Post on 18-Dec-2015
Professor Horst Cerjak, 19.12.20051
Knowledge Management Institute
Mark Kröll WSDM’09 Workshop on Web Search Click Data, Barcelona, Spain
Intentional Query Suggestion:
Making User Goals More Explicit During Search
Markus Strohmaier, Mark Kröll and Christian Körner
WSCD‘09: Workshop on Web Search Click Data
@ WSDM 2009 Barcelona, Spain
Mark Kröll
mkroell@tugraz.at
Graz University of Technology, Austria
Professor Horst Cerjak, 19.12.20052
Knowledge Management Institute
Mark Kröll WSDM’09 Workshop on Web Search Click Data, Barcelona, Spain
Related Work
on Search Intent
Understanding goals in web search, [Broder02, Rose/Levinson04] .
The Intention Behind Web Queries, [BaezaYates06].
Understanding the Relationship between Searchers’ Queries and Information Goals, [Downey08].
on Query Suggestion
Query expansion using local and global document analysis, [Xu96].
Generating query substitutions, [Jones06].
Learning latent semantic relations from clickthrough data for query suggestions, [Ma08].
Observation: There is little work that combines these two areas.
Goal Oriented Search Engine, [Liu02]. Effects of Goal-Oriented Search Suggestions, [Mostert08].
Professor Horst Cerjak, 19.12.20053
Knowledge Management Institute
Mark Kröll WSDM’09 Workshop on Web Search Click Data, Barcelona, Spain
Intentional Query Suggestion
Query Suggestion based on User Intent suggesting queries that represent potential user intentions
Initial Query Traditional Query Suggestion(MSN + Yahoo)
Intentional Query Suggestion
poker free online poker, free poker games,
cheating at poker, learn to play poker,
house houses for sale,Hugh Laurie
insure my house, build my own house
Definition of explicit intentional queries (suggested queries): contain at least one verb and
describe a plausible state of affairs that the user may want to achieve or avoid (cf.) in a recognizable way
Professor Horst Cerjak, 19.12.20054
Knowledge Management Institute
Mark Kröll WSDM’09 Workshop on Web Search Click Data, Barcelona, Spain
Research Overview
How could query suggestion based on user intent be realized?
How would queries expanded by user intent influence
Click-Through? Search results?
Professor Horst Cerjak, 19.12.20055
Knowledge Management Institute
Mark Kröll WSDM’09 Workshop on Web Search Click Data, Barcelona, Spain
Algorithm for Intentional Query Suggestion
by creating a mapping between implicit intentional queries(short) and explicit intentional queries (long)
Output: list of suggestionsExample: design your own poker chips learn to play home poker cheat at poker buy poker chips
Input:Example: poker Potential
Intentions
Professor Horst Cerjak, 19.12.20056
Knowledge Management Institute
Mark Kröll WSDM’09 Workshop on Web Search Click Data, Barcelona, Spain
Text-based Intentional Query Suggestion
dataset containing explicit intentional queries extracted from MSN log employing the algorithm from
[Strohmaier08a]
implicit intentional queries are textually compared to all explicit intentional queries
used Jaccard Similarity Measure [BaezaYates99]
BA
BABAT qq
qqqqS
),(
Professor Horst Cerjak, 19.12.20057
Knowledge Management Institute
Mark Kröll WSDM’09 Workshop on Web Search Click Data, Barcelona, Spain
Neighborhood – based Intentional Query Suggestion
based on query log data we construct a bipartite graph, where
Type Query Datequ,1 types of diet pills 2006-05-24 13:34:16qu,2 Lipo6 2006-05-24 13:36:24qu,3 lose 20 pounds in 8 weeks 2006-05-24 13:37:23qe,1 lose weight fast 2006-05-24 13:38:42qu,4 lose weight fast 2006-05-24 13:39:06qu,5 weight loss upplements 2006-05-24 13:39:51qu,6 weight loss supplements 2006-05-24 13:39:56
use neighboring queries to further describe and characterize explicit intentional queries
nodes of one type correspond to explicit intentional queries and
nodes of the second type correspond to implicit intentional queries
containing goals
containing goals
containing goals
not containing goals
not containing goals
not containing goals
not containing goals
Professor Horst Cerjak, 19.12.20058
Knowledge Management Institute
Mark Kröll WSDM’09 Workshop on Web Search Click Data, Barcelona, Spain
Parametric model
Distance Pd
size of neighborhood N
Pd=3
Type Query Date
qu,1 types of diet pills 2006-05-24 13:34:16
qu,2 Lipo6 2006-05-24 13:36:24
qu,3 lose 20 pounds in 8 weeks 2006-05-24 13:37:23
qe,1 lose weight fast 2006-05-24 13:38:42
qu,4 lose weight fast 2006-05-24 13:39:06
qu,5 weight loss upplements 2006-05-24 13:39:51
qu,6 weight loss supplements 2006-05-24 13:39:56
Excerpt of the corresponding Bipartite Graph
Pd=3
)()(
)()(),(
BA
BABAG qTqT
qTqTqqS
T(qe,1)
qe,1
T(qe,1) = {“weight”, “loss”, “supplement”, “upplements”}
Professor Horst Cerjak, 19.12.20059
Knowledge Management Institute
Mark Kröll WSDM’09 Workshop on Web Search Click Data, Barcelona, Spain
Experimental Demonstrator[joint work with student Ferdinand Wörister]
Professor Horst Cerjak, 19.12.200510
Knowledge Management Institute
Mark Kröll WSDM’09 Workshop on Web Search Click Data, Barcelona, Spain
Preliminary Evaluation of Algorithm
by conducting a human subject study categorize the 10 top-ranked suggestions for 30
queries Two relevance classes:
(i) potential user intention
(ii) clear misinterpretation
Initial Query Intentional Query Suggestions
“anime” “draw anime”, “draw manga”“playground mat” “buy playground equipment”, “build a swing set”
Initial Query Intentional Query Suggestions
“Boston herald” “care for Boston fern”, “flying to Nantucket” “playground mat” “raise money for our playground”
Average Precision: 0.71
Average Interrater Agreement Kappa: 0.6
Professor Horst Cerjak, 19.12.200511
Knowledge Management Institute
Mark Kröll WSDM’09 Workshop on Web Search Click Data, Barcelona, Spain
Step Back
presented one approach and experimental demonstrator to realize query suggestion based on user intent reasonable precision of 71% concerning quality of
suggested queries
How would queries expanded by user intent influence click through and search results?
Professor Horst Cerjak, 19.12.200512
Knowledge Management Institute
Mark Kröll WSDM’09 Workshop on Web Search Click Data, Barcelona, Spain
Influence on Search Results(1) Result Set Intersection between different Query Suggestion
Mechanisms: How many URLs intersect between URL result sets?
Compared URL result sets Avg. Intersection
Original Queries vs. Traditional Suggestions 0.1911Original Queries vs. Intentional Suggestions 0.0467
Traditional Suggestions vs. Intentional Suggestions 0.0511
10 Traditional Suggestions Original Query 10 Intentional Suggestions
5050
5050
50
5050
5050
50 5050
5050
50
5050
5050
50500 URL result sets
(unique, top-level)
Professor Horst Cerjak, 19.12.200513
Knowledge Management Institute
Mark Kröll WSDM’09 Workshop on Web Search Click Data, Barcelona, Spain
Influence on Search Results(2)
Result Set Intersection within the same Query Suggestion Mechanism:
How many URLs intersect between result sets that were retrieved by the same query suggestion mechanism regarding one original query?
Compared URL result sets Average IntersectionTraditional Suggestions 0.103Intentional Suggestions 0.026
Results suggest thatqueries that express a specific intention lead to more different results than traditional query suggestions
50
10 Traditional Suggestions 10 Intentional Suggestions
50 50. . .
. . .
50 50 50. . .
. . .
Professor Horst Cerjak, 19.12.200514
Knowledge Management Institute
Mark Kröll WSDM’09 Workshop on Web Search Click Data, Barcelona, Spain
Influence on Click-Through Do users click more frequently on suggested queries if they are
explicit intentional? Experimental Setup:
click-through events for different query lengths created different bin sizes – 5000 queries randomly drawn from MSN log corresponding click-through events were registered and counted
Implicit Intentional Queries Explicit Intent.
QueriesQuery Length
1 2 3-4 5 6-10 >10 5.33
#click-through
855,649 358,327 64,313 5,559 2,728 960 7,236
Results suggest that queries that express a specific intention retrieve more relevant results than implicit intentional queries of the same length
Professor Horst Cerjak, 19.12.200515
Knowledge Management Institute
Mark Kröll WSDM’09 Workshop on Web Search Click Data, Barcelona, Spain
Conclusions
combining Search Intent + Query Suggestion using high level search intent [Mostert08] employing search intent on a more detailed level appears to be a
natural next step
impact on search results and behavior (preliminary experiments)
results suggest higher click through for explicit intentional queries
results suggest more diverse results for explicit intentional queries
Professor Horst Cerjak, 19.12.200516
Knowledge Management Institute
Mark Kröll WSDM’09 Workshop on Web Search Click Data, Barcelona, Spain
References[BaezaYates99] Baeza-Yates R. and Ribeiro-Neto B. Modern Information Retrieval, AddisonWesley, 1999
[BaezaYates06] Baeza-Yates, R., Calderón-Benavides, L. and González-Caro, C. 2006. The Intention Behind Web Queries, in F. Crestani, P. Ferragina and M. Sanderson, ed.,Proceedings of String Processing and Information Retrieval (SPIRE ), Springer, 98-109.
[Bendersky09] Bendersky, M. and Croft, W. B. , "Analysis of Long Queries in a Large Scale Search Log," Workshop on Web Search Click Data (WSCD 2009) Barcelona, Spain, February 9, 2009.
[Broder02] Broder A. A taxonomy of web search. In ACM SIGIR Forum 36(2), pp. 3--10, 2002.
[Downey08] Downey, D.; Dumais, S.; Liebling, D. & Horvitz, E. (2008), Understanding the relationship between searchers' queries and information goals, in 'CIKM '08: Proceeding of the 17th ACM conference on Information and knowledge mining', ACM, New York, NY, USA, pp. 449--458.
[Dumais98] Dumais S., Platt J., Heckerman D., and Sahami M. "Inductive learning algorithms and representations for text categorization". In: Proceedings International Conference on Information and Knowledge Management, New York, NY, USA, ACM Press, pp 148-155, 1998
[Jansen08] Jansen, B. J., Booth, D. L. and Spink, A. Determining the informational, navigational, and transactional intent of web queries. In Inf. Process. Manage. 44(3), pp. 1251--1266, 2008.
[Jones06] Jones, R.; Rey, B.; Madani, O. & Greiner, W. (2006), Generating query substitutions, in 'WWW '06: Proceedings of the 15th international conference on World Wide Web', ACM, New York, NY, USA, pp. 387--396.
[He07] K.Y. He and Y.S. Chang and W.H. Lu. Improving Identification of Latent User Goals through Search-Result Snippet Classification. WI '07: Proceedings of the 2007 IEEE/WIC/ACM International Conference on Web Intelligence, 683-686, IEEE Computer Society,2007.
[Ma08] Ma, H.; Yang, H.; King, I. & Lyu, M. R. (2008), Learning latent semantic relations from clickthrough data for query suggestion, in 'CIKM '08: Proceeding of the 17th ACM conference on Information and knowledge mining', ACM, New York, NY, USA, pp. 709--718.
[Mostert08] Mostert, J. & Hollink, V. (2008), Effects of Goal-Oriented Search Suggestions, in 'Proceedings of the 20th Belgian-Netherlands Conference on Artificial Intelligence'.
[Liu and Lieberman02] Liu H. , Lieberman H. and Selker T.. GOOSE: A Goal-Oriented Search Engine with Commonsense. AH '02: Proceedings of the Second International Conference on Adaptive Hypermedia and Adaptive Web-Based Systems, 253--263, Springer-Verlag,London, UK,2002.
[Rose04] Rose D. E. and Levinson D., Understanding user goals in web search. In Proc. of WWW 2004, May 17-22, 2004, New York, USA, 2004.
[Strohmaier08a] Strohmaier, M., Prettenhofer, P. and Kröll, M. Acquiring explicit user goals from search query logs. In 'International Workshop on Agents and Data Mining Interaction ADMI‘ 08, in conjunction with WI '08', 2008.
[Strohmaier08b] M. Strohmaier, P. Prettenhofer, M. Lux, Different Degrees of Explicitness in Intentional Artifacts - Studying User Goals in a Large Search Query Log, CSKGOI'08 International Workshop on Commonsense Knowledge and Goal Oriented Interfaces, in conjunction with IUI'08, Canary Islands, Spain, 2008.
[Xu96] Xu, J. & Croft, W. B. (1996), Query expansion using local and global document analysis, in 'SIGIR '96: Proceedings of the 19th annual international ACM SIGIR conference on Research and development in information retrieval', ACM, New York, NY, USA, pp. 4--11.
Professor Horst Cerjak, 19.12.200517
Knowledge Management Institute
Mark Kröll WSDM’09 Workshop on Web Search Click Data, Barcelona, Spain
End of Presentation
Mark KröllGraz University of Technology, Austria
mkroell@tugraz.at
Thanks for your attention!