Professor Horst Cerjak, 19.12.2005 1 Knowledge Management Institute Mark Kröll WSDM’09 Workshop...

Professor Horst Cerjak, 19.12.20051

Knowledge Management Institute

Mark Kröll WSDM’09 Workshop on Web Search Click Data, Barcelona, Spain

Intentional Query Suggestion:

Making User Goals More Explicit During Search

Markus Strohmaier, Mark Kröll and Christian Körner

WSCD‘09: Workshop on Web Search Click Data

@ WSDM 2009 Barcelona, Spain

Mark Kröll

[email protected]

Graz University of Technology, Austria

mailto:[email protected]




Related Work

on Search Intent

Understanding goals in web search, [Broder02, Rose/Levinson04] .

The Intention Behind Web Queries, [BaezaYates06].

Understanding the Relationship between Searchers’ Queries and Information Goals, [Downey08].

on Query Suggestion

Query expansion using local and global document analysis, [Xu96].

Generating query substitutions, [Jones06].

Learning latent semantic relations from clickthrough data for query suggestions, [Ma08].

Observation: There is little work that combines these two areas.

Goal Oriented Search Engine, [Liu02]. Effects of Goal-Oriented Search Suggestions, [Mostert08].




Intentional Query Suggestion

Query Suggestion based on User Intent suggesting queries that represent potential user intentions

Initial Query Traditional Query Suggestion(MSN + Yahoo)

Intentional Query Suggestion

poker free online poker, free poker games,

cheating at poker, learn to play poker,

house houses for sale,Hugh Laurie

insure my house, build my own house

Definition of explicit intentional queries (suggested queries): contain at least one verb and

describe a plausible state of affairs that the user may want to achieve or avoid (cf.) in a recognizable way




Research Overview

How could query suggestion based on user intent be realized?

How would queries expanded by user intent influence

Click-Through? Search results?




Algorithm for Intentional Query Suggestion

by creating a mapping between implicit intentional queries(short) and explicit intentional queries (long)

Output: list of suggestionsExample: design your own poker chips learn to play home poker cheat at poker buy poker chips

Input:Example: poker Potential

Intentions




Text-based Intentional Query Suggestion

dataset containing explicit intentional queries extracted from MSN log employing the algorithm from

[Strohmaier08a]

implicit intentional queries are textually compared to all explicit intentional queries

used Jaccard Similarity Measure [BaezaYates99]

BA

BABAT qq

qqqqS

),(




Neighborhood – based Intentional Query Suggestion

based on query log data we construct a bipartite graph, where

Type Query Datequ,1 types of diet pills 2006-05-24 13:34:16qu,2 Lipo6 2006-05-24 13:36:24qu,3 lose 20 pounds in 8 weeks 2006-05-24 13:37:23qe,1 lose weight fast 2006-05-24 13:38:42qu,4 lose weight fast 2006-05-24 13:39:06qu,5 weight loss upplements 2006-05-24 13:39:51qu,6 weight loss supplements 2006-05-24 13:39:56

use neighboring queries to further describe and characterize explicit intentional queries

nodes of one type correspond to explicit intentional queries and

nodes of the second type correspond to implicit intentional queries

containing goals

containing goals

containing goals

not containing goals







Parametric model

Distance Pd

size of neighborhood N

Pd=3

Type Query Date

qu,1 types of diet pills 2006-05-24 13:34:16

qu,2 Lipo6 2006-05-24 13:36:24

qu,3 lose 20 pounds in 8 weeks 2006-05-24 13:37:23

qe,1 lose weight fast 2006-05-24 13:38:42

qu,4 lose weight fast 2006-05-24 13:39:06

qu,5 weight loss upplements 2006-05-24 13:39:51

qu,6 weight loss supplements 2006-05-24 13:39:56

Excerpt of the corresponding Bipartite Graph

Pd=3

)()(

)()(),(

BA

BABAG qTqT

qTqTqqS

T(qe,1)

qe,1

T(qe,1) = {“weight”, “loss”, “supplement”, “upplements”}




Experimental Demonstrator[joint work with student Ferdinand Wörister]




Preliminary Evaluation of Algorithm

by conducting a human subject study categorize the 10 top-ranked suggestions for 30

queries Two relevance classes:

(i) potential user intention

(ii) clear misinterpretation

Initial Query Intentional Query Suggestions

“anime” “draw anime”, “draw manga”“playground mat” “buy playground equipment”, “build a swing set”

Initial Query Intentional Query Suggestions

“Boston herald” “care for Boston fern”, “flying to Nantucket” “playground mat” “raise money for our playground”

Average Precision: 0.71

Average Interrater Agreement Kappa: 0.6




Step Back

presented one approach and experimental demonstrator to realize query suggestion based on user intent reasonable precision of 71% concerning quality of

suggested queries

How would queries expanded by user intent influence click through and search results?




Influence on Search Results(1) Result Set Intersection between different Query Suggestion

Mechanisms: How many URLs intersect between URL result sets?

Compared URL result sets Avg. Intersection

Original Queries vs. Traditional Suggestions 0.1911Original Queries vs. Intentional Suggestions 0.0467

Traditional Suggestions vs. Intentional Suggestions 0.0511

10 Traditional Suggestions Original Query 10 Intentional Suggestions

5050

5050

50

5050

5050

50 5050

5050

50

5050

5050

50500 URL result sets

(unique, top-level)




Influence on Search Results(2)

Result Set Intersection within the same Query Suggestion Mechanism:

How many URLs intersect between result sets that were retrieved by the same query suggestion mechanism regarding one original query?

Compared URL result sets Average IntersectionTraditional Suggestions 0.103Intentional Suggestions 0.026

Results suggest thatqueries that express a specific intention lead to more different results than traditional query suggestions

50

10 Traditional Suggestions 10 Intentional Suggestions

50 50. . .

. . .

50 50 50. . .

. . .




Influence on Click-Through Do users click more frequently on suggested queries if they are

explicit intentional? Experimental Setup:

click-through events for different query lengths created different bin sizes – 5000 queries randomly drawn from MSN log corresponding click-through events were registered and counted

Implicit Intentional Queries Explicit Intent.

QueriesQuery Length

1 2 3-4 5 6-10 >10 5.33

#click-through

855,649 358,327 64,313 5,559 2,728 960 7,236

Results suggest that queries that express a specific intention retrieve more relevant results than implicit intentional queries of the same length




Conclusions

combining Search Intent + Query Suggestion using high level search intent [Mostert08] employing search intent on a more detailed level appears to be a

natural next step

impact on search results and behavior (preliminary experiments)

results suggest higher click through for explicit intentional queries

results suggest more diverse results for explicit intentional queries




References[BaezaYates99] Baeza-Yates R. and Ribeiro-Neto B. Modern Information Retrieval, AddisonWesley, 1999

[BaezaYates06] Baeza-Yates, R., Calderón-Benavides, L. and González-Caro, C. 2006. The Intention Behind Web Queries, in F. Crestani, P. Ferragina and M. Sanderson, ed.,Proceedings of String Processing and Information Retrieval (SPIRE ), Springer, 98-109.

[Bendersky09] Bendersky, M. and Croft, W. B. , "Analysis of Long Queries in a Large Scale Search Log," Workshop on Web Search Click Data (WSCD 2009) Barcelona, Spain, February 9, 2009.

[Broder02] Broder A. A taxonomy of web search. In ACM SIGIR Forum 36(2), pp. 3--10, 2002.

[Downey08] Downey, D.; Dumais, S.; Liebling, D. & Horvitz, E. (2008), Understanding the relationship between searchers' queries and information goals, in 'CIKM '08: Proceeding of the 17th ACM conference on Information and knowledge mining', ACM, New York, NY, USA, pp. 449--458.

[Dumais98] Dumais S., Platt J., Heckerman D., and Sahami M. "Inductive learning algorithms and representations for text categorization". In: Proceedings International Conference on Information and Knowledge Management, New York, NY, USA, ACM Press, pp 148-155, 1998

[Jansen08] Jansen, B. J., Booth, D. L. and Spink, A. Determining the informational, navigational, and transactional intent of web queries. In Inf. Process. Manage. 44(3), pp. 1251--1266, 2008.

[Jones06] Jones, R.; Rey, B.; Madani, O. & Greiner, W. (2006), Generating query substitutions, in 'WWW '06: Proceedings of the 15th international conference on World Wide Web', ACM, New York, NY, USA, pp. 387--396.

[He07] K.Y. He and Y.S. Chang and W.H. Lu. Improving Identification of Latent User Goals through Search-Result Snippet Classification. WI '07: Proceedings of the 2007 IEEE/WIC/ACM International Conference on Web Intelligence, 683-686, IEEE Computer Society,2007.

[Ma08] Ma, H.; Yang, H.; King, I. & Lyu, M. R. (2008), Learning latent semantic relations from clickthrough data for query suggestion, in 'CIKM '08: Proceeding of the 17th ACM conference on Information and knowledge mining', ACM, New York, NY, USA, pp. 709--718.

[Mostert08] Mostert, J. & Hollink, V. (2008), Effects of Goal-Oriented Search Suggestions, in 'Proceedings of the 20th Belgian-Netherlands Conference on Artificial Intelligence'.

[Liu and Lieberman02] Liu H. , Lieberman H. and Selker T.. GOOSE: A Goal-Oriented Search Engine with Commonsense. AH '02: Proceedings of the Second International Conference on Adaptive Hypermedia and Adaptive Web-Based Systems, 253--263, Springer-Verlag,London, UK,2002.

[Rose04] Rose D. E. and Levinson D., Understanding user goals in web search. In Proc. of WWW 2004, May 17-22, 2004, New York, USA, 2004.

[Strohmaier08a] Strohmaier, M., Prettenhofer, P. and Kröll, M. Acquiring explicit user goals from search query logs. In 'International Workshop on Agents and Data Mining Interaction ADMI‘ 08, in conjunction with WI '08', 2008.

[Strohmaier08b] M. Strohmaier, P. Prettenhofer, M. Lux, Different Degrees of Explicitness in Intentional Artifacts - Studying User Goals in a Large Search Query Log, CSKGOI'08 International Workshop on Commonsense Knowledge and Goal Oriented Interfaces, in conjunction with IUI'08, Canary Islands, Spain, 2008.

[Xu96] Xu, J. & Croft, W. B. (1996), Query expansion using local and global document analysis, in 'SIGIR '96: Proceedings of the 19th annual international ACM SIGIR conference on Research and development in information retrieval', ACM, New York, NY, USA, pp. 4--11.




End of Presentation

Mark KröllGraz University of Technology, Austria

[email protected]

Thanks for your attention!




Questions and Discussion

Professor Horst Cerjak, 19.12.2005 1 Knowledge Management Institute Mark Kröll WSDM’09 Workshop...

Documents

Transcript of Professor Horst Cerjak, 19.12.2005 1 Knowledge Management Institute Mark Kröll WSDM’09 Workshop...