Post on 19-Dec-2015
Time-dependent Similarity Time-dependent Similarity Measure of Queries Using Measure of Queries Using
Historical Click-through DataHistorical Click-through Data
Time-dependent Similarity Time-dependent Similarity Measure of Queries Using Measure of Queries Using
Historical Click-through DataHistorical Click-through Data
Qiankun Zhao*, Steven C. H. Hoi*, Tie-Yan Liu, et al.
Presented by: Tie-Yan Liu
* This work was done when Zhao and Hoi were interns at Microsoft Research Asia
OutlineOutline
• Background• Observations and Motivation • Our approach• Empirical study• Future work
BackgroundBackground
• A dilemma for Web search engines2Very short queries ~2.52Inconsistency of term usages • The Web is not well-organized• Users express queries with their own
vocabulary
Background (cont’d)Background (cont’d)
• Solution: query expansion2Document term based expansion (KDD00,
SIGIR05)• a query can be expanded with top keywords in the
top-k relevant documents
2Query term based expansion (WWW02, CIKM04)• a query can be expanded with similar queries
(queries are similar if they lead to similar pages, pages are similar if they are visited by issuing similar queries)
2Click-though data were used for query expansion in many previous work.
Background (cont’d)Background (cont’d)
• Click-through data2 Log data about the interactions between users and Web search engines
• Typical Click-through data representation
(month)
Observation 1Observation 1
• Accuracy of query similarity
Calculated only from the click-through data in that time interval.
Calculated from all the click-through data before that time point
Observation 2Observation 2
• Event driven and dynamic character of query similarity
the keyword “firework” and related pages are becoming more popular one week before the
event and reach the peak on July 4th
“firework + injuries" and “firework + picture“ have a little delay in terms of the number of times being issued and visited.
“firework + injuries" and “firework + picture“ have a little delay in terms of the number of times being issued and visited.
“firework + market" and “firework + show" become
popular and reach their peaks a few days before
July 4th
MotivationsMotivations
• Exploit the click-through data for semantic similarity of queries by incorporating temporal information
• To combine explicit content similarity and implicit semantic similarity
Our ApproachOur Approach
Time-Dependent Concepts Time-Dependent Concepts
• Calendar schema and pattern
• Example2 Calendar schema <day, month, year>2 Calendar pattern <15, *,*>2 <15, 1, 2002> is contained in the pattern <15, *,*>
Time-Dependent ConceptsTime-Dependent Concepts
• Click-Through Subgroup
• Example2 Based on the schema <day, week>, and the pattern
<1,*>, <2,*>,…,<7,*>, we can partition the data into 7 groups, which correspond to Sun, Mon, Tue, …, Sat.
Similarity MeasureSimilarity Measure
• For efficiency and simplicity, we measure the query similarity in a certain time slot only based on the click-through data.2 Vector representation of queries with respect to clicked d
ocuments.
2 wi is defined by Page Frequency (PF) and Inverted Query Frequency (IQF)
Similarity MeasureSimilarity Measure• Query similarity measures
2Cosine function2Marginalized kernel
• By introducing query clusters, one can model the query similarity in a more semantic way.
•
Time-Dependent Similarity MeasureTime-Dependent Similarity Measure
Empirical EvaluationEmpirical Evaluation
• Dataset2 Click-through log of a commercial search engine:
• June 16, 2005 to July 17,2005• Total size of 22GB• Only queries from US
2 Calendar schema and pattern• <hour, day, month>, <1, *, *>, <2, *, *>, …• Divide the data into 24 subgroups• Average subgroup size: 59,400,000 query-page pairs
Empirical ExamplesEmpirical Examples
• Kids+toy, map+route
Time-dependent daily similarityIncremented daily similarity
Empirical ExamplesEmpirical Examples
• weather + forecast, fox + news
Time-dependent daily similarityIncremented daily similarity
Quality EvaluationQuality Evaluation
• Experimental Settings2 Partition 32-day dataset into two parts
• First part for model construction• Second part for model evaluation
2 Accuracy is defined as the percentage of difference between the actual similarity and the model-based prediction
2 1000 representative query pairs, similarity larger than 0.3 using the entire dataset
• Half of them are top queries of the month• Half are selected manually related to real world events such as
“hurricane”.
Experimental ResultsExperimental Results
Experimental ResultsExperimental Results
Here “distance” is the time difference between the first test data record and the last model construction data record. For example, when the distance is 1 and the training data size is 10, we summarize all the accuracy values that use the I to 10+i days as training and use the 10+1+i as testing.
Experimental ResultsExperimental Results
ConclusionConclusion
• Presented a preliminary study of the dynamic nature of query similarity using click-through data
• Observed and verified that query similarity are dynamic and event driven with real data
• Proposed an time-dependent model• For our future work, we will investigate an
adaptive way to determine the most suitable time granularity for two given queries.
Thanks!Thanks!
tyliu@microsoft.comhttp://research.microsoft.com/users/tyliu