Andorra Future Of Web Search Talk
-
Upload
mor -
Category
Technology
-
view
1.266 -
download
0
description
Transcript of Andorra Future Of Web Search Talk
![Page 1: Andorra Future Of Web Search Talk](https://reader034.fdocuments.us/reader034/viewer/2022051610/5496e841ac7959182e8b51cf/html5/thumbnails/1.jpg)
The Social Media Opportunityand Application for Landmark Search
Mor Naaman
Yahoo! Advanced Development Research
![Page 2: Andorra Future Of Web Search Talk](https://reader034.fdocuments.us/reader034/viewer/2022051610/5496e841ac7959182e8b51cf/html5/thumbnails/2.jpg)
Mor Naaman
Social Media Patterns:
Semantic space (from any text)
Activity and viewing data
User/personal data
Social network
Location/time metadata
2
![Page 3: Andorra Future Of Web Search Talk](https://reader034.fdocuments.us/reader034/viewer/2022051610/5496e841ac7959182e8b51cf/html5/thumbnails/3.jpg)
Mor Naaman
E.g., Semantic Patterns
3
![Page 4: Andorra Future Of Web Search Talk](https://reader034.fdocuments.us/reader034/viewer/2022051610/5496e841ac7959182e8b51cf/html5/thumbnails/4.jpg)
Mor Naaman
Heard of Flickr?
4
![Page 5: Andorra Future Of Web Search Talk](https://reader034.fdocuments.us/reader034/viewer/2022051610/5496e841ac7959182e8b51cf/html5/thumbnails/5.jpg)
Mor Naaman
This is not An Arch
“Noisy” data
Photographer biases
Wrong data
6 kms5 kms
5
![Page 6: Andorra Future Of Web Search Talk](https://reader034.fdocuments.us/reader034/viewer/2022051610/5496e841ac7959182e8b51cf/html5/thumbnails/6.jpg)
Mor Naaman
Tag Patterns
6
![Page 7: Andorra Future Of Web Search Talk](https://reader034.fdocuments.us/reader034/viewer/2022051610/5496e841ac7959182e8b51cf/html5/thumbnails/7.jpg)
Mor Naaman
What is “Social Media”?
Online media published or shared by individuals and organizations, in an environment that encourages significant individual participation and that promotes curation, discussion and re-use.
7
![Page 8: Andorra Future Of Web Search Talk](https://reader034.fdocuments.us/reader034/viewer/2022051610/5496e841ac7959182e8b51cf/html5/thumbnails/8.jpg)
Mor Naaman
Social Media = Context
Context is kingPredictor of content
Modifies perception of content
Social media: context also predicts activity?
8
![Page 9: Andorra Future Of Web Search Talk](https://reader034.fdocuments.us/reader034/viewer/2022051610/5496e841ac7959182e8b51cf/html5/thumbnails/9.jpg)
Mor Naaman
Social Media = Challenge
Content is still hard…
Unstructured data (no semantics)
Tags, not ground truth labels
Noise
Scale • Computation
• Long tail means no supervised learning
9
![Page 10: Andorra Future Of Web Search Talk](https://reader034.fdocuments.us/reader034/viewer/2022051610/5496e841ac7959182e8b51cf/html5/thumbnails/10.jpg)
Mor Naaman
Social Media = Opportunity
New types of data / context
Openness/need for new applications and experiences
Social environment encourages engagement
Opportunity for MM!10
User
CommunityAnalyze, extract
patterns
Support, motivate
Encourage, enable
![Page 11: Andorra Future Of Web Search Talk](https://reader034.fdocuments.us/reader034/viewer/2022051610/5496e841ac7959182e8b51cf/html5/thumbnails/11.jpg)
Mor Naaman
Approach
Analyze context to extract patternsReduce content analysis to constrained scenario/task
Leverage content to improve metadata, relevance
11
![Page 12: Andorra Future Of Web Search Talk](https://reader034.fdocuments.us/reader034/viewer/2022051610/5496e841ac7959182e8b51cf/html5/thumbnails/12.jpg)
Mor Naaman
Location-driven Modeling
12
![Page 13: Andorra Future Of Web Search Talk](https://reader034.fdocuments.us/reader034/viewer/2022051610/5496e841ac7959182e8b51cf/html5/thumbnails/13.jpg)
Mor Naaman
Extracting Knowledge
13
More “activity” in a certain locationindicates the importance of that location
Tags that are unique to a certain location can be used to represent the location
![Page 14: Andorra Future Of Web Search Talk](https://reader034.fdocuments.us/reader034/viewer/2022051610/5496e841ac7959182e8b51cf/html5/thumbnails/14.jpg)
Mor Naaman
Tag Maps - SF
14
![Page 15: Andorra Future Of Web Search Talk](https://reader034.fdocuments.us/reader034/viewer/2022051610/5496e841ac7959182e8b51cf/html5/thumbnails/15.jpg)
Mor Naaman
Make a World Explorer
15
published in JCDL 2007
http://tagmaps.research.yahoo.com
![Page 16: Andorra Future Of Web Search Talk](https://reader034.fdocuments.us/reader034/viewer/2022051610/5496e841ac7959182e8b51cf/html5/thumbnails/16.jpg)
Mor Naaman 16
Better Image Search
![Page 17: Andorra Future Of Web Search Talk](https://reader034.fdocuments.us/reader034/viewer/2022051610/5496e841ac7959182e8b51cf/html5/thumbnails/17.jpg)
Mor Naaman
Approach
Analyze context to extract patternsReduce content analysis to constrained scenario/task
Leverage content to improve metadata, relevance
17
![Page 18: Andorra Future Of Web Search Talk](https://reader034.fdocuments.us/reader034/viewer/2022051610/5496e841ac7959182e8b51cf/html5/thumbnails/18.jpg)
Mor Naaman
Rolling in Content
We identified the landmarks...
We know where they are...
We can get the matching photos...
18
![Page 19: Andorra Future Of Web Search Talk](https://reader034.fdocuments.us/reader034/viewer/2022051610/5496e841ac7959182e8b51cf/html5/thumbnails/19.jpg)
Mor Naaman
System Overview
19
published in WWW 2008
published in ACM MM 2007
![Page 20: Andorra Future Of Web Search Talk](https://reader034.fdocuments.us/reader034/viewer/2022051610/5496e841ac7959182e8b51cf/html5/thumbnails/20.jpg)
Mor Naaman
Learning from noisy labels
20
![Page 21: Andorra Future Of Web Search Talk](https://reader034.fdocuments.us/reader034/viewer/2022051610/5496e841ac7959182e8b51cf/html5/thumbnails/21.jpg)
Mor Naaman
Visual Features
•Color: moments over a 5x5 grid
•Texture: Gabor over global image
• Interest points: SIFT
21
![Page 22: Andorra Future Of Web Search Talk](https://reader034.fdocuments.us/reader034/viewer/2022051610/5496e841ac7959182e8b51cf/html5/thumbnails/22.jpg)
Mor Naaman
Ranking Clusters (1)
22
Same “objects” that appear often in cluster’s photos suggest relevance
![Page 23: Andorra Future Of Web Search Talk](https://reader034.fdocuments.us/reader034/viewer/2022051610/5496e841ac7959182e8b51cf/html5/thumbnails/23.jpg)
Mor Naaman
Ranking Clusters (2)
23
Similarity between photos inside cluster versus outside the cluster suggests coherence
Use Visual Features to compare average intra-cluster and inter-cluster similarity
![Page 24: Andorra Future Of Web Search Talk](https://reader034.fdocuments.us/reader034/viewer/2022051610/5496e841ac7959182e8b51cf/html5/thumbnails/24.jpg)
Mor Naaman
System Overview
24
![Page 25: Andorra Future Of Web Search Talk](https://reader034.fdocuments.us/reader034/viewer/2022051610/5496e841ac7959182e8b51cf/html5/thumbnails/25.jpg)
Mor Naaman
Sample Results: Golden Gate
Tags-only Tags+Location Tags+Location+Visual
XX
X
X
XX
XX
X25
![Page 26: Andorra Future Of Web Search Talk](https://reader034.fdocuments.us/reader034/viewer/2022051610/5496e841ac7959182e8b51cf/html5/thumbnails/26.jpg)
Mor Naaman
Performance: PrecisionP
re
cis
ion
@ 1
0
0
0.25
0.50
0.75
1.00
alcatraz
baybridge
coittower
deyoung
ferrybuilding
goldengatebridge
lombardstreet
palaceoffinearts
sfmoma
transamerica
average
Tag-Only Tag-Location Tag-Visual Tag-Location-Visual
26
+45% w/visual
+30% w/location
![Page 27: Andorra Future Of Web Search Talk](https://reader034.fdocuments.us/reader034/viewer/2022051610/5496e841ac7959182e8b51cf/html5/thumbnails/27.jpg)
Mor Naaman
Performance: RepresentativeR
ep
res
en
tati
ve
Ph
oto
s
0
2.5
5.0
7.5
10.0
alcatraz
baybridge
coittower
deyoung
ferrybuilding
goldengatebridge
lombardstreet
palaceoffinearts
sfmoma
transamerica
average
Tag-Only Tag-Location Tag-Visual
27
![Page 28: Andorra Future Of Web Search Talk](https://reader034.fdocuments.us/reader034/viewer/2022051610/5496e841ac7959182e8b51cf/html5/thumbnails/28.jpg)
Mor Naaman
Improve Relevance
28
![Page 29: Andorra Future Of Web Search Talk](https://reader034.fdocuments.us/reader034/viewer/2022051610/5496e841ac7959182e8b51cf/html5/thumbnails/29.jpg)
Mor Naaman
Repeated in Other Context
Analyze context to extract patternsReduce content analysis to constrained scenario/task
Leverage content to improve metadata, relevance
29
![Page 30: Andorra Future Of Web Search Talk](https://reader034.fdocuments.us/reader034/viewer/2022051610/5496e841ac7959182e8b51cf/html5/thumbnails/30.jpg)
Mor Naaman
Social Media @ Music Events
30
Analyze context to get set of media items from a single event
Use content (AF) to robustly synchronize the clips
Increase relevance,
findability
![Page 31: Andorra Future Of Web Search Talk](https://reader034.fdocuments.us/reader034/viewer/2022051610/5496e841ac7959182e8b51cf/html5/thumbnails/31.jpg)
Mor Naaman
Social Media = Opportunity
To better understand media contentAnd robustly apply content analysis
To predict and enhance use and engagement
To invent the future of multimedia systems
31
![Page 32: Andorra Future Of Web Search Talk](https://reader034.fdocuments.us/reader034/viewer/2022051610/5496e841ac7959182e8b51cf/html5/thumbnails/32.jpg)
Mor Naaman
Thanks
With: Lyndon Kennedy, Tye Rattenbury, Alex Jaffe, Shane Ahern, Rahul Nair, Jeannie Yang
Some Slides: http://slideshare.net/mor
All photos/icons CC or with permission
http://infolab.stanford.edu/~mor
[email protected] , [email protected]
32
![Page 33: Andorra Future Of Web Search Talk](https://reader034.fdocuments.us/reader034/viewer/2022051610/5496e841ac7959182e8b51cf/html5/thumbnails/33.jpg)
Mor Naaman
Ranking images: low-level similarity
Euclidean distance from cluster centroid in color and texture space.
33
![Page 34: Andorra Future Of Web Search Talk](https://reader034.fdocuments.us/reader034/viewer/2022051610/5496e841ac7959182e8b51cf/html5/thumbnails/34.jpg)
Mor Naaman
Notes: Fire Eagle
34
http://fireeagle.yahoo.net
![Page 35: Andorra Future Of Web Search Talk](https://reader034.fdocuments.us/reader034/viewer/2022051610/5496e841ac7959182e8b51cf/html5/thumbnails/35.jpg)
Mor Naaman
Landmark Graph Structure
35
Close-up Shots of TowerLess-Connected PhotosHighly-Connected Photos Taken From Tower Far-away Shots
(a) Golden Gate Bridge (b) Coit Tower
![Page 36: Andorra Future Of Web Search Talk](https://reader034.fdocuments.us/reader034/viewer/2022051610/5496e841ac7959182e8b51cf/html5/thumbnails/36.jpg)
Mor Naaman
Results: Palace of Fine Arts
XX
X
XXXX
Tags-only Tags+Location Tags+Location+Visual
36