Flickr Tag Recommendation based on Collective Knowledge BÖrkur SigurbjÖnsson, Roelof van Zwol...
-
Upload
edith-charles -
Category
Documents
-
view
216 -
download
2
Transcript of Flickr Tag Recommendation based on Collective Knowledge BÖrkur SigurbjÖnsson, Roelof van Zwol...
Flickr Tag Recommendationbased on Collective Knowledge
BÖrkur SigurbjÖnsson, Roelof van Zwol
Yahoo! Research
WWW 2008
2009. 03. 13.
Summarized and presented by Hwang Inbeom, IDS Lab., Seoul National University
Copyright 2008 by CEBT
Overview
Recommending tags for an image
More tags, more semantic meanings
Solves two questions
How much would the recommending be effective?
– Analyzing tagging behaviors
How can we recommend tags?
– Presenting some recommending strategies
2
Copyright 2008 by CEBT
Tagging
Tagging
The act of adding keywords to objects
Popular means to annotate various web resources
Web page bookmarks
Academic publications
Multimedia objects
…
3
Copyright 2008 by CEBT
Advantages of Tagging Images
Content-based image retrieval is progressing, but it has not yet succeeded in reducing semantic gap
Tagging is essential for large-scale image retrieval sys-tems to work in practice
Extension of tags
Richer semantic description
Can be used to retrieve the photofor a larger range of keyword queries
4
Sagrada Fa-miliaBarcelona
Sagrada Fa-milia
BarcelonaGaudiSpain
Catalunyaarchitecture
church
Copyright 2008 by CEBT
Analysis of Tagging Behaviors
How do users tag photos?
Distribution of tag frequency
Distribution of the number of tags per photo
What kind of tags do they provide?
Tag categorization with WordNet
5
Copyright 2008 by CEBT
Head
Tail
Tag Frequency
Distribution of tag frequency could be modeled by a power law
Tags residing in the head of power law
Too generic tags
– 2006, 2005, wedding
Tags in tail of powerlaw
Incidentally occurring words
– ambrose tompkins,ambient vector
6
)()( 15.115.1 xOaxxf
Copyright 2008 by CEBT
Head
Tail
Number of Tags per Photo
Distribution could be modeled by power law too
Photos in head of power law
Exhaustively annotated
Photos in tail of power law
Tag recommendationsystem could be useful
– Covers 64% of the photos
7
)()( 33.033.0 xObxxg
Copyright 2008 by CEBT
Number of Tags per Photo (contd.)
Photos classified by number of tags annotated
To be used to analyze the performance of recommending for different annotation levels
8
Tags per Photo Photos
Class I 1 15,500,000
Class II 2-3 17,500,000
Class III 4-6 12,000,000
Class IV >6 7,000,000
Copyright 2008 by CEBT
Tag Categorization
9
52% of tags could be catego-rized by WordNet categories
Users provide a broader con-text by tags, not only visual contents of the photo
Where / when the photo was taken
Actions people in the photo are doing
…
locations; 28%
arti-facts or
ob-jects; 16%
people or groups; 13%
actions or
events; 9%
time; 7%
other; 27%
Copyright 2008 by CEBT
Tag Recommendation System
A given photo and
user-de-fined tags
Finding candi-date tags
•Co-occur-ring tags
Tag aggre-gation
and ranking•Ranked list of candi-date tags
10
Sagrada Fa-miliaBarcelona
BarcelonaSpainGaudi2006
CatalunyaEuropetravel
Sagrada Fa-miliaBarcelonaGaudiSpainarchitectureCatalunyachurch
GaudiSpain
Catalunyaarchitecture
church
Copyright 2008 by CEBT
Tag Recommendation Strategies
Finding candidate tags based on tag co-occurrence
Symmetric measures
Asymmetric measures
Aggregation and ranking of candidate tags
Voting strategy
Summing strategy
Promotion
11
Copyright 2008 by CEBT
Tag Co-occurrence
Finding tags co-occurring with a specific tag
Co-occurring tags with higher score become candidate tags
Could be measured in two ways
Symmetric measures
Asymmetric measures
12
Copyright 2008 by CEBT
Tag Co-occurrence (contd.)
Symmetric measures
Jaccard’s coefficient
– Statistic used for computing the similarity and diversity of sample sets
Useful to identify equivalent tags
Example – Eiffel tower
– Tour Eiffel, Eiffel, Seine, La tour Eiffel, Paris
13
||
||:),(
ji
jiji tt
ttttJ
Copyright 2008 by CEBT
Tag Co-occurrence (contd.)
Asymmetric measures
Tag co-occurrence can be normalized using the frequency of one of the tags
Can provide more diverse candidates than symmetric method
Example – Eiffel Tower
– Paris, France, Tour Eiffel, Eiffel, Europe
Asymmetric tag co-occurrence will provide a more suit-able diversity
14
||
||:)|(
i
jiij t
ttttP
Copyright 2008 by CEBT
Tag Aggregation
Definitions
U is user-defined tags
Cu is top-m most co-occurring tags of a tag u in U
C is the union of all candidate tags for all user-defined tag u
R is recommended tags
15
Sagrada Fa-miliaBarcelona
BarcelonaSpainGaudi2006
CatalunyaEuropetravel
Sagrada Fa-miliaBarcelonaGaudiSpainarchitectureCatalunyachurch
GaudiSpain
Catalunyaarchitecture
church
Copyright 2008 by CEBT
Tag Aggregation (contd.)
Vote
For each candidate tag c in C, whenever c is in Cu a vote is
cast
R is obtained by sorting the candidate tags on the number of votes
16
1),( cuvote uCcif
Uu
cuvotecscore ),(:)(
BarcelonaSpainGaudi2006
CatalunyaEuropetravel
Sagrada Fa-milia
BarcelonaGaudiSpain
architectureCatalunya
church
Tag Score
Barcelona 1
Gaudi 2
Spain 2
… …
Copyright 2008 by CEBT
Tag Aggregation (contd.)
Sum
Sums over co-occurrence values of the candidate tags c in Cu
17
Uu
ucPcscore )|(:)( uCcif
Copyright 2008 by CEBT
Promotion
Stability-promotion
To make user-defined tags with low frequency less reliable
Descriptiveness-promotion
To avoid generaltags ranked too highly
18
Head
Tail
||)log(||:)(
ukk
kustability
ss
s
||)log(||:)(
ckk
kcedescriptiv
dd
d
Copyright 2008 by CEBT
Promotion (contd.)
Rank-promotion
Co-occurrence values used in summing strategy declines too fast
To make co-occurrence values work better
Applying promotion
19
)1(:),(
rk
kcurank
r
r
)()(),(),( cedescriptivustabilitycurankcupromotion
Uu
cupromotioncuvotecscore ),(),(:)(
Copyright 2008 by CEBT
Experimental Setup
For different strategies
Assessments
Top 10 recommendations from each of the four strategies make a pool
Assessors were asked to assess the descriptiveness of each tags
– Assessed as very good, good, not good, don’t know
Assessors could access and view photo directly on Flickr, to find additional context
20
vote sum
No-promotion vote sum
Promotion vote+ sum+
Copyright 2008 by CEBT
Experimental Setup (contd.)
Evaluation metrics
Mean Reciprocal Rank (MRR)
– Evaluates probability that the system returns a “relevant” tag at the top of the ranking
– Tag is relevant if its relevance score is bigger than average of relevance
Success at rank k (S@k)
– Probability of finding a good descriptive tag among the top k recommended tags
Precision at rank k (P@k)
– Proportion of retrieved tags that is relevant, averaged over all photos
21
Copyright 2008 by CEBT
Experiment Results
Promotion worked well
Without promotion, summing is better
With promotion, voting is better
22
Copyright 2008 by CEBT
Experiment Results (contd.)
Promotion acted better with more user-defined tags
23
Tags per Photo Photos
Class I 1 15,500,000
Class II 2-3 17,500,000
Class III 4-6 12,000,000
Class IV >6 7,000,000
Copyright 2008 by CEBT
Experiment Results (contd.)
Semantic analysis
Tags related to visual contents of the photo are more likely to accepted
– Higher acceptance ratio of more physical categories
24
Copyright 2008 by CEBT
Conclusions
Tag behavior in Flickr
Tag frequency follows a power law
Majority of photos are not annotated well enough
Users annotate their photos using tags with broad spectrum of the semantic space
Extending Flickr annotations
Co-occurrence model with aggregation and promotion was effective
Can incrementally updated
Future work
This model could be implemented as a recommendation system
25