MSR-Bing Image Retrieval Challenge ,written by Win
25
Text Image Retrieval Challenge -Enhance relationships between query and image Instructor： MeiChen Yeh ChenLin Yu, ChiungWei Hsu VIPLAB
-
Upload
win-yu -
Category
Technology
-
view
230 -
download
1
description
For 2014 MSR-Bing Image Retrieval Challenge written by Win
Transcript of MSR-Bing Image Retrieval Challenge ,written by Win
Text
Image Retrieval Challenge -Enhance relationships between query and image
Instructor： MeiChen Yeh ChenLin Yu, ChiungWei Hsu
VIPLAB
Outline
1. Proposed method
2. Evaluation Metric
3. Experiment Result
4. Finding and Difficulty
5. Demo
6. Conclusion
7. Future work
Proposed Method
Query
Natural Language Processing
Tokenization
POSt
QE by WordNet QE by WikipediaWordNet Wikipedia
Click_count ranking Top candidatesUser Clicklog from
MSR dataset
Apple apple apples
an apple ….
Query Processing
1. Stop word and removal
2. Tokenization
3. Stemming and Lemmatization
4. Part-of-speech Tagging
5. Wiki-suggestion (Misspelled words)
6. Expansion (wordnet and wikipeia)
Apple apple apples
an apple ….
Ranking Tablelog candidate count image
apple 1890 QYQtQsx9lH1KwA
apple 503 QJ4gfSPJYhbw0A
… … …
apple mac 490 PvfGna70qGiBIA
Click-count Ranking
MSR dataset provide real world data for user query log.
With this, generated homemade searching table by“Click-count”.
“Max click count rule”
Log data 1,000,000 (only 1/20)
We can make sure that candidate pictures are most popular.
Apple apple apples
an apple ….
Ranking Tablelog candidate count image
apple 1890 QYQtQsx9lH1KwA
apple 503 QJ4gfSPJYhbw0A
… … …
apple mac 490 PvfGna70qGiBIA
Evaluation Metric
MSR vs DIY Method
!
!
[rel]={Excellent=3,Good=2,Bad=0}
X
✔
Experiment Result
Prepare and WorkOff-line:
NLTK to process user query log
Build Ranking table (1,000,000)
Include image(base64) to Database(800,000)
On-line:
NLTK to process query input
Query expansion by word net and wikipedia
Large-scale database query processing
Single unit-query'president','frank','mars','chinese','taiwan','dargon','crash','bird','France','Eiffel','president','tony','frank','mars','chinese','taiwan','London','Mexican','ydney',
'google','yahoo','jessica','microsoft','amazon','windows','apple','line','linux','android',
'world','iphone','bacteria','cat','basketball','dog','micky','tom','jerry','christmas','table',
Test : 32 queries Acc:87.5 %
Compound word-querybook store, picture frame, the lost and bewildered tourist, ice cream, cell phone, apple pie, a story as old as time, a cool wet afternoon, many cases of infectious disease
swimming pool, the senlie old man,pencil box , long and winding road, tiddy bear , hot dog, jennifer love hewitt, some cookie shaped like stars
hello kitty coloring page, kelly osbourne drinking, micky mouse, a wet amd stinky dog
Test : 20 queries Acc:42.28 %
Finding and Difficulty
Spelling correctly can improve retrieval accuracy.
Query expansion can find more related images
!
A ambiguous query can be difficult to used.
The gap exists between users and result images, because the word is polysemic.
The user query still has a semantic problem.
Finding
In a compound word query, the relationship
between previous and next word is very
important.
Query semantic is still a challenge.
Large-scale data processing is a big problem.
How to speed up search performance?
Difficulty
Demo
Conclusion
Enhance relationships between query and image
Find relationships between query and image
Future Work
Query
Natural Language Processing
Tokenization
POSt
QE by WordNet QE by WikipediaWordNet Wikipedia
Click_count ranking Top candidates
Named Entity Recognition
User Clicklog from MSR dataset
Enhance
–ChenLin Yu, ChiungWei Hsu
“Thank you”
