Object Recognition a Machine Translation Learning a Lexicon for a Fixed Image Vocabulary Miriam...
-
Upload
jasper-mccarthy -
Category
Documents
-
view
214 -
download
1
Transcript of Object Recognition a Machine Translation Learning a Lexicon for a Fixed Image Vocabulary Miriam...
![Page 1: Object Recognition a Machine Translation Learning a Lexicon for a Fixed Image Vocabulary Miriam Miklofsky.](https://reader036.fdocuments.us/reader036/viewer/2022082819/56649f305503460f94c4aa4c/html5/thumbnails/1.jpg)
Object Recognition a Machine Translation
Learning a Lexicon for a Fixed Image Vocabulary
Miriam Miklofsky
![Page 2: Object Recognition a Machine Translation Learning a Lexicon for a Fixed Image Vocabulary Miriam Miklofsky.](https://reader036.fdocuments.us/reader036/viewer/2022082819/56649f305503460f94c4aa4c/html5/thumbnails/2.jpg)
Lexicons
A vocabulary of terms used in a subjectA specialized list of terms
Devices that predict one representation given another representation
![Page 3: Object Recognition a Machine Translation Learning a Lexicon for a Fixed Image Vocabulary Miriam Miklofsky.](https://reader036.fdocuments.us/reader036/viewer/2022082819/56649f305503460f94c4aa4c/html5/thumbnails/3.jpg)
Dataset
Aligned bitext Annotated images Images with regions Unknown which region of image goes
with which word from text
![Page 4: Object Recognition a Machine Translation Learning a Lexicon for a Fixed Image Vocabulary Miriam Miklofsky.](https://reader036.fdocuments.us/reader036/viewer/2022082819/56649f305503460f94c4aa4c/html5/thumbnails/4.jpg)
EM
![Page 5: Object Recognition a Machine Translation Learning a Lexicon for a Fixed Image Vocabulary Miriam Miklofsky.](https://reader036.fdocuments.us/reader036/viewer/2022082819/56649f305503460f94c4aa4c/html5/thumbnails/5.jpg)
Clustering
K means clustering Vector quantize the image region
representation
Kullback-Leibler divergence Relative entropy Measure of difference of two
probability distributions over the same event space
![Page 6: Object Recognition a Machine Translation Learning a Lexicon for a Fixed Image Vocabulary Miriam Miklofsky.](https://reader036.fdocuments.us/reader036/viewer/2022082819/56649f305503460f94c4aa4c/html5/thumbnails/6.jpg)
Evaluation
Auto annotate images Quantize regions Use lexicon to determine word Annotate image with word
![Page 7: Object Recognition a Machine Translation Learning a Lexicon for a Fixed Image Vocabulary Miriam Miklofsky.](https://reader036.fdocuments.us/reader036/viewer/2022082819/56649f305503460f94c4aa4c/html5/thumbnails/7.jpg)
Results - Annotation
Base results 80 words of 371 word vocabulary
could be predicted
Retraining Similar results but some words with
higher recall and precision
![Page 8: Object Recognition a Machine Translation Learning a Lexicon for a Fixed Image Vocabulary Miriam Miklofsky.](https://reader036.fdocuments.us/reader036/viewer/2022082819/56649f305503460f94c4aa4c/html5/thumbnails/8.jpg)
Results(cont.)
Null probability Recall decreases Precision increases
Clustering of like words Recall values of clusters higher than
for single words
![Page 9: Object Recognition a Machine Translation Learning a Lexicon for a Fixed Image Vocabulary Miriam Miklofsky.](https://reader036.fdocuments.us/reader036/viewer/2022082819/56649f305503460f94c4aa4c/html5/thumbnails/9.jpg)
Results -Correspondence
Base results Some good words up to 70% correct
prediction
Null prediction Predict good words with greater
probability
Word clustering Prediction rate generally increases
![Page 10: Object Recognition a Machine Translation Learning a Lexicon for a Fixed Image Vocabulary Miriam Miklofsky.](https://reader036.fdocuments.us/reader036/viewer/2022082819/56649f305503460f94c4aa4c/html5/thumbnails/10.jpg)
Evaluation
Human evaluation Images viewed by hand Somewhat subjective
![Page 11: Object Recognition a Machine Translation Learning a Lexicon for a Fixed Image Vocabulary Miriam Miklofsky.](https://reader036.fdocuments.us/reader036/viewer/2022082819/56649f305503460f94c4aa4c/html5/thumbnails/11.jpg)
![Page 12: Object Recognition a Machine Translation Learning a Lexicon for a Fixed Image Vocabulary Miriam Miklofsky.](https://reader036.fdocuments.us/reader036/viewer/2022082819/56649f305503460f94c4aa4c/html5/thumbnails/12.jpg)
EM (cont.)
![Page 13: Object Recognition a Machine Translation Learning a Lexicon for a Fixed Image Vocabulary Miriam Miklofsky.](https://reader036.fdocuments.us/reader036/viewer/2022082819/56649f305503460f94c4aa4c/html5/thumbnails/13.jpg)
KL Divergence