Learning Color Names from Real-World Images
1
0.88 0.89 0.9 0.91 0.92 0.93 0.94 0.95 0.96 10 30 50 70 90 Learning Color Names from Real-World Images Joost van de Weijer, Cordelia Schmid, Jakob Verbeek Lear Team, INRIA Grenoble, France http://lear.inrialpes.fr/ people/vandeweijer/ Basic Color Terms Pixel Classification Chip-Based vs. Real- World ABSTRACT Within a computer vision context color naming is the action of assigning linguistic color labels to image pixels. In general, research on color naming applies the following paradigm: a collection of color chips is labeled with color names within a well-defined experimental setup by multiple test subjects. The collected data set is subsequently used to label RGB values in real-world images with a color name. In this research we propose to learn color names from real-world images. We avoid test subjects by using Google Image to collect a data set. Due to limitations of Google Image this data set contains a substantial quantity of wrongly labeled data. The color names are learned using a PLSA model adapted to this task. Experimental results show that color names learned from real-world images significantly outperform color names learned from labeled color chips on retrieval and classification. black Images retrieved with Google image Color Chips named by human test subjects blue orange green brown white purple Results The English language consists of 11 basic color terms. These basic color terms are defined by the linguistics Berlin and Kay as those color names: • which are applied to diverse classes of objects. • whose meaning is not subsumable under one of the other basic color terms. • which are used consistently and with consensus by most speakers of the language. Conclusions Learning Color Names … … PLSA-bg Research Problem • Is it possible to learn color names from weakly labeled images retrieved from Google Image on the query of ‘color name+”color” ‘, e.g. “red+color” ? • How do learned color names compare to chip-based color names, i.e. the traditional way to compute color names from color chips which are labeled by multiple test subjects in a well-defined experimental setup ? white black red green yellow blue brown purple pink orange grey Development color names in languages: • Results indicate that color names can be learned from weakly labeled images returned from Google Image search. • Results show that color names returned from Google image outperform color names derived from human-named color chips. Pixel classification results improve by 17 % compared to chip-based color naming. • We illustrate that color naming based on Google images is flexible in the set of basic color terms. Labeled input images: Overview learning approach: LAB- histogram representat ion: … … Color name distribution : yellow yellow red yellow red Color names are learned with an adapted Probabilistic Latent Semantic Analysis (PLSA-bg). Flexibility Learning color names from Google has the advantage that the set of basic color terms can easily be varied. Pixels are assigned to their most probable color name. Results are given in percentage of pixels correctly classified. Data Sets Image Retrieval m ethod train-set cars shoes dressespottery overall chip-based CVC 39 60 62 50 53 SVM Google 45 61 68 56 62 P LS A Google 48 69 71 62 63 P LS A -bg Google 51 71 81 66 67 P LS A -bg G oogle+Ebay 53 73 84 71 70 m ethod train-set cars shoes dressespottery overall chip-based CVC 88 93 94 91 92 SVM Google 91 96 96 91 94 P LS A Google 89 95 94 92 93 P LS A -bg Google 92 97 99 95 96 P LS A -bg G oogle+Ebay 92 97 100 94 96 Google set: 1100 images queried with Google image, containing 100 images per color name. Color Chip set: 387 labeled color patches (CVC lab) pixel classification Image are retrieved (e.g. retrieve ‘brown shoes’) based on the percentage of pixels which has been assigned to the color name. EER are given. retrieval results for ’orange dresses’ pixel classification input-image PLSA-bg chip-based input-image PLSA-bg SVM 0.45 0.5 0.55 0.6 0.65 0.7 10 30 50 70 90 Ebay set: 440 images of four categories labeled with color names collected from the Ebay auction site. http:// lear.inrialpes.fr/ data - Added the color names beige, olive and violet. - Replaced blue by the Russian blues goluboi and siniy. number of images PLSA-bg PLSA-bg number of images PLSA-bg chip-based rank (19) (12) 1 2 3 4 5 6 EER class.
description
Development color names in languages:. purple. white. green. pink. red. blue. brown. orange. yellow. black. grey. Learning Color Names from Real-World Images. Joost van de Weijer, Cordelia Schmid, Jakob Verbeek. - PowerPoint PPT Presentation
Transcript of Learning Color Names from Real-World Images
0.88
0.89
0.9
0.91
0.92
0.93
0.94
0.95
0.96
10 30 50 70 90
Learning Color Names from Real-World ImagesJoost van de Weijer, Cordelia Schmid, Jakob VerbeekLear Team, INRIA Grenoble, Francehttp://lear.inrialpes.fr/ people/vandeweijer/
Basic Color Terms
Pixel Classification
Chip-Based vs. Real-World
ABSTRACT Within a computer vision context color naming is the action of assigning linguistic color labels to image pixels. In general, research on color naming applies the following paradigm: a collection of color chips is labeled with color names within a well-defined experimental setup by multiple test subjects. The collected data set is subsequently used to label RGB values in real-world images with a color name. In this research we propose to learn color names from real-world images. We avoid test subjects by using Google Image to collect a data set. Due to limitations of Google Image this data set contains a substantial quantity of wrongly labeled data. The color names are learned using a PLSA model adapted to this task. Experimental results show that color names learned from real-world images significantly outperform color names learned from labeled color chips on retrieval and classification.
black
Images retrieved with Google image
Color Chips named by human test subjects
blue orangegreenbrown whitepurple
ResultsThe English language consists of 11 basic color terms. These basic color terms are defined by the linguistics Berlin and Kay as those color names:
• which are applied to diverse classes of objects.
• whose meaning is not subsumable under one of the other basic color terms.
• which are used consistently and with consensus by most speakers of the language.
Conclusions
Learning Color Names
…
…
PLSA-bg
Research Problem• Is it possible to learn color names from weakly labeled images retrieved from Google Image on the query of ‘color name+”color” ‘, e.g. “red+color” ?
• How do learned color names compare to chip-based color names, i.e. the traditional way to compute color names from color chips which are labeled by multiple test subjects in a well-defined experimental setup ?
white
black
red
green
yellow
blue brown
purple
pink
orange
grey
Development color names in languages:
• Results indicate that color names can be learned from weakly labeled images returned from Google Image search.
• Results show that color names returned from Google image outperform color names derived from human-named color chips. Pixel classification results improve by 17 % compared to chip-based color naming.
• We illustrate that color naming based on Google images is flexible in the set of basic color terms.
Labeled input images:
Overview learning approach:
LAB-histogram representation:
……Color name distribution:
yellow yellow red
yellow red
Color names are learned with an adapted Probabilistic Latent Semantic Analysis (PLSA-bg).
FlexibilityLearning color names from Google has the advantage that the set of basic color terms can easily be varied.
Pixels are assigned to their most probable color name. Results are given in percentage of pixels correctly classified.
Data Sets
Image Retrieval
method train-set cars shoes dresses pottery overall
chip-based CVC 39 60 62 50 53SVM Google 45 61 68 56 62PLSA Google 48 69 71 62 63PLSA-bg Google 51 71 81 66 67PLSA-bg Google+Ebay 53 73 84 71 70
method train-set cars shoes dresses pottery overall
chip-based CVC 88 93 94 91 92SVM Google 91 96 96 91 94PLSA Google 89 95 94 92 93PLSA-bg Google 92 97 99 95 96PLSA-bg Google+Ebay 92 97 100 94 96
Google set: 1100 images queried with Google image, containing 100 images per color name.
Color Chip set: 387 labeled color patches (CVC lab)
pixel classification
Image are retrieved (e.g. retrieve ‘brown shoes’) based on the percentage of pixels which has been assigned to the color name. EER are given.
retrieval results for ’orange dresses’
pixel classification
input-image
PLSA-bg
chip-based
input-image
PLSA-bg
SVM
0.45
0.5
0.55
0.6
0.65
0.7
10 30 50 70 90
Ebay set: 440 images of four categories labeled with color names collected from the Ebay auction site.
http://lear.inrialpes.fr/data
- Added the color names beige, olive and violet. - Replaced blue by the Russian blues goluboi and siniy.
number of images
PLSA-bg
PLSA-bg
number of images
PLS
A-b
gch
ip-b
ased
rank
(19)(12)
1 2 3 4 5 6
EE
R
clas
s.
![Color Names - Inria · 2021. 1. 16. · COLOR NAMES FROM CALIBRATED DATA 15 [34]. According to this new theory, the organization of the color naming systems in dif ferent languages](https://static.fdocuments.us/doc/165x107/60f924746f562a5a6e5f1989/color-names-inria-2021-1-16-color-names-from-calibrated-data-15-34-according.jpg)
