Labeling images with a computer game
description
Transcript of Labeling images with a computer game
![Page 1: Labeling images with a computer game](https://reader033.fdocuments.us/reader033/viewer/2022051700/568161f7550346895dd22333/html5/thumbnails/1.jpg)
Center for Computational Analysis of Social and Organizational Systems
http://www.casos.cs.cmu.edu/
Labeling images with a computer game
By von Ahn & Dabbish
![Page 2: Labeling images with a computer game](https://reader033.fdocuments.us/reader033/viewer/2022051700/568161f7550346895dd22333/html5/thumbnails/2.jpg)
Motivation• ‘Our goal is ambitious: to label the majority of
images on the World Wide Web.’• Machine learning to identify and label image
components is generally ineffective.
Peter Landwehr 2
![Page 3: Labeling images with a computer game](https://reader033.fdocuments.us/reader033/viewer/2022051700/568161f7550346895dd22333/html5/thumbnails/3.jpg)
Past Work
Peter Landwehr 3
• Lempel & Soffer, 2002• Duygulu et al., 2002
![Page 4: Labeling images with a computer game](https://reader033.fdocuments.us/reader033/viewer/2022051700/568161f7550346895dd22333/html5/thumbnails/4.jpg)
Peter Landwehr 4
![Page 5: Labeling images with a computer game](https://reader033.fdocuments.us/reader033/viewer/2022051700/568161f7550346895dd22333/html5/thumbnails/5.jpg)
Implementation notes• Image reintroduction possibility• Age restriction possibility• Play alone possibility• Taboo threshold (X=1)• Image completion• Randomized partners• 350K images from random.bounceme.net• 73K word dictionary• ‘[T]he game doesn’t ask the players to describe
the image: all they are told is that they have to “think like each other” and type the same string.’
Peter Landwehr 5
![Page 6: Labeling images with a computer game](https://reader033.fdocuments.us/reader033/viewer/2022051700/568161f7550346895dd22333/html5/thumbnails/6.jpg)
August 9 – December 10, 2003
• 13,360 Players• 1,271,451 labels for 293,760 images• 80% of players returned at least once, 33 spent
50 hours playing• Mean = 3.89 labels/image/minute of play, std.
dev = 0.69.
Peter Landwehr 6
![Page 7: Labeling images with a computer game](https://reader033.fdocuments.us/reader033/viewer/2022051700/568161f7550346895dd22333/html5/thumbnails/7.jpg)
Peter Landwehr 7
Validation (1)
car dogman woman
stamp Witherspoonsmiling Aliascartoon green
![Page 8: Labeling images with a computer game](https://reader033.fdocuments.us/reader033/viewer/2022051700/568161f7550346895dd22333/html5/thumbnails/8.jpg)
1. ‘Please type the six individual words that you feel best describe the contents of this image. Type one word per line below; words should be less than 13 character’– For all of the images…
• at least 5 (83%) of the 6 labels produced by the game were entered by at least one participant.
• the three most common words entered by participants were contained among the labels produced by the game.
Peter Landwehr 8
15 participants aged 20-25, 20 images with >5 labels
Validation (2)
![Page 9: Labeling images with a computer game](https://reader033.fdocuments.us/reader033/viewer/2022051700/568161f7550346895dd22333/html5/thumbnails/9.jpg)
1. ‘How many of the words above would you use in describing this image to someone who couldn’t see it?’– Mean = 5.105, std. dev = 1.3087 (85% useful)
2. ‘How many of the words have nothing to do with the image (i.e., you don't understand why they are listed with this image)?’– Mean = 0.105, std. dev = 0.2529 (1.7%
useless)
Peter Landwehr 9
15 participants aged 20-25, 20 images with >5 labels
Validation (3)
![Page 10: Labeling images with a computer game](https://reader033.fdocuments.us/reader033/viewer/2022051700/568161f7550346895dd22333/html5/thumbnails/10.jpg)
Broad implications• ‘At this rate, 5,000 people playing the ESP game
24 hours a day would label all images on Google (425,000,000 images) [with one tag] in 31 days.’
• ‘…[O]ur main contribution stems from the way in which we attack the labeling problem. …[W]e have shown that it’s conceivable that a large-scale problem can be solved with a method that uses people playing on the Web. We’ve turned tedious work into something people want to do.’
Peter Landwehr 10
![Page 11: Labeling images with a computer game](https://reader033.fdocuments.us/reader033/viewer/2022051700/568161f7550346895dd22333/html5/thumbnails/11.jpg)
Peter Landwehr 11
‘The ESP game can be used, with minor modifications, to label sound or video clips (i.e., there is nothing inherent about
images).’
![Page 12: Labeling images with a computer game](https://reader033.fdocuments.us/reader033/viewer/2022051700/568161f7550346895dd22333/html5/thumbnails/12.jpg)
Peter Landwehr 12
‘Other problems that could be solved by having people play games include categorizing web pages into topics and monitoring security
cameras.’
![Page 13: Labeling images with a computer game](https://reader033.fdocuments.us/reader033/viewer/2022051700/568161f7550346895dd22333/html5/thumbnails/13.jpg)
More recent work
Peter Landwehr 13
Walsh & Golbeck, 2010
![Page 14: Labeling images with a computer game](https://reader033.fdocuments.us/reader033/viewer/2022051700/568161f7550346895dd22333/html5/thumbnails/14.jpg)
Recent Work• McGonigal, 2003• Law, von Ahn, Dannenberg, and Crawford, 2007• Hacker and von Ahn, 2009• Dong and Fu, 2010• Walsh and Golbeck, 2010
Peter Landwehr 14
![Page 15: Labeling images with a computer game](https://reader033.fdocuments.us/reader033/viewer/2022051700/568161f7550346895dd22333/html5/thumbnails/15.jpg)
Center for Computational Analysis of Social and Organizational Systems
http://www.casos.cs.cmu.edu/
Questions?