A SAMPLE RECOGNITION PROBLEM

Joseph Tighe University of North Carolina at Chapel Hill

What is recognition?

What is recognition?

What is recognition?

What is recognition?

He et al. (2004), Hoiem et al. (2005), Shotton et al. (2006, 2008, 2009), Verbeek and Triggs (2007), Rabinovich et al. (2007), Galleguillos et al. (2008), Gould et al. (2009), etc.

Figure from Shotton et al. (2009)

What is recognition? Have some entity

photo bounding box in photo pixel

Assign semantic meaning scene type, tags, sentence object label, action semantic class, material, geometric

orientation

Our Problem Entity: Images Goal: Assign 1 of 4 labels (airplane, car,

face, motorbike

Finding Similar Images

Ocean

Open Field

Highway

Street

Forest

Mountain

Inner City

Tall BuildingWhat is depicted in this image?

Which image is most similar?

Then assign the label from the most similar image

Pixels are a bad measure of similarity

Most similar according to pixel distance Most similar according to “Bag of Words”

Origin of the Bag of Words model

Orderless document representation: frequencies of words from a dictionary Salton & McGill (1983)

US Presidential Speeches Tag Cloudhttp://chir.ag/phernalia/preztags/

What are words for an image?

Wing Tail

WheelBuildingPropeller

Wing TailWheelBuildingPropeller Jet Engine

Wing TailWheelBuildingPropeller Jet Engine

Wing TailWheelBuildingPropeller Jet Engine

But where do the words come from?

Then where does the dictionary come from?

Example Dictionary

Source: B. Leibe

Another dictionary

………

…

Source: B. Leibe

Fei-Fei et al. 2005

Outline of the Bag of Words method

Divide the image into patches Assign a “word” for each patch Count the number of occurrences of

each “word” in the image

Matlab Demo