Michael Arbib & Laurent Itti: CS664 – USC, spring 2002. Lecture 6: Object Recognition 1 CS664,...
-
date post
21-Dec-2015 -
Category
Documents
-
view
218 -
download
0
Transcript of Michael Arbib & Laurent Itti: CS664 – USC, spring 2002. Lecture 6: Object Recognition 1 CS664,...
![Page 1: Michael Arbib & Laurent Itti: CS664 – USC, spring 2002. Lecture 6: Object Recognition 1 CS664, USC, Spring 2002 Lecture 6. Object Recognition Reading Assignments:](https://reader035.fdocuments.us/reader035/viewer/2022062421/56649d6a5503460f94a491e5/html5/thumbnails/1.jpg)
1Michael Arbib & Laurent Itti: CS664 – USC, spring 2002. Lecture 6: Object Recognition
CS664, USC, Spring 2002
Lecture 6. Object Recognition
Reading Assignments:
None
![Page 2: Michael Arbib & Laurent Itti: CS664 – USC, spring 2002. Lecture 6: Object Recognition 1 CS664, USC, Spring 2002 Lecture 6. Object Recognition Reading Assignments:](https://reader035.fdocuments.us/reader035/viewer/2022062421/56649d6a5503460f94a491e5/html5/thumbnails/2.jpg)
2Michael Arbib & Laurent Itti: CS664 – USC, spring 2002. Lecture 6: Object Recognition
![Page 3: Michael Arbib & Laurent Itti: CS664 – USC, spring 2002. Lecture 6: Object Recognition 1 CS664, USC, Spring 2002 Lecture 6. Object Recognition Reading Assignments:](https://reader035.fdocuments.us/reader035/viewer/2022062421/56649d6a5503460f94a491e5/html5/thumbnails/3.jpg)
3Michael Arbib & Laurent Itti: CS664 – USC, spring 2002. Lecture 6: Object Recognition
Four stages of representation (Marr, 1982)
1) pixel-based (light intensity)
2) primal sketch (discontinuities in intensity)
3) 2 ½ D sketch (oriented surfaces, relative depth between surfaces)
4) 3D model (shapes, spatial relationships, volumes)
problem: computationally intractable!
![Page 4: Michael Arbib & Laurent Itti: CS664 – USC, spring 2002. Lecture 6: Object Recognition 1 CS664, USC, Spring 2002 Lecture 6. Object Recognition Reading Assignments:](https://reader035.fdocuments.us/reader035/viewer/2022062421/56649d6a5503460f94a491e5/html5/thumbnails/4.jpg)
4Michael Arbib & Laurent Itti: CS664 – USC, spring 2002. Lecture 6: Object Recognition
![Page 5: Michael Arbib & Laurent Itti: CS664 – USC, spring 2002. Lecture 6: Object Recognition 1 CS664, USC, Spring 2002 Lecture 6. Object Recognition Reading Assignments:](https://reader035.fdocuments.us/reader035/viewer/2022062421/56649d6a5503460f94a491e5/html5/thumbnails/5.jpg)
5Michael Arbib & Laurent Itti: CS664 – USC, spring 2002. Lecture 6: Object Recognition
Challenges of Object Recognition
The binding problem: binding different features (color, orientation, etc) to yield a unitary percept. (see next slide)
Bottom-up vs. top-down processing: how
much is assumed top-down vs. extracted
from the image?
Perception vs. recognition vs. categorization: seeing an object vs. seeing is as something. Matching views of known objects to memory vs. matching a novel object to object categories in memory.
Viewpoint invariance: a major issue is to recognize objects irrespectively of the viewpoint from which we see them.
![Page 6: Michael Arbib & Laurent Itti: CS664 – USC, spring 2002. Lecture 6: Object Recognition 1 CS664, USC, Spring 2002 Lecture 6. Object Recognition Reading Assignments:](https://reader035.fdocuments.us/reader035/viewer/2022062421/56649d6a5503460f94a491e5/html5/thumbnails/6.jpg)
6Michael Arbib & Laurent Itti: CS664 – USC, spring 2002. Lecture 6: Object Recognition
![Page 7: Michael Arbib & Laurent Itti: CS664 – USC, spring 2002. Lecture 6: Object Recognition 1 CS664, USC, Spring 2002 Lecture 6. Object Recognition Reading Assignments:](https://reader035.fdocuments.us/reader035/viewer/2022062421/56649d6a5503460f94a491e5/html5/thumbnails/7.jpg)
7Michael Arbib & Laurent Itti: CS664 – USC, spring 2002. Lecture 6: Object Recognition
Viewpoint Invariance
Major problem for recognition.
Biederman & Gerhardstein, 1994:
We can recognize two views of an unfamiliar object as being the same object.
Thus, viewpoint invariance cannot only rely on matching views to memory.
![Page 8: Michael Arbib & Laurent Itti: CS664 – USC, spring 2002. Lecture 6: Object Recognition 1 CS664, USC, Spring 2002 Lecture 6. Object Recognition Reading Assignments:](https://reader035.fdocuments.us/reader035/viewer/2022062421/56649d6a5503460f94a491e5/html5/thumbnails/8.jpg)
8Michael Arbib & Laurent Itti: CS664 – USC, spring 2002. Lecture 6: Object Recognition
Models of Object Recognition
See Hummel, 1995, The Handbook of Brain Theory & Neural Networks
Direct Template Matching:
Processing hierarchy yields activation of view-tuned units.
A collection of view-tuned units is associated with one object.
View tuned units are built from V4-like units,
using sets of weights which differ for each object.
e.g., Poggio & Edelman, 1990; Riesenhuber & Poggio, 1999
![Page 9: Michael Arbib & Laurent Itti: CS664 – USC, spring 2002. Lecture 6: Object Recognition 1 CS664, USC, Spring 2002 Lecture 6. Object Recognition Reading Assignments:](https://reader035.fdocuments.us/reader035/viewer/2022062421/56649d6a5503460f94a491e5/html5/thumbnails/9.jpg)
9Michael Arbib & Laurent Itti: CS664 – USC, spring 2002. Lecture 6: Object Recognition
Computational Model of Object Recognition
(Riesenhuber and Poggio, 1999)
![Page 10: Michael Arbib & Laurent Itti: CS664 – USC, spring 2002. Lecture 6: Object Recognition 1 CS664, USC, Spring 2002 Lecture 6. Object Recognition Reading Assignments:](https://reader035.fdocuments.us/reader035/viewer/2022062421/56649d6a5503460f94a491e5/html5/thumbnails/10.jpg)
10Michael Arbib & Laurent Itti: CS664 – USC, spring 2002. Lecture 6: Object Recognition
the model neurons are
tuned for size
and 3D orientation
of object
![Page 11: Michael Arbib & Laurent Itti: CS664 – USC, spring 2002. Lecture 6: Object Recognition 1 CS664, USC, Spring 2002 Lecture 6. Object Recognition Reading Assignments:](https://reader035.fdocuments.us/reader035/viewer/2022062421/56649d6a5503460f94a491e5/html5/thumbnails/11.jpg)
11Michael Arbib & Laurent Itti: CS664 – USC, spring 2002. Lecture 6: Object Recognition
Models of Object Recognition
Hierarchical Template Matching:Image passed through layers of units with progressively more complex features at progressively less specific locations.
Hierarchical in that features at one stage are built from features at
earlier stages.
e.g., Fukushima & Miyake (1982)’s Neocognitron:
Several processing layers, comprising
simple (S) and complex (C) cells.
S-cells in one layer respond to conjunc-
tions of C-cells in previous layer.
C-cells in one layer are excited by
small neighborhoods of S-cells.
![Page 12: Michael Arbib & Laurent Itti: CS664 – USC, spring 2002. Lecture 6: Object Recognition 1 CS664, USC, Spring 2002 Lecture 6. Object Recognition Reading Assignments:](https://reader035.fdocuments.us/reader035/viewer/2022062421/56649d6a5503460f94a491e5/html5/thumbnails/12.jpg)
12Michael Arbib & Laurent Itti: CS664 – USC, spring 2002. Lecture 6: Object Recognition
Models of Object Recognition
Transform & Match:
First take care of rotation, translation, scale, etc. invariances.
Then recognize based on standardized pixel representation of objects.
e.g., Olshausen et al, 1993,
dynamic routing model
Template match: e.g., with
an associative memory based on
a Hopfield network.
![Page 13: Michael Arbib & Laurent Itti: CS664 – USC, spring 2002. Lecture 6: Object Recognition 1 CS664, USC, Spring 2002 Lecture 6. Object Recognition Reading Assignments:](https://reader035.fdocuments.us/reader035/viewer/2022062421/56649d6a5503460f94a491e5/html5/thumbnails/13.jpg)
13Michael Arbib & Laurent Itti: CS664 – USC, spring 2002. Lecture 6: Object Recognition
Recognition by Components
Structural approach to object recognition:
Biederman, 1987:
Complex objects are composed so simpler pieces
We can recognize a novel/unfamiliar object by parsing it in terms of its component pieces, then comparing the assemblage of pieces to those of known objects.
![Page 14: Michael Arbib & Laurent Itti: CS664 – USC, spring 2002. Lecture 6: Object Recognition 1 CS664, USC, Spring 2002 Lecture 6. Object Recognition Reading Assignments:](https://reader035.fdocuments.us/reader035/viewer/2022062421/56649d6a5503460f94a491e5/html5/thumbnails/14.jpg)
14Michael Arbib & Laurent Itti: CS664 – USC, spring 2002. Lecture 6: Object Recognition
Recognition by components (Biederman, 1987)
GEONS: geometric elements of which all objects are composed (cylinders, cones, etc). On the order of 30 different shapes.
Skips 2 ½ D sketch: Geons are directly recognized from edges, based on their nonaccidental properties (i.e., 3D features that are usually preserved by the projective imaging process).
![Page 15: Michael Arbib & Laurent Itti: CS664 – USC, spring 2002. Lecture 6: Object Recognition 1 CS664, USC, Spring 2002 Lecture 6. Object Recognition Reading Assignments:](https://reader035.fdocuments.us/reader035/viewer/2022062421/56649d6a5503460f94a491e5/html5/thumbnails/15.jpg)
15Michael Arbib & Laurent Itti: CS664 – USC, spring 2002. Lecture 6: Object Recognition
Basic Properties of GEONs
They are sufficiently different from each other to be easily discriminated
They are view-invariant (look identical from most viewpoints)
They are robust to noise (can be identified even with parts of image missing)
![Page 16: Michael Arbib & Laurent Itti: CS664 – USC, spring 2002. Lecture 6: Object Recognition 1 CS664, USC, Spring 2002 Lecture 6. Object Recognition Reading Assignments:](https://reader035.fdocuments.us/reader035/viewer/2022062421/56649d6a5503460f94a491e5/html5/thumbnails/16.jpg)
16Michael Arbib & Laurent Itti: CS664 – USC, spring 2002. Lecture 6: Object Recognition
Support for RBC: We can recognize partially occludedobjects easily if the occlusions do not obscure the setof geons which constitute the object.
![Page 17: Michael Arbib & Laurent Itti: CS664 – USC, spring 2002. Lecture 6: Object Recognition 1 CS664, USC, Spring 2002 Lecture 6. Object Recognition Reading Assignments:](https://reader035.fdocuments.us/reader035/viewer/2022062421/56649d6a5503460f94a491e5/html5/thumbnails/17.jpg)
17Michael Arbib & Laurent Itti: CS664 – USC, spring 2002. Lecture 6: Object Recognition
Potential difficulties
Edelman, 1997
A. Structural description notenough, also need metric info
B. Difficult to extract geonsfrom real images
C. Ambiguity in the structu-ral description: most oftenwe have several candidates
D. For some objects,deriving a structural repre-sentation can be difficult
![Page 18: Michael Arbib & Laurent Itti: CS664 – USC, spring 2002. Lecture 6: Object Recognition 1 CS664, USC, Spring 2002 Lecture 6. Object Recognition Reading Assignments:](https://reader035.fdocuments.us/reader035/viewer/2022062421/56649d6a5503460f94a491e5/html5/thumbnails/18.jpg)
18Michael Arbib & Laurent Itti: CS664 – USC, spring 2002. Lecture 6: Object Recognition
Geon Neurons in IT?
These are preferred
stimuli for some IT neurons.
![Page 19: Michael Arbib & Laurent Itti: CS664 – USC, spring 2002. Lecture 6: Object Recognition 1 CS664, USC, Spring 2002 Lecture 6. Object Recognition Reading Assignments:](https://reader035.fdocuments.us/reader035/viewer/2022062421/56649d6a5503460f94a491e5/html5/thumbnails/19.jpg)
19Michael Arbib & Laurent Itti: CS664 – USC, spring 2002. Lecture 6: Object Recognition
![Page 20: Michael Arbib & Laurent Itti: CS664 – USC, spring 2002. Lecture 6: Object Recognition 1 CS664, USC, Spring 2002 Lecture 6. Object Recognition Reading Assignments:](https://reader035.fdocuments.us/reader035/viewer/2022062421/56649d6a5503460f94a491e5/html5/thumbnails/20.jpg)
20Michael Arbib & Laurent Itti: CS664 – USC, spring 2002. Lecture 6: Object Recognition
Fusiform Face Area in Humans
![Page 21: Michael Arbib & Laurent Itti: CS664 – USC, spring 2002. Lecture 6: Object Recognition 1 CS664, USC, Spring 2002 Lecture 6. Object Recognition Reading Assignments:](https://reader035.fdocuments.us/reader035/viewer/2022062421/56649d6a5503460f94a491e5/html5/thumbnails/21.jpg)
21Michael Arbib & Laurent Itti: CS664 – USC, spring 2002. Lecture 6: Object Recognition
representation
• Image specific• Supports fine
discrimination• Noise tolerant
• Image invariant• Supports
generalization• Noise sensitive
visual processing
Standard View on Visual Processing
Tjan, 1999
![Page 22: Michael Arbib & Laurent Itti: CS664 – USC, spring 2002. Lecture 6: Object Recognition 1 CS664, USC, Spring 2002 Lecture 6. Object Recognition Reading Assignments:](https://reader035.fdocuments.us/reader035/viewer/2022062421/56649d6a5503460f94a491e5/html5/thumbnails/22.jpg)
22Michael Arbib & Laurent Itti: CS664 – USC, spring 2002. Lecture 6: Object Recognition
Early visual processing
Face
Place
Common objects ?(e.g. Kanwisher et al; Ishai et al)
primary visual processing
(Tjan, 1999) Multiple memory/decision sites
![Page 23: Michael Arbib & Laurent Itti: CS664 – USC, spring 2002. Lecture 6: Object Recognition 1 CS664, USC, Spring 2002 Lecture 6. Object Recognition Reading Assignments:](https://reader035.fdocuments.us/reader035/viewer/2022062421/56649d6a5503460f94a491e5/html5/thumbnails/23.jpg)
23Michael Arbib & Laurent Itti: CS664 – USC, spring 2002. Lecture 6: Object Recognition
primary visual processing
memory memory memory...“R1” “Ri” “Rn”
IndependentDecisions
1 i nDelays
Homunculus’Response the first arriving response
SensoryMemory
Tjan’s “Recognition by Anarchy”
![Page 24: Michael Arbib & Laurent Itti: CS664 – USC, spring 2002. Lecture 6: Object Recognition 1 CS664, USC, Spring 2002 Lecture 6. Object Recognition Reading Assignments:](https://reader035.fdocuments.us/reader035/viewer/2022062421/56649d6a5503460f94a491e5/html5/thumbnails/24.jpg)
24Michael Arbib & Laurent Itti: CS664 – USC, spring 2002. Lecture 6: Object Recognition
A toy visual system
Task: Identify letters from arbitrarypositions & orientations
“e”
![Page 25: Michael Arbib & Laurent Itti: CS664 – USC, spring 2002. Lecture 6: Object Recognition 1 CS664, USC, Spring 2002 Lecture 6. Object Recognition Reading Assignments:](https://reader035.fdocuments.us/reader035/viewer/2022062421/56649d6a5503460f94a491e5/html5/thumbnails/25.jpg)
25Michael Arbib & Laurent Itti: CS664 – USC, spring 2002. Lecture 6: Object Recognition
normalizeposition
normalizeorientation
Image
down-sampling
memory
![Page 26: Michael Arbib & Laurent Itti: CS664 – USC, spring 2002. Lecture 6: Object Recognition 1 CS664, USC, Spring 2002 Lecture 6. Object Recognition Reading Assignments:](https://reader035.fdocuments.us/reader035/viewer/2022062421/56649d6a5503460f94a491e5/html5/thumbnails/26.jpg)
26Michael Arbib & Laurent Itti: CS664 – USC, spring 2002. Lecture 6: Object Recognition
memory
normalizeposition
normalizeorientation
Image
down-sampling
memory memorySite 1 Site 2 Site 3
![Page 27: Michael Arbib & Laurent Itti: CS664 – USC, spring 2002. Lecture 6: Object Recognition 1 CS664, USC, Spring 2002 Lecture 6. Object Recognition Reading Assignments:](https://reader035.fdocuments.us/reader035/viewer/2022062421/56649d6a5503460f94a491e5/html5/thumbnails/27.jpg)
27Michael Arbib & Laurent Itti: CS664 – USC, spring 2002. Lecture 6: Object Recognition
Test stimuli:1) familiar (studied) views,
2) new positions, 3) new position & orientations
1800 {30%} 1500 {25%} 800 {20%} 450 {15%} 210 {10%}
Signal-to-Noise Ratio {RMS Contrast}
Study stimuli:5 orientations 20 positions at high SNR
![Page 28: Michael Arbib & Laurent Itti: CS664 – USC, spring 2002. Lecture 6: Object Recognition 1 CS664, USC, Spring 2002 Lecture 6. Object Recognition Reading Assignments:](https://reader035.fdocuments.us/reader035/viewer/2022062421/56649d6a5503460f94a491e5/html5/thumbnails/28.jpg)
28Michael Arbib & Laurent Itti: CS664 – USC, spring 2002. Lecture 6: Object Recognition
raw image
norm. pos.
norm. ori.
Site 3
Site 2
Site 1
Processing speed for each recognition module dependson recognition difficulty by that module.
![Page 29: Michael Arbib & Laurent Itti: CS664 – USC, spring 2002. Lecture 6: Object Recognition 1 CS664, USC, Spring 2002 Lecture 6. Object Recognition Reading Assignments:](https://reader035.fdocuments.us/reader035/viewer/2022062421/56649d6a5503460f94a491e5/html5/thumbnails/29.jpg)
29Michael Arbib & Laurent Itti: CS664 – USC, spring 2002. Lecture 6: Object Recognition
0
0.2
0.4
0.6
0.8
1
10 100
0
0.2
0.4
0.6
0.8
1
10 100
0
0.2
0.4
0.6
0.8
1
10 100
Pro
port
ion
Cor
rect
Contrast (%)
Familiar views Novel positionsNovel positions& orientations
raw image
norm. pos.
norm. ori.
Site 3
Site 2
Site 1
![Page 30: Michael Arbib & Laurent Itti: CS664 – USC, spring 2002. Lecture 6: Object Recognition 1 CS664, USC, Spring 2002 Lecture 6. Object Recognition Reading Assignments:](https://reader035.fdocuments.us/reader035/viewer/2022062421/56649d6a5503460f94a491e5/html5/thumbnails/30.jpg)
30Michael Arbib & Laurent Itti: CS664 – USC, spring 2002. Lecture 6: Object Recognition
0
0.2
0.4
0.6
0.8
1
10 100
0
0.2
0.4
0.6
0.8
1
10 100
0
0.2
0.4
0.6
0.8
1
10 100
Novel positionsNovel positions& orientations
Pro
port
ion
Cor
rect
raw image
norm. pos.
norm. ori.
Site 3
Site 2
Site 1
Contrast (%)
Familiar views
Black curve: full model in which recognition is based on the fastest of the responses from the three stages.
![Page 31: Michael Arbib & Laurent Itti: CS664 – USC, spring 2002. Lecture 6: Object Recognition 1 CS664, USC, Spring 2002 Lecture 6. Object Recognition Reading Assignments:](https://reader035.fdocuments.us/reader035/viewer/2022062421/56649d6a5503460f94a491e5/html5/thumbnails/31.jpg)
31Michael Arbib & Laurent Itti: CS664 – USC, spring 2002. Lecture 6: Object Recognition