9.71 Functional MRI of High-Level Vision Fall 2007 For ......(different kinds of agnosias can give...
Transcript of 9.71 Functional MRI of High-Level Vision Fall 2007 For ......(different kinds of agnosias can give...
![Page 1: 9.71 Functional MRI of High-Level Vision Fall 2007 For ......(different kinds of agnosias can give us further dissociations…) An example…. What does object recognition entail exactly,](https://reader033.fdocuments.us/reader033/viewer/2022043007/5f94c5197f7c7c0e5f38d170/html5/thumbnails/1.jpg)
MIT OpenCourseWare http://ocw.mit.edu
9.71 Functional MRI of High-Level Vision Fall 2007
For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms.
![Page 2: 9.71 Functional MRI of High-Level Vision Fall 2007 For ......(different kinds of agnosias can give us further dissociations…) An example…. What does object recognition entail exactly,](https://reader033.fdocuments.us/reader033/viewer/2022043007/5f94c5197f7c7c0e5f38d170/html5/thumbnails/2.jpg)
9.71 Lecture 3 Sept 20The Problem of Object Recognition
and The Lateral Occipital Complex (LOC)
Outline for Today
I. Demo of an fMRI scan II. Lecture: The Problem of Object Recognition:
1. Why study it, what is entailed computationally 2. The Lateral Occipital Complex (LOC)
What does LOC do? What does LOC representation?
III. A few tips on doing presentations. IV. If time: Discussion of B&Z]
![Page 3: 9.71 Functional MRI of High-Level Vision Fall 2007 For ......(different kinds of agnosias can give us further dissociations…) An example…. What does object recognition entail exactly,](https://reader033.fdocuments.us/reader033/viewer/2022043007/5f94c5197f7c7c0e5f38d170/html5/thumbnails/3.jpg)
Why Study Object Recognition?
OR is an important problem: Critical for survival We are very good at it A distinct domain of cognition
![Page 4: 9.71 Functional MRI of High-Level Vision Fall 2007 For ......(different kinds of agnosias can give us further dissociations…) An example…. What does object recognition entail exactly,](https://reader033.fdocuments.us/reader033/viewer/2022043007/5f94c5197f7c7c0e5f38d170/html5/thumbnails/4.jpg)
Object Recognition: A Distinct Domain of Cognition
Visual Agnosia: specific deficit in visual object recognition without impaired visual acuity without impaired object recognition by touch, sound, smell
The fact that visual OR can be selectively lost in brain damage implies that it is a distinct domain of cognition, with its own special neural hardware, distinct from low-level visual processing, and from knowledge of the meanings and names of objects.
(different kinds of agnosias can give us further dissociations…) An example….
![Page 5: 9.71 Functional MRI of High-Level Vision Fall 2007 For ......(different kinds of agnosias can give us further dissociations…) An example…. What does object recognition entail exactly,](https://reader033.fdocuments.us/reader033/viewer/2022043007/5f94c5197f7c7c0e5f38d170/html5/thumbnails/5.jpg)
What does object recognition entail exactly, and what is to be explained?
![Page 6: 9.71 Functional MRI of High-Level Vision Fall 2007 For ......(different kinds of agnosias can give us further dissociations…) An example…. What does object recognition entail exactly,](https://reader033.fdocuments.us/reader033/viewer/2022043007/5f94c5197f7c7c0e5f38d170/html5/thumbnails/6.jpg)
Object Recognition as Matching to MemoryVisual LTM:
World/ Eye/ thousands of Visual field Retinal image Object Recognition.
Stored shapes
Photo courtesy of Nick Devenish.
Matching
• • •
![Page 7: 9.71 Functional MRI of High-Level Vision Fall 2007 For ......(different kinds of agnosias can give us further dissociations…) An example…. What does object recognition entail exactly,](https://reader033.fdocuments.us/reader033/viewer/2022043007/5f94c5197f7c7c0e5f38d170/html5/thumbnails/7.jpg)
A Theory of Object RecognitionWould have to Specify:
World/ Eye/Visual field Retinal image
Visual Recognition
Photo courtesyof Nick Devenish.
a. the nature of the stored visual representations in LTM b. the nature of the intermediate representations
c. a computational account of how each intermediate representation can be derived from the previous one
d. a determination of whether the answers to a-d are different for different kinds of objects
Visual LTM:thousands ofStored shapes
• • •
![Page 8: 9.71 Functional MRI of High-Level Vision Fall 2007 For ......(different kinds of agnosias can give us further dissociations…) An example…. What does object recognition entail exactly,](https://reader033.fdocuments.us/reader033/viewer/2022043007/5f94c5197f7c7c0e5f38d170/html5/thumbnails/8.jpg)
Kinds of Cues Available for Visual Object Recognition
a. Characteristic motion (e.g. a fly).
b. Color/texture (e.g., lawn, ocean, beach)
c. Stored knowledge plus minimal cues (e.g. I left newspaper on dining table, that’s what that blob must be).
d. The most important cue: SHAPE! (which is the primary focus of most theories of object recognition)
![Page 9: 9.71 Functional MRI of High-Level Vision Fall 2007 For ......(different kinds of agnosias can give us further dissociations…) An example…. What does object recognition entail exactly,](https://reader033.fdocuments.us/reader033/viewer/2022043007/5f94c5197f7c7c0e5f38d170/html5/thumbnails/9.jpg)
What Makes Object Recognition (by Shape) Hard?World/ Eye/Visual field Retinal image
Photo courtesy Inverse optics problemof Nick Devenish.
1. A single image can be cast by many different 3D objects Retinal
image (Projection)
Ambiguous image (Inverse optics problem)
p
Figure by MIT OpenCourseWare.
![Page 10: 9.71 Functional MRI of High-Level Vision Fall 2007 For ......(different kinds of agnosias can give us further dissociations…) An example…. What does object recognition entail exactly,](https://reader033.fdocuments.us/reader033/viewer/2022043007/5f94c5197f7c7c0e5f38d170/html5/thumbnails/10.jpg)
What Makes Object Recognition Hard?World/ Visual field
Photo courtesyof Nick Devenish.
2. A single Object can cast many Different retinal images that differ in….
Eye/Retinal image
Viewpoint 1. A single image can be cast byDistance/size many differentOcclusion 3D objects ….
Configuration
Lighting, etc….
![Page 11: 9.71 Functional MRI of High-Level Vision Fall 2007 For ......(different kinds of agnosias can give us further dissociations…) An example…. What does object recognition entail exactly,](https://reader033.fdocuments.us/reader033/viewer/2022043007/5f94c5197f7c7c0e5f38d170/html5/thumbnails/11.jpg)
The Problem of Object Recognition
Given aRetinal What is it?Image such As this:
![Page 12: 9.71 Functional MRI of High-Level Vision Fall 2007 For ......(different kinds of agnosias can give us further dissociations…) An example…. What does object recognition entail exactly,](https://reader033.fdocuments.us/reader033/viewer/2022043007/5f94c5197f7c7c0e5f38d170/html5/thumbnails/12.jpg)
Two main Challenges:Specificity: Appreciating the distinction
Invariance/Tolerance: between different categories.Generalizing across changesin size, orientation, lighting,etc. to realize these images are all of the same thing:
Photo courtesyof Nick Devenish.
=
![Page 13: 9.71 Functional MRI of High-Level Vision Fall 2007 For ......(different kinds of agnosias can give us further dissociations…) An example…. What does object recognition entail exactly,](https://reader033.fdocuments.us/reader033/viewer/2022043007/5f94c5197f7c7c0e5f38d170/html5/thumbnails/13.jpg)
How do we solve this problem? Options:
1. Inverse Optics: Extract an abstract representation of 3D shape “invariant” to these image changes.
2. Association: Store each possible Version of an object. Brute force.
Rabbit
• an “ill-posed” problem.
• That’s a lot to store! • What about novel
views? • Alignment
3. Other intermediate descriptors e.g. image fragments, parts….
Photo courtesy of Nick Devenish.
![Page 14: 9.71 Functional MRI of High-Level Vision Fall 2007 For ......(different kinds of agnosias can give us further dissociations…) An example…. What does object recognition entail exactly,](https://reader033.fdocuments.us/reader033/viewer/2022043007/5f94c5197f7c7c0e5f38d170/html5/thumbnails/14.jpg)
A Theory of Object RecognitionWould have to Specify:
World/ Eye/Visual field Retinal image
Visual Recognition
Photo courtesyof Nick Devenish.
How can brain imaging help?
a. the nature of the stored visual representations in LTM b. the nature of the intermediate representations
c. a computational account of how each intermediate representation can be derived from the previous one
d. a determination of whether the answers to a-d are different for different kinds of objects
Visual LTM:thousands ofStored shapes
• • •
![Page 15: 9.71 Functional MRI of High-Level Vision Fall 2007 For ......(different kinds of agnosias can give us further dissociations…) An example…. What does object recognition entail exactly,](https://reader033.fdocuments.us/reader033/viewer/2022043007/5f94c5197f7c7c0e5f38d170/html5/thumbnails/15.jpg)
A Theory of Object RecognitionWould have to Specify:
World/ Eye/Visual field Retinal image
Visual Recognition
Photo courtesyof Nick Devenish.
How can brain imaging help?
a. the nature of the stored visual representations in LTM b. the nature of the intermediate representations
c. a computational account of how each intermediate representation can be derived from the previous one
d. a determination of whether the answers to a-d are different for different kinds of objects
Visual LTM:thousands ofStored shapes
• • •
![Page 16: 9.71 Functional MRI of High-Level Vision Fall 2007 For ......(different kinds of agnosias can give us further dissociations…) An example…. What does object recognition entail exactly,](https://reader033.fdocuments.us/reader033/viewer/2022043007/5f94c5197f7c7c0e5f38d170/html5/thumbnails/16.jpg)
Brain Regions Involved in Visual Cognition
Drawing Modified from Allison et al (94)
Figure by MIT OpenCourseWare. After Allison, 1994.
![Page 17: 9.71 Functional MRI of High-Level Vision Fall 2007 For ......(different kinds of agnosias can give us further dissociations…) An example…. What does object recognition entail exactly,](https://reader033.fdocuments.us/reader033/viewer/2022043007/5f94c5197f7c7c0e5f38d170/html5/thumbnails/17.jpg)
The Lateral Occipital Complex (LOC): Cortical Regions Involved in Processing Object Shape
I Malach et al (1995), “LO”
and: > Courtesy of National Academy of Sciences, U. S. A. Used with permission. Source: Malach, R. et. al. "Object-related activity revealed by functional magnetic resonance imaging in human occipital cortex." Proc. Natl. Acad. Sci. 92 (1995): 8135-8139. Copyright © 1995, National Academy of Sciences, U.S.A.
II Kanwisher et al (1996) - a similar region
and >:
Figure by MIT OpenCourseWare.
![Page 18: 9.71 Functional MRI of High-Level Vision Fall 2007 For ......(different kinds of agnosias can give us further dissociations…) An example…. What does object recognition entail exactly,](https://reader033.fdocuments.us/reader033/viewer/2022043007/5f94c5197f7c7c0e5f38d170/html5/thumbnails/18.jpg)
10-4
10-4
10-10
isphere
iew
left hemisphere
Object-Selective Regions in the Human Brain:LOC in one Subject
subject: NT
10-4
10-10
right hem
lateral v
ventral view Kalanit Grill-Spector
What does this region do?
> Photo courtesy of Enid Yu.
Photo courtesy of caspermoller.
Courtesy Elsevier, Inc., http://www.sciencedirect.com. Used with permission.
![Page 19: 9.71 Functional MRI of High-Level Vision Fall 2007 For ......(different kinds of agnosias can give us further dissociations…) An example…. What does object recognition entail exactly,](https://reader033.fdocuments.us/reader033/viewer/2022043007/5f94c5197f7c7c0e5f38d170/html5/thumbnails/19.jpg)
Components of Object Recognitionfrom Common Sence (plus a speck of data)
meaning
actionEye/Retinal image LOC?
Early sensory processing
~shape coding
“recognition”/ matching to stored reps.
word/ name
LOC: familiarity/meaning/name apparently not importantnot processing very low-level information
(Is this what is messed up in the “lock guy”?)
Is this region necessary for perceiving shape?fMRI can’t tell you this, but……
![Page 20: 9.71 Functional MRI of High-Level Vision Fall 2007 For ......(different kinds of agnosias can give us further dissociations…) An example…. What does object recognition entail exactly,](https://reader033.fdocuments.us/reader033/viewer/2022043007/5f94c5197f7c7c0e5f38d170/html5/thumbnails/20.jpg)
Patient DF: no form visual perception
Patient DF has a “ventral stream” lesion Object agnosia (a diff. Kind from the “lock guy”)
• Cannot identify line drawings of common objects • Cannot copy line drawings • Can draw from memory as long as she doesn’t lift hand from paper
Image removed due to copyright restrictions.See Figure 10.3 (p.320) in Goodale, M. A., and G. K. Humphrey, “Separate Visual Systems for Action and Perception.” Blackwell Handbook of Perception. Edited by E. Bruce Goldstein. New York, NY: Wiley-Blackwell, 2001. [Preview this content in Google Books.]
Slide adapted from Jody Culham
![Page 21: 9.71 Functional MRI of High-Level Vision Fall 2007 For ......(different kinds of agnosias can give us further dissociations…) An example…. What does object recognition entail exactly,](https://reader033.fdocuments.us/reader033/viewer/2022043007/5f94c5197f7c7c0e5f38d170/html5/thumbnails/21.jpg)
LOC in Normals and Lesion site in DF
Image removed due to copyright restrictions.Fig 4b in James et al. Brain 126, no. 11 (2003): 2463-2475.View this figure at http://brain.oxfordjournals.org/cgi/content/full/126/11/2463.
Apparently, LOC is necessaryfor object recognition. James, Culham, Humphrey, Milner, & Goodale (2003)
![Page 22: 9.71 Functional MRI of High-Level Vision Fall 2007 For ......(different kinds of agnosias can give us further dissociations…) An example…. What does object recognition entail exactly,](https://reader033.fdocuments.us/reader033/viewer/2022043007/5f94c5197f7c7c0e5f38d170/html5/thumbnails/22.jpg)
Characterizing Representations and Processes in the LOC �
Are shape representations in LOC independent of how shape is represented, i.e. independent of
form-cues (motion, luminance, texture)? contours?
Independent of changes in the size, position, viewpoint, etc?
Cool method: fMRI adaptation
![Page 23: 9.71 Functional MRI of High-Level Vision Fall 2007 For ......(different kinds of agnosias can give us further dissociations…) An example…. What does object recognition entail exactly,](https://reader033.fdocuments.us/reader033/viewer/2022043007/5f94c5197f7c7c0e5f38d170/html5/thumbnails/23.jpg)
Are common regions involved in processing object structure independent of the cues defining the object’s shape (e.g. line
contours, surface shading)?
Grayscale Grayscale Line Drawings Line DrawingsIntact Scrambled Intact Scrambled
Kourtzi & Kanwisher, 2000Courtesy of Society for Neuroscience. Used with permission.
![Page 24: 9.71 Functional MRI of High-Level Vision Fall 2007 For ......(different kinds of agnosias can give us further dissociations…) An example…. What does object recognition entail exactly,](https://reader033.fdocuments.us/reader033/viewer/2022043007/5f94c5197f7c7c0e5f38d170/html5/thumbnails/24.jpg)
Procedure
Each Scan:0 5:36 :16s :32s :48s
Line
Scr
ambl
ed
Line
Inta
ct
Gra
y Sc
ram
bled
Gra
y In
tact
Gra
y In
tact
Gra
y In
tact
Gra
y Sc
ram
bled
Gra
y Sc
ram
bled
Gra
y Sc
ram
bled
Line
Inta
ct
Line
Inta
ct
Line
Scr
ambl
ed
Line
Scr
ambl
ed
Line
Scr
ambl
ed
Line
Inta
ct
Gra
y In
tact
Each Epoch: (20 pictures per epoch)
250 msec Tasks: Passive Viewin Kourtzi & Kanwisher, 2000
550 msec
250 msec
Courtesy of Society for Neuroscience. Used with permission.
g
![Page 25: 9.71 Functional MRI of High-Level Vision Fall 2007 For ......(different kinds of agnosias can give us further dissociations…) An example…. What does object recognition entail exactly,](https://reader033.fdocuments.us/reader033/viewer/2022043007/5f94c5197f7c7c0e5f38d170/html5/thumbnails/25.jpg)
Activations in one subject for:
a. Intact versus Scrambled Grayscale images
b. Intact versus Scrambled Line Drawings
Kourtzi & Kanwisher, 2000Courtesy of Society for Neuroscience. Used with permission.
![Page 26: 9.71 Functional MRI of High-Level Vision Fall 2007 For ......(different kinds of agnosias can give us further dissociations…) An example…. What does object recognition entail exactly,](https://reader033.fdocuments.us/reader033/viewer/2022043007/5f94c5197f7c7c0e5f38d170/html5/thumbnails/26.jpg)
Experiment 1: ResultsActivation Map for Intact-Scrambled Images averaged across subjectsA big chunk of cortex is more active for intact than scrambled shapes.
% S
igna
l Cha
nge
in L
O
0
0.4
0.8 Grayscale
Intact
Line Drawings Intact
Courtesy of Society for Neuroscience. Used with permission.
![Page 27: 9.71 Functional MRI of High-Level Vision Fall 2007 For ......(different kinds of agnosias can give us further dissociations…) An example…. What does object recognition entail exactly,](https://reader033.fdocuments.us/reader033/viewer/2022043007/5f94c5197f7c7c0e5f38d170/html5/thumbnails/27.jpg)
Are common regions involved in processing object structure independent of the cues defining the object’s shape (e.g. line
contours, surface shading)? YES!
Are the same neurons responsive to photos and drawings?
Does it respond to shapes defined in other ways?
Grayscale Intact
Grayscale Scrambled
Line Drawings Intact
Line Drawings Scrambled
Kourtzi & Kanwisher, 2000Courtesy of Society for Neuroscience. Used with permission.
![Page 28: 9.71 Functional MRI of High-Level Vision Fall 2007 For ......(different kinds of agnosias can give us further dissociations…) An example…. What does object recognition entail exactly,](https://reader033.fdocuments.us/reader033/viewer/2022043007/5f94c5197f7c7c0e5f38d170/html5/thumbnails/28.jpg)
Objects from Motion Experiment
15 different images per blockpresented rate: 0.5Hz Grill-Spector et al. , Neuron 1998
Are object-selective regions preferentially activated by objects fromLuminance? Motion? Texture?
Courtesy Elsevier, Inc., http://www.sciencedirect.com. Used with permission.
![Page 29: 9.71 Functional MRI of High-Level Vision Fall 2007 For ......(different kinds of agnosias can give us further dissociations…) An example…. What does object recognition entail exactly,](https://reader033.fdocuments.us/reader033/viewer/2022043007/5f94c5197f7c7c0e5f38d170/html5/thumbnails/29.jpg)
Objects from Motion ExperimentDefine object selective regions: OFL > GFL
OFM G-MTN OFM
0
3
6
% si
gnal
0 50 100 150 200 250 300 time (s)
n=9
GFLOFL
left hemi
lateral
ventral
OFL > GFL
Grill-Spector et al. , Neuron 1998Courtesy Elsevier, Inc., http://www.sciencedirect.com. Used with permission.
![Page 30: 9.71 Functional MRI of High-Level Vision Fall 2007 For ......(different kinds of agnosias can give us further dissociations…) An example…. What does object recognition entail exactly,](https://reader033.fdocuments.us/reader033/viewer/2022043007/5f94c5197f7c7c0e5f38d170/html5/thumbnails/30.jpg)
Objects from Motion ExperimentDefine object selective regions: OFL > GFL
OFM GFLOFLMTN OFM
0
3
6
% si
gnal
0 50 100 150 200 250 300 time (s)
n=9
left hemi
lateral
ventral
OFL > GFL
Time course from object-selective regions: LOC
Grill-Spector et al. , Neuron 1998 Courtesy Elsevier, Inc., http://www.sciencedirect.com. Used with permission.
![Page 31: 9.71 Functional MRI of High-Level Vision Fall 2007 For ......(different kinds of agnosias can give us further dissociations…) An example…. What does object recognition entail exactly,](https://reader033.fdocuments.us/reader033/viewer/2022043007/5f94c5197f7c7c0e5f38d170/html5/thumbnails/31.jpg)
Objects from Motion ExperimentDefine object selective regions: OFL > GFL
OFM G-MTN OFM
0
3
6
% si
gnal
0 50 100 150 200 250 300 time (s)
n=9
GFLOFL
left hemi
lateral
ventral
OFL > GFL
Grill-Spector et al. , Neuron 1998Courtesy Elsevier, Inc., http://www.sciencedirect.com. Used with permission.
![Page 32: 9.71 Functional MRI of High-Level Vision Fall 2007 For ......(different kinds of agnosias can give us further dissociations…) An example…. What does object recognition entail exactly,](https://reader033.fdocuments.us/reader033/viewer/2022043007/5f94c5197f7c7c0e5f38d170/html5/thumbnails/32.jpg)
Conclusion: Cue-independent Representations of Object Shape Objects from motion
Left hemisphere
eral Objects from greyscale photos Objects from line drawings
Grill-Spector et al. , Neuron 1998
Courtesy of National Academy of Sciences,U. S. A. Used with permission. Source:Malach, R. et. al. "Object-related activityrevealed by functional magnetic resonanceimaging in human occipital cortex." Proc.Natl. Acad. Sci. 92 (1995): 8135-8139.Copyright © 1995, National Academy ofSciences, U.S.A.
latCourtesy Elsevier, Inc., http://www.sciencedirect.com.
Used with permission.Objects from texture
Courtesy Elsevier, Inc., http://www.sciencedirect.com.Used with permission.
Objects from luminance
![Page 33: 9.71 Functional MRI of High-Level Vision Fall 2007 For ......(different kinds of agnosias can give us further dissociations…) An example…. What does object recognition entail exactly,](https://reader033.fdocuments.us/reader033/viewer/2022043007/5f94c5197f7c7c0e5f38d170/html5/thumbnails/33.jpg)
Characterizing Representations and Processes in the LOC �
Are shape representations in LOC independent of how shape is represented, i.e. independent of
form-cues (motion, luminance, texture)? probably contours?
Independent of changes in the size, position, viewpoint, etc?
Cool method: fMRI adaptation
BUT: Have I shown you evidence that the very same neurons respond to form independent of how that form is defined?
![Page 34: 9.71 Functional MRI of High-Level Vision Fall 2007 For ......(different kinds of agnosias can give us further dissociations…) An example…. What does object recognition entail exactly,](https://reader033.fdocuments.us/reader033/viewer/2022043007/5f94c5197f7c7c0e5f38d170/html5/thumbnails/34.jpg)
Event-Related fMRI Adaptation
Basic idea: Any measure that is sensitive to the sameness vs. difference between 2 stimuli can reveal what the system takes to be the same and diff.
Example: If brain region X discriminate between two similar stimuli, say…. Then if we measure fMRI response in that region to same vs. different trials:
DIF
FER
ENT
We see this: Then region X can discriminate these 2 stimuli.
Photo courtesy of Trpster. Photo courtesy
of floridapfe.
250ms 500ms 250ms
Photo courtesy Photo courtesy of floridapfe. of floridapfe.
![Page 35: 9.71 Functional MRI of High-Level Vision Fall 2007 For ......(different kinds of agnosias can give us further dissociations…) An example…. What does object recognition entail exactly,](https://reader033.fdocuments.us/reader033/viewer/2022043007/5f94c5197f7c7c0e5f38d170/html5/thumbnails/35.jpg)
SAM
E D
IFFE
REN
TTE
ST
Event-Related fMRI Adaptation Basic idea: Any measure that is sensitive to the sameness vs. difference between 2 stimuli can reveal what the system takes to be the same.
Does region X “think” these images are the
What is the same? Photo courtesy answer if we Then region Xof floridapfe.
see this: can discriminate Photo courtesy of floridapfe. these 2 stimuli.
Photo courtesy Now we canof Trpster.
250ms 500ms 250ms also ask what images region X “thinks” are the same, e.g.…
Photo courtesy Photo courtesyof floridapfe. of floridapfe.
Photo courtesy of floridapfe.
![Page 36: 9.71 Functional MRI of High-Level Vision Fall 2007 For ......(different kinds of agnosias can give us further dissociations…) An example…. What does object recognition entail exactly,](https://reader033.fdocuments.us/reader033/viewer/2022043007/5f94c5197f7c7c0e5f38d170/html5/thumbnails/36.jpg)
Characterizing Representations and Processes in the LOC �
Are shape representations in LOC independent of how shape is represented, i.e. independent of
form-cues (motion, luminance, texture)? probably contours?
Independent of changes in the size, position, viewpoint, etc?
Cool method: fMRI adaptation
![Page 37: 9.71 Functional MRI of High-Level Vision Fall 2007 For ......(different kinds of agnosias can give us further dissociations…) An example…. What does object recognition entail exactly,](https://reader033.fdocuments.us/reader033/viewer/2022043007/5f94c5197f7c7c0e5f38d170/html5/thumbnails/37.jpg)
Is LOC “Contour Invariant”?
If the LOC represents object shape, independent of the contours defining that shape, then if the two stimuli have……..
1. Diff. Contours AdaptationBut Same Shape
2. Same Contours No AdaptationBut Different Shape
Kourtzi & Kanwisher (2001)
![Page 38: 9.71 Functional MRI of High-Level Vision Fall 2007 For ......(different kinds of agnosias can give us further dissociations…) An example…. What does object recognition entail exactly,](https://reader033.fdocuments.us/reader033/viewer/2022043007/5f94c5197f7c7c0e5f38d170/html5/thumbnails/38.jpg)
FImage removed due to copyright restrictions. Fig. 2 in "Representation of Perceived Object Shape by the HumanLateral Occipital Complex." Science, 24 AUGUST 2001 VOL 293.(http://web.mit.edu/bcs/nklab/media/pdfs/KourtziKanwisherScience01.pdf)
1. Diff. Contours But Same Shape
Is there neural adaptation in the LOC for objects that have different contours but the same
perceived shape?
Image removed due to copyright restrictions. Fig. 2 in Kourtzi, Zoe, and Nancy Kanwisher. "Representation of Perceived Object Shape by the HumanLateral Occipital Complex." Science, 24 AUGUST 2001 VOL 293.(http://web.mit.edu/bcs/nklab/media/pdfs/KourtziKanwisherScience01.pdf)
300 ms 300 ms
400 ms 2000 ms
![Page 39: 9.71 Functional MRI of High-Level Vision Fall 2007 For ......(different kinds of agnosias can give us further dissociations…) An example…. What does object recognition entail exactly,](https://reader033.fdocuments.us/reader033/viewer/2022043007/5f94c5197f7c7c0e5f38d170/html5/thumbnails/39.jpg)
Experiment 1: Results• Define the LOC for intact versus scrambled images in each subject (n=10). • Average time course of activation in the LOC.
Image removed due to copyright restrictions. Fig. 3 in Kourtzi, Zoe, and Nancy Kanwisher. "Representation of Perceived Object Shape by the Human Lateral Occipital Complex." Science, 24 AUGUST 2001 VOL 293.(http://web.mit.edu/bcs/nklab/media/pdfs/KourtziKanwisherScience01.pdf)
• Significant adaptation for identical shapes (p<0.05). Kourtzi & Kanwisher
![Page 40: 9.71 Functional MRI of High-Level Vision Fall 2007 For ......(different kinds of agnosias can give us further dissociations…) An example…. What does object recognition entail exactly,](https://reader033.fdocuments.us/reader033/viewer/2022043007/5f94c5197f7c7c0e5f38d170/html5/thumbnails/40.jpg)
Experiment 1: Results• Define the LOC for intact versus scrambled images in each subject (n=10). • Average time course of activation in the LOC.
Image removed due to copyright restrictions. Fig. 3 in Kourtzi, Zoe, and Nancy Kanwisher. "Representation of Perceived Object Shape by the Human Lateral Occipital Complex." Science, 24 AUGUST 2001 VOL 293.(http://web.mit.edu/bcs/nklab/media/pdfs/KourtziKanwisherScience01.pdf)
Kourtzi & Kanwisher • Significant adaptation for displays with the same shape but different contours (p<0.05).
![Page 41: 9.71 Functional MRI of High-Level Vision Fall 2007 For ......(different kinds of agnosias can give us further dissociations…) An example…. What does object recognition entail exactly,](https://reader033.fdocuments.us/reader033/viewer/2022043007/5f94c5197f7c7c0e5f38d170/html5/thumbnails/41.jpg)
If the LOC represents object shape, independentof the contours defining that shape, then if the two stimuli have……..
1. Diff. Contours Adaptation √Image removed due to copyright restrictions. Fig. 2 in "Representation of Perceived Object Shape by the Same Shape Human Lateral Occipital Complex." Science, 24 AUGUST 2001 VOL 293. (http://web.mit.edu/bcs/nklab/media/pdfs/KourtziKanwisherScience01.pdf)
2. Same Contours No Adaptation ?Different Shape
Kourtzi & Kanwisher
Kourtzi, Zoe, and Nancy Kanwisher.
![Page 42: 9.71 Functional MRI of High-Level Vision Fall 2007 For ......(different kinds of agnosias can give us further dissociations…) An example…. What does object recognition entail exactly,](https://reader033.fdocuments.us/reader033/viewer/2022043007/5f94c5197f7c7c0e5f38d170/html5/thumbnails/42.jpg)
2. Same Contours But Different Shape
Is there neural adaptation for stereoscopically defined shapes that share the same contours
but have different shape?
F Image removed due to copyright restrictions. Fig. 2B in Kourtzi, Zoe and Kanwisher, Nancy. "Representation of Perceived Object Shape by the Human Lateral Occipital Complex." Science, 24 AUGUST 2001 VOL 293. (http://web.mit.edu/bcs/nklab/media/pdfs/KourtziKanwisherScience01.pdf)
Kourtzi & Kanwisher
![Page 43: 9.71 Functional MRI of High-Level Vision Fall 2007 For ......(different kinds of agnosias can give us further dissociations…) An example…. What does object recognition entail exactly,](https://reader033.fdocuments.us/reader033/viewer/2022043007/5f94c5197f7c7c0e5f38d170/html5/thumbnails/43.jpg)
Experiment 2: Results• Define the LOC for intact versus scrambled images in each subject (n=10). • Average time course of activation in the LOC.
Image removed due to copyright restrictions. Fig. 4 in Kourtzi, Zoe and Kanwisher, Nancy. "Representation of Perceived Object Shape by the Human Lateral Occipital Complex." Science, 24 AUGUST 2001 VOL 293.(http://web.mit.edu/bcs/nklab/media/pdfs/KourtziKanwisherScience01.pdf)
• Significant adaptation for identical shapes (p<0.01).
![Page 44: 9.71 Functional MRI of High-Level Vision Fall 2007 For ......(different kinds of agnosias can give us further dissociations…) An example…. What does object recognition entail exactly,](https://reader033.fdocuments.us/reader033/viewer/2022043007/5f94c5197f7c7c0e5f38d170/html5/thumbnails/44.jpg)
Experiment 2: Results• Define the LOC for intact versus scrambled images in each subject (n=10). • Average time course of activation in the LOC.
Image removed due to copyright restrictions. Fig. 4 in Kourtzi, Zoe and Kanwisher, Nancy. "Representation of Perceived Object Shape by the Human Lateral Occipital Complex." Science, 24 AUGUST 2001 VOL 293.(http://web.mit.edu/bcs/nklab/media/pdfs/KourtziKanwisherScience01.pdf)
• No significant adaptation for displays with the same contours but different shape.
![Page 45: 9.71 Functional MRI of High-Level Vision Fall 2007 For ......(different kinds of agnosias can give us further dissociations…) An example…. What does object recognition entail exactly,](https://reader033.fdocuments.us/reader033/viewer/2022043007/5f94c5197f7c7c0e5f38d170/html5/thumbnails/45.jpg)
Conclusions
1. Diff. ContoursSame Shape
2. Same Contours Different Shape
AdaptationImage removed due to copyright restrictions. Fig. 2 in Kourtzi, Zoe, and Nancy Kanwisher. "Representation of Perceived Object Shape by the Human Lateral Occipital Complex." Science, 24 AUGUST 2001 VOL 293. (http://web.mit.edu/bcs/nklab/media/pdfs/KourtziKanwisherScience01.pdf)
Image removed due to copyright restrictions. Fig. 2B in Kourtzi, Zoe, and Nancy Kanwisher. "Representation of Perceived Object Shape by the Human Lateral Occipital No AdaptationComplex." Science, 24 AUGUST 2001 VOL 293. (http://web.mit.edu/bcs/nklab/media/pdfs/KourtziKanwisherScience01.pdf)
The adaptation effects in the LOC suggest that these neural populations represent object shape independent of the contours defining the shape.
Kourtzi & Kanwisher
![Page 46: 9.71 Functional MRI of High-Level Vision Fall 2007 For ......(different kinds of agnosias can give us further dissociations…) An example…. What does object recognition entail exactly,](https://reader033.fdocuments.us/reader033/viewer/2022043007/5f94c5197f7c7c0e5f38d170/html5/thumbnails/46.jpg)
Characterizing Representations and Processes in the LOC �
Are shape representations in LOC independent of how shape is represented, i.e. independent of
form-cues (motion, luminance, texture)? probably contours? Yes!
Independent of changes in the size, position, viewpoint, etc?
Uh, why does this matter again?
Cool method: fMRI adaptation
![Page 47: 9.71 Functional MRI of High-Level Vision Fall 2007 For ......(different kinds of agnosias can give us further dissociations…) An example…. What does object recognition entail exactly,](https://reader033.fdocuments.us/reader033/viewer/2022043007/5f94c5197f7c7c0e5f38d170/html5/thumbnails/47.jpg)
How do we Recognize Objects despite Variations in the Image of Each Object?
Extract one common Photo courtesy of Nick Devenish. representation from
each of these that is “invariant” to changes in size, position, viewpoint, etc.
Extract a different representation for each, then map all of these to “rabbit”.
Rabbit
![Page 48: 9.71 Functional MRI of High-Level Vision Fall 2007 For ......(different kinds of agnosias can give us further dissociations…) An example…. What does object recognition entail exactly,](https://reader033.fdocuments.us/reader033/viewer/2022043007/5f94c5197f7c7c0e5f38d170/html5/thumbnails/48.jpg)
Changes in Viewpoint
Front Profile Cheek Back
Face photos modified by OCW for privacy considerations.
• Are responses to faces tuned to specific viewpoints of faces?
Tong, Kanwisher, & Nakayama, 2000
![Page 49: 9.71 Functional MRI of High-Level Vision Fall 2007 For ......(different kinds of agnosias can give us further dissociations…) An example…. What does object recognition entail exactly,](https://reader033.fdocuments.us/reader033/viewer/2022043007/5f94c5197f7c7c0e5f38d170/html5/thumbnails/49.jpg)
1.8 1.8 1.3
Does thisFront Profile Cheek Back mean that
Face photos modified by OCW for privacy considerations.
PSC in 0.9FFA
(n=5)
the same neurons response to front & profile views of faces?
3
% MR 2
Signal 1 Change 0 In FFA -1
+ B C P F + P B F C + C F B P F P C B
0 80 160 240 320
Time Tong, Kanwisher, & Nakayama, 2000
![Page 50: 9.71 Functional MRI of High-Level Vision Fall 2007 For ......(different kinds of agnosias can give us further dissociations…) An example…. What does object recognition entail exactly,](https://reader033.fdocuments.us/reader033/viewer/2022043007/5f94c5197f7c7c0e5f38d170/html5/thumbnails/50.jpg)
Using Adaptation to Test for Invariances
Expect lower responses for blocks + + + + +
of identical imagesthan blocks of different faces/cars.
Then use that effect to test for invariancesacross changes in position, etc…. (Grill-Spector et al. 1999)
texture
+
+
identical
+
+
+
+
different
+
+
+
+
Courtesy Elsevier, Inc., http://www.sciencedirect.com. Used with permission.
![Page 51: 9.71 Functional MRI of High-Level Vision Fall 2007 For ......(different kinds of agnosias can give us further dissociations…) An example…. What does object recognition entail exactly,](https://reader033.fdocuments.us/reader033/viewer/2022043007/5f94c5197f7c7c0e5f38d170/html5/thumbnails/51.jpg)
Using Adaptation to Test for Invariances
position viewpoint different identical texture
+
+
+
+
+
+
+
+
+
+
+
+ +
+
+
+
+
+
+
+
+
+
+
+
+
+
+
Do images that vary only in position or viewpoint count as the “same” and hence get adapted, or do they count as “different” and not get adapted?
Courtesy Elsevier, Inc., http://www.sciencedirect.com. Used with permission. (Grill-Spector et al. 1999)
![Page 52: 9.71 Functional MRI of High-Level Vision Fall 2007 For ......(different kinds of agnosias can give us further dissociations…) An example…. What does object recognition entail exactly,](https://reader033.fdocuments.us/reader033/viewer/2022043007/5f94c5197f7c7c0e5f38d170/html5/thumbnails/52.jpg)
Differential Invariance in Anterior-VentralObject-Selective Areas: LOa /pFs
ventral
0.40
0.60
0.80
1.00
adap
tatio
n ra
tio
*
Identical Position Size Illumination Viewpoint
LOa/pFs 0.48 0.69 0.68 0.87 0.88
ratio = % signal condition ratio = 1.0 there is no adaptation % signal different ratio < 0.7 * significant adaptation (p<0.01)
(Grill-Spector et al. 1999)
![Page 53: 9.71 Functional MRI of High-Level Vision Fall 2007 For ......(different kinds of agnosias can give us further dissociations…) An example…. What does object recognition entail exactly,](https://reader033.fdocuments.us/reader033/viewer/2022043007/5f94c5197f7c7c0e5f38d170/html5/thumbnails/53.jpg)
Differential Invariance in Anterior-VentralObject-Selective Areas: LOa /pFs
0.40
0.60
0.80
1.00
adap
tatio
n ra
tio
LOa/pFs 0.48 0.69 0.68 0.87 0.88
Identical Position Size Illumination Viewpoint
*
* *
ventral
ratio = % signal condition ratio = 1.0 there is no adaptation % signal different ratio < 0.7 * significant adaptation (p<0.01)
(Grill-Spector et al. 1999)
![Page 54: 9.71 Functional MRI of High-Level Vision Fall 2007 For ......(different kinds of agnosias can give us further dissociations…) An example…. What does object recognition entail exactly,](https://reader033.fdocuments.us/reader033/viewer/2022043007/5f94c5197f7c7c0e5f38d170/html5/thumbnails/54.jpg)
adap
tatio
n ra
tio
ventral
lateral
Differential Invariance in Subdivisions of LOC
0.40
0.60
0.80
1.00
LO 0.64 0.92 0.88 0.85 1.01 LOa/pFs 0.48 0.69 0.68 0.87 0.88
Identical Position Size Illumination Viewpoint
*
* * *
ratio = % signal condition ratio = 1.0 there is no adaptation % signal different ratio < 0.7 * significant adaptation (p<0.01)
(Grill-Spector et al. 1999)
![Page 55: 9.71 Functional MRI of High-Level Vision Fall 2007 For ......(different kinds of agnosias can give us further dissociations…) An example…. What does object recognition entail exactly,](https://reader033.fdocuments.us/reader033/viewer/2022043007/5f94c5197f7c7c0e5f38d170/html5/thumbnails/55.jpg)
Using Adaptation to Test for Invariances
position viewpoint different identical texture
+
+
+
+
+
+
+
+
+
+
+
+ +
+
+
+
+
+
+
+
+
+
+
+
+
+
+
Do images that vary only in position or viewpoint count as the “same” and hence get adapted, or do they count as “different” and not get adapted?
Courtesy Elsevier, Inc., http://www.sciencedirect.com. Used with permission. (Grill-Spector et al. 1999)
![Page 56: 9.71 Functional MRI of High-Level Vision Fall 2007 For ......(different kinds of agnosias can give us further dissociations…) An example…. What does object recognition entail exactly,](https://reader033.fdocuments.us/reader033/viewer/2022043007/5f94c5197f7c7c0e5f38d170/html5/thumbnails/56.jpg)
Characterizing Representations and Processes in the LOC �
Are shape representations in LOC independent of how shape is represented, i.e. independent of
form-cues (motion, luminance, texture)? probably contours? Yes!
Independent of changes in the size, position, viewpoint, etc? Partly. More to size & position than viewpoint.
Cool method: fMRI adaptation
![Page 57: 9.71 Functional MRI of High-Level Vision Fall 2007 For ......(different kinds of agnosias can give us further dissociations…) An example…. What does object recognition entail exactly,](https://reader033.fdocuments.us/reader033/viewer/2022043007/5f94c5197f7c7c0e5f38d170/html5/thumbnails/57.jpg)
Face Scrambling Experiment Grill-Spector, et al (1998) car
256
64
16 X 3
4 X 4
2 vertical
whole
X 2
2 horizontal
X 2
X 4
X 4
X 4
414sec
X 4
![Page 58: 9.71 Functional MRI of High-Level Vision Fall 2007 For ......(different kinds of agnosias can give us further dissociations…) An example…. What does object recognition entail exactly,](https://reader033.fdocuments.us/reader033/viewer/2022043007/5f94c5197f7c7c0e5f38d170/html5/thumbnails/58.jpg)
Whole vs. Parts
R. Malach Weizmann. Inst.Grill-Spector, Malach, others (1998)
![Page 59: 9.71 Functional MRI of High-Level Vision Fall 2007 For ......(different kinds of agnosias can give us further dissociations…) An example…. What does object recognition entail exactly,](https://reader033.fdocuments.us/reader033/viewer/2022043007/5f94c5197f7c7c0e5f38d170/html5/thumbnails/59.jpg)
Comments on Papers 1. Writing matters in life; learn to do it well now. You cannot be a
successful scientist unless you write well. Probably true for most other professions as well.
2. One strong argument (or maybe tops 3) is much more effective than 12 weak ones. “Kitchen sink” papers are ineffective.
3. Start the paper with a statement of what was found & claimed. 4. A paper (or talk) is not a note to me >> it should pass the
“roomate test”. 5. Don’t just say X is a problem; say WHY! 6. Paragraph structure. 7. Write it, print it, get away from it, come back, and read it.
READ IT ALOUD. 8. If you have a paragraph with 3-5 separate ideas that related to
thes same point, it helps to indicate in advance, and enumerate them. E.g., “There are four problems with this design…”
9. Distinguish between design problems that matter versus those that don’t.
![Page 60: 9.71 Functional MRI of High-Level Vision Fall 2007 For ......(different kinds of agnosias can give us further dissociations…) An example…. What does object recognition entail exactly,](https://reader033.fdocuments.us/reader033/viewer/2022043007/5f94c5197f7c7c0e5f38d170/html5/thumbnails/60.jpg)
Variability Across Individual Subjects
Contrasts
moving> stationary
body parts> objects
faces> objects
scenes> objects
• some discontiguities are apparent in individual regions.
tions are overlapping but not identical across subjects
Coregister data across subjects using “spherical coordinates”, then ask which regions show a significant response in a given contrast in the same location in at least 30% of Ss.
Image removed due to copyright restrictions. See Fig. 3 in Spiridon, M., B. Fischl, and N. Kanwisher. "Location and Spatial Profile of Category-Specific Regions in Human Extrastriate Cortex." Human Brain Mapping 27 (2006): 77-89.
• loca
![Page 61: 9.71 Functional MRI of High-Level Vision Fall 2007 For ......(different kinds of agnosias can give us further dissociations…) An example…. What does object recognition entail exactly,](https://reader033.fdocuments.us/reader033/viewer/2022043007/5f94c5197f7c7c0e5f38d170/html5/thumbnails/61.jpg)
Population overlap Maps on Cortical Surface, Spherical Coords.With Bruce Fischl & Mona Spiridon
Image removed due to copyright restrictions. See Fig. 2 in Spiridon, M., B. Fischl, and N. Kanwisher. "Location and Spatial Profile of Category-Specific Regions in Human Extrastriate Cortex." Human Brain Mapping 27 (2006): 77-89.
![Page 62: 9.71 Functional MRI of High-Level Vision Fall 2007 For ......(different kinds of agnosias can give us further dissociations…) An example…. What does object recognition entail exactly,](https://reader033.fdocuments.us/reader033/viewer/2022043007/5f94c5197f7c7c0e5f38d170/html5/thumbnails/62.jpg)
Potter (1971)
Presented a random sequence of complex scenes to subjects at a rate of around 7/second. Found that subjects could get the gist of pretty much each one. (e.g., detect a “picnic”).
Implies: i) don’t need “top-down” prediction to recognize objects ii) object/scene recognition is FAST!