Fast Readout of Object Identity from Macaque Inferior Tempora Cortex
description
Transcript of Fast Readout of Object Identity from Macaque Inferior Tempora Cortex
Fast Readout of Object Identity from Macaque Inferior Tempora Cortex
Chou P. Hung, Gabriel Kreiman,Tomaso Poggio, James J.DiCarlo
McGovern Institute for Brain Research,Brain and Cognitive Sciences, MIT
Object Recognition is difficult:trade-off between selectivity and invariance Selectivity
Many different images can correspond to the same type of object
Invariance Similar activation patterns can correspond
to different objects
The end station of the ventral stream in visual cortex is IT
Can we readout what the monkey is seeing?
Single electrode recordings Anterior inferior temporal cortex: highest visual area
in the ventral “what” pathway Spiking activity in AIT shows selectivity for complex
shapes
Can we “read-out” the subject’s object percept from IT? number of sites for reliable, real-time
performance temporal properties (onset + integration scale)
of object information neural code for different tasks invariance to object position, size, pose,
illumination, clutter recognition: ‘classification’ vs. ‘identification’? spatial scale of object information (single unit, multi-
unit, LFP) stability of these neuronal codes? improvement with experience? …
77 objects, 8 classes
Recording at each recording site during passive viewing
77 visual objects 10 presentation repetitions per object presentation order randomized and counter-
balanced
One-versus-all classification g classes (g=8): G1, …, Gg (toys, monkey
faces, vehicles, etc.) For each class i, build a binary classifier fi
(toys vs. rest, monkey faces vs. rest, etc.) sj labeled examples (j=1,…,n), For each example j, compute the output of
each classifier (e.g. pi=sj. fi ) Take prediction that maximizes pi
One-versus-all is not worse than other methods (Rifkin et al, 2003)
ij Gs
Comparison of different statistical classifiers
Decoding the population response
Categorization 8 groups
Pattern of mistakes made by the classifier
Very rapid read-out of object information
Categorization and Identification
IT representation is invariant to changes in position and size
IT representation is invariant to changes in position and size
IT representation is invariant to changes in position and size
Neural code in IT: time resolution
Neural code in IT: latency and integration time
Reading out another type of object info: scale and location
How are different kinds of information coded?
Reading out another type of object info: stimulus onset
Specific wiring significantly improves classifier performance
Extrapolation to novel pictures within the same categories
Strong overlap between the best neurons for categorization and identification
The SNR for categorization and identification are positively correlated
Invariance to scale and position