Fast Readout of Object Identity from Macaque Inferior Tempora Cortex

29
Fast Readout of Object Identity from Macaque Inferior Tempora Cortex Chou P. Hung, Gabriel Kreiman, Tomaso Poggio, James J.DiCarlo McGovern Institute for Brain Research, Brain and Cognitive Sciences, MIT

description

Fast Readout of Object Identity from Macaque Inferior Tempora Cortex. Chou P. Hung, Gabriel Kreiman, Tomaso Poggio, James J.DiCarlo McGovern Institute for Brain Research, Brain and Cognitive Sciences, MIT. Object Recognition is difficult: trade-off between selectivity and invariance. - PowerPoint PPT Presentation

Transcript of Fast Readout of Object Identity from Macaque Inferior Tempora Cortex

Page 1: Fast Readout of Object Identity from Macaque Inferior Tempora Cortex

Fast Readout of Object Identity from Macaque Inferior Tempora Cortex

Chou P. Hung, Gabriel Kreiman,Tomaso Poggio, James J.DiCarlo

McGovern Institute for Brain Research,Brain and Cognitive Sciences, MIT

Page 2: Fast Readout of Object Identity from Macaque Inferior Tempora Cortex

Object Recognition is difficult:trade-off between selectivity and invariance Selectivity

Many different images can correspond to the same type of object

Invariance Similar activation patterns can correspond

to different objects

Page 3: Fast Readout of Object Identity from Macaque Inferior Tempora Cortex

The end station of the ventral stream in visual cortex is IT

Page 4: Fast Readout of Object Identity from Macaque Inferior Tempora Cortex

Can we readout what the monkey is seeing?

Page 5: Fast Readout of Object Identity from Macaque Inferior Tempora Cortex

Single electrode recordings Anterior inferior temporal cortex: highest visual area

in the ventral “what” pathway Spiking activity in AIT shows selectivity for complex

shapes

Page 6: Fast Readout of Object Identity from Macaque Inferior Tempora Cortex
Page 7: Fast Readout of Object Identity from Macaque Inferior Tempora Cortex
Page 8: Fast Readout of Object Identity from Macaque Inferior Tempora Cortex

Can we “read-out” the subject’s object percept from IT? number of sites for reliable, real-time

performance temporal properties (onset + integration scale)

of object information neural code for different tasks invariance to object position, size, pose,

illumination, clutter recognition: ‘classification’ vs. ‘identification’? spatial scale of object information (single unit, multi-

unit, LFP) stability of these neuronal codes? improvement with experience? …

Page 9: Fast Readout of Object Identity from Macaque Inferior Tempora Cortex

77 objects, 8 classes

Page 10: Fast Readout of Object Identity from Macaque Inferior Tempora Cortex

Recording at each recording site during passive viewing

77 visual objects 10 presentation repetitions per object presentation order randomized and counter-

balanced

Page 11: Fast Readout of Object Identity from Macaque Inferior Tempora Cortex

One-versus-all classification g classes (g=8): G1, …, Gg (toys, monkey

faces, vehicles, etc.) For each class i, build a binary classifier fi

(toys vs. rest, monkey faces vs. rest, etc.) sj labeled examples (j=1,…,n), For each example j, compute the output of

each classifier (e.g. pi=sj. fi ) Take prediction that maximizes pi

One-versus-all is not worse than other methods (Rifkin et al, 2003)

ij Gs

Page 12: Fast Readout of Object Identity from Macaque Inferior Tempora Cortex

Comparison of different statistical classifiers

Page 13: Fast Readout of Object Identity from Macaque Inferior Tempora Cortex

Decoding the population response

Categorization 8 groups

Page 14: Fast Readout of Object Identity from Macaque Inferior Tempora Cortex

Pattern of mistakes made by the classifier

Page 15: Fast Readout of Object Identity from Macaque Inferior Tempora Cortex

Very rapid read-out of object information

Page 16: Fast Readout of Object Identity from Macaque Inferior Tempora Cortex

Categorization and Identification

Page 17: Fast Readout of Object Identity from Macaque Inferior Tempora Cortex

IT representation is invariant to changes in position and size

Page 18: Fast Readout of Object Identity from Macaque Inferior Tempora Cortex

IT representation is invariant to changes in position and size

Page 19: Fast Readout of Object Identity from Macaque Inferior Tempora Cortex

IT representation is invariant to changes in position and size

Page 20: Fast Readout of Object Identity from Macaque Inferior Tempora Cortex

Neural code in IT: time resolution

Page 21: Fast Readout of Object Identity from Macaque Inferior Tempora Cortex

Neural code in IT: latency and integration time

Page 22: Fast Readout of Object Identity from Macaque Inferior Tempora Cortex

Reading out another type of object info: scale and location

Page 23: Fast Readout of Object Identity from Macaque Inferior Tempora Cortex

How are different kinds of information coded?

Page 24: Fast Readout of Object Identity from Macaque Inferior Tempora Cortex

Reading out another type of object info: stimulus onset

Page 25: Fast Readout of Object Identity from Macaque Inferior Tempora Cortex

Specific wiring significantly improves classifier performance

Page 26: Fast Readout of Object Identity from Macaque Inferior Tempora Cortex

Extrapolation to novel pictures within the same categories

Page 27: Fast Readout of Object Identity from Macaque Inferior Tempora Cortex

Strong overlap between the best neurons for categorization and identification

Page 28: Fast Readout of Object Identity from Macaque Inferior Tempora Cortex

The SNR for categorization and identification are positively correlated

Page 29: Fast Readout of Object Identity from Macaque Inferior Tempora Cortex

Invariance to scale and position