Contextually dependent cue realization and cue weighting ...
Form-cue invariant second-order contrast envelope...
1
S"muli Challenges Background Methods Orientation Tuning Conclusion Form-cue invariant second-order contrast envelope responses in macaque V2 G. Li 1 , Z. Wang 2 , Z. Yao 2 , N. Yuan 2 , V. Talebi 1 , J. Tan 2 , Y. Wang 2 , Y. Zhou 2 , C.L. Baker 1 1 McGill Vision Research, Dept Ophthalmology, McGill University, Montreal, QC, Canada 2 School of Life Sciences, University of Science and Technology of China, Hefei, China Supported by CIHRNSFC ChinaCanada Joint Health Ini9a9ve Collabora9on grant (CCI92217; NSFC30811120423) to CB and to YZ. Introduction • Natural images frequently contain oriented boundaries defined by differences of luminance (1 st order), or of contrast or texture (2 nd order). • 1 st order s9muli are thought to be detected by neural mechanisms ac9ng like a conven9onal linear filter. • Detec9on of 2 nd order s9muli could be explained by a ‘filterrec9fyfilter’ cascade. 1 st stage filters are tuned to the fine scale of the texture. rec9fied outputs drive a 2 nd stage, coarser scale linear filter. • 2 nd order processing has been widely studied in human psychophysics (Landy & Graham, 2004). • In primates the neural substrate s9ll remains unclear. Spatial Frequency Tuning Monkey vs. Cat Neurons In One Penetration Spatial Frequency Tuning Properties Orientation Tuning Properties Sinewave gra9ngs and contrast modula9on (CM) envelopes s9muli were presented on a gammacorrected CRT monitor, placed at a 114 cm distance. • anesthe9zed and paralyzed monkeys (Macaca mula(a) (N 2 O, Propofol, Sufentanil) + (Gallamine triethiodide, Tubocurarine chloride) • careful refrac9on and ar9ficial pupils. • extracellular recording: glasscoated tungsten electrodes (FHC): V1 > white maeer > V2 Michigan Probes (NeuroNexus): near lunate sulcus. • singleunit (SU): isolated using Plexon Offline Sorter, quan9fied as average firing rates. • mul9unit ac9vity (MUA): defined as events that exceeded 3σ of the filtered raw data (300 – 5KHz), quan9fied as average firing rates. • experiment protocol: luminance gra9ngs: spa9al frequency, orienta9on, RFmapping envelope (CM) s9muli: carrier spa9al frequency, orienta9on envelope spa9al frequency, orienta9on • 51 visually responsive neurons, 22 were 2 nd order responsive. Reference: Mareschal I & Baker CL (1999), Visual Neuroscience 16:527540. Tanaka H and Ohzawa I (2009), J Neurophysiol 101:14441462. Landy MS and Graham N (2004), The Visual Neuroscience, Chapter 73:11061118 Hallum LE, Movshon JA (2011), SfN Abstr 694.19/NN14 SU MUA Op9mal Spa9al Frequency Op9mal Orienta9on 1. Ensure that responses to 2 nd order s9muli are not due to inadvertent ac9va9on of linear/ luminance mechanisms. 2. Use s9muli that provide the analy9cal power to analyze the tuning proper9es of both early and late stage filtering. CM envelopes were generated by mul9plica9on of two gra9ng paeerns: a sta9onary high spa9al frequency (f Carrier ) gra9ng as the ‘carrier’ and a driping low spa9al frequency (f Envelope ) gra9ng as the ‘envelope’. In Fourier frequency domain, such CM s9muli consist of components all close to the ‘carrier’ spa9al frequency, with no energy at the ‘envelope’ spa9al frequency. A neuron that responds to an envelope s9mulus in which all Fourier components are clearly outside its gra9ng frequencyselec9ve range, must have been responding to the envelope of the s9mulus as a result of nonlinear processing. The nonlinearity might occur: (1) aper the 1st stage filter (2 nd order processing, FRF model), or (2) because of an early nonlinearity (before filters, e.g. s9muli ar9fact). if so, the envelope response would not be selec9ve for the carrier spa9al frequency. Criteria for a “2 nd order responsive” neuron: responds to an envelope (CM) s"mulus (blue). all the Fourier components of s"mulus are clearly outside its luminance frequency selec"ve range (black). tuned to carrier spa"al frequency (red). Electrophysiology Recording: Spa9al frequency tuning curves of three envelope responsive neurons for luminance s9muli and CM s9muli (upper row SU, boeom row MUA). Each plot shows luminance spa9al frequency (black), envelope spa9al frequency (blue) and carrier spa9al frequency (red). • Neurons are clearly tuned for luminance, envelope and carrier spa9al frequency. • Different neurons exhibited different op9mal spa9al frequencies for luminance , envelope and carrier. • The op9mal carrier spa9al frequencies were clearly dis9nct from the luminance gra9ng response range. Simple luminance gra9ngs produced liele or no response at such high frequencies. • Neurons exhibited similar, but slightly different spa9al frequency tuning curves for luminance and envelope. • MUA exhibited similar tuning curves as SUs. Orienta9on polar plots for the same three neurons (showed above). Orienta9on tuning curves for luminance (black), envelope (blue) and carrier (red) are shown in different columns. • Neurons are clearly tuned for luminance, envelope and carrier orienta9ons. • Envelope response polar plots were usually very similar to the corresponding luminance polar plots (formcue invariance). • Carrier orienta9on could be very narrowly tuned. • No clear rela9onship between the carrier orienta9on tuning and the corresponding luminance tuning. • MUA exhibited similar tuning curves as SUs. A par9cularly good penetra9on with many envelope responsive neurons recorded by tungsten electrode. • V2 was iden9fied by gray maeer (V1)white maeergray maeer (V2) transi9on. Tuning proper9es changed significantly from V1 to V2. • The recep9ve fields were at 612° eccentricity. • S9muli were confined to the neurons’ classical recep9ve fields. There was no clear rela9onship between envelope responsivity and surround suppression. • The op9mal value of carrier spa9al frequency and orienta9on varied with depth independently of luminance and envelope tuning. Scaeerplot of op9mal carrier spa9al frequency against the op9mal luminance spa9al frequency. • 2 nd order envelope responsive neurons (filled squares) had a carrierluminance ra9o higher than 5:1. • Some neurons (opened triangles) responded to CM’s with a carrierluminance ra9o around 2:1, probably due to surround modula9on (Tanaka & Ohzawa, 2009; Hallum & Movshon, 2011). • Most enveloperesponsive neurons had higher op9mal spa9al frequency for envelopes than luminance. Scaeerplot of op9mal envelope spa9al frequency against op9mal luminance spa9al frequency. Scaeerplot of op9mal carrier spa9al frequency against op9mal envelope spa9al frequency. MUA MUA MUA Scaeerplot of op9mal envelope orienta9on against op9mal luminance orienta9on. Scaeerplot of op9mal carrier orienta9on against op9mal luminance orienta9on. • Envelope responsive neurons with high carrierluminance ra9o (> 5:1, filled squares) cluster around the equality line (formcue invariance); low ra9o neurons (opened triangles) are randomly distributed. • There is no fixed rela9onship between the op9mal carrier and luminance (envelope) orienta9ons. The op9mal carrier spa9al frequencies of monkey V2 are significantly higher than those for cat area 18 (p < 0.001, MannWhitney U test). The carrier orienta9on tuning bandwidth of monkey V2 neurons is significantly narrower than in cat area 18 (p < 0.001, MannWhitney U test). In monkey V2 neurons, the op9mal spa9al frequency is usually higher for envelopes than for luminance, but in cat area 18 it is usually lower. These results show that many macaque V2 neurons are selec9vely responsive to 2 nd order s9muli in a manner that cannot be explained by quasilinear recep9ve fields, but which implies a specialized nonlinear mechanism. Unlike responses shaped by surround modula9on, these neurons exhibit formcue invariant orienta9on selec9vity and also exhibit a high carrierluminance ra9o (> 5:1). Formcue invariant 2 nd order contrast envelope responsive neurons could provide a func9onally useful, explicit representa9on of segmenta9on boundaries. SU MUA Cell 1 Cell 2 Cell 3 A B C D E F MUA Carrier Orienta9on Envelope Orienta9on Luminance Orienta9on Cell 1 Cell 2 Cell 3 0.0 0.5 1.0 1.5 2.0 2.5 0 30 60 90 120 150 180 210 240 270 300 330 0.0 0.5 1.0 1.5 2.0 2.5 Z0514 Carrier Orientation Response (spikes/sec) 0 2 4 6 8 0 30 60 90 120 150 180 210 240 270 300 330 0 2 4 6 8 Z0514 Luminance Modulated Response (spikes/sec) 0 1 2 3 4 0 30 60 90 120 150 180 210 240 270 300 330 0 1 2 3 4 Z0514 Envelope Orientation Response (spikes/sec) 0.0 0.5 1.0 1.5 0 30 60 90 120 150 180 210 240 270 300 330 0.0 0.5 1.0 1.5 Z0504 Envelope Orientation Response (spikes/sec) 0.0 2.5 5.0 7.5 10.0 0 30 60 90 120 150 180 210 240 270 300 330 0.0 2.5 5.0 7.5 10.0 Z0504 Luminance Modulated Response (spikes/sec) 0 1 2 3 0 30 60 90 120 150 180 210 240 270 300 330 0 1 2 3 Z0504 Carrier Orientation Response (spikes/sec) 0 3 6 9 12 15 0 30 60 90 120 150 180 210 240 270 300 330 0 3 6 9 12 15 Z0517 Carrier Orientation Response (spikes/sec) 0 3 6 9 12 15 0 30 60 90 120 150 180 210 240 270 300 330 0 3 6 9 12 15 Z0517 Envelope Orientation Response (spikes/sec) 0 5 10 15 20 0 30 60 90 120 150 180 210 240 270 300 330 0 5 10 15 20 Z0517 Luminance Modulated Response (spikes/sec) 0 20 40 60 0 30 60 90 120 150 180 210 240 270 300 330 0 20 40 60 MultiUnit Response (spikes/sec) Contrast Modulated 0 60 120 180 180 120 60 Z0517 Lum. Carr. Env. MultiUnit Response (spikes/sec) Luminance Modulated 0 100 200 300 0 30 60 90 120 150 180 210 240 270 300 330 0 100 200 300 MultiUnit Response (spikes/sec) Contrast Modulated 0 50 100 150 150 100 50 Z0504 Lum. Carr. Env. MultiUnit Response (spikes/sec) Luminance Modulated 0 100 200 300 400 0 30 60 90 120 150 180 210 240 270 300 330 0 100 200 300 400 MultiUnit Response (spikes/sec) Contrast Modulated 0 25 50 75 100 100 75 50 25 Z0514 Z0514 Lum. Carr. Env. Illu. Con. MultiUnit Response (spikes/sec) Luminance Modulated MUA MUA 1 st order 2 nd order narrow broad cat data: Mareschal & Baker (1999)
Transcript of Form-cue invariant second-order contrast envelope...
-
S"muli
Challenges
Background
Methods Orientation Tuning
Conclusion
Form-cue invariant second-order contrast envelope responses in macaque V2 G. Li1, Z. Wang2, Z. Yao2, N. Yuan2, V. Talebi1, J. Tan2, Y. Wang2, Y. Zhou2, C.L. Baker1
1McGill Vision Research, Dept Ophthalmology, McGill University, Montreal, QC, Canada 2School of Life Sciences, University of Science and Technology of China, Hefei, China
Supported by CIHR-‐NSFC China-‐Canada Joint Health Ini9a9ve Collabora9on grant (CCI-‐92217; NSFC-‐30811120423) to CB and to YZ.
Introduction • Natural images frequently contain oriented
boundaries defined by differences of luminance (1st-‐order), or of contrast or texture (2nd-‐order).
• 1st-‐order s9muli are thought to be detected by neural mechanisms ac9ng like a conven9onal linear filter.
• Detec9on of 2nd-‐order s9muli could be explained by a ‘filter-‐rec9fy-‐filter’ cascade.
-‐1st-‐stage filters are tuned to the fine scale of the texture. -‐rec9fied outputs drive a 2nd-‐stage, coarser scale linear filter. • 2nd-‐order processing has been widely studied in
human psychophysics (Landy & Graham, 2004). • In primates the neural substrate s9ll remains unclear.
Spatial Frequency Tuning
Monkey vs. Cat
Neurons In One Penetration
Spatial Frequency Tuning Properties
Orientation Tuning Properties
Sinewave gra9ngs and contrast modula9on (CM) envelopes s9muli were presented on a gamma-‐corrected CRT monitor, placed at a 114 cm distance.
• anesthe9zed and paralyzed monkeys (Macaca mula(a) -‐ (N2O, Propofol, Sufentanil) + (Gallamine triethiodide, Tubocurarine chloride) • careful refrac9on and ar9ficial pupils. • extracellular recording:
-‐ glass-‐coated tungsten electrodes (FHC): V1 -‐> white maeer -‐> V2 -‐ Michigan Probes (NeuroNexus): near lunate sulcus.
• single-‐unit (SU): isolated using Plexon Offline Sorter, quan9fied as average firing rates. • mul9-‐unit ac9vity (MUA): defined as events that exceeded 3-‐σ of the filtered raw data
(300 – 5KHz), quan9fied as average firing rates. • experiment protocol:
-‐ luminance gra9ngs: spa9al frequency, orienta9on, RF-‐mapping -‐ envelope (CM) s9muli: carrier spa9al frequency, orienta9on envelope spa9al frequency, orienta9on
• 51 visually responsive neurons, 22 were 2nd-‐order responsive.
Reference: Mareschal I & Baker CL (1999), Visual Neuroscience 16:527-‐540. Tanaka H and Ohzawa I (2009), J Neurophysiol 101:1444-‐1462. Landy MS and Graham N (2004), The Visual Neuroscience, Chapter 73:1106-‐1118 Hallum LE, Movshon JA (2011), SfN Abstr 694.19/NN14
SU
MUA
Op9mal Spa9al Frequency Op9mal Orienta9on
1. Ensure that responses to 2nd-‐order s9muli are not due to inadvertent ac9va9on of linear/luminance mechanisms.
2. Use s9muli that provide the analy9cal power to analyze the tuning proper9es of both early and late stage filtering.
CM envelopes were generated by mul9plica9on of two gra9ng paeerns: a sta9onary high spa9al frequency (fCarrier) gra9ng as the ‘carrier’ and a driping low spa9al frequency (fEnvelope) gra9ng as the ‘envelope’. In Fourier frequency domain, such CM s9muli consist of components all close to the ‘carrier’ spa9al frequency, with no energy at the ‘envelope’ spa9al frequency.
A neuron that responds to an envelope s9mulus in which all Fourier components are clearly outside its gra9ng frequency-‐selec9ve range, must have been responding to the envelope of the s9mulus as a result of nonlinear processing. The nonlinearity might occur: (1) aper the 1-‐st stage filter (2nd-‐order processing, F-‐R-‐F model), or (2) because of an early nonlinearity (before filters, e.g. s9muli ar9fact). -‐ if so, the envelope response would not be selec9ve for the carrier spa9al frequency.
Criteria for a “2nd-‐order responsive” neuron:
ü responds to an envelope (CM) s"mulus (blue). ü all the Fourier components of s"mulus are
clearly outside its luminance frequency-‐selec"ve range (black).
ü tuned to carrier spa"al frequency (red).
Electrophysiology Recording:
Spa9al frequency tuning curves of three envelope responsive neurons for luminance s9muli and CM s9muli (upper row à SU, boeom row à MUA). Each plot shows luminance spa9al frequency (black), envelope spa9al frequency (blue) and carrier spa9al frequency (red). • Neurons are clearly tuned for luminance,
envelope and carrier spa9al frequency. • Different neurons exhibited different op9mal
spa9al frequencies for luminance , envelope and carrier.
• The op9mal carrier spa9al frequencies were clearly dis9nct from the luminance gra9ng response range. Simple luminance gra9ngs produced liele or no response at such high frequencies.
• Neurons exhibited similar, but slightly different spa9al frequency tuning curves for luminance and envelope.
• MUA exhibited similar tuning curves as SUs.
Orienta9on polar plots for the same three neurons (showed above). Orienta9on tuning curves for luminance (black), envelope (blue) and carrier (red) are shown in different columns. • Neurons are clearly tuned for luminance,
envelope and carrier orienta9ons. • Envelope response polar plots were
usually very similar to the corresponding luminance polar plots (form-‐cue invariance).
• Carrier orienta9on could be very narrowly tuned.
• No clear rela9onship between the carrier orienta9on tuning and the corresponding luminance tuning.
• MUA exhibited similar tuning curves as SUs.
A par9cularly good penetra9on with many envelope responsive neurons recorded by tungsten electrode. • V2 was iden9fied by gray maeer (V1)-‐white
maeer-‐gray maeer (V2) transi9on. Tuning proper9es changed significantly from V1 to V2.
• The recep9ve fields were at 6-‐12° eccentricity. • S9muli were confined to the neurons’ classical
recep9ve fields. There was no clear rela9onship between envelope responsivity and surround suppression.
• The op9mal value of carrier spa9al frequency and orienta9on varied with depth independently of luminance and envelope tuning.
Scaeerplot of op9mal carrier spa9al frequency against the op9mal luminance spa9al frequency.
• 2nd-‐order envelope responsive neurons (filled squares) had a carrier-‐luminance ra9o higher than 5:1. • Some neurons (opened triangles) responded to CM’s with a carrier-‐luminance ra9o around 2:1, probably due
to surround modula9on (Tanaka & Ohzawa, 2009; Hallum & Movshon, 2011). • Most envelope-‐responsive neurons had higher op9mal spa9al frequency for envelopes than luminance.
Scaeerplot of op9mal envelope spa9al frequency against op9mal luminance spa9al frequency.
Scaeerplot of op9mal carrier spa9al frequency against op9mal envelope spa9al frequency.
MUA MUA MUA
Scaeerplot of op9mal envelope orienta9on against op9mal luminance orienta9on.
Scaeerplot of op9mal carrier orienta9on against op9mal luminance orienta9on.
• Envelope responsive neurons with high carrier-‐luminance ra9o (> 5:1, filled squares) cluster around the equality line (form-‐cue invariance); low ra9o neurons (opened triangles) are randomly distributed.
• There is no fixed rela9onship between the op9mal carrier and luminance (envelope) orienta9ons.
The op9mal carrier spa9al frequencies of monkey V2 are significantly higher than those for cat area 18 (p < 0.001, Mann-‐Whitney U test).
The carrier orienta9on tuning bandwidth of monkey V2 neurons is significantly narrower than in cat area 18 (p < 0.001, Mann-‐Whitney U test).
In monkey V2 neurons, the op9mal spa9al frequency is usually higher for envelopes than for luminance, but in cat area 18 it is usually lower.
l These results show that many macaque V2 neurons are selec9vely responsive to 2nd-‐order s9muli in a manner that cannot be explained by quasi-‐linear recep9ve fields, but which implies a specialized nonlinear mechanism.
l Unlike responses shaped by surround modula9on, these neurons exhibit form-‐cue invariant orienta9on selec9vity and also exhibit a high carrier-‐luminance ra9o (> 5:1).
l Form-‐cue invariant 2nd-‐order contrast envelope responsive neurons could provide a func9onally useful, explicit representa9on of segmenta9on boundaries.
SU
MUA
Cell 1 Cell 2 Cell 3 A
B
C
D
E
F
MUA Carrier Orienta9on Envelope Orienta9on Luminance Orienta9on
Cell 1
Cell 2
Cell 3
0.0
0.5
1.0
1.5
2.0
2.5
0
30
6090
120
150
180
210
240270
300
330
0.0
0.5
1.0
1.5
2.0
2.5
Z0514
Car
rier O
rient
atio
nR
espo
nse
(spi
kes/
sec)
0
2
4
6
8
0
30
6090
120
150
180
210
240270
300
330
0
2
4
6
8
Z0514
Lum
inan
ce M
odul
ated
Res
pons
e (s
pike
s/se
c)
0
1
2
3
4
0
30
6090
120
150
180
210
240270
300
330
0
1
2
3
4
Z0514
Env
elop
e O
rient
atio
nR
espo
nse
(spi
kes/
sec)
0.0
0.5
1.0
1.5
0
30
6090
120
150
180
210
240270
300
330
0.0
0.5
1.0
1.5Z0504
Env
elop
e O
rient
atio
nR
espo
nse
(spi
kes/
sec)
0.0
2.5
5.0
7.5
10.0
0
30
6090
120
150
180
210
240270
300
330
0.0
2.5
5.0
7.5
10.0
Z0504
Lum
inan
ce M
odul
ated
Res
pons
e (s
pike
s/se
c)
0
1
2
3
0
30
6090
120
150
180
210
240270
300
330
0
1
2
3
Z0504
Car
rier O
rient
atio
nR
espo
nse
(spi
kes/
sec)
0
3
6
9
12
15
0
30
6090
120
150
180
210
240270
300
330
0
3
6
9
12
15
Z0517
Car
rier O
rient
atio
nR
espo
nse
(spi
kes/
sec)
0
3
6
9
12
15
0
30
6090
120
150
180
210
240270
300
330
0
3
6
9
12
15
Z0517
Env
elop
e O
rient
atio
nR
espo
nse
(spi
kes/
sec)
0
5
10
15
20
0
30
6090
120
150
180
210
240270
300
330
0
5
10
15
20
Z0517
Lum
inan
ce M
odul
ated
Res
pons
e (s
pike
s/se
c)
0
20
40
60
0
30
60
90
120
150
180
210
240
270
300
330
0
20
40
60
Mul
tiUni
t Res
pons
e (s
pike
s/se
c)C
ontra
st M
odul
ated
0
60
120
180
180
120
60
Z0517
Lum. Carr. Env.
Mul
tiUni
t Res
pons
e (s
pike
s/se
c)Lu
min
ance
Mod
ulat
ed
0
100
200
300
0
30
60
90
120
150
180
210
240
270
300
330
0
100
200
300
Mul
tiUni
t Res
pons
e (s
pike
s/se
c)C
ontra
st M
odul
ated
0
50
100
150
150
100
50
Z0504
Lum. Carr. Env.
Mul
tiUni
t Res
pons
e (s
pike
s/se
c)Lu
min
ance
Mod
ulat
ed
0
100
200
300
400
0
30
60
90
120
150
180
210
240
270
300
330
0
100
200
300
400
Mul
tiUni
t Res
pons
e (s
pike
s/se
c)C
ontra
st M
odul
ated
0
25
50
75
100
100
75
50
25
Z0514
Z0514
Lum. Carr. Env. Illu. Con.
Mul
tiUni
t Res
pons
e (s
pike
s/se
c)Lu
min
ance
Mod
ulat
ed
MUA MUA
1st-‐order 2nd-‐order
narrow broad
cat data: Mareschal & Baker (1999)
