Saccadic modulation of stimulus processing in primary visual cortex · 2015-03-16 · Saccadic...
1
Saccadic modulation of stimulus processing in primary visual cortex James M. McFarland 1 , Adrian G. Bondy 2,3 , Bruce G. Cumming 2 , Daniel A. Butts 1 1) Department of Biology and Program in Neuroscience and Cognitive Science, University of Maryland, College Park, MD USA 2) Laboratory of Sensorimotor Research, National Eye Institute, National Institutes of Health, Bethesda, MD USA 3) Brown-NIH Neuroscience Graduate Partnership Program, Brown University, Providence, RI USA Introduction ‘Active sampling’ of visual inputs by saccades and microsaccades is a key component of primate vision. Such eye movements produce discontinuous changes to the image on the retina, yet the visual system assimilates this constantly shifting input into a seamless visual percept. Extra-retinal signals asso- ciated with saccades have been shown to modulate the activity of neurons throughout the visual hierarchy [1-2]. The detailed stimulus processing models that have been developed in V1 make it an ideal area to study the interactions between visual inputs and peri-saccadic modulatory signals. Such studies are complicat- ed, however, by the fact that saccades produce rapid changes in the retinal input that strongly modulate the activity of V1 neurons [3]. Previous studies have attempted to distentangle these effects by comparing V1 responses to a stimulus that is either flashed into their receptive field (RF) or introduced by a saccade. These studies have generally found little effect of saccades [4-8], leading to the prevailing belief that V1 neurons encode visual stimuli indepedently of eye movements. Interpretation of these results is invariably confounded, however, by the uncontrolled variability in the precise spatiotemporal stimulus on the retina created by saccades. Further, they have generally only looked at the effects of saccades on average firing rates, and thus could not distinguish different functional forms of saccade modulation. Here we address this question using an experimental design that decouples saccades from the visual stimulus, combined with a detailed functional modeling approach. 0 0.1 0.2 0.8 1.0 1.2 Time (s) Relative rate Time since trial onset (s) Relative depth (mm) 0 0.05 0.1 0.15 0.2 0.5 1.0 0.0 Granular Supragranular Infragranular 0.0 0.05 0.1 0.15 Time lag (s) Stimulus response profile Granular Supragranular Infragranular Peak time (ms) Suppression Enhancement Peak time (ms) 50 60 70 80 120 140 160 40 50 60 180 * * * p < 0.01 * Source Sink 0 −0.1 0 0.1 0.6 0.8 1.0 Time (s) Gain of stimulus k 1 f 1 (.) k 2 + ... ... Offset kernel r(t) f 2 (.) Saccades Gain kernel + stimulus X(t) Pre-filtering Post-filtering −0.5 0.0 0.5 (LL PRE -LL POST )/(LL POST -LL NULL ) 0 10 Number of cells POST-filtering PRE-filtering 0 40 80 120 Rel. Position (deg) −0.2 0 0.2 Latency (ms) Time (s) −0.1 0 0.1 0.2 0.3 Rel. Position (deg) 0 40 80 120 Latency (ms) 0 40 80 120 Latency (ms) 0 40 80 120 Latency (ms) Rel. Position (deg) −0.2 0 0.2 Rel. Position (deg) −0.2 0 0.2 Rel. Position (deg) −0.2 0 0.2 −0.1 0.0 0.1 0.2 0.3 0.6 0.8 1.0 Time (s) Relative I SS Rel. Position (deg) −0.2 0 0.2 Time (s) −0.1 0 0.1 0.2 0.3 Rel. Position (deg) −0.2 0 0.2 Excitatory inputs Inhibitory inputs 0 140 Latency (ms) −0.1 0.0 0.1 0.2 0.3 Time (s) 0.2 0.4 0.6 I SS (bits/spk) Subspace model Gain/offset model Linear Quadratic Rel. Position (deg) Latency (ms) 0 40 80 120 Rel. Position (deg) 0 0.2 -0.2 Conclusions −0.1 0.0 0.1 0.2 0.3 0.8 0.9 1.0 1.1 1.2 Time (s) Relative rate 0.0 0.1 0.2 0.3 Modulation strength a Modulation timing τ (s) 0.4 0.8 Enhancement Suppression Not significant τ S τ E a E a S 0.0 Robust peri-saccadic firing rate modulation Enhancement amplitude Enhancement timing Suppression timing Suppression amplitude Population mean (n=84 SUs) saccade-triggered average firing rate shows clear biphasic modulation without any sac- cade-driven changes to the stimulus. Modulation timing was largely consistent across neurons, and the magnitude of modulation was typically around 30% Gray background Nat. image background Simulated saccade Darkness Relative I SS −0.1 0.0 0.1 0.2 0.3 Time (s) 0.8 1.0 1.2 Relative rate 0.6 0.8 1.0 Visual stimulation Darkness Image-back Gray-back 0.1 0.2 Relative offset 0.0 0.8 1.0 Gain 0.6 −0.1 0 0.1 0.2 0.3 Time (s) 0.8 1.0 1.2 Relative rate 0.6 0.8 1.0 Relative I SS Real Simulated 0.1 0.2 Relative offset 0.0 -0.1 0.8 1.0 Gain 0.6 0.3 -0.1 0.3 −0.2 0.0 0.2 0.4 Time (s) 0.8 1.0 1.2 Relative rate −0.2 0.0 0.2 0.4 Time (s) 0.9 1.0 1.1 Relative rate Extra-retinal origin of saccade modulation (Left): Saccades made on natural image and gray backgrounds produced nearly identical modulation. (Middle): Simulating saccade-induced shifts of the image backgrounds produce much weaker (and qualitatively different) modulation than with real saccades. (Above): Saccades made in absolute darkness produced clear biphasic modu- lation of MUA (n=96 MUs). It was much weaker, however, and had a slower time course compared to the effects of sac- cades on visually driven responses. −0.1 0.0 0.1 0.2 0.3 0.8 1.0 1.2 Time (s) Relative rate More accurate Less accurate Intra-saccadic stimulus motion is not a factor Saccade triggered average rates were similar for saccades with more and less orthogonal stimulus displacement. 0 50 100 150 Time lag (ms) −0.1 0.0 0.1 0.2 0.3 0.8 1.0 Time (s) Relative rate 0.6 0 0.15 Time (s) 0 50 100 150 0 50 100 150 Excitatory timing (ms) Inhibitory timing (ms) Saccadic suppression Stimulus response k 1 f 1 (.) k 2 + ... ... Offset kernel stimulus X(t) r(t) f 2 (.) Saccades Gain kernels Excitatory Inhibitory Stimulus response profile 0 τ I - τ E Exc. Inh. Exc. Inh. 0 50 100 150 Latency (ms) 0 50 Time (ms) 100 150 Stimulus response Gain kernel Amplitude 0 Gain 1 0 50 100 150 20 40 60 80 100 Gain suppression timing (ms) Stim-response timing (ms) Inhibitory subunit Excitatory subunit 50 100 150 Suppression timing (ms) 40 60 Stim latency (ms) 80 20 Acknowledgements RFs Fixation target Trial onset 700 ms Reward Fixation target position Time Position RF Saccade stimulus X(t) k 1 f 1 k 2 f 2 + firing rate r(t) ... ... LN inputs generating signal g(t) Recordings from two macaques using 24-electrode linear arrays (50 μm spacing) or a 96-electrode planar Utah array (400 μm spacing). Recordings were made from both foveal and parafoveal V1 (0.5 - 4.5 deg eccentricity) Uncorrelated random bar patterns (’1D ternary noise’) were pre- sented on CRT monitors, with a frame rate of 100 Hz Eye position was monitored using scleral search coils, and the coil signals were used to detect saccades and microsaccades. In addition, we used a model-based eye-tracking procedure to infer the animals’ precise eye position along the ‘orthogonal’ di- mension [9] LNLN cascade models [10; 11] were used to describe the stimu- lus processing of V1 neurons. General Methods This experimental design ensures that the stimulus in and around the RFs is minimally affected by saccades, allow- ing us to disentangle the interactions between visual signals and sac- cade-related modulatory signals Guided saccade task Time ‘Random bar stimulus’ (sparse 1d ternary noise) Independent pattern every 10 ms frame 4 second trials Models of saccade modulation Single-spike information [12]: http://brickisland.net/cs177fa12/ Piecewise linear basis functions on 2d grid Firing rate estimated as a ‘nonparametric’ function of stimu- lus-driven input g and time τ relative to saccade onset: Incorporating temporal kernels to capture peri-saccadic multi- plicative ‘gain’ and additive ‘offset’ modulation. gain kernel associated with i th input offset kernel We also utilized mutliplicative kernels that act on the stimulus before appilcation of the stimulus filters k i s(t) is 1 if there was a saccade starting at time t and 0 otherwise • Saccades and microsaccades produce robust biphasic modulation of visually driven re- sponses in V1, even in the absence of any saccade-driven changes in the stimulus. • This modulation is composed of an initial monophasic gain suppression followed by an addi- tive increase in firing rate ‘offset’. • These changes result in a suppression of stimulus selectivity (stimulus information) follow- ing saccades that could not be predicted from firing rate modulation alone. • Similar effects were observed for microsaccades, though the suppression of stimulus selec- tivity was weaker. This is consistent with the observations of [13]. • Saccade modulation was not influenced by saccade-induced changes to the stimulus in the surround, suggesting that these effects are driven by extra-retinal signals [7,14]. • The timing of modulation (across neurons, across lamina, and across the different compo- nents of a given neuron’s stimulus tuning) all strongly suggest that saccadic suppression is in- herited from the LGN [15]. • Saccadic suppression of visual perception is thought to be weak for the high spatial frequen- cies considered here [16]. Furthermore, we did not observe suppression of PRE-saccadic stimuli. These results support the possibility of distinct pathways mediating saccadic modula- tion: one targeting LGN [15] and inherited in V1, and a direct pathway from the pulvinar to MT responsible for peri-saccadic perceptual suppression [17]. Timing of saccadic modulation is tied to stimulus response latencies Laminar profile suggests LGN origin With eye tracking Without eye tracking −0.1 0.0 0.1 0.2 0.3 Time (s) 0.8 0.9 1.0 1.1 Relative I SS 0.6 0.8 1.0 Relative I SS Saccades Microsaccades Model-based eye-tracking 0.08 0.1 0.12 Fixation error (deg) −0.1 0.0 0.1 0.2 0.3 Time (s) Saccades Microsaccades No evidence for more complex peri-saccadic modulation 0.6 0.8 1.0 Relative I SS 0.4 −0.1 0.0 0.1 0.2 0.3 Time (s) 0.0 0.1 0.2 0.3 Relative offset −0.1 0.0 0.1 0.2 0.3 Time (s) -0.1 0.4 0.4 0.6 0.8 1.0 Gain −0.1 0.0 0.1 0.2 0.3 Time (s) 0.8 0.9 1.0 1.1 1.2 Relative rate 0.4 −0.1 0.0 0.1 0.2 0.3 Time (s) Differential modulation of firing rate and stimulus selectivity We separated neurons into those with strong (blue) vs. weak (red) suppression of stimulus selectivity These groups of neurons show similar firing rate modulation Separate effects into gain and ofset Strongly suppressed Weakly suppressed Effects of saccades cannot be determined by looking at firing rate modulation alone! 0.8 1.0 1.2 Relative rate −0.1 0.0 0.1 0.2 0.3 Time (s) Guided saccade Microsaccade 0.8 1.0 Response Gain −0.1 0.0 0.1 0.2 0.3 Time (s) 0.6 0.0 0.1 0.2 Relative offset −0.1 0.0 0.1 0.2 0.3 Time (s) 0.6 0.8 1.0 Relative I SS −0.1 0.0 0.1 0.2 0.3 Time (s) −0.05 0 0.1 0.8 1.0 Time (s) Gain 0.05 0.15 Similar, but weaker, modulation by microsaccades Firing rate Stimulus selectivity Gain Rate offset Microsaccades produce similar firing rate modulation to saccades, but they produce substantially weaker reductions in stimulus selectivity. Microsaccadic suppression is also better de- scribed as acting on stimulus-driven inputs ‘up- stream’, and produces suppression of stimuli occurring during the microsaccades. ‘Upstream’ Gain Saccade duration CSD analysis to estimate laminar boundaries Stimulus filters are temporally delayed in supragranular neurons The timing of saccadic sup- pression (and enhancement) is delayed in supragranular neurons compared to granu- lar and infragranular Analysis based on 100 multi-units (58 supragranular; 55 granular; 85 infragranular) across 9 recording sessions Incorporating saccade-modulation into stimulus processing models The temporal delay in a neuron’s stimulus filters was correlated with the timing of saccadic suppression (rho=0.61; p=6.4x10 -9 ; n=76) Saccadic gain suppression of inhibitory inputs was temporally delayed relative to excitatory inputs. Temporal Filter Profiles Peri-saccadic gain kernels The timing of saccadic gain suppression of a neuron’s stimulus-driven inputs was tied to the temporal delays of the stimulus filters (rho=0.53; p=2.0x10 -25 ; n=329 subunits) Models with ‘upstream’ (pre-filtering) gain modulation performed sig- nificantly better than those with post-filtering modulation (p=2.3x10 -10 ). Saccadic suppression targeted those stimuli occuring within ~50 ms after saccadic onset (essentially stimuli during the saccades). Saccade-conditional spike-triggered average stimuli show an amplitude reduction, but no clear changes in spatial structure. Models where the stimu- lus filters are estimated as functions of time rela- tive to saccades also show no evidence of peri-saccadic changes. In these models, we rep- resent each filter as a time-dependent linear combination of basis fil- ters: Time (s) 20 40 60 100 Generating signal Firing rate (Hz) 0 Firing rate (Hz) g(X) (percentile) 0 20 40 60 80 100 0 20 40 60 80 −0.1 0.0 0.1 0.2 0.3 20 24 28 Firing rate (Hz) Example Neuron Population avg. 20 24 28 Firing rate (Hz) 0.6 0.8 1.0 Response gain 0.4 0 4 8 Firing rate offset (Hz) 12 0.3 0.4 0.5 I SS (bits/spike) 6 8 10 12 I SS rate (bits/sec) 0.8 1.0 Relative rate 1.2 0.6 0.8 1.0 Response Gain 0.0 0.1 0.2 Relative offset −0.1 0.0 0.1 0.2 0.3 Time (s) Info rate Info per spike 0.4 0.8 1.0 Relative I SS 0.6 Saccades produce gain suppression and a stimulus-inde- pendent increase in firing rate −0.1 0.0 0.1 0.2 0.3 Time (s) Excitatory inputs Inhibitory inputs 0 140 Lag (ms) + Spiking nonlinearity Poisson spike generation LNLN cascade model 0.1 deg g(X) ‘Generating signal’ Nonparametric estimation of firing rate as a function of the generat- ing signal and time relative to saccade onset: Gain suppression Firing rate ‘offset’ Saccades produce gain sup- pression, followed by an addi- tive increase in firing rate ‘offset’. We quantified multiplicative ‘gain’ and additive ‘offset’ at each time τ relative to saccade onset: Peri-saccadic firing rate modulation (top) decomposes into monophasic gain suppression (upper middle), followed by a monophasic increase in firing rate offset (lower middle). This results in nearly 2-fold reduction in stimulus information (single spike info; I SS ) fol- lowing saccades, without a corresponding ‘rebound’. Guided saccades −0.1 0.0 0.1 0.2 0.3 Time (s) 0.8 1.0 1.2 Relative Rate Parallel Orthogonal Microsaccades Orthogonal amplitude bar ori θ Microsaccade angle orthogonal Saccades generally produce < 1 pixel of translational motion (orthogonal to the bar stimuli) during a video frame −0.1 0.0 0.1 0.2 0.3 Time (s) Correcting for fixation errors using model-based eye tracking greatly im- proved the accuracy of model fits (~ 2-fold). Measures of saccade modulation were qualitatively similar without using this method (assuming perfect fixation ac- curacy in the dimension orthogonal to the bars). Note, we used leave-one-out cross-validation procedures when esti- mating eye position [9]. Tracking fixation errors was more important for measuring microsaccadic modulation, as microsaccades tended to correct fixation errors. LN cascade models References [1] Wurtz, Vis Res (2008); [2] Bremmer et al., J Neurosci (2009); [3] Muller et al., J Neurosci (2001); [4] Wurtz, J Neurophysiol (1969); [5] Judge et al., J Neurophysiol (1980); [6] Gawne and Martin, J Neurophysiol (2002); [7] Kagan et al., J Vis (2008); [8] Ruiz and Paradiso, J Neurophysiol (2012); [9] McFarland et al., Nat Commun (2014); [10] Park and Pillow, NIPS (2011); [11] McFarland et al., PLoS Comp Biol (2015); [12] Brenner et al., Neural Comp (2000); [13] Hass and Horwitz, J Vis (2011); [14] Kayama et al., J Neurophysiol (1979); [15] Reppas et al., Neuron (2002); [16] Burr et al., Nature (1994); [17] Berman and Wurtz, J Neurosci (2011). This research was supported by: NEI/NIH F32EY023921 (JMM); Intramural research program at NEI/NIH (AGB, `BGC); NSF IIS-1350990 (DAB) Spiking nonlinearity F[] gain offset Every 700 ms the fixation target jumps (3-4 deg) along the long axis of th bar stim- uli, requiring the animals to make a saccade (within 300 ms) to maintain fixation. Saccade 80 0 0.05 0.1 0 Duration (s) Relative frequency Microsaccades made parallel vs. orthogonal to the bar stimuli produced similar firing rate modulation.
Transcript of Saccadic modulation of stimulus processing in primary visual cortex · 2015-03-16 · Saccadic...
Saccadic modulation of stimulus processing in primary visual cortexJames M. McFarland1, Adrian G. Bondy2,3, Bruce G. Cumming2, Daniel A. Butts1
1) Department of Biology and Program in Neuroscience and Cognitive Science, University of Maryland, College Park, MD USA2) Laboratory of Sensorimotor Research, National Eye Institute, National Institutes of Health, Bethesda, MD USA
3) Brown-NIH Neuroscience Graduate Partnership Program, Brown University, Providence, RI USA
Introduction ‘Active sampling’ of visual inputs by saccades and microsaccades is a key component of primate vision. Such eye movements produce discontinuous changes to the image on the retina, yet the visual system assimilates this constantly shifting input into a seamless visual percept. Extra-retinal signals asso-ciated with saccades have been shown to modulate the activity of neurons throughout the visual hierarchy [1-2]. The detailed stimulus processing models that have been developed in V1 make it an ideal area to study the interactions between visual inputs and peri-saccadic modulatory signals. Such studies are complicat-ed, however, by the fact that saccades produce rapid changes in the retinal input that strongly modulate the activity of V1 neurons [3]. Previous studies have attempted to distentangle these effects by comparing V1 responses to a stimulus that is either flashed into their receptive field (RF) or introduced by a saccade. These studies have generally found little effect of saccades [4-8], leading to the prevailing belief that V1 neurons encode visual stimuli indepedently of eye movements. Interpretation of these results is invariably confounded, however, by the uncontrolled variability in the precise spatiotemporal stimulus on the retina created by saccades. Further, they have generally only looked at the effects of saccades on average firing rates, and thus could not distinguish different functional forms of saccade modulation. Here we address this question using an experimental design that decouples saccades from the visual stimulus, combined with a detailed functional modeling approach.
0 0.1 0.2
0.8
1.0
1.2
Time (s)
Rel
ativ
e ra
te
Time since trial onset (s)
Rel
ativ
e de
pth
(mm
)
0 0.05 0.1 0.15 0.2
0.5
1.0
0.0
Granular
Supragranular
Infragranular
0.0 0.05 0.1 0.15Time lag (s)
Stim
ulus
resp
onse
pro
file
GranularSupragranular
Infragranular
Peak time (ms)
Supp
ress
ion
Enha
ncem
ent
Peak time (ms)
50 60 70 80
120 140 160
40 50 60
180
*
*
*
p < 0.01*
SourceSink
0
−0.1 0 0.10.6
0.8
1.0
Time (s)
Gai
n of
stim
ulus
k1
f1(.)
k2
+
...
...
Offset kernel
r(t)
f2(.) Sacc
ades
Gai
n ke
rnel
+
stimulus X(t)
Pre-filtering
Post-filtering
−0.5 0.0 0.5(LLPRE-LLPOST)/(LLPOST-LLNULL)0
10
Num
ber o
f cel
ls
POST-filtering
PRE-filtering
0
40
80
120
Rel
. Pos
ition
(deg
)
−0.2
0
0.2Late
ncy
(ms)
Time (s)−0.1 0 0.1 0.2 0.3Rel. Position (deg)
0
40
80
120
Late
ncy
(ms)
0
40
80
120
Late
ncy
(ms)
0
40
80
120
Late
ncy
(ms)
Rel
. Pos
ition
(deg
)
−0.2
0
0.2
Rel
. Pos
ition
(deg
)
−0.2
0
0.2
Rel
. Pos
ition
(deg
)
−0.2
0
0.2
−0.1 0.0 0.1 0.2 0.3
0.6
0.8
1.0
Time (s)
Rel
ativ
e I SS
Rel
. Pos
ition
(deg
)
−0.2
0
0.2
Time (s)−0.1 0 0.1 0.2 0.3
Rel
. Pos
ition
(deg
)
−0.2
0
0.2
Excitatory inputs Inhibitory inputs
0
140
Late
ncy
(ms)
−0.1 0.0 0.1 0.2 0.3Time (s)
0.2
0.4
0.6
I SS (b
its/s
pk)
Subspace modelGain/offset model
Line
arQ
uadr
atic
Rel. Position (deg)
Late
ncy
(ms)
0
40
80
120
Rel. Position (deg)0 0.2-0.2
Conclusions
−0.1 0.0 0.1 0.2 0.3
0.8
0.9
1.0
1.1
1.2
Time (s)
Rel
ativ
e ra
te
0.0
0.1
0.2
0.3
Modulation strength a
Mod
ulat
ion
timin
g τ
(s)
0.4 0.8
EnhancementSuppressionNot significant
τS
τE
aE
aS
0.0
Robust peri-saccadic firing rate modulation
Enhancement amplitude
Enhancement timing
Suppression timing
Suppression amplitude
Population mean (n=84 SUs) saccade-triggered average firing rate shows clear biphasic modulation without any sac-cade-driven changes to the stimulus.
Modulation timing was largely consistent across neurons, and the magnitude of modulation was typically around 30%
Gray background Nat. image background Simulated saccade Darkness
Rel
ativ
e I SS
−0.1 0.0 0.1 0.2 0.3Time (s)
0.8
1.0
1.2
Rel
ativ
e ra
te
0.6
0.8
1.0
Visual stimulationDarkness
Image-backGray-back
0.1
0.2
Rel
ativ
e of
fset
0.0
0.8
1.0
Gai
n
0.6
−0.1 0 0.1 0.2 0.3Time (s)
0.8
1.0
1.2
Rel
ativ
e ra
te
0.6
0.8
1.0
Rel
ativ
e I SS
RealSimulated
0.1
0.2
Rel
ativ
e of
fset
0.0
-0.1
0.8
1.0
Gai
n
0.6
0.3
-0.1
0.3
−0.2 0.0 0.2 0.4Time (s)
0.8
1.0
1.2
Rel
ativ
e ra
te
−0.2 0.0 0.2 0.4Time (s)
0.9
1.0
1.1
Rel
ativ
e ra
te
Extra-retinal origin of saccade modulation
(Left): Saccades made on natural image and gray backgrounds produced nearly identical modulation.
(Middle): Simulating saccade-induced shifts of the image backgrounds produce much weaker (and qualitatively different) modulation than with real saccades.
(Above): Saccades made in absolute darkness produced clear biphasic modu-lation of MUA (n=96 MUs). It was much weaker, however, and had a slower time course compared to the effects of sac-cades on visually driven responses.
−0.1 0.0 0.1 0.2 0.3
0.8
1.0
1.2
Time (s)
Rel
ativ
e ra
te
More accurateLess accurate
Intra-saccadic stimulus motion is not a factor
Saccade triggered average rates were similar for saccades with more and less orthogonal stimulus displacement.
0 50 100 150Time lag (ms)
−0.1 0.0 0.1 0.2 0.3
0.8
1.0
Time (s)
Rel
ativ
e ra
te
0.6
0 0.15Time (s)
0 50 100 1500
50
100
150
Excitatory timing (ms)
Inhi
bito
ry ti
min
g (m
s)
Saccadic suppressionStimulus response
k1
f1(.)
k2
+
...
...
Offset kernel
stimulus X(t)
r(t)
f2(.)
Sacc
ades
Gain kernels
ExcitatoryInhibitory
Stim
ulus
resp
onse
pro
file
0
τI - τE
Exc.
Inh.
Exc.
Inh.
0 50 100 150Latency (ms)
0 50Time (ms)
100 150
Stimulus response
Gain kernel
Am
plitu
de
0
Gai
n
10 50 100 150
20
40
60
80
100
Gain suppression timing (ms)
Stim
-res
pons
e tim
ing
(ms)
Inhibitory subunitExcitatory subunit
50
100
150
Supp
ress
ion
timin
g (m
s)
40 60Stim latency (ms)
8020
Acknowledgements
RFsFixation target
Trial onset 700 msReward
Fixa
tion
targ
et p
ositi
on
Time
Posi
tion
RF
Saccade
stimulus X(t)
k1
f1
k2
f2
+
firing rate r(t)
...
...LN in
puts
generating signal g(t)
Recordings from two macaques using 24-electrode linear arrays (50 μm spacing) or a 96-electrode planar Utah array (400 μm spacing).
Recordings were made from both foveal and parafoveal V1 (0.5 - 4.5 deg eccentricity)
Uncorrelated random bar patterns (’1D ternary noise’) were pre-sented on CRT monitors, with a frame rate of 100 Hz
Eye position was monitored using scleral search coils, and the coil signals were used to detect saccades and microsaccades.
In addition, we used a model-based eye-tracking procedure to infer the animals’ precise eye position along the ‘orthogonal’ di-mension [9]
LNLN cascade models [10; 11] were used to describe the stimu-lus processing of V1 neurons.
General
Methods
This experimental design ensures that the stimulus in and around the RFs is minimally affected by saccades, allow-ing us to disentangle the interactions between visual signals and sac-cade-related modulatory signals
Guided saccade task
Time
‘Random bar stimulus’ (sparse 1d ternary noise)Independent pattern every 10 ms frame
4 second trials
Models of saccade modulation
Single-spike information [12]:
http://brickisland.net/cs177fa12/
Piecewise linear basis functions on 2d gridFiring rate estimated as a ‘nonparametric’ function of stimu-lus-driven input g and time τ relative to saccade onset:
Incorporating temporal kernels to capture peri-saccadic multi-plicative ‘gain’ and additive ‘offset’ modulation.
gain kernel associated with ith input offset kernel
We also utilized mutliplicative kernels that act on the stimulus before appilcation of the stimulus filters ki
s(t) is 1 if there was a saccade starting at time t and 0 otherwise
• Saccades and microsaccades produce robust biphasic modulation of visually driven re-sponses in V1, even in the absence of any saccade-driven changes in the stimulus.
• This modulation is composed of an initial monophasic gain suppression followed by an addi-tive increase in firing rate ‘offset’.
• These changes result in a suppression of stimulus selectivity (stimulus information) follow-ing saccades that could not be predicted from firing rate modulation alone.
• Similar effects were observed for microsaccades, though the suppression of stimulus selec-tivity was weaker. This is consistent with the observations of [13].
• Saccade modulation was not influenced by saccade-induced changes to the stimulus in the surround, suggesting that these effects are driven by extra-retinal signals [7,14].
• The timing of modulation (across neurons, across lamina, and across the different compo-nents of a given neuron’s stimulus tuning) all strongly suggest that saccadic suppression is in-herited from the LGN [15].
• Saccadic suppression of visual perception is thought to be weak for the high spatial frequen-cies considered here [16]. Furthermore, we did not observe suppression of PRE-saccadic stimuli. These results support the possibility of distinct pathways mediating saccadic modula-tion: one targeting LGN [15] and inherited in V1, and a direct pathway from the pulvinar to MT responsible for peri-saccadic perceptual suppression [17].
Timing of saccadic modulation is tied to stimulus response latencies
Laminar profile suggests LGN origin
With eye tracking
Without eye tracking
−0.1 0.0 0.1 0.2 0.3Time (s)
0.8
0.9
1.0
1.1
Rel
ativ
e I SS
0.6
0.8
1.0
Rel
ativ
e I SS
Saccades
Microsaccades
Model-based eye-tracking
0.08
0.1
0.12
Fixa
tion
erro
r (de
g)
−0.1 0.0 0.1 0.2 0.3Time (s)
Saccades
Microsaccades
No evidence for more complex peri-saccadic modulation
0.6
0.8
1.0
Rel
ativ
e I SS
0.4−0.1 0.0 0.1 0.2 0.3
Time (s)
0.0
0.1
0.2
0.3
Rel
ativ
e of
fset
−0.1 0.0 0.1 0.2 0.3Time (s)
-0.1
0.4
0.4
0.6
0.8
1.0
Gai
n
−0.1 0.0 0.1 0.2 0.3Time (s)
0.8
0.9
1.0
1.1
1.2
Rel
ativ
e ra
te
0.4−0.1 0.0 0.1 0.2 0.3Time (s)
Differential modulation of firing rate and stimulus selectivity
We separated neurons into those with strong (blue) vs. weak (red) suppression of stimulus selectivity
These groups of neurons show similar firing rate modulation
Separate effects into gain and ofset
Strongly suppressed Weakly suppressed
Effects of saccades cannot be determined by looking at firing rate modulation alone!
0.8
1.0
1.2
Rel
ativ
e ra
te
−0.1 0.0 0.1 0.2 0.3Time (s)
Guided saccade Microsaccade
0.8
1.0
Res
pons
e G
ain
−0.1 0.0 0.1 0.2 0.3Time (s)
0.6 0.0
0.1
0.2
Rel
ativ
e of
fset
−0.1 0.0 0.1 0.2 0.3Time (s)
0.6
0.8
1.0
Rel
ativ
e I SS
−0.1 0.0 0.1 0.2 0.3Time (s)
−0.05 0 0.1
0.8
1.0
Time (s)
Gai
n
0.05 0.15
Similar, but weaker, modulation by microsaccades
Firing rate Stimulus selectivity
Gain Rate offset
Microsaccades produce similar firing rate modulation to saccades, but they produce substantially weaker reductions in stimulus selectivity.
Microsaccadic suppression is also better de-scribed as acting on stimulus-driven inputs ‘up-stream’, and produces suppression of stimuli occurring during the microsaccades.
‘Upstream’ Gain
Saccade duration
CSD analysis to estimate laminar boundaries Stimulus filters are temporally delayed in supragranular neurons
The timing of saccadic sup-pression (and enhancement) is delayed in supragranular neurons compared to granu-lar and infragranular
Analysis based on 100 multi-units (58 supragranular; 55 granular; 85 infragranular) across 9 recording sessions
Incorporating saccade-modulation into stimulus processing models
The temporal delay in a neuron’s stimulus filters was correlated with the timing of saccadic suppression (rho=0.61; p=6.4x10-9; n=76)
Saccadic gain suppression of inhibitory inputs was temporally delayed relative to excitatory inputs.
Temporal Filter Profiles
Peri-saccadic gain kernels
The timing of saccadic gain suppression of a neuron’s stimulus-driven inputs was tied to the temporal delays of the stimulus filters(rho=0.53; p=2.0x10-25; n=329 subunits) Models with ‘upstream’ (pre-filtering) gain modulation performed sig-
nificantly better than those with post-filtering modulation (p=2.3x10-10).
Saccadic suppression targeted those stimuli occuring within ~50 ms after saccadic onset (essentially stimuli during the saccades).
Saccade-conditional spike-triggered average stimuli show an amplitude reduction, but no clear changes in spatial structure.
Models where the stimu-lus filters are estimated as functions of time rela-tive to saccades also show no evidence of peri-saccadic changes. In these models, we rep-resent each filter as a time-dependent linear combination of basis fil-ters:
Time (s)
20
40
60
100
Generating signal
Firin
g ra
te (H
z)
0
Firin
g ra
te (H
z)
g(X
) (pe
rcen
tile)
0
20
40
60
80
100
0
20
40
6080
−0.1 0.0 0.1 0.2 0.3
20
24
28
Firin
g ra
te (H
z)
Example Neuron Population avg.
20
24
28
Firin
g ra
te (H
z)
0.6
0.8
1.0
Res
pons
e ga
in
0.4
0
4
8Fi
ring
rate
off
set (
Hz)
12
0.3
0.4
0.5
I SS (b
its/s
pike
)
6
8
10
12
I SS ra
te (b
its/s
ec)
0.8
1.0
Rel
ativ
e ra
te
1.2
0.6
0.8
1.0
Res
pons
e G
ain
0.0
0.1
0.2
Rel
ativ
e of
fset
−0.1 0.0 0.1 0.2 0.3Time (s)
Info rateInfo per spike
0.4
0.8
1.0
Rel
ativ
e I SS
0.6
Saccades produce gain suppression and a stimulus-inde-pendent increase in firing rate
−0.1 0.0 0.1 0.2 0.3Time (s)
Excitatory inputs Inhibitory inputs
0
140
Lag
(ms)
+
Spiking nonlinearity Poisson spike generation
LNLN cascade model
0.1 deg
g(X) ‘Generating signal’
Nonparametric estimation of firing rate as a function of the generat-ing signal and time relative to saccade onset:
Gain suppression
Firing rate ‘offset’
Saccades produce gain sup-pression, followed by an addi-tive increase in firing rate ‘offset’.
We quantified multiplicative ‘gain’ and additive ‘offset’ at each time τ relative to saccade onset:
Peri-saccadic firing rate modulation (top) decomposes into monophasic gain suppression (upper middle), followed by a monophasic increase in firing rate offset (lower middle). This results in nearly 2-fold reduction in stimulus information (single spike info; ISS) fol-lowing saccades, without a corresponding ‘rebound’.
Guided saccades
−0.1 0.0 0.1 0.2 0.3Time (s)
0.8
1.0
1.2
Rel
ativ
e R
ate
ParallelOrthogonal
Microsaccades
Orthogonal amplitude
bar ori θ
Microsaccade angle
orthogonal
Saccades generally produce < 1 pixel of translational motion (orthogonal to the bar stimuli) during a video frame
−0.1 0.0 0.1 0.2 0.3Time (s)
Correcting for fixation errors using model-based eye tracking greatly im-proved the accuracy of model fits (~ 2-fold).
Measures of saccade modulation were qualitatively similar without using this method (assuming perfect fixation ac-curacy in the dimension orthogonal to the bars).
Note, we used leave-one-out cross-validation procedures when esti-mating eye position [9].
Tracking fixation errors was more important for measuring microsaccadic modulation, as microsaccades tended to correct fixation errors.
LN cascade models
References[1] Wurtz, Vis Res (2008); [2] Bremmer et al., J Neurosci (2009); [3] Muller et al., J Neurosci (2001); [4] Wurtz, J Neurophysiol (1969); [5] Judge et al., J Neurophysiol (1980); [6] Gawne and Martin, J Neurophysiol (2002); [7] Kagan et al., J Vis (2008); [8] Ruiz and Paradiso, J Neurophysiol (2012); [9] McFarland et al., Nat Commun (2014); [10] Park and Pillow, NIPS (2011); [11] McFarland et al., PLoS Comp Biol (2015); [12] Brenner et al., Neural Comp (2000); [13] Hass and Horwitz, J Vis (2011); [14] Kayama et al., J Neurophysiol (1979); [15] Reppas et al., Neuron (2002); [16] Burr et al., Nature (1994); [17] Berman and Wurtz, J Neurosci (2011).
This research was supported by: NEI/NIH F32EY023921 (JMM); Intramural research program at NEI/NIH (AGB, `BGC); NSF IIS-1350990 (DAB)
Spiking nonlinearity F[]
gain offset
Every 700 ms the fixation target jumps (3-4 deg) along the long axis of th bar stim-uli, requiring the animals to make a saccade (within 300 ms) to maintain fixation.
Saccade
80
0 0.05 0.10
Duration (s)
Rel
ativ
e fr
eque
ncy
Microsaccades made parallel vs. orthogonal to the bar stimuli produced similar firing rate modulation.
