Forgetting. Encoding Failure Encoding failure Encoding Failure Encoding failure.
Direct Brain Stimulation Modulates Encoding States and...
Transcript of Direct Brain Stimulation Modulates Encoding States and...
Article
Direct Brain Stimulation M
odulates Encoding Statesand Memory Performance in HumansHighlights
d Intracranial brain stimulation has variable effects on episodic
memory performance
d Stimulation increased memory performance when delivered
in poor encoding states
d Recall-related brain activity increased after stimulation of
poor encoding states
d Neural activity linked to contextual memory predicted
encoding state modulation
Ezzyat et al., 2017, Current Biology 27, 1–8May 8, 2017 ª 2017 Elsevier Ltd.http://dx.doi.org/10.1016/j.cub.2017.03.028
Authors
Youssef Ezzyat, James E. Kragel,
John F. Burke, ..., Richard Gorniak,
Daniel S. Rizzuto, Michael J. Kahana
In Brief
Direct brain stimulation is a promising
tool for modulating cognitive function.
Ezzyat et al. show that stimulation
differentially affects episodic memory
encoding depending on its timing relative
to the brain’s encoding state. The data
suggest applications for closed-loop
treatment of memory dysfunction.
Please cite this article in press as: Ezzyat et al., Direct Brain Stimulation Modulates Encoding States and Memory Performance in Humans, CurrentBiology (2017), http://dx.doi.org/10.1016/j.cub.2017.03.028
Current Biology
Article
Direct Brain Stimulation Modulates EncodingStates and Memory Performance in HumansYoussef Ezzyat,1 James E. Kragel,1 John F. Burke,2 Deborah F. Levy,1 Anastasia Lyalenko,1 Paul Wanda,1
Logan O’Sullivan,1 Katherine B. Hurley,1 Stanislav Busygin,1 Isaac Pedisich,1 Michael R. Sperling,3 Gregory A. Worrell,5
Michal T. Kucewicz,5 Kathryn A. Davis,6 Timothy H. Lucas,7 Cory S. Inman,9 Bradley C. Lega,10 Barbara C. Jobst,11
Sameer A. Sheth,12 Kareem Zaghloul,13 Michael J. Jutras,14 Joel M. Stein,8 Sandhitsu R. Das,6 Richard Gorniak,4
Daniel S. Rizzuto,1,15 and Michael J. Kahana1,15,16,*1Department of Psychology, University of Pennsylvania, Philadelphia, PA 19104, USA2Department of Neurological Surgery, University of California, San Francisco Medical Center, San Francisco, CA 94143, USA3Department of Neurology4Department of Radiology
Thomas Jefferson University Hospital, Philadelphia, PA 19107, USA5Department of Neurology, Mayo Clinic, Rochester, MN 55905, USA6Department of Neurology7Department of Neurosurgery8Department of Radiology
Hospital of the University of Pennsylvania, Philadelphia, PA 19104, USA9Department of Neurosurgery, Emory University Hospital, Atlanta, GA 30322, USA10Department of Neurosurgery, University of Texas Southwestern, Dallas, TX 75390, USA11Department of Neurology, Dartmouth-Hitchcock Medical Center, Lebanon, NH 03756, USA12Department of Neurosurgery, Columbia University Medical Center, New York, NY 10032, USA13Surgical Neurology Branch, National Institutes of Health, Bethesda, MD 20814, USA14Department of Physiology and Biophysics, University of Washington, Seattle, WA 98195, USA15These authors contributed equally16Lead Contact*Correspondence: [email protected]
http://dx.doi.org/10.1016/j.cub.2017.03.028
SUMMARY
People often forget information because they failto effectively encode it. Here, we test the hypo-thesis that targeted electrical stimulation canmodulate neural encoding states and subsequentmemory outcomes. Using recordings from neuro-surgical epilepsy patients with intracranially im-planted electrodes, we trained multivariate classi-fiers to discriminate spectral activity duringlearning that predicted remembering from forget-ting, then decoded neural activity in later ses-sions in which we applied stimulation duringlearning. Stimulation increased encoding-state es-timates and recall if delivered when the classifierindicated low encoding efficiency but had thereverse effect if stimulation was delivered whenthe classifier indicated high encoding efficiency.Higher encoding-state estimates from stimulationwere associated with greater evidence of neuralactivity linked to contextual memory encoding.In identifying the conditions under which stimu-lation modulates memory, the data suggeststrategies for therapeutically treating memorydysfunction.
INTRODUCTION
Memory depends on encoding processes that lay down neural
representations of experiences for long-term storage [1]. Re-
cordings taken during laboratory memory tasks demonstrate
that neural activity in the hippocampus, medial temporal lobe
(MTL) cortex, frontal lobe, and parietal lobe [2, 3] differentiates
learned information that is likely to be remembered from informa-
tion likely to be forgotten. These effects extend to other brain
areas [4] and exist both during and prior to when a to-be-remem-
bered stimulus is present [5–8]. This suggests that coordinated
activity in a distributed neural network generates states that
are responsible for effective memory encoding.
If variability in distributed neural network activity reflects
fluctuation of encoding states that leads to differences in
memory performance, then it should be possible to modulate
memory by perturbing the brain’s encoding state directly [9].
We test this hypothesis using electrical stimulation delivered
through electrodes implanted in the brains of epilepsy patients.
Direct electrical stimulation allows for targeting focal brain
structures in order to modulate activity in complex neural net-
works [10–12] and can be precisely timed to target specific en-
coding events, offering some advantages over non-invasive
methods [13].
We predicted that stimulation’s effects on memory would
depend on the brain’s encoding state at the time it is delivered.
If the memory network is operating efficiently, stimulation should
Current Biology 27, 1–8, May 8, 2017 ª 2017 Elsevier Ltd. 1
Recalled
Forgotten
TAPEMOLE
DADFLOOD
2+3+1
Encoding (30 sec)
Electrode
“mole” ...
Freq
uenc
y
Distractor (20 s) Recall (30 s)
“tape”
A
B
...
wTX
Pro
babi
lity
LogisticRegression
...
8+4+51+6+7
9+7+2
...
...
False Alarm Rate
Hit
Rat
e
AUC
Classifier Performance
Electrode
Freq
uenc
y
...
...
Figure 1. Experimental Design and Analysis
(A) Subjects performed delayed free recall while intracranially implanted
electrodes recorded local field potentials simultaneously across multiple re-
gions of the brain.
(B) The electrode frequency pattern of spectral power for each word-encoding
period was used as input (X) to fit a classifier to discriminate recalled from
forgotten patterns (resulting weight; w). We assessed classifier performance
using area under the receiver-operating characteristic curve (AUC).
See also Figure S1.
Please cite this article in press as: Ezzyat et al., Direct Brain Stimulation Modulates Encoding States and Memory Performance in Humans, CurrentBiology (2017), http://dx.doi.org/10.1016/j.cub.2017.03.028
interfere with the encoding process and thus later memory. How-
ever, if the memory network is not operating efficiently, we pre-
dicted that stimulation should disrupt dysfunctional encoding
activity and therefore facilitate memory. A mechanism whereby
stimulation disrupts dysfunctional brain networks is thought to
explain the success in using deep brain stimulation (DBS) of tha-
lamocortical circuits in treating motor dysfunction in Parkinson’s
disease [14, 15].
To use stimulation to modulate encoding states, we first
needed to reliably identify neural activity conducive to successful
memory. There is evidence that theta activity in the hippocam-
pus and MTL cortex prior to stimulus presentation predicts
memory [5, 6, 8], and pre-stimulus theta activity has been used
to trigger learning trials and improve performance in animal
models of classical conditioning [16]. However, similar ap-
proaches in humans using MTL activity in the form of intracranial
theta [17] have not reliably modulated memory performance. We
hypothesized that we could derive a more sensitive index of
memory function by estimating encoding states that reflect
global memory function, as opposed to specific operations car-
ried out in focal brain areas.
To do so, we simultaneously measured neural activity across
the brain. We recorded intracranial electroencephalography
(iEEG) signals from subdural and depth electrodes implanted in
patients with medically refractory epilepsy undergoing clinical
monitoring to determine seizure onset foci. Subjects performed
free recall, amemory task sensitive tomany types of neurological
dysfunction [18, 19] and whose cognitive basis has been
modeled by multiple computational mechanisms [20]. We then
used multivariate classification to test whether a classifier could
predict the probability of recall success from patterns of neural
activity recorded across the brain during encoding. In this way,
we took advantage of our access to many recording channels
2 Current Biology 27, 1–8, May 8, 2017
to derive a subject-specificmodel that could differentiate encod-
ing states likely to lead to remembering from states likely to lead
to forgetting.
Using multivariate classification was advantageous in another
way. Direct electrical stimulation has multifaceted effects on
neural activity that are local and remote relative to the site of
stimulation [21, 22], and that depend on the baseline excitability
of the targeted neural population at the time stimulation is deliv-
ered [23–26]. This poses a challenge when trying to predict the
brain structures stimulation is likely to excite or inhibit and its
consequent effects on behavior. Stimulating hippocampal and
MTL cortical targets in humans, for example, leads to inconsis-
tent and modest effects on memory, with some studies suggest-
ing memory facilitation [27–29] and others showing memory
disruption [30–34].
We addressed this problem by using each subject’s classifier
trained on record-only sessions to decode patterns of neural ac-
tivity during later stimulation sessions. The classifier served as a
model that allowed us to assess evidence for the presence of
good encoding states before and after stimulation and control
periods. The estimates from the classifier integrate information
across electrodes and frequencies, which we predicted would
account for heterogeneity in stimulation’s physiological effects
across people. We targeted stimulation to electrodes placed in
nodes of the memory network: if available within the electrode
montage, we stimulated a single MTL structure (hippocampus
or entorhinal, perirhinal, or parahippocampal cortex) in a given
session. For subjects without MTL contacts, we stimulated other
structures linked to memory encoding, such as the prefrontal
and parietal cortex [3], which we selected by identifying the con-
tact that showed the largest subsequent memory effect in the
high-frequency range (70–200 Hz), a marker of successful mem-
ory encoding that correlates with multi-unit neural firing [35].
RESULTS
Multivariate Classification of Encoding Activity PredictsLater RecallOne hundred and two subjects participated in the record-only
phase of the study. Subjects performed a free recall memory
task during which they studied at least 25 lists of 12 unrelated
words, with each list followed by a 20 smental arithmetic distrac-
tor task (a subset of subjects also performed additional sessions
of free recall with categorized word lists). Subjects then freely re-
called the words from the list in any order (Figure 1A; mean
recall = 27.2% ± 1.2%; SEM). For each encoded word, we
computed the time-frequency decomposition of the iEEG signal
for each bipolar electrode pair (50 frequencies between 1 and
200 Hz; Figure 1B). We used these estimates of spectral power
at each frequency and electrode, for each encoded word, as
input for training a logistic regression classifier. We employed
L2 penalization to avoid overfitting, then assessed performance
using N � 1 cross-validation by experimental session and area
under the receiver-operating characteristic curve (AUC), a stan-
dard measure of a classifier’s ability to generate true positives
while avoiding false positives.
Figures 2A–2D show data from two subjects for eight example
encoding lists. The classifier generated higher probabilities for
recalled than forgotten items in these periods (Figures 2A and
A B
DC
GFEWord Encoding Trial
Cla
ssifi
er O
utpu
tP
roba
bilit
y
0
0.2
0.4
0.6
0.8
1
List 1 List 2 List 3 List 4 List 5 List 6 List 7 List 8
Word Encoding Trial
Cla
ssifi
er O
utpu
tP
roba
bilit
y
0
0.2
0.4
0.6
0.8
1
List 1 List 2 List 3 List 4 List 5 List 6 List 7 List 8
False Alarm Rate0.2 0.4 0.6 0.8 1
Hit
Rat
e
0.2
0.4
0.6
0.8
1
AUC = .76
False Alarm Rate0.2 0.4 0.6 0.8 1
Hit
Rat
e
0.2
0.4
0.6
0.8
1
AUC = .73
RegionIF
GM
FGSFG
MTLC
Hipp TCIP
CSPC
OC
Fre
quen
cy (
Hz)
2
3
5
8
13
23
40
68
116
200
Mod
el-B
ased
Act
ivat
ion
-0.5
-0.25
0
0.25
0.5
RegionIF
GM
FGSFG
MTLC
Hipp TCIP
CSPC
OCF
requ
ency
(H
z)
2
3
5
8
13
23
40
68
116
200
Sub
sequ
ent M
emor
y (t
-sta
t)
-0.8
-0.4
0
0.4
0.8
False Alarm Rate0.2 0.4 0.6 0.8 1
Hit
Rat
e
0.2
0.4
0.6
0.8
1
Mean AUC: 0.63t(101) = 19.2, P < 10-10
Figure 2. Classifier Performance
(A and C) Classifier output probability for an eight-list period of the delayed free recall task in two example subjects. Dashed lines indicate the optimal decision
threshold dividing recalled from forgotten trials. Red, later recalled words; blue, later forgotten words. (A) Example patient 1. (C) Example patient 2.
(B and D) AUC for both subjects was significantly greater than chance. (B) Example patient 1. (D) Example patient 2.
(E) Individual receiver-operating characteristic (ROC) curves plotted for all subjects.
(F) Forward-model-derived estimates of classification importance for each electrode region 3 frequency feature, grouped into anatomical regions of interest.
(G) Subsequent memory analysis contrasting encoding power for later recalled words with later not recalled words.
(F andG) IFG, inferior frontal gyrus;MFG,middle frontal gyrus; SFG, superior frontal gyrus;MTLC,medial temporal lobe cortex; Hipp, hippocampus; TC, temporal
cortex; IPC, inferior parietal cortex; SPC, superior parietal cortex; OC, occipital cortex. Data are multiple comparisons corrected using false discovery rate (FDR)
at q = 0.05.
See also Figures S2 and S3.
Please cite this article in press as: Ezzyat et al., Direct Brain Stimulation Modulates Encoding States and Memory Performance in Humans, CurrentBiology (2017), http://dx.doi.org/10.1016/j.cub.2017.03.028
2C) and across all encoded words as measured using AUC (Fig-
ures 2B and 2D). Across subjects, classification performance ex-
ceeded chance (mean AUC 0.63 ± 0.07, t(101) = 19.2, p < 10�10;
Figure 2E), which indicates that our approach can identify
subject-specific features of brain activity during encoding that
predict memory. We next asked whether the features that
were important to classification were idiosyncratic or instead re-
flected consistent activity in similar brain regions and at similar
frequencies.
We derived a forward model [36] for each subject using clas-
sifier weights, accounting for covariance between features in
the input data, to estimate the relative importance of each region
3 frequency feature for classifier performance. This showed that
the classifier relied on widespread low-frequency power de-
creases simultaneous with high-frequency power increases
across the frontal, temporal, and occipital cortex, as well as in
the hippocampus, to predict successful recall (Figure 2F). We
observed a similar pattern when contrasting power for remem-
bered and forgotten words (Figure 2G), with theMTL and parietal
cortex also showing high-frequency power increases. This
echoes prior work in intracranial and scalp EEG [37] and sug-
gests consistency in the features that predict efficient memory
function across people.
Stimulation Has Heterogeneous Effects on MemoryPerformanceHaving established that classification discriminates encoding
states, we asked whether stimulation modulates these states
in a way that influences memory performance. On stimulation
(Stim) lists, we applied 50-Hz trains across a single pair of elec-
trodes at parameters previously used to modulate spatial mem-
ory in humans [28]. We then used each subject’s record-only
classifier to decode neural activity during stimulation sessions
(N = 52 stimulation datasets from 36 subjects). We first tested
classifier generalization to the stimulation sessions. Our experi-
mental design included lists without stimulation (NoStim lists)
to serve as a baseline for behavioral performance and for testing
between-session classifier generalization (Figure 3A). The classi-
fier significantly discriminated encoding activity for recalled
and forgotten words (mean AUC on NoStim lists 0.61 ± 0.01,
Current Biology 27, 1–8, May 8, 2017 3
Serial Position4 8 12
Pro
babi
lity
ofR
ecal
l
0
0.4
0.8
Lag-5 -3 -1 1 3 5
Con
d. R
espo
nse
Pro
babi
lity
0
0.4
0.8
Stim ListsNoStim Lists
Record-Only
Normalized Stim-No Stim Recall-75 -50 -25 0 25 50 75
# P
atie
nt S
timul
atio
n S
essi
ons
0
4
8
12
16 Mean = -6.8%P < 0.04
LOCK
FUR
Stim Lists
RING
HEN
NoStim Lists
50 Hz
A
B
D
C
Figure 3. Classifier Output Predicts the Effect of Stimulation on
Memory
(A) We applied stimulation across alternating pairs of words on Stim lists;
NoStim lists were devoid of stimulation.
(B) The effect of stimulation on memory performance varied across subjects
(SD 22.5%); mean, red dashed line.
(C and D) Recall probability as a function of serial position (C) and inter-item lag
(D) does not significantly differ as a function of stimulation condition.
See also Table S1.
LOCK FUR GOLD ...
PreS
timD
ata
...
Recalled?
Low High
Low-High State Difference
RegionIF
GM
FGSFG
MTL
Hipp TCIP
CSPC
OC
Fre
quen
cy (
Hz)
2
3
5
8
13
23
40
68
116
200
Mea
n t-
stat
-8
-6
-4
-2
0
2
4
6
8
P < 0.003
P < 0.03 P < 0.05
Stim-NoStim Classifier (t-stat)-5 0 5
N
0
2
4
6
8
10
12Mean = 0.67P < 0.01
Low States
-5 0 5
N
0
2
4
6
8
10
12Mean = -1.52
P < 0.001
High States
Stim-NoStim Classifier (t-stat)
Low High
Stim
-NoS
tim N
orm
aliz
ed R
ecal
l
-250
-150
-50
50
150
Preceding State
A
B
D E
C
Figure 4. The Effect of Stimulation Depends on Brain State
(A) Classifier decoding prior to stimulation onset allowed us to analyzememory
performance based on pre-stimulation brain state.
(B) Spectral power prior to stimulation onset was significantly lower at high
frequencies in frontal, temporal, parietal, and occipital cortex (FDR corrected
at q = 0.05).
(C) Recall performance increased if stimulation was delivered when the brain
was in a low encoding state (p < 0.03) and decreased if delivered in a high
encoding state (p < 0.05). The difference between low and high stimulation was
also significant (p < 0.003). Red bars show mean SE of the difference.
(D) Stimulation significantly increased classifier output when delivered at low
encoding states (p < 0.01).
(E) Stimulation significantly decreased classifier output when delivered at high
encoding states (p < 0.001).
See also Table S2.
Please cite this article in press as: Ezzyat et al., Direct Brain Stimulation Modulates Encoding States and Memory Performance in Humans, CurrentBiology (2017), http://dx.doi.org/10.1016/j.cub.2017.03.028
t(51) = 10.4, p < 10�10), even though recall performance was
slightly higher for NoStim lists compared to record-only ses-
sions (record-only 30.6% ± 1.7%, NoStim lists 33.5% ± 1.9%,
t(51) = 2.6, p < 0.02). This suggests that the relation between
neural activity and memory states was stable from the record-
only to the stimulation sessions.
We next asked whether encoding stimulation tended to facili-
tate or disrupt recall performance. Within-subject, stimulation
significantly increased recall performance in two subjects and
decreased recall performance in six subjects (c2 test, p <
0.05). Across the group, stimulation reliably decreased memory
performance (D normalized recall �6.8% ± 3.2%, p < 0.04; Fig-
ure 3B), but there was considerable variability in stimulation’s
effects across individuals, ranging from large memory disruption
to facilitation (SD = 22.5%). This heterogeneity is consistent with
past work [27–30, 32–34], and meant that the small difference in
overall recall performance was not accompanied by specific
group-level differences in recall organization, measured using
two traditional assays of human memory performance, the serial
position curve and lag conditional response probability curve
(Figures 3C and 3D).
Stimulation’s Behavioral and Neural Effects Are StateDependentAlthough stimulation had inconsistent effects overall, we pre-
dicted the pre-stimulation encoding state would account for
some of this variability. In subjects who showed above-chance
classifier generalization (N = 39 datasets from 27 subjects), we
applied the classifiers to intervals just prior to each stimulation
train (Figure 4A) and split the resulting distribution of classifier
outputs into low and high bins, based on the optimal classifica-
tion threshold from the previous record-only sessions. Low pre-
stimulation encoding states were associated with decreased
4 Current Biology 27, 1–8, May 8, 2017
high-frequency power in widespread brain areas that predicted
memory performance during word encoding, including the
frontal, temporal, and parietal cortex (Figure 4B).
Stimulation enhanced recall performance when delivered
just after low encoding states (t(38) = 2.26, p < 0.03) but
decreased performance when delivered just after high encoding
states (t(38) = �2.09, p < 0.05; low-high difference t(38) = 3.32,
p < 0.003; Figure 4C). We compared the classifier estimates of
the brain’s encoding state post- and pre-stimulation, which
showed that low-state stimulation increased evidence for good
encoding (p < 0.02; Figure 4D) whereas high-state stimulation
decreased evidence for good encoding (p < 0.001; Figure 4E).
The data suggest that stimulation influences memory function
by perturbing the brain’s encoding state relative to its status at
the time of delivery.
Evoked Spectral Tilt after Poor Encoding StatesWhen stimulation was delivered in low states, both recall perfor-
mance and classifier evidence increased. We next asked how
Figure 5. Correlation between the Stimulation-Related Change
in Classifier Output and the Spectral Tilt Effect: (High-Frequency
Activity t Stat) – (Low-Frequency Activity t Stat)
Please cite this article in press as: Ezzyat et al., Direct Brain Stimulation Modulates Encoding States and Memory Performance in Humans, CurrentBiology (2017), http://dx.doi.org/10.1016/j.cub.2017.03.028
increased classifier evidence relates to stimulation-evoked
changes in neural activity across the brain. To measure stimula-
tion’s effect on neural activity, we used an index of the spectral
tilt, which is characterized by increased high-frequency power
simultaneous with widespread decreases in low-frequency po-
wer. These spectral modulations are thought to reflect both local
increases in multi-unit firing [17] and decreased long-range low-
frequency synchrony [38]. Evidence for these patterns correlates
with the fMRI blood-oxygen-level-dependent (BOLD) signal [39],
predicts successful memory encoding [6, 40], and is related to
core memory processes such as item-context binding [41]. We
found that the change in classifier output after stimulation was
related to how much stimulation evoked the tilt pattern (r(37) =
0.54, p < 0.001; Figure 5), suggesting that stimulation increased
classifier evidence by modulating a neural marker that has been
linked to contextual memory encoding.
DISCUSSION
We applied direct electrical stimulation to nodes of the memory
network and found that stimulation reliablymodulatesmemory in
a way that depends on the brain’s encoding state. In showing
that stimulation improves memory in low encoding states and
disrupts memory in high encoding states, the findings suggest
that stimulation alters the ongoing course of memory processing
in the brain. By using brain-state-matched trials from non-stim-
ulated lists, our data show that stimulation modulates neural ac-
tivity beyond what might be expected by regression to the mean
arising from temporal autocorrelation in the brain’s encoding
state. Our data offer insight into the inconsistent effects that
have been reported in studies of how brain stimulation modu-
lates memory performance [27–34], and suggest that using
brain-state decoding can improve the ability to influence mem-
ory outcomes with stimulation.
Our results are consistent with amodel in which targeted stim-
ulation leads to changes in network activity across brain areas
that contribute to successful memory encoding. There is
growing consensus that direct electrical stimulation is likely to in-
fluence physiology across a network connected to the targeted
site [10, 12]. In using DBS for the treatment of Parkinson’s [42],
for example, researchers have had success targeting multiple
structures within the affected motor network [12], which sug-
gests that it is more important to target the relevant functional
network rather than individual structures within the network. In
the case of episodic memory, it may therefore be possible to
enhance the effectiveness of stimulation by using measures of
connectivity to identify nodes that offer a high degree of control-
lability over the memory network [43]. Resting-state data could
be leveraged to predict the stimulation targets that are most
likely to modulate the core memory network [44].
Prior work has shown that stimulation’s effects on physiology
depend not only on the excitability of the targeted neurons [26]
but also on ongoing rhythms generated by synchronous activity
in larger populations. Hippocampal stimulation, for example,
alternately promotes long-term potentiation or long-term
depression depending on whether theta phase is at the peak
or trough at the time of stimulation delivery [24, 25]. Learning it-
self is also state dependent, as shown in classical conditioning
experiments in which animals show faster learning when trials
are triggered based on theta rhythm [16]. Our data confirm the
role of pre-stimulus brain states for upcoming learning and
show that these states can be directly modulated.
We applied classification to whole-brain iEEG to decode brain
states that predict later recall. Multivariate decoding allowed us
to overcome individual differences in neural connectivity, clinical
etiology, and electrode placement that could increase variability
in stimulation’s neural and behavioral effects. Decodingmay also
have provided a more sensitive index of the encoding state than
would be possible if using a single feature to identify good and
poor memory states [17]. We then related stimulation’s effect
on physiology to its effect on memory, extending prior work
that has used stimulation to map behavior. Using direct brain re-
cordings most likely facilitated decoding, but future work should
address the extent to which non-invasive decoding and stimula-
tion methods [45] could be combined to modulate memory
states.
We used the classifier trained on encoding data to decode
pre-stimulation states. Our approach suggests that, at a broad
level, similar whole-brain patterns of neural activity predict
successful encoding during and prior to stimulus onset. In
both training and testing our classifier, we averaged spectral
power over a temporal interval of several hundred millisec-
onds, meaning our model was sensitive to consistent spectral
power fluctuations over the pre- and post-stimulus intervals.
There is evidence that assessing neural activity at a finer
temporal scale can identify distinct patterns of pre- and
post-stimulus activity that predict encoding. Increased pre-
stimulus theta power recorded using non-invasive methods,
for example, has been shown to predict successful memory
[6, 46], and increased intracranial theta power has also been
shown to predict memory pre-stimulus [5, 47], although not
during free recall. Taken together with our data, these findings
suggest that both tonic and phasic pre-stimulus signals are
predictive of memory success. Algorithms to identify good en-
coding states could therefore be improved by incorporating
time as a feature, which would allow both sustained and tran-
sient fluctuations in spectral power to influence estimates of
the encoding state.
Current Biology 27, 1–8, May 8, 2017 5
Please cite this article in press as: Ezzyat et al., Direct Brain Stimulation Modulates Encoding States and Memory Performance in Humans, CurrentBiology (2017), http://dx.doi.org/10.1016/j.cub.2017.03.028
By testing classifier generalization across days and using the
free recall task to measure memory performance, our data sup-
port the interpretation that the decoded brain states are stable in
their neural representation over time and globally predict mem-
ory function. Free recall is a complex task that recruits multiple
core episodic memory processes [20]. We show that stimulation
increases encoding states by increasing high-frequency activity
(HFA) power and decreasing low-frequency activity (LFA) power
across the brain, a pattern that predicts behavioral measures of
item-context encoding [41]. Although such encoding processes
promote memory function, free recall is also known to depend
heavily on retrieval processes, suggesting that future work may
find success influencing memory function by applying stimula-
tion during memory search.
We show an overall reduction in verbal memory performance
when stimulating a large set of brain regions, including many
outside of the MTL. This is broadly consistent with recent work
focused on the hippocampus and entorhinal cortex that showed
that stimulation tends to impair both verbal and spatial memory
[31]. However, we further test the hypothesis that stimulation’s
effects on memory depend on timing relative to the brain’s en-
coding state [48]. Our findings therefore extend prior studies of
human intracranial brain stimulation in several ways. First, we
usemultivariate decoding of neural activity to separate pre-stim-
ulation brain states, and show that stimulation counteracts low
encoding states but disrupts high encoding states. Second, we
show that stimulation at low and high encoding states differen-
tially modulates neural activity in amanner consistent with the ef-
fect on memory. Third, we show that stimulation’s effect on the
encoding state is correlated with the spectral tilt, a biomarker
of successful memory encoding. Our work therefore identifies
situations in which stimulation increases and decreases mem-
ory, and relates stimulation’s effects on behavior to its influence
on neural activity through novel use of subject-specific multivar-
iate classification.
By showing that stimulation is most likely to improve memory
when encoding efficiency is low prior to stimulation delivery, our
data provide the foundation for future work to apply stimulation
when it is most likely to improve memory function. Non-invasive
closed-loop approaches have improved attention through
training using fMRI [49] andmaximized the benefit of restudy op-
portunities using scalp EEG [50]. Closed-loop neural decoding
could thus optimally target stimulation for treatment of memory
disorders [48, 51].
EXPERIMENTAL PROCEDURES
Participants
One hundred and two patients undergoing iEEG monitoring as part of clinical
treatment for drug-resistant epilepsy were recruited to participate in this study.
Data were collected as part of a multi-center project designed to assess the
effects of electrical stimulation on memory-related brain function. Data were
collected at the following centers: Thomas Jefferson University Hospital,
Mayo Clinic, Hospital of the University of Pennsylvania, Emory University Hos-
pital, University of Texas Southwestern Medical Center, Dartmouth-Hitchcock
Medical Center, Columbia University Medical Center, National Institutes of
Health, and University of Washington Medical Center. The research protocol
was approved by the institutional review board (IRB) at each hospital and
informed consent was obtained from each participant. Electrophysiological
data were collected from electrodes implanted subdurally on the cortical sur-
face as well as deep within the brain parenchyma. In each case, the clinical
6 Current Biology 27, 1–8, May 8, 2017
team determined the placement of the electrodes so as to best localize epilep-
togenic regions. Subdural contacts were arranged in both strip and grid con-
figurations with an inter-contact spacing of 10mm.Most subjects (N = 83) also
had temporal lobe depth electrodes with 5 mm inter-contact spacing.
Verbal Memory Task
Each subject participated in a delayed free recall task in which they were in-
structed to study lists of words for a later memory test; no encoding task
was used. Lists were composed of 12 words chosen at random and without
replacement from a pool of high-frequency nouns (either English or Spanish,
depending on the participant’s native language; http://memory.psych.
upenn.edu/Word_Pools). Each word remained on the screen for 1,600 ms, fol-
lowed by a randomly jittered 750- to 1,000ms blank inter-stimulus interval (ISI).
Immediately after the final word in each list, participants performed a dis-
tractor task (20 s) consisting of a series of arithmetic problems of the form
A + B + C = ?, where A, B, and C were randomly chosen integers ranging
from 1 to 9. After the distractor task, participants were given 30 s to verbally
recall as many words as possible from the list in any order; vocal responses
were digitally recorded and later manually scored for analysis. Each session
consisted of 25 lists of this encoding-distractor-recall procedure. A subset
of subjects completed additional sessions of the free recall task using catego-
rized word lists, which were included in the electrophysiological analyses. The
categorized recall task is identical to the free recall task, with the exception that
the word pool was drawn from 25 semantic categories (e.g., fruit, furniture,
office supplies). Each list of 12 items in the categorized version of the task con-
sisted of four words drawn from each of three categories. In total, 41 patients
completed at least one session of the categorized recall task.
Stimulation Methods
At the start of each session, we determined the safe amplitude for stimulation
using a mapping procedure in which stimulation was applied at 0.5 mA while
a neurologist monitored for afterdischarges. This procedure was repeated,
incrementing the amplitude in steps of 0.5 mA, up to a maximum of
1.5 mA for depth contacts and 3.5 mA for cortical surface contacts. These
maximum amplitudes were chosen to be well below accepted safety limits
for charge density [52]. For each stimulation session, we passed electrical
current through a single pair of adjacent electrode contacts. Because the
electrode locations were determined strictly by the monitoring needs of the
clinicians, we used a combination of anatomical and functional information
to select stimulation sites. If available, we prioritized electrodes in the hippo-
campus, entorhinal cortex, perirhinal cortex, parahippocampal cortex, and
dorsolateral prefrontal cortex. To choose among these regions in cases in
which more than one was available, we selected the electrode demonstrating
the largest subsequent memory effect (SME) in the high-frequency range
(70–200 Hz) among these regions. In cases in which none of the aforemen-
tioned regions was available, we selected the contact with the largest
SME. We used a mapping procedure at the start of each session to deter-
mine the safe amplitude for stimulation. Stimulation was delivered using
charge-balanced biphasic rectangular pulses (pulse width = 300 ms) at
50 Hz frequency, and was applied continuously for 4.6 s while subjects en-
coded two consecutive words; stimulation was not applied for the following
two words. This alternation of stimulated and non-stimulated word pairs
continued until the end of the list. Stimulation onset was 200 ms prior to
word presentation and lasted until 200–450 ms after the offset of the next
word (the range is due to the variable ISI between words). Stimulation was
applied in 20 of the 25 lists in a session, and each stimulation list was
randomly chosen to begin with a stimulated or non-stimulated pair of words.
We randomized the order of the 20 stimulation lists and the remaining five
non-stimulation control lists within each session.
Statistics
Data are presented as mean ± SEM. Unless otherwise specified, all statistical
comparisons were conducted as two-tailed tests. Data distributions were
either visually inspected or assumed to be normal for parametric tests. For
both the record-only and stimulation samples, we included any enrolled sub-
ject who completed at least one full session of the task. In both cases, the sam-
ple sizes were chosen to at least match or exceed the sample sizes reported in
prior human intracranial record-only and stimulation studies. For stimulation
Please cite this article in press as: Ezzyat et al., Direct Brain Stimulation Modulates Encoding States and Memory Performance in Humans, CurrentBiology (2017), http://dx.doi.org/10.1016/j.cub.2017.03.028
analyses, we treated all sessions collected in a single patient at a single stim-
ulated bipolar pair as our unit of observation.
All other methods are described in the Supplemental Experimental
Procedures.
Data and Software Availability
All deidentified raw data and analysis code may be downloaded at http://
memory.psych.upenn.edu/Electrophysiological_Data.
SUPPLEMENTAL INFORMATION
Supplemental Information includes Supplemental Experimental Procedures,
three figures, and two tables and can be found with this article online at
http://dx.doi.org/10.1016/j.cub.2017.03.028.
AUTHOR CONTRIBUTIONS
Y.E., D.S.R., and M.J.K. designed experiments. K.B.H., D.F.L., A.L., L.O.,
P.W., M.T.K., C.S.I., and M.J.J. performed data collection and recording.
Y.E. analyzed the data. J.E.K., J.F.B., S.B., and I.P. contributed data analysis
tools. J.M.S., S.R.D., and R.G. localized the electrodes. M.R.S., G.A.W.,
K.A.D., T.H.L., B.C.L., B.C.J., S.A.S., and K.Z. recruited participants and pro-
vided general assistance. Y.E. and M.J.K. wrote the manuscript. All authors
provided feedback on the manuscript. D.S.R. and M.J.K. supervised the
research.
ACKNOWLEDGMENTS
We dedicate this paper in loving memory of Anastasia Lyalenko, without
whose contributions this work would not have been possible. This work was
supported by the DARPA Restoring Active Memory (RAM) program (coopera-
tive agreement N66001-14-2-4032). We thank Blackrock Microsystems for
providing neural recording and stimulation equipment. We thank J.A. Wachter
for suggesting the analysis in Figure 4. We are indebted to the patients and
their families for their participation and support. The views, opinions, and/or
findings contained in this material are those of the authors and should not
be interpreted as representing the official views or policies of the Department
of Defense or the U.S. Government. B.C.J. receives research funding from
NeuroPace and Medtronic not relating to this research. M.J.K. and D.S.R.
are in the process of organizing Nia Therapeutics, LLC (‘‘Nia’’), a company in-
tended to develop and commercialize brain stimulation therapies for memory
restoration. Currently, Nia has no assets and has not commenced operations.
M.J.K. and D.S.R. each holds a greater than 5% equity interest in Nia.
Received: December 5, 2016
Revised: February 24, 2017
Accepted: March 13, 2017
Published: April 20, 2017
REFERENCES
1. Paller, K.A., and Wagner, A.D. (2002). Observing the transformation of
experience into memory. Trends Cogn. Sci. 6, 93–102.
2. Davachi, L., Mitchell, J.P., and Wagner, A.D. (2003). Multiple routes to
memory: distinct medial temporal lobe processes build item and source
memories. Proc. Natl. Acad. Sci. USA 100, 2157–2162.
3. Uncapher, M.R., and Wagner, A.D. (2009). Posterior parietal cortex and
episodic encoding: insights from fMRI subsequent memory effects and
dual-attention theory. Neurobiol. Learn. Mem. 91, 139–154.
4. Kim, H. (2011). Neural activity that predicts subsequent memory and
forgetting: a meta-analysis of 74 fMRI studies. Neuroimage 54, 2446–
2461.
5. Fell, J., Ludowig, E., Staresina, B.P., Wagner, T., Kranz, T., Elger, C.E., and
Axmacher, N. (2011). Medial temporal theta/alpha power enhancement
precedes successful memory encoding: evidence based on intracranial
EEG. J. Neurosci. 31, 5392–5397.
6. Guderian, S., Schott, B.H., Richardson-Klavehn, A., and Duzel, E. (2009).
Medial temporal theta state before an event predicts episodic encoding
success in humans. Proc. Natl. Acad. Sci. USA 106, 5365–5370.
7. Park, H., and Rugg, M.D. (2010). Prestimulus hippocampal activity pre-
dicts later recollection. Hippocampus 20, 24–28.
8. Rutishauser, U., Ross, I.B., Mamelak, A.N., and Schuman, E.M. (2010).
Human memory strength is predicted by theta-frequency phase-locking
of single neurons. Nature 464, 903–907.
9. Hanslmayr, S., Matuschek, J., and Fellner, M.C. (2014). Entrainment of
prefrontal beta oscillations induces an endogenous echo and impairs
memory formation. Curr. Biol. 24, 904–909.
10. Kim, K., Ekstrom, A.D., and Tandon, N. (2016). A network approach for
modulatingmemory processes via direct and indirect brain stimulation: to-
ward a causal approach for the neural basis of memory. Neurobiol. Learn.
Mem. 134, 162–177.
11. Kringelbach, M.L., Jenkinson, N., Owen, S.L.F., and Aziz, T.Z. (2007).
Translational principles of deep brain stimulation. Nat. Rev. Neurosci. 8,
623–635.
12. McIntyre, C.C., and Hahn, P.J. (2010). Network perspectives on themech-
anisms of deep brain stimulation. Neurobiol. Dis. 38, 329–337.
13. Matzen, L.E., Trumbo, M.C., Leach, R.C., and Leshikar, E.D. (2015).
Effects of non-invasive brain stimulation on associative memory. Brain
Res. 1624, 286–296.
14. Bronstein, J.M., Tagliati, M., Alterman, R.L., Lozano, A.M., Volkmann, J.,
Stefani, A., Horak, F.B., Okun, M.S., Foote, K.D., Krack, P., et al. (2011).
Deep brain stimulation for Parkinson disease: an expert consensus and re-
view of key issues. Arch. Neurol. 68, 165.
15. Benabid, A.L., Pollak, P., Gervason, C., Hoffmann, D., Gao, D.M.,
Hommel, M., Perret, J.E., and de Rougemont, J. (1991). Long-term sup-
pression of tremor by chronic stimulation of the ventral intermediate
thalamic nucleus. Lancet 337, 403–406.
16. Seager, M.A., Johnson, L.D., Chabot, E.S., Asaka, Y., and Berry, S.D.
(2002). Oscillatory brain states and learning: impact of hippocampal
theta-contingent training. Proc. Natl. Acad. Sci. USA 99, 1616–1620.
17. Burke, J.F., Ramayya, A.G., and Kahana, M.J. (2015). Human intracranial
high-frequency activity during memory processing: neural oscillations or
stochastic volatility? Curr. Opin. Neurobiol. 31, 104–110.
18. Grober, E., Lipton, R.B., Hall, C., andCrystal, H. (2000). Memory impairment
on free and cued selective reminding predicts dementia. Neurology 54,
827–832.
19. Gershberg,F.B., andShimamura,A.P. (1995). Impaireduseoforganizational
strategies in free recall following frontal lobe damage.Neuropsychologia 33,
1305–1333.
20. Lohnas, L.J., Polyn, S.M., and Kahana, M.J. (2015). Expanding the scope
of memory search: modeling intralist and interlist effects in free recall.
Psychol. Rev. 122, 337–363.
21. Haglund, M.M., Ojemann, G.A., and Blasdel, G.G. (1993). Optical imaging
of bipolar cortical stimulation. J. Neurosurg. 78, 785–793.
22. Suh, M., Bahar, S., Mehta, A.D., and Schwartz, T.H. (2006). Blood volume
and hemoglobin oxygenation response following electrical stimulation of
human cortex. Neuroimage 31, 66–75.
23. Borchers, S., Himmelbach, M., Logothetis, N., and Karnath, H.O. (2011).
Direct electrical stimulation of human cortex - the gold standard for map-
ping brain functions? Nat. Rev. Neurosci. 13, 63–70.
24. Hyman, J.M., Wyble, B.P., Goyal, V., Rossi, C.A., and Hasselmo, M.E.
(2003). Stimulation in hippocampal region CA1 in behaving rats yields
long-term potentiation when delivered to the peak of theta and long-
term depression when delivered to the trough. J. Neurosci. 23, 11725–
11731.
25. Pavlides, C., Greenstein, Y.J., Grudman, M., and Winson, J. (1988). Long-
term potentiation in the dentate gyrus is induced preferentially on the
positive phase of theta-rhythm. Brain Res. 439, 383–387.
26. Pollen, D.A. (1977). Responses of single neurons to electrical stimulation
of the surface of the visual cortex. Brain Behav. Evol. 14, 67–86.
Current Biology 27, 1–8, May 8, 2017 7
Please cite this article in press as: Ezzyat et al., Direct Brain Stimulation Modulates Encoding States and Memory Performance in Humans, CurrentBiology (2017), http://dx.doi.org/10.1016/j.cub.2017.03.028
27. Fell, J., Staresina, B.P., Do Lam, A.T., Widman, G., Helmstaedter, C.,
Elger, C.E., and Axmacher, N. (2013). Memory modulation by weak syn-
chronous deep brain stimulation: a pilot study. Brain Stimulat. 6, 270–273.
28. Suthana, N., Haneef, Z., Stern, J., Mukamel, R., Behnke, E., Knowlton, B.,
and Fried, I. (2012). Memory enhancement and deep-brain stimulation of
the entorhinal area. N. Engl. J. Med. 366, 502–510.
29. Miller, J.P., Sweet, J.A., Bailey, C.M., Munyon, C.N., Luders, H.O., and
Fastenau, P.S. (2015). Visual-spatial memory may be enhanced with theta
burst deep brain stimulation of the fornix: a preliminary investigation with
four cases. Brain 138, 1833–1842.
30. Coleshill, S.G., Binnie, C.D., Morris, R.G., Alarcon, G., van Emde Boas,W.,
Velis, D.N., Simmons, A., Polkey, C.E., van Veelen, C.W.M., and van Rijen,
P.C. (2004). Material-specific recognition memory deficits elicited by uni-
lateral hippocampal electrical stimulation. J. Neurosci. 24, 1612–1616.
31. Jacobs, J., Miller, J., Lee, S.A., Coffey, T., Watrous, A.J., Sperling, M.R.,
Sharan, A., Worrell, G., Berry, B., Lega, B., et al. (2016). Direct electrical
stimulation of the human entorhinal region and hippocampus impairs
memory. Neuron 92, 983–990.
32. Halgren, E., and Wilson, C.L. (1985). Recall deficits produced by
afterdischarges in the human hippocampal formation and amygdala.
Electroencephalogr. Clin. Neurophysiol. 61, 375–380.
33. Lacruz, M.E., Valentın, A., Seoane, J.J.G., Morris, R.G., Selway, R.P., and
Alarcon, G. (2010). Single pulse electrical stimulation of the hippocampus
is sufficient to impair humanepisodicmemory. Neuroscience170, 623–632.
34. Perrine, K., Devinsky, O., Uysal, S., Luciano, D.J., and Dogali, M. (1994).
Left temporal neocortex mediation of verbal memory: evidence from func-
tional mapping with cortical stimulation. Neurology 44, 1845–1850.
35. Manning, J.R., Jacobs, J., Fried, I., and Kahana, M.J. (2009). Broadband
shifts in local field potential power spectra are correlated with single-
neuron spiking in humans. J. Neurosci. 29, 13613–13620.
36. Haufe, S., Meinecke, F., Gorgen, K., D€ahne, S., Haynes, J.D., Blankertz,
B., and Bießmann, F. (2014). On the interpretation of weight vectors of
linear models in multivariate neuroimaging. Neuroimage 87, 96–110.
37. Long, N.M., Burke, J.F., and Kahana, M.J. (2014). Subsequent memory
effect in intracranial and scalp EEG. Neuroimage 84, 488–494.
38. von Stein, A., and Sarnthein, J. (2000). Different frequencies for different
scales of cortical integration: from local gamma to long range alpha/theta
synchronization. Int. J. Psychophysiol. 38, 301–313.
39. Winawer, J., Kay, K.N., Foster, B.L., Rauschecker, A.M., Parvizi, J., and
Wandell, B.A. (2013). Asynchronous broadband signals are the principal
source of the BOLD response in human visual cortex. Curr. Biol. 23,
1145–1153.
8 Current Biology 27, 1–8, May 8, 2017
40. Burke, J.F., Long, N.M., Zaghloul, K.A., Sharan, A.D., Sperling, M.R.,
and Kahana, M.J. (2014). Human intracranial high-frequency activity
maps episodic memory formation in space and time. Neuroimage 85,
834–843.
41. Long, N.M., and Kahana, M.J. (2015). Successful memory formation is
driven by contextual encoding in the core memory network. Neuroimage
119, 332–337.
42. Benabid, A.L., Pollak, P., Louveau, A., Henry, S., and de Rougemont, J.
(1987). Combined (thalamotomy and stimulation) stereotactic surgery of
the VIM thalamic nucleus for bilateral Parkinson disease. Stereotact.
Funct. Neurosurg. 50, 344–346.
43. Gu, S., Pasqualetti, F., Cieslak, M., Telesford, Q.K., Yu, A.B., Kahn, A.E.,
Medaglia, J.D., Vettel, J.M., Miller, M.B., Grafton, S.T., and Bassett, D.S.
(2015). Controllability of structural brain networks. Nat. Commun. 6,
8414.
44. Fox, M.D., Buckner, R.L., Liu, H., Chakravarty, M.M., Lozano, A.M., and
Pascual-Leone, A. (2014). Resting-state networks link invasive and nonin-
vasive brain stimulation across diverse psychiatric and neurological dis-
eases. Proc. Natl. Acad. Sci. USA 111, E4367–E4375.
45. Wang, J.X., Rogers, L.M., Gross, E.Z., Ryals, A.J., Dokucu, M.E.,
Brandstatt, K.L., Hermiller, M.S., and Voss, J.L. (2014). Targeted enhance-
ment of cortical-hippocampal brain networks and associative memory.
Science 345, 1054–1057.
46. Salari, N., and Rose, M. (2016). Dissociation of the functional relevance
of different pre-stimulus oscillatory activity for memory formation.
Neuroimage 125, 1013–1021.
47. Merkow, M.B., Burke, J.F., Stein, J.M., and Kahana, M.J. (2014).
Prestimulus theta in the human hippocampus predicts subsequent recog-
nition but not recall. Hippocampus 24, 1562–1569.
48. Young, N.P., and Deisseroth, K. (2017). Cognitive neuroscience: in search
of lost time. Nature 542, 173–174.
49. deBettencourt, M.T., Cohen, J.D., Lee, R.F., Norman, K.A., and Turk-
Browne, N.B. (2015). Closed-loop training of attention with real-time brain
imaging. Nat. Neurosci. 18, 470–475.
50. Fukuda, K., and Woodman, G.F. (2015). Predicting and improving recog-
nition memory using multiple electrophysiological signals in real time.
Psychol. Sci. 26, 1026–1037.
51. Lozano, A.M., Mayberg, H.S., Giacobbe, P., Hamani, C., Craddock, R.C.,
and Kennedy, S.H. (2008). Subcallosal cingulate gyrus deep brain stimu-
lation for treatment-resistant depression. Biol. Psychiatry 64, 461–467.
52. Shannon, R.V. (1992). Amodel of safe levels for electrical stimulation. IEEE
Trans. Biomed. Eng. 39, 424–426.
Current Biology, Volume 27
Supplemental Information
Direct Brain Stimulation Modulates Encoding
States and Memory Performance in Humans
Youssef Ezzyat, James E. Kragel, John F. Burke, Deborah F. Levy, Anastasia Lyalenko, PaulWanda, Logan O'Sullivan, Katherine B. Hurley, Stanislav Busygin, IsaacPedisich, Michael R. Sperling, Gregory A. Worrell, Michal T. Kucewicz, Kathryn A.Davis, Timothy H. Lucas, Cory S. Inman, Bradley C. Lega, Barbara C. Jobst, Sameer A.Sheth, Kareem Zaghloul, Michael J. Jutras, Joel M. Stein, Sandhitsu R. Das, RichardGorniak, Daniel S. Rizzuto, and Michael J. Kahana
Figure S1: Aggregate Electrode Rendering. Related to Figure 1. All bipolar (virtual) electrodes from all102 patients rendered on normalized cortical surfaces. Red spheres indicate stimulated electrodes visiblefrom lateral and medial views.
Figure S2: Classifier performance within each time-frequency bin. Related to Figure 2E. Mean perfor-mance of classifiers trained on data from individual time-frequency pixels. Vertical dashed lines indicateword onset and offset. For each of 50 frequencies and for each of 65 timebins from -500 ms relative to wordonset to 500 ms relative to word offset, we fit an L2-penalized logistic regression classifier. We used powerat each electrode, within a given time-frequency bin, as the set of features for each classifier. We fit theseclassifiers for every subject and calculated cross-validated AUC for each time-frequency bin.
Figure S3: Whole-brain Spectral Tilt vs. Full Classifier. Related to Figure 2E. Area under the curve(AUC) for the L2-penalized logistic regression (L2 LR) classifier from the main manuscript compared withthe whole-brain spectral tilt classifier. The multivariate model outperforms the tilt model, P < 10−8. Foreach subject, we averaged spectral power above (high frequency) and below (low-frequency) a given cutofffrequency, across all electrodes. We did this for each possible cutoff frequency (from 1 to 200 Hz) and tookthe high-frequency–low-frequency difference as the measure of the tilt. We then computed area under thecurve (AUC) for this measure and did so for the measure defined using each cutoff frequency. We thenselected the cutoff frequency that maximized spectral tilt AUC for that subject and used this in the acrosssubject distribution.
Subject Site Gender Age Lang. Dom. Ictal Onset Stim Location Stim Hemisphere Amp (mA) ∆ Recall (Normalized %) ∆ Recall (Raw # Words/List)1 P F 48 L R CA1 L 1.5/1.0 -2.84 -0.052 P F 49 L B Amy L 1.5 1.21 0.05
Amy L 1.5 -7.87 -0.403 P F 39 L L CA3 L 1.5 43.22 1.854 P M 47 L L DLPFC L 1.5 -16.13 -0.405 J F 48 L R CA1 R 1.5/1.0 -1.15 -0.05
CA1 R 1.5 -2.49 -0.106 J M 24 L L CA1 L 0.5/1.0 -53.57 -1.137 J M 32 L R ACg R 1.0 -11.22 -0.35
PCg R 1.5 23.10 0.858 E F 36 L L CA3 L 1.0 -14.96 -0.789 D F 24 B L EC L 0.5 -11.03 -0.38
EC L 0.5 -35.34 -1.35Precuneus L 1.5 16.06 0.70
10 J M 48 L L CA1 L 1.0 18.95 -0.51PRC L 1.0 -10.87 -0.20
11 M F 27 NA R EC R 1.0/1.5 2.75 0.25EC R 1.0 -9.16 -0.70
12 W F 33 NA L DLPFC L 3.5 -16.67 -0.50DLPFC L 3.5 -57.29 -1.1
13 J M 23 L L PHC L 0.5 16.53 0.40PHC L 0.5 -0.35 -0.01
14 M M 40 NA L PRC R 1.5 35.09 1.3315 D F 31 L R PRC L 1.5 28.85 0.45
CA3 L 1.0 -31.25 -0.8016 M M 49 R L PRC L 1.0/0.5 -18.18 -0.7717 M F 28 R R DLPRC R 3.0 -4.57 -0.3018 E F 19 NA R CA3 L 1.5 -12.50 -0.5019 M M 34 L R CA1 R 1.5 -14.81 -0.9320 M F 36 NA R DLPFC R 3.0 -9.23 -0.2721 P F 45 L L CA3 L 1.0 -66.53 -1.6522 M M 26 L L DLPFC L 2.5 -10.42 -0.50
DLPFC L 2.5 8.85 0.4023 T F 40 L L PRC R 0.50 35.71 0.9024 M M 24 NA B ACg L 1.0 13.89 0.2525 T F 26 R L CA1 R 0.50 -27.44 -0.9026 M M 20 L L MTG L 1.5 21.01 1.62
PRC L 0.75 -15.96 -1.0027 T M 47 L L Insula R 0.5 -18.44 -0.9028 J F 49 L L DG L 0.25 -2.12 -0.0529 C M 22 L R CA1 R 0.5 -3.07 -0.1530 M F 41 L R ITG L 1.0 -39.07 -1.58
MTG L 1.0 8.7 0.3531 M F 23 L L ITG L 1.0 0.57 0.03
MTG L 1.0 13.00 0.36MTG L 1.0 -10.31 -0.40
32 M M 42 NA R Lateral Occipital Cortex R 0.75 -17.09 -0.67Inferior Parietal Cortex R 0.75 6.76 0.29
33 J M 50 NA R EC L 1.5 -3.42 -0.1034 J F 22 NA R Fusiform Gyrus R 1.0 6.25 0.2535 T F 38 L R PRC L 1.0 -15.63 -0.3836 T M 35 L R PRC L 1.0 -43.64 -1.60
Clinical Site: C: Columbia, D: Dartmouth, E: Emory, J: Jefferson, M: Mayo, P: Penn, T: Texas, W: University of WashingtonLanguage Dominance/Ictal Onset: L: Left Hemisphere, R: Right Hemisphere, B: Bilateral, NA: Not Reported/UndeterminedStimulation Location: CA1/2/3: Hippocampal subfields, DG: Dentate Gyrus, DLPFC: Dorsolateral prefrontal cortex, EC: Entorhinalcortex, ITG/MTG: Inferior/Middle temporal gyrus, PRC: Perirhinal cortex, PHC: Parahippocampal cortex, PFC: Prefrontal cortex,ACg: Anterior cingulate gyrus, PCg: Posterior Cingulate Gyrus
Table S1. Patient Demographics and Raw Data. Related to Figure 3. For each patient, the clinical datacollection site, gender, age, language dominance, ictal onset hemisphere, stimulation location, stimulationhemisphere, stimulation amplitude, Normalized ∆ %Recall, and Raw ∆ Number of Words Recalled/List.Each row presents data for a unique stimulation location (bipolar electrode pair), and encompasses one ormore sessions collected at the stimulated site.
Stim Region (N) HC (10) MTL (9) TC (7) MFG (5)Low Brain State (∆t) 0.47 ± 0.40 0.78 ± 0.59 0.83 ± 0.86 1.71 ± 0.83High Brain State (∆t) -0.96 ± 0.49 -1.47 ± 0.74 -2.68 ± 0.62 -1.82 ± 0.85
Table S2. Change in Brain State as a Function of Stimulated Region. Related to Figure 4D, E. Post-pre∆ classifier output for Low State stimulation events (Top) and High State stimulation events (Bottom) as afunction of stimulated brain region. Only regions with N > 4 are shown. Numbers indicate mean ± s.e.mwithin stimulated brain region. HC = hippocampus, MTL = medial temporal lobe cortex, TC = non-MTLtemporal lobe, MFG =middle frontal gyrus.
Supplemental Experimental Procedures
Anatomical localization
Anatomical localization of electrode placement was accomplished using a two step process. First,hippocampal subfields and MTL cortices were automatically labeled in a pre-implant 2 mm thickcoronal T2-weighted MRI using the automatic segmentation of hippocampal subfields (ASHS)multi-atlas segmentation method [S1]. Following this automatic labeling procedure, a post-implantCT scan was co-registered with the MRI using Advanced Normalization Tools [S2]. Electrodes thatare visible in the CT were then localized within the MTL subregions by a pair of neuroradiologistswith expertise in MTL anatomy. The neuroradiologists confirmed the output of the automatedpipeline and provided additional detail on localization within subfields/subregions. The wholebrain cortical surface was also obtained from a pre-implant volumetric T1-weighted MRI usingFreesurfer [S3], and subdural electrodes were separately co-registered and localized to corticalregions using an energy minimization algorithm [S4]. In cases in which automatic localizationfailed, the neuroradiologists performed manual localization of the electrodes.
Electrophysiological data processing
Intracranial data were recorded using one of the following clinical electroencephalogram (EEG)systems (depending the site of data collection): Nihon Kohden EEG-1200, Natus XLTek EMU 128or Grass Aura-LTM64. Depending on the amplifier and the preference of the clinical team, thesignals were sampled at either 500, 1000 or 1600 Hz and were referenced to a common contactplaced either intracranially, on the scalp or mastoid process. Intracranial electrophysiologicaldata were filtered to attenuate line noise (5 Hz band-stop fourth order Butterworth, centeredon 60 Hz). To eliminate potentially confounding large-scale artifacts and noise on the referencechannel, we re-refereced the data using a bipolar montage [S5]. To do so, we identified all pairs ofimmediately adjacent contacts on every depth, strip and grid and took to difference between thesignals recorded in each pair. The resulting bipolar timeseries was treated as a virtual electrode andused in all subsequent analysis. We performed spectral decomposition (50 frequencies from 1-200Hz, logarithmically-spaced; Morlet wavelets; wave number = 5) for 2600 ms epochs from -500 to to2100 ms relative to word onset. An additional 1500 ms buffer included before and after the intervalof interest was discarded to remove convolution edge effects. The resulting time-frequency datawere then log-transformed, resampled in the time domain to 50 Hz, and z-scored within sessionand frequency band across word presentation events.
Multivariate classification
We included only data collected in record-only sessions as input to a logistic regression classifiertrained to discriminate encoding-related activity predictive of whether a word was later remem-bered or forgotten. We used spectral power averaged across the time dimension for each wordencoding epoch (0-1600ms relative to word onset) as the input data. Thus, the features for eachindividual word encoding observation were the average power across time, at each of the 50 ana-lyzed frequencies×N electrodes. We used L2-penalization [S6] and selected the penalty parameterfor each subject based on the best penalty parameter (in terms of maximizing area under the curve,AUC) across the entire distribution of subjects. To do this, we began with a set of 22 possiblepenalty parameters spaced logarithmically between 10−6 and 104 and, for each penalty parameter,fit the classifier using maximum likelihood and used N − 1 cross-validation to assess performance
using AUC. For the majority of subjects that completed more than one session, N − 1 was appliedover sessions; for subjects that completed a single session N − 1 was applied over lists within thesession. This procedure yielded a distribution of AUCs across subjects for each penalty param-eter. Then, for each subject we conducted a final cross-validated training in which we selectedthe penalty parameter with the highest mean AUC across the distribution of all subjects exceptthe one whose classifier was being trained. We then computed AUC [S6] to quantify classifierperformance and repeated this procedure for all subjects. AUC measures a classifier’s ability toidentify true positives while minimizing false positives, where chance AUC = 0.50. To ensurethe classifier learned equally from both classes (given the imbalance between recalled and notrecalled exemplars), we also weighted the penalty parameter in inverse proportion to the numberof exemplars of each class [S6]. We assessed the significance of each classifier within-subject usinga permutation test in which we randomized the labels of recalled/not recalled events in the trainingdata, computed AUC and repeated the randomization 1000 times to generate a null distributionof AUCs. Classifiers that exceeded the P < 0.05 level were considered reliable for the purposes ofanalyses contingent on brain-state.
In subsequent behavioral analyses of the stimulation sessions (see Analysis of stimulation’s effect onmemory performance), we used Youden’s J statistic, which identifies the classifier output thresholdthat maximizes the difference between true and false positives. To derive Youden’s J, we fit anunequal variance signal detection model to the distribution of classifier outputs across the record-only sessions and estimated Youden’s J from the best-fit curve. These curves are plotted in Figure2E.
To assess the importance of different frequency × brain region features to classification, we gener-ated a forward model for each subject using the weights from their classifier [S7]
A =ΣxWσ2
y
where Σx is the covariance matrix of the data X, W is the vector of classifier weights obtainedfollowing maximum likelihood estimation, σ2
y is the variance of the logit-transform of the vector
of classifier outputs across all events, y, and A is the vector of parameter values for the uniqueforward model corresponding to the backward model W. The magnitude and direction of the valuesin A reflect the strength of the relation between classifier output and each input feature in X.We computed A separately for each subject and plot the average across subjects in Figure 2F. Weassessed significance across the group by computing a t-statistic against zero for the activation ofeach feature and applying threshold-free cluster enhancement to the t-map. We then generated1000 null datasets by flipping the signs of the values in A for 1000 random samples of up to Nsubjects. We computed t-statistics on the null distributions, derived P values by comparing thetrue t-statistic for each feature to the distributions generated from the null data, and corrected formultiple comparisons by controlling the false discovery rate at q = 0.05 [S8].
Spectral analysis of stimulation sessions
For the analyses of the behavioral effects of stimulation contingent on the pre-stimulation encodingstate, we used wavelet decomposition (see Electrophysiological data processing) applied to the-1100 to -100 ms intervals prior to stimulation onset. We used mirrored buffers to avoid inclusion ofdata from stimulation epochs. We averaged power values over the -900 to -100 ms pre-stimulation
interval to generate input patterns for the classifier. We applied the same analysis to the post-stimulation data (and matched periods from NoStim lists) in the interval 100 to 900 ms relative tostimulation offset.
Behavioral analysis
We used the percentage of words recalled from the encoding lists as our behavioral measureof memory performance. We first analyzed the overall effects of stimulation on memory bycomparing recall performance for the Stim and NoStim conditions using two standard assaysof memory performance in free recall tasks: the serial position curve (SPC; Figure 3A,) and thelag conditional response probability curve (CRP; Figure 3B). The SPC depicts the probability ofrecalling a studied word as a function of its position within the study list. The CRP depicts theprobability of recalling two words from the study list as a function of the lag (in number of words)between the serial positions of the two words. We also computed the same measures for therecord-only data.
Analysis of stimulation’s effect on neural activity
For the data in Figure 4B, we compared power within each region of interest at each frequencybetween trials in which the classifier indicated low or high encoding state (defined by classifieroutput being below or above the decision threshold). We computed a two-sample within-subjectt-test to quantify the effect of brain state on spectral power, then computed a one-sample t-testacross the group distribution of within-subject t-statistics. We corrected for multiple comparisonsby controlling the false discovery rate at q = 0.05.
To assess stimulation’s effect on classifier output, we extracted logit-transformed classifier es-timates pre- and post-stimulation (see Spectral analysis of stimulation sessions) and computed awithin-subject paired t-test. We plot the resulting across-subject distribution of t-statistics in Fig-ures 4D and E, separately for trials in which the pre-stimulation classifier estimate was low or high.We then used a one-sample t-test to assess stimulations effect on classifier output across the group.One subject was excluded from the high-state analysis for having only one matched NoStim trial.
To analyze neural activity evoked by stimulation during the low state (Figure 5), we computed asubsequent memory analysis contrasting power for subsequently recalled and not recalled words(i.e. Figure 2G). We corrected for multiple comparisons by controlling the false discovery rate(q = 0.05) and used the resulting map to define a set of significant brain-region × frequencyclusters. To ensure this localizer was fully independent, we did this analysis only for subjectsthat not participate in the stimulation task. We then extracted power in these clusters from thepost-stim intervals in Stim lists and matched intervals in NoStim lists from stimulation task data.For each subject we computed an unpaired t-test within-subject between Stim and NoStim lists,separately for data averaged within the high-frequency clusters and within the low-frequencyclusters. We then took the difference between these two within-subject t-statistics and computedPearson’s r between this distribution and the distribution of mean differences in classifier outputbetween post-stim and matched NoStim intervals.
Analysis of stimulation’s effect on memory performance
To quantify the effect of stimulation on encoding efficiency, we applied the classifier to the pre-and post-stimulation epochs (and matched NoStim periods). We used the inverse logit transformto convert the classifier probability estimates to log-odds. For the analyses in Figures 4 and 5, wecomputed the mean difference in classifier output between Stim lists and NoStim lists. We alsocomputed the difference in memory between Stim and NoStim lists, normalized by the patient?soverall mean recall:
∆R =RS − RNS
RAll
where RX refers to mean percent recall for either Stim lists, NoStim lists, or collapsed across All lists.
To assess the effects of stimulation on recall rate as a function of the classifier output just prior tostimulation delivery, we computed∆R but included in the numerator only those trials that followedthe termination of a stimulation train (Figure 4A) in RS and only trials in matched positions inNoStim lists in RNS. We then split the trials into two groups based on the classifier output during thepre-stimulation interval (see Analysis of stimulation sessions). We used Youden’s J (see Multivariateclassification), derived from classifier output in the record-only sessions, to separate the distributionof pre-stimulation classifier outputs into low and high bins. We then computed ∆R as a functionof state in each subject and compared ∆R to each other and to zero using paired and one-samplet-tests.
Analysis of the Spectral Tilt
For the subjects that performed the stimulation task, we used within-subject repeated measures t-tests to compare power in each region × frequency bin before and after Stim and NoStim intervals.To assess how stimulation modulated the spectral tilt in our stimulation subjects, we first usedthe subset of subjects that did not participate in stimulation sessions to define the clusters of brainregion × frequency space that significantly discriminate recalled and not recalled words in therecord-only sessions (e.g. Figure 2G). Then, in stimulation subjects, we extracted the t-statisticsin these clusters from the post-pre stimulation comparison. We then averaged the all t-stats fromclusters showing HFA increases and separately averaged t-stats from the large cluster showingLFA decreases. We took the difference (HFA-LFA) as the index of the magnitude of the spectral tilt[S5].
Supplemental References
S1. Yushkevich, P.A., Pluta, J.B., Wang, H., Xie, L., Ding, S.L., Gertje, E.C., Mancuso, L., Kliot,D., Das, S.R., and Wolk, D.A. (2015). Automated volumetry and regional thickness analysis ofhippocampal subfields and medial temporal cortical structures in mild cognitive impairment.Human Brain Mapping 36, 258–287.
S2. Avants, B.B., Epstein, C.L., Grossman, M., and Gee, J.C. (2008). Symmetric diffeomorphicimage registration with cross-correlation: evaluating automated labeling of elderly and neu-rodegenerative brain. Medical Image Analysis 12, 26–41.
S3. Fischl, B., van der Kouwe, A., Destrieux, C., Halgren, E., Segonne, F., Salat, D.H., Busa, E.,Seidman, L.J., Goldstein, J., Kennedy, D., et al. (2004). Automatically parcellating the humancerebral cortex. Cerebral Cortex 14, 11–22.
S4. Dykstra, A.R., Chan, A.M., Quinn, B.T., Zepeda, R., Keller, C.J., Cormier, J., Madsen, J.R.,Eskandar, E.N., and Cash, S.S. (2012). Individualized localization and cortical surface-basedregistration of intracranial electrodes. NeuroImage 59, 3563–3570.
S5. Burke, J.F., Long, N.M., Zaghloul, K.A., Sharan, A.D., Sperling, M.R., and Kahana, M.J. (2014).Human intracranial high-frequency activity maps episodic memory formation in space andtime. NeuroImage 85, 834–843.
S6. Hastie, T., Tibshirani, R., and Friedman, J. (2001). The elements of statistical learning (NewYork: Springer-Verlag).
S7. Haufe, S., Meinecke, F., Gorgen, K., Dahne, S., Haynes, J.D., Blankertz, B., and Bießmann, F.(2014). On the interpretation of weight vectors of linear models in multivariate neuroimaging.NeuroImage 87, 96–110.
S8. Benjamini, Y. and Hochberg, Y. (1995). Controlling the False Discovery Rate: a practical andpowerful approach to multiple testing. Journal of Royal Statistical Society, Series B 57, 289–300.