Biological Modeling of Neural Networks:

Week 14 – Dynamics and Plasticity

Wulfram GerstnerEPFL, Lausanne, Switzerland

14.1 Reservoir computing - Complex brain dynamics - Computing with rich dynamics14.2 Random Networks

- stationary state - chaos

14.3 Stability optimized circuits - application to motor data14.4. Synaptic plasticity - Hebbian - Reward-modulated 14.5. Helping Humans - oscillations - deep brain stimulation

Week 14 – Dynamics and Plasticity

Neuronal Dynamics – Brain dynamics is complex

motor cortex

frontal cortex

to motoroutput

10 000 neurons3 km wire

1mm

Week 14-part 1: Review: The brain is complex

Neuronal Dynamics – Brain dynamics is complex

motor cortex

frontal cortex

to motoroutput

Week 14-part 1: Review: The brain is complex

-Complex internal dynamics-Memory-Response to inputs-Decision making-Non-stationary-Movement planning-More than one ‘activity’ value

Liquid Computing/Reservoir Computing: exploit rich brain dynamics

Readout 1

Readout 2

Maass et al. 2002,Jaeger and Haas, 2004Review:Maass and Buonomano,

Stream of sensory inputs

Week 14-part 1: Reservoir computing

See Maass et al. 2007

Week 14-part 1: Reservoir computing

-‘calculcate’ 3 4 - if-condition on 1

ModelingHennequin et al. 2014,See also:Maass et al. 2002,Sussillo and Abbott, 2009Laje and Buonomano, 2012Shenoy et al., 2011

Experiments ofChurchland et al. 2010Churchland et al. 2012See also:Shenoy et al. 2011

Week 14-part 1: Rich dynamics Rich neuronal dynamics

-Long transients

-Reliable (non-chaotic)

-Rich dynamics (non-trivial)

-Compatible with neural data (excitation/inhibition)

-Plausible plasticity rules

Week 14-part 1: Rich neuronal dynamics: a wish list

Biological Modeling of Neural Networks:

Week 14 – Dynamics and Plasticity

Wulfram GerstnerEPFL, Lausanne, Switzerland

14.1 Reservoir computing - Complex brain dynamics - Computing with rich dynamics14.2 Random Networks

- rate model - stationary state and chaos

14.3 Hebbian Plasticity - excitatory synapses - inhibitory synapses14.4. Reward-modulated plasticity - free solution 14.5. Helping Humans - oscillations - deep brain stimulation

Week 14 – Dynamics and Plasticity

I(t)

)(tAn

Week 14-part 2: Review: microscopic vs. macroscopic

Homogeneous network:-each neuron receives input from k neurons in network-each neuron receives the same (mean) external input

Week 14-part 2: Review: Random coupling

excitation

inhibition

Stochastic spike arrival: excitation, total rate Re

inhibition, total rate Ri

u0u

EPSC IPSC

Synaptic current pulses

)()()( ttIRuuudt

d meaneq

Langevin equation,Ornstein Uhlenbeck process Fokker-Planck equation

)()()(','

''

, fk

fki

fk

fkeeq ttqttqRuuu

dt

d

Week 14-part 2: Review: integrate-and-fire/stochastic spike arrival

Firing times:Threshold crossing

( )i i ij jj

dr r F w r

dt

Fixed point with F(0)=0 0ir

stable unstable

Suppose:1 '(0) ( 0)

dF F x

dx

( )dx x F w x

dt

Suppose 1 dimension

Week 14-part 2: Dynamics in Rate Networks F-I curve of rate neuron

Slope 1

Exercise 1: Stability of fixed point

Next lecture: 9h43( )

dx x F w x

dt

Fixed point with F(0)=0 0x Suppose:

1 '(0) ( 0)d

F F xdx

( )dx x F w x

dt

Suppose 1 dimension

Calculate stability, take w as parameter

( )i i ij jj

dr r F w r

dt

Fixed point with F(0)=0 0ir

stable unstable

Suppose:1 '(0) ( 0)

dF F x

dx

( )dx x F w x

dt

w<1 w>1

Suppose 1 dimension

Week 14-part 2: Dynamics in Rate Networks

Blackboard:Two dimensions!

( )i i ij jj

dr r F w r

dt

Fixed point:0ir

stable unstable

Chaotic dynamics: Sompolinksy et al. 1988 (and many others:Amari, ...

1 '(0) ( 0)d

F F xdx

Random,10 percent connectivity

Re( ) 1 Re( ) 1

Week 14-part 2: Dynamics in RANDOM Rate Networks

Rajan and Abbott, 2006Image: Ostojic, Nat.Neurosci, 2014

( ) ( )i i ij j ij

dr r F w r t

dt

Unstable dynamics and Chaos

Image: Hennequin et al. Neuron, 2014

chaos

Image: Ostojic, Nat.Neurosci, 2014

( )fi i ij jj f

du u w t t

dt

Firing times:Threshold crossing

Week 14-part 2: Dynamics in Random SPIKING Networks

Ostojic, Nat.Neurosci, 2014

Week 14-part 2: Stationary activity: two different regimes

Switching/bursts long autocorrelations:Rate chaos

Re( ) 1 Stable rate fixed point,microscopic chaos

-Long transients

-Reliable (non-chaotic)

-Rich dynamics (non-trivial)

-Compatible with neural data (excitation/inhibition)

-Plausible plasticity rules

Week 14-part 2: Rich neuronal dynamics: a wish list

Biological Modeling of Neural Networks:

Week 14 – Dynamics and Plasticity

Wulfram GerstnerEPFL, Lausanne, Switzerland

14.1 Reservoir computing - Complex brain dynamics - Computing with rich dynamics14.2 Random Networks

- stationary state - chaos

14.3 Stability optimized circuits - application to motor data14.4. Synaptic plasticity - Hebbian - Reward-modulated 14.5. Helping Humans - oscillations - deep brain stimulation

Week 14 – Dynamics and Plasticity

Image: Hennequin et al. Neuron, 2014

Re( ) 1 Re( ) 1

Week 14-part 3: Plasticity-optimized circuit

Optimal control of transient dynamics in balanced networkssupports generation of complex movements

Hennequin et al. 2014,

Random stability-optimized circuit (SOC)

Random connectivity

Random

Week 14-part 3: Random Plasticity-optimized circuit

studydurationof transients

F-I curve of rate neuron

slope 1/linear theory

slope 1/linear theory

a1= slowest = most amplified

Week 14-part 3: Random Plasticity-optimized circuit

slope 1/linear theoryF-I curve of rate neuron

a1= slowest = most amplified

Week 14-part 3: Random Plasticity-optimized circuit

Optimal control of transient dynamics in balanced networkssupports generation of complex movements

Hennequin et al. NEURON 2014,

Churchland et al. 2010/2012 Hennequin et al. 2014

Week 14-part 3: Application to motor cortex: data and model

Quiz: experiments of Churchland et al.

[ ] Before the monkey moves his arm, neurons in motor-related areas exhibit activity[ ] While the monkey moves his arm, different neurons in motor- related area show the same activity patterns [ ] while the monkey moves his arm, he receives information which of the N targets he has to choose [ ] The temporal spike pattern of a given neuron is nearly the same, between one trial and the next (time scale of a few milliseconds)[ ] The rate activity pattern of a given neuron is nearly the same, between one trial and the next (time scale of a hundred milliseconds)

Comparison: weak random

Hennequin et al. 2014,

Week 14-part 3: Random Plasticity-optimized circuit

Random connections, fast

‘distal’ connections, slow, (Branco&Hausser, 2011) structured

12000 excitatory LIF = 200 pools of 60 neurons 3000 inhibitory LIF = 200 pools of 15 neurons

Classic sparse random connectivity (Brunel 2000)

Stabilizy-optimized random connections

Overall:20% connectivity

Fast AMPA

slow NMDA

Week 14-part 3: Stability optimized SPIKING network

Neuron 1Neuron 2Neuron 3

Spontaneous firing rate

Single neuron different initial conditions

Hennequin et al. 2014

Classic sparse random connectivity (Brunel 2000)

Week 14-part 3: Stability optimized SPIKING network

Hennequin et al. 2014

Classic sparse random connectivity (Brunel 2000)

Week 14-part 3: Stability optimized SPIKING network

-Long transients

-Reliable (non-chaotic)

-Rich dynamics (non-trivial)

-Compatible with neural data (excitation/inhibition)

-Plausible plasticity rules

Week 14-part 3: Rich neuronal dynamics: a result list

Biological Modeling of Neural Networks:

Week 14 – Dynamics and Plasticity

Wulfram GerstnerEPFL, Lausanne, Switzerland

14.1 Reservoir computing - complex brain dynamics - Computing with rich dynamics14.2 Random Networks

- stationary state - chaos

14.3 Stability optimized circuits - application to motor data14.4. Synaptic plasticity - Hebbian - Reward-modulated 14.5. Helping Humans - oscillations - deep brain stimulation

Week 14 – Dynamics and Plasticity

Hebbian Learning= all inputs/all times are equal

),( postpreFwij

prepost

ij

ijw

Week 14-part 4: STDP = spike-based Hebbian learning

Pre-before post: potentiation of synapse

Pre-after-post: depression of synapse

Modulation of Learning= Hebb+ confirmation

confirmation

( , , )ijw F pre post CONFIRM

local global

Functional PostulateUseful for learning the important stuff

Many models (and experiments) of synaptic plasticity do not take into account Neuromodulators.

Except: e.g. Schultz et al. 1997, Wickens 2002, Izhikevich, 2007; Reymann+Frey 2007; Moncada 2007, Pawlak+Kerr 2008; Pawlak et al. 2010

Consolidation of Learning

Success/reward Confirmation -Novel-Interesting-Rewarding-Surprising

Neuromodulatorsdopmaine/serotonin/Ach

‘write now’ to long-term memory’

Crow (1968), Fregnac et al (2010), Izhikevich (2007)

Plasticity

BUT - replace by inhibitory plasticity avoids chaotic blow-up of network

avoids blow-up of single neuron (detailed balance) yields stability optimized circuits

Vogels et al.,Science 2011

- here: algorithmically tuned

Stability-optimized curcuits

Plasticity

BUT

- replace by 3-factor plasticity rules

- here: algorithmically tuned

Readout

Izhikevich, 2007Fremaux et al. 2012 Success signal

Week 14-part 4: Plasticity modulated by reward

Dopamine encodes success= reward – expected reward

Dopamine-emitting neurons: Schultz et al., 1997

Izhikevich, 2007Fremaux et al. 2012 Success signal

Week 14-part 4: Plasticity modulated by reward

Week 14-part 4: STDP = spike-based Hebbian learning

Pre-before post: potentiation of synapse

Week 14-part 4: Plasticity modulated by reward

STDP with pre-before post: potentiation of synapse

Quiz: Synpatic plasticity: 2-factor and 3-factor rules

[ ] a Hebbian learning rule depends only on presynaptic and postsynaptic variables, pre and post[ ] a Hebbian learning rule can be written abstractly as [ ] STDP is an example of a Hebbian learning rule

( , , )ijw F pre post CONFIRM

[ ] a 3-factor learning rule can be written as

[ ] a reward-modulated learning rule can be written abstractly as

( , , )ijw F pre post globalfactor

( , , )ijw F pre post success

[ ] a reward-modulated learning rule can be written abstractly as ( , , )ijw F pre post neuroMODULATOR

Week 14-part 4: from spikes to movement

How can the readouts encode movement?

Population vector coding of movements

Schwartz et al. 1988

Week 14-part 5: Population vector coding

•70’000 synapses•1 trial =1 second•Output to trajectories via population vector coding•Single reward at the END of each trial based on

similarity with a target trajectory

Population vector coding of movements

Fremaux et al., J. Neurosci. 2010

Week 14-part 4: Learning movement trajectories

Per

form

ance

Fremaux et al. J. Neurosci. 2010

QuickTime™ and a decompressor

are needed to see this picture.

R-STDP LTP-only

Week 14-part 4: Learning movement trajectories

Reward-modulatedSTDP for movement learning - Readout connections, tuned by 3-factor plasticity rule

Fremaux et al. 2012 Success signal

Hebbian STDP - inhibitory connections, tuned by 2-factor STDP, for stabilization

Week 14-part 4: Plasticity can tune the network and readout

Vogels et al. 2011

Exam: - written exam 17. 06. 2014 from 16:15-19:00 - miniprojects counts 1/3 towards final grade For written exam:

-bring 1 page A5 of own notes/summary-HANDWRITTEN!

Last Lecture TODAY

Nearly the end: what can I improve for the students next year?

Integrated exercises?

Miniproject?

Overall workload ?(4 credit course = 6hrs per week)

Background/Prerequisites?

-Physics students-SV students-Math students

Quizzes?

Slides? videos?

Biological Modeling of Neural Networks:

Week 14 – Dynamics and Plasticity

Wulfram GerstnerEPFL, Lausanne, Switzerland

14.1 Reservoir computing - Review:Random Networks - Computing with rich dynamics14.2 Random Networks

- stationary state - chaos

14.3 Stability optimized circuits - application to motor data14.4. Synaptic plasticity - Hebbian - Reward-modulated 14.5. Helping Humans - oscillations - deep brain stimulation

Week 14 – Dynamics and Plasticity

Week 14-part 5: Oscillations in the brain

Individual neurons fire regularly

two groups alternate

Week 14-part 5: Oscillations in the brain

Individual neurons fire irregularly

Week 14-part 5: STDP

Week 14-part 5: STDP during oscillations

Oscillation near-synchronous spike arrival

Week 14-part 5: Helping Humans

Pfister and Tass, 2010

big weights

small weights

Week 14-part 5: Helping Humans: Deep brain stimulation

Parkinson’s disease:Symptom: Tremor

-periodic shaking- 3-6 Hz

Brain activity: - thalamus, basal ganglia- synchronized 3-6Hz in Parkinson-

Deep brain stimulation Benabid et al, 1991, 2009

Week 14-part 5: Helping Humans

healthy pathologicalAbstractDynamicalSystemsView ofBrain states

Week 14-part 5: Helping Humans: coordinated reset

Pathological, synchronous

Coordinated reset stimulus

Week 14-part 5: Helping Humans

Tass et al. 2012

The EndPlasticity and Neuronal Dynamics

Theoretical concepts can help to understand brain dynamics contribute to understanding plasticity and learning inspire medical approaches can help humans

now QUESTION SESSION!

Questions to Assistants possible until June 1

The end

… and good luck for the exam!

Hennequin et al. 2014,

Week 14-part 3: Correlations in Plasticity-optimized circuit