Learning Spatio-Temporally Encoded Pattern Transformations ... · Learning Spatio-Temporally...
Transcript of Learning Spatio-Temporally Encoded Pattern Transformations ... · Learning Spatio-Temporally...
Introduction Background Our Approach Results Summary
Learning Spatio-Temporally Encoded PatternTransformations in Structured Spiking Neural
Networks12
Andre Gruning, Brian Gardner and Ioana SporeaDepartment of Computer Science
University of SurreyGuildford, UK
9th November 2015
1http://dx.doi.org/10.6084/m9.figshare.15177792$Id: multilayerspiker.txt 1827 2015-11-09 07:39:53Z ag0015 $
Introduction Background Our Approach Results Summary
Introduction Background Our Approach Results Summary
Introduction Background Our Approach Results Summary
1 Introduction
2 Background
3 Our Approach
4 Results
5 Summary
Introduction Background Our Approach Results Summary
What are we doing?
What are we doing?
Formulate a supervised learning rule for spiking neuralnetworks that
can train spiking networks containing a hidden layer of neurons,
can map arbitrary spatio-temporal input into arbitrary outputspike patterns, ie multiple spike trains.
Why worthwhile?
Understand how spike-pattern based information processingtakes place in the brain.
A learning rule for spiking neural networks with technicalpotential.
Find a rule that is to spiking networks what is backprop torate neuron networks.
Human Brain Project
The SPIKEFRAME project:
“Structures of Learning Algorithms for Spiking NeuralNetworks”
joined HBP on 01/04/2015 through Competitive Callsprogramme
This work carried out as part of SP4 “TheoreticalNeuroscience”.
Introduction Background Our Approach Results Summary
Scientific Area
Where are we scientifically?
In the middle of nowhere between:
computational neuroscience
cognitive science
artificial intelligence / machine learning
Introduction Background Our Approach Results Summary
Introduction Background Our Approach Results Summary
1 Introduction
2 Background
3 Our Approach
4 Results
5 Summary
Introduction Background Our Approach Results Summary
Spiking Neurons
(a)
(c)
output spike
input spikes
output spike
input spikes
u
(b)
Spiking neurons: real neurons communicate with each other via sequences of pulses – spikes.
1 Dendritic tree, axon and cell body of a neuron.
2 Top: Spikes arrive from other neurons and its membrane potential rises. Bottom: incoming spikes onvarious dendrites elicit timed spikes responses as the output.
3 response of the membrane potential to incoming spikes. If the threshold θ is crossed, the membranepotential is reset to a low value, and a spike fired.
From Andre Gruning and Sander Bohte. Spiking neural networks: Principles and challenges.
In Proceedings of the 22nd European Symposium on Artificial Neural Networks, Computational Intelligence andMachine Learning – ESANN, Brugge, 2014.
Invited Contribution.
Introduction Background Our Approach Results Summary
Spiking Neurons
Spiking Information Processing
The precise timing of spikes generated by neurons conveysmeaningful information.
Synaptic plasticity forms the basis of learning.
Changes in synaptic strength depend on relative pre- andpostsynaptic spike times, and third signals.
Challenge: to relate such localised plasticity changes tolearning on the network level.
Introduction Background Our Approach Results Summary
Learning for Spiking NN
General Learning Algorithms for Spiking NN?
There is no general-purpose algorithm for spiking neuralnetworks.
Challenge: discontinuous nature of spiking events.
Various supervised learning algorithms exist, each with its ownlimitations eg:
network topology,
adaptability (e.g. reservoir computing),
limited spike encoding (e.g. latency, or spike vs no spike).
Most focus on classification rather than more challengingtasks like mapping from one spike train to another.
Introduction Background Our Approach Results Summary
Some Learning Algorithms for Spiking NN
SpikeProp 3, ReSuMe 4, Tempotron 5, Chronotron 6, SPAN 7,Urbanczik and Senn 8, Brea et al. 9, Freimaux et al. 10, . . .
3S.M. Bohte, J.N. Kok, and H. La Poutre. Spike-prop: error-backpropagation in multi-layer networks of spiking neurons.
Neurocomputing, 48(1–4):17–37, 20024
Filip Ponulak and Andrzej Kasinski. Supervised learning in spiking neural networks with ReSuMe: Sequence learning, classificationand spike shifting.Neural Computation, 22:467–510, 2010
5Robert Gutig and Haim Sompolinsky. The tempotron: a neuron that learns spike timing-based decisions.
Nature Neuroscience, 9(3), 2006.doi: 10.1038/nn1643
6Razvan V Florian. The chronotron: A neuron that learns to fire temporally precise spike patterns.
PLoS ONE, 7(8):e40233, 20127
A. Mohemmed, S. Schliebs, and N. Kasabov. SPAN: Spike pattern association neuron for learning spatio-temporal sequences.Int. J. Neural Systems, 2011
8R. Urbanczik and W. Senn. A gradient learning rule for the tempotron.
Neural Computation, 21:340–352, 20099
Johanni Brea, Walter Senn, and Jean-Pascal Pfister. Matching recall and storage in sequence learning with spiking neural networks.The Journal of Neuroscience, 33(23):9565–9575, 2013
10Nicolas Fremaux, Henning Sprekeler, and Wulfram Gerstner. Functional requirements for reward-modulated spile-timing-dependent
plasticity.The Journal of Neuroscience, 30(40):13326–13337, 10 2010
Introduction Background Our Approach Results Summary
Introduction Background Our Approach Results Summary
1 Introduction
2 Background
3 Our Approach
4 Results
5 Summary
Introduction Background Our Approach Results Summary
Our Approach
MultilayerSpiker
Generalise backpropagation to Spiking Neural Networks withhidden neurons.
Use stochastic neuron model
to connect smooth quantities (derivative exists) with discretespike trains (no derivative)
Introduction Background Our Approach Results Summary
Neuron model
Membrane potential
uo(t) :=∑
h
woh
∫ t
0Yh(t ′)ε(t−t ′)dt ′+
∫ t
0Zo(t ′)κ(t−t ′)dt ′ , (1)
o postsynaptic neurons, h presynaptic neuron
uo membrane potential of o.
woh strength of synaptic connection from h to o.
Yh(t) =∑
th<t δ(t − th) spike train of neuron h where th arethe firing times of h
Zo(t) =∑
to<t δ(t − to) spike train of neuron o where to arethe firing times of o.
Introduction Background Our Approach Results Summary
Neuron model
Spike response kernel ε and reset kernel κ
ε(s) = ε0 [e−s/τm−e−s/τs ] Θ(s) and κ(s) = κ0e−s/τm Θ(s) , (2)
spike response kernel ε0 = 4mV,
reset kernel κ0 = −15mV,
membrane time constant τm = 10ms,
the synaptic rise time τs = 5ms
Heaviside step function Θ(s).
Introduction Background Our Approach Results Summary
Neuron model
Stochastic Intensity (instantaneous firing rate) and Spikes
ρ(t) = ρ[u(t)] = ρ0 exp
(u(t)− ϑ
∆u
), (3)
firing rate at threshold ρ0 = 0.01ms−1.
“threshold” ϑ = 15mV.
smoothness of the threshold ∆uo = 0.2mV (output layer) or∆uh = 2mV (hidden layer)
Spikes are generated by a point process taking stochasticintensity ρo(t).
Ie in a small time interval [t, t + δt) a spike is generated withprobability ρo(t)δt.
Introduction Background Our Approach Results Summary
Backpropagation
Objective (“Error”) function
P(zrefo |x) = exp
(∫log (ρo(t))Z ref
o (t)− ρo(t)dt
), (4)
where Z refo (t) =
∑f δ(t − t f
o ) is the target output spike train forinput x .a
aJ. P. Pfister, T. Toyoizumi, K. Aihara, and W. Gerstner. Optimal spike-timing dependent plasticity for precise action potential firing
in supervised learning.Neural Computation, 18(6):1309–1339, 2006
Backprop approach
∆woh = ηo∂ logP(zref |x)
∂woh(5)
Introduction Background Our Approach Results Summary
Backprop approach
. . . and some ten slides later
Lots of derivatives, indices, probabilities.
Derivatives only possible due to smoothness of probabilityfunction.
Relatively freely switching between expected values and theirbest estimates to be had when you only have single cast.
Introduction Background Our Approach Results Summary
Backprop Weight Update
Backpropagated Error Signal
δo(t) :=1
∆uo
[Z ref
o (t)− ρo(t)], (6)
Hidden-to-Output Weights
∆woh = ηo
∫ T
0δo(t) (Yh ∗ ε)(t) dt . (7)
Input-to-Hidden Weights
∆whi =ηh
∆uh
∑o
woh
∫ T
0δo(t)([Yh(Xi ∗ ε)] ∗ ε)(t)dt . (8)
a
aBrian Gardner, Ioana Sporea, and Andre Gruning. Learning spatio-temporally encoded pattern transformations in structured spiking
neural networks.Neural Computation, To appear. 2015.Preprint available at http://arxiv.org/abs/1503.09129
Introduction Background Our Approach Results Summary
Introduction Background Our Approach Results Summary
1 Introduction
2 Background
3 Our Approach
4 Results
5 Summary
Introduction Background Our Approach Results Summary
Task
Task
Purpose: explore the properties of the new learning algorithm.
Map an input (given as a set of spike trains) to an output(given again as a set of spike trains).
Simulation details a.
aBrian Gardner, Ioana Sporea, and Andre Gruning. Learning spatio-temporally encoded pattern transformations in structured spiking
neural networks.Neural Computation, To appear. 2015.Preprint available at http://arxiv.org/abs/1503.09129
Introduction Background Our Approach Results Summary
Network Setup
0
20
40
60
80
100Input spike pattern
Xi
0
200
400
600
800
1000
Epis
odes
0 100 200 300 400 5000
200
400
600
800
1000
Time (ms)
Epis
odes
0 200 400 600 800 10000
1
2
3
4
Episodes
Dis
tance
E
D
A
B
C
Input
Hidden
Output
Output neuron
Hidden neuron
Left: spike rasters of input, hidden and output layers (withtargets).
Right top: network structure, bottom: van-Rossum distance.
Introduction Background Our Approach Results Summary
Network in Action
0
Xi
0 T0
(Xi ∗ ǫ)
0
uh − ϑ
0 T0
([Yh(Xi ∗ ǫ)] ∗ ǫ)
0
uo − ϑ
0 T
0
∆whi
Left: Input spike train Xi (top) and its evoked post synaptic potential Xi ∗ ε (bottom).
Middle: Fluctuations of a hidden neuron membrane potential uh relative to a firing threshold ϑ, inresponse to inputs from input layer (top). The potential dependent factor of the back propagated errorfrom hidden to input layer [Yh(Xi ∗ ∗ε)]
bottom left: corresponding PSP (according to kernel Xi ∗ ε).
Right: membrane potential of an output neuron uo , in response to hidden layer activity. Target indicatedby dotted lines (top). Weight changes of input-to-hidden weight due to learning rule.
Introduction Background Our Approach Results Summary
Experiments
0 8 16 24 32 400
20
40
60
80
100
Pe
rfo
rma
nce
(%
)
Input patterns
0 8 16 24 32 400
5000
10000
15000
Ep
iso
de
s
Input patterns
Free whi
Fixed whi
Single layer
Dependence of the performance on the number of input patterns andnetwork setup. Each input pattern mapped to a unique target output ofsingle output neuron and spike.
Left: performance as a function of the number of input patterns.
Right: Number of episodes to convergence in learning.
Blue curves: hidden weights whi updated according to learningalgorithm, red curves: fixed random weights (plus homoeostasis),green: single layer.
Introduction Background Our Approach Results Summary
Experiments
0 2 4 6 8 100
1
2
3
4
5
nh
/ n
o
Number of output spikes
no
= 10
0 0.5 1 1.5 2 2.5 30
20
40
60
80
100P
erf
orm
ance (
%)
nh
/ no
no
= 10
no
= 20
no
= 30
A B
Dependence of the performance on the ratio of hidden to output neurons,and the number of target output spikes. p = 50. A unique target outputspike pattern for each output neuron.
(Left) Performance as a function of the ratio of hidden to outputneurons.
(Right) Minimum ratio of hidden to output neurons required toachieve 90% performance.
Introduction Background Our Approach Results Summary
Introduction Background Our Approach Results Summary
1 Introduction
2 Background
3 Our Approach
4 Results
5 Summary
Introduction Background Our Approach Results Summary
Summary
Results
Compared to other learning algorithms for spiking neuronnetworks, we can learn
more input-output mappings: 20 classes or 200 individualpatterns here vs 3-4
more timed output spikes: up to 10 individually timed spikeshere vs 3-5
with multiple outputs: up to 30 here vs 1
Apply it!
MultilayerSpiker opens up the use of spiking neural networksfor technical/cognitive modelling tasks.
Spiking networks are biologically plausible.
Explore how computations can be done with neural networks.
Next step in the Human Brain Project: Implementation onSpiNNaker, and other neural hardware.
Introduction Background Our Approach Results Summary
Spiking Neural Networks
Open Questions
How do networks of spiking neurons carry out computations?
How can they learn such computations?
Does this explain how real biological neurons compute?
What is the applied killer application?