Neuromorphic VLSI Event-Based devices and systems/file/Giacomo_Indiveri.pdf · 2012-06-27 ·...
Transcript of Neuromorphic VLSI Event-Based devices and systems/file/Giacomo_Indiveri.pdf · 2012-06-27 ·...
![Page 1: Neuromorphic VLSI Event-Based devices and systems/file/Giacomo_Indiveri.pdf · 2012-06-27 · Neuromorphic VLSI systems 100µ [Nuno da Costa, INI, 2008] Goals: To exploit the physics](https://reader033.fdocuments.us/reader033/viewer/2022041906/5e63ce57376f191f0602fada/html5/thumbnails/1.jpg)
Neuromorphic VLSI Event-Based devices and systems
Giacomo Indiveri
Institute of NeuroinformaticsUniversity of Zurich and ETH Zurich
LTU, LuleaMay 28, 2012
G.Indiveri (http://ncs.ethz.ch/) Neuromorphic spiking chips 1 / 38
![Page 2: Neuromorphic VLSI Event-Based devices and systems/file/Giacomo_Indiveri.pdf · 2012-06-27 · Neuromorphic VLSI systems 100µ [Nuno da Costa, INI, 2008] Goals: To exploit the physics](https://reader033.fdocuments.us/reader033/viewer/2022041906/5e63ce57376f191f0602fada/html5/thumbnails/2.jpg)
Outline
1 “Neuromorphic Engineering”
2 Spike-based sensory systems
3 Spiking Neural NetworksSilicon NeuronsSilicon synapsesWinner-Take-All networksMulti-chip networks
4 Learning
5 Neuromorphic Cognitive Systems
G.Indiveri (http://ncs.ethz.ch/) Neuromorphic spiking chips 2 / 38
![Page 3: Neuromorphic VLSI Event-Based devices and systems/file/Giacomo_Indiveri.pdf · 2012-06-27 · Neuromorphic VLSI systems 100µ [Nuno da Costa, INI, 2008] Goals: To exploit the physics](https://reader033.fdocuments.us/reader033/viewer/2022041906/5e63ce57376f191f0602fada/html5/thumbnails/3.jpg)
Natural ComputationThe Honeybee
Energy consumption: 10−15 J/op, at least 106
more efficient than digital silicon(20watts vs. 1Mil.watts)
The brain of the workerhoneybee occupies a volumeof around 1mm3 and weighsabout 1mg.The total number of neurons inthe brain is estimated to be950,000
Flies acrobatically
Recognizes patterns
Navigates
Forages
Communicates
G.Indiveri (http://ncs.ethz.ch/) Neuromorphic spiking chips 3 / 38
![Page 4: Neuromorphic VLSI Event-Based devices and systems/file/Giacomo_Indiveri.pdf · 2012-06-27 · Neuromorphic VLSI systems 100µ [Nuno da Costa, INI, 2008] Goals: To exploit the physics](https://reader033.fdocuments.us/reader033/viewer/2022041906/5e63ce57376f191f0602fada/html5/thumbnails/4.jpg)
Natural ComputationThe Honeybee
Energy consumption: 10−15 J/op, at least 106
more efficient than digital silicon(20watts vs. 1Mil.watts)
The brain of the workerhoneybee occupies a volumeof around 1mm3 and weighsabout 1mg.The total number of neurons inthe brain is estimated to be950,000
Flies acrobatically
Recognizes patterns
Navigates
Forages
Communicates
G.Indiveri (http://ncs.ethz.ch/) Neuromorphic spiking chips 3 / 38
![Page 5: Neuromorphic VLSI Event-Based devices and systems/file/Giacomo_Indiveri.pdf · 2012-06-27 · Neuromorphic VLSI systems 100µ [Nuno da Costa, INI, 2008] Goals: To exploit the physics](https://reader033.fdocuments.us/reader033/viewer/2022041906/5e63ce57376f191f0602fada/html5/thumbnails/5.jpg)
Neocortex → Neural computation → Silicon→ Behavior
Synaptic Inputs Constant current Synapse
Soma
0 0.05 0.1 0.15 0.20
0.2
0.4
0.6
0.8
1
Time (s)
Vm
em
(V)
G.Indiveri (http://ncs.ethz.ch/) Neuromorphic spiking chips 4 / 38
![Page 6: Neuromorphic VLSI Event-Based devices and systems/file/Giacomo_Indiveri.pdf · 2012-06-27 · Neuromorphic VLSI systems 100µ [Nuno da Costa, INI, 2008] Goals: To exploit the physics](https://reader033.fdocuments.us/reader033/viewer/2022041906/5e63ce57376f191f0602fada/html5/thumbnails/6.jpg)
Neuromorphic VLSI systems
100µ
[Nuno da Costa, INI, 2008]
Goals:
To exploit the physics of silicon to reproduce the bio-physics of neuralsystems, using subthreshold analog VLSI circuits.
To develop multi-chip spike-based computing systems, using theAddress-Event Representation (AER) and asynchronous digital VLSItechnology.
To automatically configure and “program” neuromorphic processingsystems distributed across multiple chips, to carry out real–timebehavioral tasks.
G.Indiveri (http://ncs.ethz.ch/) Neuromorphic spiking chips 5 / 38
![Page 7: Neuromorphic VLSI Event-Based devices and systems/file/Giacomo_Indiveri.pdf · 2012-06-27 · Neuromorphic VLSI systems 100µ [Nuno da Costa, INI, 2008] Goals: To exploit the physics](https://reader033.fdocuments.us/reader033/viewer/2022041906/5e63ce57376f191f0602fada/html5/thumbnails/7.jpg)
Neuromorphic VLSI systems
100µ
[Nuno da Costa, INI, 2008]
Goals:
To exploit the physics of silicon to reproduce the bio-physics of neuralsystems, using subthreshold analog VLSI circuits.
To develop multi-chip spike-based computing systems, using theAddress-Event Representation (AER) and asynchronous digital VLSItechnology.
To automatically configure and “program” neuromorphic processingsystems distributed across multiple chips, to carry out real–timebehavioral tasks.
G.Indiveri (http://ncs.ethz.ch/) Neuromorphic spiking chips 5 / 38
![Page 8: Neuromorphic VLSI Event-Based devices and systems/file/Giacomo_Indiveri.pdf · 2012-06-27 · Neuromorphic VLSI systems 100µ [Nuno da Costa, INI, 2008] Goals: To exploit the physics](https://reader033.fdocuments.us/reader033/viewer/2022041906/5e63ce57376f191f0602fada/html5/thumbnails/8.jpg)
Neuromorphic VLSI systems
100µ
[Nuno da Costa, INI, 2008]
Goals:
To exploit the physics of silicon to reproduce the bio-physics of neuralsystems, using subthreshold analog VLSI circuits.
To develop multi-chip spike-based computing systems, using theAddress-Event Representation (AER) and asynchronous digital VLSItechnology.
To automatically configure and “program” neuromorphic processingsystems distributed across multiple chips, to carry out real–timebehavioral tasks.
G.Indiveri (http://ncs.ethz.ch/) Neuromorphic spiking chips 5 / 38
![Page 9: Neuromorphic VLSI Event-Based devices and systems/file/Giacomo_Indiveri.pdf · 2012-06-27 · Neuromorphic VLSI systems 100µ [Nuno da Costa, INI, 2008] Goals: To exploit the physics](https://reader033.fdocuments.us/reader033/viewer/2022041906/5e63ce57376f191f0602fada/html5/thumbnails/9.jpg)
Outline
1 “Neuromorphic Engineering”
2 Spike-based sensory systems
3 Spiking Neural NetworksSilicon NeuronsSilicon synapsesWinner-Take-All networksMulti-chip networks
4 Learning
5 Neuromorphic Cognitive Systems
G.Indiveri (http://ncs.ethz.ch/) Neuromorphic spiking chips 6 / 38
![Page 10: Neuromorphic VLSI Event-Based devices and systems/file/Giacomo_Indiveri.pdf · 2012-06-27 · Neuromorphic VLSI systems 100µ [Nuno da Costa, INI, 2008] Goals: To exploit the physics](https://reader033.fdocuments.us/reader033/viewer/2022041906/5e63ce57376f191f0602fada/html5/thumbnails/10.jpg)
AER silicon retinasTobi Delbruck
G.Indiveri (http://ncs.ethz.ch/) Neuromorphic spiking chips 7 / 38
![Page 11: Neuromorphic VLSI Event-Based devices and systems/file/Giacomo_Indiveri.pdf · 2012-06-27 · Neuromorphic VLSI systems 100µ [Nuno da Costa, INI, 2008] Goals: To exploit the physics](https://reader033.fdocuments.us/reader033/viewer/2022041906/5e63ce57376f191f0602fada/html5/thumbnails/11.jpg)
Silicon retina propertieshttp://siliconretina.ini.uzh.ch
(movie)G.Indiveri (http://ncs.ethz.ch/) Neuromorphic spiking chips 8 / 38
![Page 12: Neuromorphic VLSI Event-Based devices and systems/file/Giacomo_Indiveri.pdf · 2012-06-27 · Neuromorphic VLSI systems 100µ [Nuno da Costa, INI, 2008] Goals: To exploit the physics](https://reader033.fdocuments.us/reader033/viewer/2022041906/5e63ce57376f191f0602fada/html5/thumbnails/12.jpg)
An AER silicon cochleaShih-Chii Liu
G.Indiveri (http://ncs.ethz.ch/) Neuromorphic spiking chips 9 / 38
![Page 13: Neuromorphic VLSI Event-Based devices and systems/file/Giacomo_Indiveri.pdf · 2012-06-27 · Neuromorphic VLSI systems 100µ [Nuno da Costa, INI, 2008] Goals: To exploit the physics](https://reader033.fdocuments.us/reader033/viewer/2022041906/5e63ce57376f191f0602fada/html5/thumbnails/13.jpg)
Silicon cochlea properties
G.Indiveri (http://ncs.ethz.ch/) Neuromorphic spiking chips 10 / 38
![Page 14: Neuromorphic VLSI Event-Based devices and systems/file/Giacomo_Indiveri.pdf · 2012-06-27 · Neuromorphic VLSI systems 100µ [Nuno da Costa, INI, 2008] Goals: To exploit the physics](https://reader033.fdocuments.us/reader033/viewer/2022041906/5e63ce57376f191f0602fada/html5/thumbnails/14.jpg)
Silicon cochlea properties
G.Indiveri (http://ncs.ethz.ch/) Neuromorphic spiking chips 10 / 38
![Page 15: Neuromorphic VLSI Event-Based devices and systems/file/Giacomo_Indiveri.pdf · 2012-06-27 · Neuromorphic VLSI systems 100µ [Nuno da Costa, INI, 2008] Goals: To exploit the physics](https://reader033.fdocuments.us/reader033/viewer/2022041906/5e63ce57376f191f0602fada/html5/thumbnails/15.jpg)
Outline
1 “Neuromorphic Engineering”
2 Spike-based sensory systems
3 Spiking Neural NetworksSilicon NeuronsSilicon synapsesWinner-Take-All networksMulti-chip networks
4 Learning
5 Neuromorphic Cognitive Systems
G.Indiveri (http://ncs.ethz.ch/) Neuromorphic spiking chips 11 / 38
![Page 16: Neuromorphic VLSI Event-Based devices and systems/file/Giacomo_Indiveri.pdf · 2012-06-27 · Neuromorphic VLSI systems 100µ [Nuno da Costa, INI, 2008] Goals: To exploit the physics](https://reader033.fdocuments.us/reader033/viewer/2022041906/5e63ce57376f191f0602fada/html5/thumbnails/16.jpg)
Implement neural computation in silicon
Classical neural networks
w1
w2
w3
w4
wn
Post-synaptic Output
Pre
-syn
aptic
Inpu
ts
Neuromorphic multi-neuron networks
w2 w3 w4 wnw1
Pre-synaptic Inputs
Post-synaptic Output
G.Indiveri (http://ncs.ethz.ch/) Neuromorphic spiking chips 12 / 38
![Page 17: Neuromorphic VLSI Event-Based devices and systems/file/Giacomo_Indiveri.pdf · 2012-06-27 · Neuromorphic VLSI systems 100µ [Nuno da Costa, INI, 2008] Goals: To exploit the physics](https://reader033.fdocuments.us/reader033/viewer/2022041906/5e63ce57376f191f0602fada/html5/thumbnails/17.jpg)
Spiking multi-neuron architectures
Networks of silicon neurons with adaptation,refractory period, etc.
Silicon synapses with realistic temporal dynamics
Winner-Take-All architectures
Spike-based plasticity mechanisms
G.Indiveri (http://ncs.ethz.ch/) Neuromorphic spiking chips 13 / 38
![Page 18: Neuromorphic VLSI Event-Based devices and systems/file/Giacomo_Indiveri.pdf · 2012-06-27 · Neuromorphic VLSI systems 100µ [Nuno da Costa, INI, 2008] Goals: To exploit the physics](https://reader033.fdocuments.us/reader033/viewer/2022041906/5e63ce57376f191f0602fada/html5/thumbnails/18.jpg)
Outline
1 “Neuromorphic Engineering”
2 Spike-based sensory systems
3 Spiking Neural NetworksSilicon NeuronsSilicon synapsesWinner-Take-All networksMulti-chip networks
4 Learning
5 Neuromorphic Cognitive Systems
G.Indiveri (http://ncs.ethz.ch/) Neuromorphic spiking chips 14 / 38
![Page 19: Neuromorphic VLSI Event-Based devices and systems/file/Giacomo_Indiveri.pdf · 2012-06-27 · Neuromorphic VLSI systems 100µ [Nuno da Costa, INI, 2008] Goals: To exploit the physics](https://reader033.fdocuments.us/reader033/viewer/2022041906/5e63ce57376f191f0602fada/html5/thumbnails/19.jpg)
Silicon neuronsThe low-power adaptive exponential I&F neuron
Cahp
Cmem
Vtaua
Vahp
Vspk
Vtau
Vmem
Vrf
Positive Feedback
Refractory Period
Iin
Leak
Adaptation
Vthr
M1
M3 M2
VrestVthra
M8 M7
M4
M5
M9
M6
M10
M11
M12
M13
M14
M15
M17
M16
M19
M21
M20
M18
M22
Imem
Iahp
Ifb
DPI
DPI
τddt
Imem + Imem ≈Ig IinIτ
+ f (Imem)
[Indiveri et al., ISCAS 2010]
G.Indiveri (http://ncs.ethz.ch/) Neuromorphic spiking chips 15 / 38
![Page 20: Neuromorphic VLSI Event-Based devices and systems/file/Giacomo_Indiveri.pdf · 2012-06-27 · Neuromorphic VLSI systems 100µ [Nuno da Costa, INI, 2008] Goals: To exploit the physics](https://reader033.fdocuments.us/reader033/viewer/2022041906/5e63ce57376f191f0602fada/html5/thumbnails/20.jpg)
The low-power I&F neuronPositive Feedback
0 5 10 15 20 25 30 351
1.2
1.4
1.6
1.8
2
2.2
2.4
2.6
2.8
Time (ms)
I mem
/I 0
data
fit
G.Indiveri (http://ncs.ethz.ch/) Neuromorphic spiking chips 16 / 38
![Page 21: Neuromorphic VLSI Event-Based devices and systems/file/Giacomo_Indiveri.pdf · 2012-06-27 · Neuromorphic VLSI systems 100µ [Nuno da Costa, INI, 2008] Goals: To exploit the physics](https://reader033.fdocuments.us/reader033/viewer/2022041906/5e63ce57376f191f0602fada/html5/thumbnails/21.jpg)
The low-power I&F neuronSpike frequency adaptation
0 5 10 15 202
4
6
8
10
12
14
Spike count
Inst
anta
neou
s fir
ing
rate
(H
z)
0 1 2 3 4 50
1
2
3
4
G.Indiveri (http://ncs.ethz.ch/) Neuromorphic spiking chips 17 / 38
![Page 22: Neuromorphic VLSI Event-Based devices and systems/file/Giacomo_Indiveri.pdf · 2012-06-27 · Neuromorphic VLSI systems 100µ [Nuno da Costa, INI, 2008] Goals: To exploit the physics](https://reader033.fdocuments.us/reader033/viewer/2022041906/5e63ce57376f191f0602fada/html5/thumbnails/22.jpg)
The low-power I&F neuronBasic response properties
Leaky I&F model
0 0.05 0.1 0.15 0.20
0.2
0.4
0.6
0.8
1
Time (s)
Mem
bran
e po
tent
ial (
V)
F-F curve (note mismatch)
G.Indiveri (http://ncs.ethz.ch/) Neuromorphic spiking chips 18 / 38
![Page 23: Neuromorphic VLSI Event-Based devices and systems/file/Giacomo_Indiveri.pdf · 2012-06-27 · Neuromorphic VLSI systems 100µ [Nuno da Costa, INI, 2008] Goals: To exploit the physics](https://reader033.fdocuments.us/reader033/viewer/2022041906/5e63ce57376f191f0602fada/html5/thumbnails/23.jpg)
Outline
1 “Neuromorphic Engineering”
2 Spike-based sensory systems
3 Spiking Neural NetworksSilicon NeuronsSilicon synapsesWinner-Take-All networksMulti-chip networks
4 Learning
5 Neuromorphic Cognitive Systems
G.Indiveri (http://ncs.ethz.ch/) Neuromorphic spiking chips 19 / 38
![Page 24: Neuromorphic VLSI Event-Based devices and systems/file/Giacomo_Indiveri.pdf · 2012-06-27 · Neuromorphic VLSI systems 100µ [Nuno da Costa, INI, 2008] Goals: To exploit the physics](https://reader033.fdocuments.us/reader033/viewer/2022041906/5e63ce57376f191f0602fada/html5/thumbnails/24.jpg)
Synapses
Real synapses Artificial synapses
w1
w2
w3
w4
wn
Post-synaptic Output
Pre
-syn
aptic
Inpu
ts
Synapses are often modeled asinstantaneous multipliers.
Science and Engineering Visualization Challenge
2005 winner, Graham Johnson, Medical Media, Boulder,
Colorado.
G.Indiveri (http://ncs.ethz.ch/) Neuromorphic spiking chips 20 / 38
![Page 25: Neuromorphic VLSI Event-Based devices and systems/file/Giacomo_Indiveri.pdf · 2012-06-27 · Neuromorphic VLSI systems 100µ [Nuno da Costa, INI, 2008] Goals: To exploit the physics](https://reader033.fdocuments.us/reader033/viewer/2022041906/5e63ce57376f191f0602fada/html5/thumbnails/25.jpg)
The diff-pair integrator (DPI) circuit
Vw
Vthr
Mτ
Min Mthr
Mw
Vτ
Msyn
Isyn
Vsyn
Csyn
Mpre
Iw
Iin
Iτ
Isyn(t) = I0e−κ
UT(Vsyn(t)−Vdd )
Ithr = I0e−κ(Vthr−Vdd )
UT
Csynddt
Vsyn =−(Iin− Iτ)
τddt
Isyn + Isyn =Ithr Iw
Iτ
[Bartolozzi and Indiveri, Neural Computation, 2007]
G.Indiveri (http://ncs.ethz.ch/) Neuromorphic spiking chips 21 / 38
![Page 26: Neuromorphic VLSI Event-Based devices and systems/file/Giacomo_Indiveri.pdf · 2012-06-27 · Neuromorphic VLSI systems 100µ [Nuno da Costa, INI, 2008] Goals: To exploit the physics](https://reader033.fdocuments.us/reader033/viewer/2022041906/5e63ce57376f191f0602fada/html5/thumbnails/26.jpg)
The DPI synapseTemporal dynamics
0 0.5 1 1.5 2 2.5 3 3.50
50
100
150
200
250
300
350
400
450
Time (s)
EP
SC
(nA
)
Vw
=300mV
Vw
=320mV
Vw
=340mV
G.Indiveri (http://ncs.ethz.ch/) Neuromorphic spiking chips 22 / 38
![Page 27: Neuromorphic VLSI Event-Based devices and systems/file/Giacomo_Indiveri.pdf · 2012-06-27 · Neuromorphic VLSI systems 100µ [Nuno da Costa, INI, 2008] Goals: To exploit the physics](https://reader033.fdocuments.us/reader033/viewer/2022041906/5e63ce57376f191f0602fada/html5/thumbnails/27.jpg)
Outline
1 “Neuromorphic Engineering”
2 Spike-based sensory systems
3 Spiking Neural NetworksSilicon NeuronsSilicon synapsesWinner-Take-All networksMulti-chip networks
4 Learning
5 Neuromorphic Cognitive Systems
G.Indiveri (http://ncs.ethz.ch/) Neuromorphic spiking chips 23 / 38
![Page 28: Neuromorphic VLSI Event-Based devices and systems/file/Giacomo_Indiveri.pdf · 2012-06-27 · Neuromorphic VLSI systems 100µ [Nuno da Costa, INI, 2008] Goals: To exploit the physics](https://reader033.fdocuments.us/reader033/viewer/2022041906/5e63ce57376f191f0602fada/html5/thumbnails/28.jpg)
Recurrent cooperative-competitive architectures
Hardwired localsynapses
Local excitatoryconnections
Global inhibitoryconnections
[Chicca et al., Nips, 2006]
G.Indiveri (http://ncs.ethz.ch/) Neuromorphic spiking chips 24 / 38
![Page 29: Neuromorphic VLSI Event-Based devices and systems/file/Giacomo_Indiveri.pdf · 2012-06-27 · Neuromorphic VLSI systems 100µ [Nuno da Costa, INI, 2008] Goals: To exploit the physics](https://reader033.fdocuments.us/reader033/viewer/2022041906/5e63ce57376f191f0602fada/html5/thumbnails/29.jpg)
Local recurrent connectivityWinner-take-All architectures
0 2 4 6 8 100
5
10
15
20
25
30
Time (s)
Neu
ron
addr
ess
Input Stimulus
0 50 100Mean f (Hz)
AER INPUT Y
AER
INPU
T X
AER O
UTPU
T
Input signals are encodedwith mean firing rates
Computation andinformation transfer isdata driven
G.Indiveri (http://ncs.ethz.ch/) Neuromorphic spiking chips 25 / 38
![Page 30: Neuromorphic VLSI Event-Based devices and systems/file/Giacomo_Indiveri.pdf · 2012-06-27 · Neuromorphic VLSI systems 100µ [Nuno da Costa, INI, 2008] Goals: To exploit the physics](https://reader033.fdocuments.us/reader033/viewer/2022041906/5e63ce57376f191f0602fada/html5/thumbnails/30.jpg)
Local recurrent connectivityWinner-take-All architectures
0 2 4 6 8 100
5
10
15
20
25
30
Time (s)
Neu
ron
addr
ess
Feedforward Network
0 20 40Mean f (Hz)
Without local connectivityactivated output spike ratesrepresent linearly input spikerates (modulo mismatcheffects)
G.Indiveri (http://ncs.ethz.ch/) Neuromorphic spiking chips 25 / 38
![Page 31: Neuromorphic VLSI Event-Based devices and systems/file/Giacomo_Indiveri.pdf · 2012-06-27 · Neuromorphic VLSI systems 100µ [Nuno da Costa, INI, 2008] Goals: To exploit the physics](https://reader033.fdocuments.us/reader033/viewer/2022041906/5e63ce57376f191f0602fada/html5/thumbnails/31.jpg)
Local recurrent connectivityWinner-take-All architectures
0 2 4 6 8 100
5
10
15
20
25
30
Time (s)
Neu
ron
addr
ess
Feedback Network
0 20 40Mean f (Hz)
With local WTA connectivity thenetwork exhibits:
Selective amplification
Signal normalization
Signal restoration
[Chicca et al., Nips, 2006]
G.Indiveri (http://ncs.ethz.ch/) Neuromorphic spiking chips 25 / 38
![Page 32: Neuromorphic VLSI Event-Based devices and systems/file/Giacomo_Indiveri.pdf · 2012-06-27 · Neuromorphic VLSI systems 100µ [Nuno da Costa, INI, 2008] Goals: To exploit the physics](https://reader033.fdocuments.us/reader033/viewer/2022041906/5e63ce57376f191f0602fada/html5/thumbnails/32.jpg)
Outline
1 “Neuromorphic Engineering”
2 Spike-based sensory systems
3 Spiking Neural NetworksSilicon NeuronsSilicon synapsesWinner-Take-All networksMulti-chip networks
4 Learning
5 Neuromorphic Cognitive Systems
G.Indiveri (http://ncs.ethz.ch/) Neuromorphic spiking chips 26 / 38
![Page 33: Neuromorphic VLSI Event-Based devices and systems/file/Giacomo_Indiveri.pdf · 2012-06-27 · Neuromorphic VLSI systems 100µ [Nuno da Costa, INI, 2008] Goals: To exploit the physics](https://reader033.fdocuments.us/reader033/viewer/2022041906/5e63ce57376f191f0602fada/html5/thumbnails/33.jpg)
Spikes and Address-Event Systems
1 2 3 2 3 12
Inputs
Encode Decode
Address Event Bus
SourceChip
Outputs
DestinationChip
Action Potential
Address-Eventrepresentation ofaction potential
21
321
3
0 0.05 0.1 0.15 0.20
0.2
0.4
0.6
0.8
1
Time (s)
Vm
em (
V)
G.Indiveri (http://ncs.ethz.ch/) Neuromorphic spiking chips 27 / 38
![Page 34: Neuromorphic VLSI Event-Based devices and systems/file/Giacomo_Indiveri.pdf · 2012-06-27 · Neuromorphic VLSI systems 100µ [Nuno da Costa, INI, 2008] Goals: To exploit the physics](https://reader033.fdocuments.us/reader033/viewer/2022041906/5e63ce57376f191f0602fada/html5/thumbnails/34.jpg)
Hierarchical or multi-layer networks
The basic problem with these models is, of course, generalization:a look-up table cannot deal with new events, such as viewing a facefrom the side rather than the front, and it cannot learn in the predic-tive sense described earlier. One of the simplest and most powerfultypes of algorithm developed within learning theory corresponds tonetworks that combine the activities of ‘units’, each broadly tuned toone of the examples (Box 1). Theory (see references in Box 1) showsthat a combination of broadly tuned neurons — those that respondto a variety of stimuli, although at sub-maximal firing rates — mightgeneralize well by interpolating among the examples.
In visual cortex, neurons with a bell-shaped tuning are common.Circuits in infratemporal cortex and prefrontal cortex, which com-bine activities of neurons in infratemporal cortex tuned to differentobjects (and object parts) with weights learned from experience, mayunderlie several recognition tasks, including identification andcategorization. Computer models have shown the plausibility of thisscheme for visual recognition and its quantitative consistency withmany data from physiology and psychophysics2–5 .
Figure 2 sketches one such quantitative model, and summarizes aset of basic facts about cortical mechanisms of recognition establishedover the last decade by several physiological studies of cortex6–8. Objectrecognition in cortex is thought to be mediated by the ventral visualpathway running from primary visual cortex, V1, over extrastriatevisual areas V2 and V4 to the inferotemporal cortex. Starting fromsimple cells in V1, with small receptive fields that respond preferably tooriented bars, neurons along the ventral stream show an increase inreceptive field size as well as in the complexity of their preferred stimuli.At the top of the ventral stream, in the anterior inferotemporal cortex,neurons respond optimally to complex stimuli such as faces and otherobjects. The tuning of the neurons in anterior inferotemporal cortexprobably depends on visual experience9–19. In addition, some neuronsshow specificity for a certain object view or lighting condition13,18,20–22.For example, Logothetis et al.13 trained monkeys to perform an objectrecognition task with isolated views of novel three-dimensional objects(‘paperclips’; Fig. 1). When recording from the animals' inferotemporalcortex, they found that the great majority of neurons selectively tunedto the training objects were view-tuned (see Fig. 1) to one of the trainingobjects. About one tenth of the tuned neurons were view-invariant,consistent with an earlier computational hypothesis23.
In summary, the accumulated evidence points to a visual recog-nition system in which: (1) the tuning of infratemporal cortex cells isobtained through a hierarchy of cortical stages that successivelycombines responses from neurons tuned to simpler features; and (2)the basic ability to generalize depends on the combination of cellstuned by visual experience. Notice that in the model of Fig. 2, thetuning of the units depends on learning, probably unsupervised (forwhich several models have been suggested24; see also review in thisissue by Abbott and Regehr, page 796), since it depends only onpassive experience of the visual inputs. However, the weights of thecombination (see Fig. 3) depend on learning the task and require atleast some feedback (see Box 2).
Thus, generalization in the brain can emerge from the linear com-bination of neurons tuned to an optimal stimulus — effectivelydefined by multiple dimensions25,23,26. This is a powerful extension ofthe older computation-through-memory models of vision andmotor control. The question now is whether the available evidencesupports the existence of a similar architecture underlying general-ization in domains other than vision.
insight review articles
Figure 1 Tuned units in inferotemporal cortex. A monkey was trained to recognizea three-dimensional ‘paperclip’ from all viewpoints (pictured at top). The graphshows tuning to the multiple parameters characterizing each view summarized interms of spike rate versus rotation angle of three neurons in anterior inferotemporalcortex that are view-tuned for the specific paperclip. (The unit corresponding to thegreen tuning curve has two peaks — to a view of the object and its mirror view.) Acombination of such view-tuned neurons (Fig. 2) can provide view-invariant, objectspecific tuning as found in a small fraction of the recorded neurons. Adapted fromLogothetis et al.13.
–80 –30 –0 60 100
Sp
ikes
per
sec
ond
25
20
15
10
0
5
–180 –120 –60 0 60 120 180
213
216
239
Cells
Rotation around y axis (degrees)
Figure 2 A model of visual learning. The model summarizes in quantitative termsother models and many data about visual recognition in the ventral stream pathwayin cortex. The correspondence between the layers in the model and visual areas isan oversimplification. Circles represent neurons and arrows represent connectionsbetween them; the dots signify other neurons of the same type. Stages of neuronswith bell-shaped tuning (with black arrow inputs), that provide example-basedlearning and generalization, are interleaved with stages that perform a max-likeoperation3 (denoted by red dashed arrows), which provides invariance to positionand scale. An experimental example of the tuning postulated for the cells in thelayer labelled inferotemporal in the model is shown in Fig. 1. The model accountswell for the quantitative data measured in view-tuned inferotemporal cortex cells10
(J. Pauls, personal communication) and for other experiments55. Superposition ofgaussian-like units provides generalization to three-dimensional rotations andtogether with the soft-max stages some invariance to scale and position. IT,infratemporal cortex, AIT, anterior IT; PIT, posterior IT; PFC, prefrontal cortex.Adapted from M. Riesenhuber, personal communication.
V1
V4
V1
PFC
AIT
IT
Categ. Ident.
V4/PIT
NATURE | VOL 431 | 14 OCTOBER 2004 | www.nature.com/nature 769
14.10 Insight 768 Poggio 1/10/04 7:48 pm Page 769
© 2004 Nature Publishing Group
AER INPUT Y
AER
INPU
T X
AER O
UTPU
T
G.Indiveri (http://ncs.ethz.ch/) Neuromorphic spiking chips 28 / 38
![Page 35: Neuromorphic VLSI Event-Based devices and systems/file/Giacomo_Indiveri.pdf · 2012-06-27 · Neuromorphic VLSI systems 100µ [Nuno da Costa, INI, 2008] Goals: To exploit the physics](https://reader033.fdocuments.us/reader033/viewer/2022041906/5e63ce57376f191f0602fada/html5/thumbnails/35.jpg)
Hierarchical or multi-layer networks
The basic problem with these models is, of course, generalization:a look-up table cannot deal with new events, such as viewing a facefrom the side rather than the front, and it cannot learn in the predic-tive sense described earlier. One of the simplest and most powerfultypes of algorithm developed within learning theory corresponds tonetworks that combine the activities of ‘units’, each broadly tuned toone of the examples (Box 1). Theory (see references in Box 1) showsthat a combination of broadly tuned neurons — those that respondto a variety of stimuli, although at sub-maximal firing rates — mightgeneralize well by interpolating among the examples.
In visual cortex, neurons with a bell-shaped tuning are common.Circuits in infratemporal cortex and prefrontal cortex, which com-bine activities of neurons in infratemporal cortex tuned to differentobjects (and object parts) with weights learned from experience, mayunderlie several recognition tasks, including identification andcategorization. Computer models have shown the plausibility of thisscheme for visual recognition and its quantitative consistency withmany data from physiology and psychophysics2–5 .
Figure 2 sketches one such quantitative model, and summarizes aset of basic facts about cortical mechanisms of recognition establishedover the last decade by several physiological studies of cortex6–8. Objectrecognition in cortex is thought to be mediated by the ventral visualpathway running from primary visual cortex, V1, over extrastriatevisual areas V2 and V4 to the inferotemporal cortex. Starting fromsimple cells in V1, with small receptive fields that respond preferably tooriented bars, neurons along the ventral stream show an increase inreceptive field size as well as in the complexity of their preferred stimuli.At the top of the ventral stream, in the anterior inferotemporal cortex,neurons respond optimally to complex stimuli such as faces and otherobjects. The tuning of the neurons in anterior inferotemporal cortexprobably depends on visual experience9–19. In addition, some neuronsshow specificity for a certain object view or lighting condition13,18,20–22.For example, Logothetis et al.13 trained monkeys to perform an objectrecognition task with isolated views of novel three-dimensional objects(‘paperclips’; Fig. 1). When recording from the animals' inferotemporalcortex, they found that the great majority of neurons selectively tunedto the training objects were view-tuned (see Fig. 1) to one of the trainingobjects. About one tenth of the tuned neurons were view-invariant,consistent with an earlier computational hypothesis23.
In summary, the accumulated evidence points to a visual recog-nition system in which: (1) the tuning of infratemporal cortex cells isobtained through a hierarchy of cortical stages that successivelycombines responses from neurons tuned to simpler features; and (2)the basic ability to generalize depends on the combination of cellstuned by visual experience. Notice that in the model of Fig. 2, thetuning of the units depends on learning, probably unsupervised (forwhich several models have been suggested24; see also review in thisissue by Abbott and Regehr, page 796), since it depends only onpassive experience of the visual inputs. However, the weights of thecombination (see Fig. 3) depend on learning the task and require atleast some feedback (see Box 2).
Thus, generalization in the brain can emerge from the linear com-bination of neurons tuned to an optimal stimulus — effectivelydefined by multiple dimensions25,23,26. This is a powerful extension ofthe older computation-through-memory models of vision andmotor control. The question now is whether the available evidencesupports the existence of a similar architecture underlying general-ization in domains other than vision.
insight review articles
Figure 1 Tuned units in inferotemporal cortex. A monkey was trained to recognizea three-dimensional ‘paperclip’ from all viewpoints (pictured at top). The graphshows tuning to the multiple parameters characterizing each view summarized interms of spike rate versus rotation angle of three neurons in anterior inferotemporalcortex that are view-tuned for the specific paperclip. (The unit corresponding to thegreen tuning curve has two peaks — to a view of the object and its mirror view.) Acombination of such view-tuned neurons (Fig. 2) can provide view-invariant, objectspecific tuning as found in a small fraction of the recorded neurons. Adapted fromLogothetis et al.13.
–80 –30 –0 60 100
Sp
ikes
per
sec
ond
25
20
15
10
0
5
–180 –120 –60 0 60 120 180
213
216
239
Cells
Rotation around y axis (degrees)
Figure 2 A model of visual learning. The model summarizes in quantitative termsother models and many data about visual recognition in the ventral stream pathwayin cortex. The correspondence between the layers in the model and visual areas isan oversimplification. Circles represent neurons and arrows represent connectionsbetween them; the dots signify other neurons of the same type. Stages of neuronswith bell-shaped tuning (with black arrow inputs), that provide example-basedlearning and generalization, are interleaved with stages that perform a max-likeoperation3 (denoted by red dashed arrows), which provides invariance to positionand scale. An experimental example of the tuning postulated for the cells in thelayer labelled inferotemporal in the model is shown in Fig. 1. The model accountswell for the quantitative data measured in view-tuned inferotemporal cortex cells10
(J. Pauls, personal communication) and for other experiments55. Superposition ofgaussian-like units provides generalization to three-dimensional rotations andtogether with the soft-max stages some invariance to scale and position. IT,infratemporal cortex, AIT, anterior IT; PIT, posterior IT; PFC, prefrontal cortex.Adapted from M. Riesenhuber, personal communication.
V1
V4
V1
PFC
AIT
IT
Categ. Ident.
V4/PIT
NATURE | VOL 431 | 14 OCTOBER 2004 | www.nature.com/nature 769
14.10 Insight 768 Poggio 1/10/04 7:48 pm Page 769
© 2004 Nature Publishing Group
AER INPUT Y
AER
INPU
T X
AER O
UTPU
T
G.Indiveri (http://ncs.ethz.ch/) Neuromorphic spiking chips 28 / 38
![Page 36: Neuromorphic VLSI Event-Based devices and systems/file/Giacomo_Indiveri.pdf · 2012-06-27 · Neuromorphic VLSI systems 100µ [Nuno da Costa, INI, 2008] Goals: To exploit the physics](https://reader033.fdocuments.us/reader033/viewer/2022041906/5e63ce57376f191f0602fada/html5/thumbnails/36.jpg)
Outline
1 “Neuromorphic Engineering”
2 Spike-based sensory systems
3 Spiking Neural NetworksSilicon NeuronsSilicon synapsesWinner-Take-All networksMulti-chip networks
4 Learning
5 Neuromorphic Cognitive Systems
G.Indiveri (http://ncs.ethz.ch/) Neuromorphic spiking chips 29 / 38
![Page 37: Neuromorphic VLSI Event-Based devices and systems/file/Giacomo_Indiveri.pdf · 2012-06-27 · Neuromorphic VLSI systems 100µ [Nuno da Costa, INI, 2008] Goals: To exploit the physics](https://reader033.fdocuments.us/reader033/viewer/2022041906/5e63ce57376f191f0602fada/html5/thumbnails/37.jpg)
Spike-timing dependent plasticity (STDP)
Abbot, Nelson, 2000
1 If an input (pre-synaptic) spike arrives shortlybefore an output (post-synaptic) spike is emitted,the synaptic efficacy is increased.
2 If it arrives soon after the output spike is emitted,the synaptic efficacy is decreased.
G.Indiveri (http://ncs.ethz.ch/) Neuromorphic spiking chips 30 / 38
![Page 38: Neuromorphic VLSI Event-Based devices and systems/file/Giacomo_Indiveri.pdf · 2012-06-27 · Neuromorphic VLSI systems 100µ [Nuno da Costa, INI, 2008] Goals: To exploit the physics](https://reader033.fdocuments.us/reader033/viewer/2022041906/5e63ce57376f191f0602fada/html5/thumbnails/38.jpg)
STDP is not enoughfor learning complex spatio-temporal patterns
Senn, Biological Cybernetics, 2002[...] additional non linearities are required ifSTDP should be relevant for both encodinginformation represented in a spike correlationcode and a mean rate code without spikecorrelations.
G.Indiveri (http://ncs.ethz.ch/) Neuromorphic spiking chips 31 / 38
![Page 39: Neuromorphic VLSI Event-Based devices and systems/file/Giacomo_Indiveri.pdf · 2012-06-27 · Neuromorphic VLSI systems 100µ [Nuno da Costa, INI, 2008] Goals: To exploit the physics](https://reader033.fdocuments.us/reader033/viewer/2022041906/5e63ce57376f191f0602fada/html5/thumbnails/39.jpg)
STDP is not enoughfor learning complex spatio-temporal patterns
Spike-based learning mechanisms idealfor VLSI implementations
depend on the neuron’s membranepotential;
synaptic weights have two stable states(bi-stability);
many synapses see the same pre- andpost-synaptic mean activity (redundancy);
LTP/LTD is induced only in a randomsubset of stimulated synapses(stochasticity).
[Fusi et al. 2000]; [Gütig, Sompolinsky 2006]; [Brader et al. 2007]
G.Indiveri (http://ncs.ethz.ch/) Neuromorphic spiking chips 31 / 38
![Page 40: Neuromorphic VLSI Event-Based devices and systems/file/Giacomo_Indiveri.pdf · 2012-06-27 · Neuromorphic VLSI systems 100µ [Nuno da Costa, INI, 2008] Goals: To exploit the physics](https://reader033.fdocuments.us/reader033/viewer/2022041906/5e63ce57376f191f0602fada/html5/thumbnails/40.jpg)
Spike-driven plasticity in silicon
w2 w3 w4 wnw1
Pre-synaptic Inputs
Post-synaptic Output
Pre-synaptic component
Diff-pairIntergator
AER inputspike
V’UP
Vwth
V’DN
VWi
Vmem
Vwhi
Vwlow
Vilk
pre
~pre
Isyn
Post-synaptic component
Comparator
I&F CircuitDiff-pair
IntegratorVCa
Vcmp
Vmth
CurrentComparator
VUP
VDN
Vmem
Vmem
Vspk
G.Indiveri (http://ncs.ethz.ch/) Neuromorphic spiking chips 32 / 38
![Page 41: Neuromorphic VLSI Event-Based devices and systems/file/Giacomo_Indiveri.pdf · 2012-06-27 · Neuromorphic VLSI systems 100µ [Nuno da Costa, INI, 2008] Goals: To exploit the physics](https://reader033.fdocuments.us/reader033/viewer/2022041906/5e63ce57376f191f0602fada/html5/thumbnails/41.jpg)
Spike-driven plasticity in silicon
Dn
θ
Up
~w
eigh
tpo
st
0 0.1 0.2 0.3 0.4
pre
Time(s)
[Mitra et al. 2009]
G.Indiveri (http://ncs.ethz.ch/) Neuromorphic spiking chips 32 / 38
![Page 42: Neuromorphic VLSI Event-Based devices and systems/file/Giacomo_Indiveri.pdf · 2012-06-27 · Neuromorphic VLSI systems 100µ [Nuno da Costa, INI, 2008] Goals: To exploit the physics](https://reader033.fdocuments.us/reader033/viewer/2022041906/5e63ce57376f191f0602fada/html5/thumbnails/42.jpg)
Stochastic weight updateLTP/LTD probabilities and stop-learning
LTD consolidation
VL
θ
VH
wV
mem
0 0.1 0.2 0.3 0.4
pre
Time(s)
No LTD consolidation
VL
θ
VH
wV
mem
0 0.1 0.2 0.3 0.4
pre
Time(s)
G.Indiveri (http://ncs.ethz.ch/) Neuromorphic spiking chips 33 / 38
![Page 43: Neuromorphic VLSI Event-Based devices and systems/file/Giacomo_Indiveri.pdf · 2012-06-27 · Neuromorphic VLSI systems 100µ [Nuno da Costa, INI, 2008] Goals: To exploit the physics](https://reader033.fdocuments.us/reader033/viewer/2022041906/5e63ce57376f191f0602fada/html5/thumbnails/43.jpg)
Outline
1 “Neuromorphic Engineering”
2 Spike-based sensory systems
3 Spiking Neural NetworksSilicon NeuronsSilicon synapsesWinner-Take-All networksMulti-chip networks
4 Learning
5 Neuromorphic Cognitive Systems
G.Indiveri (http://ncs.ethz.ch/) Neuromorphic spiking chips 34 / 38
![Page 44: Neuromorphic VLSI Event-Based devices and systems/file/Giacomo_Indiveri.pdf · 2012-06-27 · Neuromorphic VLSI systems 100µ [Nuno da Costa, INI, 2008] Goals: To exploit the physics](https://reader033.fdocuments.us/reader033/viewer/2022041906/5e63ce57376f191f0602fada/html5/thumbnails/44.jpg)
Distributed event-driven systems
Neuromorphic “cognitive” systems can be assembled by using:1 Full custom hybrid analog/digital neural processing VLSI devices.2 A spike based communication protocol (e.g., the Address-Event
Representation).3 Systematic methods for parameter tuning.4 Methods for implementing state-dependent computation using spiking
neural networks.
G.Indiveri (http://ncs.ethz.ch/) Neuromorphic spiking chips 35 / 38
![Page 45: Neuromorphic VLSI Event-Based devices and systems/file/Giacomo_Indiveri.pdf · 2012-06-27 · Neuromorphic VLSI systems 100µ [Nuno da Costa, INI, 2008] Goals: To exploit the physics](https://reader033.fdocuments.us/reader033/viewer/2022041906/5e63ce57376f191f0602fada/html5/thumbnails/45.jpg)
State dependent computation
sWTA diagramExcitatory
neurons
Global
Inhibition
Inhibitory
neurons }Nearest-N
Excitation
I E E E E E
sWTA networks as building blocks
Linearbehaviors
Non linearbehaviors
Analog gain Locus invariance Gain control bycommon mode input
Selective amplification Signal restoration Multi-stability
[Douglas and Martin, 2007]
Configure key parameters of the WTA network automatically.Implement “state-holding” elements.Learn network connectivity patterns.
G.Indiveri (http://ncs.ethz.ch/) Neuromorphic spiking chips 36 / 38
![Page 46: Neuromorphic VLSI Event-Based devices and systems/file/Giacomo_Indiveri.pdf · 2012-06-27 · Neuromorphic VLSI systems 100µ [Nuno da Costa, INI, 2008] Goals: To exploit the physics](https://reader033.fdocuments.us/reader033/viewer/2022041906/5e63ce57376f191f0602fada/html5/thumbnails/46.jpg)
ConclusionsToward neuromorphic cognitive behaving systems
By using event based sensors andspike-based neural processingcircuits it is possible to implementreal-time sensory-motor systems.
By using soft WTA multi-chipnetworks it is possible toimplement real-timestate-dependent computation.
The AER communicationinfrastructure and automatedparameter tuning techniques,allow us to synthesizespike-based neural finite statemachines.
G.Indiveri (http://ncs.ethz.ch/) Neuromorphic spiking chips 37 / 38
![Page 47: Neuromorphic VLSI Event-Based devices and systems/file/Giacomo_Indiveri.pdf · 2012-06-27 · Neuromorphic VLSI systems 100µ [Nuno da Costa, INI, 2008] Goals: To exploit the physics](https://reader033.fdocuments.us/reader033/viewer/2022041906/5e63ce57376f191f0602fada/html5/thumbnails/47.jpg)
ConclusionsToward neuromorphic cognitive behaving systems
By using event based sensors andspike-based neural processingcircuits it is possible to implementreal-time sensory-motor systems.
By using soft WTA multi-chipnetworks it is possible toimplement real-timestate-dependent computation.
The AER communicationinfrastructure and automatedparameter tuning techniques,allow us to synthesizespike-based neural finite statemachines.
G.Indiveri (http://ncs.ethz.ch/) Neuromorphic spiking chips 37 / 38
![Page 48: Neuromorphic VLSI Event-Based devices and systems/file/Giacomo_Indiveri.pdf · 2012-06-27 · Neuromorphic VLSI systems 100µ [Nuno da Costa, INI, 2008] Goals: To exploit the physics](https://reader033.fdocuments.us/reader033/viewer/2022041906/5e63ce57376f191f0602fada/html5/thumbnails/48.jpg)
ConclusionsToward neuromorphic cognitive behaving systems
By using event based sensors andspike-based neural processingcircuits it is possible to implementreal-time sensory-motor systems.
By using soft WTA multi-chipnetworks it is possible toimplement real-timestate-dependent computation.
The AER communicationinfrastructure and automatedparameter tuning techniques,allow us to synthesizespike-based neural finite statemachines.
G.Indiveri (http://ncs.ethz.ch/) Neuromorphic spiking chips 37 / 38
![Page 49: Neuromorphic VLSI Event-Based devices and systems/file/Giacomo_Indiveri.pdf · 2012-06-27 · Neuromorphic VLSI systems 100µ [Nuno da Costa, INI, 2008] Goals: To exploit the physics](https://reader033.fdocuments.us/reader033/viewer/2022041906/5e63ce57376f191f0602fada/html5/thumbnails/49.jpg)
ConclusionsToward neuromorphic cognitive behaving systems
By using event based sensors andspike-based neural processingcircuits it is possible to implementreal-time sensory-motor systems.
By using soft WTA multi-chipnetworks it is possible toimplement real-timestate-dependent computation.
The AER communicationinfrastructure and automatedparameter tuning techniques,allow us to synthesizespike-based neural finite statemachines.
G.Indiveri (http://ncs.ethz.ch/) Neuromorphic spiking chips 37 / 38
![Page 50: Neuromorphic VLSI Event-Based devices and systems/file/Giacomo_Indiveri.pdf · 2012-06-27 · Neuromorphic VLSI systems 100µ [Nuno da Costa, INI, 2008] Goals: To exploit the physics](https://reader033.fdocuments.us/reader033/viewer/2022041906/5e63ce57376f191f0602fada/html5/thumbnails/50.jpg)
Acknowledgments
The Institute of NeuroinformaticsElisabetta ChiccaStefano FusiChiara BartolozziThe NCS group (http://ncs.ethz.ch/)
Rodney DouglasKevan MartinRichard Hahnloser
Funding sourcesneuroP (257219) ERCSCANDLE (ICT-231168)eMorph (ICT-231467)
nAttention (121713) SNFSoundRec (119973) SNF
G.Indiveri (http://ncs.ethz.ch/) Neuromorphic spiking chips 38 / 38