Brain Inspired Compung - Kirchhoff Institut Für Physikmeierk/myfiles/downloads/MeierLyonICT… ·...
Transcript of Brain Inspired Compung - Kirchhoff Institut Für Physikmeierk/myfiles/downloads/MeierLyonICT… ·...
BrainInspiredCompu0ngKarlheinzMeierRuprecht‐Karls‐UniversitätHeidelberg
FACETSProjectCoordinator
ICTConference,Lyon(France),2008
FutureEmergentTechnologies(FET)inIST
„FETwillexploreradicalinterdisciplinaryavenues,deliveringproofs‐of‐conceptfornewop;onsanddemonstra;ngnewpossibili;es.ItwillstrengthenEurope'sscienceandtechnologybaseinnewandemergingareas,refinenewvisionstothepointwheretheyaCractindustrialinvestment,andestablishnewinterdisciplinaryresearchcommuni;eswithinEuropeanscienceandindustry“
Especially:
“RecentadvancesinICTandneuroscienceenableasignificantpartofthehumanbraintobestudiedandmodelledin‐silico.Thisobjec;veseekstoexploitsuchadvancesinordertobeCerunderstandhowthebrainprocessesinforma;onand/orhowitcommunicateswiththeperipheralnervoussystem(PNS),andtoexplorepoten;alapplica;onsofthis”
„Complicated“and......„Complex“
Top‐DownDesign BoVom‐upEvolu0on
1 WhatistheUniversemadeof?
2 WhatistheBiologicalBasisofConsciousness?
3 WhydoHumansHaveSoFewGenes?
4 ToWhatExtentAreGene0cVaria0onandPersonalHealthLinked?
5 CantheLawsofPhysicsbeUnified?
.....120morefollowing....
125Scien0ficKeyQues0onsiden0fiedbythe
„AmericanAssocia;onfortheAdvancementofScience“
WHATDON‘TWEKNOW...
ContemporaryITsystems Processor‐memorybasedarchitectureswithserialcommandexecu0on(Turing)
Predeterminedalgorithmsdefinecapabiliesandperformance(sobware)
Basedonwelldefinedreproduciblestatesandwelldefinedreversible0meevolu0on
Electronicsimplementa0onofBooleanoperators,highpowerconsump0on
Extremelyhighyieldrequirements,liVlefaulttolerance
Limitedbyatomicdistancescaleincomponets(nm):componentlimited
WELLUNDERSTOOD
NeuralcomputaCon Maximallyparallel,non‐linearcomp0ngelementswithlargediversity
Timecorrela0onsdrivethedynamics
Learningbyinternalself‐organisa0onandstronginterac0onwithenvironment
Lowpowerconsump0onandhighfaulttolerance
Limitedbydegreeofcomplexity:architecturelimited
NOTUNDERSTOOD(listedasamajorchallengefor21.Centuryscience)
RealBiology:Neurons‐Synapses‐Dendrites‐Spikes
Ac0onPoten0al=Spike
I(t) = u(t)/R + C⋅du/dt u(t) : membrane potential I(t) : input current
Using RC time-constant :
τm ⋅ du/dt = - u(t) + R⋅ I(t)
In addition :
„Spike-Generation“ and „Reset“ of u if u = ϑ from:W.Gerstner,SpikingNeurons
Example for a Neural Circuit - here : „Integrate and Fire“!A simple approch to neuromorphic electronics!
Why neuromorphic electronics ?!TWO Answers : !
I. Research tool for neuroscience : Study neural architectures in „ANY“ detail at „ANY“ speed !
II. New type of information processing : Use massively parallel (10LARGE), potentially low yield (solve fabrication problem), low power (solve leakage current problem). Make use of self-organisation (solve software problem).!
Rela0veweigh0ng?DoesII.dependonI.?Interdisciplinary!
StatusandFutureofNeuromorphicCompu0ng
• Humanlevelperformance• Dawnofanewage
vonNeumannmachines
NeuromorphicMachines
MachineComplexitye.g.Gates;Memory;Neurons;SynapsesPower;
Size
Dawnofanewparadigm
“simple” “complex”
EnvironmentalComplexitye.g.InputCombinatorics
[log]
ProgramObjec0ve
Atradebetweenuniversalityandefficiency
TakenfromtheDARPASynapseProjectCall
Weareheretoday:Machinecomplexityofneuromorphicdevicess0llexceedstheoneofvonNeumannmachines.Currentproblemstackledwithneuromorphicdevicesares0llsimple.Newcomputa0onalparadigmsareexpectedpayoffwithwithmorecomplexinputdataandlargernetworks.
ConceptualapproachestoNeuromorphicCompu0ng
“BoVom‐up”:Basedon(simplified)reconstructedmodelsofbiologicalmorphologyandfunc0onality:cells,synapses,connec0vity,plas0citymechanisms(“atomsandtheirinterac0ons”)(e.g.FACETSproject)
“Top‐down”:Basedonlargescalefunc0onalblocksandtheirconnec0vity(e.g.JeffHawkins)
“Firstprinciples”:Designofcomputa0onalparadigms,notnecessarilybiologicalplausible(e.g.FACETS,liquidcompu0ng)
“Evolu0onary”designofcomputa0onalstructures
Speedw.r.t.biologicalreal‐0me10‐3
101
10‐2 10‐1 105
mm
3 ofb
rain0ssue
sim
ulated
orem
ulated
1mm
3 =105neu
ronsand
109synapses
100
103
102
101
FACETSVLSIApproach(upto108neurons)
12.000nodeCluster
40nodePIV
16nodePIIIBiologicalreal‐0m
e
104
Neocortex(1012neurons)
V1(109neurons)
FACETS:ComplementaritySupercomputersandVLSI‐Complexityvs.Speed
10%V1(108neurons)
TheMeritsoffast(105)neuralVLSI
Acess<10OrdersofMagnitudeinTimeinanar0ficialSystemwithaspa0alcomplexityof>105?
The FACETS
Con
sorti
um Fast Analog Computing with
Emergent Transient States
U Bordeaux, CNRS (Gif-sur-Yvette and Marseille), U Debrecen, TU Dresden, U Freiburg, TU Graz, U Heidelberg *, EPFL
Lausanne, Funetics S.a.r.l. Lausanne, U London, U Plymouth, INRIA Sophia-Antipolis, KTH Stockholm
*Coordinator
An Integrated Project in the 6th Framework Programme
Information Society Technology - Future Emergent Technologies
FP6-2004-IST-FETPI Project Reference 15879
FACETS:BasicIdea,methodologicalapproachandgoalsNeurobiology:StructuralandFunc0onalInves0ga0onofthe
Neocor0calMicrocircuitandtheCircuitElementsin‐vivoandin‐vitro
Modelling:VirtualMicrocircuitsonState‐of‐the‐ArtComputers
Hardware:Emula0oninanalogandmixed‐signalVLSIsystems
CommonGoal:Studynon‐classicaluniversalcompu0ngsolu0ons
Benchmarking(biologyvs.Modellingvs.HardwarewithvisualtasksinVI)
Methodology:ToolDevelopment(Compu0ng,VLSI)ReducConofBiologicalDetail/Complexity
synapses:
pk,l(t) exponen0alonsetanddecay(spikeshape)gk,l 0togmaxwith4bitresolu0on
effec0vemembrane0me‐constantcm /gtotalis0me‐dependent
high‐conductancestatepossible
current source, no voltage dependence
A FACETS Approach : Conductance-based Network Model
sumoverexcitatorysynapsecurrentsk
sumoverinhibitorysynapsecurrentsl
Voltage dependent part, changes membrane conductance
membranecurrent
leakagecurrent
( ) ( ) ( )∑∑ −+−+−=l llk kk EVgpEVgpEVg
dtdVc ixlleakm
FACETSmixed‐signalVLSISystemStage1(Chipbased)
100.000dynamicSynapses(PlasCcity)
384Neurons
DigitalControl
Inherentlyfast:approx.1050mesbiology
Upto16neuralnetworkboards(“crate”)canbecombinedtobuildalargernetwork
FACETSHardware
NESTSoftware
100ms
2,5µs
Equivalence between Hardware and Software!Subthreshold Membrane Potential and Spiking
WaferScaleIntegra0on:Spa0allySeparatedFunc0ons:NeuralVLSI‐ContemporaryIT
Wafer
label0mestamplabel
0mestamplabel
0mestamp
packettransportFPGA
PCB
Layer2
Layer1
HICANN
HierarchicalSetup
• Layer1:Massivelyparallelneuralcircuit,usecompletewafertoexploitinherentfaulttolerance,asynchronouspulsetransmission
• Layer2:Packetbaseddigitalcommunica0onprotocolforlong‐rangeSpiketransmission,networkset‐up,monitoringandcontrol.
J.Fieresetal.,IJCNN2008
NewTechnologyexploringtheinherentfaulttolerancsofneuralcurcuits:WaferScaleIntegra0on
J.Fieresetal.,IJCNN2008
FACETS:Wafer‐ScaleIntegra0on
J.Fieresetal.,IJCNN2008
J.Fieresetal.,IJCNN2008
SystemunderConstruc0oninHeidelberg
TechnologicalChallengesandConstraintsofCMOSNC
Automated design and verifica0on technologies for verylarge scale (50M synapses per wafer) massively parallel VLSIstructures
Distributed compact on‐chip / on‐wafer memorytechnologies for parameter storage andplas0city / learning /adapta0on mechanisms (SRAM, current memories, floa0nggates),NEW:non‐CMOSpostprocessing:magne0cstructures
On‐wafer / inter‐die horizontal high density connec0ontechnologies
Off‐waferver0calhighdensityconnec0ontechnologies
NEW : Inter‐wafer 3‐dimensional connec0on technologies(waferstacking)withlowpoweranalogdesign
NEW : Access to deep sub‐micron (< 100 nm) full waferproduc0onforhigherintegra0ondensi0esofneuralcells
Exploita0onoftheintrinsicfault‐andmismatch‐toleranceofneuralcircuits
Interna0onalTechnologyRoadmapforSemiconductors(ITRS)
Faults??
CMOL(CMOSMolecularHybrid)(Hammerström,Likharev)
UseanACTIVECMOSsubstructure(asadvancedaspossible/economical)
UsePASSIVEnanoscaleconnec0ngelements(0ps+wires)
UseACTIVEtwo‐terminalnanodevices(e.g.molecules)linkingtheconnec0ngelements
FromK.LikharevStonyBrook
LogNneuron
3
4
5
6
7
8
9
10
2008 2009 20122010 20132011
Singlechip189nm
Crate180nm
Singlewafer180nm
Mul0wafer180nm
Mul0wafer180nm
Singlewafer<180nm
WaferStack<180nm
Mul0wafer<180nm
Mul0WaferStack<180nm
DARPA„CatChallenge“
DARPA„MouseChallenge“
FACETS‐1 Possiblefutureroadmap
Bio‐ICT:Excellentopportunitytoini0atethenextHARDWARErevolu0onininforma0ontechnology
Requires:
Massivelyinterdisciplinaryapproach(includinggraduatestudenttraining) Systema0cefforttowardsbrainmapping(morphologyandfunc0on) Theoryefforttowardscomputa0onalprinciplesandphysicsofcomplexsystems Infrastructureandcapabilityforlargescalehardwaresystemdevelopment Accesstodeep‐submicrontechnologiesandcuzng‐edgeconnec0ontechnologies
Mul0‐ScaleFundingapproach:
Concepts: AdressedinIntegratedProjects(WEAREHERE) Technologies: AdressedinDesignStudies Prototypes: AdressedinPreparatoryPhases Systems: Adressedinjointprojectswithindustry
Europeisinanexcellentsitua0on,leadingthefield!