Post on 30-Jul-2018
NeuromorphicCompu-ngintheHBP
SteveFurber(UniversityofManchester)
KarlheinzMeier(HeidelbergUniversity)
INNOVATIONWORKSHOPExploita-onofNeuromorphicCompu-ngTechnologies
3February2017,Brussels
2
Outline
• Neuromorphic Computing • international perspectives • in the Human Brain Project
• SpiNNaker • BrainScaleS • Innovation Roadmap
• applications
3
Outline
• Neuromorphic Computing • international perspectives • in the Human Brain Project
• SpiNNaker • BrainScaleS • Innovation Roadmap
• applications
4
What is neuromorphic computing?
Transferring aspects of structure and function from biological substrates to electronic circuits
Structure
Cells – Networks – Connections
Function Local Processing – Communication – Learning
5
Why neuromorphic computing?
Modelling neural circuits to advance
brain science
Applying brain-like principles to cognitive computing
Achieving energy efficiency, speed,
robustness, ability to learn
REPORT TO THE PRESIDENT Ensuring Long-Term U.S.
Leadership in Semiconductors
Executive Office of the President President’s Council of Advisors on
Science and Technology
January 2017
8
Ensuring Long-Term U.S. Leadership in Semiconductors
28
Table A1. Selected component technology vectors that have a high probability of deployment in ten years (* denotes more speculative deployment within this timeframe)
Component technology vector
Time-frame to first commercial products
Approach to achieving and retaining competitive advantage
Neuromorphic Computing
Available now Continued R&D into new architectures coupled with 3D technologies and new materials, Deep Learning accelerators (for mobile and data center applications), and applications for true brain-inspired computing
Photonics Available now Foundries for tools and materials R&D; integrate photonics with CMOS and other materials
Sensors Available now Foundries for tools and materials R&D; integrate new types/classes of sensors with CMOS and other materials
CMOS (sub 7nm node size or new 3D structures)*
Advances in thermal management available with new process nodes
Deep understanding of transistor physics and chipset architecture and related design know-how; foundries and labs for transistor and materials R&D
Magnetics 1-2 years (MRAM as eFlash), 3 years (as DRAM), 5-7 years (as SRAM)
Foundries for tools and materials R&D; integrate magnetics with CMOS and other materials
3D 2-3 years (wafer-to-wafer stacking), 4-5 years (die-to-wafer stacking), 5-7 (Monolithic 3D)
Deep understanding of applications space and benefits associated use of 3D technologies and design know-how; foundries for tools and materials R&D; design automation tool R&D
Data-flow based architectures
3-4 years Continued architecture R&D, coupled with materials, integration, and manufacturing; build an ecosystem for solutions using data-flow based architectures
Ultra-high performance wireless systems
3 years (5G), 10-12 years (6G)
Continued R&D in new materials and processes, antenna design advances, chipset manufacturing, and integration
Advanced non-volatile memory as SRAM
5+ years Deep understanding of applications space and chipset architectures
Carbon nanotubes and phase change materials*
5-7 years Foundries/labs for materials R&D for hardware architectures; chipset designs to leverage these technologies
Biotech/human health
5-10 years R&D towards low power, highly integrated, high performance processing, high-data rate communications, wireless charging; couple R&D with clinical research to create, build, and evaluate on new materials and interfaces
Quantum Computing
< 10 years Pre-competitive R&D labs for new materials; foundries for new materials and hardware architectures; tools for quantum algorithms and software programming with various architectural paradigms
Point-of-Use Nanoscale 3D printing
Available now Desktop fab capabilities for rapid prototyping, additive manufacturing, moving beyond silicon and interfacing with soft matter, and small batch production
DNA for compute and storage*
10+ years Multi-disciplinary basic research in efficiently and reliably reading and writing and retrieving DNA strands
8
IBM TrueNorth • 4,096 digital neurosynaptic
cores!– one million configurable
neurons!– 256 million programmable
synapses!– ~70mW!– over 400 Mbits of
embedded SRAM!– 5.4 billion transistors!
• 16 TrueNorth Chips assembled into a 4x4 mesh!– 16 million neurons and 4
billion synapses. !
9
Stanford Neurogrid
• Neurocore Chip!– 65k neurons!– each with two
compartments and a set of configurable silicon ion channels!
• Sixteen Neurocores are assembled on a board!– million-neuron
Neurogrid!
10
11
HBP Neuromorphic Computing Concepts
PHYSICALMODELSYSTEM
Localanalogcompu-ngwith4Millionneuronsand50Millionsynapses–binary,asynchronouscommunica-on–
emula-onspeedisx10000real--me
Loca-on:Heidelberg(Germany)
MANY-CORENUMERICALMODELSYSTEM
0.5–1MillionARMprocessors–address-based,smallpacket,asynchronouscommunica-on–runningatreal--me
Loca-on:Manchester(UK)
12
Motivation and Approaches futurecompu-ngbasedon
biologicalinforma-onprocessing
understandingbiologicalinforma-onprocessing
Twofundamentallydifferentmodelingapproaches:
• NUMERICALMODEL(SpiNNaker)representsmodelparametersasbinarynumbers
• PHYSICALMODEL(BrainScaleS)representsmodelparametersasphysicalquan--es→voltage,current,charge(likethebiologicalbrain)
canbecombinedtoformahybridsystem
needmodelsystemtotestideas
13
Outline
• Neuromorphic Computing • international perspectives • in the Human Brain Project
• SpiNNaker • BrainScaleS • Innovation Roadmap
• applications
21
Outline
• Neuromorphic Computing • international perspectives • in the Human Brain Project
• SpiNNaker • BrainScaleS • Innovation Roadmap
• applications
PhysicalModelSystemCon-nuousTimeIntegra-ngNeuralCellMembrane
CmdVdt
= −gleak V −Eleak( ) Cm
R = 1/gleak
Eleak
V(t)
ΔV [V] gleak [S] Cm [F] (gΔV)/C [V/s]
Biology(*) 10-2 10-8 10-10 100 VLSI 10-1 10-6 10-13 106
(*) Brette/Gerstner, J. Neurophysiology, 2005
Inherent speed gap:
106 Volt/second → accelerated
neuron model
„Time“isimposedbyinternalphysics,notbyexternalcontrol
StructureofaBrainScaleSChipphotographoftheneuromorphicchipHICANN
512AdExneurons112650programmabledynamicsynapsesShort-termandlong-termsynap-cplas-city10000xaccelera-onwrt.biologicalreal--me
Centralelement20cmsiliconwaferwith450neuromorphicMicrochips
• analog,-me-con-nuousopera-on
• 10000fasterthanbiologicalreal-me
• 50Msynapseand230kneuroncircuitsoperateinparallel
• high-densityon-waferinterconnectsmanufacturedbymul--layerwafer-levelmetalliza-on
Physical Model, local analogue computing,
binary continuous time communication
Wafer-Scale Integration of 200.000 neurons and 50.000.000 synapses on
a single 20 cm wafer
Short term and long term plasticity, 10.000 faster
than real-time
Wafer-scaleintegra?onofanalogneuralnetworks,J.Schemmel,J,FieresandK.MeierIn:ProceedingsofIJCNN(2008),IEEEPress,431
Physical Model, local analogue computing,
binary continuous time communication
Wafer-Scale Integration of 200.000 neurons and 50.000.000 synapses on
a single 20 cm wafer
Short term and long term plasticity, 10.000 faster
than real-time
Conven-onalComputerDataorsimulatedenvironment
Rewardcalcula-on
Ac-onofmachineondataorenvironment
Stateofdataorenvironment
Reward(penalty)
NeuromorphicMachine
In-the-looplearning:2hiddenlayerfeedforwardspikingnetworkonaphysicalsubstrateforMNISTdataclassifica-on(submijedtoIJCNN2017)
Nature+Real--me Simula-on Accelerated
Model
CausalityDetec-on 10-4s 0.1s 10-8s
Synap-cPlas-city 1s 1000s 10-4s
Learning Day 1000Days 10s
Development Year 1000Years 3000s
12OrdersofMagnitude
Evolu-on >Millenia >1000Millenia >Months
>15OrdersofMagnitude
TimeScales
31
Outline
• Neuromorphic Computing • international perspectives • in the Human Brain Project
• SpiNNaker • BrainScaleS • Innovation Roadmap
• applications
Cognitive Computing ?
Finding patterns in space and time coded data Finding causal relations in data
Making predictions from learned dependencies Control of complex systems
§ Business data analysis (finance, market analysis, sales) § Science data analysis (medical, brain, astrophysics) § Communication (spatio-temporal pattern detection, security) § Entertainment (games, video creation and analysis) § Automotive (intelligent sensors, autonomous cars) § Robotics (sensory-motor loop) § Manufacturing (production line monitoring and control) § Government (intelligence, military)
HBPFrameworkPartnershipAgreement
Approvedbasisforproject
workun-l2023
SignedOctober30th,2015
Containsdetailedroadmapfor2ndgenera-onSpiNNaker-2and
BrainScales-2systems
Processing Element
Processing Element
Processing Element
Processing Element
Rout
er
SerDesSerDesSerDes
SerDes SerDes SerDes SerDes
MCU
Memory Interface
Shared Memory
Shared Memory
Ongoing2ndgenera-onprototypedevelopmentStromgemphasisonlearningcapabili-es
SpiNNaker-2PowerManagementFloa-ngpointprecisionTruerandomnumbers
BrainScales-2PowerPCplas-cityprocessorsImprovedparameterstorage
Ac-vedendrites/compartments
WhatisuniqueintheHBP?• Neuromorphictechnologydevelopmentdrivenbynewresultsfromtheore-caland
experimentalneuroscience(i.e.theonlyknownworkingneuralcomputer..).• Learningisrecognizedasakeyrequirementforallneuralnetworkbasedcompu-ng.• Groupshaveconvincinglydemonstratedtheirabilitytobuildhardwarefromthe
transistorupwardsandintegrateitintoworkingsystems.• Aconvincingroadmapreachingoutto2023(FPA)hasbeenapprovedandisreadyfor
realiza-on.
WhatischallengingintheHBP?• Exis-ngsystemslargelybasedonprevious12yearsofdevelopmentsince2005with
substan-alinvestmentsfromothersources.• Limitedfundingofapprox.4M€/abarelyfacilita-ngtheory,chipdesign,prototyping,
systemdesign,produc-oncosts,construc-on,sorwaresupportandopera-onofnovelhardwarearchitectures.Neuromorphiccompu-ngcurrentlyreceives7%oftheHBPbudget.
• Goingbeyondprototypedevelopmenttolarge-scale,compe--vesystemsrequiresone--meinvestmentof25.8M€(currentcos-ngasdetailedinFPA)between2019and2023forengineeringandproduc-oncosts.
Requiredstepstowardsexploita-onandapplica-on• Reinforceneuromorphiccompu-ngasakeycomponentofinnova-onemergingfrom
theHBPasaFETproject.Enablecon-nuedprototypedevelopmentaccordingtoFPAroadmap.
• Realizeengineeringandproduc-oncostsfor2ndgenera-onHBPsystemsaccordingtoFPAroadmap.
• Iden-fyspecificapplica-oncasesthroughexpressionsofinterests(EoIs)betweenHBPgroupsandindustrypartners.
• EvaluateEoIsandprovideECsupportforindustrialprototypedevelopmentofapplica-ons
OpenAI
Anon-profitar?ficialintelligenceresearchcompany.MissionistobuildsafeAI,ensureAI'sbenefitsareas
widelyandevenlydistributedaspossible.
Sponsors:Individuals,companies,productaccessSelec-on: ElonMusk
PeterThiel
Microsor Amazon
GitHub
ModelcaseforEoI
EUVersionorPartnership
Click to edit Master title style • Click to edit Master text styles
– Secondlevel• Third level
– Fourth level
SpiNNaker-2 Chip Roadmap
SGA2 and SGA3
SpiNNaker2 • 68 ARM M4 cores • Power management • SpiNNaker router with SerDes • HMC interface • Chip Size: 70mm², >1 billion transistors
2015
2013
2014
2016
2017
2018
2019
2020
2021
2022
NanoLink28 • SerDes Transceiver • Chip Size: 4mm²
Santos28 • 4 ARM M4 cores • AVFS, DVFS power management • SpiNNaker router with SerDes • LPDDR2 Memory Interface • Chip Size: 18mm²
Nanolink28_gen2 • 2nd Iteration SerDes • HMC Interface
SGA1
SGA1 Nanolink28_gen3 • Final SerDes • Final HMC Interface
2023
Next generation chips and small prototype systems in SGA1 and SGA2 No secured funding for next generation large-scale systems
Click to edit Master title style
• Click to edit Master text styles – Secondlevel
• Third level
– Fourth level
> SGA
1
BrainScaleS-2 Chip/Wafer Roadmap
2015
2013
2014
2016
2017
2018
2019
2020
2021
2022
HICANN-DLS NN 1st • Complete Neural Network • Plasticity Programming Unit • Chip Size: 5mm²
NanoLink65 • SerDes Transceiver • Chip Size: 4mm²
RuP
Membrane ADC • Testchip fast ADC
RuP HICANN-DLS NN 2nd • Neuron circuits with full AdEx
features
HICANN-DLS SC1 • wafer-level spike
communication
> SGA1
> SGA1
> SGA
1
Event Router 1st • inter- and intra-wafer routing • axonal delays
Event Router 2nd
HICANN-DLS • complete replacement for
180nm generation
PM2 wafer
PM3 Wafer
2023
SGA1
Next generation chips and small prototype systems in SGA1 and SGA2 No secured funding for next generation large-scale systems
Click to edit Master title style
• Click to edit Master text styles – Secondlevel
• Third level
– Fourth level
Option : Novel Wafer-Level Packaging Technologies
reduce system complexity by embedding the wafer within
the printed circuit board
PressplatePresspadandsepara-onfoilCufoil5µmWaferandFR4frameCufoil5µmPresspadandsepara-onfoilPressplate
NeuromorphicCompu-ngintheHBP
Ø Europehasstrengtheneditsleadingroleinneuromorphiccompu-ngthroughtheHBPandpreviousprojects.
Ø Twolarge-scale,neuromorphiccompu-ngsystemshavebeenbuiltaspartoftheHBPpladorminfrastructure.
Ø Twonextgenera-onsofneuromorphicchipshavebeenoutlined,prototypedandprovideapowerfulroadmapfor2023.
Ø Nextgenera-onsystemengineeringandproduc-onneedstobefacilitated.
Ø Neuromorphiccompu-nghastobereinforcedasatechnologydriverintheHBPasaFETproject.
Ø Exploita-onofHBPneuromorphictechnologycouldbeenabledthroughjointEoIsandECsupport.