Smart Data Slides: Emerging Hardware Choices for Modern AI Data Management
-
Upload
dataversity -
Category
Technology
-
view
370 -
download
0
Transcript of Smart Data Slides: Emerging Hardware Choices for Modern AI Data Management
November 10, 2016Adrian Bowles, PhD
Founder, STORM Insights, [email protected]
Emerging Hardware Choices for #ModernAI
Copyright (c) 2016 by STORM Insights Inc. All Rights reserved.
Hardware - The Final Frontier for Workload OptimizationPerformance Challenges for #ModernAIOptimizing Workloads Through Parallel ExecutionThree Architectural Paths
NeuromorphicGPU/Advanced Memory Quantum
Market Overview & Recommendations
Agenda
Copyright (c) 2016 by STORM Insights Inc. All Rights reserved.
Value Migrates to Hardware
OptimizeCommoditizeStandardize
ConventionalAI
MachineLearning
BigData
#ModernAI Scope
Copyright (c) 2016 by STORM Insights Inc. All Rights reserved.
Emerging AI Hardware Trends and OptionsA Role for Hardware Optimization
CognitiveMachine LearningReasoningUnderstandingPlanning
Human InputLanguageVisionAural
Human-Oriented OutputMachine Input
IOTMachine-Oriented Output
Emerging AI Hardware Trends and Options
Human
Machine
Input OutputNarrative Generation
Copyright (c) 2016 by STORM Insights Inc. All Rights reserved.
Data Mgmt
Learn Model
Reason
Understand
Plan
TasteSmell
Touch
Hear
See
Gestures
Emotions
Language
Visualization
Reports
Haptics
IoT IoT
Cognitive Systems: Communication & Control
Sensors
SystemsControls
Copyright (c) 2016 by STORM Insights Inc. All Rights reserved.
Hearing (audioception)~12,000 outer hair cells/ear
~3,500 inner hair cells Vision (ophthalmoception)Photoreceptors - Per Eye~120,000,000 rod cells
(triggered by single photon)~6,000,000 cone cells
(require more photons to trigger)~ 60,000 photosensitive ganglion cells
Touch (tactioception)Thermoreceptors, mechanoreceptors, chemoreceptors and nociceptors for touch, pressure, pain, temperature, vibration
Smell (olfacoception)Chemoreception
Taste (gustaoception)Chemoreception
Neurosynaptic Problem Solving Scope
Copyright (c) 2016 by STORM Insights Inc. All Rights reserved.
Hearing (audioception)~12,000 outer hair cells/ear
~3,500 inner hair cells Vision (ophthalmoception)Photoreceptors - Per Eye~120,000,000 rod cells
(triggered by single photon)~6,000,000 cone cells
(require more photons to trigger)~ 60,000 photosensitive ganglion cells
Touch (tactioception)Thermoreceptors, mechanoreceptors, chemoreceptors and nociceptors for touch, pressure, pain, temperature, vibration
Smell (olfacoception)Chemoreception
Taste (gustaoception)Chemoreception
Human Cognition~100,000,000,000 (100B) Neurons
~100-500,000,000,000,000 (100-500T) Synapses
Neurosynaptic Problem Solving Scope
Learn
ModelReason
Understand
Plan
Copyright (c) 2015 by STORM Insights Inc. All Rights reserved.
deeplearning
Deep learning refers to a biologically-inspired approach to machine learning that leverages a collection of simple processing units - analogous to neurosynaptic elements - that collaborate to solve complex problems at multiple levels of abstraction. These modern neural networks can support supervised, reinforcement, or unsupervised learning systems. In general, deep learning solutions require a high degree of parallelism, which may be implemented in hardware and/or software.
Deep Learning is Inherently Parallel
Copyright (c) 2016 by STORM Insights Inc. All Rights reserved.
Memory(Instructions & Data)
Central Processing Unit(CPU)
Control Unit
Arithmetic/Logic Unit(ALU)
InputDevice(s)
Output Device(s)
Operating System
The von Neumann Architecture
Copyright (c) 2016 by STORM Insights Inc. All Rights reserved.
Memory(Instructions & Data)
Central Processing Unit(CPU)
Control Unit
Arithmetic/Logic Unit(ALU)
InputDevice(s)
Output Device(s)
Operating System
“Speed”/Throughput Constraints
Copyright (c) 2016 by STORM Insights Inc. All Rights reserved.
Memory(Instructions & Data)
Central Processing Unit(CPU)
Control Unit
Arithmetic/Logic Unit(ALU)
InputDevice(s)
Output Device(s)
Operating System
Control Unit
Arithmetic/Logic Unit(ALU)
Parallelism With Multi-Cores
Copyright (c) 2016 by STORM Insights Inc. All Rights Reserved. 9/28/2011
IBM Power 75090 servers, 32 cores/server, 2880 Cores in 10 racks
16Tb RAM
~80TeraFLOPS
80,000,000,000,000FLOPS
IBM Watson - Parallelism for Deep QA
Copyright (c) 2016 by STORM Insights Inc. All Rights reserved. Source: https://www.top500.org/system/177999
Amdahl’s Law: The theoretical performance improvement resulting from a resource improvement for a fixed workload is limited by that part of the workload that cannot benefit from the resource improvement.
Limits to Parallelism
Copyright (c) 2015 by STORM Insights Inc. All Rights reserved.
Research Examples:The European Commission FACETS (Fast Analog Computing with Emergent Transient States)
and BrainScaleS (Brain-inspired multi scale computation in neuromorphic hybrid systems)UK SpiNNaker (Spiking Neural Network Architecture)DARPA - SyNAPSE (Systems of Neuromorphic Adaptive Plastic Scalable Electronics)
Computer, device/component -level systems modeled after biological systems or components, such as neurons and synapses. These may be implemented in analog, digital or hybrid hardware. Typically designed to learn by experience over time, rather than by programming.
Neuromorphic Architectures (“Brain-Inspired”)
Massively interconnected networks of very simple processors.
Copyright (c) 2016 by STORM Insights Inc. All Rights reserved.
Synapse 16 chip board
Neuromorphic Architectures
IBM - SyNAPSE board
“TrueNorth chips can be seamlessly tiled to create vast, scalable neuromorphic systems.” Already demonstrated 16 million neurons and 4 billion synapses. Goal is to integrate 4,096 chips in a single rack with 4 billion neurons and 1 trillion synapses while consuming ~4kW of power.
Source: Qualcomm
Copyright (c) 2015 by STORM Insights Inc. All Rights reserved.
Neuromorphic Architectures
MAY 2, 2016: Qualcomm Incorporated (NASDAQ: QCOM) today announced at the Embedded Vision Summit in Santa Clara, Calif., that its subsidiary, Qualcomm Technologies, Inc., is offering the first deep learning software development kit (SDK) for devices powered by Qualcomm® Snapdragon™ 820 processors. The SDK, called the Qualcomm Snapdragon Neural Processing Engine, is powered by the Qualcomm® Zeroth™ Machine Intelligence Platform
Copyright (c) 2016 by STORM Insights Inc. All Rights reserved.
The Nvidia M40 processor for training neural networks.Nvidia
NVIDIA Maxwell™ architectureUp to 7 Teraflops of single-precision performance with NVIDIA GPU Boost™3072 NVIDIA CUDA® cores24 GB of GDDR5 memory288 GB/sec memory bandwidthQualified to deliver maximum uptime in the datacenter
GPU/Advanced Memory Architectures
Copyright (c) 2016 by STORM Insights Inc. All Rights reserved.
GPU/Advanced Memory Architectures
Copyright (c) 2016 by STORM Insights Inc. All Rights reserved.
Server racks with TPUs used in the AlphaGo matches with Lee Sedol
GPU/Advanced Memory Architectures
Copyright (c) 2016 by STORM Insights Inc. All Rights reserved.
At Facebook, we've made great progress thus far with off-the-shelf infrastructure components and design. We've developed software that can read stories, answer questions about scenes, play games and even learn unspecified tasks through observing some examples. But we realized that truly tackling these problems at scale would require us to design our own systems. Today, we're unveiling our next-generation GPU-based systems for training neural networks, which we've code-named “Big Sur.”
• FAIR is more than tripling its investment in GPU hardware as we focus even more on research and enable other teams across the company to use neural networks in our products and services.
• As part of our ongoing commitment to open source and open standards, we plan to contribute our innovations in GPU hardware to the Open Compute Project so others can benefit from them.
Facebook Open-source AI hardware design
https://code.facebook.com/posts/1687861518126048/facebook-to-open-source-ai-hardware-design/
GPU/Advanced Memory Architectures
Copyright (c) 2016 by STORM Insights Inc. All Rights reserved.
Source: https://www.micron.com/about/emerging-technologies/automata-processing
GPU/Advanced Memory Architectures
Copyright (c) 2016 by STORM Insights Inc. All Rights reserved.
GPU/Advanced Memory Architectures
Copyright (c) 2016 by STORM Insights Inc. All Rights reserved.
http://www.research.ibm.com/quantum/
Quantum Architectures
Copyright (c) 2016 by STORM Insights Inc. All Rights reserved.
Source: https://arxiv.org/abs/1608.00263
Quantum Architectures
Copyright (c) 2016 by STORM Insights Inc. All Rights reserved.
Probabalistic Architecture?
Copyright (c) 2016 by STORM Insights Inc. All Rights reserved.
NeuromorphicGPU/Memory Acceleration Quantum
Market/Technology Positions & Maturity
Ready NowMuch More in the Pipeline
Promising -Ready Now At Handset Level
Promising -Watch But Don’t Wait
Proven approach for ||ismEasy interoperability with conventional systems
+Natural behavioral process model+Lower power requirements- Requires new software model
& skills
+Incredible compute power potential- Requires new software model
& skills- Requires interface to
conventional system for pre-processing
- Requires extremely cold (big, expensive) environment
Copyright (c) 2016 by STORM Insights Inc. All Rights reserved.
IBMQualcomm
Brain Corporation (hosted by Qualcomm)KnupathTenstorrentCirrascaleNeurogrid (Stanford)Tensilica - Cadence1026 LabsCerebrasArtificial LearningHRL LaboratoriesIsocline
NvidiaIntelAMD
Facebook (FAIR)
Nervana Systems/IntelMovidius - Intel (Vision processing)Google TPU
IBMD-WaveGoogle
NeuromorphicGPU/Memory Acceleration Quantum
Ones to Watch
On the Horizon
Ready NowMuch More in the Pipeline
Promising -Ready Now At Handset Level
Promising -Watch But Don’t Wait
Copyright (c) 2016 by STORM Insights Inc. All Rights reserved.
[email protected] @ajbowlesSkype ajbowles
Upcoming Webinar Dates & Topics
December 8 Leverage the IOT to Build a Smart Data Ecosystem
January #Modern AI and Cognitive Computing: Boundaries and OpportunitiesFebruary Artificial General Intelligence: When I Can I Get It?March Data Science and Business Analysis: A Look at Best Practices for Roles, Skills, and ProcessesApril Machine Learning: Moving Beyond Discovery to UnderstandingMay Streaming Analytics for Agile IoT-Oriented ApplicationsJune Machine Learning Case StudiesJuly Advances in Natural Language Processing I: UnderstandingAugust Organizing Data and Knowledge: The Role of Taxonomies and OntologiesSeptember Advances in Natural Language Processing II: NL GenerationOctober Choosing the Right Data Management Architecture for Cognitive ComputingNovember See Me, Feel Me, Touch Me, Heal Me: The Rise of the Cognitive InterfaceDecember The Road to Autonomous Applications
For More Information…
Copyright (c) 2016 by STORM Insights Inc. All Rights reserved.
Basilar membrane. (2016, October 28). In Wikipedia, The Free Encyclopedia. Retrieved 01:58, October 28, 2016, from https://en.wikipedia.org/w/index.php?title=Basilar_membrane&oldid=746543229
Somatosensory system. (2016, October 9). In Wikipedia, The Free Encyclopedia. Retrieved 04:59, October 9, 2016, from https://en.wikipedia.org/w/index.php?title=Somatosensory_system&oldid=743336883
Photoreceptor cell. (2016, September 19). In Wikipedia, The Free Encyclopedia. Retrieved 03:07, September 19, 2016, from https://en.wikipedia.org/w/index.php?title=Photoreceptor_cell&oldid=740108113
Copyright (c) 2016 by STORM Insights Inc. All Rights reserved.
Hardware - The Final Frontier for Workload Optimization#ModernAI DefinedPerformance Challenges Optimizing Workloads Through Parallel ExecutionThree Architecture Paths
NeuromorphicGPU/Advanced Memory Quantum
Agenda
A Role for HardwareCognitive
Machine LearningReasoningUnderstandingPlanning
Human InputLanguageVisionAural
Human-Oriented OutputMachine Input
IOTMachine-Oriented Output
Copyright (c) 2016 by STORM Insights Inc. All Rights reserved.
Copyright (c) 2015 by STORM Insights Inc. All Rights reserved.