Developing Products with Machine Learning...Developing Products with Machine Learning Cadence and...
Transcript of Developing Products with Machine Learning...Developing Products with Machine Learning Cadence and...
© 2018 Arm Limited
Developing Products with Machine Learning Cadence and Arm Seminar 2018
Petach Tikvah, Israel
Phil Burr, Director, Embedded Portfolio, ArmJacques-Olivier Piednoir, R&D Vice President, CadenceJuly 2018
2 © 2018 Arm Limited
Agenda
• What is machine learning (ML)?
• Machine learning at the edge
• Implementing machine learning
• Software, ecosystem, and resources
• Machine learning in silicon design
3 © 2018 Arm Limited
What is Machine Learning?
Artificial intelligence
Machine learning
Deep learning
Algorithms: CNNs,
RNNs, etc.
• Additional terms
• Location• Cloud – processing done in data farms• Edge – processing done in local devices
(growing much faster than Cloud ML)
• Key components of machine learning• Model – a mathematical approximation of a
collection of input data• Training – in deep learning, datasets are used
to create a “model”• Inference – in deep learning, a “model” is
used to check against new data
4 © 2018 Arm Limited
Machine Learning “Training”
Training data
Model
For each piece of data used to train the model, millions of model parameters are adjusted.
The process is repeated many times until the model delivers satisfactory performance.
Neural Network
5 © 2018 Arm Limited
Machine Learning “Inference”
✓
Input Neural Network with Model Output
96.4% confidence
97.4% confidence
When new data is presented to the trained model, large numbers of multiply-add operations
are performed using the new data and the model parameters. The process is performed once.
6 © 2018 Arm Limited
Compute Requirements Differ for Training and Inference
Apply the model
Low latencyEfficiency
Low/mid performance
SecurityPrivacy
Train the model
High performance
Throughput oriented
Large data sets
High-performance compute, servers, GPU
Embedded systems, heterogeneous processing
Clo
ud
sid
eD
evic
e si
de
© 2018 Arm Limited
Machine Learningat the Edge
8 © 2018 Arm Limited
Bandwidth ReliabilityPower SecurityCost Latency
Why ML is Moving to the Edge
9 © 2018 Arm Limited
Keyword detection
Pattern training
Voice & image recognition
Object detection
Image enhancement
Autonomous drive
Increasing power and cost (silicon)
Incr
easi
ng
per
form
ance
(O
ps/
seco
nd
)
~100W
~1W
~1-10mW
~10W
Range of “Edge” Applications
10 © 2018 Arm Limited
Machine Learning Innovation
BRAGI 'The Dash PRO'
ReindeerCam
Hive View
DriveCore PlatformNauto for Safe Driving
EZVIZ C5Si Camera
11 © 2018 Arm Limited
Keyword detection
Pattern training
Voice and image recognition
Object detection
Image enhancement
Autonomous driving
Data center
Incr
easi
ng
per
form
ance
(o
ps/
seco
nd
)
Flexible, Scalable ML Solutions
Increasing power and cost (silicon)
12 © 2018 Arm Limited
Scalable Machine Learning Processor “Architecture”
Scalable Compatible Programmable
Sensors (2 GOP/s) Servers (150 TOP/s)
IoT Mobile Industrial Automotive Networking Server
Targeting Multiple Markets with Scalable Architecture
1~3 TOP/s > 70 TOP/s~20-50 TOP/s~20 GOP/s
Typical Market Performance Requirements
© 2018 Arm Limited
Implementing Machine Learning
14 © 2018 Arm Limited
Machine Learning Tradeoffs in Embedded SystemsNeural Networks
Machine learning
System
Considerations Number of outputs
Number of inputs
Transfer function
Number of hidden layers
Neurons perhidden layer
Accuracy
HW/SW
Memory
Data type
Hardware
Software
System Considerations
Latency
Privacy Security Scalability
Energy efficiency
Communicationbandwidth
Optimization of system and machine learning requirements
15 © 2018 Arm Limited
Heterogeneous Systems Enable Efficient ML
Sophisticated end node based on a heterogeneous architecture
Interconnect
Shared L2
AHB/AXI interconnect
Local memory
DMC
DDR
ApplicationsSubsystem
Always-on subsystemSubsystem
AHB interconnect
TimerSensor SRAM
Efficient “always-on” subsystem filters and
processes data using ML
Advanced ML usingApplications processor
Co
mp
ute
per
form
ance
Time
© 2018 Arm Limited
Software, Ecosystem, and Resources Support
17 © 2018 Arm Limited
Machine Learning Needs a Strong Ecosystem
Hardware Platforms
Apps and Services Frameworks and Models
18 © 2018 Arm Limited
Frameworks simplifying ML development
Arm NN software translates existing NN frameworks:
TensorFlow, Caffe, Android NNAPI, MXNet etc.Developers maintain existing workflow and toolsReduces overall development timeAbstracts away the complexities of underlying hardware
Arm NN
CMSIS-NN
CPU GPU
Compute Library
3rd-party IP
Partner IP driver and
SW functions
Compute Library
CPUArm
ML processor
Compute Library
NN Frameworks
better efficiency and performance for NN functions
CMSIS-NN 5x
faster than other open-source software (OSS)
Compute Library 15x
19 © 2018 Arm Limited
▪ EDA wrestles with many data-driven problems▪ Need to provide answers quickly, in-design solutions▪ Heavy degree of automation is needed▪ What-if analysis to explore alternative solutions faster
Scale• Larger designs• More rules and restrictions• More data (simulation, extraction, shapes,
techfiles)
Complexity• More complicated silicon technologies (FinFET)• More complicated design and electrical rules• More interactions between chip, package and board• Thermal related impact between devices and wires
One Impactsthe others
Productivity• More uncertainty leads to re-design and
missed schedules• Limited number of capable, trained design and
layout engineers• Larger complexity and scale… more activities in
same schedule
Design Challenges That Are Faced Today
One Impactsthe others
20 © 2018 Arm Limited
Data-Driven Solutions = ML, Analytics, DMRarely is ML used in a Vacuum in Real Products and Systems
Massive Parallelization(Cloud-Enabled)
Data Preparation
Model-Based Inference
Adaptation
Steps in Producing Intelligent Solutions
• Filter• Label• Augment• Reduce• Partition
• Calibrate• Learn• Optimize
Creating an Intelligent and Adaptive Product Requires More
than Machine Learning
• Select• Train• Test• Verify
Machine Learning
Machine Learning
Machine Learning Optimization
Analyticsand DM
Prepare Data Build Models Adapt to Uncertainty
We often say “Machine Learning,” but that implies some combination of analytics, ML, optimization, and distributed processing
21 © 2018 Arm Limited
Machine Learning at Cadence
Enablement• Hardware/software co-design
Inside• Better PPA, faster engines• Improved testing/diagnostics
Outside• Automated design flow• Productivity improvement
22 © 2018 Arm Limited
Intelligent Design and Optimization
Capture more abstractions of design intent and preferences
Analytics
Optimization Goals
Observe
Recommend
Action or Decision
CPU GPU
Massive Parallelization(Cloud-Enabled)
Key Technologies for Intelligent Solutions
Create knowledgebase that can be mined and statistically analyzed in future planning and design
23 © 2018 Arm Limited
Machine Learning for Library Characterization
Standard Cell
Extracted Netlist
Stimulus (Vector)
Generation
Circuit Simulation
Simulation Results
Timing Model Extraction
Timing Library
Challenges• Many cells, many process corners• Many new effects• Millions of simulations for each new standard cell library
Machine learning improves throughput:• Learn from previous process corners• Smart interpolation by extracting critical measurements• Critical corner identification
24 © 2018 Arm Limited
Summary
• Machine Learning is already enabling innovation
• ML at the edge is enhancing intelligence in even the smallest of devices
• Strong ecosystem of ML solutions
• ML is already enhancing silicon design
• Talk to Arm and Cadence about ML solutions
2525
Thank YouDankeMerci谢谢ありがとうGraciasKiitos감사합니다धन्यवादתודה
© 2018 Arm Limited
2626
The Arm trademarks featured in this presentation are registered trademarks or trademarks of Arm Limited (or its subsidiaries) in the US and/or elsewhere. All rights reserved. All other marks featured may be trademarks of their respective owners.
www.arm.com/company/policies/trademarks
© 2018 Arm Limited