Deep Learning Meets Heterogeneous Computing...Deep Learning Meets Heterogeneous Computing Author Ren...
Transcript of Deep Learning Meets Heterogeneous Computing...Deep Learning Meets Heterogeneous Computing Author Ren...
![Page 1: Deep Learning Meets Heterogeneous Computing...Deep Learning Meets Heterogeneous Computing Author Ren Wu Subject The rise of the internet, especially mobile internet, has accelerated](https://reader035.fdocuments.us/reader035/viewer/2022081514/5fe8d5fa575e0221aa0ddcf9/html5/thumbnails/1.jpg)
Deep Learning Meets Heterogeneous Computing
Dr. Ren Wu Distinguished Scientist, IDL, Baidu [email protected]
![Page 2: Deep Learning Meets Heterogeneous Computing...Deep Learning Meets Heterogeneous Computing Author Ren Wu Subject The rise of the internet, especially mobile internet, has accelerated](https://reader035.fdocuments.us/reader035/viewer/2022081514/5fe8d5fa575e0221aa0ddcf9/html5/thumbnails/2.jpg)
Baidu Everyday 5b+ queries 500m+ users 100m+ mobile users 100m+ photos …
![Page 3: Deep Learning Meets Heterogeneous Computing...Deep Learning Meets Heterogeneous Computing Author Ren Wu Subject The rise of the internet, especially mobile internet, has accelerated](https://reader035.fdocuments.us/reader035/viewer/2022081514/5fe8d5fa575e0221aa0ddcf9/html5/thumbnails/3.jpg)
Big Data
• >2000PB Storage • 10-100PB/day Processing • 100b-1000b Webpages • 100b-1000b Index • 1b-10b/day Update • 100TB~1PB/day Log
![Page 4: Deep Learning Meets Heterogeneous Computing...Deep Learning Meets Heterogeneous Computing Author Ren Wu Subject The rise of the internet, especially mobile internet, has accelerated](https://reader035.fdocuments.us/reader035/viewer/2022081514/5fe8d5fa575e0221aa0ddcf9/html5/thumbnails/4.jpg)
Infrastructure
![Page 5: Deep Learning Meets Heterogeneous Computing...Deep Learning Meets Heterogeneous Computing Author Ren Wu Subject The rise of the internet, especially mobile internet, has accelerated](https://reader035.fdocuments.us/reader035/viewer/2022081514/5fe8d5fa575e0221aa0ddcf9/html5/thumbnails/5.jpg)
GPU服务器 – Much better performance
ARM Servers – Higher density
Data center containers – Faster deployment
Self-design switches – Much lower cost
Infrastructure
![Page 6: Deep Learning Meets Heterogeneous Computing...Deep Learning Meets Heterogeneous Computing Author Ren Wu Subject The rise of the internet, especially mobile internet, has accelerated](https://reader035.fdocuments.us/reader035/viewer/2022081514/5fe8d5fa575e0221aa0ddcf9/html5/thumbnails/6.jpg)
Big Data @Baidu
Very large scale data mining、analytics、visulization, etc
Data warehouse Deep learning
Software foundation
Servers and Data centers
A.I.“Brain” World class in size World’s first R. I.
Elastic cloud 100+PB data processing
Best in Asia Self designed Huge # of servers
![Page 7: Deep Learning Meets Heterogeneous Computing...Deep Learning Meets Heterogeneous Computing Author Ren Wu Subject The rise of the internet, especially mobile internet, has accelerated](https://reader035.fdocuments.us/reader035/viewer/2022081514/5fe8d5fa575e0221aa0ddcf9/html5/thumbnails/7.jpg)
Nine Technology Challenges
On Aug 13, 2012, CEO Robin Li gave a keynote speech at ACM KDD, and proposed nine major technological challenges to the academic research community. The top three are: 1. OCR in natural images 2. Speech recognition and understanding 3. Content-based image retrieval (visual search)
![Page 8: Deep Learning Meets Heterogeneous Computing...Deep Learning Meets Heterogeneous Computing Author Ren Wu Subject The rise of the internet, especially mobile internet, has accelerated](https://reader035.fdocuments.us/reader035/viewer/2022081514/5fe8d5fa575e0221aa0ddcf9/html5/thumbnails/8.jpg)
Deep Learning Since 2006
3/24/14 8
![Page 9: Deep Learning Meets Heterogeneous Computing...Deep Learning Meets Heterogeneous Computing Author Ren Wu Subject The rise of the internet, especially mobile internet, has accelerated](https://reader035.fdocuments.us/reader035/viewer/2022081514/5fe8d5fa575e0221aa0ddcf9/html5/thumbnails/9.jpg)
Deep Learning vs. Human Brain
pixels
edges
object parts (combination of edges)
object models Deep Architecture in the Brain
Retina
Area V1
Area V2
Area V4
pixels
Edge detectors
Primitive shape detectors
Higher level visual abstractions
Slide credit: Andrew Ng
![Page 10: Deep Learning Meets Heterogeneous Computing...Deep Learning Meets Heterogeneous Computing Author Ren Wu Subject The rise of the internet, especially mobile internet, has accelerated](https://reader035.fdocuments.us/reader035/viewer/2022081514/5fe8d5fa575e0221aa0ddcf9/html5/thumbnails/10.jpg)
Top breakthrough technology 2013
MIT Technology Review, April 23rd, 2013
![Page 11: Deep Learning Meets Heterogeneous Computing...Deep Learning Meets Heterogeneous Computing Author Ren Wu Subject The rise of the internet, especially mobile internet, has accelerated](https://reader035.fdocuments.us/reader035/viewer/2022081514/5fe8d5fa575e0221aa0ddcf9/html5/thumbnails/11.jpg)
Baidu IDL
n Announced its first research arm in Jan. 2013
n Institute of Deep Learning (IDL) n The focus is Artificial Intelligence
n Two locations: Beijing and Silicon Valley
![Page 12: Deep Learning Meets Heterogeneous Computing...Deep Learning Meets Heterogeneous Computing Author Ren Wu Subject The rise of the internet, especially mobile internet, has accelerated](https://reader035.fdocuments.us/reader035/viewer/2022081514/5fe8d5fa575e0221aa0ddcf9/html5/thumbnails/12.jpg)
Progress of Deep Learning at Baidu
• Big improvement on speech & image recognition (2013)
• Speech: error rate reduced by 25%
• OCR: error rate reduced by 30%
• Face: LFW benchmark, 94% correct
• DNN CTR for search ads was launched on May 20th 2013, serving billions of search queries everyday – substantial improvement
![Page 13: Deep Learning Meets Heterogeneous Computing...Deep Learning Meets Heterogeneous Computing Author Ren Wu Subject The rise of the internet, especially mobile internet, has accelerated](https://reader035.fdocuments.us/reader035/viewer/2022081514/5fe8d5fa575e0221aa0ddcf9/html5/thumbnails/13.jpg)
http://stu.baidu.com
Baidu – Visual Search
![Page 14: Deep Learning Meets Heterogeneous Computing...Deep Learning Meets Heterogeneous Computing Author Ren Wu Subject The rise of the internet, especially mobile internet, has accelerated](https://reader035.fdocuments.us/reader035/viewer/2022081514/5fe8d5fa575e0221aa0ddcf9/html5/thumbnails/14.jpg)
Visual Search: Faces
![Page 15: Deep Learning Meets Heterogeneous Computing...Deep Learning Meets Heterogeneous Computing Author Ren Wu Subject The rise of the internet, especially mobile internet, has accelerated](https://reader035.fdocuments.us/reader035/viewer/2022081514/5fe8d5fa575e0221aa0ddcf9/html5/thumbnails/15.jpg)
Visual Search Example
![Page 16: Deep Learning Meets Heterogeneous Computing...Deep Learning Meets Heterogeneous Computing Author Ren Wu Subject The rise of the internet, especially mobile internet, has accelerated](https://reader035.fdocuments.us/reader035/viewer/2022081514/5fe8d5fa575e0221aa0ddcf9/html5/thumbnails/16.jpg)
Visually similar images
The competition
Baidu
![Page 17: Deep Learning Meets Heterogeneous Computing...Deep Learning Meets Heterogeneous Computing Author Ren Wu Subject The rise of the internet, especially mobile internet, has accelerated](https://reader035.fdocuments.us/reader035/viewer/2022081514/5fe8d5fa575e0221aa0ddcf9/html5/thumbnails/17.jpg)
Another Example
The competition Baidu
Image uploaded
![Page 18: Deep Learning Meets Heterogeneous Computing...Deep Learning Meets Heterogeneous Computing Author Ren Wu Subject The rise of the internet, especially mobile internet, has accelerated](https://reader035.fdocuments.us/reader035/viewer/2022081514/5fe8d5fa575e0221aa0ddcf9/html5/thumbnails/18.jpg)
Baidu Google搜索结果
Image uploaded
Visually Similar Images - Comparison
The competition
![Page 19: Deep Learning Meets Heterogeneous Computing...Deep Learning Meets Heterogeneous Computing Author Ren Wu Subject The rise of the internet, especially mobile internet, has accelerated](https://reader035.fdocuments.us/reader035/viewer/2022081514/5fe8d5fa575e0221aa0ddcf9/html5/thumbnails/19.jpg)
CBIR – The Competition
![Page 20: Deep Learning Meets Heterogeneous Computing...Deep Learning Meets Heterogeneous Computing Author Ren Wu Subject The rise of the internet, especially mobile internet, has accelerated](https://reader035.fdocuments.us/reader035/viewer/2022081514/5fe8d5fa575e0221aa0ddcf9/html5/thumbnails/20.jpg)
Image Recognition - Flowers
![Page 21: Deep Learning Meets Heterogeneous Computing...Deep Learning Meets Heterogeneous Computing Author Ren Wu Subject The rise of the internet, especially mobile internet, has accelerated](https://reader035.fdocuments.us/reader035/viewer/2022081514/5fe8d5fa575e0221aa0ddcf9/html5/thumbnails/21.jpg)
Peak uploading rate at 100 million images per day! IOS APP #1 for 3 weeks
百度魔图:PK大咖
![Page 22: Deep Learning Meets Heterogeneous Computing...Deep Learning Meets Heterogeneous Computing Author Ren Wu Subject The rise of the internet, especially mobile internet, has accelerated](https://reader035.fdocuments.us/reader035/viewer/2022081514/5fe8d5fa575e0221aa0ddcf9/html5/thumbnails/22.jpg)
Deep Learning
Voice, Text
Image
User
DNN for Speech 10k hours of voice data 10b training samples Months on a GPU cluster
![Page 23: Deep Learning Meets Heterogeneous Computing...Deep Learning Meets Heterogeneous Computing Author Ren Wu Subject The rise of the internet, especially mobile internet, has accelerated](https://reader035.fdocuments.us/reader035/viewer/2022081514/5fe8d5fa575e0221aa0ddcf9/html5/thumbnails/23.jpg)
Typical scale of training data
Datasets
• Image recognition: 100 millions
• OCR: 100 millions
• Speech: 10 billions
• CTR: 100 billions
Projected training data to
grow 10x each year
Training time: Weeks to Months on GPU clusters
Big data + Deep learning + HPC = Success
![Page 24: Deep Learning Meets Heterogeneous Computing...Deep Learning Meets Heterogeneous Computing Author Ren Wu Subject The rise of the internet, especially mobile internet, has accelerated](https://reader035.fdocuments.us/reader035/viewer/2022081514/5fe8d5fa575e0221aa0ddcf9/html5/thumbnails/24.jpg)
Mobile Applications of DNN
![Page 25: Deep Learning Meets Heterogeneous Computing...Deep Learning Meets Heterogeneous Computing Author Ren Wu Subject The rise of the internet, especially mobile internet, has accelerated](https://reader035.fdocuments.us/reader035/viewer/2022081514/5fe8d5fa575e0221aa0ddcf9/html5/thumbnails/25.jpg)
“⼿手机百度 随时知道”
![Page 26: Deep Learning Meets Heterogeneous Computing...Deep Learning Meets Heterogeneous Computing Author Ren Wu Subject The rise of the internet, especially mobile internet, has accelerated](https://reader035.fdocuments.us/reader035/viewer/2022081514/5fe8d5fa575e0221aa0ddcf9/html5/thumbnails/26.jpg)
DNN – Anywhere, Anytime • DNN-based image recognition on mobile device • No connectivity needed • Real time, directly works on video stream • Everything is done within the device • What you point is what you get
• OpenCL based, highly optimized • Large deep neural network models • Thousands of objects, flowers, dogs, and bags etc • Unleashed the full potential of the device hardware • World’s first in-place mobile DNN app? • And the best!
![Page 27: Deep Learning Meets Heterogeneous Computing...Deep Learning Meets Heterogeneous Computing Author Ren Wu Subject The rise of the internet, especially mobile internet, has accelerated](https://reader035.fdocuments.us/reader035/viewer/2022081514/5fe8d5fa575e0221aa0ddcf9/html5/thumbnails/27.jpg)
DNN – Anywhere, Anytime
![Page 28: Deep Learning Meets Heterogeneous Computing...Deep Learning Meets Heterogeneous Computing Author Ren Wu Subject The rise of the internet, especially mobile internet, has accelerated](https://reader035.fdocuments.us/reader035/viewer/2022081514/5fe8d5fa575e0221aa0ddcf9/html5/thumbnails/28.jpg)
百度酷⽿耳
![Page 29: Deep Learning Meets Heterogeneous Computing...Deep Learning Meets Heterogeneous Computing Author Ren Wu Subject The rise of the internet, especially mobile internet, has accelerated](https://reader035.fdocuments.us/reader035/viewer/2022081514/5fe8d5fa575e0221aa0ddcf9/html5/thumbnails/29.jpg)
DNNs Everywhere
Supercomputers Datacenters ( cloud )
Tablets, smartphones
Wearable devices IoTs
![Page 30: Deep Learning Meets Heterogeneous Computing...Deep Learning Meets Heterogeneous Computing Author Ren Wu Subject The rise of the internet, especially mobile internet, has accelerated](https://reader035.fdocuments.us/reader035/viewer/2022081514/5fe8d5fa575e0221aa0ddcf9/html5/thumbnails/30.jpg)
DNNs Everywhere Supercomputers Datacenters Tablets, smartphones Wearable devices
IoTs
1000s GPUs 100k-1m servers 700m (in China) Billions?
Supercomputer used for training Trained DNNs then deployed to data centers (cloud), smartphones, and even wearables and IoTs
![Page 31: Deep Learning Meets Heterogeneous Computing...Deep Learning Meets Heterogeneous Computing Author Ren Wu Subject The rise of the internet, especially mobile internet, has accelerated](https://reader035.fdocuments.us/reader035/viewer/2022081514/5fe8d5fa575e0221aa0ddcf9/html5/thumbnails/31.jpg)
Heterogeneous Computing Supercomputers
Data centers (cloud) Smart phones
Wearable devices!
Big data + Deep learning + HPC HC = Success
![Page 32: Deep Learning Meets Heterogeneous Computing...Deep Learning Meets Heterogeneous Computing Author Ren Wu Subject The rise of the internet, especially mobile internet, has accelerated](https://reader035.fdocuments.us/reader035/viewer/2022081514/5fe8d5fa575e0221aa0ddcf9/html5/thumbnails/32.jpg)
OpenCL-based Open ECO-SYSTEM
• Diverse industry participation, from cell phones to supercomputers
o Processor vendors, system OEMs, middleware vendors, application developers.
• OpenCL is the industry standard embraced by many companies.
Third party names are the property of their owners. * Courtesy of Simon McIntosh-‐Smith and Tom Deakin
![Page 33: Deep Learning Meets Heterogeneous Computing...Deep Learning Meets Heterogeneous Computing Author Ren Wu Subject The rise of the internet, especially mobile internet, has accelerated](https://reader035.fdocuments.us/reader035/viewer/2022081514/5fe8d5fa575e0221aa0ddcf9/html5/thumbnails/33.jpg)
Summary
Big data + Deep learning + High performance computing =
Intelligence
Big data + Deep learning + Heterogeneous computing =
Success
![Page 34: Deep Learning Meets Heterogeneous Computing...Deep Learning Meets Heterogeneous Computing Author Ren Wu Subject The rise of the internet, especially mobile internet, has accelerated](https://reader035.fdocuments.us/reader035/viewer/2022081514/5fe8d5fa575e0221aa0ddcf9/html5/thumbnails/34.jpg)
Baidu USA
http://usa.baidu.com/
[email protected] [email protected]
And we are hiring • Heterogeneous
Computing experts • Parallel algorithm and
performance experts • CUDA/OpenCL Experts • FPGA experts • Andrios/IOS experts • Data scientist • Infrastructure Engineer …