APPLICATIONS OF DEEP LEARNING TO COMPUTER VISION AND COMPUTER … · 2015. 8. 10. · APPLICATIONS...
Transcript of APPLICATIONS OF DEEP LEARNING TO COMPUTER VISION AND COMPUTER … · 2015. 8. 10. · APPLICATIONS...
![Page 1: APPLICATIONS OF DEEP LEARNING TO COMPUTER VISION AND COMPUTER … · 2015. 8. 10. · APPLICATIONS OF DEEP LEARNING TO COMPUTER VISION AND COMPUTER GRAPHICS Mike Houston . Practical](https://reader033.fdocuments.us/reader033/viewer/2022052613/6053b9717bab064ba721c412/html5/thumbnails/1.jpg)
APPLICATIONS OF DEEP LEARNING TO COMPUTER VISION AND COMPUTER GRAPHICS Mike Houston
![Page 2: APPLICATIONS OF DEEP LEARNING TO COMPUTER VISION AND COMPUTER … · 2015. 8. 10. · APPLICATIONS OF DEEP LEARNING TO COMPUTER VISION AND COMPUTER GRAPHICS Mike Houston . Practical](https://reader033.fdocuments.us/reader033/viewer/2022052613/6053b9717bab064ba721c412/html5/thumbnails/2.jpg)
Practical DEEP LEARNING Examples
Image Classification, Object Detection, Localization, Action Recognition, Scene Understanding
Speech Recognition, Speech Translation, Natural Language Processing
Pedestrian Detection, Traffic Sign Recognition Breast Cancer Cell Mitosis Detection, Volumetric Brain Image Segmentation
![Page 3: APPLICATIONS OF DEEP LEARNING TO COMPUTER VISION AND COMPUTER … · 2015. 8. 10. · APPLICATIONS OF DEEP LEARNING TO COMPUTER VISION AND COMPUTER GRAPHICS Mike Houston . Practical](https://reader033.fdocuments.us/reader033/viewer/2022052613/6053b9717bab064ba721c412/html5/thumbnails/3.jpg)
What is DEEP LEARNING?
Input Result
![Page 4: APPLICATIONS OF DEEP LEARNING TO COMPUTER VISION AND COMPUTER … · 2015. 8. 10. · APPLICATIONS OF DEEP LEARNING TO COMPUTER VISION AND COMPUTER GRAPHICS Mike Houston . Practical](https://reader033.fdocuments.us/reader033/viewer/2022052613/6053b9717bab064ba721c412/html5/thumbnails/4.jpg)
Tree
Cat
Dog
Deep Learning Framework
“turtle”
Forward Propagation
Compute weight update to nudge
from “turtle” towards “dog”
Backward Propagation
Trained Neural
Net Model
“cat”
Repeat
Training
Inference
![Page 5: APPLICATIONS OF DEEP LEARNING TO COMPUTER VISION AND COMPUTER … · 2015. 8. 10. · APPLICATIONS OF DEEP LEARNING TO COMPUTER VISION AND COMPUTER GRAPHICS Mike Houston . Practical](https://reader033.fdocuments.us/reader033/viewer/2022052613/6053b9717bab064ba721c412/html5/thumbnails/5.jpg)
Making a vehicle classifier
PICKUP
SUV
SUV
![Page 6: APPLICATIONS OF DEEP LEARNING TO COMPUTER VISION AND COMPUTER … · 2015. 8. 10. · APPLICATIONS OF DEEP LEARNING TO COMPUTER VISION AND COMPUTER GRAPHICS Mike Houston . Practical](https://reader033.fdocuments.us/reader033/viewer/2022052613/6053b9717bab064ba721c412/html5/thumbnails/6.jpg)
![Page 7: APPLICATIONS OF DEEP LEARNING TO COMPUTER VISION AND COMPUTER … · 2015. 8. 10. · APPLICATIONS OF DEEP LEARNING TO COMPUTER VISION AND COMPUTER GRAPHICS Mike Houston . Practical](https://reader033.fdocuments.us/reader033/viewer/2022052613/6053b9717bab064ba721c412/html5/thumbnails/7.jpg)
The “Big Bang” In Deep Learning
Algorithms Data Compute Capability
![Page 8: APPLICATIONS OF DEEP LEARNING TO COMPUTER VISION AND COMPUTER … · 2015. 8. 10. · APPLICATIONS OF DEEP LEARNING TO COMPUTER VISION AND COMPUTER GRAPHICS Mike Houston . Practical](https://reader033.fdocuments.us/reader033/viewer/2022052613/6053b9717bab064ba721c412/html5/thumbnails/8.jpg)
Medical Research
Detecting Mitosis in
Breast Cancer Cells — IDSIA
Predicting the Toxicity
of New Drugs — Johannes Kepler University
Understanding Gene Mutation
to Prevent Disease — University of Toronto
![Page 9: APPLICATIONS OF DEEP LEARNING TO COMPUTER VISION AND COMPUTER … · 2015. 8. 10. · APPLICATIONS OF DEEP LEARNING TO COMPUTER VISION AND COMPUTER GRAPHICS Mike Houston . Practical](https://reader033.fdocuments.us/reader033/viewer/2022052613/6053b9717bab064ba721c412/html5/thumbnails/9.jpg)
“Automated Image Captioning with ConvNets and Recurrent Nets”
—Andrej Karpathy, Fei-Fei Li
Captioning
![Page 10: APPLICATIONS OF DEEP LEARNING TO COMPUTER VISION AND COMPUTER … · 2015. 8. 10. · APPLICATIONS OF DEEP LEARNING TO COMPUTER VISION AND COMPUTER GRAPHICS Mike Houston . Practical](https://reader033.fdocuments.us/reader033/viewer/2022052613/6053b9717bab064ba721c412/html5/thumbnails/10.jpg)
Why Are GPUs Good for Deep Learning?
GPUs deliver --
same or better prediction accuracy
faster results
smaller footprint
lower power
Neural Networks GPUs
Inherently
Parallel Matrix
Operations
FLOPS
0 0 4
60
110 28%
26%
16%
12%
7%
2010 2011 2012 2013 2014
bird
frog
person
dog
chair
![Page 11: APPLICATIONS OF DEEP LEARNING TO COMPUTER VISION AND COMPUTER … · 2015. 8. 10. · APPLICATIONS OF DEEP LEARNING TO COMPUTER VISION AND COMPUTER GRAPHICS Mike Houston . Practical](https://reader033.fdocuments.us/reader033/viewer/2022052613/6053b9717bab064ba721c412/html5/thumbnails/11.jpg)
GPU-Accelerated Deep Learning
START-UPS
![Page 12: APPLICATIONS OF DEEP LEARNING TO COMPUTER VISION AND COMPUTER … · 2015. 8. 10. · APPLICATIONS OF DEEP LEARNING TO COMPUTER VISION AND COMPUTER GRAPHICS Mike Houston . Practical](https://reader033.fdocuments.us/reader033/viewer/2022052613/6053b9717bab064ba721c412/html5/thumbnails/12.jpg)
GPU-Accelerated Deep Learning Frameworks
CAFFE TORCH THEANO CUDA-CONVNET2 KALDI
Domain Deep Learning
Framework
Scientific Computing
Framework
Math Expression
Compiler
Deep Learning
Application
Speech Recognition
Toolkit
cuDNN R2 R2 R2 -- --
Multi-GPU In Progress In Progress In Progress (nnet2)
Multi-CPU (nnet2)
License BSD-2 GPL BSD Apache 2.0 Apache 2.0
Interface(s) Text-based definition
files, Python, MATLAB Python, Lua, MATLAB Python C++ C++, Shell scripts
Embedded (TK1)
http://developer.nvidia.com/deeplearning
![Page 13: APPLICATIONS OF DEEP LEARNING TO COMPUTER VISION AND COMPUTER … · 2015. 8. 10. · APPLICATIONS OF DEEP LEARNING TO COMPUTER VISION AND COMPUTER GRAPHICS Mike Houston . Practical](https://reader033.fdocuments.us/reader033/viewer/2022052613/6053b9717bab064ba721c412/html5/thumbnails/13.jpg)
DIGITS
![Page 14: APPLICATIONS OF DEEP LEARNING TO COMPUTER VISION AND COMPUTER … · 2015. 8. 10. · APPLICATIONS OF DEEP LEARNING TO COMPUTER VISION AND COMPUTER GRAPHICS Mike Houston . Practical](https://reader033.fdocuments.us/reader033/viewer/2022052613/6053b9717bab064ba721c412/html5/thumbnails/14.jpg)
DIGITS DEEP GPU TRAINING
SYSTEM FOR DATA
SCIENTISTS
Design DNNs
Visualize activations
Manage multiple trainings GPU GPU HW Cloud GPU
Cluster Multi-GPU
USER INTERFACE
Visualize Layers
Configure DNN
Process Data
Monitor Progress
Theano Torch
Caffe cuDNN, cuBLAS
CUDA
![Page 15: APPLICATIONS OF DEEP LEARNING TO COMPUTER VISION AND COMPUTER … · 2015. 8. 10. · APPLICATIONS OF DEEP LEARNING TO COMPUTER VISION AND COMPUTER GRAPHICS Mike Houston . Practical](https://reader033.fdocuments.us/reader033/viewer/2022052613/6053b9717bab064ba721c412/html5/thumbnails/15.jpg)
DIGITS
Test Image
Monitor Progress Configure DNN Process Data Visualize Layers
![Page 16: APPLICATIONS OF DEEP LEARNING TO COMPUTER VISION AND COMPUTER … · 2015. 8. 10. · APPLICATIONS OF DEEP LEARNING TO COMPUTER VISION AND COMPUTER GRAPHICS Mike Houston . Practical](https://reader033.fdocuments.us/reader033/viewer/2022052613/6053b9717bab064ba721c412/html5/thumbnails/16.jpg)
DIGITS DEVBOX World’s fastest GPU
Max GPU out of a plug
Multi-GPU training & inference
![Page 17: APPLICATIONS OF DEEP LEARNING TO COMPUTER VISION AND COMPUTER … · 2015. 8. 10. · APPLICATIONS OF DEEP LEARNING TO COMPUTER VISION AND COMPUTER GRAPHICS Mike Houston . Practical](https://reader033.fdocuments.us/reader033/viewer/2022052613/6053b9717bab064ba721c412/html5/thumbnails/17.jpg)
Production Automotive Pipeline
![Page 18: APPLICATIONS OF DEEP LEARNING TO COMPUTER VISION AND COMPUTER … · 2015. 8. 10. · APPLICATIONS OF DEEP LEARNING TO COMPUTER VISION AND COMPUTER GRAPHICS Mike Houston . Practical](https://reader033.fdocuments.us/reader033/viewer/2022052613/6053b9717bab064ba721c412/html5/thumbnails/18.jpg)
![Page 19: APPLICATIONS OF DEEP LEARNING TO COMPUTER VISION AND COMPUTER … · 2015. 8. 10. · APPLICATIONS OF DEEP LEARNING TO COMPUTER VISION AND COMPUTER GRAPHICS Mike Houston . Practical](https://reader033.fdocuments.us/reader033/viewer/2022052613/6053b9717bab064ba721c412/html5/thumbnails/19.jpg)
![Page 20: APPLICATIONS OF DEEP LEARNING TO COMPUTER VISION AND COMPUTER … · 2015. 8. 10. · APPLICATIONS OF DEEP LEARNING TO COMPUTER VISION AND COMPUTER GRAPHICS Mike Houston . Practical](https://reader033.fdocuments.us/reader033/viewer/2022052613/6053b9717bab064ba721c412/html5/thumbnails/20.jpg)
![Page 21: APPLICATIONS OF DEEP LEARNING TO COMPUTER VISION AND COMPUTER … · 2015. 8. 10. · APPLICATIONS OF DEEP LEARNING TO COMPUTER VISION AND COMPUTER GRAPHICS Mike Houston . Practical](https://reader033.fdocuments.us/reader033/viewer/2022052613/6053b9717bab064ba721c412/html5/thumbnails/21.jpg)
TEGRA X1 CLASSIFICATION Performance
AlexNet
0
10
20
30
40
50
60
70
80
90
100
Tegra K1 Tegra X1
IMAG
ES /
SECO
ND
![Page 22: APPLICATIONS OF DEEP LEARNING TO COMPUTER VISION AND COMPUTER … · 2015. 8. 10. · APPLICATIONS OF DEEP LEARNING TO COMPUTER VISION AND COMPUTER GRAPHICS Mike Houston . Practical](https://reader033.fdocuments.us/reader033/viewer/2022052613/6053b9717bab064ba721c412/html5/thumbnails/22.jpg)
Project dave — darpa autonomous vehicle
DNN-based self-driving robot
Training data by human
driver
No hand-coded CV algorithms
IMAGENET
CHALLENGE Accuracy %
2010 2014 2012 2011 2013
74%
84%
DNN
CV
72%
![Page 23: APPLICATIONS OF DEEP LEARNING TO COMPUTER VISION AND COMPUTER … · 2015. 8. 10. · APPLICATIONS OF DEEP LEARNING TO COMPUTER VISION AND COMPUTER GRAPHICS Mike Houston . Practical](https://reader033.fdocuments.us/reader033/viewer/2022052613/6053b9717bab064ba721c412/html5/thumbnails/23.jpg)
TRAINING DATA 225K Images
![Page 24: APPLICATIONS OF DEEP LEARNING TO COMPUTER VISION AND COMPUTER … · 2015. 8. 10. · APPLICATIONS OF DEEP LEARNING TO COMPUTER VISION AND COMPUTER GRAPHICS Mike Houston . Practical](https://reader033.fdocuments.us/reader033/viewer/2022052613/6053b9717bab064ba721c412/html5/thumbnails/24.jpg)
DAVE IN ACTION
![Page 25: APPLICATIONS OF DEEP LEARNING TO COMPUTER VISION AND COMPUTER … · 2015. 8. 10. · APPLICATIONS OF DEEP LEARNING TO COMPUTER VISION AND COMPUTER GRAPHICS Mike Houston . Practical](https://reader033.fdocuments.us/reader033/viewer/2022052613/6053b9717bab064ba721c412/html5/thumbnails/25.jpg)
Data Scientist Vehicle
Active Learning
Drive PX - Deploy
Model Classification
Detection
Segmentation DIGITS - Train
Network
Solver
Dashboard
![Page 26: APPLICATIONS OF DEEP LEARNING TO COMPUTER VISION AND COMPUTER … · 2015. 8. 10. · APPLICATIONS OF DEEP LEARNING TO COMPUTER VISION AND COMPUTER GRAPHICS Mike Houston . Practical](https://reader033.fdocuments.us/reader033/viewer/2022052613/6053b9717bab064ba721c412/html5/thumbnails/26.jpg)
Deep Learning and Vision/Graphics
![Page 27: APPLICATIONS OF DEEP LEARNING TO COMPUTER VISION AND COMPUTER … · 2015. 8. 10. · APPLICATIONS OF DEEP LEARNING TO COMPUTER VISION AND COMPUTER GRAPHICS Mike Houston . Practical](https://reader033.fdocuments.us/reader033/viewer/2022052613/6053b9717bab064ba721c412/html5/thumbnails/27.jpg)
Street Number Detection
[Goodfellow 2014]
![Page 28: APPLICATIONS OF DEEP LEARNING TO COMPUTER VISION AND COMPUTER … · 2015. 8. 10. · APPLICATIONS OF DEEP LEARNING TO COMPUTER VISION AND COMPUTER GRAPHICS Mike Houston . Practical](https://reader033.fdocuments.us/reader033/viewer/2022052613/6053b9717bab064ba721c412/html5/thumbnails/28.jpg)
Object Classification
[Krizhevsky 2012]
![Page 29: APPLICATIONS OF DEEP LEARNING TO COMPUTER VISION AND COMPUTER … · 2015. 8. 10. · APPLICATIONS OF DEEP LEARNING TO COMPUTER VISION AND COMPUTER GRAPHICS Mike Houston . Practical](https://reader033.fdocuments.us/reader033/viewer/2022052613/6053b9717bab064ba721c412/html5/thumbnails/29.jpg)
Image Retrieval
[Krizhevsky 2012]
![Page 30: APPLICATIONS OF DEEP LEARNING TO COMPUTER VISION AND COMPUTER … · 2015. 8. 10. · APPLICATIONS OF DEEP LEARNING TO COMPUTER VISION AND COMPUTER GRAPHICS Mike Houston . Practical](https://reader033.fdocuments.us/reader033/viewer/2022052613/6053b9717bab064ba721c412/html5/thumbnails/30.jpg)
Pose Estimation
[Toshev, Szegedy 2014]
![Page 31: APPLICATIONS OF DEEP LEARNING TO COMPUTER VISION AND COMPUTER … · 2015. 8. 10. · APPLICATIONS OF DEEP LEARNING TO COMPUTER VISION AND COMPUTER GRAPHICS Mike Houston . Practical](https://reader033.fdocuments.us/reader033/viewer/2022052613/6053b9717bab064ba721c412/html5/thumbnails/31.jpg)
Object Detection
![Page 32: APPLICATIONS OF DEEP LEARNING TO COMPUTER VISION AND COMPUTER … · 2015. 8. 10. · APPLICATIONS OF DEEP LEARNING TO COMPUTER VISION AND COMPUTER GRAPHICS Mike Houston . Practical](https://reader033.fdocuments.us/reader033/viewer/2022052613/6053b9717bab064ba721c412/html5/thumbnails/32.jpg)
[Huval et al. 2015]
![Page 33: APPLICATIONS OF DEEP LEARNING TO COMPUTER VISION AND COMPUTER … · 2015. 8. 10. · APPLICATIONS OF DEEP LEARNING TO COMPUTER VISION AND COMPUTER GRAPHICS Mike Houston . Practical](https://reader033.fdocuments.us/reader033/viewer/2022052613/6053b9717bab064ba721c412/html5/thumbnails/33.jpg)
Face Recognition
[Taigman et al. 2014]
![Page 34: APPLICATIONS OF DEEP LEARNING TO COMPUTER VISION AND COMPUTER … · 2015. 8. 10. · APPLICATIONS OF DEEP LEARNING TO COMPUTER VISION AND COMPUTER GRAPHICS Mike Houston . Practical](https://reader033.fdocuments.us/reader033/viewer/2022052613/6053b9717bab064ba721c412/html5/thumbnails/34.jpg)
Action Recognition
[Simonyan et al. 2014]
![Page 35: APPLICATIONS OF DEEP LEARNING TO COMPUTER VISION AND COMPUTER … · 2015. 8. 10. · APPLICATIONS OF DEEP LEARNING TO COMPUTER VISION AND COMPUTER GRAPHICS Mike Houston . Practical](https://reader033.fdocuments.us/reader033/viewer/2022052613/6053b9717bab064ba721c412/html5/thumbnails/35.jpg)
Playing Games
[Mnih et al. 2013]
![Page 36: APPLICATIONS OF DEEP LEARNING TO COMPUTER VISION AND COMPUTER … · 2015. 8. 10. · APPLICATIONS OF DEEP LEARNING TO COMPUTER VISION AND COMPUTER GRAPHICS Mike Houston . Practical](https://reader033.fdocuments.us/reader033/viewer/2022052613/6053b9717bab064ba721c412/html5/thumbnails/36.jpg)
Semantic Segmentation
[Farabet et al. 2013]
![Page 37: APPLICATIONS OF DEEP LEARNING TO COMPUTER VISION AND COMPUTER … · 2015. 8. 10. · APPLICATIONS OF DEEP LEARNING TO COMPUTER VISION AND COMPUTER GRAPHICS Mike Houston . Practical](https://reader033.fdocuments.us/reader033/viewer/2022052613/6053b9717bab064ba721c412/html5/thumbnails/37.jpg)
Super Resolution
[Dong et al. 2014]
![Page 38: APPLICATIONS OF DEEP LEARNING TO COMPUTER VISION AND COMPUTER … · 2015. 8. 10. · APPLICATIONS OF DEEP LEARNING TO COMPUTER VISION AND COMPUTER GRAPHICS Mike Houston . Practical](https://reader033.fdocuments.us/reader033/viewer/2022052613/6053b9717bab064ba721c412/html5/thumbnails/38.jpg)
Ray Tracing – Monte Carlo Denoising
[Kalantari et al. 2015]
![Page 39: APPLICATIONS OF DEEP LEARNING TO COMPUTER VISION AND COMPUTER … · 2015. 8. 10. · APPLICATIONS OF DEEP LEARNING TO COMPUTER VISION AND COMPUTER GRAPHICS Mike Houston . Practical](https://reader033.fdocuments.us/reader033/viewer/2022052613/6053b9717bab064ba721c412/html5/thumbnails/39.jpg)
“Dreams”
[Mordvinstev et al. 2015]
![Page 40: APPLICATIONS OF DEEP LEARNING TO COMPUTER VISION AND COMPUTER … · 2015. 8. 10. · APPLICATIONS OF DEEP LEARNING TO COMPUTER VISION AND COMPUTER GRAPHICS Mike Houston . Practical](https://reader033.fdocuments.us/reader033/viewer/2022052613/6053b9717bab064ba721c412/html5/thumbnails/40.jpg)
“Dreams”
[Mordvinstev et al. 2015]