Deep LearningandConvolutional Neural Networks
Ronald XIE 8th May, 2014
Deep Learning and Convolutional Neural Networks | Page 1
Tensor voting system
Robot operating system
Deep learning system
Robot localization and Scene Labeling
Robot operating system
Outline
Motivation
Deep Learning
Convolutional Neural Networks
Applications
Deep Learning and Convolutional Neural Networks | Page 2
Outline
Motivation
Deep Learning
Convolutional Neural Networks
Applications
Deep Learning and Convolutional Neural Networks | Page 3
Motivation feature representation
Deep Learning and Convolutional Neural Networks | Page 4
Motivation feature representation
Deep Learning and Convolutional Neural Networks | Page 5
Motivation feature representation
Deep Learning and Convolutional Neural Networks | Page 6
Motivation feature representation
Deep Learning and Convolutional Neural Networks | Page 7
Motivation feature representation
Deep Learning and Convolutional Neural Networks | Page 8
Given a dictionary of simple non-linear functions:
Proposal 1: linear combination
Motivation learning non-linear features
Proposal 2: composition
Deep Learning and Convolutional Neural Networks | Page 9
Given a dictionary of simple non-linear functions:
Proposal 1: linear combination
Motivation learning non-linear features
Proposal 2: composition
Deep Learning and Convolutional Neural Networks | Page 10
Motivation learning non-linear features Linear Combination
BAD: It may requirean exponential number oftemplates!!!
Deep Learning and Convolutional Neural Networks | Page 11
Motivation learning non-linear features Composition
GOOD: Re-use of
intermediate parts Distributed
representations is more efficient
Deep Learning and Convolutional Neural Networks | Page 12
Motivation learning non-linear features
Deep Learning and Convolutional Neural Networks | Page 13
Motivation deep learning in practice
Google Brain
- Big success on image & speech recognition
Microsoft
- Simultaneous interpretation system
Baidu
- Institute of Deep Learning
Deep Learning and Convolutional Neural Networks | Page 14
Motivation deep learning in practice
Deep Learning and Convolutional Neural Networks | Page 15
Outline
Motivation
Deep Learning
Convolutional Neural Networks
Applications
Deep Learning and Convolutional Neural Networks | Page 16
Deep Learning definition
What is deep learning?
It's a Convolutional Net.
It's a Contrastive Divergence.
It's a Feature Learning.
It's a Unsupervised Learning.
It's just old Neural Nets.
It's a Deep Belief Net.
Deep Learning and Convolutional Neural Networks | Page 17
Deep Learning definition
What is deep learning?
A Deep Learning method is: a method which makes
predictions by using a sequence of non-linear processing
stages. The resulting intermediate representations can be
interpreted as feature hierarchies and the whole system is
jointly learned from data.
Some deep learning methods are supervised, others are
unsupervised.
It's a large family!
Deep Learning and Convolutional Neural Networks | Page 18
Three types of deep architectures:
Feed-Forward: Multilayer Neural Nets, Convolutional Nets
Feed-Back: Sparse Coding, Deconvolutional Net
Bi-Directional: Deep Boltzmann Machines, Auto-Encoders
Deep Learning types
Deep Learning and Convolutional Neural Networks | Page 19
Three types of training protocols: Purely Supervised
- Initialize parameters randomly
- Train in supervised mode
- Used in most practical systems for speech and image recognition
Unsupervised layer-wise + supervised classifier on top- Train each layer unsupervised, one after the other
- Train a supervised classifier on top, keeping the other layers fixed
- Good when very few labeled samples are available
Unsupervised layer-wise + global supervised fine-tuning- Train each layer unsupervised, one after the other
- Add a classifier layer, and retrain the whole thing supervised
- Good when label set is poor
Deep Learning types
Deep Learning and Convolutional Neural Networks | Page 20
Outline
Motivation
Deep Learning
Convolutional Neural Networks
Applications
Deep Learning and Convolutional Neural Networks | Page 30
Convolution Basic concert
CNNs key ideas
Deep Learning and Convolutional Neural Networks | Page 32
CNNs key ideas
Deep Learning and Convolutional Neural Networks | Page 33
CNNs key ideas
Deep Learning and Convolutional Neural Networks | Page 34
CNNs key ideas
Deep Learning and Convolutional Neural Networks | Page 35
CNNs key ideas
Deep Learning and Convolutional Neural Networks | Page 36
CNNs key ideas
Deep Learning and Convolutional Neural Networks | Page 37
CNNs key ideas
Deep Learning and Convolutional Neural Networks | Page 38
CNNs key ideas
Deep Learning and Convolutional Neural Networks | Page 39
CNNs key ideas
Deep Learning and Convolutional Neural Networks | Page 40
CNNs key ideas
Deep Learning and Convolutional Neural Networks | Page 41
CNNs key ideas
Deep Learning and Convolutional Neural Networks | Page 42
CNNs key ideas
Deep Learning and Convolutional Neural Networks | Page 43
CNNs typical architecture
Deep Learning and Convolutional Neural Networks | Page 44
CNNs typical architecture
Deep Learning and Convolutional Neural Networks | Page 45
CNNs conclusion
Connect each hidden unit to a small
patch of the input.
Share the weight across hidden units.
Subsampling layers are useful to reduce
computational burden and increase
invariance.
Deep Learning and Convolutional Neural Networks | Page 46
Outline
Motivation
Deep Learning
Convolutional Neural Networks
Applications
Deep Learning and Convolutional Neural Networks | Page 47
ApplicationsScene recognition:
Deep Learning and Convolutional Neural Networks | Page 52
ApplicationsLocalization:
Deep Learning and Convolutional Neural Networks | Page 52
Top Related