Convolutional Neural Networkscs230.stanford.edu/files/C4M1_old.pdfAndrew Ng Why convolutions 10 10...
41
deeplearning.ai Convolutional Neural Networks Computer vision
Transcript of Convolutional Neural Networkscs230.stanford.edu/files/C4M1_old.pdfAndrew Ng Why convolutions 10 10...
deeplearning.ai
ConvolutionalNeuralNetworks
Computervision
AndrewNg
Computer Vision ProblemsImage Classification
Cat? (0/1)
Neural Style Transfer
Object detection64x64
AndrewNg
Deep Learning on large images
Cat? (0/1)
!"!#
!$%&⋮⋮ ⋮
64x64
deeplearning.ai
ConvolutionalNeuralNetworks
Edgedetectionexample
AndrewNg
Computer Vision Problem
vertical edges
horizontal edges
AndrewNg
Vertical edge detection
3 0 1 2 7 41 5 8 9 3 12 7 2 5 1 30 1 3 1 7 84 2 1 6 2 82 4 5 2 3 9
∗ =0 -2 -4 -7
-3 -2 -3 -16
1
1
1
-1
-1
-1
0
0
0
1
1
1
-1
-1
-1
0
0
0
1
1
1
-1
-1
-1
0
0
0
1
1
1
-1
-1
-1
0
0
0
1
1
1
-1
-1
-1
0
0
0
1
1
1
-1
-1
-1
0
0
0
1
1
1
-1
-1
-1
0
0
0
1
1
1
-1
-1
-1
0
0
0
AndrewNg
Vertical edge detection
10 10 10 0 0 010 10 10 0 0 010 10 10 0 0 010 10 10 0 0 010 10 10 0 0 010 10 10 0 0 0
1 0 -11 0 -11 0 -1
=
0 30 30 00 30 30 00 30 30 00 30 30 0
∗
∗
deeplearning.ai
ConvolutionalNeuralNetworks
Moreedgedetection
AndrewNg
Vertical edge detection examples10 10 10 0 0 010 10 10 0 0 010 10 10 0 0 010 10 10 0 0 010 10 10 0 0 010 10 10 0 0 0
1 0 -11 0 -11 0 -1
=
0 30 30 00 30 30 00 30 30 00 30 30 0
∗
∗
0 0 0 10 10 100 0 0 10 10 100 0 0 10 10 100 0 0 10 10 100 0 0 10 10 100 0 0 10 10 10
=
0 -30 -30 00 -30 -30 00 -30 -30 00 -30 -30 0
1 0 -11 0 -11 0 -1
AndrewNg
Vertical and Horizontal Edge Detection1 0 -11 0 -11 0 -1
=∗
10 10 10 0 0 010 10 10 0 0 010 10 10 0 0 00 0 0 10 10 100 0 0 10 10 100 0 0 10 10 10
1 1 10 0 0-1 -1 -1
0 0 0 030 10 -10 -3030 10 -10 -300 0 0 0
Vertical
1 1 10 0 0-1 -1 -1
Horizontal
AndrewNg
Learning to detect edges
#$ #% #&#' #( #)#* #+ #,
3 0 1 2 7 41 5 8 9 3 12 7 2 5 1 30 1 3 1 7 84 2 1 6 2 82 4 5 2 3 9
1 0 -11 0 -11 0 -1
deeplearning.ai
ConvolutionalNeuralNetworks
Padding
AndrewNg
Padding
∗ =
AndrewNg
Valid and Same convolutions
“Valid”:
“Same”: Pad so that output size is the same as the input size.
deeplearning.ai
ConvolutionalNeuralNetworks
Stridedconvolutions
AndrewNg
Strided convolution
3 4 41 0 2-1 0 3
∗ =
2 3 7 4 6 2 96 6 9 8 7 4 33 4 8 3 8 9 77 8 3 6 6 3 44 2 1 8 3 4 63 2 4 1 9 8 30 1 3 9 2 1 4
3
1
-1
4
0
0
4
2
3
3
1
-1
4
0
0
4
2
3
3
1
-1
4
0
0
4
2
33
1
-1
4
0
0
4
2
3
3
1
-1
4
0
0
4
2
3
3
1
-1
4
0
0
4
2
33
1
-1
4
0
0
4
2
3
3
1
-1
4
0
0
4
2
3
3
1
-1
4
0
0
4
2
3
AndrewNg
Summary of convolutions
#×# filter&×& image
padding p stride s
'()*+,- + 1 × '()*+,
- + 1
AndrewNg
Technical note on cross-correlation vs. convolutionConvolution in math textbook:
3 4 5
1 0 2
-1 9 7
2 3 7 4 6 26 6 9 8 7 43 4 8 3 8 97 8 3 6 6 34 2 1 8 3 43 2 4 1 9 8
∗
deeplearning.ai
ConvolutionalNeuralNetworks
Convolutionsovervolumes
AndrewNg
Convolutions on RGB images
AndrewNg
Convolutions on RGB image
=∗
4x4
AndrewNg
Multiple filters
=
6x6x3
4x4
3x3x3
∗
∗
3x3x3
=
4x4
deeplearning.ai
ConvolutionalNeuralNetworks
Onelayerofaconvolutional
network
AndrewNg
Example of a layer
6x6x3
∗3x3x3
∗3x3x3
AndrewNg
Number of parameters in one layer
If you have 10 filters that are 3 x 3 x 3 in one layer of a neural network, how many parameters does that layer have?
AndrewNg
Summary of notationIf layer l is a convolution layer:" # = filter size $ # = padding% # = stride&'# = number of filtersEach filter is:Activations:Weights:bias:
Input:Output:
deeplearning.ai
ConvolutionalNeuralNetworks
Asimpleconvolutionnetworkexample
AndrewNg
Example ConvNet
AndrewNg
Types of layer in a convolutional network:
- Convolution
- Pooling
- Fully connected
deeplearning.ai
ConvolutionalNeuralNetworks
Poolinglayers
AndrewNg
Pooling layer: Max pooling
1 3 2 1
2 9 1 1
1 3 2 3
5 6 1 2
AndrewNg
Pooling layer: Max pooling
1 3 2 1 3
2 9 1 1 5
1 3 2 3 2
8 3 5 1 0
5 6 1 2 9
AndrewNg
Pooling layer: Average pooling
1 3 2 1
2 9 1 1
1 4 2 3
5 6 1 2
AndrewNg
Summary of pooling
Hyperparameters:
f : filter sizes : strideMax or average pooling
deeplearning.ai
ConvolutionalNeuralNetworks
Convolutionalneuralnetworkexample
AndrewNg
Neural network example
AndrewNg
Neural network exampleActivation shape Activation Size # parameters
Input: (32,32,3) 3,072 0
CONV1 (f=5, s=1) (28,28,8) 6,272 208
POOL1 (14,14,8) 1,568 0
CONV2 (f=5, s=1) (10,10,16) 1,600 416
POOL2 (5,5,16) 400 0
FC3 (120,1) 120 48,001
FC4 (84,1) 84 10,081
Softmax (10,1) 10 841
deeplearning.ai
ConvolutionalNeuralNetworks
Whyconvolutions?
AndrewNg
Why convolutions
AndrewNg
Why convolutions10 10 10 0 0 010 10 10 0 0 010 10 10 0 0 010 10 10 0 0 010 10 10 0 0 010 10 10 0 0 0
1 0 -11 0 -11 0 -1
0 30 30 0
0 30 30 0
0 30 30 0
0 30 30 0
∗ =
Parameter sharing: A feature detector (such as a vertical edge detector) that’s useful in one part of the image is probably useful in another part of the image.
Sparsity of connections: In each layer, each output value depends only on a small number of inputs.
AndrewNg
Putting it togetherTraining set(% & , ( & ) … (% + , ( + ).
Use gradient descent to optimize parameters to reduce ,-+
./&Cost , = &
+ ℒ((1 . , ( . )
(1
