Machine Learning Primer - Department of Computer Science ...€¦ · Machine Learning Primer Polina...
Transcript of Machine Learning Primer - Department of Computer Science ...€¦ · Machine Learning Primer Polina...
![Page 1: Machine Learning Primer - Department of Computer Science ...€¦ · Machine Learning Primer Polina Binder Slides credit: MIT Deep Learning for Self Driving Cars, Toronto CSC 321,](https://reader036.fdocuments.us/reader036/viewer/2022062917/5ed0f3a12a742537f26ea142/html5/thumbnails/1.jpg)
Machine Learning Primer
Polina Binder
Slides credit: MIT Deep Learning for Self Driving Cars, Toronto CSC 321, ICML reinforcement learning tutorial
1
![Page 2: Machine Learning Primer - Department of Computer Science ...€¦ · Machine Learning Primer Polina Binder Slides credit: MIT Deep Learning for Self Driving Cars, Toronto CSC 321,](https://reader036.fdocuments.us/reader036/viewer/2022062917/5ed0f3a12a742537f26ea142/html5/thumbnails/2.jpg)
Outline● Introduction to machine learning● How machine learning is used in self-driving cars● Deep learning
○ Neural Network Basics○ Structures in Neural Networks○ Training Neural Networks○ Challenges with deep learning
● Reinforcement learning
2
![Page 3: Machine Learning Primer - Department of Computer Science ...€¦ · Machine Learning Primer Polina Binder Slides credit: MIT Deep Learning for Self Driving Cars, Toronto CSC 321,](https://reader036.fdocuments.us/reader036/viewer/2022062917/5ed0f3a12a742537f26ea142/html5/thumbnails/3.jpg)
Outline● Introduction to machine learning● How machine learning is used in self-driving cars● Deep learning
○ Neural Network Basics○ Structures in Neural Networks○ Training Neural Networks○ Challenges with deep learning
● Reinforcement learning
3
![Page 4: Machine Learning Primer - Department of Computer Science ...€¦ · Machine Learning Primer Polina Binder Slides credit: MIT Deep Learning for Self Driving Cars, Toronto CSC 321,](https://reader036.fdocuments.us/reader036/viewer/2022062917/5ed0f3a12a742537f26ea142/html5/thumbnails/4.jpg)
What is machine learning?
● A class of algorithms that allows us to infer rules and parameters based on example data.
● In contrast to: hand coded rules
4
![Page 5: Machine Learning Primer - Department of Computer Science ...€¦ · Machine Learning Primer Polina Binder Slides credit: MIT Deep Learning for Self Driving Cars, Toronto CSC 321,](https://reader036.fdocuments.us/reader036/viewer/2022062917/5ed0f3a12a742537f26ea142/html5/thumbnails/5.jpg)
Types of Machine Learning
5
![Page 6: Machine Learning Primer - Department of Computer Science ...€¦ · Machine Learning Primer Polina Binder Slides credit: MIT Deep Learning for Self Driving Cars, Toronto CSC 321,](https://reader036.fdocuments.us/reader036/viewer/2022062917/5ed0f3a12a742537f26ea142/html5/thumbnails/6.jpg)
Supervised Learning
● Input: Training data and labels.● Goal: Learn function to map new
unlabelled data to labels
6
![Page 7: Machine Learning Primer - Department of Computer Science ...€¦ · Machine Learning Primer Polina Binder Slides credit: MIT Deep Learning for Self Driving Cars, Toronto CSC 321,](https://reader036.fdocuments.us/reader036/viewer/2022062917/5ed0f3a12a742537f26ea142/html5/thumbnails/7.jpg)
Supervised Learning: Classification vs. Regression
Input: data and discrete labelsGoal: Map data to discrete categories
Input: data and continuous valuesGoal: Learn an approximate function that maps data to values
7
![Page 8: Machine Learning Primer - Department of Computer Science ...€¦ · Machine Learning Primer Polina Binder Slides credit: MIT Deep Learning for Self Driving Cars, Toronto CSC 321,](https://reader036.fdocuments.us/reader036/viewer/2022062917/5ed0f3a12a742537f26ea142/html5/thumbnails/8.jpg)
Supervised Learning: Classification vs. Regression
Classification: labels are discrete values
Regression: labels are continuous values
8
![Page 9: Machine Learning Primer - Department of Computer Science ...€¦ · Machine Learning Primer Polina Binder Slides credit: MIT Deep Learning for Self Driving Cars, Toronto CSC 321,](https://reader036.fdocuments.us/reader036/viewer/2022062917/5ed0f3a12a742537f26ea142/html5/thumbnails/9.jpg)
Unsupervised Learning
● Input: training data without labels.● Goal: Learn structure in the data
9
![Page 10: Machine Learning Primer - Department of Computer Science ...€¦ · Machine Learning Primer Polina Binder Slides credit: MIT Deep Learning for Self Driving Cars, Toronto CSC 321,](https://reader036.fdocuments.us/reader036/viewer/2022062917/5ed0f3a12a742537f26ea142/html5/thumbnails/10.jpg)
Semi-Supervised Learning
● Input: training data, some of which is labelled● Goal: Learn function to map new unlabelled data to labels and/ or learn structure in
the data 10
![Page 11: Machine Learning Primer - Department of Computer Science ...€¦ · Machine Learning Primer Polina Binder Slides credit: MIT Deep Learning for Self Driving Cars, Toronto CSC 321,](https://reader036.fdocuments.us/reader036/viewer/2022062917/5ed0f3a12a742537f26ea142/html5/thumbnails/11.jpg)
Reinforcement Learning
● Framework for decision making○ Agent with the capacity to act○ Each action influences the
agent’s future state○ Success is measured by a
reward signal ○ Goal: Select actions to
maximize future reward
11
![Page 12: Machine Learning Primer - Department of Computer Science ...€¦ · Machine Learning Primer Polina Binder Slides credit: MIT Deep Learning for Self Driving Cars, Toronto CSC 321,](https://reader036.fdocuments.us/reader036/viewer/2022062917/5ed0f3a12a742537f26ea142/html5/thumbnails/12.jpg)
Outline● Introduction to machine learning● How machine learning is used in self-driving cars● Deep learning
○ Neural Network Basics○ Structures in Neural Networks○ Training Neural Networks○ Challenges with deep learning
● Reinforcement learning
12
![Page 13: Machine Learning Primer - Department of Computer Science ...€¦ · Machine Learning Primer Polina Binder Slides credit: MIT Deep Learning for Self Driving Cars, Toronto CSC 321,](https://reader036.fdocuments.us/reader036/viewer/2022062917/5ed0f3a12a742537f26ea142/html5/thumbnails/13.jpg)
13
![Page 14: Machine Learning Primer - Department of Computer Science ...€¦ · Machine Learning Primer Polina Binder Slides credit: MIT Deep Learning for Self Driving Cars, Toronto CSC 321,](https://reader036.fdocuments.us/reader036/viewer/2022062917/5ed0f3a12a742537f26ea142/html5/thumbnails/14.jpg)
14
![Page 15: Machine Learning Primer - Department of Computer Science ...€¦ · Machine Learning Primer Polina Binder Slides credit: MIT Deep Learning for Self Driving Cars, Toronto CSC 321,](https://reader036.fdocuments.us/reader036/viewer/2022062917/5ed0f3a12a742537f26ea142/html5/thumbnails/15.jpg)
15
![Page 16: Machine Learning Primer - Department of Computer Science ...€¦ · Machine Learning Primer Polina Binder Slides credit: MIT Deep Learning for Self Driving Cars, Toronto CSC 321,](https://reader036.fdocuments.us/reader036/viewer/2022062917/5ed0f3a12a742537f26ea142/html5/thumbnails/16.jpg)
16
![Page 17: Machine Learning Primer - Department of Computer Science ...€¦ · Machine Learning Primer Polina Binder Slides credit: MIT Deep Learning for Self Driving Cars, Toronto CSC 321,](https://reader036.fdocuments.us/reader036/viewer/2022062917/5ed0f3a12a742537f26ea142/html5/thumbnails/17.jpg)
17
![Page 18: Machine Learning Primer - Department of Computer Science ...€¦ · Machine Learning Primer Polina Binder Slides credit: MIT Deep Learning for Self Driving Cars, Toronto CSC 321,](https://reader036.fdocuments.us/reader036/viewer/2022062917/5ed0f3a12a742537f26ea142/html5/thumbnails/18.jpg)
18
![Page 19: Machine Learning Primer - Department of Computer Science ...€¦ · Machine Learning Primer Polina Binder Slides credit: MIT Deep Learning for Self Driving Cars, Toronto CSC 321,](https://reader036.fdocuments.us/reader036/viewer/2022062917/5ed0f3a12a742537f26ea142/html5/thumbnails/19.jpg)
19
![Page 20: Machine Learning Primer - Department of Computer Science ...€¦ · Machine Learning Primer Polina Binder Slides credit: MIT Deep Learning for Self Driving Cars, Toronto CSC 321,](https://reader036.fdocuments.us/reader036/viewer/2022062917/5ed0f3a12a742537f26ea142/html5/thumbnails/20.jpg)
20
![Page 21: Machine Learning Primer - Department of Computer Science ...€¦ · Machine Learning Primer Polina Binder Slides credit: MIT Deep Learning for Self Driving Cars, Toronto CSC 321,](https://reader036.fdocuments.us/reader036/viewer/2022062917/5ed0f3a12a742537f26ea142/html5/thumbnails/21.jpg)
21
![Page 22: Machine Learning Primer - Department of Computer Science ...€¦ · Machine Learning Primer Polina Binder Slides credit: MIT Deep Learning for Self Driving Cars, Toronto CSC 321,](https://reader036.fdocuments.us/reader036/viewer/2022062917/5ed0f3a12a742537f26ea142/html5/thumbnails/22.jpg)
Outline● Introduction to machine learning● How machine learning is used in self-driving cars● Deep learning
○ Neural Network Basics○ Structures in Neural Networks○ Training Neural Networks○ Challenges with deep learning
● Reinforcement learning
22
![Page 23: Machine Learning Primer - Department of Computer Science ...€¦ · Machine Learning Primer Polina Binder Slides credit: MIT Deep Learning for Self Driving Cars, Toronto CSC 321,](https://reader036.fdocuments.us/reader036/viewer/2022062917/5ed0f3a12a742537f26ea142/html5/thumbnails/23.jpg)
Outline● Introduction to machine learning● How machine learning is used in self-driving cars● Deep learning
○ Neural Network Basics○ Structures in Neural Networks○ Training Neural Networks○ Challenges with deep learning
● Reinforcement learning
23
![Page 24: Machine Learning Primer - Department of Computer Science ...€¦ · Machine Learning Primer Polina Binder Slides credit: MIT Deep Learning for Self Driving Cars, Toronto CSC 321,](https://reader036.fdocuments.us/reader036/viewer/2022062917/5ed0f3a12a742537f26ea142/html5/thumbnails/24.jpg)
24
![Page 25: Machine Learning Primer - Department of Computer Science ...€¦ · Machine Learning Primer Polina Binder Slides credit: MIT Deep Learning for Self Driving Cars, Toronto CSC 321,](https://reader036.fdocuments.us/reader036/viewer/2022062917/5ed0f3a12a742537f26ea142/html5/thumbnails/25.jpg)
Neurons - Building block of neural networks
25
![Page 26: Machine Learning Primer - Department of Computer Science ...€¦ · Machine Learning Primer Polina Binder Slides credit: MIT Deep Learning for Self Driving Cars, Toronto CSC 321,](https://reader036.fdocuments.us/reader036/viewer/2022062917/5ed0f3a12a742537f26ea142/html5/thumbnails/26.jpg)
Activation Functions
Bounded outputs Zero-centered Unbounded outputsTrains Faster 26
![Page 27: Machine Learning Primer - Department of Computer Science ...€¦ · Machine Learning Primer Polina Binder Slides credit: MIT Deep Learning for Self Driving Cars, Toronto CSC 321,](https://reader036.fdocuments.us/reader036/viewer/2022062917/5ed0f3a12a742537f26ea142/html5/thumbnails/27.jpg)
Combining neurons into layers
27
![Page 28: Machine Learning Primer - Department of Computer Science ...€¦ · Machine Learning Primer Polina Binder Slides credit: MIT Deep Learning for Self Driving Cars, Toronto CSC 321,](https://reader036.fdocuments.us/reader036/viewer/2022062917/5ed0f3a12a742537f26ea142/html5/thumbnails/28.jpg)
28
![Page 29: Machine Learning Primer - Department of Computer Science ...€¦ · Machine Learning Primer Polina Binder Slides credit: MIT Deep Learning for Self Driving Cars, Toronto CSC 321,](https://reader036.fdocuments.us/reader036/viewer/2022062917/5ed0f3a12a742537f26ea142/html5/thumbnails/29.jpg)
Outline● Introduction to machine learning● How machine learning is used in self-driving cars● Deep learning
○ Neural Network Basics○ Structures in Neural Networks○ Training Neural Networks○ Challenges with deep learning
● Reinforcement learning
29
![Page 30: Machine Learning Primer - Department of Computer Science ...€¦ · Machine Learning Primer Polina Binder Slides credit: MIT Deep Learning for Self Driving Cars, Toronto CSC 321,](https://reader036.fdocuments.us/reader036/viewer/2022062917/5ed0f3a12a742537f26ea142/html5/thumbnails/30.jpg)
30
![Page 31: Machine Learning Primer - Department of Computer Science ...€¦ · Machine Learning Primer Polina Binder Slides credit: MIT Deep Learning for Self Driving Cars, Toronto CSC 321,](https://reader036.fdocuments.us/reader036/viewer/2022062917/5ed0f3a12a742537f26ea142/html5/thumbnails/31.jpg)
Fully Connected Neural Network
● No connections within a layer● Each neuron is connected to
all neurons in the previous layer
● Used in classification problems, sometimes image recognition, etc.
Multilayer perceptron
31
![Page 32: Machine Learning Primer - Department of Computer Science ...€¦ · Machine Learning Primer Polina Binder Slides credit: MIT Deep Learning for Self Driving Cars, Toronto CSC 321,](https://reader036.fdocuments.us/reader036/viewer/2022062917/5ed0f3a12a742537f26ea142/html5/thumbnails/32.jpg)
Size of Fully Connected Neural Networks
32
![Page 33: Machine Learning Primer - Department of Computer Science ...€¦ · Machine Learning Primer Polina Binder Slides credit: MIT Deep Learning for Self Driving Cars, Toronto CSC 321,](https://reader036.fdocuments.us/reader036/viewer/2022062917/5ed0f3a12a742537f26ea142/html5/thumbnails/33.jpg)
Alternative: Convolutional Neural Networks● Not all layers are fully connected● Primarily used for image clustering, recognition, and classification● Convolutional layers - apply same filter at every location in the image● Pooling layers - reduce the size of the network and build in invariance to small
transformations
33
![Page 34: Machine Learning Primer - Department of Computer Science ...€¦ · Machine Learning Primer Polina Binder Slides credit: MIT Deep Learning for Self Driving Cars, Toronto CSC 321,](https://reader036.fdocuments.us/reader036/viewer/2022062917/5ed0f3a12a742537f26ea142/html5/thumbnails/34.jpg)
Convolution● Motivation: Learn a set of features that occur at all image locations● Apply same weights to every region of the image● Functions as a feature detector
● Example: 28x28 image, 5x5 filter - 25 shared weights
34
![Page 35: Machine Learning Primer - Department of Computer Science ...€¦ · Machine Learning Primer Polina Binder Slides credit: MIT Deep Learning for Self Driving Cars, Toronto CSC 321,](https://reader036.fdocuments.us/reader036/viewer/2022062917/5ed0f3a12a742537f26ea142/html5/thumbnails/35.jpg)
Pooling● Summarize the output of a group of units● Reduce the size of the representation ● Invariances to small perturbations in input.● Example: maximum of every 2x2 region
35
![Page 36: Machine Learning Primer - Department of Computer Science ...€¦ · Machine Learning Primer Polina Binder Slides credit: MIT Deep Learning for Self Driving Cars, Toronto CSC 321,](https://reader036.fdocuments.us/reader036/viewer/2022062917/5ed0f3a12a742537f26ea142/html5/thumbnails/36.jpg)
Convolutional neural networksPutting pooling and convolutional layers together
36
![Page 37: Machine Learning Primer - Department of Computer Science ...€¦ · Machine Learning Primer Polina Binder Slides credit: MIT Deep Learning for Self Driving Cars, Toronto CSC 321,](https://reader036.fdocuments.us/reader036/viewer/2022062917/5ed0f3a12a742537f26ea142/html5/thumbnails/37.jpg)
Higher layers capture more abstract information
37
![Page 38: Machine Learning Primer - Department of Computer Science ...€¦ · Machine Learning Primer Polina Binder Slides credit: MIT Deep Learning for Self Driving Cars, Toronto CSC 321,](https://reader036.fdocuments.us/reader036/viewer/2022062917/5ed0f3a12a742537f26ea142/html5/thumbnails/38.jpg)
38
![Page 39: Machine Learning Primer - Department of Computer Science ...€¦ · Machine Learning Primer Polina Binder Slides credit: MIT Deep Learning for Self Driving Cars, Toronto CSC 321,](https://reader036.fdocuments.us/reader036/viewer/2022062917/5ed0f3a12a742537f26ea142/html5/thumbnails/39.jpg)
Recurrent Neural Networks
● Often used for language modelling● Hard to train long term dependencies, e.g. remembering
what happened hundreds of words ago. 39
![Page 40: Machine Learning Primer - Department of Computer Science ...€¦ · Machine Learning Primer Polina Binder Slides credit: MIT Deep Learning for Self Driving Cars, Toronto CSC 321,](https://reader036.fdocuments.us/reader036/viewer/2022062917/5ed0f3a12a742537f26ea142/html5/thumbnails/40.jpg)
Long Short-Term Memory (LSTM)
● Capable of learning long-term dependencies
● Composed of memory cells which have controllers saying when to store or forget information.
● Used for time series data
● Example application: text generation
40
![Page 41: Machine Learning Primer - Department of Computer Science ...€¦ · Machine Learning Primer Polina Binder Slides credit: MIT Deep Learning for Self Driving Cars, Toronto CSC 321,](https://reader036.fdocuments.us/reader036/viewer/2022062917/5ed0f3a12a742537f26ea142/html5/thumbnails/41.jpg)
LSTM Components● x - input to LSTM● h - hidden state (output
vector)● c - cell state vector: (carries
information down the sequence of the LSTM)
● F - forget gate activation● I - input gate activation● O - output gate activation
41
![Page 42: Machine Learning Primer - Department of Computer Science ...€¦ · Machine Learning Primer Polina Binder Slides credit: MIT Deep Learning for Self Driving Cars, Toronto CSC 321,](https://reader036.fdocuments.us/reader036/viewer/2022062917/5ed0f3a12a742537f26ea142/html5/thumbnails/42.jpg)
LSTM general behaviour● I = 0, F = 1: Remember
previous value● I = 1, F = 1: Add to previous
value● I = 0, F = 0; Erase the value● I = 1, F = 0: Overwrite the
value
42
![Page 43: Machine Learning Primer - Department of Computer Science ...€¦ · Machine Learning Primer Polina Binder Slides credit: MIT Deep Learning for Self Driving Cars, Toronto CSC 321,](https://reader036.fdocuments.us/reader036/viewer/2022062917/5ed0f3a12a742537f26ea142/html5/thumbnails/43.jpg)
Forget Gatesf_t between 0 (forget previous input) and 1 (keep) previous inputFunction of previous and current input
43
![Page 44: Machine Learning Primer - Department of Computer Science ...€¦ · Machine Learning Primer Polina Binder Slides credit: MIT Deep Learning for Self Driving Cars, Toronto CSC 321,](https://reader036.fdocuments.us/reader036/viewer/2022062917/5ed0f3a12a742537f26ea142/html5/thumbnails/44.jpg)
Ignore gate and temporarily cell statei - ignore or keep new inputs C - proposed new cell state
44
![Page 45: Machine Learning Primer - Department of Computer Science ...€¦ · Machine Learning Primer Polina Binder Slides credit: MIT Deep Learning for Self Driving Cars, Toronto CSC 321,](https://reader036.fdocuments.us/reader036/viewer/2022062917/5ed0f3a12a742537f26ea142/html5/thumbnails/45.jpg)
Outputting DataCell state: output a combination of previous and new cell state.
45
![Page 46: Machine Learning Primer - Department of Computer Science ...€¦ · Machine Learning Primer Polina Binder Slides credit: MIT Deep Learning for Self Driving Cars, Toronto CSC 321,](https://reader036.fdocuments.us/reader036/viewer/2022062917/5ed0f3a12a742537f26ea142/html5/thumbnails/46.jpg)
LSTMso - Output gate’s activation vector. Decides what the next hidden/output state should be.
h - Output vector of the LSTM
46
![Page 47: Machine Learning Primer - Department of Computer Science ...€¦ · Machine Learning Primer Polina Binder Slides credit: MIT Deep Learning for Self Driving Cars, Toronto CSC 321,](https://reader036.fdocuments.us/reader036/viewer/2022062917/5ed0f3a12a742537f26ea142/html5/thumbnails/47.jpg)
Application: Visual Odometry● Combines Convolutional and Recurrent layers
47
![Page 48: Machine Learning Primer - Department of Computer Science ...€¦ · Machine Learning Primer Polina Binder Slides credit: MIT Deep Learning for Self Driving Cars, Toronto CSC 321,](https://reader036.fdocuments.us/reader036/viewer/2022062917/5ed0f3a12a742537f26ea142/html5/thumbnails/48.jpg)
Restricted Boltzmann Machine● Bipartite Graph over hidden and visible nodes● Model the joint distribution of the data and the hidden layers.● Unsupervised and semi-supervised learning● Generative graphical model
48
![Page 49: Machine Learning Primer - Department of Computer Science ...€¦ · Machine Learning Primer Polina Binder Slides credit: MIT Deep Learning for Self Driving Cars, Toronto CSC 321,](https://reader036.fdocuments.us/reader036/viewer/2022062917/5ed0f3a12a742537f26ea142/html5/thumbnails/49.jpg)
Deep Belief NetworksSimilar to RBMs with multiple hidden layers.
49
![Page 50: Machine Learning Primer - Department of Computer Science ...€¦ · Machine Learning Primer Polina Binder Slides credit: MIT Deep Learning for Self Driving Cars, Toronto CSC 321,](https://reader036.fdocuments.us/reader036/viewer/2022062917/5ed0f3a12a742537f26ea142/html5/thumbnails/50.jpg)
Outline● Introduction to machine learning● How machine learning is used in self-driving cars● Deep learning
○ Neural Network Basics○ Structures in Neural Networks○ Training Neural Networks○ Challenges with deep learning
● Reinforcement learning
50
![Page 51: Machine Learning Primer - Department of Computer Science ...€¦ · Machine Learning Primer Polina Binder Slides credit: MIT Deep Learning for Self Driving Cars, Toronto CSC 321,](https://reader036.fdocuments.us/reader036/viewer/2022062917/5ed0f3a12a742537f26ea142/html5/thumbnails/51.jpg)
51
![Page 52: Machine Learning Primer - Department of Computer Science ...€¦ · Machine Learning Primer Polina Binder Slides credit: MIT Deep Learning for Self Driving Cars, Toronto CSC 321,](https://reader036.fdocuments.us/reader036/viewer/2022062917/5ed0f3a12a742537f26ea142/html5/thumbnails/52.jpg)
52
![Page 53: Machine Learning Primer - Department of Computer Science ...€¦ · Machine Learning Primer Polina Binder Slides credit: MIT Deep Learning for Self Driving Cars, Toronto CSC 321,](https://reader036.fdocuments.us/reader036/viewer/2022062917/5ed0f3a12a742537f26ea142/html5/thumbnails/53.jpg)
Loss Function● Example Loss Function:
● Predicted labels are a function of the weights and biases in the neural network.
● Loss Function is a function of weights and biases in the neural network. ● Weights and biases can be optimized by gradient descent.
53
![Page 54: Machine Learning Primer - Department of Computer Science ...€¦ · Machine Learning Primer Polina Binder Slides credit: MIT Deep Learning for Self Driving Cars, Toronto CSC 321,](https://reader036.fdocuments.us/reader036/viewer/2022062917/5ed0f3a12a742537f26ea142/html5/thumbnails/54.jpg)
Gradient Descent: Example● Loss function: C = f(w), w is a
weight● Weight’s gradient: dC/dw● dC/dw > 0: Decreasing w
increases C● dC/dw < 0: Increasing w
increases C● For small s > 0, updating
w’ = w - s * dC/dw decreases C● Repeatedly adjust weights
looking for local minimum in C
54
![Page 55: Machine Learning Primer - Department of Computer Science ...€¦ · Machine Learning Primer Polina Binder Slides credit: MIT Deep Learning for Self Driving Cars, Toronto CSC 321,](https://reader036.fdocuments.us/reader036/viewer/2022062917/5ed0f3a12a742537f26ea142/html5/thumbnails/55.jpg)
Back propagation - forward pass
Similar as before a tiny bit more math.
55
![Page 56: Machine Learning Primer - Department of Computer Science ...€¦ · Machine Learning Primer Polina Binder Slides credit: MIT Deep Learning for Self Driving Cars, Toronto CSC 321,](https://reader036.fdocuments.us/reader036/viewer/2022062917/5ed0f3a12a742537f26ea142/html5/thumbnails/56.jpg)
Outline● Introduction to machine learning● How machine learning is used in self-driving cars● Deep learning
○ Neural Network Basics○ Structures in Neural Networks○ Training Neural Networks○ Challenges with deep learning
● Reinforcement learning
56
![Page 57: Machine Learning Primer - Department of Computer Science ...€¦ · Machine Learning Primer Polina Binder Slides credit: MIT Deep Learning for Self Driving Cars, Toronto CSC 321,](https://reader036.fdocuments.us/reader036/viewer/2022062917/5ed0f3a12a742537f26ea142/html5/thumbnails/57.jpg)
Backpropagation and gradients
● Gradients at layer n-1 are functions of gradients in layer n.
● Gradients are multiplied as they’re passed through the network
● Leads to vanishing gradients: Gradients in lower levels are close to 0.
● Exploding gradients: Update too strongly, or have numerical overflow
57
![Page 58: Machine Learning Primer - Department of Computer Science ...€¦ · Machine Learning Primer Polina Binder Slides credit: MIT Deep Learning for Self Driving Cars, Toronto CSC 321,](https://reader036.fdocuments.us/reader036/viewer/2022062917/5ed0f3a12a742537f26ea142/html5/thumbnails/58.jpg)
Activation Functions
58
![Page 59: Machine Learning Primer - Department of Computer Science ...€¦ · Machine Learning Primer Polina Binder Slides credit: MIT Deep Learning for Self Driving Cars, Toronto CSC 321,](https://reader036.fdocuments.us/reader036/viewer/2022062917/5ed0f3a12a742537f26ea142/html5/thumbnails/59.jpg)
59
![Page 60: Machine Learning Primer - Department of Computer Science ...€¦ · Machine Learning Primer Polina Binder Slides credit: MIT Deep Learning for Self Driving Cars, Toronto CSC 321,](https://reader036.fdocuments.us/reader036/viewer/2022062917/5ed0f3a12a742537f26ea142/html5/thumbnails/60.jpg)
60
![Page 61: Machine Learning Primer - Department of Computer Science ...€¦ · Machine Learning Primer Polina Binder Slides credit: MIT Deep Learning for Self Driving Cars, Toronto CSC 321,](https://reader036.fdocuments.us/reader036/viewer/2022062917/5ed0f3a12a742537f26ea142/html5/thumbnails/61.jpg)
61
![Page 62: Machine Learning Primer - Department of Computer Science ...€¦ · Machine Learning Primer Polina Binder Slides credit: MIT Deep Learning for Self Driving Cars, Toronto CSC 321,](https://reader036.fdocuments.us/reader036/viewer/2022062917/5ed0f3a12a742537f26ea142/html5/thumbnails/62.jpg)
Adversarial examples
Noise is set to be a function of the gradient in the neural network
62
![Page 63: Machine Learning Primer - Department of Computer Science ...€¦ · Machine Learning Primer Polina Binder Slides credit: MIT Deep Learning for Self Driving Cars, Toronto CSC 321,](https://reader036.fdocuments.us/reader036/viewer/2022062917/5ed0f3a12a742537f26ea142/html5/thumbnails/63.jpg)
Adversarial Stickers
Evtimov et al.Misclassified as speed signs
63
![Page 64: Machine Learning Primer - Department of Computer Science ...€¦ · Machine Learning Primer Polina Binder Slides credit: MIT Deep Learning for Self Driving Cars, Toronto CSC 321,](https://reader036.fdocuments.us/reader036/viewer/2022062917/5ed0f3a12a742537f26ea142/html5/thumbnails/64.jpg)
64
![Page 65: Machine Learning Primer - Department of Computer Science ...€¦ · Machine Learning Primer Polina Binder Slides credit: MIT Deep Learning for Self Driving Cars, Toronto CSC 321,](https://reader036.fdocuments.us/reader036/viewer/2022062917/5ed0f3a12a742537f26ea142/html5/thumbnails/65.jpg)
65
![Page 66: Machine Learning Primer - Department of Computer Science ...€¦ · Machine Learning Primer Polina Binder Slides credit: MIT Deep Learning for Self Driving Cars, Toronto CSC 321,](https://reader036.fdocuments.us/reader036/viewer/2022062917/5ed0f3a12a742537f26ea142/html5/thumbnails/66.jpg)
66
![Page 67: Machine Learning Primer - Department of Computer Science ...€¦ · Machine Learning Primer Polina Binder Slides credit: MIT Deep Learning for Self Driving Cars, Toronto CSC 321,](https://reader036.fdocuments.us/reader036/viewer/2022062917/5ed0f3a12a742537f26ea142/html5/thumbnails/67.jpg)
67
![Page 68: Machine Learning Primer - Department of Computer Science ...€¦ · Machine Learning Primer Polina Binder Slides credit: MIT Deep Learning for Self Driving Cars, Toronto CSC 321,](https://reader036.fdocuments.us/reader036/viewer/2022062917/5ed0f3a12a742537f26ea142/html5/thumbnails/68.jpg)
Outline● Introduction to machine learning● How machine learning is used in self-driving cars● Deep learning
○ Neural Network Basics○ Structures in Neural Networks○ Training Neural Networks○ Challenges with deep learning
● Reinforcement learning
68
![Page 69: Machine Learning Primer - Department of Computer Science ...€¦ · Machine Learning Primer Polina Binder Slides credit: MIT Deep Learning for Self Driving Cars, Toronto CSC 321,](https://reader036.fdocuments.us/reader036/viewer/2022062917/5ed0f3a12a742537f26ea142/html5/thumbnails/69.jpg)
69
![Page 70: Machine Learning Primer - Department of Computer Science ...€¦ · Machine Learning Primer Polina Binder Slides credit: MIT Deep Learning for Self Driving Cars, Toronto CSC 321,](https://reader036.fdocuments.us/reader036/viewer/2022062917/5ed0f3a12a742537f26ea142/html5/thumbnails/70.jpg)
70
![Page 71: Machine Learning Primer - Department of Computer Science ...€¦ · Machine Learning Primer Polina Binder Slides credit: MIT Deep Learning for Self Driving Cars, Toronto CSC 321,](https://reader036.fdocuments.us/reader036/viewer/2022062917/5ed0f3a12a742537f26ea142/html5/thumbnails/71.jpg)
71
![Page 72: Machine Learning Primer - Department of Computer Science ...€¦ · Machine Learning Primer Polina Binder Slides credit: MIT Deep Learning for Self Driving Cars, Toronto CSC 321,](https://reader036.fdocuments.us/reader036/viewer/2022062917/5ed0f3a12a742537f26ea142/html5/thumbnails/72.jpg)
72
![Page 73: Machine Learning Primer - Department of Computer Science ...€¦ · Machine Learning Primer Polina Binder Slides credit: MIT Deep Learning for Self Driving Cars, Toronto CSC 321,](https://reader036.fdocuments.us/reader036/viewer/2022062917/5ed0f3a12a742537f26ea142/html5/thumbnails/73.jpg)
73
![Page 74: Machine Learning Primer - Department of Computer Science ...€¦ · Machine Learning Primer Polina Binder Slides credit: MIT Deep Learning for Self Driving Cars, Toronto CSC 321,](https://reader036.fdocuments.us/reader036/viewer/2022062917/5ed0f3a12a742537f26ea142/html5/thumbnails/74.jpg)
74
![Page 75: Machine Learning Primer - Department of Computer Science ...€¦ · Machine Learning Primer Polina Binder Slides credit: MIT Deep Learning for Self Driving Cars, Toronto CSC 321,](https://reader036.fdocuments.us/reader036/viewer/2022062917/5ed0f3a12a742537f26ea142/html5/thumbnails/75.jpg)
75
reward at time t: rt
![Page 76: Machine Learning Primer - Department of Computer Science ...€¦ · Machine Learning Primer Polina Binder Slides credit: MIT Deep Learning for Self Driving Cars, Toronto CSC 321,](https://reader036.fdocuments.us/reader036/viewer/2022062917/5ed0f3a12a742537f26ea142/html5/thumbnails/76.jpg)
76
![Page 77: Machine Learning Primer - Department of Computer Science ...€¦ · Machine Learning Primer Polina Binder Slides credit: MIT Deep Learning for Self Driving Cars, Toronto CSC 321,](https://reader036.fdocuments.us/reader036/viewer/2022062917/5ed0f3a12a742537f26ea142/html5/thumbnails/77.jpg)
77
![Page 78: Machine Learning Primer - Department of Computer Science ...€¦ · Machine Learning Primer Polina Binder Slides credit: MIT Deep Learning for Self Driving Cars, Toronto CSC 321,](https://reader036.fdocuments.us/reader036/viewer/2022062917/5ed0f3a12a742537f26ea142/html5/thumbnails/78.jpg)
78
![Page 79: Machine Learning Primer - Department of Computer Science ...€¦ · Machine Learning Primer Polina Binder Slides credit: MIT Deep Learning for Self Driving Cars, Toronto CSC 321,](https://reader036.fdocuments.us/reader036/viewer/2022062917/5ed0f3a12a742537f26ea142/html5/thumbnails/79.jpg)
What is Deep Reinforcement Learning?
● Deep reinforcement learning is standard reinforcement learning where a deep neural network is used to approximate either a policy or a value function
● Deep neural networks require lots of real/simulated interaction with the environment to learn
● Lots of trials/interactions are possible in simulated environments, as done in ADS
●
79
![Page 80: Machine Learning Primer - Department of Computer Science ...€¦ · Machine Learning Primer Polina Binder Slides credit: MIT Deep Learning for Self Driving Cars, Toronto CSC 321,](https://reader036.fdocuments.us/reader036/viewer/2022062917/5ed0f3a12a742537f26ea142/html5/thumbnails/80.jpg)
80
![Page 81: Machine Learning Primer - Department of Computer Science ...€¦ · Machine Learning Primer Polina Binder Slides credit: MIT Deep Learning for Self Driving Cars, Toronto CSC 321,](https://reader036.fdocuments.us/reader036/viewer/2022062917/5ed0f3a12a742537f26ea142/html5/thumbnails/81.jpg)
Summary● Introduction to machine learning● How machine learning is used in self-driving cars● Deep learning
○ Neural Network Basics○ Structures in Neural Networks○ Training Neural Networks○ Challenges with deep learning
● Reinforcement learning
81