Midterm Review - CS230 Deep Learningcs230.stanford.edu/fall2018/midterm_review.pdf · Midterm...
Transcript of Midterm Review - CS230 Deep Learningcs230.stanford.edu/fall2018/midterm_review.pdf · Midterm...
![Page 1: Midterm Review - CS230 Deep Learningcs230.stanford.edu/fall2018/midterm_review.pdf · Midterm Review CS230 Fall 2018. Broadcasting. Calculating Means How would you calculate the means](https://reader034.fdocuments.us/reader034/viewer/2022042711/5f7c3f0a81e59a222e5ad42d/html5/thumbnails/1.jpg)
Midterm ReviewCS230 Fall 2018
![Page 2: Midterm Review - CS230 Deep Learningcs230.stanford.edu/fall2018/midterm_review.pdf · Midterm Review CS230 Fall 2018. Broadcasting. Calculating Means How would you calculate the means](https://reader034.fdocuments.us/reader034/viewer/2022042711/5f7c3f0a81e59a222e5ad42d/html5/thumbnails/2.jpg)
Broadcasting
![Page 3: Midterm Review - CS230 Deep Learningcs230.stanford.edu/fall2018/midterm_review.pdf · Midterm Review CS230 Fall 2018. Broadcasting. Calculating Means How would you calculate the means](https://reader034.fdocuments.us/reader034/viewer/2022042711/5f7c3f0a81e59a222e5ad42d/html5/thumbnails/3.jpg)
Calculating Means
How would you calculate the means across the rows of the following matrix? How about the columns?
![Page 4: Midterm Review - CS230 Deep Learningcs230.stanford.edu/fall2018/midterm_review.pdf · Midterm Review CS230 Fall 2018. Broadcasting. Calculating Means How would you calculate the means](https://reader034.fdocuments.us/reader034/viewer/2022042711/5f7c3f0a81e59a222e5ad42d/html5/thumbnails/4.jpg)
Calculating Means
How would you calculate the means across the rows of the following matrix? How about the columns?
Rows: row_mu = np.sum(M, axis=1) / M.shape[1]
![Page 5: Midterm Review - CS230 Deep Learningcs230.stanford.edu/fall2018/midterm_review.pdf · Midterm Review CS230 Fall 2018. Broadcasting. Calculating Means How would you calculate the means](https://reader034.fdocuments.us/reader034/viewer/2022042711/5f7c3f0a81e59a222e5ad42d/html5/thumbnails/5.jpg)
Calculating Means
How would you calculate the means across the rows of the following matrix? How about the columns?
Rows: row_mu = np.sum(M, axis=1) / M.shape[1]Cols: col_mu = np.sum(M, axis=0) / M.shape[0]
![Page 6: Midterm Review - CS230 Deep Learningcs230.stanford.edu/fall2018/midterm_review.pdf · Midterm Review CS230 Fall 2018. Broadcasting. Calculating Means How would you calculate the means](https://reader034.fdocuments.us/reader034/viewer/2022042711/5f7c3f0a81e59a222e5ad42d/html5/thumbnails/6.jpg)
Computing Softmax
How would you compute the softmax across the columns of the following matrix?
![Page 7: Midterm Review - CS230 Deep Learningcs230.stanford.edu/fall2018/midterm_review.pdf · Midterm Review CS230 Fall 2018. Broadcasting. Calculating Means How would you calculate the means](https://reader034.fdocuments.us/reader034/viewer/2022042711/5f7c3f0a81e59a222e5ad42d/html5/thumbnails/7.jpg)
Computing Softmax
How would you compute the softmax across the columns of the following matrix?
exp = np.exp(M)
![Page 8: Midterm Review - CS230 Deep Learningcs230.stanford.edu/fall2018/midterm_review.pdf · Midterm Review CS230 Fall 2018. Broadcasting. Calculating Means How would you calculate the means](https://reader034.fdocuments.us/reader034/viewer/2022042711/5f7c3f0a81e59a222e5ad42d/html5/thumbnails/8.jpg)
Computing Softmax
How would you compute the softmax across the columns of the following matrix?
exp = np.exp(M)
smx = exp / np.sum(exp, axis=0)
![Page 9: Midterm Review - CS230 Deep Learningcs230.stanford.edu/fall2018/midterm_review.pdf · Midterm Review CS230 Fall 2018. Broadcasting. Calculating Means How would you calculate the means](https://reader034.fdocuments.us/reader034/viewer/2022042711/5f7c3f0a81e59a222e5ad42d/html5/thumbnails/9.jpg)
Computing Distances
How would you compute the closest column in the matrix X to the vector V (in terms of Euclidean distance)?
![Page 10: Midterm Review - CS230 Deep Learningcs230.stanford.edu/fall2018/midterm_review.pdf · Midterm Review CS230 Fall 2018. Broadcasting. Calculating Means How would you calculate the means](https://reader034.fdocuments.us/reader034/viewer/2022042711/5f7c3f0a81e59a222e5ad42d/html5/thumbnails/10.jpg)
![Page 11: Midterm Review - CS230 Deep Learningcs230.stanford.edu/fall2018/midterm_review.pdf · Midterm Review CS230 Fall 2018. Broadcasting. Calculating Means How would you calculate the means](https://reader034.fdocuments.us/reader034/viewer/2022042711/5f7c3f0a81e59a222e5ad42d/html5/thumbnails/11.jpg)
Computing Distances
How would you compute the closest column in the matrix X to the vector V (in terms of Euclidean distance)?
![Page 12: Midterm Review - CS230 Deep Learningcs230.stanford.edu/fall2018/midterm_review.pdf · Midterm Review CS230 Fall 2018. Broadcasting. Calculating Means How would you calculate the means](https://reader034.fdocuments.us/reader034/viewer/2022042711/5f7c3f0a81e59a222e5ad42d/html5/thumbnails/12.jpg)
Computing Distances
How would you compute the closest column in the matrix X to the vector V (in terms of Euclidean distance)?
sq_diff = np.square(X-V)
![Page 13: Midterm Review - CS230 Deep Learningcs230.stanford.edu/fall2018/midterm_review.pdf · Midterm Review CS230 Fall 2018. Broadcasting. Calculating Means How would you calculate the means](https://reader034.fdocuments.us/reader034/viewer/2022042711/5f7c3f0a81e59a222e5ad42d/html5/thumbnails/13.jpg)
Computing Distances
How would you compute the closest column in the matrix X to the vector V (in terms of Euclidean distance)?
sq_diff = np.square(X-V)dists = np.sqrt(np.sum(sq_diff, axis=0))
![Page 14: Midterm Review - CS230 Deep Learningcs230.stanford.edu/fall2018/midterm_review.pdf · Midterm Review CS230 Fall 2018. Broadcasting. Calculating Means How would you calculate the means](https://reader034.fdocuments.us/reader034/viewer/2022042711/5f7c3f0a81e59a222e5ad42d/html5/thumbnails/14.jpg)
Computing Distances
How would you compute the closest column in the matrix X to the vector V (in terms of Euclidean distance)?
sq_diff = np.square(X-V)dists = np.sqrt(np.sum(sq_diff, axis=0))nearest = np.argmin(dists)
![Page 15: Midterm Review - CS230 Deep Learningcs230.stanford.edu/fall2018/midterm_review.pdf · Midterm Review CS230 Fall 2018. Broadcasting. Calculating Means How would you calculate the means](https://reader034.fdocuments.us/reader034/viewer/2022042711/5f7c3f0a81e59a222e5ad42d/html5/thumbnails/15.jpg)
L1/L2 Regularization
![Page 16: Midterm Review - CS230 Deep Learningcs230.stanford.edu/fall2018/midterm_review.pdf · Midterm Review CS230 Fall 2018. Broadcasting. Calculating Means How would you calculate the means](https://reader034.fdocuments.us/reader034/viewer/2022042711/5f7c3f0a81e59a222e5ad42d/html5/thumbnails/16.jpg)
Logistic Regression and Separable Data
What’s the issue with training a logistic regression model on the following data?
![Page 17: Midterm Review - CS230 Deep Learningcs230.stanford.edu/fall2018/midterm_review.pdf · Midterm Review CS230 Fall 2018. Broadcasting. Calculating Means How would you calculate the means](https://reader034.fdocuments.us/reader034/viewer/2022042711/5f7c3f0a81e59a222e5ad42d/html5/thumbnails/17.jpg)
![Page 18: Midterm Review - CS230 Deep Learningcs230.stanford.edu/fall2018/midterm_review.pdf · Midterm Review CS230 Fall 2018. Broadcasting. Calculating Means How would you calculate the means](https://reader034.fdocuments.us/reader034/viewer/2022042711/5f7c3f0a81e59a222e5ad42d/html5/thumbnails/18.jpg)
Logistic Regression and Separable Data
What’s the issue with training a logistic regression model on the following data?
![Page 19: Midterm Review - CS230 Deep Learningcs230.stanford.edu/fall2018/midterm_review.pdf · Midterm Review CS230 Fall 2018. Broadcasting. Calculating Means How would you calculate the means](https://reader034.fdocuments.us/reader034/viewer/2022042711/5f7c3f0a81e59a222e5ad42d/html5/thumbnails/19.jpg)
Logistic Regression and Separable Data
What’s the issue with training a logistic regression model on the following data?
The parameters will tend to plus/minus infinity! So, it will never converge.
![Page 20: Midterm Review - CS230 Deep Learningcs230.stanford.edu/fall2018/midterm_review.pdf · Midterm Review CS230 Fall 2018. Broadcasting. Calculating Means How would you calculate the means](https://reader034.fdocuments.us/reader034/viewer/2022042711/5f7c3f0a81e59a222e5ad42d/html5/thumbnails/20.jpg)
Solving the Exploding Weights Issue
What modification of the loss function can you implement so solve this issue? Write out the new loss function.
![Page 21: Midterm Review - CS230 Deep Learningcs230.stanford.edu/fall2018/midterm_review.pdf · Midterm Review CS230 Fall 2018. Broadcasting. Calculating Means How would you calculate the means](https://reader034.fdocuments.us/reader034/viewer/2022042711/5f7c3f0a81e59a222e5ad42d/html5/thumbnails/21.jpg)
Solving the Exploding Weights Issue
What modification of the loss function can you implement so solve this issue? Write out the new loss function.
Add L2 Regularization
Lkj
This new loss function will keep the magnitude of the weights from exploding!
![Page 22: Midterm Review - CS230 Deep Learningcs230.stanford.edu/fall2018/midterm_review.pdf · Midterm Review CS230 Fall 2018. Broadcasting. Calculating Means How would you calculate the means](https://reader034.fdocuments.us/reader034/viewer/2022042711/5f7c3f0a81e59a222e5ad42d/html5/thumbnails/22.jpg)
Gradient of the New Loss
Compute the gradient of the weight vector with respect to this new loss function.
![Page 23: Midterm Review - CS230 Deep Learningcs230.stanford.edu/fall2018/midterm_review.pdf · Midterm Review CS230 Fall 2018. Broadcasting. Calculating Means How would you calculate the means](https://reader034.fdocuments.us/reader034/viewer/2022042711/5f7c3f0a81e59a222e5ad42d/html5/thumbnails/23.jpg)
Gradient of the New Loss
Compute the gradient of the new loss function with respect to the weight vector.
![Page 24: Midterm Review - CS230 Deep Learningcs230.stanford.edu/fall2018/midterm_review.pdf · Midterm Review CS230 Fall 2018. Broadcasting. Calculating Means How would you calculate the means](https://reader034.fdocuments.us/reader034/viewer/2022042711/5f7c3f0a81e59a222e5ad42d/html5/thumbnails/24.jpg)
Another Solution...
What is another, similar modification to the loss function that could help with this issue? Compute its gradient?
![Page 25: Midterm Review - CS230 Deep Learningcs230.stanford.edu/fall2018/midterm_review.pdf · Midterm Review CS230 Fall 2018. Broadcasting. Calculating Means How would you calculate the means](https://reader034.fdocuments.us/reader034/viewer/2022042711/5f7c3f0a81e59a222e5ad42d/html5/thumbnails/25.jpg)
Another Solution...
What is another, similar modification to the loss function that could help with this issue? Compute its gradient?
Add L1 Regularization:
![Page 26: Midterm Review - CS230 Deep Learningcs230.stanford.edu/fall2018/midterm_review.pdf · Midterm Review CS230 Fall 2018. Broadcasting. Calculating Means How would you calculate the means](https://reader034.fdocuments.us/reader034/viewer/2022042711/5f7c3f0a81e59a222e5ad42d/html5/thumbnails/26.jpg)
Backprop
![Page 27: Midterm Review - CS230 Deep Learningcs230.stanford.edu/fall2018/midterm_review.pdf · Midterm Review CS230 Fall 2018. Broadcasting. Calculating Means How would you calculate the means](https://reader034.fdocuments.us/reader034/viewer/2022042711/5f7c3f0a81e59a222e5ad42d/html5/thumbnails/27.jpg)
![Page 28: Midterm Review - CS230 Deep Learningcs230.stanford.edu/fall2018/midterm_review.pdf · Midterm Review CS230 Fall 2018. Broadcasting. Calculating Means How would you calculate the means](https://reader034.fdocuments.us/reader034/viewer/2022042711/5f7c3f0a81e59a222e5ad42d/html5/thumbnails/28.jpg)
![Page 29: Midterm Review - CS230 Deep Learningcs230.stanford.edu/fall2018/midterm_review.pdf · Midterm Review CS230 Fall 2018. Broadcasting. Calculating Means How would you calculate the means](https://reader034.fdocuments.us/reader034/viewer/2022042711/5f7c3f0a81e59a222e5ad42d/html5/thumbnails/29.jpg)
CNN Input/Output Sizes
![Page 30: Midterm Review - CS230 Deep Learningcs230.stanford.edu/fall2018/midterm_review.pdf · Midterm Review CS230 Fall 2018. Broadcasting. Calculating Means How would you calculate the means](https://reader034.fdocuments.us/reader034/viewer/2022042711/5f7c3f0a81e59a222e5ad42d/html5/thumbnails/30.jpg)
Basic (no padding, stride 1)
Input Filter
![Page 31: Midterm Review - CS230 Deep Learningcs230.stanford.edu/fall2018/midterm_review.pdf · Midterm Review CS230 Fall 2018. Broadcasting. Calculating Means How would you calculate the means](https://reader034.fdocuments.us/reader034/viewer/2022042711/5f7c3f0a81e59a222e5ad42d/html5/thumbnails/31.jpg)
Basic
![Page 32: Midterm Review - CS230 Deep Learningcs230.stanford.edu/fall2018/midterm_review.pdf · Midterm Review CS230 Fall 2018. Broadcasting. Calculating Means How would you calculate the means](https://reader034.fdocuments.us/reader034/viewer/2022042711/5f7c3f0a81e59a222e5ad42d/html5/thumbnails/32.jpg)
Basic
![Page 33: Midterm Review - CS230 Deep Learningcs230.stanford.edu/fall2018/midterm_review.pdf · Midterm Review CS230 Fall 2018. Broadcasting. Calculating Means How would you calculate the means](https://reader034.fdocuments.us/reader034/viewer/2022042711/5f7c3f0a81e59a222e5ad42d/html5/thumbnails/33.jpg)
Basic
![Page 34: Midterm Review - CS230 Deep Learningcs230.stanford.edu/fall2018/midterm_review.pdf · Midterm Review CS230 Fall 2018. Broadcasting. Calculating Means How would you calculate the means](https://reader034.fdocuments.us/reader034/viewer/2022042711/5f7c3f0a81e59a222e5ad42d/html5/thumbnails/34.jpg)
Basic
![Page 35: Midterm Review - CS230 Deep Learningcs230.stanford.edu/fall2018/midterm_review.pdf · Midterm Review CS230 Fall 2018. Broadcasting. Calculating Means How would you calculate the means](https://reader034.fdocuments.us/reader034/viewer/2022042711/5f7c3f0a81e59a222e5ad42d/html5/thumbnails/35.jpg)
Basic
![Page 36: Midterm Review - CS230 Deep Learningcs230.stanford.edu/fall2018/midterm_review.pdf · Midterm Review CS230 Fall 2018. Broadcasting. Calculating Means How would you calculate the means](https://reader034.fdocuments.us/reader034/viewer/2022042711/5f7c3f0a81e59a222e5ad42d/html5/thumbnails/36.jpg)
Basic
![Page 37: Midterm Review - CS230 Deep Learningcs230.stanford.edu/fall2018/midterm_review.pdf · Midterm Review CS230 Fall 2018. Broadcasting. Calculating Means How would you calculate the means](https://reader034.fdocuments.us/reader034/viewer/2022042711/5f7c3f0a81e59a222e5ad42d/html5/thumbnails/37.jpg)
Basic
![Page 38: Midterm Review - CS230 Deep Learningcs230.stanford.edu/fall2018/midterm_review.pdf · Midterm Review CS230 Fall 2018. Broadcasting. Calculating Means How would you calculate the means](https://reader034.fdocuments.us/reader034/viewer/2022042711/5f7c3f0a81e59a222e5ad42d/html5/thumbnails/38.jpg)
Basic
![Page 39: Midterm Review - CS230 Deep Learningcs230.stanford.edu/fall2018/midterm_review.pdf · Midterm Review CS230 Fall 2018. Broadcasting. Calculating Means How would you calculate the means](https://reader034.fdocuments.us/reader034/viewer/2022042711/5f7c3f0a81e59a222e5ad42d/html5/thumbnails/39.jpg)
Basic
Input Filter Conv Output
![Page 40: Midterm Review - CS230 Deep Learningcs230.stanford.edu/fall2018/midterm_review.pdf · Midterm Review CS230 Fall 2018. Broadcasting. Calculating Means How would you calculate the means](https://reader034.fdocuments.us/reader034/viewer/2022042711/5f7c3f0a81e59a222e5ad42d/html5/thumbnails/40.jpg)
Basic
Input Filter Conv Output
Shape = n - f + 1
![Page 41: Midterm Review - CS230 Deep Learningcs230.stanford.edu/fall2018/midterm_review.pdf · Midterm Review CS230 Fall 2018. Broadcasting. Calculating Means How would you calculate the means](https://reader034.fdocuments.us/reader034/viewer/2022042711/5f7c3f0a81e59a222e5ad42d/html5/thumbnails/41.jpg)
Padding0 0 0 0 0 0 0 0
0 0
0 0
0 0
0 0
0 0
0 0
0 0 0 0 0 0 0 0
Input Filter
![Page 42: Midterm Review - CS230 Deep Learningcs230.stanford.edu/fall2018/midterm_review.pdf · Midterm Review CS230 Fall 2018. Broadcasting. Calculating Means How would you calculate the means](https://reader034.fdocuments.us/reader034/viewer/2022042711/5f7c3f0a81e59a222e5ad42d/html5/thumbnails/42.jpg)
Padding0 0 0 0 0 0 0 0
0 0
0 0
0 0
0 0
0 0
0 0
0 0 0 0 0 0 0 0
Input Filter Conv Output
![Page 43: Midterm Review - CS230 Deep Learningcs230.stanford.edu/fall2018/midterm_review.pdf · Midterm Review CS230 Fall 2018. Broadcasting. Calculating Means How would you calculate the means](https://reader034.fdocuments.us/reader034/viewer/2022042711/5f7c3f0a81e59a222e5ad42d/html5/thumbnails/43.jpg)
Padding0 0 0 0 0 0 0 0
0 0
0 0
0 0
0 0
0 0
0 0
0 0 0 0 0 0 0 0
Input Filter Conv Output
Shape = n + 2p- f + 1
![Page 44: Midterm Review - CS230 Deep Learningcs230.stanford.edu/fall2018/midterm_review.pdf · Midterm Review CS230 Fall 2018. Broadcasting. Calculating Means How would you calculate the means](https://reader034.fdocuments.us/reader034/viewer/2022042711/5f7c3f0a81e59a222e5ad42d/html5/thumbnails/44.jpg)
Valid and Same Convolutions
● Valid ○ No padding○ Output shape -> n - f + 1
● Same ○ Pad so that input is same as output size○ Output shape -> n + 2p - f + 1
![Page 45: Midterm Review - CS230 Deep Learningcs230.stanford.edu/fall2018/midterm_review.pdf · Midterm Review CS230 Fall 2018. Broadcasting. Calculating Means How would you calculate the means](https://reader034.fdocuments.us/reader034/viewer/2022042711/5f7c3f0a81e59a222e5ad42d/html5/thumbnails/45.jpg)
Stride
Input Filter
![Page 46: Midterm Review - CS230 Deep Learningcs230.stanford.edu/fall2018/midterm_review.pdf · Midterm Review CS230 Fall 2018. Broadcasting. Calculating Means How would you calculate the means](https://reader034.fdocuments.us/reader034/viewer/2022042711/5f7c3f0a81e59a222e5ad42d/html5/thumbnails/46.jpg)
Stride
![Page 47: Midterm Review - CS230 Deep Learningcs230.stanford.edu/fall2018/midterm_review.pdf · Midterm Review CS230 Fall 2018. Broadcasting. Calculating Means How would you calculate the means](https://reader034.fdocuments.us/reader034/viewer/2022042711/5f7c3f0a81e59a222e5ad42d/html5/thumbnails/47.jpg)
Stride
![Page 48: Midterm Review - CS230 Deep Learningcs230.stanford.edu/fall2018/midterm_review.pdf · Midterm Review CS230 Fall 2018. Broadcasting. Calculating Means How would you calculate the means](https://reader034.fdocuments.us/reader034/viewer/2022042711/5f7c3f0a81e59a222e5ad42d/html5/thumbnails/48.jpg)
Stride
![Page 49: Midterm Review - CS230 Deep Learningcs230.stanford.edu/fall2018/midterm_review.pdf · Midterm Review CS230 Fall 2018. Broadcasting. Calculating Means How would you calculate the means](https://reader034.fdocuments.us/reader034/viewer/2022042711/5f7c3f0a81e59a222e5ad42d/html5/thumbnails/49.jpg)
Stride
![Page 50: Midterm Review - CS230 Deep Learningcs230.stanford.edu/fall2018/midterm_review.pdf · Midterm Review CS230 Fall 2018. Broadcasting. Calculating Means How would you calculate the means](https://reader034.fdocuments.us/reader034/viewer/2022042711/5f7c3f0a81e59a222e5ad42d/html5/thumbnails/50.jpg)
Basic
Input Filter Conv Output
Shape = (n - f)/s + 1
![Page 51: Midterm Review - CS230 Deep Learningcs230.stanford.edu/fall2018/midterm_review.pdf · Midterm Review CS230 Fall 2018. Broadcasting. Calculating Means How would you calculate the means](https://reader034.fdocuments.us/reader034/viewer/2022042711/5f7c3f0a81e59a222e5ad42d/html5/thumbnails/51.jpg)
With Stride
n x n image
f x f filter
p padding
s stride
Output size -> (n + 2p - f)/s + 1
![Page 52: Midterm Review - CS230 Deep Learningcs230.stanford.edu/fall2018/midterm_review.pdf · Midterm Review CS230 Fall 2018. Broadcasting. Calculating Means How would you calculate the means](https://reader034.fdocuments.us/reader034/viewer/2022042711/5f7c3f0a81e59a222e5ad42d/html5/thumbnails/52.jpg)
Maxpool
![Page 53: Midterm Review - CS230 Deep Learningcs230.stanford.edu/fall2018/midterm_review.pdf · Midterm Review CS230 Fall 2018. Broadcasting. Calculating Means How would you calculate the means](https://reader034.fdocuments.us/reader034/viewer/2022042711/5f7c3f0a81e59a222e5ad42d/html5/thumbnails/53.jpg)
Forward prop
1 3 2 1
4 10 5 1
1 6 6 5
2 4 2 9
2 x 2 Pooling layer with stride 2
Input
![Page 54: Midterm Review - CS230 Deep Learningcs230.stanford.edu/fall2018/midterm_review.pdf · Midterm Review CS230 Fall 2018. Broadcasting. Calculating Means How would you calculate the means](https://reader034.fdocuments.us/reader034/viewer/2022042711/5f7c3f0a81e59a222e5ad42d/html5/thumbnails/54.jpg)
Forward prop
1 3 2 1
4 10 5 1
1 6 6 5
2 4 2 9
2 x 2 Pooling layer with stride 2
Input Output
Size of output (n-f)/s + 1
![Page 55: Midterm Review - CS230 Deep Learningcs230.stanford.edu/fall2018/midterm_review.pdf · Midterm Review CS230 Fall 2018. Broadcasting. Calculating Means How would you calculate the means](https://reader034.fdocuments.us/reader034/viewer/2022042711/5f7c3f0a81e59a222e5ad42d/html5/thumbnails/55.jpg)
Forward prop
1 3 2 1
4 10 5 1
1 6 6 5
2 4 2 9
2 x 2 Pooling layer with stride 2
Input
10
Output
![Page 56: Midterm Review - CS230 Deep Learningcs230.stanford.edu/fall2018/midterm_review.pdf · Midterm Review CS230 Fall 2018. Broadcasting. Calculating Means How would you calculate the means](https://reader034.fdocuments.us/reader034/viewer/2022042711/5f7c3f0a81e59a222e5ad42d/html5/thumbnails/56.jpg)
Forward prop
1 3 2 1
4 10 5 1
1 6 6 5
2 4 2 9
2 x 2 Pooling layer with stride 2
Input
10 5
Output
![Page 57: Midterm Review - CS230 Deep Learningcs230.stanford.edu/fall2018/midterm_review.pdf · Midterm Review CS230 Fall 2018. Broadcasting. Calculating Means How would you calculate the means](https://reader034.fdocuments.us/reader034/viewer/2022042711/5f7c3f0a81e59a222e5ad42d/html5/thumbnails/57.jpg)
Forward prop
1 3 2 1
4 10 5 1
1 6 6 5
2 4 2 9
2 x 2 Pooling layer with stride 2
Input
10 5
6
Output
![Page 58: Midterm Review - CS230 Deep Learningcs230.stanford.edu/fall2018/midterm_review.pdf · Midterm Review CS230 Fall 2018. Broadcasting. Calculating Means How would you calculate the means](https://reader034.fdocuments.us/reader034/viewer/2022042711/5f7c3f0a81e59a222e5ad42d/html5/thumbnails/58.jpg)
Forward prop
1 3 2 1
4 10 5 1
1 6 6 5
2 4 2 9
2 x 2 Pooling layer with stride 2
Input
10 5
6 9
Output
![Page 59: Midterm Review - CS230 Deep Learningcs230.stanford.edu/fall2018/midterm_review.pdf · Midterm Review CS230 Fall 2018. Broadcasting. Calculating Means How would you calculate the means](https://reader034.fdocuments.us/reader034/viewer/2022042711/5f7c3f0a81e59a222e5ad42d/html5/thumbnails/59.jpg)
Forward prop
1 3 2 1
4 10 5 1
1 6 6 5
2 4 2 9
2 x 2 Pooling layer with stride 2
Input
10 5
6 9
Output
![Page 60: Midterm Review - CS230 Deep Learningcs230.stanford.edu/fall2018/midterm_review.pdf · Midterm Review CS230 Fall 2018. Broadcasting. Calculating Means How would you calculate the means](https://reader034.fdocuments.us/reader034/viewer/2022042711/5f7c3f0a81e59a222e5ad42d/html5/thumbnails/60.jpg)
Backprop
1 3 2 1
4 10 5 1
1 6 6 5
2 4 2 9
Input to maxpool layer
10 5
6 9
Output of maxpool layer
-4 7
5 -6
Gradient w.r.t output
![Page 61: Midterm Review - CS230 Deep Learningcs230.stanford.edu/fall2018/midterm_review.pdf · Midterm Review CS230 Fall 2018. Broadcasting. Calculating Means How would you calculate the means](https://reader034.fdocuments.us/reader034/viewer/2022042711/5f7c3f0a81e59a222e5ad42d/html5/thumbnails/61.jpg)
Backprop
? ? ? ?
? ? ? ?
? ? ? ?
? ? ? ?
Gradient w.r.t input
-4 7
5 -6
Gradient w.r.t output
![Page 62: Midterm Review - CS230 Deep Learningcs230.stanford.edu/fall2018/midterm_review.pdf · Midterm Review CS230 Fall 2018. Broadcasting. Calculating Means How would you calculate the means](https://reader034.fdocuments.us/reader034/viewer/2022042711/5f7c3f0a81e59a222e5ad42d/html5/thumbnails/62.jpg)
ReLU
![Page 63: Midterm Review - CS230 Deep Learningcs230.stanford.edu/fall2018/midterm_review.pdf · Midterm Review CS230 Fall 2018. Broadcasting. Calculating Means How would you calculate the means](https://reader034.fdocuments.us/reader034/viewer/2022042711/5f7c3f0a81e59a222e5ad42d/html5/thumbnails/63.jpg)
Maxpool
if mij ≠ max(m) Maxpool(mij) Maxpool(mij)
Maxpool(mij) if mij ≠ max(m)
if mij = max(m)
if mij = max(m)
![Page 64: Midterm Review - CS230 Deep Learningcs230.stanford.edu/fall2018/midterm_review.pdf · Midterm Review CS230 Fall 2018. Broadcasting. Calculating Means How would you calculate the means](https://reader034.fdocuments.us/reader034/viewer/2022042711/5f7c3f0a81e59a222e5ad42d/html5/thumbnails/64.jpg)
Backprop
1 3 2 1
4 10 5 1
1 6 6 5
2 4 2 9
Keep track of where the maximum value is
Input
10 5
6 9
Output
0 0 0 0
0 1 1 0
0 1 0 0
0 0 0 1
Mask
![Page 65: Midterm Review - CS230 Deep Learningcs230.stanford.edu/fall2018/midterm_review.pdf · Midterm Review CS230 Fall 2018. Broadcasting. Calculating Means How would you calculate the means](https://reader034.fdocuments.us/reader034/viewer/2022042711/5f7c3f0a81e59a222e5ad42d/html5/thumbnails/65.jpg)
Backprop
0 0 0 0
0 1 1 0
0 1 0 0
0 0 0 1
Mask
-4 7
5 -6
Gradient w.r.t output
0 0 0 0
0 -4 5 0
0 5 0 0
0 0 0 -6
Gradient w.r.t input
![Page 66: Midterm Review - CS230 Deep Learningcs230.stanford.edu/fall2018/midterm_review.pdf · Midterm Review CS230 Fall 2018. Broadcasting. Calculating Means How would you calculate the means](https://reader034.fdocuments.us/reader034/viewer/2022042711/5f7c3f0a81e59a222e5ad42d/html5/thumbnails/66.jpg)
Error Analysis
![Page 67: Midterm Review - CS230 Deep Learningcs230.stanford.edu/fall2018/midterm_review.pdf · Midterm Review CS230 Fall 2018. Broadcasting. Calculating Means How would you calculate the means](https://reader034.fdocuments.us/reader034/viewer/2022042711/5f7c3f0a81e59a222e5ad42d/html5/thumbnails/67.jpg)
Dog Classifier
Trying to predict dog vs not dog.
![Page 68: Midterm Review - CS230 Deep Learningcs230.stanford.edu/fall2018/midterm_review.pdf · Midterm Review CS230 Fall 2018. Broadcasting. Calculating Means How would you calculate the means](https://reader034.fdocuments.us/reader034/viewer/2022042711/5f7c3f0a81e59a222e5ad42d/html5/thumbnails/68.jpg)
Improving performance
Two kinds of errors - misclassification on muffins and fried chicken
https://medium.freecodecamp.org/chihuahua-or-muffin-my-search-for-the-best-computer-vision-api-cbda4d6b425d
https://barkpost.com/doodle-or-fried-chicken-twitter/
![Page 69: Midterm Review - CS230 Deep Learningcs230.stanford.edu/fall2018/midterm_review.pdf · Midterm Review CS230 Fall 2018. Broadcasting. Calculating Means How would you calculate the means](https://reader034.fdocuments.us/reader034/viewer/2022042711/5f7c3f0a81e59a222e5ad42d/html5/thumbnails/69.jpg)
Error analysis
- Get 100 examples on dev set
Image number Classified as muffin
Classified as chicken
... Comments
1 Y -
2 - Y
... - -
100 - Y
5% 50%
![Page 70: Midterm Review - CS230 Deep Learningcs230.stanford.edu/fall2018/midterm_review.pdf · Midterm Review CS230 Fall 2018. Broadcasting. Calculating Means How would you calculate the means](https://reader034.fdocuments.us/reader034/viewer/2022042711/5f7c3f0a81e59a222e5ad42d/html5/thumbnails/70.jpg)
Error Analysis
![Page 71: Midterm Review - CS230 Deep Learningcs230.stanford.edu/fall2018/midterm_review.pdf · Midterm Review CS230 Fall 2018. Broadcasting. Calculating Means How would you calculate the means](https://reader034.fdocuments.us/reader034/viewer/2022042711/5f7c3f0a81e59a222e5ad42d/html5/thumbnails/71.jpg)
Strategic Data Acquisition
![Page 72: Midterm Review - CS230 Deep Learningcs230.stanford.edu/fall2018/midterm_review.pdf · Midterm Review CS230 Fall 2018. Broadcasting. Calculating Means How would you calculate the means](https://reader034.fdocuments.us/reader034/viewer/2022042711/5f7c3f0a81e59a222e5ad42d/html5/thumbnails/72.jpg)
Trigger Word Detection
![Page 73: Midterm Review - CS230 Deep Learningcs230.stanford.edu/fall2018/midterm_review.pdf · Midterm Review CS230 Fall 2018. Broadcasting. Calculating Means How would you calculate the means](https://reader034.fdocuments.us/reader034/viewer/2022042711/5f7c3f0a81e59a222e5ad42d/html5/thumbnails/73.jpg)
Classification
![Page 74: Midterm Review - CS230 Deep Learningcs230.stanford.edu/fall2018/midterm_review.pdf · Midterm Review CS230 Fall 2018. Broadcasting. Calculating Means How would you calculate the means](https://reader034.fdocuments.us/reader034/viewer/2022042711/5f7c3f0a81e59a222e5ad42d/html5/thumbnails/74.jpg)
Dropout
![Page 75: Midterm Review - CS230 Deep Learningcs230.stanford.edu/fall2018/midterm_review.pdf · Midterm Review CS230 Fall 2018. Broadcasting. Calculating Means How would you calculate the means](https://reader034.fdocuments.us/reader034/viewer/2022042711/5f7c3f0a81e59a222e5ad42d/html5/thumbnails/75.jpg)
![Page 76: Midterm Review - CS230 Deep Learningcs230.stanford.edu/fall2018/midterm_review.pdf · Midterm Review CS230 Fall 2018. Broadcasting. Calculating Means How would you calculate the means](https://reader034.fdocuments.us/reader034/viewer/2022042711/5f7c3f0a81e59a222e5ad42d/html5/thumbnails/76.jpg)
![Page 77: Midterm Review - CS230 Deep Learningcs230.stanford.edu/fall2018/midterm_review.pdf · Midterm Review CS230 Fall 2018. Broadcasting. Calculating Means How would you calculate the means](https://reader034.fdocuments.us/reader034/viewer/2022042711/5f7c3f0a81e59a222e5ad42d/html5/thumbnails/77.jpg)
Batchnorm
![Page 78: Midterm Review - CS230 Deep Learningcs230.stanford.edu/fall2018/midterm_review.pdf · Midterm Review CS230 Fall 2018. Broadcasting. Calculating Means How would you calculate the means](https://reader034.fdocuments.us/reader034/viewer/2022042711/5f7c3f0a81e59a222e5ad42d/html5/thumbnails/78.jpg)