Final Exam Reviemgormley/courses/10601bd-f18/slides/final-revie… · Final Exam • Time /...
Transcript of Final Exam Reviemgormley/courses/10601bd-f18/slides/final-revie… · Final Exam • Time /...
![Page 1: Final Exam Reviemgormley/courses/10601bd-f18/slides/final-revie… · Final Exam • Time / Location – Time: Evening Exam Thu, Dec 13 at 1:00pm –4:00pm – Room: We will contact](https://reader036.fdocuments.us/reader036/viewer/2022070822/5f28779a55ef57646920a36a/html5/thumbnails/1.jpg)
Final Exam Review
1
10-601 Introduction to Machine Learning
Matt GormleyLecture 31
May 2, 2018
Machine Learning DepartmentSchool of Computer ScienceCarnegie Mellon University
![Page 2: Final Exam Reviemgormley/courses/10601bd-f18/slides/final-revie… · Final Exam • Time / Location – Time: Evening Exam Thu, Dec 13 at 1:00pm –4:00pm – Room: We will contact](https://reader036.fdocuments.us/reader036/viewer/2022070822/5f28779a55ef57646920a36a/html5/thumbnails/2.jpg)
Reminders
• Homework 9: Learning Paradigms
– Out: Fri, Nov 30
– Due: Fri, Dec 7 at 11:59pm
2
![Page 3: Final Exam Reviemgormley/courses/10601bd-f18/slides/final-revie… · Final Exam • Time / Location – Time: Evening Exam Thu, Dec 13 at 1:00pm –4:00pm – Room: We will contact](https://reader036.fdocuments.us/reader036/viewer/2022070822/5f28779a55ef57646920a36a/html5/thumbnails/3.jpg)
Outline
1. Exam Logistics
2. Sample Questions
3. Overview
3
![Page 4: Final Exam Reviemgormley/courses/10601bd-f18/slides/final-revie… · Final Exam • Time / Location – Time: Evening Exam Thu, Dec 13 at 1:00pm –4:00pm – Room: We will contact](https://reader036.fdocuments.us/reader036/viewer/2022070822/5f28779a55ef57646920a36a/html5/thumbnails/4.jpg)
EXAM LOGISTICS
4
![Page 5: Final Exam Reviemgormley/courses/10601bd-f18/slides/final-revie… · Final Exam • Time / Location – Time: Evening Exam Thu, Dec 13 at 1:00pm –4:00pm – Room: We will contact](https://reader036.fdocuments.us/reader036/viewer/2022070822/5f28779a55ef57646920a36a/html5/thumbnails/5.jpg)
Final Exam
• Time / Location– Time: Evening Exam
Thu, Dec 13 at 1:00pm – 4:00pm– Room: We will contact each student individually with your room assignment. The
rooms are not based on section. – Seats: There will be assigned seats. Please arrive early. – Please watch Piazza carefully for announcements regarding room / seat
assignments.
• Logistics– Format of questions:
• Multiple choice• True / False (with justification)• Derivations• Short answers• Interpreting figures• Implementing algorithms on paper
– No electronic devices– You are allowed to bring one 8½ x 11 sheet of notes (front and back)
5
![Page 6: Final Exam Reviemgormley/courses/10601bd-f18/slides/final-revie… · Final Exam • Time / Location – Time: Evening Exam Thu, Dec 13 at 1:00pm –4:00pm – Room: We will contact](https://reader036.fdocuments.us/reader036/viewer/2022070822/5f28779a55ef57646920a36a/html5/thumbnails/6.jpg)
Final Exam
• How to Prepare
– Attend (or watch) this final exam review session
– Review prior year’s exams and solutions• We already posted these for the midterm
• Disclaimer: This year’s 10-601 is not the same as prior offerings, so review both midterm and final
– Review this year’s homework problems
– Consider whether you have achieved the “learning objectives” for each lecture / section
– Attend the Final Exam Recitation (Friday)
6
![Page 7: Final Exam Reviemgormley/courses/10601bd-f18/slides/final-revie… · Final Exam • Time / Location – Time: Evening Exam Thu, Dec 13 at 1:00pm –4:00pm – Room: We will contact](https://reader036.fdocuments.us/reader036/viewer/2022070822/5f28779a55ef57646920a36a/html5/thumbnails/7.jpg)
Final Exam
• Advice (for during the exam)– Solve the easy problems first
(e.g. multiple choice before derivations)• if a problem seems extremely complicated you’re likely
missing something
– Don’t leave any answer blank!
– If you make an assumption, write it down
– If you look at a question and don’t know the answer:• we probably haven’t told you the answer
• but we’ve told you enough to work it out
• imagine arguing for some answer and see if you like it
7
![Page 8: Final Exam Reviemgormley/courses/10601bd-f18/slides/final-revie… · Final Exam • Time / Location – Time: Evening Exam Thu, Dec 13 at 1:00pm –4:00pm – Room: We will contact](https://reader036.fdocuments.us/reader036/viewer/2022070822/5f28779a55ef57646920a36a/html5/thumbnails/8.jpg)
Final Exam
• Exam Contents
– ~20% of material comes from topics covered before the midterm exam
– ~80% of material comes from topics covered after the midterm exam
8
![Page 9: Final Exam Reviemgormley/courses/10601bd-f18/slides/final-revie… · Final Exam • Time / Location – Time: Evening Exam Thu, Dec 13 at 1:00pm –4:00pm – Room: We will contact](https://reader036.fdocuments.us/reader036/viewer/2022070822/5f28779a55ef57646920a36a/html5/thumbnails/9.jpg)
Topics covered before Midterm
• Foundations– Probability, Linear Algebra,
Geometry, Calculus
– MLE
– Optimization
• Important Concepts– Regularization and
Overfitting
– Experimental Design
• Classifiers– Decision Tree
– KNN
– Perceptron
– Logistic Regression
• Regression– Linear Regression
• Feature Learning– Neural Networks
– Basic NN Architectures
– Backpropagation
• Learning Theory– PAC Learning
9
![Page 10: Final Exam Reviemgormley/courses/10601bd-f18/slides/final-revie… · Final Exam • Time / Location – Time: Evening Exam Thu, Dec 13 at 1:00pm –4:00pm – Room: We will contact](https://reader036.fdocuments.us/reader036/viewer/2022070822/5f28779a55ef57646920a36a/html5/thumbnails/10.jpg)
Topics covered after Midterm
• Learning Theory– PAC Learning
• Generative Models– Generative vs.
Discriminative– MLE / MAP– Naïve Bayes
• Graphical Models– HMMs– Learning and Inference– Bayesian Networks
• Reinforcement Learning– Value Iteration– Policy Iteration– Q-Learning– Deep Q-Learning
• Unsupervised Learning– K-Means– PCA
• Other Learning Paradigms– SVM (large-margin)– Kernels– Ensemble Methods /
AdaBoost
10
![Page 11: Final Exam Reviemgormley/courses/10601bd-f18/slides/final-revie… · Final Exam • Time / Location – Time: Evening Exam Thu, Dec 13 at 1:00pm –4:00pm – Room: We will contact](https://reader036.fdocuments.us/reader036/viewer/2022070822/5f28779a55ef57646920a36a/html5/thumbnails/11.jpg)
SAMPLE QUESTIONSMaterial Covered Before Midterm Exam
11
![Page 12: Final Exam Reviemgormley/courses/10601bd-f18/slides/final-revie… · Final Exam • Time / Location – Time: Evening Exam Thu, Dec 13 at 1:00pm –4:00pm – Room: We will contact](https://reader036.fdocuments.us/reader036/viewer/2022070822/5f28779a55ef57646920a36a/html5/thumbnails/12.jpg)
Matching Game
Goal: Match the Algorithm to its Update Rule
12
1. SGD for Logistic Regression
2. Least Mean Squares
3. Perceptron
4.
5.
6.
A. 1=5, 2=4, 3=6B. 1=5, 2=6, 3=4C. 1=6, 2=4, 3=4D. 1=5, 2=6, 3=6E. 1=6, 2=6, 3=6
![Page 13: Final Exam Reviemgormley/courses/10601bd-f18/slides/final-revie… · Final Exam • Time / Location – Time: Evening Exam Thu, Dec 13 at 1:00pm –4:00pm – Room: We will contact](https://reader036.fdocuments.us/reader036/viewer/2022070822/5f28779a55ef57646920a36a/html5/thumbnails/13.jpg)
Sample Questions
13
![Page 14: Final Exam Reviemgormley/courses/10601bd-f18/slides/final-revie… · Final Exam • Time / Location – Time: Evening Exam Thu, Dec 13 at 1:00pm –4:00pm – Room: We will contact](https://reader036.fdocuments.us/reader036/viewer/2022070822/5f28779a55ef57646920a36a/html5/thumbnails/14.jpg)
Sample Questions
14
Dataset
![Page 15: Final Exam Reviemgormley/courses/10601bd-f18/slides/final-revie… · Final Exam • Time / Location – Time: Evening Exam Thu, Dec 13 at 1:00pm –4:00pm – Room: We will contact](https://reader036.fdocuments.us/reader036/viewer/2022070822/5f28779a55ef57646920a36a/html5/thumbnails/15.jpg)
15
![Page 16: Final Exam Reviemgormley/courses/10601bd-f18/slides/final-revie… · Final Exam • Time / Location – Time: Evening Exam Thu, Dec 13 at 1:00pm –4:00pm – Room: We will contact](https://reader036.fdocuments.us/reader036/viewer/2022070822/5f28779a55ef57646920a36a/html5/thumbnails/16.jpg)
16
Topographical Maps
![Page 17: Final Exam Reviemgormley/courses/10601bd-f18/slides/final-revie… · Final Exam • Time / Location – Time: Evening Exam Thu, Dec 13 at 1:00pm –4:00pm – Room: We will contact](https://reader036.fdocuments.us/reader036/viewer/2022070822/5f28779a55ef57646920a36a/html5/thumbnails/17.jpg)
Sample Questions
17
Dataset
![Page 18: Final Exam Reviemgormley/courses/10601bd-f18/slides/final-revie… · Final Exam • Time / Location – Time: Evening Exam Thu, Dec 13 at 1:00pm –4:00pm – Room: We will contact](https://reader036.fdocuments.us/reader036/viewer/2022070822/5f28779a55ef57646920a36a/html5/thumbnails/18.jpg)
Sample Questions
18
Dataset
![Page 19: Final Exam Reviemgormley/courses/10601bd-f18/slides/final-revie… · Final Exam • Time / Location – Time: Evening Exam Thu, Dec 13 at 1:00pm –4:00pm – Room: We will contact](https://reader036.fdocuments.us/reader036/viewer/2022070822/5f28779a55ef57646920a36a/html5/thumbnails/19.jpg)
Sample Questions
19
Dataset
![Page 20: Final Exam Reviemgormley/courses/10601bd-f18/slides/final-revie… · Final Exam • Time / Location – Time: Evening Exam Thu, Dec 13 at 1:00pm –4:00pm – Room: We will contact](https://reader036.fdocuments.us/reader036/viewer/2022070822/5f28779a55ef57646920a36a/html5/thumbnails/20.jpg)
20
![Page 21: Final Exam Reviemgormley/courses/10601bd-f18/slides/final-revie… · Final Exam • Time / Location – Time: Evening Exam Thu, Dec 13 at 1:00pm –4:00pm – Room: We will contact](https://reader036.fdocuments.us/reader036/viewer/2022070822/5f28779a55ef57646920a36a/html5/thumbnails/21.jpg)
Robotic Farming
21
Deterministic Probabilistic
Classification(binary output)
Is this a picture of a wheat kernel?
Is this plant drought resistant?
Regression(continuous output)
How many wheat kernels are in this picture?
What will the yield of this plant be?
![Page 22: Final Exam Reviemgormley/courses/10601bd-f18/slides/final-revie… · Final Exam • Time / Location – Time: Evening Exam Thu, Dec 13 at 1:00pm –4:00pm – Room: We will contact](https://reader036.fdocuments.us/reader036/viewer/2022070822/5f28779a55ef57646920a36a/html5/thumbnails/22.jpg)
Multinomial Logistic Regression
polar bears
sea lions
sharks
22
![Page 23: Final Exam Reviemgormley/courses/10601bd-f18/slides/final-revie… · Final Exam • Time / Location – Time: Evening Exam Thu, Dec 13 at 1:00pm –4:00pm – Room: We will contact](https://reader036.fdocuments.us/reader036/viewer/2022070822/5f28779a55ef57646920a36a/html5/thumbnails/23.jpg)
Sample Questions
23
![Page 24: Final Exam Reviemgormley/courses/10601bd-f18/slides/final-revie… · Final Exam • Time / Location – Time: Evening Exam Thu, Dec 13 at 1:00pm –4:00pm – Room: We will contact](https://reader036.fdocuments.us/reader036/viewer/2022070822/5f28779a55ef57646920a36a/html5/thumbnails/24.jpg)
Handcrafted Features
24
NNP : VBN NNP VBD
PERLOC
Egypt - born Proyas directed
S
NP VP
ADJP VPNP
egypt - born proyas direct
p(y|x) ∝exp(Θyf( ))
born-in
![Page 25: Final Exam Reviemgormley/courses/10601bd-f18/slides/final-revie… · Final Exam • Time / Location – Time: Evening Exam Thu, Dec 13 at 1:00pm –4:00pm – Room: We will contact](https://reader036.fdocuments.us/reader036/viewer/2022070822/5f28779a55ef57646920a36a/html5/thumbnails/25.jpg)
Example: Linear Regression
25x
y
Goal: Learn y = wT f(x) + bwhere f(.) is a polynomial basis function
true “unknown” target function is linear with negative slope and gaussian noise
![Page 26: Final Exam Reviemgormley/courses/10601bd-f18/slides/final-revie… · Final Exam • Time / Location – Time: Evening Exam Thu, Dec 13 at 1:00pm –4:00pm – Room: We will contact](https://reader036.fdocuments.us/reader036/viewer/2022070822/5f28779a55ef57646920a36a/html5/thumbnails/26.jpg)
Samples Questions
26
![Page 27: Final Exam Reviemgormley/courses/10601bd-f18/slides/final-revie… · Final Exam • Time / Location – Time: Evening Exam Thu, Dec 13 at 1:00pm –4:00pm – Room: We will contact](https://reader036.fdocuments.us/reader036/viewer/2022070822/5f28779a55ef57646920a36a/html5/thumbnails/27.jpg)
Samples Questions
27
![Page 28: Final Exam Reviemgormley/courses/10601bd-f18/slides/final-revie… · Final Exam • Time / Location – Time: Evening Exam Thu, Dec 13 at 1:00pm –4:00pm – Room: We will contact](https://reader036.fdocuments.us/reader036/viewer/2022070822/5f28779a55ef57646920a36a/html5/thumbnails/28.jpg)
Sample Questions
28
![Page 29: Final Exam Reviemgormley/courses/10601bd-f18/slides/final-revie… · Final Exam • Time / Location – Time: Evening Exam Thu, Dec 13 at 1:00pm –4:00pm – Room: We will contact](https://reader036.fdocuments.us/reader036/viewer/2022070822/5f28779a55ef57646920a36a/html5/thumbnails/29.jpg)
Sample Questions
29
Can the neural network in Figure (b) correctly classify the dataset given in Figure (a)?
Neural Networks
![Page 30: Final Exam Reviemgormley/courses/10601bd-f18/slides/final-revie… · Final Exam • Time / Location – Time: Evening Exam Thu, Dec 13 at 1:00pm –4:00pm – Room: We will contact](https://reader036.fdocuments.us/reader036/viewer/2022070822/5f28779a55ef57646920a36a/html5/thumbnails/30.jpg)
Sample Questions
30
Apply the backpropagation algorithm to obtain
the partial derivative of the mean-squared error
of y with the true value y* with respect to the
weight w22 assuming a sigmoid nonlinear
activation function for the hidden layer.
Neural Networks
![Page 31: Final Exam Reviemgormley/courses/10601bd-f18/slides/final-revie… · Final Exam • Time / Location – Time: Evening Exam Thu, Dec 13 at 1:00pm –4:00pm – Room: We will contact](https://reader036.fdocuments.us/reader036/viewer/2022070822/5f28779a55ef57646920a36a/html5/thumbnails/31.jpg)
SAMPLE QUESTIONSMaterial Covered After Midterm Exam
31
![Page 32: Final Exam Reviemgormley/courses/10601bd-f18/slides/final-revie… · Final Exam • Time / Location – Time: Evening Exam Thu, Dec 13 at 1:00pm –4:00pm – Room: We will contact](https://reader036.fdocuments.us/reader036/viewer/2022070822/5f28779a55ef57646920a36a/html5/thumbnails/32.jpg)
Samples Questions
33
![Page 33: Final Exam Reviemgormley/courses/10601bd-f18/slides/final-revie… · Final Exam • Time / Location – Time: Evening Exam Thu, Dec 13 at 1:00pm –4:00pm – Room: We will contact](https://reader036.fdocuments.us/reader036/viewer/2022070822/5f28779a55ef57646920a36a/html5/thumbnails/33.jpg)
Samples Questions
34
![Page 34: Final Exam Reviemgormley/courses/10601bd-f18/slides/final-revie… · Final Exam • Time / Location – Time: Evening Exam Thu, Dec 13 at 1:00pm –4:00pm – Room: We will contact](https://reader036.fdocuments.us/reader036/viewer/2022070822/5f28779a55ef57646920a36a/html5/thumbnails/34.jpg)
Sample Questions
35
![Page 35: Final Exam Reviemgormley/courses/10601bd-f18/slides/final-revie… · Final Exam • Time / Location – Time: Evening Exam Thu, Dec 13 at 1:00pm –4:00pm – Room: We will contact](https://reader036.fdocuments.us/reader036/viewer/2022070822/5f28779a55ef57646920a36a/html5/thumbnails/35.jpg)
Sample Questions
36
![Page 36: Final Exam Reviemgormley/courses/10601bd-f18/slides/final-revie… · Final Exam • Time / Location – Time: Evening Exam Thu, Dec 13 at 1:00pm –4:00pm – Room: We will contact](https://reader036.fdocuments.us/reader036/viewer/2022070822/5f28779a55ef57646920a36a/html5/thumbnails/36.jpg)
Sample Questions
37
![Page 37: Final Exam Reviemgormley/courses/10601bd-f18/slides/final-revie… · Final Exam • Time / Location – Time: Evening Exam Thu, Dec 13 at 1:00pm –4:00pm – Room: We will contact](https://reader036.fdocuments.us/reader036/viewer/2022070822/5f28779a55ef57646920a36a/html5/thumbnails/37.jpg)
Sample Questions
38
![Page 38: Final Exam Reviemgormley/courses/10601bd-f18/slides/final-revie… · Final Exam • Time / Location – Time: Evening Exam Thu, Dec 13 at 1:00pm –4:00pm – Room: We will contact](https://reader036.fdocuments.us/reader036/viewer/2022070822/5f28779a55ef57646920a36a/html5/thumbnails/38.jpg)
Sample Questions
39
![Page 39: Final Exam Reviemgormley/courses/10601bd-f18/slides/final-revie… · Final Exam • Time / Location – Time: Evening Exam Thu, Dec 13 at 1:00pm –4:00pm – Room: We will contact](https://reader036.fdocuments.us/reader036/viewer/2022070822/5f28779a55ef57646920a36a/html5/thumbnails/39.jpg)
Sample Questions
40
![Page 40: Final Exam Reviemgormley/courses/10601bd-f18/slides/final-revie… · Final Exam • Time / Location – Time: Evening Exam Thu, Dec 13 at 1:00pm –4:00pm – Room: We will contact](https://reader036.fdocuments.us/reader036/viewer/2022070822/5f28779a55ef57646920a36a/html5/thumbnails/40.jpg)
Samples Questions
41
![Page 41: Final Exam Reviemgormley/courses/10601bd-f18/slides/final-revie… · Final Exam • Time / Location – Time: Evening Exam Thu, Dec 13 at 1:00pm –4:00pm – Room: We will contact](https://reader036.fdocuments.us/reader036/viewer/2022070822/5f28779a55ef57646920a36a/html5/thumbnails/41.jpg)
Samples Questions
42
Circle the image which depicts the
cluster center positions after 1
iteration of Lloyd’s algorithm.
![Page 42: Final Exam Reviemgormley/courses/10601bd-f18/slides/final-revie… · Final Exam • Time / Location – Time: Evening Exam Thu, Dec 13 at 1:00pm –4:00pm – Room: We will contact](https://reader036.fdocuments.us/reader036/viewer/2022070822/5f28779a55ef57646920a36a/html5/thumbnails/42.jpg)
Samples Questions
43
Circle the image which depicts the
cluster center positions after 1
iteration of Lloyd’s algorithm.
![Page 43: Final Exam Reviemgormley/courses/10601bd-f18/slides/final-revie… · Final Exam • Time / Location – Time: Evening Exam Thu, Dec 13 at 1:00pm –4:00pm – Room: We will contact](https://reader036.fdocuments.us/reader036/viewer/2022070822/5f28779a55ef57646920a36a/html5/thumbnails/43.jpg)
Sample Questions
44
![Page 44: Final Exam Reviemgormley/courses/10601bd-f18/slides/final-revie… · Final Exam • Time / Location – Time: Evening Exam Thu, Dec 13 at 1:00pm –4:00pm – Room: We will contact](https://reader036.fdocuments.us/reader036/viewer/2022070822/5f28779a55ef57646920a36a/html5/thumbnails/44.jpg)
Sample Questions
45
![Page 45: Final Exam Reviemgormley/courses/10601bd-f18/slides/final-revie… · Final Exam • Time / Location – Time: Evening Exam Thu, Dec 13 at 1:00pm –4:00pm – Room: We will contact](https://reader036.fdocuments.us/reader036/viewer/2022070822/5f28779a55ef57646920a36a/html5/thumbnails/45.jpg)
Sample Questions
46
![Page 46: Final Exam Reviemgormley/courses/10601bd-f18/slides/final-revie… · Final Exam • Time / Location – Time: Evening Exam Thu, Dec 13 at 1:00pm –4:00pm – Room: We will contact](https://reader036.fdocuments.us/reader036/viewer/2022070822/5f28779a55ef57646920a36a/html5/thumbnails/46.jpg)
Course Level ObjectivesYou should be able to…1. Implement and analyze existing learning algorithms, including well-studied
methods for classification, regression, structured prediction, clustering, and representation learning
2. Integrate multiple facets of practical machine learning in a single system: data preprocessing, learning, regularization and model selection
3. Describe the the formal properties of models and algorithms for learning and explain the practical implications of those results
4. Compare and contrast different paradigms for learning (supervised, unsupervised, etc.)
5. Design experiments to evaluate and compare different machine learning techniques on real-world problems
6. Employ probability, statistics, calculus, linear algebra, and optimization in order to develop new predictive models or learning methods
7. Given a description of a ML technique, analyze it to identify (1) the expressive power of the formalism; (2) the inductive bias implicit in the algorithm; (3) the size and complexity of the search space; (4) the computational properties of the algorithm: (5) any guarantees (or lack thereof) regarding termination, convergence, correctness, accuracy or generalization power.
47
![Page 47: Final Exam Reviemgormley/courses/10601bd-f18/slides/final-revie… · Final Exam • Time / Location – Time: Evening Exam Thu, Dec 13 at 1:00pm –4:00pm – Room: We will contact](https://reader036.fdocuments.us/reader036/viewer/2022070822/5f28779a55ef57646920a36a/html5/thumbnails/47.jpg)
Q&A
48