The most stupid classifier - unipi.itdidawiki.cli.di.unipi.it/.../dm/classification_nnandsvm.pdf ·...
Transcript of The most stupid classifier - unipi.itdidawiki.cli.di.unipi.it/.../dm/classification_nnandsvm.pdf ·...
![Page 1: The most stupid classifier - unipi.itdidawiki.cli.di.unipi.it/.../dm/classification_nnandsvm.pdf · 2018. 4. 20. · A quick look on Deep Learning ... to automatically discover the](https://reader035.fdocuments.us/reader035/viewer/2022063017/5fd8b4bd611a23394d3f1c63/html5/thumbnails/1.jpg)
![Page 2: The most stupid classifier - unipi.itdidawiki.cli.di.unipi.it/.../dm/classification_nnandsvm.pdf · 2018. 4. 20. · A quick look on Deep Learning ... to automatically discover the](https://reader035.fdocuments.us/reader035/viewer/2022063017/5fd8b4bd611a23394d3f1c63/html5/thumbnails/2.jpg)
The most stupid classifier
Rote learner
To classify object X, check if there is a labelled
example in the training set identical to X
Yes X has the same label
No I don’t know
?
![Page 3: The most stupid classifier - unipi.itdidawiki.cli.di.unipi.it/.../dm/classification_nnandsvm.pdf · 2018. 4. 20. · A quick look on Deep Learning ... to automatically discover the](https://reader035.fdocuments.us/reader035/viewer/2022063017/5fd8b4bd611a23394d3f1c63/html5/thumbnails/3.jpg)
Classify by similarity
K-Nearest Neighbors
Decide label based on K most similar examples
K=3
![Page 4: The most stupid classifier - unipi.itdidawiki.cli.di.unipi.it/.../dm/classification_nnandsvm.pdf · 2018. 4. 20. · A quick look on Deep Learning ... to automatically discover the](https://reader035.fdocuments.us/reader035/viewer/2022063017/5fd8b4bd611a23394d3f1c63/html5/thumbnails/4.jpg)
Build a model
Decision Trees
Cut space by lines orthogonal to the axes
Income > 15k€ ?
Age > 50y ?
Age
Income
yes no
yes no
![Page 5: The most stupid classifier - unipi.itdidawiki.cli.di.unipi.it/.../dm/classification_nnandsvm.pdf · 2018. 4. 20. · A quick look on Deep Learning ... to automatically discover the](https://reader035.fdocuments.us/reader035/viewer/2022063017/5fd8b4bd611a23394d3f1c63/html5/thumbnails/5.jpg)
![Page 6: The most stupid classifier - unipi.itdidawiki.cli.di.unipi.it/.../dm/classification_nnandsvm.pdf · 2018. 4. 20. · A quick look on Deep Learning ... to automatically discover the](https://reader035.fdocuments.us/reader035/viewer/2022063017/5fd8b4bd611a23394d3f1c63/html5/thumbnails/6.jpg)
![Page 7: The most stupid classifier - unipi.itdidawiki.cli.di.unipi.it/.../dm/classification_nnandsvm.pdf · 2018. 4. 20. · A quick look on Deep Learning ... to automatically discover the](https://reader035.fdocuments.us/reader035/viewer/2022063017/5fd8b4bd611a23394d3f1c63/html5/thumbnails/7.jpg)
![Page 8: The most stupid classifier - unipi.itdidawiki.cli.di.unipi.it/.../dm/classification_nnandsvm.pdf · 2018. 4. 20. · A quick look on Deep Learning ... to automatically discover the](https://reader035.fdocuments.us/reader035/viewer/2022063017/5fd8b4bd611a23394d3f1c63/html5/thumbnails/8.jpg)
Sample model on 2-D
Linear separation line
General case: lines can be oblique
![Page 9: The most stupid classifier - unipi.itdidawiki.cli.di.unipi.it/.../dm/classification_nnandsvm.pdf · 2018. 4. 20. · A quick look on Deep Learning ... to automatically discover the](https://reader035.fdocuments.us/reader035/viewer/2022063017/5fd8b4bd611a23394d3f1c63/html5/thumbnails/9.jpg)
![Page 10: The most stupid classifier - unipi.itdidawiki.cli.di.unipi.it/.../dm/classification_nnandsvm.pdf · 2018. 4. 20. · A quick look on Deep Learning ... to automatically discover the](https://reader035.fdocuments.us/reader035/viewer/2022063017/5fd8b4bd611a23394d3f1c63/html5/thumbnails/10.jpg)
Notice: activation function is fundamental !!!
With Identity (= no activation function) the ANN
reduces to a simple perceptron
Proof: a linear sum of linear sums, is just another linear sum
![Page 11: The most stupid classifier - unipi.itdidawiki.cli.di.unipi.it/.../dm/classification_nnandsvm.pdf · 2018. 4. 20. · A quick look on Deep Learning ... to automatically discover the](https://reader035.fdocuments.us/reader035/viewer/2022063017/5fd8b4bd611a23394d3f1c63/html5/thumbnails/11.jpg)
![Page 12: The most stupid classifier - unipi.itdidawiki.cli.di.unipi.it/.../dm/classification_nnandsvm.pdf · 2018. 4. 20. · A quick look on Deep Learning ... to automatically discover the](https://reader035.fdocuments.us/reader035/viewer/2022063017/5fd8b4bd611a23394d3f1c63/html5/thumbnails/12.jpg)
A quick look on Deep Learning
Various approaches exist
Basic examples equivalent to ANNs
with several levelsAI
Machine Learning
Repres.
Learning
Deep Learning
![Page 13: The most stupid classifier - unipi.itdidawiki.cli.di.unipi.it/.../dm/classification_nnandsvm.pdf · 2018. 4. 20. · A quick look on Deep Learning ... to automatically discover the](https://reader035.fdocuments.us/reader035/viewer/2022063017/5fd8b4bd611a23394d3f1c63/html5/thumbnails/13.jpg)
Deep learning
Representation learning methods that
allow a machine to be fed with raw data and
to automatically discover the representations needed for detection or classification.
Repres.
Learning
Deep Learning
• Age
• Weight
• Income
• Children
• Likes sport
• Likes reading
• Education
• …
Raw representation Higher-level representation
• Young parent
• Fit sportsman
• High-educated reader
• Rich obese
• …
35
65
23 k€
2
0.3
0.6
high
…
0.9
0.1
0.8
0.0
…
![Page 14: The most stupid classifier - unipi.itdidawiki.cli.di.unipi.it/.../dm/classification_nnandsvm.pdf · 2018. 4. 20. · A quick look on Deep Learning ... to automatically discover the](https://reader035.fdocuments.us/reader035/viewer/2022063017/5fd8b4bd611a23394d3f1c63/html5/thumbnails/14.jpg)
Multiple Levels Of Abstraction
![Page 15: The most stupid classifier - unipi.itdidawiki.cli.di.unipi.it/.../dm/classification_nnandsvm.pdf · 2018. 4. 20. · A quick look on Deep Learning ... to automatically discover the](https://reader035.fdocuments.us/reader035/viewer/2022063017/5fd8b4bd611a23394d3f1c63/html5/thumbnails/15.jpg)
Why now?
(Big) Data
GPU
Theory
![Page 16: The most stupid classifier - unipi.itdidawiki.cli.di.unipi.it/.../dm/classification_nnandsvm.pdf · 2018. 4. 20. · A quick look on Deep Learning ... to automatically discover the](https://reader035.fdocuments.us/reader035/viewer/2022063017/5fd8b4bd611a23394d3f1c63/html5/thumbnails/16.jpg)
Support Vector Machines
Support Vector Machine
![Page 17: The most stupid classifier - unipi.itdidawiki.cli.di.unipi.it/.../dm/classification_nnandsvm.pdf · 2018. 4. 20. · A quick look on Deep Learning ... to automatically discover the](https://reader035.fdocuments.us/reader035/viewer/2022063017/5fd8b4bd611a23394d3f1c63/html5/thumbnails/17.jpg)
![Page 18: The most stupid classifier - unipi.itdidawiki.cli.di.unipi.it/.../dm/classification_nnandsvm.pdf · 2018. 4. 20. · A quick look on Deep Learning ... to automatically discover the](https://reader035.fdocuments.us/reader035/viewer/2022063017/5fd8b4bd611a23394d3f1c63/html5/thumbnails/18.jpg)
![Page 19: The most stupid classifier - unipi.itdidawiki.cli.di.unipi.it/.../dm/classification_nnandsvm.pdf · 2018. 4. 20. · A quick look on Deep Learning ... to automatically discover the](https://reader035.fdocuments.us/reader035/viewer/2022063017/5fd8b4bd611a23394d3f1c63/html5/thumbnails/19.jpg)
![Page 20: The most stupid classifier - unipi.itdidawiki.cli.di.unipi.it/.../dm/classification_nnandsvm.pdf · 2018. 4. 20. · A quick look on Deep Learning ... to automatically discover the](https://reader035.fdocuments.us/reader035/viewer/2022063017/5fd8b4bd611a23394d3f1c63/html5/thumbnails/20.jpg)
![Page 21: The most stupid classifier - unipi.itdidawiki.cli.di.unipi.it/.../dm/classification_nnandsvm.pdf · 2018. 4. 20. · A quick look on Deep Learning ... to automatically discover the](https://reader035.fdocuments.us/reader035/viewer/2022063017/5fd8b4bd611a23394d3f1c63/html5/thumbnails/21.jpg)
![Page 22: The most stupid classifier - unipi.itdidawiki.cli.di.unipi.it/.../dm/classification_nnandsvm.pdf · 2018. 4. 20. · A quick look on Deep Learning ... to automatically discover the](https://reader035.fdocuments.us/reader035/viewer/2022063017/5fd8b4bd611a23394d3f1c63/html5/thumbnails/22.jpg)
![Page 23: The most stupid classifier - unipi.itdidawiki.cli.di.unipi.it/.../dm/classification_nnandsvm.pdf · 2018. 4. 20. · A quick look on Deep Learning ... to automatically discover the](https://reader035.fdocuments.us/reader035/viewer/2022063017/5fd8b4bd611a23394d3f1c63/html5/thumbnails/23.jpg)
x = data points (n-dimensional vectors)
w = direction of line (n-dimensional vector)
b = displacement of line
![Page 24: The most stupid classifier - unipi.itdidawiki.cli.di.unipi.it/.../dm/classification_nnandsvm.pdf · 2018. 4. 20. · A quick look on Deep Learning ... to automatically discover the](https://reader035.fdocuments.us/reader035/viewer/2022063017/5fd8b4bd611a23394d3f1c63/html5/thumbnails/24.jpg)
w1
w2
Example
Each data point is projected over direction w
Projections along w2 have much larger margin than w1
![Page 25: The most stupid classifier - unipi.itdidawiki.cli.di.unipi.it/.../dm/classification_nnandsvm.pdf · 2018. 4. 20. · A quick look on Deep Learning ... to automatically discover the](https://reader035.fdocuments.us/reader035/viewer/2022063017/5fd8b4bd611a23394d3f1c63/html5/thumbnails/25.jpg)
![Page 26: The most stupid classifier - unipi.itdidawiki.cli.di.unipi.it/.../dm/classification_nnandsvm.pdf · 2018. 4. 20. · A quick look on Deep Learning ... to automatically discover the](https://reader035.fdocuments.us/reader035/viewer/2022063017/5fd8b4bd611a23394d3f1c63/html5/thumbnails/26.jpg)
![Page 27: The most stupid classifier - unipi.itdidawiki.cli.di.unipi.it/.../dm/classification_nnandsvm.pdf · 2018. 4. 20. · A quick look on Deep Learning ... to automatically discover the](https://reader035.fdocuments.us/reader035/viewer/2022063017/5fd8b4bd611a23394d3f1c63/html5/thumbnails/27.jpg)
• Basically, each point is “moved” by a a specific amount
along the w direction
• The cost function “pays” for each extra movement
![Page 28: The most stupid classifier - unipi.itdidawiki.cli.di.unipi.it/.../dm/classification_nnandsvm.pdf · 2018. 4. 20. · A quick look on Deep Learning ... to automatically discover the](https://reader035.fdocuments.us/reader035/viewer/2022063017/5fd8b4bd611a23394d3f1c63/html5/thumbnails/28.jpg)
Support Vector Machines
Non-linear separation line
![Page 29: The most stupid classifier - unipi.itdidawiki.cli.di.unipi.it/.../dm/classification_nnandsvm.pdf · 2018. 4. 20. · A quick look on Deep Learning ... to automatically discover the](https://reader035.fdocuments.us/reader035/viewer/2022063017/5fd8b4bd611a23394d3f1c63/html5/thumbnails/29.jpg)
![Page 30: The most stupid classifier - unipi.itdidawiki.cli.di.unipi.it/.../dm/classification_nnandsvm.pdf · 2018. 4. 20. · A quick look on Deep Learning ... to automatically discover the](https://reader035.fdocuments.us/reader035/viewer/2022063017/5fd8b4bd611a23394d3f1c63/html5/thumbnails/30.jpg)
• Key problem: find the most appropriate set of extra dimensions
• They are derived from original attributes
• Most common: x^2, (x+y)^2, and other polynomials