Support Vector machine
-
Upload
anandha-l-ranganathan -
Category
Technology
-
view
311 -
download
2
description
Transcript of Support Vector machine
![Page 1: Support Vector machine](https://reader036.fdocuments.us/reader036/viewer/2022062419/55788764d8b42a02618b4be3/html5/thumbnails/1.jpg)
SEN – 935 DATA MINING
Anandha L Ranganathan
Support Vector Machine
17. Support Vector Machine 1
![Page 2: Support Vector machine](https://reader036.fdocuments.us/reader036/viewer/2022062419/55788764d8b42a02618b4be3/html5/thumbnails/2.jpg)
Pre 1980s - Almost all learning methods learned linear decision surfaces. - Linear learning methods have nice theoretical properties
1980s - Almost all learning methods learned linear decision surfaces. - Linear learning methods have nice theoretical properties
1990’s - Efficient learning algorithms for non-linear functions based on
computational learning theory developed - Nice theoretical properties.
History
17. Support Vector Machine 2
![Page 3: Support Vector machine](https://reader036.fdocuments.us/reader036/viewer/2022062419/55788764d8b42a02618b4be3/html5/thumbnails/3.jpg)
Two independent developments within last decade
– Computational learning theory– New efficient separability of non-linear functions that use “kernel
functions”
The resultant learning algorithm is optimization algorithm rather than a greedy search.
What is greedy search ?
History
17. Support Vector Machine 3
![Page 4: Support Vector machine](https://reader036.fdocuments.us/reader036/viewer/2022062419/55788764d8b42a02618b4be3/html5/thumbnails/4.jpg)
Find largest sum by traversing through path.
Greedy search
17. Support Vector Machine 4
![Page 5: Support Vector machine](https://reader036.fdocuments.us/reader036/viewer/2022062419/55788764d8b42a02618b4be3/html5/thumbnails/5.jpg)
Find largest sum by traversing through path.
Greedy search
17. Support Vector Machine 5
![Page 6: Support Vector machine](https://reader036.fdocuments.us/reader036/viewer/2022062419/55788764d8b42a02618b4be3/html5/thumbnails/6.jpg)
Find largest sum by traversing through path.
Greedy search
17. Support Vector Machine 6
![Page 7: Support Vector machine](https://reader036.fdocuments.us/reader036/viewer/2022062419/55788764d8b42a02618b4be3/html5/thumbnails/7.jpg)
A system receives data as input. Output a function that can be predict some
features of future data.
Learning Theory
f x yest
f(x,w,b) = sign(w. x - b)
17. Support Vector Machine 7
![Page 8: Support Vector machine](https://reader036.fdocuments.us/reader036/viewer/2022062419/55788764d8b42a02618b4be3/html5/thumbnails/8.jpg)
Not affected by local minima. Do not suffer from the curse of dimensionality. Have modular design that allows one to
separately implement and design other component.
Various properties of the SVM solution help avoid over fitting, even in very high dimensional feature spaces
Features of SVM's
17. Support Vector Machine 8
![Page 9: Support Vector machine](https://reader036.fdocuments.us/reader036/viewer/2022062419/55788764d8b42a02618b4be3/html5/thumbnails/9.jpg)
Support vectors are data points that lie closes to the decision surface.
But they are difficult to classify. They have direct bearing of optimum location
on the surface.
Support Vectors
17. Support Vector Machine 9
![Page 10: Support Vector machine](https://reader036.fdocuments.us/reader036/viewer/2022062419/55788764d8b42a02618b4be3/html5/thumbnails/10.jpg)
d1 and d2 are 2 vectors. And sum of their distance is d1+d2=q.
d1=2x+5y and d2=3x+2y q=d1+d2=5x+7y
Vector Space – Primer
17. Support Vector Machine 10
![Page 11: Support Vector machine](https://reader036.fdocuments.us/reader036/viewer/2022062419/55788764d8b42a02618b4be3/html5/thumbnails/11.jpg)
Computing the inner products between the vectors in the featured space.
Kernal – Primer
17. Support Vector Machine 11
![Page 12: Support Vector machine](https://reader036.fdocuments.us/reader036/viewer/2022062419/55788764d8b42a02618b4be3/html5/thumbnails/12.jpg)
Linear Classifiers
denotes +1
denotes -1
How would you classify this data?
17. Support Vector Machine 12
![Page 13: Support Vector machine](https://reader036.fdocuments.us/reader036/viewer/2022062419/55788764d8b42a02618b4be3/html5/thumbnails/13.jpg)
denotes +1
denotes -1
How would you classify this data?
Linear Classifiers
17. Support Vector Machine 13
![Page 14: Support Vector machine](https://reader036.fdocuments.us/reader036/viewer/2022062419/55788764d8b42a02618b4be3/html5/thumbnails/14.jpg)
denotes +1
denotes -1
How would you classify this data?
Linear Classifiers
17. Support Vector Machine 14
![Page 15: Support Vector machine](https://reader036.fdocuments.us/reader036/viewer/2022062419/55788764d8b42a02618b4be3/html5/thumbnails/15.jpg)
denotes +1
denotes -1
How would you classify this data?
Linear Classifiers
17. Support Vector Machine 15
![Page 16: Support Vector machine](https://reader036.fdocuments.us/reader036/viewer/2022062419/55788764d8b42a02618b4be3/html5/thumbnails/16.jpg)
denotes +1
denotes -1
How would you classify this data?
Linear Classifiers
17. Support Vector Machine 16
![Page 17: Support Vector machine](https://reader036.fdocuments.us/reader036/viewer/2022062419/55788764d8b42a02618b4be3/html5/thumbnails/17.jpg)
denotes +1
denotes -1
Any of these would be fine..
..but which is best?
Linear Classifiers
17. Support Vector Machine 17
![Page 18: Support Vector machine](https://reader036.fdocuments.us/reader036/viewer/2022062419/55788764d8b42a02618b4be3/html5/thumbnails/18.jpg)
Classifier Margin
denotes +1
denotes -1 Define the margin of a linear classifier as the width that the boundary could be increased by before hitting a datapoint.
17. Support Vector Machine 18
![Page 19: Support Vector machine](https://reader036.fdocuments.us/reader036/viewer/2022062419/55788764d8b42a02618b4be3/html5/thumbnails/19.jpg)
Maximum Margin
denotes +1
denotes -1 The maximum margin linear classifier is the linear classifier with the maximum margin.
This is the simplest kind of SVM (Called an LSVM)
Linear SVM17. Support Vector Machine 19
![Page 20: Support Vector machine](https://reader036.fdocuments.us/reader036/viewer/2022062419/55788764d8b42a02618b4be3/html5/thumbnails/20.jpg)
Formulating SVM
17. Support Vector Machine 20
![Page 21: Support Vector machine](https://reader036.fdocuments.us/reader036/viewer/2022062419/55788764d8b42a02618b4be3/html5/thumbnails/21.jpg)
Formulating SVM
17. Support Vector Machine 21
![Page 22: Support Vector machine](https://reader036.fdocuments.us/reader036/viewer/2022062419/55788764d8b42a02618b4be3/html5/thumbnails/22.jpg)
Formulating SVM
17. Support Vector Machine 22
![Page 23: Support Vector machine](https://reader036.fdocuments.us/reader036/viewer/2022062419/55788764d8b42a02618b4be3/html5/thumbnails/23.jpg)
Idea: map to higher dimensional feature space
Kernal - polynomial
17. Support Vector Machine 23
![Page 24: Support Vector machine](https://reader036.fdocuments.us/reader036/viewer/2022062419/55788764d8b42a02618b4be3/html5/thumbnails/24.jpg)
License Plate Recognition
17. Support Vector Machine 24
![Page 25: Support Vector machine](https://reader036.fdocuments.us/reader036/viewer/2022062419/55788764d8b42a02618b4be3/html5/thumbnails/25.jpg)
Pre-process the image of number plate. Segment the image into several parts of
which each contains only a single character.
License Plate Recognition
17. Support Vector Machine 25
![Page 26: Support Vector machine](https://reader036.fdocuments.us/reader036/viewer/2022062419/55788764d8b42a02618b4be3/html5/thumbnails/26.jpg)
Extract the feature vector of each normalized candidate
Recognizes the single character (a digit or a letter) by the set of SVMs trained in advance.
License Plate Recognition
17. Support Vector Machine 26
![Page 27: Support Vector machine](https://reader036.fdocuments.us/reader036/viewer/2022062419/55788764d8b42a02618b4be3/html5/thumbnails/27.jpg)
License Plate Recognition
17. Support Vector Machine 27
![Page 28: Support Vector machine](https://reader036.fdocuments.us/reader036/viewer/2022062419/55788764d8b42a02618b4be3/html5/thumbnails/28.jpg)
License Plate Recognition
17. Support Vector Machine 28
![Page 29: Support Vector machine](https://reader036.fdocuments.us/reader036/viewer/2022062419/55788764d8b42a02618b4be3/html5/thumbnails/29.jpg)
License Plate Recognition
17. Support Vector Machine 29
![Page 30: Support Vector machine](https://reader036.fdocuments.us/reader036/viewer/2022062419/55788764d8b42a02618b4be3/html5/thumbnails/30.jpg)
License Plate Recognition
17. Support Vector Machine 30
![Page 31: Support Vector machine](https://reader036.fdocuments.us/reader036/viewer/2022062419/55788764d8b42a02618b4be3/html5/thumbnails/31.jpg)
License Plate Recognition
If there are no more unclassified samples, then STOP. Otherwise, then repeat the process of recognition of character.
Add these test samples into their corresponding database for further training.
Recognize number plate by bringing all characters used together
17. Support Vector Machine 31
![Page 32: Support Vector machine](https://reader036.fdocuments.us/reader036/viewer/2022062419/55788764d8b42a02618b4be3/html5/thumbnails/32.jpg)
SVM is widely used as classify spam detection in the market.
It supports for Linear and Non-Linear spectrum.
Conclusion
17. Support Vector Machine 32
![Page 33: Support Vector machine](https://reader036.fdocuments.us/reader036/viewer/2022062419/55788764d8b42a02618b4be3/html5/thumbnails/33.jpg)
http://www.cs.ucf.edu/courses/cap6412/fall2009/papers/Berwick2003.pdf
http://physiology.med.cornell.edu/people/banfelder/qbio/resources_2011/2011_Leslie.pdf
http://physiology.med.cornell.edu/people/banfelder/qbio/resources_2011/2011_Leslie.pdf
http://www.cs.columbia.edu/~kathy/cs4701/documents/jason_svm_tutorial.pdf
http://www.slideshare.net/analog76/savedfiles?s_title=svm-12978262&user_login=wltongxing
17. Support Vector Machine 33