Lecture 17 ML supp - GitHub Pages · Lecture 17: Linear Classifiers and Support Vector Machines...
Transcript of Lecture 17 ML supp - GitHub Pages · Lecture 17: Linear Classifiers and Support Vector Machines...
![Page 1: Lecture 17 ML supp - GitHub Pages · Lecture 17: Linear Classifiers and Support Vector Machines Theodoros Rekatsinas (lecture by AnkurGoswamimany slides from David Sontag) 1. Today’s](https://reader036.fdocuments.us/reader036/viewer/2022062506/5fb47bc829be6b736b3e3795/html5/thumbnails/1.jpg)
CS639:DataManagementfor
DataScienceLecture17:LinearClassifiersandSupportVectorMachines
TheodorosRekatsinas(lecturebyAnkur Goswami manyslidesfromDavidSontag)
1
![Page 2: Lecture 17 ML supp - GitHub Pages · Lecture 17: Linear Classifiers and Support Vector Machines Theodoros Rekatsinas (lecture by AnkurGoswamimany slides from David Sontag) 1. Today’s](https://reader036.fdocuments.us/reader036/viewer/2022062506/5fb47bc829be6b736b3e3795/html5/thumbnails/2.jpg)
Today’sLecture
1. Linearclassifiers
2. ThePerceptronAlgorithm
3. SupportVectorMachines
2
![Page 3: Lecture 17 ML supp - GitHub Pages · Lecture 17: Linear Classifiers and Support Vector Machines Theodoros Rekatsinas (lecture by AnkurGoswamimany slides from David Sontag) 1. Today’s](https://reader036.fdocuments.us/reader036/viewer/2022062506/5fb47bc829be6b736b3e3795/html5/thumbnails/3.jpg)
Linearclassifier• Let’ssimplifylifebyassuming:
• Everyinstanceisavectorofrealnumbers,x=(x1,…,xn).(Notation:boldfacexisavector.)
• Thereareonlytwoclasses,y=(+1) andy=(-1)
• Alinearclassifier isvectorw ofthesamedimensionasx thatisusedtomakethisprediction:
)(sign)...sign(ˆ 2211 xw ×=+++= nnxwxwxwy
þýü
îíì
<-³+
=0if10 if1
)(signx
x
![Page 4: Lecture 17 ML supp - GitHub Pages · Lecture 17: Linear Classifiers and Support Vector Machines Theodoros Rekatsinas (lecture by AnkurGoswamimany slides from David Sontag) 1. Today’s](https://reader036.fdocuments.us/reader036/viewer/2022062506/5fb47bc829be6b736b3e3795/html5/thumbnails/4.jpg)
Example:Linearclassifier
![Page 5: Lecture 17 ML supp - GitHub Pages · Lecture 17: Linear Classifiers and Support Vector Machines Theodoros Rekatsinas (lecture by AnkurGoswamimany slides from David Sontag) 1. Today’s](https://reader036.fdocuments.us/reader036/viewer/2022062506/5fb47bc829be6b736b3e3795/html5/thumbnails/5.jpg)
ThePerceptronAlgorithmtolearnaLinearClassifier
![Page 6: Lecture 17 ML supp - GitHub Pages · Lecture 17: Linear Classifiers and Support Vector Machines Theodoros Rekatsinas (lecture by AnkurGoswamimany slides from David Sontag) 1. Today’s](https://reader036.fdocuments.us/reader036/viewer/2022062506/5fb47bc829be6b736b3e3795/html5/thumbnails/6.jpg)
Definition:Linearlyseparabledata
![Page 7: Lecture 17 ML supp - GitHub Pages · Lecture 17: Linear Classifiers and Support Vector Machines Theodoros Rekatsinas (lecture by AnkurGoswamimany slides from David Sontag) 1. Today’s](https://reader036.fdocuments.us/reader036/viewer/2022062506/5fb47bc829be6b736b3e3795/html5/thumbnails/7.jpg)
Doestheperceptronalgorithmwork?
![Page 8: Lecture 17 ML supp - GitHub Pages · Lecture 17: Linear Classifiers and Support Vector Machines Theodoros Rekatsinas (lecture by AnkurGoswamimany slides from David Sontag) 1. Today’s](https://reader036.fdocuments.us/reader036/viewer/2022062506/5fb47bc829be6b736b3e3795/html5/thumbnails/8.jpg)
Propertiesoftheperceptronalgorithm
![Page 9: Lecture 17 ML supp - GitHub Pages · Lecture 17: Linear Classifiers and Support Vector Machines Theodoros Rekatsinas (lecture by AnkurGoswamimany slides from David Sontag) 1. Today’s](https://reader036.fdocuments.us/reader036/viewer/2022062506/5fb47bc829be6b736b3e3795/html5/thumbnails/9.jpg)
Problemswiththeperceptronalgorithm
[Wewillseenextweek]
![Page 10: Lecture 17 ML supp - GitHub Pages · Lecture 17: Linear Classifiers and Support Vector Machines Theodoros Rekatsinas (lecture by AnkurGoswamimany slides from David Sontag) 1. Today’s](https://reader036.fdocuments.us/reader036/viewer/2022062506/5fb47bc829be6b736b3e3795/html5/thumbnails/10.jpg)
Linearseparators
![Page 11: Lecture 17 ML supp - GitHub Pages · Lecture 17: Linear Classifiers and Support Vector Machines Theodoros Rekatsinas (lecture by AnkurGoswamimany slides from David Sontag) 1. Today’s](https://reader036.fdocuments.us/reader036/viewer/2022062506/5fb47bc829be6b736b3e3795/html5/thumbnails/11.jpg)
SupportVectorMachines
![Page 12: Lecture 17 ML supp - GitHub Pages · Lecture 17: Linear Classifiers and Support Vector Machines Theodoros Rekatsinas (lecture by AnkurGoswamimany slides from David Sontag) 1. Today’s](https://reader036.fdocuments.us/reader036/viewer/2022062506/5fb47bc829be6b736b3e3795/html5/thumbnails/12.jpg)
Normaltoaplane
![Page 13: Lecture 17 ML supp - GitHub Pages · Lecture 17: Linear Classifiers and Support Vector Machines Theodoros Rekatsinas (lecture by AnkurGoswamimany slides from David Sontag) 1. Today’s](https://reader036.fdocuments.us/reader036/viewer/2022062506/5fb47bc829be6b736b3e3795/html5/thumbnails/13.jpg)
Scaleinvariance
![Page 14: Lecture 17 ML supp - GitHub Pages · Lecture 17: Linear Classifiers and Support Vector Machines Theodoros Rekatsinas (lecture by AnkurGoswamimany slides from David Sontag) 1. Today’s](https://reader036.fdocuments.us/reader036/viewer/2022062506/5fb47bc829be6b736b3e3795/html5/thumbnails/14.jpg)
Scaleinvariance
![Page 15: Lecture 17 ML supp - GitHub Pages · Lecture 17: Linear Classifiers and Support Vector Machines Theodoros Rekatsinas (lecture by AnkurGoswamimany slides from David Sontag) 1. Today’s](https://reader036.fdocuments.us/reader036/viewer/2022062506/5fb47bc829be6b736b3e3795/html5/thumbnails/15.jpg)
Whatisγasafunctionofw?
![Page 16: Lecture 17 ML supp - GitHub Pages · Lecture 17: Linear Classifiers and Support Vector Machines Theodoros Rekatsinas (lecture by AnkurGoswamimany slides from David Sontag) 1. Today’s](https://reader036.fdocuments.us/reader036/viewer/2022062506/5fb47bc829be6b736b3e3795/html5/thumbnails/16.jpg)
SupportVectorMachines(SVMs)
![Page 17: Lecture 17 ML supp - GitHub Pages · Lecture 17: Linear Classifiers and Support Vector Machines Theodoros Rekatsinas (lecture by AnkurGoswamimany slides from David Sontag) 1. Today’s](https://reader036.fdocuments.us/reader036/viewer/2022062506/5fb47bc829be6b736b3e3795/html5/thumbnails/17.jpg)
Whatifthedataisnotseparable?
![Page 18: Lecture 17 ML supp - GitHub Pages · Lecture 17: Linear Classifiers and Support Vector Machines Theodoros Rekatsinas (lecture by AnkurGoswamimany slides from David Sontag) 1. Today’s](https://reader036.fdocuments.us/reader036/viewer/2022062506/5fb47bc829be6b736b3e3795/html5/thumbnails/18.jpg)
Allowingforslack:“Softmargin”SVM
![Page 19: Lecture 17 ML supp - GitHub Pages · Lecture 17: Linear Classifiers and Support Vector Machines Theodoros Rekatsinas (lecture by AnkurGoswamimany slides from David Sontag) 1. Today’s](https://reader036.fdocuments.us/reader036/viewer/2022062506/5fb47bc829be6b736b3e3795/html5/thumbnails/19.jpg)
Allowingforslack:“Softmargin”SVM
![Page 20: Lecture 17 ML supp - GitHub Pages · Lecture 17: Linear Classifiers and Support Vector Machines Theodoros Rekatsinas (lecture by AnkurGoswamimany slides from David Sontag) 1. Today’s](https://reader036.fdocuments.us/reader036/viewer/2022062506/5fb47bc829be6b736b3e3795/html5/thumbnails/20.jpg)
EquivalentHingeLossFormulation
![Page 21: Lecture 17 ML supp - GitHub Pages · Lecture 17: Linear Classifiers and Support Vector Machines Theodoros Rekatsinas (lecture by AnkurGoswamimany slides from David Sontag) 1. Today’s](https://reader036.fdocuments.us/reader036/viewer/2022062506/5fb47bc829be6b736b3e3795/html5/thumbnails/21.jpg)
HingeLossvs0-1Loss
![Page 22: Lecture 17 ML supp - GitHub Pages · Lecture 17: Linear Classifiers and Support Vector Machines Theodoros Rekatsinas (lecture by AnkurGoswamimany slides from David Sontag) 1. Today’s](https://reader036.fdocuments.us/reader036/viewer/2022062506/5fb47bc829be6b736b3e3795/html5/thumbnails/22.jpg)
MulticlassSVM
![Page 23: Lecture 17 ML supp - GitHub Pages · Lecture 17: Linear Classifiers and Support Vector Machines Theodoros Rekatsinas (lecture by AnkurGoswamimany slides from David Sontag) 1. Today’s](https://reader036.fdocuments.us/reader036/viewer/2022062506/5fb47bc829be6b736b3e3795/html5/thumbnails/23.jpg)
Oneversusallclassification
![Page 24: Lecture 17 ML supp - GitHub Pages · Lecture 17: Linear Classifiers and Support Vector Machines Theodoros Rekatsinas (lecture by AnkurGoswamimany slides from David Sontag) 1. Today’s](https://reader036.fdocuments.us/reader036/viewer/2022062506/5fb47bc829be6b736b3e3795/html5/thumbnails/24.jpg)
MulticlassSVM
![Page 25: Lecture 17 ML supp - GitHub Pages · Lecture 17: Linear Classifiers and Support Vector Machines Theodoros Rekatsinas (lecture by AnkurGoswamimany slides from David Sontag) 1. Today’s](https://reader036.fdocuments.us/reader036/viewer/2022062506/5fb47bc829be6b736b3e3795/html5/thumbnails/25.jpg)
MulticlassSVM
![Page 26: Lecture 17 ML supp - GitHub Pages · Lecture 17: Linear Classifiers and Support Vector Machines Theodoros Rekatsinas (lecture by AnkurGoswamimany slides from David Sontag) 1. Today’s](https://reader036.fdocuments.us/reader036/viewer/2022062506/5fb47bc829be6b736b3e3795/html5/thumbnails/26.jpg)
Whatyouneedtoknow