Nearest Neighbour - University College Londonweb4.cs.ucl.ac.uk › staff › O.Macaodha › ml_intro...
Transcript of Nearest Neighbour - University College Londonweb4.cs.ucl.ac.uk › staff › O.Macaodha › ml_intro...
![Page 1: Nearest Neighbour - University College Londonweb4.cs.ucl.ac.uk › staff › O.Macaodha › ml_intro › 1.3_Nearest Neighbour.pdfEach datapoint has some features and a class label](https://reader030.fdocuments.us/reader030/viewer/2022041100/5ed8d7bd6714ca7f4768ab9a/html5/thumbnails/1.jpg)
Nearest Neighbour
![Page 2: Nearest Neighbour - University College Londonweb4.cs.ucl.ac.uk › staff › O.Macaodha › ml_intro › 1.3_Nearest Neighbour.pdfEach datapoint has some features and a class label](https://reader030.fdocuments.us/reader030/viewer/2022041100/5ed8d7bd6714ca7f4768ab9a/html5/thumbnails/2.jpg)
Here is our 2D dataset, with 3 different classes
![Page 3: Nearest Neighbour - University College Londonweb4.cs.ucl.ac.uk › staff › O.Macaodha › ml_intro › 1.3_Nearest Neighbour.pdfEach datapoint has some features and a class label](https://reader030.fdocuments.us/reader030/viewer/2022041100/5ed8d7bd6714ca7f4768ab9a/html5/thumbnails/3.jpg)
Each datapoint has some features and a class label
(length, width), (species)
![Page 4: Nearest Neighbour - University College Londonweb4.cs.ucl.ac.uk › staff › O.Macaodha › ml_intro › 1.3_Nearest Neighbour.pdfEach datapoint has some features and a class label](https://reader030.fdocuments.us/reader030/viewer/2022041100/5ed8d7bd6714ca7f4768ab9a/html5/thumbnails/4.jpg)
Given a new datapoint, how can we determine its class?
![Page 5: Nearest Neighbour - University College Londonweb4.cs.ucl.ac.uk › staff › O.Macaodha › ml_intro › 1.3_Nearest Neighbour.pdfEach datapoint has some features and a class label](https://reader030.fdocuments.us/reader030/viewer/2022041100/5ed8d7bd6714ca7f4768ab9a/html5/thumbnails/5.jpg)
Find its “Nearest Neighbour” in the feature space
![Page 6: Nearest Neighbour - University College Londonweb4.cs.ucl.ac.uk › staff › O.Macaodha › ml_intro › 1.3_Nearest Neighbour.pdfEach datapoint has some features and a class label](https://reader030.fdocuments.us/reader030/viewer/2022041100/5ed8d7bd6714ca7f4768ab9a/html5/thumbnails/6.jpg)
Computing “similarity” between two points
(l1,w1)
(l2,w2)
![Page 7: Nearest Neighbour - University College Londonweb4.cs.ucl.ac.uk › staff › O.Macaodha › ml_intro › 1.3_Nearest Neighbour.pdfEach datapoint has some features and a class label](https://reader030.fdocuments.us/reader030/viewer/2022041100/5ed8d7bd6714ca7f4768ab9a/html5/thumbnails/7.jpg)
Euclidean distance in 2D
(l1,w1)
(l2,w2)
b
ac
c2 = a2 + b2 i.e Pythagoras' theorem
![Page 8: Nearest Neighbour - University College Londonweb4.cs.ucl.ac.uk › staff › O.Macaodha › ml_intro › 1.3_Nearest Neighbour.pdfEach datapoint has some features and a class label](https://reader030.fdocuments.us/reader030/viewer/2022041100/5ed8d7bd6714ca7f4768ab9a/html5/thumbnails/8.jpg)
Euclidean distance in 2D
(l1,w1)(l2,w2)
ba
cc2 = a2 + b2
![Page 9: Nearest Neighbour - University College Londonweb4.cs.ucl.ac.uk › staff › O.Macaodha › ml_intro › 1.3_Nearest Neighbour.pdfEach datapoint has some features and a class label](https://reader030.fdocuments.us/reader030/viewer/2022041100/5ed8d7bd6714ca7f4768ab9a/html5/thumbnails/9.jpg)
Euclidean distance in 2D
(l1,w1)(l2,w2)
ba
cc2 = a2 + b2
c = √a2 + b2
![Page 10: Nearest Neighbour - University College Londonweb4.cs.ucl.ac.uk › staff › O.Macaodha › ml_intro › 1.3_Nearest Neighbour.pdfEach datapoint has some features and a class label](https://reader030.fdocuments.us/reader030/viewer/2022041100/5ed8d7bd6714ca7f4768ab9a/html5/thumbnails/10.jpg)
Euclidean distance in 2D
(l1,w1)(l2,w2)
ba
cc2 = a2 + b2
c = √a2 + b2
c = √(l1-l2)2 + (w1-w2)
2
![Page 11: Nearest Neighbour - University College Londonweb4.cs.ucl.ac.uk › staff › O.Macaodha › ml_intro › 1.3_Nearest Neighbour.pdfEach datapoint has some features and a class label](https://reader030.fdocuments.us/reader030/viewer/2022041100/5ed8d7bd6714ca7f4768ab9a/html5/thumbnails/11.jpg)
Euclidean distance in 2D
In R:p1 <- c(0, 0)p2 <- c(1, 1)distance <- sqrt(sum((p1-p2)^2))
(l1,w1)(l2,w2)
ba
cc2 = a2 + b2
c = √a2 + b2
c = √(l1-l2)2 + (w1-w2)
2
![Page 12: Nearest Neighbour - University College Londonweb4.cs.ucl.ac.uk › staff › O.Macaodha › ml_intro › 1.3_Nearest Neighbour.pdfEach datapoint has some features and a class label](https://reader030.fdocuments.us/reader030/viewer/2022041100/5ed8d7bd6714ca7f4768ab9a/html5/thumbnails/12.jpg)
Choose the closest point
(l1,w1)
(l2,w2)
(l3,w3)
![Page 13: Nearest Neighbour - University College Londonweb4.cs.ucl.ac.uk › staff › O.Macaodha › ml_intro › 1.3_Nearest Neighbour.pdfEach datapoint has some features and a class label](https://reader030.fdocuments.us/reader030/viewer/2022041100/5ed8d7bd6714ca7f4768ab9a/html5/thumbnails/13.jpg)
Choose the closest point
(l1,w1)
(l2,w2)
(l3,w3)
c1,2
c1,3
![Page 14: Nearest Neighbour - University College Londonweb4.cs.ucl.ac.uk › staff › O.Macaodha › ml_intro › 1.3_Nearest Neighbour.pdfEach datapoint has some features and a class label](https://reader030.fdocuments.us/reader030/viewer/2022041100/5ed8d7bd6714ca7f4768ab9a/html5/thumbnails/14.jpg)
Choose the closest point
(l1,w1)
(l2,w2)
(l3,w3)
c1,2
c1,3
c1,3 < c1,2
![Page 15: Nearest Neighbour - University College Londonweb4.cs.ucl.ac.uk › staff › O.Macaodha › ml_intro › 1.3_Nearest Neighbour.pdfEach datapoint has some features and a class label](https://reader030.fdocuments.us/reader030/viewer/2022041100/5ed8d7bd6714ca7f4768ab9a/html5/thumbnails/15.jpg)
Find its “Nearest Neighbour” in the feature space
![Page 16: Nearest Neighbour - University College Londonweb4.cs.ucl.ac.uk › staff › O.Macaodha › ml_intro › 1.3_Nearest Neighbour.pdfEach datapoint has some features and a class label](https://reader030.fdocuments.us/reader030/viewer/2022041100/5ed8d7bd6714ca7f4768ab9a/html5/thumbnails/16.jpg)
“Nearest Neighbour” in the feature space
![Page 17: Nearest Neighbour - University College Londonweb4.cs.ucl.ac.uk › staff › O.Macaodha › ml_intro › 1.3_Nearest Neighbour.pdfEach datapoint has some features and a class label](https://reader030.fdocuments.us/reader030/viewer/2022041100/5ed8d7bd6714ca7f4768ab9a/html5/thumbnails/17.jpg)
Nearest Neighbour Algorithm
Given a test point x
Compute the distance between x and every other datapoint
The class of x is set as the same as the closest datapoint
![Page 18: Nearest Neighbour - University College Londonweb4.cs.ucl.ac.uk › staff › O.Macaodha › ml_intro › 1.3_Nearest Neighbour.pdfEach datapoint has some features and a class label](https://reader030.fdocuments.us/reader030/viewer/2022041100/5ed8d7bd6714ca7f4768ab9a/html5/thumbnails/18.jpg)
Again our 2D dataset
![Page 19: Nearest Neighbour - University College Londonweb4.cs.ucl.ac.uk › staff › O.Macaodha › ml_intro › 1.3_Nearest Neighbour.pdfEach datapoint has some features and a class label](https://reader030.fdocuments.us/reader030/viewer/2022041100/5ed8d7bd6714ca7f4768ab9a/html5/thumbnails/19.jpg)
Let’s try a different test point
![Page 20: Nearest Neighbour - University College Londonweb4.cs.ucl.ac.uk › staff › O.Macaodha › ml_intro › 1.3_Nearest Neighbour.pdfEach datapoint has some features and a class label](https://reader030.fdocuments.us/reader030/viewer/2022041100/5ed8d7bd6714ca7f4768ab9a/html5/thumbnails/20.jpg)
Here is it’s neighbour
![Page 21: Nearest Neighbour - University College Londonweb4.cs.ucl.ac.uk › staff › O.Macaodha › ml_intro › 1.3_Nearest Neighbour.pdfEach datapoint has some features and a class label](https://reader030.fdocuments.us/reader030/viewer/2022041100/5ed8d7bd6714ca7f4768ab9a/html5/thumbnails/21.jpg)
Resulting Nearest Neighbour classification
For every point in the space we colour it with the class of the datapoint it is closest to.
![Page 22: Nearest Neighbour - University College Londonweb4.cs.ucl.ac.uk › staff › O.Macaodha › ml_intro › 1.3_Nearest Neighbour.pdfEach datapoint has some features and a class label](https://reader030.fdocuments.us/reader030/viewer/2022041100/5ed8d7bd6714ca7f4768ab9a/html5/thumbnails/22.jpg)
Resulting Nearest Neighbour classification
For every point in the space we colour it with the class of the datapoint it is closest to.
![Page 23: Nearest Neighbour - University College Londonweb4.cs.ucl.ac.uk › staff › O.Macaodha › ml_intro › 1.3_Nearest Neighbour.pdfEach datapoint has some features and a class label](https://reader030.fdocuments.us/reader030/viewer/2022041100/5ed8d7bd6714ca7f4768ab9a/html5/thumbnails/23.jpg)
Resulting Nearest Neighbour classification
For every point in the space we colour it with the class of the datapoint it is closest to.
![Page 24: Nearest Neighbour - University College Londonweb4.cs.ucl.ac.uk › staff › O.Macaodha › ml_intro › 1.3_Nearest Neighbour.pdfEach datapoint has some features and a class label](https://reader030.fdocuments.us/reader030/viewer/2022041100/5ed8d7bd6714ca7f4768ab9a/html5/thumbnails/24.jpg)
Practical example
2_nearest_neighbour.R