Nearest Neighbor Algorithm Zaffar Ahmed
-
Upload
zaffar-ahmed-shaikh -
Category
Technology
-
view
1.245 -
download
3
Transcript of Nearest Neighbor Algorithm Zaffar Ahmed
![Page 1: Nearest Neighbor Algorithm Zaffar Ahmed](https://reader036.fdocuments.us/reader036/viewer/2022082605/5562da4cd8b42aac778b4de9/html5/thumbnails/1.jpg)
Nearest Neighbor Algorithm
Zaffar Ahmed Shaikh
![Page 2: Nearest Neighbor Algorithm Zaffar Ahmed](https://reader036.fdocuments.us/reader036/viewer/2022082605/5562da4cd8b42aac778b4de9/html5/thumbnails/2.jpg)
Topics
• Introduction – Memory-based algorithms• K-nearest neighbor (KNN) algorithm• How KNN works?• KNN Example• Different types of KNN
![Page 3: Nearest Neighbor Algorithm Zaffar Ahmed](https://reader036.fdocuments.us/reader036/viewer/2022082605/5562da4cd8b42aac778b4de9/html5/thumbnails/3.jpg)
Introduction
• Memory-based algorithms utilize the entire user-item database to generate a prediction. They find a set of users, known as neighbors, that have a history of agreeing with the target user. Once a neighborhood of users is formed, the preferences of neighbors are combined to produce a prediction or top-K recommendation for the active user.
![Page 4: Nearest Neighbor Algorithm Zaffar Ahmed](https://reader036.fdocuments.us/reader036/viewer/2022082605/5562da4cd8b42aac778b4de9/html5/thumbnails/4.jpg)
K-nearest neighbor (KNN)• The nearest neighbor algorithm measures the distance
dE(Xi,Xj) between query points Xi and a set of training samples Xj to classify a new object based on majority of K-nearest neighbor category of Y attributes of training samples.
Query point Xi = x1, x2, x3, ……….., xn
Training Sample Xj= x1, x2, x3, ……….., xn
Dist(c1,c2) attr
i(c1) attr
i(c2) 2
i1
N
k NearestNeighbors k MIN(Dist(ci,ctest))
predictiontest
1
kclass
ii1
k (or
1
kvalue
ii1
k )
![Page 5: Nearest Neighbor Algorithm Zaffar Ahmed](https://reader036.fdocuments.us/reader036/viewer/2022082605/5562da4cd8b42aac778b4de9/html5/thumbnails/5.jpg)
How KNN works?
1. Determine K (no of nearest neighbors)2. Calculate distance (Euclidean, Manhattan)3. Determine K-minimum distance neighbors4. Gather category Y values of nearest neighbors 5. Use simple majority of nearest neighbors to
predict value of query instance
![Page 6: Nearest Neighbor Algorithm Zaffar Ahmed](https://reader036.fdocuments.us/reader036/viewer/2022082605/5562da4cd8b42aac778b4de9/html5/thumbnails/6.jpg)
KNN Example• Predict who will win today’s Cricket match between India
and Pakistan based on users rating and previous results of matches played between the two teams.
Matches/Teams Who will win?Pakistan?
Who will win?India? Neutral Y (Winner)
1 7 2 1 +
2 3 5 2 -
3 2 6 2 +
4 6 3 1 +
5 4 4 2 -
6 7 2 1 -
7 2 3 4 +
8 4 3 3 ?
![Page 7: Nearest Neighbor Algorithm Zaffar Ahmed](https://reader036.fdocuments.us/reader036/viewer/2022082605/5562da4cd8b42aac778b4de9/html5/thumbnails/7.jpg)
1. Determine K
1. Determine value of K Suppose K = 32. Calculate distanceCoordinates of query instance are (4,3,3)Coordinates of training instance(1) are (7,2,1)D = SQRT ((7-4)2+(2-3) 2+(1-3) 2) = 3.74165
![Page 8: Nearest Neighbor Algorithm Zaffar Ahmed](https://reader036.fdocuments.us/reader036/viewer/2022082605/5562da4cd8b42aac778b4de9/html5/thumbnails/8.jpg)
2. Calculate distance
Matches/Teams
Who will win?Pakistan?
Who will win?India?
Neutral Y (Winner) distance
1 7 2 1 + 3.741657
2 3 5 2 - 2.44949
3 2 6 2 + 3.741657
4 6 3 1 + 2.828427
5 4 4 2 - 1.414214
6 7 2 1 - 3.741657
7 2 3 4 + 2.236068
8 4 3 3 ?
![Page 9: Nearest Neighbor Algorithm Zaffar Ahmed](https://reader036.fdocuments.us/reader036/viewer/2022082605/5562da4cd8b42aac778b4de9/html5/thumbnails/9.jpg)
3. Determine K-minimum distance neighbors
K = 3
Matches/Teams
Who will win?Pakistan?
Who will win?India?
Neutral Y (Winner) distance
1 7 2 1 + 3.741657
2 3 5 2 - 2.44949 (3)
3 2 6 2 + 3.741657
4 6 3 1 + 2.828427
5 4 4 2 - 1.414214 (1)
6 7 2 1 - 3.741657
7 2 3 4 + 2.236068 (2)
8 4 3 3 ?
![Page 10: Nearest Neighbor Algorithm Zaffar Ahmed](https://reader036.fdocuments.us/reader036/viewer/2022082605/5562da4cd8b42aac778b4de9/html5/thumbnails/10.jpg)
4. Gather category Y values of nearest neighbors
Matches/Teams
Who will win?Pakistan?
Who will win?India?
Neutral Y (Winner)
2 3 5 2 -5 4 4 2 -7 2 3 4 +8 4 3 3
![Page 11: Nearest Neighbor Algorithm Zaffar Ahmed](https://reader036.fdocuments.us/reader036/viewer/2022082605/5562da4cd8b42aac778b4de9/html5/thumbnails/11.jpg)
5. Use simple majority of nearest neighbors to predict value of query instance
• Here India has won 2 matches 2 (-) signs and Pakistan has won 1 match 1 (+) sign
• We conclude that India will win today’s match
Matches/Teams Who will win?Pakistan?
Who will win?India? Neutral Y (Winner)
1 7 2 1 +2 3 5 2 -3 2 6 2 +4 6 3 1 +5 4 4 2 -6 7 2 1 -7 2 3 4 +
8 4 3 3 (-)
![Page 12: Nearest Neighbor Algorithm Zaffar Ahmed](https://reader036.fdocuments.us/reader036/viewer/2022082605/5562da4cd8b42aac778b4de9/html5/thumbnails/12.jpg)
Different types of KNN
• KNN for Classification• KNN for Prediction• KNN for Smoothing
![Page 13: Nearest Neighbor Algorithm Zaffar Ahmed](https://reader036.fdocuments.us/reader036/viewer/2022082605/5562da4cd8b42aac778b4de9/html5/thumbnails/13.jpg)
Thank you