Week 2 - Hacettepe Üniversitesi · Week 2 Aysun Koçak k Nearest 2 Neighbor(k-NN) Linear...

23
Week 2 RA : Aysun Koçak Instructor : Dr. Aykut Erdem Department of Computer Engineering Hacettepe University

Transcript of Week 2 - Hacettepe Üniversitesi · Week 2 Aysun Koçak k Nearest 2 Neighbor(k-NN) Linear...

Page 1: Week 2 - Hacettepe Üniversitesi · Week 2 Aysun Koçak k Nearest 2 Neighbor(k-NN) Linear Regression Note for Assignment Dept. of Computer Engineering Hacettepe University k-Nearest

Week 2

RA : Aysun KoçakInstructor : Dr. Aykut Erdem

Department of Computer EngineeringHacettepe University

Page 2: Week 2 - Hacettepe Üniversitesi · Week 2 Aysun Koçak k Nearest 2 Neighbor(k-NN) Linear Regression Note for Assignment Dept. of Computer Engineering Hacettepe University k-Nearest

22

Week 2

Aysun Koçak

k NearestNeighbor(k-NN)

Linear Regression

Note for Assignment

Dept. of ComputerEngineering

Hacettepe University

Outline

k Nearest Neighbor(k-NN)

Linear Regression

Note for Assignment

Page 3: Week 2 - Hacettepe Üniversitesi · Week 2 Aysun Koçak k Nearest 2 Neighbor(k-NN) Linear Regression Note for Assignment Dept. of Computer Engineering Hacettepe University k-Nearest

22

Week 2

Aysun Koçak

2k NearestNeighbor(k-NN)

Linear Regression

Note for Assignment

Dept. of ComputerEngineering

Hacettepe University

k-Nearest Neighbor(k-NN)

I Nonparametric algorithmI To classify a new point, find its K nearest neighbors from the

training data.

credit - Siddharth Deokar slides

Page 4: Week 2 - Hacettepe Üniversitesi · Week 2 Aysun Koçak k Nearest 2 Neighbor(k-NN) Linear Regression Note for Assignment Dept. of Computer Engineering Hacettepe University k-Nearest

22

Week 2

Aysun Koçak

3k NearestNeighbor(k-NN)

Linear Regression

Note for Assignment

Dept. of ComputerEngineering

Hacettepe University

Code

http://scikit-learn.org/stable/auto_examples/neighbors/plot_classification.html

http://scikit-learn.org/stable/auto_examples/neighbors/plot_classification.html
Page 5: Week 2 - Hacettepe Üniversitesi · Week 2 Aysun Koçak k Nearest 2 Neighbor(k-NN) Linear Regression Note for Assignment Dept. of Computer Engineering Hacettepe University k-Nearest

22

Week 2

Aysun Koçak

4k NearestNeighbor(k-NN)

Linear Regression

Note for Assignment

Dept. of ComputerEngineering

Hacettepe University

k-NN Results

Page 6: Week 2 - Hacettepe Üniversitesi · Week 2 Aysun Koçak k Nearest 2 Neighbor(k-NN) Linear Regression Note for Assignment Dept. of Computer Engineering Hacettepe University k-Nearest

22

Week 2

Aysun Koçak

5k NearestNeighbor(k-NN)

Linear Regression

Note for Assignment

Dept. of ComputerEngineering

Hacettepe University

Drawbacks

I Keep all data to classify a new inputI All neighbors have same importanceI Determining hyper-parameters

I What is the best distance to use?I What is the best value of k to use?

I Distance usually relates to all the attributes and assumes all ofthem have the same effects on distance(curse of dimensionality)

Page 7: Week 2 - Hacettepe Üniversitesi · Week 2 Aysun Koçak k Nearest 2 Neighbor(k-NN) Linear Regression Note for Assignment Dept. of Computer Engineering Hacettepe University k-Nearest

22

Week 2

Aysun Koçak

6k NearestNeighbor(k-NN)

Linear Regression

Note for Assignment

Dept. of ComputerEngineering

Hacettepe University

Determining Parameters

Credit - Fei-Fei Li & Andrej Karpathy slides

Page 8: Week 2 - Hacettepe Üniversitesi · Week 2 Aysun Koçak k Nearest 2 Neighbor(k-NN) Linear Regression Note for Assignment Dept. of Computer Engineering Hacettepe University k-Nearest

22

Week 2

Aysun Koçak

7k NearestNeighbor(k-NN)

Linear Regression

Note for Assignment

Dept. of ComputerEngineering

Hacettepe University

Determining Parameters

Accuracy = (# of correctly classified examples / # of examples) X 100Credit - Fei-Fei Li & Andrej Karpathy slides

Page 9: Week 2 - Hacettepe Üniversitesi · Week 2 Aysun Koçak k Nearest 2 Neighbor(k-NN) Linear Regression Note for Assignment Dept. of Computer Engineering Hacettepe University k-Nearest

22

Week 2

Aysun Koçak

k NearestNeighbor(k-NN)

8Linear Regression

Note for Assignment

Dept. of ComputerEngineering

Hacettepe University

Linear Regression

Credit - Andrew Ng slides

Page 10: Week 2 - Hacettepe Üniversitesi · Week 2 Aysun Koçak k Nearest 2 Neighbor(k-NN) Linear Regression Note for Assignment Dept. of Computer Engineering Hacettepe University k-Nearest

22

Week 2

Aysun Koçak

k NearestNeighbor(k-NN)

9Linear Regression

Note for Assignment

Dept. of ComputerEngineering

Hacettepe University

Linear Regression

Credit - Andrew Ng slides

Page 11: Week 2 - Hacettepe Üniversitesi · Week 2 Aysun Koçak k Nearest 2 Neighbor(k-NN) Linear Regression Note for Assignment Dept. of Computer Engineering Hacettepe University k-Nearest

22

Week 2

Aysun Koçak

k NearestNeighbor(k-NN)

10Linear Regression

Note for Assignment

Dept. of ComputerEngineering

Hacettepe University

Linear Regression

Credit - Andrew Ng slides

Page 12: Week 2 - Hacettepe Üniversitesi · Week 2 Aysun Koçak k Nearest 2 Neighbor(k-NN) Linear Regression Note for Assignment Dept. of Computer Engineering Hacettepe University k-Nearest

22

Week 2

Aysun Koçak

k NearestNeighbor(k-NN)

11Linear Regression

Note for Assignment

Dept. of ComputerEngineering

Hacettepe University

Linear Regression

Credit - Andrew Ng slides

Page 13: Week 2 - Hacettepe Üniversitesi · Week 2 Aysun Koçak k Nearest 2 Neighbor(k-NN) Linear Regression Note for Assignment Dept. of Computer Engineering Hacettepe University k-Nearest

22

Week 2

Aysun Koçak

k NearestNeighbor(k-NN)

12Linear Regression

Note for Assignment

Dept. of ComputerEngineering

Hacettepe University

Linear Regression

Credit - Andrew Ng slides

Page 14: Week 2 - Hacettepe Üniversitesi · Week 2 Aysun Koçak k Nearest 2 Neighbor(k-NN) Linear Regression Note for Assignment Dept. of Computer Engineering Hacettepe University k-Nearest

22

Week 2

Aysun Koçak

k NearestNeighbor(k-NN)

13Linear Regression

Note for Assignment

Dept. of ComputerEngineering

Hacettepe University

Linear Regression

Credit - Andrew Ng slides

Page 15: Week 2 - Hacettepe Üniversitesi · Week 2 Aysun Koçak k Nearest 2 Neighbor(k-NN) Linear Regression Note for Assignment Dept. of Computer Engineering Hacettepe University k-Nearest

22

Week 2

Aysun Koçak

k NearestNeighbor(k-NN)

14Linear Regression

Note for Assignment

Dept. of ComputerEngineering

Hacettepe University

Linear Regression

Credit - Andrew Ng slides

Page 16: Week 2 - Hacettepe Üniversitesi · Week 2 Aysun Koçak k Nearest 2 Neighbor(k-NN) Linear Regression Note for Assignment Dept. of Computer Engineering Hacettepe University k-Nearest

22

Week 2

Aysun Koçak

k NearestNeighbor(k-NN)

15Linear Regression

Note for Assignment

Dept. of ComputerEngineering

Hacettepe University

Linear Regression

Credit - Andrew Ng slides

Page 17: Week 2 - Hacettepe Üniversitesi · Week 2 Aysun Koçak k Nearest 2 Neighbor(k-NN) Linear Regression Note for Assignment Dept. of Computer Engineering Hacettepe University k-Nearest

22

Week 2

Aysun Koçak

k NearestNeighbor(k-NN)

16Linear Regression

Note for Assignment

Dept. of ComputerEngineering

Hacettepe University

Linear Regression

Credit - Andrew Ng slides

Page 18: Week 2 - Hacettepe Üniversitesi · Week 2 Aysun Koçak k Nearest 2 Neighbor(k-NN) Linear Regression Note for Assignment Dept. of Computer Engineering Hacettepe University k-Nearest

22

Week 2

Aysun Koçak

k NearestNeighbor(k-NN)

17Linear Regression

Note for Assignment

Dept. of ComputerEngineering

Hacettepe University

Linear Regression

Credit - Andrew Ng slides

Page 19: Week 2 - Hacettepe Üniversitesi · Week 2 Aysun Koçak k Nearest 2 Neighbor(k-NN) Linear Regression Note for Assignment Dept. of Computer Engineering Hacettepe University k-Nearest

22

Week 2

Aysun Koçak

k NearestNeighbor(k-NN)

18Linear Regression

Note for Assignment

Dept. of ComputerEngineering

Hacettepe University

Linear Regression

Credit - Andrew Ng slides

Page 20: Week 2 - Hacettepe Üniversitesi · Week 2 Aysun Koçak k Nearest 2 Neighbor(k-NN) Linear Regression Note for Assignment Dept. of Computer Engineering Hacettepe University k-Nearest

22

Week 2

Aysun Koçak

k NearestNeighbor(k-NN)

19Linear Regression

Note for Assignment

Dept. of ComputerEngineering

Hacettepe University

Linear Regression

OR

Credit - Andrew Ng slides

Page 21: Week 2 - Hacettepe Üniversitesi · Week 2 Aysun Koçak k Nearest 2 Neighbor(k-NN) Linear Regression Note for Assignment Dept. of Computer Engineering Hacettepe University k-Nearest

22

Week 2

Aysun Koçak

k NearestNeighbor(k-NN)

20Linear Regression

Note for Assignment

Dept. of ComputerEngineering

Hacettepe University

Linear Regression

Page 22: Week 2 - Hacettepe Üniversitesi · Week 2 Aysun Koçak k Nearest 2 Neighbor(k-NN) Linear Regression Note for Assignment Dept. of Computer Engineering Hacettepe University k-Nearest

22

Week 2

Aysun Koçak

k NearestNeighbor(k-NN)

21Linear Regression

Note for Assignment

Dept. of ComputerEngineering

Hacettepe University

Linear Regression

Page 23: Week 2 - Hacettepe Üniversitesi · Week 2 Aysun Koçak k Nearest 2 Neighbor(k-NN) Linear Regression Note for Assignment Dept. of Computer Engineering Hacettepe University k-Nearest

22

Week 2

Aysun Koçak

k NearestNeighbor(k-NN)

Linear Regression

22Note for Assignment

Dept. of ComputerEngineering

Hacettepe University

PCA


Slide DM 09 K-Nearest Neighbour

Slide DM 09 K-Nearest Neighbour

Hierarchical k-nearest neighbours classification and ...

Hierarchical k-nearest neighbours classification and ...

Classification with Nearest Neighbors · Classification with Nearest Neighbors CMSC 422 ... K Nearest Neighbor Classification Training Data K: ... – Classification can be very slow.

Classification with Nearest Neighbors · Classification with Nearest Neighbors CMSC 422 ... K Nearest Neighbor Classification Training Data K: ... – Classification can be very slow.

K Nearest Neighbor Classifiersseem5470/lecture/K-Nearest... · K‐Nearest‐Neighbor Classifiers Framework • Classifiers: ‐memory‐based ‐require no model to be fit • Given

K Nearest Neighbor Classifiersseem5470/lecture/K-Nearest... · K‐Nearest‐Neighbor Classifiers Framework • Classifiers: ‐memory‐based ‐require no model to be fit • Given

Exploiting k-Nearest Neighbor Information with Many Data...Classification with Nearest Neighbors • Use majority voting (k-nearest neighbor classification) • k = 9 (five / four

Exploiting k-Nearest Neighbor Information with Many Data...Classification with Nearest Neighbors • Use majority voting (k-nearest neighbor classification) • k = 9 (five / four

PERBANDINGAN METODE K-NEAREST NEIGHBOR (K-NN ......TESIS – SM 142501 PERBANDINGAN METODE K-NEAREST NEIGHBOR (K-NN) DAN LEARNING VECTOR QUANTIZATION (LVQ) UNTUK PERMASALAHAN KLASIFIKASI

PERBANDINGAN METODE K-NEAREST NEIGHBOR (K-NN ......TESIS – SM 142501 PERBANDINGAN METODE K-NEAREST NEIGHBOR (K-NN) DAN LEARNING VECTOR QUANTIZATION (LVQ) UNTUK PERMASALAHAN KLASIFIKASI

Decreasing Radius K-Nearest Neighbor Search using Mapping ... · decreasing radius k-nearest neighbor search algorithm with the increasing radius k-nearest neighbor search algorithm

Decreasing Radius K-Nearest Neighbor Search using Mapping ... · decreasing radius k-nearest neighbor search algorithm with the increasing radius k-nearest neighbor search algorithm

K-nearest neighbors (Ch. 18.8)

K-nearest neighbors (Ch. 18.8)

K Nearest Neighbors Saed Sayad 1.

K Nearest Neighbors Saed Sayad 1.

L8: Nearest neighbors - Texas A&M Universitycourses.cs.tamu.edu/rgutier/csce666_f16/l8.pdf · L8: Nearest neighbors • Nearest neighbors density estimation • The k nearest neighbors

L8: Nearest neighbors - Texas A&M Universitycourses.cs.tamu.edu/rgutier/csce666_f16/l8.pdf · L8: Nearest neighbors • Nearest neighbors density estimation • The k nearest neighbors

Optimasi K-Nearest Neighbour Menggunakan Particle Swarm ...

Optimasi K-Nearest Neighbour Menggunakan Particle Swarm ...

IMPLEMENTATION OF K-NEAREST NEIGHBOR ALGORITHM IN ...

IMPLEMENTATION OF K-NEAREST NEIGHBOR ALGORITHM IN ...

Scanned Document - oyp.hacettepe.edu.tr · Universite Hacettepe Ûniversites1 Hacettepe ÛniverS1tesi Hacettepe Universitesi Hacettepe Ûniversitesi Haccttepe Üniversitesi Hacettcpe

Scanned Document - oyp.hacettepe.edu.tr · Universite Hacettepe Ûniversites1 Hacettepe ÛniverS1tesi Hacettepe Universitesi Hacettepe Ûniversitesi Haccttepe Üniversitesi Hacettcpe

Exact bootstrap k-nearest neighbor learners

Exact bootstrap k-nearest neighbor learners

Adaptive k-Nearest-Neighbor Classification Using a …delab.csd.auth.gr/~apostol/pubs/adbis2007.pdfAdaptive k-Nearest-Neighbor Classification Using a Dynamic Number of Nearest Neighbors?

Adaptive k-Nearest-Neighbor Classification Using a …delab.csd.auth.gr/~apostol/pubs/adbis2007.pdfAdaptive k-Nearest-Neighbor Classification Using a Dynamic Number of Nearest Neighbors?

Overfitting and K-Nearest Neighbour · 2019. 10. 16. · Overfitting and K-Nearest Neighbour Dr. Xiaowei Huang xiaowei

Overfitting and K-Nearest Neighbour · 2019. 10. 16. · Overfitting and K-Nearest Neighbour Dr. Xiaowei Huang xiaowei

Classification of Stock Index movement using k-Nearest Neighbours (k-NN) algorithm · 2012. 12. 10. · Classification of Stock Index movement using k-Nearest Neighbours (k-NN) algorithm

Classification of Stock Index movement using k-Nearest Neighbours (k-NN) algorithm · 2012. 12. 10. · Classification of Stock Index movement using k-Nearest Neighbours (k-NN) algorithm

CLASSIFICATION & PREDICTION K-NEAREST NEIGHBORSmaukar.staff.gunadarma.ac.id/Downloads/files/44232/... · CLASSIFICATION & PREDICTION K-NEAREST NEIGHBORS Ericks Universitas Gunadarma

CLASSIFICATION & PREDICTION K-NEAREST NEIGHBORSmaukar.staff.gunadarma.ac.id/Downloads/files/44232/... · CLASSIFICATION & PREDICTION K-NEAREST NEIGHBORS Ericks Universitas Gunadarma

K-Nearest Neighborsjbhuang/teaching/ECE...K-Nearest Neighbors Jia-Bin Huang ECE-5424G / CS-5824 Virginia Tech Spring 2019. ... •K-Nearest Neighbor (kNN) •Distance metric •Pros/Cons

K-Nearest Neighborsjbhuang/teaching/ECE...K-Nearest Neighbors Jia-Bin Huang ECE-5424G / CS-5824 Virginia Tech Spring 2019. ... •K-Nearest Neighbor (kNN) •Distance metric •Pros/Cons

K-Nearest Neighbors (kNN)

K-Nearest Neighbors (kNN)