Introduction
-
Upload
estevan-gutierrez -
Category
Documents
-
view
23 -
download
1
description
Transcript of Introduction
![Page 2: Introduction](https://reader035.fdocuments.us/reader035/viewer/2022062717/56812a8c550346895d8e33a4/html5/thumbnails/2.jpg)
Pattern recognition and Machine Learning
Syllabus Introduction,Linear Models for classificationNeural Networks (MLP, RBF, SOM, LVQ, ADALINE) Kernel Methods & Support Vector Machines Statistical Pattern Recognition ? (HMM,EM, Clustering and unsupervised learning ? Feature Selection and Dimension reduction ?
![Page 3: Introduction](https://reader035.fdocuments.us/reader035/viewer/2022062717/56812a8c550346895d8e33a4/html5/thumbnails/3.jpg)
Pattern recognition and Machine Learning
TextsR. O. Duda, P. E. Hart, D. G. Stork,
Pattern Classification, 2nd edition, John Wiley & Sons, Inc., 2000.
M. Bishop, Pattern Recognition and Machine Learning, Springer, 2006.
![Page 4: Introduction](https://reader035.fdocuments.us/reader035/viewer/2022062717/56812a8c550346895d8e33a4/html5/thumbnails/4.jpg)
• Midterm 25%• Final 40%• Computer assignments 10%• Final Programming Project 15%• Seminar 10%
Evaluation
![Page 5: Introduction](https://reader035.fdocuments.us/reader035/viewer/2022062717/56812a8c550346895d8e33a4/html5/thumbnails/5.jpg)
Human Perception
Humans have developed highly sophisticated skills for sensing their environment and taking actions according to what they observe, e.g
Understanding spoken wordsreading handwritingdistinguishing fresh food from its smell
We would like to give similar capabilities to machines
![Page 6: Introduction](https://reader035.fdocuments.us/reader035/viewer/2022062717/56812a8c550346895d8e33a4/html5/thumbnails/6.jpg)
What is Pattern Recognition? A pattern is an entity, vaguely defined, that could be given a name, e.g.,
fingerprint image, handwritten word, human face, speech signal, DNA sequence,
Pattern recognition is the study of how machines can observe the environment, learn to distinguish patterns of interest, make sound and reasonable decisions about the categories of the
patterns.
![Page 7: Introduction](https://reader035.fdocuments.us/reader035/viewer/2022062717/56812a8c550346895d8e33a4/html5/thumbnails/7.jpg)
Human and Machine Perception
We are often influenced by the knowledge of how patterns are modeled and recognized in nature when we develop pattern recognition algorithms. Research on machine perception also helps us gain deeper understanding and appreciation for pattern recognition systems in nature. Yet, we also apply many techniques that are purelynumerical and do not have any correspondence in naturalsystems.
![Page 8: Introduction](https://reader035.fdocuments.us/reader035/viewer/2022062717/56812a8c550346895d8e33a4/html5/thumbnails/8.jpg)
![Page 9: Introduction](https://reader035.fdocuments.us/reader035/viewer/2022062717/56812a8c550346895d8e33a4/html5/thumbnails/9.jpg)
Pattern Recognition Applications
![Page 10: Introduction](https://reader035.fdocuments.us/reader035/viewer/2022062717/56812a8c550346895d8e33a4/html5/thumbnails/10.jpg)
Pattern Recognition Applications
![Page 11: Introduction](https://reader035.fdocuments.us/reader035/viewer/2022062717/56812a8c550346895d8e33a4/html5/thumbnails/11.jpg)
Pattern Recognition Applications
![Page 12: Introduction](https://reader035.fdocuments.us/reader035/viewer/2022062717/56812a8c550346895d8e33a4/html5/thumbnails/12.jpg)
Pattern Recognition Applications
![Page 13: Introduction](https://reader035.fdocuments.us/reader035/viewer/2022062717/56812a8c550346895d8e33a4/html5/thumbnails/13.jpg)
Pattern Recognition Applications
![Page 14: Introduction](https://reader035.fdocuments.us/reader035/viewer/2022062717/56812a8c550346895d8e33a4/html5/thumbnails/14.jpg)
Pattern Recognition Applications
![Page 15: Introduction](https://reader035.fdocuments.us/reader035/viewer/2022062717/56812a8c550346895d8e33a4/html5/thumbnails/15.jpg)
Pattern Recognition Applications
![Page 16: Introduction](https://reader035.fdocuments.us/reader035/viewer/2022062717/56812a8c550346895d8e33a4/html5/thumbnails/16.jpg)
Pattern Recognition Applications
![Page 17: Introduction](https://reader035.fdocuments.us/reader035/viewer/2022062717/56812a8c550346895d8e33a4/html5/thumbnails/17.jpg)
Pattern Recognition Applications
Figure 9: Clustering of Microarray Data
![Page 18: Introduction](https://reader035.fdocuments.us/reader035/viewer/2022062717/56812a8c550346895d8e33a4/html5/thumbnails/18.jpg)
Pattern Recognition Applications
Figure 10: Brain Control Interface
![Page 19: Introduction](https://reader035.fdocuments.us/reader035/viewer/2022062717/56812a8c550346895d8e33a4/html5/thumbnails/19.jpg)
![Page 20: Introduction](https://reader035.fdocuments.us/reader035/viewer/2022062717/56812a8c550346895d8e33a4/html5/thumbnails/20.jpg)
![Page 21: Introduction](https://reader035.fdocuments.us/reader035/viewer/2022062717/56812a8c550346895d8e33a4/html5/thumbnails/21.jpg)
![Page 22: Introduction](https://reader035.fdocuments.us/reader035/viewer/2022062717/56812a8c550346895d8e33a4/html5/thumbnails/22.jpg)
![Page 23: Introduction](https://reader035.fdocuments.us/reader035/viewer/2022062717/56812a8c550346895d8e33a4/html5/thumbnails/23.jpg)
![Page 24: Introduction](https://reader035.fdocuments.us/reader035/viewer/2022062717/56812a8c550346895d8e33a4/html5/thumbnails/24.jpg)
![Page 25: Introduction](https://reader035.fdocuments.us/reader035/viewer/2022062717/56812a8c550346895d8e33a4/html5/thumbnails/25.jpg)
![Page 26: Introduction](https://reader035.fdocuments.us/reader035/viewer/2022062717/56812a8c550346895d8e33a4/html5/thumbnails/26.jpg)
![Page 27: Introduction](https://reader035.fdocuments.us/reader035/viewer/2022062717/56812a8c550346895d8e33a4/html5/thumbnails/27.jpg)
![Page 28: Introduction](https://reader035.fdocuments.us/reader035/viewer/2022062717/56812a8c550346895d8e33a4/html5/thumbnails/28.jpg)
![Page 29: Introduction](https://reader035.fdocuments.us/reader035/viewer/2022062717/56812a8c550346895d8e33a4/html5/thumbnails/29.jpg)
![Page 30: Introduction](https://reader035.fdocuments.us/reader035/viewer/2022062717/56812a8c550346895d8e33a4/html5/thumbnails/30.jpg)
![Page 31: Introduction](https://reader035.fdocuments.us/reader035/viewer/2022062717/56812a8c550346895d8e33a4/html5/thumbnails/31.jpg)
![Page 32: Introduction](https://reader035.fdocuments.us/reader035/viewer/2022062717/56812a8c550346895d8e33a4/html5/thumbnails/32.jpg)
![Page 33: Introduction](https://reader035.fdocuments.us/reader035/viewer/2022062717/56812a8c550346895d8e33a4/html5/thumbnails/33.jpg)
![Page 34: Introduction](https://reader035.fdocuments.us/reader035/viewer/2022062717/56812a8c550346895d8e33a4/html5/thumbnails/34.jpg)
![Page 35: Introduction](https://reader035.fdocuments.us/reader035/viewer/2022062717/56812a8c550346895d8e33a4/html5/thumbnails/35.jpg)
Regression: Polynomial Curve Fitting
t is continuous
![Page 36: Introduction](https://reader035.fdocuments.us/reader035/viewer/2022062717/56812a8c550346895d8e33a4/html5/thumbnails/36.jpg)
Sum-of-Squares Error Function
* min ( )w Arg E w Optimization Problem
![Page 37: Introduction](https://reader035.fdocuments.us/reader035/viewer/2022062717/56812a8c550346895d8e33a4/html5/thumbnails/37.jpg)
0th Order Polynomial
![Page 38: Introduction](https://reader035.fdocuments.us/reader035/viewer/2022062717/56812a8c550346895d8e33a4/html5/thumbnails/38.jpg)
1st Order Polynomial
![Page 39: Introduction](https://reader035.fdocuments.us/reader035/viewer/2022062717/56812a8c550346895d8e33a4/html5/thumbnails/39.jpg)
3rd Order Polynomial
![Page 40: Introduction](https://reader035.fdocuments.us/reader035/viewer/2022062717/56812a8c550346895d8e33a4/html5/thumbnails/40.jpg)
9th Order Polynomial
![Page 41: Introduction](https://reader035.fdocuments.us/reader035/viewer/2022062717/56812a8c550346895d8e33a4/html5/thumbnails/41.jpg)
Over-fitting
Root-Mean-Square (RMS) Error:
![Page 42: Introduction](https://reader035.fdocuments.us/reader035/viewer/2022062717/56812a8c550346895d8e33a4/html5/thumbnails/42.jpg)
Polynomial Coefficients
![Page 43: Introduction](https://reader035.fdocuments.us/reader035/viewer/2022062717/56812a8c550346895d8e33a4/html5/thumbnails/43.jpg)
Data Set Size: 9th Order Polynomial
![Page 44: Introduction](https://reader035.fdocuments.us/reader035/viewer/2022062717/56812a8c550346895d8e33a4/html5/thumbnails/44.jpg)
Data Set Size: 9th Order Polynomial
![Page 45: Introduction](https://reader035.fdocuments.us/reader035/viewer/2022062717/56812a8c550346895d8e33a4/html5/thumbnails/45.jpg)
Regularization ;ridge regression
Penalize large coefficient values
Shrinkage: reduce the order of method
~* min ( )w Arg E w
![Page 46: Introduction](https://reader035.fdocuments.us/reader035/viewer/2022062717/56812a8c550346895d8e33a4/html5/thumbnails/46.jpg)
Regularization:
![Page 47: Introduction](https://reader035.fdocuments.us/reader035/viewer/2022062717/56812a8c550346895d8e33a4/html5/thumbnails/47.jpg)
Regularization:
![Page 48: Introduction](https://reader035.fdocuments.us/reader035/viewer/2022062717/56812a8c550346895d8e33a4/html5/thumbnails/48.jpg)
Regularization: vs.
![Page 49: Introduction](https://reader035.fdocuments.us/reader035/viewer/2022062717/56812a8c550346895d8e33a4/html5/thumbnails/49.jpg)
Polynomial Coefficients
Optimization Problem: Finding optimum ,M
![Page 50: Introduction](https://reader035.fdocuments.us/reader035/viewer/2022062717/56812a8c550346895d8e33a4/html5/thumbnails/50.jpg)
Classification example: Handwritten Digit Recognition
28*28 Pixel image : 784 real numbers, training set: 1{ ,.... }Nx x1x
( ),y x t {1,..,9}t
![Page 51: Introduction](https://reader035.fdocuments.us/reader035/viewer/2022062717/56812a8c550346895d8e33a4/html5/thumbnails/51.jpg)
![Page 52: Introduction](https://reader035.fdocuments.us/reader035/viewer/2022062717/56812a8c550346895d8e33a4/html5/thumbnails/52.jpg)
![Page 53: Introduction](https://reader035.fdocuments.us/reader035/viewer/2022062717/56812a8c550346895d8e33a4/html5/thumbnails/53.jpg)
![Page 54: Introduction](https://reader035.fdocuments.us/reader035/viewer/2022062717/56812a8c550346895d8e33a4/html5/thumbnails/54.jpg)
Pattern recognition approaches
![Page 55: Introduction](https://reader035.fdocuments.us/reader035/viewer/2022062717/56812a8c550346895d8e33a4/html5/thumbnails/55.jpg)
Statistical Pattern recognition
![Page 56: Introduction](https://reader035.fdocuments.us/reader035/viewer/2022062717/56812a8c550346895d8e33a4/html5/thumbnails/56.jpg)
Statistical Pattern recognition
![Page 57: Introduction](https://reader035.fdocuments.us/reader035/viewer/2022062717/56812a8c550346895d8e33a4/html5/thumbnails/57.jpg)
Structural Pattern Recognition
![Page 58: Introduction](https://reader035.fdocuments.us/reader035/viewer/2022062717/56812a8c550346895d8e33a4/html5/thumbnails/58.jpg)
Neural Pattern Recognition