A New Unsupervised Competive Learning Algorithm for Vector Quantization
Statistical Learning Theory•Support Vector Machines: Support Vector Classifier, Kernels and...
Transcript of Statistical Learning Theory•Support Vector Machines: Support Vector Classifier, Kernels and...
![Page 1: Statistical Learning Theory•Support Vector Machines: Support Vector Classifier, Kernels and Support Vector Machines. •Unsupervised Learning: Unsupervised Learning and Principal](https://reader033.fdocuments.us/reader033/viewer/2022042219/5ec4dc5871cf24623f2b0765/html5/thumbnails/1.jpg)
Statistical Learning TheoryProf. Giuseppe De Nicolao
![Page 2: Statistical Learning Theory•Support Vector Machines: Support Vector Classifier, Kernels and Support Vector Machines. •Unsupervised Learning: Unsupervised Learning and Principal](https://reader033.fdocuments.us/reader033/viewer/2022042219/5ec4dc5871cf24623f2b0765/html5/thumbnails/2.jpg)
Course Material
• Hastie&Tibshirani’s Slideshttps://www.r-bloggers.com/in-depth-introduction-to-machine-learning-in-15-hours-of-expert-videos/
• Textbook (basic)http://faculty.marshall.usc.edu/gareth-james/ISL/
• Textbook (advanced)https://web.stanford.edu/~hastie/ElemStatLearn//download.html
![Page 3: Statistical Learning Theory•Support Vector Machines: Support Vector Classifier, Kernels and Support Vector Machines. •Unsupervised Learning: Unsupervised Learning and Principal](https://reader033.fdocuments.us/reader033/viewer/2022042219/5ec4dc5871cf24623f2b0765/html5/thumbnails/3.jpg)
Books: basic
![Page 4: Statistical Learning Theory•Support Vector Machines: Support Vector Classifier, Kernels and Support Vector Machines. •Unsupervised Learning: Unsupervised Learning and Principal](https://reader033.fdocuments.us/reader033/viewer/2022042219/5ec4dc5871cf24623f2b0765/html5/thumbnails/4.jpg)
Books: advanced
![Page 5: Statistical Learning Theory•Support Vector Machines: Support Vector Classifier, Kernels and Support Vector Machines. •Unsupervised Learning: Unsupervised Learning and Principal](https://reader033.fdocuments.us/reader033/viewer/2022042219/5ec4dc5871cf24623f2b0765/html5/thumbnails/5.jpg)
Exam
• Written exam• 2 questions about theory (definitions, methods, proofs, ...)• 2 problems
• In addition, there might be• Mini-projects• Contests(1-3 points each)
![Page 6: Statistical Learning Theory•Support Vector Machines: Support Vector Classifier, Kernels and Support Vector Machines. •Unsupervised Learning: Unsupervised Learning and Principal](https://reader033.fdocuments.us/reader033/viewer/2022042219/5ec4dc5871cf24623f2b0765/html5/thumbnails/6.jpg)
Contents (1/2)• Introduction: Supervised and Unsupervised Learning.
• Statistical Learning: Statistical Learning and Regression, Curse of Dimensionality and Parametric Models, Assessing Model Accuracy and Bias-Variance Trade-off, Classification Problems and K-Nearest Neighbors.
• Linear Regression: Simple Linear Regression and Confidence Intervals, Hypothesis Testing, Multiple Linear Regression, Model Selection, Interactions and Nonlinearity.
• Classification: Introduction to Classification, Logistic Regression and Maximum Likelihood, Linear Discriminant Analysis and Bayes Theorem, Naive Bayes.
• Resampling Methods: Estimating Prediction Error and Validation Set Approach, K-fold Cross-Validation, Cross-Validation: The Right and Wrong Ways, The Bootstrap.
• Linear Model Selection and Regularization: Linear Model Selection and Best Subset Selection, Stepwise Selection, Estimating Test Error Using Mallow’s Cp, AIC, BIC, Adjusted R-squared, Cross-Validation, Shrinkage Methods and Ridge Regression, The Lasso, Principal Components Regression and Partial Least Squares.
![Page 7: Statistical Learning Theory•Support Vector Machines: Support Vector Classifier, Kernels and Support Vector Machines. •Unsupervised Learning: Unsupervised Learning and Principal](https://reader033.fdocuments.us/reader033/viewer/2022042219/5ec4dc5871cf24623f2b0765/html5/thumbnails/7.jpg)
Contents (2/2)
• Moving Beyond Linearity: Polynomial Regression, Piecewise Polynomials and Splines, Smoothing Splines, Local Regression and Generalized Additive Models.
• Tree-Based Methods: Decision Trees, Classification Trees and Comparison with Linear Models, Bootstrap Aggregation (Bagging) and Random Forests, Boosting.
• Support Vector Machines: Support Vector Classifier, Kernels and Support Vector Machines.• Unsupervised Learning: Unsupervised Learning and Principal Components Analysis, K-means
Clustering.
• The fallacies of learning: regression to mediocrity, the covariate shift, statistical significance vs practical significance, correlation is not causation, observational vs experimental studies.
![Page 8: Statistical Learning Theory•Support Vector Machines: Support Vector Classifier, Kernels and Support Vector Machines. •Unsupervised Learning: Unsupervised Learning and Principal](https://reader033.fdocuments.us/reader033/viewer/2022042219/5ec4dc5871cf24623f2b0765/html5/thumbnails/8.jpg)
The blog: https://statisticallearningtheory.wordpress.com/
![Page 9: Statistical Learning Theory•Support Vector Machines: Support Vector Classifier, Kernels and Support Vector Machines. •Unsupervised Learning: Unsupervised Learning and Principal](https://reader033.fdocuments.us/reader033/viewer/2022042219/5ec4dc5871cf24623f2b0765/html5/thumbnails/9.jpg)
Links to books, slides, video, etc
![Page 10: Statistical Learning Theory•Support Vector Machines: Support Vector Classifier, Kernels and Support Vector Machines. •Unsupervised Learning: Unsupervised Learning and Principal](https://reader033.fdocuments.us/reader033/viewer/2022042219/5ec4dc5871cf24623f2b0765/html5/thumbnails/10.jpg)
The Facebook page
![Page 11: Statistical Learning Theory•Support Vector Machines: Support Vector Classifier, Kernels and Support Vector Machines. •Unsupervised Learning: Unsupervised Learning and Principal](https://reader033.fdocuments.us/reader033/viewer/2022042219/5ec4dc5871cf24623f2b0765/html5/thumbnails/11.jpg)
Statistical learning
• Supervised statistical learning: building a statistical model for predicting, or estimating, an output based on one or more inputs.
• Unsupervised statistical learning: there are inputs but no supervising output; nevertheless we can learn relationships and structure from such data.