Artificial Intelligence - An Introduction to Machine...
Transcript of Artificial Intelligence - An Introduction to Machine...
![Page 1: Artificial Intelligence - An Introduction to Machine Learningstephane.ayache.perso.luminy.univ-amu.fr/zoom/cours/... · 2019-03-12 · Python: Key Concepts Identifiers Python entities](https://reader030.fdocuments.us/reader030/viewer/2022040120/5e7c78b2e88d3a2e807b5d02/html5/thumbnails/1.jpg)
Artificial Intelligence- An Introduction to Machine Learning -
Hachem Kadri
Aix-Marseille University, CNRSLaboratoire d’Informatique et des Systemes, LIS
QARMA teamhttps://qarma.lis-lab.fr/
March, 2019
![Page 3: Artificial Intelligence - An Introduction to Machine Learningstephane.ayache.perso.luminy.univ-amu.fr/zoom/cours/... · 2019-03-12 · Python: Key Concepts Identifiers Python entities](https://reader030.fdocuments.us/reader030/viewer/2022040120/5e7c78b2e88d3a2e807b5d02/html5/thumbnails/3.jpg)
Published Papers: Papers by topic
I AI papers on Scopus have increased 8x since 1996I CS papers increased 6x during the same timeframe
![Page 4: Artificial Intelligence - An Introduction to Machine Learningstephane.ayache.perso.luminy.univ-amu.fr/zoom/cours/... · 2019-03-12 · Python: Key Concepts Identifiers Python entities](https://reader030.fdocuments.us/reader030/viewer/2022040120/5e7c78b2e88d3a2e807b5d02/html5/thumbnails/4.jpg)
Published Papers: AI papers by region
I In 2017, 28% of AI papers on Scopus were affiliated withEuropean authors, followed by China (25%) and theU.S. (17%).
![Page 5: Artificial Intelligence - An Introduction to Machine Learningstephane.ayache.perso.luminy.univ-amu.fr/zoom/cours/... · 2019-03-12 · Python: Key Concepts Identifiers Python entities](https://reader030.fdocuments.us/reader030/viewer/2022040120/5e7c78b2e88d3a2e807b5d02/html5/thumbnails/5.jpg)
Published Papers: AI papers by subcategory
I 56% of papers fell into the Machine Learning and ProbabilisticReasoning category in 2017, compared to 28% in 2010.
![Page 6: Artificial Intelligence - An Introduction to Machine Learningstephane.ayache.perso.luminy.univ-amu.fr/zoom/cours/... · 2019-03-12 · Python: Key Concepts Identifiers Python entities](https://reader030.fdocuments.us/reader030/viewer/2022040120/5e7c78b2e88d3a2e807b5d02/html5/thumbnails/6.jpg)
Participation: Large AI conferences
![Page 7: Artificial Intelligence - An Introduction to Machine Learningstephane.ayache.perso.luminy.univ-amu.fr/zoom/cours/... · 2019-03-12 · Python: Key Concepts Identifiers Python entities](https://reader030.fdocuments.us/reader030/viewer/2022040120/5e7c78b2e88d3a2e807b5d02/html5/thumbnails/7.jpg)
Participation: Small AI conferences
![Page 8: Artificial Intelligence - An Introduction to Machine Learningstephane.ayache.perso.luminy.univ-amu.fr/zoom/cours/... · 2019-03-12 · Python: Key Concepts Identifiers Python entities](https://reader030.fdocuments.us/reader030/viewer/2022040120/5e7c78b2e88d3a2e807b5d02/html5/thumbnails/8.jpg)
Jobs: Openings by AI skill
![Page 9: Artificial Intelligence - An Introduction to Machine Learningstephane.ayache.perso.luminy.univ-amu.fr/zoom/cours/... · 2019-03-12 · Python: Key Concepts Identifiers Python entities](https://reader030.fdocuments.us/reader030/viewer/2022040120/5e7c78b2e88d3a2e807b5d02/html5/thumbnails/9.jpg)
Act I: Machine Learning
Overview and basic concepts
![Page 10: Artificial Intelligence - An Introduction to Machine Learningstephane.ayache.perso.luminy.univ-amu.fr/zoom/cours/... · 2019-03-12 · Python: Key Concepts Identifiers Python entities](https://reader030.fdocuments.us/reader030/viewer/2022040120/5e7c78b2e88d3a2e807b5d02/html5/thumbnails/10.jpg)
What is Machine Learning?
“Field of study that gives computers the ability to learnwithout being explicitly programmed.”
– Arthur Samuel, 1959
![Page 11: Artificial Intelligence - An Introduction to Machine Learningstephane.ayache.perso.luminy.univ-amu.fr/zoom/cours/... · 2019-03-12 · Python: Key Concepts Identifiers Python entities](https://reader030.fdocuments.us/reader030/viewer/2022040120/5e7c78b2e88d3a2e807b5d02/html5/thumbnails/11.jpg)
Learning versus Programming
AlgorithmA sequence of instructions that are carried out to transform theinput to the output.
Sorting
5 1 12 -5 16 -5 1 5 12 16
unsorted sorted
Letter Recognition
Alphabet in a bitmap representation
![Page 12: Artificial Intelligence - An Introduction to Machine Learningstephane.ayache.perso.luminy.univ-amu.fr/zoom/cours/... · 2019-03-12 · Python: Key Concepts Identifiers Python entities](https://reader030.fdocuments.us/reader030/viewer/2022040120/5e7c78b2e88d3a2e807b5d02/html5/thumbnails/12.jpg)
When do we need Machine Learning?
Tasks that are too complex to program
Spam filter (from A. Geron, 2017)
![Page 13: Artificial Intelligence - An Introduction to Machine Learningstephane.ayache.perso.luminy.univ-amu.fr/zoom/cours/... · 2019-03-12 · Python: Key Concepts Identifiers Python entities](https://reader030.fdocuments.us/reader030/viewer/2022040120/5e7c78b2e88d3a2e807b5d02/html5/thumbnails/13.jpg)
Machine Learning
“is the study of computer algorithms that improveautomatically through experience.”
– Tom Mitchell, 1997
![Page 14: Artificial Intelligence - An Introduction to Machine Learningstephane.ayache.perso.luminy.univ-amu.fr/zoom/cours/... · 2019-03-12 · Python: Key Concepts Identifiers Python entities](https://reader030.fdocuments.us/reader030/viewer/2022040120/5e7c78b2e88d3a2e807b5d02/html5/thumbnails/14.jpg)
Recommendation System
![Page 16: Artificial Intelligence - An Introduction to Machine Learningstephane.ayache.perso.luminy.univ-amu.fr/zoom/cours/... · 2019-03-12 · Python: Key Concepts Identifiers Python entities](https://reader030.fdocuments.us/reader030/viewer/2022040120/5e7c78b2e88d3a2e807b5d02/html5/thumbnails/16.jpg)
Collaborative Filtering
https://en.wikipedia.org/wiki/Collaborative_filtering
![Page 17: Artificial Intelligence - An Introduction to Machine Learningstephane.ayache.perso.luminy.univ-amu.fr/zoom/cours/... · 2019-03-12 · Python: Key Concepts Identifiers Python entities](https://reader030.fdocuments.us/reader030/viewer/2022040120/5e7c78b2e88d3a2e807b5d02/html5/thumbnails/17.jpg)
Collaborative Filtering
https://en.wikipedia.org/wiki/Collaborative_filtering
![Page 18: Artificial Intelligence - An Introduction to Machine Learningstephane.ayache.perso.luminy.univ-amu.fr/zoom/cours/... · 2019-03-12 · Python: Key Concepts Identifiers Python entities](https://reader030.fdocuments.us/reader030/viewer/2022040120/5e7c78b2e88d3a2e807b5d02/html5/thumbnails/18.jpg)
Collaborative Filtering
https://en.wikipedia.org/wiki/Collaborative_filtering
![Page 19: Artificial Intelligence - An Introduction to Machine Learningstephane.ayache.perso.luminy.univ-amu.fr/zoom/cours/... · 2019-03-12 · Python: Key Concepts Identifiers Python entities](https://reader030.fdocuments.us/reader030/viewer/2022040120/5e7c78b2e88d3a2e807b5d02/html5/thumbnails/19.jpg)
Collaborative Filtering
https://en.wikipedia.org/wiki/Collaborative_filtering
![Page 20: Artificial Intelligence - An Introduction to Machine Learningstephane.ayache.perso.luminy.univ-amu.fr/zoom/cours/... · 2019-03-12 · Python: Key Concepts Identifiers Python entities](https://reader030.fdocuments.us/reader030/viewer/2022040120/5e7c78b2e88d3a2e807b5d02/html5/thumbnails/20.jpg)
Collaborative Filtering
https://en.wikipedia.org/wiki/Collaborative_filtering
![Page 21: Artificial Intelligence - An Introduction to Machine Learningstephane.ayache.perso.luminy.univ-amu.fr/zoom/cours/... · 2019-03-12 · Python: Key Concepts Identifiers Python entities](https://reader030.fdocuments.us/reader030/viewer/2022040120/5e7c78b2e88d3a2e807b5d02/html5/thumbnails/21.jpg)
Collaborative Filtering
https://en.wikipedia.org/wiki/Collaborative_filtering
![Page 22: Artificial Intelligence - An Introduction to Machine Learningstephane.ayache.perso.luminy.univ-amu.fr/zoom/cours/... · 2019-03-12 · Python: Key Concepts Identifiers Python entities](https://reader030.fdocuments.us/reader030/viewer/2022040120/5e7c78b2e88d3a2e807b5d02/html5/thumbnails/22.jpg)
Collaborative Filtering
https://en.wikipedia.org/wiki/Collaborative_filtering
![Page 23: Artificial Intelligence - An Introduction to Machine Learningstephane.ayache.perso.luminy.univ-amu.fr/zoom/cours/... · 2019-03-12 · Python: Key Concepts Identifiers Python entities](https://reader030.fdocuments.us/reader030/viewer/2022040120/5e7c78b2e88d3a2e807b5d02/html5/thumbnails/23.jpg)
Collaborative Filtering
https://en.wikipedia.org/wiki/Collaborative_filtering
![Page 24: Artificial Intelligence - An Introduction to Machine Learningstephane.ayache.perso.luminy.univ-amu.fr/zoom/cours/... · 2019-03-12 · Python: Key Concepts Identifiers Python entities](https://reader030.fdocuments.us/reader030/viewer/2022040120/5e7c78b2e88d3a2e807b5d02/html5/thumbnails/24.jpg)
Collaborative Filtering
https://en.wikipedia.org/wiki/Collaborative_filtering
![Page 25: Artificial Intelligence - An Introduction to Machine Learningstephane.ayache.perso.luminy.univ-amu.fr/zoom/cours/... · 2019-03-12 · Python: Key Concepts Identifiers Python entities](https://reader030.fdocuments.us/reader030/viewer/2022040120/5e7c78b2e88d3a2e807b5d02/html5/thumbnails/25.jpg)
Collaborative Filtering
https://en.wikipedia.org/wiki/Collaborative_filtering
![Page 26: Artificial Intelligence - An Introduction to Machine Learningstephane.ayache.perso.luminy.univ-amu.fr/zoom/cours/... · 2019-03-12 · Python: Key Concepts Identifiers Python entities](https://reader030.fdocuments.us/reader030/viewer/2022040120/5e7c78b2e88d3a2e807b5d02/html5/thumbnails/26.jpg)
Collaborative Filtering
https://en.wikipedia.org/wiki/Collaborative_filtering
![Page 27: Artificial Intelligence - An Introduction to Machine Learningstephane.ayache.perso.luminy.univ-amu.fr/zoom/cours/... · 2019-03-12 · Python: Key Concepts Identifiers Python entities](https://reader030.fdocuments.us/reader030/viewer/2022040120/5e7c78b2e88d3a2e807b5d02/html5/thumbnails/27.jpg)
Collaborative Filtering
https://en.wikipedia.org/wiki/Collaborative_filtering
![Page 28: Artificial Intelligence - An Introduction to Machine Learningstephane.ayache.perso.luminy.univ-amu.fr/zoom/cours/... · 2019-03-12 · Python: Key Concepts Identifiers Python entities](https://reader030.fdocuments.us/reader030/viewer/2022040120/5e7c78b2e88d3a2e807b5d02/html5/thumbnails/28.jpg)
Collaborative Filtering
https://en.wikipedia.org/wiki/Collaborative_filtering
![Page 29: Artificial Intelligence - An Introduction to Machine Learningstephane.ayache.perso.luminy.univ-amu.fr/zoom/cours/... · 2019-03-12 · Python: Key Concepts Identifiers Python entities](https://reader030.fdocuments.us/reader030/viewer/2022040120/5e7c78b2e88d3a2e807b5d02/html5/thumbnails/29.jpg)
Collaborative Filtering
https://en.wikipedia.org/wiki/Collaborative_filtering
![Page 30: Artificial Intelligence - An Introduction to Machine Learningstephane.ayache.perso.luminy.univ-amu.fr/zoom/cours/... · 2019-03-12 · Python: Key Concepts Identifiers Python entities](https://reader030.fdocuments.us/reader030/viewer/2022040120/5e7c78b2e88d3a2e807b5d02/html5/thumbnails/30.jpg)
Collaborative Filtering
https://en.wikipedia.org/wiki/Collaborative_filtering
![Page 31: Artificial Intelligence - An Introduction to Machine Learningstephane.ayache.perso.luminy.univ-amu.fr/zoom/cours/... · 2019-03-12 · Python: Key Concepts Identifiers Python entities](https://reader030.fdocuments.us/reader030/viewer/2022040120/5e7c78b2e88d3a2e807b5d02/html5/thumbnails/31.jpg)
Collaborative Filtering
https://en.wikipedia.org/wiki/Collaborative_filtering
![Page 32: Artificial Intelligence - An Introduction to Machine Learningstephane.ayache.perso.luminy.univ-amu.fr/zoom/cours/... · 2019-03-12 · Python: Key Concepts Identifiers Python entities](https://reader030.fdocuments.us/reader030/viewer/2022040120/5e7c78b2e88d3a2e807b5d02/html5/thumbnails/32.jpg)
Collaborative Filtering
https://en.wikipedia.org/wiki/Collaborative_filtering
![Page 33: Artificial Intelligence - An Introduction to Machine Learningstephane.ayache.perso.luminy.univ-amu.fr/zoom/cours/... · 2019-03-12 · Python: Key Concepts Identifiers Python entities](https://reader030.fdocuments.us/reader030/viewer/2022040120/5e7c78b2e88d3a2e807b5d02/html5/thumbnails/33.jpg)
. . . , among others
Machine LearningFoundations
ComputationalIntelligence
Neural Networks
Optimization
Support Vector Machines
Linear Algebra
Spectral Methods
ComputationalStatistics
Sparsity & Feature Selection
![Page 34: Artificial Intelligence - An Introduction to Machine Learningstephane.ayache.perso.luminy.univ-amu.fr/zoom/cours/... · 2019-03-12 · Python: Key Concepts Identifiers Python entities](https://reader030.fdocuments.us/reader030/viewer/2022040120/5e7c78b2e88d3a2e807b5d02/html5/thumbnails/34.jpg)
. . . , among others
AppliedMachine Learning
Speech and AudioProcessing
Speech Recognition
Neuroscience
Brain Imaging
Computer Vision
Image Annotation
Natural LanguageProcessing
Machine Translation
![Page 35: Artificial Intelligence - An Introduction to Machine Learningstephane.ayache.perso.luminy.univ-amu.fr/zoom/cours/... · 2019-03-12 · Python: Key Concepts Identifiers Python entities](https://reader030.fdocuments.us/reader030/viewer/2022040120/5e7c78b2e88d3a2e807b5d02/html5/thumbnails/35.jpg)
Image Annotation (from L. Ralaivola)
(from Farabet et al, 2013)
(from Haxby et al, 2001)
![Page 36: Artificial Intelligence - An Introduction to Machine Learningstephane.ayache.perso.luminy.univ-amu.fr/zoom/cours/... · 2019-03-12 · Python: Key Concepts Identifiers Python entities](https://reader030.fdocuments.us/reader030/viewer/2022040120/5e7c78b2e88d3a2e807b5d02/html5/thumbnails/36.jpg)
P300 Speller (from L. Ralaivola)
Vintage P300 Speller
(from Breaking bad)
Modern P300 Speller (from A. Rakotomamonjy)
![Page 37: Artificial Intelligence - An Introduction to Machine Learningstephane.ayache.perso.luminy.univ-amu.fr/zoom/cours/... · 2019-03-12 · Python: Key Concepts Identifiers Python entities](https://reader030.fdocuments.us/reader030/viewer/2022040120/5e7c78b2e88d3a2e807b5d02/html5/thumbnails/37.jpg)
AlphaGo (Silver et al. 2016)
![Page 38: Artificial Intelligence - An Introduction to Machine Learningstephane.ayache.perso.luminy.univ-amu.fr/zoom/cours/... · 2019-03-12 · Python: Key Concepts Identifiers Python entities](https://reader030.fdocuments.us/reader030/viewer/2022040120/5e7c78b2e88d3a2e807b5d02/html5/thumbnails/38.jpg)
Main Types of Learning
Machine Learning
UnsupervisedSupervised Reinforcement
Task Driven Learn from MistakesData Driven
This course
![Page 39: Artificial Intelligence - An Introduction to Machine Learningstephane.ayache.perso.luminy.univ-amu.fr/zoom/cours/... · 2019-03-12 · Python: Key Concepts Identifiers Python entities](https://reader030.fdocuments.us/reader030/viewer/2022040120/5e7c78b2e88d3a2e807b5d02/html5/thumbnails/39.jpg)
Supervised Learning
Generalize from training to testing
![Page 40: Artificial Intelligence - An Introduction to Machine Learningstephane.ayache.perso.luminy.univ-amu.fr/zoom/cours/... · 2019-03-12 · Python: Key Concepts Identifiers Python entities](https://reader030.fdocuments.us/reader030/viewer/2022040120/5e7c78b2e88d3a2e807b5d02/html5/thumbnails/40.jpg)
Supervised Learning – Generalization
From a training set consisting of randomly sampled pairs of (input,target), learn a function or a predictor which predicts well thetarget of a new data.
Supervised learning / Generalization
−→ Given l training examples (x1, y1), . . . , (xl, yl) ∈ (X × Y) andu test data xl+1, . . . , xl+u ∈ X
−→ Learn f : X → Y to generalize from training to testing
![Page 41: Artificial Intelligence - An Introduction to Machine Learningstephane.ayache.perso.luminy.univ-amu.fr/zoom/cours/... · 2019-03-12 · Python: Key Concepts Identifiers Python entities](https://reader030.fdocuments.us/reader030/viewer/2022040120/5e7c78b2e88d3a2e807b5d02/html5/thumbnails/41.jpg)
Focus: binary classification
Important notions in learning to classifyI a number of training data (pictures, emails, etc.)I learning algorithm (how to build the classifier?)I generalization: the classifier should correctly classify test data
Quick formalizationI X (e.g. Rd , d > 0) is the
space of data, called inputspace
I Y (e.g. toxic/not toxic, or{−1, +1}) is the targetspace
I f : X → Y is the classifier
?class +1class -1
f(x) = 0X (= R 2)
![Page 42: Artificial Intelligence - An Introduction to Machine Learningstephane.ayache.perso.luminy.univ-amu.fr/zoom/cours/... · 2019-03-12 · Python: Key Concepts Identifiers Python entities](https://reader030.fdocuments.us/reader030/viewer/2022040120/5e7c78b2e88d3a2e807b5d02/html5/thumbnails/42.jpg)
Perceptron (Rosenblatt, 1958)
Inspiration: biological neural networkMotivations:
I Learning systemcomposed by associatingsimple processing units
I Efficiency, scalability, andadaptability
Perceptron: a linear classifier, X = Rd , Y = {−1, +1}biais : activation = 1
σ(∑d
i=1wixi + w0)x1
x2x =
σ
w0
w1
w2
![Page 43: Artificial Intelligence - An Introduction to Machine Learningstephane.ayache.perso.luminy.univ-amu.fr/zoom/cours/... · 2019-03-12 · Python: Key Concepts Identifiers Python entities](https://reader030.fdocuments.us/reader030/viewer/2022040120/5e7c78b2e88d3a2e807b5d02/html5/thumbnails/43.jpg)
Perceptron (Rosenblatt, 1958)
A linear classifier, X = Rd , Y = {−1, +1}
I Classifier weights: w ∈ Rd
I Classifier prediction: f (x) = sign〈w, x〉I Question: how to learn w from training data
Algorithm: Perceptron
Input: S = {(Xn, Yn)}Nn=1w← 0while it exists (Xn, Yn): Yn〈w, Xn〉 ≤ 0 do
w← w + YnXnend while
![Page 44: Artificial Intelligence - An Introduction to Machine Learningstephane.ayache.perso.luminy.univ-amu.fr/zoom/cours/... · 2019-03-12 · Python: Key Concepts Identifiers Python entities](https://reader030.fdocuments.us/reader030/viewer/2022040120/5e7c78b2e88d3a2e807b5d02/html5/thumbnails/44.jpg)
Perceptron in action
![Page 45: Artificial Intelligence - An Introduction to Machine Learningstephane.ayache.perso.luminy.univ-amu.fr/zoom/cours/... · 2019-03-12 · Python: Key Concepts Identifiers Python entities](https://reader030.fdocuments.us/reader030/viewer/2022040120/5e7c78b2e88d3a2e807b5d02/html5/thumbnails/45.jpg)
Perceptron in action
![Page 46: Artificial Intelligence - An Introduction to Machine Learningstephane.ayache.perso.luminy.univ-amu.fr/zoom/cours/... · 2019-03-12 · Python: Key Concepts Identifiers Python entities](https://reader030.fdocuments.us/reader030/viewer/2022040120/5e7c78b2e88d3a2e807b5d02/html5/thumbnails/46.jpg)
Perceptron in action
![Page 47: Artificial Intelligence - An Introduction to Machine Learningstephane.ayache.perso.luminy.univ-amu.fr/zoom/cours/... · 2019-03-12 · Python: Key Concepts Identifiers Python entities](https://reader030.fdocuments.us/reader030/viewer/2022040120/5e7c78b2e88d3a2e807b5d02/html5/thumbnails/47.jpg)
Perceptron in action
![Page 48: Artificial Intelligence - An Introduction to Machine Learningstephane.ayache.perso.luminy.univ-amu.fr/zoom/cours/... · 2019-03-12 · Python: Key Concepts Identifiers Python entities](https://reader030.fdocuments.us/reader030/viewer/2022040120/5e7c78b2e88d3a2e807b5d02/html5/thumbnails/48.jpg)
Perceptron: limitations
Theorem (XOR, Minsky, Papert, 1969)The perceptron algorithm cannot solve the XOR problem
How to address this issue? See Part III
![Page 49: Artificial Intelligence - An Introduction to Machine Learningstephane.ayache.perso.luminy.univ-amu.fr/zoom/cours/... · 2019-03-12 · Python: Key Concepts Identifiers Python entities](https://reader030.fdocuments.us/reader030/viewer/2022040120/5e7c78b2e88d3a2e807b5d02/html5/thumbnails/49.jpg)
In the following . . .
“A computer program is said to learn from experience E withrespect to some class of tasks T and performance measure P, if itsperformance at tasks in T, as measured by P, improves withexperience E”
Task Supervised learning, binaryand multi-class classification
Expe-rience
Training data, instances, labels
Perfor-mance
Correct classificationrate on the test set
![Page 50: Artificial Intelligence - An Introduction to Machine Learningstephane.ayache.perso.luminy.univ-amu.fr/zoom/cours/... · 2019-03-12 · Python: Key Concepts Identifiers Python entities](https://reader030.fdocuments.us/reader030/viewer/2022040120/5e7c78b2e88d3a2e807b5d02/html5/thumbnails/50.jpg)
Supervised Machine Learning Pipeline
(from Sarkar et al., 2018)
![Page 51: Artificial Intelligence - An Introduction to Machine Learningstephane.ayache.perso.luminy.univ-amu.fr/zoom/cours/... · 2019-03-12 · Python: Key Concepts Identifiers Python entities](https://reader030.fdocuments.us/reader030/viewer/2022040120/5e7c78b2e88d3a2e807b5d02/html5/thumbnails/51.jpg)
Validation Set
(from Raschka, 2017)
![Page 52: Artificial Intelligence - An Introduction to Machine Learningstephane.ayache.perso.luminy.univ-amu.fr/zoom/cours/... · 2019-03-12 · Python: Key Concepts Identifiers Python entities](https://reader030.fdocuments.us/reader030/viewer/2022040120/5e7c78b2e88d3a2e807b5d02/html5/thumbnails/52.jpg)
k-fold Cross-validation (k=10)
(from Raschka, 2017)
![Page 53: Artificial Intelligence - An Introduction to Machine Learningstephane.ayache.perso.luminy.univ-amu.fr/zoom/cours/... · 2019-03-12 · Python: Key Concepts Identifiers Python entities](https://reader030.fdocuments.us/reader030/viewer/2022040120/5e7c78b2e88d3a2e807b5d02/html5/thumbnails/53.jpg)
Act II: Training Machine Learning Algorithmsfor Classification and Regression
![Page 54: Artificial Intelligence - An Introduction to Machine Learningstephane.ayache.perso.luminy.univ-amu.fr/zoom/cours/... · 2019-03-12 · Python: Key Concepts Identifiers Python entities](https://reader030.fdocuments.us/reader030/viewer/2022040120/5e7c78b2e88d3a2e807b5d02/html5/thumbnails/54.jpg)
Decision Trees
Should we play tennis or not?
Play Wind Humidity Outlook
No Low High Sunny
No High Normal Rain
Yes Low High Overcast
Yes Weak Normal Rain
Yes Low Normal Sunny
Yes Low Normal Overcast
Yes High Normal Sunny
![Page 55: Artificial Intelligence - An Introduction to Machine Learningstephane.ayache.perso.luminy.univ-amu.fr/zoom/cours/... · 2019-03-12 · Python: Key Concepts Identifiers Python entities](https://reader030.fdocuments.us/reader030/viewer/2022040120/5e7c78b2e88d3a2e807b5d02/html5/thumbnails/55.jpg)
Decision Trees
I breaking down our data by making decision based on asking aseries of questions
![Page 56: Artificial Intelligence - An Introduction to Machine Learningstephane.ayache.perso.luminy.univ-amu.fr/zoom/cours/... · 2019-03-12 · Python: Key Concepts Identifiers Python entities](https://reader030.fdocuments.us/reader030/viewer/2022040120/5e7c78b2e88d3a2e807b5d02/html5/thumbnails/56.jpg)
K-nearest neighbors
I Choose the number of k and a distance metric
I Find the k-nearest neighbors of the sample to be classified
I Assign the class label by majority vote
![Page 57: Artificial Intelligence - An Introduction to Machine Learningstephane.ayache.perso.luminy.univ-amu.fr/zoom/cours/... · 2019-03-12 · Python: Key Concepts Identifiers Python entities](https://reader030.fdocuments.us/reader030/viewer/2022040120/5e7c78b2e88d3a2e807b5d02/html5/thumbnails/57.jpg)
Perceptron
1. Initialize the weights w to 0 or small random numbers
2. For each training sample Xn :
a. Compute the output value Yn , sign(w>Xn)
b. Update the weights if Yn 6= Yn,
![Page 58: Artificial Intelligence - An Introduction to Machine Learningstephane.ayache.perso.luminy.univ-amu.fr/zoom/cours/... · 2019-03-12 · Python: Key Concepts Identifiers Python entities](https://reader030.fdocuments.us/reader030/viewer/2022040120/5e7c78b2e88d3a2e807b5d02/html5/thumbnails/58.jpg)
Support Vector Machines (SVM)
I Maximize the margin which is equal to 2‖w‖
I under the constraint that the samples are classified correctly:
a. w>Xn ≥ 1 if Yn = 1
b. w>Xn ≤ −1 if Yn = −1
∀n = 1, . . . , N
![Page 59: Artificial Intelligence - An Introduction to Machine Learningstephane.ayache.perso.luminy.univ-amu.fr/zoom/cours/... · 2019-03-12 · Python: Key Concepts Identifiers Python entities](https://reader030.fdocuments.us/reader030/viewer/2022040120/5e7c78b2e88d3a2e807b5d02/html5/thumbnails/59.jpg)
Kernel SVM for linearly inseparable data
I Project the data in a high-dimensional feature space using akernel function
I Apply a linear classifier in the feature space
![Page 60: Artificial Intelligence - An Introduction to Machine Learningstephane.ayache.perso.luminy.univ-amu.fr/zoom/cours/... · 2019-03-12 · Python: Key Concepts Identifiers Python entities](https://reader030.fdocuments.us/reader030/viewer/2022040120/5e7c78b2e88d3a2e807b5d02/html5/thumbnails/60.jpg)
Multilayer Perceptron and Convolutional Neural Networks
biais : activation = 1
i kj = yx1
x2x =
y1
y2
(de By Aphex34 - Own work, CC BY-SA 4.0,https://commons.wikimedia.org/w/index.php?curid=45679374)
![Page 61: Artificial Intelligence - An Introduction to Machine Learningstephane.ayache.perso.luminy.univ-amu.fr/zoom/cours/... · 2019-03-12 · Python: Key Concepts Identifiers Python entities](https://reader030.fdocuments.us/reader030/viewer/2022040120/5e7c78b2e88d3a2e807b5d02/html5/thumbnails/61.jpg)
Linear Regression
I model the relation between features (explanatory variable x)and a continuous valued response (target variable y)
I Linear model:
y = w0 + w1x (1) + . . . + wdx (d) =d∑
i=1wix (i) = 〈w , x〉
![Page 62: Artificial Intelligence - An Introduction to Machine Learningstephane.ayache.perso.luminy.univ-amu.fr/zoom/cours/... · 2019-03-12 · Python: Key Concepts Identifiers Python entities](https://reader030.fdocuments.us/reader030/viewer/2022040120/5e7c78b2e88d3a2e807b5d02/html5/thumbnails/62.jpg)
. . . How these work in practice
See the notebookML_5GSC2019.ipynb
![Page 63: Artificial Intelligence - An Introduction to Machine Learningstephane.ayache.perso.luminy.univ-amu.fr/zoom/cours/... · 2019-03-12 · Python: Key Concepts Identifiers Python entities](https://reader030.fdocuments.us/reader030/viewer/2022040120/5e7c78b2e88d3a2e807b5d02/html5/thumbnails/63.jpg)
Act III: Programming with Python forMachine Learning
![Page 64: Artificial Intelligence - An Introduction to Machine Learningstephane.ayache.perso.luminy.univ-amu.fr/zoom/cours/... · 2019-03-12 · Python: Key Concepts Identifiers Python entities](https://reader030.fdocuments.us/reader030/viewer/2022040120/5e7c78b2e88d3a2e807b5d02/html5/thumbnails/64.jpg)
Packages for Scientific Computing and Machine Learning
Throughout this course, we will mainly use
I NumPy: multidimensional array package
I SciPy: scientific computing package
I Matplotlib: plotting library for visualization
I pandas: data analysis library
I scikit-learn: machine learning library
Python-based Ecosystem for Machine Learning
![Page 65: Artificial Intelligence - An Introduction to Machine Learningstephane.ayache.perso.luminy.univ-amu.fr/zoom/cours/... · 2019-03-12 · Python: Key Concepts Identifiers Python entities](https://reader030.fdocuments.us/reader030/viewer/2022040120/5e7c78b2e88d3a2e807b5d02/html5/thumbnails/65.jpg)
Installing Python and ML packages
Anaconda Python distributionis a free –including for commercial use– enterprise-ready Pythondistribution that bundles all the essential Python packages formachine learning, data science, math, and engineering in oneuser-friendly cross-platform distribution.
Anaconda installer can be downloaded athttps://www.anaconda.com/download/
Anaconda quick-start guide available athttps://conda.io/docs/user-guide/getting-started.html
![Page 66: Artificial Intelligence - An Introduction to Machine Learningstephane.ayache.perso.luminy.univ-amu.fr/zoom/cours/... · 2019-03-12 · Python: Key Concepts Identifiers Python entities](https://reader030.fdocuments.us/reader030/viewer/2022040120/5e7c78b2e88d3a2e807b5d02/html5/thumbnails/66.jpg)
Jupyter Notebooks
I formerly known as IPython notebooksI an interactive computational environmentI can contain code, text, images, output, . . .I installed by default with the Anaconda distribution
We can invoke the Jupyter notebook by executing the followingcommand at the command prompt or terminal:$ jupyter notebook
![Page 67: Artificial Intelligence - An Introduction to Machine Learningstephane.ayache.perso.luminy.univ-amu.fr/zoom/cours/... · 2019-03-12 · Python: Key Concepts Identifiers Python entities](https://reader030.fdocuments.us/reader030/viewer/2022040120/5e7c78b2e88d3a2e807b5d02/html5/thumbnails/67.jpg)
Python and scikit-learn: checking your installation
Open a Python console or a Jupyter notebook and execute thefollowing:
>>> import sys>>> print (sys.version)3.6.1 |Anaconda 4.4.0 (x86_64)| (default, May 11 2017, 13:04:09)[GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.57)]
I to verify the Python version
>>> import sklearn>>> sklearn.__version__’0.18.1’
I to check that scikit-learn has been installed correctly
![Page 68: Artificial Intelligence - An Introduction to Machine Learningstephane.ayache.perso.luminy.univ-amu.fr/zoom/cours/... · 2019-03-12 · Python: Key Concepts Identifiers Python entities](https://reader030.fdocuments.us/reader030/viewer/2022040120/5e7c78b2e88d3a2e807b5d02/html5/thumbnails/68.jpg)
Colaboratory
Google internal tool for data science and machine learningworkflow available at https://colab.research.google.com/
![Page 69: Artificial Intelligence - An Introduction to Machine Learningstephane.ayache.perso.luminy.univ-amu.fr/zoom/cours/... · 2019-03-12 · Python: Key Concepts Identifiers Python entities](https://reader030.fdocuments.us/reader030/viewer/2022040120/5e7c78b2e88d3a2e807b5d02/html5/thumbnails/69.jpg)
Python: Key Concepts
Identifiers Python entities such as class, functions, and variablesI can be a combination of upper-or lower-case letters (a to z or A to Z)I can be any digits (0 to 9) or an underscore ( )I cannot start with a digit. For example, 1variable is not valid,
whereas variable1 is validI special symbols like , , #, $, % etc cannot be part of the identifiers
Keywords set of reserved words used in Python to define the syntax andstructure of the language
![Page 70: Artificial Intelligence - An Introduction to Machine Learningstephane.ayache.perso.luminy.univ-amu.fr/zoom/cours/... · 2019-03-12 · Python: Key Concepts Identifiers Python entities](https://reader030.fdocuments.us/reader030/viewer/2022040120/5e7c78b2e88d3a2e807b5d02/html5/thumbnails/70.jpg)
Python: Key Concepts
A First Python Program>>> print("Welcome to the 5G&SC 2019 days!")Welcome to the 5G&SC 2019 days!
IndentationEach line of code must be indented by the same amount to denote ablock of code in Python
# Correct indentationx = 1if x == 1:print (’x has a value of 1’)else:print (’x does NOT have a value of 1’)
# incorrect indentation, program will generate a syntax error# due to the wrong indentation in the else statementx = 1if x == 1:print (’x has a value of 1’)else:print (’x does NOT have a value of 1’)
![Page 71: Artificial Intelligence - An Introduction to Machine Learningstephane.ayache.perso.luminy.univ-amu.fr/zoom/cours/... · 2019-03-12 · Python: Key Concepts Identifiers Python entities](https://reader030.fdocuments.us/reader030/viewer/2022040120/5e7c78b2e88d3a2e807b5d02/html5/thumbnails/71.jpg)
Python: Key ConceptsBasic Object Types
Types Examples Types ExamplesNone None String ’this is a string’, ”also me”Boolean True, False Tuple (1, True, ’ML’)Integer -1, 0, sys.maxint List [1, True, ‘ML’]Long 1L, 9787L Set set(1, True, ‘ML’)Float 3.141592654 Dictionary {’1’:’A’, ’2’:’AA’, True:1}Complex 2+8j File f = open(‘filename’, ‘rb’)
Float = 3.14List = [1, True, ’ML’] # Values can be changed
# lets print the object type and the valueprint(type(Float), Float)print(type(List), List)
----- output ------<type ’float’> 3.14<type ’list’> [1, True, ’ML’]
![Page 72: Artificial Intelligence - An Introduction to Machine Learningstephane.ayache.perso.luminy.univ-amu.fr/zoom/cours/... · 2019-03-12 · Python: Key Concepts Identifiers Python entities](https://reader030.fdocuments.us/reader030/viewer/2022040120/5e7c78b2e88d3a2e807b5d02/html5/thumbnails/72.jpg)
Python: Key Concepts
When to Use List vs. Tuples vs. Set vs. Dictionary
I List: Use when you need an ordered sequence of homogeneouscollections, whose values can be changed later in the program
I Tuple: Use when you need an ordered sequence ofheterogeneous collections whose values need not be changedlater in the program
I Set: It is ideal for use when you don’t have to store duplicatesand you are not concerned about the order or the items. Youjust want to know whether a particular value already exists ornot
I Dictionary: It is ideal for use when you need to relate valueswith keys, in order to look them up efficiently using a key
![Page 73: Artificial Intelligence - An Introduction to Machine Learningstephane.ayache.perso.luminy.univ-amu.fr/zoom/cours/... · 2019-03-12 · Python: Key Concepts Identifiers Python entities](https://reader030.fdocuments.us/reader030/viewer/2022040120/5e7c78b2e88d3a2e807b5d02/html5/thumbnails/73.jpg)
Python: Key Concepts
Basic Operators
I Arithmetic OperatorsI Comparison or Relational OperatorsI Assignment OperatorsI Bitwise OperatorsI Logical OperatorsI Membership OperatorsI Identity Operators
See the associated notebook for a collection of examplesillustrating these operators
![Page 74: Artificial Intelligence - An Introduction to Machine Learningstephane.ayache.perso.luminy.univ-amu.fr/zoom/cours/... · 2019-03-12 · Python: Key Concepts Identifiers Python entities](https://reader030.fdocuments.us/reader030/viewer/2022040120/5e7c78b2e88d3a2e807b5d02/html5/thumbnails/74.jpg)
Python: Key ConceptsIterationA loop control statement enables us to execute a single or a set ofprogramming statements multiple times until a given condition is satisfied# First Exampleprint "First Example"for item in [1,2,3,4,5]:
print(’item :’, item)
# Second Exampleprint "Second Example"letters = [’A’, ’B’, ’C’]for letter in letters:
print(’ First loop letter :’, letter)
# Third Exampleprint "Third Example"for index in range(len(letters)):
print(’First loop letter :’, letters[index])
# Example code for while loop statementcount = 0while (count <3):
print(’The count is:’, count)count = count + 1
----- output ------First Exampleitem : 1item : 2item : 3item : 4item : 5Second ExampleFirst loop letter : AFirst loop letter : BFirst loop letter : CThird ExampleFirst loop letter : AFirst loop letter : BFirst loop letter : C
![Page 75: Artificial Intelligence - An Introduction to Machine Learningstephane.ayache.perso.luminy.univ-amu.fr/zoom/cours/... · 2019-03-12 · Python: Key Concepts Identifiers Python entities](https://reader030.fdocuments.us/reader030/viewer/2022040120/5e7c78b2e88d3a2e807b5d02/html5/thumbnails/75.jpg)
Python: Key Concepts
ListsPython list is a collection of items which is ordered and changeable. Notethat the items in the list need not be of the same data type
Description Python Expression Example ResultsCreating a list [item1, item2,. . . ] list = [’a’,’b’,’c’,’d’] [’a’,’b’,’c’,’d’]Accessing items list[index] list[2] cLength len(list) len([1, 2, 3]) 3Concatenation list 1 + list 2 [1, 2, 3] + [4, 5] [1, 2, 3, 4, 5]Iteration for x in list:print x for x in [1, 2, 3]:print x 1 2 3Slicing list[index:] list = [1,2,3]; list[1:] [2,3]Append object list.append(obj) [1,2,3].append(5) [1,2,3,5]Remove object list.remove(obj) [’a’,’c’,1].remove(’c’) [’a’, 1]Count occurrence list.count(obj) [1,1,2,3].count(1) 2
![Page 76: Artificial Intelligence - An Introduction to Machine Learningstephane.ayache.perso.luminy.univ-amu.fr/zoom/cours/... · 2019-03-12 · Python: Key Concepts Identifiers Python entities](https://reader030.fdocuments.us/reader030/viewer/2022040120/5e7c78b2e88d3a2e807b5d02/html5/thumbnails/76.jpg)
Python: Key Concepts
Defining a FunctionI The keyword def denotes the beginning of a function block,
which will be followed by the name of the function and open,close parentheses. After this a colon (:) to be put to indicatethe end of the function header
I Functions can accept arguments or parameters. Any suchinput arguments or parameters should be placed within theparentheses in the header of the parameter
I The main code statements are to be put below the functionheader and should be indented, which indicates that the codeis part of the same function
I Functions can return an expression to the caller using return
![Page 77: Artificial Intelligence - An Introduction to Machine Learningstephane.ayache.perso.luminy.univ-amu.fr/zoom/cours/... · 2019-03-12 · Python: Key Concepts Identifiers Python entities](https://reader030.fdocuments.us/reader030/viewer/2022040120/5e7c78b2e88d3a2e807b5d02/html5/thumbnails/77.jpg)
Python: Key Concepts
Syntax for Creating Functions
without argument
# Simple functiondef someFunction():
print("IA&ML @ 5GSC 2019")
# Call the functionsomeFunction()
----- output -----IA&ML @ 5GSC 2019
with argument
# Simple function to add two numbersdef sum_two_numbers(x, y):
return x + y
# after this line x will hold the value 3print(sum_two_numbers(1,2))
----- output -----3
![Page 78: Artificial Intelligence - An Introduction to Machine Learningstephane.ayache.perso.luminy.univ-amu.fr/zoom/cours/... · 2019-03-12 · Python: Key Concepts Identifiers Python entities](https://reader030.fdocuments.us/reader030/viewer/2022040120/5e7c78b2e88d3a2e807b5d02/html5/thumbnails/78.jpg)
I Numpy basics
(NumPy documentation: http://docs.scipy.org/doc/)
I Pandas basics
(pandas documentation: http://pandas.pydata.org/
I Machine learning method using scikit-learn
(scikit-learn documentation:http://scikit-learn.org/stable/documentation.html)
![Page 79: Artificial Intelligence - An Introduction to Machine Learningstephane.ayache.perso.luminy.univ-amu.fr/zoom/cours/... · 2019-03-12 · Python: Key Concepts Identifiers Python entities](https://reader030.fdocuments.us/reader030/viewer/2022040120/5e7c78b2e88d3a2e807b5d02/html5/thumbnails/79.jpg)
to be continued...
Go to Google Colaboratory or Jupyter notebook andrun the notebookML_5GSC2019.ipynb