[Lecture 2] AI and Deep Learning: Logistic Regression (Theory)
-
Upload
kobkrit-viriyayudhakorn -
Category
Education
-
view
109 -
download
3
Transcript of [Lecture 2] AI and Deep Learning: Logistic Regression (Theory)
![Page 1: [Lecture 2] AI and Deep Learning: Logistic Regression (Theory)](https://reader031.fdocuments.us/reader031/viewer/2022021816/5aacee8e7f8b9a003b8b456b/html5/thumbnails/1.jpg)
Introduction toArtificial Intelligence & Deep Learning
Lecture 2: Logistic Regression
Dr. Kobkrit Viriyayudhakorn
![Page 2: [Lecture 2] AI and Deep Learning: Logistic Regression (Theory)](https://reader031.fdocuments.us/reader031/viewer/2022021816/5aacee8e7f8b9a003b8b456b/html5/thumbnails/2.jpg)
Following slides are based on
• Andrew Ng’s Coursera Deep Learning course.
• Stanford CS231n: Convolutional Neural Networks for Visual Recognition course.
• Goodwill’s Deep Learning Books
• Prof. Thanaruk’s Introduction to Concepts and Techniques in Data Mining and Application to Text Mining Book.
• Andreas’s Introduction to Machine Learning with Python Book.
• Giancarlo Zaccone’s Getting Start with Tensorflow Book.
• Justine Johnson’s Python Numpy Tutorial
• ETC…
![Page 3: [Lecture 2] AI and Deep Learning: Logistic Regression (Theory)](https://reader031.fdocuments.us/reader031/viewer/2022021816/5aacee8e7f8b9a003b8b456b/html5/thumbnails/3.jpg)
What is Neural Network (Regression)?P
rice
Size of House
![Page 4: [Lecture 2] AI and Deep Learning: Logistic Regression (Theory)](https://reader031.fdocuments.us/reader031/viewer/2022021816/5aacee8e7f8b9a003b8b456b/html5/thumbnails/4.jpg)
What is Neural Network (Classification)?R
ed-d
ish
Round-dish
![Page 5: [Lecture 2] AI and Deep Learning: Logistic Regression (Theory)](https://reader031.fdocuments.us/reader031/viewer/2022021816/5aacee8e7f8b9a003b8b456b/html5/thumbnails/5.jpg)
House Price Prediction with 4-3-1 NN
Size (x1)
#Bed Room (x2)
Wealth (x4)
Zip Code (x3)Price
![Page 6: [Lecture 2] AI and Deep Learning: Logistic Regression (Theory)](https://reader031.fdocuments.us/reader031/viewer/2022021816/5aacee8e7f8b9a003b8b456b/html5/thumbnails/6.jpg)
Supervised Learning in Neural Network
Input (X) Output (Y) Application
House Features Price Real Estate Agent
Patient conditions Disease Physician Assistant
Image 10,000 Objects Photo Recognition
CCTV Camera Footage Person Name / Car License Number
Security / Robot
Audio Text Transcript Speech Recognition, Subtitle Generation
Text of Thai Language Text of English Language Machine Translation
Radar Signal, Images Position of Obstacle Autonomous Driving
![Page 7: [Lecture 2] AI and Deep Learning: Logistic Regression (Theory)](https://reader031.fdocuments.us/reader031/viewer/2022021816/5aacee8e7f8b9a003b8b456b/html5/thumbnails/7.jpg)
Neural Network Example
Feed Forward Neural Network Convolutional Neural Network (CNN)
Recurrent Neural Network (RNN)
![Page 8: [Lecture 2] AI and Deep Learning: Logistic Regression (Theory)](https://reader031.fdocuments.us/reader031/viewer/2022021816/5aacee8e7f8b9a003b8b456b/html5/thumbnails/8.jpg)
Supervised Learning
• Structure Data • Unstructured Data
Sepal Length
SepalWidth
… Species
5.1 3.5 I. setosa
6.3 3.3 I. virginica
Iris Flower Dataset
![Page 9: [Lecture 2] AI and Deep Learning: Logistic Regression (Theory)](https://reader031.fdocuments.us/reader031/viewer/2022021816/5aacee8e7f8b9a003b8b456b/html5/thumbnails/9.jpg)
Why Deep Learning is so popular?
Labelled data (m)
Perf
orm
ance
![Page 10: [Lecture 2] AI and Deep Learning: Logistic Regression (Theory)](https://reader031.fdocuments.us/reader031/viewer/2022021816/5aacee8e7f8b9a003b8b456b/html5/thumbnails/10.jpg)
• Data
• Computation
• Algorithm
Why Deep Learning is taking off?
![Page 11: [Lecture 2] AI and Deep Learning: Logistic Regression (Theory)](https://reader031.fdocuments.us/reader031/viewer/2022021816/5aacee8e7f8b9a003b8b456b/html5/thumbnails/11.jpg)
Logistic Regression
![Page 12: [Lecture 2] AI and Deep Learning: Logistic Regression (Theory)](https://reader031.fdocuments.us/reader031/viewer/2022021816/5aacee8e7f8b9a003b8b456b/html5/thumbnails/12.jpg)
Binary Classification
1 (Flower) or 0 (Non-Flower)
![Page 13: [Lecture 2] AI and Deep Learning: Logistic Regression (Theory)](https://reader031.fdocuments.us/reader031/viewer/2022021816/5aacee8e7f8b9a003b8b456b/html5/thumbnails/13.jpg)
65 10 14 78
43 21 4 25
5 20 130 60
90 120 90 35
78 28 120 30
35 25 1 18
33 12 24 250
65 0 120 45
85 123 4 50
78 88 123 33
Binary Classification
1 (Flower) or 0 (Non-Flower)
255 245 91 128
123 1 244 255
55 80 120 45
85 123 44 39
88 88 123 33
Red
Blue
Green
64
64
![Page 14: [Lecture 2] AI and Deep Learning: Logistic Regression (Theory)](https://reader031.fdocuments.us/reader031/viewer/2022021816/5aacee8e7f8b9a003b8b456b/html5/thumbnails/14.jpg)
Notations
![Page 15: [Lecture 2] AI and Deep Learning: Logistic Regression (Theory)](https://reader031.fdocuments.us/reader031/viewer/2022021816/5aacee8e7f8b9a003b8b456b/html5/thumbnails/15.jpg)
Logistic Regression
![Page 16: [Lecture 2] AI and Deep Learning: Logistic Regression (Theory)](https://reader031.fdocuments.us/reader031/viewer/2022021816/5aacee8e7f8b9a003b8b456b/html5/thumbnails/16.jpg)
Logistic Regression Cost Function
Loss (error) function:
𝑦 = 𝜎 𝑤𝑇𝑥 + 𝑏 𝜎 𝑧 =1
1 + ⅇ−𝑧
𝑥 ⅈ , 𝑦 ⅈ … , 𝑥1𝑚
𝑦 𝑚 𝑦 ⅈ ≈ 𝑦 ⅈ
, where
Given want
![Page 17: [Lecture 2] AI and Deep Learning: Logistic Regression (Theory)](https://reader031.fdocuments.us/reader031/viewer/2022021816/5aacee8e7f8b9a003b8b456b/html5/thumbnails/17.jpg)
![Page 18: [Lecture 2] AI and Deep Learning: Logistic Regression (Theory)](https://reader031.fdocuments.us/reader031/viewer/2022021816/5aacee8e7f8b9a003b8b456b/html5/thumbnails/18.jpg)
Gradient Descent
Want to find W, b that minimize 𝐽 W, 𝑏
𝑦 = 𝜎 𝑊𝑇𝑥 + 𝑏 𝜎 𝑧 =1
1 + ⅇ−𝑧, where
𝐽 W, 𝑏 =1
𝑚
𝑖=1
𝑚
ℒ 𝑦 𝑖 , 𝑦 𝑖 =−1
𝑚
𝑖=1
𝑚
𝑦 𝑖 log 𝑦+𝑖
1 − 𝑦 𝑖 log 1 − 𝑦 𝑖
𝐽 W, 𝑏
𝑊
𝑏
![Page 19: [Lecture 2] AI and Deep Learning: Logistic Regression (Theory)](https://reader031.fdocuments.us/reader031/viewer/2022021816/5aacee8e7f8b9a003b8b456b/html5/thumbnails/19.jpg)
Gradient Descent (1D)
w
![Page 20: [Lecture 2] AI and Deep Learning: Logistic Regression (Theory)](https://reader031.fdocuments.us/reader031/viewer/2022021816/5aacee8e7f8b9a003b8b456b/html5/thumbnails/20.jpg)
![Page 21: [Lecture 2] AI and Deep Learning: Logistic Regression (Theory)](https://reader031.fdocuments.us/reader031/viewer/2022021816/5aacee8e7f8b9a003b8b456b/html5/thumbnails/21.jpg)
Computation Graph
![Page 22: [Lecture 2] AI and Deep Learning: Logistic Regression (Theory)](https://reader031.fdocuments.us/reader031/viewer/2022021816/5aacee8e7f8b9a003b8b456b/html5/thumbnails/22.jpg)
Computing derivatives
b = 3
a = 5
c = 2u = bc
v = a + u J = 3v
11
6
33
![Page 23: [Lecture 2] AI and Deep Learning: Logistic Regression (Theory)](https://reader031.fdocuments.us/reader031/viewer/2022021816/5aacee8e7f8b9a003b8b456b/html5/thumbnails/23.jpg)
Computing derivatives
b = 3
a = 5
c = 2u = bc
v = a + u J = 3v
11
6
33
![Page 24: [Lecture 2] AI and Deep Learning: Logistic Regression (Theory)](https://reader031.fdocuments.us/reader031/viewer/2022021816/5aacee8e7f8b9a003b8b456b/html5/thumbnails/24.jpg)
Logistic Regression Gradient Descent
𝑧 = 𝑤𝑇𝑥 + 𝑏
𝑦 = 𝑎 = 𝜎 𝑧
ℒ 𝑎, 𝑦 = − 𝑦 log 𝑎 + 1 − 𝑦 log 𝑎
![Page 25: [Lecture 2] AI and Deep Learning: Logistic Regression (Theory)](https://reader031.fdocuments.us/reader031/viewer/2022021816/5aacee8e7f8b9a003b8b456b/html5/thumbnails/25.jpg)
![Page 26: [Lecture 2] AI and Deep Learning: Logistic Regression (Theory)](https://reader031.fdocuments.us/reader031/viewer/2022021816/5aacee8e7f8b9a003b8b456b/html5/thumbnails/26.jpg)
Logistic Regression Gradient Descent
𝑧 = 𝑤1𝑥1 + 𝑤2𝑥2 + 𝑏
𝑤1
𝑥1
𝑤2
𝑥2
𝑏
𝑎 = 𝜎 𝑧 ℒ 𝑎, 𝑦
![Page 27: [Lecture 2] AI and Deep Learning: Logistic Regression (Theory)](https://reader031.fdocuments.us/reader031/viewer/2022021816/5aacee8e7f8b9a003b8b456b/html5/thumbnails/27.jpg)
Gradient Descent in 𝑚 examples
![Page 28: [Lecture 2] AI and Deep Learning: Logistic Regression (Theory)](https://reader031.fdocuments.us/reader031/viewer/2022021816/5aacee8e7f8b9a003b8b456b/html5/thumbnails/28.jpg)
What is Vectorization?
![Page 29: [Lecture 2] AI and Deep Learning: Logistic Regression (Theory)](https://reader031.fdocuments.us/reader031/viewer/2022021816/5aacee8e7f8b9a003b8b456b/html5/thumbnails/29.jpg)
Algorithm for Logistic Regression