Introduction toArtificial Intelligence & Deep Learning
Lecture 2: Logistic Regression
Dr. Kobkrit Viriyayudhakorn
Following slides are based on
• Andrew Ng’s Coursera Deep Learning course.
• Stanford CS231n: Convolutional Neural Networks for Visual Recognition course.
• Goodwill’s Deep Learning Books
• Prof. Thanaruk’s Introduction to Concepts and Techniques in Data Mining and Application to Text Mining Book.
• Andreas’s Introduction to Machine Learning with Python Book.
• Giancarlo Zaccone’s Getting Start with Tensorflow Book.
• Justine Johnson’s Python Numpy Tutorial
• ETC…
What is Neural Network (Regression)?P
rice
Size of House
What is Neural Network (Classification)?R
ed-d
ish
Round-dish
House Price Prediction with 4-3-1 NN
Size (x1)
#Bed Room (x2)
Wealth (x4)
Zip Code (x3)Price
Supervised Learning in Neural Network
Input (X) Output (Y) Application
House Features Price Real Estate Agent
Patient conditions Disease Physician Assistant
Image 10,000 Objects Photo Recognition
CCTV Camera Footage Person Name / Car License Number
Security / Robot
Audio Text Transcript Speech Recognition, Subtitle Generation
Text of Thai Language Text of English Language Machine Translation
Radar Signal, Images Position of Obstacle Autonomous Driving
Neural Network Example
Feed Forward Neural Network Convolutional Neural Network (CNN)
Recurrent Neural Network (RNN)
Supervised Learning
• Structure Data • Unstructured Data
Sepal Length
SepalWidth
… Species
5.1 3.5 I. setosa
6.3 3.3 I. virginica
Iris Flower Dataset
Why Deep Learning is so popular?
Labelled data (m)
Perf
orm
ance
• Data
• Computation
• Algorithm
Why Deep Learning is taking off?
Logistic Regression
Binary Classification
1 (Flower) or 0 (Non-Flower)
65 10 14 78
43 21 4 25
5 20 130 60
90 120 90 35
78 28 120 30
35 25 1 18
33 12 24 250
65 0 120 45
85 123 4 50
78 88 123 33
Binary Classification
1 (Flower) or 0 (Non-Flower)
255 245 91 128
123 1 244 255
55 80 120 45
85 123 44 39
88 88 123 33
Red
Blue
Green
64
64
Notations
Logistic Regression
Logistic Regression Cost Function
Loss (error) function:
𝑦 = 𝜎 𝑤𝑇𝑥 + 𝑏 𝜎 𝑧 =1
1 + ⅇ−𝑧
𝑥 ⅈ , 𝑦 ⅈ … , 𝑥1𝑚
𝑦 𝑚 𝑦 ⅈ ≈ 𝑦 ⅈ
, where
Given want
Gradient Descent
Want to find W, b that minimize 𝐽 W, 𝑏
𝑦 = 𝜎 𝑊𝑇𝑥 + 𝑏 𝜎 𝑧 =1
1 + ⅇ−𝑧, where
𝐽 W, 𝑏 =1
𝑚
𝑖=1
𝑚
ℒ 𝑦 𝑖 , 𝑦 𝑖 =−1
𝑚
𝑖=1
𝑚
𝑦 𝑖 log 𝑦+𝑖
1 − 𝑦 𝑖 log 1 − 𝑦 𝑖
𝐽 W, 𝑏
𝑊
𝑏
Gradient Descent (1D)
w
Computation Graph
Computing derivatives
b = 3
a = 5
c = 2u = bc
v = a + u J = 3v
11
6
33
Computing derivatives
b = 3
a = 5
c = 2u = bc
v = a + u J = 3v
11
6
33
Logistic Regression Gradient Descent
𝑧 = 𝑤𝑇𝑥 + 𝑏
𝑦 = 𝑎 = 𝜎 𝑧
ℒ 𝑎, 𝑦 = − 𝑦 log 𝑎 + 1 − 𝑦 log 𝑎
Logistic Regression Gradient Descent
𝑧 = 𝑤1𝑥1 + 𝑤2𝑥2 + 𝑏
𝑤1
𝑥1
𝑤2
𝑥2
𝑏
𝑎 = 𝜎 𝑧 ℒ 𝑎, 𝑦
Gradient Descent in 𝑚 examples
What is Vectorization?
Algorithm for Logistic Regression
Top Related