AdaBoost - cmp.felk.cvut.czcmp.felk.cvut.cz/cmp/courses/recognition/AdaBoost2/adaboost_talk.… ·...
Transcript of AdaBoost - cmp.felk.cvut.czcmp.felk.cvut.cz/cmp/courses/recognition/AdaBoost2/adaboost_talk.… ·...
![Page 1: AdaBoost - cmp.felk.cvut.czcmp.felk.cvut.cz/cmp/courses/recognition/AdaBoost2/adaboost_talk.… · AdaBoost variants. 3/12 What is AdaBoost? AdaBoost is an algorithm for constructing](https://reader034.fdocuments.us/reader034/viewer/2022052019/6032ce98264d8d49d92220bd/html5/thumbnails/1.jpg)
AdaBoost
Lecturer:Jan Sochman
Authors:Jan Sochman, Jirı Matas
Centre for Machine PerceptionCzech Technical University, Prague
http://cmp.felk.cvut.cz
![Page 2: AdaBoost - cmp.felk.cvut.czcmp.felk.cvut.cz/cmp/courses/recognition/AdaBoost2/adaboost_talk.… · AdaBoost variants. 3/12 What is AdaBoost? AdaBoost is an algorithm for constructing](https://reader034.fdocuments.us/reader034/viewer/2022052019/6032ce98264d8d49d92220bd/html5/thumbnails/2.jpg)
2/12
Presentation
Outline:
� AdaBoost algorithm
• Why is of interest?
• How it works?
• Why it works?
� AdaBoost variants
![Page 3: AdaBoost - cmp.felk.cvut.czcmp.felk.cvut.cz/cmp/courses/recognition/AdaBoost2/adaboost_talk.… · AdaBoost variants. 3/12 What is AdaBoost? AdaBoost is an algorithm for constructing](https://reader034.fdocuments.us/reader034/viewer/2022052019/6032ce98264d8d49d92220bd/html5/thumbnails/3.jpg)
3/12
What is AdaBoost?
AdaBoost is an algorithm for constructing a “strong” classifier as linear combination
f(x) =T∑
t=1
αtht(x)
of “simple” “weak” classifiers ht(x).
Terminology
� ht(x) ... “weak” or basis classifier, hypothesis, “feature”
� H(x) = sign(f(x)) ... ‘’strong” or final classifier/hypothesis
Interesting properties
� AB is a linear classifier with all its desirable properties.
� AB output converges to the logarithm of likelihood ratio.
� AB has good generalisation properties.
� AB is a feature selector with a principled strategy (minimisation of upper bound onempirical error).
� AB close to sequential decision making (it produces a sequence of gradually morecomplex classifiers).
![Page 4: AdaBoost - cmp.felk.cvut.czcmp.felk.cvut.cz/cmp/courses/recognition/AdaBoost2/adaboost_talk.… · AdaBoost variants. 3/12 What is AdaBoost? AdaBoost is an algorithm for constructing](https://reader034.fdocuments.us/reader034/viewer/2022052019/6032ce98264d8d49d92220bd/html5/thumbnails/4.jpg)
4/12
The AdaBoost Algorithm
Given: (x1, y1), . . . , (xm, ym);xi ∈ X , yi ∈ {−1,+1}
![Page 5: AdaBoost - cmp.felk.cvut.czcmp.felk.cvut.cz/cmp/courses/recognition/AdaBoost2/adaboost_talk.… · AdaBoost variants. 3/12 What is AdaBoost? AdaBoost is an algorithm for constructing](https://reader034.fdocuments.us/reader034/viewer/2022052019/6032ce98264d8d49d92220bd/html5/thumbnails/5.jpg)
4/12
The AdaBoost Algorithm
Given: (x1, y1), . . . , (xm, ym);xi ∈ X , yi ∈ {−1,+1}Initialise weights D1(i) = 1/m.
![Page 6: AdaBoost - cmp.felk.cvut.czcmp.felk.cvut.cz/cmp/courses/recognition/AdaBoost2/adaboost_talk.… · AdaBoost variants. 3/12 What is AdaBoost? AdaBoost is an algorithm for constructing](https://reader034.fdocuments.us/reader034/viewer/2022052019/6032ce98264d8d49d92220bd/html5/thumbnails/6.jpg)
4/12
The AdaBoost Algorithm
Given: (x1, y1), . . . , (xm, ym);xi ∈ X , yi ∈ {−1,+1}Initialise weights D1(i) = 1/m.For t = 1, ..., T :
![Page 7: AdaBoost - cmp.felk.cvut.czcmp.felk.cvut.cz/cmp/courses/recognition/AdaBoost2/adaboost_talk.… · AdaBoost variants. 3/12 What is AdaBoost? AdaBoost is an algorithm for constructing](https://reader034.fdocuments.us/reader034/viewer/2022052019/6032ce98264d8d49d92220bd/html5/thumbnails/7.jpg)
4/12
The AdaBoost Algorithm
Given: (x1, y1), . . . , (xm, ym);xi ∈ X , yi ∈ {−1,+1}Initialise weights D1(i) = 1/m.For t = 1, ..., T :
� Find ht = arg minhj∈H
εj =m∑
i=1
Dt(i)Jyi 6= hj(xi)K
t = 1
![Page 8: AdaBoost - cmp.felk.cvut.czcmp.felk.cvut.cz/cmp/courses/recognition/AdaBoost2/adaboost_talk.… · AdaBoost variants. 3/12 What is AdaBoost? AdaBoost is an algorithm for constructing](https://reader034.fdocuments.us/reader034/viewer/2022052019/6032ce98264d8d49d92220bd/html5/thumbnails/8.jpg)
4/12
The AdaBoost Algorithm
Given: (x1, y1), . . . , (xm, ym);xi ∈ X , yi ∈ {−1,+1}Initialise weights D1(i) = 1/m.For t = 1, ..., T :
� Find ht = arg minhj∈H
εj =m∑
i=1
Dt(i)Jyi 6= hj(xi)K
� If εt ≥ 1/2 then stop
t = 1
![Page 9: AdaBoost - cmp.felk.cvut.czcmp.felk.cvut.cz/cmp/courses/recognition/AdaBoost2/adaboost_talk.… · AdaBoost variants. 3/12 What is AdaBoost? AdaBoost is an algorithm for constructing](https://reader034.fdocuments.us/reader034/viewer/2022052019/6032ce98264d8d49d92220bd/html5/thumbnails/9.jpg)
4/12
The AdaBoost Algorithm
Given: (x1, y1), . . . , (xm, ym);xi ∈ X , yi ∈ {−1,+1}Initialise weights D1(i) = 1/m.For t = 1, ..., T :
� Find ht = arg minhj∈H
εj =m∑
i=1
Dt(i)Jyi 6= hj(xi)K
� If εt ≥ 1/2 then stop
� Set αt = 12 log(1−εt
εt)
t = 1
![Page 10: AdaBoost - cmp.felk.cvut.czcmp.felk.cvut.cz/cmp/courses/recognition/AdaBoost2/adaboost_talk.… · AdaBoost variants. 3/12 What is AdaBoost? AdaBoost is an algorithm for constructing](https://reader034.fdocuments.us/reader034/viewer/2022052019/6032ce98264d8d49d92220bd/html5/thumbnails/10.jpg)
4/12
The AdaBoost Algorithm
Given: (x1, y1), . . . , (xm, ym);xi ∈ X , yi ∈ {−1,+1}Initialise weights D1(i) = 1/m.For t = 1, ..., T :
� Find ht = arg minhj∈H
εj =m∑
i=1
Dt(i)Jyi 6= hj(xi)K
� If εt ≥ 1/2 then stop
� Set αt = 12 log(1−εt
εt)
� Update
Dt+1(i) =Dt(i)exp(−αtyiht(xi))
Zt
t = 1
![Page 11: AdaBoost - cmp.felk.cvut.czcmp.felk.cvut.cz/cmp/courses/recognition/AdaBoost2/adaboost_talk.… · AdaBoost variants. 3/12 What is AdaBoost? AdaBoost is an algorithm for constructing](https://reader034.fdocuments.us/reader034/viewer/2022052019/6032ce98264d8d49d92220bd/html5/thumbnails/11.jpg)
4/12
The AdaBoost Algorithm
Given: (x1, y1), . . . , (xm, ym);xi ∈ X , yi ∈ {−1,+1}Initialise weights D1(i) = 1/m.For t = 1, ..., T :
� Find ht = arg minhj∈H
εj =m∑
i=1
Dt(i)Jyi 6= hj(xi)K
� If εt ≥ 1/2 then stop
� Set αt = 12 log(1−εt
εt)
� Update
Dt+1(i) =Dt(i)exp(−αtyiht(xi))
Zt
Output the final classifier:
H(x) = sign
(T∑
t=1
αtht(x)
)
t = 1
0 5 10 15 20 25 30 35 400
0.05
0.1
0.15
0.2
0.25
0.3
0.35
![Page 12: AdaBoost - cmp.felk.cvut.czcmp.felk.cvut.cz/cmp/courses/recognition/AdaBoost2/adaboost_talk.… · AdaBoost variants. 3/12 What is AdaBoost? AdaBoost is an algorithm for constructing](https://reader034.fdocuments.us/reader034/viewer/2022052019/6032ce98264d8d49d92220bd/html5/thumbnails/12.jpg)
4/12
The AdaBoost Algorithm
Given: (x1, y1), . . . , (xm, ym);xi ∈ X , yi ∈ {−1,+1}Initialise weights D1(i) = 1/m.For t = 1, ..., T :
� Find ht = arg minhj∈H
εj =m∑
i=1
Dt(i)Jyi 6= hj(xi)K
� If εt ≥ 1/2 then stop
� Set αt = 12 log(1−εt
εt)
� Update
Dt+1(i) =Dt(i)exp(−αtyiht(xi))
Zt
Output the final classifier:
H(x) = sign
(T∑
t=1
αtht(x)
)
t = 2
0 5 10 15 20 25 30 35 400
0.05
0.1
0.15
0.2
0.25
0.3
0.35
![Page 13: AdaBoost - cmp.felk.cvut.czcmp.felk.cvut.cz/cmp/courses/recognition/AdaBoost2/adaboost_talk.… · AdaBoost variants. 3/12 What is AdaBoost? AdaBoost is an algorithm for constructing](https://reader034.fdocuments.us/reader034/viewer/2022052019/6032ce98264d8d49d92220bd/html5/thumbnails/13.jpg)
4/12
The AdaBoost Algorithm
Given: (x1, y1), . . . , (xm, ym);xi ∈ X , yi ∈ {−1,+1}Initialise weights D1(i) = 1/m.For t = 1, ..., T :
� Find ht = arg minhj∈H
εj =m∑
i=1
Dt(i)Jyi 6= hj(xi)K
� If εt ≥ 1/2 then stop
� Set αt = 12 log(1−εt
εt)
� Update
Dt+1(i) =Dt(i)exp(−αtyiht(xi))
Zt
Output the final classifier:
H(x) = sign
(T∑
t=1
αtht(x)
)
t = 3
0 5 10 15 20 25 30 35 400
0.05
0.1
0.15
0.2
0.25
0.3
0.35
![Page 14: AdaBoost - cmp.felk.cvut.czcmp.felk.cvut.cz/cmp/courses/recognition/AdaBoost2/adaboost_talk.… · AdaBoost variants. 3/12 What is AdaBoost? AdaBoost is an algorithm for constructing](https://reader034.fdocuments.us/reader034/viewer/2022052019/6032ce98264d8d49d92220bd/html5/thumbnails/14.jpg)
4/12
The AdaBoost Algorithm
Given: (x1, y1), . . . , (xm, ym);xi ∈ X , yi ∈ {−1,+1}Initialise weights D1(i) = 1/m.For t = 1, ..., T :
� Find ht = arg minhj∈H
εj =m∑
i=1
Dt(i)Jyi 6= hj(xi)K
� If εt ≥ 1/2 then stop
� Set αt = 12 log(1−εt
εt)
� Update
Dt+1(i) =Dt(i)exp(−αtyiht(xi))
Zt
Output the final classifier:
H(x) = sign
(T∑
t=1
αtht(x)
)
t = 4
0 5 10 15 20 25 30 35 400
0.05
0.1
0.15
0.2
0.25
0.3
0.35
![Page 15: AdaBoost - cmp.felk.cvut.czcmp.felk.cvut.cz/cmp/courses/recognition/AdaBoost2/adaboost_talk.… · AdaBoost variants. 3/12 What is AdaBoost? AdaBoost is an algorithm for constructing](https://reader034.fdocuments.us/reader034/viewer/2022052019/6032ce98264d8d49d92220bd/html5/thumbnails/15.jpg)
4/12
The AdaBoost Algorithm
Given: (x1, y1), . . . , (xm, ym);xi ∈ X , yi ∈ {−1,+1}Initialise weights D1(i) = 1/m.For t = 1, ..., T :
� Find ht = arg minhj∈H
εj =m∑
i=1
Dt(i)Jyi 6= hj(xi)K
� If εt ≥ 1/2 then stop
� Set αt = 12 log(1−εt
εt)
� Update
Dt+1(i) =Dt(i)exp(−αtyiht(xi))
Zt
Output the final classifier:
H(x) = sign
(T∑
t=1
αtht(x)
)
t = 5
0 5 10 15 20 25 30 35 400
0.05
0.1
0.15
0.2
0.25
0.3
0.35
![Page 16: AdaBoost - cmp.felk.cvut.czcmp.felk.cvut.cz/cmp/courses/recognition/AdaBoost2/adaboost_talk.… · AdaBoost variants. 3/12 What is AdaBoost? AdaBoost is an algorithm for constructing](https://reader034.fdocuments.us/reader034/viewer/2022052019/6032ce98264d8d49d92220bd/html5/thumbnails/16.jpg)
4/12
The AdaBoost Algorithm
Given: (x1, y1), . . . , (xm, ym);xi ∈ X , yi ∈ {−1,+1}Initialise weights D1(i) = 1/m.For t = 1, ..., T :
� Find ht = arg minhj∈H
εj =m∑
i=1
Dt(i)Jyi 6= hj(xi)K
� If εt ≥ 1/2 then stop
� Set αt = 12 log(1−εt
εt)
� Update
Dt+1(i) =Dt(i)exp(−αtyiht(xi))
Zt
Output the final classifier:
H(x) = sign
(T∑
t=1
αtht(x)
)
t = 6
0 5 10 15 20 25 30 35 400
0.05
0.1
0.15
0.2
0.25
0.3
0.35
![Page 17: AdaBoost - cmp.felk.cvut.czcmp.felk.cvut.cz/cmp/courses/recognition/AdaBoost2/adaboost_talk.… · AdaBoost variants. 3/12 What is AdaBoost? AdaBoost is an algorithm for constructing](https://reader034.fdocuments.us/reader034/viewer/2022052019/6032ce98264d8d49d92220bd/html5/thumbnails/17.jpg)
4/12
The AdaBoost Algorithm
Given: (x1, y1), . . . , (xm, ym);xi ∈ X , yi ∈ {−1,+1}Initialise weights D1(i) = 1/m.For t = 1, ..., T :
� Find ht = arg minhj∈H
εj =m∑
i=1
Dt(i)Jyi 6= hj(xi)K
� If εt ≥ 1/2 then stop
� Set αt = 12 log(1−εt
εt)
� Update
Dt+1(i) =Dt(i)exp(−αtyiht(xi))
Zt
Output the final classifier:
H(x) = sign
(T∑
t=1
αtht(x)
)
t = 7
0 5 10 15 20 25 30 35 400
0.05
0.1
0.15
0.2
0.25
0.3
0.35
![Page 18: AdaBoost - cmp.felk.cvut.czcmp.felk.cvut.cz/cmp/courses/recognition/AdaBoost2/adaboost_talk.… · AdaBoost variants. 3/12 What is AdaBoost? AdaBoost is an algorithm for constructing](https://reader034.fdocuments.us/reader034/viewer/2022052019/6032ce98264d8d49d92220bd/html5/thumbnails/18.jpg)
4/12
The AdaBoost Algorithm
Given: (x1, y1), . . . , (xm, ym);xi ∈ X , yi ∈ {−1,+1}Initialise weights D1(i) = 1/m.For t = 1, ..., T :
� Find ht = arg minhj∈H
εj =m∑
i=1
Dt(i)Jyi 6= hj(xi)K
� If εt ≥ 1/2 then stop
� Set αt = 12 log(1−εt
εt)
� Update
Dt+1(i) =Dt(i)exp(−αtyiht(xi))
Zt
Output the final classifier:
H(x) = sign
(T∑
t=1
αtht(x)
)
t = 40
0 5 10 15 20 25 30 35 400
0.05
0.1
0.15
0.2
0.25
0.3
0.35
![Page 19: AdaBoost - cmp.felk.cvut.czcmp.felk.cvut.cz/cmp/courses/recognition/AdaBoost2/adaboost_talk.… · AdaBoost variants. 3/12 What is AdaBoost? AdaBoost is an algorithm for constructing](https://reader034.fdocuments.us/reader034/viewer/2022052019/6032ce98264d8d49d92220bd/html5/thumbnails/19.jpg)
5/12
Reweighting
Effect on the training set
Dt+1(i) =Dt(i)exp(−αtyiht(xi))
Zt
exp(−αtyiht(xi)){
< 1, yi = ht(xi)> 1, yi 6= ht(xi)
−2 −1.5 −1 −0.5 0 0.5 1 1.5 2
0
0.5
1
1.5
2
2.5
3
yf(x)
err
⇒ Increase (decrease) weight of wrongly (correctly) classified examples
⇒ The weight is the upper bound on the error of a given example.
⇒ All information about previously selected “features” is captured in Dt.
![Page 20: AdaBoost - cmp.felk.cvut.czcmp.felk.cvut.cz/cmp/courses/recognition/AdaBoost2/adaboost_talk.… · AdaBoost variants. 3/12 What is AdaBoost? AdaBoost is an algorithm for constructing](https://reader034.fdocuments.us/reader034/viewer/2022052019/6032ce98264d8d49d92220bd/html5/thumbnails/20.jpg)
5/12
Reweighting
Effect on the training set
Dt+1(i) =Dt(i)exp(−αtyiht(xi))
Zt
exp(−αtyiht(xi)){
< 1, yi = ht(xi)> 1, yi 6= ht(xi)
−2 −1.5 −1 −0.5 0 0.5 1 1.5 2
0
0.5
1
1.5
2
2.5
3
yf(x)
err
⇒ Increase (decrease) weight of wrongly (correctly) classified examples
⇒ The weight is the upper bound on the error of a given example.
⇒ All information about previously selected “features” is captured in Dt.
![Page 21: AdaBoost - cmp.felk.cvut.czcmp.felk.cvut.cz/cmp/courses/recognition/AdaBoost2/adaboost_talk.… · AdaBoost variants. 3/12 What is AdaBoost? AdaBoost is an algorithm for constructing](https://reader034.fdocuments.us/reader034/viewer/2022052019/6032ce98264d8d49d92220bd/html5/thumbnails/21.jpg)
5/12
Reweighting
Effect on the training set
Dt+1(i) =Dt(i)exp(−αtyiht(xi))
Zt
exp(−αtyiht(xi)){
< 1, yi = ht(xi)> 1, yi 6= ht(xi)
−2 −1.5 −1 −0.5 0 0.5 1 1.5 2
0
0.5
1
1.5
2
2.5
3
yf(x)
err
⇒ Increase (decrease) weight of wrongly (correctly) classified examples
⇒ The weight is the upper bound on the error of a given example.
⇒ All information about previously selected “features” is captured in Dt.
![Page 22: AdaBoost - cmp.felk.cvut.czcmp.felk.cvut.cz/cmp/courses/recognition/AdaBoost2/adaboost_talk.… · AdaBoost variants. 3/12 What is AdaBoost? AdaBoost is an algorithm for constructing](https://reader034.fdocuments.us/reader034/viewer/2022052019/6032ce98264d8d49d92220bd/html5/thumbnails/22.jpg)
5/12
Reweighting
Effect on the training set
Dt+1(i) =Dt(i)exp(−αtyiht(xi))
Zt
exp(−αtyiht(xi)){
< 1, yi = ht(xi)> 1, yi 6= ht(xi)
−2 −1.5 −1 −0.5 0 0.5 1 1.5 2
0
0.5
1
1.5
2
2.5
3
yf(x)
err
⇒ Increase (decrease) weight of wrongly (correctly) classified examples
⇒ The weight is the upper bound on the error of a given example.
⇒ All information about previously selected “features” is captured in Dt.
![Page 23: AdaBoost - cmp.felk.cvut.czcmp.felk.cvut.cz/cmp/courses/recognition/AdaBoost2/adaboost_talk.… · AdaBoost variants. 3/12 What is AdaBoost? AdaBoost is an algorithm for constructing](https://reader034.fdocuments.us/reader034/viewer/2022052019/6032ce98264d8d49d92220bd/html5/thumbnails/23.jpg)
6/12
Upper Bound Theorem
Theorem: The following upper bound holds on the training error of H
1m|{i : H(xi) 6= yi}| ≤
T∏t=1
Zt
Proof: By unravelling the update rule
DT+1(i) =exp(−
∑t αtyiht(xi))
m∏
t Zt
=exp(−yif(xi))
m∏
t Zt.
If H(xi) 6= yi then yif(xi) ≤ 0 implying that exp(−yif(xi)) > 1, thus
JH(xi) 6= yiK ≤ exp(−yif(xi))1m
∑i
JH(xi) 6= yiK ≤ 1m
∑i
exp(−yif(xi))
=∑
i
(∏
t
Zt)DT+1(i) =∏
t
Zt−2 −1.5 −1 −0.5 0 0.5 1 1.5 2
0
0.5
1
1.5
2
2.5
3
yf(x)
err
![Page 24: AdaBoost - cmp.felk.cvut.czcmp.felk.cvut.cz/cmp/courses/recognition/AdaBoost2/adaboost_talk.… · AdaBoost variants. 3/12 What is AdaBoost? AdaBoost is an algorithm for constructing](https://reader034.fdocuments.us/reader034/viewer/2022052019/6032ce98264d8d49d92220bd/html5/thumbnails/24.jpg)
7/12
Consequences of the Theorem
� Instead of minimising the training error, its upper bound can be minimised.
� This can be done by minimising Zt in each training round by:
• Choosing optimal ht, and
• Finding optimal αt.
� Can be proved to maximise margin.
![Page 25: AdaBoost - cmp.felk.cvut.czcmp.felk.cvut.cz/cmp/courses/recognition/AdaBoost2/adaboost_talk.… · AdaBoost variants. 3/12 What is AdaBoost? AdaBoost is an algorithm for constructing](https://reader034.fdocuments.us/reader034/viewer/2022052019/6032ce98264d8d49d92220bd/html5/thumbnails/25.jpg)
8/12
Choosing αt
We attempt to minimise Zt =∑
i Dt(i)exp(−αtyiht(xi)):
dZ
dα= −
m∑i=1
D(i)yih(xi)e−yiαih(xi) = 0
−∑
i:yi=h(xi)
D(i)e−α +∑
i:yi 6=h(xi)
D(i)eα = 0
−e−α(1− ε) + eαε = 0
α =12
log1− ε
ε
⇒ The minimisator of the upper bound is αt = 12 log 1−εt
εt.
![Page 26: AdaBoost - cmp.felk.cvut.czcmp.felk.cvut.cz/cmp/courses/recognition/AdaBoost2/adaboost_talk.… · AdaBoost variants. 3/12 What is AdaBoost? AdaBoost is an algorithm for constructing](https://reader034.fdocuments.us/reader034/viewer/2022052019/6032ce98264d8d49d92220bd/html5/thumbnails/26.jpg)
9/12
Choosing ht
Weak classifier examples
� Decision tree, Perceptron – H infinite
� Selecting the best one from given finite set H
Justification of the weighted error minimisation
Having αt = 12 log 1−εt
εt
Zt =m∑
i=1
Dt(i)e−yiαiht(xi)
=∑
i:yi=ht(xi)
Dt(i)e−αt +∑
i:yi 6=ht(xi)
Dt(i)eαt
= (1− εt)e−αt + εteαt
= 2√
εt(1− εt)
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
εt
Z t
⇒ Zt is minimised by selecting ht with minimal weighted error εt.
![Page 27: AdaBoost - cmp.felk.cvut.czcmp.felk.cvut.cz/cmp/courses/recognition/AdaBoost2/adaboost_talk.… · AdaBoost variants. 3/12 What is AdaBoost? AdaBoost is an algorithm for constructing](https://reader034.fdocuments.us/reader034/viewer/2022052019/6032ce98264d8d49d92220bd/html5/thumbnails/27.jpg)
10/12
The Algorithm Recapitulation
Given: (x1, y1), . . . , (xm, ym);xi ∈ X , yi ∈ {−1,+1}
![Page 28: AdaBoost - cmp.felk.cvut.czcmp.felk.cvut.cz/cmp/courses/recognition/AdaBoost2/adaboost_talk.… · AdaBoost variants. 3/12 What is AdaBoost? AdaBoost is an algorithm for constructing](https://reader034.fdocuments.us/reader034/viewer/2022052019/6032ce98264d8d49d92220bd/html5/thumbnails/28.jpg)
10/12
The Algorithm Recapitulation
Given: (x1, y1), . . . , (xm, ym);xi ∈ X , yi ∈ {−1,+1}Initialise weights D1(i) = 1/m.
![Page 29: AdaBoost - cmp.felk.cvut.czcmp.felk.cvut.cz/cmp/courses/recognition/AdaBoost2/adaboost_talk.… · AdaBoost variants. 3/12 What is AdaBoost? AdaBoost is an algorithm for constructing](https://reader034.fdocuments.us/reader034/viewer/2022052019/6032ce98264d8d49d92220bd/html5/thumbnails/29.jpg)
10/12
The Algorithm Recapitulation
Given: (x1, y1), . . . , (xm, ym);xi ∈ X , yi ∈ {−1,+1}Initialise weights D1(i) = 1/m.For t = 1, ..., T :
![Page 30: AdaBoost - cmp.felk.cvut.czcmp.felk.cvut.cz/cmp/courses/recognition/AdaBoost2/adaboost_talk.… · AdaBoost variants. 3/12 What is AdaBoost? AdaBoost is an algorithm for constructing](https://reader034.fdocuments.us/reader034/viewer/2022052019/6032ce98264d8d49d92220bd/html5/thumbnails/30.jpg)
10/12
The Algorithm Recapitulation
Given: (x1, y1), . . . , (xm, ym);xi ∈ X , yi ∈ {−1,+1}Initialise weights D1(i) = 1/m.For t = 1, ..., T :
� Find ht = arg minhj∈H
εj =m∑
i=1
Dt(i)Jyi 6= hj(xi)K
t = 1
![Page 31: AdaBoost - cmp.felk.cvut.czcmp.felk.cvut.cz/cmp/courses/recognition/AdaBoost2/adaboost_talk.… · AdaBoost variants. 3/12 What is AdaBoost? AdaBoost is an algorithm for constructing](https://reader034.fdocuments.us/reader034/viewer/2022052019/6032ce98264d8d49d92220bd/html5/thumbnails/31.jpg)
10/12
The Algorithm Recapitulation
Given: (x1, y1), . . . , (xm, ym);xi ∈ X , yi ∈ {−1,+1}Initialise weights D1(i) = 1/m.For t = 1, ..., T :
� Find ht = arg minhj∈H
εj =m∑
i=1
Dt(i)Jyi 6= hj(xi)K
� If εt ≥ 1/2 then stop
t = 1
![Page 32: AdaBoost - cmp.felk.cvut.czcmp.felk.cvut.cz/cmp/courses/recognition/AdaBoost2/adaboost_talk.… · AdaBoost variants. 3/12 What is AdaBoost? AdaBoost is an algorithm for constructing](https://reader034.fdocuments.us/reader034/viewer/2022052019/6032ce98264d8d49d92220bd/html5/thumbnails/32.jpg)
10/12
The Algorithm Recapitulation
Given: (x1, y1), . . . , (xm, ym);xi ∈ X , yi ∈ {−1,+1}Initialise weights D1(i) = 1/m.For t = 1, ..., T :
� Find ht = arg minhj∈H
εj =m∑
i=1
Dt(i)Jyi 6= hj(xi)K
� If εt ≥ 1/2 then stop
� Set αt = 12 log(1−εt
εt)
t = 1
![Page 33: AdaBoost - cmp.felk.cvut.czcmp.felk.cvut.cz/cmp/courses/recognition/AdaBoost2/adaboost_talk.… · AdaBoost variants. 3/12 What is AdaBoost? AdaBoost is an algorithm for constructing](https://reader034.fdocuments.us/reader034/viewer/2022052019/6032ce98264d8d49d92220bd/html5/thumbnails/33.jpg)
10/12
The Algorithm Recapitulation
Given: (x1, y1), . . . , (xm, ym);xi ∈ X , yi ∈ {−1,+1}Initialise weights D1(i) = 1/m.For t = 1, ..., T :
� Find ht = arg minhj∈H
εj =m∑
i=1
Dt(i)Jyi 6= hj(xi)K
� If εt ≥ 1/2 then stop
� Set αt = 12 log(1−εt
εt)
� Update
Dt+1(i) =Dt(i)exp(−αtyiht(xi))
Zt
t = 1
![Page 34: AdaBoost - cmp.felk.cvut.czcmp.felk.cvut.cz/cmp/courses/recognition/AdaBoost2/adaboost_talk.… · AdaBoost variants. 3/12 What is AdaBoost? AdaBoost is an algorithm for constructing](https://reader034.fdocuments.us/reader034/viewer/2022052019/6032ce98264d8d49d92220bd/html5/thumbnails/34.jpg)
10/12
The Algorithm Recapitulation
Given: (x1, y1), . . . , (xm, ym);xi ∈ X , yi ∈ {−1,+1}Initialise weights D1(i) = 1/m.For t = 1, ..., T :
� Find ht = arg minhj∈H
εj =m∑
i=1
Dt(i)Jyi 6= hj(xi)K
� If εt ≥ 1/2 then stop
� Set αt = 12 log(1−εt
εt)
� Update
Dt+1(i) =Dt(i)exp(−αtyiht(xi))
Zt
Output the final classifier:
H(x) = sign
(T∑
t=1
αtht(x)
)
t = 1
0 5 10 15 20 25 30 35 400
0.05
0.1
0.15
0.2
0.25
0.3
0.35
![Page 35: AdaBoost - cmp.felk.cvut.czcmp.felk.cvut.cz/cmp/courses/recognition/AdaBoost2/adaboost_talk.… · AdaBoost variants. 3/12 What is AdaBoost? AdaBoost is an algorithm for constructing](https://reader034.fdocuments.us/reader034/viewer/2022052019/6032ce98264d8d49d92220bd/html5/thumbnails/35.jpg)
10/12
The Algorithm Recapitulation
Given: (x1, y1), . . . , (xm, ym);xi ∈ X , yi ∈ {−1,+1}Initialise weights D1(i) = 1/m.For t = 1, ..., T :
� Find ht = arg minhj∈H
εj =m∑
i=1
Dt(i)Jyi 6= hj(xi)K
� If εt ≥ 1/2 then stop
� Set αt = 12 log(1−εt
εt)
� Update
Dt+1(i) =Dt(i)exp(−αtyiht(xi))
Zt
Output the final classifier:
H(x) = sign
(T∑
t=1
αtht(x)
)
t = 2
0 5 10 15 20 25 30 35 400
0.05
0.1
0.15
0.2
0.25
0.3
0.35
![Page 36: AdaBoost - cmp.felk.cvut.czcmp.felk.cvut.cz/cmp/courses/recognition/AdaBoost2/adaboost_talk.… · AdaBoost variants. 3/12 What is AdaBoost? AdaBoost is an algorithm for constructing](https://reader034.fdocuments.us/reader034/viewer/2022052019/6032ce98264d8d49d92220bd/html5/thumbnails/36.jpg)
10/12
The Algorithm Recapitulation
Given: (x1, y1), . . . , (xm, ym);xi ∈ X , yi ∈ {−1,+1}Initialise weights D1(i) = 1/m.For t = 1, ..., T :
� Find ht = arg minhj∈H
εj =m∑
i=1
Dt(i)Jyi 6= hj(xi)K
� If εt ≥ 1/2 then stop
� Set αt = 12 log(1−εt
εt)
� Update
Dt+1(i) =Dt(i)exp(−αtyiht(xi))
Zt
Output the final classifier:
H(x) = sign
(T∑
t=1
αtht(x)
)
t = 3
0 5 10 15 20 25 30 35 400
0.05
0.1
0.15
0.2
0.25
0.3
0.35
![Page 37: AdaBoost - cmp.felk.cvut.czcmp.felk.cvut.cz/cmp/courses/recognition/AdaBoost2/adaboost_talk.… · AdaBoost variants. 3/12 What is AdaBoost? AdaBoost is an algorithm for constructing](https://reader034.fdocuments.us/reader034/viewer/2022052019/6032ce98264d8d49d92220bd/html5/thumbnails/37.jpg)
10/12
The Algorithm Recapitulation
Given: (x1, y1), . . . , (xm, ym);xi ∈ X , yi ∈ {−1,+1}Initialise weights D1(i) = 1/m.For t = 1, ..., T :
� Find ht = arg minhj∈H
εj =m∑
i=1
Dt(i)Jyi 6= hj(xi)K
� If εt ≥ 1/2 then stop
� Set αt = 12 log(1−εt
εt)
� Update
Dt+1(i) =Dt(i)exp(−αtyiht(xi))
Zt
Output the final classifier:
H(x) = sign
(T∑
t=1
αtht(x)
)
t = 4
0 5 10 15 20 25 30 35 400
0.05
0.1
0.15
0.2
0.25
0.3
0.35
![Page 38: AdaBoost - cmp.felk.cvut.czcmp.felk.cvut.cz/cmp/courses/recognition/AdaBoost2/adaboost_talk.… · AdaBoost variants. 3/12 What is AdaBoost? AdaBoost is an algorithm for constructing](https://reader034.fdocuments.us/reader034/viewer/2022052019/6032ce98264d8d49d92220bd/html5/thumbnails/38.jpg)
10/12
The Algorithm Recapitulation
Given: (x1, y1), . . . , (xm, ym);xi ∈ X , yi ∈ {−1,+1}Initialise weights D1(i) = 1/m.For t = 1, ..., T :
� Find ht = arg minhj∈H
εj =m∑
i=1
Dt(i)Jyi 6= hj(xi)K
� If εt ≥ 1/2 then stop
� Set αt = 12 log(1−εt
εt)
� Update
Dt+1(i) =Dt(i)exp(−αtyiht(xi))
Zt
Output the final classifier:
H(x) = sign
(T∑
t=1
αtht(x)
)
t = 5
0 5 10 15 20 25 30 35 400
0.05
0.1
0.15
0.2
0.25
0.3
0.35
![Page 39: AdaBoost - cmp.felk.cvut.czcmp.felk.cvut.cz/cmp/courses/recognition/AdaBoost2/adaboost_talk.… · AdaBoost variants. 3/12 What is AdaBoost? AdaBoost is an algorithm for constructing](https://reader034.fdocuments.us/reader034/viewer/2022052019/6032ce98264d8d49d92220bd/html5/thumbnails/39.jpg)
10/12
The Algorithm Recapitulation
Given: (x1, y1), . . . , (xm, ym);xi ∈ X , yi ∈ {−1,+1}Initialise weights D1(i) = 1/m.For t = 1, ..., T :
� Find ht = arg minhj∈H
εj =m∑
i=1
Dt(i)Jyi 6= hj(xi)K
� If εt ≥ 1/2 then stop
� Set αt = 12 log(1−εt
εt)
� Update
Dt+1(i) =Dt(i)exp(−αtyiht(xi))
Zt
Output the final classifier:
H(x) = sign
(T∑
t=1
αtht(x)
)
t = 6
0 5 10 15 20 25 30 35 400
0.05
0.1
0.15
0.2
0.25
0.3
0.35
![Page 40: AdaBoost - cmp.felk.cvut.czcmp.felk.cvut.cz/cmp/courses/recognition/AdaBoost2/adaboost_talk.… · AdaBoost variants. 3/12 What is AdaBoost? AdaBoost is an algorithm for constructing](https://reader034.fdocuments.us/reader034/viewer/2022052019/6032ce98264d8d49d92220bd/html5/thumbnails/40.jpg)
10/12
The Algorithm Recapitulation
Given: (x1, y1), . . . , (xm, ym);xi ∈ X , yi ∈ {−1,+1}Initialise weights D1(i) = 1/m.For t = 1, ..., T :
� Find ht = arg minhj∈H
εj =m∑
i=1
Dt(i)Jyi 6= hj(xi)K
� If εt ≥ 1/2 then stop
� Set αt = 12 log(1−εt
εt)
� Update
Dt+1(i) =Dt(i)exp(−αtyiht(xi))
Zt
Output the final classifier:
H(x) = sign
(T∑
t=1
αtht(x)
)
t = 7
0 5 10 15 20 25 30 35 400
0.05
0.1
0.15
0.2
0.25
0.3
0.35
![Page 41: AdaBoost - cmp.felk.cvut.czcmp.felk.cvut.cz/cmp/courses/recognition/AdaBoost2/adaboost_talk.… · AdaBoost variants. 3/12 What is AdaBoost? AdaBoost is an algorithm for constructing](https://reader034.fdocuments.us/reader034/viewer/2022052019/6032ce98264d8d49d92220bd/html5/thumbnails/41.jpg)
10/12
The Algorithm Recapitulation
Given: (x1, y1), . . . , (xm, ym);xi ∈ X , yi ∈ {−1,+1}Initialise weights D1(i) = 1/m.For t = 1, ..., T :
� Find ht = arg minhj∈H
εj =m∑
i=1
Dt(i)Jyi 6= hj(xi)K
� If εt ≥ 1/2 then stop
� Set αt = 12 log(1−εt
εt)
� Update
Dt+1(i) =Dt(i)exp(−αtyiht(xi))
Zt
Output the final classifier:
H(x) = sign
(T∑
t=1
αtht(x)
)
t = 40
0 5 10 15 20 25 30 35 400
0.05
0.1
0.15
0.2
0.25
0.3
0.35
![Page 42: AdaBoost - cmp.felk.cvut.czcmp.felk.cvut.cz/cmp/courses/recognition/AdaBoost2/adaboost_talk.… · AdaBoost variants. 3/12 What is AdaBoost? AdaBoost is an algorithm for constructing](https://reader034.fdocuments.us/reader034/viewer/2022052019/6032ce98264d8d49d92220bd/html5/thumbnails/42.jpg)
11/12
AdaBoost variants
Freund & Schapire 1995
� Discrete (h : X → {0, 1})
� Multiclass AdaBoost.M1 (h : X → {0, 1, ..., k})
� Multiclass AdaBoost.M2 (h : X → [0, 1]k)
� Real valued AdaBoost.R (Y = [0, 1], h : X → [0, 1])
Schapire & Singer 1997
� Confidence rated prediction (h : X → R, two-class)
� Multilabel AdaBoost.MR, AdaBoost.MH (different formulation of minimised loss)
... Many other modifications since then (WaldBoost, cascaded AB, online AB, ...)
![Page 43: AdaBoost - cmp.felk.cvut.czcmp.felk.cvut.cz/cmp/courses/recognition/AdaBoost2/adaboost_talk.… · AdaBoost variants. 3/12 What is AdaBoost? AdaBoost is an algorithm for constructing](https://reader034.fdocuments.us/reader034/viewer/2022052019/6032ce98264d8d49d92220bd/html5/thumbnails/43.jpg)
12/12
Pros and cons of AdaBoost
Advantages
� Very simple to implement
� General learning scheme - can be used for various learning tasks
� Feature selection on very large sets of features
� Fairly good generalisation
Disadvantages
� Suboptimal solution (greedy learning)
� Can overfit in presence of noise (has been addressed recently)
![Page 44: AdaBoost - cmp.felk.cvut.czcmp.felk.cvut.cz/cmp/courses/recognition/AdaBoost2/adaboost_talk.… · AdaBoost variants. 3/12 What is AdaBoost? AdaBoost is an algorithm for constructing](https://reader034.fdocuments.us/reader034/viewer/2022052019/6032ce98264d8d49d92220bd/html5/thumbnails/44.jpg)
![Page 45: AdaBoost - cmp.felk.cvut.czcmp.felk.cvut.cz/cmp/courses/recognition/AdaBoost2/adaboost_talk.… · AdaBoost variants. 3/12 What is AdaBoost? AdaBoost is an algorithm for constructing](https://reader034.fdocuments.us/reader034/viewer/2022052019/6032ce98264d8d49d92220bd/html5/thumbnails/45.jpg)
![Page 46: AdaBoost - cmp.felk.cvut.czcmp.felk.cvut.cz/cmp/courses/recognition/AdaBoost2/adaboost_talk.… · AdaBoost variants. 3/12 What is AdaBoost? AdaBoost is an algorithm for constructing](https://reader034.fdocuments.us/reader034/viewer/2022052019/6032ce98264d8d49d92220bd/html5/thumbnails/46.jpg)
![Page 47: AdaBoost - cmp.felk.cvut.czcmp.felk.cvut.cz/cmp/courses/recognition/AdaBoost2/adaboost_talk.… · AdaBoost variants. 3/12 What is AdaBoost? AdaBoost is an algorithm for constructing](https://reader034.fdocuments.us/reader034/viewer/2022052019/6032ce98264d8d49d92220bd/html5/thumbnails/47.jpg)
![Page 48: AdaBoost - cmp.felk.cvut.czcmp.felk.cvut.cz/cmp/courses/recognition/AdaBoost2/adaboost_talk.… · AdaBoost variants. 3/12 What is AdaBoost? AdaBoost is an algorithm for constructing](https://reader034.fdocuments.us/reader034/viewer/2022052019/6032ce98264d8d49d92220bd/html5/thumbnails/48.jpg)
![Page 49: AdaBoost - cmp.felk.cvut.czcmp.felk.cvut.cz/cmp/courses/recognition/AdaBoost2/adaboost_talk.… · AdaBoost variants. 3/12 What is AdaBoost? AdaBoost is an algorithm for constructing](https://reader034.fdocuments.us/reader034/viewer/2022052019/6032ce98264d8d49d92220bd/html5/thumbnails/49.jpg)
![Page 50: AdaBoost - cmp.felk.cvut.czcmp.felk.cvut.cz/cmp/courses/recognition/AdaBoost2/adaboost_talk.… · AdaBoost variants. 3/12 What is AdaBoost? AdaBoost is an algorithm for constructing](https://reader034.fdocuments.us/reader034/viewer/2022052019/6032ce98264d8d49d92220bd/html5/thumbnails/50.jpg)
![Page 51: AdaBoost - cmp.felk.cvut.czcmp.felk.cvut.cz/cmp/courses/recognition/AdaBoost2/adaboost_talk.… · AdaBoost variants. 3/12 What is AdaBoost? AdaBoost is an algorithm for constructing](https://reader034.fdocuments.us/reader034/viewer/2022052019/6032ce98264d8d49d92220bd/html5/thumbnails/51.jpg)
![Page 52: AdaBoost - cmp.felk.cvut.czcmp.felk.cvut.cz/cmp/courses/recognition/AdaBoost2/adaboost_talk.… · AdaBoost variants. 3/12 What is AdaBoost? AdaBoost is an algorithm for constructing](https://reader034.fdocuments.us/reader034/viewer/2022052019/6032ce98264d8d49d92220bd/html5/thumbnails/52.jpg)
0 5 10 15 20 25 30 35 400
0.05
0.1
0.15
0.2
0.25
0.3
0.35
![Page 53: AdaBoost - cmp.felk.cvut.czcmp.felk.cvut.cz/cmp/courses/recognition/AdaBoost2/adaboost_talk.… · AdaBoost variants. 3/12 What is AdaBoost? AdaBoost is an algorithm for constructing](https://reader034.fdocuments.us/reader034/viewer/2022052019/6032ce98264d8d49d92220bd/html5/thumbnails/53.jpg)
![Page 54: AdaBoost - cmp.felk.cvut.czcmp.felk.cvut.cz/cmp/courses/recognition/AdaBoost2/adaboost_talk.… · AdaBoost variants. 3/12 What is AdaBoost? AdaBoost is an algorithm for constructing](https://reader034.fdocuments.us/reader034/viewer/2022052019/6032ce98264d8d49d92220bd/html5/thumbnails/54.jpg)
0 5 10 15 20 25 30 35 400
0.05
0.1
0.15
0.2
0.25
0.3
0.35
![Page 55: AdaBoost - cmp.felk.cvut.czcmp.felk.cvut.cz/cmp/courses/recognition/AdaBoost2/adaboost_talk.… · AdaBoost variants. 3/12 What is AdaBoost? AdaBoost is an algorithm for constructing](https://reader034.fdocuments.us/reader034/viewer/2022052019/6032ce98264d8d49d92220bd/html5/thumbnails/55.jpg)
![Page 56: AdaBoost - cmp.felk.cvut.czcmp.felk.cvut.cz/cmp/courses/recognition/AdaBoost2/adaboost_talk.… · AdaBoost variants. 3/12 What is AdaBoost? AdaBoost is an algorithm for constructing](https://reader034.fdocuments.us/reader034/viewer/2022052019/6032ce98264d8d49d92220bd/html5/thumbnails/56.jpg)
0 5 10 15 20 25 30 35 400
0.05
0.1
0.15
0.2
0.25
0.3
0.35
![Page 57: AdaBoost - cmp.felk.cvut.czcmp.felk.cvut.cz/cmp/courses/recognition/AdaBoost2/adaboost_talk.… · AdaBoost variants. 3/12 What is AdaBoost? AdaBoost is an algorithm for constructing](https://reader034.fdocuments.us/reader034/viewer/2022052019/6032ce98264d8d49d92220bd/html5/thumbnails/57.jpg)
![Page 58: AdaBoost - cmp.felk.cvut.czcmp.felk.cvut.cz/cmp/courses/recognition/AdaBoost2/adaboost_talk.… · AdaBoost variants. 3/12 What is AdaBoost? AdaBoost is an algorithm for constructing](https://reader034.fdocuments.us/reader034/viewer/2022052019/6032ce98264d8d49d92220bd/html5/thumbnails/58.jpg)
0 5 10 15 20 25 30 35 400
0.05
0.1
0.15
0.2
0.25
0.3
0.35
![Page 59: AdaBoost - cmp.felk.cvut.czcmp.felk.cvut.cz/cmp/courses/recognition/AdaBoost2/adaboost_talk.… · AdaBoost variants. 3/12 What is AdaBoost? AdaBoost is an algorithm for constructing](https://reader034.fdocuments.us/reader034/viewer/2022052019/6032ce98264d8d49d92220bd/html5/thumbnails/59.jpg)
![Page 60: AdaBoost - cmp.felk.cvut.czcmp.felk.cvut.cz/cmp/courses/recognition/AdaBoost2/adaboost_talk.… · AdaBoost variants. 3/12 What is AdaBoost? AdaBoost is an algorithm for constructing](https://reader034.fdocuments.us/reader034/viewer/2022052019/6032ce98264d8d49d92220bd/html5/thumbnails/60.jpg)
0 5 10 15 20 25 30 35 400
0.05
0.1
0.15
0.2
0.25
0.3
0.35
![Page 61: AdaBoost - cmp.felk.cvut.czcmp.felk.cvut.cz/cmp/courses/recognition/AdaBoost2/adaboost_talk.… · AdaBoost variants. 3/12 What is AdaBoost? AdaBoost is an algorithm for constructing](https://reader034.fdocuments.us/reader034/viewer/2022052019/6032ce98264d8d49d92220bd/html5/thumbnails/61.jpg)
![Page 62: AdaBoost - cmp.felk.cvut.czcmp.felk.cvut.cz/cmp/courses/recognition/AdaBoost2/adaboost_talk.… · AdaBoost variants. 3/12 What is AdaBoost? AdaBoost is an algorithm for constructing](https://reader034.fdocuments.us/reader034/viewer/2022052019/6032ce98264d8d49d92220bd/html5/thumbnails/62.jpg)
0 5 10 15 20 25 30 35 400
0.05
0.1
0.15
0.2
0.25
0.3
0.35
![Page 63: AdaBoost - cmp.felk.cvut.czcmp.felk.cvut.cz/cmp/courses/recognition/AdaBoost2/adaboost_talk.… · AdaBoost variants. 3/12 What is AdaBoost? AdaBoost is an algorithm for constructing](https://reader034.fdocuments.us/reader034/viewer/2022052019/6032ce98264d8d49d92220bd/html5/thumbnails/63.jpg)
![Page 64: AdaBoost - cmp.felk.cvut.czcmp.felk.cvut.cz/cmp/courses/recognition/AdaBoost2/adaboost_talk.… · AdaBoost variants. 3/12 What is AdaBoost? AdaBoost is an algorithm for constructing](https://reader034.fdocuments.us/reader034/viewer/2022052019/6032ce98264d8d49d92220bd/html5/thumbnails/64.jpg)
0 5 10 15 20 25 30 35 400
0.05
0.1
0.15
0.2
0.25
0.3
0.35
![Page 65: AdaBoost - cmp.felk.cvut.czcmp.felk.cvut.cz/cmp/courses/recognition/AdaBoost2/adaboost_talk.… · AdaBoost variants. 3/12 What is AdaBoost? AdaBoost is an algorithm for constructing](https://reader034.fdocuments.us/reader034/viewer/2022052019/6032ce98264d8d49d92220bd/html5/thumbnails/65.jpg)
![Page 66: AdaBoost - cmp.felk.cvut.czcmp.felk.cvut.cz/cmp/courses/recognition/AdaBoost2/adaboost_talk.… · AdaBoost variants. 3/12 What is AdaBoost? AdaBoost is an algorithm for constructing](https://reader034.fdocuments.us/reader034/viewer/2022052019/6032ce98264d8d49d92220bd/html5/thumbnails/66.jpg)
0 5 10 15 20 25 30 35 400
0.05
0.1
0.15
0.2
0.25
0.3
0.35
![Page 67: AdaBoost - cmp.felk.cvut.czcmp.felk.cvut.cz/cmp/courses/recognition/AdaBoost2/adaboost_talk.… · AdaBoost variants. 3/12 What is AdaBoost? AdaBoost is an algorithm for constructing](https://reader034.fdocuments.us/reader034/viewer/2022052019/6032ce98264d8d49d92220bd/html5/thumbnails/67.jpg)
![Page 68: AdaBoost - cmp.felk.cvut.czcmp.felk.cvut.cz/cmp/courses/recognition/AdaBoost2/adaboost_talk.… · AdaBoost variants. 3/12 What is AdaBoost? AdaBoost is an algorithm for constructing](https://reader034.fdocuments.us/reader034/viewer/2022052019/6032ce98264d8d49d92220bd/html5/thumbnails/68.jpg)
![Page 69: AdaBoost - cmp.felk.cvut.czcmp.felk.cvut.cz/cmp/courses/recognition/AdaBoost2/adaboost_talk.… · AdaBoost variants. 3/12 What is AdaBoost? AdaBoost is an algorithm for constructing](https://reader034.fdocuments.us/reader034/viewer/2022052019/6032ce98264d8d49d92220bd/html5/thumbnails/69.jpg)
![Page 70: AdaBoost - cmp.felk.cvut.czcmp.felk.cvut.cz/cmp/courses/recognition/AdaBoost2/adaboost_talk.… · AdaBoost variants. 3/12 What is AdaBoost? AdaBoost is an algorithm for constructing](https://reader034.fdocuments.us/reader034/viewer/2022052019/6032ce98264d8d49d92220bd/html5/thumbnails/70.jpg)
![Page 71: AdaBoost - cmp.felk.cvut.czcmp.felk.cvut.cz/cmp/courses/recognition/AdaBoost2/adaboost_talk.… · AdaBoost variants. 3/12 What is AdaBoost? AdaBoost is an algorithm for constructing](https://reader034.fdocuments.us/reader034/viewer/2022052019/6032ce98264d8d49d92220bd/html5/thumbnails/71.jpg)
![Page 72: AdaBoost - cmp.felk.cvut.czcmp.felk.cvut.cz/cmp/courses/recognition/AdaBoost2/adaboost_talk.… · AdaBoost variants. 3/12 What is AdaBoost? AdaBoost is an algorithm for constructing](https://reader034.fdocuments.us/reader034/viewer/2022052019/6032ce98264d8d49d92220bd/html5/thumbnails/72.jpg)
![Page 73: AdaBoost - cmp.felk.cvut.czcmp.felk.cvut.cz/cmp/courses/recognition/AdaBoost2/adaboost_talk.… · AdaBoost variants. 3/12 What is AdaBoost? AdaBoost is an algorithm for constructing](https://reader034.fdocuments.us/reader034/viewer/2022052019/6032ce98264d8d49d92220bd/html5/thumbnails/73.jpg)
![Page 74: AdaBoost - cmp.felk.cvut.czcmp.felk.cvut.cz/cmp/courses/recognition/AdaBoost2/adaboost_talk.… · AdaBoost variants. 3/12 What is AdaBoost? AdaBoost is an algorithm for constructing](https://reader034.fdocuments.us/reader034/viewer/2022052019/6032ce98264d8d49d92220bd/html5/thumbnails/74.jpg)
0 5 10 15 20 25 30 35 400
0.05
0.1
0.15
0.2
0.25
0.3
0.35
![Page 75: AdaBoost - cmp.felk.cvut.czcmp.felk.cvut.cz/cmp/courses/recognition/AdaBoost2/adaboost_talk.… · AdaBoost variants. 3/12 What is AdaBoost? AdaBoost is an algorithm for constructing](https://reader034.fdocuments.us/reader034/viewer/2022052019/6032ce98264d8d49d92220bd/html5/thumbnails/75.jpg)
![Page 76: AdaBoost - cmp.felk.cvut.czcmp.felk.cvut.cz/cmp/courses/recognition/AdaBoost2/adaboost_talk.… · AdaBoost variants. 3/12 What is AdaBoost? AdaBoost is an algorithm for constructing](https://reader034.fdocuments.us/reader034/viewer/2022052019/6032ce98264d8d49d92220bd/html5/thumbnails/76.jpg)
0 5 10 15 20 25 30 35 400
0.05
0.1
0.15
0.2
0.25
0.3
0.35
![Page 77: AdaBoost - cmp.felk.cvut.czcmp.felk.cvut.cz/cmp/courses/recognition/AdaBoost2/adaboost_talk.… · AdaBoost variants. 3/12 What is AdaBoost? AdaBoost is an algorithm for constructing](https://reader034.fdocuments.us/reader034/viewer/2022052019/6032ce98264d8d49d92220bd/html5/thumbnails/77.jpg)
![Page 78: AdaBoost - cmp.felk.cvut.czcmp.felk.cvut.cz/cmp/courses/recognition/AdaBoost2/adaboost_talk.… · AdaBoost variants. 3/12 What is AdaBoost? AdaBoost is an algorithm for constructing](https://reader034.fdocuments.us/reader034/viewer/2022052019/6032ce98264d8d49d92220bd/html5/thumbnails/78.jpg)
0 5 10 15 20 25 30 35 400
0.05
0.1
0.15
0.2
0.25
0.3
0.35
![Page 79: AdaBoost - cmp.felk.cvut.czcmp.felk.cvut.cz/cmp/courses/recognition/AdaBoost2/adaboost_talk.… · AdaBoost variants. 3/12 What is AdaBoost? AdaBoost is an algorithm for constructing](https://reader034.fdocuments.us/reader034/viewer/2022052019/6032ce98264d8d49d92220bd/html5/thumbnails/79.jpg)
![Page 80: AdaBoost - cmp.felk.cvut.czcmp.felk.cvut.cz/cmp/courses/recognition/AdaBoost2/adaboost_talk.… · AdaBoost variants. 3/12 What is AdaBoost? AdaBoost is an algorithm for constructing](https://reader034.fdocuments.us/reader034/viewer/2022052019/6032ce98264d8d49d92220bd/html5/thumbnails/80.jpg)
0 5 10 15 20 25 30 35 400
0.05
0.1
0.15
0.2
0.25
0.3
0.35
![Page 81: AdaBoost - cmp.felk.cvut.czcmp.felk.cvut.cz/cmp/courses/recognition/AdaBoost2/adaboost_talk.… · AdaBoost variants. 3/12 What is AdaBoost? AdaBoost is an algorithm for constructing](https://reader034.fdocuments.us/reader034/viewer/2022052019/6032ce98264d8d49d92220bd/html5/thumbnails/81.jpg)
![Page 82: AdaBoost - cmp.felk.cvut.czcmp.felk.cvut.cz/cmp/courses/recognition/AdaBoost2/adaboost_talk.… · AdaBoost variants. 3/12 What is AdaBoost? AdaBoost is an algorithm for constructing](https://reader034.fdocuments.us/reader034/viewer/2022052019/6032ce98264d8d49d92220bd/html5/thumbnails/82.jpg)
0 5 10 15 20 25 30 35 400
0.05
0.1
0.15
0.2
0.25
0.3
0.35
![Page 83: AdaBoost - cmp.felk.cvut.czcmp.felk.cvut.cz/cmp/courses/recognition/AdaBoost2/adaboost_talk.… · AdaBoost variants. 3/12 What is AdaBoost? AdaBoost is an algorithm for constructing](https://reader034.fdocuments.us/reader034/viewer/2022052019/6032ce98264d8d49d92220bd/html5/thumbnails/83.jpg)
![Page 84: AdaBoost - cmp.felk.cvut.czcmp.felk.cvut.cz/cmp/courses/recognition/AdaBoost2/adaboost_talk.… · AdaBoost variants. 3/12 What is AdaBoost? AdaBoost is an algorithm for constructing](https://reader034.fdocuments.us/reader034/viewer/2022052019/6032ce98264d8d49d92220bd/html5/thumbnails/84.jpg)
0 5 10 15 20 25 30 35 400
0.05
0.1
0.15
0.2
0.25
0.3
0.35
![Page 85: AdaBoost - cmp.felk.cvut.czcmp.felk.cvut.cz/cmp/courses/recognition/AdaBoost2/adaboost_talk.… · AdaBoost variants. 3/12 What is AdaBoost? AdaBoost is an algorithm for constructing](https://reader034.fdocuments.us/reader034/viewer/2022052019/6032ce98264d8d49d92220bd/html5/thumbnails/85.jpg)
![Page 86: AdaBoost - cmp.felk.cvut.czcmp.felk.cvut.cz/cmp/courses/recognition/AdaBoost2/adaboost_talk.… · AdaBoost variants. 3/12 What is AdaBoost? AdaBoost is an algorithm for constructing](https://reader034.fdocuments.us/reader034/viewer/2022052019/6032ce98264d8d49d92220bd/html5/thumbnails/86.jpg)
0 5 10 15 20 25 30 35 400
0.05
0.1
0.15
0.2
0.25
0.3
0.35
![Page 87: AdaBoost - cmp.felk.cvut.czcmp.felk.cvut.cz/cmp/courses/recognition/AdaBoost2/adaboost_talk.… · AdaBoost variants. 3/12 What is AdaBoost? AdaBoost is an algorithm for constructing](https://reader034.fdocuments.us/reader034/viewer/2022052019/6032ce98264d8d49d92220bd/html5/thumbnails/87.jpg)
![Page 88: AdaBoost - cmp.felk.cvut.czcmp.felk.cvut.cz/cmp/courses/recognition/AdaBoost2/adaboost_talk.… · AdaBoost variants. 3/12 What is AdaBoost? AdaBoost is an algorithm for constructing](https://reader034.fdocuments.us/reader034/viewer/2022052019/6032ce98264d8d49d92220bd/html5/thumbnails/88.jpg)
0 5 10 15 20 25 30 35 400
0.05
0.1
0.15
0.2
0.25
0.3
0.35
![Page 89: AdaBoost - cmp.felk.cvut.czcmp.felk.cvut.cz/cmp/courses/recognition/AdaBoost2/adaboost_talk.… · AdaBoost variants. 3/12 What is AdaBoost? AdaBoost is an algorithm for constructing](https://reader034.fdocuments.us/reader034/viewer/2022052019/6032ce98264d8d49d92220bd/html5/thumbnails/89.jpg)
−2 −1.5 −1 −0.5 0 0.5 1 1.5 2
0
0.5
1
1.5
2
2.5
3
yf(x)
err
![Page 90: AdaBoost - cmp.felk.cvut.czcmp.felk.cvut.cz/cmp/courses/recognition/AdaBoost2/adaboost_talk.… · AdaBoost variants. 3/12 What is AdaBoost? AdaBoost is an algorithm for constructing](https://reader034.fdocuments.us/reader034/viewer/2022052019/6032ce98264d8d49d92220bd/html5/thumbnails/90.jpg)
![Page 91: AdaBoost - cmp.felk.cvut.czcmp.felk.cvut.cz/cmp/courses/recognition/AdaBoost2/adaboost_talk.… · AdaBoost variants. 3/12 What is AdaBoost? AdaBoost is an algorithm for constructing](https://reader034.fdocuments.us/reader034/viewer/2022052019/6032ce98264d8d49d92220bd/html5/thumbnails/91.jpg)
![Page 92: AdaBoost - cmp.felk.cvut.czcmp.felk.cvut.cz/cmp/courses/recognition/AdaBoost2/adaboost_talk.… · AdaBoost variants. 3/12 What is AdaBoost? AdaBoost is an algorithm for constructing](https://reader034.fdocuments.us/reader034/viewer/2022052019/6032ce98264d8d49d92220bd/html5/thumbnails/92.jpg)
![Page 93: AdaBoost - cmp.felk.cvut.czcmp.felk.cvut.cz/cmp/courses/recognition/AdaBoost2/adaboost_talk.… · AdaBoost variants. 3/12 What is AdaBoost? AdaBoost is an algorithm for constructing](https://reader034.fdocuments.us/reader034/viewer/2022052019/6032ce98264d8d49d92220bd/html5/thumbnails/93.jpg)
−2 −1.5 −1 −0.5 0 0.5 1 1.5 2
0
0.5
1
1.5
2
2.5
3
yf(x)
err
![Page 94: AdaBoost - cmp.felk.cvut.czcmp.felk.cvut.cz/cmp/courses/recognition/AdaBoost2/adaboost_talk.… · AdaBoost variants. 3/12 What is AdaBoost? AdaBoost is an algorithm for constructing](https://reader034.fdocuments.us/reader034/viewer/2022052019/6032ce98264d8d49d92220bd/html5/thumbnails/94.jpg)
−2 −1.5 −1 −0.5 0 0.5 1 1.5 2
0
0.5
1
1.5
2
2.5
3
yf(x)
err
![Page 95: AdaBoost - cmp.felk.cvut.czcmp.felk.cvut.cz/cmp/courses/recognition/AdaBoost2/adaboost_talk.… · AdaBoost variants. 3/12 What is AdaBoost? AdaBoost is an algorithm for constructing](https://reader034.fdocuments.us/reader034/viewer/2022052019/6032ce98264d8d49d92220bd/html5/thumbnails/95.jpg)
![Page 96: AdaBoost - cmp.felk.cvut.czcmp.felk.cvut.cz/cmp/courses/recognition/AdaBoost2/adaboost_talk.… · AdaBoost variants. 3/12 What is AdaBoost? AdaBoost is an algorithm for constructing](https://reader034.fdocuments.us/reader034/viewer/2022052019/6032ce98264d8d49d92220bd/html5/thumbnails/96.jpg)
−2 −1.5 −1 −0.5 0 0.5 1 1.5 2
0
0.5
1
1.5
2
2.5
3
yf(x)
err
![Page 97: AdaBoost - cmp.felk.cvut.czcmp.felk.cvut.cz/cmp/courses/recognition/AdaBoost2/adaboost_talk.… · AdaBoost variants. 3/12 What is AdaBoost? AdaBoost is an algorithm for constructing](https://reader034.fdocuments.us/reader034/viewer/2022052019/6032ce98264d8d49d92220bd/html5/thumbnails/97.jpg)
![Page 98: AdaBoost - cmp.felk.cvut.czcmp.felk.cvut.cz/cmp/courses/recognition/AdaBoost2/adaboost_talk.… · AdaBoost variants. 3/12 What is AdaBoost? AdaBoost is an algorithm for constructing](https://reader034.fdocuments.us/reader034/viewer/2022052019/6032ce98264d8d49d92220bd/html5/thumbnails/98.jpg)
−2 −1.5 −1 −0.5 0 0.5 1 1.5 2
0
0.5
1
1.5
2
2.5
3
yf(x)
err
![Page 99: AdaBoost - cmp.felk.cvut.czcmp.felk.cvut.cz/cmp/courses/recognition/AdaBoost2/adaboost_talk.… · AdaBoost variants. 3/12 What is AdaBoost? AdaBoost is an algorithm for constructing](https://reader034.fdocuments.us/reader034/viewer/2022052019/6032ce98264d8d49d92220bd/html5/thumbnails/99.jpg)
![Page 100: AdaBoost - cmp.felk.cvut.czcmp.felk.cvut.cz/cmp/courses/recognition/AdaBoost2/adaboost_talk.… · AdaBoost variants. 3/12 What is AdaBoost? AdaBoost is an algorithm for constructing](https://reader034.fdocuments.us/reader034/viewer/2022052019/6032ce98264d8d49d92220bd/html5/thumbnails/100.jpg)
−2 −1.5 −1 −0.5 0 0.5 1 1.5 2
0
0.5
1
1.5
2
2.5
3
yf(x)
err
![Page 101: AdaBoost - cmp.felk.cvut.czcmp.felk.cvut.cz/cmp/courses/recognition/AdaBoost2/adaboost_talk.… · AdaBoost variants. 3/12 What is AdaBoost? AdaBoost is an algorithm for constructing](https://reader034.fdocuments.us/reader034/viewer/2022052019/6032ce98264d8d49d92220bd/html5/thumbnails/101.jpg)
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
εt
Z t
![Page 102: AdaBoost - cmp.felk.cvut.czcmp.felk.cvut.cz/cmp/courses/recognition/AdaBoost2/adaboost_talk.… · AdaBoost variants. 3/12 What is AdaBoost? AdaBoost is an algorithm for constructing](https://reader034.fdocuments.us/reader034/viewer/2022052019/6032ce98264d8d49d92220bd/html5/thumbnails/102.jpg)
![Page 103: AdaBoost - cmp.felk.cvut.czcmp.felk.cvut.cz/cmp/courses/recognition/AdaBoost2/adaboost_talk.… · AdaBoost variants. 3/12 What is AdaBoost? AdaBoost is an algorithm for constructing](https://reader034.fdocuments.us/reader034/viewer/2022052019/6032ce98264d8d49d92220bd/html5/thumbnails/103.jpg)
![Page 104: AdaBoost - cmp.felk.cvut.czcmp.felk.cvut.cz/cmp/courses/recognition/AdaBoost2/adaboost_talk.… · AdaBoost variants. 3/12 What is AdaBoost? AdaBoost is an algorithm for constructing](https://reader034.fdocuments.us/reader034/viewer/2022052019/6032ce98264d8d49d92220bd/html5/thumbnails/104.jpg)
![Page 105: AdaBoost - cmp.felk.cvut.czcmp.felk.cvut.cz/cmp/courses/recognition/AdaBoost2/adaboost_talk.… · AdaBoost variants. 3/12 What is AdaBoost? AdaBoost is an algorithm for constructing](https://reader034.fdocuments.us/reader034/viewer/2022052019/6032ce98264d8d49d92220bd/html5/thumbnails/105.jpg)
![Page 106: AdaBoost - cmp.felk.cvut.czcmp.felk.cvut.cz/cmp/courses/recognition/AdaBoost2/adaboost_talk.… · AdaBoost variants. 3/12 What is AdaBoost? AdaBoost is an algorithm for constructing](https://reader034.fdocuments.us/reader034/viewer/2022052019/6032ce98264d8d49d92220bd/html5/thumbnails/106.jpg)
![Page 107: AdaBoost - cmp.felk.cvut.czcmp.felk.cvut.cz/cmp/courses/recognition/AdaBoost2/adaboost_talk.… · AdaBoost variants. 3/12 What is AdaBoost? AdaBoost is an algorithm for constructing](https://reader034.fdocuments.us/reader034/viewer/2022052019/6032ce98264d8d49d92220bd/html5/thumbnails/107.jpg)
![Page 108: AdaBoost - cmp.felk.cvut.czcmp.felk.cvut.cz/cmp/courses/recognition/AdaBoost2/adaboost_talk.… · AdaBoost variants. 3/12 What is AdaBoost? AdaBoost is an algorithm for constructing](https://reader034.fdocuments.us/reader034/viewer/2022052019/6032ce98264d8d49d92220bd/html5/thumbnails/108.jpg)
![Page 109: AdaBoost - cmp.felk.cvut.czcmp.felk.cvut.cz/cmp/courses/recognition/AdaBoost2/adaboost_talk.… · AdaBoost variants. 3/12 What is AdaBoost? AdaBoost is an algorithm for constructing](https://reader034.fdocuments.us/reader034/viewer/2022052019/6032ce98264d8d49d92220bd/html5/thumbnails/109.jpg)
0 5 10 15 20 25 30 35 400
0.05
0.1
0.15
0.2
0.25
0.3
0.35
![Page 110: AdaBoost - cmp.felk.cvut.czcmp.felk.cvut.cz/cmp/courses/recognition/AdaBoost2/adaboost_talk.… · AdaBoost variants. 3/12 What is AdaBoost? AdaBoost is an algorithm for constructing](https://reader034.fdocuments.us/reader034/viewer/2022052019/6032ce98264d8d49d92220bd/html5/thumbnails/110.jpg)
![Page 111: AdaBoost - cmp.felk.cvut.czcmp.felk.cvut.cz/cmp/courses/recognition/AdaBoost2/adaboost_talk.… · AdaBoost variants. 3/12 What is AdaBoost? AdaBoost is an algorithm for constructing](https://reader034.fdocuments.us/reader034/viewer/2022052019/6032ce98264d8d49d92220bd/html5/thumbnails/111.jpg)
0 5 10 15 20 25 30 35 400
0.05
0.1
0.15
0.2
0.25
0.3
0.35
![Page 112: AdaBoost - cmp.felk.cvut.czcmp.felk.cvut.cz/cmp/courses/recognition/AdaBoost2/adaboost_talk.… · AdaBoost variants. 3/12 What is AdaBoost? AdaBoost is an algorithm for constructing](https://reader034.fdocuments.us/reader034/viewer/2022052019/6032ce98264d8d49d92220bd/html5/thumbnails/112.jpg)
![Page 113: AdaBoost - cmp.felk.cvut.czcmp.felk.cvut.cz/cmp/courses/recognition/AdaBoost2/adaboost_talk.… · AdaBoost variants. 3/12 What is AdaBoost? AdaBoost is an algorithm for constructing](https://reader034.fdocuments.us/reader034/viewer/2022052019/6032ce98264d8d49d92220bd/html5/thumbnails/113.jpg)
0 5 10 15 20 25 30 35 400
0.05
0.1
0.15
0.2
0.25
0.3
0.35
![Page 114: AdaBoost - cmp.felk.cvut.czcmp.felk.cvut.cz/cmp/courses/recognition/AdaBoost2/adaboost_talk.… · AdaBoost variants. 3/12 What is AdaBoost? AdaBoost is an algorithm for constructing](https://reader034.fdocuments.us/reader034/viewer/2022052019/6032ce98264d8d49d92220bd/html5/thumbnails/114.jpg)
![Page 115: AdaBoost - cmp.felk.cvut.czcmp.felk.cvut.cz/cmp/courses/recognition/AdaBoost2/adaboost_talk.… · AdaBoost variants. 3/12 What is AdaBoost? AdaBoost is an algorithm for constructing](https://reader034.fdocuments.us/reader034/viewer/2022052019/6032ce98264d8d49d92220bd/html5/thumbnails/115.jpg)
0 5 10 15 20 25 30 35 400
0.05
0.1
0.15
0.2
0.25
0.3
0.35
![Page 116: AdaBoost - cmp.felk.cvut.czcmp.felk.cvut.cz/cmp/courses/recognition/AdaBoost2/adaboost_talk.… · AdaBoost variants. 3/12 What is AdaBoost? AdaBoost is an algorithm for constructing](https://reader034.fdocuments.us/reader034/viewer/2022052019/6032ce98264d8d49d92220bd/html5/thumbnails/116.jpg)
![Page 117: AdaBoost - cmp.felk.cvut.czcmp.felk.cvut.cz/cmp/courses/recognition/AdaBoost2/adaboost_talk.… · AdaBoost variants. 3/12 What is AdaBoost? AdaBoost is an algorithm for constructing](https://reader034.fdocuments.us/reader034/viewer/2022052019/6032ce98264d8d49d92220bd/html5/thumbnails/117.jpg)
0 5 10 15 20 25 30 35 400
0.05
0.1
0.15
0.2
0.25
0.3
0.35
![Page 118: AdaBoost - cmp.felk.cvut.czcmp.felk.cvut.cz/cmp/courses/recognition/AdaBoost2/adaboost_talk.… · AdaBoost variants. 3/12 What is AdaBoost? AdaBoost is an algorithm for constructing](https://reader034.fdocuments.us/reader034/viewer/2022052019/6032ce98264d8d49d92220bd/html5/thumbnails/118.jpg)
![Page 119: AdaBoost - cmp.felk.cvut.czcmp.felk.cvut.cz/cmp/courses/recognition/AdaBoost2/adaboost_talk.… · AdaBoost variants. 3/12 What is AdaBoost? AdaBoost is an algorithm for constructing](https://reader034.fdocuments.us/reader034/viewer/2022052019/6032ce98264d8d49d92220bd/html5/thumbnails/119.jpg)
0 5 10 15 20 25 30 35 400
0.05
0.1
0.15
0.2
0.25
0.3
0.35
![Page 120: AdaBoost - cmp.felk.cvut.czcmp.felk.cvut.cz/cmp/courses/recognition/AdaBoost2/adaboost_talk.… · AdaBoost variants. 3/12 What is AdaBoost? AdaBoost is an algorithm for constructing](https://reader034.fdocuments.us/reader034/viewer/2022052019/6032ce98264d8d49d92220bd/html5/thumbnails/120.jpg)
![Page 121: AdaBoost - cmp.felk.cvut.czcmp.felk.cvut.cz/cmp/courses/recognition/AdaBoost2/adaboost_talk.… · AdaBoost variants. 3/12 What is AdaBoost? AdaBoost is an algorithm for constructing](https://reader034.fdocuments.us/reader034/viewer/2022052019/6032ce98264d8d49d92220bd/html5/thumbnails/121.jpg)
0 5 10 15 20 25 30 35 400
0.05
0.1
0.15
0.2
0.25
0.3
0.35
![Page 122: AdaBoost - cmp.felk.cvut.czcmp.felk.cvut.cz/cmp/courses/recognition/AdaBoost2/adaboost_talk.… · AdaBoost variants. 3/12 What is AdaBoost? AdaBoost is an algorithm for constructing](https://reader034.fdocuments.us/reader034/viewer/2022052019/6032ce98264d8d49d92220bd/html5/thumbnails/122.jpg)
![Page 123: AdaBoost - cmp.felk.cvut.czcmp.felk.cvut.cz/cmp/courses/recognition/AdaBoost2/adaboost_talk.… · AdaBoost variants. 3/12 What is AdaBoost? AdaBoost is an algorithm for constructing](https://reader034.fdocuments.us/reader034/viewer/2022052019/6032ce98264d8d49d92220bd/html5/thumbnails/123.jpg)
0 5 10 15 20 25 30 35 400
0.05
0.1
0.15
0.2
0.25
0.3
0.35
![Page 124: AdaBoost - cmp.felk.cvut.czcmp.felk.cvut.cz/cmp/courses/recognition/AdaBoost2/adaboost_talk.… · AdaBoost variants. 3/12 What is AdaBoost? AdaBoost is an algorithm for constructing](https://reader034.fdocuments.us/reader034/viewer/2022052019/6032ce98264d8d49d92220bd/html5/thumbnails/124.jpg)
![Page 125: AdaBoost - cmp.felk.cvut.czcmp.felk.cvut.cz/cmp/courses/recognition/AdaBoost2/adaboost_talk.… · AdaBoost variants. 3/12 What is AdaBoost? AdaBoost is an algorithm for constructing](https://reader034.fdocuments.us/reader034/viewer/2022052019/6032ce98264d8d49d92220bd/html5/thumbnails/125.jpg)
![Page 126: AdaBoost - cmp.felk.cvut.czcmp.felk.cvut.cz/cmp/courses/recognition/AdaBoost2/adaboost_talk.… · AdaBoost variants. 3/12 What is AdaBoost? AdaBoost is an algorithm for constructing](https://reader034.fdocuments.us/reader034/viewer/2022052019/6032ce98264d8d49d92220bd/html5/thumbnails/126.jpg)
![Page 127: AdaBoost - cmp.felk.cvut.czcmp.felk.cvut.cz/cmp/courses/recognition/AdaBoost2/adaboost_talk.… · AdaBoost variants. 3/12 What is AdaBoost? AdaBoost is an algorithm for constructing](https://reader034.fdocuments.us/reader034/viewer/2022052019/6032ce98264d8d49d92220bd/html5/thumbnails/127.jpg)
![Page 128: AdaBoost - cmp.felk.cvut.czcmp.felk.cvut.cz/cmp/courses/recognition/AdaBoost2/adaboost_talk.… · AdaBoost variants. 3/12 What is AdaBoost? AdaBoost is an algorithm for constructing](https://reader034.fdocuments.us/reader034/viewer/2022052019/6032ce98264d8d49d92220bd/html5/thumbnails/128.jpg)
![Page 129: AdaBoost - cmp.felk.cvut.czcmp.felk.cvut.cz/cmp/courses/recognition/AdaBoost2/adaboost_talk.… · AdaBoost variants. 3/12 What is AdaBoost? AdaBoost is an algorithm for constructing](https://reader034.fdocuments.us/reader034/viewer/2022052019/6032ce98264d8d49d92220bd/html5/thumbnails/129.jpg)
![Page 130: AdaBoost - cmp.felk.cvut.czcmp.felk.cvut.cz/cmp/courses/recognition/AdaBoost2/adaboost_talk.… · AdaBoost variants. 3/12 What is AdaBoost? AdaBoost is an algorithm for constructing](https://reader034.fdocuments.us/reader034/viewer/2022052019/6032ce98264d8d49d92220bd/html5/thumbnails/130.jpg)
![Page 131: AdaBoost - cmp.felk.cvut.czcmp.felk.cvut.cz/cmp/courses/recognition/AdaBoost2/adaboost_talk.… · AdaBoost variants. 3/12 What is AdaBoost? AdaBoost is an algorithm for constructing](https://reader034.fdocuments.us/reader034/viewer/2022052019/6032ce98264d8d49d92220bd/html5/thumbnails/131.jpg)
0 5 10 15 20 25 30 35 400
0.05
0.1
0.15
0.2
0.25
0.3
0.35
![Page 132: AdaBoost - cmp.felk.cvut.czcmp.felk.cvut.cz/cmp/courses/recognition/AdaBoost2/adaboost_talk.… · AdaBoost variants. 3/12 What is AdaBoost? AdaBoost is an algorithm for constructing](https://reader034.fdocuments.us/reader034/viewer/2022052019/6032ce98264d8d49d92220bd/html5/thumbnails/132.jpg)
![Page 133: AdaBoost - cmp.felk.cvut.czcmp.felk.cvut.cz/cmp/courses/recognition/AdaBoost2/adaboost_talk.… · AdaBoost variants. 3/12 What is AdaBoost? AdaBoost is an algorithm for constructing](https://reader034.fdocuments.us/reader034/viewer/2022052019/6032ce98264d8d49d92220bd/html5/thumbnails/133.jpg)
0 5 10 15 20 25 30 35 400
0.05
0.1
0.15
0.2
0.25
0.3
0.35
![Page 134: AdaBoost - cmp.felk.cvut.czcmp.felk.cvut.cz/cmp/courses/recognition/AdaBoost2/adaboost_talk.… · AdaBoost variants. 3/12 What is AdaBoost? AdaBoost is an algorithm for constructing](https://reader034.fdocuments.us/reader034/viewer/2022052019/6032ce98264d8d49d92220bd/html5/thumbnails/134.jpg)
![Page 135: AdaBoost - cmp.felk.cvut.czcmp.felk.cvut.cz/cmp/courses/recognition/AdaBoost2/adaboost_talk.… · AdaBoost variants. 3/12 What is AdaBoost? AdaBoost is an algorithm for constructing](https://reader034.fdocuments.us/reader034/viewer/2022052019/6032ce98264d8d49d92220bd/html5/thumbnails/135.jpg)
0 5 10 15 20 25 30 35 400
0.05
0.1
0.15
0.2
0.25
0.3
0.35
![Page 136: AdaBoost - cmp.felk.cvut.czcmp.felk.cvut.cz/cmp/courses/recognition/AdaBoost2/adaboost_talk.… · AdaBoost variants. 3/12 What is AdaBoost? AdaBoost is an algorithm for constructing](https://reader034.fdocuments.us/reader034/viewer/2022052019/6032ce98264d8d49d92220bd/html5/thumbnails/136.jpg)
![Page 137: AdaBoost - cmp.felk.cvut.czcmp.felk.cvut.cz/cmp/courses/recognition/AdaBoost2/adaboost_talk.… · AdaBoost variants. 3/12 What is AdaBoost? AdaBoost is an algorithm for constructing](https://reader034.fdocuments.us/reader034/viewer/2022052019/6032ce98264d8d49d92220bd/html5/thumbnails/137.jpg)
0 5 10 15 20 25 30 35 400
0.05
0.1
0.15
0.2
0.25
0.3
0.35
![Page 138: AdaBoost - cmp.felk.cvut.czcmp.felk.cvut.cz/cmp/courses/recognition/AdaBoost2/adaboost_talk.… · AdaBoost variants. 3/12 What is AdaBoost? AdaBoost is an algorithm for constructing](https://reader034.fdocuments.us/reader034/viewer/2022052019/6032ce98264d8d49d92220bd/html5/thumbnails/138.jpg)
![Page 139: AdaBoost - cmp.felk.cvut.czcmp.felk.cvut.cz/cmp/courses/recognition/AdaBoost2/adaboost_talk.… · AdaBoost variants. 3/12 What is AdaBoost? AdaBoost is an algorithm for constructing](https://reader034.fdocuments.us/reader034/viewer/2022052019/6032ce98264d8d49d92220bd/html5/thumbnails/139.jpg)
0 5 10 15 20 25 30 35 400
0.05
0.1
0.15
0.2
0.25
0.3
0.35
![Page 140: AdaBoost - cmp.felk.cvut.czcmp.felk.cvut.cz/cmp/courses/recognition/AdaBoost2/adaboost_talk.… · AdaBoost variants. 3/12 What is AdaBoost? AdaBoost is an algorithm for constructing](https://reader034.fdocuments.us/reader034/viewer/2022052019/6032ce98264d8d49d92220bd/html5/thumbnails/140.jpg)
![Page 141: AdaBoost - cmp.felk.cvut.czcmp.felk.cvut.cz/cmp/courses/recognition/AdaBoost2/adaboost_talk.… · AdaBoost variants. 3/12 What is AdaBoost? AdaBoost is an algorithm for constructing](https://reader034.fdocuments.us/reader034/viewer/2022052019/6032ce98264d8d49d92220bd/html5/thumbnails/141.jpg)
0 5 10 15 20 25 30 35 400
0.05
0.1
0.15
0.2
0.25
0.3
0.35
![Page 142: AdaBoost - cmp.felk.cvut.czcmp.felk.cvut.cz/cmp/courses/recognition/AdaBoost2/adaboost_talk.… · AdaBoost variants. 3/12 What is AdaBoost? AdaBoost is an algorithm for constructing](https://reader034.fdocuments.us/reader034/viewer/2022052019/6032ce98264d8d49d92220bd/html5/thumbnails/142.jpg)
![Page 143: AdaBoost - cmp.felk.cvut.czcmp.felk.cvut.cz/cmp/courses/recognition/AdaBoost2/adaboost_talk.… · AdaBoost variants. 3/12 What is AdaBoost? AdaBoost is an algorithm for constructing](https://reader034.fdocuments.us/reader034/viewer/2022052019/6032ce98264d8d49d92220bd/html5/thumbnails/143.jpg)
0 5 10 15 20 25 30 35 400
0.05
0.1
0.15
0.2
0.25
0.3
0.35
![Page 144: AdaBoost - cmp.felk.cvut.czcmp.felk.cvut.cz/cmp/courses/recognition/AdaBoost2/adaboost_talk.… · AdaBoost variants. 3/12 What is AdaBoost? AdaBoost is an algorithm for constructing](https://reader034.fdocuments.us/reader034/viewer/2022052019/6032ce98264d8d49d92220bd/html5/thumbnails/144.jpg)
![Page 145: AdaBoost - cmp.felk.cvut.czcmp.felk.cvut.cz/cmp/courses/recognition/AdaBoost2/adaboost_talk.… · AdaBoost variants. 3/12 What is AdaBoost? AdaBoost is an algorithm for constructing](https://reader034.fdocuments.us/reader034/viewer/2022052019/6032ce98264d8d49d92220bd/html5/thumbnails/145.jpg)
0 5 10 15 20 25 30 35 400
0.05
0.1
0.15
0.2
0.25
0.3
0.35