Classification with AdaBoost
Transcript of Classification with AdaBoost
![Page 1: Classification with AdaBoost](https://reader030.fdocuments.us/reader030/viewer/2022021419/58a1a9961a28ab244d8c460a/html5/thumbnails/1.jpg)
v.1.1, Kai Arras, Jan 12, including material by Luciano Spinello and Oscar Martinez Mozos
Robotics 2 AdaBoost for People and Place Detection
Kai Arras, Cyrill Stachniss, Maren Bennewitz, Wolfram Burgard
![Page 2: Classification with AdaBoost](https://reader030.fdocuments.us/reader030/viewer/2022021419/58a1a9961a28ab244d8c460a/html5/thumbnails/2.jpg)
Chapter Contents
Machine Learning: A Survey
Classification
AdaBoost
People Detection with Boosted Features
Place Recognition with Boosted Features
![Page 3: Classification with AdaBoost](https://reader030.fdocuments.us/reader030/viewer/2022021419/58a1a9961a28ab244d8c460a/html5/thumbnails/3.jpg)
Machine Learning: Survey What is Machine Learning?
Learning a model from data Fundamentally different than model-based
approaches where the model is derived from domain knowledge, e.g. physics, social science
Often it is too complex, too costly, or impossible to model a process in “closed form” (e.g. financial market, consumer behavior in on-line store)
Thus, we can collect data and hope to extract the process or pattern that explains the observed data
Even if we are unable to describe the complete process, an approximate model may be enough
![Page 4: Classification with AdaBoost](https://reader030.fdocuments.us/reader030/viewer/2022021419/58a1a9961a28ab244d8c460a/html5/thumbnails/4.jpg)
Machine Learning: Survey Machine Learning Taxonomy:
Supervised Learning: Inferring a function from labelled training data Examples: Classification, Regression
Unsupervised Learning: Try to find hidden structures in unlabeled data Examples: Clustering, Outlier Detection
Semi-supervised Learning: Learn a function from both, labelled and unlabelled data
Reinforcement Learning: Learn how to act guided by feedback (rewards) from the world
![Page 5: Classification with AdaBoost](https://reader030.fdocuments.us/reader030/viewer/2022021419/58a1a9961a28ab244d8c460a/html5/thumbnails/5.jpg)
Machine Learning: Survey Machine Learning Examples:
Classification Support Vector Machines (SVM), naive Bayes, LDA,
Decision trees, k-nearest neighbor, ANNs, AdaBoost
Regression Gaussian Processes, Least Squares Estimation, Gauss-
Newton
Clustering GMMs, Hierarchical clustering, k-means
Reinforcement Learning Q-Learning
![Page 6: Classification with AdaBoost](https://reader030.fdocuments.us/reader030/viewer/2022021419/58a1a9961a28ab244d8c460a/html5/thumbnails/6.jpg)
Machine Learning: Survey Machine Learning in Robotics Examples:
Perception: people/object/speech recognition from sensory data, learning of dynamic objects
Modeling: human behavior modeling and analysis Planning: on learned cost maps, e.g. for human-
aware coverage Action (learning motions by imitating people, e.g.
ping-pong playing)
Machine Learning has become a very popular tool for many robotics tasks Can make systems adaptive to changing environments
![Page 7: Classification with AdaBoost](https://reader030.fdocuments.us/reader030/viewer/2022021419/58a1a9961a28ab244d8c460a/html5/thumbnails/7.jpg)
Chapter Contents
Machine Learning: A Survey
Classification
AdaBoost
People Detection with Boosted Features
Place Recognition with Boosted Features
![Page 8: Classification with AdaBoost](https://reader030.fdocuments.us/reader030/viewer/2022021419/58a1a9961a28ab244d8c460a/html5/thumbnails/8.jpg)
Classification Classification algorithms are supervised algorithms
to predict categorical labels
Differs from regression which is a supervised technique to predict real-valued labels
Formal problem statement:
Produce a function that maps
Given a training set
C
label
training sample
![Page 9: Classification with AdaBoost](https://reader030.fdocuments.us/reader030/viewer/2022021419/58a1a9961a28ab244d8c460a/html5/thumbnails/9.jpg)
Classification Error types
Precision = TP / (TP + FP)
Recall = TP / (TP + FN)
Many more measures...
True Positive False Positive
False Negative True Negative
True value
Pred
icte
d va
lue
of
the
cla
ssifi
er
T'
N'
T N
Detected Not Detected
![Page 10: Classification with AdaBoost](https://reader030.fdocuments.us/reader030/viewer/2022021419/58a1a9961a28ab244d8c460a/html5/thumbnails/10.jpg)
Classification Linear vs. Non-Linear Classifier, Margin
NL L
![Page 11: Classification with AdaBoost](https://reader030.fdocuments.us/reader030/viewer/2022021419/58a1a9961a28ab244d8c460a/html5/thumbnails/11.jpg)
Overfitting
Overfitting occurs when a model begins to memorize the training data rather than learning the underlying relationship
Occurs typically when fitting a statistical model with too many parameters
Overfitted models explain training data perfectly but they do not generalize!
There are techniques to avoid overfitting such as regularization or cross- validation
Classification
![Page 12: Classification with AdaBoost](https://reader030.fdocuments.us/reader030/viewer/2022021419/58a1a9961a28ab244d8c460a/html5/thumbnails/12.jpg)
Chapter Contents
Machine Learning: A Survey
Classification
AdaBoost
People Detection with Boosted Features
Place Recognition with Boosted Features
![Page 13: Classification with AdaBoost](https://reader030.fdocuments.us/reader030/viewer/2022021419/58a1a9961a28ab244d8c460a/html5/thumbnails/13.jpg)
Boosting An ensemble technique (a.k.a. committee method) Supervised learning: given <samples x, labels y>
Learns an accurate strong classifier by combining an ensemble of inaccurate “rules of thumb”
Inaccurate rule h(xi): “weak” classifier, weak learner, basis classifier, feature
Accurate rule H(xi): “strong” classifier, final classifier
Other ensemble techniques exist: Bagging, Voting,
Mixture of Experts, etc.
![Page 14: Classification with AdaBoost](https://reader030.fdocuments.us/reader030/viewer/2022021419/58a1a9961a28ab244d8c460a/html5/thumbnails/14.jpg)
AdaBoost Most popular algorithm: AdaBoost
[Freund et al. 95], [Schapire et al. 99]
Given an ensemble of weak classifiers , the combined strong classifier is obtained by a weighted majority voting scheme
AdaBoost in Robotics: [Viola et al. 01], [Treptow et al. 04], [Martínez-Mozos et al. 05], [Rottmann et al. 05] , [Monteiro et al. 06] , [Arras et al. 07]
!
h(xi)
!
H(xi)
f (xi ) = !tht (xi )t=1
T
! H (xi ) = sgn f (xi )( )
![Page 15: Classification with AdaBoost](https://reader030.fdocuments.us/reader030/viewer/2022021419/58a1a9961a28ab244d8c460a/html5/thumbnails/15.jpg)
AdaBoost Why is AdaBoost interesting?
1. It tells you what the best "features" are
2. What the best thresholds are, and
3. How to combine them to a classifier
AdaBoost can be seen as a principled feature selection strategy
Classifier design becomes science, not art
![Page 16: Classification with AdaBoost](https://reader030.fdocuments.us/reader030/viewer/2022021419/58a1a9961a28ab244d8c460a/html5/thumbnails/16.jpg)
AdaBoost AdaBoost is a non-linear classifier
Has good generalization properties: can be proven to maximize the margin
Quite robust to overfitting
Very simple to implement
Prerequisite:
weak classifier must be better than chance: error < 0.5 in a binary classification problem
![Page 17: Classification with AdaBoost](https://reader030.fdocuments.us/reader030/viewer/2022021419/58a1a9961a28ab244d8c460a/html5/thumbnails/17.jpg)
AdaBoost Possible Weak Classifiers:
Decision stump: Single axis-parallel partition of space
Decision tree: Hierarchical partition of space
Multi-layer perceptron: General non-linear function approximators
Support Vector Machines (SVM): Linear classifier with RBF Kernel
Trade-off between diversity among weak learners versus their accuracy. Can be complex, see literature
Decision stumps are a popular choice
![Page 18: Classification with AdaBoost](https://reader030.fdocuments.us/reader030/viewer/2022021419/58a1a9961a28ab244d8c460a/html5/thumbnails/18.jpg)
1x
2x
θ
AdaBoost: Weak Classifier Decision stump
Simple-most type of decision tree
Equivalent to linear classifier defined by affine hyperplane
Hyperplane is orthogonal to axis with which it intersects in threshold θ
Commonly not used on its own
Formally,
where x is (d-dim.) training sample, j is dimension
!
h(x; j,") =+1 x j > "
#1 else
$ % &
![Page 19: Classification with AdaBoost](https://reader030.fdocuments.us/reader030/viewer/2022021419/58a1a9961a28ab244d8c460a/html5/thumbnails/19.jpg)
AdaBoost: Weak Classifier Train a decision stump on weighted data
This consists in...
Finding an optimum parameter θ* for each dimension j =1…d and then select the j* for which the weighted error is minimal.
( j*, ! *) = argminj, ! wt (i) !(yi " ht (xi ))
i =1
n
#$%&
'&
()&
*&
1x*θ
2x
![Page 20: Classification with AdaBoost](https://reader030.fdocuments.us/reader030/viewer/2022021419/58a1a9961a28ab244d8c460a/html5/thumbnails/20.jpg)
A simple training algorithm for stumps:
∀ j = 1...d
Sort samples xi in ascending order along dimension j
∀ i = 1...n
Compute n cumulative sums
end
Threshold θj is at extremum of
Sign of extremum gives direction pj of inequality
end
Global extremum in all d sums gives threshold θ* and dimension j*
AdaBoost: Weak Classifier
!
wcum
j(i) = wk yk
k=1
i
"
!
wcum
j
!
wcum
![Page 21: Classification with AdaBoost](https://reader030.fdocuments.us/reader030/viewer/2022021419/58a1a9961a28ab244d8c460a/html5/thumbnails/21.jpg)
AdaBoost: Weak Classifier Training algorithm for stumps: Intuition
Label y : red: + blue: –
Assuming all
weights = 1
1x
!
"*, j* = 1
2x
!
wcum
j(i) = wk yk
k=1
i
"
![Page 22: Classification with AdaBoost](https://reader030.fdocuments.us/reader030/viewer/2022021419/58a1a9961a28ab244d8c460a/html5/thumbnails/22.jpg)
AdaBoost: Algorithm Given the training data
1. Initialize weights
2. For t = 1,...,T Train a weak classifier on weighted training data
minimizing the error
Compute voting weight of :
Recompute weights:
3. Make predictions using the final strong classifier
!
wt(i) =1 n
!
ht(x)
!
ht(x) !
t= 12log((1!!
t) !
t)
wt+1(i) = wt (i) exp !!t yi ht (xi ){ } Zt
!t = wt (i) !(yi " ht (xi ))i =1
n
#
![Page 23: Classification with AdaBoost](https://reader030.fdocuments.us/reader030/viewer/2022021419/58a1a9961a28ab244d8c460a/html5/thumbnails/23.jpg)
AdaBoost: Voting Weight Computing the voting weight of a weak classifier measures the importance assigned to
!t
! !"# !"$ !"% !"& !"' !"(!!"'
!
!"'
#
#"'
$
$"'
error
!t= 12log((1!"
t) "
t)
!t
chance: e = 0.5
!t h
t(x
i)
![Page 24: Classification with AdaBoost](https://reader030.fdocuments.us/reader030/viewer/2022021419/58a1a9961a28ab244d8c460a/html5/thumbnails/24.jpg)
AdaBoost: Weight Update Looking at the weight update step:
➔ Weights of misclassified training samples are increased ➔ Weights of correctly classified samples are decreased
Algorithm generates weak classifier by training the next learner on the mistakes of the previous one
Now we understand the name: AdaBoost comes from adaptive Boosting
exp !!t yi ht (xi ){ }=<1, yi = ht (xi )
>1, yi " ht (xi )
#$%
&% !
Zt:
Normalizer such that wt+1 is a prob. distribution
wt+1(i) = wt (i) exp !!t yi ht (xi ){ } Zt
![Page 25: Classification with AdaBoost](https://reader030.fdocuments.us/reader030/viewer/2022021419/58a1a9961a28ab244d8c460a/html5/thumbnails/25.jpg)
AdaBoost: Strong Classifier Training is completed...
The weak classifiers and their voting weight are now fix
The resulting strong classifier is
Weighted majority voting scheme
Put your data here
Class Result {+1, -1}
h1...T(x)
!1....T
!
H(xi) = sgn "
tht(x
i)
t=1
T
#$
% & &
'
( ) )
![Page 26: Classification with AdaBoost](https://reader030.fdocuments.us/reader030/viewer/2022021419/58a1a9961a28ab244d8c460a/html5/thumbnails/26.jpg)
AdaBoost: Algorithm Given the training data
1. Initialize weights
2. For t = 1,...,T Train a weak classifier on weighted training data
minimizing the error
Compute voting weight of :
Recompute weights:
3. Make predictions using the final strong classifier
!
wt(i) =1 n
!
ht(x)
!
ht(x) !
t= 12log((1!!
t) !
t)
wt+1(i) = wt (i) exp !!t yi ht (xi ){ } Zt
!t = wt (i) !(yi " ht (xi ))i =1
n
#
![Page 27: Classification with AdaBoost](https://reader030.fdocuments.us/reader030/viewer/2022021419/58a1a9961a28ab244d8c460a/html5/thumbnails/27.jpg)
Training data
AdaBoost: Step-By-Step
![Page 28: Classification with AdaBoost](https://reader030.fdocuments.us/reader030/viewer/2022021419/58a1a9961a28ab244d8c460a/html5/thumbnails/28.jpg)
Iteration 1, train weak classifier 1
AdaBoost: Step-By-Step
Threshold θ* = 0.37
Dimension j* = 1
Weighted error et = 0.2
Voting weight αt = 1.39
Total error = 4
![Page 29: Classification with AdaBoost](https://reader030.fdocuments.us/reader030/viewer/2022021419/58a1a9961a28ab244d8c460a/html5/thumbnails/29.jpg)
Iteration 1, recompute weights
AdaBoost: Step-By-Step
Threshold θ* = 0.37
Dimension j* = 1
Weighted error et = 0.2
Voting weight αt = 1.39
Total error = 4
![Page 30: Classification with AdaBoost](https://reader030.fdocuments.us/reader030/viewer/2022021419/58a1a9961a28ab244d8c460a/html5/thumbnails/30.jpg)
Iteration 2, train weak classifier 2
AdaBoost: Step-By-Step
Threshold θ* = 0.47
Dimension j* = 2
Weighted error et = 0.16
Voting weight αt = 1.69
Total error = 5
![Page 31: Classification with AdaBoost](https://reader030.fdocuments.us/reader030/viewer/2022021419/58a1a9961a28ab244d8c460a/html5/thumbnails/31.jpg)
Iteration 2, recompute weights
AdaBoost: Step-By-Step
Threshold θ* = 0.47
Dimension j* = 2
Weighted error et = 0.16
Voting weight αt = 1.69
Total error = 5
![Page 32: Classification with AdaBoost](https://reader030.fdocuments.us/reader030/viewer/2022021419/58a1a9961a28ab244d8c460a/html5/thumbnails/32.jpg)
Iteration 3, train weak classifier 3
AdaBoost: Step-By-Step
Threshold θ* = 0.14
Dimension, sign j* = 2 , neg
Weighted error et = 0.25
Voting weight αt = 1.11
Total error = 1
![Page 33: Classification with AdaBoost](https://reader030.fdocuments.us/reader030/viewer/2022021419/58a1a9961a28ab244d8c460a/html5/thumbnails/33.jpg)
Iteration 3, recompute weights
AdaBoost: Step-By-Step
Threshold θ* = 0.14
Dimension, sign j* = 2 , neg
Weighted error et = 0.25
Voting weight αt = 1.11
Total error = 1
![Page 34: Classification with AdaBoost](https://reader030.fdocuments.us/reader030/viewer/2022021419/58a1a9961a28ab244d8c460a/html5/thumbnails/34.jpg)
Iteration 4, train weak classifier 4
AdaBoost: Step-By-Step
Threshold θ* = 0.37
Dimension j* = 1
Weighted error et = 0.20
Voting weight αt = 1.40
Total error = 1
![Page 35: Classification with AdaBoost](https://reader030.fdocuments.us/reader030/viewer/2022021419/58a1a9961a28ab244d8c460a/html5/thumbnails/35.jpg)
Iteration 4, recompute weights
AdaBoost: Step-By-Step
Threshold θ* = 0.37
Dimension j* = 1
Weighted error et = 0.20
Voting weight αt = 1.40
Total error = 1
![Page 36: Classification with AdaBoost](https://reader030.fdocuments.us/reader030/viewer/2022021419/58a1a9961a28ab244d8c460a/html5/thumbnails/36.jpg)
Iteration 5, train weak classifier 5
AdaBoost: Step-By-Step
Threshold θ* = 0.81
Dimension j* = 1
Weighted error et = 0.28
Voting weight αt = 0.96
Total error = 1
![Page 37: Classification with AdaBoost](https://reader030.fdocuments.us/reader030/viewer/2022021419/58a1a9961a28ab244d8c460a/html5/thumbnails/37.jpg)
Iteration 5, recompute weights
AdaBoost: Step-By-Step
Threshold θ* = 0.81
Dimension j* = 1
Weighted error et = 0.28
Voting weight αt = 0.96
Total error = 1
![Page 38: Classification with AdaBoost](https://reader030.fdocuments.us/reader030/viewer/2022021419/58a1a9961a28ab244d8c460a/html5/thumbnails/38.jpg)
Iteration 6, train weak classifier 6
AdaBoost: Step-By-Step
Threshold θ* = 0.47
Dimension j* = 2
Weighted error et = 0.29
Voting weight αt = 0.88
Total error = 1
![Page 39: Classification with AdaBoost](https://reader030.fdocuments.us/reader030/viewer/2022021419/58a1a9961a28ab244d8c460a/html5/thumbnails/39.jpg)
Iteration 6, recompute weights
AdaBoost: Step-By-Step
Threshold θ* = 0.47
Dimension j* = 2
Weighted error et = 0.29
Voting weight αt = 0.88
Total error = 1
![Page 40: Classification with AdaBoost](https://reader030.fdocuments.us/reader030/viewer/2022021419/58a1a9961a28ab244d8c460a/html5/thumbnails/40.jpg)
Iteration 7, train weak classifier 7
AdaBoost: Step-By-Step
Threshold θ* = 0.14
Dimension, sign j* = 2 , neg
Weighted error et = 0.29
Voting weight αt = 0.88
Total error = 1
![Page 41: Classification with AdaBoost](https://reader030.fdocuments.us/reader030/viewer/2022021419/58a1a9961a28ab244d8c460a/html5/thumbnails/41.jpg)
Iteration 7, recompute weights
AdaBoost: Step-By-Step
Threshold θ* = 0.14
Dimension, sign j* = 2 , neg
Weighted error et = 0.29
Voting weight αt = 0.88
Total error = 1
![Page 42: Classification with AdaBoost](https://reader030.fdocuments.us/reader030/viewer/2022021419/58a1a9961a28ab244d8c460a/html5/thumbnails/42.jpg)
Iteration 8, train weak classifier 8
AdaBoost: Step-By-Step
Threshold θ* = 0.93
Dimension, sign j* = 1 , neg
Weighted error et = 0.25
Voting weight αt = 1.12
Total error = 0
![Page 43: Classification with AdaBoost](https://reader030.fdocuments.us/reader030/viewer/2022021419/58a1a9961a28ab244d8c460a/html5/thumbnails/43.jpg)
Iteration 8, recompute weights
AdaBoost: Step-By-Step
Threshold θ* = 0.93
Dimension, sign j* = 1 , neg
Weighted error et = 0.25
Voting weight αt = 1.12
Total error = 0
![Page 44: Classification with AdaBoost](https://reader030.fdocuments.us/reader030/viewer/2022021419/58a1a9961a28ab244d8c460a/html5/thumbnails/44.jpg)
Final Strong Classifier
AdaBoost: Step-By-Step
Total training error = 0 (Rare in practice)
![Page 45: Classification with AdaBoost](https://reader030.fdocuments.us/reader030/viewer/2022021419/58a1a9961a28ab244d8c460a/html5/thumbnails/45.jpg)
AdaBoost: Why Does it Work? AdaBoost minimizes the training error Upper bound theorem: the following upper bound holds on
the training error of H
Proof: By unravelling the weight update rule
!
1
ni :H(xi) " yi{ } # Zt
t=1
T
$
Sochman, Matas
![Page 46: Classification with AdaBoost](https://reader030.fdocuments.us/reader030/viewer/2022021419/58a1a9961a28ab244d8c460a/html5/thumbnails/46.jpg)
AdaBoost: Why Does it Work? Ergo... Instead of minimizing the training error directly, its
upper bound can be minimized We have to minimize the normalizer
in each training round.
This is achieved by
Finding the optimal voting weight Finding the optimal weak classifier
Zt= w
t(i)
i
! exp "!t yi ht (xi ){ }
!
"t
!
ht(x)
![Page 47: Classification with AdaBoost](https://reader030.fdocuments.us/reader030/viewer/2022021419/58a1a9961a28ab244d8c460a/html5/thumbnails/47.jpg)
AdaBoost: Why Does it Work? Optimal voting weight
Theorem: The minimizer of the bound is
Proof:
Optimal weak classifier
Theorem: Zt is minimized by selecting with minimal weighted error
Proof:
Sochman, Matas
!
ht(x)
!
"t
!
"t
= 12log((1#$
t) $
t)
![Page 48: Classification with AdaBoost](https://reader030.fdocuments.us/reader030/viewer/2022021419/58a1a9961a28ab244d8c460a/html5/thumbnails/48.jpg)
AdaBoost in Action
![Page 49: Classification with AdaBoost](https://reader030.fdocuments.us/reader030/viewer/2022021419/58a1a9961a28ab244d8c460a/html5/thumbnails/49.jpg)
AdaBoost: Summary Misclassified samples receive higher weight.
The higher the weight the "more attention" a training sample receives
Algorithm generates weak classifier by training the next learner on the mistakes of the previous one
AdaBoost minimizes the upper bound of the training error by properly choosing the optimal weak classifier and voting weight. AdaBoost can further be shown to maximize the margin (proof in literature)
Large impact on ML community and beyond
![Page 50: Classification with AdaBoost](https://reader030.fdocuments.us/reader030/viewer/2022021419/58a1a9961a28ab244d8c460a/html5/thumbnails/50.jpg)
Chapter Contents
Machine Learning: A Survey
Classification
AdaBoost
People Detection with Boosted Features
Place Recognition with Boosted Features
![Page 51: Classification with AdaBoost](https://reader030.fdocuments.us/reader030/viewer/2022021419/58a1a9961a28ab244d8c460a/html5/thumbnails/51.jpg)
Motivation: People Detection People detection and tracking is a key
component for many vision systems and for all robots in human environments:
Human-Robot-Interaction (HRI)
Social Robotics: social learning, learning by imitation and observation
Motion planning in populated environments
Human activity and intent recognition
Abnormal behavior detection
Crowd behavior analysis and control
![Page 52: Classification with AdaBoost](https://reader030.fdocuments.us/reader030/viewer/2022021419/58a1a9961a28ab244d8c460a/html5/thumbnails/52.jpg)
Motivation: People Detection Where are the people?
![Page 53: Classification with AdaBoost](https://reader030.fdocuments.us/reader030/viewer/2022021419/58a1a9961a28ab244d8c460a/html5/thumbnails/53.jpg)
Motivation: People Detection Where are the people?
Why is it hard?
Range data contain little information on people
Hard in cluttered environments
![Page 54: Classification with AdaBoost](https://reader030.fdocuments.us/reader030/viewer/2022021419/58a1a9961a28ab244d8c460a/html5/thumbnails/54.jpg)
Motivation: People Detection Appearance of humans in range data changes
drastically with: - Body pose - Distance to sensor - Occlusion and self-occlusion
2D range data from a SICK laser scanner
![Page 55: Classification with AdaBoost](https://reader030.fdocuments.us/reader030/viewer/2022021419/58a1a9961a28ab244d8c460a/html5/thumbnails/55.jpg)
Motivation: People Detection Appearance of humans in 3D range data
(Velodyne scanner)
![Page 56: Classification with AdaBoost](https://reader030.fdocuments.us/reader030/viewer/2022021419/58a1a9961a28ab244d8c460a/html5/thumbnails/56.jpg)
Motivation: People Detection
![Page 57: Classification with AdaBoost](https://reader030.fdocuments.us/reader030/viewer/2022021419/58a1a9961a28ab244d8c460a/html5/thumbnails/57.jpg)
Motivation: People Detection Freiburg Main Station data set: raw data
![Page 58: Classification with AdaBoost](https://reader030.fdocuments.us/reader030/viewer/2022021419/58a1a9961a28ab244d8c460a/html5/thumbnails/58.jpg)
Motivation: People Detection Freiburg Main Station data set: annotations
![Page 59: Classification with AdaBoost](https://reader030.fdocuments.us/reader030/viewer/2022021419/58a1a9961a28ab244d8c460a/html5/thumbnails/59.jpg)
Approach Can we find robust features for people, legs and
groups of people in 2D range data? What are the best features for people detection? Can we find people that do not move?
Approach:
Classifying groups of adjacent beams (segments) Computing a set of scalar features on these groups Boosting the features
![Page 60: Classification with AdaBoost](https://reader030.fdocuments.us/reader030/viewer/2022021419/58a1a9961a28ab244d8c460a/html5/thumbnails/60.jpg)
Related Work People Tracking
[Fod et al. 2002] [Kleinhagenbrock et al. 2002] [Schulz et al. 2003] [Scheutz et al. 2004] [Topp et al. 2005] [Cui et al. 2005] [Schulz 2006] [Mucientes et al. 2006]
SLAM in dynamic env. [Montemerlo et al. 2002] [Hähnel et al. 2003] [Wang et al. 2003] ...
People detection done with very simple classifiers: manual feature selection, heuristic thresholds
Typically: narrow local-minima blobs that move
![Page 61: Classification with AdaBoost](https://reader030.fdocuments.us/reader030/viewer/2022021419/58a1a9961a28ab244d8c460a/html5/thumbnails/61.jpg)
Segmentation Divide the scan into segments
Range image segmentation
![Page 62: Classification with AdaBoost](https://reader030.fdocuments.us/reader030/viewer/2022021419/58a1a9961a28ab244d8c460a/html5/thumbnails/62.jpg)
Segmentation
Feature profiles
Segmented scan Raw scan
Method: Jump distance condition
Size filter: rejection of too small segments
![Page 63: Classification with AdaBoost](https://reader030.fdocuments.us/reader030/viewer/2022021419/58a1a9961a28ab244d8c460a/html5/thumbnails/63.jpg)
Segmentation
Feature profiles
Segmented scan Raw scan
Method: Jump distance condition
Size filter: rejection of too small segments
![Page 64: Classification with AdaBoost](https://reader030.fdocuments.us/reader030/viewer/2022021419/58a1a9961a28ab244d8c460a/html5/thumbnails/64.jpg)
Segmentation
Feature profiles
Segmented scan Raw scan
Method: Jump distance condition
Size filter: rejection of too small segments
![Page 65: Classification with AdaBoost](https://reader030.fdocuments.us/reader030/viewer/2022021419/58a1a9961a28ab244d8c460a/html5/thumbnails/65.jpg)
Features Segment
1. Number of points
2. Standard Deviation
3. Mean avg. deviation from median
4. Jump dist. to preceding segment
5. Jump dist. to succeeding segment
6. Width
![Page 66: Classification with AdaBoost](https://reader030.fdocuments.us/reader030/viewer/2022021419/58a1a9961a28ab244d8c460a/html5/thumbnails/66.jpg)
Features Segment
7. Linearity
8. Circularity
9. Radius
rc
![Page 67: Classification with AdaBoost](https://reader030.fdocuments.us/reader030/viewer/2022021419/58a1a9961a28ab244d8c460a/html5/thumbnails/67.jpg)
Features Segment
10. Boundary Length
11. Boundary Regularity
12. Mean curvature
13. Mean angular difference
14. Mean speed
![Page 68: Classification with AdaBoost](https://reader030.fdocuments.us/reader030/viewer/2022021419/58a1a9961a28ab244d8c460a/html5/thumbnails/68.jpg)
Features Resulting feature signature for each segment
![Page 69: Classification with AdaBoost](https://reader030.fdocuments.us/reader030/viewer/2022021419/58a1a9961a28ab244d8c460a/html5/thumbnails/69.jpg)
Training: Data Labeling Mark segments that correspond to people
Either manually or automatically
![Page 70: Classification with AdaBoost](https://reader030.fdocuments.us/reader030/viewer/2022021419/58a1a9961a28ab244d8c460a/html5/thumbnails/70.jpg)
Training: Data Labeling Automatic labeling: obvious approach, define area
of interest
Here: discrimination from background is relevant information, includes spatial relation between fore- and background. Thus: labeling is done by hand
4 m
3 m
![Page 71: Classification with AdaBoost](https://reader030.fdocuments.us/reader030/viewer/2022021419/58a1a9961a28ab244d8c460a/html5/thumbnails/71.jpg)
Training Resulting Training Set
Segments corresponding to people
(foreground segments)
Segments corresponding to other objects
(background segments)
+1 -1
![Page 72: Classification with AdaBoost](https://reader030.fdocuments.us/reader030/viewer/2022021419/58a1a9961a28ab244d8c460a/html5/thumbnails/72.jpg)
example1 . . . exampleN
Boosting
f#1 . . . f#14
w1 h1
wT hT
Σ
Weighted majority vote classifier
{-1,1} . . .
Strong Binary Classifier
Vocabulary of features
AdaBoost: Final Strong Classifier
![Page 73: Classification with AdaBoost](https://reader030.fdocuments.us/reader030/viewer/2022021419/58a1a9961a28ab244d8c460a/html5/thumbnails/73.jpg)
Experiments Env. 1: Corridor, no clutter
Env. 2: Office, very cluttered
![Page 74: Classification with AdaBoost](https://reader030.fdocuments.us/reader030/viewer/2022021419/58a1a9961a28ab244d8c460a/html5/thumbnails/74.jpg)
Experiments Env. 1 & 2: Corridor and Office
Env. 1→2: Cross-evaluation Trained in corridor, applied in office
![Page 75: Classification with AdaBoost](https://reader030.fdocuments.us/reader030/viewer/2022021419/58a1a9961a28ab244d8c460a/html5/thumbnails/75.jpg)
Experiments Adding motion feature (mean speed, f#14)
Experimental setup:
Robot Herbert
SICK 2D laser range finder, 1 degree resolution
→ Motion feature has no contribution
![Page 76: Classification with AdaBoost](https://reader030.fdocuments.us/reader030/viewer/2022021419/58a1a9961a28ab244d8c460a/html5/thumbnails/76.jpg)
Experiments Comparison with hand-tuned classifier
Jump distance θδ = 30 cm
Width θw,m = 5 cm, θw,M = 50 cm
Number of points θn = 4
Standard deviation θσ = 50 cm
Motion of points θv = 2 cm
People are often not detected
![Page 77: Classification with AdaBoost](https://reader030.fdocuments.us/reader030/viewer/2022021419/58a1a9961a28ab244d8c460a/html5/thumbnails/77.jpg)
Experiments Five best features:
1: Radius of LSQ-fitted circle, robust size measure (#9)
2: Mean angular difference Convexity measure (#13)
3/4: Jump distances Local minima measure (#4 and #5)
5: Mad from median Robust compactness measure (#3)
![Page 78: Classification with AdaBoost](https://reader030.fdocuments.us/reader030/viewer/2022021419/58a1a9961a28ab244d8c460a/html5/thumbnails/78.jpg)
Result: Classification T F
T
F
![Page 79: Classification with AdaBoost](https://reader030.fdocuments.us/reader030/viewer/2022021419/58a1a9961a28ab244d8c460a/html5/thumbnails/79.jpg)
Chapter Contents
Machine Learning: A Survey
Classification
AdaBoost
People Detection with Boosted Features
Place Recognition with Boosted Features
![Page 80: Classification with AdaBoost](https://reader030.fdocuments.us/reader030/viewer/2022021419/58a1a9961a28ab244d8c460a/html5/thumbnails/80.jpg)
Place Labeling: Motivation A map is a metric and topological model
of the environment
![Page 81: Classification with AdaBoost](https://reader030.fdocuments.us/reader030/viewer/2022021419/58a1a9961a28ab244d8c460a/html5/thumbnails/81.jpg)
Place Labeling: Motivation
Room
Corridor Doorway
Wanted: semantic information about places
![Page 82: Classification with AdaBoost](https://reader030.fdocuments.us/reader030/viewer/2022021419/58a1a9961a28ab244d8c460a/html5/thumbnails/82.jpg)
Scenario Example
I am in the corridor!
User: Albert, where are you?
![Page 83: Classification with AdaBoost](https://reader030.fdocuments.us/reader030/viewer/2022021419/58a1a9961a28ab244d8c460a/html5/thumbnails/83.jpg)
Scenario Example 2 Semantic mapping
Human-Robot Interaction of type: "Robot, get out of my room, go into the corridor!"
Room Corridor
Doorway
![Page 84: Classification with AdaBoost](https://reader030.fdocuments.us/reader030/viewer/2022021419/58a1a9961a28ab244d8c460a/html5/thumbnails/84.jpg)
Problem Statement Classification of the position of the robot using
a single observation: a 360° laser range scan
![Page 85: Classification with AdaBoost](https://reader030.fdocuments.us/reader030/viewer/2022021419/58a1a9961a28ab244d8c460a/html5/thumbnails/85.jpg)
Observations
![Page 86: Classification with AdaBoost](https://reader030.fdocuments.us/reader030/viewer/2022021419/58a1a9961a28ab244d8c460a/html5/thumbnails/86.jpg)
Observations
Room Room
![Page 87: Classification with AdaBoost](https://reader030.fdocuments.us/reader030/viewer/2022021419/58a1a9961a28ab244d8c460a/html5/thumbnails/87.jpg)
Observations
Room Room
![Page 88: Classification with AdaBoost](https://reader030.fdocuments.us/reader030/viewer/2022021419/58a1a9961a28ab244d8c460a/html5/thumbnails/88.jpg)
Observations
Room Room Doorway Doorway
![Page 89: Classification with AdaBoost](https://reader030.fdocuments.us/reader030/viewer/2022021419/58a1a9961a28ab244d8c460a/html5/thumbnails/89.jpg)
Observations
Room Room Doorway Doorway
![Page 90: Classification with AdaBoost](https://reader030.fdocuments.us/reader030/viewer/2022021419/58a1a9961a28ab244d8c460a/html5/thumbnails/90.jpg)
Observations
Room Room Corridor Corridor Doorway Doorway
![Page 91: Classification with AdaBoost](https://reader030.fdocuments.us/reader030/viewer/2022021419/58a1a9961a28ab244d8c460a/html5/thumbnails/91.jpg)
Similar Observations
![Page 92: Classification with AdaBoost](https://reader030.fdocuments.us/reader030/viewer/2022021419/58a1a9961a28ab244d8c460a/html5/thumbnails/92.jpg)
Similar Observations
Corridor Doorway
![Page 93: Classification with AdaBoost](https://reader030.fdocuments.us/reader030/viewer/2022021419/58a1a9961a28ab244d8c460a/html5/thumbnails/93.jpg)
Classification Problem
![Page 94: Classification with AdaBoost](https://reader030.fdocuments.us/reader030/viewer/2022021419/58a1a9961a28ab244d8c460a/html5/thumbnails/94.jpg)
Classification Problem
![Page 95: Classification with AdaBoost](https://reader030.fdocuments.us/reader030/viewer/2022021419/58a1a9961a28ab244d8c460a/html5/thumbnails/95.jpg)
Classification Problem
Room Room Corridor Corridor Doorway Doorway
?
![Page 96: Classification with AdaBoost](https://reader030.fdocuments.us/reader030/viewer/2022021419/58a1a9961a28ab244d8c460a/html5/thumbnails/96.jpg)
Representing the Observations How we represent the 360 laser beams for our
classification task?
As a list of beams Problem: which beam is the first beam?
Not invariant to rotation!
!=
![Page 97: Classification with AdaBoost](https://reader030.fdocuments.us/reader030/viewer/2022021419/58a1a9961a28ab244d8c460a/html5/thumbnails/97.jpg)
Representing the Observations A list of scalar geometrical features of the scan
The features are all invariant to rotation
=
![Page 98: Classification with AdaBoost](https://reader030.fdocuments.us/reader030/viewer/2022021419/58a1a9961a28ab244d8c460a/html5/thumbnails/98.jpg)
Simple Features
• Gap = d > θ • f = # Gaps
Minimum
• f =Area • f =Perimeter • f = d
d di
N1f =
d
• f = d •
d
Σ di
![Page 99: Classification with AdaBoost](https://reader030.fdocuments.us/reader030/viewer/2022021419/58a1a9961a28ab244d8c460a/html5/thumbnails/99.jpg)
Simple Features Features of the raw beams
![Page 100: Classification with AdaBoost](https://reader030.fdocuments.us/reader030/viewer/2022021419/58a1a9961a28ab244d8c460a/html5/thumbnails/100.jpg)
Simple Features Features of the closed polynom P(z)
made up by the beams
![Page 101: Classification with AdaBoost](https://reader030.fdocuments.us/reader030/viewer/2022021419/58a1a9961a28ab244d8c460a/html5/thumbnails/101.jpg)
Multiple Classes
Room Room Corridor Corridor Doorway Doorway 1 3 2
![Page 102: Classification with AdaBoost](https://reader030.fdocuments.us/reader030/viewer/2022021419/58a1a9961a28ab244d8c460a/html5/thumbnails/102.jpg)
Multiple Classes
Room Room Corridor Corridor Doorway Doorway 1 3 2
![Page 103: Classification with AdaBoost](https://reader030.fdocuments.us/reader030/viewer/2022021419/58a1a9961a28ab244d8c460a/html5/thumbnails/103.jpg)
Multiple Classes Sequence of binary classifiers in a decision list
Alternative to AdaBoost.M2, the multi-class variant of AdaBoost
Order matters, chosen to be according to error rate One-vs-all learning
Corridor Classifier
Room Classifier
H(x)=–1 H(x)=–1
H(x)=1 H(x)=1
Corridor Corridor Room Room
Doorway Doorway
![Page 104: Classification with AdaBoost](https://reader030.fdocuments.us/reader030/viewer/2022021419/58a1a9961a28ab244d8c460a/html5/thumbnails/104.jpg)
Experiments Training (top) # examples:
16045
Test (bottom) # examples:
18726 classification: 93.94%
Building 079 Uni. Freiburg
Room Room Corridor Corridor Doorway Doorway
![Page 105: Classification with AdaBoost](https://reader030.fdocuments.us/reader030/viewer/2022021419/58a1a9961a28ab244d8c460a/html5/thumbnails/105.jpg)
Training (left) # examples:
13906
Test (right) # examples:
10445 classification: 89.52%
Building 101 Uni. Freiburg
Room Room Corridor Corridor Doorway Doorway Hallway Hallway
Experiments
![Page 106: Classification with AdaBoost](https://reader030.fdocuments.us/reader030/viewer/2022021419/58a1a9961a28ab244d8c460a/html5/thumbnails/106.jpg)
Application to New Environment
Training map
Intel Research Lab in Seattle
![Page 107: Classification with AdaBoost](https://reader030.fdocuments.us/reader030/viewer/2022021419/58a1a9961a28ab244d8c460a/html5/thumbnails/107.jpg)
Application to New Environment
Training map
Intel Research Lab in Seattle
Room Room Corridor Corridor Doorway Doorway
![Page 108: Classification with AdaBoost](https://reader030.fdocuments.us/reader030/viewer/2022021419/58a1a9961a28ab244d8c460a/html5/thumbnails/108.jpg)
Summary People detection and place recognition phrased as
a classification problem using (geometrical and statistical) features that characterize range data (entire scans, groups of neighboring beams)
AdaBoost allows for a systematic approach to perform this task
Both, single-frame people detection and place recognition with around 90% accuracy
Learned classifier clearly superior to hand-tuned classifier