Face Alignment Using Cascaded Boosted Regression Active Shape Models Michael Dixon.
-
Upload
jared-caldwell -
Category
Documents
-
view
219 -
download
1
Transcript of Face Alignment Using Cascaded Boosted Regression Active Shape Models Michael Dixon.
Faces in computer vision
• What problems do people work on?– Detection– Alignment– High-level analysis
• Face recognition• Facial expression
recognition• Face tracking
2
Face alignment
• Given an image of a face and an initial guess, localize key facial features
• Approaches– Active Shape Model,
1992– Boosted Regression
ASM, 2007
3
Learning a shape model
• Represent as a linear subspace
Mean face shape
Principal variations from the mean8
Feature model
• Given a patch near a facial feature, predict the correct position of that feature
Given Predict
10
Learning a feature model
• Generate training examples with known feature positions
• Train a regression model to predict the correct displacement
11
Boosted regression
• Goal: Learn a function to predict a set of target values
• Boosting builds a strong regression model from many weak models– Evaluate a large pool of possible weak regression
functions– Select the function with the lowest error and add
it to the strong regression model– Update the target values and repeat
12
Weak regression model
13
bthaf mm Weak regression functionHaar wavelet features
hm =The sum of all pixel values under the white box minus the sum of all pixel values under the black box
Haar wavelet response
Weak regression example
bthaf mm
fit weak regression function to data
disp
lace
men
thm
a = -0.027b = 0.012t = 21.7
disp
lace
men
thm
14
Strong regression model
15
Predicted displacement
Gro
und-
trut
h di
spla
cem
ent
25 weak regression functions combined into a strong regression function
The Active Shape Model framework
• Combining the shape and feature models
Shape Features Alignment16
Fitting using Boosted Regression ASM
• Initialize the feature positions
• Iteratively– Predict feature
positions using regression model
– Constrain to fit the shape model
– Update feature positions
17
Limitations of the previous work
• How often does the boosted regression feature model improve on the initial estimate?
Some improvement
Significantimprovement
0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 50
10
20
30
40
50
60
70
80
90
100
Displacement (in pixels)
Perc
ent t
hat i
mpr
oved
Improved by at least 50%
Any improvement
18
Predicted position vs. actual position
Accuracy trade-off
• Regression model can’t accurately predict both large and small displacements
0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 50
10
20
30
40
50
60
70
80
90
100
Displacement (in pixels)
Some improvement
Significantimprovement
Perc
ent t
hat i
mpr
oved
0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 50
10
20
30
40
50
60
70
80
90
100
Displacement (in pixels)
Some improvement
Significantimprovement
Perc
ent t
hat i
mpr
oved
Model trained on large displacements Model trained on small displacements
19
Proposed solution
• Train multiple models (coarse to fine) and apply them in sequence
Coarse regression model
Fine regression model
Displacement (in pixels)
Perc
ent t
hat i
mpr
oved
20
Cascaded Boosted Regression ASM
21
FaceDetector
FaceDetector
Boosted Regression ASM15 iterations
Stage 15 iterations
Stage 25 iterations
Stage 35 iterations
Cascaded Boosted Regression ASM
Image
Image
Alignment
Alignment
Learning an alignment cascade• Train a new stage of the
cascade using the output of the previous stage– Use a face detector as the
initial stage
• For each stage– Measure error distribution of
each feature– Generate training examples
from the error distribution– Train new feature models– Align all images using the
updated model to get a new error distribution
22
Quantitative evaluation• Error metric:
where:– di is the distance between the estimated position
and the ground truth position of the ith point– s is the inter-ocular distance
• An alignment is only as good as its worst point
s
de i
i 20,,1max
Alignment vs.Ground-truth
24
Results
• Evaluated on 500 unseen test images
0 0.05 0.1 0.15 0.2 0.25 0.30
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Cum
ulati
ve e
rror
dis
trib
ution
Alignment error
CascadedStandardAverage face
25
73%
19%
3%
Results
• Alignment accuracy after each stage
26
0 0.05 0.1 0.15 0.2 0.25 0.30
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Stage 1Stage 2Stage 3Cu
mul
ative
err
or d
istr
ibuti
on
Alignment error Stage
0 1 2 30
0.02
0.04
0.06
0.08
0.1
0.12
0.14
0.16
0.18
0.2
Med
ian
alig
nmen
t err
or
Conclusions
• Boosted Regression ASMs are a newly proposed method for performing face alignment
• Training a cascade of Boosted Regression ASMs can significantly improve alignment accuracy
27