Learning the Appearance of Faces: A Unifying Approach for the Analysis and Synthesis of Images....
-
date post
22-Dec-2015 -
Category
Documents
-
view
216 -
download
0
Transcript of Learning the Appearance of Faces: A Unifying Approach for the Analysis and Synthesis of Images....
Learning the Appearance of Faces:A Unifying Approach for the Analysis and Synthesis of Images.
Thomas VetterThomas Vetter
Germany GermanyUniversity of FreiburgUniversity of Freiburg
http://graphics.informatik.uni-freiburg.dehttp://graphics.informatik.uni-freiburg.de
Computer Vision & Computer Graphics
Computer Graphics can help to solve Computer Vision!
| G (p) - I |2 = min Parameters
G ( image ) Parameters-1
Vision ( image ) parameters
image Graphics ( parameters )
Analysis by Synthesis
3DWorld Image
Analysis
Synthesis
Image Model
Image Description
model parameter
Synthesis of Faces
Input ImageInput Image
Modeler
ResultResult
Database
FaceAnalyzer
3D Head3D Head
Morphable
Face Model
Approach: Example based modeling of faces
2D Image 3D Face Models2D Image 3D Face Models
= w1 * + w2 * + w3 * + w4 * +. . .
Cylindrical Coordinates
red(h,)green(h,)
blue(h,)
red(h,)green(h,)
blue(h,)
h
radius(h,)radius(h,)
h
Morphing 3D Faces
3D Blend
3D Morph
1__2
1__2
+ =
Correspondence: A two step process!
Correspondence between
1. two examples ( Optical Flow like algorithms).
2. many examples ( Morphable Model )
Reference
Example
2nd Example
=
a1 * + a2 * + a3 * + a4 * +. . .
b1 * + b2 * + b3 * + b4 * +. . .
Vector space of 3D faces.
A Morphable Model can generate new faces.
Manipulation of Faces
Modeler
Modelling in Face Space
Caricatur
OriginalOriginal
AverageAverage
Modelling the Appearance of Faces
A face is represented as a point in face space.Which directions code for specific attributes ?
Learning from Labeled Example Faces
Fitting a (linear) regression function
Facial Attributes
WeightWeightWeightWeight
OriginalOriginalOriginalOriginal
Subjective Subjective AttractivenessAttractiveness
Subjective Subjective AttractivenessAttractiveness
Transfer of Facial Expressions
= Smile= Smile--
Originals:Originals:
+ Smile =+ Smile =Novel Face:Novel Face:
Facial Expressions
OriginalOriginalOriginalOriginal
3D Shape from Images
FaceAnalyzer
3D Head3D HeadInput ImageInput Image
Matching a Morphable 3D-Face-Model
= R
Optimization problem!Optimization problem!Optimization problem!Optimization problem!
a1 * + a2 * + a3 * + a4 * +. .
b1 * + b2 * + b3 * + b4 * +. .
Error Function
• Image difference
• Plausible parameters
• Minimize
priorimage EEE priorimage EEE
2
,inputmodelImage ),(),(
yx
yxyxE II 2
,inputmodelImage ),(),(
yx
yxyxE II
),...),(log(prior ii bapE ),...),(log(prior ii bapE
Optimization Strategies
• Stochastic Gradient Decent
new old
old
i i ci
dEc c i
dc
new old
old
i i ci
dEc c i
dc
I c A I c A
• Difference Decomposition • Difference Decomposition
Future Challenges
• Which Object Classes are linear ?
• How to built them automatically?