Recognizing Action Units for Facial Expression Analysis (ppt).pdf
-
Upload
felice-chew -
Category
Documents
-
view
218 -
download
0
Transcript of Recognizing Action Units for Facial Expression Analysis (ppt).pdf
-
7/28/2019 Recognizing Action Units for Facial Expression Analysis (ppt).pdf
1/26
Recognizing Action Units forFacial Expression Analysis
Y.-l. Tian, T. Kanade, and J. F.
Cohn, PAMI 23(2), 2001Presented by Wei-Kai Liao
-
7/28/2019 Recognizing Action Units for Facial Expression Analysis (ppt).pdf
2/26
-
7/28/2019 Recognizing Action Units for Facial Expression Analysis (ppt).pdf
3/26
Automatic Facial Expression
Recognition
Key issues:
Facial features extraction & representation Appearance-based, geometric feature-based, or hybrid
Facial expression classification NN, SVM, BN, HMM, rule-based,
Description of facial expressions Expression prototypes Facial Action Coding System
-
7/28/2019 Recognizing Action Units for Facial Expression Analysis (ppt).pdf
4/26
Expression Prototypes
6 universal expression prototype (emotion
prototype): disgust, fear, joy, surprise,sadness, anger
-
7/28/2019 Recognizing Action Units for Facial Expression Analysis (ppt).pdf
5/26
Facial Action Coding System
Defined Action Unit (AU), which is a description of avisible facial change, in the face
Totally 44 AUs in the face: 30 are related to the contractions of specific facial muscles
12 for upper face and 18 for lower face
Initially, it is designed for trained human expert todetect the change of facial appearance
AUs could be additive or non-additive In this paper, the combined AUs are treated as a new AU
AUs in upper and lower face are relativelyindependent
-
7/28/2019 Recognizing Action Units for Facial Expression Analysis (ppt).pdf
6/26
Upper Face AU
-
7/28/2019 Recognizing Action Units for Facial Expression Analysis (ppt).pdf
7/26
Lower Face AU
-
7/28/2019 Recognizing Action Units for Facial Expression Analysis (ppt).pdf
8/26
System Overview Face and facial features are
automatically detected and thenmanually adjusted in the firstframe
Detect and track the facialfeatures.
Map the extracted facialfeatures into 2 sets ofparameters These parameters are
geometrically normalized tocompensate for image scaleand in-plane head motion
Feed these 2 sets of parametersinto 2 NN-based classifiers
-
7/28/2019 Recognizing Action Units for Facial Expression Analysis (ppt).pdf
9/26
Multistate Facial Component
Models Detect and track facial
components in near frontalfaces
Facial features could bedivided into 2 classes: Permanent: brow, cheek,
lip, eye Transient
Facial features have severalstates
Brow and cheek Modeled by a triangular
template with 6 parameters Use Lucas-Kanade algorithm
to track these templates
-
7/28/2019 Recognizing Action Units for Facial Expression Analysis (ppt).pdf
10/26
3-State Lip Model Use L-K algorithm to track
points of lip template For open and tightly
closed, there are non-lippixels inside the lip contour
Use the Gaussian mixturemodel to represent the colordistribution of the pixelsinside the lip contour
Determine the state basedon the shape and the color
-
7/28/2019 Recognizing Action Units for Facial Expression Analysis (ppt).pdf
11/26
Dual-State Eye Model The state is controlled by iris
Iris detection:
Edge maps: Canny edge operator
Fit the edge maps with iris mask
Eye corners Inner corners: LK tracking
Outer corners: determined by inner
corners
-
7/28/2019 Recognizing Action Units for Facial Expression Analysis (ppt).pdf
12/26
Transient Features Nasolabial furrows,
crows-feet wrinkles,Nose wrinkles
These areas are locatedusing the trackedlocations of thecorrespondingpermanent features
A Canny edge detectorto quantify the amountand orientation offurrows
-
7/28/2019 Recognizing Action Units for Facial Expression Analysis (ppt).pdf
13/26
Permanent Features Tracking
Results
-
7/28/2019 Recognizing Action Units for Facial Expression Analysis (ppt).pdf
14/26
Permanent Features Tracking
Results
-
7/28/2019 Recognizing Action Units for Facial Expression Analysis (ppt).pdf
15/26
Transient Features Tracking
Results
-
7/28/2019 Recognizing Action Units for Facial Expression Analysis (ppt).pdf
16/26
Upper Face Feature
Representation 15 parameters for
upper face features
12 for motion and shapeof the eyes, brows, and
cheek 2 for crows-feet wrinkles
1 for distance betweenthe brows
Computed as ratios tothe first frame
-
7/28/2019 Recognizing Action Units for Facial Expression Analysis (ppt).pdf
17/26
Lower Face Feature
Representation 9 parameters for lower face
features:
6 for lip shape, state, motion
3 for furrows in the nasolabial
and nasal root regions.
Normalized to the neutral
-
7/28/2019 Recognizing Action Units for Facial Expression Analysis (ppt).pdf
18/26
Neural Network Classifier A 3 layer neural
networks with onehidden layer
Separate NNs are usedfor upper and lowerface
6 12 hidden units are
used
Back-propagationalgorithm is used
-
7/28/2019 Recognizing Action Units for Facial Expression Analysis (ppt).pdf
19/26
Experimental Results
Left: a same subject could appear in both
training and test sets Right: no subject appears in both training and
test sets
-
7/28/2019 Recognizing Action Units for Facial Expression Analysis (ppt).pdf
20/26
Combined AUs Recognition
Result: Upper Face
-
7/28/2019 Recognizing Action Units for Facial Expression Analysis (ppt).pdf
21/26
Combined AUs Recognition
Result: Lower Face
-
7/28/2019 Recognizing Action Units for Facial Expression Analysis (ppt).pdf
22/26
Generalized to Different
Databases
Train on one database and test on the other
database
-
7/28/2019 Recognizing Action Units for Facial Expression Analysis (ppt).pdf
23/26
-
7/28/2019 Recognizing Action Units for Facial Expression Analysis (ppt).pdf
24/26
Problems (1) Only handling limited head motion (in-plane
motion) This is not sufficient for a practical application
Robustness issues: Complex environment Various lighting conditions Occlusions
Image resolution Not fully automatic need to manually adjust
the detected features in the first frame
-
7/28/2019 Recognizing Action Units for Facial Expression Analysis (ppt).pdf
25/26
-
7/28/2019 Recognizing Action Units for Facial Expression Analysis (ppt).pdf
26/26
Further References
CMU Automated Face Analysis website: http://www-2.cs.cmu.edu/~face/index2.htm http://www-2.cs.cmu.edu/afs/cs/project/face/www/Facial.htm
Survey paper Y.-l. Tian, T. Kanade, and J. F. Cohn, Facial Expression
Analysis, Handbook of Face Recognition, S. Z. Li and A. K.Jain, ed., Springer, October 2003.
FACS P. Ekman and W. Friesen, The Facial Action Coding System:
A Technique for the Measurement of Facial Movement,Consulting Psychologists Press, San Francisco, 1978
http://www-2.cs.cmu.edu/~face/index2.htmhttp://www-2.cs.cmu.edu/afs/cs/project/face/www/Facial.htmhttp://www-2.cs.cmu.edu/afs/cs/project/face/www/Facial.htmhttp://www-2.cs.cmu.edu/afs/cs/project/face/www/Facial.htmhttp://www-2.cs.cmu.edu/~face/index2.htmhttp://www-2.cs.cmu.edu/~face/index2.htm