CSCI480/582 Lecture 31 Chap 5.3 Performance Driven Facial Animation Apr, 13, 2009
description
Transcript of CSCI480/582 Lecture 31 Chap 5.3 Performance Driven Facial Animation Apr, 13, 2009
![Page 1: CSCI480/582 Lecture 31 Chap 5.3 Performance Driven Facial Animation Apr, 13, 2009](https://reader035.fdocuments.us/reader035/viewer/2022070416/5681520b550346895dc04d86/html5/thumbnails/1.jpg)
CSCI480/582 Lecture 31 Chap 5.3Performance Driven Facial Animation
Apr, 13, 2009
![Page 2: CSCI480/582 Lecture 31 Chap 5.3 Performance Driven Facial Animation Apr, 13, 2009](https://reader035.fdocuments.us/reader035/viewer/2022070416/5681520b550346895dc04d86/html5/thumbnails/2.jpg)
Outline What differs facial animation from body
animation? Face model Facial motions and FACS Facial motion synchronization with audio
production Performance-driven techniques
![Page 3: CSCI480/582 Lecture 31 Chap 5.3 Performance Driven Facial Animation Apr, 13, 2009](https://reader035.fdocuments.us/reader035/viewer/2022070416/5681520b550346895dc04d86/html5/thumbnails/3.jpg)
Face Model for Facial Animation
In modeling face, Four aspects needs to be considered Structure: vertex distribution and topology to
represent face in geometry
Appearance: color, texture, light reflection of face
![Page 4: CSCI480/582 Lecture 31 Chap 5.3 Performance Driven Facial Animation Apr, 13, 2009](https://reader035.fdocuments.us/reader035/viewer/2022070416/5681520b550346895dc04d86/html5/thumbnails/4.jpg)
Face Model - Geometry Dense mesh around the “holes” The density of the mesh determines the level of details in facial deformation Eyes, ears, and mouth cavity Teeth and tongue
![Page 5: CSCI480/582 Lecture 31 Chap 5.3 Performance Driven Facial Animation Apr, 13, 2009](https://reader035.fdocuments.us/reader035/viewer/2022070416/5681520b550346895dc04d86/html5/thumbnails/5.jpg)
Face Model - Appearance
Using Texture Mapping
Unwinding the “cylinder” surface by concatenating multiple camera views to form the texture map
![Page 6: CSCI480/582 Lecture 31 Chap 5.3 Performance Driven Facial Animation Apr, 13, 2009](https://reader035.fdocuments.us/reader035/viewer/2022070416/5681520b550346895dc04d86/html5/thumbnails/6.jpg)
More Advanced Skin Appearance Model
The skin is translucent Light is scattered through
multiple layers, then reflected from a sub-layer
Bidirectional Reflectance Distribution Function is often used
![Page 7: CSCI480/582 Lecture 31 Chap 5.3 Performance Driven Facial Animation Apr, 13, 2009](https://reader035.fdocuments.us/reader035/viewer/2022070416/5681520b550346895dc04d86/html5/thumbnails/7.jpg)
Bidirectional Reflectance Distribution Function(BRDF)
A 4-dimensional function that defines how light is reflected at an opaque surface
Input:
incoming light direction wi
Outgoing direction wo
Output:
Ratio of reflected radiance along wo to irradiance incident along wi
![Page 8: CSCI480/582 Lecture 31 Chap 5.3 Performance Driven Facial Animation Apr, 13, 2009](https://reader035.fdocuments.us/reader035/viewer/2022070416/5681520b550346895dc04d86/html5/thumbnails/8.jpg)
Facial Motion Expressions involving
All front facial mesh
Eye ball orientation
Speeches involving Lip and jaw
Tongue
The combination of expressions and speeches
![Page 9: CSCI480/582 Lecture 31 Chap 5.3 Performance Driven Facial Animation Apr, 13, 2009](https://reader035.fdocuments.us/reader035/viewer/2022070416/5681520b550346895dc04d86/html5/thumbnails/9.jpg)
Facial Acting Code System
Originally developed by Ekman and Friessen in 70s
Based on the physiology of facial muscles
A common standard to systematically categorize the physical expression of emotions
![Page 10: CSCI480/582 Lecture 31 Chap 5.3 Performance Driven Facial Animation Apr, 13, 2009](https://reader035.fdocuments.us/reader035/viewer/2022070416/5681520b550346895dc04d86/html5/thumbnails/10.jpg)
FACS – The Codes for Action Units
Action Descriptors
Inner Brow Raiser -- Frontalis (pars medialis)
2 Outer Brow Raiser -- Frontalis (pars lateralis)
4 Brow Lowerer -- Corrugator supercilii, Depressor supercilii
5 Upper Lid Raiser -- Levator palpebrae superioris
6 Cheek Raiser -- Orbicularis oculi (pars orbitalis)
7 Lid Tightener -- Orbicularis oculi (pars palpebralis)
9 Nose Wrinkler -- Levator labii superioris alaeque nasi
10 Upper Lip Raiser -- Levator labii superioris
…
66 Action descriptors
Demo
![Page 11: CSCI480/582 Lecture 31 Chap 5.3 Performance Driven Facial Animation Apr, 13, 2009](https://reader035.fdocuments.us/reader035/viewer/2022070416/5681520b550346895dc04d86/html5/thumbnails/11.jpg)
Lip-Sync by Viseme Morphing Use reference video to rotoscope
the shape of the 3D model for key gestures
Use a set of viseme corresponding to phonemes, then interpolate between viseme gestures
Such approach often is not detailed enough for finer visual speech animation
![Page 12: CSCI480/582 Lecture 31 Chap 5.3 Performance Driven Facial Animation Apr, 13, 2009](https://reader035.fdocuments.us/reader035/viewer/2022070416/5681520b550346895dc04d86/html5/thumbnails/12.jpg)
Facial Motion Driven by Mocap Data
Captures subtle details of facial expressions
But, Mocap Data need be to cleaned for good rendering effect. Often this step is labor intensive.
How can we make Mocap more flexible?
![Page 13: CSCI480/582 Lecture 31 Chap 5.3 Performance Driven Facial Animation Apr, 13, 2009](https://reader035.fdocuments.us/reader035/viewer/2022070416/5681520b550346895dc04d86/html5/thumbnails/13.jpg)
FACS-Viseme Solver Use a small set of Mocap data to synthesize a
much larger range of facial motions Each piece of mocap data in the small recording set
represent an Action Unit
Use Action Unit analysis to decompose recorded Mocap data for easy post editing
Use audio-visual mapping analysis to align mocap segment for new speech
N
kkkAUwimocap
1
)(