Cootes Paradise: Environmental Impact Evaluation Hamilton ...
52425 Digital Video Technology Mikkel B. Stegmann Active … · 2002. 2. 5. · A deformable...
Transcript of 52425 Digital Video Technology Mikkel B. Stegmann Active … · 2002. 2. 5. · A deformable...
![Page 1: 52425 Digital Video Technology Mikkel B. Stegmann Active … · 2002. 2. 5. · A deformable template model introduced by Cootes, Edwards and Taylor (1998). – A priori knowledge](https://reader035.fdocuments.us/reader035/viewer/2022081410/60a09ed6ad56874b980522b2/html5/thumbnails/1.jpg)
Extraction of MPEG-4 Facial Animation Parameters with
Active Appearance Models
Mikkel B. Stegmann
52425 Digital Video Technology
TELE February 27th 2001
![Page 2: 52425 Digital Video Technology Mikkel B. Stegmann Active … · 2002. 2. 5. · A deformable template model introduced by Cootes, Edwards and Taylor (1998). – A priori knowledge](https://reader035.fdocuments.us/reader035/viewer/2022081410/60a09ed6ad56874b980522b2/html5/thumbnails/2.jpg)
Aim & Method
� To extract MPEG-4 Facial Animation Parameters (FAPs) from a detailed map of facialfeatures in an unknown image
� Active Appearance Models (AAMs):
– A deformable template model introduced by Cootes, Edwards and Taylor (1998).
– A priori knowledge is learned through statistical analysis of a training set
![Page 3: 52425 Digital Video Technology Mikkel B. Stegmann Active … · 2002. 2. 5. · A deformable template model introduced by Cootes, Edwards and Taylor (1998). – A priori knowledge](https://reader035.fdocuments.us/reader035/viewer/2022081410/60a09ed6ad56874b980522b2/html5/thumbnails/3.jpg)
Presentation outline
� Active Appearance Models
� Segmentation of faces in the IMM face database• Building the Active Appearance Model
• Example results[face database provided by Janusz Sierakowski, Michael M. Nordstrøm &
Mads Larsen]
� Real-time tracking using an AAM
� Near real-time extraction of MPEG-4 facial parameters• MPEG-4 AAM
• Results[presentation based on work by Jörgen Ahlberg, Linköbing University]
![Page 4: 52425 Digital Video Technology Mikkel B. Stegmann Active … · 2002. 2. 5. · A deformable template model introduced by Cootes, Edwards and Taylor (1998). – A priori knowledge](https://reader035.fdocuments.us/reader035/viewer/2022081410/60a09ed6ad56874b980522b2/html5/thumbnails/4.jpg)
Active Appearance Models
� Aim: To locate non-rigid objects with large variability – but distinct features – in digital images
� Method: A generative deformable model providing a photo-realistic model representation
� Characteristics of the object class is learned through statistical analysis of shape and grey-level variation in a training set
� A compact object class description is derived and used to rapidly search images for new object instances
� Properties: general, specific, robust, parameter-less and fast
Ref: Cootes, T. F. and Edwards, G. J. and Taylor, C. J., Active Appearance Models, Proc. European Conf. On Computer Vision, Vol. 2, pp. 484-498, 1998.
![Page 5: 52425 Digital Video Technology Mikkel B. Stegmann Active … · 2002. 2. 5. · A deformable template model introduced by Cootes, Edwards and Taylor (1998). – A priori knowledge](https://reader035.fdocuments.us/reader035/viewer/2022081410/60a09ed6ad56874b980522b2/html5/thumbnails/5.jpg)
Face training set
� 35 frontal images of 35 different human faces(the total database consists of 240 face images)
� Each face is annotated with 58 corresponding points
Annotated training exampleTraining example AAM mesh model
![Page 6: 52425 Digital Video Technology Mikkel B. Stegmann Active … · 2002. 2. 5. · A deformable template model introduced by Cootes, Edwards and Taylor (1998). – A priori knowledge](https://reader035.fdocuments.us/reader035/viewer/2022081410/60a09ed6ad56874b980522b2/html5/thumbnails/6.jpg)
Face AAM
� Shape and texture variation are extracted by a principal component analysis of the shape points and pixel intensities
� Resulting model: ~7600 pixels, 26 combined parameters
Registration 1st shape mode 1st texture mode 1st combined mode
![Page 7: 52425 Digital Video Technology Mikkel B. Stegmann Active … · 2002. 2. 5. · A deformable template model introduced by Cootes, Edwards and Taylor (1998). – A priori knowledge](https://reader035.fdocuments.us/reader035/viewer/2022081410/60a09ed6ad56874b980522b2/html5/thumbnails/7.jpg)
AAM Search
![Page 8: 52425 Digital Video Technology Mikkel B. Stegmann Active … · 2002. 2. 5. · A deformable template model introduced by Cootes, Edwards and Taylor (1998). – A priori knowledge](https://reader035.fdocuments.us/reader035/viewer/2022081410/60a09ed6ad56874b980522b2/html5/thumbnails/8.jpg)
0
Multi-resolution AAMs
64x64
128x128
256x256
� Normal AAMs are built at each level of a scale-pyramid
� Search-based initialization at top level
� AAM search results propagates down the pyramid
3 level Cardiac AAM
Segmentation time:
~200 ms
Texture model:
Level 0: 2196 pixelsLevel 1: 549 pixelsLevel 2: 136 pixels
0
1
2
![Page 9: 52425 Digital Video Technology Mikkel B. Stegmann Active … · 2002. 2. 5. · A deformable template model introduced by Cootes, Edwards and Taylor (1998). – A priori knowledge](https://reader035.fdocuments.us/reader035/viewer/2022081410/60a09ed6ad56874b980522b2/html5/thumbnails/9.jpg)
Real-time AAMs� Purpose
– To demonstrate the general nature and the performance of AAMs
� Equipment– PC & 30$ web-cam
� LQ-AAM
– 5 images / 12 landmarks
– Texture model: 9.100 pixels
– Initialized by a 2 level pyramid
� Performance– 7-10 frames/sec.
– No temporal filtering performed
![Page 10: 52425 Digital Video Technology Mikkel B. Stegmann Active … · 2002. 2. 5. · A deformable template model introduced by Cootes, Edwards and Taylor (1998). – A priori knowledge](https://reader035.fdocuments.us/reader035/viewer/2022081410/60a09ed6ad56874b980522b2/html5/thumbnails/10.jpg)
Near real-time AAMs
![Page 11: 52425 Digital Video Technology Mikkel B. Stegmann Active … · 2002. 2. 5. · A deformable template model introduced by Cootes, Edwards and Taylor (1998). – A priori knowledge](https://reader035.fdocuments.us/reader035/viewer/2022081410/60a09ed6ad56874b980522b2/html5/thumbnails/11.jpg)
MPEG-4 AAM
� AAM built on 257 face images of 5 different persons
� AAM mesh corresponds to the MPEG-4 Facial Feature Points, thus making extration of MPEG-4 Facial Animation Parameters (FAPs) easy
� Processing time: 0.1–1.5 seconds/frame (500 MHz PIII)
Ref: Ahlberg, Jörgen, An Experiment on 3D Face Model Adaption using the Active Appearance Algorithm, LinköbingUniversity, Image Coding Group, 2001, http://www.icg.isy.liu.se /~ahlberg
![Page 12: 52425 Digital Video Technology Mikkel B. Stegmann Active … · 2002. 2. 5. · A deformable template model introduced by Cootes, Edwards and Taylor (1998). – A priori knowledge](https://reader035.fdocuments.us/reader035/viewer/2022081410/60a09ed6ad56874b980522b2/html5/thumbnails/12.jpg)
Results
AAM tracking of head movements Synthesized MPEG-4 FAP movie
![Page 13: 52425 Digital Video Technology Mikkel B. Stegmann Active … · 2002. 2. 5. · A deformable template model introduced by Cootes, Edwards and Taylor (1998). – A priori knowledge](https://reader035.fdocuments.us/reader035/viewer/2022081410/60a09ed6ad56874b980522b2/html5/thumbnails/13.jpg)
Other AAM Models
![Page 14: 52425 Digital Video Technology Mikkel B. Stegmann Active … · 2002. 2. 5. · A deformable template model introduced by Cootes, Edwards and Taylor (1998). – A priori knowledge](https://reader035.fdocuments.us/reader035/viewer/2022081410/60a09ed6ad56874b980522b2/html5/thumbnails/14.jpg)
The end
� Acknowledgements
– Face database built by:
• Janusz Sierakowski, Michael M. Nordstrøm & Mads Larsen
– MPEG-4 facial parameter extraction by:
• Jörgen Ahlberg, Linköbing University - http://www.icg.isy.liu.se/~ahlberg
� References– Tim Cootes (co-inventor of AAM) - http://www.isbe.man.ac.uk/~bim
– The AAM-Site - http://www.imm.dtu.dk/~aam
– Mikkel B. Stegmann, [email protected]