Facial Feature Tracking under Varying Facial Expressions and Face Poses based on Restricted...
Embed Size (px)
Transcript of Facial Feature Tracking under Varying Facial Expressions and Face Poses based on Restricted...
- 1. JAMES COOK AUSTRALIA INSTITUTE OF HIGHER LEARNING IN SINGAPORE HEALTH DIAGNOSTIC BY ANALYSING FACE IMAGES USING MOBILE DEVICES Instructor : Dr. Insu Song Student : Ho Thi Hoang Yen Email:
- 2. INTRODUCTION Previously : A robust, highly accurate method for detecting 20 facial points in images of expressionless faces
- 3. BACKGROUND A restricted Boltzmann machine (RBM) is a generative stochastic artificial neural network that can learn a probability distribution over its set of inputs. Inventor : under the name Harmonium by Paul Smolensky in 1986. Fast learning algorithms : mid-2000s by Geoffrey Hinton & collaborators. RBMs have found applications in dimensionality reduction, classification, collaborative filtering, feature learning and topic modelling.
- 4. INTRODUCTION Track 26 feature points different facial expressions, varying poses, or occlusion
- 5. INTRODUCTION Other methods : - track facial feature points independently or - build a shape model to capture the variations of face shape or - appearance regardless of the facial expressions and face poses This method : capture the distinctions & variations of face shapes due to facial expression and pose change in a UNIFIED framework
- 6. CONTENT - MEDOTHOLOGY 1. Related work 2. FrontalRBM & PoseRBM 3. Facial feature tracking based on face shape prior model 4. Experimental results
- 7. 1. RELATED WORK Facial feature localization: 2 categories : Without shape prior models : track each facial feature point independently and ignore the prior knowledge about the face => sensitive with expression & pose With shape prior models : capture the dependence between facial feature points by explicitly modeling the general properties as well as the variations of facial shape or appearance
- 8. 1. RELATED WORK Facial feature localization: Recently methods: Active Shape Model (ASM)  and Active Appearance Model (AAM) : linear generative models Facial point detection using boosted regression and graph models : facial feature points are detected independently based on the response of the support vector regressor. Gaussian Process Latent Variable model : a single Gaussian is used for each facial component. Multi-State Facial Component Model of Tian and Cohn .
- 9. 1. RELATED WORK Restricted Boltzmann Machines based shape prior model: Deep Belief Networks(DBNs)-like model : S. Eslami, N. Heess, and J. Winn. (2012) - a strong model of object shape. Implicit mixture of Conditional Restricted Boltzmann Machines : G. Taylor, L. Sigal, D. Fleet, and G. Hinton (2010) - capture the human poses and motions (imRBM) under different activities such as walking, running etc
- 10. CONTENT - MEDOTHOLOGY 1. Related work 2. FrontalRBM & PoseRBM 3. Facial feature tracking based on face shape prior model 4. Experimental results
- 11. 2. FRONTAL-RBM & POSE-RBM the locations of facial feature points for frontal face when subjects show different facial expressions the corresponding locations of facial feature points for non- frontal face under the same facial expression H1 & H2 are two sets of hidden nodes
- 12. FACIAL FEATURE TRACKING BASED ON FACE SHAPE PRIOR MODEL Gaussian assumption : estimate the prior probability by calculating the mean vector p and covariance matrix p from the samples. Kernel Density Function: to estimate the probability.
- 13. CONTENT - MEDOTHOLOGY 1. Related work 2. FrontalRBM & PoseRBM 3. Facial feature tracking based on face shape prior model 4. Experimental results
- 14. RESULT Experiments on synthetic data FrontalRBM shows strong power as a face shape prior model.
- 15. RESULT Experiments on CK+ database: Error rate reduce.
- 16. RESULT Experiments on MMI database: comparable to Facial point detection using boosted regression and graph models (rate error of 5.3 on 400 images ). 19 : Robust facial feature tracking under varying face pose and facial expression (Y. Tong, Y. Wang, Z. Zhu, and Q. Ji. - Nov 2007)
- 17. RESULT Tracking under occlusion : ISL db - happiness
- 18. RESULT Tracking under occlusion : ISL db - suprised
- 19. RESULT Tracking under occlusion : ASL database, sequence 1
- 20. RESULT Tracking under occlusion :ASL database, sequence 2
- 21. CONCLUSION Improving The Accuracy And Robustness Of Facial Feature Tracking Under Simultaneous Pose And Expression Variations 1st : A face shape prior model to capture the face shape patterns under varying facial expressions for near-frontal face based on deep belief networks 2nd : Extend the frontal face prior model by a 3-way RBM to capture face shape patterns under simultaneous expression and pose variation. 3rd : Systematically combine the face prior models with image measurements of facial feature points to perform facial feature point tracking.
- 22. SWOT STRENGTH ? Experiments on many methods & do well comparing. WEAKNESS? The steps of the methods are not very clear. There is no specific correct detection rate. OPPORTUNITY ? Can be very useful for face detection related programs THREAT ? There are more than 6 basic expression. The training data must be labelled manually.
- 23. OPINION This paper has described a good method for face detection under varying expression and specially with occlusions. It is valuable for all kinds of researches related to face, to the system for interacting between human & computer , or the face recognition and to FACE ANALYSIS FOR HEALTH PURPOSE.