JAMES COOK AUSTRALIA INSTITUTE OF HIGHER LEARNING

IN SINGAPORE

HEALTH DIAGNOSTIC BY ANALYSING FACE IMAGES USING MOBILE DEVICES

Instructor : Dr. Insu Song

Student : Ho Thi Hoang YenEmail: hyenho91@gmail.com

INTRODUCTION

Previously : A robust, highly accurate method for detecting 20 facial points in images of

expressionless faces

BACKGROUND

A restricted Boltzmann machine (RBM) is a generative stochastic artificial neural network that can learn a probability distribution over its set of inputs.

Inventor : under the name Harmonium by Paul Smolensky in 1986.Fast learning algorithms : mid-2000s by Geoffrey Hinton & collaborators.

RBMs have found applications in dimensionality reduction, classification, collaborative filtering, feature learning and topic modelling.

INTRODUCTION

Track 26 feature points

different facial expressions, varying poses, or occlusion

INTRODUCTIONOther methods : - track facial feature points independently or - build a shape model to capture the variations of face shape or - appearance regardless of the facial expressions and face poses

This method : capture the distinctions & variations of face shapes due to facial expression and pose change in a UNIFIED framework

CONTENT - MEDOTHOLOGY

1. Related work2. FrontalRBM & PoseRBM3. Facial feature tracking based on face shape prior

model 4. Experimental results

1. RELATED WORKFacial feature localization: 2 categories :

• Without shape prior models : track each facial feature point independently and ignore the prior knowledge about the face => sensitive with expression & pose

• With shape prior models : capture the dependence between facial feature points by explicitly modeling the general properties as well as the variations of facial shape or appearance

1. RELATED WORK

Facial feature localization:Recently methods:

• Active Shape Model (ASM) [2] and Active Appearance Model (AAM) : linear generative models

• Facial point detection using boosted regression and graph models : facial feature points are detected independently based on the response of the support vector regressor.

• Gaussian Process Latent Variable model : a single Gaussian is used for each facial component.

• Multi-State Facial Component Model of Tian and Cohn• ….

1. RELATED WORK

Restricted Boltzmann Machines based shape prior model:• Deep Belief Networks(DBNs)-like model :  S. Eslami, N.

Heess, and J. Winn. (2012) - a strong model of object shape.

• Implicit mixture of Conditional Restricted Boltzmann Machines :  G. Taylor, L. Sigal, D. Fleet, and G. Hinton (2010) - capture the human poses and motions (imRBM) under different activities such as walking, running etc

• …

CONTENT - MEDOTHOLOGY

1. Related work2. FrontalRBM & PoseRBM3. Facial feature tracking based on face shape prior

model 4. Experimental results

2. FRONTAL-RBM & POSE-RBM

the locations of facial feature points for frontal face when subjects show different facial expressions

the corresponding locations of facial feature points for non-frontal face under the same facial expression

H1 & H2 are two sets of hidden nodes

FACIAL FEATURE TRACKING BASED ON FACE SHAPE PRIOR MODEL

Gaussian assumption : estimate the prior probability by calculating the mean vector μp and covariance matrix Σp from the samples.

Kernel Density Function: to estimate the probability.

CONTENT - MEDOTHOLOGY

1. Related work2. FrontalRBM & PoseRBM3. Facial feature tracking based on face shape prior

model 4. Experimental results

RESULT

Experiments on synthetic data

FrontalRBM shows strong power as a face shape prior model.

RESULT

Experiments on CK+ database: Error rate reduce.

RESULT

Experiments on MMI database: comparable to Facial point detection using boosted regression and graph models (rate error of 5.3 on 400 images ).

19 : Robust facial feature tracking under varying face pose and facial expression (Y. Tong, Y. Wang, Z. Zhu, and Q. Ji. - Nov 2007)

RESULTTracking under occlusion : ISL db - happiness

RESULTTracking under occlusion : ISL db - suprised

RESULTTracking under occlusion : ASL database, sequence 1

RESULTTracking under occlusion :ASL database, sequence 2

CONCLUSION• Improving The Accuracy And Robustness Of Facial Feature Tracking

Under Simultaneous Pose And Expression Variations

• 1st : A face shape prior model to capture the face shape patterns under

varying facial expressions for near-frontal face based on deep belief

networks

• 2nd : Extend the frontal face prior model by a 3-way RBM to capture face

shape patterns under simultaneous expression and pose variation.

• 3rd : Systematically combine the face prior models with image

measurements of facial feature points to perform facial feature point

tracking.

SWOT• STRENGTH ?Þ Experiments on many methods & do well comparing.

• WEAKNESS?

Þ The steps of the methods are not very clear.

Þ There is no specific correct detection rate.

• OPPORTUNITY ?

Þ Can be very useful for face detection related programs

• THREAT ?

Þ There are more than 6 basic expression.

ÞThe training data must be labelled manually.

OPINION

This paper has described a good method for face

detection under varying expression and specially with

occlusions.

It is valuable for all kinds of researches related to face, to

the system for interacting between human & computer , or the

face recognition and to FACE ANALYSIS FOR HEALTH

PURPOSE.

THANK YOU