Facial Feature DetectionFacial Feature Detection

Levente SajóUniversity of Debrecen

04.07.2008CSCS

2

Introduction

Emotion Detection

Feature Localization

Shape Templates

Human Computer InteractionHuman Computer Interaction

• In multi-modal human-computer interaction takes an important part– face detection/recognition– extracting facial features– emotion detection – age recognition

04.07.2008CSCS

3

Introduction

Emotion Detection

Feature Localization

Shape Templates

Face DetectionFace Detection

• For detecting faces, many different techniques appeared over the years– Template based– Appearance based (neural networks, SVM)

• Probably the most successful is the one based on cascaded Haar-classifiers (Boosted Cascade Detector - BCD)

• On the localized face further steps can be performed for recognizing gender, age or facial gestures

04.07.2008CSCS

4

Introduction

Emotion Detection

Feature Localization

Shape Templates

Emotion DetectionEmotion Detection

• 6 different facial emotions: neutral, happy, sad, surprised, angry, fear, disgust

• Classification methods used in face detection can be used for emotion detection, too:– Gabor-transformed image is classified using

SVM or BCD– A feature vector formed by manually defined

facial landmarks is passed to SVM classifier

04.07.2008CSCS

5

Introduction

Emotion Detection

Feature Localization

Shape Templates

Emotion DetectionEmotion Detection

• Emotion detection is sensitive for changes of illumination and different rotation of the face

• Using 2 cameras, 3D feature points can be used for constructing the feature vectors, with these more accurate classifiers can be created

04.07.2008CSCS

6

Introduction

Emotion Detection

Feature Localization

Shape Templates

Localizing Facial FeaturesLocalizing Facial Features

• Local feature detectors (SVM, BCD) can be used to detect facial features

• Since facial features contains less information then the whole face, individual feature detectors seemed to be unreliable

04.07.2008CSCS

7

Introduction

Emotion Detection

Feature Localization

Shape Templates

Localizing Facial FeaturesLocalizing Facial Features

• Shape models can be used to – reduce the number of false

detections by only selecting plausible configurations of feature matches

– correcting the false detection of the local feature detectors

• Statistical Shape Model– For each landmarks their mean

position and variance are determined

• Distance Shape Template

04.07.2008CSCS

8

Introduction

Emotion Detection

Feature Localization

Shape Templates

Distance TemplateDistance Template

• The template is described by template rules

• A rule defines the estimated distance between template points

• If a template point does not satisfy the conditions of a rule, a penalty value is calculated

• The sum of the penalties gives the overall penalty of the template

04.07.2008CSCS

9

Introduction

Emotion Detection

Feature Localization

Shape Templates

Distance TemplateDistance Template

• By replacing the feature points, the overall penalty of the template can be minimized

04.07.2008CSCS

10

Introduction

Emotion Detection

Feature Localization

Shape Templates

ConclusionConclusion

• Emotion detection is a complex task

• Single techniques proved to have several weaknesses

• Combination of techniques can result a robust emotion detection

Thanks for attention!Thanks for attention!