Project 10 Facial Emotion Recognition Based On Mouth Analysis SSIP 08, Vienna 1 .

Project 10Facial Emotion Recognition Based On Mouth Analysis

SSIP 08, Vienna1

http://www.we-hope-project10-will-win.info

The Project

Objective : To recognize emotional state / expression using mouth information

Input: Mouth images (no make-up)

Output: Emotional State/ Expression Happy, Neutral, Sad

2


The Team

3

Kornélprogrammer

Kornélprogrammer

PéterWeb programmer

PéterWeb programmer

Kamalprogrammer

Kamalprogrammer

Naiemresearcher

Naiemresearcher

Sofiaprogrammer

Sofiaprogrammer


The Tasks

Create facial expressions photographic database

Segment the mouth in the input image

Use suitable features for expression characterization

Design a reliable classifier to distinguish between different mouth expressions

4


SSIP Lips database

Happy, Neutral and Sad Photos of SSIP students and lecturers

Thank you all!!!

Happy Neutral Sad

5


Mouth Segmentation

6

Input Image HSV Space - Hue Thresholding

Morphological Operations


Segmentation Results…

And Segmentation Problems…

7


Lips Features Extraction

Detect the leftmost and rightmost lip points

Normalize images (rotation, translation and scaling)

Calculate features Eccentricity Convex Area Minor Axis Ratio of Upper to Lower Lip

8


Expression Classification

SVM Classifier

Two Stage Classification

Mouth Features

☺

9


Results 1

Differences between different classes were found to be statistically significant (p<0.01)

Classification Accuracy Stage 1 (Sad / Not Sad) 88% Stage 2 (Happy/ Neutral) 62%

10


Results 2

11


Future Work Acquire larger database for training and testing

Test different facial expressions (such as anger and disgust)

Other classifiers: NN, FIS

Conclusion Mouth information is often insufficient for

recognizing facial expression / emotional state

Other face features such as eyes and eyebrows can contribute in emotional state recognition

12


GUI

13


References M. Gordan, C. Kotropoulos, I. Pitas, “Pseudoautomatic Lip Contour Detection Based on Edge

Direction Patterns”

J. Kim, S. Na, R. Cole, “Lip Detection Using Confidence-Based Adaptive Thresholding”

F. Tang, “Facial Expression Recognition using AAM and Local Facial Features”

M. Pantic, M. Tomc, L. Rothkrantz , “A Hybrid Approcah to Mouth Features Detection”

14


Thank you for your attention!!!

15


Project 10 Facial Emotion Recognition Based On Mouth Analysis SSIP 08, Vienna 1 .

Documents

Transcript of Project 10 Facial Emotion Recognition Based On Mouth Analysis SSIP 08, Vienna 1 .