HANDY: A Configurable Gesture Recognition System

Mahsa Teimourikia1, Hassan Saidinejad2,,and Sara Comai3

Politecnico di Milano

1mahsa.teimourikia@polimi.it, 2hassan.saidinejad@polimi.it, 3sara.comai@polimi.it March 2014

Polo Territoriale di Como

Motivations

2

• To embed computer systems in the environments we need more natural ways of Human Computer Interactions

• Gesture-based interfaces are one of the natural ways that humans can use to interact with computers

• Gesture-based interfaces are still in an infant stage. And still not so natural!

• More fine grained gestures should be recognized

• Users should be able to define their preferred gestures

Polo Territoriale di Como

Objectives

•Design and development of a gesture-based interface, adaptable to a specific user

•Recognition of the dynamic gestures based on the posture of the hand

3

Polo Territoriale di Como

Methodology

4

Polo Territoriale di Como

Hand Localization and Segmentation

5

The environment setting:

Hand Localization and segmentation:

Polo Territoriale di Como

Feature Extraction

6

Skeletonization: Using Voronoi Diagrams on Boundary Points

Polo Territoriale di Como

Hand Pose Estimation

• Resulted skeletons are not the same even in the same hand poses

7

• Dynamic Time Warping (DTW) for pose estimation

Polo Territoriale di Como

Gesture Recognition

8

Polo Territoriale di Como

The Testing Application

9

Polo Territoriale di Como

Results

10

Average Recognition Accuracy

•Static Hand Pose Estimation : 83.53% •Gesture Recognition: 95.57%

• 7 persons, ages between 24 to 60, 5 right handed and 2 tested the system with the left hand.

Polo Territoriale di Como

Conclusions

• This work has presented an approach for a flexible hand pose and gesture recognition system that can be adapted to the special needs of the users.

The interface:

• Is configurable for different hand poses

• Acceptable accuracy

• Can be trained with minimal effort to recognize user defined gestures and sequence of poses

• Can be used in uncontrolled environments, with dynamic background and low illumination. And the user does not need to use any wearable devices.

11

Polo Territoriale di Como

Future Works

• As future work, hand pose estimator will be improved.

• Further extension of this system may include also facial expression recognition and vocal.

• This work can be used in several applications such as home automation.

• Since the proposed system is customizable it can be used for HCI purposes for people with different disabilities or needs.

14

Polo Territoriale di Como

Thank You

15