GESTURE RECOGNITION / SIGN LANGUAGE

Lukas BloderJohannes BannhoferSE09 MUS2 SS10

Overview

- Hardware- Sign Language

- Live Demo

- System Architecture- System Tools - Technologies- Problems- Fazit

Hardware

P5 GloveAPI: http://www.robotgroup.org/index.cgi/P5Glove

CyberGlove:http://www.immersion.com

http://www.golem.de/0512/42086.html

MIT Color Glove Handtrackinghttp://people.csail.mit.edu/rywang/handtracking/

Hardware

CyberGlove :

Hardware

Number of sensors: 18 or 22Sensor Resolution: 0.5 degrees (typical)Sensor Data Rate: 90 records/sec minimum (100 records/sec typical).Operating system and hosts: Windows 2000 and XPOperating Range: 30 ft radius from USB portInterface: USB port for the wireless receiver

CyberGlove II:

Sign Language

American manual alphabet

Sign Language

Substitution signs

-Dynamic signs: J, Z

Additional SignsSpace, enter, delete, various commands

Demo Time!

System

C++ API (Partially from original source of 1998)JNI Bridge

Application:Exchangeable Processing (Matlab, weka)Rules (substitution signs, comamnds)Clients (Commandline, TTS, Graphical)

System Architecture

Classification using ANN

Matlabnntool

Classification using ANN

Matlab – Erros recognizing letters

Processing Rules

Rules to process more complex signsRecognition splitted to Wrist/FingersEvaluation with rules

System Tools

- Data Collector- Data Aggregator

Technologies Used

-C++ / Java-Matlab-MaryTTS

Problems

-Old API -Matlab /generating JAR Files-API license problems-Training data-Inconsistent sensor data

Fazit

-Old Hardware still does the job -Don’t touch machine generated code-Generating good training data -> hard work

THANKS FOR YOUR ATTENTION!

Lukas BloderJohannes BannhoferSE09 PEG SS10