Hand motion and gesture recognition system for PCHand motion and gesture

recognition system for PC

by Gayan Denaindra perera (cb005044)

Line up….

• Problem overview.• Solution and objective.• Application scope.• Similar system comparison.• Implementation .• Techniques are used for implementing.• Strength of this system.• Conclusion and future improvements .

Problem Overiew

• Reduceing the distance between human and computer and filling the gap by new input modality .

• Controling pc where user’s hand become busy .

Solution and Objectives

Recognize Hand Gestures, Motion and interact with the PC application.

• Memorable gestures• Applications that mostly using toady. • Provide 90% accuracy than other systems

Similar Systems Features

Application

Available gestures

Available motion

Graphical user

interface

Support application

Accuracy User friendly

Flutter 1 0

Point grab 2 4Wave Control 1 2Control Air 2 2ADM gesture control

1 4

Samsung Smart TV

4 4

Proposed approach

6 4

Capture image Smoothing Subtraction Color

conversion Grayscale Threshold

Remove arm Calculate total pixels

Detect thumb

Extract features

Recognize posture

Calculates x and y

coordinates

Calculates new x and y coordinates

Detect motion

Check motion count

Recognize motion

Hand Detection

Gesture Recognition

Motion Detection and Recognition

Call Command

Hand gestures

Selected hand gestures

• Gestures should be memorable.

• Flexible.• Simple.

Selected hand motion

Horizontal

Vertical

Straight horizontal

Straight vertical

Curve horizontal Zigzag horizontal

Curve vertical Zigzag vertical

Selected application and controls

Play, Pause, forward and backword

Play, Pause, forward and backword

forward and backword

Answer call, Mute mic and

ignore and hang-up

Volume up and down, Mic

mute

forward and backword,zoo

m-in and zoom out

22 Commands

Capture devices

Leap motion

• 30 fps• 3 MP• Friendly price• HD 720p

Logitech c270

Motion detection

and recognitio

n

Gesture recogniti

onHand

detection

Implemetation

Image smoothing

• Gaussian blur and median filters.

Smoothing Subtraction Color conversion Grayscale Threshold

Background substrction

• Back-ground subtrction.• Temporal detection.• Optical flow.

Smoothing Subtraction Color conversion Grayscale Threshold

Colour space convertion

• YCbCr and HSV.

Smoothing Subtraction Color conversion Grayscale Threshold

Image thresholding   

• Otus algorihtum and K-means algorithum.

Smoothing Subtraction Color conversion Grayscale Threshold

Gesture recognition trditional method

Gesture recogniti

onHand

detection

Gesture recognition

400 for detecting400 for recognizing

400+ (400 x 6) Total templates2800

Gesture recognition Author’s approach

Remove arm Calculate total pixels

Detect thumb

Extract features

Recognize posture

1 2 3 4 5

25%25%

25%

How author’s algorithm works

Not depending object size

• Detect thumb region• Detect feature

regions

Thumb region given by{12,13,12.3,11.89,12.7,13.1,12.8,12.9,13,12.1,12.4}Average – 12.5 ==== 12 Max value - 13.1 %Mini value – 11.89 %

Diversity of Hand

Motion detection and recognition

Calculates x and y

coordinates

Calculates new x and y coordinates

Detect motion

Check motion count

Recognize motion

1 2 3 4 5

Testing

• Unit testing.• Scenario testing.• Performance testing.• Scalability testing.• Environment testing • Accuracy testing.

8000 +10

0

Accuracy and Environment testing

Scenario Actual result (millisecond) Expected result (millisecond)

Hand detection 620-340 700Gesture recognition

980-655 1000

Motion detection 1166-876 1200

Motion recognition

366-244 500

Overall performance

3164-2136 3500

Test Evaluation

Conclusions and future improvement• Successfully managed the all user requirements . • Introduce new algorithm .• Introduce innovative system .

• Training the system for all universal hand diversities • Add features for disable peoples.• Add voice recognition .