HUMAN COMPUTER INTERACTION USING HAND GESTURES BY AFFORDABLE ALTERNATIVE TO

DEPTH CAMERA

INTRODUCTION

For the last three decades we are stuck at the tradtional mouse keyboard setup . Recently with the introduction of touch screen in smart phones and the emergance of Augmented Reality and

Virtual Reality devices and state of the art sensors like Leap Motion and Kinect we are taking a leap

into the future of human computer interaction

THE TIMELINE

1977THE FIRST MASS MARKETED PERSONALCOMPUTER APPLE II

1982THE FIRST PC TO USE MODERN TRACKBALL BASEDMOUS

2015MODERN PC STILL USE THE THREE DECADE OLD MOUSEKEYBOARD SETUP

WHY CHANGE?

With The introduction of Virtual Reality and Augmented Reality deivecs this tradtional mouse

keyboard setup is of no use . We just can't intteract with a Virtual Reality System with a mouse and

keyboard

THE OBJECTIVE

Our Objective is to built a low cost system with help of low cost hardware and open source

software to provide robust and accurate hand gesture recognition and tracking.

THE INSPIRATION

THE CHALLENGES - HARDWARE

KINECT SENSOR

Rs 14,900Depth Camera

LEAP MOTION

Rs 8,900 Motion Sensor

LEAP MOTION

Rs 12,000 3D Sensor

OUR APPROACH - HARDWARE

IR ILLUMINATOR

Rs 120For IR Illumination

WebCam

Rs 1000For Vision Based Motion Sensing And Gesture recognistion

OUR APPROACH SOFTWARE

OPEN COMPUTER VISION LIBRARY WITH PYTHON

THE PROJECT To build a hand gesture recognition system that

doesn’t get affected by external factors such as light , distance and movements.

To build a system that can recognize with high accuracy.

To build a system that can help us interact with a computer with ease.

HOW WE DO IT HARDWARE SOFTWARE

WHY USE THIS HARDWARE SETUP

BUILD THE CORE SOFTWARE

TRAINING THE SVM MODEL AND CHECKING THE ACCURACY

THE WORK FLOW

DETECTING AND

PREDICTING THE HAND POSE

EXPERIMENTS AND RESULTS

HAND CONTROLLER

VIRTUAL WHEEL

MULTI TOUCH

LIMITATIONS AND IMPROVEMENT

LIMITATION FPS limitations cannot detect fast moving hand The System makes an assumption that the hand is

the closest and the brightest object to the camera

LIMITATION AND IMPROVEMENT

IMPROVEMENTS Use a common Light source Use a IR illuminator and a band pass filter to filter

out unnecessary background objects Use of IR illuminator to estimate depth Use of High FPS webcam as image source Using a depth camera to understand the scene Using machine learning techniques to estimate

hand pose

FUTURE IMPROVEMENTS

• FUTURE SCOPE• Using more sophisticated machine learning

techniques• Try to build a complete product .• Miniaturization of the whole system.• Running the device with least power.

THANK YOU