ASL using dynamic gesture recognisation
-
Upload
vinod-bazari -
Category
Documents
-
view
220 -
download
0
Transcript of ASL using dynamic gesture recognisation
-
7/29/2019 ASL using dynamic gesture recognisation
1/30
PRESENTED
BY:-
-
7/29/2019 ASL using dynamic gesture recognisation
2/30
IIT-BHU.
Indian Institute of Technology (BHU)Varanasi [IIT(BHU)], formerly knownas Institute of Technology, BanarasHindu University (IT-BHU), is apublic engineering institution locatedin Varanasi, Uttar Pradesh, India.Founded in 1919 as part ofthe Banaras Hindu University, it wasdesignated anIndian Institute ofTechnology in 2012. IIT(BHU) has 13departments and three inter-
disciplinary schools.
-
7/29/2019 ASL using dynamic gesture recognisation
3/30
Sixth Sense Technology..
Sixth Sense allows people to use thecomputer without a screen or a keyboard
It can turn any surface into a touch screen It is made with store-bought components The software program processes the video
stream data captured by the camera andtracks the locations of the colored markers[that you wear on your fingers] using simple
computer-vision techniques(http://www.pranavmistry.com/projects/sixthsense
/index.htm#ABOUT) According to a number of sources, graduate
students at MIT have been working on this forthe past four months day and night.
http://www.pranavmistry.com/projects/sixthsense/index.htmhttp://www.pranavmistry.com/projects/sixthsense/index.htmhttp://www.pranavmistry.com/projects/sixthsense/index.htmhttp://www.pranavmistry.com/projects/sixthsense/index.htmhttp://www.pranavmistry.com/projects/sixthsense/index.htmhttp://www.pranavmistry.com/projects/sixthsense/index.htmhttp://www.pranavmistry.com/projects/sixthsense/index.htmhttp://www.pranavmistry.com/projects/sixthsense/index.htm -
7/29/2019 ASL using dynamic gesture recognisation
4/30
AT A GLANCE.
The device recognizes items in stores, and thengathers and projects information about productsto the user. It also is able to provide quick signalsto let users know what the best products arebased on their needs.
Another amazing feature is the devices ability tolook at an airplane ticket and let the user knowwhether the flight is delayed or on time. It is alsoable to recognize books (based on an image orRFID) in a book store and then project readerreviews or author information from the Internetonto blank pages.
The device can also recognize articles innewspapers, retrieve the latest related stories orvideo from the Internet and then display them onpages for the user. Say goodbye to yourdictionary and encyclopedia
-
7/29/2019 ASL using dynamic gesture recognisation
5/30
Educational Use
Engagement
Research
Collaboration
Cost-effectiveness
-
7/29/2019 ASL using dynamic gesture recognisation
6/30
Engagement
.
If this technology does nothing else, the idea thatstudents can simply use their hands to draw images
on walls and take pictures with their hands will be (atleast initially) quite motivating
-
7/29/2019 ASL using dynamic gesture recognisation
7/30
Research
Sixth Sense will give new meaning tolooking something up.
Since we are essentially talking about
the internet, students will be given theopportunity to portably research theworld.
They can all go on a virtual field trip
and project the images on their owndesks, for example
-
7/29/2019 ASL using dynamic gesture recognisation
8/30
Collaboration
Students will be given an opportunitythrough this technology to feel moreconnected to others.
Although there does seem to be thesense that individuals could walk throughlife wired and lonely, I like to think theclassroom would serve as theenvironment in which students andteachers using Sixth Sense would wantto share.
Not only could the sharing take place inthe classroom, I suppose since the usersare connected online, sharing could takeplace around the world.
-
7/29/2019 ASL using dynamic gesture recognisation
9/30
Cost-effectivenes
As mentioned earlier, this productcurrently costs only $350. From aneducators stand point, this is
minimum given the amount of moneyspent on textbooks, computer labsand LCD projectors.
For educators, Sixth Sense would be
a way to put technology (literally) intoeach students hands, because of itssize and its cost.
-
7/29/2019 ASL using dynamic gesture recognisation
10/30
Sign Language
Used primarily by hearing-impairedpeople,
Uses a different medium: hands, face,
and eyes (rather than vocal tract orears).
NOT derived from spoken language
Study ofsign language providesunique insight into the nature oflanguage itself.
Brain function similarities indicate
-
7/29/2019 ASL using dynamic gesture recognisation
11/30
American sign language
-
7/29/2019 ASL using dynamic gesture recognisation
12/30
Structure of ASL
Five Basic Parameters Shape of the hand
Place of articulation (location)
Movement Palm Orientation Region of the hand contacting body
Orientation of the hand to body
Orientation of hands to each other Facial Expressions
-
7/29/2019 ASL using dynamic gesture recognisation
13/30
INTRODUCTION TO IMAGE PROCESSING
1 pixel
-
7/29/2019 ASL using dynamic gesture recognisation
14/30
Types Of Image
GREY SCALE IMAGE (IntensityImage
-
7/29/2019 ASL using dynamic gesture recognisation
15/30
RGB IMAGE
-
7/29/2019 ASL using dynamic gesture recognisation
16/30
Black and White Image(BinaryImage)
-
7/29/2019 ASL using dynamic gesture recognisation
17/30
Conversion of Image
-
7/29/2019 ASL using dynamic gesture recognisation
18/30
DEMO
-
7/29/2019 ASL using dynamic gesture recognisation
19/30
Dynamic gesture and Computervision
Human-computer interaction (HCI) has become anincreasingly important part of our daily lives.Keyboards and mice are the most popular mode ofHCI.
Virtual Reality and Wearable Computing requirenovel interaction modalities.
in a way that humans communicate with each other.
Hand gesture is a natural and intuitive
communication mode.Other applications: Sign Language Recognition,video transmission, and so on.
C t Vi i i
-
7/29/2019 ASL using dynamic gesture recognisation
20/30
Computer Vision usinggesture
Replace mouse and keyboardPointing gesturesNavigate in a virtual environmentPick up and manipulate virtual objectsInteract with a 3D worldNo physical contact with computer
Communicate at a distance
-
7/29/2019 ASL using dynamic gesture recognisation
21/30
Gesture for Human-computerInteraction
-
7/29/2019 ASL using dynamic gesture recognisation
22/30
Gesture
What is a gesture? Body motion used for communication.
There are different types of gestures.
Hand gestures (e.g., waving goodbye). Head gestures (e.g., nodding).
Body gestures (e.g., kicking).
Example applications: Human-computer interaction. Controlling robots, appliances, via gestures.
Sign language recognition.
-
7/29/2019 ASL using dynamic gesture recognisation
23/30
Dynamic Gestures
-
7/29/2019 ASL using dynamic gesture recognisation
24/30
Gesture Analysis
1 Gesture detection and featureextraction
- skin color clues based approaches
- motion clues based approaches- multiple clues based approaches
- features include gray image, binary
silhouette, moving region, edge, contour,and so on.
-
7/29/2019 ASL using dynamic gesture recognisation
25/30
Decomposing GestureRecognition We need modules for:
Computing how the person moved. Person detection/tracking.
Hand detection/tracking.
Articulated tracking (tracking each body part). Handshape recognition.
Recognizing what the motion means.
Motion estimation and recognition are
quite different tasks. When we see someone signing in ASL, we
know how they move, but not what themotion means.
-
7/29/2019 ASL using dynamic gesture recognisation
26/30
GESTURE SPOTTING
Obtaining the trajectory of dynamic gesture fromeach frame
Applying mean shift algorithm for each block.
Comparing the out with data base made bycontinues samples
Finally giving the out put in the voice form.
21 3
4
56 7 8
2
1
34
5
7 8 9
6
-
7/29/2019 ASL using dynamic gesture recognisation
27/30
Project in a glance.
Hand Detection +Feature Extraction
Gesture Models+Mean shift algorithm
Spotting Rules +Trajectory comparing
Input trajectory Spotting
VideoStream Feature
Vector
Gesture id,sound output
-
7/29/2019 ASL using dynamic gesture recognisation
28/30
About my professor
PhD
ProfessorDepartment of ElectricalEngineeringInstitute of Technology, BHU
Dr. S. K. Nagar
Areas of Interest:Control Systems
AI Applications
-
7/29/2019 ASL using dynamic gesture recognisation
29/30
Career at IIT
Summer cum research intern with thestipend of 7000+HRA.
Project Associate for MHRD projects
accomplished by IIT with pay of10,000+HRA.
JRF for particular field with pay of
18000+HRA. SRF with pay of 26000+HRA.
-
7/29/2019 ASL using dynamic gesture recognisation
30/30
ANY QUERies??
THANK YOU.