3d based hand gesture recognition and tracking
-
Upload
venkatesh-g-mowni -
Category
Documents
-
view
227 -
download
0
Transcript of 3d based hand gesture recognition and tracking
8/7/2019 3d based hand gesture recognition and tracking
http://slidepdf.com/reader/full/3d-based-hand-gesture-recognition-and-tracking 1/31
WELCOME TO THE TECHNICAL
SEMINAR ON
3D MODEL3D MODEL--BASED HANDBASED HAND
GESTURE RECOGNITION ANDGESTURE RECOGNITION AND
TRACKINGTRACKING
3/15/2011 1:51:27 PM
BY
VENKATESH.G(1MV06EC411)
8/7/2019 3d based hand gesture recognition and tracking
http://slidepdf.com/reader/full/3d-based-hand-gesture-recognition-and-tracking 2/31
Topic
3/15/2011 1:51:27 PM2
y 1. Introduction
y 2. Human Hand Modeling
y 3. Feature Selection and Extraction
y 4. Model-Based Hand Posture Recognition
y 5. Hand Motion Tracking
y 6. Conclusion
y
Refs.
8/7/2019 3d based hand gesture recognition and tracking
http://slidepdf.com/reader/full/3d-based-hand-gesture-recognition-and-tracking 3/31
3/15/2011 1:51:28 PM3
y Hand gestures:y Purpose of human gestures: conversational, controlling,
manipulative, and communicative.
yObjective is to track the hand in a general background and to beable to recognize dynamic gestures in real time.
y Three objectives-simulation, real world video streaming & multiplecamera data fusion.
y Suggest a road map for future development to reach the final goal.
8/7/2019 3d based hand gesture recognition and tracking
http://slidepdf.com/reader/full/3d-based-hand-gesture-recognition-and-tracking 4/31
8/7/2019 3d based hand gesture recognition and tracking
http://slidepdf.com/reader/full/3d-based-hand-gesture-recognition-and-tracking 5/31
2. Human Hand Modeling
3/15/2011 1:51:28 PM5
y Representations of a hand and 3-D model
y Human hand motion has 26 DOF
y Global configuration: six DOF, representing the pose of the hand (position
and orientation).y Local configuration: 20 angular DOF of fingers
y DIP and PIP joint each has one degree of freedom for rotation
y MCP joint has two degrees of freedom
y Finger motion constraints are applied to define the ranges each
finger may move within.
8/7/2019 3d based hand gesture recognition and tracking
http://slidepdf.com/reader/full/3d-based-hand-gesture-recognition-and-tracking 6/31
2. Human Hand Modeling
3/15/2011 1:51:28 PM6
8/7/2019 3d based hand gesture recognition and tracking
http://slidepdf.com/reader/full/3d-based-hand-gesture-recognition-and-tracking 7/31
2. Human Hand Modeling
3/15/2011 1:51:28 PM7
y Kinematic model is augmented with shape information to
generate appearances of a hand seen in 2D images
y A 3-D model has been built in graphic programming
environment as shown.y Palm is represented by a flat, chamfered rectangular
y Each segment of fingers was approximated by a sphere-ended
cylinder with a unique dimention.
y Each joint is modeled using a rotation matrix, with a pre-defined
range.
8/7/2019 3d based hand gesture recognition and tracking
http://slidepdf.com/reader/full/3d-based-hand-gesture-recognition-and-tracking 8/31
3/15/2011 1:51:28 PM8
2. Human Hand Modeling
3-D hand model:
pose1: open palm, pose2: fist
8/7/2019 3d based hand gesture recognition and tracking
http://slidepdf.com/reader/full/3d-based-hand-gesture-recognition-and-tracking 9/31
3/15/2011 1:51:28 PM9
2. Human Hand Modeling
3-D hand model:
pose3: pointing, pose4: victory
8/7/2019 3d based hand gesture recognition and tracking
http://slidepdf.com/reader/full/3d-based-hand-gesture-recognition-and-tracking 10/31
2. Human Hand Modeling
3/15/2011 1:51:28 PM10
y All 21 local DOFs are modeled with static and dynamic
constraints.
y Different fingers are color-coded just for easy identification.
Actual models will use skin color.y 2-D projections of any posture at any angle can be easily obtained
by manipulating the model in 3-D space and performing a
perspective projection.
y For global configuration, only one DOF is implemented: rotationalong virtical axis.
8/7/2019 3d based hand gesture recognition and tracking
http://slidepdf.com/reader/full/3d-based-hand-gesture-recognition-and-tracking 11/31
8/7/2019 3d based hand gesture recognition and tracking
http://slidepdf.com/reader/full/3d-based-hand-gesture-recognition-and-tracking 12/31
3/15/2011 1:51:28 PM12
3. Feature Selection and Extraction
Hand feature: silhouette images
pose1: open palm, pose2: fist
8/7/2019 3d based hand gesture recognition and tracking
http://slidepdf.com/reader/full/3d-based-hand-gesture-recognition-and-tracking 13/31
3/15/2011 1:51:28 PM13
3. Feature Selection and Extraction
Hand feature: silhouette images
pose3: pointing, pose4: victory
8/7/2019 3d based hand gesture recognition and tracking
http://slidepdf.com/reader/full/3d-based-hand-gesture-recognition-and-tracking 14/31
3. Feature Selection and Extraction
3/15/2011 1:51:28 PM14
y Skin color segmentation
y Canny edge detector (Implemented)
y Hand shape normalization (dimension)
y 3D features:
y Stereo cameras obtain 3D images
y Depth info helping for cluttered backgrounds
y Acquired surface is matched to the model surface
8/7/2019 3d based hand gesture recognition and tracking
http://slidepdf.com/reader/full/3d-based-hand-gesture-recognition-and-tracking 15/31
8/7/2019 3d based hand gesture recognition and tracking
http://slidepdf.com/reader/full/3d-based-hand-gesture-recognition-and-tracking 16/31
4. Model-Based Hand Posture
Recognition
3/15/2011 1:51:28 PM16
y Edge model of the target image is
superimposed onto the distance
image.
y Avg/Max of distance values that edge
model hits gives Chamfer Distance.
8/7/2019 3d based hand gesture recognition and tracking
http://slidepdf.com/reader/full/3d-based-hand-gesture-recognition-and-tracking 17/31
3/15/2011 1:51:28 PM17
4. Model-Based Hand Posture Recognition
An example of an image (for the V pose)
8/7/2019 3d based hand gesture recognition and tracking
http://slidepdf.com/reader/full/3d-based-hand-gesture-recognition-and-tracking 18/31
8/7/2019 3d based hand gesture recognition and tracking
http://slidepdf.com/reader/full/3d-based-hand-gesture-recognition-and-tracking 19/31
8/7/2019 3d based hand gesture recognition and tracking
http://slidepdf.com/reader/full/3d-based-hand-gesture-recognition-and-tracking 20/31
8/7/2019 3d based hand gesture recognition and tracking
http://slidepdf.com/reader/full/3d-based-hand-gesture-recognition-and-tracking 21/31
Block Diagram of the Prototype
3/15/2011 1:51:28 PM21
8/7/2019 3d based hand gesture recognition and tracking
http://slidepdf.com/reader/full/3d-based-hand-gesture-recognition-and-tracking 22/31
5. Hand Motion Tracking
3/15/2011 1:51:28 PM22
y Bayesian tracking
y Multi-resolution partitioning of the state space.
y Particle filtering
yApproximate arbitrary distributions with a set of random samples.
y Deal with clutter and ambiguous situations more effectively, by
multiple hypotheses.
y Tree-based filtering and searching
y
Cluster prototype: a group of similar shape templates.
8/7/2019 3d based hand gesture recognition and tracking
http://slidepdf.com/reader/full/3d-based-hand-gesture-recognition-and-tracking 23/31
8/7/2019 3d based hand gesture recognition and tracking
http://slidepdf.com/reader/full/3d-based-hand-gesture-recognition-and-tracking 24/31
Simulation
3/15/2011 1:51:28 PM24
Fingertips: Finger posture and Joint angles
y Finger posture: use detected finger markers to determine a
reachable range by the finger along the camera view
directiony The reachable linear segment is then sampled at constant lengths
to calculate a finger posture hypothesis .
y Thumb: binary search of a lookup table of all feasible end
positions
y Other fingers: solved by error model analysis technique
8/7/2019 3d based hand gesture recognition and tracking
http://slidepdf.com/reader/full/3d-based-hand-gesture-recognition-and-tracking 25/31
8/7/2019 3d based hand gesture recognition and tracking
http://slidepdf.com/reader/full/3d-based-hand-gesture-recognition-and-tracking 26/31
8/7/2019 3d based hand gesture recognition and tracking
http://slidepdf.com/reader/full/3d-based-hand-gesture-recognition-and-tracking 27/31
8/7/2019 3d based hand gesture recognition and tracking
http://slidepdf.com/reader/full/3d-based-hand-gesture-recognition-and-tracking 28/31
8/7/2019 3d based hand gesture recognition and tracking
http://slidepdf.com/reader/full/3d-based-hand-gesture-recognition-and-tracking 29/31
Comments:
3/15/2011 1:51:28 PM29
y A prototype of 3-D model-based hand tracking in a general
environment with unconstrained background.
y Recognize dynamic gestures in real-time.
y Dataglove is used to validate the proposed framework.y Colored markers are used to assist palm and finguretip
recognition.
8/7/2019 3d based hand gesture recognition and tracking
http://slidepdf.com/reader/full/3d-based-hand-gesture-recognition-and-tracking 30/31
8/7/2019 3d based hand gesture recognition and tracking
http://slidepdf.com/reader/full/3d-based-hand-gesture-recognition-and-tracking 31/31
3D Model-Based Hand Gesture
Recognition and Tracking
3/15/2011 1:51:28 PM3
1