Personalized Bare Hand Modeling Based on Stereo
Vision and Geometry Deformation
Quan Yu State Key Lab of CAD&CG
Zhejiang University
Outline Introduction System Overview Algorithm
Input Features Alignment & Deformation
Result & Feature Work Conclusion
Introduction
Personalized Hand Modeling(NSFC, No.60970078) No markers nor gloves Low-end devices (web cameras)
Challenge Lack of strong features Lack of solution
Introduction(Cont.) Idea
Corse features come from stereo vision Fine features come from a template
Solution Deform a template under constrains of
vision data step by step
System Overview
Fig. 1 System overview. (a)extract convexity defects of contours and build a local coordinate; (b)align the template with defects and refine alignment with ICP algorithm; (c)laplacian deformation under constrains of defects (point level); (d)generate contour points with a single image; (e)laplacian deformation under constrains of contours (line level); (f)extract surface features and construct a point cloud; (g)laplacian deformation under constrains of surface points (surface level).
Input Two pairs of stereo images
front and back faces of a hand A generic template
Denote contours and defects Camera parameters
Web cameras No markers
Features: defect points Convexity defects of contours
Stable Strong
Used to determine: Size Position Alignment with the template
Features: contour points
Generate contour points from a single image Contours of left and right image are
different. Assume the depths of contour points are
constant A non-linear interpolation between defects
( , , ) ( / , / , / )X Y Z X W Y W Z W
1
x X
y YQ
d Z
W
Disparities are unknown.
1 0 0
0 1 0
0 0 0
0 0 1/ 0
x
y
x
c
cQ
f
T
Features: contour points(Cont.)
Approximate contours as Correspondences: arc length
matching
1contours= (front back)
2
mean of contours arc length matching
Features: surface points Image enhancement
Contrast-Limited Adaptive Histogram Equalization Hard to extract robust features of hand
skin. SIFT SURF GLOH ? DAISY ?
Efficient Large-Scale Stereo Matching(ACCV 2010)
Sobel responses on a regular grid
Features: surface points(Cont.)
Find correspondences Estimate normals (MLS) Project 3D points onto the template Split the template at the projection point
Features: surface points(Cont.)
Iterative deformation to eliminate outliers
Reject Threshold
correspondences
deformation iter=1
iter=2
iter=3Details or Outliers?
H=k , 0,1,2,Z
kd
Alignment
Extract defects Local coordinate
Refine: ICP Efficient Variants of the ICP Algorithm[S. R. 2001]
defect templateM M M
/defect templates L L
,arg min( )template defect
R tR P t P
Laplacian Deformation Laplacian Mesh Processing
Ogla Sorkine, 2005
Laplacian Mesh Optimization Andrew Nealen, 2006
2 2( ) 2arg min xj j
x j C
x Lx x c
Conclusion
A novel approach to construct personalized hand model with low-end equipments;
Generate 3D contour points from a single image;
Eliminate outliers with an iterative deformation.
Top Related