Wrist Recognition and the Center of the Palm Estimation Based on Depth Camera

Wrist Recognition and the Center of the Palm Estimation Based on Depth CameraZhengwei Yao ; Zhigeng Pan ; Shuchang XuVirtual Reality and Visualization (ICVRV), 2013 International Conference on1OutlineIntroductionRelated workProposed methodExperimental resultsConclusion2Introduction3IntroductionProblem: Can not separateahandfroma forearm using color and depth information

Solution: Find wrist to recognize hand

4Related Work5Related WorkHand segmentation and extractionColor [11,12]Depth threshold [13,14]The location of other body parts [15~17]WristWear wristband[14]Palm detection[18]

6Reference[13] D. Uebersax, J. Gall, and M. Van den Bergh, and L. Van Gool, Realtime sign language letter and word recognition from depth data. International Conference on Computer Vision Workshops (ICCV Workshops), 2011 [14] Z. Ren, J. Yuan, and Z. Zhang, Robust hand gesture recognition based on finger-earth mover's distance with a commodity depth camera. ACM international conference on Multimedia, 2011[15] T. I. Cerlinca and S. P. Pentiuc, Robust 3D Hand Detection for Gestures Recognition. Proc. the 5th International Symposium on Intelligent Distributed Computing, Delft, 2012[16] M. Van den Bergh and L. Van Gool, Combining RGB and ToF cameras for real-time 3D hand gesture interaction. Workshop on Applications of Computer Vision (WACV), Kona, 2011 [17] K. Fujimura and L. Xia, Sign recognition using depth image streams. Automatic Face and Gesture Recognition, 2006[18] U. Lee and J. Tanaka, Hand Controller: Image Manipulation Interface Using Fingertips and Palm Tracking with Kinect Depth Data. Proc. of 10th Asia Pacific Conference on Computer Human Interaction(APCHI), Matsue, 20127Related WorkCandescent NUI(Natural User Interface) projectHand and finger tracking Develop by Stefan Stegmueller, SwissOpen source: Use the OpenNI framework with the Kinect sensor

http://blog.candescent.ch/http://candescentnui.codeplex.com/

Finger direction detection Blue : cluster centroid Green : palm center Red : fingertips Yellow : hand contour Long lines : finger directions A Robust Method of Detecting Hand Gestures Using Depth Sensors, Yan Wen; Chuanyan Hu; Guanghui Yu; Changbo Wang, 2012 IEEE International Workshop on HAVE http://www.camdemy.com/media/11513 8Proposed Method9Proposed MethodHand segmentation and palm estimation Wrist recognitionThe center of the palm estimation

10Hand Segmentation and Palm Estimation (1/5)a. Cluster the hand dataK-means clustering algorithmSpecify the depth range: 0.5~0.8m

11Hand Segmentation and Palm Estimation (2/5)b. Compute the Convex hull of the handsThe Graham scan algorithm

12Hand Segmentation and Palm Estimation (3/5)c. Detect the hand contoursMoor-Neighbor tracking algorithm

13Hand Segmentation and Palm Estimation (4/5)d. Detect the fingertipsFind all candidate points that are both on the convex hull and the contourThe distance of P0 and P> threshold

14Hand Segmentation and Palm Estimation (5/5)e. Estimate the center of the palmThe biggest circle inside the hand contour

15Wrist RecognitionWrist: pit pointsFind an obvious pit point in the contour of hand Create an appropriate to find another wrist point. inscribed rectangle

16Wrist Recognition (1/4)Step 1: Find candidate lines of wristThe ends of the candidate line should not be both fingertips.The distance of the candidate line should not be less than a specific value.

17Wrist Recognition (2/4)Step 2: Find the corresponding candidate contours whose ends are the ends of the candidate lines.

18

Wrist Recognition (3/4)Step 3: Find one of the wrist pointsCalculate the maximum distance between the candidate line and the corresponding candidate contour.The largest distance from these maximum distances. The point with the largest distance is one of the wrist points

19

Wrist Recognition (4/4)Step 4: Find another wrist pointConnect this wrist point to each point in the hand contour, and take these connecting lines as the diagonals of rectangles. If the rectangle is not inside the hand contour, the corresponding point in the contour is not another wrist point.Find out the point with the shortest rectangle diagonal as another wrist point.

20Wrist RecognitionCandidate linesCorresponding contourFind one of the wrist pointsFind another wrist point

21Estimating the Center of the Palm (1/4)Step 1: Select three points from the hand contourThe three points (P1, P2, P3) form an acute triangle.

22Estimating the Center of the Palm (2/4)Step 2: Find circumcenter Oj of the triangleThe Oj coordinate

The radius of circle :

23Step 3: Determine the center Calculate the distances from each point in the hand contour to the center Formula.

Condition A: The number of distance Rji > Rj is bigger than the thresholdCondition B: Rj >minR (minR=> the minimum radius of the palm )If A or B is not satisfied, Step 4Estimating the Center of the Palm (3/4)

24Step 4: Find another appropriate palm centerOne end-point of these two intersectant chords is replaced by point Pmin Repeat step 2 to step 4 until the ending condition is trueEstimating the Center of the Palm (3/4)

25Proposed MethodHand segmentation and palm estimation Wrist recognitionThe center of the palm estimation

Fingertips detection

26Experimental Results27

Experimental ResultsDevice: AMD Athlon(tm)Formula Dual Core Processor Formula CPU, 4GB RAM, NVIDIA GeForce 9600GT Graphics card and Window7 32bit OSThreshold settingDepth: 0.5-0.8mMinimum distance of line: 50Minimum radius of the palm: 33#hand contour inside circle: 25~5028

Experimental Results

BeforeAfter29Experimental ResultsDivide into three groups based on the number of the points inside the hand contour.

#contour#inside contoura6257085b4705919c5057335d107411205e135612362f115513047g140716716h147817961i10241899030Experimental Results

31

31Experimental ResultsImproved original algorithm: every 8th pointThe new algorithm: proposed method

32Conclusion33ConclusionPropose the wrist recognition algorithm to separate the hand from the forearm, Propose a new algorithm of estimating the center of the palm to reduce the computing time. Without Kinect SDK34