A Recognition Method of RestrictedA Recognition Method of Restricted Hand Shapes in Still Image and Moving Image Hand Shapes in Still Image and Moving Image

as a Man-Machine Interfaceas a Man-Machine Interface

Speaker : Meng-Shun SuSpeaker : Meng-Shun SuAdviser : Chih-Hung LinAdviser : Chih-Hung Lin Ten-Chuan HsiaoTen-Chuan HsiaoDate : 2010/03/23Date : 2010/03/23

©2010 STUT.©2010 STUT. CSIE.CSIE. Multimedia and Information Security Lab. J205-Multimedia and Information Security Lab. J205-11

Human System Interactions, 2008 Conference onNobuharu Yasukochi, Aya Mitome, and Rokuya Ishii, Fellow, IEEE

Yokohama National University, Yokohama, Japan

OutlineOutline

Experimental ResultsExperimental Results33

IntroductionIntroduction11

Method DescriptionMethod Description22

ConclusionsConclusions44©2010 STUT.©2010 STUT. CSIE.CSIE. Multimedia and Information Security Lab. J205-Multimedia and Information Security Lab. J205-11

2

IntroductionIntroduction11

©2010 STUT.©2010 STUT. CSIE.CSIE. Multimedia and Information Security Lab. J205-Multimedia and Information Security Lab. J205-11

Some interfaces employ to wear magnetic sensor, use infrared camera, multi-cameras, and special input device with marker and grove.

Those interfaces are very expensive & expensive & complicatedcomplicated, and not suitable for everyone's use.

3

IntroductionIntroduction11

©2010 STUT.©2010 STUT. CSIE.CSIE. Multimedia and Information Security Lab. J205-Multimedia and Information Security Lab. J205-11

Featuring a simple user interface, this paper presents a simple recognition algorithm of restricted hand shapes from an image taken by only a (not multi-) camera.

The proposed method can be divided into two parts: one is the hand region extraction process hand region extraction process from an input imagefrom an input image; another is the hand shape hand shape recognition process from the extracted image.recognition process from the extracted image.

4

IntroductionIntroduction11

©2010 STUT.©2010 STUT. CSIE.CSIE. Multimedia and Information Security Lab. J205-Multimedia and Information Security Lab. J205-11

In the hand shape recognition process, we first make a mask image from the extracted hand region, and we recognize hand shapes based on the image with uneven hand surface by using the mask.

The effectiveness of the proposed method is evaluated by recognition success rate and computation time.

5

©2010 STUT.©2010 STUT. CSIE.CSIE. Multimedia and Information Security Lab. J205-Multimedia and Information Security Lab. J205-11

Method DescriptionMethod Description22 Hand Region ExtractionHand Region Extraction

RGB values of hand region and grayscale background region have the following relationship.

BGR In hand region.

BGR In background region.

6

©2010 STUT.©2010 STUT. CSIE.CSIE. Multimedia and Information Security Lab. J205-Multimedia and Information Security Lab. J205-11

Method DescriptionMethod Description22 Hand Region ExtractionHand Region Extraction

Therefore, an input image (RGB: 256 values) is transformed into (511 values) pxV

2552

BGRVpx

7

©2010 STUT.©2010 STUT. CSIE.CSIE. Multimedia and Information Security Lab. J205-Multimedia and Information Security Lab. J205-11

Method DescriptionMethod Description22 Hand Region ExtractionHand Region Extraction

Fig.1.Distribution of reference images in RGB color space.

Calculate skin color vector. is determined such that 95% of skin color pixels.

2thd

8

©2010 STUT.©2010 STUT. CSIE.CSIE. Multimedia and Information Security Lab. J205-Multimedia and Information Security Lab. J205-11

Method DescriptionMethod Description22

For recognition of finger shapes, we employ the luminosity Value in HSV color space. The value R for each pixel in a recognized image is calculated.

4)511(

ValueV

R px

pxV = 門檻Value = HSV色彩空間亮度值

A. Marking a mask image

9

©2010 STUT.©2010 STUT. CSIE.CSIE. Multimedia and Information Security Lab. J205-Multimedia and Information Security Lab. J205-11

Method DescriptionMethod Description22

Fig.3.Distribution of values R for pixels in recognized hand shape image

10

©2010 STUT.©2010 STUT. CSIE.CSIE. Multimedia and Information Security Lab. J205-Multimedia and Information Security Lab. J205-11

Method DescriptionMethod Description22

B. Normalization of values of pixels in a hand shape region

1) Position normalization2) Angle normalization 3) Eliminating a wrist region 4) Normalization (size)

11

©2010 STUT.©2010 STUT. CSIE.CSIE. Multimedia and Information Security Lab. J205-Multimedia and Information Security Lab. J205-11

Method DescriptionMethod Description22

C. Hand Shape Recognition Algorithm

Here we prepare an ellipse curve that crosses all the fingers. The number of fingers can be detected from the pixel value R on the ellipse curve. Then hand shapes can be recognized by the angle of fingers from List point.

12

©2010 STUT.©2010 STUT. CSIE.CSIE. Multimedia and Information Security Lab. J205-Multimedia and Information Security Lab. J205-11

Method DescriptionMethod Description22

C. Hand Shape Recognition Algorithm

Fig.5.Recognition algorithm. 13

©2010 STUT.©2010 STUT. CSIE.CSIE. Multimedia and Information Security Lab. J205-Multimedia and Information Security Lab. J205-11

Experimental ResultsExperimental Results33

A. Recognition Rate

We evaluate the recognition rate. Recognition ratio is given by:

[%]100#

#. frames of

sized frameof recognaratiorecog

14

©2010 STUT.©2010 STUT. CSIE.CSIE. Multimedia and Information Security Lab. J205-Multimedia and Information Security Lab. J205-11

Experimental ResultsExperimental Results33

In the case of the grayscale background, average recognition rate was 96.8%.

Note that the recognition cannot be successful when the background contains skin color. That problem still remains as one of future studies.

The recognition rate by the method of Ref. [3] was 87.9% for grayscale, and 83.1% for color images. The proposed algorithm achieves higher recognition rate, the reason would be that Ref. [3] did not employ normalization process.

15

©2010 STUT.©2010 STUT. CSIE.CSIE. Multimedia and Information Security Lab. J205-Multimedia and Information Security Lab. J205-11

Experimental ResultsExperimental Results33

B. Processing Speed

we see that the total recognition processing time is about 30[ms] at most, which can be considered as fast enough processing time to be used as hand shape recognition system.

16

©2010 STUT.©2010 STUT. CSIE.CSIE. Multimedia and Information Security Lab. J205-Multimedia and Information Security Lab. J205-11

ConclusionsConclusions44

The processing time was around several 10 of milliseconds which can be regarded enough to recognize a hand shape. So, this method enables real-time hand shape recognition.

The present algorithm could recognize 9 hand shapes at the accuracy of 96.8% for the case of grayscale backgrounds.

Cannot be successful when the background contains skin color.

17

©2010 STUT.©2010 STUT. CSIE.CSIE. Multimedia and Information Security Lab. J205-Multimedia and Information Security Lab. J205-11