MALAYSIA SIGN LANGUAGE RECOGNITION USING IMAGE PROCESSINGBy: Siti Nor Nabilah binti Yahya

HK2006-2096

OVERVIEW

Convert sign language into verbal sound

Plot the area of the gesture by using a color glove

For project 1, only try to detect color and gesture in static form

OBJECTIVE

To develope software for converting sign language into speech(verbal sound).

LITRETURE REVIEW

COLOR REGION DETECTION

In sign language, hand is the important part of human body that delivered message to others.

Yellow glove is use to cover our hand and from this idea, we are detecting gesture by detecting the color of the glove.

Thus, message is easy to be converting.

RGB COLOR MODEL

The RGB color model is an additive color model in which red, green, and blue light are added together in various ways to reproduce a broad array of colors.

RGB allows the user to access a set of 21 colors via their common English names.

COLOR SEGMENTATION

RGB color space is no very efficient for the detection of skin pixels but efficient for color segmentation

METHODOLOGIES

DESCRIPTION OF GLOVE COLOR REGION SEGMENTATION

Using glove to mark the movement of the gesture

Used one color of glove to specify the selected area

To avoid from noise, image is taken start from neck.

METHOD OF DETECTING COLOR REGION

RGB data method

Day light color illumination

RGB DATA METHOD

After converting video into image, the RGB from the image can be classified.

From the RGB data we are taken, we make a calculation make small conclusion for the RGB data

For example:G>R>B

From it, we can say that G is the biggest and B is the smaller

Find every minimum data for R, G and B for each frame

FLASHLIGHT OR DAYLIGHT LATERAL ILLUMINATION

Using equation below and applied in MATLAB code

R>(min value of R from data taken), G>(min value of G from data taken), B>(min value of B from data taken)

|R-G|255(since maximum value of color in RGB is 255), B<R, B<G

C=R(R)+G(G)+B(B)

RESULT

OVERVIEW RESULT

After convert it into frames, we choose different gesture to be analyze

Result of the color detection is also come to the failure result

Cause by the problem occur from the video and also from the surrounding such as background color, color of cloth, overlapping gesture and also blurry image

result is divide into two that is success case and also failure case

COLOR SEGMENTATION USING RGB DATA

23<R<25537<G<2552<B<255

This is the range we get from the experiment

SUCCESS CASE

FAILURE CASE

COLOR SEGMENTATION USING RGB DAYLIGHT

Result for this method also divide into two group. Success and fail.

SUCCESS

FAILURE CASE

DISCUSSION FOR RESULT FROM TWO METHOD From the test we are doing, we can see the result of the

segmented color as below:a) Some detection area contain holeb) Overlapping gesture is hard to detectc) Bright background may affect the result of color

detections; result contains noised) Surrounding color cannot be almost same like glove or

else it will give a noise to the result or the result will be fail

e) It is better to take more reading of RGB. This because is better if we could get the minimum value of RGB of the image

f) Unbalance light while taking the image may affect the result in detection color process

g) Blurry image may cause the gesture cannot be detected

CONCLUSION

Archive a good result for this stage Cause by using the method we can detect the

gesture without noise after make an analysis from the result.

In future, there many things we can improve refer from the failure case

By detecting color region is the first step before we can detect the motion of a gesture

THANK YOU…..