Dynamic hand gesture recognition using cbir

International Journal of Computer Engineering and Technology (IJCET), ISSN 0976-

6367(Print), ISSN 0976 – 6375(Online) Volume 4, Issue 3, May – June (2013), © IAEME

340

DYNAMIC HAND GESTURE RECOGNITION USING CBIR

Mr.Shivamurthy.R.C

Research Scholar, Department of Computer Science & Engineering

Akshaya Institute of Technology, Tumkur-572106, Karnataka, India

Dr. B.P. Mallikarjunaswamy

Professor, Department of Computer Science & Engg,

Sri Siddharatha Institute of Technology, Maralur, Tumkur: 572105, Karnataka, India

Mr.Pradeep kumar B.P.

Department of Electronics & communication Engineering

Akshaya Institute of Technology, Tumkur-572106, Karnataka, India

ABSTRACT

Image Databases and archives provide lot of research areas. Significant among them

is ; The Contentbased image retrieval (CBIR) research area for manipulating large amount of

image databases and archives. CBIR is mainly based on the way that the image is extracted.

The main focus of the proposed system is on the color and shape feature extractions for hand

tracking that is intended as a step towards palm and face tracking for a perceptual user

interface. This paper extends a default implementation to allow tracking on type of feature

spaces and arbitrary number by reviewing the k-means clustering algorithm. We weigh the

multidimensional histogram with a simple monotonically decreasing kernel profile prior to

histogram back projection in order to compute the new portability that a pixel value belongs

to the target model. We examine the effectiveness of the K-means clustering algorithm as a

general-purpose hand and face tracking approach in the case where no assumptions have been

made about the palm to tracked in this paper image retrieval

Keywords: Content–based image retrieval (CBIR), histogram back projection, k means

clustering algorithm

INTERNATIONAL JOURNAL OF COMPUTER ENGINEERING

& TECHNOLOGY (IJCET)

ISSN 0976 – 6367(Print) ISSN 0976 – 6375(Online) Volume 4, Issue 3, May-June (2013), pp. 340-352 © IAEME: www.iaeme.com/ijcet.asp Journal Impact Factor (2013): 6.1302 (Calculated by GISI) www.jifactor.com

IJCET

© I A E M E



341

1.INTRODUCTION

There has been a significant growth in the IT field for medical imaging where various

techniques and process are involved to create images of human body for clinical procedure.

This rapid growth has kept the medical science at a very high level. The developments of

large database of medical image are the results of collaborative approaches of handling

medical procedures. An intelligent, fast and accurate medical image retrieval system would

be the ultimate goal of medical imaging for it to be succeeded. With rapidly growing data in

size and semantically distinguishable as the features of various medical images are fuzzy in

nature for different organs, the retrieval system should be very adaptive.

As compared to general purpose images (GPI), the medical images are distinguished

in its characteristics. Hence, in medical image processing systems the process adopted for

searching GPI cannot be adopted.The most visually similar images to a give query image

from a database of images are importantly adopted in context-based image retrieval (CBIR).

CBIR will assist him/her in diagnosis; it does not target at replacing the physician by

predicting the disease of a particular case. The diagnostic information can be derived by the

visual characteristics of a disease. Sometimes it can also be derived out of similar images

correspond to the same disease category. The physician can use the output of CBIR system to

obtain more confidence in his/her decision or to consider other possibilities as well.

The advances in CBIR systems have led the researchers for new approaches in

information retrieval for image databases. It has already met some degree of success in

constrained problems in medical applications.Not withstanding the progress already achieved

in the few frameworks available here is still a lot of work to be done in order to develop a

commercial system able to fulfill image retrieval/diagnosis comprehending a broader image

domain.

Recently, advances in Content Based Image Retrieval prompted researchers towards

new approaches in information retrieval for image databases. In medical applications it

already met some degree of success in constrained problems. The generic framework is

shown in Figure 1.

Figure 1: Diagram for content-based image retrieval system



342

Notwithstanding the progress already achieved in the few frameworks available here

is still a lot of work to be done in order to develop a commercial system able to fulfil image

retrieval/diagnosis comprehending a broader image domain.

2.METHODS USED FOR IMPLEMENTING CBIR

2.1Shape based Method The image feature extracted for shape based image retrieval is usually an N-

dimensional feature vector that can be regarded as a point in N-dimensional space. After the

images are stored into the database using the extracted feature vectors, the image will be

retieved to determine the similarity between the query image and the target images in the

databases. This is essentially the determination of distance between the feature vectors that

represents the images.

The desirable distance measure should reflect human perception. In image retrieval

various similarity measure have been exploited. We have used Euclidean distance for

similarity measurement in our implementation.

2.2Texture Based Method We find larger variety of texture measures as compared to color measures. Wavelets

and Gabor filters are some of the common measures for capturing the texture of images

where Gabor filters perform better and correspond well to. The characteristics of images or

image parts with respect to changes in certain directions and scale of changes are capture by

the texture measure. This is most useful for regions or images with homogeneous texture.

2.3Using Low-Level Visual Features Preprocessing phase and retrieval phase are the two main phases of the image

retrieval process. The description of of both phases as follows.

A feature extraction model and a classification model are the two main components of

pre-processing phase.The original image database i.e. images from the image CLEF medical

collection, with more than 66,000 medical images is the input of the preprocessing phase. An

index relating each image to its modularity and a feature database is the output of the

preprocessing phase.

3.LOW LEVEL IMAGE FEATURE

Content-based image retrieval is based principally on low level image feature. As it

has been found that users are usually more interested in specific region rather than entire

image, most current content-based image retrieval systems are region based. This means that

the image is divided in regions on whuich the other operation are

performed. To carry out this first step we need to perform a segmentation of the original

image. To specify queries we will consider several classes of features: color, texture, shape.

3.1 Image segmentation Automatic image segmentation is a difficult task to compute. A lot of techniques have

been proposed in the past, such as curve evolution [11] and graph partitioning [12]. A lot of

known techniques works well for images that have homogeneous color regions. Let's see an

example of segmentation on one picture in figure 2. However, picture from the real world are



343

richer of color and shades. In literature there are know many segmentation algorithm, like

JSEG [10], blobworld [9] or KMCC [13].

Figure 2: Example of segmentation of a picture. (the red lines divided the regions)

3.2 Color feature Color features are easy to obtain, often directly from the pixels intensities like color

histogram over the whole image, over a fixed sub image, or over a segmented region. It is one

of the most used features in image retrieval. The colors are described by their color space:

RGB, LAB, LUV, HSV,

RGB is the best known space color and is it commonly used for visualization. The acronym

stands for Red Green Blue. This space color can be seen as a cube where the horizontal x-axis

as red values increasing to the left, y-axis as blue increasing to the lower right and the vertical

z-axis as green increasing towards the top, as in figure 3.

Figure 3: The RGB color model mapped to a cube.

The origin, black, is hidden behind the cube. RGB is a convenient color model for

computer graphics because the human visual system works in a way that is similar, though

not quite identical, to an RGB color space.

Another famous space color is the HSV. The acronym stands for Hue Saturation

Value. Refering to the image 3 we can see the color space as a cylinder, where the angle

around the central vertical axis corresponds to hue, the distance from the axis corresponds to

saturation, and the distance along the axis corresponds to value (also called brightness).



344

Figure 4: The HSV space color.

3.3 Shape feature There's no universal definition of what a shape is. Impressions of shape can be

conveyed by color or intensity patterns, or texture, from which a geometrical representation

can be derived, like Plato's Meno, where a Socratic dialog is made around the word “figure".

Figure is the only existing thing that is found always following color – Socrate. Shape feature

(like aspect ratio, circularity, Fourier descriptor, consecutive boundary segments, . . . ) are

very important image feature, even if they are not so commonly used in Region-Based Image

Retrieval Systems. Due to the inaccuracy of segmentation step, it is more difficult to apply

shape features instead of color or texture feature. However in literature are know some

Content-Based Image Retrieval Systems that use this feature, like [15], [13] e [14].

4. INTRODUCTION TO GESTURE

Human hand gestures have been a mode of nonverbal interaction widely used. It

ranges from simple action of using our finger to point at and using hands to move objects

around for more complex expressions for the feelings and communicating with others. The

pursuance for the Human Computer Interaction research is moved by the central dogma of

removing the complex and cumbersome interaction devices and replacing them with more

obvious and expressive means of interaction by minimizing interaction

4.1 Background Registration and foreground segmentation When making skin color detection in real situation where it is very important process

in gesture recognition, so that divide the images into foreground and background regions

according to the dynamic characteristic of the film. The background regions such as face,

neck, areas of adjacent skin color, etc. do not change during the recognition process, In

addition to the background region, we call the hand gestures area foreground region. We

subtract the background from the images to get foreground regions. The foreground image is

used to conduct "and" operation with skin color images to get a more accurate image of hand

region.

Differential techniques are used in video image processing, that is current image

subtracts background image, the result image is called difference foreground image. Select

appropriate threshold. The difference foreground image was then be binarized to produce

foreground binary image. The basic principle of differential image processing is to carry out



345

differences calculation between the image after gray scale transformation in the detect area

with the background image in the spatial domain.

That can be expressed as

Where f(x,y,ti) and f(x,y,tj) represent the brightness values of the pixel at position of

(x,y) on the moment of ti and tj respectively. The range of the value is from 0 to 255. The

significant differences of pixel's brightness at position (x,y) is described as:

Where mk and sk(k=i,j) are the mean and variance of f(x,y,tk) within a small

neighborhood q(x,y) at position of (x,y). t is a threshold.

Segmentation is a process of partitioning an image into multiple segments based on

certain attributes. The ultimate goal of the segmentation is to convert the image into a

simplified form that is more useful as compared to the original image. The results of multiple

segmentation techniques depend upon the requirement for segmentation. Background

subtraction provides an effective means of segmenting objects moving in front of a static

background. Researchers have traditionally used combinations of morphological operations

to remove the noise inherent in the background-subtracted result.

4.2 Tracking Tracking starts with interest of color space used for skin modeling. RGB is a

convenient color model for computer graphic because the human visual system works in a

way that is Similar to an RGB color space, when it was convenient to express color as a

combination of three based colored rays (red, green and blue). Normalized RGB skin color

model is considered to be more proper for hand skin, and can be easily obtained from the

RGB values by a simple normalization processing.

Format of YCbCr has been considered to be better in describing the properties than

RGB color space The clustering characteristic for YCbCr is better than RGB YCbCr is used

to separate out a luminance signal (Y) and two chrominance components (Cb and Cr). YCbCr

can challenge various illumination conditions by discarding the signal Y, which not only

improve the performance and also reduce the data dimension than RGB.



346

The properties of skin color can be characterized by Gaussian distribution. The single

Gaussian model is one of the simplest models to model the distribution of the certain objects

which is widely used in computer vision and pattern recognition. Gaussian distribution is

given by

Where is the mean value of the samples and the variance value. Using the

Gaussian model to model skin color is actually a process that matching each pixel of the

image to the model. If matched, it is consider the pixel as a skin pixel, else it is consider it

background.

Highlighting of the hand posture particularly in the film is done considering several steps

including skin detection are camshaft algorithm, calibration, motion detection etc.

5. BLOCK DIAGRAM

Figure 5: Proposed Hand Gesture Recognition System

Gesture recognition is important for developing an attractive alternative to prevalent

human–computer interaction modalities. The system design shown above depicts the

techniques used for designing a dynamic user interface which initializes by acquiring image



347

or video of the gestures from user. The different hand gestures considered are the sequences

of distinct hand gestures. A given hand gesture can undergo motion and discrete change.

These gestures are distinguished based on the nature of motion. The real time

recognition engine is developing which can reliably recognize these gestures despite

individual variations. The engine also has the ability to detect start and end of gesture

sequences in an automated fashion.

6. ALGORITHMS

6.1 Algorithm for calibration In calibration part, take the snapshot of the hand region from the webcam then prompt

user has to select the hand region from the snapshot. Convert the input image from RGB

color format to LAB and HSV format for further operation on selected skin region. Then

calculate and store the mean values of A, B, H&S[7].

6.2 Skin Detection Skin detection is very important process in detecting the hand posture so to perform with

skin detection we need to do calibration to get hand color pixels as samples then

comparing those pixels with current image to detect the hand. Hence following

processing steps can be observed in skin detection.

a. Load color samples, and update the background so as to take the median. Calculate

the mean 'a*' and 'b*' value for each area that you extracted with roipoly. These values

serve as your color markers in 'a*b*' space.

b. Trigger the image and take the difference of current image and the background.

watever the output we obtain we have to take it in 3 dimension.

c. Next process is Thersholding. This is in simple converting image into binary. Then

doing some morphological operations like Dilation defined as some kind of

expansion.Fill the circular region.

d. Convert the output to LAB format and HSV format where LAB defined as L for

luminosity, A for chrominocity layer a, B for chrominocity layer b.

e. Calculate Euclidian distance between input pixels and sample values, these values are

nothing but the inputs A,B,am,bm which are stored in calibration.

distance1 = ((a - am).^2 + (b - bm).^2 ).^0.5;

distance2 = ((H - hm).^2 +(S - sm).^2 ).^0.5;

If distance between the pixels is less than threshold value 15 for LAB

format(D1<THRESHOLD (15)) then image mask 1 is generated.

f. If distance between the pixels is less than threshold value 0.5 for HSV

format(D1<THRESHOLD (0.5)) then image mask 2 is generated.

Distance between the pixels is less if we have similar pixels and more if pixels are

different. In terms of HSV if the distance is less than 0.1 then it is matching. Get BW and

BW2. Get the mask for skin segmentation. Perform OR operation for both color space

segmentation output so that to get the skin mask [8].



348

7. RESULTS AND DISCUSSION

Figure 6: Acquired Image with subplot

Figure 7: A sample output after background registration and foreground segmentation



349

Figure 8: Selecting ROI to get skin color samples in calibration

Figure 9: Dynamic single palm Picture with Segmentation

Figure 10: two palm after skin segmentation



350

Figure 11: detection of palm and face after skin segmentation

8. INTERPRETATION OF RESULTS

Figure 6 shows the hand figure, is captured through the camera by using the Matlab

Program. Figure 8 shows the hand figure with area selected. Here, the values from area

selected are stored for the dynamic hand segmentation depending on the values (color)

calculated earlier. Figure 7 A sample output after background registration and foreground

segmentation. Figure 9,10 shows the dynamic segmentation by matching the color values.

Pictures are saved while hand in motion dynamically. Figure 11 shows there will be a

detection of face also due to the skin color segmentation.

9. CONCLUSION

This paper describes the design and implementation of a bare dynamic hand gesture

recognition system using just one color camera. The developed system can obtain a high

recognition rate of bare hand gestures. Our current research mainly focuses on the single

hand static gestures for the simplicity. However, we are far from building a general-purpose

gesture recognition system. The extensive experiments and evaluation in outdoor

environment where more uncertainties such as changing backgrounds, sunshine and shadows

may bring the hand gesture complicated. The dynamic gestures and two-handed gestures

should also be explored in the future, since they are more expressive and allow more natural

interaction. the system implementation shown above adopted by the detection of hand or

highlighting hand posture in the input image using calibration, motion detection, and resulted

with segmented hand posture is defined with results.

As in case of dynamic recognition technics video processing needs speed and higher

intensity in providing experience of virtual and real time experience.



351

REFERENCES

[1] Yoo-Joo Choi, Je-Sung Lee and We-Duke Cho, “A Robust Hand Recognition In

Varying Illumination,” Advances in Human Computer Interaction, Shane Pinder

(Ed.), 2006.

[2] N. Conci, P. Cerseato and F. G. B. De Natale, “Natural Human- Machine Interface

using an Interactive Virtual Blackboard,” In Proceeding of ICIP 2007, pp. 181-184,

2007.

[3] S. Mitra, and T. Acharya, “Gesture Recognition: A survey,” IEEE Transactions on

Systems, Man and Cybernetics (SMC) - Part C: Applications and Reviews, vol. 37(3),

pp. 211-324, 2007

[4] Xiujuan Chai, Yikai Fang and Kongqiao Wang, “Robust hand gesture analysis and

application in gallery browsing,” In Proceeding of ICME, New York, pp. 938-94,

2009.[5] Ayman Atia and Jiro Tanaka, “Interaction with Tilting Gestures in Ubiquitous

Environments,” In International Journal of UbiComp (IJU), Vol.1, No.3, 2010.

[6] José Miguel Salles Dias, Pedro Nande, Pedro Santos, Nuno Barata and André Correia,

“Image Manipulation through Gestures,” In Proceedings of AICG’04, pp. 1-8, 2004.

[7] Pradeep kumar B.P Shailesh M.L, Shankar B.B, , Kumudeesh.K.C. “Dynamic Hand

Gesture Recognition”, International Journal of Graphics And Image Processing,

published in ijgip, volume 2, issue 1, feb 2012

[8] Pradeep kumar B.P “Design and development of human computer interface (HCI)

system based on gesture recognition using svm”, International Journal of Graphics And

Image Processing, published in ijgip, volume 2, issue 2, may 2012

[9] C. Carson, S. Belongie, H. Greenspan, and J. Malik. Blobworld: im-age segmentation

using expectation-maximization and its application to image querying. Pattern Analysis

and Machine Intelligence, IEEE Transactions on, 24(8):1026{1038, 2002.

[10] Y. Deng and B. S. Manjunath. Unsupervised segmentation of color- texture regions in

images and video. Pattern Analysis and Machine Intelligence, IEEE Transactions on,

23(8):800{810, 2001.

[11] Haihua Feng, D.A. Castanon, and W.C. Karl. A curve evolution approach for image

segmentation using adaptive ows. In ICCV 2001. Proceedings. Eighth IEEE

International Conference on computer vision, volume 2, 2001.

[12] W.Y. Ma and B.S. Majunath. Edge ow: a framework of boundary detection and image

segmentation. In ICCV 2001. Proceedings. Eighth IEEE International Conference on

computer vision, 1997.

[13] V. Mezaris, I. Kompatsiaris, and M. Strintzis. An ontology approach to object-based

image retrieval, 2003.

[14] Christopher Town and David Sinclair. Content based image retrieval using semantic

visual categories. Technical report, 2001.

[15] J. Wang, J. Li, D. Chan, and G. Wiederhold. Semantics-sensitive retrieval for digital

picture libraries. D-LIB Magazine, 5(11), November 1999.

[16] Shaikh Shabnam Shafi Ahmed, Dr.Shah Aqueel Ahmed and Sayyad Farook Bashir,

“Fast Algorithm for Video Quality Enhancing using Vision-Based Hand Gesture

Recognition”, International Journal of Computer Engineering & Technology (IJCET),

Volume 3, Issue 3, 2012, pp. 501 - 509, ISSN Print: 0976 – 6367, ISSN Online: 0976 –

6375.



352

ABOUT THE AUTHOR

Mr. Shivamurthy R C received the BE degree from PDA college of

Engineering, Gulbarga University and received the M.Tech degree in

Computer Science & Engineering from Malnad College of Engineering,

Visvesvaraya Technological University, Belgaum. Currently working as

professor in the department of Computer Science at A.I.T, Tumkur,

Karnataka, and he is also a Ph.D scholar in CMJ University, India.

Dr.B.P Mallikarjunaswamy. working as a professor in the

Department of Computer Science & Engineering, Sri Siddhartha Institute

of Technology, affiliated to Sri Siddhartha University. He has more than 20

years of Experience in teaching and 5 years of R & D. He is guiding many

Ph.D scholars. He has published more than 30 technical papers in national

and International Journals and conferences. His current research interests

are in pattern Recognition and Image Processing.

Mr.Pradeep Kumar.B.P, He is presently working as a Assistant

Professor in Akshaya Institute of Technology, Tumkur and perusing his

PhD in Jain university in electronics & communication Engineering

department. He has published more than 20 technical papers in national and

International Journals and conferences. His areas of interests are signal

processing, Medical Imaging, pattern recognition, video processing,

Multimedia Communication systems.

Dynamic hand gesture recognition using cbir

Technology

Transcript of Dynamic hand gesture recognition using cbir