Feature Level Fusion of Palm Veins and Signature Biometrics · on multimodal palm vein and...

International Journal of Video & Image Processing and Network Security IJVIPNS-IJENS Vol: 12 No: 01 28

I J E N S IJENS © February 2012 IJENS-IJVIPNS 7474-016812

Feature Level Fusion of Palm Veins and Signature Biometrics

Hassan Soliman

Faculty of Engineering, Mansoura University,

Mansoura, Egypt [email protected]

Abdelnasser Saber Mohamed Information Systems Department, El-Mehalla Technical College,

Mansoura, Egypt [email protected]

Ahmed Atwan Faculty of Computer & Information Sciences,

Mansoura University, Mansoura, Egypt

[email protected]

Abstract- Traditional biometric systems that based on single biometric usually suffer from problems likeimposters' attack or hacking, unacceptable error rate and low performance. So the need of using multimodal biometric system occurred .In this paper, a study of multimodal palm veins and signature identification is presented.Features of both modalities are extracted by using morphological operations and Scale Invariant Features Transform (SIFT)algorithm and a comparison for both methods is developed.Feature level fusion for both modalitiesis achieved by using a simple sum rule. Fused features vectors are subjected to discrete cosine transform (DCT) to reduce their dimensionalities. Linear Vector Quantization (LVQ) classifier is used with changed parameters to classify the different people in the database. Preliminary results are encouraging and supporting using palm veins and signature as a robust and reliable identification system.

Key Words: Multimodal biometrics, feature fusion level, Palm veins recognition, Signature recognition, Morphological operations, SIFT algorithm, DCT, LVQ classifier.

I. INTRODUCTION

Multimodal biometric systems provide anti-

spoofing measures by making it difficult for

intruders to spoof multiple biometric traits

simultaneously [1]. The benefit of multimodal

biometrics may become even more evident in the

case of a larger database of users. Palm vein

recognition is good by itself and has high

acceptance rate with low false acceptance rate, but

due to increasing online transactions and

communication the demand to increase the

security against imposters or hackers [2]. So by

combining multiple modalities enhanced

performance, more security and reliability could

be achieved.

In our proposed approach for identification based

on multimodal palm vein and signature, the

features of both modalities were extractedby using

morphological operations and SIFT algorithm and

a comparison for both methods was developed.

Both modalities' features can be fused at three

different levels: fusion at the feature level,

matching level or decision level [3]. In this paper,

the focus is on fusion at feature level, whichis

believed to be very promising as feature sets can

provide more information about the input

biometrics than other levels.Feature level fusion

refers to combining different feature vectors that

are obtained by either using multiple sensor data.

We used Fujitsu’s PalmSecure™ scanner to obtain

the vein pattern database and collect regular

signatures from employees of Center of Scientific

Computing located in Mansoura University.

Each modality contains its feature vector. These

feature vectors contain non-homogenous features,

so normalizing should be madefor features vectors

for both modalities before concatenate them to

form a single one. Concatenating two features

vectors may result in a feature vector with very

large dimensionality [4]. In this paper, DCT

(discrete cosine transforms) is usedto reduce the

dimensionality of both modalities' feature vectors.

II. REVIEW OF RELATED WORKS

A number of studies showing the advantages of

multimodal biometrics fusion have appeared in

the literature. Brunelli and Falavigna [5] used

hyperbolic tangent (tanh) for normalization and

weighted geometric average for fusion of voice and



face biometrics. They also proposed a hierarchical

combination scheme for a multimodal

identification system. Kittler et al. [6] have

experimented with several fusion techniques for

face and voice biometrics, including sum, product,

minimum, median, and maximum rules and they

have found that the sum rule outperformed others.

Kittler et al. [6] note that the sum rule is not

significantly affected by the probability estimation

errors and this explains its superiority.

Hong and Jain [7] proposed an identification

system based on face and fingerprint, where

fingerprint matching is applied after pruning the

database via face matching. Ben-Yacoub et al. [8]

considered several fusion strategies, such as

support vector machines, tree classifiers and

multi-layer perceptron, for face and voice

biometrics. The Bayes classifier is found to be the

best method. Ross and Jain [9] combined face,

fingerprint and hand geometry biometrics with

sum, decision tree and linear discriminant-based

methods. The authors report that sum rule

outperforms others.

This paper contains the following sections: first

section is the introduction, second one is review of

related works, third section investigates

multimodal biometric system using palm veins

and signature recognition systems and discuss the

two methods of feature extraction and forth

section introduces the experimental results based

on LVQ classifier, and the last section introduces

conclusions.

III. MULTIMODAL BIOMETRIC SYSTEM

Most of the problems and limitations of biometrics

are imposed by unimodal biometric systems,

which rely on the evidence of only a single

biometric trait. Some of these problems may be

overcome by multi biometric systems and an

efficient fusion scheme to combine the information

presented in multiple biometric traits. In this

paper, we introduce a novel multimodal biometric

system using palm veins and signature modalities.

The following framework (see Figure 1) explains

the workflow of the system.

A. Palm Veins Recognition System

This system uses an infrared beam to penetrate

the users hand as it is held over the sensor; the

veins within the palm of the user are returned as

black lines. Palm vein authentication has a high

level of authentication accuracy due to the

uniqueness and complexity of vein patterns of the

palm. Because the palm vein patterns are internal

to the body, this is a difficult method to forge. In

addition, the system is contactless and hygienic

for use in public areas [2].

1. Palm vein authentication based on morphological operations:

1.1 Vein pattern

The sensing technology used for vein patterns is

based on near-infrared spectroscopy (NIRS) and

imaging, and has been developed through in vivo

measurements during the last 10 or so years. That

is, the vein pattern in the subcutaneous tissue of

the palm is captured using near-infrared rays [2].

In this work, we used Fujitsu’s PalmSecure™

scanner for sensing and image acquisition of

images that we tested in our research. Decide

whether or not the user will hold his/her palm

over the palm vein authentication sensor. If the

palm is held, capture the infrared-ray image,

which contains palm vein patterns (see Fig. 2).

Fig.1. Workflow of the proposed system.

International Journal of Video & Image Processing and Network Security IJVIPNS

(a)

Fig. 2. (a) Capture device. (b) an infrared palm image captured by PalmSecureTM scanner

The following workflow (see Figure 3) shows the

whole processes of the identification system usi

Palm veins biometrics based on morphological

operations.

Fig.3. Palm Vein Authentication Operations Workflow Diagram based on morphological operations.

1.2 ROI extraction from palm vein patterns After image capture, asmall area (128*128 pixels)

of a palm image is located as the region of interest

(ROI) to extract the features and to compare

different palms. Using the features within ROI for

recognition can improve the computation

efficiency significantly. Further, because this ROI

is located by a normalized coordinate based on the

palm boundaries, the recognition error caused by a

user who slightly rotate or shift his/her hand is


IJENS © February 2012 IJENS-IJVIPNS 7474-016812

(b)

infrared palm image

3) shows the

whole processes of the identification system using

Palm veins biometrics based on morphological

Palm Vein Authentication Operations Workflow

Diagram based on morphological operations.

ROI extraction from palm vein patterns

After image capture, asmall area (128*128 pixels)

f a palm image is located as the region of interest

(ROI) to extract the features and to compare

different palms. Using the features within ROI for

recognition can improve the computation

efficiency significantly. Further, because this ROI

normalized coordinate based on the

palm boundaries, the recognition error caused by a

user who slightly rotate or shift his/her hand is

minimized. Figure (4) illustrates the procedure of

ROI locating:

1). Binarized the input image (Fig. 4(a));

2). Obtain the boundaries of the gaps, (Fixj; Fiyj)

(Fig. 4(b));

3). Compute the tangent of the two gaps (Fig.

4(b)), use this tangent (the line connect (x1, y1)

and (x2, y2)) as the Y

coordinate;

4). Use a line passing through the midpoint of the

two points (x1, y1) and (x2, y2), which is also

perpendicular to the Y-axis, as the X

line perpendicular to the tangent in Fig. 4(b));

5). The ROI is located as a square of fixed size

whose center has a fixed distance to the palm

coordinate origin (Fig. 4(c));

6). Extract the subimage within the ROI (Fig.

4(d)).

(a)

(c)

Fig. 4. Locate ROI. (a) Binarized image; (b) boundaries and ROI locating; (c) ROI

locating; (d) the subimage in ROI.

1.3 Image processing steps: Usually, in the image-based biometric systems, a

number of processing tasks to produce a better

quality of image that will be used on the later

stage as an input image and assuring that

relevant information can be detected [10]. In this

paper, we will use Matlab functions an

Processing Toolbox to do the entire required image

processing. We will follow the lines set out by Gao

et al [11] for fingerprint recognition and

subsequently applied to hand veins by Malki

[12].


I J E N S

minimized. Figure (4) illustrates the procedure of

1). Binarized the input image (Fig. 4(a));

he boundaries of the gaps, (Fixj; Fiyj)

3). Compute the tangent of the two gaps (Fig.

4(b)), use this tangent (the line connect (x1, y1)

and (x2, y2)) as the Y-axis of the palm

4). Use a line passing through the midpoint of the

points (x1, y1) and (x2, y2), which is also

axis, as the X-axis (the

line perpendicular to the tangent in Fig. 4(b));

5). The ROI is located as a square of fixed size

whose center has a fixed distance to the palm

Fig. 4(c));

6). Extract the subimage within the ROI (Fig.

(b)

(d)

Locate ROI. (a) Binarized image; (b) boundaries and ROI locating; (c) ROI

locating; (d) the subimage in ROI.

based biometric systems, a

number of processing tasks to produce a better

quality of image that will be used on the later

stage as an input image and assuring that

relevant information can be detected [10]. In this

paper, we will use Matlab functions and Image

Processing Toolbox to do the entire required image

processing. We will follow the lines set out by Gao

for fingerprint recognition and

subsequently applied to hand veins by Malki et al



Normally, the captured hand-vein pattern is gray-

scale and subject to noise. Noise Reduction and

Contrast Enhancement are crucial to ensure the

quality of the subsequent steps of feature

extraction [12]. This is achieved by means of:

a. Binarization that transforms the gray-

scale pattern into a black and white

image,

b. Skeletonization that reduces the width

of lines to one pixel and

c. Finally,Isolated Pixel Removal that

eliminates the unwanted isolated points.

These three steps constitute the procedure of

image preprocessing, as shown in Figure 5.

Fig.5. three steps of Image Preprocessing

1.4 Palm vein features extraction Hand vein patterns have two main features:

endings (end points) and bifurcations (branch

points). The former is the end point of a thinned

line, while the latter is the junction point of three

lines Figure6 illustrates the idea.

Fig.6. Palm vein features: endings and

bifurcations

The detection of bifurcations and endings in the

preprocessed image can be performed in

parallel. Intermediate results are summed by a

simple OR logic before the feature of false

is eliminated. Figure (8) illustrates all the steps

for Palm vein feature extraction.

(i)

Fig.8. (a)Original image contains the vein pattern,

(b) after Binarization, (c) after filer (d) after

Skeletonization, (e) after isolated pixel removal,

leaving ,(f) after thinning (g) the bifurcations, (h)

endings and (i) OR operation between bifurcations

and endings(merging).

B. Off-Line Signature Recognition Signature verification is an important research

area in the field of authentication of a person as

well as documents in e-Commerce and banking.

We can generally distinguish between two

different categories of signature verification

systems: online, for which the signature signal is

captured during the writing process, thus making

the dynamic information available, and offline for

which the signature is captured once the writing

process is over and thus, only a static image is

available. In this paper, we deal with an Off-line

signature verification System. We design a system

capable of verifying authenticity of a signature

based on test performed with genuine signatures

(Verification Mode) and person identification from

signature (Recognition Mode).[13]

Various approaches are possible for signature

recognition with a lot of scope of research. In this

paper, we deal with an Off-line signature

recognition technique, where the signature is

captured and presented to the user in the format


of image only. We use various image

techniques to extract the parameters of signatu

and verify the signature based on these

parameters. Signature recognition is a two

pattern classification problem, where authentic

signatures belong to one class and the forged

signatures belong to the other class[13].

illustrates the signature authentication operations

workflow diagram.

1. Signature Authentication based on Morphological Operations Workflow

Fig.9. Signature Authentication Operations

Workflow Diagram

1.1 Signatures Preprocess

In this paper, we divided the preprocess

procedures intoImage normalization processes and

image processing processes. Preprocess procedures

will be applied on both training and testing

images as follows:

� Image normalization

The signature images must be normalized to

determined standards because of the variation

they may have. We do the following processes to

normalize images:

a) Resizing: signature dimensions may have

intrapersonal and interpersonal

differences. Therefore, the image should be

adjusted to a default size. We proposed it

175 X 200



of image only. We use various image-processing

techniques to extract the parameters of signatures

and verify the signature based on these

parameters. Signature recognition is a two-class

pattern classification problem, where authentic

signatures belong to one class and the forged

signatures belong to the other class[13]. Figure (9)

ion operations

Signature Authentication based on Morphological

ion Operations

In this paper, we divided the preprocess

procedures intoImage normalization processes and

image processing processes. Preprocess procedures

oth training and testing

The signature images must be normalized to

of the variation

they may have. We do the following processes to

: signature dimensions may have

intrapersonal and interpersonal

differences. Therefore, the image should be

adjusted to a default size. We proposed it

b) Rotating:moving the signature to the

origin and passing the new co

c) Strip out the white area

non-data pixels surrounded the

image.

(a)

(c) Fig.10. (a) original scanned signature's image,

(b) rotated image for its origina

Co-ordinates and (c) image without its

surrounded white area.

� Image processing

The purpose in this phase is to make

signature images ready for feature

extraction phase. The processing stage

includes the followings:

a) Binarization:Converts the ima

black and white equivalency pixels.

b) Noise Elimination: Removes single

black pixels on white background.

c) Skeletonization: Reduces the width of

lines to one pixel. Figure(11) shows

these operations.

d)

(a)

(c)

Fig.11. (a) Binaries image

filtration (c) after Skeletonization

Y


I J E N S

:moving the signature to the

origin and passing the new co-ordinates.

Strip out the white area:stripping

data pixels surrounded the

(b)

(a) original scanned signature's image,

(b) rotated image for its original

ordinates and (c) image without its

surrounded white area.

The purpose in this phase is to make

signature images ready for feature

extraction phase. The processing stage

includes the followings:

Binarization:Converts the image to its

black and white equivalency pixels.

Noise Elimination: Removes single

black pixels on white background.

Skeletonization: Reduces the width of

lines to one pixel. Figure(11) shows

(b)

(c)

a) Binaries image (b) after noise

c) after Skeletonization

X

Y

X


1.2 Signatures Features Extraction : In this phase, we will focus on global features

extractions, which provide information about

specific cases of the signature shape.

described as the following:

1) Signature height-to-width ratio:It is

approximately equal for person's

signatures. It can be obtained by dividing

height to width of the signature image.

2) Signaturearea: it provides information

about the signature's pixels density.

3) Maximum horizontal histogram (MHH)

and maximum vertical histogram (MVH):

horizontal histogram is calculated for each

row and takes the highest value as MHH.

Vertical histogram is calculated for each

column and takes the highest value as

MVH.

4) End point numbers of the signature: It

the pixel, which has only one neighbor.

(a) (b)

(c)

(e)

Fig.12. (a) Processed signature (b) Maximum Vertical Histogram (c) Maximum Horizontal Histogram (d) end

points (f) Intermediate Signature merg

C. Multimodal Biometrics Fusion based on Morphological Operations

There are many levels of biometric fusion such as

[14] sensor level, feature level, match score level,

rank level, and decision level. In feature level, the

feature sets extracted from multiple data source

can be fuse to create a new feature set to

represent the individual. It may result in a new



Signatures Features Extraction :

In this phase, we will focus on global features

extractions, which provide information about

specific cases of the signature shape. It is

width ratio:It is

approximately equal for person's

signatures. It can be obtained by dividing

height to width of the signature image.

Signaturearea: it provides information

about the signature's pixels density.

gram (MHH)

and maximum vertical histogram (MVH):

horizontal histogram is calculated for each

row and takes the highest value as MHH.

Vertical histogram is calculated for each

column and takes the highest value as

End point numbers of the signature: It is

the pixel, which has only one neighbor.

(d)

(f)

a) Processed signature (b) Maximum Vertical Histogram (c) Maximum Horizontal Histogram (d) end

points (f) Intermediate Signature merging

cs Fusion based on

There are many levels of biometric fusion such as

[14] sensor level, feature level, match score level,

rank level, and decision level. In feature level, the

feature sets extracted from multiple data source

e fuse to create a new feature set to

represent the individual. It may result in a new

high-dimension feature vector. In order to reduce

the high dimensionality of the feature vectors, we

used the DCT (Discrete Cosine Transform). The

D discrete cosine transform (DCT) is defined

as:[15]

Biometric multimodality can be studied as a

classifier combination problem [6]. Kittler et al [6]

consider in the task of combining classifiers in a

probabilistic Bayesian framework. Several ways to

merge the modalities are obtained (sum,

product,max,min,) based on the Bayesian theorem

and certain hypothesis, from which the Sum Rule

outperformed the remainder in the experimental

comparison [16]. In this paper, we will apply Sum

Rule to concatenate the features from two

modalities (palm veins and off

then introduce the new feature vectors to the LVQ

classifier.

In this paper, we employ Arun and Rohin [

procedures for feature level fusion is accomplished

by a simple concatenation of feature sets obtain

from multiple modalities. Let X = { x1 , x2,……,

xm} and Y = { y1,y2,…..,yn} denote feature vectors

(X ∈ Rm and Y ∈ Rn) representing the information

extracted via two different resources. The

objective is to combine these two feature sets in

order to yield a new feature vector, Z that would

better represent the individual. The vector Z is

generated by first augmenting vectors X and Y,

and then performing feature selection on the

resultant feature vector. The fusion of feature

level data from any two biometric sources in this

paper will follow the similar procedure. The

different stage in this algorithm is d

below.

1. Feature Normalization

In our experiments, we used the median

normalization scheme due to its robustness to

outliers. Normalizing the feature values via this

technique results in modified feature vectors. [17]

The median normalization sch

hand, is relatively robust to the presence of noise


I J E N S

dimension feature vector. In order to reduce

the high dimensionality of the feature vectors, we

used the DCT (Discrete Cosine Transform). The 2-

ansform (DCT) is defined

Biometric multimodality can be studied as a

classifier combination problem [6]. Kittler et al [6]

consider in the task of combining classifiers in a

probabilistic Bayesian framework. Several ways to

re obtained (sum,

product,max,min,) based on the Bayesian theorem

and certain hypothesis, from which the Sum Rule

outperformed the remainder in the experimental

comparison [16]. In this paper, we will apply Sum

Rule to concatenate the features from two

alities (palm veins and off-line signatures),

then introduce the new feature vectors to the LVQ

In this paper, we employ Arun and Rohin [17]

procedures for feature level fusion is accomplished

by a simple concatenation of feature sets obtain

from multiple modalities. Let X = { x1 , x2,……,

Y = { y1,y2,…..,yn} denote feature vectors

Rn) representing the information

extracted via two different resources. The

objective is to combine these two feature sets in

order to yield a new feature vector, Z that would

better represent the individual. The vector Z is

t augmenting vectors X and Y,

and then performing feature selection on the

resultant feature vector. The fusion of feature

level data from any two biometric sources in this

similar procedure. The

different stage in this algorithm is described

In our experiments, we used the median

normalization scheme due to its robustness to

outliers. Normalizing the feature values via this

technique results in modified feature vectors. [17]

The median normalization scheme, on the other

hand, is relatively robust to the presence of noise


in thetraining data. In this case, x' is computed as,

x' = ��

��| �� . Where Fx is the function

generates x. The denominator is known as the

Median Absolute deviation (MAD) and is an

estimate of the scale parameter of the feature

value.

2. Feature Selection

Augmenting the two feature vectors, x' and Y',

results in a new feature vector,

Z' = {x'1, x'2,….x'm, y'1,y'2,….,y'n} Z'

curse-of-dimensionality'18 dictates that the

augmented vector needs not necessity

improved matching performance. Further, some of

the feature values may be `noisy' compared to the

others.

The feature selection process entails choosing a

minimal feature set of si

k < (m + n), that improves classification

performance on a training set of feature vectors.

The sequential forward floating selection

technique is employed to perform feature selection

on the feature values of Z'. This results in a new

feature vector Z = {Z1.Z2, ….Zk}. The criterion

function to perform feature selection is defined to

be the average of the Genuine Accept Rate (GAR)

at four different False Accept Rate (FAR) values

(0:05%, 0:1%, 1%, 10%) in the ROC (Receiver

Operating Characteristics) curve pertaining to the

training data. The reason for choosing this

criterion is explained below[17].

The following figure (see figure 13) shows

intermediate fusion of palm veins features (X'),

intermediate fusion of signature features and the

final fusion matrix of both modalities based on

morphological operations.

Fig.13. Morphological features fusionof both modalities

+ X’ (Palm veins features) Y ‘(Signature features)



in thetraining data. In this case, x' is computed as,

Fx is the function

generates x. The denominator is known as the

deviation (MAD) and is an

estimate of the scale parameter of the feature

Augmenting the two feature vectors, x' and Y',

results in a new feature vector,

Z' = {x'1, x'2,….x'm, y'1,y'2,….,y'n} Z' ∈ Rm+n. The

ity'18 dictates that the

nted vector needs not necessity result in an

improved matching performance. Further, some of

the feature values may be `noisy' compared to the

The feature selection process entails choosing a

minimal feature set of size k,

k < (m + n), that improves classification

performance on a training set of feature vectors.

The sequential forward floating selection

technique is employed to perform feature selection

on the feature values of Z'. This results in a new

r Z = {Z1.Z2, ….Zk}. The criterion

function to perform feature selection is defined to

be the average of the Genuine Accept Rate (GAR)

at four different False Accept Rate (FAR) values

(0:05%, 0:1%, 1%, 10%) in the ROC (Receiver

urve pertaining to the

training data. The reason for choosing this

The following figure (see figure 13) shows

intermediate fusion of palm veins features (X'),

intermediate fusion of signature features and the

n matrix of both modalities based on

Fig.13. Morphological features fusionof both modalities

D. Palm Veins and Signature Features Extraction based on SIFT Algorithm:

Scale-invariant feature transform

algorithm in computer vision

describe local features in images. The algorithm

was published by David Lowe

Invariant Feature Transform (SIFT) is an

approach for detecting and extracting local feature

descriptors that are reasonably invariant to

changes in illumination, image noise, rotation,

scaling, and small changes in viewpoint [18]. SIFT

algorithm was deployed to extract features from

tow modalities( see figure (14))

Fig.14. SIFT Algorithm Overview diagram [19]

In this work, we considered spatial, orientation

and Keypoints descriptor i

extracted SIFT point.For palm vein, the input to

the system is the palm vein image.The output is

the set of extracted SIFT features sp= (sp1, sp2....

....spm) where each feature point spi=(x ,y , •,k)

consist of the (x, y) spatial locati

orientation • and k is the key descriptor of size

1x128. For signature image, the input to the

system is the signature image. The output is the

set of extracted SIFT features ss=(ss1, ss2.... ....

ssm) where each feature point ssi=(x ,y , •,k

consist of the (x, y) spatial location, the local


1x128.

1. Multimodal Biometrics Fusion based on SIFT

Keypoints

1.1 Feature set normalization

The Keypoints descriptors of each palm vein and

signature points are then normalized using the

min-max normalization technique (SPnorm and

SSnorm), to scale all the 128 values of each

Keypoints descriptor within the range 0 to 1. This

normalization can also apply the same threshold

=

(Signature features)

Z’Fusion matrix


I J E N S

Palm Veins and Signature Features Extraction

invariant feature transform (or SIFT) is an

computer vision to detect and

describe local features in images. The algorithm

David Lowe in 1999. Scale

Invariant Feature Transform (SIFT) is an

approach for detecting and extracting local feature

descriptors that are reasonably invariant to

changes in illumination, image noise, rotation,

ing, and small changes in viewpoint [18]. SIFT

algorithm was deployed to extract features from

( see figure (14)).

Fig.14. SIFT Algorithm Overview diagram [19]

we considered spatial, orientation

and Keypoints descriptor information of each

extracted SIFT point.For palm vein, the input to

the system is the palm vein image.The output is

the set of extracted SIFT features sp= (sp1, sp2....

....spm) where each feature point spi=(x ,y , •,k)



1x128. For signature image, the input to the

system is the signature image. The output is the

set of extracted SIFT features ss=(ss1, ss2.... ....

ssm) where each feature point ssi=(x ,y , •,k)


he key descriptor of size

Multimodal Biometrics Fusion based on SIFT

The Keypoints descriptors of each palm vein and

are then normalized using the

max normalization technique (SPnorm and

SSnorm), to scale all the 128 values of each

Keypoints descriptor within the range 0 to 1. This

normalization can also apply the same threshold



on the palm vein and signature Keypoints

descriptors, when the corresponding pair of points

is found for matching the fused pointsets of

database and query palm vein and signature

images.[19]

1.2 Feature Concatenation and Reduction

The feature level fusion is performed by

concatenating the two feature pointsets. These

results in a fused feature pointsetconcat=

(SPlnorm, SP2norm, SPnorm,…….SSlnorm,

SS2norm, SSmnorm).

Feature reduction strategy to eliminate irrelevant

features can be applied either before or after

feature concatenation.In this paper, DCT (Discrete

Cosine Transform) is used to reduce the

dimensionality of the concatenated feature vector.

1.3 SIFT Feature Matching

1) Find nearest neighbor in a database of SIFT

features from training images.

2) For robustness, use ratio of nearest

neighbor to ratio of second nearest neighbor.

3) Neighbor with minimum Euclidean

distance!Expensive search.

4) Use an approximate, fast method to find

nearest neighbor with high probability.[19]

(a)

(b)

Fig.15. SIFT Keypoints matching for (a) Palm

veins pattern and (b) Signature image

IV. EXPERIMENTAL RESULTS

A. Database Description

We collected our own database from employees of

Center of Scientific Computing located in

Mansoura University who have different jobs,

genders, and ages. 37 signers assigned on the form

we have designed to collect 10 signatures from

each person. Therefore, we have 370signatures in

our signature database and by usingPalmSecure

scanner; we collect 5*37 = 185 palmveins images

from 37 persons.

For the signatures images we first doing the

normalization and then image processing steps to

enhance the database, then after filtering the

images by computing the coefficients variance for

each singer and taking a threshold. We discard

signer who above it (those who have no sufficient

quality (consistency) in signing) and accept the

rest. After databasefiltration for signatures,we

selected 30 consistence signatures. We used 5

signatures in training phase and 5 signatures in

testing phase. We dedicated 30 people as genuine

signatures and the other 7 persons as imposters.

In addition, we dedicated 30palm vein image as

genuine images and the other 7 images as

imposters.

By Fusion5 palm vein imageswith the 5 images of

signature in both phases training and testing, we

get the fused features and introduce them to DCT

and then to the classifier.

B. LVQ Classifier

The LVQ (Linear Vector Quantization) network

which consist of an input layer and an output

layer, it is used as the basic building block of our

classifier.The input layer consists of a number of

neurons corresponding to the number of training

patterns the network will be trainedon. For

example, we have 30 fused features vectors (e.g.

30 classes) with 5 training pattern for each, we

will have 150 input neurons each corresponding to

an input pattern.Output layer consist of a

specified number of neurons per class. We have 3

neurons per class and 80 classes then we have

30 x 3 = 90 output neurons.

C. Output layer initialization

The neurons in the output layer may be initialized

to random values, may be initialized to one of the

training samples (each neuron representing a

class initialized to one of the training samples for

this class. The neurons which representing a class

may be initialized by taking the mean of training

sample for this class. The last method of

initialization is the best one because it reduces the

training time and is more reliable because the

mean is unbiased to any of the given training

samples.

D. Training the network

Each pattern in the input layer has a response in

the output layer. The neurons in the output layer

with minimum Euclidean distance from this

pattern are considered the winner. The winner

may or not represent the correct class. In either

case, the winner is trained. If it represents a

correct class, its weights are moved toward this

pattern; otherwise, they are moved away from the

pattern.



This process is repeated for all patterns in the

input layer. This makes one epoch of training. A

large number of epochs (100 epochs) are

performed. Each epoch changes the weights of a

number of neurons (it makes them stronger in

classifying a pattern or stronger in not being

confused with patterns of other classes). Finally,

the network is tested with testing set of pattern.

The pattern is classified by measuring the

distance between it and between each neuron in

the trained output layer. The class of the winner is

considered the class of the pattern.

Now, we will make 60 classifiers (whichusing the

same network architecture, but using different

parameters for each. We have varied 3

parameters:

1. The learning rate (which controls how

many a neurons moves towards or away

from a pattern) is5 values.

2. The number of training epochs is4 values.

3. The number of neurons representing each

class is3 values.

Therefore, we should have 4x3x5= 60 classifiers.

Each classifier is tested against the training set

and the test set. Each individual classifier gives

average classification strength against each class.

It forms the following table,

TABLE I 60 Classifiers Voting Strength For 8 Classes

Classifier No.[learning rate, training

epochs, neurons per

class]

Voting strength for classes

Classifier 1 [ 0.05, 40, 1]

0.6 0.8 0.7 1 0.8 0.8 0.8 0.6

Classifier 2 [0.05 , 80,3]

0.8 0.8 0.5 0.9 0.8 0.8 0.7 0.4

.

.

Classifier 19 [ 0.1,120,1]

0.6 0.8 0.7 0.8 0.9 0.8 0.8 0.6

.

.

Classifier 60 [ 0.25, 160,5]

0.9 0.7 0.5 1 0.8 0.7 0.7 0.3

This table shows a relation between 60 classifiers

and their average strength in individually

classifying 8 classes. Each classifier is shown

along with its training parameters enclosed in

square brackets [learning rate, training epochs,

neurons per class]. Each classifier gives a vote for

a class so a class with the greatest number of

voting becomes the class of the given pattern. To

do so we combined the classifiers. It simply

explained as the following:

TABLE II Votes For Some Classes By Some Classifiers

Classifier No.

Suggested class

Voting strength

1 13 0.8 2 17 0.7 3 12 1 4 13 0.9 5 17 0.8 6 12 0.9 7 12 0.9 8 12 1 9 13 0.7 10 17 0.8

In this table, classifier no. 1 suggested class 13 for

the given pattern and gave vote (0.8),classifier no.

2 suggested class 17 for the given pattern and

gave vote (0.7) and classifier no.3 suggested class

12 for the given pattern and gave vote (0.9) and so

on.

To get the final classification we sort distinct

classes and sum the vote given for each as in the

following table:

TABLE III Result Of Summing Voting

The question arises now is, how can we choose the

combination of the classifier? The answer would

be; an accumulative classification test was used.

We begin by a combination of semi-strong

classifiers, testing their average classification

strength as a single combined classifier. We

gradually add stronger classifiers and retest the

average classification strength. After adding a

sufficient number of classifiers, we note that the

classification strength become stable. Adding more

classifiers no longer contributes to the

classification strength. At this point, we have a

robust and economical number of classifiers to use

the final combined classifier.In the following

figure, we can deduce that when both modalities

are being fusedthe Genuine Acceptance Rate

(GAR) will be increased and the False Acceptance

Rate (FAR) will be decreased. Table IV contains the GAR results of using palm veins and signature as separated and fused modalities.

Class Total voting 12 3.8 13 2.4 17 2.3


TABLE IV. Variation of the System Recognition Accuracy when

using Palm Veins and Signature as Separated and Fused Modalities.

Alp

ha(l

earn

ing

rate

)

Neu

rons

per

cl

ass

Tra

inin

g ep

ochs

GA

R u

sing

C

ombi

ned

LV

Q

for

Sign

atur

e

GA

R u

sing

C

ombi

ned

LV

Q

for

palm

vei

n

0.05

1

40 88.35% 90.77%80 88.58% 91.75%120 88.05% 92.73%160 89.13% 93.04%

2

40 88.23% 91.25%80 88.45% 92.12%120 89.60% 92.93%160 89.69% 91.79%

3

40 87.88% 93.76%80 87.98% 92.26%120 88.22% 93.60%160 89.38% 93.77%

0.1

1

40 87.93% 91.76%80 88.47% 91.73%120 88.03% 92.78%160 88.98% 94.03%

2

40 87.97% 91.93%80 89.27% 91.92%120 89.80% 93.16%160 89.91% 94.66%

3

40 87.98% 94.36%80 88.10% 94.13%120 88.45% 94.04%160 90.00% 94.14%

0.25

1

40 89.89% 91.02%80 90.34% 92.04%120 90.06% 93.26%160 90.91% 93.24%

2

40 88.32% 92.15%80 90.39% 93.03%120 90.93% 93.14%160 90.80% 93.37%

3

40 88.29% 93.79%80 88.46% 93.76%120 88.57% 95.68%160 89.59% 95.50%

As shown above in TABLE IV, when both

modalities are being fused the Genuine

Acceptance Rate (GAR) will be increased and the

False Acceptance Rate (FAR) will be decreased.

Figure 16 illustrates the idea of the research and

summarizes the above table.



Variation of the System Recognition Accuracy when using Palm Veins and Signature as Separated and Fused

for

palm

vei

n

GA

R u

sing

C

ombi

ned

LV

Q

for

Fea

ture

L

evel

Fus

ion

90.77% 93.50% 91.75% 94.52% 92.73% 95.01% 93.04% 95.98% 91.25% 92.86% 92.12% 94.85% 92.93% 95.43% 91.79% 94.66% 93.76% 96.71% 92.26% 94.90% 93.60% 95.24% 93.77% 95.99% 91.76% 94.79% 91.73% 94.01% 92.78% 95.31% 94.03% 95.98% 91.93% 94.47% 91.92% 94.82% 93.16% 95.89% 94.66% 96.14% 94.36% 96.12%

.13% 97.03% 94.04% 96.92% 94.14% 96.72% 91.02% 93.71% 92.04% 94.84% 93.26% 95.52% 93.24% 95.62% 92.15% 94.73% 93.03% 95.40%

4% 95.25% 93.37% 95.02% 93.79% 96.60% 93.76% 95.99% 95.68% 96.98% 95.50% 96.96%

As shown above in TABLE IV, when both

modalities are being fused the Genuine

e increased and the

False Acceptance Rate (FAR) will be decreased.

illustrates the idea of the research and

Fig.16. the relation between GAR and FAR in case

of feature level fusion based on Morpholog

Operations

E. SIFT Keypoints and Morphological Features Results for Both Modalities

As mentioned above, we used morphological

operations to extract two main features

(bifurcations and endpoints) from palm veins, used

global features of the signature a

concatenate two modalities features, and

introduced them to DCT algorithm to reduce the

dimensionality of the vector and we got the above

results using Combined LVQ classifier. Here, a

comparison between SIFT Keypointsand

Morphological features after applying DCT for

both fused modalities is developed. Table (4)

illustrate the variation of the system recognition

accuracy based on SIFT Keypoints and

Morphological features after applying DCT using

Combined LVQ classifier.


I J E N S

the relation between GAR and FAR in case

of feature level fusion based on Morphological

Operations

SIFT Keypoints and Morphological Features Results for Both Modalities

As mentioned above, we used morphological

operations to extract two main features

(bifurcations and endpoints) from palm veins, used

global features of the signature and then

concatenate two modalities features, and

introduced them to DCT algorithm to reduce the

dimensionality of the vector and we got the above

results using Combined LVQ classifier. Here, a

comparison between SIFT Keypointsand

er applying DCT for

both fused modalities is developed. Table (4)

illustrate the variation of the system recognition

accuracy based on SIFT Keypoints and

Morphological features after applying DCT using



TABLE V Variation of The System Recognition Accuracy based on

SIFT Keypoints and Morphological Features after applying DCT Using Combined LVQ Classifier.

Alpha(learning rate)

Neurons per

class

Training

epochs

GAR using

Combined LVQ

based on Morphol

ogical features

GAR using

Combined LVQ

based on SIFT

Keypoints

Descriptors

0.05

1

40 93.50% 95.54% 80 94.52% 96.56%

120 95.01% 97.05% 160 95.98% 97.88%

2

40 92.86% 94.76% 80 94.85% 96.75%

120 95.43% 97.33% 160 94.66% 96.56%

3

40 96.71% 98.78% 80 94.90% 96.97%

120 95.24% 97.31% 160 95.99% 98.06%

0.1

1

40 94.79% 96.86% 80 94.01% 96.08%

120 95.31% 97.38% 160 95.98% 98.05%

2

40 94.47% 96.54% 80 94.82% 96.77%

120 95.89% 97.84% 160 96.14% 98.09%

3

40 96.12% 98.07% 80 97.03% 98.98%

120 96.92% 98.87% 160 96.72% 98.67%

0.25

1

40 93.71% 95.66% 80 94.84% 96.89%

120 95.52% 97.57% 160 95.62% 97.67%

2

40 94.73% 96.78% 80 95.40% 97.45%

120 95.25% 97.30% 160 95.02% 97.07%

3

40 96.60% 98.65% 80 95.99% 98.04%

120 96.98% 99.03% 160 96.96% 99.01%

Figure (17) summarizes the above table. It

indicates the result that is SIFT algorithm is

more accurate,and does not need more

preprocessing steps to identify people.

Fig.17.GAR Comparison between SIFT features

and Morphological features Accuracy.

V. CONCLUSION In this paper, we have attempt to present new

insights and experimental results for palm vein

and signature recognition. We proposed different

approaches of morphological techniques to

enhance features extraction in palm vein and

global features extraction of signature images.In

addition, we deploy SIFT algorithm to extract the

features from two modalities. We used the simple

sum rule to fuse both modalities' features based on

both morphological operations and SIFT

Keypoints descriptors. We applied DCT algorithm

to reduce the feature vectors dimensionalities of

both feature extraction techniques

(morphologicaloperations and SIFT Algorithm).

Feature vectors after DCT operations are ready to

be introduced to the LVQ classifier, which

contains the following parameters: learning rate,

training epochs, neurons per class. We combined

the 60 classifiers and then made voting for their

strength and chose the high voting class to be the

class of the pattern.Finally we can deduce that

SIFT algorithm is more accurate and does not

need more preprocessing steps to identify people.

84.00%

86.00%

88.00%

90.00%

92.00%

94.00%

96.00%

98.00%

100.00%

GAR using Combined LVQ based on Morphological features

GAR using Combined LVQ based on SIFT keypoint Descriptors

0.050.10.25α

◌Accurac

y



References 1. S. THEODORIDIS and K. KOUTROUMBAS,

Pattern recognition. Academic Press, 31 Oct 2008. 2. Palm Vein Authentication Technology White

Paper march of Fujistu 2008. 3. M. H. KHAN, R. K. SUBRAMANIAN, and N.

M. KHAN," Representation of hand dorsal vein features using a low dimensional representation integrating cholesky decomposition,". In the IEEE 2nd International Congress on Image and Signal Processing, Vol.3, 17-19 October 2009, Tianjin, China.

4. F. KUO-CHIN, C. L. LINAND, and L. WIN-LONG, "Biometric verification using thermal images of palm-dorsa vein-patterns," Institute of Computer Science and Information Engineering, National Central University.16th IPPR Conference on Computer Vision, Graphics and Image Processing (CVGIP 2003).

5. R. BRUNELLI, D. FALAVIGNA, "Person identification using multiple cues," IEEE Transactions on Pattern Analysis and Machine Intelligence 1995.

6. J. KITTLER, R. P.W. DUIN, "The combining classifier: to train or not to train," in Proceedings of the International Conference on Pattern Recognition, vol. 16, no. 2, pp. 765–770, 2002.

7. L. HONG and A. K. JAIN, "Integrating faces and fingerprints for personal identification," IEEE Trans. PAMI, vol. 20, no. 12, pp. 1295-1307, 1998.

8. S. BEN-YACOUB, Y. ABDELJAOUED, and E. MAYORAZ, "Fusion of face and speech data for person identity verification," IEEE Trans. Neural Networks, vol. 10, no. 5, pp. 1065-1075, 1999.

9. A. ROSS, and A. K. JAIN, "Information fusion in biometrics," Pattern Recognition Letters, vol. 24, no. 13, pp. 2115-2125, 2003.

10. D. Bhattacharyya, P. Das, K. Tai-Hoon, and B. S. Kumar, "Vascular pattern analysis towards pervasive palm vein authentication," Heritage Institute of Technology, Kolkata, India and Hannam University, Daejeon, Korea ,Journal of Universal Computer Science, vol. 15, no. 5 2009.

11. Q. Gao, and G. S. Moschytz, "Fingerprint feature extraction using cnns," European Conference on Circuit Theory and Design, pp. 97-100, Espoo, Finland, 2001.

12. S. Malki, , Y. Fuqianga and L. Spaanenburg, " Vein feature extraction using dt-cnns.10th International Workshop on Cellular Neural Networks and Their Applications," Istanbul, Turkey, 28-30, pp.1-6. August 2006

13. H.B. Kekre, and V. A. Bharadi,

"Signature Recognition Using Cluster Based Global Feature," IEEE International Conference (Iacc 2009).

14. P. Verlinde, G. Chollet, and M. Acheroy, "Multi-Modal Identity Verification Using Expert Fusion," Information Fusion. no. 1, Pp. 17-33, Elsevier, 2000.

15. Http://En.Wikipedia.Org/Wiki/Discrete_Cosine_Transform.

16. G. Rodriguez, and Others, " A Comparative Evaluation Of Fusion Strategies For Multimodal Biometric Verification," Avbpa Conference, 23 August 2010.

17. A. ROSS, And R. Govindarajan, " Feature Level Fusion Using Hand And Face Biometrics," Motorola Inc., Anaheim, Ca 92806 USA, June 2011.

18. L. David, "Object Recognition From Local Scale-Invariant Features," Proceedings Of The International Conference On Computer Vision. 2. Pp. 1150–1157.Doi:10.1109/Iccv, 1999.

19. A. Rattani, D. R. Kisku, M. Bicego, and M. Tistarelli, "Feature Level Fusion Of Face And Fingerprint Biometrics," Arxiv:1002.2523v1, 12 Feb 2010

Feature Level Fusion of Palm Veins and Signature Biometrics · on multimodal palm vein and...

Documents

Transcript of Feature Level Fusion of Palm Veins and Signature Biometrics · on multimodal palm vein and...