8/13/2019 2008 - Face Recognition by Combining Gabor Wavelets and Nearest Neighbor Discriminant Analysis - In

1/5

Face Recognition by Combining Gabor Wavelets and Nearest Neighbor

Discriminant Analysis

Kadir Krta, Onur Dolu, Muhittin Gkmen

Istanbul Technical University, Computer Engineering Dept.

34469, Maslak, Istanbul, Turkey

{kirtac, doluo, gokmen}@itu.edu.tr

Abstract

One of the successful approaches in face recognition is

the Gabor wavelets-based approach. The importanceof the Gabor wavelets lie under the fact that the ker-

nels are similar to the 2-D receptive field profiles of

the man visual neurons, offering spatial locality, spatial

frequency and orientation selectivity. In this work, we

propose a new combination of a Gabor wavelets-based

method for illumination and expression invariant face

recognition. It applies the Nearest Neighbor Discrim-

inant Analysis to the augmented Gabor feature vectors

obtained by the Gabor wavelets representation of fa-

cial images. To make use of all the features provided

by different Gabor kernels, each kernel output is con-

catenated to form an augmented Gabor feature vector.The feasibility of the proposed method has been suc-

cessfully tested on Yale database by giving a compari-

son with its predecessor, NNDA. The effectiveness of the

method is shown by a comparative performance study

against standard face recognition methods such as the

combination of Gabor and Eigenfaces and the combi-

nation of the Gabor and Fisherfaces, using a subset of

the FERET database containing a total of 600 facial

images of 200 subjects exhibiting both illumination and

facial expression variations. The achieved recognition

rate of 98 percent in the FERET test shows the efficiency

of the proposed method.

1. Introduction

Due to the highly informative and discriminative nature

of the face stimuli, face recognition has been regarded

as a biometric identification application in computer vi-

sion community. However, because of the 3-D nature of

the stimuli, the appearance of a single face can change

dramatically due to the variations of light, pose and ex-

pressions. One of the successful approaches for robust

face recognition is the Gabor wavelets-based approach.

After the pioneering work of Daugman [1], extending

1-D Gabor wavelets to 2-D, Gabor wavelets have beenextensively used both in many image processing and

computer vision applications. The efficiency of the Ga-

bor wavelets lie under the fact that they exhibit similar-

ity to the 2-D receptive field profiles of the man visual

neurons, offering spatial locality, spatial frequency and

orientation selectivity. Therefore, the Gabor wavelets

representation of facial images should be robust to illu-

mination and facial expression variations.

In [2], Lades et al. introduced the use of Gabor

wavelets for face recognition using the Dynamic Link

Architecture. The DLA forms and stores deformable

model graphs whose vertices are labeled by Gabor jets

computed from a rectangular subgrid centered over the

object to be stored. It then applies a flexible graph

matching between model graphs and the image graph.

In [3], Wiskott et al. extended the DLA architecture

to Elastic Bunch Graph Matching. They have utilized

phase information for accurate node localization and

introduced the bunch graph data structure which com-

bined jets of a small set of individuals. In recognition,

matching takes place between a bunch graph and an im-

age graph. In [4], Donato et al. utilized Gabor wavelets

for facial action classification. They achieved the best

results with the Gabor wavelets approach and with an

ICA-based scheme. In [5], Liu et al. combined Ga-bor wavelets and Enhanced Fisher Linear Discriminant

Model for face recognition. They applied a set of Ga-

bor kernels to whole face image and resulted with aug-

mented Gabor feature vectors. They then applied fea-

ture extraction on the augmented Gabor feature vectors

using EFM.

In this work, we propose a new combination of a Ga-

bor wavelets-based method, Gabor+NNDA. It applies

the Nearest Neighbor Discriminant Analysis [6] to the

978-1-4244-2881-6/08/$25.00 2008 IEEE

8/13/2019 2008 - Face Recognition by Combining Gabor Wavelets and Nearest Neighbor Discriminant Analysis - In

2/5

augmented Gabor feature vectors obtained by the Ga-

bor wavelets representation of facial images. To make

use of all the features provided by different Gabor ker-

nels, each kernel output is concatenated to form an aug-

mented Gabor feature vector. The feasibility of the

proposed method has been successfully tested on Yale

database [7] by giving a comparison with its predeces-sor, NNDA. The effectiveness of the proposed method

is shown by a comparative performance study against

standard face recognition methods such as the combi-

nation of Gabor and Eigenfaces and the combination of

the Gabor and Fisherfaces, using a subset of the FERET

database containing a total of 600 facial images of 200

subjects exhibiting both illumination and facial expres-

sion variations. The achieved recognition rate of 98 per-

cent in the FERET test shows the efficiency of the pro-

posed method.

2. Two-dimensional Gabor wavelets

Gabor wavelets(kernels) are a set of filters ,, whichare defined as,

,(z) =k,2

2 e

( k,2z2

22 )

[eik,z e2/2], (1)

whereand indicates the orientation and scale of thekernels, z= (x,y) denotes the image coordinates,.denotes the norm operator andk, is the wave vector.

Each kernel is a product of a Gaussian envelope func-

tion and a complex plane wave.The wave vectork,is defined as,

k,= kei, (2)

where k=kmax/f and= /8. kmax is defined

as maximum frequency, is defined as the width ofthe Gaussian along the x and y axis and f is the spac-

ing factor between kernels in the frequency domain [2].

Lades et al. investigated=2, f=

2 andkmax=/2 yielding with optimal results along with 5 scales, {0, . . . , 4}, and 8 orientations, {0, . . . , 7}. In[8], Shen et al. also discussed tuning the Gabor ker-

nel parameters and after two experiments they showed

that 5 scales and 8 orientations yielded with optimal

recognition performance. The first exponential term in

the square brackets in (1) indicates the oscillatory part

while the second exponential term compensates for the

DC value of the kernel, to make the filter independent

from the absolute intensity of the image. The kernel, ex-

hibiting complex response, combines a real part(cosine

part) and an imaginary part(sine part). The wavelets are

parameterized byk,, which controls the width of the

Gaussian window and scale and orientation of the os-

cillatory part. Theparameter determines the ratio ofwindow width to scale, in other words, the number of

the oscillations under the envelope function [2].

2.1. Two-dimensional Gabor wavelets-based

feature representation

The 2-D Gabor wavelets representation of an image is

the convolution of the image with a family of kernels

,, where is the orientation and is the spatialscale of the kernel. The convolution of an image I(z)with a Gabor kernel, results withG,(z)which isdefined as,

G,(z) =I(z) ,, (3)where z = (x,y) and denotes the convolution opera-tion. Thus, the set S={G,(z): {0, . . . , 7},

{0, . . . , 4

}}forms the Gabor wavelets representation of

an imageI(z).

3. Nearest Neighbor Discriminant Analysis

Nearest neighbor discriminant analysis(NNDA) is a

nonparametric feature extraction method which forms

the between-class and within-class scatter matrices in

a nonparametric way [6]. Considering a c-class prob-

lem with classes Ci{i = 1, 2, . . . , c} and training samples{x1,x2, . . . ,xN}, the extra-class and intra-class neighborof a sample xnCi is defined as,

xEn =argminz z xn,z Ci, (4)

xIn=argminz

z xn,z Ci,z =xn. (5)

The nonparametric extra-class and intra-class distances

are defined as,

En =xn xEn, (6)In=xn xIn. (7)

Using the extra-class and intra-class distances defined

above, the nonparametric between-class and within-

class scatter matrices are defined as follows,

SB=N

n=1

wn(En)(

En)

T, (8)

SW=N

n=1

wn(In)(

In)

T, (9)

wherewnis defined as,

wn= In

In+En . (10)

8/13/2019 2008 - Face Recognition by Combining Gabor Wavelets and Nearest Neighbor Discriminant Analysis - In

3/5

wn is introduced to emphasize the samples in class

boundaries and deemphasize the samples in class cen-

ters. Utilizing the fact that the accuracy of the nearest

neighbor classification can be directly computed by,

n=

En

2

In

2, (11)

Qiu and Wu [6] reached to the following solution for

the computation of the projection matrix W,

W=argmaxW

tr(WT(SB SW)W). (12)

Thus, the columns of the projection matrixWare them

leading eigenvectors ofSB SW, corresponding to themgreatest eigenvalues. Due to the computational ineffi-

ciency of obtaining extra- and intra-class nearest neigh-

bors when the data dimensionality is high, first PCA is

applied to reduce the dimension of the data to N-1(the

rank of the total scatter matrix) by removing the null

space of the total scatter matrix.

To keep the nonparametric extra-class and intra-class

differences of the high dimensional space consisted

with the projected extra-class and intra-class differ-

ences, Qiu and Wu [6] proposed stepwise dimensional-

ity reduction process. In this scheme, the nonparamet-

ric between-class and within-class matrices are recom-

puted each time in the current dimensionality. This pro-

cess is repeated until reaching the desired dimensional-

ity.

They also utilizedk-NN classification criterion in the

training phase. The nonparametric extra- and intra-class

differences are rewritten as,

E =x xE[k/2], (13)

I =x xI([k/2]+1), (14)where,xI([k/2]+1)is defined as the intra-class([k/2] + 1)-

th nearest neighbor and xE[k/2] is defined as the [k/2]-

th extra-class nearest neighbor of the sample x. If the

distance fromx to xI([k/2]+1) is smaller than the distance

from x to xE[k/2], x will be classified correctly by k-NN

classifier.

4. Face recognition by combining Gabor

wavelets and Nearest Neighbor Discrim-

inant Analysis

After obtaining the feature representation described

in (3), eachG,(z)is downsampled by a scale factorand normalized by zero-mean and unit variance opera-

tion. Downsampling is carried out by first smoothing

G,(z) image using a 5 5 Gaussian window and

then picking out every (k.) th sample both in x

and y directions, where 1k( w), k Z and wis the width of the G,(z). The training algorithm ofGabor+NNDA is given in Table 1.

Table 1: Training algorithm of Gabor+NNDA.

(1)GivenD dimensional samples{x1,x2, . . . ,xN},d-dimensional discriminant subspace is searched.

(2)Normalize each sample xi to zero-mean and unit

variance.

(3)Apply a set of 40 Gabor kernels(5 scales and 8

orientations) to each samplexi, resulting with

Gi,,(z); i {1, . . . ,N}, {0, . . . ,7}, {0, . . . ,4}.

(4)Downsample each filter output Gi,,(z) with a

factor of to achieve G()i,,(z), and normalize the

finalG()i,,(z)to zero-mean and unit variance.

(5)Concatenate the rows(or columns) of each resul-

tant G()i,,(z) to form an augmented feature vector

x()i .

x()i ={G()i,0,0|G()i,0,1| . . . |G()i,7,4}t.

(6)Form the final Gabor feature matrix X(), by as-

sembling eachx()i in columns, side by side.

X() = {x()1 |x()2 | . . . |x()N}.

(7)Apply PCA to X() to learn the PCA projection

matrixTpca.

Tpca= [1|2| . . . |N1], TpcaN1D.

(8)Project feature matrixX() with the learned PCA

model.Ypca=Tt

pcaX().

(9)Apply NNDA on Ypcato learn the NNDA projec-

tion matrix.

Tnnda= [1|2| . . . |d], TnndadN1.

(10)Project Gabor+PCA feature matrix Ypcawith the

learned NNDA model.Y=TtnndaYpca.

8/13/2019 2008 - Face Recognition by Combining Gabor Wavelets and Nearest Neighbor Discriminant Analysis - In

4/5

In recognition, steps (2) to (5) of the training algorithm

is applied similarly to a test image y, and y() is ob-

tained. Then,y() is projected using the projection ma-

trixTnnda,

y=Ttnnday(). (15)

Finally, L2 distance measure is applied to identify ywith the label of the closest feature vector in Ga-

bor+NNDA feature space,

L2( y,y) = ( y y)t( y y). (16)

5. Experimental Results

We performed the first set of experiments on Yale

database [7], to compare the proposed method with its

predecessor, NNDA.

Yale database contains 165 images of 15 subjects.

There are 11 images per subject, one for each of the

following facial expressions or configurations: center-light, w/glasses, happy, left-light, w/no glasses, normal,

right-light, sad, sleepy, surprised, and wink.

Scaled images of size 64x64 are used in this exper-

iment. The downsampling factor of Gabor+NNDA is

64, i.e., = 64. 5 partitions are formed by randomlyselecting 5 images for training and leaving the remain-

ing 6 images for testing in each trial. Thestep size of

NNDA is set to 5. The reduced subspace is of 14 dimen-

sions. Note that the original dimensionality is 64x64

= 4,096. While the step size is constant, (k, alpha)tuple is changed as, (k, alpha) : k {1, 3, 5}, alpha{0, 1, 2, 3}.The result is shown in Fig. 2.

(1,0) (1,1) (1,2) (1,3) (3,0) (3,1) (3,2) (3,3) (5,0) (5,1) (5,2) (5,3)

0.76

0.78

0.8

0.82

0.84

0.86

0.88

0.9

0.92

(k,alpha)

recognition

rate

NNDA

Gabor+NNDA

Figure 1: Relative face recognition performance of

Gabor+NNDA and NNDA on Yale database. Re-

duced feature dimension is 14. Step size is set to 5.

From Figure 1, it can be easily observed that Ga-

bor+NNDA outperformed NNDA on all (k, alpha) tu-ples. Gabor+NNDA reaches 90 percent accuracy in 14

feature dimension, along with parametersstep size = 5,

alpha=1, andk=5.The next experiment is the performance compari-

son of the two standard methods Gabor+Eigenfaces and

Gabor+Fisherfaces [5], with the proposed method Ga-

bor+NNDA, using a subset of the FERET database [9].

The FERET database contains 1564 sets of images fora total of 14,126 images that includes 1199 individuals

and 365 duplicate sets of images.

The experiment includes 600 facial images of 200

subjects such that each subject has three images of size

256x384 with 256 gray scale levels. Each of these three

images shows different variations with the following

configuration: the first image is in neutral expression,

the second image shows different expression than the

first one and the last image contains different illumina-

tion than first two images. First, the centers of the eyes

are manually detected, then they are aligned to prede-

fined locations by rotation and scaling transformations.Finally, face image is cropped to the size of 128x128

to extract the facial region, which is further normalized

by zero-mean and unit variance operation. The training

parameters of Gabor+NNDA is set as, step size = 13,

alpha=1, andk=1. The result is shown in Figure 2.

0 20 40 60 80 1000

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

number of features

recognitionr

ate

Gabor+Eigenfaces

Gabor+Fisherfaces

Gabor+NNDA

Figure 2:Comparative face recognition performance

of Gabor+Eigenfaces, Gabor+Fisherfaces and Ga-

bor+NNDA, on the augmented Gabor feature vector

X

()

, downsampled by a factor of 64, i.e., = 64.

In the experiment, Gabor+NNDA achieves the highest

recognition rate by 98 percent in 65 feature dimensions

while Gabor+Fisherfaces achieves a 92.6 percent accu-

racy and Gabor+Eigenfaces achieves 40.6 percent accu-

racy in the same feature dimension. Figure 2 shows the

overall superior performance of the proposed method

over Gabor+Eigenfaces and Gabor+Fisherfaces meth-

ods.

8/13/2019 2008 - Face Recognition by Combining Gabor Wavelets and Nearest Neighbor Discriminant Analysis - In

5/5

6. Discussion and Conclusion

2-D Gabor wavelets are optimally localized in the space

and frequency domains, and result with illumination,

expression and pose invariant image features.

Nearest Neighbor Discriminant Analysis(NNDA) [6]was shown to be an efficient nonparametric feature ex-

traction tool from the point of view of nearest neighbor

classification. It does not suffer from the small sam-

ple size problem and it does not need to estimate any

parametric distribution because of its nonparametric na-

ture. Moreover, it does not suffer from the singularity

of the within-class scatter matrix as no matrix inversion

is required in eigenvector computation. Due to the non-

parametric structure and stepwise dimensionality reduc-

tion process, the training time complexity of NNDA is

greater than that of Fisherfaces method [7]. However,

NNDA gives higher performance than Fisherfaces in

terms of classification accuracy and both methods have

the same time complexity in recognition phase.

Gabor+NNDA extracts important discriminant fea-

tures both utilizing the power of Gabor wavelets and

NNDA. The efficiency of the approach is shown with

experiments both in Yale and FERET database. It

achieved a 98 percent classification accuracy in 65 fea-

ture dimension, outperforming both standard methods

such as Gabor+Fisherfaces and Gabor+Eigenfaces [5],

using a 200 class subset of FERET database exhibiting

both illumination and expression variations. We also

evidenced the increase of classification accuracy in the

Yale test, with increasing k and decreasing alpha val-ues, which also agrees with the theoretical results of

[6]. The FERET test also showed the favor of stepwise

dimensionality reduction process, by reaching 98 per-

cent accuracy after 13 steps. However, the chosen step

sizes greater than 13 did not increase the classification

accuracy and did not seem to be effective in terms of

time complexity. In our experiments, training with 400

FERET images took 80 seconds for 13 steps by using

Matlab R14 on our Intel P4 3.2 Ghz configuration.

It should be noted that no suggestion is given in the

original NNDA work [6], on how to select parameters

alpha, k and step size. An optimization scheme such

as Evolutionary Computing can be suggested to pro-

vide the optimal parameter selection for training. But

this would also increase the time complexity of the sys-

tem. Due to the effectiveness of kernel approaches in

Gabor wavelets-based face recognition [8], NNDA can

be extended to a kernel approach and Gabor features

then can be utilized in kernel-NNDA space. Moreover,

instead of applying a simple distance measure like L2

norm in classification, more sophisticated classification

schemes such as Support Vector Machines or Neural

Networks can be applied. However, it is no guarantee

that SVM and Neural Network classifiers would be ef-

fective in Gabor+NNDA feature space.

Acknowledgments

This work has been partially supported by

TUBITAK(National Science Council of Turkey)

under the grant National Scholarship Programme

for MSc. Students and TUBITAK project num-

bered 104E121. The authors would like to thank

Fatih Kahraman and Abdulkerim apar for valuable

discussions.

References

[1] J. G. Daugman, Uncertainty relation for resolution

in space, spatial frequency, and orientation optimized

by two-dimensional visual cortical filters, J. Opt. Soc.Amer., vol. 2, no. 7, pp. 11601169, 1985.

[2] M. Lades, J. C. Vorbruggen, J. M. Buhmann, J. M. Lange,

C. von der Malsburg, R. P. Wrtz, and W. Konen, Dis-

tortion Invariant Object Recognition in the Dynamic Link

Architecture, IEEE Trans. Computers, vol. 42, no. 3,

pp. 300311, 1993.

[3] L. Wiskott, J.-M. Fellous, N. Krger, and C. von der

Malsburg, Face Recognition by Elastic Bunch Graph

Matching, IEEE Trans. on Pattern Analysis and Machine

Intelligence, vol. 19, no. 7, pp. 775779, 1997.

[4] G. Donato, M. S. Bartlett, J. C. Hager, P. Ekman, and T. J.

Sejnowski, Classifying Facial Actions, IEEE Trans.

Pattern Anal. Mach. Intell., vol. 21, no. 10, pp. 974989,

1999.

[5] C. Liu and H. Wechsler, Gabor feature based classifica-

tion using the enhanced fisher linear discriminant model

for face recognition, IEEE Transactions on Image Pro-

cessing, vol. 11, no. 4, pp. 467476, 2002.

[6] X. Qiu and L. Wu, Nearest Neighbor Discriminant Anal-

ysis,IJPRAI, vol. 20, no. 8, pp. 12451260, 2006.

[7] P. N. Belhumeur, J. P. Hespanha, and D. J. Kriegman,

Eigenfaces vs. Fisherfaces: Recognition Using Class

Specific Linear Projection, IEEE Trans. Pattern Anal.

Mach. Intell., vol. 19, no. 7, pp. 711720, 1997.

[8] L. Shen, L. Bai, and M. C. Fairhurst, Gabor wavelets

and General Discriminant Analysis for face identification

and verification, Image Vision Comput., vol. 25, no. 5,pp. 553563, 2007.

[9] P. J. Phillips, H. Wechsler, J. Huang, and P. J. Rauss,

The FERET database and evaluation procedure for face-

recognition algorithms, Image Vision Comput., vol. 16,

no. 5, pp. 295306, 1998.