Comparative study of face recognition techniquesthat use joint transform correlation and principalcomponent analysis

A. Alsamman and Mohammad S. Alam

Face recognition based on principal component analysis (PCA) that uses eigenfaces is popular in facerecognition markets. We present a comparison between various optoelectronic face recognition tech-niques and a PCA-based technique for face recognition. Computer simulations are used to study theeffectiveness of the PCA-based technique, especially for facial images with a high level of distortion.Results are then compared with various distortion-invariant optoelectronic face recognition algorithmssuch as synthetic discriminant functions (SDF), projection-slice SDF, optical-correlator-based neuralnetworks, and pose-estimation-based correlation. © 2005 Optical Society of America

OCIS codes: 150.0150, 250.0250.

1. Introduction

The task of face recognition is complicated by varia-tions in a three-dimensional face, which translateinto distortions in two-dimensional (2D) images. Dis-tortions that hinder successful recognition are causedby scale variations, changes in illumination sourcesand direction, background clutter, occlusion, in-planerotation, and out-of-plane rotation.

The use of principal component analysis (PCA), orthe Karhunen–Loeve expansion, on facial images wasfirst introduced by Turk and Pentland in 1991 (Refs. 1and 2) and has been gaining in popularity ever since.The process assumes that 2D facial images belong to aface space that can be spanned by its principal compo-nents. The goal of the process is to reduce the compu-tational complexity of face matching by exploitation ofthe similarities in facial images. Although the reduc-tion of this space into principal eigenvectors, calledeigenfaces, should enable electronic PCA-based sys-tems to operate in near real time, in reality, this de-pends on factors such as the number of eigenfaces, thepreprocessing time, and the database size.

The speed and massive parallelism of optoelec-tronic correlators diminish the need for image-spacereduction. To overcome distortions, we use algo-rithms to generate composite images or filters such assynthetic discriminant functions (SDF),3,4 projection-slice SDF,5,6 and optical-correlator-based neural net-works.7,8 Recently9 we proposed a faster and moreaccurate algorithm that uses pose estimation to re-place composite image generation.

A variety of optical correlators have been describedin the literature,10–13 varying in design complexityand correlation-output quality. All our research hasbeen based on the fringe-adjusted joint transform cor-relator (JTC) because it operates in a 5f system, usesreal-valued images, and employs a real-valued pole-less fringe-adjusted filter that is computed a priori.

In this paper we study the effectiveness of the PCA-based face recognition technique on a database con-taining 15 classes of images and 320 facial images. Theimages contain a variety of distortion effects, most no-tably in-plane and out-of-plane rotation. The perfor-mance of this technique is then compared with theclassification results obtained from optical correlationsbased on SDF, projection-slice SDF, optical neural net-works, and pose estimation, as reported in Ref. 14.

In Section 2 we describe how eigenfaces are pro-duced and used for classification. The principles offringe-adjusted JTC are presented in Section 3. Sec-tion 4 contains a summary of the simulation experi-ment and its results. The conclusions are outlined inSection 5.

A. Alsamman (a.alsamman@uno.edu) is with the Department ofElectrical Engineering, University of New Orleans, Louisiana70148. M. S. Alam is with the Department of Electrical Engineer-ing, University of South Alabama, Mobile, Alabama 36688-0002.

Received 20 May 2004; revised manuscript received 27 Septem-ber 2004; accepted 1 October 2004.

0003-6935/05/050688-05$15.00/0© 2005 Optical Society of America

688 APPLIED OPTICS � Vol. 44, No. 5 � 10 February 2005

2. Eigenfaces

Eigenfaces are the principal components of the facial-image training database. By treating 2D facialimages as one-dimensional facial vectors, the eigen-vectors and their corresponding eigenvalues are cal-culated from the covariance matrix of these facialvectors. The vectors with the highest values are se-lected as principal components, also known as eigen-faces. The mathematical steps are described below.

Given a set of N training images, ti�x, y�, of size X� Y, the 2D images are first converted into one-dimensional vectors, �i�x�, of size XY. The differencevectors, �i, are then calculated by subtracting theaverage, �, from each vector, described by the follow-ing equations:

�i � (1�N)�i�1

N

�i, (1)

�i � �i � �i, i � 1, 2, . . . N. (2)

The computation of the covariance matrix of thedifference vectors is then performed:

C � (1�N) �i � 1

N

�i�iT�(1�N)AAT, (3)

where the superscript T denotes the transpose oper-ation and A is the matrix containing all the differencevectors, given as

A � [�1 �2 . . . �N]. (4)

The complexity of calculating the eigenvectors ofthe covariance matrix C, which is an XY � XY ma-trix, can be reduced by solving the eigenvectors ofATA, which is a small N � N matrix [2]. Thus theeigenvectors of C are defined as

ui � Avi � �k�1

N

vik�k, (5)

where vi is the eigenvector of the ATA matrix. As seenin Eq. (5), each eigenvector is simply a weighted sumof the training images. The most significant eigenvec-tors, N�, are then selected based on the largest eig-envalues as the principal components, or eigenfaces,which span the face subspace.

Any facial image in that subspace can be described asa linear combination of eigenfaces. The contribution ofeach eigenface to a facial image, �, in the subspace can becalculated as a projection coefficient, �k, such that

�k � ukT(� � �), k � 1, 2, . . . N�. (6)

For a certain class of images, the projectionweights, �k, form a vector �k. A test image is classi-fied as belonging to a class k if the Euclidian distance,ki, between the kth projection weight vectors, �k, and

the test image, �i, is minimal, where ki is defined as

ki � � �i � �k � . (7)

3. Fringe-Adjusted Joint Transform Correlator

This section briefly describes the optical fringe-adjusted JTC design. In a JTC system a referenceimage, r�x, y�, and a target image, t�x, y�, are spatiallymultiplexed in a joint image, f�x, y�, as follows:

f(x, y) � r(x, y y�) t(x, y � y�). (8)

The Fourier transform, F�u, v�, is produced opticallyby use of a converging lens, and the joint power spec-trum (JPS) that is recorded by a square-law device is

|F(u, v)|2 � |R(u, v)|2 |T(u, v)|2

2|R(u, v) � T(u, v)|cos[�r(u, v)� �t(u, v) 2vy�]. (9)

To protect the correlation output against broad side-lobes and noise, the JPS is multiplied by a fringe-adjusted filter (FAF) expressed as12

HFAF(u, v) �B(u, v)

A(u, v) |R(u, v)|2, (10)

where A�u, v� and B�u, v� may be constants of func-tions and are used to control the gain and to eliminatethe problems related to the poles. An inverse Fouriertransform is then performed on the fringe-adjustedJPS by means of a converging lens. In the case of amatch, �u, v� � T�u, v�, and the modified JPS becomes

G(u, v) � 2 2 cos[�r(u, v) � �r(u, v) 2vy0].(11)

The overall process results in sharper, larger, andmore distinct correlation peaks. It should also be notedthat the FAF is a real-valued function and does notinvolve phase terms. The FAF can be computed a pri-ori and does not affect the system’s processing speed.This is advantageous compared with binary JTC sys-tems in which filters must be computed on-line.

The correlation output can be further improved byremoving the dc terms and the autocorrelation termsfrom the JPS. This is accomplished by subtracting theFourier transform of the target and reference imagesfrom the JPS13 as follows:

|F(u, v)|2 � |R(u, v)|2 |T(u, v)|2

2|R(u, v) � T(u, v)|cos[�r(u, v)

� �t(u, v) 2vy�] � |R(u, v)|2

� |T(u, v)|2

� 2|R(u, v) � T(u, v)|cos[�r(u, v)� �t(u, v) 2vy�]. (12)

10 February 2005 � Vol. 44, No. 5 � APPLIED OPTICS 689

This is particularly useful in the presence of multipletargets in the input scene.

The diagram in Fig. 1 depicts the components of anoptoelectronic implementation of the fringe-adjustedJTC by use of the Fourier image-subtraction tech-nique. In this system multiplication of the JPS by theFAF and Fourier plane image subtraction are imple-mented digitally by a computer. An all-optical imple-

mentation is possible but expensive, and alignmentissues must be addressed.

4. Simulation Results

A database was constructed specifically for this re-search. The database images were collected from 15subjects and divided into a training database with 9training images per class (see Fig. 2) and a testingdatabase (see Fig. 3). Images in the testing databaseincluded extreme distortions in rotation, scale, andocclusion and included subjects without their glassesand wearing hats or sunglasses. Two different exper-iments were conducted. In the first, only the firstthree training images were used; in the second, allnine images were used. As seen in Fig. 2, the firstthree images in each class contained the least distor-tion of all the images.

Table 1 summarizes the results obtained. Thetraining database was used to generate eigenfacesand composite images for the optoelectronic tech-niques, as described in Ref. 14. The table shows thatthe PCA-based eigenface technique was the least suc-cessful, with a success rate of 58% when three train-ing images are used from each class. The success rateincreases to 81% when the full training database isused but is still less effective when compared with theSDF technique.

The PCA technique was found to be less effectivethan the optoelectronic face recognition techniquesfor databases containing images with a variety ofdistortion effects. The PCA technique assumes thatall distortions from the face space can be reduced tothe principal vectors of that space. From our test we

Fig. 1. Fringe-adjusted JTC: BS, beam splitter; L1–L3, lenses;LS, laser source; SLM, spatial light modulator.

Fig. 2. Training database.

690 APPLIED OPTICS � Vol. 44, No. 5 � 10 February 2005

discovered that the number of dominant vectors wereclose to the number of training images for mostclasses. Thus the face space is more complex and isdifficult to linearize by means of PCA. Even when allthe eigenvectors were included, the results improvedonly slightly. It is also noted that the PCA techniqueinvolved converting facial images to vectors, makingthe process of recognition highly susceptible even toslight spatial shifts. Electronically, the PCA-basedtechniques are faster. However, in the presence ofvarious image distortions or in large databases, thenumber of eigenfaces increases dramatically suchthat the computational intensity causes latency.

5. Conclusion

A comparative study of face recognition techniqueshas been presented in this paper. The optical-basedtechniques have been found to perform better thanthe electronic PCA-based technique in the case ofimages that exhibit a variety of extreme distortions.It was also noted that PCA was not effective in re-ducing the number of dominant vectors for imageswith a high degree of distortion levels. The distancemeasure was not effective in measuring the similar-ity between images. The correlation-based optoelec-tronic techniques were more effective, especiallywhen more training images were included in the ex-periment. The pose-estimation method was shown tobe superior to the other optoelectronic techniques ow-ing to its capability of incorporating three-dimensional information into the recognition process.

References1. M. Turk and A. Pentland, “Face recognition using eigenfaces,”

J. Cogn. Neurosci. 3, 586–591 (1991).2. M. Turk and A. Pentland, “Eigenfaces for recognition,” J. Cogn.

Neurosci. 3, 71–86 (1991).3. C. F. Hester and D. Casasent, “Multivariant technique for mul-

ticlass pattern recognition,” Appl. Opt. 19, 1758–1761 (1980).4. M. S. Alam and A. R. Alsamman, “Invariant face recognition,”

Microwave Opt. Techn. Lett. 27, 418–423 (2001).

Fig. 3. Testing database.

Table 1. Summary of the Successful Classification Rate for VariousAlgorithms

Technique

Success Rate (%)

Using 3 Images Using 9 Images

Eigenfaces 58 81SDF 66 85PSDFa 78 88Neural net — 93Pose estimation 75 95

aPSDF, projection-slice SDF.

10 February 2005 � Vol. 44, No. 5 � APPLIED OPTICS 691

5. V. R. Riasati and M. Abushagur, “Projection-slice syntheticdiscriminant functions for optical pattern recognition,” Appl.Opt. 36, 3022–3034 (1997).

6. A. Alsamman and M. S. Alam, “Invariant face recognitionusing a projection slice synthetic discriminant function basedalgorithm,” Opt. Eng. 41, 87–93 (2002).

7. B. Javidi, J. Li, and Q. Tang, “Optical implementation of neu-ral networks for face recognition by the use of joint transformcorrelators,” Appl. Opt. 34, 3950–3962 (1995).

8. A. R. Alsamman and M. S. Alam, “Face recognition usingfringe-adjusted joint transform correlator based neural net-work,” J. Opt. Eng. 41, 2768–2777 (2002).

9. A. Alsamman and M. S. Alam, “Face recognition through poseestimation and fringe-adjusted joint transform correlation,” J.Opt. Eng. 42, 560–567 (2003).

10. J. W. Goodman, Introduction to Fourier Optics (McGraw-Hill,New York, 1968).

11. F. T. S. Yu and X. J. Lu, “A real-time programmablejoint transform correlator,” Opt. Commun. 52, 10–16(1984).

12. M. S. Alam and M. A. Karim, “Fringe-adjusted joint transformcorrelation,” Appl. Opt. 32, 4344–4350 (1993).

13. O. Perez and M. A. Karim, “Optical enhancements of jointFourier transform correlator by image subtraction,” in Auto-matic Object Recognition, F. A. Sadjadi, ed., Proc. SPIE 1471,255–264 (1991).

14. A. Alsamman and M. S. Alam, “Performance of invariant facerecognition techniques,” in Algorithms and Systems for OpticalInformation Processing VI, B. Javidi and D. Psaltis, eds., Proc.SPIE 4789, 167–176 (2002).

692 APPLIED OPTICS � Vol. 44, No. 5 � 10 February 2005