EE Dept. IIT Delhi 1

Face Recognition Using Fuzzy Fisherface Classifier

Presenters:Nilesh PadwalVivek K.Rajat Rastogi

EE Dept. IIT Delhi 2

ContentsIntroductionEigenface Based Approach (PCA)Disadvantages of Eigenfaces ApproachFisherface Approach (FLD)Disadvantage of Fisherface ApproachFuzzy Fisherface Approach (F-FLD)Comparison of PCA, FLD, F-FLDConclusionReferences

EE Dept. IIT Delhi 3

Introduction

Growing interest in biometric authenticationNational ID cards, Airport security, Surveillance.Fingerprint, iris, hand geometry, gait, voice, vein and face.

Face recognition offers several advantages over other biometrics:

Covert operation.Public acceptance.Data required is easily obtained and readily available.

Approaches include:Feature analysis, Appearance-Based.

EE Dept. IIT Delhi 4

PCA Based Approach (Eigenface)

Developed in 1991 by M.TurkRelatively simplePCA seeks directions that are efficient for representing the dataReduces the dimension of the dataSpeeds up the computational time

EE Dept. IIT Delhi 5

Eigenfaces, the algorithm

Original Images (I1,I2,……….,IM)

2

1

2

N

hh

h

⎛ ⎞⎜ ⎟⎜ ⎟=⎜ ⎟⎜ ⎟⎜ ⎟⎝ ⎠

M

2

1

2

N

bb

b

⎛ ⎞⎜⎜=⎜⎜⎜⎝ ⎠

M2

1

2

N

aa

a

⎛ ⎞⎜⎜=⎜⎜⎜⎝ ⎠

M……

2

1

2

N

aa

a

⎛ ⎞⎜ ⎟⎜ ⎟=⎜ ⎟⎜ ⎟⎜ ⎟⎝ ⎠

M

2

1

2

N

bb

b

⎛ ⎞⎜⎜=⎜⎜⎜⎝ ⎠

M2

1

2

N

hh

h

⎛ ⎞⎜ ⎟⎜ ⎟= ⎜ ⎟⎜ ⎟⎜ ⎟⎝ ⎠

M

EE Dept. IIT Delhi 6

Eigenfaces, the algorithm

The mean face can be computed as:

Mean-Face

2 2 2

1 1 1

2 2 21 , 8

N N N

a b ha b h

whereMM

a b h

+ + +⎛ ⎞⎜ ⎟+ + +⎜ ⎟Ψ= =⎜ ⎟⎜ ⎟⎜ ⎟+ + +⎝ ⎠

LL

M M ML

EE Dept. IIT Delhi 7

Eigenfaces, The Algorithm

Then subtract it from the training faces (Φi=Ii-Ψ)

2 2 2 2 2 2 2 2

2 2

1 1 1 1 1 1 1 1

2 2 2 2 2 2 2 2

1 1 1 1

2 2

, , , ,

,

m m m m

N N N N N N N N

m m

N N

a m b m c m d ma m b m c m d m

a b c d

a m b m c m d m

e m f me m f

e f

e m

− − − −⎛ ⎞ ⎛ ⎞ ⎛ ⎞ ⎛ ⎞⎜ ⎟ ⎜ ⎟ ⎜ ⎟ ⎜ ⎟− − − −⎜ ⎟ ⎜ ⎟ ⎜ ⎟ ⎜ ⎟= = = =⎜ ⎟ ⎜ ⎟ ⎜ ⎟ ⎜ ⎟⎜ ⎟ ⎜ ⎟ ⎜ ⎟ ⎜ ⎟⎜ ⎟ ⎜ ⎟ ⎜ ⎟ ⎜ ⎟− − − −⎝ ⎠ ⎝ ⎠ ⎝ ⎠ ⎝ ⎠

− −⎛ ⎞⎜ ⎟−⎜ ⎟= =⎜ ⎟⎜ ⎟⎜ ⎟−⎝ ⎠

r rr rM M M M M M M M

rrM M

2 2 2 2 2 2

1 1 1 1

2 2 2 2 2 2, ,m m

N N N N N N

g m h mm g m h m

g h

f m g m h m

− −⎛ ⎞ ⎛ ⎞ ⎛ ⎞⎜ ⎟ ⎜ ⎟ ⎜ ⎟− − −⎜ ⎟ ⎜ ⎟ ⎜ ⎟= =⎜ ⎟ ⎜ ⎟ ⎜ ⎟⎜ ⎟ ⎜ ⎟ ⎜ ⎟⎜ ⎟ ⎜ ⎟ ⎜ ⎟− − −⎝ ⎠ ⎝ ⎠ ⎝ ⎠

rrM M M M M M

EE Dept. IIT Delhi 8

Eigenfaces, The Algorithm

Now we build the matrix which is N2 by M

The covariance matrix which is N2 by N2

m m m m m m m mA a b c d e f g h⎡ ⎤= ⎣ ⎦r r r rr r r r

C o v A A Τ=

EE Dept. IIT Delhi 9

Eigenfaces, The Algorithm

Compute the Eigenvectors(N2), ui of AAT

Matrix AAT is very large, so computing all Eigenvectors not practicalCompute the Eigenvectors(M), vi of ATAAAT and ATA have the same eigenvalues and their eigenvectors are related as follows!!

ui=Avi

Keep only K eigenvectors (corresponding to K largest values)

EE Dept. IIT Delhi 10

Eigenfaces, The Algorithm

Each training image is projected to face space using

wj=ujTΦi

Each Φ, can be represented as a vector Ωi as follows!!!

EE Dept. IIT Delhi 11

Eigenfaces, The Algorithm

For each test image Ω is foundei=|Ω- Ωi| Test image is assigned to nearest training sample in the face space

EE Dept. IIT Delhi 12

Disadvantages of EigenfaceApproach

Sensitive to large variations in lightingFacial Expressions

Because it maximizes the total scatter across all classes but it retains the unwanted variations due to lighting and facial expressions

EE Dept. IIT Delhi 13

Different Lighting Conditions

Same person appears different and PCA suffers

courtesy:Source [4]

EE Dept. IIT Delhi 14

Fisherface ApproachIt is class specific method

Shapes the scatters in order to make it more reliable for classification

Principle:Projects the image set to a lower dimension space using PCA , followed by the FLD phasePCA helps us achieve non-singularity of SW prior to computation of optimal projection WFLD

EE Dept. IIT Delhi 15

Comparison of PCA and FLD for Two Class Data

courtesy:Source [1]

EE Dept. IIT Delhi 16

Fisher Linear DiscriminantBetween Class Scatter Matrix SB

Ni- Number of samples in class Ximi- mean image of class Xi

-mean of all the images

Within Class Scatter Matrix SW

c- number of classes in training samples

m

EE Dept. IIT Delhi 17

Fisher Linear DiscriminantOptimal Projection Matrix

It maximizes the ratio of the determinant of between class scatter matrix of projected patterns to the determinant of within class scatter matrix of projected patterns

Where is the set of eigenvectors of SB and SWcorresponding to m largest eigenvalues

Rank of SB is c-1 and rank of SW is at most N-c

EE Dept. IIT Delhi 18

Fisherface Approach

In the face recognition problem SW matrix is always singular (number of images in learning set N is much smaller the number of pixels in each image)Fisherface avoids this problem by projecting the image set to a lower dimension space using PCA and then applying standard FLD

EE Dept. IIT Delhi 19

Fisherface Approach

where

• Optimization for WPCA is performed over matrices with orthonormal columns

• While the optimization for WFLD is performed over matrices with orthonormal columns

• Optimal projection matrix

EE Dept. IIT Delhi 20

Fuzzy Fisherface Approach

More sophisticated usage of class assignment of patterns (faces)

Classification results affect the within-class and between-class scatter matrices

EE Dept. IIT Delhi 21

AlgorithmGiven set of feature vectors transformed by the PCA,

X = x1, x2, . . . , xN,Partition matrix

Which satisfies,

U=[ ] for 1, 2,..., and 1, 2, ...,ij i c j Nµ = =

11

c

i ji

µ=

=∑

10

N

ijj

Nµ=

< <∑

EE Dept. IIT Delhi 22

The Computations Of Membership DegreesCompute the Euclidean distance matrix between pairs of feature vectors in the training,Set diagonal elements of this matrix to infinity,Sort the distance matrix in ascending order,Collect the class labels of the patterns located in the closest neighborhood of the pattern,

EE Dept. IIT Delhi 23

The Computations Of Membership Degrees

Compute the membership grade to class for pattern ,

0.51 0.49( / ) if same as the label of the pattern

0.49( / ) if same as the label of the patternij

ij

n k i jth

n k i jth

+ =⎧⎪⎨ ≠⎪⎩

i jth

where is number of the neighbors of theijnjth data that belong to ith class

ijµ

EE Dept. IIT Delhi 24

FKNN Initialization

courtesy:Source [1]

EE Dept. IIT Delhi 25

AlgorithmResults of FKNN are used in computations of mean value and scatter covariance matrices,Mean vector of each class

The between class and within class fuzzy scatter matrices are respectively,

~1

1

N

i j jj

i N

i jj

Xm

µ

µ

=

=

=∑

∑

~ ~

1

~ ~

1 1

( )( )

( )( )i

k i

cT

i iFB ii

c cT

i iFW k k FWi x C i

S N m m m m

S x m x m S

=

= ∈ =

= − −

= − − =

∑

∑ ∑ ∑

EE Dept. IIT Delhi 26

AlgorithmThe optimal fuzzy projection WF-FLD and feature vector transformed by fuzzy fisherfacemethod are given by

~

arg max

( )

TFB

F FLD TWFB

T T Ti F FLD i F FLD i

W S WW

W S W

v W X W E z z

−

− −

=

= = −

EE Dept. IIT Delhi 27

Flowchart

courtesy:Source [1]

EE Dept. IIT Delhi 28

ComparisonEigenface

Fisherface

Fuzzy Fisherface

Test Image Recognized Image

courtesy:Source [1]

EE Dept. IIT Delhi 29

Comparison of Recognition Rates

EE Dept. IIT Delhi 30

ConclusionFuzzy fisherface approach outperform the other two methods for the datasets considered.Sensitivity variations in illumination and facial expression reduced substantially.Fuzzy sets can efficiently manage the vagueness and ambiguity of face images degraded by poor illumination component.

EE Dept. IIT Delhi 31

ReferencesKeun-Chang Kwak, Witold Pedrycz : Face Recognition Using Fuzzy Fisherface Classifier, Science Direct Journal Of Pattern Recognition Society 38(2005),1717-1732Turk, M., Pentland, A.: Eignefaces for Recognition. Journal of Cognitive Neuroscience, Vol.3, (1991) 72-86Turk, M., Pentland, A.: Face Recognition Using Eignefaces. In Proc. IEEE Conf. On Computer Vision and Pattern Recognition. (1991) 586-591Belhumeur, P., Hespanha, J., Kriegman, D.: Eigenfacesvs. Fisherfaces: Face Recognition using class specific linear projection. In Proc. ECCV, (1996) 45-58