Eigenface for Eigenface for Face RecognitionFace Recognition

Presenter: Trần Đức MinhPresenter: Trần Đức Minh

OutlineOutline

OverviewOverview Eigenfaces for RecognitionEigenfaces for Recognition Conclusion

OverviewOverview Face RepresentationFace Representation

Template-based approachesTemplate-based approaches Feature-based approachesFeature-based approaches Appearance-based approachesAppearance-based approaches

Face DetectionFace Detection Utilization of elliptical shape of human head Utilization of elliptical shape of human head

((applicable only to front views applicable only to front views ) [5]) [5] Manipulation of images in “face space” [1]Manipulation of images in “face space” [1]

Face IdentificationFace Identification Performance affected by scale, pose, illumination, Performance affected by scale, pose, illumination,

facial expression, and disguise, etc.facial expression, and disguise, etc.

Face spaceFace space

An image is a point in a high dimensional spaceAn image is a point in a high dimensional space An N x M image is a point in RAn N x M image is a point in RNMNM

We can define vectors in this space as we did in the 2D caseWe can define vectors in this space as we did in the 2D case

+=

OutlineOutline

OverviewOverview Eigenfaces for RecognitionEigenfaces for Recognition Conclusion

Eigenfaces ApproachEigenfaces Approach

In the language of information theory……In the language of information theory…… Efficient encoding followed by comparing one face Efficient encoding followed by comparing one face

encoding with a database of models encoded encoding with a database of models encoded similarlysimilarly

Eigenfaces Approach Eigenfaces Approach (Contd.)(Contd.)

In mathematical terms…….In mathematical terms……. Find the principal components of the face Find the principal components of the face

distribution, or the eigenvectors of the covariance distribution, or the eigenvectors of the covariance matrix of the set of face images, called eigenfacesmatrix of the set of face images, called eigenfaces

Eigenfaces are a set of features that characterize Eigenfaces are a set of features that characterize the variation between face imagesthe variation between face images

Each training face image can be represented in Each training face image can be represented in terms of a linear combination of the eigenfaces, so terms of a linear combination of the eigenfaces, so can the new input imagecan the new input image

Compare the feature weights of the new input Compare the feature weights of the new input image with those of the known individualsimage with those of the known individuals

Example for eigenfaceExample for eigenface

Eigenfaces look somewhat like generic faces.

Major StepsMajor Steps

1.1. Initialization: acquire the training set of face Initialization: acquire the training set of face images and calculate the eigenfaces, which define images and calculate the eigenfaces, which define the face spacethe face space

2.2. Given an image to be recognized, calculate a set of Given an image to be recognized, calculate a set of weights of the weights of the MM eigenfaces by projecting it onto eigenfaces by projecting it onto each of the eigenfaceseach of the eigenfaces

3.3. Determine if the image is a face at all by checking Determine if the image is a face at all by checking if the image is sufficiently close to the face spaceif the image is sufficiently close to the face space

4.4. If it is a face, classify the weight pattern as either a If it is a face, classify the weight pattern as either a known person of as unknownknown person of as unknown

5.5. (Optional) If the same unknown face is seen several (Optional) If the same unknown face is seen several times, update the eigenfaces / weight patterns, times, update the eigenfaces / weight patterns, calculate its characteristic weight pattern and calculate its characteristic weight pattern and incorporate into the known facesincorporate into the known faces

Calculating EigenfacesCalculating Eigenfaces

Set of training images ( is a column Set of training images ( is a column vector of size )vector of size )

Average face of the training set:Average face of the training set:

Each training image differs from the average face by:Each training image differs from the average face by:

A total number of pairs of eigenvectors and A total number of pairs of eigenvectors and eigenvalues of the covariance matrix eigenvalues of the covariance matrix

((CC: : matrix) Eq. (1) matrix) Eq. (1)

where (where (AA: matrix): matrix)

Computationally IntractableComputationally Intractable!!

1 2, ,......, M n 2 1N

1

1 M

nnM

n n

nn

1

1 MT T

n nn

C AAM

1 2[ , ,......, ]MA

2 2N N

2N M

2N

Calculating EigenfacesCalculating Eigenfaces

For Computational FeasibilityFor Computational Feasibility

Only Only M M - 1- 1 eigenvectors are eigenvectors are meaningfulmeaningful ( ) ( )

Eigenvectors and associated eigenvalues of Eigenvectors and associated eigenvalues of : :

Eq. (2)Eq. (2)

Therefore, are the eigenvalues of , are Therefore, are the eigenvalues of , are the associated eigenvaluesthe associated eigenvalues

Eq. (3)Eq. (3)

The associated eigenvalues allow us to rank the The associated eigenvalues allow us to rank the eigenvectors according to their usefulness in eigenvectors according to their usefulness in characterizing the variation among the imagescharacterizing the variation among the images

2M Nn TL A An

Tn n nA A

Tn n nAA A A

nA TC AA n

1

M

n nk k nk

A

Using Eigenfaces for Using Eigenfaces for IdentificationIdentification

Construction of Known Individuals’ Face Construction of Known Individuals’ Face ClassesClasses

-- Images of known individuals are projected onto “face -- Images of known individuals are projected onto “face space” by a simple operation , space” by a simple operation , where i=1, 2, ……, Mwhere i=1, 2, ……, M represents the represents the iith individual, th individual, and and k k =1, 2, ……, =1, 2, ……, M’ M’ represents the weight coefficient represents the weight coefficient of eigenvector . The pattern vector of the of eigenvector . The pattern vector of the iith th individualindividual

-- If an individual has more than one image, take the -- If an individual has more than one image, take the average of the pattern vectors of this personaverage of the pattern vectors of this person

( )Tik k i

k1 2 '[ , ,......, ]i i i iM

Using Eigenfaces for Using Eigenfaces for Identification (Contd.)Identification (Contd.)

Given a new imageGiven a new image -- Project onto face space, and get its pattern vector -- Project onto face space, and get its pattern vector

-- Determine whether is a face image: -- Determine whether is a face image: Eq. (4) Eq. (4)

If < a predefined threshold , it is a face image; If < a predefined threshold , it is a face image; otherwise, nototherwise, not

-- Classify either as a known individual or as unknown: -- Classify either as a known individual or as unknown: Eq. (5)Eq. (5)

If < a predefined If < a predefined threshold , is identified as the threshold , is identified as the k’k’th face class; th face class; otherwise, it is identified as unknownotherwise, it is identified as unknown

1 2 '[ , ,......, ]M '

2 2

1

( )M

n nn

2 2( )k k

'min{ , 1,2,......, '}k kk M

OutlineOutline

OverviewOverview Eigenfaces for RecognitionEigenfaces for Recognition Conclusion

AdvantagesAdvantages

Ease of implementationEase of implementation No knowledge of geometry or specific No knowledge of geometry or specific

feature of the face requiredfeature of the face required Little preprocessing workLittle preprocessing work

LimitationsLimitations

Sensitive to head scaleSensitive to head scale Applicable only to front viewApplicable only to front view Good performance only under controlled Good performance only under controlled

background (not including natural scenes)background (not including natural scenes)

ReferenceReference

1.1. ““Eigenfaces for recognition”, M. Turk and A. Eigenfaces for recognition”, M. Turk and A. Pentland, Pentland, Journal of Cognitive Neuroscience, Journal of Cognitive Neuroscience, vol.3, No.1, 1991vol.3, No.1, 1991

2.2. ““Face recognition using eigenfaces”, M. Turk and Face recognition using eigenfaces”, M. Turk and A. Pentland, A. Pentland, Proc. IEEE Conf. on Computer Vision Proc. IEEE Conf. on Computer Vision and Pattern Recognitionand Pattern Recognition, pages 586-591, 1991, pages 586-591, 1991

3.3. Lindsay. I. Smith. A tutorial on principal Lindsay. I. Smith. A tutorial on principal components analysis, February 2002components analysis, February 2002

4.4. Ilker Atalay M.Sc Thesis: Face Recognition Using Ilker Atalay M.Sc Thesis: Face Recognition Using Eigenfaces Istanbul Technical University – Eigenfaces Istanbul Technical University – January 1996January 1996

5.5. Dimitri Pissarenko, Eigenface-based facial Dimitri Pissarenko, Eigenface-based facial recognition, Feb 06, 2003recognition, Feb 06, 2003

DemoDemo

Thank youThank you