CONTENT BASED CONTENT BASED FACE FACE RECOGNITIONRECOGNITION

Ankur JainAnkur Jain 01D05007 01D05007Pranshu SharmaPranshu Sharma 01005026 01005026Prashant BaroniaPrashant Baronia 01D05005 01D05005Swapnil ZarekarSwapnil Zarekar 01D05001 01D05001

Under the guidance

of

Prof. Pushpak Bhattacharya

IntroductionIntroduction

Problem Statement :Problem Statement : Given an image, to Given an image, to identify it as a faceidentify it as a face

and/or and/or extract face imagesextract face images from it. from it. To retrieve the To retrieve the similar imagessimilar images (based on a (based on a

heuristic) heuristic) from the given databasefrom the given database of face of face images.images.

Why face recognition ?Why face recognition ?

Various potential applications, such asVarious potential applications, such as

person identification. person identification. human-computer interaction.human-computer interaction. security systems. security systems.

Faces are complex, multidimensional and meaningful visual stimuli.

Face Recognition is difficult.

Face Images are similar in overall configuration.

Difference From Image RecognitionDifference From Image Recognition

Approach Approach

Similar toSimilar to Content BasedContent Based Image Retrieval (CImage Retrieval (CBIR).BIR).

Neural Networks andNeural Networks and Self Organizing MapsSelf Organizing Maps (SOMs).(SOMs).

Principal Component AnalysisPrincipal Component Analysis (PCA).(PCA). RelevanceRelevance feed back.feed back.

Stages of Face RecognitionStages of Face Recognition(1) face location detection(1) face location detection(2) feature extraction(2) feature extraction(3) facial image classification(3) facial image classification

Approaches of Feature ExtractionApproaches of Feature Extraction(1) (1) local featurelocal feature : eyes, nose, mouth information : eyes, nose, mouth information easily affected by irrelevant information .easily affected by irrelevant information .(2) (2) global featureglobal feature : :

• extract feature from whole image .extract feature from whole image .

Face Recognition Using Face Recognition Using EigenfacesEigenfaces

Face Images are projected into a Face Images are projected into a feature feature spacespace (“ (“Face SpaceFace Space”) that ”) that best encodes the best encodes the variationvariation among known face images. among known face images.

The face space is defined by the The face space is defined by the “eigenfaces”,“eigenfaces”, which are the which are the eigenvectorseigenvectors of of the set of faces. the set of faces.

Eigen Space and Eigen FacesEigen Space and Eigen Faces

Initialization :

Acquire the training set and calculate eigenfaces (using PCA projections) which define eigenspace.

When a new face is encountered, calculate its weight.

Determine if the image is face.

If yes, classify the weight pattern as known or unknown.

(Learning) If the same unknown face is seen several times incorporate it into known faces.

Steps In Face RecognitionSteps In Face Recognition

PCAPCA

Main assumption of PCA Main assumption of PCA approach:approach:

Face spaceFace space forms a clusterforms a cluster in image spacein image space.. PCA givesPCA gives suitable representationsuitable representation..

Eigenfaces (1)Eigenfaces (1)

Calculation of EigenfacesCalculation of Eigenfaces(1) Calculate (1) Calculate average faceaverage face : : v.v.

(2) Collect (2) Collect difference between training imagesdifference between training images and average and average face in matrix A (M by N), where M is the number of pixels and face in matrix A (M by N), where M is the number of pixels and N is the number of images.N is the number of images.

(3) The (3) The eigenvectors of covariance matrix Ceigenvectors of covariance matrix C (M by M) give (M by M) give the eigenfaces. the eigenfaces.

M is usually big, so this process would be time consuming.M is usually big, so this process would be time consuming.

What to do?What to do? TAAC

Eigenfaces (2)Eigenfaces (2)

Calculation of Eigenvectors of CCalculation of Eigenvectors of CIf the If the number of data pointsnumber of data points is smaller than the dimension (N< is smaller than the dimension (N<M), then there will be M), then there will be only N-1 meaningful eigenvectorsonly N-1 meaningful eigenvectors..

Instead of directly calculating the eigenvectors of C, we can Instead of directly calculating the eigenvectors of C, we can ccalculate the eigenvalues and the corresponding eigenvectalculate the eigenvalues and the corresponding eigenvectors of a much smaller matrix L (N by N).ors of a much smaller matrix L (N by N).

if λif λii are the eigenvectors of L are the eigenvectors of L then A λthen A λii are the eigenvectors fo are the eigenvectors fo

r C.r C. The eigenvectors are in the The eigenvectors are in the descent order of the corresponding eigenvdescent order of the corresponding eigenv

alues.alues.

AAL T

Eigenfaces (3)Eigenfaces (3)

Representation of Face Images using Representation of Face Images using EigenfacesEigenfaces

The The training face imagestraining face images and and new face imagesnew face images can be can be represented as represented as linear combination of the eigenfaceslinear combination of the eigenfaces..

When we have a face image When we have a face image u u ::

Since the Since the eigenvectors are orthogonaleigenvectors are orthogonal : :

i

iiau

iT

i ua

Eigenfaces (4)Eigenfaces (4)

Experiment and ResultsExperiment and Results

Data used here are from the ORL database of faces. Data used here are from the ORL database of faces. Facial images of Facial images of 16 persons16 persons each with 10 viewseach with 10 views are are used. - used. - Training set contains 16×7 images. Training set contains 16×7 images.

- - Test set contains 16×3 images.Test set contains 16×3 images.

First threeFirst three eigenfaces : eigenfaces :

Classification UsingClassification Using Nearest NeighborNearest Neighbor Save average coefficients for each person. ClassifySave average coefficients for each person. Classify new new

face as the person with the closest average.face as the person with the closest average. Recognition accuracyRecognition accuracy increases with number of eigenfaincreases with number of eigenfa

ces till 15. ces till 15.

　　 Later eigenfacesLater eigenfaces do not help muchdo not help much with recognition.with recognition.

Best recognition ratesBest recognition rates

Training set 99%Training set 99%

Test set 89%Test set 89%0.4

0.6

0.8

1

0 50 100 150

number of eigenfaces

accu

racy

validation set training set

Neural Networks Neural Networks and and

TS-SOMTS-SOM

What are Neural Networks ?What are Neural Networks ?

Individual units to Individual units to simulate Neuronssimulate Neurons ParallelParallel Processing Processing Many inputsMany inputs and single output and single output Organization/structure of theOrganization/structure of the TLU’s TLU’s is is

importantimportant

What is SOM ?What is SOM ?

TS-SOM TS-SOM :- :- Tree structureTree structure self-organizing self-organizing mapsmaps

CompetitiveCompetitive learning ANN learning ANN Each unit of map Each unit of map receivesreceives identical identical inputsinputs Units competeUnits compete for selection for selection ModificationModification of selected node and its of selected node and its

neighbors neighbors

Training of SOMTraining of SOM

Randomly initialized Randomly initialized Selection based on some Selection based on some query parameterquery parameter On selection On selection a node and its neighborsa node and its neighbors are are

modifiedmodified Degree of modification Degree of modification reduces with each reduces with each

iterationiteration

Example of a two-dimensional TS-SOM structure of 3 levelsExample of a two-dimensional TS-SOM structure of 3 levels

AlgorithmAlgorithm

CalculateCalculate weight vectorweight vector for first level.for first level. InitializeInitialize weight vectors of other levelsweight vectors of other levels.. CalculateCalculate centroid centroid associated to each associated to each

node as mean of closest training samples.node as mean of closest training samples. IterateIterate to the next level.to the next level.

Relevance FeedbackRelevance Feedback

SystemSystem content basedcontent based retrieval.retrieval. Point of human interventionPoint of human intervention User analysis of system output.User analysis of system output. User selectsUser selects most relevantmost relevant Query iteratedQuery iterated if output not satisfactoryif output not satisfactory

InteractionInteraction Between User & SystemBetween User & System

1.1. A random set of faces is presented to the A random set of faces is presented to the user.user.

2.2. User interactive selection of faces.User interactive selection of faces.3.3. SystemSystem content-based face retrievalcontent-based face retrieval..4.4. User analysis of retrieved faces.User analysis of retrieved faces.

Requested face was found -> ExitRequested face was found -> Exit Similar faces were found. -> Go to 2 Similar faces were found. -> Go to 2 No similar faces were found. No similar faces were found.

User tired -> Exit User tired -> Exit User not tired (re initialization -> Go to 1User not tired (re initialization -> Go to 1

Comparison of the Two ApproachesComparison of the Two Approaches Training timeTraining time

Nearest neighbor is much faster.Nearest neighbor is much faster. StorageStorage

About the same.About the same. Classification timeClassification time

Nearest neighborNearest neighbor is slightly slower. is slightly slower. AccuracyAccuracy

Neural network is able to achieve the same accuracy Neural network is able to achieve the same accuracy using 5 eigenfaces with nearest neighbor using 15, and a using 5 eigenfaces with nearest neighbor using 15, and a higher accuracy when using 15. higher accuracy when using 15.

Neural network models the problem Neural network models the problem betterbetter, , but takes but takes more trainingmore training time time..

Future WorkFuture Work

Face Detection in motion pictures.Face Detection in motion pictures. Detailed study of the proposed systemDetailed study of the proposed system

assuming PCA assumptions not to be true.assuming PCA assumptions not to be true. Investigate whetherInvestigate whether eigenfaceseigenfaces is a good is a good

solution for this problem by comparing with solution for this problem by comparing with other feature extraction techniques such asother feature extraction techniques such as DCTDCT

ReferencesReferences Navarrete P. and Ruiz-del-Solar J. (2002), “Navarrete P. and Ruiz-del-Solar J. (2002), “Interactive Face Interactive Face

Retrieval using Self-Organizing Maps”, Retrieval using Self-Organizing Maps”, 2002 Int. Joint Conf. on 2002 Int. Joint Conf. on

Neural Networks – IJCNN 2002Neural Networks – IJCNN 2002, May 12-17, Honolulu, USA., May 12-17, Honolulu, USA. ““A tutorial on Principal Components Analysis”, By Lindsay I A tutorial on Principal Components Analysis”, By Lindsay I

Smith.Smith. ““Eigenfaces for Recognition”, Turk, M. and Pentland A., (1991)Eigenfaces for Recognition”, Turk, M. and Pentland A., (1991)

Journal of Cognitive Neuroscience, Vol. 3, No. 1, pp. 71-86.Journal of Cognitive Neuroscience, Vol. 3, No. 1, pp. 71-86. Ruiz-del-Solar, J., and Navarrete, P. (2002). “Ruiz-del-Solar, J., and Navarrete, P. (2002). “Towards a Towards a

Generalized Eigenspace-based Face Recognition Framework”, Generalized Eigenspace-based Face Recognition Framework”, 4th Int. Workshop on Statistical Techniques in Pattern 4th Int. Workshop on Statistical Techniques in Pattern

RecognitionRecognition, August 6-9, Windsor, Canada, August 6-9, Windsor, Canada.. Simulating Neural Networks by James A. Freeman.Simulating Neural Networks by James A. Freeman. Artificial Intelligence by Neil J. Nielsson.Artificial Intelligence by Neil J. Nielsson.