Face Recognition and Biometric Systemssun.aei.polsl.pl/~mkawulok/stud/fr/lect/07.pdfThe Eigenfaces...

The Eigenfaces methodThe Eigenfaces method

Face Recognition and Biometric Systems

Plan of the lecturePlan of the lecture

Principal Components Analysisi idmain idea

Feature extraction by PCAyface recognition

EigenfacesEigenfacestrainingfeature extraction


LiteratureLiterature

M.A.Turk, A.P.PentlandFace Recognition Using Eigenfaces


Phases of recognitionPhases of recognition

Detection Normalisation

FeatureFeatureFeature vectors Feature extractionFeature

extractionFeature vectors

comparison


ExampleExample...


PCA main issuesPCA – main issues

Dimensionality input spaceinput spacedata

C di t t it d t tCoordinates system suited to a setDimensionality reductiony

reduction error

Principal Components Analysis (PCA)Principal Components Analysis (PCA)statistical method



Input data setvectors / pointsvectors / pointsinput space

S th l b iSpace orthonormal basiseach vector from the input space can be expressed as a linear combination of basis vectorsbasis vector defines a dimension



Dimensions are sortedeach basis vector assigned with a weight

Main directions of variance in theMain directions of variance in the input set

each direction generates one dimensioneach direction generates one dimension

Dimension importance proportional to variance


Input face spaceInput face spaceNormalised image

described by pixel values

Image as a point in a spacee.g. 64x75 pixels – 4800 dimensionsg p

Information excesstoo many dimensionstoo many dimensions

Elimination of redundant informationrelevant information must remainrelevant information must remainfeature extraction


PCA for feature extractionPCA for feature extractionSet of normalised imagesSet of normalised images

input vectors (a.k.a. training set)PCA:PCA:

finds new orthogonal basisnumber of dimensions can be reducednumber of dimensions can be reducedcreates „face space”

Feature extraction by PCAFeature extraction by PCAfind a corresponding point in a new spacepworks for any face image (not only from training set)

Eigenfaces method – PCA for face imagesFace Recognition and Biometric Systems

g g

The Eigenfaces methodThe Eigenfaces methodTrainingTraining

1. create covariance matrix for the training set2 calculate eigenvalues and eigenvectors2. calculate eigenvalues and eigenvectors

eigenvectors are the orthonormal basis and define the face spaceeigenvalues are associated with eigenvectorseigenvalues proportional to variance

3 select eigenvectors3. select eigenvectorsFeature extraction

face image mapped to the new spaceface image mapped to the new space (coordinates must be found)any face image may be processed


y g y p

Eigenfaces: trainingEigenfaces: training Input vectors: uuInput vectors:

u – N-dimensional vectorMuu ,...,1

M – number of vectors in the setM1

Average vector: ∑==i

iM 1

1 uμ

Covariance matrix:M1 1

or∑ )−−==

ΤM

iiiM 1

)((1 μuμuC T

MAAC ⋅=

1


Training exampleTraining – example

Training set (M=4, N=3):[1, 0, 2][0, 3, 1][ , , ][4, 1, 2][3 0 -1][3, 0, -1]

Average vector, covariance matrix


Eigenfaces: trainingEigenfaces: trainingCharacteristic equation:Characteristic equation:

0)det( =⋅− IC λeigenvalues

( )

)()(λ

Eigenvectors (v)one for each eigenvalueg

vvC ⋅=⋅ λJacobi method (numerical method)

OpenCV - cvEigenVV functionFace Recognition and Biometric Systems

p g

Training optimisationTraining – optimisation Problem: large size of covarianceProblem: large size of covariance matrix (NxN), e.g. 4800x4800Trick: vvAA ⋅=⋅⋅ λT

''' vvAA λT ''' vvAA ⋅=⋅⋅ λ)'(')'( vAvAAA ⋅⋅=⋅⋅⋅ λT )()( vv λ

'λλ = 'vAv ⋅=

Av’ – desired eigenvectorsFace Recognition and Biometric Systems

Eigenfaces: trainingEigenfaces: trainingEi t tiEigenvectors – properties

dimensionality equal to input vectorsth l (l th 1)orthonormal (length = 1)

sorted by corresponding eigenvaluesmay be scaled to pixel value rangemay be scaled to pixel value range

Eigenfaces – eigenvectors transformed to imagesto images

exampledimensionality reductiondimensionality reduction

New space, less dimensions


Eigenfaces: trainingEigenfaces: training

C00 C0n...

C C

... ......

Normalised

Cn0 Cnn...

Covariance EigenfacesNormalisedimages

Covariancematrix

Eigenfaces


Eigenfaces: feature extractionEigenfaces: feature extractionF t t ti i tFeature extraction input:

set of eigenvectors and eigenvalues (delivered by training)normalised imagenormalised image

Projection:t i ith i t

xψx ⋅= T'ψ - matrix with eigenvectorsx – normalised image after average face subtractionx’ – transformed vector


Feature extraction exampleFeature extraction - example

2-dimensional space:eigenvectors:

]2;2[ ]2;2[

Vectors projection:

]2

;2

[ ]2

;2

[−

Vectors projection: [3; 1], [-2; -2], [10, 9]

Di i lit d tiDimensionality reduction


Eigenfaces: feature extractionEigenfaces: feature extraction

ψ matrix can be cut to reduce dimensions

ψ ψ’ ψ’’

Feature vector element is a scalar product:

xv ⋅= Tiiw xψw ⋅= T'

Feature vector – cut projected vector x’


Eigenfaces: feature extractionEigenfaces: feature extraction

K1

K2

K3

...Scalar products between

...

Feature vectornormalised image andeigenvectors

Feature vector


Back projection: face imageBack projection: face image

Feature vector – face descriptioninformation reduction

Back projection: face image recovered from feature vectorfrom feature vector

reduced information are lostP j tiProjection error:

depends on similarity to the training set2D exampleface images


g

Back projection: detectionBack projection: detection

Back projection of images:face -> slightly modified face imageflower -> image similar to a faceg

Back projection error is higher for non face imagesnon-face imagesCan be used as a verifier

threshold of accepted projection error


SummarySummary

Eigenfaces – a basic face recognition methodmethod

many derived methodsTraining and feature extractionTraining and feature extractionHolistic approachHigh speedAverage / low effectivenessAverage / low effectiveness

may be improved


Thank you for your attention!Thank you for your attention!


Face Recognition and Biometric Systemssun.aei.polsl.pl/~mkawulok/stud/fr/lect/07.pdfThe Eigenfaces...

Documents

Transcript of Face Recognition and Biometric Systemssun.aei.polsl.pl/~mkawulok/stud/fr/lect/07.pdfThe Eigenfaces...