Human Face Recognition - University of Windsor Face-Recognition… · • An inverted face is much...

RESEARCH CENTRE FOR INTEGRATED MICROSYSTEMS - UNIVERSITY OF WINDSORRESEARCH CENTRE FOR INTEGRATED MICROSYSTEMS - UNIVERSITY OF WINDSOR

Human Face Recognition

PhD ProposalAmirhosein Nabatchian

March 2008

RESEARCH CENTRE FOR INTEGRATED MICROSYSTEMS - UNIVERSITY OF WINDSOR

Outline• Research objective• Introduction• Applications of this research• Challenges• Literature review• My approach• Conclusions and future work

2


Research Objective• Face recognition:

Given still or video images of a scene, identify or verify one or more persons in the scene using a stored database of faces

• Areas of interest• Feature extraction and Classification fusion• Faces with non-uniform illumination• Faces with Occlusion• One sample per person problem

3


Introduction

• One of the most used biometrics

• Does not need participation• There is feasible

technology

Human Identification

Biometrics

Face Recognition

4


Authentication vs. Identification

• Face Authentication/Verification (1:1 matching)

5

• Face Identification/Recognition (1:N matching)


Applications

Areas ApplicationsBiometrics Driver’s Licences, Entitlement Programs, Immigration, National

ID, Passports, Voter Registration, Welfare FraudInformation Security

Desktop Logon, Application Security, Database Security, File Encryption, Intranet Security, Internet Access, Medical Records,Secure Trading Terminals

Law Enforcement and Surveillance

Advanced Video Surveillance, CCTV Control, Portal Control, Post-Event Analysis, Shoplifting and Suspect Tracking and InvestigationSmart Cards Stored Value Security, User Authentication

Access Control Facility Access, Vehicular Access

6


Challenges• automatically locate the face

• Identify similar faces (inter-class similarity)

• Accommodate intra-class variability due to:

• head pose

• illumination conditions

• expressions

• facial accessories

• aging effects

7


What is Biometrics?• The study of automated identification by use of physical

or behavioral traits• Physical:

Face, Fingerprint, Iris, Ear, Retina, Hands• Behavioral:

Signature, Walking gait, Typing patterns• Both:

Voice• Biometric is used in all aspects of present day life for

identifying individuals uniquely • ATM, car, house, work, computer

• A convenient and secure way of producing valid ID• Can never forget it or lose it• Growing field due to applicability to many aspects of everyday

life

8


Essential Properties of a Biometric

• UniversalEveryone should have the characteristic

• UniquenessNo two persons have the same characteristic

• PermanenceCharacteristic should be unchangeable

• CollectabilityCharacteristic must be measurable

9


Psychophysics/Neuroscience Issues• Face recognition is a dedicated process• Both holistic and feature information are crucial for face perception

and recognition• Human quickly focus on odd features• An inverted face is much harder to recognize

• Hair, face outline, eyes and mouth are important for perceiving faces• Nose plays an insignificant role

• But has the most features in profile face recognition• Upper part of the face is much more important than the lower part• pleasant faces, unpleasant faces, mid-range• It’s view point dependent• Top lighting• Movement (even if negated, inverted or thresholded) • Facial expression is accomplished in parallel with face recognition

10


A Face Recognition System (FRS)

11

Image Enhancement

FaceDetection

FeatureExtraction

FeatureDatabase

ClassificationInput Image

Image Database


Image Enhancement• Filtering & Noise removal• Histogram equalization• Color adjust

12


Face Detection/Segmentation• Face template• Deformable feature-based template• Skin color• Neural network • Support Vector Machine (SVM)• First order moment invariants

13


Face Detection

• Scan window over image• Classify window as either:

– Face– Non-face

ClassifierWindowFace

Non-face


An example

• Skin has a very small range of (intensity independent) colors, and little texture– Compute an intensity-independent color measure,

check if color is in this range, check if there is little texture (median filter)

– See this as a classifier - we can set up the tests by hand, or learn them.

– get class conditional densities (histograms), priors from data (counting)

• Classifier is

15


Feature Extraction• Extraction of applicable features from the human face images is an

important part of the recognition. • There are two main approaches for this feature extraction problem

– extracting geometrical and structural facial features like the shapes of the eyes, nose, mouth and the distance between these.

• not affected by irrelevant information in the image, but sensitive to unpredictability of face appearance and environmental conditions.

– the Holistic approach, • features from the whole image are extracted. • Since the features are global in the whole image, irrelevant

elements like background and hair might affect the feature vectors and cause erroneous recognition results.

16


Feature Extraction• Eigenfaces• Generalized symmetry operator

– Eyes, nose, mouth, • Template-based eye detection• Edge-based approach• Gabor wavelet decomposition• Moment Invariants

17


EigenfacesEigenfaces

• Use Principle Component Analysis (PCA) to reduce the dimensionality

• Principal components are eigenvectors of covariance matrix

18


Eigenfaces• Modeling

1. Given a collection of n labeled training images,2. Compute mean image and covariance matrix.

Subtract the mean.3. Compute k Eigenvectors (note that these are

images) of covariance matrix corresponding to k largest Eigenvalues.

4. Project the training images to the k-dimensional Eigenspace.

• Recognition1. Given a test image, project to Eigenspace.2. Perform classification to the projected training

images.19


Eigenfaces: Training Images

20

N*Nimage

N2*1 V

ector


Eigenfaces

Mean Image Basis Images

21


Difficulties with PCA• Projection may suppress important detail

– smallest variance directions may not be unimportant

• Method does not take discriminative task into account– typically, we wish to compute features that

allow good discrimination– not the same as largest variance

22


PCA vs Fisher’s Linear Discriminant

• Between-class scatter

• Within-class scatter

• Total scatter

• Where– c is the number of classes– µi is the mean of class χi

– | χi | is number of samples of χi..

Tii

c

iiBS ))((

1µµµµχ −−=∑

=

∑ ∑= ∈

−−=c

i x

TikikW

ik

xS1

))((χ

µµµ

WB

c

i x

TkkT SSxS

ik

+=−−=∑ ∑= ∈1

))((χ

µµµµ1

µ2

µ

χ1χ2

23


PCA & Fisher’s Linear Discriminant• PCA (Eigenfaces)

Maximizes projected total scatter

• Fisher’s Linear Discriminant

Maximizes ratio of projected between-class to projected within-class scatter

WSWW TT

WPCA maxarg=

WSW

WSWW

WT

BT

Wfld maxarg=

χ1χ2

PCA

FLD

24


Moment Invariants• Moment features are invariant under

scaling, translation rotation and reflection• Moment Invariants have been proven to

be useful in pattern recognition applications due to their sensitivity to the pattern features

25


Moment invariants• In 1961 Hu introduced the first set of algebraic moment invariants • Simply when calculating an image’s center we are using first order Regular

moment invariants (RMI). • Bamieh and De Figueiredo derived another set of moment invariants (BMI)

which have small feature vectors that are computationally efficient. • Zernike and pseudo Zernike orthogonal polynomials are the basis of the

Zernike and pseudo Zernike moment invariants (ZMI) and (PZMI). • Teague-Zernike Moment invariants (TZMI) were proposed by Teauge as a

method for calculating ZMIs. • Zernike and pseudo Zernike moment invariants have been normalized to

generate NZMI and NPMZIs. The normalized forms have yielded better results in simple recognition applications

26


Regular Moment Invariants

( , )p qpqM x y f x y dxdy

+∞ +∞

−∞ −∞= ∫ ∫

10 01

00 00

, M Mx yM M

= =

( ) ( ) ( , )p qpqM x x y y f x y dxdy

+∞ +∞

−∞ −∞= − −∫ ∫

20 02

1 ( ) ( ) ( , )p qpq x x y y f x y dxdy

M M γµ+∞ +∞

−∞ −∞= − −

+⎡ ⎤⎣ ⎦∫ ∫

27


Regular Moment Invariants

( )

( ) ( ) ( )

0 0

,

( ) ( 1) cos

cos

j kj r sk s

jkr s

k r sj k r s r s

j kRMI

r sθ

θ µ

− +−

= =+ −

+ − − +

⎛ ⎞⎛ ⎞= − ⎜ ⎟⎜ ⎟

⎝ ⎠⎝ ⎠

×

∑∑

1 11

20 02

21 tan2

MM M

θ − ⎡ ⎤= ⎢ ⎥−⎣ ⎦

28


Hu Moment Invariants

, 2 ,20 0

2( )

2 0, 1

r rl

p r r p k l k ll k

p l rI j

l k

p r j

µ− − − += =

−⎡ ⎤ ⎡ ⎤= − ⎢ ⎥ ⎢ ⎥

⎣ ⎦ ⎣ ⎦

− > = −

∑ ∑

2,( ) , 1, 2,3,...., 2r p r rHMI I r p r−= = −

29


Bamieh Moment Invariants

( )1 2 1 2... ,ijk i j kM x x x f x x dx dx+∞ +∞

−∞ −∞= ∫ ∫

2nd order

3rd order

4th order

( ) 220 02 111BMI µ µ µ= −

( ) ( )

( )( )2

30 03 12 2122 2

03 12 21 21 30 124

BMI µ µ µ µ

µ µ µ µ µ µ

= −

− − −

( ) 240 04 13 31 223 4 3BMI µ µ µ µ µ= − +

( ) 40 22 04 13 22 3142 2 3

13 40 04 31 22

2BMI µ µ µ µ µ µ

µ µ µ µ µ

= −

− − −

Table 1: Bamieh Moment Invariants [10]

30


Zernike Moment Invariants

( ) ( )2

0 0

1 cos , sin jLnL nL

nA f r r R r e rdrdπ ϕϕ ϕ ϕ

π

∞ −+= ×∫ ∫

( )n

knL nLk

k L

R r B r=

=∑

( ) 21 !2

! ! !2 2 2

n k

nLk

n k

Bn k L k k L

− +⎛ ⎞− × ⎜ ⎟⎝ ⎠=

− + −⎛ ⎞ ⎛ ⎞ ⎛ ⎞× ×⎜ ⎟ ⎜ ⎟ ⎜ ⎟⎝ ⎠ ⎝ ⎠ ⎝ ⎠

31


Pseudo Zernike Moment Invariants

( )

( )

( )

( )

, ,0

2 2 ,20 0

, ,0

2 2 ,20 0

1

1

nmn m

n m ssn m s even

k mb

k m a b a ba b

n m

n m ssn m s odd

d mb

d m a b a ba b

PZMI

n D

k mj CM

a b

n D

d mj RM

a b

π

π

−

=− −

+ − − += =

−

=− −

+ − − += =

=

+

⎛ ⎞⎛ ⎞× −⎜ ⎟⎜ ⎟

⎝ ⎠⎝ ⎠

++

⎛ ⎞⎛ ⎞× −⎜ ⎟⎜ ⎟

⎝ ⎠⎝ ⎠

∑

∑∑

∑

∑∑

( ) $ $ $ $2 2

( 2) / 200

,p q

x ypq p q

f x y x y x yRM

M + +

+=∑ ∑

( ) ( )( ) ( ), ,

1 2 1 !! ! 1 !

s

n m sn s

Ds n m s n m s

− × + −=

× − − × − − +

( 2) / 200

pqpq p qCM

M

µ+ +

=

32


Classification• k-NN• Neural Network• SVM• RBF• Fuzzy Logic

33


k-NN• K-nearest neighbors:

– Given x, take vote among its k nearest neighbors

• Advantages:– Training is very fast– Learns complex target functions– Does not loose information

• Disadvantages:– Slow at query time– Easily fooled by irrelevant attributes

34


Support Vector Machines

35

Small Margin Large Margin

• When not linearly separable, transform to a higher order space, separate linearly, and transform to the output space


The Database• The AT&T face image database (formerly known

as the ORL database) • A set of face images taken between April 1992

and April 1994 from 40 people• 10 different images of each person taken at

different time, illuminations, and facial expressions like open or closed eyes, smiling or not smiling and facial details like glasses.

• dark homogeneous background, the subject is in an upright, frontal position.

• The size of each image is 92×112 pixels, with 256 grey levels per pixel

36


The Database

37


LOCATING THE FACE AND CREATING SUBIMAGES

• we use second order moment invariants

1 11 13 38 84 4max min

min max

I I4 4,I I

α βπ π

⎛ ⎞ ⎛ ⎞⎛ ⎞ ⎛ ⎞= =⎜ ⎟ ⎜ ⎟⎜ ⎟ ⎜ ⎟ ⎜ ⎟⎜ ⎟⎝ ⎠ ⎝ ⎠ ⎝ ⎠⎝ ⎠

( ) ( )

( ) ( )

2min 0 0

2max 0 0

I cos sin

I sin cos

x y

x y

x x y y

x x y y

θ θ

θ θ

⎡ ⎤= − − −⎣ ⎦

⎡ ⎤= − − −⎣ ⎦

∑∑

∑∑

38


• result of face localization on images of the database

Figure 2: result of face localization on images of the database in figure 1

39


Feature Vector and Feature Matrix

( )( )

, ,,

,varMI n MI n

MI nMI n

FV mean FVWFV

FV

−=

1 2 3 4, , ,BMIFV BMI BMI BMI BMI= ⎡ ⎤⎣ ⎦

, 1 2 3, , ,...,MI n nFV MI MI MI MI= ⎡ ⎤⎣ ⎦ Calculating this vector for all the images in our training database will result in a M×n feature space matrix where M is the number of images in the training database and n is the number of the moments that we are going to use for each type of moment invariants.

40


• Artificial neural networks (ANN) have been widely used as the classifier in many face recognition systems

• We have used a three layer perceptron neural network. • The number of neurons in input layer is equal to the size of

related feature vector for each experiment • The number of output neurons is equal to the number of classes

which is 40

41


Recognition Results

Method Classification accuracy

Number of features order

Number of neurons in input layer

Number of neurons in

hidden layerR Number of

epochs

PZMI 94 % 120 10 120 108 0.9 1000

NZMI 91 % 34 7 34 68 2.0 6000

NPZMI (10) 90.5 % 118 10 118 106 0.9 1000

ZMI 87 % 76 11 76 152 2.0 4000

NPZMI (7) 86 % 61 7 61 122 2.0 2000

RMI 85.5 % 75 11 75 150 2.0 2000

TZMI 67.5 % 32 7 32 96 3.0 10000

HMI 61 % 32 7 32 96 3.0 6000

BMI 22 % 4 4 4 12 3.0 4000

classification parameters for different moment invariants

42


Recognition Speed

Moment Invariant

Recognition time (s) order Number of Features

BMI 0.15 4 4

HMI 0.62 7 32

RMI 4.53 7 33

PZMI 10.75 7 64 (36 Re+28 Im)

ZMI 124 7 33 (18 Re+15 Im)

recognition time for different methods

43


Conclusion• Performance of different moment invariants for feature extraction in face recognition

was studied• The moment invariants used in this study were HMI, BMI, RMI, ZMI, TZMI, PZMI,

NZMI and NPZMI. • A three layer perceptron neural network was used as the classifier in the recognition

system. • The performance of the system can be optimized by using proper value for ρ and also

by choosing proper orders of the moment invariants.• High order PZMIs contain useful information, therefore lead to the best results.• BMIs were the fastest to be computed but had very poor recognition accuracy for

face images• calculating ZMIs was the most time consuming. • The best recognition rate of 95% was achieved with PZMI of order 1-7.

44


Future work• Feature extractor and classifier fusion• Non-uniformed illumination• Occlusion• one sample problem

45


References• [1] Heisele, B., Ho, P., Wu, J., Poggio, T., "Face recognition: component-based versus global

approaches", Computer Vision and Image Understanding, Vol. 91 No.1/2, pp.6-21.2003• [2] Zhao, W., Chellappa, R., Phillips, P.J., Rosenfeld, A. (2003), "Face recognition: a literature

survey", ACM Computing Surveys, Vol. 35 No.4, pp.339-458.• [3] J. Haddadnia, M.,Ahmadi, K.,Faez.: An efficient feature extraction method with pseudo-Zernike

moment in RBF neural network-based human face recognition system. EURASIP Journal on Applied Signal Processing (2003), 890–901.

• [4] E. Hjelmas and B. K. Low, “Face detection: a survey,” Computer Vision and Image Understanding, vol. 83, no. 3, pp. 236– 274, 2001.

• [5] M. Hu, visual Pattern recognition by Moment Invariants, IRE Trans. Inf. Theory 8, 179-187, 1962• [6] S. Reddi, Radial and Angular Moment Invariants for image Identification, IEEE Trans. Pattern

Analysis and Machine intelligence, 3,240-242, 1981• [7] R.Bamieh, R. De Figueiredo, A General Moment Invariants/Attributed-graph method for the three

dimensional object recognition from a single image, IEEE Journal of Robotics Automation 2, 31-41,1986• [8] C. Teh, R. T. Chin, On Image Analysis by the Methods of Moments, IEEE Transactions on

Pattern Analysis and Machine Intelligence, v.10 n.4, p.496-513, July 1988• [9] M. Teague, "Image analysis via the general theory of moments,"J. Opt. Soc. Amer., vol. 70, no.

8, pp. 920-930, Aug. 1980.

46


References• [10] S. O. Belkasim , M. Shridhar , M. Ahmadi, Pattern recognition with moment invariants: a

comparative study and new results, Pattern Recognition, v.24 n.12, p.1117-1138, Dec. 1991• [11] http://www.cl.cam.ac.uk/research/dtg/attarchive/facedatabase.html• [12] J. Wang and T. Tan, “A new face detection method based on shape information,” Pattern

Recognition Letter, vol. 21, no. 6-7, pp. 463–471, 2000.• [13] J. Haddadnia and K. Faez, “Human face recognition using radial basis function neural network,”

in Proc. 3rd International Conference On Human and Computer (HC ’00), pp. 137–142, Aizu, Japan, September 2000.

• [14] J. Haddadnia, M. Ahmadi, and K. Faez, “A hybrid learning RBF neural network for human face recognition with pseudo Zernike moment invariant,” in IEEE International Joint Conference On Neural Network (IJCNN ’02), pp. 11–16, Honolulu, Hawaii, USA, May 2002.

• [15] S. Gutta, J. R. J. Huang, P. Jonathon, and H. Wechsler, “Mixture of experts for classification of gender, ethnic origin, and pose of human faces,” IEEE Transactions on Neural Networks, vol. 11, no. 4, pp. 948–960, 2000.

• [16] S. Z. Li and J. Lu, “Face recognition using the nearest feature line method,” IEEE Transactions on Neural Networks, vol. 10, no. 2, pp. 439–443, 1999.

• [17] S. Lawrence, C. L. Giles, A. C. Tsoi, and A. D. Back, “Face recognition: a convolutional neural network approach,” IEEE Transactions on Neural Networks, vol. 8, no. 1, pp. 98–113, 1997.

47


Thanks for your kind attention

48


Inter-Class Similarity• Different persons may have very similar appearance

Twins Father and son

49


Intra-Class Variability

• Faces with intra-subject variations in pose, illumination, expression, accessories, color, occlusions, and brightness

50


How to construct Eigenspace?

[ ] [ ][ x1 x2 x3 x4 x5 ] W

Construct data matrix by stacking vectorized images and then apply Singular Value Decomposition (SVD)

51


Fisherfaces: Class specific linear projection

• An n-pixel image x∈Rn can be projected to a low-dimensional feature space y∈Rm by

y = Wx

where W is an n by m matrix.

• Recognition is performed using nearest neighbor in Rm.

• How do we choose a good W?52

Human Face Recognition - University of Windsor Face-Recognition… · • An inverted face is much...

Documents

Transcript of Human Face Recognition - University of Windsor Face-Recognition… · • An inverted face is much...