Lecture’17:’’ Face’Recogni2on’ - Stanford...

Lecture 17 - !!!

Fei-Fei Li!

Lecture 17: Face Recogni2on

Professor Fei-‐Fei Li Stanford Vision Lab

15-‐Nov-‐14 1

Lecture 17 - !!!

Fei-Fei Li!

What we will learn today •  Introduc2on to face recogni2on •  Principal Component Analysis (PCA) •  The Eigenfaces Algorithm •  Linear Discriminant Analysis (LDA)

15-‐Nov-‐14 2

Turk and Pentland, Eigenfaces for Recogni2on, Journal of Cogni-ve Neuroscience 3 (1): 71–86. P. Belhumeur, J. Hespanha, and D. Kriegman. "Eigenfaces vs. Fisherfaces: Recogni2on Using Class Specific Linear Projec2on". IEEE Transac-ons on pa7ern analysis and machine intelligence 19 (7): 711. 1997.

Lecture 17 - !!!

Fei-Fei Li!

Courtesy of Johannes M. Zanker

15-‐Nov-‐14

“Faces” in the brain

3

Lecture 17 - !!!

Fei-Fei Li!

“Faces” in the brain

15-‐Nov-‐14

Kanwisher, et al. 1997

4

Lecture 17 - !!!

Fei-Fei Li!

•  Digital photography

Face Recogni2on

15-‐Nov-‐14 5

Lecture 17 - !!!

Fei-Fei Li!

Face Recogni2on •  Digital photography •  Surveillance

15-‐Nov-‐14 6

Lecture 17 - !!!

Fei-Fei Li!

•  Digital photography •  Surveillance •  Album organiza2on

Face Recogni2on

15-‐Nov-‐14 7

Lecture 17 - !!!

Fei-Fei Li!

Face Recogni2on •  Digital photography •  Surveillance •  Album organiza2on •  Person tracking/id.

15-‐Nov-‐14 8

Lecture 17 - !!!

Fei-Fei Li!

•  Digital photography •  Surveillance •  Album organiza2on •  Person tracking/id. •  Emo2ons and expressions

Face Recogni2on

15-‐Nov-‐14 9

Lecture 17 - !!!

Fei-Fei Li!

•  Digital photography •  Surveillance •  Album organiza2on •  Person tracking/id. •  Emo2ons and expressions

•  Security/warfare •  Tele-‐conferencing •  Etc.

Face Recogni2on

15-‐Nov-‐14 10

Lecture 17 - !!!

Fei-Fei Li!

The Space of Faces

•  An image is a point in a high dimensional space –  If represented in grayscale

intensity, an N x M image is a point in RNM

–  E.g. 100x100 image = 10,000 dim

15-‐Nov-‐14 11

Slide credit: Chuck Dyer, Steve Seitz, Nishino

Lecture 17 - !!!

Fei-Fei Li!

The Space of Faces

•  An image is a point in a high dimensional space –  If represented in grayscale

intensity, an N x M image is a point in RNM

–  E.g. 100x100 image = 10,000 dim

•  However, rela2vely few high dimensional vectors correspond to valid face images

•  We want to effec2vely model the subspace of face images

15-‐Nov-‐14 12

Slide credit: Chuck Dyer, Steve Seitz, Nishino

Lecture 17 - !!!

Fei-Fei Li!

Image space

Face space

•  Maximize the scaier of the training images in face space

•  Computes n-‐dim subspace such that the projec2on of the data points onto the subspace has the largest variance among all n-‐dim subspaces.

15-‐Nov-‐14 13

Lecture 17 - !!!

Fei-Fei Li!

•  So, compress them to a low-‐dimensional subspace that captures key appearance characteris2cs of the visual DOFs.

Key Idea

•  USE PCA for es2ma2ng the sub-‐space (dimensionality reduc2on)

• Compare two faces by projec2ng the images into the subspace and measuring the EUCLIDEAN distance between them.

15-‐Nov-‐14 14

Lecture 17 - !!!

Fei-Fei Li!


15-‐Nov-‐14 15


Lecture 17 - !!!

Fei-Fei Li!

PCA Formula2on •  Basic idea:

–  If the data lives in a subspace, it is going to look very flat when viewed from the full space, e.g.

–  This means that if we fit a Gaussian to the data the equiprobability contours are going to be highly skewed ellipsoids

15-‐Nov-‐14 16

technical

note

Slide inspired by N. Vasconcelos

1D subspace in 2D 2D subspace in 3D

Lecture 17 - !!!

Fei-Fei Li!

PCA Formula2on •  If x is Gaussian with covariance Σ, the equiprobability

contours are the ellipses whose

–  Principal components φi are the eigenvectors of Σ –  Principal lengths λi are the eigenvalues of Σ

•  by compu2ng the eigenvalues we know the data is

–  Not flat if λ1 ≈ λ2 –  Flat if λ1 >> λ2

15-‐Nov-‐14 17

technical

note


x1

x2

λ1 λ2

φ1

φ2

Lecture 17 - !!!

Fei-Fei Li!

PCA Algorithm (training)

15-‐Nov-‐14 18

technical

note


Lecture 17 - !!!

Fei-Fei Li!

PCA Algorithm (tes2ng)

15-‐Nov-‐14 19

technical

note


Lecture 17 - !!!

Fei-Fei Li!

PCA by SVD •  An alterna2ve manner to compute the principal components,

based on singular value decomposi2on •  Quick reminder: SVD

–  Any real n x m matrix (n>m) can be decomposed as

–  Where M is an (n x m) column orthonormal matrix of let singular vectors (columns of M)

–  Π is an (m x m) diagonal matrix of singular values –  NT is an (m x m) row orthonormal matrix of right singular vectors

(columns of N)

15-‐Nov-‐14 20

technical

note


Lecture 17 - !!!

Fei-Fei Li!

PCA by SVD •  To relate this to PCA, we consider the data matrix

•  The sample mean is

15-‐Nov-‐14 21

technical

note


Lecture 17 - !!!

Fei-Fei Li!

PCA by SVD •  Center the data by subtrac2ng the mean to each column of X •  The centered data matrix is

15-‐Nov-‐14 22

technical

note


Lecture 17 - !!!

Fei-Fei Li!

PCA by SVD •  The sample covariance matrix is

where xic is the ith column of Xc •  This can be wriien as

15-‐Nov-‐14 23

technical

note


Lecture 17 - !!!

Fei-Fei Li!

PCA by SVD •  The matrix

is real (n x d). Assuming n>d it has SVD decomposi2on

and

15-‐Nov-‐14 24

technical

note


Lecture 17 - !!!

Fei-Fei Li!

PCA by SVD

•  Note that N is (d x d) and orthonormal, and Π2 is diagonal. This is just the eigenvalue decomposi2on of Σ

•  It follows that –  The eigenvectors of Σ are the columns of N –  The eigenvalues of Σ are

•  This gives an alterna2ve algorithm for PCA

15-‐Nov-‐14 25

technical

note


Lecture 17 - !!!

Fei-Fei Li!

PCA by SVD •  In summary, computa2on of PCA by SVD •  Given X with one example per column

–  Create the centered data matrix

–  Compute its SVD

–  Principal components are columns of N, eigenvalues are

15-‐Nov-‐14 26

technical

note


Lecture 17 - !!!

Fei-Fei Li!

Rule of thumb for finding the number of PCA components

•  A natural measure is to pick the eigenvectors that explain p% of the data variability –  Can be done by ploxng the ra2o rk as a func2on of k

–  E.g. we need 3 eigenvectors to cover 70% of the variability of this dataset

15-‐Nov-‐14 27

technical

note


Lecture 17 - !!!

Fei-Fei Li!


15-‐Nov-‐14 28


Lecture 17 - !!!

Fei-Fei Li!

Eigenfaces: key idea

•  Assume that most face images lie on a low-‐dimensional subspace determined by the first k (k<<d) direc2ons of maximum variance

•  Use PCA to determine the vectors or “eigenfaces” that span that subspace

•  Represent all face images in the dataset as linear combina2ons of eigenfaces

15-‐Nov-‐14 29

M. Turk and A. Pentland, Face Recognition using Eigenfaces, CVPR 1991

Lecture 17 - !!!

Fei-Fei Li!

Eigenface algroithm •  Training

1.  Align training images x1, x2, …, xN

2.  Compute average face

3.  Compute the difference image (the centered data matrix)

15-‐Nov-‐14 30

technical

note

Note that each image is formulated into a long vector!

µ =1N

xi∑

Lecture 17 - !!!

Fei-Fei Li!

Eigenface algroithm

4.  Compute the covariance matrix

5.  Compute the eigenvectors of the covariance matrix Σ 6.  Compute each training image xi ‘s projec2ons as

7.  Visualize the es2mated training face xi

15-‐Nov-‐14 31

technical

note

xi → xic ⋅φ1, xi

c ⋅φ2,..., xic ⋅φK( ) ≡ a1,a2, ... ,aK( )

xi ≈ µ + a1φ1 + a2φ2 +...+ aKφK

Lecture 17 - !!!

Fei-Fei Li! 15-‐Nov-‐14 32

6.  Compute each training image xi ‘s projec2ons as

7.  Visualize the es2mated training face xi

xi → xic ⋅φ1, xi

c ⋅φ2,..., xic ⋅φK( ) ≡ a1,a2, ... ,aK( )

xi ≈ µ + a1φ1 + a2φ2 +...+ aKφK

a1φ1 a2φ2 aKφK...

Eigenface algroithm technical

note

Lecture 17 - !!!

Fei-Fei Li!

Eigenface algroithm

•  Tes2ng 1.  Take query image t 2.  Project y into eigenface space and compute projec2on

3.  Compare projec2on w with all N training projec2ons •  Simple comparison metric: Euclidean •  Simple decision: K-‐Nearest Neighbor (note: this “K” refers to the k-‐NN algorithm, is different from the previous K’s referring to the # of principal components)

15-‐Nov-‐14 33

technical

note

t→ (t −µ) ⋅φ1, (t −µ) ⋅φ2,..., (t −µ) ⋅φK( ) ≡ w1,w2,...,wK( )

Lecture 17 - !!!

Fei-Fei Li!

Eigenfaces look somewhat like generic faces.

15-‐Nov-‐14 34

Visualiza2on of eigenfaces

Lecture 17 - !!!

Fei-Fei Li!

•  Only selec2ng the top K eigenfaces à reduces the dimensionality. •  Fewer eigenfaces result in more informa2on loss, and hence less

discrimina2on between faces.

Reconstruc2on and Errors

K = 4

K = 200

K = 400

15-‐Nov-‐14 35

Lecture 17 - !!!

Fei-Fei Li!

Summary for Eigenface

Pros •  Non-‐itera2ve, globally op2mal solu2on

Limita2ons

• PCA projec2on is op&mal for reconstruc&on from a low dimensional basis, but may NOT be op&mal for discrimina&on…

• See supplementary materials for “Linear Discrimina2ve Analysis”, aka “Fisherfaces”

15-‐Nov-‐14 36

Lecture 17 - !!!

Fei-Fei Li!


15-‐Nov-‐14 37


Lecture 17 - !!!

Fei-Fei Li!

Fischer’s Linear Discriminant Analysis

•  Goal: find the best separa2on between two classes

15-‐Nov-‐14 38


Lecture 17 - !!!

Fei-Fei Li! 15-‐Nov-‐14 39

Basic intui2on: PCA vs. LDA

Lecture 17 - !!!

Fei-Fei Li!

Linear Discriminant Analysis (LDA)

•  We have two classes such that

•  We want to find the line z that best separates them

•  One possibility would be to maximize

15-‐Nov-‐14 40


technical

note

Lecture 17 - !!!

Fei-Fei Li!


•  However, this difference

can be arbitrarily large by simply scaling w •  We are only interested in the direc2on, not the magnitude •  Need some type of normaliza2on •  Fischer suggested

15-‐Nov-‐14 41


technical

note

Lecture 17 - !!!

Fei-Fei Li!


•  We have already seen that

•  also

15-‐Nov-‐14 42


technical

note

Lecture 17 - !!!

Fei-Fei Li!


•  And

•  which can be wriien as

15-‐Nov-‐14 43


technical

note

Lecture 17 - !!!

Fei-Fei Li!

2S

1S

BS

21 SSSW +=

x1

x2 Within class scaier

Between class scaier

15-‐Nov-‐14 44

Visualiza2on techn

ical

note

Lecture 17 - !!!

Fei-Fei Li!


•  Maximizing the ra2o

–  Is equivalent to maximizing the numerator while keeping the denominator constant, i.e.

–  And can be accomplished using Lagrange mul2pliers, where we define the Lagrangian as

–  And maximize with respect to both w and λ

15-‐Nov-‐14 45


technical

note

Lecture 17 - !!!

Fei-Fei Li!

Linear Discriminant Analysis (LDA) •  Sexng the gradient of

With respect to w to zeros we get

or •  This is a generalized eigenvalue problem •  The solu2on is easy when exists

15-‐Nov-‐14 46


technical

note

Lecture 17 - !!!

Fei-Fei Li!


•  In this case

•  And using the defini2on of S

•  No2ng that (μ1-‐μ0)Tw=α is a scalar this can be wriien as

•  and since we don’t care about the magnitude of w

15-‐Nov-‐14 47


technical

note

Lecture 17 - !!!

Fei-Fei Li!

PCA vs. LDA

•  Eigenfaces exploit the max scaier of the training images in face space

•  Fisherfaces aiempt to maximise the between class sca<er, while minimising the within class sca<er.

15-‐Nov-‐14 48

Lecture 17 - !!!

Fei-Fei Li!

Results: Eigenface vs. Fisherface (1)

•  Variation in Facial Expression, Eyewear, and Lighting

•  Input: 160 images of 16 people •  Train: 159 images •  Test: 1 image

With glasses

Without glasses

3 Lighting conditions 5 expressions

15-‐Nov-‐14 49

Lecture 17 - !!!

Fei-Fei Li!

Eigenface vs. Fisherface (2)

15-‐Nov-‐14 50

Lecture 17 - !!!

Fei-Fei Li!

What we have learned today •  Introduc2on to face recogni2on •  Principal Component Analysis (PCA) •  The Eigenfaces Algorithm •  Linear Discriminant Analysis (LDA)

15-‐Nov-‐14 51


Lecture’17:’’ Face’Recogni2on’ - Stanford...

Documents

Transcript of Lecture’17:’’ Face’Recogni2on’ - Stanford...