Invariant-Based Face Recognition

Nigel Boston

Departments of Mathematics and Electrical and Computer Engineering University of Wisconsin, Madison

Credits

• Professor Yu Hen Hu

• Wei-Yang Lin

• Ryan Kin Hong Wong

• UW Face Recognition Group

• www.ece.wisc.edu/~facerec/

• National Science Foundation

Summary of Talk

• Applications of algebra

• Challenges of face recognition

• Invariants of Lie groups

• Integral and summation invariants

• Implementation and results

Applications of Algebra to Engineering & Computer Science• (Hyper)elliptic curve cryptography - algebraic

attacks on AES• Space-time codes from cyclic division algebras

over number fields and fixed-point-free matrix groups (for wireless communication)

• Low-discrepancy sequences from ray class fields of function fields (for quasi-Monte Carlo methods)

Biometric Authentication

Biometric Authentication IIBiometrics is catching on in a big way as the previous slide showed.

For overt authentication, hand outlines and fingerprints have been very popular, but iris recognition is the most dependable in terms of giving a threshold, with few false negatives and false positives.

For covert authentication, a huge amount of research has gone intoface recognition but has been relatively unsuccessful. Visionics tested its FaceIt system in Palm Beach Airport for a month in 2002. 15 employees were compared against a database of 250 mug shots of airport workers. Out of 958 attempts, only 455 successful identifications were made. The Tampa police department also tried FaceIt over 6 months and failed to make any match with a database of known criminals.

Applications of Face Recognition

• Homeland security

• Cataloguing old photos

• Casinos

• Politicians

• Professors

The Problem We Face

Given the gallery photos on the left, identify the person in the new photo on the right.

The Problems We Face1. Detection of the face.2. Normalization of the face.3. Parametrization of the face.4. Lighting variation.5. Pose variation.6. Beards, disguises.7. Glasses, occlusion.8. Aging, weight gain.

Problems of Recognition I

Problems of Recognition II

Pawan Sinha gave an Identikit operator photographs of celebrities and asked him to create the best likenesses he could. The operator thought he did very well. Who are these people?

EigenFace Method

• Baseline method- PCA

Psychophysics of Face Recognition

There are three traditional approaches to face-recognition.

(a) Transformationist (popular in machine vision community): compute optimal transformation to bring image and model in register, then match. Problem - computationally expensive.

(b) View-based approach: viewing parameters in learning phase linked with performance on recognition tasks - storage expensive.

(c) Invariant-based approach: encode object views into a compact description of their invariant attributes. Inexpensive on both computation and storage. Simple and intuitively appealing.

Lie GroupsManifold = object locally like an open subset of Euclidean spaceMaps between manifolds locally like maps between Euclidean spaces.Smooth = infinitely differentiable.Lie Group = smooth manifold that is also a group.Interested in Lie groups acting on manifolds.

Example. SO(2) = group of all rotations acting on SE(2) = group generated by all translations and rotations, acting on

In general, a Lie group G acting on a manifold M will be given by a smooth map

InvariantsAn invariant of G is a real-valued function I from M to such that I(g.z) = I(z) for all z in M and all g in G.

Think of M as parametrizing various faces and G as a group of rotations and translations. We want to consider two faces as being the same if one is just the other moved in space, i.e. two faces will be the same if and only if they are in the same orbit. Invariant features are real functions constant on orbits. Call such faces equivalent.

Invariants IINote that if are invariants and anyreal-valued function, then is an invariant. We seek a complete set of fundamental invariants, i.e. any otherinvariant can be written as a function of these.

The idea is that in face recognition, given a face, represented by x in M, we store and then to check if a face y is equivalent to x, simply compute and compare.

Example: For SO(2), is the fundamental invariant.

Moving FramesA moving frame is a smooth G-equivariant map, i.e.

Theorem: A moving frame exists in a neighborhood of a point z in M if and only if G acts freely and regularly near z.

Let for all z (canonical forms). Let G be r-dimensional, M m-dimensional, r < m. Pick coordinates of M such that

If then the fundamental invariants are

For SO(2), r=1, m=2, so one fundamental invariant, I above.

Jet SpaceProblem - the group action might not be free. For instance, SE(2) acting on

Solution - increase the dimension of the manifold acted upon, by including derivatives (jet space) or by M x … x M (joint invariants).

Example - the signature curve of a curve C in is the curve whose points are curvature, s arc length.

Theorem - Two curves are equivalent under the action of SE(2) on if and only if their signature curves are equal.

Good news - just store and compare signature curves.Bad news - derivatives are too sensitive to noise.

Integral InvariantsDespite the large history and literature on differential invariants, it was only 2002 when Hann and Hickman extended actions to integrals rather than derivatives. If G acts on = {(x,u)}, then the monomial potential of order k ( ) is given by:

Hann-Hickman defined potential jet space with local coordinates and proved that the action of G prolongs to an action on potential jet space. We defined and implemented simpler integral invariants by using parameters. This extends to the 3D case. .

Summation Invariants

In practice, summation rather than integral invariants are employed. In 2D suppose the boundary of an object is parametrized by(x[n],y[n]) (n = 0, …, N). The potentials of order k are given by

Potential jet space consists of potentials up to some order. Example: say g.(x,y) = (ax + by + c, dx + ey + f) where ae-bd is nonzero. The group action sends the order 1 potentials to Solve for a,b,c,d,e,f setting

Then plugging that {a,b,c,d,e,f} into e.g. yields an invariant.

Semi-local Summation Invariants

• In order to extract local characteristics of shape, we compute summation invariants locally.

N

Fish RecognitionRandomly selected 100 fish contours from the SQUID database and resampled each 2D contour curve so that the number of points is 512. For each curve generated 20 variations by applying random affine transformations and Gaussian distributed noise.

Fish contours are shown after adding Gaussian distributed noise with increasing standard deviations.Compared with e.g. wavelet-based techniques, our method had about one eighth the number of errors.

-350 -300 -250 -200 -150 -100 -50 0-180

-160

-140

-120

-100

-80

-60

-40

-20

-350 -300 -250 -200 -150 -100 -50 0-180

-160

-140

-120

-100

-80

-60

-40

-20

Implementation Issues

• Face detection (RGB values, eyes)

• 3D reconstruction from stereo images

• Parametrization (conformal mapping)

Our goal is to handle a semi-controlled environment (e.g. someone walking along a corridor with controlled lighting).

Conformal MappingsWe use ideas from “Genus zero surface conformal mapping and its application to brain surface mapping” by Gu, Wang, Tony Chan, Thompson, and Shing-Tung Yau.

A conformal equivalence is bijective and angle-preserving and so retains local geometric information. Conformal parametrizations work well.

3D Face Recognition

• Preprocessing - crop region around nose tip

• Feature extraction from 3D depth map

• Computation of similarity measures

Tested with 3D range data from the Face Recognition Grand Challenge, we observed significant performance over their baseline.

Face Recognition Grand Challenge

• Goal: to advance performance of face recognition by 10-fold (20% 2% verification rate @0.1% false alarm rate)

• Focus on five different scenarios.

• Status: on-going, to be concluded by the end of 2005

Matching Score Protocol

Comparison with 3D Baseline

Conclusions1. Traditionally engineers and computer scientists have used analysis (DE’s, probability, …) but algebra use is increasing.

2. Face recognition is a hot topic but far from being solved.

3. New mathematics (integral and summation invariants) results from this work.

4. Engineering feedback (sensitivity to noise) drives this.

5. New feature gives superior results to 3D baseline algorithm.