Ongoing Challenges in Face Recognition Peter Belhumeur Columbia University New York City.
-
Upload
candice-houston -
Category
Documents
-
view
216 -
download
0
Transcript of Ongoing Challenges in Face Recognition Peter Belhumeur Columbia University New York City.
![Page 1: Ongoing Challenges in Face Recognition Peter Belhumeur Columbia University New York City.](https://reader035.fdocuments.us/reader035/viewer/2022062714/56649cf85503460f949c8fa3/html5/thumbnails/1.jpg)
Ongoing Challenges in Face Recognition
Peter BelhumeurPeter Belhumeur
Columbia UniversityColumbia UniversityNew York CityNew York City
![Page 2: Ongoing Challenges in Face Recognition Peter Belhumeur Columbia University New York City.](https://reader035.fdocuments.us/reader035/viewer/2022062714/56649cf85503460f949c8fa3/html5/thumbnails/2.jpg)
How are people identified?
People are identified by three basic means:
– Something they have (identity document or token)
– Something they know (password, PIN)
– Something they are (human body)
![Page 3: Ongoing Challenges in Face Recognition Peter Belhumeur Columbia University New York City.](https://reader035.fdocuments.us/reader035/viewer/2022062714/56649cf85503460f949c8fa3/html5/thumbnails/3.jpg)
Iris
![Page 4: Ongoing Challenges in Face Recognition Peter Belhumeur Columbia University New York City.](https://reader035.fdocuments.us/reader035/viewer/2022062714/56649cf85503460f949c8fa3/html5/thumbnails/4.jpg)
Retina
Every eye has its own totally unique pattern of blood vessels.
![Page 5: Ongoing Challenges in Face Recognition Peter Belhumeur Columbia University New York City.](https://reader035.fdocuments.us/reader035/viewer/2022062714/56649cf85503460f949c8fa3/html5/thumbnails/5.jpg)
Hand
![Page 6: Ongoing Challenges in Face Recognition Peter Belhumeur Columbia University New York City.](https://reader035.fdocuments.us/reader035/viewer/2022062714/56649cf85503460f949c8fa3/html5/thumbnails/6.jpg)
Fingerprint
![Page 7: Ongoing Challenges in Face Recognition Peter Belhumeur Columbia University New York City.](https://reader035.fdocuments.us/reader035/viewer/2022062714/56649cf85503460f949c8fa3/html5/thumbnails/7.jpg)
Ear
![Page 8: Ongoing Challenges in Face Recognition Peter Belhumeur Columbia University New York City.](https://reader035.fdocuments.us/reader035/viewer/2022062714/56649cf85503460f949c8fa3/html5/thumbnails/8.jpg)
Face
![Page 9: Ongoing Challenges in Face Recognition Peter Belhumeur Columbia University New York City.](https://reader035.fdocuments.us/reader035/viewer/2022062714/56649cf85503460f949c8fa3/html5/thumbnails/9.jpg)
Who are these people?
[Sinha and Poggio 1996]
![Page 10: Ongoing Challenges in Face Recognition Peter Belhumeur Columbia University New York City.](https://reader035.fdocuments.us/reader035/viewer/2022062714/56649cf85503460f949c8fa3/html5/thumbnails/10.jpg)
Who are these people?
[Sinha and Poggio 2002]
![Page 11: Ongoing Challenges in Face Recognition Peter Belhumeur Columbia University New York City.](https://reader035.fdocuments.us/reader035/viewer/2022062714/56649cf85503460f949c8fa3/html5/thumbnails/11.jpg)
Images as Points in Euclidean Space
• Let an n-pixel image to be a point in an n-D space, x Rn.
• Each pixel value is a coordinate of x.
x1
x2
xn
![Page 12: Ongoing Challenges in Face Recognition Peter Belhumeur Columbia University New York City.](https://reader035.fdocuments.us/reader035/viewer/2022062714/56649cf85503460f949c8fa3/html5/thumbnails/12.jpg)
Face Recognition: Euclidean Distances
D2 > 0
D3 0
D1 > 0
~
![Page 13: Ongoing Challenges in Face Recognition Peter Belhumeur Columbia University New York City.](https://reader035.fdocuments.us/reader035/viewer/2022062714/56649cf85503460f949c8fa3/html5/thumbnails/13.jpg)
Face Recognition: Euclidean Distances
D2 > 0
D3 > D1 or 2
D1 > 0
[Hallinan 1994] [Adini, Moses, and Ullman 1994]
![Page 14: Ongoing Challenges in Face Recognition Peter Belhumeur Columbia University New York City.](https://reader035.fdocuments.us/reader035/viewer/2022062714/56649cf85503460f949c8fa3/html5/thumbnails/14.jpg)
Same Person or
Different People
![Page 15: Ongoing Challenges in Face Recognition Peter Belhumeur Columbia University New York City.](https://reader035.fdocuments.us/reader035/viewer/2022062714/56649cf85503460f949c8fa3/html5/thumbnails/15.jpg)
![Page 16: Ongoing Challenges in Face Recognition Peter Belhumeur Columbia University New York City.](https://reader035.fdocuments.us/reader035/viewer/2022062714/56649cf85503460f949c8fa3/html5/thumbnails/16.jpg)
Same Person or
Different People
![Page 17: Ongoing Challenges in Face Recognition Peter Belhumeur Columbia University New York City.](https://reader035.fdocuments.us/reader035/viewer/2022062714/56649cf85503460f949c8fa3/html5/thumbnails/17.jpg)
![Page 18: Ongoing Challenges in Face Recognition Peter Belhumeur Columbia University New York City.](https://reader035.fdocuments.us/reader035/viewer/2022062714/56649cf85503460f949c8fa3/html5/thumbnails/18.jpg)
Why is Face Recognition Hard?
![Page 19: Ongoing Challenges in Face Recognition Peter Belhumeur Columbia University New York City.](https://reader035.fdocuments.us/reader035/viewer/2022062714/56649cf85503460f949c8fa3/html5/thumbnails/19.jpg)
Challenges: Image Variability
Expression
Illumination
PosePoseShort Term
•Facial Hair•Makeup•Eyewear
•Hairstyle•Piercings•Aging
Long Term
![Page 20: Ongoing Challenges in Face Recognition Peter Belhumeur Columbia University New York City.](https://reader035.fdocuments.us/reader035/viewer/2022062714/56649cf85503460f949c8fa3/html5/thumbnails/20.jpg)
Illumination Invariants?
f ( ) = f ( ) = f ( ) = a
f ( ) = f ( ) = f ( ) = b
and
Does there exist a function f s.t.
?
![Page 21: Ongoing Challenges in Face Recognition Peter Belhumeur Columbia University New York City.](https://reader035.fdocuments.us/reader035/viewer/2022062714/56649cf85503460f949c8fa3/html5/thumbnails/21.jpg)
a
I(x,y)
a
J(x,y)
Can Any Two Images Arise from a Single Surface?
Same Albedo and Surface Different Lighting
I(x,y) = a(x,y) n(x,y) s
J(x,y) = a(x,y) n(x,y) l
ns
nl
![Page 22: Ongoing Challenges in Face Recognition Peter Belhumeur Columbia University New York City.](https://reader035.fdocuments.us/reader035/viewer/2022062714/56649cf85503460f949c8fa3/html5/thumbnails/22.jpg)
The Surface PDE
Nonlinear PDE Linear PDE
I(x,y) = a(x,y) n(x,y) s
J(x,y) = a(x,y) n(x,y) l( I l – J s ) n = 0
![Page 23: Ongoing Challenges in Face Recognition Peter Belhumeur Columbia University New York City.](https://reader035.fdocuments.us/reader035/viewer/2022062714/56649cf85503460f949c8fa3/html5/thumbnails/23.jpg)
Illumination invariants for 3-D objects do not exist.
This result does not ignore attached and cast shadows, as well as
surface interreflection.
Non-Existence Theorem for Illumination Invariants
[Chen, Belhumeur, and Jacobs 2000]
![Page 24: Ongoing Challenges in Face Recognition Peter Belhumeur Columbia University New York City.](https://reader035.fdocuments.us/reader035/viewer/2022062714/56649cf85503460f949c8fa3/html5/thumbnails/24.jpg)
Geometric Invariants?
f ( ) = f ( ) = f ( ) = a
f ( ) = f ( ) = f ( ) = b
and
Does there exist a function f s.t.
?
![Page 25: Ongoing Challenges in Face Recognition Peter Belhumeur Columbia University New York City.](https://reader035.fdocuments.us/reader035/viewer/2022062714/56649cf85503460f949c8fa3/html5/thumbnails/25.jpg)
Geometric invariants for rigid transformations of 3-D
objects viewed under perspective projective projection do
not exist.
Non-Existence Theorem for Geometric Invariants
[Burns, Weiss, and Riseman 1992]
![Page 26: Ongoing Challenges in Face Recognition Peter Belhumeur Columbia University New York City.](https://reader035.fdocuments.us/reader035/viewer/2022062714/56649cf85503460f949c8fa3/html5/thumbnails/26.jpg)
Image Variability: Appearance Manifolds
x2
x1
xn
Lighting x Pose[Murase and Nayar 1993]
![Page 27: Ongoing Challenges in Face Recognition Peter Belhumeur Columbia University New York City.](https://reader035.fdocuments.us/reader035/viewer/2022062714/56649cf85503460f949c8fa3/html5/thumbnails/27.jpg)
Modeling Image Variability
Can we model illumination and pose variability in images of a face?
Yes, if we can determine the shape and texture of the face. But how?
![Page 28: Ongoing Challenges in Face Recognition Peter Belhumeur Columbia University New York City.](https://reader035.fdocuments.us/reader035/viewer/2022062714/56649cf85503460f949c8fa3/html5/thumbnails/28.jpg)
Modeling Image Variability: 3-D Faces
Laser Range Scanners
Stereo Cameras
Structured Light
Photometric Stereo
][Atick, Griffin, Redlich 1996] [Georghiades, Belhumeur, Kriegman 1996] [Blanz and Vetter 1999] [Zhao and Chellepa 1999] [Kimmel and Sapiro 2003] [Geometrix 2001] [MERL 2005]
![Page 29: Ongoing Challenges in Face Recognition Peter Belhumeur Columbia University New York City.](https://reader035.fdocuments.us/reader035/viewer/2022062714/56649cf85503460f949c8fa3/html5/thumbnails/29.jpg)
Illumination Variation Reveals Object Shape
ns1
a
I3
s2
s3
I2I1
[Woodham 1984]
![Page 30: Ongoing Challenges in Face Recognition Peter Belhumeur Columbia University New York City.](https://reader035.fdocuments.us/reader035/viewer/2022062714/56649cf85503460f949c8fa3/html5/thumbnails/30.jpg)
Illumination Movie
Illumination Movie
![Page 31: Ongoing Challenges in Face Recognition Peter Belhumeur Columbia University New York City.](https://reader035.fdocuments.us/reader035/viewer/2022062714/56649cf85503460f949c8fa3/html5/thumbnails/31.jpg)
Shape Movie
Shape Movie
![Page 32: Ongoing Challenges in Face Recognition Peter Belhumeur Columbia University New York City.](https://reader035.fdocuments.us/reader035/viewer/2022062714/56649cf85503460f949c8fa3/html5/thumbnails/32.jpg)
Image Variability: From Few to Many
x2
x1
xn
Lighting x Pose
[Georghiades, Belhumeur, and Kriegman 1999]
Real
Synthetic
![Page 33: Ongoing Challenges in Face Recognition Peter Belhumeur Columbia University New York City.](https://reader035.fdocuments.us/reader035/viewer/2022062714/56649cf85503460f949c8fa3/html5/thumbnails/33.jpg)
Illumination Dome
![Page 34: Ongoing Challenges in Face Recognition Peter Belhumeur Columbia University New York City.](https://reader035.fdocuments.us/reader035/viewer/2022062714/56649cf85503460f949c8fa3/html5/thumbnails/34.jpg)
Real vs. Synthetic
Real Synthetic
![Page 35: Ongoing Challenges in Face Recognition Peter Belhumeur Columbia University New York City.](https://reader035.fdocuments.us/reader035/viewer/2022062714/56649cf85503460f949c8fa3/html5/thumbnails/35.jpg)
Real vs. Synthetic
Real
Synthetic
![Page 36: Ongoing Challenges in Face Recognition Peter Belhumeur Columbia University New York City.](https://reader035.fdocuments.us/reader035/viewer/2022062714/56649cf85503460f949c8fa3/html5/thumbnails/36.jpg)
A Step Back in Time
![Page 37: Ongoing Challenges in Face Recognition Peter Belhumeur Columbia University New York City.](https://reader035.fdocuments.us/reader035/viewer/2022062714/56649cf85503460f949c8fa3/html5/thumbnails/37.jpg)
Albrecht Dürer, “Four Books on Human Proportion” (1528)
![Page 38: Ongoing Challenges in Face Recognition Peter Belhumeur Columbia University New York City.](https://reader035.fdocuments.us/reader035/viewer/2022062714/56649cf85503460f949c8fa3/html5/thumbnails/38.jpg)
D’arcy Thompson, “On Growth and Form” (1917)
![Page 39: Ongoing Challenges in Face Recognition Peter Belhumeur Columbia University New York City.](https://reader035.fdocuments.us/reader035/viewer/2022062714/56649cf85503460f949c8fa3/html5/thumbnails/39.jpg)
D’arcy Thompson, “On Growth and Form” (1917)
![Page 40: Ongoing Challenges in Face Recognition Peter Belhumeur Columbia University New York City.](https://reader035.fdocuments.us/reader035/viewer/2022062714/56649cf85503460f949c8fa3/html5/thumbnails/40.jpg)
D’arcy Thompson, “On Growth and Form” (1917)
![Page 41: Ongoing Challenges in Face Recognition Peter Belhumeur Columbia University New York City.](https://reader035.fdocuments.us/reader035/viewer/2022062714/56649cf85503460f949c8fa3/html5/thumbnails/41.jpg)
But what if we could ….?
[Blanz and Vetter 1999, 2003]
![Page 42: Ongoing Challenges in Face Recognition Peter Belhumeur Columbia University New York City.](https://reader035.fdocuments.us/reader035/viewer/2022062714/56649cf85503460f949c8fa3/html5/thumbnails/42.jpg)
Building a Morphable Face Model
[Blanz and Vetter 1999, 2003]
![Page 43: Ongoing Challenges in Face Recognition Peter Belhumeur Columbia University New York City.](https://reader035.fdocuments.us/reader035/viewer/2022062714/56649cf85503460f949c8fa3/html5/thumbnails/43.jpg)
3-D Morphaple Models: Semi-Automatic
[Blanz and Vetter 1999, 2003]
![Page 44: Ongoing Challenges in Face Recognition Peter Belhumeur Columbia University New York City.](https://reader035.fdocuments.us/reader035/viewer/2022062714/56649cf85503460f949c8fa3/html5/thumbnails/44.jpg)
Building Morphable Face Models
[Blanz and Vetter 1999, 2003]
![Page 45: Ongoing Challenges in Face Recognition Peter Belhumeur Columbia University New York City.](https://reader035.fdocuments.us/reader035/viewer/2022062714/56649cf85503460f949c8fa3/html5/thumbnails/45.jpg)
Fitting Morphable Face Models
[Blanz and Vetter 1999, 2003]
![Page 46: Ongoing Challenges in Face Recognition Peter Belhumeur Columbia University New York City.](https://reader035.fdocuments.us/reader035/viewer/2022062714/56649cf85503460f949c8fa3/html5/thumbnails/46.jpg)
National Geographic 1984 and 2002
1985 2002
![Page 47: Ongoing Challenges in Face Recognition Peter Belhumeur Columbia University New York City.](https://reader035.fdocuments.us/reader035/viewer/2022062714/56649cf85503460f949c8fa3/html5/thumbnails/47.jpg)
Identity Confirmed by IRIS
=
[Daugman 2002]