Anil K. Jain Michigan State University Biometrics:...
Transcript of Anil K. Jain Michigan State University Biometrics:...
![Page 1: Anil K. Jain Michigan State University Biometrics: …biometrics.cse.msu.edu/Presentations/Haifa_workshop_Feb...3 Months, 13 days 3 Months, 15 days 6 Months, 5 days 3 Months, 13 days](https://reader036.fdocuments.us/reader036/viewer/2022063002/5f2320d335fa24741a7f603c/html5/thumbnails/1.jpg)
Biometrics: 2020 Vision
Anil K. JainMichigan State
University
http://biometrics.cse.msu.edu
1st Israeli Biometrics Winter School, Haifa
February 11, 2020
![Page 2: Anil K. Jain Michigan State University Biometrics: …biometrics.cse.msu.edu/Presentations/Haifa_workshop_Feb...3 Months, 13 days 3 Months, 15 days 6 Months, 5 days 3 Months, 13 days](https://reader036.fdocuments.us/reader036/viewer/2022063002/5f2320d335fa24741a7f603c/html5/thumbnails/2.jpg)
Outline
• Background and milestones
• Drivers (applications) and enablers
• State-of-the-art
• 2020 Vision
• Summary
2
![Page 3: Anil K. Jain Michigan State University Biometrics: …biometrics.cse.msu.edu/Presentations/Haifa_workshop_Feb...3 Months, 13 days 3 Months, 15 days 6 Months, 5 days 3 Months, 13 days](https://reader036.fdocuments.us/reader036/viewer/2022063002/5f2320d335fa24741a7f603c/html5/thumbnails/3.jpg)
Security Concerns
We now live in a society where individuals
cannot be trusted based on their ID
documents and PIN/passwords
![Page 4: Anil K. Jain Michigan State University Biometrics: …biometrics.cse.msu.edu/Presentations/Haifa_workshop_Feb...3 Months, 13 days 3 Months, 15 days 6 Months, 5 days 3 Months, 13 days](https://reader036.fdocuments.us/reader036/viewer/2022063002/5f2320d335fa24741a7f603c/html5/thumbnails/4.jpg)
![Page 5: Anil K. Jain Michigan State University Biometrics: …biometrics.cse.msu.edu/Presentations/Haifa_workshop_Feb...3 Months, 13 days 3 Months, 15 days 6 Months, 5 days 3 Months, 13 days](https://reader036.fdocuments.us/reader036/viewer/2022063002/5f2320d335fa24741a7f603c/html5/thumbnails/5.jpg)
Methods of Person Recognition
3.14159
What you have?
What you know?
Access Method
What you are?
✓ Unique
✓ Stable
✓ Natural
❖ Lost
❖ Stolen
❖ Forgot
❖ Cracked
![Page 6: Anil K. Jain Michigan State University Biometrics: …biometrics.cse.msu.edu/Presentations/Haifa_workshop_Feb...3 Months, 13 days 3 Months, 15 days 6 Months, 5 days 3 Months, 13 days](https://reader036.fdocuments.us/reader036/viewer/2022063002/5f2320d335fa24741a7f603c/html5/thumbnails/6.jpg)
Biometrics
• Morris (1875): Bios: life and Metron: a measure
• Pollack (1981): What makes each person unique?
• ISO/IEC JTC1 2382-37:2012: “Automated
recognition of individuals based on their
behavioral and biological characteristics”
• Foundation of biometrics laid over 150 years back
Stigler, “The Problematic Unity of Biometries”, BIOMETRIC5, S6 , Sept. 2000 Pollack, “Technology: Recognizing the Real You”, NYT, Sep. 24, 1981
![Page 7: Anil K. Jain Michigan State University Biometrics: …biometrics.cse.msu.edu/Presentations/Haifa_workshop_Feb...3 Months, 13 days 3 Months, 15 days 6 Months, 5 days 3 Months, 13 days](https://reader036.fdocuments.us/reader036/viewer/2022063002/5f2320d335fa24741a7f603c/html5/thumbnails/7.jpg)
Biometric Milestones
300
B.C.
1839 1869 1883 1900 1905 1924 1972 19991963 2001 2003 2008 2013 2014 20171858 2018 2019
Fingerprint as a personal mark
Habitual Criminals Act“What is wanted is a means ofclassifying the records ofhabitual criminal, such that assoon as the particulars of thepersonality of any prisoner(whether description,measurements, marks, orphotographs) are received, itmay be possible to ascertainreadily, and with certainty,whether his case is in theregister, and if so, who he is”
First use of fingerprints in
British criminal case
Bertillonage invented
Galton / Henry fingerprint system adopted by Scotland Yard
• Seventeen classes
• Whorl (double loop), loop (left and right)
& arch cover 99% of fingerprints
Delta Core
Identimat: First commercial use of biometrics
FBI inaugurates full operation
of “IAFIS”
State AFIS
State AFIS
State AFIS
State AFIS IAFIS
Forensics Other operations
Criminal booking
9/11 terrorist attacks lead to govt. mandates to use biometrics in
regulating intl. travel
Apple Pay
FaceID
US Congress authorizes DOJ to collect fingerprints and arrest information
Biometric Payment Cards
Aadhaar
FBI Next Generation Identification
Fingerprint: M. Trauring, Nature, 1961 Face: W. W. Bledsoe, “Man-Machine Facial Recognition”, Tech. Report PRI 22, Panoramic Res., 1966Iris: L. Flom and A. Safir, “Iris Recognition System”, US Patent 4641349 A, 1987Voice: S. Pruzansky, “Pattern-Matching Procedure for Automatic Talker Recognition”, J. Acoustic Society of America, 1963
TouchID
First In-display Fingerprint Sensor in
smartphones
US-VISITA Chinese deed of sale with a fingerprint
Early use of fingerprints for Civil applications
(Bengal, India)
Y. Taigman, M. Yang, M. Ranzato, L. Wolf, “DeepFace: Closing the Gap to Human-Level Performance in Face Verification”, CVPR 2014
![Page 8: Anil K. Jain Michigan State University Biometrics: …biometrics.cse.msu.edu/Presentations/Haifa_workshop_Feb...3 Months, 13 days 3 Months, 15 days 6 Months, 5 days 3 Months, 13 days](https://reader036.fdocuments.us/reader036/viewer/2022063002/5f2320d335fa24741a7f603c/html5/thumbnails/8.jpg)
Biometric Characteristics (Traits)
8
![Page 9: Anil K. Jain Michigan State University Biometrics: …biometrics.cse.msu.edu/Presentations/Haifa_workshop_Feb...3 Months, 13 days 3 Months, 15 days 6 Months, 5 days 3 Months, 13 days](https://reader036.fdocuments.us/reader036/viewer/2022063002/5f2320d335fa24741a7f603c/html5/thumbnails/9.jpg)
Rejected Traits
![Page 10: Anil K. Jain Michigan State University Biometrics: …biometrics.cse.msu.edu/Presentations/Haifa_workshop_Feb...3 Months, 13 days 3 Months, 15 days 6 Months, 5 days 3 Months, 13 days](https://reader036.fdocuments.us/reader036/viewer/2022063002/5f2320d335fa24741a7f603c/html5/thumbnails/10.jpg)
• Uniqueness (distinctive across users)
• Permanence (does not change over time)
• Universality (every user has it)
• Collectability (ease of capture)
• Performance (error rate, throughput)
• User experience (acceptable to the users)
• Circumvention (resistant to spoofing)
• Integration (easy to embed in an application)
Which Biometric Trait?
10
![Page 11: Anil K. Jain Michigan State University Biometrics: …biometrics.cse.msu.edu/Presentations/Haifa_workshop_Feb...3 Months, 13 days 3 Months, 15 days 6 Months, 5 days 3 Months, 13 days](https://reader036.fdocuments.us/reader036/viewer/2022063002/5f2320d335fa24741a7f603c/html5/thumbnails/11.jpg)
Most Frequently Used Biometric Traits
• Legacy databases
• Capability for 1:N search for large N
• Uniqueness (capacity) and stability
• High accuracy in NIST evaluations
![Page 12: Anil K. Jain Michigan State University Biometrics: …biometrics.cse.msu.edu/Presentations/Haifa_workshop_Feb...3 Months, 13 days 3 Months, 15 days 6 Months, 5 days 3 Months, 13 days](https://reader036.fdocuments.us/reader036/viewer/2022063002/5f2320d335fa24741a7f603c/html5/thumbnails/12.jpg)
Biometrics Applications
12
![Page 13: Anil K. Jain Michigan State University Biometrics: …biometrics.cse.msu.edu/Presentations/Haifa_workshop_Feb...3 Months, 13 days 3 Months, 15 days 6 Months, 5 days 3 Months, 13 days](https://reader036.fdocuments.us/reader036/viewer/2022063002/5f2320d335fa24741a7f603c/html5/thumbnails/13.jpg)
Applications in Israel
https://www.gov.il/en/service/biometric_smart_id_request
ID card with Face and two index fingerprints Speed Gate: West Bank Check posts
https://www.npr.org/2019/08/22/752765606/face-recognition-lets-palestinians-cross-israeli-checkposts-fast-but-raises-conc
![Page 14: Anil K. Jain Michigan State University Biometrics: …biometrics.cse.msu.edu/Presentations/Haifa_workshop_Feb...3 Months, 13 days 3 Months, 15 days 6 Months, 5 days 3 Months, 13 days](https://reader036.fdocuments.us/reader036/viewer/2022063002/5f2320d335fa24741a7f603c/html5/thumbnails/14.jpg)
Application Requirements
Walt Disney Theme Park (2005)
• Throughput
• Embedded
• Ease of use
• Low cost
• Spoof detection
![Page 15: Anil K. Jain Michigan State University Biometrics: …biometrics.cse.msu.edu/Presentations/Haifa_workshop_Feb...3 Months, 13 days 3 Months, 15 days 6 Months, 5 days 3 Months, 13 days](https://reader036.fdocuments.us/reader036/viewer/2022063002/5f2320d335fa24741a7f603c/html5/thumbnails/15.jpg)
Biometrics Recognition System
• Enrollment; Template
• Authentication (claim an identity); Search
FeatureExtractor
TemplateDatabase
Authentication Enrollment
Similarity
computation(Threshold)
Yes/No
Preprocessor Preprocessor
![Page 16: Anil K. Jain Michigan State University Biometrics: …biometrics.cse.msu.edu/Presentations/Haifa_workshop_Feb...3 Months, 13 days 3 Months, 15 days 6 Months, 5 days 3 Months, 13 days](https://reader036.fdocuments.us/reader036/viewer/2022063002/5f2320d335fa24741a7f603c/html5/thumbnails/16.jpg)
Biometric Template
Enrollment
Trained
Deep Network
d-dim
![Page 17: Anil K. Jain Michigan State University Biometrics: …biometrics.cse.msu.edu/Presentations/Haifa_workshop_Feb...3 Months, 13 days 3 Months, 15 days 6 Months, 5 days 3 Months, 13 days](https://reader036.fdocuments.us/reader036/viewer/2022063002/5f2320d335fa24741a7f603c/html5/thumbnails/17.jpg)
Face Authentication
Same Person?Similarity > T
Probe Gallery
![Page 18: Anil K. Jain Michigan State University Biometrics: …biometrics.cse.msu.edu/Presentations/Haifa_workshop_Feb...3 Months, 13 days 3 Months, 15 days 6 Months, 5 days 3 Months, 13 days](https://reader036.fdocuments.us/reader036/viewer/2022063002/5f2320d335fa24741a7f603c/html5/thumbnails/18.jpg)
Search
Probe
Who is this?
One of them?
Search System
Gallery (N faces)
Manual Inspection
Wang, Otto, Jain, “Face Search at Scale: 80 Million Gallery“, arXiv:1507.07242v2 [cs.CV] 18
![Page 19: Anil K. Jain Michigan State University Biometrics: …biometrics.cse.msu.edu/Presentations/Haifa_workshop_Feb...3 Months, 13 days 3 Months, 15 days 6 Months, 5 days 3 Months, 13 days](https://reader036.fdocuments.us/reader036/viewer/2022063002/5f2320d335fa24741a7f603c/html5/thumbnails/19.jpg)
Authentication Performance
Similarity Score Distribution ROC Curve
![Page 20: Anil K. Jain Michigan State University Biometrics: …biometrics.cse.msu.edu/Presentations/Haifa_workshop_Feb...3 Months, 13 days 3 Months, 15 days 6 Months, 5 days 3 Months, 13 days](https://reader036.fdocuments.us/reader036/viewer/2022063002/5f2320d335fa24741a7f603c/html5/thumbnails/20.jpg)
Closed-set IdentificationTop-5 RetrievalsProbe Image
● All the probes are guaranteed a mate (green) in the gallery
● Search is a success if the mate is in the top-K retrievals
● Rank-K retrieval rate is the percentage of successful searches
...
...
Success
Failure
![Page 21: Anil K. Jain Michigan State University Biometrics: …biometrics.cse.msu.edu/Presentations/Haifa_workshop_Feb...3 Months, 13 days 3 Months, 15 days 6 Months, 5 days 3 Months, 13 days](https://reader036.fdocuments.us/reader036/viewer/2022063002/5f2320d335fa24741a7f603c/html5/thumbnails/21.jpg)
State of the Art: Authentication
21
Fingerprint: TAR = 99.964% @ FAR =0.01% (FVC-ongoing)
Iris: TAR = 99.82% @ FAR = 0.01% (NIST IREX II)
Face: TAR = 99.7% @ FAR = 0.1% (NIST FRVT 2010)
![Page 22: Anil K. Jain Michigan State University Biometrics: …biometrics.cse.msu.edu/Presentations/Haifa_workshop_Feb...3 Months, 13 days 3 Months, 15 days 6 Months, 5 days 3 Months, 13 days](https://reader036.fdocuments.us/reader036/viewer/2022063002/5f2320d335fa24741a7f603c/html5/thumbnails/22.jpg)
State of the Art: Unconstrained FR Performance
NIST IJB-A (2015) NIST IJB-S (2018)YTF (2012)LFW (2009)
TAR
@FA
R=0
.1%
DifficultyLFW
(2009)
99.92%
IJB-A(2015)
95.67%
82.27%
IJB-S(2018)
4.86%
YTF(2012)
Frame-to-Frame VerificationTemplate Identification
Gallery Size Rank1 Rank5
IJB-A 112 97.5 98.4
IJB-S (S2B) 202 62.0 67.1
S2B: Surveillance to Booking protocol
![Page 23: Anil K. Jain Michigan State University Biometrics: …biometrics.cse.msu.edu/Presentations/Haifa_workshop_Feb...3 Months, 13 days 3 Months, 15 days 6 Months, 5 days 3 Months, 13 days](https://reader036.fdocuments.us/reader036/viewer/2022063002/5f2320d335fa24741a7f603c/html5/thumbnails/23.jpg)
World’s Largest Biometric System
“To empower residents of India with a unique identity and
a digital platform to authenticate anytime, anywhere.”
• Established in 2009
• Enrolment of ~1.3 billion residents
![Page 24: Anil K. Jain Michigan State University Biometrics: …biometrics.cse.msu.edu/Presentations/Haifa_workshop_Feb...3 Months, 13 days 3 Months, 15 days 6 Months, 5 days 3 Months, 13 days](https://reader036.fdocuments.us/reader036/viewer/2022063002/5f2320d335fa24741a7f603c/html5/thumbnails/24.jpg)
Aadhaar (Foundation)
A 12-digit randomly generated Identifier, linked to your biometrics
![Page 25: Anil K. Jain Michigan State University Biometrics: …biometrics.cse.msu.edu/Presentations/Haifa_workshop_Feb...3 Months, 13 days 3 Months, 15 days 6 Months, 5 days 3 Months, 13 days](https://reader036.fdocuments.us/reader036/viewer/2022063002/5f2320d335fa24741a7f603c/html5/thumbnails/25.jpg)
Aadhaar Verification
Aadhaar Biometric Modules
Biometric Data Acquisition
(Enrollment)
Biometric Deduplication (search)Module
Verification ModuleBiometric Data
for Verification
Aadhaar Generation System & ID Repository
D1
D2
D3
M1 M2 M3
Aadhaar Enrollment
Aadhaar Generation
![Page 26: Anil K. Jain Michigan State University Biometrics: …biometrics.cse.msu.edu/Presentations/Haifa_workshop_Feb...3 Months, 13 days 3 Months, 15 days 6 Months, 5 days 3 Months, 13 days](https://reader036.fdocuments.us/reader036/viewer/2022063002/5f2320d335fa24741a7f603c/html5/thumbnails/26.jpg)
Enrollment
Minimal documentation
for enrollment
![Page 27: Anil K. Jain Michigan State University Biometrics: …biometrics.cse.msu.edu/Presentations/Haifa_workshop_Feb...3 Months, 13 days 3 Months, 15 days 6 Months, 5 days 3 Months, 13 days](https://reader036.fdocuments.us/reader036/viewer/2022063002/5f2320d335fa24741a7f603c/html5/thumbnails/27.jpg)
De-duplication (Find duplicates)
New Applicant
…
Enrollment database
Alre
ad
y E
xis
ts in
the
Data
base
?
Fusion of face, fingerprint and iris improves de-duplication
![Page 28: Anil K. Jain Michigan State University Biometrics: …biometrics.cse.msu.edu/Presentations/Haifa_workshop_Feb...3 Months, 13 days 3 Months, 15 days 6 Months, 5 days 3 Months, 13 days](https://reader036.fdocuments.us/reader036/viewer/2022063002/5f2320d335fa24741a7f603c/html5/thumbnails/28.jpg)
Biometric Fusion for De-duplication
![Page 29: Anil K. Jain Michigan State University Biometrics: …biometrics.cse.msu.edu/Presentations/Haifa_workshop_Feb...3 Months, 13 days 3 Months, 15 days 6 Months, 5 days 3 Months, 13 days](https://reader036.fdocuments.us/reader036/viewer/2022063002/5f2320d335fa24741a7f603c/html5/thumbnails/29.jpg)
Authentication
Over 30 million biometric authentication transactions/day
![Page 30: Anil K. Jain Michigan State University Biometrics: …biometrics.cse.msu.edu/Presentations/Haifa_workshop_Feb...3 Months, 13 days 3 Months, 15 days 6 Months, 5 days 3 Months, 13 days](https://reader036.fdocuments.us/reader036/viewer/2022063002/5f2320d335fa24741a7f603c/html5/thumbnails/30.jpg)
Fingerprint Quality Challenges
No. of false minutiae = 27No. of false minutiae = 7No. of false minutiae = 0
![Page 31: Anil K. Jain Michigan State University Biometrics: …biometrics.cse.msu.edu/Presentations/Haifa_workshop_Feb...3 Months, 13 days 3 Months, 15 days 6 Months, 5 days 3 Months, 13 days](https://reader036.fdocuments.us/reader036/viewer/2022063002/5f2320d335fa24741a7f603c/html5/thumbnails/31.jpg)
Systems operating at the edge while maintaining privacy, cost,….
2020 Vision
Accuracy
Scalability
Noise
Low
Medium
High
101
103
105
107
90%
99%
99.99%
99.999%
No Noise
109
![Page 32: Anil K. Jain Michigan State University Biometrics: …biometrics.cse.msu.edu/Presentations/Haifa_workshop_Feb...3 Months, 13 days 3 Months, 15 days 6 Months, 5 days 3 Months, 13 days](https://reader036.fdocuments.us/reader036/viewer/2022063002/5f2320d335fa24741a7f603c/html5/thumbnails/32.jpg)
Scalability
• Population of Israel = 9 m
• #Enrollments in FBI database = 100 m (70 m criminals)
• #Enrollments in Aadhaar = 1.3 bn
• Current World Population = 7.8 bn (11 bn in 2100)
• #New births/year = 130 m/year (250/minute)
• The United Nations (UN) Member States have adopted Sustainable Development Goal (SDG) Target 16.9: “to provide legal identity for all, including birth registration” by 2030.
• That would be 10 times larger than Aadhaar
32
![Page 33: Anil K. Jain Michigan State University Biometrics: …biometrics.cse.msu.edu/Presentations/Haifa_workshop_Feb...3 Months, 13 days 3 Months, 15 days 6 Months, 5 days 3 Months, 13 days](https://reader036.fdocuments.us/reader036/viewer/2022063002/5f2320d335fa24741a7f603c/html5/thumbnails/33.jpg)
How Do We Get Biometric Data?
• Data is necessary for
– Model training (learning)
– Evaluation of trained model
• Larger the data sample from population of interest, more
robust and generalizable the model
• Data privacy makes it difficult to collect and share data
• Venmo demands end to harvesting of images by Clearview
AI (which has collected 3 bn images), NYT Feb 6, 2020
33
![Page 34: Anil K. Jain Michigan State University Biometrics: …biometrics.cse.msu.edu/Presentations/Haifa_workshop_Feb...3 Months, 13 days 3 Months, 15 days 6 Months, 5 days 3 Months, 13 days](https://reader036.fdocuments.us/reader036/viewer/2022063002/5f2320d335fa24741a7f603c/html5/thumbnails/34.jpg)
Synthetic Fingerprint Generator
![Page 35: Anil K. Jain Michigan State University Biometrics: …biometrics.cse.msu.edu/Presentations/Haifa_workshop_Feb...3 Months, 13 days 3 Months, 15 days 6 Months, 5 days 3 Months, 13 days](https://reader036.fdocuments.us/reader036/viewer/2022063002/5f2320d335fa24741a7f603c/html5/thumbnails/35.jpg)
3 Months, 13 days 3 Months, 15 days 6 Months, 5 days3 Months, 13 days 3 Months, 15 days 6 Months, 5 days
6 days 8 days 2 Months, 27 days 6 days 8 days 2 Months, 27 days
1 Month, 12 days 1 Month, 14 days 4 Months, 5 days1 Month, 12 days 1 Month, 14 days
4 Months, 5 days
2 Months, 13 days 2 Months, 15 days 5 Months, 5 days 2 Months, 13 days 2 Months, 15 days 5 Months, 5 days
Longitudinal
![Page 36: Anil K. Jain Michigan State University Biometrics: …biometrics.cse.msu.edu/Presentations/Haifa_workshop_Feb...3 Months, 13 days 3 Months, 15 days 6 Months, 5 days 3 Months, 13 days](https://reader036.fdocuments.us/reader036/viewer/2022063002/5f2320d335fa24741a7f603c/html5/thumbnails/36.jpg)
In-situ EvaluationSaran Ashram Hospital, Dayalbagh, Agra, India
• TAR @ FAR = 0.1% (1.0%)
• Authenticate with 3-months gap
• Fusion of three matchers
Enrollment 0-1 Months 1-3 Months 2-3 Months
Left Thumb 59.0% (65.4%)
62.3% (69.6%)
76.5% (82.4%)
Right Thumb 55.8% (58.4%)
60.9% (63.8%)
68.4% (74.5%)
Thumbs Fused 66.7% (78.2%)
75.4% (85.1%)
90.2% (94.1%)
![Page 37: Anil K. Jain Michigan State University Biometrics: …biometrics.cse.msu.edu/Presentations/Haifa_workshop_Feb...3 Months, 13 days 3 Months, 15 days 6 Months, 5 days 3 Months, 13 days](https://reader036.fdocuments.us/reader036/viewer/2022063002/5f2320d335fa24741a7f603c/html5/thumbnails/37.jpg)
Infant ID
• Is this the infant (0-12 mos.) his parents claim him to be?
• Have we seen this infant before?Engelsma, Deb, Jain, Sudhish, Bhatnagar, "Infant-Prints: Fingerprints for Reducing Infant Mortality", CVPR Workshop on CV4GC, 2019Jain, Arora, Cao, Best-Rowden Bhatnagar, "Fingerprint Recognition of Young Children", IEEE TIFS, 2016
![Page 38: Anil K. Jain Michigan State University Biometrics: …biometrics.cse.msu.edu/Presentations/Haifa_workshop_Feb...3 Months, 13 days 3 Months, 15 days 6 Months, 5 days 3 Months, 13 days](https://reader036.fdocuments.us/reader036/viewer/2022063002/5f2320d335fa24741a7f603c/html5/thumbnails/38.jpg)
Vaccination ClinicFood distribution
Why Infant Identity?
https://bit.ly/1oIVAOf
Birth registry
Vaccination
Nutrition
Baby swapping
In many Sub-Saharan African countries rural birth registration for children under 5 is less than 50 percent
![Page 39: Anil K. Jain Michigan State University Biometrics: …biometrics.cse.msu.edu/Presentations/Haifa_workshop_Feb...3 Months, 13 days 3 Months, 15 days 6 Months, 5 days 3 Months, 13 days](https://reader036.fdocuments.us/reader036/viewer/2022063002/5f2320d335fa24741a7f603c/html5/thumbnails/39.jpg)
Fingerprint
Palmprint
Face
Footprint
IrisFingerprint
Ballprint
Palmprint
Face
Footprint
IrisFingerprint
Ballprint
Face
Footprint or Ballprint
Iris
Palmprint
Palmprint
Face
Footprint
IrisFingerprint
Ballprint
Requirements: permanence, uniqueness, ergonomic, acceptability, low cost, lifelong usability
Which Biometric for Infants?
![Page 40: Anil K. Jain Michigan State University Biometrics: …biometrics.cse.msu.edu/Presentations/Haifa_workshop_Feb...3 Months, 13 days 3 Months, 15 days 6 Months, 5 days 3 Months, 13 days](https://reader036.fdocuments.us/reader036/viewer/2022063002/5f2320d335fa24741a7f603c/html5/thumbnails/40.jpg)
Fundamental Premise of Biometrics
• Individuality: Do different individuals have
different biometric features? Does the same
person always has the same feature vector?
• Permanence: How does recognition accuracy
change with increasing time gap between
enrolled and query biometric?
![Page 41: Anil K. Jain Michigan State University Biometrics: …biometrics.cse.msu.edu/Presentations/Haifa_workshop_Feb...3 Months, 13 days 3 Months, 15 days 6 Months, 5 days 3 Months, 13 days](https://reader036.fdocuments.us/reader036/viewer/2022063002/5f2320d335fa24741a7f603c/html5/thumbnails/41.jpg)
Large Intra-Person Face Variability
![Page 42: Anil K. Jain Michigan State University Biometrics: …biometrics.cse.msu.edu/Presentations/Haifa_workshop_Feb...3 Months, 13 days 3 Months, 15 days 6 Months, 5 days 3 Months, 13 days](https://reader036.fdocuments.us/reader036/viewer/2022063002/5f2320d335fa24741a7f603c/html5/thumbnails/42.jpg)
Large Inter-Person Face Similarity
42
Ilham Anas, a photographer from Java, travels the world cashing
in on his uncanny resemblance to the former US president
![Page 43: Anil K. Jain Michigan State University Biometrics: …biometrics.cse.msu.edu/Presentations/Haifa_workshop_Feb...3 Months, 13 days 3 Months, 15 days 6 Months, 5 days 3 Months, 13 days](https://reader036.fdocuments.us/reader036/viewer/2022063002/5f2320d335fa24741a7f603c/html5/thumbnails/43.jpg)
Fingerprint Individuality
Pankanti, Prabhakar and Jain, On the individuality of fingerprints, PAMI, 2002
m = n = q = 26, PRC = 2.40 x 10-30
m = n=26, q=10, PRC = 5.49 x 10-4
• PRC = Prob. of two fingerprints
with m and n minutiae sharing
q points in common
• "Two Like Fingerprints Would
be Found Only Once Every
1048 Years” (Scientific American,
1911)
![Page 44: Anil K. Jain Michigan State University Biometrics: …biometrics.cse.msu.edu/Presentations/Haifa_workshop_Feb...3 Months, 13 days 3 Months, 15 days 6 Months, 5 days 3 Months, 13 days](https://reader036.fdocuments.us/reader036/viewer/2022063002/5f2320d335fa24741a7f603c/html5/thumbnails/44.jpg)
Fingerprint Persistence
Yoon and Jain, "Longitudinal Study of Fingerprint Recognition", PNAS, 2015
• Fingerprint records of 16K subjects over 12 years
• Longitudinal model showed: (i) Accuracy is stable over time;
(ii) Accuracy depends on the fingerprint image quality
![Page 45: Anil K. Jain Michigan State University Biometrics: …biometrics.cse.msu.edu/Presentations/Haifa_workshop_Feb...3 Months, 13 days 3 Months, 15 days 6 Months, 5 days 3 Months, 13 days](https://reader036.fdocuments.us/reader036/viewer/2022063002/5f2320d335fa24741a7f603c/html5/thumbnails/45.jpg)
Capacity of Face Recognition
Gong, Boddeti, Jain, "On the Intrinsic Dimensionality of Face Representation", arXiv:1803.09672, 2018
• #Identities resolved at a specified
FAR and feature space (FAR of
0.01% and SpehereFace model)
IJB-A, capacity = 1.1 x 10^4
IJB-C, capacity = 2.0 x 10^2
![Page 46: Anil K. Jain Michigan State University Biometrics: …biometrics.cse.msu.edu/Presentations/Haifa_workshop_Feb...3 Months, 13 days 3 Months, 15 days 6 Months, 5 days 3 Months, 13 days](https://reader036.fdocuments.us/reader036/viewer/2022063002/5f2320d335fa24741a7f603c/html5/thumbnails/46.jpg)
Persistence of Face
Best-Rowden and Jain, "Longitudinal Study of Automatic Face Recognition", PAMI, 2017
• Longitudinal face data of 20K subjects
• Findings: 99% of the subjects could be recognized @ 0.01%
FAR up to 6 years irrespective of age, gender & race
![Page 47: Anil K. Jain Michigan State University Biometrics: …biometrics.cse.msu.edu/Presentations/Haifa_workshop_Feb...3 Months, 13 days 3 Months, 15 days 6 Months, 5 days 3 Months, 13 days](https://reader036.fdocuments.us/reader036/viewer/2022063002/5f2320d335fa24741a7f603c/html5/thumbnails/47.jpg)
Soft BiometricsProvide some discriminatory information; can be
used alone or in conjunction with primary traits
Ethnicity, Gender, Skin & Hair color(Sub-Saharan African, Indian, Southern European,
and Northwest European)
Height
Scars, Marks, and Tattoos (SMT)
Eye color
WeightPeriocular
![Page 48: Anil K. Jain Michigan State University Biometrics: …biometrics.cse.msu.edu/Presentations/Haifa_workshop_Feb...3 Months, 13 days 3 Months, 15 days 6 Months, 5 days 3 Months, 13 days](https://reader036.fdocuments.us/reader036/viewer/2022063002/5f2320d335fa24741a7f603c/html5/thumbnails/48.jpg)
A Tattoo Reveals True Identity
The suspect gave his name as “Darnell Lewis”, but his real name, “Frazier” was tattoed on his neck! He was arrested on four misdemeanor warrants (Dec. 2008, St. Paul)
![Page 49: Anil K. Jain Michigan State University Biometrics: …biometrics.cse.msu.edu/Presentations/Haifa_workshop_Feb...3 Months, 13 days 3 Months, 15 days 6 Months, 5 days 3 Months, 13 days](https://reader036.fdocuments.us/reader036/viewer/2022063002/5f2320d335fa24741a7f603c/html5/thumbnails/49.jpg)
Tattoo Caught in Surveillance Camera
Detroit police linked at least six armed robberies at an ATMon the city’s west side after matching up a tipster’sdescription of the suspect’s distinctive tattoos
www.DetroitisScrap.com/2009/09/567/
![Page 50: Anil K. Jain Michigan State University Biometrics: …biometrics.cse.msu.edu/Presentations/Haifa_workshop_Feb...3 Months, 13 days 3 Months, 15 days 6 Months, 5 days 3 Months, 13 days](https://reader036.fdocuments.us/reader036/viewer/2022063002/5f2320d335fa24741a7f603c/html5/thumbnails/50.jpg)
50
Security of Biometric Systems
SensorFeature
Extractor MatcherApplication Device
(e.g.,cash dispenser)
StoredTemplates
1. FakeBiometric
2. ReplayOld Data
3. OverrideFeature Extractor
Yes/No
8. Override Final Decision
5. OverrideMatcher
4. SynthesizedFeature Vector
7. Interceptthe Channel
6. ModifyTemplate
Ratha, Connell, Bolle, “Enhancing security and privacy in biometrics-based authentication systems”, IBM Systems Journal, 2001
![Page 51: Anil K. Jain Michigan State University Biometrics: …biometrics.cse.msu.edu/Presentations/Haifa_workshop_Feb...3 Months, 13 days 3 Months, 15 days 6 Months, 5 days 3 Months, 13 days](https://reader036.fdocuments.us/reader036/viewer/2022063002/5f2320d335fa24741a7f603c/html5/thumbnails/51.jpg)
Biometric Spoofs
Face AttacksPrinted, Replay, 3D Mask
Fingerprint AttacksDifferent materials and fabrication techniques
Iris AttacksVan Dyke, Pattern Contact Lenses, Glass eyes
![Page 52: Anil K. Jain Michigan State University Biometrics: …biometrics.cse.msu.edu/Presentations/Haifa_workshop_Feb...3 Months, 13 days 3 Months, 15 days 6 Months, 5 days 3 Months, 13 days](https://reader036.fdocuments.us/reader036/viewer/2022063002/5f2320d335fa24741a7f603c/html5/thumbnails/52.jpg)
D. Deb, J. Zhang, and A. K. Jain, "AdvFaces: Adversarial Face Synthesis", arXiv:1908.05008, 2019.
Digitally Altered Faces
![Page 53: Anil K. Jain Michigan State University Biometrics: …biometrics.cse.msu.edu/Presentations/Haifa_workshop_Feb...3 Months, 13 days 3 Months, 15 days 6 Months, 5 days 3 Months, 13 days](https://reader036.fdocuments.us/reader036/viewer/2022063002/5f2320d335fa24741a7f603c/html5/thumbnails/53.jpg)
![Page 54: Anil K. Jain Michigan State University Biometrics: …biometrics.cse.msu.edu/Presentations/Haifa_workshop_Feb...3 Months, 13 days 3 Months, 15 days 6 Months, 5 days 3 Months, 13 days](https://reader036.fdocuments.us/reader036/viewer/2022063002/5f2320d335fa24741a7f603c/html5/thumbnails/54.jpg)
Template Protection
Minutiae-based ISO/IEC 19794-2:2005 standard template
Template Reconstructed Image Spoof
![Page 55: Anil K. Jain Michigan State University Biometrics: …biometrics.cse.msu.edu/Presentations/Haifa_workshop_Feb...3 Months, 13 days 3 Months, 15 days 6 Months, 5 days 3 Months, 13 days](https://reader036.fdocuments.us/reader036/viewer/2022063002/5f2320d335fa24741a7f603c/html5/thumbnails/55.jpg)
GOODIX IN-DISPLAY FINGERPRINT SENSORTM
ARM TrustZone®
Serial Peripheral Interface (SPI)
Encrypt Raw Data
To secure fingerprint,
processing is done in TEE
(Trusted Execution
Environment):
• Image pre-processing
• Feature extraction
• Alignment and recognition
Match on Device
![Page 56: Anil K. Jain Michigan State University Biometrics: …biometrics.cse.msu.edu/Presentations/Haifa_workshop_Feb...3 Months, 13 days 3 Months, 15 days 6 Months, 5 days 3 Months, 13 days](https://reader036.fdocuments.us/reader036/viewer/2022063002/5f2320d335fa24741a7f603c/html5/thumbnails/56.jpg)
Matching in the Encrypted Domain
Probe Fingerprint Aligned Fingerprint
Alignment
Network
Stem
Network
Texture
Network
Minutiae
Network
Fixed-Length
Representation
(192-dim)
EncryptedMatching
Encrypt
DecryptScores
score = 0.96
Encrypted
Database
Encrypted Templates have been enrolled into the database offline
![Page 57: Anil K. Jain Michigan State University Biometrics: …biometrics.cse.msu.edu/Presentations/Haifa_workshop_Feb...3 Months, 13 days 3 Months, 15 days 6 Months, 5 days 3 Months, 13 days](https://reader036.fdocuments.us/reader036/viewer/2022063002/5f2320d335fa24741a7f603c/html5/thumbnails/57.jpg)
Biometrics: Privacy Concerns
User consent, data security, retention policy, linking to other databases
![Page 58: Anil K. Jain Michigan State University Biometrics: …biometrics.cse.msu.edu/Presentations/Haifa_workshop_Feb...3 Months, 13 days 3 Months, 15 days 6 Months, 5 days 3 Months, 13 days](https://reader036.fdocuments.us/reader036/viewer/2022063002/5f2320d335fa24741a7f603c/html5/thumbnails/58.jpg)
Chongqing: World’s Most Heavily Surveilled
2.58m cameras covering 15.35 million people – equal to one camera for every six residentshttps://www.theguardian.com/cities/2019/dec/02/big-brother-is-watching-chinese-city-with-26m-cameras-is-worlds-most-heavily-surveilled
![Page 59: Anil K. Jain Michigan State University Biometrics: …biometrics.cse.msu.edu/Presentations/Haifa_workshop_Feb...3 Months, 13 days 3 Months, 15 days 6 Months, 5 days 3 Months, 13 days](https://reader036.fdocuments.us/reader036/viewer/2022063002/5f2320d335fa24741a7f603c/html5/thumbnails/59.jpg)
Facial Recognition Gains Entry Into Schools
59
A camera, connected to the Aegis system (Clearview AI), in the ceiling of a hallway at the Lockport Board of Education building. Libby March, NYT, Feb 6, 2020
• PRO: A crime-fighting tool, to help prevent mass shootings and stop sexual predators.
• CON: False matches “can lead to very dangerous and completely avoidable situations,” said Jayde McDonald, a former Lockport student.
![Page 60: Anil K. Jain Michigan State University Biometrics: …biometrics.cse.msu.edu/Presentations/Haifa_workshop_Feb...3 Months, 13 days 3 Months, 15 days 6 Months, 5 days 3 Months, 13 days](https://reader036.fdocuments.us/reader036/viewer/2022063002/5f2320d335fa24741a7f603c/html5/thumbnails/60.jpg)
Security v. Privacy
• Citizens right to protest and independent judiciary
• What recourse do you have if you are incorrectly recognized?
![Page 61: Anil K. Jain Michigan State University Biometrics: …biometrics.cse.msu.edu/Presentations/Haifa_workshop_Feb...3 Months, 13 days 3 Months, 15 days 6 Months, 5 days 3 Months, 13 days](https://reader036.fdocuments.us/reader036/viewer/2022063002/5f2320d335fa24741a7f603c/html5/thumbnails/61.jpg)
Social Good v. Privacy
“Aadhaar gives dignity to the marginalized. Dignity to the marginalizedoutweighs privacy” - Justice Sikri, Indian Supreme Court (Sept 2018)
![Page 62: Anil K. Jain Michigan State University Biometrics: …biometrics.cse.msu.edu/Presentations/Haifa_workshop_Feb...3 Months, 13 days 3 Months, 15 days 6 Months, 5 days 3 Months, 13 days](https://reader036.fdocuments.us/reader036/viewer/2022063002/5f2320d335fa24741a7f603c/html5/thumbnails/62.jpg)
Kenya’s High Court Delays Biometrics ID Program
A Kenyan being photographed for a national ID, NY Times, Jan 29, 2020)
….until the government enacts laws to protect the security of the data and prevent discrimination against minorities (marginalization)
![Page 63: Anil K. Jain Michigan State University Biometrics: …biometrics.cse.msu.edu/Presentations/Haifa_workshop_Feb...3 Months, 13 days 3 Months, 15 days 6 Months, 5 days 3 Months, 13 days](https://reader036.fdocuments.us/reader036/viewer/2022063002/5f2320d335fa24741a7f603c/html5/thumbnails/63.jpg)
Summary• Reliable and automated person identification is becoming
a necessity; No substitute to biometrics for effectiveperson identification
• It can enhance security, eliminate fraud and offerconvenience to the users
• Biometric sensors are inexpensive-fingerprint, face andvoice sensors are embedded in laptops & mobile phones;system performance is not meeting the expectations
• Deployed systems should not infringe on civil liberties,……