Human Recognition Using Biometrics - IEEE · The Biometrics Revolution Over 1 billion people have...

Human Recognition Using Biometrics:

Past, Present, Future

Arun RossProfessor

Michigan State [email protected]

http://www.cse.msu.edu/~rossarun

Presented by Arun Ross, 2016

§ Whoisthisperson?

Who is This?


§ Arethesetwoprintsfromthesamefinger?

Are These The Same?


§ AreanyoftheBostonBombersinthisscene?

Is He There?


§ IsthisreallyaphotographofAbrahamLincoln?

Is This Really Him?

LINCOLN


§ Ishetheownerofthissmartphone?

Is He Allowed Access?


§ IsthisreallyElvisPresley’svoice?(Andifso,ishestillalive?!)

Who is Singing?

https://www.youtube.com/watch?v=HGsssVWiu54


§ Doesthispersonalreadyhaveadriver’slicenseunderadifferentname?

Is She In The Database?


§ Find all video frames in which Odette appears

Where is She?

© Nest Entertainment


Where’s Waldo?


§ Automatedrecognitionofindividualsbasedontheirbiological andbehavioralcharacteristics

§ Biologicalandbehavioralcharacteristicofanindividualfromwhichdistinguishing,repeatablebiometricfeaturescanbeextracted

Biometric Recognition

H.T.F.Rhodes,AlphonseBertillon:FatherofScientificDetection,Harrap,1956


First Biometric System


Biometric Traits


Biometric Applications

Fingerprint: US OBIM

Iris: Frankfurt Airport

Finger Vein: Japan ATMs

http://ww

w.ubergizm

o.com/

Fingerprint: Privaris Key Fob

Fingerprint: Apple Touch ID


The Biometrics Revolution

Over 1 billion people have been covered by biometric identification programs in the Low Middle Income Countries

*Identification for Development: The Biometrics Revolution, A. Gelb and J. Clark, Center for Global Development, NW, Washington DC, Working Paper 315, Jan. 2013, http://www.cgdev.org/sites/default/files/1426862_file_Biometric_ID_for_Development.pdf


• “Rich countries have long used biometrics for forensics and security but fewer have incorporated them into their national identity systems or used them to underpin public service delivery.”

• “In contrast, we have seen a proliferation of non-security applications in low- and middle-income countries, from civil registries to voter rolls, health records to social transfers, public payrolls to pension payments and beyond.”

• “This divergence in purpose partly reflects the different identification baselines in rich and poor countries—the identity gap.”

Application Domain for Biometrics

Alan Gelb and Julia Clark. 2013. “Identification for Development: The Biometrics Revolution.” CGD Working Paper 315. http://www.cgdev.org/content/publications/detail/1426862


Biometrics for Refugees

https://www.youtube.com/watch?v=44nLWR4V-lc


Mobile Phone Market


Smartphone Authentication


Smartphone Payment Systems


Smartphone Sensors


Obtrusive versus Non-obtrusive


Identification Without Biometric Data!DeMontjoye,Hidalgo,Verleysen &Blondel,“UniqueintheCrowd:ThePrivacyBoundsofHumanMobility”,ScientificReports,vol.3,2013

Withjustanonymouslocationdata,itispossibletofigureout“whoyouare”bytrackingyoursmartphone• 15monthsofmobilitydatafor1.5millionindividualsandfoundthathuman

mobilitytracesarehighlyunique.• 4spatio-temporalpointsareenoughtouniquelyidentify95%ofthe

individuals


§ We do not necessarily want to elicit identity§ We want to recognize a person

Identity vs Recognition

Based on a single fingerprint image, we cannot say this belongs to Jane Doe

We need a referencefingerprint image that is known to belong to Jane Doe in order to make this assessment

Jane Doe

???

REFERENCE

INPUT


§ Age, Gender, Ethnicity, can be automatically derived from the face image

§ That is, a trained classifier or a regressor may be used to automatically deduce certain soft biometric attributes

Information from a Single Image

• Gender:Male• Age:25• Health:Verygood• EyeSight:Wearsglasses• Ethnicity:AsianIndian• Name:Rohin

Also see, Dantcheva, Elias, Ross, “"What Else Does Your Biometric Data Reveal? A Survey on Soft Biometrics,” TIFS 2016


• Viewingtheirisasatexturalentityratherthanjustabinary code

What else is revealed in an iris image?


• Biographical:Age,Gender,Race

• Anatomical:Distributionofcrypts,Wolfflinnodules,pigmentationspots

• Environmental:Sensor,Illuminationwavelength,Indoor/Outdoor

• Pathological:StromalAtrophy

• Other:Pupildilationlevel,ContactLens

Iris: Levels of Information

Notallinformationcanbereliablyextracted

Butinformationcanbeaggregated


Semantic Description of Iris

• SubjectisaMale (90%),White(85%)

• ImagetakenusinganAoptixcamera

• Irisstromaisplaintextured• Highlyconstrictedpupilsuggestsstrong ambientillumination


Information from a Single Image


§ Estimate the probability that two biometric samples are of the same person

Biometric Matching


§ Compute the similarity between two instances of biometric data corrupted by noise

Real-world Matching


§ Sensor: To acquire biometric data

§ Feature extractor: To extract a set of discriminative features from the data

§ Matcher: To compare two extracted feature sets

§ Database: To store biometric templates of individuals

Components of a Biometric System


§ Ensuring that the input data is uncorrupted and from a real person

§ Protecting the biometric templates in the database

§ Ensuring the privacy of an individual

Beyond Pattern Recognition


Intra-user variations

Rn

§ FNMR: False Non-Match Rate


Inter-user similarity

§ FMR: False Match Rate

TWIN BROTHERS© Martin Schoeller

MOTHER DAUGHTER© PleasantonWeekly.Com


Changes Due to Illumination

nachoguzman.net


Biometric Ageing


Heterogeneous Face RecognitionPhoto vs Sketch

RGB vs NIR vs THM

Before vs After Makeup

Young vs Old 2D vs 3D

FundamentalDifferencesin

ImageFormationCharacteristics


Spoofing: Presentation Attack

Imagesfromhttps://www.idiap.ch/dataset/3dmad

§ Spoofing: Altering one’s trait or creating a physicalartifact in order to “spoof” another person’s trait


§ Obfuscation: Masking one’s own identity by altering the trait

Obfuscation: Presentation Attack

Dantcheva et al, “Can Facial Cosmetics Affect the Matching Accuracy of Face Recognition Systems?”, BTAS 2012

BEFORE MAKEUP AFTER MAKEUP


§ 1995: Alexander Guzman was arrested by Florida officials for possessing a false passport

§ He was found to have mutilated fingerprints

§ After a two-week search based on manually reconstructing the damaged fingerprints and searching the FBI database, the reconstructed fingerprints were linked to the fingerprints of Jose Izquiredo who was an absconding drug criminal

Fingerprint Alteration


§ His fingerprint mutilation process consisted of three steps: making a ‘Z’ shaped cut on the fingertip; lifting and switching two triangles; and stitching them back.

The “Z”-cut


Face Recognition Progress


§ “Privacy is the right to be let alone” [Samuel Warren and Louis Brandeis (1890)]

§ “Privacy is the claim of individuals, groups, or institutions to determine for themselves when, how, and to what extent information about them is communicated to others” [Alan Westin (1970)]

§ “Privacy is the right of people to conceal information about themselves that others might use to their disadvantage” [Richard Posner (1983)]

Importance of Privacy

PRIVACY IS DIFFERENT FROM SECURITY


§ Some biometric systems may store the raw images of an individual as a reference image

§ e.g., face or fingerprint or iris image

§ From a visual standpoint, face images are perceived to divulge more information about a person

Reference Biometric Images


§ Biometric data of an individual is sometimes stored in a central database with an identifier

§ Cross-database matching may be done to track individuals

§ Biometric data mining may be performed to glean information about identity

§ Large-scale processing of biometric data

Linking Across Databases


§ Faces of Facebook: Privacy in the Age of Augmented Reality (Alessandro Acquisti et al 2011)

§ Convergence of three technologies:

§ face recognition, cloud computing, online social networks

§ They investigated whether combination of publicly available Web 2.0 data and off-the-shelf face recognition software may allow large-scale, automated, end-user individual re-identification

§ Started from an anonymous face in the street, and ended up with very sensitive information about that person, in a process of data "accretion”

§ Combined face recognition with the algorithms they developed in 2009 to predict SSNs from public data

Identifying People on the Web


Privacy Visor

https://www.youtube.com/watch?v=LRj8whKmN1M


Anti-Face!

https://cvdazzle.com/


De-identification via Collaboration


§ The input face image is decomposed and stored in two separate servers: either server will be unable to deduce original face image by themselves

Decomposing Face Images

SERVER 1 SERVER 2

A. Ross and A. Othman, "Visual Cryptography for Biometric Privacy," TIFS 2011


§ Given an original binary image T, it is encrypted in n images, such that:

where ⊕ is a Boolean operation , Shi is an image which appears as noise, k ≤ n, and n is the number of noisy images

§ This is referred to as k-out-of-n VCS

Visual Cryptography*

* M. Naor and A. Shamir, “Visual cryptography,” in EUROCRYPT, pp. 1–12, 1994.


§ Decomposing a fingerprint into two random images using Visual Cryptography

Decomposing a Binary Image


HOSTS (PUBLIC IMAGES)PRIVATE IMAGE

HOSTS AFTER ENCRYPTIONPRIVATE IMAGE AFTER DECRYPTION

Gray-level Extended Visual Cryptography Scheme (GEVCS)

Ross and Othman, “Visual Cryptography for Biometrics Privacy”, TIFS 2011


Face Visual Cryptography

Actual Face HOST IMAGE IN SERVER 1

HOST IMAGE IN SERVER 2


Simple XOR operator


§ Method to protect privacy of face images by decomposing it into two independent host (public) face images

§ Original face image can be reconstructed only when both host images are available

§ Each host image does not expose the identity of the original face image

Face De-identification: Results



De-identification via Mixing


§ An input fingerprint image is mixed with another fingerprint (e.g., from a different finger)

§ produces a new mixed fingerprint image that obscures the identity of the original fingerprint

§ We consider the problem of mixing two fingerprint images in order to generate a new cancelable fingerprint image

Mixing Fingerprints

Othman and Ross, “On Mixing Fingerprints”, TIFS 2013


§ Mixing fingerprints creates a new entity that looks like a plausible fingerprint:

§ It can be processed by conventional fingerprint algorithms

§ An eavesdropper may not be able to determine if a given fingerprint is mixed or not

Mixing FingerprintsSecret Transformation

FunctionMixed

Fingerprint


Decomposition: Whorl

Original Spiral Phase Continuous Phase



Mixing Fingerprints

§ Let F1 and F2 be two different fingerprint images from different fingers, and let Ψci(x, y) and Ψsi(x, y) be the pre-aligned continuous and spiral phases, i = 1,2.

MF1 = cos[Ψc2(x, y)+ Ψs1(x, y)]

MF2 = cos[Ψc1(x, y)+ Ψs2(x, y)]

§ The continuous phase of F2 is combined with the spiral phase of F1 which generates a new fused fingerprint image MF1



Mixed Fingerprint Images



§ Can the mixed fingerprint be used as a new biometric identity? (Yes)

§ Are the original fingerprint and the mixed fingerprint correlated? (No)

§ Does mixing result in cancelable templates? (Yes)

§ If two different fingerprints are mixed with a common fingerprint, are the mixed fingerprints similar? (No)

Mixing Fingerprints: Results



“Differential” Privacy


Soft Biometric Privacy

Othman and Ross, “Privacy of Facial Soft Biometrics,” ECCVW 2014


§ Gender attribute of an input face image is progressivelysuppressed

§ With respect to a face matcher the identity is preserved

Soft Biometric Privacy

Name Alice Alice Alice Alice

Gender Female(confident)

Female(less confident)

Male(less confident)

Male(confident)

Othman and Ross, “Privacy of Facial Soft Biometrics: Suppressing Gender But Retaining Identity”, ECCV Workshop, 2014


Face Morphing§ To generate a mixed face image, the principle of

face morphing is used

§ The mixed face image can be anywhere along a continuum from F1 to F2

F1

F2

MF



Similarity to the original images§ The resultant rank-1 accuracy is 95% and the EER is 5%

The identities of the originals have been preserved in the mixed faces



Gender Perturbation


ORIGINAL IMAGES

MODIFIED IMAGES


§ We explored the possibility of generating mixed face images that perturb the gender of a face image to different degrees

§ Experiments on MUCT demonstrate that:

- The new mixed face can potentially suppress the gender of an input face to different degrees (gender classifier)

- The new mixed face image exhibits similarity with the original (face matcher)

Differential Privacy: Results


n How can the biometric trait of an individual be effectively modeled using biologically tenable models?

n How can the uniqueness of a biometric trait, as it pertains to an individual, be deduced based on such models?

n What is the impact of age and disease on the stability and permanence of biometric characteristics?

Biometric Science Questions


n What types of signal enhancement and matching models are necessary to conduct biometric recognition using severely degraded biometric data?

n How can biometric templates be stored and transmitted securely?

n What types of statistical and mathematical models are required to predict matching performance of large-scale biometric systems?

n How can large biometric databases be efficiently searched in order to rapidly locate an identity of interest?

Engineering Questions


n What constitutes the identity of an individual?

n What are the societal implications of machines identifying humans?

n What are the moral and ethical implications of a biometric system misidentifying an individual in high-risk environments such as a combat zone?

Philosophical Musings


Problem Solving

n We can't solve problems by using the same kind of thinking we used when we created them


Relevant Papers

§ A. K. Jain and A. Ross, "Bridging the Gap: From Biometrics to Forensics," Philosophical Transactions of The Royal Society B, Vol. 370, Issue 1674, August 2015

§ A. K. Jain, K. Nandakumar, A. Ross, "50 Years of Biometric Research: Accomplishments, Challenges, and Opportunities," Pattern Recognition Letters, Vol. 79, pp. 80 -105, August 2016

§ A. Dantcheva, P. Elia, A. Ross, "What Else Does Your Biometric Data Reveal? A Survey on Soft Biometrics," IEEE Transactions on Information Forensics And Security (TIFS), Vol. 11, No. 3, pp. 441 - 467, March 2016


[Funded by NSF CAREER Award]

§ A. Ross and A. Othman, "Visual Cryptography for Biometric Privacy," IEEE Transactions on Information Forensics and Security (TIFS), Vol. 6, Issue 1, pp. 70 - 81, March 2011

§ A. Othman and A. Ross, "On Mixing Fingerprints," IEEE Transactions on Information Forensics and Security, Vol. 8, Issue 1, pp. 260 - 267, January 2013

§ A. Ross and A. Othman, "Mixing Fingerprints for Template Security and Privacy," Proc. of the 19th European Signal Processing Conference (EUSIPCO), (Barcelona, Spain), August/September 2011

§ A. Othman and A. Ross, "Privacy of Facial Soft Biometrics: Suppressing Gender But Retaining Identity," Proc. of ECCV Workshop on Soft Biometrics, (Zurich, Switzerland), September 2014

Biometrics Privacy


The i-PRoBe Lab

§ Currently: 8PhDStudents+2 MSStudents

§ Graduated:24MSStudents+5PhDStudents

http://www.cse.msu.edu/~rossarun/i-probe/


Current Research• Biometric Fusion:

• Biometrics + Demographics, Score + Quality + Liveness

• Fingerprints:• Anti-Spoofing, De-identification, Soft Biometrics

• Ocular Biometrics:• Cross-spectral Iris, Pupil Dilation, Iris Forensics

• Face:• Cross-spectral Face Recognition, Privacy

Human Recognition Using Biometrics:

Past, Present, Future

Arun RossProfessor

Michigan State [email protected]

http://www.cse.msu.edu/~rossarun

Human Recognition Using Biometrics - IEEE · The Biometrics Revolution Over 1 billion people have...

Documents

Transcript of Human Recognition Using Biometrics - IEEE · The Biometrics Revolution Over 1 billion people have...