Multimodal User Authentication: From Theory to Practice 1 TUTORIAL Conference IEEE ICME 2003 ...

Multimodal User Authentication: From Theory to PracticeMultimodal User Authentication: From Theory to Practice

1

Multimodal User Authentication:Multimodal User Authentication:From Theory to PracticeFrom Theory to Practice

TUTORIAL

ConferenceIEEE ICME 2003

SpeakersJean-Luc DUGELAY

Jean-Claude JUNQUA

Location Baltimore

Date & TimeSunday, 6 July 2003, 13:30 - 17:00


2

Who we are…Who we are…

Jean-Luc DUGELAY Ph.D. 92 Professor at Eurécom Sophia Antipolis, France Security Imaging

Watermarking Biometrics

Jean-Claude JUNQUA Ph.D. 89 Director of PSTL (Panasonic Speech

Technology Laboratory) Santa Barbara, California, U.S.A. Speech

Recognition Synthesis Multimodal Dialogue Speaker Verification


3

Outline (1/3)Outline (1/3)

Multimodal user authentication: Background Introduction Is there a universal biometric identifier? What are the factors influencing the reliability of biometric systems? Why are there still very few biometric systems in use today? Physiological versus behavioral biometrics Why multimodal biometrics? Can multimodal biometrics improve

performance? Tradeoffs between robustness (security) and convenience


4


Main individual modalities Signature Voice Hand Fingerprint Face

Frontal Specific acquisition (infra red, profile, dynamic, range) Specific parts (eyes: iris & retina, ears, teeth)

[Specificities, Pros & cons, Open problems, Sensing technologies, Major algorithms, Database examples, …]

Towards Multimodal Biometric Systems Sequence Fusion


5


Applications, Standards and Evaluation Main application areas Biometrics and privacy Important criteria to deploy multimodal authentication systems Biometric standards Multimodal databases Best practices in testing biometric systems Examples of multimodal user authentication systems Perspectives and future challenges Demonstrations Forthcoming events Bibliography


6


Multimodal user authentication: Background Introduction Is there a universal biometric identifier? What are the factors influencing the reliability of biometric systems? Why are there still very few biometric systems in use today? Physiological versus behavioral biometrics Why multimodal biometrics? Can multimodal biometrics improve

performance? Tradeoffs between robustness (security) and convenience


7

IntroductionIntroduction Drawbacks of traditional identification(knowledge- or token- based):

PIN may be forgotten or guessed by imposter Physical keys may be misplaced or lost It is not possible to differentiate between an authorized person and an imposter

Biometric system Pattern recognition system which establishes the authenticity of a specific

physiological or behavioral user’s characteristic Relies on “who you are or what you do” to make a positive personal identification Comprises an enrollment stage and an identification/verification stage Identification (1:N matching, who am I?) & Verification (1:1 matching, Am I who I

claim I am?)

→ Well-known example: login + passwd


8

Seven types of authentication:Something you know (1)

e.g. PIN code, mother’s maiden name, birthday

Something you have (2)

e.g. Card, key

Something you know + something you have (3)

e.g. ATM card + PIN

Something you are – Biometrics (4)

no PIN to remember, no PIN to forget

Something you have + something you are (5)

Smart Card

Something you know + something you have (6)

Something you know + something you have + something you are (7)

Types

Securitylevel

1, 23

4

5,6

7


9

SensorSensor

Sensor (e.g. Microphone,

Camera)

Sensor (e.g. Microphone,

Camera)

Feature Extraction

Feature Extraction

Feature Extraction

Feature Extraction

Identification/Verification


Enrollment andTemplate Storage

Enrollment andTemplate Storage

ActionAction

TemplateAdaptation

TemplateAdaptation

Enrollment


A Generic Biometric SystemA Generic Biometric System


10

How to measure performance Biometric systems are not perfect. They make errors in identifying or true claimants and in rejecting imposters The probability of committing these two types of errors are called

False Rejection Rate (FRR)False Acceptance Rate (FAR)

ROC: Receiver Operating Characteristic

FRR is user-dependent


11

Performance & EvaluationPerformance & Evaluation


12

Performance & EvaluationPerformance & Evaluation

100%

10%

1%

0.1%0.0001% 0.001% 0.01% 0.1% 1% 10% 100%

FAR

FRR

Better performance

Detection error trade-off (DET) curves (uniform treatment of both types of error)

Martin et al., « The DET curve in assessment of detection task performance »

Proc. EuroSpeech 1997.


13

State-of-the-art error rates State-of-the-art error rates Test data FRR FAR

Fingerprint 20 years

(average age)

0.2% 0.2%

Face 11 to 13 months spaced

10-20% 0.1-20%

Text-dependent Speaker verification

Text-dependent (entrance door, 3 months period )

1-3% 1-3%

Text-independent speaker verification

Text-Independent (NIST 2000)

10-20% 2-5%

Table adapted from http://akhisar.sdsu.edu/abut/BC2002talk_jain.pdf


14

Some other performance criteriaSome other performance criteria

Performance

Failure and difficulties* toenroll (e.g. amount of data)

Failure and difficultiesto acquire

False rejection rate False acceptance rate

* 4% of fingerprints are of poor quality


15

Is there a universal biometric identifier? Is there a universal biometric identifier? There are many biometric identifiers:

Fingerprint

Voice

Image

Hand geometry

Retina

Iris

Signature

Keystroke dynamics

Gait

DNA (requires physical sample)

Wrist/hand veins

Brain activity

etc.Ideally, a biometric identifier should be universal, unique, permanent and measurableHowever, in practice each biometric identifier depends on factors such as users’ attitudes,Personality, operational environment, etc.

In theory many of these biometric identifiersshould be universal. However, in practicethis is not the case.


16

Characteristics Fingerprints Hand Geometry

Retina Iris Face Signature Voice

Ease to use High High Low Medium Medium High High

Error incidence Dryness, dirt, age

Hand injury, age

Glasses Poor Lighting Lighting, age, glasses, hair

Changing signatures (inconsistencies)

Noise, Colds, weather

Accuracy High High Very high Very high High High High

Cost Medium High Medium High Medium Medium Low

User acceptance Medium Medium Medium Medium Medium Medium High

Required security level

High Medium High Very high Medium Medium Medium

Long-term stability

High Medium High High Medium Medium Medium

Template size * (bytes)

200+ 9 96 512 84 (1:n) 1300 (1:1) 3.5 k

500+

History of automatic ID **

(1880) 1963/1974

1972 (1935) 1976

1994 (1888) 1972/1987

(1929) 1983

1964

…

Each biometric identifier has its strengths and weaknesses

Adapted from Source: http://www.computer.org/itpro/homepage/Jan_Feb/Security3.html

* Also, www.biometricsmi.com (Vol. 1, Issue 01) & ** www.engr.sjsu.deu/biometrics (Dr. J. Wayman)


17

What are the factors influencing the What are the factors influencing the reliability of biometric systemsreliability of biometric systems

The factors influencing the reliability of biometric systems depend on the biometric identifier used. Understanding therequirements, the users and the environment is the key

However, some general factors can be identified User behavior/cooperativeness Stability (time and environment) of the biometric identifier How easy is it to use the system? Is the user accustomed to the use of the biometric sensor? Quality of the enrollment Population demographics User interface


18

Influence of the user and the Influence of the user and the environmentenvironment

The user Behavior Consistency Physiology Appearance Familiarity with the equipment

The environment Lighting Background noise Weather (e.g. humidity)


19

Influence of timeInfluence of time

As the time between enrollment and testing increases, the biometric features enrolled are generally becoming less reliable

e.g. - 50% decrease in performance after a period of 1 year for a biometric system based on faces.

However, as the user keeps using the biometric system he/she tends to adapt to the biometric system

Supervised or unsupervised adaptation helps dealing with the mismatch between enrollment and testing


20

Why are there still very few biometric Why are there still very few biometric systems in use today? systems in use today?

Main reasons Accuracy User acceptance & social factors (e.g. lack of familiarity, privacy) Standards Every biometric has its limitations Cost of deployment Ease of use Ease of development (e.g. standards) Lack of understanding on how to combine biometric identifiers Difficulties to enroll a large set of individuals Lack of large scale deployments


21

Why are there still very few biometric Why are there still very few biometric systems in use today? systems in use today?

Social factors

Informational privacy (collection, storage and use of the user information)

Personal privacy (how invasive or intrusive is the biometric identifier used?)

Political will/cultural climate

User acceptance/familiarity of the technology


22

• Physiological: what we AreAre or HaveHave

• Behavioral: what we DoDo

Thus Physiological is static, Behavioral is dynamic

Combinations provide potential for robustness

Physiological versus behavioral Physiological versus behavioral biometricsbiometrics

From: The Role of Dynamics in Visual Speech Biometrics

Presentation by John Mason and Jason Brand, ICASSP 2002


23

Physiological: Behavioral: what we Are/HaveAre/Have what we DoDo

Eye scans are

Fingerprints are

Face are

Handwriting do

Gait do

Speech (audio)Speech (audio) do

Speech (visual)Speech (visual) are do

Examples of physiological and Examples of physiological and behavioral biometricsbehavioral biometrics




24

Audio Behavioral

Instantaneous ‘snap-shots’ inherently Behavioral

(information signal is a function of time)

Behavioral & PhysiologicalBehavioral & Physiological Biometrics Biometrics

Visual Physiological

Instantaneous ‘snap-shots’ inherently Physiological

(information signal is not a function of time, though with speech it can become so)




25

Dynamics in BiometricsDynamics in Biometrics

Behavioral: what we

DoDo

Essential

Inherent/ unavoidable(nuisance)

Multi-session / adaptive to capture inherent variation

Physiological: what we

AreAre or HaveHave

Possible / maybe detrimental

Slow/small/nil

One-off possible

Implications

Movement

Signature Variation

(undesirable)

System Enrollment/

Training

Of course, ‘signatures’ must always relate to physical properties, some less than others, e.g. gait – highly so, speech or handwriting – perhaps less so




26

Speech as a BiometricSpeech as a Biometric

Behavioral:

DoDo (Dynamics)

Physiological:

AreAre / HaveHaveSpeech

Visual (lips)

Acoustic

Err

or

rate

sE

rro

r ra

tes

Quantity/quality of dataQuantity/quality of data




27

Visual Speech BiometricVisual Speech Biometric

Instantaneous lip contours from a series of framesin speaking mode




28

Why Multimodal Biometrics? Why Multimodal Biometrics? Introduction

No single biometric is generally considered sufficiently accurate and user-acceptable for any given application

Authentication systems that are robust in natural environments (e.g. in the presence of noise and illumination changes) cannot rely on a single modality

Multimodal user authentication can provide a more balanced solution to the security and convenience requirements of many applications

There is a clear requirement for the system to be able to adapt to the user needs and conditions and, especially, to be able to determine and maintain an acceptable balance between confidence and convenience for its users.

Each individual biometric can operate in either a verification mode or an identification mode


29

Multimodal Biometrics Multimodal Biometrics

Generic architecture

Biometric

sensor 1

Biometric

sensor N

Decision

fusion

Biometric

database

Claimed identity

accepted or rejected

Biometric feature 1

Biometric feature N


30

Can multimodal biometrics improve Can multimodal biometrics improve performance? performance?

Introduction Multimodal user authentication provides a practically viable approach for

overcoming the performance and acceptability barriers to the widespread adoption of authentication systems

Integration of multimodal biometric modalities is strongly based on a thorough understanding of each of the modalities and the different sensing technologies

A fully successful multimodal fusion can only be obtained through a careful investigation of these technologies and their interaction

Multimodal biometrics can improve accuracy or speed (e.g. face recognition, can be used to index a template database and fingerprint verification can be used to ensure the overall accuracy)

There is the perception that if a a strong test is combined with a weaker test, the resulting decision is averaged. However, the performance improvement comes from a well-designed fusion algorithm which should take advantage of additional information


31

Can multimodal biometrics improve Can multimodal biometrics improve performance? performance?

A

B

A

B

Fusion

If A rejects then B is used

The FRR is determined by both systems

A and B provides separate scores

The fusion algorithm decides

Can produce a very low FAR as well as a very low FRR

A B

The FAR is determined by the FAR of both systems

Sequential

Parallel

Fusion


32

Pros and Cons of Multimodal Pros and Cons of Multimodal BiometricsBiometrics

Pros Can overcome weaknesses of individual biometric identifiers Can extend the operation range to a larger target user population Can increase the reliability of the decision made by a single biometric

system Is generally more robust to fraudulent technologies (it is more difficult to

forge multiple biometric characteristics) If well-designed can improve performance and speed

Cons Can make the interaction longer Cost of deployment is generally higher Integration of multiple biometrics is more complex (score normalization,

etc.)


33

Trade-off between robustness (security) Trade-off between robustness (security) and convenienceand convenience

A very secure system will have a higher rejection rate Or it will have different passes to increase security at the expense of user

convenience For some modalities (e.g. voice) the amount of enrollment data is directly

related to this tradeoff

Robustness(Security)

Convenience


34


Main individual modalities Signature Voice Hand Fingerprint Face

Frontal Specific acquisition (infra red, profile, dynamic, range) Specific parts (eyes: iris & retina, ears)

[Specificities, Pros & cons, Open problems, Sensing technologies, Major algorithms, Database examples, …]

Towards Multimodal Biometric Systems Sequence Fusion


35

Modality by modalityModality by modality

Signature Voice Hand Fingerprint Face


36

Signatures: introductionSignatures: introduction

Several types of signatures American

Close to recursive handwritten

European Includes a graphical component

Arabic etc.

Is a signature authentic or not?

Main difficulty –Intra-class variations

(a signature of one individual) Easier to forge than other biometric

attributes

European

American

M. EL Yassa et al., « ETAT DE L'ART SUR LA VÉRIFICATION OFF-LINE DE SIGNATURES MANUSCRITES» SETIT 2003, Tunis.


37

Signatures: static vs. dynamicSignatures: static vs. dynamic

Off-line versus on-line signature Off-line

Only spatial information is available

Static: shape of the signature

On-line

Add. features: velocity, pressure, etc.

Dynamic: the dynamics of how you sign

(such as speed, pressure, and timing)

+ time


38

Signatures: acquisitionsSignatures: acquisitions

Acquisition

Off-line Scans

On-line Special hardware:

Digitizing tablet Pressure sensitive pen

“e-pad”


39

Signatures: ForgeriesSignatures: Forgeries

Forgeries

Random forgeries The forger has either no knowledge about the original signature, or does not try to imitate the shape of the signature. Zero-effort forgeries Skilled forgeries

Others: Counter-drawing Disguise


40

Signatures: Classical criteriaSignatures: Classical criteria

Classical criteria

Alignments: baseline & envelope Drawing characteristics: upward / downward drawing Speed Proportions Pressure


41

Signatures: Example of featuresSignatures: Example of features

Features: centroid, baseline, top/down envelopsFeatures: centroid, baseline, top/down envelops


42





43

Types of Speaker Recognition SystemsThree types of speech-based authentication:

•Text-dependent systems. User enrolls & authenticates with same password (template approach) Advantage: low memory, low processing power, low cost. Disadvantage: impostor can record client’s voice

•Prompted phrases/passwords system (HMM phoneme models)•Advantage: combination of recognition and adaptation can improve

performance•Disadvantage: less natural

•Text-independent systems. Enrollment speech and authentication speech are different (single state HMM, GMM)

•Advantage: very secure•Disadvantage: lower accuracy, need more resources, more enrollment and

authentication speech


44

Text-Dependent Authentication

“Santa Barbara, California”

Simple Model for P

Rightful user P

Enrollment speech

Speaker Verification

Claimant C

Test speech S

Compare scores

Speaker Enrollment

“Santa Barbara, California”

Simple Model for P Impostor Model

Accept C Reject C


45

Text-Independent Authentication

“April is the cruellest month”

Model for P (GMM)

Rightful user P

Enrollment speech

Speaker Verification

“Do I dare to eat a peach?”

Impostor model IModel for P

Claimant C

Test speech S

log p(S|P)-log p(S|I) > T?yes no

Speaker Enrollment

Accept C Reject C


46





47

Hand geometry: acquisitionHand geometry: acquisition

Fairly simple and accurate But human hand is not unique; only used for verification (not descriptive

enough for identification) Usage: Some people are reluctant to put their hands on the same support

used previously by others Special hardware; the hand is properly aligned by the pegs

“Hand punch”

fingers


48

Hand geometryHand geometry

14 axes along which features values are computed.

(5 pegs serve as control points and assist in choosing these axes).

Ps and Pe refer to the end points (using gray scale profile)→ feature vector

Reference: A prototype Hand Geometry-based

Verification System, Arun Ross


49

Hand/Finger geometryHand/Finger geometry

Measurements (4 categories) Widths: Palm, plus each of the four fingers is measured in different heights Heights: middle, little and palm. Deviations: distance between a middle point of the finger and the middle point of the straight line

between the interfinger point and the last height where the finger width is measured. Angles: between the interfinger points and the horizontal.

Classification and Verification Euclidean, Hamming, Gaussian Mixture Models (GMMs), Radial Basis Function Neural

Networks (RBF).

Reference: Biometric Identification through

Hand Geometry Measurements

R. Sanchez-Reillonet al.

IEEE PAMI Vol. 22 no. 10, 2000.


50





51

Fingerprints: acquisitionFingerprints: acquisition

Inked fingerprints: finger is rolled or dabbed on a sheet of paper

Live-scan fingerprints: no need of an intermediate medium like paper; systems are optical, thermal, electromagnetic or ultrasound based

Quality fingerprint acquisition is extremely challenging:

elastic distortion of the finger on the acquisition surface dry/wet skin scars, cuts, presence of dirt/ grease, etc.

Exact position of the finger on the scanner machine (i.e. slight rotation are possible)

Pressure of the finger on the surface of the acquisition machine

Degree of finger moisture at the contact area

Source:


52

Fingerprints: image examplesFingerprints: image examples

Subset of ST Microelectronics’ Fingerprint Image Database


53

Fingerprints: global classificationFingerprints: global classification

Arch Left Loop Right Loop Whorl

6% 34% 32% 28%

Global patterns of ridges and furrows form special configurations in the central region of fingerprints

• Class information is not sufficient to carry out recognition• Can be used for clustering: once a fingerprint is classified, it can be matched only with a subset of the database

(Courtesy of ST Microelectronics)


54

Fingerprints: local classificationFingerprints: local classification


bifurcation ending bridge lake island

• Ridge endings and bifurcations are usually used for their robustness and stability •Most automatic fingerprint matching algorithms mimic the process used by forensic experts to perform recognition:

minutiae extraction template matching

Local ridge characteristics determine the uniqueness of a fingerprint


55

Fingerprints:Fingerprints:Typical algorithm for Minutiae extractionTypical algorithm for Minutiae extraction

Minutiae Extraction Smoothing Filter Oriented Field Estimation Fingerprint Region Localization Ridge Extraction Thinning Minutiae extraction


Reference: On-Line Fingerprint Verification

Anil Jain & Ruud Bolle

IEEE T-PAMI, Vol. 19, No. 4, April 1997


56

Fingerprints: MatchingFingerprints: Matching

Alignment stage (global) Adjustment (local)



57



Frontal Specific acquisition (infra red, profile, dynamic, range) Specific parts (eyes: iris & retina, ears)

Reference:

Human and Machine Recognition of Faces: A Survey

R. Chellappa et al.

Proc. of the IEEE Vol. 83., No. 5, May 1995


58

Face: frontal face recognitionFace: frontal face recognition

Face detection - Is there a face? Face segmentation - Where? Face tracking (if video)

Face size and positionFace size and position. In practice, it is very difficult to control the position of the subject with respect to the camera

→ “Normalize” inter-ocular distance… Changes in illuminationChanges in illumination. If a spotlight is not used, lighting variations occur.

For example, close to a window, the lighting depends strongly on the time of the day and the weather

→ “Normalize” gray-scale histogram… Facial expressionsFacial expressions. In practice, it is almost impossible to control the mood

of the subject. The smile causes probably the largest variation of facial expressions

Others. Others. Glasses, Hats, Facial hair, etc.


59

Face (frontal): Image ExamplesFace (frontal): Image Examplesdatabase: ORL, FERET, M2VTS & XM2VTSdatabase: ORL, FERET, M2VTS & XM2VTS

Subset of AT&T’s Face Image Database


60

Face detection & segmentationFace detection & segmentation

Reference:

Detecting Faces in Images: A Survey

M.-H. Yang at al.

IEEE t-PAMI, Vol. 21, No. 1, Jan. 2002


61

Face: frontal face recognitionFace: frontal face recognition

Two successful classes of algorithmsAccording to the last round of NIST evaluations, current best solutions are

derived either from Eigenfaces or Elastic Graph Matching approaches.

Projection-based approaches: Eigenfaces→ Fisherfaces. Deformable models: Elastic Graph Matching (EGM)→ Elastic Bunch Graph Matching.


62

EigenfacesEigenfaces(eigeneyes, eigenmouths, eigenvoices, eigenears, etc.)(eigeneyes, eigenmouths, eigenvoices, eigenears, etc.)

Reference: Face Recognition Using Eigenfaces

M. Turk & A. Pentland

IEEE 1991


63

EigenfacesEigenfaces

(Originally designed for compression, not recognition)

Eigenspace can be built using the clients (higher performances but less flexible) or not

I(x,y) can be considered as a two-dimensional NxN array of pixels

(if N=256; can be seen as a point in a 65,536 dimensional space)

“face space”

The space of variation between photographs of human faces with the same

orientation and scale lit in the same way can be described by a relatively low

dimensional subspace.

Individual face image ≈ linear combination of a small number of face components

I1, I2 … IN: set of reference or training faces

(eigenface 0)



IEEE 1991

N

iiI

NE

10

1


64


Each face differs from the other faces by the vector Di = Ii – E0.

Covariance matrix C:

Eigenvectors of C (variation between face images) → Eigenfaces Ek

Dimensionality Reduction Technique (DRT)Principal Component Analysis (PCA)→ eigenvectors ordered by the magnitude of their contribution to the variation between

the training images

Extract the R eigenvalues; Order them from largest to smallest, 1, 2, …r Order corresponding eigenvectors E1, E2, …Er→ « principal components »

Weighted sums of a small collection of characteristic images



IEEE 1991

K

kkk

iki EEI

10

N

i

Tii DD

NC

1

1


65




IEEE 1991

Euclidean distances between the K coordinates representing the new faceand each of the K-dimensional vectors representing the stored faces,

→ the stored image yielding the smallest distance

PCA in a 2-D space


66

Eigenfaces vs. FisherfacesEigenfaces vs. Fisherfaces

• Eigenfaces [Kirby & Sirovich, Turk & Pentland]: - no distinction between inter- and intra-class variabilities

Average face (eigenface 0) and first four eigenfaces

• Fisherfaces [Belhumeur, Hespanha & Kriegman]:- discriminative approach: find a sub-space which maximizes the ratio of

inter-class and intra-class variability- same intra-class variability for all classes

Average face (Fisherface 0) and first four Fisherfaces


67

EGMEGMElastic Graph MatchingElastic Graph Matching

(a) (b) (c)

(a) Model grid for person A (1 feature vector / node)

(b) Best grid for test person A after elastic graph matching with the model grid.

(c) Best grid for test person B after elastic graph matching with the model grid for person A.

• Vertex labels (local mappings costs)• Edge labels (local distortions costs)• λ controls the rigidity of the image graph

vetotal CCC

Distortion Invariant Object Recognition

In the Dynamic Link Architecture

M. Lades et al.

IEEE Trans. On Computers, Vol. 42, no. 3 1993.

Courtesy of Thessaloniki Univ.


68



Frontal Specific acquisition (infra red, profile, dynamic, range) Specific parts (eyes: iris & retina, ears, dental)


69

ProfileProfile

distance and angle between fiducial points

Source:


70

Dynamic Video BiometricsDynamic Video Biometrics

Higher potential of video w.r.t. still images

More clues (Abundant data) Face / Facial feature tracking

New opportunities

Visual speech data → correlation between speech and lip motion) Dynamic facial Expression→ behavior (not physical only) Shape/Structure from motion

Useful for covert surveillance (but non-cooperative with low resolution)


71

3-D Faces, range data3-D Faces, range data

Advantages: Access to the depth information (i.e. shape) Pose and light conditions may be compensated Higher robustness (e.g. the system cannot be trapped by an impostor using a 2D picture of

someone else)

Disadvantages: Acquisition process is slow and highly expensive, e.g. 3-D Scanner, 2 calibrated video

cameras Cooperation of the users is required

Little literature available on the topic (novel facial biometrics)Some published works…

Extension of existing algorithms from 2-D to 3-D (e.g. eigenfaces) Adaptation of a generic 3-D Deformable Model to 2-D images of users to provide a set of

parameters associated with a person Segmentation of range data into 4 surface regions; Normalization based on the location

of eyes, nose and mouth; Distance computed from the volume between surfaces …

- Face Recognition using range images B. Achermann et al., VSMM 97.

- Face Identification by Fitting a 3D Morphable Model using Linear Shape and Texture Error Functions, S. Romdhani et al., ECCV 2002

- Face Recognition based on depth maps and surface curvature G. Gordon, SPIE Proc., vol. 1570, 1991.


72

Facial Thermogram (IR imaging)Facial Thermogram (IR imaging)The facial heat emission patterns can used

to characterize a person

Patterns depend on 9 factors including:- Location of major blood vessels- Skeleton thickness- Amount of tissue- Muscle- Fat

Advantages: Unique (even for identical twins) Stable over time Cannot be altered through plastic surgery Independent of the lighting conditions

Disadvantages: IR imagery depend on the temperature Opaque to glass

- Thermal pattern recoginition systems faces security challenges head on

M. Lawlor, Signal Magazine Nov. 97

- Comparison of visible and infra-red imagery for face recognition

J. Wilder et al., Int. Conf. On Automatic Face and Gesture Recognition, Oct. 96

Source: Other Biometric Techniques

Chapter 10. D. Baik and I. Kim


73



Frontal Specific acquisition (infra red, profile, dynamic, range) Specific parts (eyes: iris & retina, ears, dental)


74

IrisIris

How Iris Recognition Works

J. Daugmann http://www.cl.cam.ac.uk/users/jgd1000/


75

IrisIris

4 steps

1. Acquisition ( < 1 meter)2. Find iris in the image (edge detection)

3. Features extraction: - Local regions of an iris are projected

onto quadrature 2D Gabor wavelets, generating complex-valued coefficients

whose real and imaginary parts specify the coordinates of a phasor in the complex plane

- The angle of each phasor is quantized to one of the four quadrants, setting two bits of phase information

- This process is repeated all across the iris with many wavelet sizes, frequencies & orientations

→ the Iris-Code (1024 phase bits are computed)

4. Verification

How Iris Recognition Works

J. Daugmann http://www.cl.cam.ac.uk/users/jgd1000/


76

RetinaRetina Unique

Number/Pattern of blood vessels, that emanate from the optic nerve and disperse throughout the retina Relative angle w.r.t. optical nerve Bifurcations No two retinas are the sameeven in identical twins Vascular pattern does not change over the

course of life Glasses, contact lenses, existing medical

conditions (e.g. cataracts) do not interfere

At the moment, identification based on retina is used for animals (bovines)

‘Uncomfortable’ acquisition Eye has to fix a lighting point Projected lighting source on the center of

the optical nerve Light is absorbed by red vessels but

reflected by retina tissues

Coutesy of J. Leroux des Jardins


77

RetinaRetina

Source: http://www.retinaltech.com

Extracting Intensity Profiles Performing Scan

Locating Blood Vessels Generating Circular Bar Code


78

EarEar

Sources- On the use of Outer Ear Images for Personal Identification in Security Applications

B. Moreno et al., IEEE 1999.- http://www.dcs.shef.ac.uk/~miguel/papers/msc-thesis.html- Ear Biometrics for Machine Vision M. Burge and W. Burger

Advantages of ears over faces Uniform distribution of colors Reduced surface Less variability vs. pose/expressions, Shape and appearance fixes.

Passive identification (≠ fingerprints)

Algo. Eigenears Feature points


79

Iannarelli’s Ear BiometricsIannarelli’s Ear Biometrics

Source: Ear Biometrics for Machine Vision M. Burge and W. Burger

Iannarelli System (1949) is based on 12 measurements.(External anatomy)

1 Helix Rim,

2 Lobule,

3 Antihelix,

4 Concha,

5 Tragus,

6 Antigrus

7 Crus of Helix

8 Triangular Fossa

9 Incisure Intertragica

Distance between each

of the numbered areas

Segmented outer ear Segmented inner ear


80

EigenearsEigenears

Sources:

- On the use of Outer Ear Images

For Personal Identification in Security Applications

B. Moreno et al., IEEE 1999.- http://www.dcs.shef.ac.uk/~miguel/papers/msc-thesis.html


81

« Liveness » and countermeasure« Liveness » and countermeasure

Impostors may use a fake biometric, Photography of a face Recorded voice Plaster hand etc.

Countermeasure: To use a « liveness » test to check the presence of a “real” biometric, e.g. cardiac activity, heart rate


82

Diverse facets of MultimodalDiverse facets of Multimodal

Source: Ph. D., Fingerprint classification and matching using a filterbank,

S. Prabhakar


83

Multimodality & fusionMultimodality & fusione.g. Some possible scenarios in Facese.g. Some possible scenarios in Faces

frontal

profile

Fusion

Face

IrisIf needed only

less comfortable

but more accurate

By default

Visible

IRIf case of darkness

By default

In case of different shots available

profile

EarIf needed only

less comfortable

but more accurate

By default


84

FusionFusion

At 3 possible levels:

Abstract level→ Output of each module is a list of labels without any confidence

information,Identification: ID of the person

Verification: binary response

Rank level Output of each module is a set of possible labels ranked by

decreasing confidence values Measurement level A measure of confidence is associated with each label

Person Identification Using Multiple Cues

R. Brunelli & D. Falavigna

IEEE T-PAMI, Vol. 17, no. 10, pp. 955-966, Oct. 95


85

FusionFusion

{wi} i=1..N

the set of possible classes Identification: Number of persons present

inside the database; Verification: Authentic and Impostor.

{xj} j=1..M

the set of biometrics

Abstract:

Vote based on majority

Ranks:

Maximum, minimum and median

Scores:

Averaging & Weighed averaging

)xw(Pmaxmax jiM

1jN

1i

Person Identification Using Multiple Cues

J. Kittler et al.

IEEE T-PAMI, Vol. 20, no. 3, pp. 226-239, 98.

)xw(Pminmax jiM

1jN

1i

)xw(Pmedmax jiM

1jN

1i

M

1jji

N1i )xw(P

M

1max

M

1jjii

N1i )xw(Pmax


86

(hard/soft) Fusion: AND/OR(hard/soft) Fusion: AND/OR

Score #1

Sco

re #

2

Operator AND Operator OR Arithmetic Operator(mean)

Accepted User

Advanced fusion: SVM (Support Vector Machine)

http://www.cl.ac.uk/users/jgd1000/combine/combine.html

A Tutorial on Support Vector Machines for Pattern Recognition

C. Burges ([email protected])


87

Example: Face + FingerprintExample: Face + Fingerprint

Goal: To overcome the limitations of both systems i- Pre-selection of N persons using Face Recognition (top 5 matches) ii- Fingerprint Verification only performed on pre-selected persons

Reminder Face recognition is fast but not reliable while fingerprint verification is reliable

but inefficient in database retrieval

Reported results

FAR FRRface fingerprint integration

1% 15.8% 3.9% 1.8%0.01% 61.2% 10.6 6.6%

(0.9 sec.) (3.2 sec.) (4.1 sec.)

Integrating Faces and Fingerprints for Personal Identification

L. Hong & A. Jain

IEEE T-PAMI, Vol. 20, No.12, pp. 1295-1307, 1998.


88

Example: Frontal + Profile + SpeechExample: Frontal + Profile + Speech

Method EER (%)

Frontal 12.2

Profile 8.5

Speech 1.4

Sum 0.7

Product 1.4

Maximum 12.2

Median 1.2

Minimum 4.5

KITTLER et al. « On Combining Classifiers »

IEEE T-PAMI, Vol. 20, No. 3, March 98


89

Example: Face + SpeechExample: Face + Speech

- B. Duc et al., « Fusion of audio and video information for multimodal person authentication »

Pattern recognition Letters 18 (1997) 835-843

- S. Ben-Yacoub, « Multi-Modal Data Fusion for Person Authentication using SVM »

IDIAP RR 98-07

Supervisor FA(%) FR(%) TE(%)

Face(EGM)

3.6 7.4 11.0

Speech (Text-dependent)

6.7 0.0 6.7

Arithmetic mean 1.2 2.1 3.3

Bayesian conciliation

0.54 0.0 0.54

Linear-SVM 0.07 0.0 0.07

Polynomial-SVM 0.21 0.0 0.21

RBF-SVM 0.12 0.0 0.12

MLP-SVM 0.15 0.0 0.15


90

Example: FaceExample: Facess

J. Kittler et al., « Enhancing the performance of personal identity authentication systems by fusion of face verification experts”

Expert Evaluation Test Test Set

UniS-gdm 97.83 97.15

UniS-noc 96.46 96.90

Unis-eucl 88.80 91.15

UCL-lda 96.11 96.68

UCL-pm1 94.43 95.34

UCL-pm2 95.29 96.14

N. Experts DT BKS

2 97.02 97.92

3 97.51 98.15

4 96.46 98.21

5 96.12 98.34

6 95.81 98.43

DATABASE: XM2VTS DT: decision Templates BKS: Behavior Knowledge Space ASR: Average Success Rate of client

acceptance and impostor rejectionon the Evaluation set (top), on the Test set (down).

BKS >> DT By adding experts, the performance of the

multimodal system will not be degraded. For a sufficient number of experts, optimal

configuration selected on the evaluation set, also a posteriori optimal on the test set.

N. Experts DT BKS

2A posteriori

87.7987.79

97.3997.39

3A posteriori

97.797.7

97.7897.78

4A posteriori

97.2897.8

97.6898.05

5A posteriori

96.696.6

98.0898.19

6A posteriori

96.2496.33

98.4998.49


91

Note on Biometrics vs. Data HidingNote on Biometrics vs. Data Hiding

Goal: To combine watermarking and biometrics, for example by hiding the minutia of a passport’s owner inside his/her id picture present on the document

→ In order to enforce the security of documents (harder to falsify thanks to cross-security)

Embedding of eigenface data (associated with a face image) in a fingerprint image (cover) of a given person

Problem: Relevant characteristics of host image must remain unchanged (e.g. location and nature of minutia), i.e. the same map of minutia must be obtained either from the original fingerprint image or from its watermarked version

Basic recall on watermarking

The aim of digital watermarking is to include a subliminal information (i.e. imperceptible) in a multimedia document for security purpose (e.g. copyright)

It would then be possible to recover the embedded message using a secret key, at any time, even if the document was altered

Trade-off: capacity, visibility and robustness

- Information Hiding, S. Katzenbeisser and F. Petitcolas, Eds. 2000 Artech House

- Hide a Face in a Fingerprint Image Jain et al.


92


Applications, Standards and Evaluation Main application areas Biometrics and privacy Important criteria to deploy multimodal authentication systems Biometric standards Multimodal databases Best practices in testing biometric systems Examples of multimodal user authentication systems Perspectives and future challenges Demonstrations Forthcoming events Bibliography


93

Main application areasMain application areas

Biometric applications can be classified as follows:

Forensics: criminal investigation and prison security Retail/ATM/point of sale Civilian applications: electronic commerce and electronic banking (e.g. Visa cards) Information system/ computer network security: user authentication, remote access

to databases Physical access/time and attendance (e.g. cellular phone, workstations, door

entrance, automobile) Citizen identification (e.g. interaction with government agencies) Surveillance (identify or verify the identity of individuals present in a given

space/area, e.g. airport)

Different Markets require different biometric levels of security

S. Nanavati, M. Thieme, and R. Nanavati: Biometrics, Identity Verification

in a Networked World, Wiley Computer Publishing, 2002


94

Biometrics in Airports Biometrics in Airports (USA)(USA)Airport Biometric First Installed, Trial Population

Staff, travellers

Supplier

Charlotte/Douglas Iris US Airways employees entering secure ares

Chicago Finger Cargo truck drivers who deliver to the airport Identix

Fresno Face

Idaho Hand Trial Reco. Systems Inc.

JFK Hand

Lincoln Finger Identix

Logan Iris Nov. 02 Trier Technologies

LAX Hand

Manchester Face

Miami Hand

Mineta San Jose Hand Trial Reco. Systems Inc.

Portland Hand Trial Reco. Systems Inc.

Salt Lake City Hand Reco. Systems Inc.

SF Hand

Springfield Finger Identix

St. Petersburg Face

St. Petersburg- Clearwater

Finger Identix

…


95

Biometrics in Airports Biometrics in Airports (EU, Others)(EU, Others)Airport Biometric First Installed, Trial Population

Staff, passengers

Supplier Country

Charles de Gaulle Trial France

Orly Trial

Berlin 1-1 Face reco. Jan. 03 ZN Vision Techn. AG Germany

Frankfurt Iris

Amsterdam Schiphol

Iris Oct. 01 Joh. Enschede BV Netherlands

London Heathrow Iris Aug. 02 UK

Keflavik 1-n Face Jun. 01 Crowd Identix Inc., Visionics FaceIt Software

Iceland

Ben Gurion 1-1 Hand 1998 Passenger moving through custom

Reco. Syst. Inc. Israel

Narita Face and Iris NTT DoCoMo

King Abdul Aziz Iris Feb. 02 Saudi Arabia

Singapore 1-1 Finger

Thunder Bay Face Trial NExus AcSys Canada

Toronto Hand Canada

Vancouver Finger and Hand Canada

…


96

In practice…In practice…

INPASS program Enrollment procedure about 30 min. Inpass benefits: insignificant (?)

Palm Beach Airport Airport Face scanner failed… Error rate of 53% (455 success out of 958 attempts) Vendor argues system was not used

properly (i.e. incorrect lighting)


97

Vertical MarketsVertical Markets

Law enforcement

Government sector

Financial sector

Healthcare

Travel and immigration

However, biometric deployments in these markets are not always very different

S. Nanavati, M. Thieme, and R. Nanavati: Biometrics, Identity Verification

in a Networked World, Wiley Computer Publishing, 2002


98

Published September 2001, International Biometrics Group

Fingerprint technology is the only biometric that has been implemented

within a large scale (IAFIS)


99

Smart Cards and BiometricsSmart Cards and Biometrics

A Smart Card is a portable secure storage(can contain computer chip)

Smart Cards are excellent support for privacy

Smart Cards can verify the biometric identity

Smart cards can update the biometric template

Smart Cards prevent the need for a big centralized database (support privacy)


100

PrivacyPrivacy

“your privacy is important to us. How much would you pay to preserve it?”

The Wall Street Journal, November 14th, 2001


101

PrivacyPrivacy

“Privacy is the claim of individuals, groups, or institutionsto determine for themselves, when, how and to what extentinformation about them is communicated to others”

Definition by Alan Westin


102

PrivacyPrivacy

Use biometric data in accordance with privacy needs

General requirements

Technical requirements

Do not store biometric raw data in a database Do not use the biometric data outside the specified purpose Do not collect unnecessary personal data Use adequate algorithms for the calculation of biometric signatures


103

Privacy ConcernsPrivacy Concerns

High

Low HighAm

ount

of

data

Sensitivity of the data

Very High

Factors affecting privacy

Privacy is becoming an increasingly important issue especiallyin large systems


104

Important criteria to deploy multimodal Important criteria to deploy multimodal user authentication systems user authentication systems

Enrollment User acceptance Privacy/Civil liberties ID management/ID theft Database management/Integrity Political and cultural environment System complexity Cost


105

Important criteria to deploy multimodal Important criteria to deploy multimodal user authentication systemsuser authentication systems

Before introducing this technology to customers, a number of fundamental questions about consumer understanding, expectations and concerns need to be answered. The answers to these questions will help the development of solutions that are accepted by the consumers

Understanding consumer attitudes towards this technology is essential to business managers as they study the ROI and design the user interface“


106

Biometric standardsBiometric standards What are standards and what are they good for?

Standards (a general set of rules to which all complying procedures, products or research must adhere) offer a myriad of benefits. They reduce differences between products and promote an aura of stability, maturity and quality to both consumers and potential investors (http://www.biometrics.org/html/standards.html)

Who establishes them? Standard Bodies, e.g.

American National Standards Institute (ANSI)

International Standards Organization (ISO)

National Institute of Standards and Technology (NIST)

What biometric standards are available? Several already exist (http://www.biometrics.org/html/standards.html)


107

Biometric standardsBiometric standards BioAPI

BioAPI (March 2002: BioAPI Version 1.1 was approved as ANSI/INCITS 358-2002)

Biometric Consortium took the lead to merge the efforts of several vendors under BioAPI with strong support from NIST

Defines a generic way of interfacing to a broad range of biometric technologies

Founded in 1988 by Compaq, Microsoft, Novell, IBM, Identicator, Miros. Merged with other efforts in 1999

Purpose: Development of a standard biometric API to bring platform and device independence to application developers, integrators and end-users

Benefits Easy substitution of biometric technologies Use of biometric technologies across multiple applications Easy integration of multiple biometrics using the same interface Rapid application development – increased competition (tends to lower costs) Application compatibility / interoperability www.bioapi.org


108

Biometric standardsBiometric standards CBEFF

CBEFF (NISTIR 6529, Jan.3, 2001)

Common Biometric Exchange File Format Describes a set of data elements necessary to support biometric

technologies in a common way Features

Facilitate biometric data interchange between different system components or systems

Promotes interoperability of biometric-based application programs and systems

Provides forward compatibility for technology improvements Simplifies the hardware and software integration process www.itl.nist.gov/div895/isis/cbeff


109

Multimodal databasesMultimodal databases

There are very few They are costly to record Many parameters need to be taken into account (e.g. for one modality such

as voice: speaker population, environment, age, text to say, etc.) Realistic data (from real-world application is very difficult to collect and it is

generally difficult to control the different factors) Nature of the imposters is an issue, etc.

→ One possibility could consist in building a multimodal database by artificially combining different unimodal ones.

There is no correlation between most of biometrics


110

Multimodal databases: XM2VTSMultimodal databases: XM2VTS

http://www.ee.surrey.ac.uk/Research/VSSP/xm2vtsdb/ The database was recorded within the M2VTS project (Multimodal Verification

for Teleservices and Security applications), a part of the EU ACTS program, which deals with access control by the use of multimodal identification of human faces

The goal of using a multimodal recognition scheme is to improve the recognition efficiency by combining single modalities, namely face and voice features

The XM2VTSDB contains four recordings of 295 subjects taken over a period of four months. Each recording contains a speaking head shot and a rotating head shot. Sets of data taken from this database are available including high quality colour images, 32 KHz 16-bit sound files, video sequences and a 3D Model


111

Multimodal databases: BT-DAVIDMultimodal databases: BT-DAVID http://galilee.swan.ac.uk

The BT-DAVID (Digital Audio-Visual Integrated Database) audio-visual database is designed for undertaking research in speech or person recognition, as well as synthesis and communication of audio-visual signals

Expected areas of application are: automatic speech/person recognition for terminal interfaces or automated transaction machines, voice control of video-conferencing resources, speech-assisted video coding, and synthesis of talking heads

The BT-DAVID database contains full-motion video, showing a full-face and a profile view of talking subjects, together with the associated synchronous sound. BT-DAVID includes audio-visual material from more than 100 subjects including 30 clients recorded on 5 sessions spaced over several months

The BT-DAVID database was compiled by the Speech and Image Research Group at University of Wales Swansea under a contract to BT Labs


112

Best practices in testing biometric Best practices in testing biometric systemssystems

Fact: It is still very difficult to predict real-world error rates

Besides performance (which includes both false positive and false negative decisions along with failure to enroll and failure to acquire rates across the test population) the following criteria should also be taken into account

Reliability, availability and maintainability Vulnerability Security User acceptance Human factors Cost/benefit Privacy regulation compliance


113

Best practices in testing biometric Best practices in testing biometric systemssystems

Biometric technical performance testing can be of three types: Technology (database evaluation) Scenario (overall system performance in a prototype or simulated

application. Could be a combination of offline and online testings) Operational evaluation (performance of a complete system in a specific

application environment with a specific target population. In general not repeatable)

Each type of test requires a different protocol and produces different results

The nature of impostors is an important part of the testing of biometric systems

A.J. Mansfield and J.L. Wayman: Best Practices in Texting and

Reporting Performance of Biometric Devices, NPL Report CMSC 14/02

P.J. Phillips, A. Martin, C.L. Wilson, and M. Przybocki: An introductionto evaluating biometric systems. Computer, (Feb. 2000), 56-63


114

ExampleExampleK. Jain, L. Hong, and Y. Kulkarni: A Multimodal Biometric System Using Fingerprint,Face, and Speech, Technical Report MSU-CPS-98-32, Department of Computer Science,Michigan State University

False acceptance rate (%)

Acc

ep

tan

ce r

ate

(%

)


115

ExampleExample BioID SDK by HumanScan (Germany)

http://www.humanscan.de/products/bioid/index.php

BioID SDK offers multimodal biometrics in the form of a software development kit

BioID SDK offers three biometrics: • Face recognition • Voice recognition • Lip movement recognition

Since BioID uses true multimodality, the preferred way of using it is by using all of the three biometrics together. But BioID can be easily configured (e.g. using the Control Panel) to use any of the above three biometrics alone or in any combination

Basic features include:• User enrollment wizard • User recognition (verification or identification) • User template and authorization management • Enrollment management • Template storage to database, local PC or Smart Card • Support BioAPI

R.W. Frischholz and U. Dieckmann: “BioID: A Multimodal Biometric Identification

System”, IEEE Computer, vol. 33, no. 2, pp. 64-68, February 2000


116

Perspectives and future challengesPerspectives and future challenges

Multimodal biometrics will play vital roles in the next generation of automatic identification systems

Future challenges in multimodal biometric systems

Accuracy is still an issue for most of existing biometrics Feature extraction Dealing with dynamic information using a small amount of training data How to combine information (fusion) and make use of the strengths of each modality Collection of a multimodal and realistic database (most of the existing databases are

unimodal) Integrating higher level of information (e.g. for speech, prosodic modeling,

word/phrase usage) Scalability Establishment of common standards along the lines of GSM in the mobile world Dealing with privacy concerns Ease of use and development


117

DemonstrationsDemonstrations

Iris recognition

Speaker and Fingerprint recognition for door entrance system


118

Forthcoming eventsForthcoming events

IEE PROCEEDINGS VISION, IMAGE AND SIGNAL PROCESSING, Special Issue on BIOMETRICS ON THE INTERNET, Aladdin Ariyaeeinia, University of Hertfordshire, UK, Guest Editor (http://www.iee.org/Publish/Support/Auth/Authproc.cfm)

Multimodal User Authentication workshop, Santa Barbara, CA, U.S.A. December 2003 (http://mmua.cs.ucsb.edu )

International Conference on Biometric Authentication, Hong Kong, January 2004 (http://www4.comp.polyu.edu.hk/~icba/)

EURASIP, Applied Signal Processing,Special Issue on Biometric Signal Processing (4th quarter 2003) (http://asp.hindawi.com)


119

AcknowledgmentsAcknowledgments(for inputs, fruitful discussions and help)(for inputs, fruitful discussions and help)

Institut Eurécom (Florent Perronnin) Panasonic Speech Technology Laboratory University of Thessaloniki


120

BibliographyBibliography

BOOKS S. Nanavati, M. Thieme, and R. Nanavati: “Biometrics, Identity Verification in a Networked World”,

Wiley Computer Publishing, 2002. J. Ashbourn: “Biometrics, Advanced Identity Verification, The Complete Guide”, Springer, 2000. L.C. Jain, U. Halici, I. Hayashi, S.B. Lee, and S. Tsutsui, editors: “Intelligent Biometric Techniques in

Fingerprint and Face Recognition”, The CRC Press International Series on Computational Intelligence, 1999.

A. Jain, R. Bolle, and S. Pankanti, editors: “Biometrics, Personal Identification in Networked Society”, Kluwer Academic Publishers, 1998.


121


INTRODUCTION Y. W. Yun The ‘123’ of Biometric Technology.

OVERVIEW J.-L. Dugelay, J.-C. Junqua, C. Kotropoulos, R. Kuhn, F. Perronnin, and I. Pitas: “Recent Advances in

Biometric Person Authentication”, ICASSP 2002, pp. IV 4060-IV 4063.

COURSE (slides) J. Wayman (San José State University) Biometrics & How they Work.


122


MULTIMODAL A.J. Mansfield and J.L. Wayman: “Best Practices in Texting and Reporting Performance of Biometric

Devices”, NPL Report CMSC 14/02. K. Jain, L. Hong, and Y. Kulkarni: “A Multimodal Biometric System Using Fingerprint, Face, and

Speech”, Technical Report MSU-CPS-98-32, Department of Computer Science, Michigan State University.

L. Hong and A. Jain: “Integrating Faces and Fingerprints for Personal Identification”, IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 20, no. 12, pp. 1295-1307, December 1998.


123


FUSION R. Brunelli and D. Falavigna: “Person Identification Using Multiple Cues”, IEEE Trans. Pattern Analysis

and Machine Intelligence, vol. 17, no. 10, pp. 955-966, October 1995. J. Kittler, M. Hatef, R.P.W. Duin, and J. Matas: “On Combining Classifiers”, IEEE Trans. Pattern Analysis

and Machine Intelligence, vol. 20, no. 3, pp. 226-239, March 1998. R.W. Frischholz and U. Dieckmann: “BioID: A Multimodal Biometric Identification System”, IEEE

Computer, vol. 33, no. 2, pp. 64-68, February 2000. V. Chatzis, A.G. Bors, and I. Pitas: “Multimodal Decision-Level Fusion for Person Authentication”, IEEE

Trans. Systems, Man and Cybernetics, Part A, vol. 29, pp. 674-680, November 1999. C. Burges: “A Tutorial on Support Vector Machines For Pattern Recognition”.


124


LIPREADING – audio/video B. Duc, E.S. Bigun, J. Bigun, G. Maitre, and S. Fischer: “Fusion of Audio and Video Information for Multi-

Modal Person Authentication”, Pattern Recognition Letters, vol. 18, pp. 835-843, 1997. S. Ben-Yacoub, Y. Abdeljaoued, and E. Mayoraz: “Fusion of Face and Speech Data for Person Identity

Verification”, IEEE Trans. Neural Networks, vol. 10, no. 5, pp. 1065-1074, September 1999.


125


SPEECH D.A. Reynolds and L.P. Heck: “Speaker Verification: From Research to Reality”, Tutorial, ICASSP, Salt

Lake City, Utah, May 7, 2001. G. Doddington: “Speaker Recognition Based on Idiolectal Differences between Speakers”, Eurospeech

2001, V. 4, pp. 2521-2524, Aalborg, Denmark, Sept. 3-7, 2001. O. Thyes, R. Kuhn, P. Nguyen, and J-C. Junqua: “Speaker Identification and Verification Using

Eigenvoices”, ICSLP-2000, V. 2, pp. 242-245, Beijing China, Oct. 2000.


126


FACE R. Chellappa, C.L. Wilson, and S. Sirohey: “Human and Machine Recognition of Faces: A Survey”,

Proceedings of the IEEE, vol. 83, no. 5, pp. 705-740, May 1995. M. Turk and A. Pentland: “Eigenfaces for Recognition”, J. Cognitive Neuroscience, vol. 3, no. 1, pp. 71-

86, 1991. S. Pigeon and L. Vandendorpe: “Image-based Multi-Modal Face Authentication”, Signal Processing, vol.

69, pp. 59-79, August 1998.

IRIS J. Daugman: “Recognizing Persons by their Iris Patterns”, in Biometrics, Personal Identification in

Networked Society, pp. 103-121, A. Jain, R. Bolle and S. Pankanti, editors, Kluwer Academic Publishers, 1998.

RETINA http://www.retinaltech.com.

EAR B. Moreno et al.: « On the Use of Outer Ear Images for Personal Identification in Security

Applications » 1999 IEEE. M. Burge and W. Burge: « Ear Biometrics for Machine Vision ».


127


FINGERPRINT R. Adhami and P. Meenen: “Fingerprinting for Security”, IEEE Potentials, Vol. 20, no. 3, pp. 33-38,

Aug.-Sept. 2001. A. Jain and S. Pankanti: “Automated Fingerprint Identification and Imaging Systems”, Advances in

Fingerprint Technology, 2nd Ed., Elsevier Science, New York, 2001.


128


HAND GEOMETRY R. Sanchez-Reillonet et al. : “Biometric Identification through Hand Geometry Measurements”, IEEE

PAMI Vol. 22, No. 10, 2000.

SIGNATURE A. Jain et al. : “On-line Fingerprint Verification”, IEEE T-PAMI Vol. 19, No. 4, April 97.


129


DATABASE S. Pigeon and L.Vandendorpe: “The M2VTS multimodal Face Database”, Lecture Notes in Computer

Science: Audio- and Video-Based Biometric Person Authentication (J. Bigun, G. Chollet, and G. Borgefors Eds.), vol. 1206, pp. 403-409, 1997.

K. Messer, J. Matas, J. Kittler, J. Luettin, and G. Maitre: “XM2VTSDB: The Extended M2VTS Database”, in Proc. 2nd Int. Conf. on Audio- and Video-Based Biometric Person Authentication, March 1999.

EVALUATION P. J. Philips, et al.: « The Feret evaluation methodology for face-recognition algorithms », IEEE T-PAMI,

Vol. 22, No. 10, Oct. 2000.


130

End.End.

[email protected] (Jean-Claude JUNQUA) [email protected] (Jean-Luc DUGELAY)

Multimodal User Authentication: From Theory to Practice 1 TUTORIAL Conference IEEE ICME 2003 ...

Documents

Transcript of Multimodal User Authentication: From Theory to Practice 1 TUTORIAL Conference IEEE ICME 2003 ...