Advances in Fingerprint Recognition at CUBS

Venu Govindaraju Director of Center for Unified Biometrics and Sensors

University at Buffalo, State University of New York govind@buffalo.edu

Abstract

The Center for Unified Biometrics and Sensors (CUBS) at the University at Buffalo, SUNY is developing biometrics identification systems for civilian, law enforcement and homeland security applications. The center is involved in traditional research developing matching algorithms for several biometric modalities (e.g., fingerprint, signature, face) and in identifying exploratory areas of research that will address problems anticipated in wide- scale deployment of biometrics. In this paper we describe some of our recent research in fingerprint verification. In particular, we will describe a new non-stationary fingerprint enhancement algorithm based on Fourier domain analysis and a partial fingerprint matching algorithm that uses novel approaches from graph and optimization theory. We will also present new score computation algorithm that treats fingerprint verification as a classification problem. Finally, we will describe a novel technique for protecting the privacy and security of fingerprint templates that is based on symmetric hash functions.

INTRODUCTION In an increasingly digital world, reliable personal authentication is an important human computer interface activity. National security, e-commerce, and access to computer networks are some examples where establishing a person's identity is vital. Existing security measures rely on knowledge-based approaches like passwords or token-based approaches such as swipe cards and passports to control access to physical and virtual spaces. Though ubiquitous, such methods are not very secure. Tokens such as badges and access cards may be shared or stolen. Passwords and PIN numbers may be stolen electronically. Further, they cannot differentiate between authorized user and a person having access to the tokens or knowledge. Biometrics such as fingerprint, face and voice print offers means of reliable personal authentication that can address these problems and is gaining citizen and government acceptance. Biometrics is the science of verifying the identity of an individual through physiological measurements or behavioral traits. Since biometric identifiers are associated permanently with the user they are more reliable that token or knowledge based authentication methods.

Figure 1: (a)Various biometric modalities: Fingerprint, Voice, Signature, Face recognition, Hand Geometry and chemical biometrics (b) General architecture of a biometric authentication system

Depending on the application, biometrics can be used for identification or for verification. In verification, the biometrics is used to validate the claim made by the individual. The biometric of the user is compared with the biometrics of the claimed individual in the databases. The claim is rejected or accepted based on the match. (In essence, the system tries to answer the question, “Am I whom I claim to be?”). In identification, the system recognizes an individual by comparing his biometrics with every record in the database. (In essence, the system tries to answer the question, “Who am I?”). In this paper, we will be dealing solely with the problem of verification using fingerprints. In general, biometric verification consists of two stages (Figure 1b) (i) Enrollment and (ii) Authentication. During enrollment, the biometrics of the user is captured and the extracted features (template) are stored in the database. During authentication, the biometrics of the user is captured again and the extracted features are compared with the ones already existing in the database to determine a match. The specific record to fetch from the database is determined by using the claimed identity of the user. Furthermore, the database itself may be central or distributed with each user carrying his template on a smart card. Biometrics offers several advantages over traditional security measures. These include (i) Non-repudiation: With token and password based approaches, the perpetrator can always deny committing the crime pleading his/her password or ID was stolen or compromised in some fashion. Therefore a user can repudiate or deny the use of a service even when an electronic record exists. However, biometrics is indefinitely associated with a user and hence it cannot be lent or stolen making

repudiation infeasible. (ii) Accuracy and Security: Password based systems are prone to dictionary and brute force attacks. Furthermore, the system is as vulnerable as its weakest password. Biometric authentication requires the physical presence of the user and therefore cannot be circumvented through a dictionary or brute force style attack. Biometrics have also been shown to possess a higher bit strength compared to password based systems (Jea, Chavan et al. 2004) and are therefore inherently secure (iii) Screening : In screening applications, we are interested in preventing the users from assuming multiple identities (e.g. a terrorist using multiple passports to enter a foreign country). This requires we ensure that a person has not already enrolled under another assumed identity before enrolling his new record into the database. Such screening is not possible using traditional authentication mechanisms and biometrics provides the only available solution. Fingerprints were one of the first biometrics to be adopted and have currently become synonymous with reliable personal identification. Fingerprints were accepted formally as valid personal identifier in the early twentieth century and have since then become a de-facto authentication technique in law-enforcement agencies world wide. The FBI currently maintains more than 200 million fingerprint records on file. Fingerprints have several advantages over other biometrics, such as the following: (i) High universality: A large majority of the human population has legible fingerprints and can

therefore be easily authenticated. This exceeds the extent of the population who possess passports, ID cards or any other form of tokens.

(ii) High distinctiveness: Even identical twins who share the same DNA have been shown to have different fingerprints, since the ridge structure on the finger is not encoded in the genes of an individual. Thus, fingerprints represent a stronger authentication mechanism than DNA. Furthermore, there has been no evidence of identical fingerprints in more than a century of forensic practice. There are also mathematical models (Pankanti, Prabhakar et al. 2002) that justify the high distinctiveness of fingerprint patterns.

(iii) High permanence: The ridge patterns on the surface of the finger are formed in the womb and remain invariant until death except in the case of severe burns or deep physical injuries.

(iv) Easy collectability : The process of collecting fingerprints has become very easy with the advent of online sensors. These sensors are capable of capturing high resolution images of the finger surface within a matter of seconds(Maio, Maltoni et al. 2003). This process requires minimal or no user training and can be collected easily from co-operative or non co-operative users. In contrast, other accurate modalities like iris recognition require very co-operative users and have considerable learning curve in using the identification system.

(v) High performance: Fingerprints remain one of the most accurate biometric modalities available to date with jointly optimal FAR (false accept rate) and FRR (false reject rate). Forensic systems are currently capable of achieving FAR of less than 10-4(NIST).

(vi) Wide acceptability: While a minority of the user population is reluctant to give their fingerprints due to the association with criminal and forensic fingerprint databases, it is by far the most widely used modality for biometric authentication.

The fingerprint surface is made up of a system of ridges and valleys that serve as friction surface when we are gripping the objects. The surface exhibits very rich structural information when examined as an image. The fingerprint images can be represented by both global as well as local features (Figure 2). The global features include the ridge orientation, ridge spacing and singular points such as core and delta. The singular points are very useful from the classification perspective. However, verification usually relies exclusively on minutiae features. Minutiae are local features marked by ridge discontinuities. There are about 18 distinct types of minutiae features that include ridge endings, bifurcations, crossovers and islands. Among these, ridge endings and bifurcation are the commonly used features. A ridge ending occurs when the ridge flow abruptly terminates and a ridge bifurcation is marked by a fork in the ridge flow. Most matching algorithms do not even differentiate between these two types since they can easily get exchanged under different pressures during acquisition. Global features do not have sufficient discriminative power on their own and are therefore used for binning and during intermediate steps before the extraction of the local minutiae features.

Figure 2: (a) Fingerprint image showing various type of ridge features (b) Core (c) Delta (d) Lake

(e) Island (f) Ridge ending (g) Ridge bifurcation

The various stages of a typical fingerprint recognition system is shown in Figure 3. The fingerprint image is acquired using off-line methods such as creating an inked impression on

paper or through a live capture device consisting of an optical, capacitive, ultrasound or thermal sensor (Maio, Maltoni et al. 2003). The first stage consists of standard image processing algorithms such as noise removal and smoothening. However, it is to be noted that unlike regular images, the fingerprint image represents a system of oriented texture and has very rich structural information within the image. Furthermore, the definition of noise and unwanted artifacts are also specific to fingerprints. The fingerprint image enhancement algorithms are specifically designed to exploit the periodic and directional nature of the ridges. Finally, the minutiae features are extracted from the image and are subsequently used for matching.

Figure 3: Shows the various stages of a fingerprint recognition algorithm

Although research in fingerprint verification research has been pursued for several years now, there are several open research challenges still remaining, some of which will be addressed in the ensuing sections of this paper. FINGERPRINT ENHANCEMENT:NON-STATIONARY FOURIER FILTERING It can be seen from Figure 4 that the quality of fingerprint encountered during verification varies over a wide range. The robustness of the verification system depends on its ability to enhance poor quality images. General-purpose image processing algorithms are not very useful in this regard but serve as a preprocessing step in the overall enhancement scheme. A majority of the techniques are based on the use of contextual filters whose parameters depend on the local ridge frequency and orientation. The filters themselves may be spatial (O'Gormann and J.V.Nickerson 1989; Hong, Wang et al. August 1998) or based on Fourier domain analysis (B.G.Sherlock, D.M.Monro et al. 1994; Watson, Candela et al. 1994) .

Figure 4 . Fingerprint images of different quality. The quality decreases from left to right. (a) Good quality image with high contrast between the ridges and valleys (b) Insufficient distinction between

ridges and valleys in the center of the image (c) Dry print

The estimation of local ridge orientation and the local ridge frequency play an important role in all subsequent stages of a fingerprint verification system (B.G.Sherlock, D.M.Monro et al. 1994). There have been several approaches to estimate the orientation map of a fingerprint image. These include the use of gradients (Hong, Wang et al.1998), template comparison (Kawagoe and Tojo 1987), and ridge projection based methods. The orientation estimation obtained by these methods is noisy and has to be processed further. The local ridge frequency indicates the average inter ridge distance within a block. This can be estimated based on the projection sum (Hong, Wang et al. August 1998), or variation of gray level in different directions (Maio and Maltoni 1997). Both methods also depend upon the reliable extraction of the local ridge orientation. In this paper we present a new fingerprint image enhancement algorithm based on contextual filtering in the Fourier domain. The proposed algorithm is able to simultaneously yield the local ridge orientation and ridge frequency information. The algorithm is also able to successfully segment the fingerprint images. The proposed approach obviates the need for disparate and different algorithms used for fingerprint segmentation, ridge orientation estimation, ridge frequency estimation and image enhancement replacing it with a single unified approach. The fingerprint image may be thought of as a system of oriented texture. The direction of the ridges in each region is given by the local ridge orientation and the ridge spacing is given by the local ridge frequency. With the exception of the singularities such as core and delta any local

region in the fingerprint image has a consistent orientation and frequency. Therefore, the local region can be modeled as a surface wave that is characterized completely by its orientation φ and frequency f. This approximation model does not account for the presence of local discontinuities such as minutiae but has been found useful in Fourier domain based fingerprint image enhancement techniques. Figure 5 illustrates the validity of this model. A local region of the image can be modeled as a surface wave according to Eqn.(1) .

))sincos(2cos(),( φφπ yxfAyxi += (1)

Figure 5 (a) Local region in a fingerprint image (b) Surface wave approximation (c,d) Fourier spectrum of Fig. 2a, Fig. 2b. The symmetric nature of the Fourier spectrum arrives from the

properties of the Fourier transform for real signals.

The parameters of the surface wave (f, φ) may be easily obtained from its Fourier spectrum that consists of two impulses whose distance from the origin indicates the frequency and its angular location indicates the orientation of the wave (Figure 5). The surface wave model is only an approximation, and the Fourier spectrum of the real fingerprint images is characterized by a distribution of energies across all frequencies and orientations. To approximate each block by a single orientation and frequency, a probabilistic approximation is used. We can represent the Fourier spectrum in polar form as F(r,φ). We can define a probability density function f(r, φ) and the marginal density functions f(φ), f(r) as

∫ ∫

=

r

drdrFrF

rf

φ

φφφ

φ 2

2

),(

),(),( (2)

∫=

r

drrff ),()( φφ , ∫=φ

φφ drfrf ),()( (3)

We assume that the orientation φ is a random variable that has the probability density function f (φ). The expected value of the orientation may then be obtained by performing a vector averaging according to Eq.(4). The terms )2sin( φ and )2cos( φ are used to resolve the orientation ambiguity between orientations φ and (φ +180).

⋅

⋅=Ε

∫

∫−

φ

φ

φφφ

φφφφ

df

df

)()2cos(

)()2sin(tan5.0}{ 1 (4)

The average ridge frequency is estimated in a manner similar to the ridge orientation. We can assume the ridge frequency to be a random variable with the probability density function f(r) as in Eq. (3). The expected value of the ridge frequency is given by ∫ ⋅=Ε

r

drrfrr )(}{ (5)

The frequency map so obtained is smoothened by applying a 3x3 Gaussian mask. Furthermore, the fingerprint image may be easily segmented based on the observation that the surface wave model does not hold in regions where ridges do not exist. In the areas of background and noisy regions, there is very little structure and hence very little energy content in the Fourier spectrum.

2),(∑ ∑=

u vvuFE (6)

We define an energy image E(x, y) Eq. (6), where each value indicates the energy content of the corresponding block. The fingerprint region may be differentiated from the background by thresholding the energy image. We take the logarithm values of the energy to obtain a linear scale. Figure 6 illustrates the energy map obtained by this method.

Figure 6: Results of the Fourier domain analysis (a) Original Image (b) Energy map (c) Frequency map (d) Orientation map

During enhancement, the image is divided into 16x16 overlapping blocks that is filtered in the Fourier domain by a frequency and orientation selective filter whose parameters are based on the estimated local ridge orientation and frequency. However, block-wise approaches have problems around the singularities where direction of the ridges cannot be approximated by a single value. The bandwidth of the directional filter has to be increased around these regions. We use the directional histogram obtained in the estimation of the orientation image for this purpose. We assume that f(φ) is unimodal and centered around E{φ} and define the bandwidth as the angular extent where P{|φ-E{φ}|<φBW}= 0.5. Thus, the angular bandwidth of the filter adapts itself in regions of high curvature. The filters are defined as specified in (B.G.Sherlock, D.M.Monro et al. 1994) and are given by Eq (7). The filter H is separable in angular and frequency domains and is obtained by multiplying separate frequency and angular band pass filters of order n.

≤−

−=

−+=

=

otherwise 0

if 2

)(cos)(

)()(

)()(

)()(),(

2

2222

2

BWcBW

C

nBW

nBW

nBW

r

r

H

rrrrrrrH

HrHrH

φφφφ

φφπφ

φφ

φ

φ

(7)

n.orientatiomean :bandwidth,angular :bandwidth, radial: cBWBWr φφ

The results of the enhancement for different quality of fingerprints are shown in Figure 7. It can be seen that the quality of reconstruction is not affected even around the points of high curvature marked by the presence of the singularities.

Figure 7: Results of the contextual filtering algorithm on various types of fingerprints (a) Creases. (b) Poor ridge continuity (c) Poor ridge contrast.

FEATURE EXTRACTION: CHAIN CODE REPRESENTATION The representation of the fingerprint feature represents the most important decision during the design of a fingerprint verification system. The representation largely determines the accuracy and scalability of the system. Existing systems rely on one of the following representations (i) Minutiae features: This is by far the most widely used method of fingerprint representation. The template simply consists of a list of minutiae location and their orientations. However, many systems extract additional information such as minutiae type and secondary features such as pairs of closest neighbors and the geometrical features associated with this triplets(Jea, Chavan et al. 2004) , ridge count to nearest neighbors etc. (ii)Texture descriptors: In this approach the feature consists of texture descriptors such as the response of a circularly or rectangular tessellated image blocks to a bank of directionally selective filters (Jain, Prabhakar et al. 2000).

However, the two fingerprints have to be aligned prior to comparing the texture descriptor and this represents a major problem with this approach (iii) Raw image: In this method, optical correlators (Rodberg, Soutar et al. 1998; Venkataramani and Kumar 2003; Baze, Verwaaijen et al. 2000) are used to directly find the correspondence between raw images. However, correlation is not invariant to shift, rotation, scaling or distortion. Furthermore, this technique requires the entire image to be stored leading to large template sizes. For these reasons, minutiae features remain, by far, the most widely used representation and will be the subject of discussion in this paper. Minutiae feature extraction algorithms can be categorized as those based on gray level and those dealing with binarized images. Approaches working with binary images gray detect the minutiae after thinning the binary image. The binarization approaches include peak detection and adaptive thersholding. Ratha et al. (Ratha, Chen et al. 1995) proposed an adaptive flow orientation based segmentation or binarization algorithm. In this approach the orientation field is computed to obtain the ridge directions at each point in the image. To segment the ridges, a 16x16 window oriented along the ridge direction is considered around each pixel. The projection sum along the ridge direction is computed. The centers of the ridges appear as peak points in the projection. The ridge skeleton thus obtained is smoothened by morphological operation. Finally minutiae are detected by locating the end points and bifurcations in the thinned binary image. Approaches working with gray level images are mostly based on ridge following. Maio and Maltoni (Maio and Maltoni 1997) proposed a feature extraction algorithm that directly operates on gray scale image. This algorithm is based on tracking the ridges by following the location of the local maxima along the flow direction. The ridge line algorithm attempts to locate at each step, the local maxima relative to a section perpendicular to the local ridge direction. Unlike Ratha’s approach which is a point wise operation, Maio and Maltoni’s approach is based on ridge pursuit where each ridge is sequentially traced along its full length. While such approaches are fairly popular, minutiae extraction algorithms that are based on thinning are iterative, computationally expensive and produce artifacts such as spurs and bridges. With the ridge following approaches, the local maxima cannot be reliably located in poor quality images and therefore, false positives are still introduced. We propose a new feature extraction algorithm based on chain code contours. Chain codes are a loss less representation of contours and yield a wide range of information about the contour such as curvature, direction, length etc (Govindaraju, Shi et al. 2003). As the contour of the ridges is traced consistently in a counter-clockwise direction, the minutiae points are encountered as

locations where the contour has a significant turn. Specifically, the ridge end occurs as significant left turn and the bifurcation as a significant right turn in the contour. Analytically the turning direction may be determined by considering the sign of the cross product of the incoming and outgoing vectors at each point, (Figure 8). The product is right handed if the sign of Eq. (8) is positive and left handed if the sign is negative. The turn is termed significant only if their dot product, Tyxyx <=+ 2211 . The threshold T is chosen to have a small value. In practice a group of points along the turn satisfy this condition. We define the minutia point as the center of this group.

)sgn()sgn( 1221 yxyxPP outin −=×rr

(8) This feature extraction process results in two forms of errors. The enhancement process may introduce artifacts that are detected as spurious minutia or the feature extractor may miss some of the genuine minutiae. While nothing can be done about missing minutiae, spurious minutiae can be eliminated by including a post-processing stage after the feature extraction. There have been several efforts at eliminating spurious minutia though pruning techniques. These can be broadly classified into (i) Structural post processing and (ii) Gray level image based filtering.

Figure 8 (a) Minutiae marked by significant turn in the contour (b) Left turn (b) Right turn

Structural post processing methods prune spurious minutia based on heuristics rules or ad hoc steps specific to the feature extraction algorithm. Xiao and Raafat (Xiao and Raafat 1991) provided taxonomy of structures resulting from thinning that lead to spurious minutia and proposed heuristic rules to eliminate them. Hung (Hung 1993) proposed a graph-based algorithm that exploits the duality of the ridges and bifurcation. The binarization and thinning is carried on positive and negative gray level images resulting in ridge skeleton and its complementary valley skeleton. Only the features with a corresponding counterpart are retained while eliminating the false positives. Gray scale based techniques use the gray scale values in the immediate neighborhood to verify the presence of a real minutia. Prabhakar et al (Prabhakar, Jain et al.

2000) proposed a gray scale image based approach to eliminate false minutiae. A 64x64 block surrounding a minutia is taken and is normalized with respect to orientation, brightness and variance. The block is then filtered using horizontally oriented Gabor filter. The central 32x32 pixels are taken as features and the resulting 1024 dimensional vectors is used to train a supervised classifier based on Learning Vector Quantization. Maio and Maltoni (Maio and Maltoni 1998) also proposed a neural network based approach for minutiae filtering relying on gray scale features. The minutia and non-minutia neighborhoods are normalized with respect to orientation and are passed to a multi-layer shared weights neural network that classifies them as ridge ending, bifurcation or non-minutia We use a set of simple heuristic rules to eliminate false minutiae (i) We merge the minutiae that are within a certain distance of each other and have similar angles. This is to merge the false positives that arise out of discontinuities along the significant turn of the contour. (ii) If the direction of the minutiae is not consistent with the local ridge orientation, then it is discarded. This removes minutiae that arise out of noise in the contour. (iii) We discard all minutiae that are within a distance of the border of the fingerprint area. The border is determined based on the energy map. This rule removes spurious minutiae that occur along the border of the fingerprint image (iv) We remove the pair of opposing minutiae that are within a certain distance of each other. This rule removes the minutiae that occur at either ends of a ridge break.

Figure 9 Post processing rules, (a) Fingerprint image with locations of spurious minutiae marked (b)

Types of spurious minutiae removed by applying heuristic rules (i-iv)

We measure two quantities namely “Sensitivity“ and “Specificity“ to objectively evaluate the algorithm (B.G.Sherlock, D.M.Monro et al. 1994). The sensitivity and specificity indicates the ability of the algorithm to detect the true minutiae and reject false minutiae respectively. 150 fingerprint images were randomly selected from the FVC2002 DB1 database (FVC2002) for

evaluation. The ground truth minutiae for these images were marked using a semi-automated truthing tool. The minutiae features were then detected using our feature extraction algorithm and the sensitivity and specificity were measured over the entire database. We compared our performance with the NIST open source feature extractor (Garris, Watson et al. 2002). The algorithm was executed on a PC with AMD Athlon 1.1GHz processor running Windows XP. It takes an average of 0.98s over the test images. We found that the number of true positives due to the proposed methods exceeds the number of true positives found by the NIST method in 110 of the 150 test images. Furthermore, the number of minutiae that whose type were flipped during feature extraction is less in 110 of the 150 test images when compared to the NIST method. Table 2 and Table 1 provide summary results of the objective evaluation.

Table 1 Summary Results

Metric NIST Proposed

Sensitivity(%) 82.8 83.5

Specificity(%) 77.2 76.8

Flipped(%) 12.0 10.9

Table 2 Feature Extraction Results over Sample Images from DB1

(TP – True positives, FP- False positives, M – Missing minutiae, F – Flipped) File Name

NIST Proposed method

Actual TP FP M F TP FP M F 10_8.tif 18 16 8 2 1 17 0 1 1 11_6.tif 50 40 4 10 2 41 4 9 4 12_8.tif 29 22 5 7 3 22 3 7 1 13_6.tif 35 28 10 7 4 28 10 7 2 14_6.tif 44 34 12 10 6 37 13 7 5 15_7.tif 38 37 7 1 5 37 3 1 0 16_7.tif 41 35 12 6 5 36 8 5 8 17_6.tif 43 35 16 8 11 36 7 8 11

18_8.tif 34 31 7 3 4 32 6 2 1 19_7.tif 35 26 8 9 3 31 6 4 5

PARTIAL FINGERPRINT RECOGNITION Matching incomplete or partial fingerprints continues to be an important challenge today, despite the advances made in fingerprint identification techniques. While the introduction of compact silicon chip-based sensors that capture only part of the fingerprint has made this problem important from a commercial perspective, there is also considerable interest in processing partial and latent fingerprints obtained at crime scenes. When the partial print does not include structures such as core and delta, common matching methods based on alignment of singular structures fail. We present an approach that uses localized secondary features derived from relative minutiae information. A flow network-based matching technique is introduced to obtain one-to-one correspondence of secondary features. Our method balances the tradeoffs between maximizing the number of matches and minimizing total feature distance between query and reference fingerprints. We also present a novel approach for similarity score computation. A two-hidden-layer fully connected neural network is trained to generate the final similarity score based on minutiae matched in the overlapping areas. Since the minutia-based fingerprint representation is an ANSI-NIST standard, our approach has the advantage of being directly applicable to existing databases. We present results of testing on FVC2002’s DB1 and DB2 databases. Matching small (partial) fingerprints to full pre-enrolled images in the database has several problems: (i) the number of minutia points available in such prints is few, thus reducing its discriminating power; (ii) loss of singular points (core and delta) is likely and therefore, a robust algorithm independent of these singularities is required; and (iii) uncontrolled impression environments result in unspecified orientations of partial fingerprints, and distortions like elasticity and humidity are introduced due to characteristics of the human skin. There are two major types of features that are used in fingerprint matching: local and global features. Local features, such as the minutiae information and our secondary features, containthe information that is in a local area only and invariant with respect to global transformation. Moreover, localized features have the ability to tolerate more distortions. Kovács-Vajna (Kovács-Vajna 2000) has shown that the geometric deformations on local areas can be more easily controlled than global deformations.

Jiang and Yau (Jiang and Yau 2000) use relative distance, radial angle, and minutia orientation along with the ridge count and minutia type to generate the features for local matching. The secondary features that we use are similar but the minutiae type and ridge count elements are removed. Minutiae types are difficult to distinguish when impression pressure varies on different applications (Figure 10). Furthermore, ridge count is not universally available and not all minutiae representations in existing databases contain this information.

(a)

(b).

Figure 10. The same minutiae extracted from two different impressions. In (a) it appears as a bifurcation but a ridge ending in (b).

We generate a five-element secondary feature vector (Figure 11). For each minutiae Mi (xi, yi, θi) and its two nearest neighbors N0 (xn0, yn0, θn0) and N1 (xn1, yn1, θn1), we construct a secondary feature vector Si (ri0, ri1, φi0, φi1, δi) in which ri0 and ri1 are the Euclidean distances between the central minutia Mi and its neighbors N0 and N1 respectively. φik is the orientation difference between Mi and Nk, where k is 0 or 1. δi represents the acute angle between the line segments MiN0 and MiN1. Note that N0 and N1 are the two nearest neighbors of the central minutia Mi and

ordered not by their Euclidean distances but by satisfying the condition: 010 ≥× ii MNMN

Figure 11 Secondary feature of Mi. Where ri0 and ri1 are the Euclidean distances between central

minutia Mi and its neighbors N0 and N1 respectively. φik is the orientation difference between Mi and Nk where k is 0 or 1. δi represents the acute angle between the line segments MiN0 and MiN1.

N0 is the first and N1 is the second minutia that we encounter when we traverse the angle, ∠N0MiN1. This arrangement is again different from the feature vector proposed by (Jiang and Yau 2000) where the Euclidean distance to the central minutiae orders neighboring minutiae. However, this increases the chance of flipping the order of the neighboring minutiae. Distortions are inevitable when mapping a 3-dimensional fingertip onto a 2-dimensional plane. These can be caused by vertical pressures, shear forces and varying impression conditions. As values of ri0 and ri1 increase, we observe that a secondary feature, Si (ri0, ri1, φi0, φi1, δi), has larger distortions of φi0, φi1 and δi. Thus, we make the assumption that the distortions of distance are less when the values of ri0 and ri1 are small. However, the distortions of the angle and orientation tend to be larger when ri0 and ri1 are small. Due to these factors, it is reasonable to adjust the tolerance areas according to the values of ri0 and ri1. A tolerance area is decided by three threshold functions Thldr(·), Thldδ(·), and Thldθ(·). The distance thresholds (decided by Thldr(·)) should be more restrictive (smaller) when ri0 and ri1 are smaller and more flexible when ri0 and ri1 are larger. On the other hand, the thresholds on angles should be larger in order to allow large distortions when ri0 and ri1 are small, but smaller when ri0 and ri1 are large (Figure 12). Since the thresholds change with the length of the line segment of central minutia and its neighbors, we use functions Thldr(·) and Thldδ(·) instead of fixed numbers to represent the thresholds. Threshold Thldθ(·) is used for the orientation differences between the central minutiae and its neighbors. Thldθ(·) has the same characteristics as Thldδ(·).

Figure 12 Dynamic tolerance areas. The gray areas around the two neighbors, N0 and N1 of Mi, are

decided according to the thresholds Thldr(ri0), Thldδ(ri0), Thldr(ri1), and Thldδ(ri1).

We use different matching schemes according to the number of minutiae on the query (I) and the reference (R). A ‘full’ fingerprint implies an image that is about 0.5”x0.7” and usually leads to greater than α (a pre-defined threshold) minutiae. There are three matching scenarios: 1) both number of minutiae on I and R are less than α; 2) either I or R has number of minutiae less than α; and 3) both I and R contain more than α minutiae. In the first two cases, we have fewer minutiae on at least one fingerprint. Finding the two nearest neighbors to construct a secondary feature makes it difficult to discover a match when the fingerprint is small. In such cases, we match (by brute-force) all the feature points directly by examining all the possible solutions and finding the most matches. A brute-force matching technique tries all possible correspondences between the minutiae on query and reference fingerprints. This technique is usually very time-consuming. To make it practical, our system uses brute-force matching only when there are small numbers of minutiae, which commonly occurs when matching partial fingerprints. When, we have more than α minutiae, we use a secondary feature-based matching method instead of the brute-force method to improve speed and accuracy. More details of the matching approach are provided in (Jea and Govindaraju 2004).

iM

0N

1N

0ir

1ir

)( 0ir rThld

)( 1ir rThld)( 0irThldδ

)( 1irThldδ

Figure 13: Minutiae of I and R are aligned with respect to the reference point r. Minutiae from I

are denoted by Xs, and the minutiae form R are denoted as Os. If 2Im were matched with

1Rm , which

is the closest minutia to 2Im , then

2Rm would stay unmatched.

Matching the feature points on two fingerprints is equivalent to finding the correspondences between the feature points. The numbers of feature points on query and reference fingerprints are rarely equal and therefore not every feature point finds a matched feature point. Thus, obtaining an optimal pairing is not trivial even when two fingerprints are aligned. The most important rule of matching feature points is to guarantee one feature point can match to at most one feature point. To comply with this constraint, one can mark the minutiae that have already been matched to avoid matching it twice or more. But, it is hard to find the optimal pairing of the feature points. For example, given that the feature point (

2Im ) of a query fingerprint, I, can

fall within the tolerance area of more than one feature point of the template fingerprint, R. (Figure 13) The best pairing is the configuration that can maximize the final number of matched minutia pairs. A more sophisticated method should be used to obtain the optimum pairing.

1Tm

2Tm

1Im

2Im

r

Figure 14: Flow network representation of minutia matching problem. All the edges in the network have capacity 1. The edges between source node s and nodes from I have zero cost as well as the edges from R to sink node t. Costs of the edges between nodes from I and nodes from R are from the cost matrix c.

We translate the feature point-matching problem into a Minimum Cost Flow (MCF) problem. Suppose we have two sets of feature points from different fingerprint images (I and R) and they are already aligned with respect to a pair of reference points in each image. We add one extra point (node), say the source s, into the set of I and add the point (node), say sink t, into the set of R. We also set up the links (edges) between nodes by obeying the following rules:

• There is one and only one link that connects s to every point in the first set. • There is one and only one link that connects t to every point in the second set. • There is no link between the points within the same set. • There is exactly one link between every point in first set and every point in the second set. • Every link is associated with a capacity and a cost. The cost matrix ),(),( ji mmdistjic ′= , where INi ≤≤1 and RNj ≤≤1 , represents the costs of the edges between I and R. IN and RN are the numbers of feature points on I and R , respectively. The function, ),( badist , is the distance measure of two feature points, a and b, on I and T, respectively. For efficiency purposes, we remove the edge between mi and jm′ if mi is not in the tolerance area of jm′ . There is a total of 2++ TI NN (with the source s and sink t

s t

NI nodes from I NR nodes from R

1 (cij)

1 (0) 1 (0)

nodes) nodes in the network. In our application, the capacity on every edge is set to 1, and the costs associated with the edges that come from s and going to t are set to 0. The configuration of the fingerprint-matching problem is shown in Figure 14. The optimal flow value in this network is the number of matched feature points. Because the capacity of every edge is set to 1, there will be no two feature points on I that match with the same feature point of R and vice versa, thus, the one-to-one matching of feature points is guaranteed. Thus, solving the minimum cost flow problem of the generated flow network is equivalent to finding the maximum number of matched feature points (flow) with the minimum total feature distance (cost).

A traditional way to calculate the similarity scores for a minutiae-based system is n2/ (sizeI× sizeR). Where sizeI and sizeR represent the numbers of minutiae on query and reference fingerprints, and n is the number of matched minutiae on both prints. (Bazen and Gerez 2003) claim using 2n/( sizeI+ sizeR) to compute the similarity scores will give better results. In our observation, we found both methods are unreliable, especially when matching fingerprints of different sizes. We use the number of matched minutiae, the numbers of minutiae points on overlapping areas, and the average feature distances to calculate reliable similarity scores. A two-hidden-layer fully connected neural network is trained to take six values as input and return a similarity score between 0 and 1. Our experiments show about 1.21% and 0.68% improvement on minimum total error rate on the FVC 2002 DB1 and DB2 databases by simply using this similarity score calculation method. Further studies about score computation can be found in (Jea, Chikkerur et al. 2005) . The effects of the new score computation scheme are illustrated in Table 3 and Figure 15.

Table 3 Comparison of different similarity score calculations. Results are calculated on the images

from FVC2002 DB1 database. Images 57_2 and 57_4 are from the same finger but different impressions. Images 23_1 and 45_1 are from different fingers.

n sizeI sizeR RI sizesize

n2

RI sizesize

n+2 Heuristic NN

57_2 vs 57_4 10 41 21 0.12 0.32 0.55 0.999996

23_1 vs 45_1 14 39 45 0.13 0.35 0.18 0.005490

In order to test the influence of the size of partial fingerprints, we generated a series of partial fingerprint databases with different sizes (in percentage) at random positions from the FVC 2002 DB1 data set. For every fingerprint, we generate 5 partial fingerprint templates of each target size. The target sizes are 10%, 15%, 20%, 25%, 30%, 35%, 40%, 45%, 50%, 55%, 60%, 65%, 70%, 75%, 80%, 85%, and 90% of the fingerprint foreground areas. We tested the system by

matching the partial fingerprint templates of the second impressions against the first impression of every finger in DB1. Table 4 shows the performance over these different sizes.

90

91

92

93

94

95

96

97

98

99

100

0 2 4 6 8 10 12 14 16 18 20

False positive

True

pos

itive

NN Score Heuristic Score Figure 15: ROC graph of system testing result on FVC2002 DB2 database. With heuristic rules

for similarity scores, the system reaches the minimum total error rate at 3.17% (with FAR at 1.24% and FRR at 1.93%), and EER at 1.69%. With NN scores, the system reaches the minimum total

error rate at 2.49% (with FAR at 0.85% and FRR at 1.64%), and EER at 1.57%.

Table 4 System performance with difference sized images.

Size Avg. width Avg. height Avg. minu. num. FAR FRR EER

90 196.61 283.30 34.89 1.844871 1.090909 1.71 80 185.32 267.07 32.00 1.988324 0.909091 1.74 70 173.35 249.78 28.74 1.721435 1.818182 1.77 60 160.42 231.22 25.28 2.675563 2.363636 2.52 50 146.41 211.03 21.56 2.378649 3.636364 3.17 40 130.90 188.70 17.49 2.905755 7.454545 5.25 30 113.31 163.36 13.36 4.647206 13.09091 9.12 20 92.42 133.30 8.84 9.611343 19.63636 17.11 10 65.20 94.11 4.23 15.23937 45.63636 38.21

SECURING FINGERPRINT TEMPLATES Biometrics is secure but not secret. Fingerprints, face, voice can be captured covertly and used in replay attack. Unlike password and PINs that can be reset or replaced, biometrics signal and templates are indefinitely and uniquely associated with the user. If a biometric is stolen or lost, it is compromised forever across multiple applications and cannot be replaced. Also, since biometrics contains private information of individuals, there is a privacy concern regarding the misuse of such data. Once collected, biometrics may be used for other purposes without the user’s specific knowledge or approval. The possibility that a database with biometric data is compromised is one of the main concerns in implementing biometric identification systems. For authentication based on physical possessions, e.g., keys and badges, a token can be easily cancelled and the user can be reassigned a new one. Similarly, logical entities, such as user IDs and passwords, can be changed as often as required. Yet, a user has only a limited number of biometrics such as one face, ten fingers and two eyes! Another major concern is the possible sharing of such a database of biometrics signals with law enforcement agencies, commercial organizations for other applications without the user’s knowledge.

Figure 16 : Cancelable biometric using a non-invertible transformation function H().

Cancelable or private biometrics are proposed to alleviate such concerns. A cancelable biometric template is obtained by transforming the regular features using a non-invertible transform (Figure 16). The most commonly used non-invertible transform is a one-way hash function. Different transforms can be used in each application to protect the biometric data. Thus, even if the stored biometric is compromised; the person can be re-enrolled simply by changing the transformation.

The problem we are dealing with is well described in (Maio, Maltoni et al. 2003). Plaintext passwords are hashed, and only hash values are stored in the database and transmitted across networks with the original passwords not used in any part of the authentication process. We want to devise a similar way for biometric data, in particular fingerprint data, to be hashed, and the biometric identification to be performed using hashed biometric data. Previous attempts to solve this problem include works of (Soutar et al.,1998), which deal directly with images, and (Davida, Frankel et al. 1998) where error-correcting codes were used. In our work we use ideas similar to (Jea, Chavan et al. 2004) to combine results of localized matches into the whole fingerprint recognition algorithm.

Figure 17 Problems associated with hashing fingerprint features

The main difficulty in producing hash functions for fingerprint minutiae is the inability to somehow normalize fingerprint data, for example, by finding specific fingerprint orientation and center. If fingerprint data is not normalized, then the values of any hashing functions are destined to be orientation/position- dependent. The way to overcome this difficulty is to have hash functions as well as matching algorithm deal with transformations of fingerprint data. . In this paper we present a method of hashing fingerprint minutia information and performing fingerprint identification in a new space. Only hashed data is transmitted and stored in the server database, and it is not possible to restore fingerprint minutia locations using hashed data.

We represent minutia points as complex numbers }{ ic . We assume that two fingerprints of the same finger can have different position, rotation and scale, coming from possibly different scanners and different ways to put the finger on scanner. Thus the transformation of one

fingerprint to the other can be described by the complex function trzzf +=)( . In our approach we construct hash functions and corresponding matching algorithm, so that this transformation function is taken into account. Additionally we cannot set specific order of minutiae, so we want our hash functions be independent of this order. Thus we consider symmetric complex functions as our hash functions.

Figure 18 (a)Complex representation for a minutiae point (b) Effect of affine transformation

Given n minutia points },,,{ 21 nccc K we construct following m symmetric hash functions

mn

mmnm

nn

nn

cccccch

cccccch

cccccch

+++=

+++=

+++=

KK

KK

KK

2121

222

21212

21211

),,,(

),,,(

),,,(

(9)

If another set of minutia points is obtained by applying trzzf +=)( , trccfc iii +==′ )( , then

K

KK

KK

KKK

KK

KKK

2211212

2

221

222

21

2

222

21

222

21212

21121

2121211

),,,(2),,,(

)(2)(

)()()(),,,(

),,,()()()()(),,,(

ntcccrthccchrntcccrtcccr

trctrctrccccccch

ntcccrhntcccrtrctrctrccccccch

nn

nn

nnn

nn

nnn

++=

++++++++=

++++++=′++′+′=′′′

+=++++=++++++=′++′+′=′′′

(10)

Given values of hash functions of two minutia sets, ),,,( 21 nii ccchh K= and ),,,( 21 nii ccchh ′′′=′ K , it is possible using first two relations in (10) to find transformation

parameters r and t. Other equations can be used to verify the similarity of two minutia sets using found parameters r and t. During our experiments we found that the better way to verify the matching between two minutia sets using symmetric hash functions is to define some distance

function K+−−−′+−−′=′ |2|||),,,( 212

222111 ntrthhrhntrhhtrhhdist ii αα and find r

and t that give minimum of this distance function. The value of the distance function serves as a measure of matching confidence between two sets of minutiae. Since directions of minutia points are important information for fingerprint matching, similar symmetric functions of unit vectors of directions are also constructed and stored together with symmetric functions of minutia positions. Thus distance formula above actually contains also terms with hash values of minutia directions. Usually since fingerprints have different minutia points it is not possible to use above described matching method for global matching. Instead we generate localized minutia subsets and perform matching for these subsets. The results of local matching are subsequently combined together. So far we have done experiments with following configurations: • n=2, m=1: for each minutia point we find its nearest neighbor, and

2)(),( 21

211cccch +=

• n=3, m=1: for each minutia point we find two nearest neighbors and

3)(

),,( 3213211

cccccch ++=

• n=3, m=2: for each minutia point find three nearest neighbors, and for each minutia triplet including original minutia point construct 2 hash functions as in (2) (and similar formulae for directions). We compared performance with fingerprint matching algorithm developed in (Jea and Govindaraju 2004) and using same set of fingerprints with identically extracted minutiae points. Also, since in configurations 1 and 2 we simply get another set of minutia points, we used matching algorithm of (Jea and Govindaraju 2004) to perform matching. We used fingerprint database of FVC2002 with 2800 genuine tests and 4950 impostor tests.

Although current performance of fingerprint matching algorithm with hash functions is worse than referenced algorithm employing all information about minutia points, we expect to achieve similar performance with additional algorithm and parameter optimization. Furthermore, the current choice of hash functions is determined by simplicity and ability to get good performance. Generally, in case stored hash information is compromised, it will be possible to choose different hash functions, and reenroll all persons with new hash values. Or each person can have its own set of hash functions for additional security. The main purpose of the proposed algorithm is to conceal the original fingerprint minutiae locations from an attacker. Since the number of hash

values generated by each set is less than the number of minutiae in the set, it is not possible to recover the original points even when we have full knowledge of the hashed locations. CONCLUSION In this paper we presented the recent research at CUBS that advance the state of the art in fingerprint recognition. We presented a new non-stationary filtering algorithm to enhance the fingerprint image. It was shown that the algorithm is capable of adapting to the high curvature regions around the singular points. Furthermore it was shown that the algorithm is capable of simultaneously extracting orientation, frequency and segmentation information through Fourier domain analysis. We also presented a novel feature extraction approach based on chain code contour analysis. Unlike thinning algorithms that shift the minutiae locations, the chain code algorithm provides exact localization of the minutiae positions. We described a new approach for matching partial prints based on secondary features and outlined a novel approach for score computation. Further we demonstrated the efficacy of the proposed approached through objective evaluation over FVC 2002 databases. We described the first successful implementation of cancelable fingerprint templates based on symmetric hash functions and showed that it increases the security of the template at acceptable loss in accuracy. Our current research is focused on real time matching for large-scale national civilian databases and also improving the performance of the symmetric hashing functions. REFERENCES

1. B.G.Sherlock, D.M.Monro and K.Millard (1994). Fingerprint Enhancement by directional Fourier Filtering. Visual Image Signal Processing. 141: 87--94.

2. Baze, A. M., G. T. B. Verwaaijen, S. H. Garez, L. P. J. Veelunturf and B. J. v. d. Zwaag (Nov 2000). A correlation-based fingerprint verification system. ProRISC2000 Workshops on Circuits, Systems and Signal Processing.

3. Bazen, A. M. and S. H. Gerez (2003). "Fingerprint matching by thin-plate spline modeling of elastic deformations." Pattern Recognition 36: 1859-1867.

4. Davida, G. I., Y. Frankel and B. Matt (1998). On Enabling Secure Applications Through Off-line Biometric. Symposium on Security and Privacy.

5. FVC2002 "Fingerprint Verification Competition." 6. Garris, M. D., C. I. Watson, R. M. McCabe and C. L. Wilson (2002). "User’s Guide to

NIST Fingerprint Image Software (NFIS)." (NISTIR 6813).

7. Govindaraju, V., Z. Shi and J. Schneider (2003). Feature Extraction Using a Chaincoded Contour Representation. International Conference on Audio and Video Based Biometric Person Authentication.

8. Hong, L., Y. Wang and A. K. Jain (August 1998). "Fingerprint Image Enhancement: Algorithm and Performance Evaluation." Transactions on PAMI 21(4): 777--789.

9. Hung, D. C. (1993). "Enhancement and Feature Purification of Fingerprint Images." Pattern Recognition 26(11): 1661--1671.

10. Jain, A., S. Prabhakar, L. Hong and S. Pankanti (May 2000). Filterbank-based fingerprint matching. Transactions on Image Processing. 9: 846--859.

11. Jea, T., V. K. Chavan, V. Govindaraju and J. K. Schneider (2004). Security and matching of partial fingerprint recognition systems. Proceeding of SPIE: 39--50.

12. Jea, T., S. Chikkerur and V. Govindaraju (2005). A Study on the Generation of Similarity Scores for Minutiae-based Fingerprint Matchers. AVBPA 2005.

13. Jea, T. and V. Govindaraju (2004). "A minutia-based partial fingerprint recognition system." Pattern Recognition.

14. Jea, T. Y., V. S. Chavan, V. Govindaraju and J. K. Schneider (2004). Security and matching of partial fingerprint recognition systems. SPIE Defense and Security Symposium.

15. Jiang, X. and W.-Y. Yau (2000). Fingerprint Minutiae Matching Based on the Local And Global Structures. International Conference on Pattern Recognition: 1038--1041.

16. Kawagoe and Tojo (1987). "Fingerprint Pattern Classification." Pattern Recogntion 17(3): 295--303.

17. Kovács-Vajna, Z. M. (2000). "A Fingerprint Verification System Based on Triangular Matching and Dynamic Time Warping." Transactions on Pattern Analysis and Machine Intelligence 22(11): 1266-1276.

18. Maio, D. and D. Maltoni (1997). "Direct Gray Scale Minutia Detection in Fingerprints." Transactions on PAMI 19(1).

19. Maio, D. and D. Maltoni (1998). Neural Network based Minutiae Filtering in Fingerprint Images. 14th International Conference on Pattern Recognition: 1654-1658.

20. Maio, D., D. Maltoni, A. K. Jain and S. Prabhakar (2003). "Handbook of Fingerprint Recognition."

21. NIST "Fingerprint Vendor Technology Evaluation." 22. O'Gormann, L. and J.V.Nickerson (1989). "An Approach to Fingerprint Filter Design."

Pattern Recognition 22(1): 29--38. 23. Pankanti, S., S. Prabhakar and A. K. Jain (2002). "On the Individuality of Fingerprints."

Transactions on PAMI 24(8): 1010--1025.

24. Prabhakar, S., A. Jain, J. Wang, S. Pankanti and R. Bolle (2000). Minutiae Verification and Classification for Fingerprint Matching. International Conference on Pattern Recognition. 1: 25--29.

25. Ratha, N., S. Chen and A. K. Jain (1995). "Adaptive Flow Orientation Based Feature Extraction in Fingerprint Images." Pattern Recognition 28: 1657--1672.

26. Rodberg, D., C. Soutar and B. V. K. V. Kumar (1998). High-speed fingerprint verification using an optical correlator. Optical Pattern Recognition IX. 3386: 123--133.

27. Venkataramani, K. and B. V. K. V. Kumar (August 2003). "Fingerprint Verification Using Correlation Filters." Lecture Notes in Computer Science 2668: 886-894.

28. Watson, C. I., G. T. Candela and P. J. Grother (1994). "Comparison of FFT Fingerprint Filtering Methods for Neural Network Classification." NISTIR 5493.

29. Xiao, Q. and H. Raafat (1991). Combining Statistical and Structural Information for Fingerprint Image Processing Classification and Identification. Pattern Recognition: Architectures, Algorithms and Applications: 335-354.