Face Identification by Deformation Measure

8/2/2019 Face Identification by Deformation Measure

1/5

Face Identification by Deform ation Me:asureBertrand LEROY' Isabelle L. H E K I N ' Laurent D. COHEN2

'INRIA - Projet A IR , Rocquencourt - B.P. 105,78153Le Chesnay Cedex, France.'Universitk Paris IX Dauphine - CEREMADE,Place du M ark ha1 de Lattre De Tassigny - 75775 Paris Cedex 16.E-mail:Bertrand.Leroy~inria.fr,Isabelle.Heirlin~inria.fr,

[email protected]

AbstractThis paper studies the problem of ace identification or

the particular application of an automatic cash machinewithdrawal: the problem is to decide i fa person identifyinghimself by a secret code is the same person registered in thedata base. The identification process consists of three mainslages. The localization of salient features is obtained byusing morphological operators and spatio-temporal infor-mation. The location o these features are used to achievea normalization of the fac e image with regard to the cor-responding ace in the data base. Facial features, such aseyes, mouth and nose, are extracted by an active contourmodel which is able t o incorporate information about theglobal shape of each object. Finally the identijication isachieved by ace warping including a deformation measure.

1. IntroductionResearch studies related to face recognition from 2-Dfrontal views may be shared in two main classes accordingto whether they refer to a feature based approach or to a pixelapproach. The first class consists in extracting the positionof several facial features in order to restrict the data space.Several statistical methods, generally classical algorithmsof pattern recognition, are then used to identify faces usingthese measures [131. The pixel approach is a global onethat requires few knowledg e about face geometry. Somestudies are linked to pattern m atching: the measures usedto identify faces are correlations between template re,' ions.Other methods use principal component analysis applied on

pixel's grey level values [121.The app lication, described in this paper, is a system simi-lar to an autom atic cash machine withdraw al where the useridentifies himself by using a credit card with a secret code.It is possible to enhance the secu rity of such a transactionby verifying that the observed face is identical to a stored

1015-4 651/96 $5.00 0 1996 IEEEProceedings of ICPR '96

face of the same person or to compare the face to informa-tion (points, curves, quantitative values, . . obtained bylearning and stored in the data base.We propose an identity check sy stem using warping be-tween faces as a measure of likeness. Kn owing the assumedidentity of a face (supposed by the secret code), the systemcompares this face with that on e in the database correspond-ing to the same person. This is achieved by warping oneface in order to fit the other one.Since we have chos en to use the geometry and the loca-tions of facial features (such as the mouth, no se and eyes) toachiev e warp ing, each1 perso n of the d atabas e is representedby this information.The face identification scheme consists of three mainstages . In the first step, in section 2, facial features arelocated and the image is normalized so that features locationmatch those of the face Corresponding to this person in th edatabase. In section 3 , he features bo undaries are extracted

using a s pecific active: contour mod el. This model is basedon Fourier descriptors and is able to incorporate informationabout the global shape of each object. The last stage isdescribed in section 4 and consists in measuring the warpingbetween the normalizled face and the correspond ing face inthe database.2. Face normalization

In the con text of our application, the image acquisitionis achieved with a fixed lightning and is acquired from anear frontal view throughout the temporal sequence. Thenormalization process, is based on this constraint.The first step of our facial features localization processconcerns the eyes. The nose and the mouth will be furtherdetected using some facial geometry hypothesis.The presence of spec ular spots (usually in the neighbor-hood of the irises) due to the direct lightning allows to locatethe eyes using a "peak" morphological operator which has

63 3

Authorized licensed use limited to: UNIVERSITY TEKNOLOGY PETRONAS. Downloaded on October 15, 2009 at 23:49 from IEEE Xplore. Restrictions apply.
mailto:[email protected]:[email protected]


2/5

the effect of bringing out the local maxima of the grey levelvalues function. A rough estimation of the eyes positionis consequently obtained by looking for the two regionscorresponding to the highest values of the peak image.Since the lightning used for the acquisition is frontal, theseestimations correspond approxim ately to the irises positions.In order to locate the nose and the mouth, we then usea spatio-temporal information by computing the su m of thespatio-temporal gradient norm o ver the sequence:

I , I

where I ( z , : t) represents the grey level value, TC an d ythe spatial compo nents, t the temporal component and N th enumbe r of images of the sequence. p is a normalization termdepending on the temporal sampling of the sequence and onthe motion of the different featur es during the sequence.

Figure 1. Opposite of the sum of the spatio-temporal gradient norm calculated on animage sequence.As it is shown on figure 1 , the image I,,, permits to pickup regions corresponding to a weak contrast but having ahigh deformation, like the mouth, and regions having a highcontrast like the shade due to the n ose relief. The localization

of the nose and the mouth in the image I,,, consists insearching, along the mediatrix of the segment joining thetwo irises, the regions having the two higher values of theimage I,,, . In order to minimize the computing time, theimage I,,, is computed in small windows on both sides ofthe mediatrix between the tw o irises. Figure 2shows theresults obtained by this algorithm and it can be seen that thelocalization of the lower part of the nose is obtained with agood accuracy. The point correspon ding to the position ofthe mouth depends on its motion during the sequence andis not always the same physical point. When the motionis important, this point is located within the mouth andotherwise it is generally located on the boundary of the lips.

In order to compute the image Isum,we use the first30 images of the sequence (with a sam pling rate of 25 hz).Using a shorter sequence leads to less accurate results forthe temporal information will be less significant.In next section we make use for the system of an activecontour model to extract facial features boundaries: in orderto use the sa me initialization a nd parameters for the active

Figure 2. Results of the localization of theeyes, nose and mouth.

contou r model with different images acquired from one face,we normalize the image.This is achieve by apply ing a similarity using the iriseslocation as references. Since the ac tive contour parametersand the curves used for initialization are not the same fordifferent faces, these features will be stored in the databasewith the facial characteristics.3. Facial feature extraction

An active contour model is an energy minimizing curveguided by internal constraints and influenced by imageforces that pull it towards ima ge edges. Sin ce the globalregularity constraints used by generic active contour mod-els (such as snakes [ 8 ] [ 3 ] ) re included w ithin the internalenergy function, these models do not allow to refine thespace of admissible curves to a specific shape class. In or-der to take into account the s hape of facial features, severalmethods based on deformable templates have been devel-oped.Yuille, Cohen and Hallinan [131 proposed a set of de-

formab le templates for this purpose. Each one of thesetemplates is designed to extract a specific feature. For ex-ample the template used to extract the eye boundary consistsof a circle bounded by two parab olas and possesses 11 pa-rameters.Craw, Tock and Bennett [6] escribed amod el which con-sists of a set of 40 landmark points, each one representingthe position of a particular part of the structure to be located.Several algo rithms are designed to deal with a particular fea-ture using a statistical know ledge of th e landmark positions.A global algorithm controls the interaction between them.A similar method has been proposed by Cootes, Tailor andLanitis [4] using statistically based models which iterativelymov e towards structures in im ages after a learning processon similar images.In this section we propose a parametrical active contou rmod el using Fourier descriptors. Th e aim is to design amodel sufficiently general to extract the boundary of sev-eral facial features while taking into account the specificgeometry of each feature. This model needs to tune few

634



3/5

parameters.Actually, this model is able to extract facial features su b-jected to high deformation (in order to take lips motion intoaccount) and having corners (like the mouth and the eyes).3.1.Active contour model using Fourier descriptors

The use of F ourier descriptors for active contou r modelshas been introduced by Staib and D uncan [111 in o rder toextract an object boundary . Their method is based on the useof probability distributions on the parameters. In order tobe less sensitive to the initial parameter value, we propose avariational app roach sim ilar to the method used in the snakemodel [8].An elliptic Fourier representation of a closed curve is aparametrical cur ve v defined by:

where AI, s a 2x2 matrix, N the number of harmonics usedto describe the curve and 8 the angular parameterizationindex.Such a class of curves does not permit to represent effec-tively the boundaries of objects having an irregularity pointwithin a regular section, such as the corner of the mouth oreyes. Thus, the extraction of this kind of boundary requiresthe use of high frequency harmonics which h ave the effectof creating oscillations. I n order to solve this problem wepropose a new model mad e of two open curves joined attheir ends:

where V A and V B respect the conditions:V A ( O ) = vg(27r) andvA(7r) = V B ( ~ ) .

In order to use orthogonal bases on the interval [O , 7r] , anopen curve can b e defined by:

where a k an d b k are the parameters model, N is thenumber of harmonics used to describe the curve and R,$, sthe rotation matrix of angle 4. Thus, the phase shift betweenthe two bases used fo r ~ ( 0 )nd ~ ( 0 )ermits to describe ahalf-ellipse with four parameters.The curve modeling the boundary of the object, is ob-tained by minimizing a functional similar to the snake en-ergy:

E(V) = P (UA ( 8 ) )+ x(v;)'d91;+ P (VB ( 8 ) )+ X(v&)2d8 .

The first term P inside each one of the integrals is animage potential equal to the opposite of the square of theimage gradient (P = -1VII') and the second one is anelasticity term associated to the curv e tension.In order to extract boundaries of the mouth and eyes weuse one harmonic to describe the lower boundary, VB andN harmonics for the upper boundary, V A . Thus V A has theform defined in equation (2 ) nd VB s defined by:

Toextract the loweir boundary of the nos e, w e use a singleopen curve described by equation (2).3.2. Results

The algorithm used to extract features is a fine to coarsescheme (see figure 3 ) . Defining two half-ellipses as ini-tialization, the convergence is first achieved using a singleharmonic for the curve V A , hen the first result is used as aninitialization to get a curve with two h armonics u ntil we ob-tain a cur ve VA described by N harmonics fitting the objectboundary.As the number of harmonics increases, the regularity

constraints are smaller and the boundary can be describedmore precisely with a better stability.4. Face Warping for identification

Several methods have been proposed to achieve facerecognition using features location. So me studies use thisinformation in order to calculate the correlation betweenthe regio ns corresponlding to a same feature in two differentimages [ 11. The measure is then equal to the mean of thecorrelation values. Other methods utilize a mo re geomet-rical approach by defining a discriminating measure usingphysiological knowledge [7],results of principal componentanalysis [ 5 ] or topological graphs 191.In this paper, two faces are compar ed by a deformationmeasure. Given a set of landmark points on the face, themeasure is a quantification of the bending that has to beapplied to the face, so that its landmark points match theones of the other face. To thisissue, we utilize a class of

635



4/5

database we use a sequence of images of people utteringa particular sentence while blinking. Then , on each imageof the seq uence , the features boundaries are extracted on anormalized image (as it is shown in sections 2 and 3) an dfor each landmark poin t we calculate the distribution of itsposition. Normality tests showed that these distributionsmay be represented by Gaussian laws. Thu s, a person isrepresented in the database by a set of 4 values:

{G,f s z , YZ l f f Y 3 1where ( x 2 , z) is the mean position of the ith landmarkpoint and (gSz,Y3) ts standard deviation on the horizontal

Figure 3. initialization (first column) and re-sul ts obtained with 1 (se con d column) ndthen 3 (third co lumn) harmonics for the upperboundary of mouth and eyes and the lowerboundary of n o s e .

and vertical axes.4.2. Deformation measure

Bookstein [2] show ed the interest of the thin plate splinesfor image registration purpose sinc e they can be used to solvethe problem of scattereddata interpolation while minimizingthe warping.

The total amount of bending of a surface (z ,y, f(z,))(f being a twicely differentiable function f : IR2 -+ E ) ,can be quantified by the value of the functional:

deformations similar to the one used for normalization byReisfeld [lo]. While his aim was to normalize faces inorder to apply a classification on the image data, we proposeto achiev e the identification using the deformation m easurevalue as the discriminating information.4.1. Face characterization and database

The landmark points used to represent the face are lo-cated on the the features boundaries which have been ex-tracted as described in section 3. Using all the points of theboundaries for the matching process will be useless since theinformation is redundant (like neighbor points on the sam ebound ary) and some points cannot be easily extracted fromthe curves describing the boundaries (du e to the parameter-ization which is not intrinsic). We made the ch oice of 15points displayed on figure 4. These points characterize theposition of the features as well as their global geometries.

Figure 4. point s characterizing a fac eHowever some of the chosen landmark points are mo-bile since they are located on a feature which is subject totemporal deformation, There fore, in order to construct the

Given a set of landmark points { p , = ( z i , y z ) } on oneface and { p i = (x:,yL)} on a second one, the functionT = (T,,T,) : R2+ R2,hat matches each point p , top i with respect to its standard deviation (oz2, y%)nd thatis least ben t according to the functional (3), minimizes thefunctional:

X is a positive weighting param eter and h is defined by:h ( z )= 2 if 121> 1 and 1 otherwise.

Thus , a weight is associated to each landmark point whichdecreases the influence of a point if its standard deviationis large. T he function h is constructed so that J r is alwaysdefined and the points having small standard deviation arenot too influent.

The transformation T , hat minimizes J T , has the fol-lowing form:

636



5/5

References

Several types of function U can be used in order to ob-tain a unique minimum for JT . In particular the use ofU ( z )= x2 og(z) (with U(O)=O) leads to an unique solu-tion verifying the prop erty of the thin plate spline. Th ecoefficients a l , a2 , a3 , b l , bz, b3, w1 , . . . W N an d z1 . . Z Ncan be calculated by solving a linear system as describedin [2] [lo].

The quantity J ( T )may be used to measu re the likenessbetween two faces. Th e assumed identity w ill then be con-firmed if this value is sma ller than a given threshold. Th ethreshold value is linked to the accu racy of the system thatis wished for. If it is small, it can hapen that the identity o fa person will be w rongly rejected; if it is larg e, the identifi-cation test will then be less restrictive. Lastely, the cho iceof the treshold value must take sev eral parameters into ac-count such as the image acquisition and the accuracy of thelocalization and ex traction processes.

5. ConclusionThis paper presents a set of methods that are used to co n-struct an identity check system. Th e localization of facialfeatures (such as the eyes, nose and mouth) is achieved us-ing a morphological operator as well as the spatio-temporal

information. This metho d is robust provided that lightningis frontal. It is used to normalize the face ima ge with re-gard to the corresponding face in the data base. Th e activecontour model proposed for feature boundary extraction is aparametrically deformable model u sing Fourier descriptors.This model is designed so that few parameters are neededto extract irregular objects like the eyes or the mouth andthe stability is increased by applying a coars e to fine schemeto the convergence process. A face is represented in thedatabase by a set of landmark points b elonging to the fea-tures boundaries as well as the standard deviations of thepoints calculated on a s equence of images. Finally the like-ness between two faces is quantified using a deformationmeasure. This deformation corresponds to the bending tobe applied on a face, so that its landmark points match theones of the other face. To this issue, an enhance ment of astandard method used to interpolate scattered data has beenproposed which allows to associate a weight to each land-mark point so that points having a small stand ard deviationhave more effect on the bending.

[ I ] D. J. Beymer. Face recognition under varying pose. In IEEEProceedings of Computer vision and Pattern Recognition,pages 556-761, 1994.[2] F. L. Bookstein. Principal warps: Thin-plate splines anddecomposition of deformations.IEEE Transactionson Pat-tern AnalysisandMachine Intelligence, 11 6):567-585, June1989.[3] L. D. ohen and I. Cohen. F inite element methods for activecontour models and balloons for 2-D and 3-D images. IEEETransactionson Pattern Analysis and Machine Intelligence,15(11): 1131-1 147, November 1993.[4] T. F. Cootes, C. J. Taylor, and A. Lanitis. Multi-resolutionwith active shape models. In Proceedings of the IntemationalConference of Pa,ttem Recognition, pages 610-612, 1994.[5] N. Costen, I. Craw, G. Robertson, and S . Akamatsu. Auto-matic face recognition: What representation? In Proceed-ings of the First European Conferenceon Computer vision,1996.[6] I. Craw, D. Tock, and A. Bennett. Finding face features. InProceedingsof the First European Conferenceon ComputerVision,pages 92-96, 1992.[7] T. Kanade. Picture processing system by computer complexand recognitionqf human faces. PhD thesis, Kyoto Univer-sity, Department of Information Science, November 1973.[8] M. Kass, A. Witkin, and D. Terzopoulos. Snakes: Activecontour models. In IEEE Proceedings of the IntemationalConference on Computer Msion, pages 259-268, London,June 1987.[9] B. Manjunath, R. Chellappa, and C. von der Malsburg. Afeature based approach to face recognition. In IEEE Pro-ceedingsofComp,uter isionand Pattem Recognition, pages

[101 D. Reisfe ld. Generalized Symmetry Transfoms: AttentionalMechanisms and Face Recognition. PhD thesis, Tel-AvivUniversity, January 1994.[ I I ] L. H. Staib and J . S . Duncan. Boundary finding with para-metrically deformable models. lEEE Transactionson Pat-tern Analysis and Machine Intelligence, 14(11):1061-1075,November 1992.[121 M. Turk. Interactive-time vision: face recognitionas a visualbehavior. PhD thes is, Massachusetts Institute ofTechnology,September 1991.[131 A. Yuille, D. Cohen, and P. Hallinan. Facial feature extractionby deformable templates. Technical Report 88-2, HarvardRobotics Laboratory, 1988.

373-378, 1992.

637

A h i d li d li i d UNIVERSITY TEKNOLOGY PETRONAS D l d d O b 15 2009 23 49 f IEEE X l R i i l

Face Identification by Deformation Measure

Documents

Transcript of Face Identification by Deformation Measure