Illumination-Based Image Synthesis: Creating Novel Images ofHuman Faces Under Di�ering Pose and Lighting�

Athinodoros S.Georghiades Peter N. Belhumeur David J. Kriegman

Center for Computational Vision and Control Beckman InstituteYale University University of Illinois, Urbana-Champaign

New Haven, CT 06520-8267 Urbana, IL 61801

AbstractWe present an illumination-based method for synthe-sizing images of an object under novel viewing condi-tions. Our method requires as few as three images ofthe object taken under variable illumination, but froma �xed viewpoint. Unlike multi-view based image syn-thesis, our method does not require the determinationof point or line correspondences. Furthermore, ourmethod is able to synthesize not simply novel view-points, but novel illumination conditions as well. Wedemonstrate the e�ectiveness of our approach by gen-erating synthetic images of human faces.

1 IntroductionWe present an illumination-based method for creat-ing novel images of an object under di�ering pose andlighting. This method uses as few as three imagesof the object taken under variable lighting but �xedpose to estimate the object's albedo and generate itsgeometric structure. Our approach does not requireany knowledge about the light source directions in themodeling images, or the establishment of point or linecorrespondences.

In contrast, nearly all approaches to view synthesisor image-based rendering take a set of images gath-ered from multiple viewpoints and apply techniquesakin to structure from motion [17, 28, 6], stereopsis[21, 9], image transfer [3], image warping [18, 20, 24],or image morphing [7, 23]. Each of these methodsrequires the establishment of correspondence betweenimage data (e.g. pixels) across the set. (Unlike othermethods, the Lumigraph [12, 19] exhaustively sam-ples the ray space and renders images of an object fromnovel viewpoints by taking 2�D slices of the 4�D light�eld at the appropriate directions.) Since dense corre-spondence is di�cult to obtain, most methods extractsparse image features (e.g. corners, lines), and mayuse multi-view geometric constraints (e.g. the trifocaltensor [2, 1]) or scene-dependent geometric constraints

�P. N. Belhumeur and A. S. Georghiades were supported bya Presidential Early Career Award, an NSF Career Award IRI-9703134, and ARO grant DAAH04-95-1-0494. D. J. Kriegmanwas supported by NSF under NYI, IRI-9257990, and by AROgrant DAAG55-98-1-0168.

[9, 8] to reduce the search process and constrain the es-timates. By using a sequence of images taken at nearbyviewpoints, incremental tracking can further simplifythe process, particularly when features are sparse.

For these approaches to be e�ective, there must besu�cient texture or viewpoint-independent scene fea-tures, such as albedo discontinuities or surface nor-mal discontinuities. From sparse correspondence, theepipolar geometry can be established and stereo tech-niques can be used to provide dense reconstruction.Underlying nearly all such stereo algorithms is a con-stant brightness assumption { that is, the intensity (ir-radiance) of corresponding pixels should be the same.In turn, constant brightness implies two seldom statedassumptions: (1) The scene is Lambertian, and (2) thelighting is static with respect to the scene { only theviewpoint is changing.

In the presented illumination-based approach, wealso assume that the surface is Lambertian, althoughthis assumption is very explicit. As a dual to the sec-ond point listed above, our method requires that thecamera remains static with respect to the scene { onlythe lighting is changing. As a consequence, geomet-ric correspondence is trivially established, and so themethod can be applied to scenes where it is di�cultto establish multi-viewpoint correspondence, namelyscenes that are highly textured (i.e. where image fea-tures are not sparse) or scenes that completely lacktexture (i.e. where there are insu�cient image fea-tures).

At the core of our approach for generating novelviewpoints is a variant of photometric stereo [27, 29,14, 13, 30] which simultaneously estimates geometryand albedo across the scene. However, the main limi-tation of classical photometric stereo is that the lightsource positions must be accurately known, and thisnecessitates a �xed lighting rig as might be possible inan industrial setting. Instead, the proposed methoddoes not require knowledge of light source locations,and so illumination could be varied by simply waivinga light around the scene.

In fact, our method derives from work by Belhumeurand Kriegman in [5] where they showed that a smallset of images with unknown light source directions can

be used to generate a representation { the illuminationcone { which models the complete set of images of anobject (in �xed pose) under all possible illumination.This method had as its pre-cursor the work of Shashua[25] who showed that, in the absence of shadows, theset of images of an object lies in a 3 �D subspace inthe image space. Generated images from the illumi-nation cone representation accurately depict shadingand attached shadows under extreme lighting; in [11]the cone representation was extended to include castshadows (shadows the object casts on itself) for ob-jects with non-convex shapes. Unlike attached shad-ows, cast shadows are global e�ects, and their predic-tion requires the reconstruction of the object's surface.

In generating the geometric structure, multi-viewpoint methods typically estimate depth directlyfrom corresponding image points [21, 9]. It is wellknown that without sub-pixel correspondence, stereop-sis provides a modest number of disparities over the ef-fective operating range, and so smoothness or regular-ization constraints are used to interpolate and providesmooth surfaces. The presented illumination-basedmethod estimates surface normals which are then in-tegrated to generate a surface. As a result, very subtlechanges in depth are recovered as demonstrated in thesynthetic images in Figures 4 and 5. Those imagesshow also the e�ectiveness of our approach in gener-ating realistic images of faces under novel pose andillumination conditions.

2 Illumination ModelingIn [5], Belhumeur and Kriegman have shown that, fora convex object with a Lambertian re ectance func-tion, the set of all images under an arbitrary combina-tion of point light sources forms a convex polyhedralcone in the image space IRn which can be constructedwith as few as three images.

Let x 2 IRn denote an image with n pixels of aconvex object with a Lambertian re ectance functionilluminated by a single point source at in�nity. LetB 2 IRn�3 be a matrix where each row in B is theproduct of the albedo with the inward pointing unitnormal for a point on the surface projecting to a partic-ular pixel in the image. A point light source at in�nitycan be represented by s 2 IR3 signifying the productof the light source intensity with a unit vector in thedirection of the light source. A convex Lambertian sur-face with normals and albedo given by B, illuminatedby s, produces an image x given by

x = max(Bs;0); (1)

where max(Bs;0) sets to zero all negative componentsof the vector Bs. The pixels set to zero correspond tothe surface points lying in an attached shadow. Con-vexity of the object's shape is assumed at this pointto avoid cast shadows. It should be noted that whenno part of the surface is shadowed, x lies in the 3-Dsubspace L given by the span of the columns of B.

If an object is illuminated by k light sources at in-�nity, then the image is given by the superposition ofthe images which would have been produced by theindividual light sources, i.e.,

x =

kXi=1

max(Bsi;0) (2)

where si is a single light source. Due to the inherentsuperposition, it follows that the set of all possible im-ages C of a convex Lambertian surface created by vary-ing the direction and strength of an arbitrary numberof point light sources at in�nity is a convex cone. It isalso evident from Equation 2 that this convex cone iscompletely described by matrix B.

This suggests a way to construct the illuminationmodel for an individual: gather three or more im-ages of the face without shadowing illuminated by asingle light source at unknown locations but viewedunder �xed pose, and use them to estimate the three-dimensional illumination subspaceL. This can be doneby �rst normalizing the images to unit length and thenestimating the best three-dimensional orthogonal basisB� using a least-squares minimization technique suchas singular value decomposition (SVD). Note that thebasis B� di�ers from B by an unknown linear transfor-mation, i.e., B = B�A where A 2 GL(3) [10, 13, 22];for any light source s, x = Bs = (B�A)(A�1s). Nev-ertheless, both B� and B de�ne the same illuminationcone and represent valid illumination models.

Unfortunately, using SVD in the above procedureleads to an inaccurate estimate of B�. For even aconvex object whose Gaussian image covers the Gausssphere, there is only one light source direction (theviewing direction) for which no point on the surface isin shadow. For any other light source direction, shad-ows will be present. If the object is non-convex, suchas a face, then shadowing in the modeling images islikely to be more pronounced. When SVD is used to�nd B� from images with shadows, these systematicerrors bias its estimate signi�cantly. Therefore, an al-ternative way is needed to �nd B� that takes into ac-count the fact that some data values should not beused in the estimation.

We have implemented a variation of [26] (see also[28, 16]) that �nds a basis B� for the 3-D linear sub-space L from image data with missing elements. Tobegin, de�ne the data matrix for c images of an indi-vidual to be X = [x1 : : :xc]. If there were no shad-owing, X would be rank 3 (assuming no image noise),and we could use SVD to factorize X into X = B�S�

where S� is a 3�c matrix the columns of which are thelight source directions scaled by the light intensities sifor all c images.

Since the images have shadows (both cast and at-tached), and possibly saturations, we �rst have to de-termine which data values are invalid. Unlike satura-tions which can be trivially determined, �nding shad-ows is more involved. In our implementation, a pixel is

assigned to be in shadow if its value divided by its cor-responding albedo is below a threshold. As an initialestimate of the albedo, we use the average of the mod-eling (or training) images. A conservative thresholdis then chosen to determine shadows making it almostcertain no invalid data is included in the estimationprocess, at the small expense of throwing away somevalid data. After �nding the invalid data, the followingestimation method is used: without doing any row orcolumn permutations sift out all the full rows (with no

invalid data) of matrix X to form a full sub-matrix ~X.Note that the number of pixels in an image (i.e. thenumber of rows of X) is much larger than the numberof images (i.e. the number of columns of X), whichmeans we can always �nd a large number of full rowsso that the number of rows of ~X is larger than itsnumber of columns. Therefore, perform SVD on ~Xto get a fairly good initial estimate of S�. Fix S�

and estimate each of the rows of B� independently us-ing least squares. Then, �x B� and update each ofthe light source direction si independently, again us-ing least squares. Repeat these last two steps untilestimates converge. In our experiments, the algorithmis very well behaved, converging to the global mini-mum within 10-15 iterations. Though it is possible toconverge to a local minimum, we never observed thiseither in simulation or in practice.

Figure 1 demonstrates the process for constructingthe illumination model. Figure 1.a shows six of theoriginal single light source images of a face used in theestimation of B�. Note that the light source in eachimage moves only by a small amount (�15o in eitherdirection) about the viewing axis. Despite this, theimages do exhibit shadowing, e.g. left and right ofthe nose. In fact, there is a tradeo� in the image ac-quisition process: the smaller the motion of the lightsource, meaning fewer shadows present in the images,the worse the conditioning of the estimation problem.If, on the other hand, the light source moves exces-sively, despite the improvement in the conditioning,more extensive shadowing can increase the possibilityof having too few (less than three) valid measurementswith a �xed number of images for some parts of theface. Therefore, the light source should move in mod-eration as in the images shown in Figure 1.a.

Figure 1.b shows the basis images of the estimatedmatrix B�. These basis images encode not only thealbedo (re ectance) of the face but also its surface nor-mal �eld. They can be used to construct images ofthe face under arbitrary and quite extreme illumina-tion conditions. However, the image formation modelin Equation 1 does not account for cast shadows ofnon-convex objects such as faces. In order to deter-mine which parts of the image are in cast shadows,given a light source direction, we need to reconstructthe surface of the face (see next section) and then useray-tracing techniques.

a.

b.Figure 1: a) Six of the original single light source im-ages used to estimate B�. Note that the light sourcein each image moves only by a small amount (�15o ineither direction) about the viewing axis. Despite this,the images do exhibit shadowing. b) The basis imagesof B�.

3 Surface ReconstructionIn this section, we demonstrate how we can generatean object's surface from B� after enforcing the inte-grability constraint on the surface normal �eld. It hasbeen shown [4, 31] that from multiple images, in whichthe light source directions are unknown, one can onlyrecover a Lambertian surface up to a three-parameterfamily given by the generalized bas-relief (GBR) trans-formation. This family scales the relief ( attens or ex-trudes) and introduces an additive plane. It has alsobeen shown that the family of GBR transformations isthe only one that preserves integrability.

3.1 Enforcing Integrability

The vector �eld B� estimated in Section 2 may notbe integrable, i.e., it may not correspond to a smoothsurface. So, prior to reconstructing the surface up toGBR, the integrability constraint must be enforced onB�. Since no method has been developed to enforce theintegrability during the estimation of B�, we enforceit afterwards. That is, given B� estimate a matrixA 2 GL(3) such that B = B�A corresponds to anintegrable normal �eld; the development follows [31].

Consider a continuous surface de�ned as the graphof z(x; y), and let b(x; y) be the corresponding nor-

mal �eld scaled by an albedo �eld. The integrabilityconstraint for a surface is zxy = zyx where subscriptsdenote partial derivatives. In turn, b(x; y) must sat-isfy: �

b1

b3

�y

=

�b2

b3

�x

To estimate A such that bT (x; y) = b�T

(x; y)A, weexpand this out. Letting the columns of A be denotedby A1; A2; A3 yields

(b�T

A3)(b�T

x A2)� (b�T

A2)(b�T

x A3) =

(b�T

A3)(b�T

y A1)� (b�T

A1)(b�T

y A3)

which can be expressed as

b�TS1b

�

x = b�TS2b

�

y (3)

where S1 = A3AT2 �A2A

T3 and S2 = A3A

T1 �A1A

T3 .

S1 and S2 are skew-symmetric matrices and havethree degrees of freedom. Equation 3 is linear in thesix elements of S1 and S2. From the estimate of B�

discrete approximations of the partial derivatives (b�xand b

�

y) are computed, and then SVD is used to solvefor the six elements of S1 and S2. In [31], it was shownthat the elements of S1 and S2 are cofactors of A, and asimple method for computing A from the cofactors waspresented. This procedure only determines six degreesof freedom of A. The other three correspond to theGBR transformation [4] and can be chosen arbitrarilybecause a GBR transformation preserves integrability.The surface corresponding to B = B�A di�ers from thetrue surface by GBR, i.e., z(x; y) = �z(x; y)+�x+ �yfor arbitrary �; �; � with � 6= 0.

3.2 Generating a GBR surfaceAfter enforcing integrability, we can now reconstructthe corresponding surface z(x; y). Note that z(x; y) isnot a Euclidean reconstruction of the face, but a rep-resentative element of the orbit under a GBR transfor-mation. Despite this, both the shading and the shad-owing will be correct for images synthesized from sucha surface [4].

To �nd z(x; y), we use the variational approach pre-sented in [15]. A surface z(x; y) is �t to the givencomponents of the gradient p and q by minimizing thefunctional Z Z

(zx � p)2 + (zy � q)2 dx dy:

the Euler equation of which reduces to r2z = px + qy.By enforcing the right natural boundary conditionsand employing an iterative scheme that uses a discreteapproximation of the Laplacian, we can reconstructthe surface z(x; y) [15].

Recall that a GBR transformation scales the re-lief ( attens or extrudes) and introduces an additive

plane. To resolve this GBR ambiguity, we take ad-vantage of the fact that we are dealing with humanfaces which constitute a well known class of objects.We can therefore exploit the left-to-right symmetry offaces and the fairly constant ratios of distances be-tween facial features such as the eyes, the nose, andthe forehead. (In the case when the class of objects isnot well de�ned, the issue of resolving the GBR ambi-guity becomes more subtle and is essentially an openproblem.) A surface of a face that has undergone aGBR transformation will have di�erent distance ratiosand can be asymmetric. These di�erences allow us toestimate the three parameters of the GBR transfor-mation which we can then invert. Note that this in-verse transformation is applied to both the estimatedsurface z(x; y) and B. Even though this inverse oper-ation (which is also a GBR transformation) may notcompletely resolve the ambiguity of the relief becauseof errors in the estimation of the GBR parameters, itnevertheless comes very close to that e�ect. After all,our purpose is not to reconstruct the exact Euclideansurface of the face, but to create realistic images of aface under di�ering pose and illumination. Moreover,since shadows are preserved under GBR transforma-tions [4], images synthesized under an arbitrary lightsource from a surface whose normal �eld has been GBRtransformed will have correct shadowing. This meansthat the residual GBR transformation (after resolvingthe ambiguity) will not a�ect the image synthesis withvariable illumination.

Figure 2 shows the reconstructed surface of the faceshown in Figure 1 after resolving the GBR ambigu-ity. The �rst basis image of B� shown in Figure 1.bhas been texture-mapped on the surface. Even thoughwe cannot recover the exact Euclidean structure of theface (i.e. resolve the ambiguity completely), we canstill generate synthetic images of a face under variablepose where the shape distortions due to the residualGBR ambiguity are quite small and not visually de-tectable.

4 Image SynthesisWe �rst demonstrate the ability of our method to gen-erate images of an object under novel illumination con-ditions but �xed pose. Figure 3 shows sample singlelight source images of a face generated with the im-age formation model in Equation 1 which has beenextended to account for cast shadows. To determinecast shadows, we employ ray-tracing that uses the re-constructed surface of the face z(x; y) after resolvingthe GBR ambiguity. Speci�cally, a point on the surfaceis in cast shadow if, for a given light source direction,a ray emanating from that point parallel to the lightsource direction intersects the surface at some otherpoint. With this extended image formation model,the generated images exhibit realistic shading and, de-spite the small presence of shadows in the images inFigure 1.a, have strong attached and cast shadows.

Figure 4 displays a set of synthesized images of the

Figure 2: The reconstructed surface.

the face viewed under variable pose but with �xedlighting. The images were created by rigidly rotat-ing the reconstructed surface shown in Figure 2 �rstabout the horizontal and then about the vertical axis.Along the rows from left to right, the azimuth varies(in 10 degree intervals) from 30 degrees to the right ofthe face to 10 degrees to the left. Down the columns,the elevation varies (again in 10 degree intervals) from20 degrees above the horizon to 30 degrees below. Forexample, in the bottom image of the second columnfrom the left the surface has an azimuth of 20 degreesto the right and an elevation of 30 degrees below thehorizon. The single light source is following the facearound as it changes pose. This implies that a patchon the surface has the same intensity in all poses. It isinteresting to see that the images look quite realisticwith maybe the exception of the three right images inthe bottom row which appear to be a little attened.This is not due to any errors during the geometric orphotometric modeling but probably due to our visualpriors; we are not used to looking at a face from above.

In Figure 5, we combine both variations in viewingconditions to synthesize images of the face under novelpose and lighting. We used the same poses as in Fig-ure 4 but now the light from the single point source is�xed to come along the gaze direction of the face in thetop-right image. Therefore, as the face moves aroundand its gaze direction changes with respect to the lightsource direction, the shading of the surface changesand both attached and cast shadows are formed, asone would expect. The synthesized images seem toagree with our visual intuition.

Figure 3: Sample images of the face under novel illu-mination conditions but �xed pose.

5 Discussion

Appearance variation of an object caused by smallchanges in illumination under �xed pose can provideenough information to estimate (under the assumptionof a Lambertian re ectance function) the object's sur-face normal �eld scaled by its albedo. In the presentedmethod, as few as three images with no knowledge ofthe light source directions can be used in the estima-tion. The estimated surface normal �eld can then beintegrated to reconstruct the object's surface. Unlikemulti-view based image synthesis, our approach doesnot require the determination of point or line corre-spondences to do the surface reconstruction. Since weare dealing with a well known class of objects, we canacceptably resolve the GBR ambiguity of the recon-structed surface. Then, the surface together with thesurface normal �eld scaled by the albedo are su�cientfor synthesizing images of the object under novel poseand lighting.

The e�ectiveness of our approach stems from threereasons. First, the estimation of the illuminationmodel B� does not use any invalid data (such as shad-ows) which would otherwise lead to large biases. Sec-

ond, the integrability constraint is enforced on the sur-face normal �eld which signi�cantly improves the sur-face reconstruction. Last, unlike classical photomet-ric stereo, our method requires no knowledge of lightsource locations. This obviates the need of error-pronecalibration of a �xed lighting rig where any errors inestimating the position of the light sources can propa-gate to the estimation of the illumination model caus-ing large inaccuracies. These reasons have to led toimproved performance and we have demonstrated thisby synthesizing realistic images of human faces.

References[1] S. Avidan, T. Evgeniou, A. Shashua, and T. Pog-

gio. Image-based view synthesis by combining trilineartensors and learning techniques. In ACM Symposiumon Virtual Reality Software and Technology, 1997.

[2] S. Avidan and A. Shashua. Novel view synthesis intensor space. In Proc. IEEE Conf. on Comp. Visionand Patt. Recog., pages 1034{1040, 1997.

[3] E. Barett, M. Brill, N. Haag, and P. Payton. Invariantlinear methods in photogrammetry and model match-ing. In J. Mundy and A. Zisserman, editors, GeometricInvariance in Computer Vision, pages 277{292. MITPress, 1992.

[4] P. Belhumeur, D. Kriegman, and A. Yuille. The bas-relief ambiguity. In Proc. IEEE Conf. on Comp. Vi-sion and Patt. Recog., pages 1040{1046, 1997.

[5] P. N. Belhumeur and D. J. Kriegman. What is theset of images of an object under all possible lightingconditions? In Proc. IEEE Conf. on Comp. Visionand Patt. Recog., pages 270{277, 1996.

[6] R. Carceroni and K. Kutulakos. Shape and motionof 3-d curves from multi-view image scenes. In ImageUnderstanding Workshop, pages 171{176, 1998.

[7] S. Chen and L. Williams. View interpolation for imagesynthesis. In Computer Graphics (SIGGRAPH), pages279{288, 1993.

[8] G. Chou and S. Teller. Multi-image correspondenceusing geometric and structural constraints. In ImageUnderstanding Workshop, pages 869{874, 1997.

[9] P. Debevec, C. Taylor, and J. Malik. Modeling andrendering architecture from photographs: A hybridgeometry- and image-based approach. In ComputerGraphics (SIGGRAPH), pages 11{20, 1996.

[10] R. Epstein, A. Yuille, and P. N. Belhumeur. Learningand recognizing objects using illumination subspaces.In Proc. of the Int. Workshop on Object Representa-tion for Computer Vision, 1996.

[11] A. Georghiades, D. Kriegman, and P. Belhumeur. Il-lumination cones for recognition under variable light-ing: Faces. In Proc. IEEE Conf. on Comp. Vision andPatt. Recog., 1998.

[12] S. J. Gortler, R. Grzeszczuk, R. Szeliski, and M. F.Cohen. The Lumigraph. In Computer Graphics (SIG-GRAPH), pages 43{54, 1996.

[13] H. Hayakawa. Photometric stereo under a light-sourcewith arbitrary motion. JOSA-A, 11(11):3079{3089,Nov. 1994.

[14] B. Horn. Computer Vision. MIT Press, Cambridge,Mass., 1986.

[15] B. Horn and M. Brooks. The variational approach toshape from shading. Computer Vision, Graphics andImage Processing, 35:174{208, 1992.

[16] D. Jacobs. Linear �tting with missing data: Appli-cations to structure from motion and characterizingintensity images. In Proc. IEEE Conf. on Comp. Vi-sion and Patt. Recog., 1997.

[17] J. Koenderink and A. Van Doorn. A�ne structurefrom motion. JOSA-A, 8(2):377{385, 1991.

[18] S. Laveau and O. Faugeras. 3-D scene representa-tion as a collection of images and fundamental matri-ces. Technical Report 2205, INRIA-Sophia Antipolis,February 1994.

[19] M. Levoy and P. Hanrahan. Light �eld rendering. InComputer Graphics (SIGGRAPH), pages 31{42, 1996.

[20] W. R. Mark, L. McMillan, and G. Bishop. Post-rendering 3d warping. In Computer Graphics (SIG-GRAPH), pages 39{46, 1997.

[21] L. Matthies, R. Szeliski, and T. Kanade. Kalman�lter-based algorithms for estimating depth from im-age sequences. Int. J. Computer Vision, 3:293{312,1989.

[22] R. Rosenholtz and J. Koenderink. A�ne structure andphotometry. In Proc. IEEE Conf. on Comp. Visionand Patt. Recog., pages 790{795, 1996.

[23] S. Seitz and C. Dyer. View morphing. In ComputerGraphics (SIGGRAPH), pages 21{30, 1996.

[24] J. Shade, S. Gortler, L. wei He, and R. Szeliski.Layered depth maps. In Computer Graphics (SIG-GRAPH), pages 251{258, 1998.

[25] A. Shashua. Geometry and Photometry in 3D VisualRecognition. PhD thesis, MIT, 1992.

[26] H. Shum, K. Ikeuchi, and R. Reddy. Principal com-ponent analysis with missing data and its applicationto polyhedral object modeling. IEEE Trans. PatternAnal. Mach. Intelligence, 17(9):854{867, September1995.

[27] W. Silver. Determining Shape and Re ectance UsingMultiple Images. PhD thesis, MIT, Cambridge, MA,1980.

[28] C. Tomasi and T. Kanade. Shape and motion fromimage streams under orthography: A factorizationmethod. Int. J. Computer Vision, 9(2):137{154, 1992.

[29] R. Woodham. Analysing images of curved surfaces.Arti�cial Intelligence, 17:117{140, 1981.

[30] Y. Yu and J. Malik. Recovering photometric proper-ties of architectural scenes from photographs. In Com-puter Graphics (SIGGRAPH), pages 207{218, 1998.

[31] A. Yuille and D. Snow. Shape and albedo from multi-ple images using integrability. In Proc. IEEE Conf. onComp. Vision and Patt. Recog., pages 158{164, 1997.

Figure 4: Synthesized images under variable pose but with �xed lighting; the single light source is following theface.

Figure 5: Synthesized images under both variable pose and lighting. As the face moves around the single lightsource stays �xed resulting to image variability due to changes in pose and illumination conditions.