debevec
description
Transcript of debevec
![Page 1: debevec](https://reader035.fdocuments.us/reader035/viewer/2022070419/56815aa2550346895dc8298f/html5/thumbnails/1.jpg)
www.debevec.orgwww.debevec.org
Paul Debevec, Tim Hawkins, Chris Tchou, H.P. Duiker, Westley Sarokin, and Mark Sagar
Paul Debevec, Tim Hawkins, Chris Tchou, H.P. Duiker, Westley Sarokin, and Mark Sagar
Acquiring the Reflectance Fieldof a Human Face
Acquiring the Reflectance Fieldof a Human Face
UC Berkeley / USC Institute for Creative Technologies / LifeF/XUC Berkeley / USC Institute for Creative Technologies / LifeF/X
![Page 2: debevec](https://reader035.fdocuments.us/reader035/viewer/2022070419/56815aa2550346895dc8298f/html5/thumbnails/2.jpg)
![Page 3: debevec](https://reader035.fdocuments.us/reader035/viewer/2022070419/56815aa2550346895dc8298f/html5/thumbnails/3.jpg)
![Page 4: debevec](https://reader035.fdocuments.us/reader035/viewer/2022070419/56815aa2550346895dc8298f/html5/thumbnails/4.jpg)
Related WorkRelated WorkHanrahan and Krueger. Reflection from
Layered Surfaces due to Subsurface Scattering. SIGGRAPH 93
Bregler et al. Video Rewrite. SIGGRAPH 97
Guenter et al. Making Faces. SIGGRAPH 98
Pighin et al. Synthesizing Realistic Facial Expressions from Photographs. SIGGRAPH 98
Sagar et al. The Jester. SIGGRAPH 99 ET
Marschner et al. Reflectance Measurements of Human Skin. 1999
Hanrahan and Krueger. Reflection from Layered Surfaces due to Subsurface Scattering. SIGGRAPH 93
Bregler et al. Video Rewrite. SIGGRAPH 97
Guenter et al. Making Faces. SIGGRAPH 98
Pighin et al. Synthesizing Realistic Facial Expressions from Photographs. SIGGRAPH 98
Sagar et al. The Jester. SIGGRAPH 99 ET
Marschner et al. Reflectance Measurements of Human Skin. 1999
![Page 5: debevec](https://reader035.fdocuments.us/reader035/viewer/2022070419/56815aa2550346895dc8298f/html5/thumbnails/5.jpg)
The Reflectance FieldThe Reflectance Field
![Page 6: debevec](https://reader035.fdocuments.us/reader035/viewer/2022070419/56815aa2550346895dc8298f/html5/thumbnails/6.jpg)
The Reflectance FieldThe Reflectance Field
![Page 7: debevec](https://reader035.fdocuments.us/reader035/viewer/2022070419/56815aa2550346895dc8298f/html5/thumbnails/7.jpg)
The Reflectance FieldThe Reflectance Field
![Page 8: debevec](https://reader035.fdocuments.us/reader035/viewer/2022070419/56815aa2550346895dc8298f/html5/thumbnails/8.jpg)
The Reflectance FieldThe Reflectance Field
Ri( ui ,vi ,i ,i )Ri( ui ,vi ,i ,i )incident light fieldincident light field
![Page 9: debevec](https://reader035.fdocuments.us/reader035/viewer/2022070419/56815aa2550346895dc8298f/html5/thumbnails/9.jpg)
The Reflectance FieldThe Reflectance Field
Rr ( ur ,vr ,r ,r )Rr ( ur ,vr ,r ,r )Ri( ui ,vi ,i ,i )Ri( ui ,vi ,i ,i )incident light fieldincident light field radiant light fieldradiant light field
![Page 10: debevec](https://reader035.fdocuments.us/reader035/viewer/2022070419/56815aa2550346895dc8298f/html5/thumbnails/10.jpg)
The Reflectance FieldThe Reflectance Field
Ri( ui ,vi ,i ,i ; ur ,vr ,r ,r )Ri( ui ,vi ,i ,i ; ur ,vr ,r ,r )8D reflectance field8D reflectance field
![Page 11: debevec](https://reader035.fdocuments.us/reader035/viewer/2022070419/56815aa2550346895dc8298f/html5/thumbnails/11.jpg)
Ri( ui ,vi ,i ,i ; ur ,vr ,r ,r )Ri( ui ,vi ,i ,i ; ur ,vr ,r ,r )
4D Slices of the 8D
Reflectance Field
4D Slices of the 8D
Reflectance Field
![Page 12: debevec](https://reader035.fdocuments.us/reader035/viewer/2022070419/56815aa2550346895dc8298f/html5/thumbnails/12.jpg)
The Light StageThe Light Stage
![Page 13: debevec](https://reader035.fdocuments.us/reader035/viewer/2022070419/56815aa2550346895dc8298f/html5/thumbnails/13.jpg)
The Light Stage:60-second exposure
The Light Stage:60-second exposure
![Page 14: debevec](https://reader035.fdocuments.us/reader035/viewer/2022070419/56815aa2550346895dc8298f/html5/thumbnails/14.jpg)
Light Stage DataLight Stage DataOriginal
Resolution: 6432
Original Resolution:
6432
Lighting through image recombination: Haeberli ‘92, Nimeroff ‘94, Wong ‘97Lighting through image recombination: Haeberli ‘92, Nimeroff ‘94, Wong ‘97
![Page 15: debevec](https://reader035.fdocuments.us/reader035/viewer/2022070419/56815aa2550346895dc8298f/html5/thumbnails/15.jpg)
Light Stage ResultsLight Stage Results
Environments from the Light Probe Image Gallerywww.debevec.org
Environments from the Light Probe Image Gallerywww.debevec.org
![Page 16: debevec](https://reader035.fdocuments.us/reader035/viewer/2022070419/56815aa2550346895dc8298f/html5/thumbnails/16.jpg)
Reflectance FunctionsReflectance Functions
![Page 17: debevec](https://reader035.fdocuments.us/reader035/viewer/2022070419/56815aa2550346895dc8298f/html5/thumbnails/17.jpg)
Lighting Reflectance FunctionsLighting Reflectance Functions
normalized light map
normalized light map
reflectance function
reflectance function
lighting productlighting product
rendered pixel
rendered pixel
1
1
DCT BasisDCT BasisSmith and Rowe. Compressed domain processing of JPEG-encoded images. 1996Smith and Rowe. Compressed domain processing of JPEG-encoded images. 1996
![Page 18: debevec](https://reader035.fdocuments.us/reader035/viewer/2022070419/56815aa2550346895dc8298f/html5/thumbnails/18.jpg)
Interactive Lighting DemoSIGGRAPH 2000 Creative Applications Laboratory
Interactive Lighting DemoSIGGRAPH 2000 Creative Applications Laboratory
![Page 19: debevec](https://reader035.fdocuments.us/reader035/viewer/2022070419/56815aa2550346895dc8298f/html5/thumbnails/19.jpg)
Interactive Lighting DemoSIGGRAPH 2000 Creative Applications Laboratory
Interactive Lighting DemoSIGGRAPH 2000 Creative Applications Laboratory
![Page 20: debevec](https://reader035.fdocuments.us/reader035/viewer/2022070419/56815aa2550346895dc8298f/html5/thumbnails/20.jpg)
Changing the ViewpointChanging the Viewpoint
![Page 21: debevec](https://reader035.fdocuments.us/reader035/viewer/2022070419/56815aa2550346895dc8298f/html5/thumbnails/21.jpg)
Reflection of Light from SkinReflection of Light from Skin
Specular Component:Color of light, shiny, brighter near grazing, maintains polarization
Subsurface Component:Color of skin, diffuse, desaturated near grazing, scrambles polarization
Specular Component:Color of light, shiny, brighter near grazing, maintains polarization
Subsurface Component:Color of skin, diffuse, desaturated near grazing, scrambles polarization
After Hanrahan ‘93After Hanrahan ‘93
![Page 22: debevec](https://reader035.fdocuments.us/reader035/viewer/2022070419/56815aa2550346895dc8298f/html5/thumbnails/22.jpg)
Separating Reflectance Components using Crossed Polarizers
Separating Reflectance Components using Crossed Polarizers
Normal ImageNormal Image Subsurface ComponentSubsurface Component
Specular Component
Specular Component
Colorspace techniques - Sato ‘94, Nayar ‘97Colorspace techniques - Sato ‘94, Nayar ‘97
![Page 23: debevec](https://reader035.fdocuments.us/reader035/viewer/2022070419/56815aa2550346895dc8298f/html5/thumbnails/23.jpg)
Transforming a Reflectance Function
Transforming a Reflectance Function
Subsurface ComponentSubsurface Component
Specular ComponentSpecular Component
Final RFFinal RF Comparison RFComparison RF
Shifted and ScaledSpecular
Shifted and ScaledSpecular
=> Torrance-Sparrow microfacet distribution
=> Torrance-Sparrow microfacet distribution
Surface Normal EstimateSurface Normal Estimate
Original RFOriginal RF
![Page 24: debevec](https://reader035.fdocuments.us/reader035/viewer/2022070419/56815aa2550346895dc8298f/html5/thumbnails/24.jpg)
Point-Source ComparisonPoint-Source Comparison
Original ImageOriginal Image Novel ViewpointNovel Viewpoint
![Page 25: debevec](https://reader035.fdocuments.us/reader035/viewer/2022070419/56815aa2550346895dc8298f/html5/thumbnails/25.jpg)
Spatially-VaryingReflectance Parameters
Spatially-VaryingReflectance Parameters
Surface Normals
n
Surface Normals
n
Diffuse Albedo
d
Diffuse Albedo
d
SpecularIntensity
ks
SpecularIntensity
ks
Specular Roughness
Specular Roughness
![Page 26: debevec](https://reader035.fdocuments.us/reader035/viewer/2022070419/56815aa2550346895dc8298f/html5/thumbnails/26.jpg)
Compositing TestCompositing Test
Original ImageOriginal Image Light Probe Light Probe
Rendered FaceRendered Face Composite Composite
![Page 27: debevec](https://reader035.fdocuments.us/reader035/viewer/2022070419/56815aa2550346895dc8298f/html5/thumbnails/27.jpg)
4. Ongoing Work4. Ongoing Work
• Animate the faces
• Capture more spectral samples
• Use high-speed cameras to achieve real-time capture
• Animate the faces
• Capture more spectral samples
• Use high-speed cameras to achieve real-time capture
![Page 28: debevec](https://reader035.fdocuments.us/reader035/viewer/2022070419/56815aa2550346895dc8298f/html5/thumbnails/28.jpg)
5. Conclusion5. Conclusion
We have presented:
The light stage apparatus for capturing slices of the reflectance field of the human face
A technique for correctly relighting faces and objects with arbitrary illumination
A technique for extrapolating human reflectance to novel viewpoints
We have presented:
The light stage apparatus for capturing slices of the reflectance field of the human face
A technique for correctly relighting faces and objects with arbitrary illumination
A technique for extrapolating human reflectance to novel viewpoints
![Page 29: debevec](https://reader035.fdocuments.us/reader035/viewer/2022070419/56815aa2550346895dc8298f/html5/thumbnails/29.jpg)
ThanksThanksDigital Media Innovation Program
Interactive Pictures Corporation
Alias|Wavefront
UCB Digital Digital Media/New Genre Program
ONR/BMDO
Cornell Program of Computer Graphics
Berkeley Millennium Project
Digital Media Innovation Program
Interactive Pictures Corporation
Alias|Wavefront
UCB Digital Digital Media/New Genre Program
ONR/BMDO
Cornell Program of Computer Graphics
Berkeley Millennium Project
and
Shawn Brixey, Bill Buxton, Larry Rowe, Jessica Vallot, Patrick Wilson, Melanie Levine, Eric Paulos, Christine Waggoner, Holly Cim, Eliza Ra, Bryan Musson,
David Altenau, Marc Levoy, Maryann Simmons, Henrik Wann Jensen, Don Greenberg, Pat Hanrahan, Randal Kleiser, Chris Bregler, Michael Naimark,
Dan Maas, Steve Marschner, and Kevin Binkert.
and
Shawn Brixey, Bill Buxton, Larry Rowe, Jessica Vallot, Patrick Wilson, Melanie Levine, Eric Paulos, Christine Waggoner, Holly Cim, Eliza Ra, Bryan Musson,
David Altenau, Marc Levoy, Maryann Simmons, Henrik Wann Jensen, Don Greenberg, Pat Hanrahan, Randal Kleiser, Chris Bregler, Michael Naimark,
Dan Maas, Steve Marschner, and Kevin Binkert.
![Page 30: debevec](https://reader035.fdocuments.us/reader035/viewer/2022070419/56815aa2550346895dc8298f/html5/thumbnails/30.jpg)
![Page 31: debevec](https://reader035.fdocuments.us/reader035/viewer/2022070419/56815aa2550346895dc8298f/html5/thumbnails/31.jpg)
![Page 32: debevec](https://reader035.fdocuments.us/reader035/viewer/2022070419/56815aa2550346895dc8298f/html5/thumbnails/32.jpg)
![Page 33: debevec](https://reader035.fdocuments.us/reader035/viewer/2022070419/56815aa2550346895dc8298f/html5/thumbnails/33.jpg)
In-plane Reflectometry Measurements
In-plane Reflectometry Measurements
Subsurface exhibits chromaticity falloff
Specular is monochromatic with Torrance-Sparrow microfacet behavior
Subsurface exhibits chromaticity falloff
Specular is monochromatic with Torrance-Sparrow microfacet behavior
SubsurfaceSubsurface
Subsurface + SpecularSubsurface + Specular
ii
rr
ii
rr
![Page 34: debevec](https://reader035.fdocuments.us/reader035/viewer/2022070419/56815aa2550346895dc8298f/html5/thumbnails/34.jpg)
Reflectometry ExperimentReflectometry Experiment
![Page 35: debevec](https://reader035.fdocuments.us/reader035/viewer/2022070419/56815aa2550346895dc8298f/html5/thumbnails/35.jpg)
The JesterSIGGRAPH 99 Electronic Theater
Mark Sagar et al. - LifeF/X, Inc.Performance and Text: Jessica Vallot
The JesterSIGGRAPH 99 Electronic Theater
Mark Sagar et al. - LifeF/X, Inc.Performance and Text: Jessica Vallot
![Page 36: debevec](https://reader035.fdocuments.us/reader035/viewer/2022070419/56815aa2550346895dc8298f/html5/thumbnails/36.jpg)
Changing the ViewpointChanging the ViewpointModel from Structured Lighting
![Page 37: debevec](https://reader035.fdocuments.us/reader035/viewer/2022070419/56815aa2550346895dc8298f/html5/thumbnails/37.jpg)
Traditional ApproachTraditional Approach
Derive reflectance parameters for each point on the face’s surface
Map the parameters onto a geometric model of the face
Render using traditional methods
Derive reflectance parameters for each point on the face’s surface
Map the parameters onto a geometric model of the face
Render using traditional methods
![Page 38: debevec](https://reader035.fdocuments.us/reader035/viewer/2022070419/56815aa2550346895dc8298f/html5/thumbnails/38.jpg)
Reflectance Function Mosaic
Reflectance Function Mosaic