RECONSTRUCTION. Republicans split over Reconstruction Radicals vs.Moderates.
Multi-Flash 3D Photography: Capturing Shape and...
1
Photometric Reconstruction Geometric Reconstruction Multi-Flash 3D Photography: Capturing Shape and Appearance Douglas Lanman* Peter G. Sibley † Daniel Crispell* Yong Zhao* Gabriel Taubin* *{dlanman, daniel_crispell, yong_zhao, taubin}@brown.edu, † [email protected] We describe a new 3D scanning system which exploits the depth discontinuity information captured by a multi-flash camera [3]. In contrast to existing differential and global shape-from- silhouette algorithms, our method can reconstruct the position and orientation of points located deep inside concavities. Extending our prior work [2], we model the appearance of each surface point by fitting a Phong reflectance model to the BRDF samples acquired from a turntable sequence. A computer-controlled turntable and 8 megapixel digital camera are used to acquire data from 670 viewpoints in a circular path around an object. For each viewpoint, we capture four images using illumination from the top, left, right, and bottom flashes. The camera is intrinsically calibrated and its pose with respect to the turntable is determined using a calibration grid. Acquisition hardware: An Olympus C-8080 8 megapixel digital camera with 8 LED flashes and a computer-controlled turntable. [1] Cipolla R., Blake A. 1992. Surface shape from the deformation of apparent contours. International Journal of Computer Vision 9, 2, 83–112. [2] Crispell, D., Lanman, D., Sibley, P. G., Zhao, Y., and Taubin, G. 2006. Beyond silhouettes: Surface reconstruction using multi-flash photography. In Third International Symposium on 3D Data Processing, Visualization and Transmission. [3] Raskar, R., Tan, K.-H., Feris, R., Yu, J., and Turk, M. 2004. Non-photorealistic camera: Depth edge detection and stylized rendering using multi-flash imaging. ACM Trans. Graph. 23, 3, 679–688. [4] Sadlo, F., Weyrich, T., Peikert, R., and Gross, M. 2005. A practical structured light acquisition system for point-based geometry and texture. In Eurographics Symposium on Point-Based Graphics, 89–98. Before resampling After resampling Input image Reconstruction Input image Reconstruction Input image Reconstruction Input image Reconstructions Point-based reconstruction of the “woman with fruit basket” using 67 viewpoints and a per-point Phong appearance model. Input image Median albedo Phong (specular) Phong (diffuse) Estimated Phong appearance model As presented, the multi-flash 3D photography system represents a new, self-contained method for acquiring point-based models of both shape and appearance. Future work will explore both the quantitative accuracy of this system as well as its synergistic combination with existing approaches (e.g. laser scanners and structured light systems), especially with regard to planning optimal viewpoints for 3D scanning. References Experimental Results Conclusions and Future Work The tangent ray from the camera to the object slides over the surface as the camera moves. Depth can be estimated based on the apparent motion of the resulting contour. Using the implicit surface, we can determine which points are visible from each viewpoint. Despite the relatively large number of viewpoints, the BRDF remains sparsely-sampled since the illumination sources and camera are nearly coincident. As a result, we fit a per-point Phong model to the set of reflectance observations (using 67 viewpoints). Note that, experimentally, we found that the median diffuse albedo was a visually- acceptable substitute for the diffuse component of the Phong model. Four objects were reconstructed with varying geometric and material complexity. For the “hand” model, the multi-flash approach successfully reconstructed the fingers – regions that could not be recovered using existing shape-from-silhouette or visual hull algorithms. In addition, the diffuse wood grain and specular highlights were modeled accurately. 1) Using a sequence of four images captured with different illumination at each turntable position, we are able to robustly compute depth edges and surface normals using the method introduced by Raskar et al. [3]. 2) Next, we stack the sequence of depth edge images and “slice” across a single image scanline, forming an epipolar slice, which approximates the epipolar constraint in local regions. 3) We estimate the subpixel position of the set of depth discontinuities (observed along each scanline over time) by fitting a quadratic polynomial to the ridges in each epipolar slice. 4) Using the set of tracked depth discontinuities, we apply differential reconstruction to estimate an oriented point cloud using the method of Cipolla and Blake [1]. 5) The reconstructed point cloud will exhibit errors near frontier points and will possess several gaps. To minimize these effects, we filter outliers by backprojecting the point cloud into a small number of foreground-segmented images. Gap-filling is accomplished by introducing samples from the implicit surface generated using the method of Sibley and Taubin [2]. Input image Depth edges Edge orientation Geometric reconstruction follows the approach of Crispell et al. [2], as shown in the system flowchart. Multi-Flash 3D Photography: System flowchart Overview System Architecture
Transcript of Multi-Flash 3D Photography: Capturing Shape and...
-
Photometric Reconstruction
Geometric Reconstruction
Multi-Flash 3D Photography: Capturing Shape and Appearance
Douglas Lanman* Peter G. Sibley† Daniel Crispell* Yong Zhao* Gabriel Taubin**{dlanman, daniel_crispell, yong_zhao, taubin}@brown.edu, †[email protected]
We describe a new 3D scanning system which exploits the depth discontinuity information captured by a multi-flash camera [3]. In contrast to existing differential and global shape-from-silhouette algorithms, our method can reconstruct the position and orientation of points located deep inside concavities. Extending our prior work [2], we model the appearance of each surface point by fitting a Phong reflectance model to the BRDF samples acquired from a turntable sequence.
A computer-controlled turntable and 8 megapixel digital camera are used to acquire data from 670 viewpoints in a circular path around an object. For each viewpoint, we capture four images using illumination from the top, left, right, and bottom flashes. The camera is intrinsically calibrated and its pose with respectto the turntable is determined using a calibration grid.
Acquisition hardware: An Olympus C-8080 8 megapixel digital camera with 8 LED flashes and a computer-controlled turntable.
[1] Cipolla R., Blake A. 1992. Surface shape from the deformation of apparent contours. International Journal of Computer Vision 9, 2, 83–112.
[2] Crispell, D., Lanman, D., Sibley, P. G., Zhao, Y., and Taubin, G. 2006. Beyond silhouettes: Surface reconstruction using multi-flash photography. In Third International Symposium on 3D Data Processing, Visualization and Transmission.
[3] Raskar, R., Tan, K.-H., Feris, R., Yu, J., and Turk, M. 2004. Non-photorealistic camera: Depth edge detection and stylized rendering using multi-flash imaging. ACM Trans. Graph. 23, 3, 679–688.
[4] Sadlo, F., Weyrich, T., Peikert, R., and Gross, M. 2005. A practical structured light acquisition system for point-based geometry and texture. In Eurographics Symposium on Point-Based Graphics, 89–98.
Before resampling After resampling
Input image Reconstruction Input image Reconstruction
Input image Reconstruction Input image Reconstructions
Point-based reconstruction of the “woman with fruit basket”using 67 viewpoints and a per-point Phong appearance model.
Input image Median albedo
Phong (specular) Phong (diffuse) Estimated Phong appearance model
As presented, the multi-flash 3D photography system represents a new, self-contained method for acquiring point-based models of both shape and appearance. Future work will explore both the quantitative accuracy of this system as well as its synergistic combination with existing approaches (e.g. laser scanners and structured light systems), especially with regard to planning optimal viewpoints for 3D scanning.
References
Experimental Results
Conclusions and Future Work
The tangent ray from the camera to the object slides over the surface as the camera moves. Depth can be estimated based on the apparent motion of the resulting contour.
Using the implicit surface, we can determine which points are visible from each viewpoint. Despite the relatively large number of viewpoints, the BRDF remains sparsely-sampled since the illumination sources and camera are nearly coincident. As a result, we fit a per-point Phong model
to the set of reflectance observations (using 67 viewpoints). Note that, experimentally, we found that the median diffuse albedo was a visually-acceptable substitute for the diffuse component of the Phong model.
Four objects were reconstructed with varying geometric and material complexity. For the “hand” model, the multi-flash approach successfully reconstructed the fingers – regions that could not be recovered using existing shape-from-silhouette or visual hull algorithms. In addition, the diffuse wood grain and specular highlights were modeled accurately.
1) Using a sequence of four images captured with different illumination at each turntable position, we are able to robustly compute depth edges and surface normals using the method introduced by Raskar et al. [3].
2) Next, we stack the sequence of depth edge images and “slice” across a single image scanline, forming an epipolar slice, which approximates the epipolar constraint in local regions.
3) We estimate the subpixel position of the set of depth discontinuities (observed along each scanline over time) by fitting a quadratic polynomial to the ridges in each epipolar slice.
4) Using the set of tracked depth discontinuities, we apply differential reconstruction to estimate an oriented point cloud using the method of Cipolla and Blake [1].
5) The reconstructed point cloud will exhibit errors near frontier points and will possess several gaps. To minimize these effects, we filter outliers by backprojecting the point cloud into a small number of foreground-segmented images. Gap-filling is accomplished by introducing samples from the implicit surface generated using the method of Sibley and Taubin [2].
Input image Depth edges Edge orientation
Geometric reconstruction follows the approach of Crispell et al. [2], as shown in the system flowchart.
Multi-Flash 3D Photography: System flowchart
Overview
System Architecture
/ColorImageDict > /JPEG2000ColorACSImageDict > /JPEG2000ColorImageDict > /AntiAliasGrayImages false /CropGrayImages true /GrayImageMinResolution 300 /GrayImageMinResolutionPolicy /OK /DownsampleGrayImages true /GrayImageDownsampleType /Bicubic /GrayImageResolution 300 /GrayImageDepth -1 /GrayImageMinDownsampleDepth 2 /GrayImageDownsampleThreshold 1.50000 /EncodeGrayImages true /GrayImageFilter /DCTEncode /AutoFilterGrayImages true /GrayImageAutoFilterStrategy /JPEG /GrayACSImageDict > /GrayImageDict > /JPEG2000GrayACSImageDict > /JPEG2000GrayImageDict > /AntiAliasMonoImages false /CropMonoImages true /MonoImageMinResolution 1200 /MonoImageMinResolutionPolicy /OK /DownsampleMonoImages true /MonoImageDownsampleType /Bicubic /MonoImageResolution 1200 /MonoImageDepth -1 /MonoImageDownsampleThreshold 1.50000 /EncodeMonoImages true /MonoImageFilter /CCITTFaxEncode /MonoImageDict > /AllowPSXObjects false /CheckCompliance [ /None ] /PDFX1aCheck false /PDFX3Check false /PDFXCompliantPDFOnly false /PDFXNoTrimBoxError true /PDFXTrimBoxToMediaBoxOffset [ 0.00000 0.00000 0.00000 0.00000 ] /PDFXSetBleedBoxToMediaBox true /PDFXBleedBoxToTrimBoxOffset [ 0.00000 0.00000 0.00000 0.00000 ] /PDFXOutputIntentProfile () /PDFXOutputConditionIdentifier () /PDFXOutputCondition () /PDFXRegistryName () /PDFXTrapped /False
/Description > /Namespace [ (Adobe) (Common) (1.0) ] /OtherNamespaces [ > /FormElements false /GenerateStructure true /IncludeBookmarks false /IncludeHyperlinks false /IncludeInteractive false /IncludeLayers false /IncludeProfiles true /MultimediaHandling /UseObjectSettings /Namespace [ (Adobe) (CreativeSuite) (2.0) ] /PDFXOutputIntentProfileSelector /NA /PreserveEditing true /UntaggedCMYKHandling /LeaveUntagged /UntaggedRGBHandling /LeaveUntagged /UseDocumentBleed false >> ]>> setdistillerparams> setpagedevice
![Texture Synthesis and Manipulation - Brown Universitymesh.brown.edu/dlanman/courses/en256/Lanman-Quilting-Proposal.pdfDouglas Lanman 9 Pixel-based Texture Synthesis References: [3,4,5,6]](https://static.fdocuments.us/doc/165x107/6032bb1b4fb0814f6e4304e7/texture-synthesis-and-manipulation-brown-douglas-lanman-9-pixel-based-texture.jpg){kind=tracking}
![Welcome to alumni.media.mit.edu - Layered 3D: tomographic ...alumni.media.mit.edu/~dlanman/research/compressive...man et al. [2010]. Compared to parallax barriers, our multi-layer](https://static.fdocuments.us/doc/165x107/6094c2309d133867c471f812/welcome-to-layered-3d-tomographic-dlanmanresearchcompressive-man-et.jpg)
![Texture Synthesis and Manipulationalumni.media.mit.edu/~dlanman/courses/en256/Lanman-Quilting-Proposal.pdfDouglas Lanman 9 Pixel-based Texture Synthesis References: [3,4,5,6] Pixel-based](https://static.fdocuments.us/doc/165x107/5e75a00f12fe8151cb7f28c9/texture-synthesis-and-dlanmancoursesen256lanman-quilting-proposalpdf-douglas.jpg)