Projection Defocus Analysis for Scene Capture and Image Display Li Zhang Shree Nayar Columbia...
-
date post
21-Dec-2015 -
Category
Documents
-
view
214 -
download
0
Transcript of Projection Defocus Analysis for Scene Capture and Image Display Li Zhang Shree Nayar Columbia...
![Page 1: Projection Defocus Analysis for Scene Capture and Image Display Li Zhang Shree Nayar Columbia University IIS-00-85864 SIGGRAPH Conference July 2006, Boston,](https://reader036.fdocuments.us/reader036/viewer/2022062714/56649d605503460f94a40926/html5/thumbnails/1.jpg)
Projection Defocus Analysis for Scene Capture and Image
Display
Li Zhang Shree NayarColumbia University
IIS-00-85864
SIGGRAPH ConferenceJuly 2006, Boston, USA
![Page 2: Projection Defocus Analysis for Scene Capture and Image Display Li Zhang Shree Nayar Columbia University IIS-00-85864 SIGGRAPH Conference July 2006, Boston,](https://reader036.fdocuments.us/reader036/viewer/2022062714/56649d605503460f94a40926/html5/thumbnails/2.jpg)
Modeling Complex Scenes is Hard
Courtesy of Maia C @ flickr.com
![Page 3: Projection Defocus Analysis for Scene Capture and Image Display Li Zhang Shree Nayar Columbia University IIS-00-85864 SIGGRAPH Conference July 2006, Boston,](https://reader036.fdocuments.us/reader036/viewer/2022062714/56649d605503460f94a40926/html5/thumbnails/3.jpg)
LimitationGoal
The Limits of 3D Photography
Scene
Depth Map
Structured Light & Stereo
Depth from Camera Defocus
Photometric Stereo
Missing Data near
Occlusions
Inaccurate at Discontinuitie
s
Only for Single
Surface
Image-Complete
Accurate
for Complex Scenes
![Page 4: Projection Defocus Analysis for Scene Capture and Image Display Li Zhang Shree Nayar Columbia University IIS-00-85864 SIGGRAPH Conference July 2006, Boston,](https://reader036.fdocuments.us/reader036/viewer/2022062714/56649d605503460f94a40926/html5/thumbnails/4.jpg)
Projection Defocus Analysis
Projector Defocus Model
Robust Depth Recovery
Image Refocusing Video Composition
Enhanced Image DisplayIncreasing Depth of Field
Depixelation
![Page 5: Projection Defocus Analysis for Scene Capture and Image Display Li Zhang Shree Nayar Columbia University IIS-00-85864 SIGGRAPH Conference July 2006, Boston,](https://reader036.fdocuments.us/reader036/viewer/2022062714/56649d605503460f94a40926/html5/thumbnails/5.jpg)
Camera Defocus Cue
Courtesy of Koesbong @ flickr.com
![Page 6: Projection Defocus Analysis for Scene Capture and Image Display Li Zhang Shree Nayar Columbia University IIS-00-85864 SIGGRAPH Conference July 2006, Boston,](https://reader036.fdocuments.us/reader036/viewer/2022062714/56649d605503460f94a40926/html5/thumbnails/6.jpg)
Limitation of Depth from Camera Defocus
Focal Plane Surface Camera
Camera Defocus:
![Page 7: Projection Defocus Analysis for Scene Capture and Image Display Li Zhang Shree Nayar Columbia University IIS-00-85864 SIGGRAPH Conference July 2006, Boston,](https://reader036.fdocuments.us/reader036/viewer/2022062714/56649d605503460f94a40926/html5/thumbnails/7.jpg)
Depth from Projector Defocus
Focal Plane
Lamp
Projector
![Page 8: Projection Defocus Analysis for Scene Capture and Image Display Li Zhang Shree Nayar Columbia University IIS-00-85864 SIGGRAPH Conference July 2006, Boston,](https://reader036.fdocuments.us/reader036/viewer/2022062714/56649d605503460f94a40926/html5/thumbnails/8.jpg)
Depth from Projector Defocus
Focal Plane
Lamp
Surface Projector
![Page 9: Projection Defocus Analysis for Scene Capture and Image Display Li Zhang Shree Nayar Columbia University IIS-00-85864 SIGGRAPH Conference July 2006, Boston,](https://reader036.fdocuments.us/reader036/viewer/2022062714/56649d605503460f94a40926/html5/thumbnails/9.jpg)
Depth from Projector Defocus
Focal Plane
Lamp
Surface Projector
Projector Defocus:
![Page 10: Projection Defocus Analysis for Scene Capture and Image Display Li Zhang Shree Nayar Columbia University IIS-00-85864 SIGGRAPH Conference July 2006, Boston,](https://reader036.fdocuments.us/reader036/viewer/2022062714/56649d605503460f94a40926/html5/thumbnails/10.jpg)
Camera Defocus vs. Projector Defocus
Projector Defocus:
Camera Defocus:
![Page 11: Projection Defocus Analysis for Scene Capture and Image Display Li Zhang Shree Nayar Columbia University IIS-00-85864 SIGGRAPH Conference July 2006, Boston,](https://reader036.fdocuments.us/reader036/viewer/2022062714/56649d605503460f94a40926/html5/thumbnails/11.jpg)
Temporal Defocus MethodFocal Plane
LampSh
ift
Surface ProjectorCamera
Projector Defocus Model:
![Page 12: Projection Defocus Analysis for Scene Capture and Image Display Li Zhang Shree Nayar Columbia University IIS-00-85864 SIGGRAPH Conference July 2006, Boston,](https://reader036.fdocuments.us/reader036/viewer/2022062714/56649d605503460f94a40926/html5/thumbnails/12.jpg)
Experimental Setup
Projector
Half-mirror
Camera
Projector
Camera
Half-mirror
![Page 13: Projection Defocus Analysis for Scene Capture and Image Display Li Zhang Shree Nayar Columbia University IIS-00-85864 SIGGRAPH Conference July 2006, Boston,](https://reader036.fdocuments.us/reader036/viewer/2022062714/56649d605503460f94a40926/html5/thumbnails/13.jpg)
Experimental Setup
Projector
Half-mirror
Camera
![Page 14: Projection Defocus Analysis for Scene Capture and Image Display Li Zhang Shree Nayar Columbia University IIS-00-85864 SIGGRAPH Conference July 2006, Boston,](https://reader036.fdocuments.us/reader036/viewer/2022062714/56649d605503460f94a40926/html5/thumbnails/14.jpg)
Books and Gate: Sharp Discontinuities
Scene Depth Map
3D Model
![Page 15: Projection Defocus Analysis for Scene Capture and Image Display Li Zhang Shree Nayar Columbia University IIS-00-85864 SIGGRAPH Conference July 2006, Boston,](https://reader036.fdocuments.us/reader036/viewer/2022062714/56649d605503460f94a40926/html5/thumbnails/15.jpg)
Books and Gate: Sharp Discontinuities
Scene Depth Map
Refocus on Gate
+
![Page 16: Projection Defocus Analysis for Scene Capture and Image Display Li Zhang Shree Nayar Columbia University IIS-00-85864 SIGGRAPH Conference July 2006, Boston,](https://reader036.fdocuments.us/reader036/viewer/2022062714/56649d605503460f94a40926/html5/thumbnails/16.jpg)
Books and Gate: Sharp Discontinuities
Scene Depth Map
Refocus on Back
+
![Page 17: Projection Defocus Analysis for Scene Capture and Image Display Li Zhang Shree Nayar Columbia University IIS-00-85864 SIGGRAPH Conference July 2006, Boston,](https://reader036.fdocuments.us/reader036/viewer/2022062714/56649d605503460f94a40926/html5/thumbnails/17.jpg)
Application: Refocus Synthesis
![Page 18: Projection Defocus Analysis for Scene Capture and Image Display Li Zhang Shree Nayar Columbia University IIS-00-85864 SIGGRAPH Conference July 2006, Boston,](https://reader036.fdocuments.us/reader036/viewer/2022062714/56649d605503460f94a40926/html5/thumbnails/18.jpg)
Wrestlers: Curved and Specular Surfaces
Scene Depth Map
3D Model
![Page 19: Projection Defocus Analysis for Scene Capture and Image Display Li Zhang Shree Nayar Columbia University IIS-00-85864 SIGGRAPH Conference July 2006, Boston,](https://reader036.fdocuments.us/reader036/viewer/2022062714/56649d605503460f94a40926/html5/thumbnails/19.jpg)
Fence and Leaves: Complex Occlusions
Scene Depth Map
3D Model
![Page 20: Projection Defocus Analysis for Scene Capture and Image Display Li Zhang Shree Nayar Columbia University IIS-00-85864 SIGGRAPH Conference July 2006, Boston,](https://reader036.fdocuments.us/reader036/viewer/2022062714/56649d605503460f94a40926/html5/thumbnails/20.jpg)
Application: Refocus Synthesis
![Page 21: Projection Defocus Analysis for Scene Capture and Image Display Li Zhang Shree Nayar Columbia University IIS-00-85864 SIGGRAPH Conference July 2006, Boston,](https://reader036.fdocuments.us/reader036/viewer/2022062714/56649d605503460f94a40926/html5/thumbnails/21.jpg)
Playing Cards: Object Insertion
Scene Depth Map
Green Screen Capture Depth-based Composition
![Page 22: Projection Defocus Analysis for Scene Capture and Image Display Li Zhang Shree Nayar Columbia University IIS-00-85864 SIGGRAPH Conference July 2006, Boston,](https://reader036.fdocuments.us/reader036/viewer/2022062714/56649d605503460f94a40926/html5/thumbnails/22.jpg)
Projection Defocus Analysis
Projector Defocus Model
Robust Depth Recovery
Image Refocusing Video Composition
Enhanced Image DisplayIncreasing Depth of Field
Depixelation
![Page 23: Projection Defocus Analysis for Scene Capture and Image Display Li Zhang Shree Nayar Columbia University IIS-00-85864 SIGGRAPH Conference July 2006, Boston,](https://reader036.fdocuments.us/reader036/viewer/2022062714/56649d605503460f94a40926/html5/thumbnails/23.jpg)
Focused Projection at Multiple Depths
![Page 24: Projection Defocus Analysis for Scene Capture and Image Display Li Zhang Shree Nayar Columbia University IIS-00-85864 SIGGRAPH Conference July 2006, Boston,](https://reader036.fdocuments.us/reader036/viewer/2022062714/56649d605503460f94a40926/html5/thumbnails/24.jpg)
Focused Projection at Multiple Depths
A Multi-Projector Solution [Bimber & Emmerling 2006]
![Page 25: Projection Defocus Analysis for Scene Capture and Image Display Li Zhang Shree Nayar Columbia University IIS-00-85864 SIGGRAPH Conference July 2006, Boston,](https://reader036.fdocuments.us/reader036/viewer/2022062714/56649d605503460f94a40926/html5/thumbnails/25.jpg)
Focused Projection at Multiple Depths
Our Single Projector Solution Defocus Compensation
30 inches
30 inches
3 meters
![Page 26: Projection Defocus Analysis for Scene Capture and Image Display Li Zhang Shree Nayar Columbia University IIS-00-85864 SIGGRAPH Conference July 2006, Boston,](https://reader036.fdocuments.us/reader036/viewer/2022062714/56649d605503460f94a40926/html5/thumbnails/26.jpg)
?
?
Defocused ProjectionOriginal ImageCompensation Image
Defocus CompensationDefocus Kernel
![Page 27: Projection Defocus Analysis for Scene Capture and Image Display Li Zhang Shree Nayar Columbia University IIS-00-85864 SIGGRAPH Conference July 2006, Boston,](https://reader036.fdocuments.us/reader036/viewer/2022062714/56649d605503460f94a40926/html5/thumbnails/27.jpg)
Compensation Image
?
Defocused Projection
Defocus CompensationDefocus Kernel
Simple Sharpening Does NOT work.
Result
![Page 28: Projection Defocus Analysis for Scene Capture and Image Display Li Zhang Shree Nayar Columbia University IIS-00-85864 SIGGRAPH Conference July 2006, Boston,](https://reader036.fdocuments.us/reader036/viewer/2022062714/56649d605503460f94a40926/html5/thumbnails/28.jpg)
?
Defocus CompensationOriginal ImageDefocus KernelCompensation Image
![Page 29: Projection Defocus Analysis for Scene Capture and Image Display Li Zhang Shree Nayar Columbia University IIS-00-85864 SIGGRAPH Conference July 2006, Boston,](https://reader036.fdocuments.us/reader036/viewer/2022062714/56649d605503460f94a40926/html5/thumbnails/29.jpg)
CompensatedProjection
Front Plane Middle Plane Back Plane
Original Image
Compensation Results
Normal Projection
![Page 30: Projection Defocus Analysis for Scene Capture and Image Display Li Zhang Shree Nayar Columbia University IIS-00-85864 SIGGRAPH Conference July 2006, Boston,](https://reader036.fdocuments.us/reader036/viewer/2022062714/56649d605503460f94a40926/html5/thumbnails/30.jpg)
Defocus-Based Depixelation
Screen-Door Effect
![Page 31: Projection Defocus Analysis for Scene Capture and Image Display Li Zhang Shree Nayar Columbia University IIS-00-85864 SIGGRAPH Conference July 2006, Boston,](https://reader036.fdocuments.us/reader036/viewer/2022062714/56649d605503460f94a40926/html5/thumbnails/31.jpg)
Depixelation Results
DepixelatedProjection
Original Image
Normal Projection
![Page 32: Projection Defocus Analysis for Scene Capture and Image Display Li Zhang Shree Nayar Columbia University IIS-00-85864 SIGGRAPH Conference July 2006, Boston,](https://reader036.fdocuments.us/reader036/viewer/2022062714/56649d605503460f94a40926/html5/thumbnails/32.jpg)
Contributions
• Reducing Pixelation for Digital Projectors
• Increasing DoF without Changing Optics
• Estimating Depth from Projector Defocus
![Page 33: Projection Defocus Analysis for Scene Capture and Image Display Li Zhang Shree Nayar Columbia University IIS-00-85864 SIGGRAPH Conference July 2006, Boston,](https://reader036.fdocuments.us/reader036/viewer/2022062714/56649d605503460f94a40926/html5/thumbnails/33.jpg)
Limitations and Future Work
Real-time Compensation
Real-time Depth Recovery
Translucency Acquisition
Matte Estimation
![Page 34: Projection Defocus Analysis for Scene Capture and Image Display Li Zhang Shree Nayar Columbia University IIS-00-85864 SIGGRAPH Conference July 2006, Boston,](https://reader036.fdocuments.us/reader036/viewer/2022062714/56649d605503460f94a40926/html5/thumbnails/34.jpg)
The End