Diffusion Coding Photography for Extended Depth of Field SIGGRAPH 2010 Ollie Cossairt, Changyin...
-
date post
20-Dec-2015 -
Category
Documents
-
view
214 -
download
0
Transcript of Diffusion Coding Photography for Extended Depth of Field SIGGRAPH 2010 Ollie Cossairt, Changyin...
![Page 1: Diffusion Coding Photography for Extended Depth of Field SIGGRAPH 2010 Ollie Cossairt, Changyin Zhou, Shree Nayar Columbia University.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d425503460f94a1dca1/html5/thumbnails/1.jpg)
Diffusion Coding Photography
for Extended Depth of Field
SIGGRAPH 2010
Ollie Cossairt, Changyin Zhou, Shree NayarColumbia University
![Page 2: Diffusion Coding Photography for Extended Depth of Field SIGGRAPH 2010 Ollie Cossairt, Changyin Zhou, Shree Nayar Columbia University.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d425503460f94a1dca1/html5/thumbnails/2.jpg)
Conventional Camera (F/1.8)
![Page 3: Diffusion Coding Photography for Extended Depth of Field SIGGRAPH 2010 Ollie Cossairt, Changyin Zhou, Shree Nayar Columbia University.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d425503460f94a1dca1/html5/thumbnails/3.jpg)
Focused Image PSF Image Noise Captured Image
Camera Blur ModelSpati
al dom
ain
Focused Image MTF Image Noise Captured ImageFrequency
dom
ain
![Page 4: Diffusion Coding Photography for Extended Depth of Field SIGGRAPH 2010 Ollie Cossairt, Changyin Zhou, Shree Nayar Columbia University.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d425503460f94a1dca1/html5/thumbnails/4.jpg)
Deblurring Problems
MTFCaptured image
Low SNR
low MTF values
Problem 1:
Focused image
Lens
Sensor
P
Object
Q
Problem 2:
Variation with depth
![Page 5: Diffusion Coding Photography for Extended Depth of Field SIGGRAPH 2010 Ollie Cossairt, Changyin Zhou, Shree Nayar Columbia University.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d425503460f94a1dca1/html5/thumbnails/5.jpg)
Extending Depth of Field: Previous Work
Focus Sweep Cameras[Hausler ’72]
[Nagahara et al. ’08]Lens Focal PlaneSensor
Wavefront Coding Cameras
[Dowski and Cathey ’95] [Chi and George ’01] [Garcia-Guerrero et al. ‘07]
Other Related Work[Levin et al. ’07][Veeraraghavan et al. ’07] [Levin et al. ’09]
![Page 6: Diffusion Coding Photography for Extended Depth of Field SIGGRAPH 2010 Ollie Cossairt, Changyin Zhou, Shree Nayar Columbia University.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d425503460f94a1dca1/html5/thumbnails/6.jpg)
Focus Sweep Camera
Lens
Scene
Sensor
+ + + + + + =InstantaneousPSF
t = 1 t = 2 t = 3 t = 4 t = 5 t = 6 t = 7
Final PSF
[Hausler ’72][Nagahara et al. ’08]
![Page 7: Diffusion Coding Photography for Extended Depth of Field SIGGRAPH 2010 Ollie Cossairt, Changyin Zhou, Shree Nayar Columbia University.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d425503460f94a1dca1/html5/thumbnails/7.jpg)
Focus Sweep Camera
+ + + + + + =InstantaneousPSF
t = 1 t = 2 t = 3 t = 4 t = 5 t = 6 t = 7
Final PSF
InstantaneousPSF
t = 1
+
t = 2
+
t = 3
+
t = 4 t = 5
+
t = 6
+
t = 7
+
depth
1depth
2
[Levin et al. ’09]
2D MTF
=
Final PSF
Lens
Scene
Sensor
[Hausler ’72][Nagahara et al. ’08]
![Page 8: Diffusion Coding Photography for Extended Depth of Field SIGGRAPH 2010 Ollie Cossairt, Changyin Zhou, Shree Nayar Columbia University.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d425503460f94a1dca1/html5/thumbnails/8.jpg)
Wavefront Coding
Lens Scene
Sensor
[Levin et al. ’09]
2D MTF
Cubic Phase Plate
x u
[Dowski and Cathey ’95]
y
Ambiguity Function
slice
MTF
![Page 9: Diffusion Coding Photography for Extended Depth of Field SIGGRAPH 2010 Ollie Cossairt, Changyin Zhou, Shree Nayar Columbia University.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d425503460f94a1dca1/html5/thumbnails/9.jpg)
Lens Resolution Target
Sensor
Focu
s S
weep
Wavefr
on
tC
od
ing
EDOF Camera Comparison
depth
![Page 10: Diffusion Coding Photography for Extended Depth of Field SIGGRAPH 2010 Ollie Cossairt, Changyin Zhou, Shree Nayar Columbia University.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d425503460f94a1dca1/html5/thumbnails/10.jpg)
Focu
s S
weep
Wavefr
on
tC
od
ing
Deblurred image
EDOF Camera Comparison
![Page 11: Diffusion Coding Photography for Extended Depth of Field SIGGRAPH 2010 Ollie Cossairt, Changyin Zhou, Shree Nayar Columbia University.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d425503460f94a1dca1/html5/thumbnails/11.jpg)
Deblurring Error vs. Depth
Deblu
rrin
g E
rror
noise
Deblurring Error
Deblurring Error
Depth
Wavefront Coding
Focus Sweep
Focus Sweep Wavefront Coding
![Page 12: Diffusion Coding Photography for Extended Depth of Field SIGGRAPH 2010 Ollie Cossairt, Changyin Zhou, Shree Nayar Columbia University.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d425503460f94a1dca1/html5/thumbnails/12.jpg)
Is it possible to achieve the performance of focus sweep
without moving parts?
![Page 13: Diffusion Coding Photography for Extended Depth of Field SIGGRAPH 2010 Ollie Cossairt, Changyin Zhou, Shree Nayar Columbia University.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d425503460f94a1dca1/html5/thumbnails/13.jpg)
Optical DiffusersCircular diffuser
[http://www.luminitco.com]
Diffuser sheets
w
SensorDiffuser
Light ray
x
x
Scatter function
w
SEM image
![Page 14: Diffusion Coding Photography for Extended Depth of Field SIGGRAPH 2010 Ollie Cossairt, Changyin Zhou, Shree Nayar Columbia University.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d425503460f94a1dca1/html5/thumbnails/14.jpg)
Diffuser Kernels
LensSensor
x u
A
u
x
A/2
-A/2
With diffuser
A/2
-A/2
u
x
Without diffuser
Light
field
space
![Page 15: Diffusion Coding Photography for Extended Depth of Field SIGGRAPH 2010 Ollie Cossairt, Changyin Zhou, Shree Nayar Columbia University.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d425503460f94a1dca1/html5/thumbnails/15.jpg)
w w
A/2
-A/2
u
x
Without diffuser
Diffuser Kernels
LensSensor
w
u
x
With diffuser
w
Light
field
space
x u
![Page 16: Diffusion Coding Photography for Extended Depth of Field SIGGRAPH 2010 Ollie Cossairt, Changyin Zhou, Shree Nayar Columbia University.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d425503460f94a1dca1/html5/thumbnails/16.jpg)
Diffuser Kernels
LensSensor
w
u
x
A/2
-A/2
u
x
u
x
Diffuser kernel
Light field Diffuser kernel
Coded light field
x u
Without diffuser With diffuser
Light
field
space
![Page 17: Diffusion Coding Photography for Extended Depth of Field SIGGRAPH 2010 Ollie Cossairt, Changyin Zhou, Shree Nayar Columbia University.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d425503460f94a1dca1/html5/thumbnails/17.jpg)
Diffusion Coded PSF
A/2
-A/2
u
x
Without diffuser
u
x
With diffuser
u
x
Diffuser kernel
x
project
x
project
Light
field
space
x
project
Senso
r sp
ace
Camera PSF Scatter function
Coded PSF
![Page 18: Diffusion Coding Photography for Extended Depth of Field SIGGRAPH 2010 Ollie Cossairt, Changyin Zhou, Shree Nayar Columbia University.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d425503460f94a1dca1/html5/thumbnails/18.jpg)
Radially Symmetric Light Field
Lensv
u
Sensor
For an on-axis, isotropic point source:
![Page 19: Diffusion Coding Photography for Extended Depth of Field SIGGRAPH 2010 Ollie Cossairt, Changyin Zhou, Shree Nayar Columbia University.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d425503460f94a1dca1/html5/thumbnails/19.jpg)
Radially Symmetric Diffuser
For a radially-symmetric diffuser kernel
Lens Sensor Sensor
![Page 20: Diffusion Coding Photography for Extended Depth of Field SIGGRAPH 2010 Ollie Cossairt, Changyin Zhou, Shree Nayar Columbia University.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d425503460f94a1dca1/html5/thumbnails/20.jpg)
Radially Symmetric Diffuser PSFs
Radially symmetric diffuser
Coded PSF Scatter function
Camera PSFCoded PSF Scatter function
Camera PSF
Conventional diffuser
Normalized frequencyNormalized frequencyNormalized frequencyNormalized frequency
PSF Vs. Depth
MTF Vs. Depth
k7
k9
k11
k13
K7
K9
K11
K13
-50px 50px -50px 50px -50px 50px -50px 50px
depth
depth
![Page 21: Diffusion Coding Photography for Extended Depth of Field SIGGRAPH 2010 Ollie Cossairt, Changyin Zhou, Shree Nayar Columbia University.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d425503460f94a1dca1/html5/thumbnails/21.jpg)
Diffusion Coding Performance
Deblurring Error vs. Depth
Depth
Wavefront CodingFocus Sweep
noise
Diffusion Coding (light field)Diffusion Coding (wave optics)
Similar performance to focus sweep without moving parts
![Page 22: Diffusion Coding Photography for Extended Depth of Field SIGGRAPH 2010 Ollie Cossairt, Changyin Zhou, Shree Nayar Columbia University.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d425503460f94a1dca1/html5/thumbnails/22.jpg)
Diffuser ImplementationDiffuser scatter function
)(rf
r (mm)110
Th
ickn
ess
(u
m)
3
863
2
1
)(rt
r (mm)
[www.rpcphotonics.com]
Diffuser heightmap Fabricated Diffuser
Diffuser surface profile
[Sales et al. ‘03]
![Page 23: Diffusion Coding Photography for Extended Depth of Field SIGGRAPH 2010 Ollie Cossairt, Changyin Zhou, Shree Nayar Columbia University.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d425503460f94a1dca1/html5/thumbnails/23.jpg)
Garcia-Guerrero
Comparison with Prior Work
Deblurring Error vs. Depth
Depth
Diffusion Coding
Diffusion coding significantly outperforms prior work
![Page 24: Diffusion Coding Photography for Extended Depth of Field SIGGRAPH 2010 Ollie Cossairt, Changyin Zhou, Shree Nayar Columbia University.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d425503460f94a1dca1/html5/thumbnails/24.jpg)
Diffusion Coding Experiments
Fabricated DiffuserCannon 50mm EF lensCannon 450D Sensor
Experimental Setup
Measured PSFs
depth
Without diffuser
with diffuser
BM3D Deblurring Algorithm
[Dabov et al. ‘08]
![Page 25: Diffusion Coding Photography for Extended Depth of Field SIGGRAPH 2010 Ollie Cossairt, Changyin Zhou, Shree Nayar Columbia University.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d425503460f94a1dca1/html5/thumbnails/25.jpg)
Examples
![Page 26: Diffusion Coding Photography for Extended Depth of Field SIGGRAPH 2010 Ollie Cossairt, Changyin Zhou, Shree Nayar Columbia University.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d425503460f94a1dca1/html5/thumbnails/26.jpg)
Conventional Camera
f-number = 1.8, exposure time = 16ms
![Page 27: Diffusion Coding Photography for Extended Depth of Field SIGGRAPH 2010 Ollie Cossairt, Changyin Zhou, Shree Nayar Columbia University.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d425503460f94a1dca1/html5/thumbnails/27.jpg)
Conventional Camera
f-number = 18 , exposure time = 16ms
![Page 28: Diffusion Coding Photography for Extended Depth of Field SIGGRAPH 2010 Ollie Cossairt, Changyin Zhou, Shree Nayar Columbia University.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d425503460f94a1dca1/html5/thumbnails/28.jpg)
Diffusion Coding
Captured
f-number = 1.8, exposure time = 16ms
![Page 29: Diffusion Coding Photography for Extended Depth of Field SIGGRAPH 2010 Ollie Cossairt, Changyin Zhou, Shree Nayar Columbia University.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d425503460f94a1dca1/html5/thumbnails/29.jpg)
Diffusion Coding
Deblurred
f-number = 1.8, exposure time = 16ms
![Page 30: Diffusion Coding Photography for Extended Depth of Field SIGGRAPH 2010 Ollie Cossairt, Changyin Zhou, Shree Nayar Columbia University.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d425503460f94a1dca1/html5/thumbnails/30.jpg)
Conventional Camera
• f-number = 1.8• exposure time = 10ms
![Page 31: Diffusion Coding Photography for Extended Depth of Field SIGGRAPH 2010 Ollie Cossairt, Changyin Zhou, Shree Nayar Columbia University.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d425503460f94a1dca1/html5/thumbnails/31.jpg)
Diffusion Coding
• f-number = 1.8• exposure time = 10ms
Captured
![Page 32: Diffusion Coding Photography for Extended Depth of Field SIGGRAPH 2010 Ollie Cossairt, Changyin Zhou, Shree Nayar Columbia University.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d425503460f94a1dca1/html5/thumbnails/32.jpg)
Diffusion Coding
• f-number = 1.8• exposure time = 10ms
Deblurred
![Page 33: Diffusion Coding Photography for Extended Depth of Field SIGGRAPH 2010 Ollie Cossairt, Changyin Zhou, Shree Nayar Columbia University.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d425503460f94a1dca1/html5/thumbnails/33.jpg)
Conventional Camera
• f-number = 1.8• exposure time =
12.5ms
![Page 34: Diffusion Coding Photography for Extended Depth of Field SIGGRAPH 2010 Ollie Cossairt, Changyin Zhou, Shree Nayar Columbia University.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d425503460f94a1dca1/html5/thumbnails/34.jpg)
Diffusion Coding
• f-number = 1.8• exposure time =
12.5ms
Captured
![Page 35: Diffusion Coding Photography for Extended Depth of Field SIGGRAPH 2010 Ollie Cossairt, Changyin Zhou, Shree Nayar Columbia University.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d425503460f94a1dca1/html5/thumbnails/35.jpg)
Diffusion Coding
• f-number = 1.8• exposure time =
12.5ms
Deblurred
![Page 36: Diffusion Coding Photography for Extended Depth of Field SIGGRAPH 2010 Ollie Cossairt, Changyin Zhou, Shree Nayar Columbia University.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d425503460f94a1dca1/html5/thumbnails/36.jpg)
Conventional Camera
f-number = 1.8, exposure time = 16ms
![Page 37: Diffusion Coding Photography for Extended Depth of Field SIGGRAPH 2010 Ollie Cossairt, Changyin Zhou, Shree Nayar Columbia University.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d425503460f94a1dca1/html5/thumbnails/37.jpg)
Diffusion Coding
Captured
f-number = 1.8, exposure time = 16ms
![Page 38: Diffusion Coding Photography for Extended Depth of Field SIGGRAPH 2010 Ollie Cossairt, Changyin Zhou, Shree Nayar Columbia University.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d425503460f94a1dca1/html5/thumbnails/38.jpg)
Diffusion Coding
Deblurred
f-number = 1.8, exposure time = 16ms
![Page 39: Diffusion Coding Photography for Extended Depth of Field SIGGRAPH 2010 Ollie Cossairt, Changyin Zhou, Shree Nayar Columbia University.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d425503460f94a1dca1/html5/thumbnails/39.jpg)
LimitationsConventional Camera Diffusion Coding
•Loss of image texture
•Loss of contrast
•Occlusion errors
![Page 40: Diffusion Coding Photography for Extended Depth of Field SIGGRAPH 2010 Ollie Cossairt, Changyin Zhou, Shree Nayar Columbia University.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d425503460f94a1dca1/html5/thumbnails/40.jpg)
Conclusions
Diffusion Coding Theory
Diffusion Coding Examples
Radially Symmetric Diffusers
LensSensor
Diffuser
Diffusion Coding Implementation