Spectral Focal Sweep (SFS): Extended Depth of Field from Chromatic Aberrations ICCP 2010 Ollie...
-
Upload
bernadette-patterson -
Category
Documents
-
view
216 -
download
1
Transcript of Spectral Focal Sweep (SFS): Extended Depth of Field from Chromatic Aberrations ICCP 2010 Ollie...
Spectral Focal Sweep (SFS):
Extended Depth of Field from Chromatic Aberrations
ICCP 2010
Ollie Cossairt, Shree NayarColumbia University
Conventional Camera (F/4)
Spectral Focal Sweep: ICCP 2010
BLURRY
Conventional Camera (F/16)
Spectral Focal Sweep: ICCP 2010
NOISY
EDOF Camera (F/4)
Spectral Focal Sweep: ICCP 2010
CAPTURED
EDOF Camera (F/4)
Spectral Focal Sweep: ICCP 2010
DEBLURRED
Focus Sweep[Nagahara et al. ’08][Hausler ’72] Lens Focal PlaneSensor
Coded Aperture[Welford ’60] [Levin et al. ‘07][Veeraraghavan et al. ’07] [Zhou and Nayar ’09]
Related Work
Spectral Focal Sweep: ICCP 2010
Phase Plates[Dowski and Cathey ’95] [Chi and George ’01] [Levin et al. ’09]
Color[Guichard et al. ’09]
LensFocal Planes
RGB Sensor RedGreenBlue
Spectral Focal Sweep Idea
Spectral Focal Sweep: ICCP 2010
No moving parts or custom elements
PoorlyCorrected
Lens
+
B&WSensor
+
Focus Sweep
Lens
Focal Plane
Sensor
=
Broadband reflectance
spectra
Broadband reflectance spectra• Works exactly for black and white scenes• Many naturally occurring reflectance spectra
are also broadband [Parkkinen ‘89]
Poorly corrected lens• Simpler lens design optimization
Black & White Sensor• Can also approximate with luminance channel
from color sensor
Spectral Focal Sweep Idea
Spectral Focal Sweep: ICCP 2010
Conventional Focus Sweep PSF
Spectral Focal Sweep: ICCP 2010
v1u
u = object depth (fixed)v = sensor distance (varying)
LensEFL = f
FocalPlane
InstantaneousPSF Sensor
Scene Depth Range
Conventional Focus Sweep PSF
Spectral Focal Sweep: ICCP 2010
u = object depth (fixed)v = sensor distance (varying)
LensEFL = f
FocalPlane
InstantaneousPSF Sensor
v2u
Scene Depth Range
d2
Conventional Focus Sweep PSF
Spectral Focal Sweep: ICCP 2010
v3u
d3
u = object depth (fixed)v = sensor distance (varying)
LensEFL = f
FocalPlane
InstantaneousPSF Sensor
Scene Depth Range
Conventional Focus Sweep PSF
Spectral Focal Sweep: ICCP 2010
Varying either object/sensor distance, or focal length will produce the same
PSF
Integrate over time to get final PSF
Circ/Top-hat function
Defocus disk diameter (A = aperture diameter)
Refractive Dispersion (prism)
Spectral Focal Sweep: ICCP 2010
Snell’s Law of refraction
Refraction depends
on wavelengt
h
Refractive Dispersion (lens)
Spectral Focal Sweep: ICCP 2010
uv
SensorPSFFocalPlanes
Scene Depth Range
For an uncorrected lens, Focal length depends on wavelength
R1 , R2 = lens radii of curvature
Spectral Focal Sweep PSF
Spectral Focal Sweep: ICCP 2010
Each wavelength produces a different defocus disk
A B&W Sensor integrates over wavelength, weighted by the spectral reflectance of the scene
For white objects, a SFS PSF is identical to a mechanical focal sweep PSF
SFS Lens Design
Spectral Focal Sweep: ICCP 2010
Maximum defocus diameter 100um at 400nm and 700nm
Design Criteria• Wavelength range 400-
700nm• 75mm EFL F/4• 1/3” Sensor, 10um pixel size
Optimization Criteria• Zemax Optical Design
Software• Maximize chromatic defocus
across wavelength range• Minimize spot size at 550nm• Fit to off-the-shelf lenses
Zemax SFS PSFs(white spectrum)
dep
th
field
SFS Lens Corrected lens
Spectral Focal Sweep: ICCP 2010
15x
Zemax SFS PSFs(white spectrum)
dep
th
Corrected lens
Spectral Focal Sweep: ICCP 2010
field
SFS Lens
1
15x
• Munsell database of 1250 measured reflectance spectra
• Zemax simulation of SFS PSF at 12 depth locations (1.5m-3m, 100um max defocus dia.)
• For each Munsell spectrum– Calculate average deblurring error when a
white spectrum is used for deblurring
Simulating Real Spectra
Spectral Focal Sweep: ICCP 2010
Average Deblurring Error
Spectral Focal Sweep: ICCP 2010
Input Image:
Munsell color PSFs:
White PSF at center depth:
PSF Distance Metric:
Average Deblurring Error:
[Zhou and Nayar ‘09]
Averaged over many images
Munsell Spectra PSFs
Spectral Focal Sweep: ICCP 2010
Corrected lens
dep
th
Most Munsell spectra show little PSF variation
1
15x
Average Deblurring Error(%95 of Munsell spectra)
Spectral Focal Sweep: ICCP 2010
Most Munsell PSFs introduce negligible deblurring artifacts
Doublet SFS Implementation
Spectral Focal Sweep: ICCP 2010
Edmund Optics #481841” Dia. 100mm FL Plano-convex asphere
Thorlabs #LE19291” Dia. 300mm FL positive meniscus
Zemax Design Implementation
Experimental Setup
Spectral Focal Sweep: ICCP 2010
B&W Examples(Basler 1/3” sensor)
Color Examples(Canon 450D sensor)
Corrected Lens SFS Doublet Lens
F/1.4
75
mm
EFL C
osm
icar
F/2.8
10
0m
m
EFL C
annon
F/4 S
FS 7
5m
mEFL D
ouble
tF/4
SFS
75
mm
EFL D
ouble
t
Measured SFS PSFs(white spectrum)
Spectral Focal Sweep: ICCP 2010
Measured SFS PSF approximately depth-invariant
SFS
lens
Cosm
ica
r
depth
Deblurring
Deblur with measured white PSF
B&W Images• Wiener deconvolution
Color Images• Transform from RGB to YUV space• Wiener deconvolution of Y image• Transform back to RGB space
Spectral Focal Sweep: ICCP 2010
EDOF Examples
Spectral Focal Sweep: ICCP 2010
F/4 Corrected Lens
Spectral Focal Sweep: ICCP 2010
8ms exposure
F/4 SFS Camera
Spectral Focal Sweep: ICCP 2010
8ms exposure
(Captured)
F/4 SFS Camera
Spectral Focal Sweep: ICCP 2010
8ms exposure
(Deblurred)
F/4 Corrected Lens
Spectral Focal Sweep: ICCP 2010
8ms exposure
Background Foreground
F/4 SFS Lens
Spectral Focal Sweep: ICCP 2010
8ms exposure
(Captured)
F/4 SFS Lens
Spectral Focal Sweep: ICCP 2010
(Deblurred)
8ms exposure
F/4 Corrected Lens
Spectral Focal Sweep: ICCP 2010
16ms exposure
Background
Foreground
F/4 SFS Lens
Spectral Focal Sweep: ICCP 2010
16ms exposure
(Captured)
F/4 SFS Lens
Spectral Focal Sweep: ICCP 2010
16ms exposure
(Deblurred)
F/4 Corrected Lens
Spectral Focal Sweep: ICCP 2010
16ms exposure
BackgroundForeground
F/4 SFS Lens
Spectral Focal Sweep: ICCP 2010
16ms exposure
(Captured)
F/4 SFS Lens
Spectral Focal Sweep: ICCP 2010
16ms exposure
(Deblurred)
F/4 Corrected Lens
Spectral Focal Sweep: ICCP 2010
16ms exposure
Background
Foreground
F/4 SFS Lens
Spectral Focal Sweep: ICCP 2010
16ms exposure
(Captured)
F/4 SFS Lens
Spectral Focal Sweep: ICCP 2010
16ms exposure
(Deblurred)
F/4 Corrected Lens
Spectral Focal Sweep: ICCP 2010
16ms exposure
Background
Foreground
F/4 SFS Lens
Spectral Focal Sweep: ICCP 2010
16ms exposure
(Captured)
F/4 SFS Lens
Spectral Focal Sweep: ICCP 2010
16ms exposure
(Deblurred)
Limitations
Spectral Focal Sweep: ICCP 2010
• Broadband assumption does not always hold (e.g. %5 of Munsell Spectra)– deblurring artifacts introduced
• Color deblurring method does not correct blur in chrominance channels
• Works exactly for B&W scenes• Works well for many naturally occurring
spectra• Results for color images “look” good• Easy to build EDOF camera
Conclusion
Spectral Focal Sweep: ICCP 2010
Broadbandspectra
Chromaticaberrations
B&Wsensor
Focussweep+ + ==