High Dynamic Range from Multiple Images: Which Exposures to Combine? Michael Grossberg and Shree...
-
date post
20-Dec-2015 -
Category
Documents
-
view
217 -
download
0
Transcript of High Dynamic Range from Multiple Images: Which Exposures to Combine? Michael Grossberg and Shree...
![Page 1: High Dynamic Range from Multiple Images: Which Exposures to Combine? Michael Grossberg and Shree Nayar CAVE Lab, Columbia University ICCV Workshop on CPMCV.](https://reader035.fdocuments.us/reader035/viewer/2022062407/56649d4c5503460f94a2a1e1/html5/thumbnails/1.jpg)
High Dynamic Range from Multiple High Dynamic Range from Multiple Images: Images: Which Exposures to Combine?Which Exposures to Combine?
Michael Grossberg and Shree NayarMichael Grossberg and Shree Nayar
CAVE Lab, Columbia UniversityCAVE Lab, Columbia University
ICCV Workshop on CPMCV ICCV Workshop on CPMCV October, 2003, Nice, FranceOctober, 2003, Nice, France
Partially funded by NSF ITR Award, DARPA/ONR MURIPartially funded by NSF ITR Award, DARPA/ONR MURI
![Page 2: High Dynamic Range from Multiple Images: Which Exposures to Combine? Michael Grossberg and Shree Nayar CAVE Lab, Columbia University ICCV Workshop on CPMCV.](https://reader035.fdocuments.us/reader035/viewer/2022062407/56649d4c5503460f94a2a1e1/html5/thumbnails/2.jpg)
Combining Different ExposuresCombining Different Exposures
[Ginosar and Zeevi, 88, Madden, 93,Mann and Picard, 95, Debevec and Malik, 97,Mitsunaga and Nayar, 99]
Low
Dyn
amic
Ran
ge E
xpos
ures
+
Combination Yields High Dynamic Range
![Page 3: High Dynamic Range from Multiple Images: Which Exposures to Combine? Michael Grossberg and Shree Nayar CAVE Lab, Columbia University ICCV Workshop on CPMCV.](https://reader035.fdocuments.us/reader035/viewer/2022062407/56649d4c5503460f94a2a1e1/html5/thumbnails/3.jpg)
The Camera The Camera ResponseResponse
Scene Radiance
Linear Function(Optical Attenuation)
Image Irradiance
L s E
0 255
Camera Response
ImageIntensit
y
Bf
![Page 4: High Dynamic Range from Multiple Images: Which Exposures to Combine? Michael Grossberg and Shree Nayar CAVE Lab, Columbia University ICCV Workshop on CPMCV.](https://reader035.fdocuments.us/reader035/viewer/2022062407/56649d4c5503460f94a2a1e1/html5/thumbnails/4.jpg)
From Response To Measured Irradiance From Response To Measured Irradiance LevelsLevels
Measured Irradiance Levels
Irradiance E
Bri
ghtn
ess
Levels
B
Response Function f
![Page 5: High Dynamic Range from Multiple Images: Which Exposures to Combine? Michael Grossberg and Shree Nayar CAVE Lab, Columbia University ICCV Workshop on CPMCV.](https://reader035.fdocuments.us/reader035/viewer/2022062407/56649d4c5503460f94a2a1e1/html5/thumbnails/5.jpg)
Where do you want your bits?Where do you want your bits?
High Dynamic Range
Coarse quantization
Low Dynamic Range
Fine quantization
Bri
ghtn
ess
BB
rightn
ess
B
Irradiance E
Response Function f
Irradiance E
Response Function f
![Page 6: High Dynamic Range from Multiple Images: Which Exposures to Combine? Michael Grossberg and Shree Nayar CAVE Lab, Columbia University ICCV Workshop on CPMCV.](https://reader035.fdocuments.us/reader035/viewer/2022062407/56649d4c5503460f94a2a1e1/html5/thumbnails/6.jpg)
Effective Camera from Multiple Effective Camera from Multiple ExposuresExposures
CaptureHighDynamic Range
CaptureIrradianceUniformly +
+
Acquired Images Image from Effective CameraGoal
![Page 7: High Dynamic Range from Multiple Images: Which Exposures to Combine? Michael Grossberg and Shree Nayar CAVE Lab, Columbia University ICCV Workshop on CPMCV.](https://reader035.fdocuments.us/reader035/viewer/2022062407/56649d4c5503460f94a2a1e1/html5/thumbnails/7.jpg)
Flexible Dynamic Range Imaging:Flexible Dynamic Range Imaging:
• Can we create an effective camera with a Can we create an effective camera with a desired response?desired response?
– How many exposures are needed?How many exposures are needed?
– Which exposures to acquire? Which exposures to acquire?
– How to combine the acquired images?How to combine the acquired images?
![Page 8: High Dynamic Range from Multiple Images: Which Exposures to Combine? Michael Grossberg and Shree Nayar CAVE Lab, Columbia University ICCV Workshop on CPMCV.](https://reader035.fdocuments.us/reader035/viewer/2022062407/56649d4c5503460f94a2a1e1/html5/thumbnails/8.jpg)
Irradiance Levels From Multiple Irradiance Levels From Multiple ExposuresExposures
f(E)
Irradiance E
E10 E2 E3
21
3
0
2
01
3
6
45
h(E)
f(e2E)
E3 / e3 E2 / e2 E1 / e1 0
21
3
0
Sum
E1
^E6
^E5
^E4
^E3
^E2
^0
Exposure e1 = 1
Effective Camera
Exposure e2
![Page 9: High Dynamic Range from Multiple Images: Which Exposures to Combine? Michael Grossberg and Shree Nayar CAVE Lab, Columbia University ICCV Workshop on CPMCV.](https://reader035.fdocuments.us/reader035/viewer/2022062407/56649d4c5503460f94a2a1e1/html5/thumbnails/9.jpg)
Response of the Effective CameraResponse of the Effective Camera
1
)()(j
EfEh
Number of exposures
Exposures
Camera Response
Irradiance
Effective Response
Theorem: The sum of a set of images of a scene taken at different exposures includes all the information in the individual exposures.
jen
![Page 10: High Dynamic Range from Multiple Images: Which Exposures to Combine? Michael Grossberg and Shree Nayar CAVE Lab, Columbia University ICCV Workshop on CPMCV.](https://reader035.fdocuments.us/reader035/viewer/2022062407/56649d4c5503460f94a2a1e1/html5/thumbnails/10.jpg)
Camera Response EmulationCamera Response Emulation
• How can we tell if How can we tell if hh emulates emulates gg well? well?
Naïve answer:Naïve answer: | h – g | < | h – g | <
gf , ee = ( = (ee11, … ,, … ,eenn))
Level spacing characterizes similarity
Bri
ghtn
ess
Levels
B
Irradiance E
h
Desired Response
Emulated Responsedepends on:
h~
![Page 11: High Dynamic Range from Multiple Images: Which Exposures to Combine? Michael Grossberg and Shree Nayar CAVE Lab, Columbia University ICCV Workshop on CPMCV.](https://reader035.fdocuments.us/reader035/viewer/2022062407/56649d4c5503460f94a2a1e1/html5/thumbnails/11.jpg)
How Response Determines Level How Response Determines Level SpacingSpacing
Measured Irradiance Levels
Irradiance E
Bri
ghtn
ess
Levels
B
h
Observation: The derivative determines the distances between levels.
Dense Spacing
LargerDerivative
Sparse Spacing
SmallerDerivatives
![Page 12: High Dynamic Range from Multiple Images: Which Exposures to Combine? Michael Grossberg and Shree Nayar CAVE Lab, Columbia University ICCV Workshop on CPMCV.](https://reader035.fdocuments.us/reader035/viewer/2022062407/56649d4c5503460f94a2a1e1/html5/thumbnails/12.jpg)
The Objective FunctionThe Objective Function
MAX
MIN
E
E
dEwhg 2||),(
Spacing Based Comparison:
Weight prevents penalizing success:
Desired Response Effective Response
Weight
Number of Exposures Exposure Values
en
)(Ew { )()( EhEg 0,
1, otherwise
B
rightn
ess
B
g
Irradiance E
h
w=0 w=1
![Page 13: High Dynamic Range from Multiple Images: Which Exposures to Combine? Michael Grossberg and Shree Nayar CAVE Lab, Columbia University ICCV Workshop on CPMCV.](https://reader035.fdocuments.us/reader035/viewer/2022062407/56649d4c5503460f94a2a1e1/html5/thumbnails/13.jpg)
Which Exposures and How Many?Which Exposures and How Many?
• For fixed For fixed nn, find minimizing exposure , find minimizing exposure values values ee
• Choose min Choose min nn such that error within such that error within tolerance tolerance
• Method: Exhaustive searchMethod: Exhaustive search
– Objective function not continuousObjective function not continuous– Only need to search actual settings Only need to search actual settings – Offline build table of exposuresOffline build table of exposures
![Page 14: High Dynamic Range from Multiple Images: Which Exposures to Combine? Michael Grossberg and Shree Nayar CAVE Lab, Columbia University ICCV Workshop on CPMCV.](https://reader035.fdocuments.us/reader035/viewer/2022062407/56649d4c5503460f94a2a1e1/html5/thumbnails/14.jpg)
Flexible Dynamic Range ImagingFlexible Dynamic Range Imaging
Linear
Gamma =1/2
Constantcontrast(log)
1111111111111111
21.0031.0031.0151.0031.0033.0941.0031.0031.0031.00320.249.914.6894.8313.979
3
2.9851.0191.0071.007
5.1463.0191.0061.006
88.3837.2329.1816.01
4
3.6721.1661.031
11.235.0492.866
280144.964.22
5
4.9751.078
18.568.564
763.5305.4
6
5.636
33.7
1130
EffectiveCamera
Number ofExposures
Exposure Values
234562345623456
![Page 15: High Dynamic Range from Multiple Images: Which Exposures to Combine? Michael Grossberg and Shree Nayar CAVE Lab, Columbia University ICCV Workshop on CPMCV.](https://reader035.fdocuments.us/reader035/viewer/2022062407/56649d4c5503460f94a2a1e1/html5/thumbnails/15.jpg)
Desired Response
Constant Contrast
Gamma 1/2
Linear
High 1:16,000
Low 1:256
Dynamic Range
Medium 1:1,000
Desired Response
Constant Contrast
Gamma 1/2
Linear
High 1:16,000
Low 1:256
Dynamic Range
Linear
Gamma =1/2
Constantcontrast(log)
1111111111111111
21.0031.0031.0151.0031.0033.0941.0031.0031.0031.00320.249.91
4.6894.8313.979
3
2.9851.0191.0071.007
5.1463.0191.0061.006
88.3837.2329.1816.01
4
3.6721.1661.031
11.235.0492.866
280144.964.22
5
4.9751.078
18.568.564
763.5305.4
6
5.636
33.7
1130
EffectiveCamera
Number ofExposures
Exposure Values
2345623456
23456
Flexible Dynamic Range ImagingFlexible Dynamic Range Imaging
![Page 16: High Dynamic Range from Multiple Images: Which Exposures to Combine? Michael Grossberg and Shree Nayar CAVE Lab, Columbia University ICCV Workshop on CPMCV.](https://reader035.fdocuments.us/reader035/viewer/2022062407/56649d4c5503460f94a2a1e1/html5/thumbnails/16.jpg)
Baseline Exposure ValuesBaseline Exposure Values
• Typically exposures are doubledTypically exposures are doubled
• Baseline: Baseline: Combine the exposures Combine the exposures e e =(1,2,4)=(1,2,4)
[Ginosar and Zeevi, 88, Madden, 93,Mann and Picard, 95, Debevec and Malik, 97,Mitsunaga and Nayar, 99]
![Page 17: High Dynamic Range from Multiple Images: Which Exposures to Combine? Michael Grossberg and Shree Nayar CAVE Lab, Columbia University ICCV Workshop on CPMCV.](https://reader035.fdocuments.us/reader035/viewer/2022062407/56649d4c5503460f94a2a1e1/html5/thumbnails/17.jpg)
Increased Dynamic Range Linear Increased Dynamic Range Linear CameraCamera
• Real Camera: Real Camera: f f linear linear • Desired Camera: Desired Camera: gg linear (greater dynamic linear (greater dynamic
range) range)
Baseline Response (1,2,4)
Desired Response
0.8
0.6
0.4
0.2
0.0
1.0
0.80.60.40.20.0 1.0
Irradiance E
Bri
ghtn
ess
B
4-bit real camera
Baseline Response (1,2,4)
0.8
0.6
0.4
0.2
0.0
1.0
0.80.60.40.20.0 1.0
Desired Response
Irradiance E
Bri
ghtn
ess
B
8-bit real camera
Computed Response(1, 1.003, 2.985)
Computed Response(1, 1.05, 1.11)
![Page 18: High Dynamic Range from Multiple Images: Which Exposures to Combine? Michael Grossberg and Shree Nayar CAVE Lab, Columbia University ICCV Workshop on CPMCV.](https://reader035.fdocuments.us/reader035/viewer/2022062407/56649d4c5503460f94a2a1e1/html5/thumbnails/18.jpg)
Linear Camera: Synthetic Ramp Linear Camera: Synthetic Ramp ImageImage
From computed exposures (1,1.05,1.11)From baseline exposures (1,2,4)
High Dynamic Range Linear Camera
![Page 19: High Dynamic Range from Multiple Images: Which Exposures to Combine? Michael Grossberg and Shree Nayar CAVE Lab, Columbia University ICCV Workshop on CPMCV.](https://reader035.fdocuments.us/reader035/viewer/2022062407/56649d4c5503460f94a2a1e1/html5/thumbnails/19.jpg)
Linear Camera: Image of ClothLinear Camera: Image of Cloth
Computed Exposures (1,1.05,1.11)
Ground Truth (HDR image)
Baseline Exposures (1,2,4)
![Page 20: High Dynamic Range from Multiple Images: Which Exposures to Combine? Michael Grossberg and Shree Nayar CAVE Lab, Columbia University ICCV Workshop on CPMCV.](https://reader035.fdocuments.us/reader035/viewer/2022062407/56649d4c5503460f94a2a1e1/html5/thumbnails/20.jpg)
Constant Contrast from Linear Constant Contrast from Linear CamerasCameras
• Real Camera: Real Camera: f f linear linear • Desired Camera: Desired Camera: gg log response (constant log response (constant
contrast)contrast)
Baseline Response (1,2,4)
0.8
0.6
0.4
0.2
0.0
1.0
0.80.60.40.20.0 1.0
Irradiance E
Brig
htne
ss B
Desired Response
3 Exposures
Baseline Response
Irradiance E
1.0
0.8
0.6
0.4
0.2
0.00.80.60.40.20.0 1.0
Brig
htne
ss B
Desired Response
5 Exposures
Computed Response (1,9.91,88.38)
ComputedResponse
![Page 21: High Dynamic Range from Multiple Images: Which Exposures to Combine? Michael Grossberg and Shree Nayar CAVE Lab, Columbia University ICCV Workshop on CPMCV.](https://reader035.fdocuments.us/reader035/viewer/2022062407/56649d4c5503460f94a2a1e1/html5/thumbnails/21.jpg)
Constant Contrast : Image of TilesConstant Contrast : Image of Tiles
Computed Exposures
Baseline Exposures
![Page 22: High Dynamic Range from Multiple Images: Which Exposures to Combine? Michael Grossberg and Shree Nayar CAVE Lab, Columbia University ICCV Workshop on CPMCV.](https://reader035.fdocuments.us/reader035/viewer/2022062407/56649d4c5503460f94a2a1e1/html5/thumbnails/22.jpg)
Linear Camera from Non-linear CameraLinear Camera from Non-linear Camera
Baseline Input Exposures
Iso-brightnessCombined Iso-brightness
Computed Input Exposures
Combined
• Real Camera: Real Camera: f f non-linear (Nikon 990) non-linear (Nikon 990)• Desired Camera: Desired Camera: gg linear linear
![Page 23: High Dynamic Range from Multiple Images: Which Exposures to Combine? Michael Grossberg and Shree Nayar CAVE Lab, Columbia University ICCV Workshop on CPMCV.](https://reader035.fdocuments.us/reader035/viewer/2022062407/56649d4c5503460f94a2a1e1/html5/thumbnails/23.jpg)
SummarySummary
• Combine images using Combine images using summationsummation
• Method finds Method finds number of exposuresnumber of exposures and and exposure valuesexposure values to use to use
• Emulation of a variety of cameras:Emulation of a variety of cameras:
Flexible Dynamic Range ImagingFlexible Dynamic Range Imaging