High dynamic images between devices and vision limits
-
Upload
media-integration-and-communication-center -
Category
Technology
-
view
903 -
download
0
description
Transcript of High dynamic images between devices and vision limits
Le immagini ad alta dinamica tra i limiti dei dispositivi e quelli
della visioneAlessandro Rizzi
Dipartimento di Informatica e ComunicazioneUniversità degli Studi di Milano
Friday, June 10, 2011
Outline
HDR imaging
HDR in practice: measuring the limits
Using HDR
Friday, June 10, 2011
The dynamic range
Friday, June 10, 2011
Friday, June 10, 2011
Define HDR ?
do we need a threshold number ?
Friday, June 10, 2011
Define HDR ?
do we need a threshold number ?
NO
Friday, June 10, 2011
Define HDR
A rendition of a scene with greater dynamic range than
the reproduction media
Friday, June 10, 2011
That is ?
Friday, June 10, 2011
Friday, June 10, 2011
Annibale Carracci (1560-‐1609) PaesaggioFriday, June 10, 2011
Photo: C. OleariFriday, June 10, 2011
Photo: C. OleariFriday, June 10, 2011
Annibale Carracci (1560-‐1609) PaesaggioFriday, June 10, 2011
Source/lamp Average Luminance cd/m2
Xenon short arc 200 000 ÷ 5 000 000 000Sun 1 600 000 000Metal halide 10 000 000 ÷ 60 000 000Incandescent 20 000 000 ÷ 26 000 000compact Fluorescent 20 000 ÷ 70 000Fluorescent 5 000 ÷ 30 000Sunlit clouds 10 000Candle 7 500blue sky 5 000Preferred values for indoor lighIng
50 ÷ 500
White paper at sun 10 000White paper at 500 lx 100White paper at 5 lx 1
Courtesy: C. Oleari
Light levels
Friday, June 10, 2011
Dynamic ranges
Friday, June 10, 2011
Dynamic ranges
?Friday, June 10, 2011
Range limits and quantization: the ‘salame’ metaphor
Friday, June 10, 2011
Friday, June 10, 2011
Range compression from incorrect pixel perspective
Friday, June 10, 2011
Range compression from incorrect pixel perspective
Friday, June 10, 2011
Range compression from incorrect pixel perspective
Very wide range obtained with isolated stimuliimpossible to obtain in an image
Friday, June 10, 2011
The “salame” metaphor
Dynamic range Quantization
Friday, June 10, 2011
The “salame” metaphor
Dynamic range Quantization
More bits do not mean wider rangeLess bits do not mean shorter range
Friday, June 10, 2011
SceneDR
SensorDR
16 bit
28=2562-3 log unit
216=65536 4-5 log unit
8 bit
Friday, June 10, 2011
SceneDR
SensorDR
16 bit
28=2562-3 log unit
216=65536 4-5 log unit
8 bit
NO
Friday, June 10, 2011
SceneDR
SensorDR
8 bit
2-3 log unit
4-5 log unit
8 bit
SceneDR
SensorDR
16 bit
16 bit
Friday, June 10, 2011
SceneDR
SensorDR
8 bit 16 bit
SceneDR
SensorDR
Friday, June 10, 2011
SceneDR
SensorDR
8 bit 16 bit
SceneDR
SensorDR
Friday, June 10, 2011
The HDR idea
http://www.adolfo.trinca.name/public/2010/11/ahdrdiagram.jpg
Friday, June 10, 2011
The HDR ideaHow ?
general solution ?rendering intent ?
http://www.adolfo.trinca.name/public/2010/11/ahdrdiagram.jpg
Friday, June 10, 2011
http://www.digitalcameratracker.com/how-to-create-high-definition-range-hdr-photos/
Friday, June 10, 2011
Two sides of the coin
• Objective data: recording/displaying physical light colorimetric distribution
• Subjective data: reproducing appearance (or different rendering intent)
Friday, June 10, 2011
Mapping the world: the characteristic curve
Friday, June 10, 2011
H & D curve
Friday, June 10, 2011
H & D curve
Friday, June 10, 2011
H & D curve
Friday, June 10, 2011
H & D curve
Friday, June 10, 2011
http://www.dpreview.com/reviews/olympuse3/page21.asp
Olympus E-3
Friday, June 10, 2011
Exposure problem
Friday, June 10, 2011
Friday, June 10, 2011
Friday, June 10, 2011
History of HDR imaging
Friday, June 10, 2011
HDR 1858H.P. Robinson “Fading Away
Friday, June 10, 2011
Mees (1920) 2 negative print
“The Fundamentals of Photography”
Friday, June 10, 2011
Ansel Adams
Friday, June 10, 2011
ISCC 11/05-McCann
Ansel Adams - Zone System
Friday, June 10, 2011
Jones and Condit, 1941Measurements of dynamic range of real scenes
0.0 3.01.5log range
REFLECTANCE RANGE OF PRINTS
Maximum
Average of 126 outdoor scenes
Minimum
SCENE RANGE OF WORLD
Friday, June 10, 2011
L.A.Jones & H.R.Condit, JOSA,1941
Friday, June 10, 2011
Retinex starting idea
Green record
5588
ratio = 0.62
146
ratio = 0.62
230
digit ~ luminance 119 119
Ratios are constant in sun and shadeFriday, June 10, 2011
1980Friday, June 10, 2011
Retinex cameraFriday, June 10, 2011
Capturing and reproducing the scene
Friday, June 10, 2011
Friday, June 10, 2011
Sensors dynamic range
Limited !
Friday, June 10, 2011
Is HDR a technological problem ?
Friday, June 10, 2011
Expanding sensors dynamic range• Sensors that compress their response to light due to their
logarithmic transfer function;• Multimode sensors that have a linear and a logarithmic
response at dark and bright illumination levels, (switches between linear and logarithmic modes of operation);
• Sensors with a capacity well adjustment method;• Frequency-based sensors, sensor output is converted into
pulse frequency; • Time-to-saturation [(TTS); time-to-first spike] sensors,
signal is the time the to saturated pixel; • Sensors with global control over the integration time; • Sensors with autonomous control over the integration time,
where each pixel has control over its own exposure.Spivak A, Belenky A, Fish A & Yadid-Pecht O (2009) Wide dynamic-range CMOS image sensors:
A comparative performance analysis, IEEE Trans. on Electron Devices, 56, 2446-2461.Friday, June 10, 2011
Friday, June 10, 2011
Friday, June 10, 2011
The HDR idea
http://www.adolfo.trinca.name/public/2010/11/ahdrdiagram.jpg
Friday, June 10, 2011
The HDR ideaHow ?
http://www.adolfo.trinca.name/public/2010/11/ahdrdiagram.jpg
Friday, June 10, 2011
Multiple image acquisition
Friday, June 10, 2011
•Multiple Exposures
•Use Multiple Times
•Recover scene radiances at all pixelsfrom camera digits
€
CameraDigit = radiance* time( )
New goal: Accurately measure radiances
Friday, June 10, 2011
Multiple Exposures
Flux = Luminance * time
Scene Luminance = Flux / time
Scene Luminance = Camera Digit / time
Friday, June 10, 2011
Multiple Exposures
One Spot (ScaleD)
0
50
100
150
200
250
0.0001 0.0010 0.0100 0.1000 1.0000 10.0000 100.0000 1000.0000Exposure Flux [(cd/m2) * sec]
Cam
era
Dig
it
1/8 sec1/4 sec1/2 sec1 sec2 sec4 sec8 sec16 sec32 sec64 secFIT
Flux = Luminance * time
Camera Digit
Friday, June 10, 2011
HDR file formats
Source: Reinhard et al., High Dynamic Range Imaging: Acquisition, Display, and Image-Based Lighting (The Morgan Kaufmann Series in Computer Graphics)
Friday, June 10, 2011
HDR file formats
Source: Reinhard et al., High Dynamic Range Imaging: Acquisition, Display, and Image-Based Lighting (The Morgan Kaufmann Series in Computer Graphics)
Friday, June 10, 2011
Acquisition limits
Friday, June 10, 2011
Friday, June 10, 2011
The glare problem
Friday, June 10, 2011
The glare problem
Friday, June 10, 2011
Friday, June 10, 2011
Effect of illumination
Assumes 0.0 glare
1.0 refl * 1.0 illum = 1.0 cd/m2
0.2 refl *0.01 illum = 0.002 cd/m2
Friday, June 10, 2011
Glare is image dependent
Assumes 0.001 glare
1.0 refl * 1.0 illum = 1.0 cd/m2
0.2 refl *0.01 illum = 0.002 cd/m2
0.002 cd/m2 *0.001 = 0.000002
1.0 cd/m2 *0.001 = 0.001
0.001
0.001
Friday, June 10, 2011
Ratio Signal/Glare
Assumes 0.001 glare
1.0 cd/m2)/(0.000002) = 5*10^5
( 0.002 cd/m2)) / (0.001) = 2
Friday, June 10, 2011
Sowerby, “Dictionary of Photography”, 1956
Friday, June 10, 2011
Parasitic Images
Friday, June 10, 2011
Camera limits•Glare•Unwanted scattered light in camera
•air - glass reflections •lens (number of elements)•aperture •angle off optical axis
•camera wall reflections•sensor surface reflections
•We must measure actual veiling glare limit
Friday, June 10, 2011
Measuring overall camera glare
Friday, June 10, 2011
Friday, June 10, 2011
HDR Test Setup
Friday, June 10, 2011
Text
Synthetic HDR(High-Dynamic Range)
Images
18,619:1
digit 255 = 2094.2 cd/m2
= 18,619
digit 0 = 0.11 cd/m2
2094.2 cd/m2
0.11 cd/m2
Goal Image
Friday, June 10, 2011
Targets
18,619:1
20:1
Friday, June 10, 2011
16 sec exposure - Target 1scaleBlackFriday, June 10, 2011
16 sec exposure - Target 4scaleBlackFriday, June 10, 2011
16 sec exposure - Target 4scaleBlackFriday, June 10, 2011
16 sec exposure
TextTarget 1B
Target 4B
Target 4W
Text
Friday, June 10, 2011
Constant Luminance - Variable SurroundFriday, June 10, 2011
Minimum Glare
Friday, June 10, 2011
Mild Glare
Friday, June 10, 2011
Maximum Glare
Friday, June 10, 2011
Friday, June 10, 2011
Friday, June 10, 2011
Measure In-camera Accuracy
SceneScene
Dynamic Range
In-cameraAccurate
Range
MaximumError
(% radiance)
1scaleB 20:1 20:1 0
4scaleB 18,619:1 3,000:1 300% Min
4scaleW 18,619:1 100:1 10,000% Max
4.3 log10 scene ----> 3.0 log10 image
1
Friday, June 10, 2011
Side Dupe FilmFriday, June 10, 2011
Slide Dupe FilmFriday, June 10, 2011
One Negative Capture4scale Black - Single Negative
1.50
1.70
1.90
2.10
2.30
2.50
-1.00 -0.50 0.00 0.50 1.00 1.50 2.00 2.50 3.00 3.50Log Cd/m2
Log
dig
it
3.5 Log10 units
Friday, June 10, 2011
Dynamic Range (OD)Friday, June 10, 2011
HDR from cameras
• Range of usable captured information
• Range of accurate luminance information
(much smaller)
• Scene dependent
Friday, June 10, 2011
Courtesy: M. Fairchild
Friday, June 10, 2011
Glare insertion
Gregory Ward Larson, Holly Rushmeier, and Christine Piatko, “A Visibility Matching Tone Reproduction Operator for High Dynamic Range Scenes”, IEEE Trans on VISUALIZATION AND COMPUTER GRAPHICS, VOL. 3, NO. 4, oct-dec 1997
Friday, June 10, 2011
Display: measuring the human limits
Friday, June 10, 2011
Friday, June 10, 2011
Magnitude estimates (100-1)Friday, June 10, 2011
•Luminance does not correlate uniquely with appearance
•No global tone scale can render the appearance
Friday, June 10, 2011
Magnitude Estimation of AppearanceChange Surrounds
0102030405060708090
100
0.10 1.00 10.00 100.00 1000.00 10000.00Log Luminance (cd/m2)
Mag
nit
ud
e Es
tim
atio
n
Min [0 cd/m2] Max [2094 cd/m2]
Friday, June 10, 2011
We need a new range target
•White surround•adds glare•changes surround
(simultaneous contrast)
•Vary dynamic range with•constant glare•contrast surround
Friday, June 10, 2011
Center/Surround Basic Unit
Fixed contrast surround 88%
Gray test areas 12%(small differences)
Friday, June 10, 2011
90o rotation
Friday, June 10, 2011
Friday, June 10, 2011
Testing different glares% of white surround
100%
50%0%
Friday, June 10, 2011
Single =
Double =(superimposed)
5.4 log10 range
2.7 log10 range
Single & Double Density Transparencies
Friday, June 10, 2011
5.4 & 2.7 log10 Ranges Constant Glare & Surround
Friday, June 10, 2011
0.0
10.0
20.0
30.0
40.0
50.0
60.0
70.0
80.0
90.0
100.0
0123456relative optical density
mag
nitu
de e
stim
atio
n
50% Single Density
50% whitesurround
White[100] = 0.0 rOD - Black [1] = 2.89 rOD
Friday, June 10, 2011
0
10
20
30
40
50
60
70
80
90
100
0123456relative optical density
mag
nitu
de e
stim
atio
n
50% Double Density 50% Single Density
50% whitesurround
2.3 log10 units
Friday, June 10, 2011
0
10
20
30
40
50
60
70
80
90
100
0123456relative optical density
mag
nitu
de e
stim
atio
n
White Double Density White Single Density
2.0 log10 units
100% whitesurround
Friday, June 10, 2011
0
10
20
30
40
50
60
70
80
90
100
0123456relative optical density
mag
nitu
de e
stim
atio
n
Black Double Density Black Single Density
0% whitesurround
5.0 log10 units
Friday, June 10, 2011
0
10
20
30
40
50
60
70
80
90
100
0123456relative optical density
mag
nitu
de e
stim
atio
n
Black Double Density Black Single Density
0% whitesurround
5.0 log10 units
Over 20 not big improvement
Friday, June 10, 2011
Measurements of apparent range
(depends on area of white)
•100% = 2.0 log10 units
• 50% = 2.3 log10 units
• 8% = 2.9 log10 units
Friday, June 10, 2011
DD DD DD DDFriday, June 10, 2011
Test summary
•Double transmission contrast
•Double dynamic range
•very small change in appearance range
•Visual limit ~ area of white surround
•area of white controls glare
Friday, June 10, 2011
What is on the retina: calculated retinal luminance
Friday, June 10, 2011
Friday, June 10, 2011
What comes to the retina is different from the image
High glare Low glare
Friday, June 10, 2011
Glare vs. Contrast
Veiling glare increases gray luminance
Contrast decreases gray appearance
Contrast offsets glare
Friday, June 10, 2011
Discussion
• Glare lowers the physical contrast • Spatial comparisons increase the
contrast of appearance.
• The two act in opposition. • Change with distance are different and
the cancellation is far from exact.
Friday, June 10, 2011
1 Vos, J.J. and van den Berg, T.J.T.P, CIE Research note 135/1, “Disability Glare”, ISBN 3900734976 (1999).
PIGMENTBlue eyed Caucasian 1.21Blue green Caucasian 1.02Mean over all Caucasian 1.00Brown eyed Caucasian 0.50Non Caucasian with pigmented skin and dark brown eyes 0.00
Glare Spread Function
Friday, June 10, 2011
Glare Spread Function
Plotted in log scale
Friday, June 10, 2011
False-color LookUpTable (LUT)
Dynamic Range = 5.4 ODor 251,189:1
Friday, June 10, 2011
Visualize HDR targets
Same LUT applied to SD & DD
Friday, June 10, 2011
Retinal image
Friday, June 10, 2011
Same LUT applied to SD & DD
Visualize Retinal ImagesFriday, June 10, 2011
Same LUT applied to SD & DD
Change LUT for Retinal ImagesFriday, June 10, 2011
Change LUT for Retinal ImagesFriday, June 10, 2011
Two scene-dependent spatial mechanisms:glare and contrast
Glare masks the strength of spatial contrast
Scene Retina Appearance 1,000,000:1 100:1 1,000:1
SpatialGlare
SpatialContrast
Friday, June 10, 2011
Ranges
Friday, June 10, 2011
Tone-rendering problem and spatial comparisons
Friday, June 10, 2011
Friday, June 10, 2011
Choosing a rendering intent
Friday, June 10, 2011
124
Friday, June 10, 2011
124
Friday, June 10, 2011
Friday, June 10, 2011
Friday, June 10, 2011
Friday, June 10, 2011
Land experiment
Friday, June 10, 2011
Land experiment
Friday, June 10, 2011
Land experiment
Projector
Friday, June 10, 2011
Land experiment
Projector
Friday, June 10, 2011
Land experiment
Projector
ES=100 EL=100EM=100
Friday, June 10, 2011
Land experiment
Projector Colorimeter
ES=100 EL=100EM=100
Friday, June 10, 2011
Land experiment
LS=255 LL=255LM=115
Projector Colorimeter
ES=100 EL=100EM=100
Friday, June 10, 2011
Land experiment
LS=255 LL=255LM=115
Projector Colorimeter
ES=100 EL=100EM=100
Observer
Friday, June 10, 2011
Land experiment
LS=255 LL=255LM=115
Projector Colorimeter
ES=100 EL=100EM=100
Observer
PINK
Friday, June 10, 2011
Projector Colorimeter
Observer
Land experiment
Friday, June 10, 2011
Projector Colorimeter
ES=50 EL=50EM=111
Observer
Land experiment
Friday, June 10, 2011
Projector Colorimeter
ES=50 EL=50EM=111
Observer
LS=128 LL=128LM=128
Land experiment
Friday, June 10, 2011
PINK
Projector Colorimeter
ES=50 EL=50EM=111
Observer
LS=128 LL=128LM=128
Land experiment
Friday, June 10, 2011
PINKGRAY
Projector Colorimeter
ES=50 EL=50EM=111
Observer
LS=128 LL=128LM=128
Land experiment
Friday, June 10, 2011
visual sensation
Friday, June 10, 2011
HVS: local compression of range
Friday, June 10, 2011
HVS: local compression of range
Friday, June 10, 2011
Tone mapping vs Tone rendering
No tone mapping operator (global) can mimic vision
We need an image dependent tone renderer operator (local)
Friday, June 10, 2011
Black and White Mondrian
Friday, June 10, 2011
HP 945 Images without “Frames of Reference”Friday, June 10, 2011
Some examples
Friday, June 10, 2011
Friday, June 10, 2011
Bob Sobol, HP
R. Sobol, “ Improving the Retinex algorithm for rendering
wide dynamic range photographs”, in Human Vision and Electronic
Imaging VII, B. E. Rogowitz and T. N. Pappas, ed., Proc. SPIE 4662-41, 341-348,
2002.
Friday, June 10, 2011
Friday, June 10, 2011
Original ACEOriginal ACE
ACE
Friday, June 10, 2011
STRESS Tone Rendering
Friday, June 10, 2011
Judging the results
Friday, June 10, 2011
Beauty contest
C. Gatta, A. Rizzi, D. Marini, “Perceptually inspired HDR images tone mapping with color correction”, Journal of Imaging Systems and Technology, Volume 17 Issue 5, pp. 285-294 (2007).
Friday, June 10, 2011
HDR is in the middle
Imagein CPUmemory
Scene DisplaySpatialImagein CPU
SpatialAlgorithm
GlareSensor
Pre-LUT
Post-LUTgraphics
card
Friday, June 10, 2011
Summary
Friday, June 10, 2011
• To understand HDR we need a new perspective!
1. Veiling glare limits the range on the retina 2. Neural processing (spatial) determines appearance 3. Neural is stronger than it appears [neural cancels glare] 4. General Solution requires spatial process [mimic vision] 5. Tone-Scale is limited, we need Tone-rendering [scene dependent]
Friday, June 10, 2011
Take home points
• HDR limits are not (only) technological
• Glare limits both acquisition and vision
• Glare is scene dependent
• Human vision use spatial comparison to overcome this limit
• Tone renderer operator can use the same approach
Friday, June 10, 2011
Take home points
HDR works very well
• because preserves image information
• not because are more accurate (not possible)
Friday, June 10, 2011
References• J. J. McCann, A. Rizzi, “Camera and visual veiling glare in HDR images”
Journal of the Society for Information Display 15/9, 721–730 (2007).
• J. J. McCann, “Art, Science and Appearance in HDR” Journal of the Society for Information Display 15/9, 709–719 (2007).
• A. Rizzi, J. J. McCann, “Glare-limited Appearances in HDR Images”, Journal of the Society for Information Display, 17/1, pp. 3-12, (2009).
• J. J. McCann, A. Rizzi, “Retinal HDR Images: Intraocular Glare and Object Size” Journal of the Society for Information Display, 17/11, pp. 913-920, (2009).
Friday, June 10, 2011
The art and science of HDR imagingJ.J. McCann, A. Rizzi
(expected publication date autumn 2011)
Friday, June 10, 2011