Automatic Compensation for Camera Settings Automatic Compensation for Camera Settings for Images Taken under Different Illuminantsfor Images Taken under Different Illuminants

School of Electrical Engineering and Computer Science School of Electrical Engineering and Computer Science Kyungpook National Univ.Kyungpook National Univ.

FourteenthFourteenth Color Imaging ConferenceColor Imaging Conference

20062006

Cheng Lu and Mark S. DrewCheng Lu and Mark S. Drew

22 / 16 / 16

Combination of two images for the same scene under different illumination– Estimating scene illumination

– Enhancing photographs shot for shadow removal

Proposed method– Adjusting parametrically two images to compensate

for the difference in camera setting• Exposure speed, ISO, aperture size, and white balance

– Existing effect of illumination

Solving by using shadow effect

AbstractAbstract

33 / 16 / 16

Work on image pairs taken for the same scene under different illumination– Analysis of the difference image (with flash image and

no-flash image)• Estimating surface reflectance and illumination in the no-flash image

• Enhancing photographs in dark environments and removing red-eye effects

• Removing the ambient shadow from the no-flash image

– Inference of the contribution of the flash • Must be in same camera settings

• Different lighting conditions

Causing changes of camera settings

• Not investigated in this problem

IntroductionIntroduction

44 / 16 / 16

Proposed method– Masking model to describe camera settings

• Assuming additivity and proportionality of the different factors involved

• Using 2nd-order masking model with 9 parameters

– Training phase• Image A ; only one light source, illuminant 1

• Image B ; two light sources, illuminant 1 and illuminant 2

55 / 16 / 16

– An example of images

Fig. 1. (a,b): Ambient-light image A, unscaled, and scaled for display.(c,d): “Both” image B (ambient + flash), unscaled, and scaled.

(a)

(c)

(b)

(d)

66 / 16 / 16

Additivity property– Subtracting the image A from image B

• Taken under illuminant 2 only

– Situation with flash image and no-flash image• Difference image

– Pure flash image

– Disappearance of shadow caused by sunlight

77 / 16 / 16

Consumer-grade cameras settings– Focal length

– Exposure time (shutter speed)

– Aperture (f-number)

– ISO (film speed)

– White balance

Other settings– Fixing the focal length

– Using a tripod

– Using RAW format

Camera Settings and Camera Settings and Image AcquisitionImage Acquisition

88 / 16 / 16

Exposure value (EV)– Sum of Aperture Value (AV) and Time Value (TV)

• AV

• TV

ISO (Film speed)– Measure of a photographic film’s sensitivity to light

– Using 100 and 200

White balance settings– Auto, Daylight, Fluorescent, and Tungsten

TVAVEV

NNAV 22

2 log2log

tTV t 21

2 loglog

Where N ; the f-number

Where t ; the shutter time in seconds

99 / 16 / 16

Goal ; Finding a model – Describing the relation between the difference of the

two images and the camera settings

– Transforming A to A’• Shadows caused by illuminant 1 will be removed in the

difference image B−A’

– ; shadow region, ; out-of-shadow region

A Masking Model for Compensating A Masking Model for Compensating for Camera Settingsfor Camera Settings

s ns

Fig. 2. Ambient-light image A with in-shadow and out-of-shadow regions.

1010 / 16 / 16

– Transforming A to A’ via a 3-coefficient vector M• M ; a coefficient for each color channel

– Using compensation of the camera settings of the two images

– Function of the ratios of exposure value, ISO, and white balance between the two images

,)()(

)()())()(()()(

)(')(')()()(')()(')(

sAnsA

sBnsBM

sAnsAMsBnsBLetsAnsAsBnsB

sAsBnsAnsB

no shadow effect

M is a 3-vector.

1111 / 16 / 16

– 2nd-order masking model • Describing the difference by camera settings

• Proposed originally for characterizing color printers– Using logarithms

– Assuming additivity and proportionality

• The form of the 2nd-order model

A

B

iA

iB

iA

iB

A

B

A

B

A

B

iA

iB

A

B

A

B

iA

iB

A

B

A

B

ii

ii

EVEV

mean

mean

mean

mean

ISOISO

ISOISO

EVEV

mean

mean

ISOISO

EVEV

mean

mean

ISOISO

EVEV

isAnsAsBnsB

ccc

bbb

aaa

iM

loglogloglogloglog

logloglog

logloglog

3..1,loglog

321

2

3

2

2

2

1

321

)()()()(

1212 / 16 / 16

– Difference image

Fig. 3. (a): Pure-flash image (B−A) without compensation(b): With compensation (B-A’)

(a) (b)

1313 / 16 / 16

Experimental imaging environment– Using five lighting sources

• Direct sunlight

• Cloudy daylight

• Tungsten light lamp

• Incandescent lamp

• Xenon flash light

– Using five objects with different colors on five different tablecloths

Experiments and ResultsExperiments and Results

Fig. 4. Experimental imaging environment

1414 / 16 / 16

– Capture of images under five situations• Using direct sunlight as illuminant 1 to create shadows and

adding the flash as illuminant 2

• Using tungsten light as illuminant 1 and adding the flash as illuminant 2

• Using tungsten light as illuminant 1 and adding cloudy daylight as illuminant 2

• Using incandescent light as illuminant 1 and adding the flash as illuminant 2

• Using incandescent light as illuminant 1 and adding cloudy daylight as illuminant 2

1515 / 16 / 16

Examples of the results

Fig. 5,6. Results: (a) images A, (b) image B, (c) image B−A’

(a) (b) (c)

1616 / 16 / 16

Addressing the problem– Compensation for camera settings for image pairs for

the same scene under different lighting conditions

– Difficulty• Difference of the two images

– Composite of camera settings and scene illumination

Proposed method– Simple masking model to describe camera settings

SummarySummary