Evaluating a CameraEvaluating a Camera

KarniyatiKarniyatiResearch Advisor: Dr Roy S. Research Advisor: Dr Roy S. BernsBerns

May 14, 2005May 14, 2005

OutlinesOutlines

Project GoalsProject GoalsBackgroundBackgroundMethods and Results Methods and Results ConclusionsConclusions

Project GoalsProject Goals

Evaluate a camera for cultural heritage Evaluate a camera for cultural heritage application (artwork preservation)application (artwork preservation)

Perform a usability studyPerform a usability study

BackgroundBackground

Why do art conservators want to Why do art conservators want to preserve artwork now?preserve artwork now?–– The artworks are changing over timeThe artworks are changing over time–– Accuracy and ease of preservation have Accuracy and ease of preservation have

improvedimproved

BackgroundBackground

A method for characterizing a camera was A method for characterizing a camera was proposed by Erin P. Murphyproposed by Erin P. MurphyThe individual methods that she used are The individual methods that she used are standards published by standards published by –– ANSI (American National Standards Institute)ANSI (American National Standards Institute)–– ISO (International Organization for ISO (International Organization for

Standardization)Standardization)–– IEC (International IEC (International ElectrotechnicalElectrotechnical Commission). Commission).

Camera setCamera set--upup

SD9

43”

95” 95”

45° 45°

Black Board

Tungsten light inside of Softbox

Tungsten light inside of Softbox

Black Walls

Top ViewTop View

System Spatial UniformitySystem Spatial Uniformity

Description of ideal spatial uniformityDescription of ideal spatial uniformity

Testing Method:Testing Method:Divide image into 36 Divide image into 36 patchespatchesCompare each patch to Compare each patch to the entire imagethe entire imageCalculate % differenceCalculate % difference

System Spatial UniformitySystem Spatial Uniformity

Ideal spatial uniformity would be 0% Ideal spatial uniformity would be 0% difference (perfectly flat in the graphs)difference (perfectly flat in the graphs)

This test was very easy to performThis test was very easy to perform

Tone ReproductionTone Reproduction

A relationship between luminance (input) A relationship between luminance (input) and digital count (output) of the cameraand digital count (output) of the camera

Testing Method:Testing Method:Measure the luminance Measure the luminance of the patches of the patches Rescale the underRescale the under-- and and over exposed imagesover exposed imagesFit a gamma functionFit a gamma function

Tone ReproductionTone Reproduction

A perfectly linear input output relationship A perfectly linear input output relationship has a has a γ γ = 1= 1

This was not hard to performThis was not hard to perform

Spectral SensitivitiesSpectral Sensitivities

The testing method:The testing method:Measure the radiance Measure the radiance LinearizeLinearize the digital countsthe digital countsCalculate relative spectral Calculate relative spectral sensitivitysensitivityRotate the relative spectral Rotate the relative spectral sensitivity curves to fit CIE 2sensitivity curves to fit CIE 2°°observer color matching observer color matching functionsfunctionsCalculate Calculate µµ--factorfactor

The cameraThe camera’’s sensitivity to different colorss sensitivity to different colors

Spectral SensitivitiesSpectral Sensitivities

Perfect spectral sensitivity has a Perfect spectral sensitivity has a µµ--factor = 1factor = 1

The The µµ--factor of the whole factor of the whole imaging system is 0.83imaging system is 0.83

A A µµ--factor > 0.9 is desirablefactor > 0.9 is desirable

The tungsten lights affect the The tungsten lights affect the µµ--factor slightlyfactor slightly

This is easy to perform, however it takes a while This is easy to perform, however it takes a while to collect the imagesto collect the images

Color Reproduction Color Reproduction AccuracyAccuracyColor reproduction accuracy measures how closely a Color reproduction accuracy measures how closely a reproduced color matches the originalreproduced color matches the original

Color Reproduction Color Reproduction AccuracyAccuracy

It is desirable to have It is desirable to have ∆∆E*E*abab less than 2, but 4less than 2, but 4--6 6 is acceptableis acceptable

The The imaged targets have imaged targets have ∆∆E*E*abab values between values between 3.463.46--13.4613.46

It is easy to performIt is easy to perform

The testing method:The testing method:

Calculate the color differences Calculate the color differences

MetamerismMetamerism

Observer Observer metamerismmetamerism and illuminant and illuminant metamerismmetamerism

The testing method:The testing method:

Calculate the lowest Calculate the lowest ∆∆EE0000 match on the match on the D&H color ruleD&H color rule

MetamerismMetamerism

This is easy to performThis is easy to perform

The camera does not The camera does not ““seesee”” the same colors the same colors on the D&H color rule as the CIE 2on the D&H color rule as the CIE 2°° observerobserver

Noise Noise –– Image NoiseImage Noise

Testing methods:Testing methods:ISO 15739ISO 157394 types of noise:4 types of noise:–– Total noiseTotal noise–– Fixed pattern Fixed pattern

noisenoise–– Temporal noiseTemporal noise–– Black temporal Black temporal

noise noise

Noise Noise –– Image NoiseImage Noise

51.751.7Temporal Signal to Temporal Signal to

Noise RatioNoise Ratio

6767Fixed Pattern Signal Fixed Pattern Signal

to Noise Ratioto Noise Ratio

39.939.9Total Signal to Noise Total Signal to Noise

RatioRatio

Mean of RGB Mean of RGB ChannelsChannels

It is desirable to have low average noise and a It is desirable to have low average noise and a high signal to noise ratiohigh signal to noise ratio

348Black Temporal

Average Noise

329Temporal Average

Noise

223Fixed Pattern

Average Noise

401Total Average

Noise

Mean of RGB Channels

This is also not too difficult to performThis is also not too difficult to perform

Noise Noise –– Color NoiseColor Noise

Testing method:Testing method:Calculate the % standard Calculate the % standard deviation deviation Calculate the mean color Calculate the mean color difference from the mean difference from the mean (MCDM)(MCDM)

It is desirable to have very small % standard It is desirable to have very small % standard deviation and MCDMdeviation and MCDM

Noise Noise –– Color NoiseColor Noise

It is desirable to have very small % standard It is desirable to have very small % standard deviation and MCDMdeviation and MCDM

% standard deviation: % standard deviation:

•• least color noise least color noise –– red channelred channel

•• most color noise most color noise –– blue channel blue channel

This is not hard to performThis is not hard to perform

MCDM:MCDM:•• least color noise least color noise –– magenta and white patchesmagenta and white patches•• most color noise most color noise –– black patchblack patch

Dynamic RangeDynamic Range

The range of irradiance values that the The range of irradiance values that the camera can capture in one image camera can capture in one image

Testing Method:Testing Method:Calculate the logCalculate the log1010 of the of the luminance ratioluminance ratio

Dynamic RangeDynamic Range

3.6123.612Theoretical Dynamic Range (Density)Theoretical Dynamic Range (Density)

2.8842.884ISO Digital Still Camera Dynamic Range (Density)ISO Digital Still Camera Dynamic Range (Density)

Mean of RGB ChannelsMean of RGB Channels

It is desirable to have a large dynamic It is desirable to have a large dynamic rangerange

It is also not hard to performIt is also not hard to perform

Spatial CrossSpatial Cross--talktalk

The testing method:The testing method:Calculate the mean DC values Calculate the mean DC values of the 15 gray patches from of the 15 gray patches from the two images the two images LinearizeLinearize the mean the mean DCsDCsCalculate % relative maximum Calculate % relative maximum differencedifference

The signal from one bright pixel spills The signal from one bright pixel spills over into the surrounding pixelsover into the surrounding pixels

Spatial CrossSpatial Cross--talktalk

It is desirable to have a maximum difference It is desirable to have a maximum difference as close to 0% as possibleas close to 0% as possible

This is not hard to performThis is not hard to perform

The percent relative maximum difference is The percent relative maximum difference is 8.54%8.54%

Color Channel Color Channel RegistrationRegistration

Correct color channel registrationCorrect color channel registration

The testing method:The testing method:

• BurnBurn’’s sfrmat2 MATLAB s sfrmat2 MATLAB programprogram

Color Channel Color Channel RegistrationRegistration

It is desirable to have a color channel It is desirable to have a color channel misregistrationmisregistration error < 0.5 pixelserror < 0.5 pixels

The mean of The mean of misregistrationmisregistration shift of RGB shift of RGB channels is 0.09 with maximum 0.33channels is 0.09 with maximum 0.33

ItIt’’s easy to performs easy to perform

DifficultiesDifficulties

ItIt’’s hard to keep the target flat against s hard to keep the target flat against the black boardthe black boardDust in the Dust in the cameracameraSince the targets are not all the same Since the targets are not all the same size, it took a while to center themsize, it took a while to center themDesirable values were usually vagueDesirable values were usually vague–– More specific guidelines are neededMore specific guidelines are needed

ConclusionsConclusions

The camera does not meet the criteria The camera does not meet the criteria for artwork preservationfor artwork preservation

MurphyMurphy’’s method is easy to perform, s method is easy to perform, but could be improvedbut could be improved