Computational Colour Vision

Stephen WestlandCentre for Colour Design Technology

University of Leedss.westland@leeds.ac.uk

June 2005

http://www.colourtech.org

Oxford Brookes University

Computational Colour Vision

Introduce some basic concepts - the physical basis of colour

Computational approaches to how colour vision works

Phenomenology of colour perception (the problem)

Computational and psychophysical studies of transparency perception

The Physical Basis of Colour

C() = E()P()

The colour signal C() is the product at each wavelength of the power in the light source and the reflectance of the object

E()

P()

E()P()

Cone spectral sensitivity

LM

S L = E()P()L()

M = E()P()M()

S = E()P()S()

∫

∫

∫

Cone Responses

L = E()P()L()

M = E()P()M()

S = E()P()S()

∫

∫

∫Each cone produces a univariant response

LMS

Colour perception stems from the comparative responses of the three cone responses

Colour is a perception – ‘the rays are not coloured’

Colour Constancy

Objects tend to retain their approximate daylight appearance when viewed under a wide range of different light sources

P

0.01

0.99

Indoors (100 cd/m2)

1

99

Outdoors (10,000 cd/m2)

100

9900

The visual system is able to discount changes in the intensity or spectral composition of the illumination

WHY? / HOW?

noonnoon sunsetsunset

X

X

Computational Explanation

L1 = E1()P()L()

M1 = E1()P()M()

S1 = E1()P()S()

∫

∫

∫

L2 = E2()P()L()

M2 = E2()P()M()

S2 = E2()P()S()

∫

∫

∫

L1 / L1W = L2 / L2W

M1 / M1W = M2 / M2W

S1 / S1W = S2 / S2W

e1 = De2

D = L1W/L2W

M1W/M2W

S1W/S2W

000

000

e1 = L1

M1

S1

e2 = L2

M2

S2

Practical Use – Colour Correction

Camera RGB values vary for a scene depending upon the light source

colour correction

In order to correct the images we need an estimate of the light source under which the original image was taken

brightest pixel is white grey-world hypothesis

Colour Constancy

Adaptation is too slow to explain colour constancy

“Any visual system that achieves colour constancy is making use of the constraints in the statistics of surfaces and lights” – Maloney (1986)

Is it possible for the visual system to recover the spectral reflectance factors of the surfaces in scenes from the cone responses?

L = E()P()L()

M = E()P()M()

S = E()P()S()

∫

∫

∫

P wiBi()

Using a process such as SVD or PCA we can compute a set of basis functions Bi() such that each reflectancespectrum may be represented by a linear sum of basis functions - a linear model of low dimensionality.

If we use n basis functions then each spectrum can be represented by just n scalars or weights.

Basis Functions

1 Basis Function

Original 1 BF

P() = w1B1()

400 450 500 550 600 650 700-0.4

-0.2

0

0.2

0.4

0.6

0.8

Wavelength

Val

ue

400 450 500 550 600 650 7000.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

0.45

0.5

0.55

Ref

lect

ance

val

ue

Wavelength

2 Basis Functions

Original 1 BF 2 BF

P() = w1B1() + w2B2()

400 450 500 550 600 650 700-0.4

-0.2

0

0.2

0.4

0.6

0.8

Wavelength

Val

ue

400 450 500 550 600 650 7000.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

0.45

0.5

0.55

Ref

lect

ance

val

ue

Wavelength

3 Basis Functions

400 450 500 550 600 650 700-0.4

-0.2

0

0.2

0.4

0.6

0.8

Wavelength

Val

ue

400 450 500 550 600 650 7000.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

0.45

0.5

0.55

Ref

lect

ance

val

ue

Wavelength

Original 1 BF 2 BF 3 BF

P() = w1B1() + w2B2() + w3B3()

PC Variance Total Variance

1 76.77 76.77 2 15.83 92.60 3 5.96 98.56

4 0.76 99.32

5 0.37 99.68

6 0.12 99.80

7 0.09 99.89

8 0.04 99.93

About 99% of the variancecan be accounted for by a 3-D model (Maloney & Wandell, 1986)

But what proportion of the variance do we need to account for?

How many Basis Functions are Required?

6-9 basis functions are required

Simultaneous Contrast

original original covered by filter

original with small filter

Colour Constancy - spatial comparisons

“For the qualities of lights and colours are perceived by theeye only by comparing them with one another” (Alhazen, 1025)

“… object colour depends upon the ratios of light reflected from the various parts of the visual field rather than on the absolute amount of light reflected” (Marr)

ei,1/ei,2 = e'i,1/e'i,2 (Foster)

i = {L, M, S} Ratio under first light source

Ratio under second light source

Spatial Comparison of Cone Excitations

Retinex – Land and McCann (1971)

Foster and Nascimento (1994)

L1

L2=k

L’1

L’2

=k

Transparency Perception

(Ripamonti and Westland, 2001)

e’1 e’2

e1 e2

e1/e2 = e’1/e’2

What is transparency?

An object is (physically) transparent if some proportion of the incident radiation that falls upon the object is able to pass through the object.

What is perceptual transparency?

Perceptual transparency is the process ‘of seeing one objectthrough another’ (Helmholz, 1867)

Physical transparency is neither a necessary or sufficient condition for perceptual transparency (Metelli, 1974)

Even in the complete absence of any physical transparencyit is possible to experience perceptual transparency

Perceptual transparency

Research Questions

What mechanisms could drive perceptual transparency?

What are the chromatic conditions that cause transparency?

Could transparency and colour constancy be linked?

Perceptual transparency

Transparency and Spatial Ratios

ei,1/ei,2 = e'i,1/e'i,2

ei,1 ei,2

T()

e'i,1e'i,2

Experimental

Computational analysis to investigate whether for physical transparency the cone ratios are preserved

Psychophysical study to investigate whether the invariance of spatial ratios can predict chromatic conditions for perceptual transparency

Psychophysical study to compare the performance of the ratio-invariance model when the number of surfaces is varied

opaque surface P

T

(1-b)bP2T4(1-b)PT2

(1-b)b2P3T6

Physical Model of Transparency

b

P'() = P()[T()(1-b)2]2(Wyszecki & Stiles, 1982)

Monte Carlo Simulation

4. Steps 1-3 repeated 1000 times

1. A pair of surfaces P1() and P2() were randomly selected

2. A filter was randomly selected (defined by a gaussiandistribution)

m

3. The cone excitations were computed for the surfaces viewed directly (under D65) and through the filter

ei,1/ei,2 e'i,1/e'i,2

P1() P2()

ei,1/ei,2

e' i,

1/e

' i,2

i,2e

'i,1e

i,2'e

i,1eMonte Carlo Results

i,2e

'i,1e

i,2'e

i,1eMonte Carlo Results

The ratios are approximately invariant

Invariance is slightly better for the S cones

Invariance decreases as the spectral transmittancedecreases

filter S cones M cones L cones = 10nm 0.9988 (0.9968) 0.9964 (0.9951) 0.9996 (0.9955) = 50nm 0.9978 (0.9231) 1.0037 (0.8666) 0.9788 (0.8599) = 200nm 0.9231 (0.8032) 0.8885 (0.7154) 0.9144 (0.7284)

Da Pos, 1989, D’Zmura et al., 1997

xB

xP xQ

g

xA

xP

xB

xQ

xP = xA + (1-) gxQ = xB +(1- ) g

xA

Convergence

(a) convergent(deviation

0)

(b) invariant

(deviation = 0)

deviationi = 1 - [ei,1/ei,2]/[ e'i,1/e'i,2]

Psychophysical Stimuli I

d'<0 indicates subjects' preference for convergent filter; d'=0 no preference; d'>0 indicates subjects' preference for invariant filter

-3

-1

1

3

5

0 0.1 0.2 0.3 0.4 0.5 0.6

LMS deviations

d'

L M S Log. (L) Log. (S) Log. (M)

Psychophysical Results I

(a)realfilter

number of surfaces

2 4 6 8 12

(b)filter

with noise

Psychophysical Stimuli II

y = 1.1 Ln (x + 2.23)

-1

0

1

2

3

4

5

0 2 4 6 8 10 12 14

number of surfaces

d'L

M

LMS

Psychophysical Results II

Computational and pyschophysical studies show that the invariance of cone-excitation ratiosmay be a useful cue driving transparency perception

Conclusions

Colour constancy and transparency perceptionmay be related. Could they result from similar mechanisms, perhaps even similar groups of neurones?

There are still a mass of unsolved problems in computational colour vision including the relationship between cone excitations and the actual sensation of colour

xB

xP xQ

g

xA

xP

xB

xQ

xA

xP = xA + (1-) gxQ = xB +(1- ) g

xP = xA xQ = xB

Cone excitations are transformed by a a diagonal matrix whose diagonal elements are all equal

xB

xP xQ

xA xP = xA

xQ = xB

Cone excitations are transformed by a a diagonal matrix whose diagonal elements are not necessarily all equal

The two models can be made to be the same if the convergence model has no additive component and if the invariance model has equal cone scaling