Computational Colour Vision
-
Upload
shea-buckley -
Category
Documents
-
view
30 -
download
0
description
Transcript of Computational Colour Vision
![Page 1: Computational Colour Vision](https://reader035.fdocuments.us/reader035/viewer/2022062421/56812a7a550346895d8e0586/html5/thumbnails/1.jpg)
Computational Colour Vision
Stephen WestlandCentre for Colour Design Technology
University of [email protected]
June 2005
http://www.colourtech.org
Oxford Brookes University
![Page 2: Computational Colour Vision](https://reader035.fdocuments.us/reader035/viewer/2022062421/56812a7a550346895d8e0586/html5/thumbnails/2.jpg)
Computational Colour Vision
Introduce some basic concepts - the physical basis of colour
Computational approaches to how colour vision works
Phenomenology of colour perception (the problem)
Computational and psychophysical studies of transparency perception
![Page 3: Computational Colour Vision](https://reader035.fdocuments.us/reader035/viewer/2022062421/56812a7a550346895d8e0586/html5/thumbnails/3.jpg)
The Physical Basis of Colour
C() = E()P()
The colour signal C() is the product at each wavelength of the power in the light source and the reflectance of the object
E()
P()
E()P()
![Page 4: Computational Colour Vision](https://reader035.fdocuments.us/reader035/viewer/2022062421/56812a7a550346895d8e0586/html5/thumbnails/4.jpg)
Cone spectral sensitivity
LM
S L = E()P()L()
M = E()P()M()
S = E()P()S()
∫
∫
∫
![Page 5: Computational Colour Vision](https://reader035.fdocuments.us/reader035/viewer/2022062421/56812a7a550346895d8e0586/html5/thumbnails/5.jpg)
Cone Responses
L = E()P()L()
M = E()P()M()
S = E()P()S()
∫
∫
∫Each cone produces a univariant response
LMS
Colour perception stems from the comparative responses of the three cone responses
Colour is a perception – ‘the rays are not coloured’
![Page 6: Computational Colour Vision](https://reader035.fdocuments.us/reader035/viewer/2022062421/56812a7a550346895d8e0586/html5/thumbnails/6.jpg)
Colour Constancy
Objects tend to retain their approximate daylight appearance when viewed under a wide range of different light sources
P
0.01
0.99
Indoors (100 cd/m2)
1
99
Outdoors (10,000 cd/m2)
100
9900
The visual system is able to discount changes in the intensity or spectral composition of the illumination
WHY? / HOW?
![Page 7: Computational Colour Vision](https://reader035.fdocuments.us/reader035/viewer/2022062421/56812a7a550346895d8e0586/html5/thumbnails/7.jpg)
![Page 8: Computational Colour Vision](https://reader035.fdocuments.us/reader035/viewer/2022062421/56812a7a550346895d8e0586/html5/thumbnails/8.jpg)
noonnoon sunsetsunset
![Page 9: Computational Colour Vision](https://reader035.fdocuments.us/reader035/viewer/2022062421/56812a7a550346895d8e0586/html5/thumbnails/9.jpg)
X
![Page 10: Computational Colour Vision](https://reader035.fdocuments.us/reader035/viewer/2022062421/56812a7a550346895d8e0586/html5/thumbnails/10.jpg)
X
![Page 11: Computational Colour Vision](https://reader035.fdocuments.us/reader035/viewer/2022062421/56812a7a550346895d8e0586/html5/thumbnails/11.jpg)
Computational Explanation
L1 = E1()P()L()
M1 = E1()P()M()
S1 = E1()P()S()
∫
∫
∫
L2 = E2()P()L()
M2 = E2()P()M()
S2 = E2()P()S()
∫
∫
∫
L1 / L1W = L2 / L2W
M1 / M1W = M2 / M2W
S1 / S1W = S2 / S2W
e1 = De2
D = L1W/L2W
M1W/M2W
S1W/S2W
000
000
e1 = L1
M1
S1
e2 = L2
M2
S2
![Page 12: Computational Colour Vision](https://reader035.fdocuments.us/reader035/viewer/2022062421/56812a7a550346895d8e0586/html5/thumbnails/12.jpg)
Practical Use – Colour Correction
Camera RGB values vary for a scene depending upon the light source
colour correction
In order to correct the images we need an estimate of the light source under which the original image was taken
brightest pixel is white grey-world hypothesis
![Page 13: Computational Colour Vision](https://reader035.fdocuments.us/reader035/viewer/2022062421/56812a7a550346895d8e0586/html5/thumbnails/13.jpg)
Colour Constancy
Adaptation is too slow to explain colour constancy
“Any visual system that achieves colour constancy is making use of the constraints in the statistics of surfaces and lights” – Maloney (1986)
Is it possible for the visual system to recover the spectral reflectance factors of the surfaces in scenes from the cone responses?
L = E()P()L()
M = E()P()M()
S = E()P()S()
∫
∫
∫
![Page 14: Computational Colour Vision](https://reader035.fdocuments.us/reader035/viewer/2022062421/56812a7a550346895d8e0586/html5/thumbnails/14.jpg)
P wiBi()
Using a process such as SVD or PCA we can compute a set of basis functions Bi() such that each reflectancespectrum may be represented by a linear sum of basis functions - a linear model of low dimensionality.
If we use n basis functions then each spectrum can be represented by just n scalars or weights.
Basis Functions
![Page 15: Computational Colour Vision](https://reader035.fdocuments.us/reader035/viewer/2022062421/56812a7a550346895d8e0586/html5/thumbnails/15.jpg)
1 Basis Function
Original 1 BF
P() = w1B1()
400 450 500 550 600 650 700-0.4
-0.2
0
0.2
0.4
0.6
0.8
Wavelength
Val
ue
400 450 500 550 600 650 7000.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
0.45
0.5
0.55
Ref
lect
ance
val
ue
Wavelength
![Page 16: Computational Colour Vision](https://reader035.fdocuments.us/reader035/viewer/2022062421/56812a7a550346895d8e0586/html5/thumbnails/16.jpg)
2 Basis Functions
Original 1 BF 2 BF
P() = w1B1() + w2B2()
400 450 500 550 600 650 700-0.4
-0.2
0
0.2
0.4
0.6
0.8
Wavelength
Val
ue
400 450 500 550 600 650 7000.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
0.45
0.5
0.55
Ref
lect
ance
val
ue
Wavelength
![Page 17: Computational Colour Vision](https://reader035.fdocuments.us/reader035/viewer/2022062421/56812a7a550346895d8e0586/html5/thumbnails/17.jpg)
3 Basis Functions
400 450 500 550 600 650 700-0.4
-0.2
0
0.2
0.4
0.6
0.8
Wavelength
Val
ue
400 450 500 550 600 650 7000.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
0.45
0.5
0.55
Ref
lect
ance
val
ue
Wavelength
Original 1 BF 2 BF 3 BF
P() = w1B1() + w2B2() + w3B3()
![Page 18: Computational Colour Vision](https://reader035.fdocuments.us/reader035/viewer/2022062421/56812a7a550346895d8e0586/html5/thumbnails/18.jpg)
PC Variance Total Variance
1 76.77 76.77 2 15.83 92.60 3 5.96 98.56
4 0.76 99.32
5 0.37 99.68
6 0.12 99.80
7 0.09 99.89
8 0.04 99.93
About 99% of the variancecan be accounted for by a 3-D model (Maloney & Wandell, 1986)
But what proportion of the variance do we need to account for?
How many Basis Functions are Required?
6-9 basis functions are required
![Page 19: Computational Colour Vision](https://reader035.fdocuments.us/reader035/viewer/2022062421/56812a7a550346895d8e0586/html5/thumbnails/19.jpg)
Simultaneous Contrast
![Page 20: Computational Colour Vision](https://reader035.fdocuments.us/reader035/viewer/2022062421/56812a7a550346895d8e0586/html5/thumbnails/20.jpg)
![Page 21: Computational Colour Vision](https://reader035.fdocuments.us/reader035/viewer/2022062421/56812a7a550346895d8e0586/html5/thumbnails/21.jpg)
![Page 22: Computational Colour Vision](https://reader035.fdocuments.us/reader035/viewer/2022062421/56812a7a550346895d8e0586/html5/thumbnails/22.jpg)
original original covered by filter
original with small filter
![Page 23: Computational Colour Vision](https://reader035.fdocuments.us/reader035/viewer/2022062421/56812a7a550346895d8e0586/html5/thumbnails/23.jpg)
Colour Constancy - spatial comparisons
“For the qualities of lights and colours are perceived by theeye only by comparing them with one another” (Alhazen, 1025)
“… object colour depends upon the ratios of light reflected from the various parts of the visual field rather than on the absolute amount of light reflected” (Marr)
ei,1/ei,2 = e'i,1/e'i,2 (Foster)
i = {L, M, S} Ratio under first light source
Ratio under second light source
![Page 24: Computational Colour Vision](https://reader035.fdocuments.us/reader035/viewer/2022062421/56812a7a550346895d8e0586/html5/thumbnails/24.jpg)
Spatial Comparison of Cone Excitations
Retinex – Land and McCann (1971)
Foster and Nascimento (1994)
L1
L2=k
L’1
L’2
=k
![Page 25: Computational Colour Vision](https://reader035.fdocuments.us/reader035/viewer/2022062421/56812a7a550346895d8e0586/html5/thumbnails/25.jpg)
Transparency Perception
(Ripamonti and Westland, 2001)
e’1 e’2
e1 e2
e1/e2 = e’1/e’2
![Page 26: Computational Colour Vision](https://reader035.fdocuments.us/reader035/viewer/2022062421/56812a7a550346895d8e0586/html5/thumbnails/26.jpg)
What is transparency?
An object is (physically) transparent if some proportion of the incident radiation that falls upon the object is able to pass through the object.
![Page 27: Computational Colour Vision](https://reader035.fdocuments.us/reader035/viewer/2022062421/56812a7a550346895d8e0586/html5/thumbnails/27.jpg)
What is perceptual transparency?
Perceptual transparency is the process ‘of seeing one objectthrough another’ (Helmholz, 1867)
Physical transparency is neither a necessary or sufficient condition for perceptual transparency (Metelli, 1974)
Even in the complete absence of any physical transparencyit is possible to experience perceptual transparency
![Page 28: Computational Colour Vision](https://reader035.fdocuments.us/reader035/viewer/2022062421/56812a7a550346895d8e0586/html5/thumbnails/28.jpg)
Perceptual transparency
![Page 29: Computational Colour Vision](https://reader035.fdocuments.us/reader035/viewer/2022062421/56812a7a550346895d8e0586/html5/thumbnails/29.jpg)
Research Questions
What mechanisms could drive perceptual transparency?
What are the chromatic conditions that cause transparency?
Could transparency and colour constancy be linked?
![Page 30: Computational Colour Vision](https://reader035.fdocuments.us/reader035/viewer/2022062421/56812a7a550346895d8e0586/html5/thumbnails/30.jpg)
Perceptual transparency
![Page 31: Computational Colour Vision](https://reader035.fdocuments.us/reader035/viewer/2022062421/56812a7a550346895d8e0586/html5/thumbnails/31.jpg)
Transparency and Spatial Ratios
ei,1/ei,2 = e'i,1/e'i,2
ei,1 ei,2
T()
e'i,1e'i,2
![Page 32: Computational Colour Vision](https://reader035.fdocuments.us/reader035/viewer/2022062421/56812a7a550346895d8e0586/html5/thumbnails/32.jpg)
Experimental
Computational analysis to investigate whether for physical transparency the cone ratios are preserved
Psychophysical study to investigate whether the invariance of spatial ratios can predict chromatic conditions for perceptual transparency
Psychophysical study to compare the performance of the ratio-invariance model when the number of surfaces is varied
![Page 33: Computational Colour Vision](https://reader035.fdocuments.us/reader035/viewer/2022062421/56812a7a550346895d8e0586/html5/thumbnails/33.jpg)
opaque surface P
T
(1-b)bP2T4(1-b)PT2
(1-b)b2P3T6
Physical Model of Transparency
b
P'() = P()[T()(1-b)2]2(Wyszecki & Stiles, 1982)
![Page 34: Computational Colour Vision](https://reader035.fdocuments.us/reader035/viewer/2022062421/56812a7a550346895d8e0586/html5/thumbnails/34.jpg)
Monte Carlo Simulation
4. Steps 1-3 repeated 1000 times
1. A pair of surfaces P1() and P2() were randomly selected
2. A filter was randomly selected (defined by a gaussiandistribution)
m
3. The cone excitations were computed for the surfaces viewed directly (under D65) and through the filter
ei,1/ei,2 e'i,1/e'i,2
P1() P2()
![Page 35: Computational Colour Vision](https://reader035.fdocuments.us/reader035/viewer/2022062421/56812a7a550346895d8e0586/html5/thumbnails/35.jpg)
ei,1/ei,2
e' i,
1/e
' i,2
i,2e
'i,1e
i,2'e
i,1eMonte Carlo Results
![Page 36: Computational Colour Vision](https://reader035.fdocuments.us/reader035/viewer/2022062421/56812a7a550346895d8e0586/html5/thumbnails/36.jpg)
i,2e
'i,1e
i,2'e
i,1eMonte Carlo Results
The ratios are approximately invariant
Invariance is slightly better for the S cones
Invariance decreases as the spectral transmittancedecreases
filter S cones M cones L cones = 10nm 0.9988 (0.9968) 0.9964 (0.9951) 0.9996 (0.9955) = 50nm 0.9978 (0.9231) 1.0037 (0.8666) 0.9788 (0.8599) = 200nm 0.9231 (0.8032) 0.8885 (0.7154) 0.9144 (0.7284)
![Page 37: Computational Colour Vision](https://reader035.fdocuments.us/reader035/viewer/2022062421/56812a7a550346895d8e0586/html5/thumbnails/37.jpg)
Da Pos, 1989, D’Zmura et al., 1997
xB
xP xQ
g
xA
xP
xB
xQ
xP = xA + (1-) gxQ = xB +(1- ) g
xA
Convergence
![Page 38: Computational Colour Vision](https://reader035.fdocuments.us/reader035/viewer/2022062421/56812a7a550346895d8e0586/html5/thumbnails/38.jpg)
(a) convergent(deviation
0)
(b) invariant
(deviation = 0)
deviationi = 1 - [ei,1/ei,2]/[ e'i,1/e'i,2]
Psychophysical Stimuli I
![Page 39: Computational Colour Vision](https://reader035.fdocuments.us/reader035/viewer/2022062421/56812a7a550346895d8e0586/html5/thumbnails/39.jpg)
d'<0 indicates subjects' preference for convergent filter; d'=0 no preference; d'>0 indicates subjects' preference for invariant filter
-3
-1
1
3
5
0 0.1 0.2 0.3 0.4 0.5 0.6
LMS deviations
d'
L M S Log. (L) Log. (S) Log. (M)
Psychophysical Results I
![Page 40: Computational Colour Vision](https://reader035.fdocuments.us/reader035/viewer/2022062421/56812a7a550346895d8e0586/html5/thumbnails/40.jpg)
(a)realfilter
number of surfaces
2 4 6 8 12
(b)filter
with noise
Psychophysical Stimuli II
![Page 41: Computational Colour Vision](https://reader035.fdocuments.us/reader035/viewer/2022062421/56812a7a550346895d8e0586/html5/thumbnails/41.jpg)
y = 1.1 Ln (x + 2.23)
-1
0
1
2
3
4
5
0 2 4 6 8 10 12 14
number of surfaces
d'L
M
LMS
Psychophysical Results II
![Page 42: Computational Colour Vision](https://reader035.fdocuments.us/reader035/viewer/2022062421/56812a7a550346895d8e0586/html5/thumbnails/42.jpg)
Computational and pyschophysical studies show that the invariance of cone-excitation ratiosmay be a useful cue driving transparency perception
Conclusions
Colour constancy and transparency perceptionmay be related. Could they result from similar mechanisms, perhaps even similar groups of neurones?
There are still a mass of unsolved problems in computational colour vision including the relationship between cone excitations and the actual sensation of colour
![Page 43: Computational Colour Vision](https://reader035.fdocuments.us/reader035/viewer/2022062421/56812a7a550346895d8e0586/html5/thumbnails/43.jpg)
xB
xP xQ
g
xA
xP
xB
xQ
xA
xP = xA + (1-) gxQ = xB +(1- ) g
xP = xA xQ = xB
Cone excitations are transformed by a a diagonal matrix whose diagonal elements are all equal
![Page 44: Computational Colour Vision](https://reader035.fdocuments.us/reader035/viewer/2022062421/56812a7a550346895d8e0586/html5/thumbnails/44.jpg)
xB
xP xQ
xA xP = xA
xQ = xB
Cone excitations are transformed by a a diagonal matrix whose diagonal elements are not necessarily all equal
The two models can be made to be the same if the convergence model has no additive component and if the invariance model has equal cone scaling