Introduction of the intrinsic image. Intrinsic Images The method of Finlayson & Hordley ( 2001 )...
-
date post
22-Dec-2015 -
Category
Documents
-
view
232 -
download
1
Transcript of Introduction of the intrinsic image. Intrinsic Images The method of Finlayson & Hordley ( 2001 )...
![Page 1: Introduction of the intrinsic image. Intrinsic Images The method of Finlayson & Hordley ( 2001 ) Two assumptions 1. the camera ’ s sensors are sufficiently.](https://reader035.fdocuments.us/reader035/viewer/2022081419/56649d765503460f94a57fde/html5/thumbnails/1.jpg)
Introduction of the intrinsic image
![Page 2: Introduction of the intrinsic image. Intrinsic Images The method of Finlayson & Hordley ( 2001 ) Two assumptions 1. the camera ’ s sensors are sufficiently.](https://reader035.fdocuments.us/reader035/viewer/2022081419/56649d765503460f94a57fde/html5/thumbnails/2.jpg)
Intrinsic Images
The method of Finlayson & Hordley ( 2001 )
Two assumptions 1. the camera’s sensors are sufficiently narrow band. 2. the illuminant can be approximated by a black-body radiator.
![Page 3: Introduction of the intrinsic image. Intrinsic Images The method of Finlayson & Hordley ( 2001 ) Two assumptions 1. the camera ’ s sensors are sufficiently.](https://reader035.fdocuments.us/reader035/viewer/2022081419/56649d765503460f94a57fde/html5/thumbnails/3.jpg)
The power spectrum of a black-body radiator The radiance of a black-body
radiator
temperature T ( Kelvin ) , wavelength , Planck’s constant
Boltzmann’s constant ,
the speed of light
![Page 4: Introduction of the intrinsic image. Intrinsic Images The method of Finlayson & Hordley ( 2001 ) Two assumptions 1. the camera ’ s sensors are sufficiently.](https://reader035.fdocuments.us/reader035/viewer/2022081419/56649d765503460f94a57fde/html5/thumbnails/4.jpg)
Finlayson & Hordley : many fluorescent light sources can be approximated by a black-body radiator.
![Page 5: Introduction of the intrinsic image. Intrinsic Images The method of Finlayson & Hordley ( 2001 ) Two assumptions 1. the camera ’ s sensors are sufficiently.](https://reader035.fdocuments.us/reader035/viewer/2022081419/56649d765503460f94a57fde/html5/thumbnails/5.jpg)
Simplification of the equation
The temperature T < 10000 K. The visual spectrum ranges : 400 n
m ~ 700 nm.
![Page 6: Introduction of the intrinsic image. Intrinsic Images The method of Finlayson & Hordley ( 2001 ) Two assumptions 1. the camera ’ s sensors are sufficiently.](https://reader035.fdocuments.us/reader035/viewer/2022081419/56649d765503460f94a57fde/html5/thumbnails/6.jpg)
Different intensities of the black-body radiator : another constant k.
The intensity measured by the sensor
![Page 7: Introduction of the intrinsic image. Intrinsic Images The method of Finlayson & Hordley ( 2001 ) Two assumptions 1. the camera ’ s sensors are sufficiently.](https://reader035.fdocuments.us/reader035/viewer/2022081419/56649d765503460f94a57fde/html5/thumbnails/7.jpg)
Assumption 1. the illuminant can be
approximated by a black-body radiator. 2. pixel colors are linearly related to
the data measured by the sensor.
![Page 8: Introduction of the intrinsic image. Intrinsic Images The method of Finlayson & Hordley ( 2001 ) Two assumptions 1. the camera ’ s sensors are sufficiently.](https://reader035.fdocuments.us/reader035/viewer/2022081419/56649d765503460f94a57fde/html5/thumbnails/8.jpg)
Logarithm
The first term : the power of the illuminant and the scene geometry but independent of the color of the illuminant and the reflectance.
The second term : the wavelength to which the sensor responds and on the reflectance of the object.
The last term : the color of illuminant.
![Page 9: Introduction of the intrinsic image. Intrinsic Images The method of Finlayson & Hordley ( 2001 ) Two assumptions 1. the camera ’ s sensors are sufficiently.](https://reader035.fdocuments.us/reader035/viewer/2022081419/56649d765503460f94a57fde/html5/thumbnails/9.jpg)
Compute the differences
Let and .
![Page 10: Introduction of the intrinsic image. Intrinsic Images The method of Finlayson & Hordley ( 2001 ) Two assumptions 1. the camera ’ s sensors are sufficiently.](https://reader035.fdocuments.us/reader035/viewer/2022081419/56649d765503460f94a57fde/html5/thumbnails/10.jpg)
As a 2-D vector. If the temperature T is a
parameter, the two equations define a line where
is a point on the line and other points on the line are reached by adding with varying amounts.
![Page 11: Introduction of the intrinsic image. Intrinsic Images The method of Finlayson & Hordley ( 2001 ) Two assumptions 1. the camera ’ s sensors are sufficiently.](https://reader035.fdocuments.us/reader035/viewer/2022081419/56649d765503460f94a57fde/html5/thumbnails/11.jpg)
The constants only depend on the wavelength to which the sensor responds.
is independent of reflectance.
![Page 12: Introduction of the intrinsic image. Intrinsic Images The method of Finlayson & Hordley ( 2001 ) Two assumptions 1. the camera ’ s sensors are sufficiently.](https://reader035.fdocuments.us/reader035/viewer/2022081419/56649d765503460f94a57fde/html5/thumbnails/12.jpg)
T : 1000 K ~ 10000 K ( in steps of 20 %) Several different surfaces ( red -> whit
e ) were illuminanted by a black-body radiator of different temperatures.
sensors : The direction of the lines only depends on t
he type of sensors used in the camera.
![Page 13: Introduction of the intrinsic image. Intrinsic Images The method of Finlayson & Hordley ( 2001 ) Two assumptions 1. the camera ’ s sensors are sufficiently.](https://reader035.fdocuments.us/reader035/viewer/2022081419/56649d765503460f94a57fde/html5/thumbnails/13.jpg)
Gamma correction
Pixel colors : logarithm : The influence of any gamma correctio
n does not exist for the line in log-chromaticity difference space.
![Page 14: Introduction of the intrinsic image. Intrinsic Images The method of Finlayson & Hordley ( 2001 ) Two assumptions 1. the camera ’ s sensors are sufficiently.](https://reader035.fdocuments.us/reader035/viewer/2022081419/56649d765503460f94a57fde/html5/thumbnails/14.jpg)
Removal of the temperature T
Project the data points in a direction orthogonal to the line.
The equation is independent of the illuminant.
![Page 15: Introduction of the intrinsic image. Intrinsic Images The method of Finlayson & Hordley ( 2001 ) Two assumptions 1. the camera ’ s sensors are sufficiently.](https://reader035.fdocuments.us/reader035/viewer/2022081419/56649d765503460f94a57fde/html5/thumbnails/15.jpg)
Color circle log-chromaticity difference coordinates
![Page 16: Introduction of the intrinsic image. Intrinsic Images The method of Finlayson & Hordley ( 2001 ) Two assumptions 1. the camera ’ s sensors are sufficiently.](https://reader035.fdocuments.us/reader035/viewer/2022081419/56649d765503460f94a57fde/html5/thumbnails/16.jpg)
If we project the data points along the invariant direction, some information will be lost.
Mix of blue and red or cyan and yellow.
The direction in which to project the log-chromaticity difference is unique for each camera.
![Page 17: Introduction of the intrinsic image. Intrinsic Images The method of Finlayson & Hordley ( 2001 ) Two assumptions 1. the camera ’ s sensors are sufficiently.](https://reader035.fdocuments.us/reader035/viewer/2022081419/56649d765503460f94a57fde/html5/thumbnails/17.jpg)
Drawback
& : the response of the chosen channel may be very low, leading to noisy results.
Which channel should be chosen ?
![Page 18: Introduction of the intrinsic image. Intrinsic Images The method of Finlayson & Hordley ( 2001 ) Two assumptions 1. the camera ’ s sensors are sufficiently.](https://reader035.fdocuments.us/reader035/viewer/2022081419/56649d765503460f94a57fde/html5/thumbnails/18.jpg)
Modification
Finlayson & Drew ( 2001 ): dividing by the geometric mean of the three channels to remove the dependence on the shading information G and the dependence on the intensity k.
![Page 19: Introduction of the intrinsic image. Intrinsic Images The method of Finlayson & Hordley ( 2001 ) Two assumptions 1. the camera ’ s sensors are sufficiently.](https://reader035.fdocuments.us/reader035/viewer/2022081419/56649d765503460f94a57fde/html5/thumbnails/19.jpg)
![Page 20: Introduction of the intrinsic image. Intrinsic Images The method of Finlayson & Hordley ( 2001 ) Two assumptions 1. the camera ’ s sensors are sufficiently.](https://reader035.fdocuments.us/reader035/viewer/2022081419/56649d765503460f94a57fde/html5/thumbnails/20.jpg)
the vector is orthogonal to the vector ,
all points are located on the 2-D plane defined by u.
![Page 21: Introduction of the intrinsic image. Intrinsic Images The method of Finlayson & Hordley ( 2001 ) Two assumptions 1. the camera ’ s sensors are sufficiently.](https://reader035.fdocuments.us/reader035/viewer/2022081419/56649d765503460f94a57fde/html5/thumbnails/21.jpg)
Finlayson ( 2004 ) define the following coordinate system for the geometric mean chromaticity space.
: the two basis vectors Find the two vectors
![Page 22: Introduction of the intrinsic image. Intrinsic Images The method of Finlayson & Hordley ( 2001 ) Two assumptions 1. the camera ’ s sensors are sufficiently.](https://reader035.fdocuments.us/reader035/viewer/2022081419/56649d765503460f94a57fde/html5/thumbnails/22.jpg)
The geometric mean 2-D chromaticity space
Color circle Geometric mean 2-D chromaticity space
![Page 23: Introduction of the intrinsic image. Intrinsic Images The method of Finlayson & Hordley ( 2001 ) Two assumptions 1. the camera ’ s sensors are sufficiently.](https://reader035.fdocuments.us/reader035/viewer/2022081419/56649d765503460f94a57fde/html5/thumbnails/23.jpg)
![Page 24: Introduction of the intrinsic image. Intrinsic Images The method of Finlayson & Hordley ( 2001 ) Two assumptions 1. the camera ’ s sensors are sufficiently.](https://reader035.fdocuments.us/reader035/viewer/2022081419/56649d765503460f94a57fde/html5/thumbnails/24.jpg)
Geometric mean chromaticities If we project along a line in
geometric mean 2-D chromaticity space, we’ll lose some information about the color of the objects.
Using the method, we can transform the input image of a calibrated camera to an intrinsic image.
![Page 25: Introduction of the intrinsic image. Intrinsic Images The method of Finlayson & Hordley ( 2001 ) Two assumptions 1. the camera ’ s sensors are sufficiently.](https://reader035.fdocuments.us/reader035/viewer/2022081419/56649d765503460f94a57fde/html5/thumbnails/25.jpg)
Intrinsic image The intrinsic images only depend on the
reflectance of the object points. The 2-D coordinates have to be
projected onto a line that is orthogonal to the vector .
For a given camera, this vector can be found experimentally by imaging a calibration image under several different illuminants.
![Page 26: Introduction of the intrinsic image. Intrinsic Images The method of Finlayson & Hordley ( 2001 ) Two assumptions 1. the camera ’ s sensors are sufficiently.](https://reader035.fdocuments.us/reader035/viewer/2022081419/56649d765503460f94a57fde/html5/thumbnails/26.jpg)
Uncalibrated image Finlayson ( 2004 ): compute intrin
sic images for an uncalibrated camera from a single image
The projection has to be done onto a line inside the geometric mean 2-D chromaticity space.
Which line is the correct one ?
![Page 27: Introduction of the intrinsic image. Intrinsic Images The method of Finlayson & Hordley ( 2001 ) Two assumptions 1. the camera ’ s sensors are sufficiently.](https://reader035.fdocuments.us/reader035/viewer/2022081419/56649d765503460f94a57fde/html5/thumbnails/27.jpg)
The projection of the data points for two different lines.
![Page 28: Introduction of the intrinsic image. Intrinsic Images The method of Finlayson & Hordley ( 2001 ) Two assumptions 1. the camera ’ s sensors are sufficiently.](https://reader035.fdocuments.us/reader035/viewer/2022081419/56649d765503460f94a57fde/html5/thumbnails/28.jpg)
Entropy
Finlayson : the correct orientation is the orientation where the resulting image has minimum entropy.
Let be the set of lines for which we compute the entropy.
![Page 29: Introduction of the intrinsic image. Intrinsic Images The method of Finlayson & Hordley ( 2001 ) Two assumptions 1. the camera ’ s sensors are sufficiently.](https://reader035.fdocuments.us/reader035/viewer/2022081419/56649d765503460f94a57fde/html5/thumbnails/29.jpg)
Entropy
For each line, the invariant data points
A gray-scale image is formed from the projected data points that are transformed to the range[0, 1].
A histogram is computed for the image.
![Page 30: Introduction of the intrinsic image. Intrinsic Images The method of Finlayson & Hordley ( 2001 ) Two assumptions 1. the camera ’ s sensors are sufficiently.](https://reader035.fdocuments.us/reader035/viewer/2022081419/56649d765503460f94a57fde/html5/thumbnails/30.jpg)
Compute the probability that the gray value occurs in the image.
Compute the entropy
The correct direction for the line will be the one where the entropy is minimal.
![Page 31: Introduction of the intrinsic image. Intrinsic Images The method of Finlayson & Hordley ( 2001 ) Two assumptions 1. the camera ’ s sensors are sufficiently.](https://reader035.fdocuments.us/reader035/viewer/2022081419/56649d765503460f94a57fde/html5/thumbnails/31.jpg)
The invariant direction is located at an angle of 150.75°, which corresponds to a projection direction of 60.75°.
![Page 32: Introduction of the intrinsic image. Intrinsic Images The method of Finlayson & Hordley ( 2001 ) Two assumptions 1. the camera ’ s sensors are sufficiently.](https://reader035.fdocuments.us/reader035/viewer/2022081419/56649d765503460f94a57fde/html5/thumbnails/32.jpg)
The intrinsic image that is based only on reflectance can be used for object recognition.
Can we move from an intrinsic image back to a full color image ?
As we have known, some color information is lost.
![Page 33: Introduction of the intrinsic image. Intrinsic Images The method of Finlayson & Hordley ( 2001 ) Two assumptions 1. the camera ’ s sensors are sufficiently.](https://reader035.fdocuments.us/reader035/viewer/2022081419/56649d765503460f94a57fde/html5/thumbnails/33.jpg)
Color image
Drew ( 2003 ): go back to a color image
The projected coordinates don’t have to be interpreted as a gray-scale image.
The coordinates along the projection line are 2-D.
![Page 34: Introduction of the intrinsic image. Intrinsic Images The method of Finlayson & Hordley ( 2001 ) Two assumptions 1. the camera ’ s sensors are sufficiently.](https://reader035.fdocuments.us/reader035/viewer/2022081419/56649d765503460f94a57fde/html5/thumbnails/34.jpg)
Compute the corresponding 3-D coordinates in geometric mean chromaticity space by multiplying the projected coordinates by .
![Page 35: Introduction of the intrinsic image. Intrinsic Images The method of Finlayson & Hordley ( 2001 ) Two assumptions 1. the camera ’ s sensors are sufficiently.](https://reader035.fdocuments.us/reader035/viewer/2022081419/56649d765503460f94a57fde/html5/thumbnails/35.jpg)
Drew : adding a little illuminant in order to obtain a color image from the 1-D data points.
Compute the and project these points onto the invariant direction .
Compute the median values of this data.
![Page 36: Introduction of the intrinsic image. Intrinsic Images The method of Finlayson & Hordley ( 2001 ) Two assumptions 1. the camera ’ s sensors are sufficiently.](https://reader035.fdocuments.us/reader035/viewer/2022081419/56649d765503460f94a57fde/html5/thumbnails/36.jpg)
An illuminanted image is obtained by moving all projected data points by along the direction .
1% of the brightest image pixels
= the median of the brightest 1% of the image pixels
![Page 37: Introduction of the intrinsic image. Intrinsic Images The method of Finlayson & Hordley ( 2001 ) Two assumptions 1. the camera ’ s sensors are sufficiently.](https://reader035.fdocuments.us/reader035/viewer/2022081419/56649d765503460f94a57fde/html5/thumbnails/37.jpg)
![Page 38: Introduction of the intrinsic image. Intrinsic Images The method of Finlayson & Hordley ( 2001 ) Two assumptions 1. the camera ’ s sensors are sufficiently.](https://reader035.fdocuments.us/reader035/viewer/2022081419/56649d765503460f94a57fde/html5/thumbnails/38.jpg)
Exponentiate to compute output colors.
Intrinsic RGB image
Rescale the intrinsic image to have the original lightness of the input image.
![Page 39: Introduction of the intrinsic image. Intrinsic Images The method of Finlayson & Hordley ( 2001 ) Two assumptions 1. the camera ’ s sensors are sufficiently.](https://reader035.fdocuments.us/reader035/viewer/2022081419/56649d765503460f94a57fde/html5/thumbnails/39.jpg)
Segment the input image Compute edges of the segmented input ima
ge and the RGB chromaticity intrinsic image Take the threshold : shadow edges 1. edge values > a threshold in the segmented image 2. edge values < another threshold in the RGB color space
![Page 40: Introduction of the intrinsic image. Intrinsic Images The method of Finlayson & Hordley ( 2001 ) Two assumptions 1. the camera ’ s sensors are sufficiently.](https://reader035.fdocuments.us/reader035/viewer/2022081419/56649d765503460f94a57fde/html5/thumbnails/40.jpg)
Apply the first derivative to the logarithm of the input image
Use a morphological operator to binarize and thicken the shadow edges
Replace iteratively unknown derivative values on the boundary of the shadow edges by the median of known ones in the vicinity
![Page 41: Introduction of the intrinsic image. Intrinsic Images The method of Finlayson & Hordley ( 2001 ) Two assumptions 1. the camera ’ s sensors are sufficiently.](https://reader035.fdocuments.us/reader035/viewer/2022081419/56649d765503460f94a57fde/html5/thumbnails/41.jpg)
( e )=( b )-( d )
Marc Ebner
![Page 42: Introduction of the intrinsic image. Intrinsic Images The method of Finlayson & Hordley ( 2001 ) Two assumptions 1. the camera ’ s sensors are sufficiently.](https://reader035.fdocuments.us/reader035/viewer/2022081419/56649d765503460f94a57fde/html5/thumbnails/42.jpg)
![Page 43: Introduction of the intrinsic image. Intrinsic Images The method of Finlayson & Hordley ( 2001 ) Two assumptions 1. the camera ’ s sensors are sufficiently.](https://reader035.fdocuments.us/reader035/viewer/2022081419/56649d765503460f94a57fde/html5/thumbnails/43.jpg)
Graham D. Finlayson
![Page 44: Introduction of the intrinsic image. Intrinsic Images The method of Finlayson & Hordley ( 2001 ) Two assumptions 1. the camera ’ s sensors are sufficiently.](https://reader035.fdocuments.us/reader035/viewer/2022081419/56649d765503460f94a57fde/html5/thumbnails/44.jpg)
![Page 45: Introduction of the intrinsic image. Intrinsic Images The method of Finlayson & Hordley ( 2001 ) Two assumptions 1. the camera ’ s sensors are sufficiently.](https://reader035.fdocuments.us/reader035/viewer/2022081419/56649d765503460f94a57fde/html5/thumbnails/45.jpg)
![Page 46: Introduction of the intrinsic image. Intrinsic Images The method of Finlayson & Hordley ( 2001 ) Two assumptions 1. the camera ’ s sensors are sufficiently.](https://reader035.fdocuments.us/reader035/viewer/2022081419/56649d765503460f94a57fde/html5/thumbnails/46.jpg)
Reference Marc Ebner. Color Constancy
John Wiley & Sons, England, 2007. Finlayson, G.D. and Drew, M.S. and Lu, C., "Intrinsic
Images by Entropy Minimisation", In 8th European Conference on Computer Vision III, pp. 582-595, 2004.
G.D. Finlayson, S.D. Hordley, and M.S. Drew. Removing shadows from images. In ECCV 2002: European Conference on Computer Vision, pages 4:823–836, 2002. Lecture Notes in Computer Science Vol. 2353, http://www.cs.sfu.ca/∼mark/ftp/Eccv02/shadowless.pdf.