International Journal of Information Technology and Knowledge ManagementJuly-December 2011, Volume 4, No. 2, pp. 371-377

RETINEX IMAGE PROCESSING: IMPROVING THE VISUAL REALISMOF COLOR IMAGES

Mrs. Anjali Chandra1, Bibhudendra Acharya2, Mohammad Imroze Khan3

Sensations of color show a strong correlation with reflectance, even though the amount of visible light reaching the eye dependson the product of reflectance and illumination. The visual system must achieve this remarkable result by a scheme that does notmeasure flux. Such a scheme is described as the basis of Retinex theory. This paper deals with image enhancement techniquesbasically Retinex and its constituents, namely Small Scale Retinex (SSR), Multi Scale Retinex (MSR) and Multi Scale Retinexwith Color Restoration (MSRCR). The techniques enable color rendition and dynamic range compression in case of ‘dark’ or‘low lit’ images. The Multi Scale Retinex (MSR) is a generalization of the SSR, which, in turn, is based upon the last versionof Land’s center/surround retinex. The current version, the multi scale Retinex with color restoration (MSRCR), combines thedynamic range compression and color constancy of the MSR with a color ‘restoration’ filter that provides excellent colorrendition. The MSRCR has been tested on a very large suite of images. We provide a general overview of the types of operationsthat can be performed on the image in order to enhance its quality.

Department of Electronics & TelecommunicationNational Institute of Technology, Raipur, (C.G.)E-mail:1chatanishq@gmail.com, 2bacharya.etc@nitrr.ac.in,

3imroze786@gmail.com

1. INTRODUCTION

Image Enhancement improves the quality (clarity) of imagesfor human viewing. Our paper deals with a few techniquesbased on Retinex theory contributing to the enhancementof an image especially dark or low-lit images. The RetinexImage Enhancement Algorithm is an automatic imageenhancement method that enhances a digital image in termsof dynamic range compression, color independence fromthe spectral distribution of the scene illuminant, and color/lightness rendition. The digital image enhanced by theRetinex Image Enhancement Algorithm is much closer tothe scene perceived by the human visual system, under allkinds and levels of lighting variations, than the digitalimage enhanced by any other method. The comparison ofthis technique with other image enhancement techniqueshas been studied and codes for homomorphic filtering aredeveloped. The result section contains comparison oforiginal image, SSR image, MSR image, MSRCR imageand the output of homomorphic filtering. Finally it isconcluded that output of MSRCR is better than othertechniques.

2. RETINEX THEORY

A human observer can easily see individual objects both inthe sunlight and shadowed areas, since the eye locallyadapts while scanning the different regions of the scene.When attempting to display the image on a display, eitherthe low intensity areas are underexposed and look black orthe high intensity areas are overexposed and cannot be seen.

Images taken from digital cameras suffer from a loss in clarityof details and color as it depends on the illuminance whichin term varies with distance from source. This problem ofColor Constancy in images is solved using the basis ofRetinex Theory.

A. Retinex Image Enhancement

The Retinex takes an input digital image I and produces anoutput image R on a pixel by pixel basis as in the followingequation (1):

R(x, y) =log( ( , )

log( ( , )* ( , )I x y

I x y M x y(1)

where M(x, y) = exp (x2 + y2)/σ2) is a constant which controlsthe extent of M, and σ represent spatial convolutions.

B. Reflectance–Illuminance

The input image can be written as the product of twocomponents:σ(x, y)the reflectance component whichrepresents the light reflected from all the objects in the scenebeing imaged, and i(x, y) which represents the illuminationcomponent as in equation (2):

I(x, y) = i(x, y) ρ(x, y) (2)

• Since the illumination component varies veryslowly across the scene in equation (3) and (4),

I(x, y) = I0ρ(x, y) (3)

And R(x, y) = log ( I0ρ(x, y) / I

0ρ(x, y) * M(x, y) (4)

• By performing the same operation on each colorchannel, the output color image can be written asequation (5).

372 ANJALI CHANDRA, BIBHUDENDRA CHARYA MOHAMMAD IMROZE KHAN

R(x, y) = log (I1 (x, y)/ I

1 (x, y)* M(x, y))...ie{R, G, B}

(5)

Ri(x, y) is dependent upon the size of the surround mask

which is parameterized by σ.

• Different values of σ enhance different featuresof the input image, equation (6).

Ri (x, y) = 1/k (

0

( , )(log ( ))

( , )* ( , )k i

ki

I x y

I x y M x y=∑ (6)

C. Image Formation

Based on the image formation mode shows equation (7):

S(x, y) = L(x, y) R(x, y) (7)

An image S is the pixel-by-pixel product of the ambientillumination L and the scene reflectance R. The Retinexalgorithm deals with the problem of separating the twoquantities: first estimating the illumination information andthen obtaining the reflectance by division. Working in alogarithmic domain, the above relation can be expressedas-Figure1.

Fig. 1: Basic of Retinex Algorithm

D. Categories of Retinex Algorithms

• Set of path-based algorithms where the new pixelvalue depends on the multiplication of ratios alongthe path.

• Recursive comparison of new pixel withsurrounding pixels.

• Centre/surround Retinex where the new pixelvalue depends on the comparison of a given pixeland the surrounding average pixel values.

• Mathematical formulas which converts theconstraints of illumination and reflectance intomathematical problem and then obtains the newpixel values.

E. Development of Retinex Techniques

• Single Scale Retinex (SSR)

• Multi-Scale Retinex (MSR)

• Multi-Scale Retinex with Color Restoration(MSRCR)

• Multi-Scale Retinex with canonical gain/offset

3. SINGLE SCALE RETINEX

Jobson and his co-worker defined a single-scale Retinex(SSR), which is an implementation of center/surround

Retinex. The Single-scale retinex is given by equation (8).

Ri(x, y) = log I

i(x, y) – log [F(x, y) * I

i(x, y)] (8)

Where Ri (x, y) is the retinex output, I

i (x, y) is image

distribution in the i th color band,’’*’’ denotes theconvolution operation is the normalized Gaussian functiongiven by equation (10).

F(x, y) = ke–(x2 + y2)c2 (10)

c is the Gaussian surround space constant, or the scale andis selected such that, equation (11).

( , )F x y dxdy∫∫ = 1 (11)

The image distribution is the product of scenesreflectance and illumination given by equation (12)

Ii (x, y) = S

i(x, y) r

i(x, y) (12)

Where Si(x, y) is the spatial distribution of illumination

and the distribution of scene reflectance. The convolutionwith surround function works as averaging in theneighborhood. Generally the illumination has slow spatialvariation, which means equations (13-a, 13-b, 13-c).

Ri(x, y) = log {(Si (x, y) r

i (x, y)/

( , ) ( , )}i iS x y r x y (13 -a)

Si (x, y) ≈ ( , )iS x y (13 -b)

Ri (x, y) ≈ log {r

i (x, y)/ ( , )}ir x y (13-c)

Hence the illuminance term can be eliminated from theretinex obtained making color constancy possible.

A. Characteristics of SSR

SSR has been defined to have the following characteristicsand properties-

• The functional form of the surround is a Gaussian.

• The placement of the log function is after surroundformation.

• The post retinex signal processing is a canonicalgain offset rather than an automatic gain offset.

• There is a trade off between dynamic rangecompression and tonal rendition which is governedby the Gaussian surround space constant. A spaceconstant of 80 pixel is a reasonable compromisebetween dynamic range compression and rendition.

• A single scale seems incapable of simultaneouslyproviding sufficient dynamic range compressionand tonal rendition.

• Violations of the gray world assumptions has ledto retinex images which were either greyed outlocally or globally or more rarely suffered fromcolour distortion.

RETINEX IMAGE PROCESSING: IMPROVING THE VISUAL REALISM OF COLOUR IMAGES 373

4. MULTISCALE RETINEX

Recent work advocates MSR as a method to bridge the gapbetween what a camera sees and what a human sees, withthe goal being to provide image reproductions which arevery similar to what the human viewer would have seen,were they present when the picture was taken.

We believe that the chief conceptual problem usingMSR in this way is that a number of image-processing tasksare performed simultaneously without sufficient regard tothe interactions occurring between them.

The main practical consequence of this is that MSR isnot appropriate for applications which are sensitive to color.In the image processing/image enhancement context, MSRserves a subset of the following four image processing goals,depending on the circumstances:

(i) Compensating for uncalibrated devices (gammacorrection)

(ii) Color constancy processing

(iii) Dynamic range compression

(iv) Color enhancement

Because of the tradeoff between dynamic range compressionand color rendition, we have to choose a good scale c in theformula of F(X, Y) in SSR. If we do not want to sacrificeeither dynamic range compression or color rendition, multi-scale retinex, which is a combination of weighted differentscale of SSR, is a good solution, given by equation (14) and(15).

RMSRi

=1

N

n nin

w R=

∑ (14)

RMSRi

=1

{log ( , )N

n in

w I x y=

∑– log[ ( , )* ( , )]}n iF x y I x y (15)

Where N is the number of the scales, Rni is the ith

component of the nth scale. The obvious question aboutMSR is the number of scales needed, scale values, andweight values. Experiments showed that three scales areenough for most of the images, and the weights can be equal.Generally fixed scales of 15, 80 and 250 can be used, orscales of fixed portion of image size can be used. The MSRoutput is different from existing techniques in that theoverall effect of processing is scene dependent but theprocessing itself is not. In other words, though the overalleffect adapts itself to lighting variations within the sceneprocess, with exactly the same control parameters can beused for any image. This is not true for other adaptivetechniques since variations in lighting conditions implyvariations in the control parameters.

5. MSR TO MSRCR

A. MSR Style Algorithms and Color Fidelity:

Color fidelity in image reproduction is a complex and activeresearch topic. First we assume that changing the intensitydoes not change the perceived non-intensity aspects ofcolor. This assumption is, of course, only an approximation.Even for isolated colors, color appearance does change withintensity.

We also note that the original MSR also does not addressthese problems, and that our modified version of MSR isthe MSRCR is more suited to adding corrections fordeviations from the above assumption. We thusacknowledge that an important area for future work is toinclude more complex color appearance models, includingones for simultaneous contrast as already discussed.

For the purposes of this work, then, we assume that afirst approximation of faithful color reproduction is topreserve chromaticity, as defined by equation (16).

X =X

X Y Z+ + and

Y =Y

X Y Z+ +(16)

If we assume that the region’s reproduced chromaticityX, Y matches the region’s scene chromaticity XY,then itfollows immediately that, equation (17).

X = KX, Y = KY, Z = KZ (17)

where K =X Y ZX Y Z

′ ′ ′+ ++ +

Similarly, scaling the reproduction X, Y, Z by a constantk preserves chromaticity. Thus to preserve chromaticity wecan manipulate k on a region-by-region basis, but nototherwise. Normally we deal with intermediate variablessuch as camera RGB. In order to be confident that a goodapproximation of scene chromaticity is being reproduced,we need to know the relationship between the input sceneand, XYZ and RGB, the reproduction X, Y, Z . If we knowthese relationships, then we can map intermediate variablesinto ones which are linearly related to XYZ input and outputX YZ . Having done so, we can manipulate the reproducedcontrast while preserving chromaticity by scaling pixelRGB. Linearity from RGB to X Y Z ensures that thechromaticity reproduced by (kR, kG, kB) is the same as thatreproduced by (R, G, B). Applying gamma correction to theresult of MSR processing normally gives poor results.Specifically the images look washed out and over gammacorrected. The problem with just accepting and using thiscoincidence as a conveniently provided gamma correctionis that device calibration (gamma correction) is meant to

374 ANJALI CHANDRA, BIBHUDENDRA CHARYA MOHAMMAD IMROZE KHAN

compensate for devices, but now one is committed to a singlemethod, and thus the result is device dependent. Regardless,since MSR can, to some extent, play the role of gammacorrection, it is important to ensure proper gamma correctionis being applied to the original image when being comparedto MSR results on a monitor. Based on visual inspection,and the chromaticity results presented below, we suggestthat MSR effectively applies a gamma correction in therange of 2 to 2.5, which we will refer to as the MSR equivalentgamma.

B. Color Restoration

The general effect of retinex processing on the images witha regional or global gray world violation is a graying out ofthe image either in specific regions or globally. More rarelythe gray world violations can simply produce an unexpectedcolor distortion. In this paper, we extend a previouslydesigned single scale centre/surround retinex to a multi-scale version that achieves simultaneous dynamic rangecompression/color consistency/lightness rendition.

C. Chromaticity Preserving MSR

We now outline the approach to MSRCR. As mentionedearlier, the main idea is to separate the processing goals ofMSR so that each one can be done more optimally. First weensure that the input is linear. Then we optionally applycolor constancy processing to correct for mismatchesbetween the imaging system and the illumination. This isfollowed by MSR style processing on an appropriatelydefined image luminance. The RGB of the output imagepixels are then set so that their chromaticity is the same asin the original linear image, but their luminance are theresult of the previous processing step.

D. Gain-offset Adjustment, Color Enhancementand Output

The next step is to apply the offset part of the offset-gainalgorithm. Here the range of the luminance result is offsetso that some of the dark pixels are clipped at zero. If colorenhancement is desired, then it is best added at this stage.The chromaticity of the pixels that are clipped will be aslightly incorrect, but this is not normally noticeable. It isnot recommended, however, to do the same with the bottomof the range, as this can affect the chromaticity of all thepixels. Instead it is generally better to increase the amountof clipping on the bottom by doing so when the luminancerange is adjusted, as described above.

6. MULTISCALE RETINEX WITH COIOR

RESTORATION ALGORITHM

The current version, the multi scale retinex with colorrestoration (MSRCR), combines the dynamic range

compression and color constancy of the MSR with a color‘restoration’ filter that provides excellent color rendition.The MSRCR has been tested on a very large suite of images.The general form of the MSRCR can be summarized by thefollowing equation (18):

RMi

(x, y) = GrF

i (x, y)

1

(log[ ( , )] N

s in

w I x y=

∑

– log[ ( , )* ( , )])i sI x y M x y – Or

(18)

Where i =1 to N, Where Rmi

is the ith band of the MSRCRoutput, s is the number of scales being used, w

s is the weight

of the scale, Iiis the ith band of the input image, and B is the

number of bands in the input images. The surround functionM

sis defined by equation (19).

Ms (x, y) = K exp

2

2 2s

x y

ρ+

(19)

where ρs is the standard deviation of the sth surround

function, and

2

2 2exp sK

x y

ρ+∫∫ dx dy = 1

Fi(x, y) is the color restoration functions defined by

equation (20).

Fi(x, y) = G

f log[I

i (x, y)/(

11 ( , )) – ]

N

n fnx y O

=∑ (20)

Gfand O

f: are, respectively, the final gain and offset

values needed to scale the output of the log domainoperations to the (R, G, B) color space, and G

f and O

f and

control the degree to which the color restoration functionF(x, y) affects the overallcolor of the output image. Theseconstants, the number of scales S and the widths of thesurround functions, are image independent in the sense thatwe apply the same (canonical) set of constants to everyimage that we process. The color restoration can also becalculated using the expression given by equation (21).

Ci (x, y) = β{log [αI

i (x, y)] – log [ΣI

i(x, y)]} (21)

where β – Gain Constant and α – Controls the strengthof nonlinearity. The final representation of MSRCR isrepresented as equation (22):

RMSRCRi

(x, y) = G[Ci(x, y)* R

MSRi(x, y) + b (22)

Where, G – Gain Constant, b – Gain Offset value;

G – Final Gain – 192;, b – Offset Value – 30;

α – Strength of non-linearity – 125;

β – Control gain constant - 46

Although these constants are not optimal for most images,they yield serviceable results for many images, and thus weagree that widespread applicability of the constants isstrength of MSRCR.

RETINEX IMAGE PROCESSING: IMPROVING THE VISUAL REALISM OF COLOUR IMAGES 375

Fig.2: Block Diagram of MSRCR

As a start, we have experimented with some input dependentmethods for the gain offset adjustment, where heuristics areapplied to the image histogram to find gain offsetparameters. The block diagram of MSRCR is shown aboveas fig.2.

7. COMPARISON WITH OTHER TECHNIQUES

A. Non-linear Gamma Correction

Good visual representations seem to be based upon somecombination of high regional visual lightness and contrast.To compute the regional parameters, we divide the imageinto non overlapping blocks that are 50×50 pixels. For eachblock, a mean, I, and a standard deviation, σ

f, are computed.

A first approach was to postulate that for visually goodrendition the contrast & lightness product should be abovea minimum value, with the additional constraint that eachcomponent cannot fall below an absolute minimum value.

This regional scale is sufficiently granular to capturethe visual sense of regional contrast. Both the contrast andthe lightness can be measured in terms of the regionalparameters.

The coupling of the constraints of minimum contrast-lightness product with minimum contrast and lightness asseparate entities defines the zone in figure labeled “visualgood”. Further, this figure suggests that there may exist acontour of much higher contrast-lightness, which can beconsidered a “visual ideal”.

Fig.3 : Variation of Image Intensity and Contrast

When images are displayed on monitors, their intensityprofile is typically modified using the gamma-transformation given by equation (23): (23)

Fig.4: Comparison Between Original and Retinex Image

Fig.5: Showing Visually Optimal Area

Where, Ii(x, y) is the input value, and I

0 (X, Y) is the modified

value. A value of γ–1 is the linear transform. In order to gaugeour results against a linear baseline for the original imagedata, we determined that most digital images are super-linearand should be corrected to approximate linearity by gammatransforming the processed image using γ = 0.63. Whilethis has negligible effect on standard deviation values, itjust adjusts the mean downward from about 165 to about128. as shown in figure in (3) (4) & (5).

B. Histogram EqualizationA global technique that works well for a wide variety ofimages is histogram equalization. This technique is basedon the idea of remapping the histogram of the scene to ahistogram that has a near-uniform probability densityfunction. This results in reassigning dark regions to brightervalues and bright regions to darker values.

376 ANJALI CHANDRA, BIBHUDENDRA CHARYA MOHAMMAD IMROZE KHAN

C. Homomorphic FilteringThe technique that most resembles retinex algorithm,conceptually and functionally, is homomorphic filtering.Homomorphic filter is used for image enhancement. Itsimultaneously normalizes the brightness across an imageand increases contrast. Here homomorphic filtering is usedto remove multiplicative noise. The high-pass filtering isused to suppress low frequencies and amplify highfrequencies, in the log-intensity domain. Mathematically,it cab be represented as shown in equation (24 to 28)

Si(x, y) = ln[I

i(x, y)] (24)

Si’(v, w) = F[S

i(x, y)] (25)

Si”(v, w) = S

i’(v, w)H(v, w) (26)

Si”’(x, y) = F-1[S

i”(v, w)] (27)

Ii’(x, y) = exp[S

i’”(x, y)] (28)

Where F[] and F–1[] represent the Fourier and inverseFourier transforms respectively. H represents thehomomorphic filter. It is in its final exponential transformthat the homomorphic filter differs the most from theMSRCR. MSRCR does not apply a final inverse transformto go back to the original domain.

Fig.6: Block Diagram for Homomorphic Filtering

The homomorphic filter consistently provided excellentdynamic range compression but is lacking in final colorrendition. The output of the homomorphic filter in effectappears extremely hazy compared with the output of theMSRCR though the dynamic range compression of the twomethods appears to be comparable as shown in figure (7).

Fig.7: Characteristics of Homomorphic Filter Used

D. Manual Image Enhancement

We have provided a brief description of the commonlyencountered “problems” introduced inevitably in a digitalimage due to the nature of the acquisition process and thepre-processing algorithms. The MSRCR has proven to bequite resilient to many of the arbitrary operations that areused in digital image formation and can thus be trulyconsidered a fully automatic process.

8. CONCLUSION AND RESULTS

The SSR provides a good mechanism for enhancing certainaspects of images and providing dynamic range

compression. However it is limited in its use because it caneither provide good tonal rendition or dynamic rangecompression. The MSR comprises of 3 scales-small,intermediate and large- overcomes this limitation and isfound to synthesize-dynamic range compression, colorconstancy, tonal rendition and produce results whichcompare favorably with human visual perception exceptfor the scenes which contain violations of the gray worldassumption. As compared with other image enhancementtechniques, the Retinex enhancement has the followingadvantage:

• Image enhancement is a process independentof inputs;

• Has general application on all pictures.

• Good dynamic range compression and colorrendition effect.

MSRCR provides proportional RGB components incolor images which is an improvement over MSR techniquewhich is preferred for gray images. Optimized scale, gainand offset parameters have been investigated and analyzedfor better results. We have provided a brief description ofthe most commonly used image enhancement techniquesand compared their operation with the multiscale retinexwith color restoration. Application to various fields has beenshown (8) & (9).

Fig.8 (a) Original Image (b) SSR Image (c) MSR Image(d) MSRCR Image (e) Homomorphic Filtering

Fig.9: X – Ray Image (a) Original Image (b) SSR Image(c) MSR Image (d) MSRCR Image (e) Homomorphic

Filtering

9. REFERENCES

[1] D. J. Jobson, Z. Rahman, and G. A. Woodell, “A Multi-Scale Retinex For Bridging the Gap Between Color Imagesand the Human Observation of Scenes”, IEEE Transactionson Image Processing: Special Issue on Color Processing,July 1997.

[2] Edwin H. Land, John J. Mccann, “Lightness and RetinexTheory, Polaroid Corporation”, Cambridge, Massachusetts02139(1970)

RETINEX IMAGE PROCESSING: IMPROVING THE VISUAL REALISM OF COLOUR IMAGES 377

[3] Lei Ling, Zhou Yinqing, Li Jingwen “An Investigation ofRetinex Algorithm For Image Enhancement”, 201 Lab, BeijingUniversity of Aeronautics and Astronautics, Beijing 100083,China.

[4] Kobus Barnard and Brian Funt “Investigations Into Multi-Scale Retinex”, Vision and Technology, pp. 9-17 John Wileyand Sons (1999).

[5] Z. Rahman, D. Jobson, and G. A. Woodell “A Multi ScaleRetinex For Color Rendition and Dynamic RangeCompression”.

[6] Z. Rahman, D. Jobson, and G. A. Woodell “Resiliency ofthe Multiscale Retinex Image Enhancement Algorithm”,NASA Langley Research Center.

[7] Z. Rahman, D. Jobson, and G. A. Woodell “A Comparisonof the Multiscale Retinex with Other Image EnhancementTechniques”, NASA Langley Research Center.

[8] Brian Funt, Kobus Barnard, Michael Brockington, VladCardei “Luminance-Based Multi-Scale Retinex”, May 1997.

[9] Z. Rahman, D. Jobson, and G. A. Woodell, “Multi-ScaleRetinex For Color Image Enhancement,” In Proceedings ofthe IEEE International Conference on Image Processing,IEEE, 1996.

[10] Lei Ling Zhou Yinqing Li Jingwen, “An Investigation ofRetinex Algorithms For Image Enhancement” (201 Lab,Beijing University of Aeronautics and Astronautics, Beijing100083, China) (2006).