Symbolic fusion of luminance-hue-chroma features for region segmentation

*Corresponding author.E-mail address: [email protected] (P. Lambert)

Pattern Recognition 32 (1999) 1857}1872

Symbolic fusion of luminance-hue-chroma featuresfor region segmentation

Patrick Lambert*, Thierry Carron

Laboratoire d 'Automatique et de MicroInformatique Industrielle LAMII / CESALP } Universite& de Savoie } BP 806-74016Annecy Cedex, France et CNRS-GDR-PRC ISIS Information Signal Images, France

Received 5 November 1998

Abstract

Segmentation is a very important pre-processing step towards image analysis or image compression. In this paper, wepropose an original region segmentation method applied to color images. The two key elements of this method are:

f the use of a color space where hue is explicitly de"ned and processed according to its relevance which can be linked tochroma,

f the use of symbolic representations and rule-base systems combining color and luminance features in order to de"nehomogeneity between pixels.

As an illustration, this approach is applied to an iterative region growing method classically used in grey level imageprocessing. Experimental results on real images are given. ( 1999 Pattern Recognition Society. Published by ElsevierScience Ltd. All rights reserved.

Keywords: Color; Segmentation; Region growing; Fuzzy segmentation; Symbolic representation

1. Introducion

Image segmentation is one of the "rst and most di$-cult tasks of any image analysis system. Whatever thesubsequent step may be, either object detection, or fea-ture extraction, object recognition, scene interpretationor image coding, it relies heavily on the quality of thesegmentation process. Many methods have been pro-posed in the literature [1,2]. Segmentation techniquescan roughly be divided into two approaches: the regionapproach and the frontier approach. The goal of regionsegmentation is to split the image into disjoint sets ofconnected pixels similar according to a homogeneitycriterion. On the contrary, frontier segmentation is a dualapproach looking for lines corresponding to the bound-

aries of objects within the image. Of course, these dualapproaches do not give exactly the same result: the fron-tiers found do not "t the limits of the regions fully. Somemixed approaches, using region and frontier segmenta-tion jointly, can be used to improve performance [3].Despite the large number of segmentation techniquesavailable, no general method is "tted to a large anddiverse set of imagery and research e!orts are alwaysintense in this area.

Most of the existing research in image processing dealswith monochrome images. However, constant technicalprogress and the decreasing cost of cameras, memorycapacities, acquisition modules and computers have in-duced an increasing interest in the area of color image"ltering and analysis over the past ten years. It is nowwidely accepted that color information can be used tofurther re"ne the performance of an image processingsystem. Nevertheless, grey level techniques cannot beused in a straightforward way because of the vectorialnature of color images. Typically, pixels in a color images

0031-3203/99/$20.00 ( 1999 Pattern Recognition Society. Published by Elsevier Science Ltd. All rights reserved.PII: S 0 0 3 1 - 3 2 0 3 ( 9 9 ) 0 0 0 1 0 - 2

are represented by the classical three Red}Green}Blue(RGB) components. Using the RGB basis, interestingvectorial approaches have been proposed [4,5]. Theseapproaches are not fully satisfactory because they donot use the inherent nature of color images: chromaticinformation is split into the three RGB components.Competitive approaches consist in using another spacerepresentation better suited to color perception andexhibiting explicitly chromatic information [6].

Concurrently, the place of fuzzy sets in segmentationtechniques is getting more and more important [7}10].Essentially, there is a double interest in using fuzzy sets.First, it reduces the criticality of choice of many thresh-olds inherent to segmentation methods. Secondly, thecomplex mechanism of segmentation techniques can beconveniently and simply expressed by using the fuzzy settheory, and more speci"cally, by symbolic representa-tions associated with rule-base systems. In the speci"ccase of color images, fuzzy sets bring a third advantage:the aggregation of semantically di!erent complementaryinformation, like luminance and chrominance, can beperformed in an e$cient and #exible way.

Therefore, the fuzzy set theory seems to be an interest-ing tool to address the problem of segmentation of colorimage segmentation. Huntsberger uses such a strategy inRef. [11]. He de"nes color edges as the zero crossings ofdi!erences between membership values at each pixel.These fuzzy membership values k

ik, which represent how

much a pixel k belongs to a region i, are generated by aniterative segmentation scheme based on Bezdeck's fuzzyc-means algorithm [7]. Results are encouraging but themethod is time-consuming because of the fuzzy c-meansalgorithm and detected lines are not always continuous.Lim [12] proposed an automatic region segmentationmethod using a coarse-"ne concept. A "rst coarse seg-mentation is performed using a thresholding techniqueapplied on the histogram of each color band. Then, thefuzzy c-mean algorithm is only applied to pixels whichare not yet segmented. However, the marginal processingof each band scatters color information. Moghaddam-zadeh [13] combined fuzzy edge detection and regiongrowing approaches. Depending on the di!erent stepsused and on the parameter values within each step, acoarse segmentation algorithm and a "ne one are pro-posed. This interesting method is only applied with theRGB color space in which the contrast between twopixels is measured by the fuzzi"cation of the Euclideandistance.

A few studies proposed segmentation techniques per-formed in a color space explicitly exhibiting color at-tributes. Celenk [14], Schettini [15] and Tominaga [16],used techniques based on a recursive analysis of the threeone-dimension color histograms. The di!erences betweenthese methods lie in the choice of the color space and inthe way the histograms are analysed. Miyawaki [17]divides images into 3 classes: the high chromatic regions,

the low chromatic ones and the achromatic ones. He alsouses a histogram thresholding which is applied on theHue component, the three Red}Green}Blue componentsor the intensity component according to the chromaticitytypes of the regions the pixels belong to. Chassery [18]proposes a region growing strategy which combines localcolor similarity and global spatial cohesion. The interest-ing point is the design of the color similarity measurewhere the hue di!erence is weighted according to the huerelevance.

For a few years, our work has also focussed on thesegmentation of color images. First used in edge detec-tion [19], the proposed strategy has then been applied toregion segmentation [20,21]. In both cases, the two keyelements of the method are:

f the use of a color space where hue is explicitly de"ned.But, as the hue feature is not relevant for pixels nearthe achromatic axis (grey pixels), the hue informationis used according to its relevance which can be linkedto the chroma information.

f the use of symbolic representations and rule-base sys-tems combining color and luminance features in orderto de"ne homogeneity or inhomogeneity betweenpixels.

The originality of this contribution comes from theconjunction of these two key points to get a global fuzzymeasure of similarity, or contrast. Compared with otherrelated papers, this approach brings the following threemain advantages:

f the use of a well-suited color space giving more em-phasis to the hue information which is generally splitbetween the RGB components (this is the main pointwhich di!erentiates the proposed approach fromMoghaddamzadeh),

f the joint use of the color attributes which avoids a sep-arate processing of each component,

f the use of fuzzy representations avoiding the thresholde!ect inherent to a crisp decision, as is done inMiyawaki's and Chassery's methods.

As an illustration, this approach is applied to an iter-ative region growing method classically used in grey levelimage processing. However, this concept could be ex-tended to any color segmentation algorithm based ona homogeneity measure between color pixels or colorregions.

The color space used is presented and analyzed inSection 2. Section 3 gives a general symbolic descriptionof the method. In Section 4, technical points about fuz-zi"cations and inference mechanisms are detailed. Sec-tion 5 presents some results on real images and discussedthe segmentation evaluation process. Finally, Section 6concludes this work.

1858 P. Lambert, T. Carron / Pattern Recognition 32 (1999) 1857}1872

Fig. 1. Circular representation of hue.

2. Choice and properties of the color space

2.1. The proposed color space

Color images are classically obtained by using theRed}Green}Blue (RGB) representation. However, usingthis representation presents some drawbacks which makeit inappropriate for certain image segmentation tasks:RGB components are highly correlated and then, chro-matic information is not directly "t for use.

So, the "rst step is the de"nition of a 3-dimensionluminance chrominance space. Many such spaces havebeen proposed in literature [6]. The most common goalis to get an approximately uniform space in which per-ceptual di!erences between colors correspond to equaldistances in the tristimulus space. Such spaces are, e.g.CIELab and CIELuv ones. Based on nonlinear trans-forms, these spaces su!er from nonlinear e!ects (singular-ity, instability) [22] and generally have a nonuniformnoise sensitivity. So, we adopt a color space minimizingthis sensitivity and de"ning colors in terms of perceptualattributes of hue, chroma and intensity (H}C}I in thefollowing). Intensity is the perceived brightness, hue cor-responds to the general notion of color and chroma isa measure of the `deepnessa of color. The chosen spacehas been de"ned as follows:

f a "rst linear transform gives >, Ch1, Ch

2components:

C>

Ch1

Ch2D"C

1/3 1/3 1/3

1 !1/2 !1/2

0 !J3/2 J3/2D]CR

G

BD (1)

f a second nonlinear transform gives H, C, I compo-nents:

I"> (2)

C"JCh21#Ch2

2(3)

if Ch1'0 then

H"cos~1(Ch1/C)

elseH"2p!cos~1(Ch

1/C) (4)

The special case Ch1"0 leads to H"p/2 or 3p/2 de-

pending on the sign of Ch2.

Di!erent coe$cients are added to these equations sothat H, C and I values range from 0 to 255 with a 8-bitrepresentation. Then, C and I images can be processed asclassical grey level images. It must be noticed too that thecircular representation of hue (see Fig. 1) has to be takeninto account when hue di!erences or hue averages arecalculated.

A further interesting property of the proposed trans-form is its reversibility. However, the segmentationmethod which is presented below could be applied to any

other color spaces as long as color information is ex-pressed in terms of hue, chroma, intensity.

2.2. Noise sensitivity

Noise is a corrupting factor present in any imageprocessing task. So, it is important to evaluate the noisesensitivity of data used in any processing. In order tomeasure this sensitivity on H}C}I components, the fol-lowing commonly used model is proposed:

Rn"R#n

R, G

n"G#n

G, B

n"B#n

B(5)

R, G, B denote the uncorrupted data, nR, n

G, n

Bwhite

additive noises and Rn, G

n, B

nthe corrupted data. Ap-

plying the above H}C}I transform on the three compo-nents R

n, G

n, B

n, we get H

n, C

nand I

n, components. The

noise sensitivity of H}C}I space is analyzed through thevariance of the H

n, C

n, I

ncomponents. If noises are sup-

posed to be both uncorrelated and gaussian with thesame variance, p2

0, we have the following results:

f p21n

"p20/3. It is a classical result for the mean estima-

tion. Of course, this result does not depend on the H, Cand I values.

f p2Cn

value does not depend on the H and I values.Experimentally, it can be observed that p2

Cn

rangesfrom 0.5]p2

0, for a low value of C, to 1.5]p2

0for a high

value of C.f p2

Hn

value does not depend on the H value, but dependsstrongly on the C value. Fig. 2 displays the evolution ofp2Hn

versus the C value. Experimentation has beenperformed with a constant H value while the C valueincreases.

Beyond a threshold value C2, p2

Hn

is small and lowerthan the Intensity variance p2

Ln

. It means that the huecomponent is less noisy than the intensity component.On the contrary, beneath a threshold value C

1, p2

Hn

is highand greater than the noise variance p2

0. Furthermore,

P. Lambert, T. Carron / Pattern Recognition 32 (1999) 1857}1872 1859

Fig. 2. p2Hn

evolution versus C value.

when the chroma value decreases beneath this valueC

1, increases in a very signi"cant way. Then, the hue

component is very noisy, far more than the Intensitycomponent.

2.3. Hue relevance

As a "rst conclusion, these results suggest thatin a segmentation processing, H}C}I componentsmust be utilized carefully: intensity and chroma arealways relevant and anyway can contribute to thesegmentation processing; on the contrary, hue has avery variable reliability. So, its contribution to thesegmentation processing will have to depend on itsrelevance.

The second conclusion is that the hue relevance can bemeasured by the chroma value. This can be summarizedby de"ning three areas in such a way:

f Small chroma valueHue is not relevant and cannot be utilized in thesegmentation processing

f Medium chroma valueHue is approximately as relevant as Chroma andIntensity. The three H}C}I components could be usedwith similar weights

f Large chroma valueHue is very relevant and brings reliable information tothe segmentation processing

Of course, the problem is to determine limits betweenthese three areas. These values depend on image charac-teristics, essentially chroma distribution, and on the typeof H}C}I transforms. Use of symbolic representationsmakes this choice less critical. This point will be dis-cussed in Section 4.1.2.

3. General description of the method

The segmentation technique which has been chosenhere is a region growing one. It requires the de"nition ofa homogeneity criterion Hom(P

1, P

2) between two pixels

P1and P

2. Based on this criterion, an iterative segmenta-

tion is proposed. The segmentation algorithm will bepresented in Section 3.1. The homogeneity criterion is aunique criterion mixing H}C}I features. As it is di$cultto set up a crisp numerical function Hom, the formulationof the criterion will be done by using fuzzy variables andthe fuzzy set theory. The general principle of the de"nitionof the fuzzy homogeneity criterion will be given in Sec-tion 3.2.

3.1. Segmentation algorithm

The basic principle of the algorithm is the classical`blob coloringa algorithm using a `L-shapea mask.The image is scanned line by line from the top left tothe bottom right. Let P

cbe the current pixel to be

processed and Pa and Pj the above and left neighboringpixels (Fig. 3). Pa (respectively Pj) is supposedto be already processed and belongs to region Ra(respectively Rj). Eventually, regions Ra and Rjare the same region. The region growing principle can beexpressed in a plain linguistic form by the followingstatements:

if Hom(Pc, Pa) is TRUE and Hom(P

c, Pj) is FALSE then

Pc

is included in Raif Hom(P

c, Pa) is FALSE and Hom(P

c, Pj) is TRUE then

Pc

is included in Rjif Hom(P

c, Pa) is TRUE and Hom(P

c, Pj) is TRUE then

Pc

is included in Rj andequivalence is performed between regions Ra and Rj


1 Readers who are not familiar with fuzzy logic and symbolicrepresentation and aggregation could "nd elementary notionswhich are necessary to understand this paper in Refs. [23,24].

Fig. 3. Segmentation mask used in region growing processing.

if Hom(Pc, Pa) is FALSE and Hom(P

c, Pj) is FALSE then

a new region Rc

is created.

This basic algorithm is rather too simple to give goodresults. But it can be improved by iterative applicationaccording to the strategy de"ned in Fig. 4: after eachsegmentation, a resulting segmented image is built up. Inthis image, H}C}I values of a region are obtained bycomputing averages of original H}C}I values for all thepixels of this region.

Improvement is obtained by starting with a soft cri-terion Hom

1, which gives an over segmentation, and by

progressively hardening the homogeneity criterion ateach iteration, thus reducing over-segmentation. In a fewsteps (typically 3 iterations) a good segmentation is gen-erally obtained.

3.2. General description of the fuzzy homogeneity criteriondexnition1

The homogeneity criterion measures the similarity be-tween two pixels P

1and P

2characterized by their three

components in the H}C}I color space. These two pixelsare a couple of pixels chosen among the three pixelsPc, Pa and Pj de"ned in Fig. 3. To build up this criterion,

the following "ve numerical segmentation attributes areused:

f c1

and c2, the chroma values at pixels P

1and P

2,

f dH, dC and dI, the absolute numerical di!erences be-tween the H}C}I features of pixels P

1and P

2(the hue

di!erence has to be calculated taking into account thecircular representation of this feature. It can be noticedthat the criterion is a very local one, depending only onthe two pixels P

1and P

2, and without any information

from the neighborhood).

The proposed method is based on human way ofreasoning. The basic idea is to aggregate the two pixelswhen diwerences are rather small, but weighting these dif-

ferences according to their relevances. The fuzzy signi"-cance of the previous sentence naturally induces the useof symbolic representations associated to fuzzy sets.

Fig. 5 gives the general symbolic description of themechanism building up the criterion. This diagram iscomposed of three main parts. The "rst one, denoted I inFig. 5, deals with the fuzzi"cation of the numerical valuesinto fuzzy subsets of symbols. The second one, Part II inFig. 5, describes the symbols de"ning the di!erent co-operation methods for the aggregation of the H}C}Isymbolic informations. These symbols are built up withthe symbols describing the chroma values. The lastone, Part III in Fig. 5, de"nes two symbols whichqualify the homogeneity of the two pixels. These twosymbols are obtained by combining the symbols describ-ing the numerical di!erences and the symbols inferred inPart II.

3.2.1. Fuzzixcation and symbolic representations of thenumerical segmentation attributes (Part I in Fig. 5)

Let C, *H, *C, *I be, respectively, the universes ofdiscourse of chroma values, hue di!erences, chroma dif-ferences and intensity di!erences. For a 3]8 bit-depthimage, these universes are:

C"*C"*I"[0, 255] and *H"[0, 127].

The chroma values are used to characterize the huerelevance areas introduced in Section 2.3. Therefore, thesymbolic fuzzi"cation of chroma is performed by de"n-ing a set of three symbols corresponding to small, me-dium or large chroma values:

f when chroma is low, hue is irrelevant. It correspondsto achromatic pixels which can be associated to thehuman notion of GREY.

f when chroma is high, hue is very relevant and can belinked to the notion of PURE color.

f when chroma is medium, hue is moderately relevant.The intermediate notion of PASTEL can be utilized.

Then, the set of symbols describing the chroma feature is:

L(C)"MGRE>, PAS¹E¸, P;REN.

The hue, chroma and intensity di!erences are symboli-cally fuzzi"ed in a classical way by using three symbols:

L(*H)"L(*C)"L(*I)

"MSMA¸¸, MEDI;M, ¸ARGEN.

Of course, the choice of these symbols is the result ofvarious experimentations on di!erent images combinedwith our expertise. In Section 5, some results will illustratee!ects of the choice of these symbols. More technicaldetails will be given about these symbolic descriptions inSection 4.1.


Fig. 4. Iterative segmentation principle.

Fig. 5. General symbolic description of the method.

3.2.2. Cooperation method dexnition (Part II of Fig. 5)The aim is to de"ne di!erent strategies, or methods,

performing the aggregation between descriptions ofsegmentation attributes. Each method corresponds toparticular in#uence given to the three subsets of symbolsdescribing H, C and I di!erences in the segmentationprocess. Reproducing human reasoning, these methodsare built up by using a set of if}then rules. A very simplerule could be expressed thus:

if hue is relevant thenhue diwerence is essentially used in segmentationprocess

elseintensity and chroma diwerences are both used insegmentation process

Of course a slightly more complex set of rules is neces-sary to take the in#uence of di!erent features into ac-count in a more discriminating way. Once more, theserules are obtained thanks to various experimentationsand human expertise. Table 1 proposed one possible setof rules (denoted R

1in Fig. 5) de"ning six cooperation

methods:

M"MI, I#c, i#C#h, i#C#H, C#H, SepN


Table 1Rule-base R

1arranged in a decision table

SC3L(C)

S@C3L(C)

GREY PASTEL PURE

GREY I I#c SepPASTEL I#c i#C#h i#C#HPURE Sep i#C#H C#H

and corresponding to nine if}then rules (the nine combi-nations of the two symbolic descriptions of chromavalues c

1and c

2). It can be noted that these rules only use

symbolic descriptions of chroma values c1

and c2.

The meanings of these symbols are:

I: Segmentation process uses only intensityI#c: Segmentation process uses both intensity and

chroma, but intensity is privileged,i#C#h: Segmentation process uses the three com-

ponents, but chroma is privileged.i#C#H: Segmentation process uses the three com-

ponents, but chromatic components are privileged.C#H: Segmentation process uses hue and chroma.SEP: Chroma of the two pixels are so di!erent that

they must be separated.

The way a method is denoted can be understood in thefollowing way: an upper letter means that the corre-sponding component is privileged. On the contrary,a lower letter means that the corresponding componentis used with a lower in#uence. When a component is notmentioned in a method name, this component is not usedat all in the segmentation. In that way, the bold cell ofTable 1 is the result of the following rule:

if (c1

is PURE and c2

is PASTEL) or (c1

is PASTEL andc2

is PURE) thenH, C, I diwerences are used in segmentation process,but hue and chroma are privileged

Of course, this table is symmetric. The way the set ofrules is designed has a great in#uence on the segmenta-tion result. The reasons of choice of the proposed rule-base and further considerations about this problem willbe discussed in Section 5.4.

The mechanism privileging a component will be ex-plained in Section 4.2 by detailing the fuzzy inferenceproducing the fuzzy descriptions of the methods.

3.2.3. Homogeneity measure dexnition (Part III of Fig. 5)A second rule-base (denotedR

2in Fig. 5) builds up the

homogeneity criterion in order to realize the fuzzy ag-

gregation between the two pixels. Let Q be the set ofsymbols qualifying the homogeneity:

Q"MHOMOGENEO;S, HE¹EROGENEO;SN

This rule-base is de"ned by a set of six if}then}elserules (one for each method) combining the fuzzydescriptions of the three di!erences dH, dC and dIand the cooperation methods de"ned in Section 3.2.2.The if}then}else rules are set up according to the mean-ing of the cooperation method following such astrategy:

f if a component is privileged in a method, the homo-geneity will be achieved only if the correspondingdi!erence is small.

f if a component is used in a method but without beingprivileged, the homogeneity activation is less strict andwill be set if the corresponding di!erence is small ormedium.

Thus, for the method i#C#H corresponding to thegrey cell in Table 1, the if}then}else rule has the followingexpression:

if method is i#C#H thenif (dI is small or medium) and (dH is small) and (dCis small) then

Pixels are homogeneouselsePixels are heterogeneous

Table 2, summarizing the six if}then}else rules asso-ciated to the six cooperation methods, presents the situ-ations where pixels are declared to be homogeneous.

It is important to mention that the design of these rulesis absolutely automatic and does not require any expert-ise. With the chosen strategy, it only depends on thenames of the methods. In fact, it means that the key pointof the proposed segmentation is the design of the R

1set

of rules.Fuzzy inference producing the fuzzy description of

homogeneity symbols will be detailed in Section 4.2.

4. Fuzzi5cation techniques and inference mechanisms

The general description of the method was given aboveonly in a symbolic manner. It now needs to be detailed ina more technical way. In a "rst part, the fuzzi"cationof the di!erent numerical data that are used will bepresented. The second part will describe the inferencemechanisms associated to the two sets of rules R

1and

R2. Finally, a general diagram will summarize the algo-

rithm.


Table 2Rule-base R

2arranged in a decision table

FeaturesMethods

dL

dH

dC

I SMALL Not used Not usedI#c SMALL Not used SMALL or MEDIUMi#C#h SMALL or MEDIUM SMALL SMALL or MEDIUMi#C#H SMALL or MEDIUM SMALL SMALLC#H Not used SMALL SMALLSep Whatever di!erence descriptions there may be, no homogeneity is available

Fig. 6. Fuzzy meanings of the three symbols.

Fig. 7. Fuzzy descriptions of dI0.

Fig. 8. Fuzzy meanings of Chroma.

4.1. Fuzzixcation of the numerical domains

4.1.1. Fuzzixcation of H, C and I diwerencesLet us consider "rst the intensity di!erence.The absolute intensity di!erence ranges from 0 to 255,

and three symbols SMALL, MEDIUM and LARGE areused to symbolically characterize this di!erence. In a "rststep, we can de"ne the fuzzy meanings of each symbolwhich are characterized by their membership functions,respectively labelled k

SMALL(dI), k

MEDIUM(dI), k

LARGE(dI).

The design of these functions is performed in a veryclassical way using piece-wise linear functions given inFig. 6.

This de"nition is based on the choice of two thresholdsdI

SMALLand dI

LARGE. Actually these values are tuned

with the help of human expertise after several experi-mentations. Thanks to the linear shape of membershipfunctions, segmentation results are not too sensitive tolittle changes of these two values. Typical values aredI

SMALL"8 and dI

LARGE"16.

In a second step, the fuzzy descriptions of a speci"cvalue dI0 (see Fig. 6) are de"ned for the three symbols.They are, respectively, denoted k

dI0(SMA¸¸), k

dI0

(MEDI;M), kdI

0(¸ARGE). Illustration is given in Fig. 7.Of course, the link between the fuzzy meaning and the

fuzzy description is formalized by the following relation-ship: k

dI0(SMA¸¸)"k

SMALL(dI0).

The fuzzi"cations of hue and chroma di!erences areperformed in a very similar way. These de"nitions alsorequire the choice of two thresholds dH

SMALLand

dHLARGE

for hue (with 6 and 10 as typical values) and twothresholds dC

SMALLand dC

LARGEfor chroma (with 8 and

16 as typical values).The hue di!erence is calculated modulo 128 so as to

take the circular representation of this feature into ac-count. So, the universe of discourse *H is limited to[0, 127].

4.1.2. Fuzzy partition of chromaChroma values range from 0 to 255 and three symbols

GREY, PASTEL and PURE are used to characterize huerelevance or, it is equivalent, chroma magnitude. Thefuzzy meanings of these three symbols are representedby their membership functions, respectively labelledkGREY

(c), kPASTEL

(c), kPURE

(c). The shape of these func-tions is also triangular, as shown in Fig. 8.

This partition of chroma is built up with two para-meters h

GREYand h

PURE. These two parameters are linked

to the limits between the three areas de"ned in Section


2.3. The values of these parameters can be tuned bylooking at the chroma histogram and by taking theoret-ical hue noise sensitivity into account (see Fig. 2). Typicalvalues are h

GREY"35 and h

PURE"85. The fuzzy descrip-

tions of a speci"c value c0

of chroma are respectivelykc0(GRE>), k

c0(PAS¹E¸), k

c0(P;RE).

4.2. Inferences of the rule-bases

4.2.1. Inference mechanism of rule base R1

determiningcooperation methods

Six symbols have been de"ned to characterize thecooperation methods between the di!erent features of thetwo pixels P

1and P

2. To get the fuzzy descriptions of each

of these symbols, an inference is performed according tothe combination/projection principle (also referred to asthe generalized modus ponens). So,

∀SM3M"MI, I#c, i#C#h, i#C#H, C#H, SEPN,

∀SC, S@

C3 L(C)"MGRE>, PAS¹E¸, P;REN,

kC1, C2

(SM)"o

SC, S{C|L(C)T MT [k

C1(S

C), k

C2(S@

C)],

kC1(S

M, S

C, S@

C)N (6)

where T denotes a conjunction operator, o a disjunctionoperator and C

1the crisp graph of rule base R

1corre-

sponding to Table 1.The choice of T and o operators is important. In

Section 5.5, it will be shown that the segmentation pro-cess is less sensitive to inherent noise if the t-norm T is theproduct and the t-conorm o is the bounded sum to 1.Furthermore, as all partitions are strict and the graphC1

is crisp, we can replace the bounded sum by a simplesum.

So, as an example using these operators, the fuzzymeaning of symbol `I#ca becomes:

kc1, c2

(I#c)"kc1(GRE>) Hk

c2(PAS¹E¸)

#kc2(GRE>) Hk

c1(PAS¹E¸). (7)

Fuzzy meanings of other symbols can be easily obtainedin the same manner.

4.2.2. Inference mechanism of rule base R2

determininghomogeneity

Two symbols de"ne the homogeneity between the twopixels P

1and P

2. These symbols depend on H, C, I di!er-

ences and cooperation methods. To get the fuzzy descrip-tions of each of these symbols, an inference is againperformed according to the projection principle. So,∀S

H0.3Q"MHOMOGENEO;S,HE¹EROGENEO;SN

∀SM3M(depending on c

1and c

2values), ∀S*I3L(*I),

∀S*H3L(*H), ∀S*C3L(*C),

kC1, C2

, dI, dH, dC(SH0.)

"oSM|SCM, S*I | L(*I), S*H | L(*H), S*C | L(C)

2

2TMT [kC1, C2

(SM), k

dI(S*I), kdH

(S*H), kdC

(S*C)],2

2kC2(S

M, S*I, S*H, S*C, SH0.

)N (8)

where T and o denote conjunction and disjunction oper-ators and C

2the crisp graph of rule base R

2correspond-

ing to Table 2. Using again product as t-norm T andsimple sum as t-conorm o, the fuzzy meaning of symbolhomogeneous is given by:

kc1, c2

, dI, dH, dC (HOMOGENEO;S)

"kc1, c2

(I)HkdI(SMA¸¸)

#kc1, c2

(I#c)H[kdI

(SMA¸¸)

* [kdC

(SMA¸¸)#kdC

(MEDI;M)]]

#kc1, c2

(I#c#H)

* [[kdI(SMA¸¸)#k

dI(MEDI;M)]Hk

dC(SMA¸¸)

* [kdH

(SMA¸¸)]#kdH

(MEDI;M)]]

#kc1, c2

(i#C#H)

* [[kdI(SMA¸¸)#k

dI(MEDI;M)]Hk

dC(SMA¸¸)

*kdC

(SMA¸¸)]#kc1, c2

(C#H) H [kdC

(SMA¸¸)

*kdC

(SMA¸¸)] (9)

It can be demonstrated that the fuzzy meaning expres-sion of symbol HETEROGENEOUS is:

kc1, c2

, dI, dH, dC (HE¹EROGENEO;S)

"1!kc1, c2

, dI, dH, dC (HOMOGENEO;S). (11)

4.3. Summary

The diagram in Fig. 9 summarizes the fuzzy mecha-nisms used to get the fuzzy descriptions of the Homo-geneity symbol. To keep this diagram clear, only thecontribution of the `I#ca method is shown:

5. Application on real images

The proposed method is applied to three real images.Two of them, `peppersa and `housea, are classical ones.The third one is a microscope color image of an alloy(provided by the Ugine Savoie company). As regiongrowing generally tends to produce an over-segmenta-tion, a pre-processing is used in order to both reduce thenoise in homogeneous areas and enhance edges betweenthe regions. The pre-processing which is used is a non-linear vectorial "lter, called dab "lter, detailed in Ref. [25].


Fig. 9. Contribution of method `I#ca to the fuzzy meaning of HOMOGENEITY symbol.

To get a simpli"ed image only composed with the largerregions, a post-processing can eventually be utilized. Itconsists in merging the small regions (sizes under a "xedthreshold) into the region which is both close from a spa-tial point of view and similar according to the H}C}Icomponents.

5.1. Experimental results on the `peppersa image

The general behavior of the method is illustrated withthe `peppersa image. Fig. 10 presents the original image"ltered by the dab "lter (Fig. 10a), the frontiers of theregions detected by the method (Fig. 10b), the frontiers ofthe regions after merging the small regions } whose areasare smaller than 80 pixels } (Fig. 10c), and the corre-sponding segmented image (Fig. 10d). It can be seen (Fig.10b) that over-segmentation occurs in many cases. How-ever, after applying the post-processing step mentionedabove, the main regions "t the main objects of the imagecorrectly. Nevertheless some frontiers are lost (black ar-rows in Fig. 10c). In fact, whatever the component maybe, the contrasts along these frontiers are too small. Onthe contrary, the green pepper in the center of the imageis divided in two because of a shadow e!ect (grey arrowin Fig. 10c). Four iterations were required to get this

result, and the time processing (without pre-processingand post-processing) was about 7 s with a Sun Sparc-station Ultra-170 (the 24-bit image size is 512]512 pixels).

5.2. Comparison with another method

Comparison with other methods is a di$cult exercise.To be valid, the comparison must be performed underthe best conditions for each method. These conditionsare not always easy to ful"l for competitive methods.

First, the result obtained on the above `peppersa im-age can be qualitatively compared with the one presentedby Moghaddamzadeh [13]. In the latter, the proposed"ne segmentation gives a result close to the result pre-sented in Fig. 10d. The small noisy regions are lessnumerous, but the three main peppers are better de"nedwith our approach.

Second, because there are no parameters to set, ourmethod is compared to the algorithm presented by Lim[12] and introduced in Section 1. The original image isthe `alloya image (Fig. 11a) and the aim is to detect allthe grains. Fig. 11b presents the result of our algorithm.Fig. 11c and Fig. 11d present Lim's results using respec-tively the RGB color space and the YUV color space.


Fig. 10. Segmentation results with the `peppersa image. (a) Image "ltered by dab "lter. (b) Frontiers of the regions. (c) Frontiers of largeregions ('80 pixels). (d) Segmented image.

The better results of our method are due to twocauses:

f First, our method explicitly uses the relevant hue in-formation which is less sensitive to noise. This avoidsthe breaking up of certain noisy grains (black arrowsin Fig. 11c and Fig. 11d).

f Secondly, Lim uses automatic thresholding and, con-sequently, the di!erence between regions with closefeatures cannot be detected (grey arrows in Fig. 11d).

5.3. Inyuence of the choice of the symbolic representations

The segmentation results are sensitive to the choice ofthe symbols used to represent the di!erent features neces-sary to the segmentation processing.

In Fig. 12, a "rst result is shown with the `alloya image.The aim is to present the in#uence of the choice of themethods de"ning the way the fusion between the di!erentH}C}I features is performed. Two sets of methods arecompared. The "rst one, M

1"MI, I#c, i#C#h,

i#C#H, c#h, sepN, "nds the separation between twosimilar grains (black arrow in Fig. 12a). However someareas of the segmented image are noisy (grey arrow inFig. 12a). On the contrary, the second set of methods,M

2"MI, I#c, I#c#h, i#C#H, C#H, SepN, gives

a less noisy segmentation but does not "nd the limitbetween the two similar grains (grey and black arrows inFig. 12b). Table 3 summarizes the two sets of rulesassociated to the two sets of cooperation methods:

The di!erence between the two sets of methods isslight. When the two chroma features are medium, the


Fig. 11. Comparison between Lim's algorithm and our algorithm. (a) Image "ltered by dab "lter. (b) Frontiers of the regions.(c) Frontiers of the regions with Lim's algorithm with RGB basis. (d) Frontiers of the regions with Lim's algorithm with YUV basis.

chroma di!erence is privileged in M1. In the same situ-

ation, it is the intensity di!erence which is privileged inM

2. Nevertheless, this minor di!erence produces a major

e!ect on the segmentation.A second experimentation has been performed to show

that a speci"c aim can be reached with a speci"c set ofsymbols. In the example presented below with the`housea image, the aim which has been de"ned is to getregions without taking shadow e!ect into account.Consequently, a speci"c symbolic representation isused. First, chroma is symbolically described in a moredetailed way by using four symbols, GREY, PASTEL-LOW, PASTEL-HIGH and PURE. Then, only hue andintensity di!erences are used in the de"nition of the

homogeneity criterion. Table 4 presents the sets of rulesassociated.

It can be noted that when the two chroma are de-scribed by the symbol PURE, only the Hue information isutilized. It is this strategy which allows to avoid theshadow e!ect. This method has been applied to the`housea image (Fig. 13a).

The results are presented in Fig. 13b (small regions whosesizes are lower than 20 pixels have been merged). It can beseen that areas split by a shadow e!ect are generally seg-mented within a single region (see white arrows in Fig. 13aand b). When the shadow e!ect is too important, the huefeatures are not so relevant and the segmentation processgives noisy frontiers (see grey arrow in Fig. 13a and b).


Fig. 12. Results with di!erent cooperation methods. (a) Frontiers of the regions with the set of methodsM1. (b) Frontiers of the regions

with the set of methods M2.

Table 3Rule-bases associated to the two sets of methods

M1

GREY PASTEL PURE M2

GREY PASTEL PURE

GREY I I#c Sep GREY I I#c SepPASTEL I#c i#C#h i#C#H PASTEL I#c I#c#h i#C#HPURE Sep i#C#H C#H PURE Sep i#C#H C#H

Table 4Speci"c rule-base arranged in a decision table

¸3L(C)¸@3L(C)

GREY PASTEL-LOW PASTEL-HIGH PURE

GREY I I SEP SEPPASTEL-LOW I I#h I#h SEPPASTEL-HIGH SEP I#h I#H i#HPURE SEP SEP i#H H

5.4. Parameter and symbol selection

Looking at the di!erent applications presented above,it can be seen that the parameter values and the symboldesigns play signi"cant roles in the results of the segmen-tation. Moreover, selecting the appropriate set of para-meters and symbols is di$cult and is often the key tosegmenting the image e!ectively. Three answers can begiven to solve this problem.

First, as has been already said, using symbolic repres-entations associated with fuzzy sets reduces the criticality

of parameter and symbol selection. Presented in Ref.[26], this property is illustrated in the example in Fig. 14where a crisp segmentation is compared to the samesegmentation designed in a fuzzy way. Fig. 14b showsthe result of the crisp segmentation with correct para-meter values. Fig. 14c shows the result of the samecrisp segmentation with parameter values increasedby 20%. Fig. 14d shows the result of the fuzzysegmentation with correct parameter values. Fig. 14eshows the result of the same crisp segmentationwith parameter values increased by 20%. Obviously,


Fig. 13. Segmentation results with the `housea image. (a) Original image. (b) Frontiers of the regions.

Fig. 14. Comparison between the crisp and the fuzzy segmentation robustness. (a) Original image. (b) Frontiers of the regions witha crisp segmentation and correct parameter values. (c) Frontiers of the regions with a crisp segmentation and parameter valuesincreased by 20%. (d) Frontiers of the regions with a fuzzy segmentation and correct parameter values. (e) Frontiers of the regions witha fuzzy segmentation and parameter values increased by 20%.


Fig. 15. Performance evaluation for di!erent sets of T ando operators.

decrease in the performance is less important in the fuzzycase.

Secondly, although real-world applications proposea wide diversity of images, a speci"c problem is generallyassociated with a very small diversity of images. So, thesetup of the technique can be performed only once for thewhole application, thanks to human expertise and toseveral experimentations with a typical sample of images.Thus, a more formal training process, based on neuralnetworks for instance, could be used to give an objectivede"nition of parameters and symbols.

Thirdly, parameter and symbol selections can be real-ized by the processing itself. Di!erent approaches arethen possible. It can be a global set up according tocharacteristics evaluated from the whole image. Lim'smethod [12] is an example of such a strategy. As imagecharacteristics are not always stationary, an adaptive setup can be de"ned according to local characteristics. Onthe otherhand, the segmentation results can be evaluatedin order to control the parameter and symbol selections,giving a closed-loop image segmentation. Such an orig-inal strategy is presented in Ref. [27].

5.5. Performance evaluation

A performance evaluation has been experimented. It isbased on the measure of a distance [28] between theresult of the segmentation and a referenced segmentation.As this implies that the result of the segmentation isknown, it has been used on a synthetic image. This isa strong hypothesis which limits the interest of this evalu-ation. The use of this measure is illustrated in the valida-tion of the choice of T and o operators. Fig. 15 gives theevolution of this measure versus the noise variance of theadditive gaussian noise corrupting the synthetic image. Ithas been done for two sets of T and o operators. Beyonddoubt, the `product/suma set gives better results than the`min/maxa set.

This measure could also be used as a tool to comparethe proposed method to other ones. This is not so easy.The "rst problem is to dispose of the segmentation re-

sults of the di!erent methods applied on a same represen-tative image. The second problem is in the choice of such arepresentative image, because performances of a methodgenerally depend on the image type and on the applica-tion. Finally, a unique global measure obtained withsimulations on synthetic images is not fully satisfactorybecause it does not always "t with human opinion.

6. Conclusion

In this paper, a segmentation method for color imageshas been proposed. Performed in a hue, chroma, intensitybasis, this method is based on a symbolic representationof di!erent features necessary to the segmentation pro-cess. In the same way, the homogeneity criterion, which isthe core of the segmentation process, is also expressed ina symbolic way using if}then}else rule-bases.

The key point of this algorithm is the weighted ag-gregation of hue, chroma, intensity features to de"ne thehomogeneity criterion. Di!erent aggregation strategies(or methods) are used to realize this fusion. The design ofstrategies is essentially performed by taking the hue rel-evance, which is evaluated thanks to the Chroma magni-tude, into account. Human expertise is also a parameterplaying a signi"cant role in this design.

This strategy has been applied to an iterative growingregion technique. Despite the simplicity of this technique,quite good results are obtained in a short processingtime.

With the same strategy, further developments can beconsidered. A "rst one consists in introducing adaptivityinto method set up. By considering some local statisticalcharacteristics in the neighborhood of the current pixel, itwill be possible to decide which is the best method toutilize. In the application shown in Fig. 12, the improve-ment provided by such an adaptive system could be a lessnoisy segmentation with more signi"cant frontiers. A "rstattempt in that direction has highlighted the di$culty to"nd consistent local statistical characteristics.

A second extension could be the association of theproposed method with a frontier approach. Ref. [19]proposed a color edge detection using a similar HCIfuzzy fusion strategy. It could be combined to the regiongrowing approach to improve the measure of homo-geneity.

Furthermore, the general principle of the methodcould be applied to other region segmentation algo-rithms. For instance, similarity between two regionscould be measured by using the proposed homogeneitycriterion: H}C}I means, standard deviations, etc. of thetwo regions could take the place of H}C}I values of thetwo pixels P

1and P

2. Then, techniques like split and

merge, could be performed using our strategy.


References

[1] N.R. Pal, S.K. Pal, A review on image segmentation tech-niques, Pattern Recognition 26 (9) (1993) 1277}1294.

[2] R.M. Haralick, L.G. Shapiro, Image segmentation tech-niques, Comput. Vision Graphics Image Process. 29 (1986)100}132.

[3] Y. Xiaohan, Yla-Jaaski, Image segmentation combiningregion growing and edge detection, in: 11th IAPR, TheNetherlands, 1992, pp. 481}484.

[4] P.E. Trahanias, D. Karakos, A.N. Venetsanopoulos,Directional processing of color images: theory and ex-perimental results, IEEE Trans. Image Process. 5 (1996)868}880.

[5] K. Tang, J. Astola, Y. Neuvo, Nonlinear multivariateimage "ltering techniques, IEEE Trans. Image Process.4 (1995) 788}798.

[6] G. Wyszecki, W.S. Stiles, Color sciences: Concepts andMethods, Quantitative Data and Formulae, Wiley, NewYork, 1982.

[7] J.C. Bezdeck, Pattern Recognition with Fuzzy ObjectiveFunction Algorithms, Plenum Press, New York, 1981.

[8] L.A. Zadeh, Fuzzy Sets and their Applications to Classi-"cation and Clustering, Academic Press, New York, 1977,pp. 251}299.

[9] W. Pedrycz and all, Special issue on fuzzy set technologyin pattern recognition, Pattern Recognition Lett. 17(1996).

[10] F. Russo, Nonlinear fuzzy "lters: an overview, in: Proceed-ings of EUSIPCO'96, Trieste, Italy, 1996, pp. 1709}1712.

[11] T.L. Huntsberger, M.F. Descalzi, Color edge detection,Pattern Recognition Lett. 3 (1985) 205}209.

[12] Y.W. Lim, S.U. Lee, On the color image segmentationalgorithm based on the thresholding and the fuzzy c-Means techniques, Pattern Recognition 23 (9) (1990)935}952.

[13] A. Moghaddamzadeh, N. Bourbakis, A fuzzy region grow-ing approach for segmentation of color images, PatternRecognition 30 (6) (1997) 867}881.

[14] M. Celenk, A color clustering technique for image segmen-tation, Comput. Vision Graphics Image Process. 52 (1990)145}170.

[15] R. Schettini, A segmentation algorithm for color image,Pattern Recognition Lett. 14 (1993) 499}506.

[16] S. Tominaga, Color image segmentation using three percep-tual attributes, in: Proceedings of CVPR, 1986, pp. 628}63.

[17] T. Miyawaki, S. Ishibashi, F. Kishino, A region segmenta-tion method using color information, in: Proceedings ofIMAGECOM 90, Bordeaux, France, 1990, pp. 288}292.

[18] J.M. Chassery, C. Garbay, An iterative segmentationmethod based on contextual color and shape criterion,IEEE Trans. Pattern Anal. Mach. Intell. 6 (6) (1984)794}800.

[19] T. Carron, P. Lambert, Fuzzy color edge extraction byinference rules, Quantitative study and evaluation of per-formances, in: Proceedings of ICIP 95, Washington,USA, 1995, pp. 977}981.

[20] T. Carron, P. Lambert, Integration of linguistic knowl-edge for color image segmentation, in: Proceedings ofEUSIPCO'96, Trieste, Italy, 1996, pp. 1729}1732.

[21] T. Carron, P. Lambert, Symbolic fusion of Hue-Chroma-Intensity features for region segmentation, in: Proceedingsof ICIP 96, Lausanne, Switzerland, 1996, pp. 971}974.

[22] J. Kender, Saturation, hue, and normalized color: calcu-lation digitization e!ects and use, Master's Thesis, Dep. ofComput. Sci., Carnegie-Mellon University, 1976.

[23] L.A. Zadeh, Fuzzy sets, Inform. Control 8 (1965) 338}353.[24] G. Mauris, E. Benoit, L. Foulloy, The aggregation of

complementary information via fuzzy sensors, J. Int. Meas.Confederation 17 (4) (1996) 235}249.

[25] W. Alshatti, Ph. Bolon, P. Lambert, A vector rank "lter:application to multi-spectral image "ltering, in: Proceed-ings of 1995 IEEE Workshop on Nonlinear Signal andImage Processing, Neos Marmaras, Halkidiki, Greece, vol.2, 1995, pp. 630}633.

[26] M. Barni, S. Rossi, A fuzzy expert system for low levelimage segmentation, in: Proceedings of EUSIPCO'96,Trieste, Italy, 1996, pp. 1725}1728.

[27] B. Bahnu, S. Lee, J. Ming, Adaptive image segmentationusing a genetic algorithm, IEEE Trans. Systems ManCybernet. 25 (12) (1995) 1543}1567.

[28] P. Zamperoni, V. Starovoitov, On measures of dissimilar-ity between arbitrary grey scale images?, Int. J. ShapeModeling 2 (2&3) (1996) 189}213.

About the Author*PATRICK LAMBERT graduated from the Ecole d'IngeH nieurs Electriciens de Grenoble in 1978. He received thePh.D. degree in Signal Processing from the Institut National Polytechnique de Grenoble in 1983.

He is currently Assistant Professor of Electrical Engineering at the University of Savoie (Institut Universitaire de Technologie) andworks in the Laboratoire d'Automatique et de MicroInformatique Industrielle in Annecy } France. His research interests are onmulti-component Image Processing and Analysis.

About the Author*THIERRY CARRON received the M.S. degree in Electrical Engineering of the University of Savoie in 1990 and hisPh.D. degree in Electronics, Electrotechniques and Control from the University of Savoie in 1995. His research interests are color imagesegmentation. He is currently working as a Software Engineer in the Silicomp Ingenierie company in Grenoble.


Symbolic fusion of luminance-hue-chroma features for region segmentation

Documents

Transcript of Symbolic fusion of luminance-hue-chroma features for region segmentation