261610096

8/3/2019 261610096

1/4

A Novel Unsupervised Salient Region Segmentation for Color Images

Yu-Hsin Kuan, Shih-Ting Chen, Chung Ming Kuo, and Chaur-Heh HsiehDepartment of Information Engineering, I-Shou UniversityTahsu, 840, Kaohsiung, Taiwan

Abstract

In this paper, we propose a novel unsupervised

algorithm for the segmentation of salient regions in

color images. There are two phases in this algorithm.

In the first phase, we use nonparametric density

estimation to extract dominant colors in an image,which are then used for the quantization of the image.

The label map of the quantized image forms initialregions of segmentation. In the second phase, a region

merging approach is performed. It merges the initial

regions using a novel region attraction rule to form

salient regions. Experimental results show that the

proposed method achieves excellent segmentation

performance for most of our test images. In addition,the computation is very efficient.

1. Introduction

Nowadays, color images are extensively used in

multimedia applications. The use of low-level visual

features to retrieve relevant information from imageand video databases has received much attention in

recent years. For last two decades, many content-basedimage retrieval systems have been established [1, 2].

Usually, low-level feature descriptors (color, shape,

texture, etc.) retrieve too many unrelated images andconsequently their performances are unsatisfactory.

Using high-level semantic descriptors such as object,

scene, place, etc. should be more consistent with

human perception. Nevertheless, the semantic image

segmentation is still a challenging problem.A trade off solution to narrow down the gap

between low-level features and human perception is to

use spatial local features instead of global features ofimages. This means that we need the perceptually

relevant regions in an image and the extracted features

for matching are not from the entire image but from

segmented regions. Therefore, a suitable image

segmentation technique, which effectively partitionsimage into salient regions, is an important issue.

The main purpose of this paper is not to precisely

segment every single object in an image but to find thesalient regions that are relatively meaningful to human

perception. In the past few years, some methods have

been proposed for finding salient regions in images [3,

4], but the computational complexity and the

segmentation results are not satisfactory. The proposed

method will effectively address these drawbacks. Theremainder of this paper is organized as follows. Section

2 explains the basic idea of our model. Section 3describes the experiments and demonstrates the

experimental results. Section 4 concludes the paper.

2. The proposed method

Our method consists of two major portions: the first

is dominant color extraction and image quantizationand the second is spatial segmentation. The flow charts

of the proposed method are illustrated in Figure 1.

2.1. Dominant color extraction and image

quantization

Because the luminance and chrominance are mixed

together in RGB color space, we adopt YUV color

space to take advantage of decorrelating the luminance

and chrominance. The dominant colors are extracted based on nonparametric density estimation [3, 5].

Given an n-dimensional dataset {xiRn

; i =1N} , thenonparametric density f (x) is obtained by convolving

the dataset with a unimodal density kernelK(x),

=

=

N

i

ixxKN

xf1

)(1

)(

(1)

where is the bandwidth for the kernel. In our work,we selected a Gaussian kernel as

22 2/2

1

22

1)(

xexK

= (2)

where 2 is its variance. The density of each channel (Y,

U, and V) is estimated by (1). To speed up thecomputation of estimation, the convolution of Gaussian

Proceedings of the First International Conference on Innovative Computing, Information and Control (ICICIC'06)0-7695-2616-0/06 $20.00 2006

8/3/2019 261610096

2/4

kernel does not directly apply to the source image but

to the histogram of each channel. This tremendously

increases the processing speed and achieves equivalentresults. After the convolution, the local maxima of each

channel are obtained by using the gradient ascent

scheme. The candidates of dominant colors are thecombination of the local maxima of the 3 channels. The

number of candidates depends on the bandwidth of

the Gaussian kernelThe number of the candidates will

decrease when a large bandwidth is selected and vice

versa. Figure 2 gives an example of the dominant color

extraction. To avoid over-smoothing of the densities, a

smaller bandwidth is preferred. We assign the image

pixels to one of the candidates according to the color

distance between them. Because it may cause too manycandidates of dominant colors, we eliminate the

candidates that the image pixels assignment is lowerthan a pre-defined threshold. Then we merge the

similar colors to obtain the final dominant colors.

After the dominant color extraction, each pixel ofthe image has already been replaced by the nearest

dominant color. Consequently, a quantized color imageis obtained and a label map is created as well.

2.2. Region merging strategy

The initial regions are obtained by region growing[6] according to the quantization label map. Some of

them may be very small and less important.

Therefore, not all the initial regions are salient. In the

following, we will define the property of salient region,

and then a new region merging strategy will be

proposed.

2.2.1. Important index computation. Our goal is to

find the salient regions in an image. Therefore, we

should define saliency first. A salient region should be

compact, complete and significant enough. Based onthe definition, neither a small region nor a fragmentary

region can be important. Thus, the important index of

region is defined as follows:

( ) (3)

1 11

=

= ==

n

i

m

jR

R

m

jR

Ri

ji

ij

ij

i

ij

ij

N

N

N

NRImp

Rij : a region with color label i , region indexj.

Imp(Rij) : Important index ofRij.

: The number of pixels ofRij.

: Total number of pixels of all regions withcolor label i.

:Total number of pixels of an image.

If the important index of a region is less than merge

threshold Tm then it should be merged into an adjacent

neighbor with greatest attraction. Moreover, Tm is

proportional to the important index of the second

important region.

2.2.2. Attraction computation. The idea of attraction

computation is analog to Newtons law of universal

gravitation. For any two objects with mass m1 and m2,

separated by a distance D, attract each other with aforce

)4(2

21

D

mmGF =

For any two connected regionsR1,R2 in an image, D in

(4) is replaced by the color distance betweenR1 andR2,

and m1, m2 are replaced by the region size ofR1, R2respectively and the universal gravitation constant G is

set to one. Hence, the attraction can be expressed as

follows:

)5()2,1(

)2,1(21

RRnceColorDista

RgionSizeReRgionSizeReRRAttraction2

=

The Euclidean distance is used to compute the

distance between two colors.

( ) )6(222

2,1212121

+

+

=RRRRRR

vvuuyyRRd

Assume a is a region to be merged and b, c, dare its

neighboring regions. We should compute theattractions between a and each of b, c, d to decide

where a should be merged into. Since a is common toall attraction computations the region size of a can be

neglected. Thus (5) can be rewritten as

)7(),(

),(kanceColorDista

gionSizeRekaAttraction

2k

=

where k{b, c, d}.To make it more reasonable for the computation of

attraction, the region size is quantized into ten levels,from one to 10, to decrease the influence of the

variation of region size. Furthermore, theColorDistance in (7) is defined as

( ) ( )(8)

),(,),(

,,),(max2)(

>=

dTkadkad

dTkadkada,knceColorDista

where ( )2

),(max kaddT = , and k{b, c, d}.

When a region is merged into another region, just

simply change its color label and region index and add

its region size to the target region.

3. Experiments and results

Three parameters in our algorithm need to be preset.The first is the bandwidth of the convolution kernel.

ijR

N

=

i

ij

m

jR

N

1

= =

n

i

m

jR

Ni

ij

1 1


8/3/2019 261610096

3/4

This is set proportional to the standard deviation of

each color component histogram. The second is the

allowed maximal number of dominant colors. Based onour experiments, 10 to 20 colors are suitable for most

images. The third is the merge threshold Tm, which is

proportional to the important index of the secondimportant region. These parameter values are decided

ahead of our experiments and identical setting is used

for all images. There is no human intervention during

the process. Therefore, our approach is fullyunsupervised.

We have implemented the proposed algorithm on a

Pentium 4 PC, 2.66 GHz CPU with 512 MB RAM. The

computational efficiency of the algorithm is very good.

For CIF format images, the average speed is around 0.6

second for each image.Here, we present some of our test images and

segmentation results to demonstrate the power andpotential of our work. Figure 3 shows the experimental

results. Because we are interested in salient regions in

image, the region boundaries, which do not preciselymatch the contours of objects, are acceptable.

4. Conclusions

We presented a new salient region segmentation

approach for color images based on dominant color

extraction and region merging. A nonparametricdensity estimation was first employed to extract

dominant colors and quantize images in an efficient

way. A region attraction rule was then developed to

merge the initial regions generated in the quantization

step. The proposed approach effectively extracts salient

regions in color images. Experiments show that the

segmentation results satisfied our definition of saliency,

and the proposed method effectively addressed the

over-segmentation problem in traditional segmentationalgorithms.

5. References

[1] Y. Rui, T.S. Huang, S.F. Chang (1999), Image Retrieval:Current Techniques, Promising, Directions, and OpenIssues. Journal of Visual Communication and Image

Representation, 10(1): 39-62.

[2] B. Johansson (2000), A Survey on: Contents BasedSearch in Image Databases. Technical ReportLiTH-ISY-R-2215.

[3] E. J. Pauwels, G. Frederix. Finding salient regions inimages: Nonparametric clustering for image segmentationand grouping. Computer Vision and Image Understanding,75(1/2):73-85, 1999.

[4] A. Dimai, Unsupervised Extraction of SalientRegion-Descriptors for Content Based Image Retrieval.

Proceedings of the 10th International Conference on Image

Analysis and Processing, pages 686-691, September 1999.

[5] A. Elgammal, R. Duraiswami, D. Harwood, L. S. Davis,Background and Foreground Modeling Using

Nonparametric Kernel Density Estimation for VisualSurveillance. Proceedings of the IEEE, vol. 90, Issue 7, PP.1151 1163, July 2002.

[6] R. Adams, and L. Bischof, Seeded Region Growing. IEEE Transactions on Pattern Analysis and MachineIntelligence, June 1994, vol. 16, no. 6, pp. 641-647.

Digitalimage

Color space transformation(RGB to YUV)

Yhistogram

Uhistogram Vhistogram

Kernel densityestimator



local maxFinding local maxFinding

local maxFinding

Color combination(Color candidates)

Color selection

(Quantized colors)

Image labelingusing quantized colors

Quantizedimage

Quantizedimage

Important indexcomputation

Important index < Threshold

Compute attraction betweencurrent region and its adjacent

regions

yes

Merge current region into theregion with maximum attraction

Segmentationresult

no

Region growing

(a) (b)Figure 1. Proposed segmentation method (a) Dominant color extraction and image quantization(b) Spatial segmentation.


8/3/2019 261610096

4/4

(a)

2 local maxima 1 local maximumCandidates = 2 x 2 x 1 = 4 colors

2 local maxima

(b)

Figure 2. (a) Original densities (b) Nonparametric densities

(a) (b) (a) (b)

(a) (b) (a) (b)

(a) (b) (a) (b)

(a) (b) (a) (b)

Figure 3. (a) Source images (b) Segmentation results


261610096

Documents

Transcript of 261610096