Human Color Perception in the HSV Space and its Application in

Histogram Generation for Image Retrieval

A. VadivelDepartment of Computer Science & Engineering

Indian Institute of Technology Kharagpur 721302 India.

CBIR – An Introduction

• Content Based Image Retrieval involves retrieval of images from a database which are “similar” to a query image.

• The similarity metric is usually based on image features such as color, texture, shape, etc.,.

• Some existing CBIR systems:- QBIC, VisualSeek, NeTra, MARS, Blobworld, PicToSeek,SIMPLIcity.

• Color based feature are found to be most effective.

Chose an appropriate Color Space such as RGB, HSV, etc.

To find NN with respect to Color Perception

HSV color representation separates Chrominance and luminance components

Hue

Saturation

Intensity

Developing color based CBIR Applications

View Of HSV Color space

Typical Color Histogram

RGB Color Histogram LSB is truncated

Various BINs

One update for one pixel

A Pixel

When S=0, Hue is undefined

When I=0, Saturation is undefined

Pixel is “True Color”, when H is defined

Pixel is “Gray Color”, when H is Undefined

Updated Here OR Updated Here

TRUE COLOR GRAY COLOR

Useful properties of HSV color space

Histogram

S=0, No Color PerceivedS is increased color perceived

Saturation (S)

I=0, Black Pixel I is increased for S= 1.0

Intensity (I)

Saturation and Intensity values of a pixel determine “True color” pixel or a

“Gray color” pixel.

The variation in perceived color with changes in S and I

min max min max

Fovea centralis(only cones) Very high – resolution color vision

but low light sensitivity

Saturation

Saturation(S)

Color weight

0.250.0

Near Saturation threshold

Transition is gradual

The color is perceived gradually

min

max

Saturation Threshold to determine“True Color and Gray Color”

Color weight = 1 if S>0.25 = 0 if S<=0.25

Color weight is added to the respective BIN’s value of the Color Histogram

wH(S,I) should satisfy the following conditions

(a) wH(S,I) is in the range [0,1]

(b) For S1 > S2, wH(S1,I) > wH(S2,I)

(c) For I1 > I2, wH(S,I1) > wH(S,I2)

(d) wH(S,I) changes slowly with S for high values of I

(e) wH(S,I) changes sharply with S for low values of I

A function WH(S,I)to capture the variation in perceived color with changes in S and I

Justification

High I and low S –All types of cells are excited yet WH

is low due to sharp vision, i.e. simultaneous excitation

of different types of cones in the fovea centralis due to

white light mix leads to loss in color. – captured by condition (d)

Low I and high S– Visual perception is predominantly

by rod cells very low contribution by cone cells.

=> loss of color perception.

- captured by condition (e)

A Typical Choice of WH(S,I).

WI (S,I) = 1 – WH (S,I) WH – True color degree of a

pixel.

WI - Gray color degree of a

pixel.

A Pixel

WH

WI

Here r1 and r2 are constants

with 0.2 and 1.5 as values

+ = 1

0Ifor 0

0Ifor )/255(,

21

rIrsISWH

AND

TRUE COLOR GRAY COLOR

BIN VALUE =

BIN VALUE

+WH

BIN VALUE =

BIN VALUE

+WI

A Pixel

WI

WH

Histogram Generation

Histogram

For each pixel in the imageRead the RGB valueConvert RGB to Hue (H), Saturation (S) and Intensity Value

(I)

Determine wH(S,I) and wI(S,I)

Update histogram as follows:HCPH[Round(H.MULT_FCTR)]=

HCPH[Round(H.MULT_FCTR)]+ wH(S,I)

HCPH[NT+Round(I/DIV_FCTR)]=HCPH[NT+ Round(I/DIV_FCTR)]+wI(S,I)

Histogram Generation Procedure (HCPH)

NT = Round (2MULT_FCTR) +1

Ng = 1DIV_FCTR

Imax

Here NT is total number of True color BIN Ng is total number of Gray color BINS

WEB BASED IMAGE RETRIEVAL SYSTEM

Interested user can visit http://www.imagedb.iitkgp.ernet.in/hcph.php

0

0.5

1

2 5 10 15 20

Nearest Neighbors : Euclidean Distance

Pe

rce

ive

d P

rec

isio

n HSVSN

HSVHD

HCPH

0

0.5

1

2 5 10 15 20

Nearest Neighbors : Manhattan Distance

Pe

rce

ive

d P

rec

isio

n

HSVSN

HSVHD

HCPH

0

0.5

1

2 5 10 15 20

Nearest Neighbors : Vector Cosine Angle Distance

Pe

rce

ive

d P

rec

isio

n HSVSN

HSVHD

HCPH

0

0.5

1

2 5 10 15 20

Nearest Neighbors : Histogram Intersection Distance

Pe

rce

ive

d P

rec

isio

n

HSVSN

HSVHD

HCPH

HSVSN – HSV NormalHSVSD – HSV Hard DecisionHCPH – Human color Perception Histogram

Performance comparison with HSV color histograms.

Variation of Perceived Precision with Nearest Neighbor based on different distance measures

Observation : HCPH scheme leads to higher precision compared to other HSV color histogram for all types of distance measure

(a) (b)

(c) (d)

0

0.5

1

2 5 10 15 20

Nearest Neighbors : RGB Histogram

Per

ceiv

ed P

reci

sion

EU MH

VCAD HI

0

0.5

1

2 5 10 15 20

Nearest Neighbors : JV

Pe

rce

ive

d P

rec

isio

n

EU

HI

0

0.5

1

2 5 10 15 20

Nearest Neighbors : QBIC

Pe

rce

ive

d P

rec

isio

n

EU MH

VCAD HI

0

0.5

1

2 5 10 15 20

Nearest Neighbors : HCPH

Per

ceiv

ed P

reci

sion

EU MH

VCAD HI

(a) (b)

(c) (d)

Performance of color histogram based schemes for different distance measures

EU-Euclidean DistanceMH – Manhattan DistanceVCAD – Vector Cosine Angle DistanceHI- Histogram IntersectionJV- Jain & Vailaya’s scheme

Precision variation with nearest neighbor

Observation : Histogram Intersection distance measure leads to higher precision for most of the color histograms

Comparison of HCPH using Histogram Intersection based distance measure with some existing color

histogram based schemes

0

2

4

6

2 5 10 15 20

Nearest Neighbors : Histogram Intersection Distance

SD o

f Per

ceiv

ed P

reci

sion

RGB JV

QBIC HCPH

JV – Jain & Vailaya’s Method

HCPH- Human color Perception Histogram

Observation: (i) HCPH scheme leads to higher precision for most cases (ii ) HCPH scheme leads to uniformly high correct retrieval -> Lower standard deviation of perceived precision higher Precision

0

0.5

1

2 5 10 15 20

Nearest Neighbors : Histogram Intersection distance

Pe

rce

ive

d P

rec

isio

n

RGB JV

QBIC HCPH

Precision(P) of retrieval of HCPH and some recently proposed CBIR schemes. 

Scheme used N=10 N=20 N=50 N=100

Local Fourier Transform (LFT) Quantization (YUV)

27.59 19.76 13.42 9.89

Color Texture Moments (HSV) 32.36 25.16 16.87 12.29

Color Texture Moments (SvcosH, SvsinH, V)

35.81 26.59 18.24 13.40

Multimedia Retrieval Markup Language with Four-Level Relevance Feedback

34.00 30.00 15.48 12.01

2 Systems combSUM Merge 40.00 33.00 20.00 17.48

Color-Spatial Feature (36 Colors) 34.98 29.00 15.99 12.60

Human Color Perception Histogram (HCPH)

46.00 35.90 25.40 19.98

N= No. of images retrieved for query

Average Precision (%)

Conclusion• A Simple color weight function WH of S and I is proposed to

estimate True color degree and Gray color degree of a pixel.

• HCPH scheme has lower histogram dimension (2-D).

• HCPH scheme tries to capture human visual perception of the

color of a pixel for grouping similar pixels in the histogram.

• Histogram Intersection distance metric gives higher precision

compared to other distance metrics.

• Among different color histogram based schemes HCPH leads to higher precision is most cases.

• Standard Deviation of observed Precision is smaller with HCPH scheme.

References

[1]     R. Brunelli and O. Mich, “Histograms Analysis for Image Retrieval”, Pattern Recognition 34, pp., 1625-1637, 2001.[2]     Y. Deng, B. S. Manjunath, C. Kenney, M. S. Moore and H. Shin, “An Efficient Color Representation for Image Retrieval”, IEEE Transactions on Image Processing, 10, pp., 140-147, 2001. [3]     J. C. French, J. V. S. Watson, X. Jin and W. N. Martin, “Integrating Multiple Multi-Channel CBIR Systems”, Inter. Workshop on Multimedia Information Systems (MIS 2003), pp., 85-95, 2003.[4]     T. Gevers and H. M. G. Stokman, “Robust Histogram Construction from Color Invariants for Object Recognition”, IEEE Trans. on Pattern Analysis and Machine Intelligence (PAMI) 26(1), pp., 113-118, 2004.[5]     R. C. Gonzalez and R. E. Woods, “Digital Image Processing “, II Ed. Pearson Education Asia, First Indian Reprint, 2002.[6]     Z. Lei, L. Fuzong, and Z. Bo, “A CBIR Method Based Color-Spatial Feature”, Proc. IEEE Region 10th Annual International Conference, pp., 166-169, 1999.[7]     H. Mueller, W. Mueller, S. Marchand-Maillet, D. Squire and T. Pun, “ A Web-Based Evaluation System for CBIR”, Third Intl. Workshop on Multimedia Information Retrieval (MIR2001), 2001.[8]     T. Ojala, M. Rautiainen, E. Matinmikko and M. Aittola, “Semantic Image Retrieval with HSV Correlograms”, Scandinavian Conference on Image Analysis, pp., 621-627, 2001.[9]     H.Yu, M. Li, H J. Zhang and J. Feng, “Color Texture Moments for Content-Based Image Retrieval”, Proc. Int. Conference on Image Processing, Volume III, pp., 929-931, 2002.[10]  F. Zhou,J. Feng and Q. Shi, “Texture Feature Based on Local Fourier Transform”, Proc. Int. Conference on Image Processing, pp., 7-10, 2001.

Representation of colors in the histogram.

True Color Components

Gray Color Components

Circular representation of “true colors” and linear representation of “gray colors”.

Pixels when Hue is undefined

Saturation

Pixels when Hue is defined

Weight = 1 if S>0.25 = 0 if S<=0.25

Saturation(S)

Weight

0.250.0

Weight is added to the respective BIN’s value of the Histogram

Saturation Threshold to determine“True Color and Gray Color”min

max