Content-based Image Retrieval Using Gabor

Texture Features

Dengsheng Zhang, Aylwin Wong, Maria Indrawan, Guojun Lu

Gippsland School of Computing and Information Technology Monash University

Churchill, Victoria, 3842, Australia Email: dengsheng.zhang, aylwin.wong, maria.indrawan,

guojun.lu@infotech.monash.edu.au

Outline

• Motivations

• Problems

• Gabor Filters

• Experiment

• Gabor Texture Features

• Rotation Invariance

• Retrieval Test

• Conclusions, Issues and Future

Work

Motivations

• Content Based Image RetrievalVOD, D-Library, MPEG-7

• TextureCBIR, Texture analysis • WaveletMPEG-7, JPEG-2000

Problems IImage retrieval

a. Object (shape) b. Color (histogram) c. Texture

?

Problems IITexture retrieval

?

Texture

• Difficult to model. (No formal definition of texture exists, intuitively texture provides measures of properties such as smoothness, coarseness and regularity etc.)

• Tamura textures. (coarseness, contrast, directionality, line-likeness, regularity and roughness.)

Texture Analysis

• Structural. Textures composed of a number of copies of a primitive placed at various locations in the image plane

• Probabilistic. Co-occurrence, Markov models, Moments

• Spectral. Fourier, Wavelet

• Fractal. Surface roughness(Box Counting), Fractal code (IFS)

Gabor texture analysis joint-space approach (among the best method)

• Psychological evidence Visual discrimination is a local process. Pattern grouping depends on both shape similarity (spatial properties) and organization (spectral properties), this leads to the need for high resolution in both the spatial and frequency domains. [Reed & Wechsler90]

• Theoretical evidence achieve lower

boundary of uncertainty in joint-space • Experimental evidence

Outperforms most other methods[Tan92, Manjunath & Ma96] • Biological evidence 2D Gabor filters

best describe the 2D receptive-field profiles of simple cells found in the visual cortex of vertebrate animals [Daugman 85, Wiskott99], therefore, they are consistent with human vision system (HSV) [Reed & Wechsler90]

Gabor Filters IDefinition

Gabor filters are a group of wavelets. For a given image I(x, y) with size P××××Q, its discrete Gabor wavelet transform is given by a convolution:

∑∑ −−=s t

mnmn tstysxIyxG ),(),(),( *ψ

where, s and t are the filter mask size variables, and

*mnψ is the complex

conjugate of ψψψψmn.

Gabor Filters IIWavelet Generation

A. Mother Wavelet

Gabor impulse response function, a Gaussian envelope modulated by a sinusoidal plane wave along the x-axis:

)2exp()](21exp[

21),( 2

2

2

2

Wxjyxyxyxyx

πσσσπσ

ψ ⋅+−=

where W is called the modulation frequency.

Transfer function of Mother Wavelet -- frequency response of Gabor function ψψψψ(x, y), i.e., its 2-D Fourier transform is:

])([21exp),( 2

2

2

2

vu

vWuvuσσ

+−−=Ψ

where σσσσu = (2ππππσσσσx)-1 and σσσσv = (2ππππσσσσy)-1.

Gabor Filters

NotationsGabor Transform and Short-time Fourier Transform (SFT)

• Fourier transform:

∫∫ +−= dydxvyuxjyxIvuF )](2exp[),(),( π (localization in frequency domain only)

• Gabor transform:

∫∫ +−−−= dtdsvyuxjtysxgyxIvuG yx ])(2exp[),(),(),(, π

∫∫ −−−= dtdswxjtysxgyxIwG yx )2exp(),(),()(, π

(simultaneous localization in spatial/frequency or time/frequency domain)

Gabor Filters IIWavelet Generation

B. Children Wavelets

• Conventional wavelets generation form:

ψψψψ(x, y) a, b ),(1),(, aby

abx

ayxba

−−= ψψ

where a = 2 j

• Gabor wavelets generation form:

ψψψψ(x, y) m, n )~,~(),(, yxayx mnm ψψ −=

)cossin(~)sincos(~

θθθθ

yxayyxax

m

m

+−=

+=−

−

where a >1 and θθθθ = nππππ/N.

Mother Wavelet

Dilation & Translation

Children Wavelets

Mother Wavelet

Dilation & Rotation

Children Wavelets

Gabor Filters IIIProperties of Gabor filters

• Scale (frequency) and orientation selectivity

• Achieve the lower boundary of the

uncertainty relationship

! Uncertainty principle: ∆∆∆∆x · ∆∆∆∆u ≥ 1/4ππππ ∆∆∆∆y · ∆∆∆∆v ≥ 1/4ππππ

! Other filters: ∆∆∆∆x · ∆∆∆∆u > 1/4ππππ ∆∆∆∆y · ∆∆∆∆v > 1/4ππππ

! Gabor filters: ∆∆∆∆x · ∆∆∆∆u = 1/4ππππ ∆∆∆∆y · ∆∆∆∆v = 1/4ππππ

• Complete but non-orthogonal

Magnitude of a Gabor filter set for N=4 in direction of the modulation frequency

Half-peak plot of the Gabor filters in the frequency plane tuned to different frequencies (with bandwith of Uh -Ul) and orientations (30 degree resolution)

Implementation of Gabor filters

• Parameters selection

m = 5 (scales), n = 6 (orientations), Ul= 0.05, Uh=0.4, s and t range form 0 to 60, i.e, filter mask size is 60x60.

• Convolution

∫∫ −−−= dtdswxjtysxgyxIwG yx )2exp(),(),()(, π

F (I ⊗⊗⊗⊗ G ) = F ( I ) · F (G )

Experimental results

Gabor texture feature & similarity measure

• Gabor texture feature

After applying Gabor filters on the image with different orientation at different scale, we obtain an array of magnitudes:

∑∑=x y

mn yxGnmE |),(|),(, m = 0, 1, , M-1; n = 0, 1, , N-1

QP

yxG

QPnmE

x ymnmn

mn

mn

×

−=

×=

∑∑ 2)|),((|

),(

µσ

µ

Gabor Feature Vector: f = (µµµµ00 , σσσσ00 , µµµµ01 , σσσσ01 , , µµµµ45, σσσσ45)

• Similarity measure

∑∑ −+−=m n

Tnn

Qnn

Tnn

QmnTQD 22 )()(),( σσµµ

Rotation Invariance-I

(a) (b)

(c) (d)

Figure 1. (a) a straw image, (b) energy map of (a), (c) rotated image of (a), (d) energy map of (c)

Rotation Invariance-II

Assume the original image is I(x, y) with dominant orientation at iππππ/N. I'(x, y) is the rotate version of I(x, y) so that its dominant orientation is at 0. If at a particular scale m, the energy distribution of I(x, y) is

(E m,0, E m,1, … E m,i, …E m,N-1)

then the energy distribution of I'(x, y) is

(E'm,-i, E' m,1-i, … E' m,0, …E' m,N-1-i)

where E m,0= E' m,-i, E m,1= E' m, 1-i, and so forth. Because E' m,n = E' m,n+N (an image has the same energy distribution after rotating 180o.), we have E' m,-i+N =E' m,-i, E' m,1-i+N = E'm, 1-i, etc. (Negative orientations are added by N). We then have the following energy distribution of I'(x, y):

(E'm,-i+N, E'm,1-i+N,…,E'm,0 ,…,E'm,N-1-i) Reorder the above distribution according to orientation values

(E'm,0 , E'm,1 ,…E'm,N-1-i , E'm,N-i , E'm,N-i+1…,E'm,N-1) this is the circular rotation of the original feature vector

Retrieval Test

• Database creation 112 512××××512 Brodatz texture images, each is cut into 16 128××××128

sub textures, to create a database composed of 1792 textures, plus some deliberately rotated textures.

• Java client-server retrieval framework retrieval online.

Conclusions and Future work

• Preliminary results show Gabor filter is very promising for texture retrieval

• Circular shift solves rotation

invariance problem • Scale invariance

• Bandwidthimage (application)

dependent

• General problemsegmentation images into texture patches