Center for Image Processing

Fourier Transform andTexture Filtering

Lucas J. van Vlietwww.ph.tn.tudelft.nl/~lucas

Center for Image ProcessingCenter for Image Processing

CBP course: Fourier & Texture 2

Center for Image Processing

Image Analysis Paradigm

Texture filtering

analysis

segmentation

Imagerestoration

Imageenhancement

sensorImageformation scene

pre-processing

classification

Texture properties:scale

orientationLocal variance

regularity

Center for Image ProcessingCenter for Image Processing

CBP course: Fourier & Texture 3

Center for Image Processing

Texture properties

� Texture is domain property rather than a pixel property.

� Fourier analysis offers a powerful tool to analyze and describe domain properties

� Properties� Local power = local variance� Scale (from fine detail to course structures)� Anisotropy (orientation preference)

Center for Image ProcessingCenter for Image Processing

CBP course: Fourier & Texture 4

Center for Image Processing

Discrete Fourier Transform

� Each image can be decomposed in weighted sum of complex exponentials (sines and cosines) of frequency fand angle φ. (or two frequency components u and v)

( ) ( )( )

( ) ( )( )

1 1 2

0 0

1 1 2

0 0

1, ,

1, ,

N N j ux vyN

u v

N N j ux vyN

u v

g x y G u v eN

G u v g x y eN

π

π

− −+

= =

− −− +

= =

=

=

∑∑

∑∑

� For real-valued images: ( ){ } ( ){ }

( ){ } ( ){ }

Ev , Re ,

Od , Im ,

F

F

g x y G u v

g x y j G u v

←→

←→

image sizeNxN

Center for Image ProcessingCenter for Image Processing

CBP course: Fourier & Texture 5

Center for Image Processing

Getting used to Fourier (2)

An image is a weighted sum of cos (even) and sin (odd) images.

a+jb

Fourier domain with complex amplitude: a+jb

Center for Image ProcessingCenter for Image Processing

CBP course: Fourier & Texture 6

Center for Image Processing

Getting used to Fourier (1)

� Graphite surface by Scanning Tunneling Microscopy� Atomic structure of graphite shows a hexagonal surface

column

row

0

0N-1

N-1

(c,r) = (½N,½N)(u,v) = (0,0)

x

y

u

v

(c,r) = (0,0)(u,v) = (-½N,-½N)

(c,r) = (N-1,N-1)(u,v) = (½N-1, ½N-1)

Center for Image ProcessingCenter for Image Processing

CBP course: Fourier & Texture 7

Center for Image Processing

Superposition� Fourier spectrum

= + + +

= + + +

F F F F F

Center for Image ProcessingCenter for Image Processing

CBP course: Fourier & Texture 8

Center for Image Processing

Two domains

� The coordinate systems (x,y) and (u,v) have their origin at the position (column,row) = (½N, ½N)

u

v

x

y

column

row

column

row

0

0N-1

N-1

(u,v) = (0,0)

Center for Image ProcessingCenter for Image Processing

CBP course: Fourier & Texture 9

Center for Image Processing

Local variance filter� Recipe: local variance filter(filter size = n)

� Compute the local mean (blurring filter of size n)� Subtract (if necessary) the local mean.� Compute the square of each pixel value.� Suppress the “double” response by local averaging (blurring filter

of size n).� Local variance is a measure for the local squared-contrast.

Center for Image ProcessingCenter for Image Processing

CBP course: Fourier & Texture 10

Center for Image Processing

Segmenting textured objects� Simple thresholding fails!� The objects differ in texture from the background� Local variance followed by a (morphological) closing yields …

Center for Image ProcessingCenter for Image Processing

CBP course: Fourier & Texture 11

Center for Image Processing

Scaling: local vs global

� Problem: Choosing the proper scale is an important, but tedious task.� Scale too small: Texture characteristics are missed

which yields an incomplete data description.� Scale too large: Confusion (mixing) of adjacent texture

characteristics, lack of localization, and blindness for small and subtle textured regions.

� Solution: Multi-scale analysis.� Analyze the image as function of scale: from fine detail

to course “image-filling” objects.

( ) ( )( ) ( )

2 22 2 22 2

22 22

12

, ,u vx y N N

Fg x y e G u v eπ π

σ

σ

πσ

− + + −

= ←→ =

Center for Image ProcessingCenter for Image Processing

CBP course: Fourier & Texture 12

Center for Image Processing

Multi-Scale

Series of images filtered of decreasing scale: Scale-space� Sample the scales logarithmically using filters of size = basescale

yields n scales per octave

Input imagescale 0

scale 1

scale 2

scale 3

scale 4

scale 5

var (scale 1)

var (scale 2)

var (scale 3)

var (scale 4)

var (scale 5)

Local variance between scales n and n-1.

{ }11 13212 ,2 ,2 ,...,2 nbase ∈

Scale difference ≈ scale derivative

Scal

e sp

ace

Scal

e sp

ace

Center for Image ProcessingCenter for Image Processing

CBP course: Fourier & Texture 13

Center for Image Processing

Scale-spaces

� Morphological scale-space: Use openings (closings)� Gaussian scale-space: Use Gaussian filters

Increasing scales

Fourier domain with “footprints” of Gaussianfilters of increasing scale

Filter size is inversely proportional to “footprint” in Fourier domain

12

345

Center for Image ProcessingCenter for Image Processing

CBP course: Fourier & Texture 14

Center for Image Processing

Chirp exampleSc

ale-

spac

e

Scale derivative Spatial variance

Scal

e-sp

ace

Center for Image ProcessingCenter for Image Processing

CBP course: Fourier & Texture 15

Center for Image Processing

Gabor: scale and orientation

� Problem: Crossing and touching of oriented patterns.� Isotropic scale cannot discriminate based on orientation.

� Solution: Multi-orientation analysis� Orientation selective filtering

� Multi-scale and multi-orientation analysis by Gabor filtering.� Orientation selectivity

( ) ( )2 2

2 222

1, 2

,x y

i iNi i

j u x v yu vg x y e e

π

σ

πσ

+−

+=

Gaussian window of size σSelected frequency (ui,vi)

Center for Image ProcessingCenter for Image Processing

CBP course: Fourier & Texture 16

Center for Image Processing

Gabor filter bank

� Linear sampling of orientation� Logarithmic sampling of scale� (Remember: reciprocal relation of size in space and freq.

A single input imageyields many output images.

Question: Which output image(s) discriminate between textures?

( )

( )( ) ( )( )

( )

2 2 22 2

2, ,,

u u v vi iN N

i iu v

i i

G u v eG u u v v

π πσ

− − + −

=

= − −(ui,vi) σ−1

Center for Image ProcessingCenter for Image Processing

CBP course: Fourier & Texture 17

Center for Image Processing

Gabor filtering

� Each Gabor filter selects a (Gaussian-shaped) region of the Fourier spectrum of the image.

� A Gabor filter can be implemented by convolution!

( ) ( )( ) ( )( )[ ]2 2

222

2 2 1, 2

, cos sini i i ii i N N

x yu x v y u x v yu vg x y j eπ π σ

πσ

+−

+ += +

Center for Image ProcessingCenter for Image Processing

CBP course: Fourier & Texture 18

Center for Image Processing

How to go further …?

� Problem: How to segment� textured objects from background� adjacent textures

Each pixel is described by many texture attributes: a feature vectorPixels from a single texture yield a cluster in feature space.

� Solutions:� Unsupervised: Cluster analysis� Supervised: Pattern recognition

allows training a classifier based on examples