EE4H, M.Sc 0407191Computer VisionDr. Mike Spann

m.spann@bham.ac.ukhttp://www.eee.bham.ac.uk/spannm

IntroductionImages may suffer from the following

degradations: Poor contrast due to poor illumination or finite

sensitivity of the imaging device Electronic sensor noise or atmospheric

disturbances leading to broad band noise Aliasing effects due to inadequate samplingFinite aperture effects or motion leading to

spatial

IntroductionWe will consider simple algorithms for image

enhancement based on lookup tablesContrast enhancement

We will also consider simple linear filtering algorithmsNoise removal

Histogram equalisation In an image of low contrast, the image has

grey levels concentrated in a narrow band Define the grey level histogram of an

image h(i) where : h(i)=number of pixels with grey level = i

For a low contrast image, the histogram will be concentrated in a narrow bandThe full greylevel dynamic range is not used

h i( )

i

Histogram equalisationCan use a sigmoid lookup to map input to

output grey levelsA sigmoid function g(i) controls the mapping

from input to output pixelCan easily be implemented in hardware for

maximum efficiency

Histogram equalisationh i( )

g i( )

h i' ( )

h i h g i' ( ) ( ( )) 1

g ii

( )exp

1

1

Histogram equalisationθ controls the position of maximum slope λ controls the slope Problem - we need to determine the

optimum sigmoid parameters and for each image A better method would be to determine the

best mapping function from the image data

Histogram equalisationA general histogram stretching algorithm is

defined in terms of a transormation g(i)We require a transformation g(i) such that

from any histogram h(i) :

constant)()(')(:

jgij

jhih

Histogram equalisationConstraints (N x N x 8 bit image)No ‘crossover’ in grey levels after

transformation

2)(' Nihi

i i g i g i1 2 1 2 ( ) ( )

Histogram equalisationAn adaptive histogram equalisation algorithm

can be defined in terms of the ‘cumulative histogram’ H(i) :

H i( ) = number of pixels with grey levels i

i

j

jhiH0

)()(

Histogram equalisationSince the required h(i) is flat, the required

H(i) is a ramp:

h(i) H(i)

Histogram equalisationLet the actual histogram and cumulative

histogram be h(i) and H(i)Let the desired histogram and desired

cumulative histogram be h’(i) and H’(i)Let the transformation be g(i)

H g iN g i

' ( ( ))( )

2

255

( ' ( ) , ' ( ) )H N H255 0 02

Histogram equalisationSince g(i) is an ‘ordered’ transformation

i i g i g i1 2 1 2 ( ) ( )

H g i H iN g i

' ( ( )) ( )( )

2

255

g iH i

N( )

( )

2552

Histogram equalisationWorked example, 32 x 32 bit image with grey

levels quantised to 3 bits

g iH i

( )( )

7

1024

h i h jj i g j

' ( ) ( ): ( )

Histogram equalisationi h i( ) H i( ) g i( )

0 197 197 1.351 -

1 256 453 3.103 197

2 212 665 4.555 -

3 164 829 5.676 256

4 82 911 6.236 -

5 62 993 6.657 212

6 31 1004 6.867 246

7 20 1024 7.07 113

h i' ( )

Histogram equalisation

0 1 2 3 4 5 6 70

50

100

150

200

250

300

0 1 2 3 4 5 6 7

Original histogram

Stretched histogram

0.00

500.00

1000.00

1500.00

2000.00

0.00 50.00 100.00 150.00 200.00 250.00

i

h(i)

0.00

500.00

1000.00

1500.00

2000.00

0.00 50.00 100.00 150.00 200.00 250.00

i

h(i)

0.00

500.00

1000.00

1500.00

2000.00

2500.00

3000.00

0.00 50.00 100.00 150.00 200.00 250.00

i

h(i)

0.00

500.00

1000.00

1500.00

2000.00

2500.00

3000.00

0.00 50.00 100.00 150.00 200.00 250.00

i

h(i)

Histogram equalisationImageJ demonstration

http://rsb.info.nih.gov/ij/signed-applet

Image FilteringSimple image operators can be classified as

'pointwise' or 'neighbourhood' (filtering) operators

Histogram equalisation is a pointwise operation

More general filtering operations use neighbourhoods of pixels

(x,y) (x,y)

Input image Output image

(x,y) (x,y)

pointwisetransformation

neighbourhoodtransformation

Input image Output image

Image FilteringThe output g(x,y) can be a linear or non-

linear function of the set of input pixel grey levels {f(x-M,y-M)…f(x+M,y+M}.

(x,y) (x,y)

Input image f(x,y) Output image g(x,y)

(x-1,y-1)

(x+1,y+1)

Image FilteringExamples of filters:

g x y h f x y h f x y

h f x y

( , ) ( , ) ( , )

..... ( , )

1 2

9

1 1 1

1 1

g x y medianf x y f x y

f x y( , )

( , ), ( , )

..... ( , )

1 1 1

1 1

Linear filtering and convolutionExample

3x3 arithmetic mean of an input image (ignoring floating point byte rounding)

(x,y) (x,y)

Input image f(x,y) Output image g(x,y)

(x-1,y-1)

(x+1,y+1)

Linear filtering and convolutionConvolution involves ‘overlap – multiply –

add’ with ‘convolution mask’

H

1

9

1

9

1

91

9

1

9

1

91

9

1

9

1

9

(x,y) (x,y)

Input image f(x,y) Output image g(x,y)

Image point

Filter mask point

Linear filtering and convolutionWe can define the convolution operator

mathematicallyDefines a 2D convolution of an image f(x,y)

with a filter h(x,y)

g x y h x y f x x y y

f x x y y

yx

yx

( , ) ( ' , ' ) ( ' , ' )

( ' , ' )

''

''

1

1

1

1

1

1

1

11

9

Linear filtering and convolutionExample – convolution with a Gaussian filter

kernel σ determines the width of the filter and hence

the amount of smoothing

g x yx y

g x g y

g xx

( , ) exp(( )

)

( ) ( )

( ) exp( )

2 2

2

2

2

2

2

0.00

0.20

0.40

0.60

0.80

1.00

-6 -4 -2 0 2 4 x

g(x)

σ

Linear filtering and convolution

Original Noisy

Filtered

σ=1.5

Filtered

σ=3.0

Linear filtering and convolutionImageJ demonstration

http://rsb.info.nih.gov/ij/signed-applet

Linear filtering and convolutionWe can also convolution to be a frequency

domain operationBased on the discrete Fourier transform F(u,v)

of the image f(x,y)

F u v f x yj

Nux vy

y

N

x

N

( , ) ( , )exp( ( ))

2

0

1

0

1

u v N, .. 0 1

Linear filtering and convolutionThe inverse DFT is defined by

f x yN

F u vj

Nux vy

y

N

x

N

( , ) ( , )exp( ( ))

1 22

0

1

0

1

x y N, .. 0 1

x

y

f(x,y)

(0.0)

(N-1,N-1)

v

u(0,0)

(N-1,N-1)

F(u,v)

DFT IDFT

log( ( , ) )1 F u v

Linear filtering and convolutionF(u,v) is the frequency content of the image

at spatial frequency position (u,v)Smooth regions of the image contribute low

frequency components to F(u,v) Abrupt transitions in grey level (lines and

edges) contribute high frequency components to F(u,v)

Linear filtering and convolutionWe can compute the DFT directly using the

formulaAn N point DFT would require N2 floating

point multiplications per output point Since there are N2 output points , the

computational complexity of the DFT is N4

N4=4x109 for N=256Bad news! Many hours on a workstation

Linear filtering and convolutionThe FFT algorithm was developed in the 60’s

for seismic explorationReduced the DFT complexity to 2N2log2N

2N2log2N~106 for N=256A few seconds on a workstation

Linear filtering and convolutionThe ‘filtering’ interpretation of convolution

can be understood in terms of the convolution theorem

The convolution of an image f(x,y) with a filter h(x,y) is defined as:

g x y h x y f x x y y

f x y h x yy

M

x

M

( , ) ( ' , ' ) ( ' , ' )

( , )* ( , )''

0

1

0

1

(x,y)

Input image f(x,y) Output image g(x,y)

(x,y)

Filter mask h(x,y)

Linear filtering and convolutionNote that the filter mask is shifted and

inverted prior to the ‘overlap multiply and add’ stage of the convolution

Define the DFT’s of f(x,y),h(x,y), and g(x,y) as F(u,v),H(u,v) and G(u,v)

The convolution theorem states simply that :

G u v H u v F u v( , ) ( , ) ( , )

Linear filtering and convolutionAs an example, suppose h(x,y) corresponds to

a linear filter with frequency response defined as follows:

Removes low frequency components of the image

H u v u v R( , )

0

1

2 2 for

otherwise

DFT

IDFT

Linear filtering and convolutionFrequency domain implementation of

convolutionImage f(x,y) N x N pixelsFilter h(x,y) M x M filter mask pointsUsually M<<NIn this case the filter mask is 'zero-padded'

out to N x NThe output image g(x,y) is of size N+M-1 x

N+M-1 pixels. The filter mask ‘wraps around’ truncating g(x,y) to an N x N image

Filter mask h(x,y)

Input image f(x,y) zero padding

x x

x x

x x

DFT DFT H(u,v) F(u,v)

H(u,v)F(u,v)

IDFT

f(x,y) * h(x,y)

x x x x x

x x x x x

Input image f(x,y) Output image g(x,y)

(x,y)

Filter mask h(x,y)

(x',y')

x' = x modulo N

y' = y modulo N

Linear filtering and convolutionWe can evaluate the computational

complexity of implementing convolution in the spatial and spatial frequency domains

N x N image is to be convolved with an M x M filterSpatial domain convolution requires M 2 floating

point multiplications per output point or N 2 M 2 in total

Frequency domain implementation requires 3x(2N 2 log 2 N) + N 2 floating point multiplications ( 2 DFTs + 1 IDFT + N 2 multiplications of the DFTs)

Linear filtering and convolutionExample 1, N=512, M=7

Spatial domain implementation requires 1.3 x 107 floating point multiplications

Frequency domain implementation requires 1.4 x 107 floating point multiplications

Example 2, N=512, M=32Spatial domain implementation requires 2.7

x 108 floating point multiplications Frequency domain implementation requires

1.4 x 107 floating point multiplications

Linear filtering and convolutionFor smaller mask sizes, spatial and

frequency domain implementations have about the same computational complexity

However, we can speed up frequency domain interpretations by tessellating the image into sub-blocks and filtering these independentlyNot quite that simple – we need to overlap the

filtered sub-blocks to remove blocking artefactsOverlap and add algorithm

Linear filtering and convolutionWe can look at some examples of linear filters

commonly used in image processing and their frequency responsesIn particular we will look at a smoothing filter

and a filter to perform edge detection

Linear filtering and convolutionSmoothing (low pass) filter

Simple arithmetic averagingUseful for smoothing images corrupted by

additive broad band noise

H H3 5

1

9

1 1 1

1 1 1

1 1 1

1

25

1 1 1 1 1

1 1 1 1 1

1 1 1 1 1

1 1 1 1 1

1 1 1 1 1

etc

h x( ) H u( )

Spatial domain Spatial frequency domain

ux

Linear filtering and convolutionEdge detection filter

Simple differencing filter used for enhancing edged

Has a bandpass frequency response

H

1 0 1

1 0 1

1 0 1

Linear filtering and convolutionImageJ demonstration

http://rsb.info.nih.gov/ij/signed-applet

f x( )

p

x

f x( )* ( ) 1 0 1

p

x

Linear filtering and convolutionWe can evaluate the (1D) frequency

response of the filter h(x)={1,0,-1 } from the DFT definition

H u h xjux

Nju

Nju

N

ju

N

ju

N

jju

N

u

N

x

N

( ) ( )exp( )

exp( )

exp( ) exp( ) exp( )

exp( )sin( )

2

14

2 2 2

22 2

0

1

Linear filtering and convolutionThe magnitude of the response is therefore:

This has a bandpass characteristic

H uu

N( ) sin( )

2

H u( )

u

ConclusionWe have looked at basic (low level) image

processing operationsEnhancementFiltering

These are usually important pre-processing steps carried out in computer vision systems (often in hardware)