Digital Image Analysis and ProcessingCPE 0907544CPE 0907544

Image Enhancement –Part IISpatial Filtering

Chapter 3S ti 3 4 3 7

D I d J f

Sections : 3.4-3.7

Dr. Iyad Jafar

OutlineIntroduction

Mechanics of Spatial Filtering

Correlation and Convolution

Linear Spatial FilteringLinear Spatial Filtering

Spatial filters for Smoothing

Spatial filters for Sharpening

S Nonlinear Spatial Filtering

Combining Spatial Enhancement Techniques Combining Spatial Enhancement Techniques 2

BackgroundFiltering is borrowed from the frequency domainprocessing and refers to the process of passing orprocessing and refers to the process of passing orrejecting certain frequency components

Highpass lowpass band-reject and bandpassHighpass, lowpass, band-reject , and bandpassfilters

Filtering is achieved in the frequency domain bydesigning the proper filter (Chapter 4)g g p p ( p )

Filtering can be done in the spatial domain also byusing filter masks (kernels, templates, or windows)

Unlike frequency domain filters, spatial filters canbe nonlinear !

3

Spatial Filtering MechanicsA spatial filter is characterized by

A rectangular neighborhood of size mxn (usually m and n are odd)A rectangular neighborhood of size mxn (usually m and n are odd)A predefined operation that is specified by the mask values at each position. Origin y

w(-1,-1) w(-1,0) w(-1,1)

y

w(0,-1) w(0,0) w(0,1)

w(1,-1) w(1,0) w(1,1) (x, y)

Neighbourhood

3x3 filter mask example

x Image f (x, y)

Spatial filtering OperationThe filter mask is centered at each pixel in the image and the outputpixel value is computed based on the operation specified by the

g f y

pixel value is computed based on the operation specified by themask4

Spatial Filtering MechanicsO i iOrigin

y

x3x3 Filter

w(-1,-1) w(-1,0) w(-1,1) f(x-1,y-1) f(x-1,y) f(x-1,y+1)

xImage f (x, y) xw(0,-1) w(0,0) w(0,1) f(x,y-1) f(x,y) f(x,y+1)

Filter Original Image Pixels

w(1,-1) w(1,0) w(1,1) f(x+1,y-1) f(x+1,y) f(x+1,y+11)

a b

g( x y ) w( s t ) f ( x s y t )= + +∑∑ ∑5 s a t b

g( x, y ) w( s,t ) f ( x s, y t )=− =−

= + +∑∑ ∑

Vector Representation of S ti l Filt iSpatial Filtering

The previous filtering equation can be written as The previous filtering equation can be written as

w1 w2 w31 1 2 2 mn mnR w z w z ... w z= + + +

w4 w5 w6

w7 w8 w9 1

mn

k kk

= w z ∑

If we represent the coefficients of the filter mask as a

w7 w8 w9 1k=

row vector w = [w1 w2 w3 w4 w5 w6 w7 w8 w9] and the image pixels under the mask by z = [z1 z2 z3 z4 z5

] h h f l b z6 z7 z8 z9], then the filtering equation can be written as

TR wz=6

R wz=

Spatial CorrelationThe filtering operation just described is calledcorrelationIn 1-D

f wOrigin

0 0 0 1 0 0 0 0 1 2 3 2 81) Function and mask

1 2 3 2 8

0 0 0 1 0 0 0 02) Initial Alignment

1 2 3 2 8

0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 03) Zero Padding for f by m-1

1 2 3 2 8

0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0

7 1 2 3 2 8

0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 04) Position after one shift

Spatial Correlation

1 2 3 2 8

0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 05) Position after 4 shifts

1 2 3 2 8

.6) Repeat .

..

1 2 3 2 8

0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0Final Position

0 0 0 8 2 3 2 1 0 0 0 0

1 2 3 2 8

Full correlation result 0 0 0 8 2 3 2 1 0 0 0 0

8 Cropped correlation result

0 8 2 3 2 1 0 0

Spatial Convolution

A strongly related operation to correlation isconvolution (an important tool in linear systemstheory)

The mechanics of convolution is similar to thoseThe mechanics of convolution is similar to thoseof correlation, except that the filter mask isrotated by 180o before slidingrotated by 180 before sliding

9

Spatial Convolution

0 0 0 1 0 0 0 0

f wOrigin

1 2 3 2 81) Function and mask 0 0 0 1 0 0 0 0

0 0 0 1 0 0 0 0

1 2 3 2 81) Function and mask

8 2 3 2 12) Initial Alignment

after rotation ofmask

8 2 3 2 1

0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 03) Zero Padding for f by m-1

0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 04) Position after one shift

8 2 3 2 14) Position after one shift

10

Spatial Convolution

8 2 3 2 1

0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 05) Position after 4 shifts

8 2 3 2 1

.6) Repeat .

..

8 2 3 2 1

0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0Final Position

0 0 0 1 2 3 2 8 0 0 0 0

8 2 3 2 1

Full convolution result 0 0 0 1 2 3 2 8 0 0 0 0

11 Cropped convolutionresult

0 1 2 3 2 8 0 0

Extension to 2DExtension to 2D is straight forward

In convolution, the mask is flipped vertically and , pp yhorizontally Zero padding is done in both directions by m-1 and n-1

0 0 0 0 0 0 0 0 00 0 0 0 0 0 0 0 0

0 0 0 0 00 0 0 0 0 Zero padding by m 1 and n 1 0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 00 0 0 0 0 0 0 0 0

0 0 0 0 00 0 1 0 00 0 0 0 0

f

Zero padding by m-1 and n-1

0 0 0 0 1 0 0 0 00 0 0 0 0 0 0 0 0

0 0 0 0 0

1 2 3 9 8 7 0 0 0 0 0 0 0 0 00 0 0 0 0 0 0 0 00 0 0 0 0 0 0 0 0

1 2 34 5 67 8 9

9 8 76 5 43 2 1

w Mask rotation

12

0 0 0 0 0 0 0 0 0

Extension to 2D

9 8 7 0 0 0 0 0 06 5 4 0 0 0 0 0 0

1 2 3 0 0 0 0 0 04 5 6 0 0 0 0 0 0

3 2 1 0 0 0 0 0 00 0 0 0 0 0 0 0 0

7 8 9 0 0 0 0 0 00 0 0 0 0 0 0 0 0

0 0 0 0 1 0 0 0 00 0 0 0 0 0 0 0 00 0 0 0 0 0 0 0 0

0 0 0 0 1 0 0 0 00 0 0 0 0 0 0 0 00 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 00 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 00 0 0 0 0 0 0 0 00 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

Initial position in convolution

0 0 0 0 0 0 0 0 0

Initial position in correlation

13

Extension to 2D

0 0 0 0 0 0 0 0 00 0 0 0 0 0 0 0 00 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 00 0 0 0 0 0 0 0 00 0 0 0 0 0 0 0 00 0 0 0 0 0 0 0 0

0 0 0 9 8 7 0 0 00 0 0 6 5 4 0 0 0

0 0 0 0 0 0 0 0 00 0 0 1 2 3 0 0 00 0 0 4 5 6 0 0 00 0 0 6 5 4 0 0 0

0 0 0 3 2 1 0 0 00 0 0 0 0 0 0 0 0

0 0 0 4 5 6 0 0 00 0 0 7 8 9 0 0 00 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 00 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 00 0 0 0 0 0 0 0 0

Full convolution resultFull correlation result

14

Extension to 2D0 0 0 0 00 1 2 3 0

0 0 0 0 00 1 2 3 00 4 5 6 00 7 8 9 0

0 9 8 7 00 6 5 4 00 3 2 1 0

Cropped convolution result

0 7 8 9 00 0 0 0 0

Cropped correlation result

0 3 2 1 00 0 0 0 0

ppCropped correlation result

a b

Correaltion

( ) ( ) f ( )∑∑a b

Convolution

( ) ( ) f ( )∑∑s a t b

g( x, y ) w( s,t ) f ( x s, y t )=− =−

= + +∑∑s a t b

g( x, y ) w( s,t ) f ( x s, y t )=− =−

= − −∑∑

15

Treatment of Pixels at Edges

In the previous slides, we padded the image with zerosin both directions in order to compensate forunavailable values. x

Other approachesReplicate edge pixels

e e

Consider only available pixels that fall under the e

mask in the computation of the new values

e

Allow pixels to wrap around ee e

16y

Image f (x, y)

Treatment of Pixels at EdgesThe result depends on the used approachE l 3 6Example 3.6

Filtered Image: Z P ddiZero Padding

OriginalImage

Filtered Image: Replicate Edge PixelsImage Replicate Edge Pixels

Filtered Image: Wrap Around Edge Pixels

17

Wrap Around Edge Pixels

Smoothing Spatial FiltersA smoothing (averaging, blurring) filter replaces each i l i h h l f ll i l d h pixel with the average value of all pixels under the

mask

Uses of smoothing filtersThe smoothed image correspond to the gross details of the image or the low frequency content of the image C b d t d i it i h t i d ith Can be used to reduce noise as it is characterized with sharp transitions

However, smoothing usually result in degradation of ed e alit (sha p ess of the i a e)edge quality (sharpness of the image)

18

Smoothing Spatial FiltersCommon smoothing masks

1 1 1 1 2 1

1 1 1

1 1 1

X 1/9 2 4 2

1 2 1

X 1/16

Standard averaging mask Weighted average mask

NotesThe weighted average filter gives more weight to pixels near the centercenterIt is hard to see the visual difference between the processingresults of the two filters; however, the weighted average filtersummation is 16, which make it more attractive for computers

19

Smoothing Spatial FiltersExample: 3.7 Smoothing with different mask sizes. Notice the loss of details as the mask size increases

Original Smoothing by 3x3 Mask Smoothing by 5x5 Mask

20Smoothing by 9x9 Mask Smoothing by 15x15 Mask Smoothing by 35x35 Mask

Smoothing Spatial FiltersExample 3.8: smoothing highlights gross details. C ld b f l i idi b tt t ti Could be useful in providing better segmentation results

Original Image Smoothed Image Thresholded Image

21

Smoothing Spatial FiltersExample 3.9: Noise Reduction

22

Original Smoothed

Sharpening Spatial FiltersThe principle objective of sharpening is to highlighttransitions in intensity which usually correspond totransitions in intensity which usually correspond toedges in images; thus sharpening is the opposite ofsmoothingsmoothing

If we examine the smoothing operation we can thinkf it i t tiof it as integration

Thus to perform sharpening in the spatial domain, it isintuitive to use differentiation

In the following few slides we examine theIn the following few slides we examine thefundamental properties of the first and secondderivative when applied on different intensitypp ytransitions

23

Sharpening Spatial FiltersLets discuss differentiation for 1-D caseWe are concerned about the behavior of 1st and 2nd

derivatives in the following areas Constant intensityOnset and end of discontinuities (ramps and steps discontinuities)

Intensity ramps

Properties of 1st derivativeZero in areas of constant intensityNonzero at the onset of a step and intensity rampNonzero along intensity ramp

P 2 d dProperties 2nd derivativeZero in areas of constant intensityN t th t d d f t d i t it Nonzero at the onset and end of a step and intensity rampZero along intensity ramp 24

Sharpening Spatial FiltersDerivatives can be approximated as differences

1st derivative at x

1f f ( ) f ( )∂ 1f f ( x ) f ( x )x= + −

∂

2nd derivative at x

2

2 1 1 2f f ( x ) f ( x ) f ( x )∂= + + − −

∂25

2x∂

Sharpening Spatial FiltersInvestigation of derivatives behavior

26

Sharpening Spatial FiltersNotes

Examining the 1st and 2nd derivatives plots shows that allof their properties are satisfied

1st derivative produce thicker edges than 2nd derivatives

2nd derivative produce double edge separated by a zerocrossingcrossing

2nd derivative is commonly used in sharpening since it2nd derivative is commonly used in sharpening since ithas simpler implementation and finer edges

27

Sharpening Using 2nd

D i tiDerivativeThe second derivative (Laplacian) in 2-D is defined as

2 22

2 2f ff ∂ ∂

∇ = +

If we define

2 2fx y∂ ∂

If we define 2

2 1 1 2f f ( x , y ) f ( x , y ) f ( x, y )x

∂= + + − −

∂

2

2 1 1 2f f ( x, y ) f ( x, y ) f ( x, y )y

∂= + + − −

∂

Then the discrete second derivative can be approximated by

x∂ y∂

approximated by

2f∇ =1 1 1 1 4f ( x , y ) f ( x , y ) f ( x, y ) f ( x, y ) f ( x, y )+ + − + + + − −

28

Sharpening Using 2nd

D i tiDerivativeThe Laplacian can be implemented as a filter mask

0 1 0

1 -4 1

0 1 0

OrOr

1 1 1

1 -8 1

1 1 1

29

Sharpening Using 2nd

D i tiDerivativeComputing the Laplacian doesn’t produce a sharpenedp g p p pimage. However, grayish edge lines and discontinuitiessuperimposed on a dark background

It is common practice to scale the Laplacian image to [0,255] for better display30

Sharpening Using 2nd

D i tiDerivativeAlternatively, to obtain a sharpened image g(x,y), y p g g( y)subtract the Laplacian image from the original image

2( ) f ( ) f ( )2g( x, y ) f ( x, y ) f ( x, y )= − ∇

- =

Original Laplacian Sharpened

31

OriginalImage

LaplacianFiltered Image

SharpenedImage

Sharpening Using 2nd

D i iDerivativeThe two steps required to achieve sharpening can beThe two steps required to achieve sharpening can be

combined into a single filtering operation

ff 2)()( ∇)1()1([)( yxfyxfyxf −++−=

fyxfyxg 2),(),( ∇−=),1(),1([),( yxfyxfyxf −++−=)1,()1,( −+++ yxfyxf )()( yfyf

)],(4 yxf−),1(),1(),(5 yxfyxfyxf −−+−=)1()1( + ff )1,()1,( −−+− yxfyxf

32

Sharpening Using 2nd

DerivativeAs a filter mask the previous equation can be represented As a filter mask, the previous equation can be represented

as0 -1 0-1 5 -10 -1 0

33

Sharpening Using 1st DerivativeThe first derivative in image processing is implemented

as a gradient which is defined as a vectorg

⎥⎤

⎢⎡∂∂

⎤⎡f

G

⎥⎥⎥⎥

⎦⎢⎢⎢⎢

⎣∂∂∂=⎥

⎦

⎤⎢⎣

⎡=∇

yfx

GG

y

xf

The gradient points in the direction of greatest rate ofh l i ( )

⎥⎦⎢⎣∂y

change at location (x,y)The magnitude of the gradient is defined as

Or, approximately1

2 22f fM( x, y )⎡ ⎤⎛ ⎞∂ ∂⎛ ⎞⎢ ⎥= + ⎜ ⎟⎜ ⎟⎢ ⎥⎝ ⎠

34

( , y )x y⎜ ⎟⎜ ⎟∂ ∂⎢ ⎥⎝ ⎠ ⎝ ⎠⎣ ⎦ | | x yf G G∇ ≈ +

Sharpening Using 1st DerivativeComputation of the gradient using Roberts

cross-gradient operatorsg pRoberts operators for computing Gx and Gy are shown

below. The upper-left pixel in the operator is overlaid overthe pixel (x,y) or z5 in the original image

1 0 0 1Z1 Z2 Z3 -1 0

0 1

0 -1

1 0

Z1 Z2 Z3

Z4 Z5 Z6

Z7 Z8 Z9

[ ]2

Horizontal OperatorZ7 Z8 Z9

Pixel z5 and its neighboursVertical Operator

[ ][ ]

29 5 9 5

28 6 8 6

x

y

G ( x, y ) z z z z

G ( x, y ) z z z z

= − ≈ −

= − ≈ −

35

[ ]8 6 8 6

9 5 8 6

y( , y )

M( x, y ) z z z z≈ − + −

Sharpening Using 1st DerivativeComputation of the gradient using Sobel

operatorspSobel operators for computing Gx and Gy are shown

below. The upper-left pixel in the operator is overlaid overthe pixel (x,y) or z5

-1 0 1-1 -2 -1Z1 Z2 Z3 1 0 1

-2 0 2

1 0 1

1 2 1

0 0 0

1 2 1

Z1 Z2 Z3

Z4 Z5 Z6

Z7 Z8 Z9 -1 0 11 2 1

Mask to Compute Gx

Z7 Z8 Z9

Pixel z5 and its neighbours Mask to Compute Gy

7 8 9 1 2 3

3 6 9 1 4 7

2 2

2 2x

y

G ( x, y ) ( z z z ) ( z z z )

G ( x, y ) ( z z z ) ( z z z )

≈ + + − + +

= + + − + +

36

y

x yM( x, y ) G ( x, y ) G ( x, y )= +

Sharpening Using 1st DerivativeExample 3.10

Gradient is widely used in industrial inspection as ity pproduce thicker edges in the result, which make it easierfor machines to detect artifacts

37Optical Image for a contact lens Gradient Image

Nonlinear (Order-statistic) S ti l FiltSpatial Filters

Order-statistic filters are nonlinear filters whose response is based on ordering the pixels under the mask and then replacing the centre pixel with the value determined by the ranking result

Examples

Z1 Z2 Z3

Z4 Z5 Z6 Rankpminimum filtermaximum filter

Z7 Z8 Z9

median filter popular and useful in removing impulse or salt-and-pepper noise

New value

with less blurring than linear smoothing38

Nonlinear (Order-statistic) S ti l FiltSpatial Filters

Example 3.11p

Original Image Image After Image After

Note how the median filter has reduced the noise

With Noise Averaging Filter Median Filter

significantly with less blurring than smoothing filter39

Combining Spatial hEnhancements

It is very common to combine It is very common to combine different enhancement techniques in order to achieve techniques in order to achieve the desired goal

Example 3.12Example 3.12We want to enhance/sharpen

the bones in this imagegDirect application of the

Laplacian may result in noise amplification

40

Combining Spatial hEnhancements

Compare to enhancement by single method

Processed by a combination of

Sharpened by Laplacian

Histogram Equalization

Power-Law with gamma = 0 5

41

combination of methods

Laplacian Equalization gamma = 0.5

Combining Spatial E h tEnhancement

Enhancement by multiple methods

(a)

Laplacian filter of bone scan (a) (b)

Sharpened version of bone scan achieved by subtracting (a)

(c)y g ( )

and (b) Sobel filter of bone scan (a) (d)42

Combining Spatial E h tEnhancement

Result of applying a power law trans to

(h)

Sharpened image which is sum of (a) and (f)

power-law trans. to (g)

(g)The product of (c) and (e) which will be used as a mask

and (f)(f)

(e)

Image (d) smoothed witha 5*5 averaging filter

43

Combining Spatial E h tEnhancement

Compare the original and final imagesp g g

44

Readings

Read section 3 6 3 titled “Unsharp Masking Read section 3.6.3 titled Unsharp Masking and Highboost filtering”.

45

Related Matlab Functions

Check Matlab documentation for the following functions

2conv2

corr2

fillter2

imfilter

padarraypadarray

median

medfilt246