Mask Operation

11

Linear systems and linear filtering

Smoothing operations

di il iMedian Filtering

Sharpening operationsSharpening operations

Derivative operations

Correlation22

Linear systems & Linear Filteringy gAny process that accepts an (signal) image ‘I’ as input and transform it by an act of linear convolution is called a lineartransform it by an act of linear convolution is called a linear system

Output (filtered)Digital Image Filter,

characterized by i l

Digital Image IOutput (filtered) Image J = I*H

impulse response H

Goals of linear filtering is to improve image quality (by someGoals of linear filtering is to improve image quality (by some criteria) by

Enhancing certain featuresEnhancing certain featuresDe-emphasizing or eradicating certain features

33

Specific GoalsSpecific Goals

Smoothing – remove noise from bit errors, transmission, thermal noise

Enhancement – enhance image quality or emphasize significant features, such as edgesg g

D bl i i h f bl d iDeblurring – increase sharpness of blurred images

44

Characterizing Linear SystemsCharacterizing Linear SystemsAny linear digital image processing system can be characterized in one of two equivalent ways:characterized in one of two equivalent ways:

by system unit pulse responseby system frequency response

Unit pulse response p p

(1) the response of the system to the unit pulse

(2) It is an effective way to model a system’s response to an image, because any image is a sum of weighted unit pulses

55

F RFrequency Response

(1) d ib tl h th t ff t h f(1) describes exactly how the system effects each frequency component in an image that is passed through a system

(2) Note that an unit pulse has all frequency components

(3) Image frequency components are amplified or attenuated and phase-shiftedattenuated and phase shifted

66

Image Enhancement: Spatial Filtering

Image enhancement in the spatial domain can be represented age e a ce e t t e spat a do a ca be ep ese ted

as: )()( nmfTnmg ),(),( nmfTnmg

Enhanced Image Transformation Given ImageEnhanced Image Transformation g

The transformation T maybe linear or nonlinear. We willThe transformation T maybe linear or nonlinear. We will mainly study linear operators T but will see one important nonlinear operation.

There are two closely related concepts that must be understood when performing linear spatial filtering. One is correlation, the 7p g p g ,other is convolution.

7

How to specify TIf the operator T is linear and shift invariant (LSI), characterized by the point-spread sequence (PSS) h(m, n), y p p q ( ) ( , ),then (recall convolution)

nmfnmhnmg ),(),(),(

l k

lkflnkmh

fg

),(),(

),(),(),(

l k

lkhlnkmf ),(),( l k

In practice, to reduce computations, h( n , m ) is of “finite extent:”

where is a small set (called neighborhood). is also called as h( n , m ) =0, for (k,l)

8the support of h.

8

But Equ.(3.5-1) in text book is given

nmfnmhnmg ),(),(),(

a

al

b

bklmkmfnmh

fg

),(),(

)()()(

al bk

The above expression is called correlation.

Mathematically, convolution is the same process, except that h is rotated by 180 degree.

When h(m,n) is symmetric convolution is equal to correlation

99

1010

In the frequency domain, this can be represented as:q y p

G(u, v ) He (u, v) Fe (u, v )h H ( ) d F ( ) bt i d ft i twhere are He (u, v) and Fe (u, v ) obtained after appropriate

zero-padding.

Many LSI operations can be interpreted in the frequency domain as a “filtering operation.” It has the effect of filtering frequency components (passing certain frequency components and stopping others).

The term filtering is generally associated with such operations.p

1111

Examples of some common filters (1-D case):Examples of some common filters (1 D case):

Lowpass filter Highpass filter12p 12

Lowpass filterLowpass filter

(1) fil ll b h “l ” f i(1) filter attenuates all but the “lower” frequencies(2) smooth noise(3) blur image details to emphasize gross features

1313

Bandpass filter

(1) attenuates all but an intermediate range of “middle” frequencies(2)usually highly specialized filters

Highpass filterHighpass filter

(1)attenuates all but the “higher” frequencies(2)enhance image details & contrast

(3) remove image blur 14(3) remove image blur 14

If h(m,n) is a 3 by 3 mask given by

w1 w3w2

h yf(-1,-1) f(-1, 0) f(-1, 1)

f( 0 1) f( 0 0) f( 0 1)w4 w6w5

w7 w9w8

h =(x,y)

y

x

f( 0,-1) f( 0, 0) f( 0, 1)

f( 1,-1) f( 1, 0) f( 1, 1)

( 0 0)

),( nmg

x(m=0,n=0)

)1()()1()1,1(),1()1,1( 321

nmfwnmfwnmfwnmfwnmfwnmfw

)1,1(),1()1,1()1,(),()1,(

987

654

nmfwnmfwnmfwnmfwnmfwnmfw

1515

The output g(m, n) is computed by sliding the mask over p g( ) p y geach pixel of the image f(m, n). This filtering procedure is sometimes referred to as moving average filter.

Special care is required for the pixels at the border of image f(m n) This depends on the so-called boundary conditionf(m, n). This depends on the so called boundary condition. Common choices are:

(1) The mask is truncated at the border (free boundary)(1) The mask is truncated at the border (free boundary).

For one dimension

0

)()(~ xf

xfNx 0

01)2/( xL 1)2/( LNxN

MATLAB option is ‘P’1616

(2)The image is extended by appending extra rows/columns at the boundaries The extension is done by repeating the first/lastboundaries. The extension is done by repeating the first/last row/column or by setting them to some constant (fixed boundary).

For one dimensionFor one dimension

)()0(

)(~xf

fxf

01)2/( xL

Nx 0

)1()()(

Nfxfxf

1)2/( LNxNNx 0

MATLAB option is ‘replicate’

(3)The boundaries “wrap around” (periodic boundary).

MATLAB option is replicate

)d)(( NNf 01)2/( L

For one dimension

)mod(

)()mod)((

)(~

Nxfxf

NNxfxf

1)2/( LNxN

01)2/( xL

Nx 0

17 )(f

MATLAB option is ‘symmetric’

17

In any case, the final output g(m, n) is restricted to the support y p g( ) ppof the original image f(m, n).The mask operation can be implemented in Matlab using the imfilter command, which is based on the conv2command.

1818

Smoothing Filters

Image smoothing refers to any image-to-image transformation designed to “smooth” or flatten the image by reducing the rapiddesigned to “smooth” or flatten the image by reducing the rapid pixel-to-pixel variation in gray values.

Smoothing filters are used for:

(1) Blurring: This is usually a preprocessing step for removing(1) Blurring: This is usually a preprocessing step for removing small (unwanted) details before extracting the relevant (large) object, bridging gaps in lines/curves,

(2)Noise reduction: Mitigate the effect of noise by linear or nonlinear operations.p

Image smoothing by averaging (lowpass spatial filtering)1919

Smoothing is accomplished by applying an averaging mask.Smoothing is accomplished by applying an averaging mask.

An averaging mask is a mask with positive weights, which sum to 1 It computes a weighted average of the pixel valuessum to 1. It computes a weighted average of the pixel values in a neighborhood. This operation is sometimes called neighborhood averaging.

Some 3 x 3 averaging masks:

111010

1

141010

1

111111

1

3163131

1

010

1115

010

1418

111

1119

131

32

This operation is equivalent to lowpass filtering. 20

Why?20

Example: Consider a spatial filtering operation which averages the four p g p gnearest neighbors of pixels with a 3X3 window.

The output y(m,n) is give e ou pu y( , ) s g ve

)1,()1,(),1(),1(41),( nmxnmxnmxnmxnmy

010

1

x(m,n) is the input of image.

010101

41

2121

Example of Image Blurring

11

1

NN

N

11

12

Average maskNN

Original image N=5Original image N=5

22

N=9

22

Example of noise reduction

Noise free image I ith iNoise free image Image with zero mean gaussian noise, variance=0.01

23

Noise reduced image23

Separable filters

Some filter can be implemented by the successive application of two simple filters For example:

1 1 1 11 1 1

two simple filters. For example:

1 1 11 1 1 1 1 1 19 3 3

1 1 1 1

The above 2-D average filter is thus separable into two 1-D small filters.

Advantages: reduce multiplications and additions

2-D filter: 2n Multiplications; 2 1n additions

Two 1 D filters: 2n Multiplications; 2 2n additions 24Two 1-D filters: 2n Multiplications; 2 2n additions 24

Median Filtering

The averaging filter is best suited for noise whose distribution

is Gaussian:)

2exp(

21)( 2

2

xxpnoise

22

The averaging filter typically blurs edges and sharp details.

The median filter usually does a better job of preserving edges.

Median filter is particularly suited if the noise pattern exhibits strong (positive and negative) spikes. Example: salt and pepper noise.

2525

Median filter is a nonlinear filter, that also uses a mask.Median filter is a nonlinear filter, that also uses a mask. Each pixel is replaced by the median of the pixel values in a neighborhood of the given pixel.

Suppose A {a1, a2,…, aK } are the pixel values in a

neighborhood of a given pixel with a1 a2 … aK . Then

2/Ka for K even

2/)1(

2/)(K

K

aa

Amedianfor K odd

for K even

Note: Median of a set of values is the “center value,” after sorting.g

For example: If A {0,1,2,4,6,6,10,12,15}, then median(A) = 6 26median(A) = 6. 26

Example of noise reduction

Noise free image Salt & pepper :prob=0.02

27

Output of 3 by 3 averaging27

Image Sharpening This involves highlighting fine details or enhancing details that have been blurredhave been blurred.

Basic highpass spatial filtering

This can be accomplished by a linear shift-invariant operator, implemented by means of a mask, with positive and negativeimplemented by means of a mask, with positive and negative coefficients.

Thi i ll d h i k i it t d t hThis is called a sharpening mask, since it tends to enhance abrupt gray level changes in the image.

2828

The mask should have a positive coefficient at the center and ti ffi i t t th i h Th ffi i t h ldnegative coefficients at the periphery. The coefficients should

sum to zero. Example:

181111

91

1119

This is equivalent to highpass filtering Why?This is equivalent to highpass filtering.

A highpass filtered image g can be thought of as the difference

Why?

A highpass filtered image g can be thought of as the difference between the original image f and a lowpass filtered version of f :

g(m n) = f(m n) – lowpass(f(m n))g(m,n) = f(m,n) – lowpass(f(m,n))

2929

Example:

3030

High-boost filteringg g

This is a filter whose output g is produced by subtracting a lowpass (blurred) version of f from an amplified version of flowpass (blurred) version of f from an amplified version of f

g(m,n) = A f(m,n) – lowpass(f(m,n))

This is also referred to as unsharp masking.Observe that

g(m,n) = A f(m,n) – lowpass(f(m,n))= (A-1) f(m,n) + f(m,n) – lowpass(f(m,n))( ) ( ) ( ) p ( ( ))= (A-1) f(m,n) + highpass(f(m,n))

For A 1 part of the original image is added back to theFor A 1, part of the original image is added back to the highpass filtered version of f.

3131

The result is the original image with the edges enhanced relative to th i i l ithe original image.

Example:

3232

3333

Derivative filter

Averaging tends to blur details in an image. Averaging involves summation or integrationsummation or integration.

Naturally, differentiation or “differencing” would tend to enhance abrupt changes, i.e., sharpen edges.Foundation

)()1( xfxff

)1()( xfxff

1-D case

First derivative: )()1( xfxfx

)()()1(

ffx

2 fff

First derivative:

Second derivative:

))1()(())()1(()1(2

xfxfxfxfxxx

Second derivative:

34)(2)1()1( xfxfxf 34

)()1( xfxfxf

First derivatives produces thicker edges in an imagex

2 f

Second derivatives have a stronger response to fine details

)(2)1()1(2 xfxfxfxf

3535

2-D case ),(),1(),( yxfyxfx

yxf

),()1,(),( yxfyxf

yyxf

x y

)(2)1()1(2

yxfyxfyxff

),(2),1(),1(2 yxfyxfyxfx

)(2)1()1(2

yxfyxfyxff

),(2)1,()1,(2 yxfyxfyxfy

)(4)1()1()1()1(22

2 yxfyxyyxfyxfyxffff

),(4)1,()1,(),1(),1(22 yxfyxyyxfyxfyxfyx

f

This can implemented using the masks:

141010

or

141010

The resulting marks are called Laplacian operators

010 010

36The resulting marks are called Laplacian operators. 36

Example:Example:

3737

Gradient

Most common differentiation operator is the gradient.

Gradient

xyxf

f

),(

)(

yyxf

xyxf ),(),(

3838

The magnitude of the gradient is:g g2/122 ),(),(),(

y

yxfx

yxfyxf

Discrete approximations to the magnitude of the gradient is ll d C id h f ll i i i

yx

normally used. Consider the following image region:

z1 z3z2 )()1( xfxff

yz4 z6z5

z7 z9z8

)()( ffxy

x7 98

We may use the approximation

2/1265

285 )()(),( zzzzyxf

3939

This can be implemented using the masks:

1

11h and 112 h

As follows:

2/122 )()()( hfhfyxf 21 )()(),( hfhfyxf

Alternatively, we may use the approximation:

2/1286

295 )()(),( zzzzyxf

U i h kUsing the masks:

and

01

1h

10

1hand

As follows:

101

011

40

2/122

21 )()(),( hfhfyxf

40

The resulting marks are called Roberts cross-gradient g goperators.

Better approximations to the gradient can be obtained by:Better approximations to the gradient can be obtained by:

2/12741963

2321987 ))()(())()((),( zzzzzzzzzzzzyxf

This can be implemented using the masks:

101

111000111

1h

101101101

2hand 111 101

As follows:

2/12

21 )()(),( hfhfyxf

The resulting masks are called Prewitt operators 41The resulting masks are called Prewitt operators. 41

Another approximation is given by the masks:

000121

1h and

202101

2h

121

1 d

101

2022h

The resulting masks are called Sobel operatorsThe resulting masks are called Sobel operators.

The Roberts operators and the Prewitt/Sobel operators are used for edge detection and are sometimes called edge detectors.

4242

Example:

4343

Laplacian enhanced not only fine details but also noiseLaplacian enhanced not only fine details but also noise

This noise is most objectionable in smooth areas, where it 44

tends to be more visible44

Sobel enhanced edges, but less on fine details. However, the noise is not as enhanced as in LaplacianN i i th S b l t d i b f th d d b 45Noise in the Sobel-operated image can be further reduced by

smoothing it with an averaging filter45

Comments

A single technique is not suitable for solving a real-life problem

The approach just described is representative of the types of processes that can be linked in order to achieve results that are not possible with a single techniquenot possible with a single technique

The way, in which the processes (image processing tools) areThe way, in which the processes (image processing tools) are used, depends on the application

4646

Correlation• Correlation is similar to convolution

1 1*1 M N

where: f* is the conjugate of f As we usually deal with

*

0 0

1( , ) ( , ) ( , ) ( , )m n

f x y h x y f m n h x m y nMN

where: f is the conjugate of f. As we usually deal with real images, f*=f

• Correlation Theorem:• Correlation Theorem:*( , ) ( , ) ( , ) ( , )f x y h x y F u v H u v

• Autocorrelation Theorem2( ) ( ) | ( ) |f f F 2( , ) ( , ) | ( , ) |f x y f x y F u v

4747

Application: pattern matching

• We want to identify the location of the letter ‘T’

• We need the ‘template’ of ‘T’ to do pattern matching

• We can find the cross-correlation between the two images

• Obviously, at the location of T, the cross correlation will be

imaximum• Matlab command: corr2

i• Image size = 256 x 256• Template size = 38 x 42

4848

Zero-paddingp g

• Before convolving, we need to zero-pad the image to prevent g, p g paliasing

4949

Cross-correlation result

• It is easy to do cross-correlation in frequency domains e sy o do c oss co e o eque cy do• Highest cross-correlation occurs, when the template is

exactly over the letter ‘T’y

5050