Mestrado em Ciência de Computadores

Mestrado Integrado em Engenharia de Redes e

Sistemas Informáticos

VC 15/16 – TP7

Spatial Filters

Miguel Tavares Coimbra

VC 15/16 - TP7 - Spatial Filters

Outline

• Spatial filters

• Frequency domain filtering

• Edge detection

Acknowledgements: Most of this course is based on the excellent courses offered by Prof. Shree Nayar at

Columbia University, USA and by Prof. Srinivasa Narasimhan at CMU, USA. Please acknowledge the original

source when reusing these slides for academic purposes.

VC 15/16 - TP7 - Spatial Filters

Topic: Spatial filters

• Spatial filters

• Frequency domain filtering

• Edge detection

VC 15/16 - TP7 - Spatial Filters

Images are Discrete and Finite

yxf , yxg , yxh ,

f

MN

i

j

h

Convolution

M

m

N

n

njmihnmfjig1 1

,,,

Fourier Transform

M

m

N

n

N

nv

M

mui

enmfvuF1 1

2

,,

Inverse Fourier Transform

M

u

N

v

N

lv

M

kui

evuFMN

lkf1 1

2

,1

,

VC 15/16 - TP7 - Spatial Filters

Spatial Mask

• Simple way to

process an image.

• Mask defines the

processing function.

• Corresponds to a

multiplication in

frequency domain. Convolution – Mask

‘slides’ over the image

Mask Image

VC 15/16 - TP7 - Spatial Filters

Example

• Each mask position

has weight w.

• The result of the

operation for each

pixel is given by:

1 2 1

0 0 0

-1 -2 -1

2 2 2

4 4 4

4 5 6

Mask Image

=1*2+2*2+1*2+…

=8+0-20

=-12

a

as

b

bt

tysxftswyxg ),(),(),(

VC 15/16 - TP7 - Spatial Filters

Definitions

• Spatial filters

– Use a mask (kernel) over an image region.

– Work directly with pixels.

– As opposed to: Frequency filters.

• Advantages

– Simple implementation: convolution with the kernel function.

– Different masks offer a large variety of functionalities.

VC 15/16 - TP7 - Spatial Filters

Averaging

Let’s think

about

averaging

pixel values

For n=2, convolve pixel values with 1 2 1

2D images:

1 2 1 (a) use then

1

2

1

1 2 1 or (b) use

1

2

1

1

2

1

1

2

1

2

4

2

Which is faster? 2

1 12 nObnOa

VC 15/16 - TP7 - Spatial Filters

The convolution kernel 2n

8n

nlarge

Repeated averaging Gaussian smoothing

Averaging

VC 15/16 - TP7 - Spatial Filters

Gaussian Smoothing

2

22

2

1

22

1,

ji

ejihGaussian

kernel

N pixels

Filter size N …can be very large

(truncate, if necessary)

1 1

2

1

2,

2

1,

2

22

m n

nm

njmifejig

2D Gaussian is separable!

1 1

2

1

2

1

2,

2

1,

2

2

2

2

m n

nm

njmifeejig

Use two 1D

Gaussian

Filters!

VC 15/16 - TP7 - Spatial Filters

• A Gaussian kernel gives less weight to pixels further from the center of the window

• This kernel is an approximation of a Gaussian function:

Gaussian Smoothing

1 2 1

2 4 2

1 2 1

VC 15/16 - TP7 - Spatial Filters

2

8.2 4

original

VC 15/16 - TP7 - Spatial Filters

Mean Filtering

• We are degrading the

energy of the high spatial

frequencies of an image

(low-pass filtering).

– Makes the image

‘smoother’.

– Used in noise reduction.

• Can be implemented with

spatial masks or in the

frequency domain. 1/9 1/9 1/9

1/9 1/9 1/9

1/9 1/9 1/9

VC 15/16 - TP7 - Spatial Filters Mean filter Gaussian filter

VC 15/16 - TP7 - Spatial Filters

http://www.michaelbach.de/ot/cog_blureffects/index.html

VC 15/16 - TP7 - Spatial Filters

http://www.michaelbach.de/ot/cog_blureffects/index.html

VC 15/16 - TP7 - Spatial Filters

Median Filter

• Smoothing is averaging

(a) Blurs edges

(b) Sensitive to outliers

(a)

(b)

– Sort values around the pixel

– Select middle value (median)

– Non-linear (Cannot be implemented with convolution)

• Median filtering

12 N

sort median

VC 15/16 - TP7 - Spatial Filters

3x3

5x5

7x7

Salt and pepper noise Gaussian noise

VC 15/16 - TP7 - Spatial Filters

Border Problem

What a computer sees

1 2 1

2 4 2

1 2 1

How do we apply

our mask to this

pixel?

VC 15/16 - TP7 - Spatial Filters

Border Problem

• Ignore

– Output image will be smaller than original

• Pad with constant values

– Can introduce substantial 1st order derivative values

• Pad with reflection

– Can introduce substantial 2nd order derivative values

VC 15/16 - TP7 - Spatial Filters

Topic: Frequency domain filtering

• Spatial filters

• Frequency domain filtering

• Edge detection

VC 15/16 - TP7 - Spatial Filters

Image Processing in the Fourier

Domain

Does not look anything like what we have seen

Magnitude of the FT

VC 15/16 - TP7 - Spatial Filters

Convolution in the Frequency Domain

f(x,y)

h(x,y)

g(x,y)

|F(sx,sy)|

|H(sx,sy)|

|G(sx,sy)|

VC 15/16 - TP7 - Spatial Filters

Lets the low frequencies

pass and eliminates the

high frequencies.

Generates image with overall

shading, but not much detail

Low-pass Filtering

VC 15/16 - TP7 - Spatial Filters

Lets through the high

frequencies (the detail),

but eliminates the low

frequencies (the overall

shape). It acts like an

edge enhancer.

High-pass Filtering

VC 15/16 - TP7 - Spatial Filters

Boosting High Frequencies

VC 15/16 - TP7 - Spatial Filters

VC 15/16 - TP7 - Spatial Filters

VC 15/16 - TP7 - Spatial Filters

The Ringing Effect

http://homepages.inf.ed.ac.uk/rbf/HIPR2/freqfilt.htm

An ideal low-pass filter causes ‘rings’

in the spatial domain!

VC 15/16 - TP7 - Spatial Filters

Topic: Edge detection

• Spatial filters

• Frequency domain filtering

• Edge detection

VC 15/16 - TP7 - Spatial Filters

Edge Detection

• Convert a 2D image into a set of curves – Extracts

salient features of the scene

– More compact than pixels

VC 15/16 - TP7 - Spatial Filters

Origin of Edges

• Edges are caused by a variety of factors

depth discontinuity

surface color discontinuity

illumination discontinuity

surface normal discontinuity

VC 15/16 - TP7 - Spatial Filters

How can you tell that a pixel is

on an edge?

VC 15/16 - TP7 - Spatial Filters

Edge Types

Step Edges

Roof Edge Line Edges

VC 15/16 - TP7 - Spatial Filters

Real Edges

Noisy and Discrete!

We want an Edge Operator that produces:

– Edge Magnitude

– Edge Orientation

– High Detection Rate and Good Localization

VC 15/16 - TP7 - Spatial Filters

Gradient

• Gradient equation:

• Represents direction of most rapid change in intensity

• Gradient direction:

• The edge strength is given

by the gradient magnitude

VC 15/16 - TP7 - Spatial Filters

Theory of Edge Detection

1B

2B tx

yIdeal edge

0cossin, yxyxL

0,:1 yxLB

0,:2 yxLB

Unit step function:

0for 0

0for 2

1

0for 1

t

t

t

tu dsstut

Image intensity (brightness):

c o ss in, 121 yxuBBByxI

VC 15/16 - TP7 - Spatial Filters

• Partial derivatives (gradients):

cossincos

cossinsin

12

12

yxBBy

I

yxBBx

I

• Squared gradient:

212

22

cossin,

yxBB

y

I

x

Iyxs

Edge Magnitude:

Edge Orientation:

yxs ,

x

I

y

I/arctan

Rotationally symmetric, non-linear operator

(normal of the edge)

Theory of Edge Detection

VC 15/16 - TP7 - Spatial Filters

• Laplacian:

cossin'

122

2

2

22 yxBB

y

I

x

II

Rotationally symmetric, linear operator

I

x

x

I

2

2

x

I

x

x

zero-crossing

Theory of Edge Detection

VC 15/16 - TP7 - Spatial Filters

Discrete Edge Operators

• How can we differentiate a discrete image?

Finite difference approximations:

1, jiI 1,1 jiI

jiI , jiI ,1

jijijiji IIIIx

I,,11,1,1

2

1

jijijiji IIII

y

I,1,,11,1

2

1

1 1

1 12

1

x

I 1 1

1 12

1

y

I

Convolution masks :

VC 15/16 - TP7 - Spatial Filters

1, jiI 1,1 jiI

jiI , jiI ,1

1,1 jiI

jiI ,1

1,1 jiI 1, jiI 1,1 jiI

• Second order partial derivatives:

jijiji III

x

I,1,,122

2

21

1,,1,22

2

21

jijiji III

y

I

• Laplacian :

2

2

2

22

y

I

x

II

2

2 1

I

Convolution masks :

1 0

4 1

0

1

0 1 0

or 26

1

4 1

20 4

1

4

1 4 1

Discrete Edge Operators

(more accurate)

VC 15/16 - TP7 - Spatial Filters

The Sobel Operators

• Better approximations of the gradients exist

– The Sobel operators below are commonly used

-1 0 1

-2 0 2

-1 0 1

1 2 1

0 0 0

-1 -2 -1

VC 15/16 - TP7 - Spatial Filters

Comparing Edge Operators

-1 0 1

-2 0 2

-1 0 1

1 2 1

0 0 0

-1 -2 1

Gradient:

Roberts (2 x 2):

Sobel (3 x 3):

Sobel (5 x 5): -1 -2 0 2 1

-2 -3 0 3 2

-3 -5 0 5 3

-2 -3 0 3 2

-1 -2 0 2 1

1 2 3 2 1

2 3 5 3 2

0 0 0 0 0

-2 -3 -5 -3 -2

-1 -2 -3 -2 -1

0 1

-1 0

1 0

0 -1

Good Localization

Noise Sensitive

Poor Detection

Poor Localization

Less Noise Sensitive

Good Detection

VC 15/16 - TP7 - Spatial Filters

Effects of Noise

• Consider a single row or column of the image

– Plotting intensity as a function of position gives a signal

Where is

the edge??

VC 15/16 - TP7 - Spatial Filters Where is the edge?

Solution: Smooth First

Look for peaks in

VC 15/16 - TP7 - Spatial Filters

Derivative Theorem of Convolution

…saves us one operation.

VC 15/16 - TP7 - Spatial Filters

Laplacian of Gaussian (LoG)

Laplacian of Gaussian operator

Where is the edge? Zero-crossings of bottom graph !

fhx

fhx

2

2

2

2

Laplacian of Gaussian

VC 15/16 - TP7 - Spatial Filters

2D Gaussian Edge Operators

Laplacian of Gaussian Gaussian

Derivative of Gaussian (DoG)

Mexican Hat (Sombrero)

• is the Laplacian operator:

VC 15/16 - TP7 - Spatial Filters

Canny Edge Operator

• Smooth image I with 2D Gaussian:

• Find local edge normal directions for each pixel

• Compute edge magnitudes

• Locate edges by finding zero-crossings along the edge normal

directions (non-maximum suppression)

IG

IG

n

IG

0

2

2

n

IG

IG

VC 15/16 - TP7 - Spatial Filters

Non-maximum Suppression

• Check if pixel is local maximum along gradient direction

– requires checking interpolated pixels p and r

VC 15/16 - TP7 - Spatial Filters original image

VC 15/16 - TP7 - Spatial Filters magnitude of the gradient

VC 15/16 - TP7 - Spatial Filters

After non-maximum suppression

VC 15/16 - TP7 - Spatial Filters

Canny Edge Operator

Canny with Canny with original

• The choice of depends on desired behavior

– large detects large scale edges

– small detects fine features

VC 15/16 - TP7 - Spatial Filters

Difference of Gaussians (DoG)

• Laplacian of Gaussian can be approximated by the

difference between two different Gaussians

VC 15/16 - TP7 - Spatial Filters

DoG Edge Detection

1(a) 2(b) (b)-(a)

VC 15/16 - TP7 - Spatial Filters

Unsharp Masking

200 400 600 800

100

200

300

400

500

– =

= + a

VC 15/16 - TP7 - Spatial Filters

Resources

• Gonzalez & Woods – Chapter 3