Lec003 Filtering Convolution

7/28/2019 Lec003 Filtering Convolution

1/18

CSE397/497-011 Real-time Image Processingfor Autonomous Robot Systems

Department of

Computer Science & Engineering

P.C. Rossin

College of Engineering

and Applied Science

Lecture 003:

Image Filtering & Convolution

CSE397/497 Real-time Image Processing

JR Spletzer

Administration

Lab sessions to start on Thursday

If you dont hear from me via e-mail, meet in PL322

Lab example


2/18


JR Spletzer

References Supporting Todays Lecture

Digital Image Processing, R. Gonzalez & R. Woods

HIPR2 Image Processing Learning Resources

http://homepages.inf.ed.ac.uk/rbf/HIPR2/

Introductory Techniques for 3-D Computer Vision, E.

Trucco & A. Verri


JR Spletzer

Images are Discrete Functions!

Digital images are discrete functions that correspond to

the average scene luminance as perceived by the camera

over a period of time

Discrete spatially

Discrete quantization


3/18


JR Spletzer

Images are Discrete Signals!

Signal is visible light (the scene radiance)

Collector is our CCD array

While the signal is continuous, we sample at discrete

intervals (e.g. 30 Hz)

Both spatial and frequency representations

Image formed in the spatial domain

Can obtain frequency info through Discrete Fourier

Transform (DFT) of the image

We will focus on the spatial domain for now


JR Spletzer

Since Images are Signals

They can be corrupted

Subject to random and additive noise (e.g. from

electronics)

Often assumed to be zero-mean Gaussian

Mathematical convenience

Some theoretical basis

2

2

2

)(

22

1)(

=

x

exf


4/18


JR Spletzer

Effects of Gaussian Noise (=3)


JR Spletzer

Also from Impulsive Noise

Aka salt & pepper noise

Less chronic but more acute than Gaussian noise

Causes:

Transmission errors

Faulty CCD elements

External noise in AD conversion

Algorithm artifacts


5/18


JR Spletzer

Effects of Salt & Pepper Noise


JR Spletzer

Noise Filtering

Q: Given a noise corrupted image, how to we go

about attenuating the noise while minimizing the

impact on the true signal ?

A: Signal processing.

We will look at filters for handling both random and

impulsive noise

First, we need to go way back in our minds to

calculus and rediscover


6/18


JR Spletzer

Convolution

Definition:

Q: What is the result of the convolution of 2

functions?

A: Another function.

dtgfgf )()(

= dtgfgfth )()()(


JR Spletzer

So What Exactly does Convolution Do?

For a given value oft

Take the mirror ofg

Shift it by a given value oft

Multiply byf()

Integrate from

Repeat for every value oft from

Since our images are only defined over a finite

region, our range oft will be limited

= dtgfth )()()(

)( g

)( tg

)()( tgf

],[

],[


7/18


JR Spletzer

Sample Convolutions

http://mathworld.wolfram.com/Convolution.html


JR Spletzer

Fun with Gaussians!

The convolution of 2 gaussians function is itself a

gaussian

Convolution can be used to motivate the central limit

theorem

Convolve 2 equally sized rectangle functions

* =


8/18


JR Spletzer

More fun with Gaussians!

Now convolve the rectangle function with the resultof the initial convolution

Is this starting to look familiar?

* =


JR Spletzer

Discrete Convolution

For discrete functions, integration is replaced by a

summation

Discrete convolution can then be defined as:

In our case, the functionfwill correspond to our

image andg the filter kernel that we will use to

suppress noise

=

)()()( tgfth


9/18


JR Spletzer

Kernels are just Discrete Functions

A filter kernel or mask is an n x m array of numbers

It is no different than an image in that it is a discrete

function defined over the n x m array and 0

everywhere else

1-D Example:

Kernel is [-1 0 1]

2551313

1515012

111110

000

000

010

1)1(11)0(11)1(10

=

++=

Im2Im1


JR Spletzer

So What Exactly does Convolution Do?

For a given value oft

Take the mirror ofg

Shift it by a given value oft

Multiply byf()

Integrate from

Repeat for every value oft from

Since our images are only defined over a finite

region, our range oft will be limited

= dtgfth )()()(

)( g

)( tg

)()( tgf

],[

],[

Original Image

Kernel

Resulting

Image


10/18


JR Spletzer

Linear Filter Takes a weighted average of the neighborhood. For a

3x3 we have

Also referred to as the box filter

Noise Filtering: Mean Smoothing

=

111

111

111

9

1k


JR Spletzer

Noise Filtering: Mean Smoothing

In the frequency domain, this corresponds to the sinc

function

This acts as a low-pass filter by weighting frequencies

in the main lobes higher

Because of the secondary lobes, some noise can still

enter the filtered image


11/18


JR Spletzer

Noise Filtering: Gaussian Smoothing

Linear filter, low-pass filter

Based upon Gaussian distributions

Typically use a 5x5 (min and max)

Subtends 98.8% of the area when =1 pixel

Because of discretization, some high frequency noise is

not attenuated


JR Spletzer

Generating Gaussian Kernels

1

1 1

1 2 1

1 3 3 1

1 4 6 4 1

Discrete 1-D kernel coefficients can be generated from

Pascals triangle

2-D coefficients can be obtained from convolving 2 1-D

kernels (horizontal & vertical components)

[ ]

=

121

242

121

16

1

1

2

1

121


12/18


JR Spletzer

3x3 Gaussian

Aka Tent Filter

Reduces high frequencies

more

Weights center sample more

Other samples weighted

linearly

[ ]

=

121242

121

16

1

12

1

121

* Filtering, R. Lin 2004


JR Spletzer

Properties of Convolution

Commutative

Associative

Distributive

Linear

fggf =

hgfhgf = )()(

hfgfhgf +=+ )(

)()()( agfgafgfa ==


13/18


JR Spletzer

Separability of the 2-D Gaussian

12 IGI =

=

=

=

h k

kh

h k

kh

h k

kjhiIee

kjhiIe

kjhiIkhG

),(

),(

),(),(

1

22

12

1

2

2

2

2

2

22


JR Spletzer

Separability of the 2-D Gaussian

This means that rather than convolve the image with

a 2-D n x n Gaussian kernel, we can first convolve it

by a horizontal 1-D gaussian, followed by a 1-D

vertical kernel

Theoretically, computation then will only increase by

a factor of2n vice n2

In practice, we will use the 2-D convolution because

Moving image data to/from memory costs instructions

We typically will use a 5x5 (or even 3x3) kernel


14/18


JR Spletzer

Synthetic Example: Gaussian Smoothing


JR Spletzer

Gaussian Filtering Demo


15/18


JR Spletzer

Noise Filtering: Median Filter

Non-linear Filter useful for impulsive noise

Generate an n x n neighborhood around each pixel in

the original image

Take the median value of this neighborhood as the

value in the new image

n is typically small (3-5 are common)

));:,:(1(),(2

;2

mjmjmimiimmedianjiim

nfloorm

++=

=


JR Spletzer

Noise Filtering: Median Filter

1010111110

1010111110

14142551313

14141515012

1010111110

1010111110

1010111110

14142551313

14141515012

1010111110

(10,11,11,12,13,13,15,150,255)

1010111110

1010111110

14142551313

1414151312

1010111110

Image 1

Image 2


16/18


JR Spletzer

Synthetic Image Example


JR Spletzer

Handling Edge Effects

No perfect solution

Only convolve valid areas

Mirror the borders outside the image

Pad the image border with zeros & adapt kernel

1010111110

1010111110

14142551313

14141515012

1010111110

000000

0

0

0

0

0

1010111110

1010111110

14142551313

14141515012

1010111110


17/18


JR Spletzer

Putting the Pieces Together: Laboratory 1

Construct a vision system for automatic scene

surveillance

Part 1: Establish the Background of the Scene

Part 2: Separate the background from the

foreground

Part 3: Eliminate outliers from the data


JR Spletzer

POP QUIZ : How do we do this???

Part 1:

?

Part 2:

?

?

Part 3:

?


18/18


JR Spletzer

Other Applications of Convolution: Edge Detection

Convolution with an appropriate kernel can yield the

image gradient

Discontinuities in the gradient correspond to edges.

Well talk more about this later


JR Spletzer

Summary

Images are discrete digital signals subjected to additive,

random and impulse noise

We can attenuate this noise through a variety of filters

(mean, gaussian, median, et al)

This is accomplished by convolving the original image with

a discrete kernel generated from the filter

Convolution associativity can be used to speed upprocessing when multiple convolutions are required

(sometimes)

Because we are operating in a discrete world, results are

imperfect

Lec003 Filtering Convolution

Documents

Transcript of Lec003 Filtering Convolution