Dr. Iyad Jafar

Digital Image Analysis and ProcessingCPE 0907544

Image Restoration

Chapter 5

Sections : 5.1 – 5.8

Outline

Introduction Degradation/Restoration Model Noise Models Restoration in the Presence of Noise only –

Spatial Filtering Mean Filters Order-statistic Filters Adaptive Filters

Periodic Noise Reduction Using Frequency Filtering

Estimation of Degradation Function 2

Introduction The principle goal of image restoration is similar to

that of image enhancement in obtaining an improvedimage according to some criterion

Image enhancement is subjective in the sense that theused techniques are heuristic and are applied to takeadvantage of the psychophysical aspects of the humanvisual system

Image restoration is objective in the sense that thedegradation introduced into the image is known basedon a prior knowledge of the degradation phenomena

Restoration can be performed in spatial or frequencydomains. Spatial treatment is applicable only when thedegradation is additive noise

3

A Degradation /Restoration Model

The primary objective of restoration is to obtain anestimate of the original image based on the knowledge ofthe degradation and noise functions

The degraded image g(x,y) can be modeled as

4

g( x, y ) h( x, y ) f ( x, y ) η( x, y )

Or

G( u,v ) H( u,v )F( u,v ) N( u,v )

The degradation model is expressed as

The operator H is said to be linear if

The homogeneity property of H

The Operator H is said to be position-invariant if

This definition implies that the response at any point in the image depends only on the value at that point, regardless of its position

A Degradation /Restoration Model

5

g( x, y ) H f ( x, y ) η( x, y )

1 2 1 2H f ( x, y ) f ( x, y ) H f ( x, y ) H f ( x, y )

H af ( x , y ) aH f ( x , y )

H f ( x α , y β ) g ( x α , y β )

Noise Models Noise is introduced into a digital image during image

acquisition and/or transmission

Noise introduced during acquisition is dependant onthe quality of sensing elements and the environmentalconditions (Light levels and temperature)

Image corruption during transmission is primarily dueto channel interference (lightning and atmosphericdisturbances)

In our discussion, we assume that the noiseintroduced into the image is a random variable that isuncorrelated with the spatial coordinates and valuesof the pixels

6

Noise Models Gaussian Noise

z0 is the mean valueσ2 is the variance

It is useful in characterizingnoise of electronic circuits and sensors

7

2 2201

2( z z ) / σp( z ) e

πσ

Noise Models Rayleigh Noise

The mean is

The variance

It is useful in characterizing noise in range imaging It is useful in approximating skewed histograms 8

22

0

( z a ) / b ( z a )e , z ap( z ) b

, z

Noise Models Erlang (Gamma) Noise

The mean is

The variance

a>0 and b is a positive integer

It is useful in characterizing noise in laser imaging

9

1

1

0

b baza z e , z 0

p( z ) ( b )!

, z

Noise Models Exponential Noise

The mean is

The variance

It is useful in characterizing noise in laser imaging 10

0

az ae , z 0p( z )

, z

Noise Models Uniform Noise

The mean is

The variance

It is the least descriptive of all types 11

1

10

, a z bp( z ) b

, otherwise

0 2

b az

22

12

( b a )σ

Noise Models Impulse (salt-and-pepper) Noise

Usually represented as black and white dots in the image

It appears in situations with quick transitions such as faulty switching

12

a

b

P , z=a

p( z ) P , z=b

0 , otherwise

Noise Models Example 5.1.

13

OriginalG

au

ssia

n

Ra

yle

igh

Ga

mm

a

Noise Models Example 5.2.

14

Original

Un

iform

Exp

one

ntia

l

Sa

lt-a

nd-P

epp

er

Noise Models Periodic Noise

Periodic noise is usually added to the image fromelectrical and electromechanical interference duringimage acquisition

It is the only type of spatially dependent noise that wewill discuss

Periodic noise can be dealt with using frequency domainfiltering

15

Noise Models Estimating Noise Parameters

Periodic noise parameters can be estimated by Infer periodicity of noise components from the image

(difficult) Inspection of the Fourier transform of the image since

periodic noise tend to produce spikes that can be easilydetected visually

The parameters of noise PDFs may be Known partially from sensor specifications Estimated from acquired images by examining the

histograms of small patches of reasonably constantbackground intensity.

The approximate shape of the histogram determines thetype of noise and we can compute the mean and varianceby

16

12 2

0

L

s i

i

σ ( zi z ) p ( z )

1

0

L

s i

i

z zi p ( z )

Noise Models Example 5.3. Estimating Noise Parameters

Histograms of reasonably constant intensity regionextracted from images corrupted by Gaussian, Rayleigh,and uniform noise

17

Gaussian Rayleigh Uniform

Restoration in the Presence of Noise Only

If noise is the only degradation present, the degradationmodel becomes

Restoration can be simply done by subtraction if thenoise values are known!

Use spatial filtering when only additive noise is present In what follows, we explore new types of spatial filters.

Filtering using the new filters is done in the same waydiscussed in Chapter 3

18

g( x, y ) f ( x, y ) η( x, y )

Or

G( u,v ) F( u,v ) N( u,v )

Restoration in the Presence of Noise Only

Mean Filters For a rectangular subimage window Sxy of size mxn that is centered at

pixel (x,y), we define the following mean filters

Arithmetic Mean Filter

It’s capable of reducing noise levels, but it smoothes local variationsin an image

Geometric Mean Filter

Achieves similar smoothing results as the arithmetic mean filter, buttends to lose less details in the image19

1^

( s,t ) Sxy

f ( x, y ) g( s,t )mn

1

m n^

( s ,t ) S xy

f ( x , y ) g ( s ,t )

Restoration in the Presence of Noise Only Mean Filters

Harmonic Mean Filter

The harmonic mean filter works well for salt noise, but fails forpepper noise. It does well also for other types of noise.

Contraharmonic Mean Filter Q is the order of the filter

Well suited for reducing or virtually

eliminating salt-and-pepper noise.

Positive Q eliminates pepper noise while

negative Q eliminates salt noise

It is important to select the proper sign of the filter20

1

^

( s,t ) Sxy

mnf ( x, y )

g( s,t )

1Q

^ ( s ,t ) S x y

Q

( s ,t ) S x y

g ( s ,t )

f ( x , y )g ( s ,t )

Restoration in the Presence of Noise Only

Example 5.4. Mean Filters

21

Original

Corrupted by Gaussian

NoiseMean = 0

Variance = 400

Filtered by 3x3

Arithmetic Mean Filter

Filtered by 3x3

Geometric Mean Filter

Restoration in the Presence of Noise Only

Example 5.5. Mean Filters

22

Image corrupted

with pepper noise

Filtered by 3x3 Contraharmonic

Mean FilterQ = 1.5

Image corrupted with salt

noise

Filtered by 3x3 Contraharmonic

Mean FilterQ = -1.5

Restoration in the Presence of Noise Only

Example 5.6. Mean Filters Choosing wrong sign for the contraharmonic filter

23

Image with pepper noise after filtering with

contraharmonic Mean FilterQ = -1.5

Image with salt noise after filtering with contraharmonic

Mean FilterQ = +1.5

Restoration in the Presence of Noise Only

Order-Statistic FiltersOrder-statistic filters are spatial filters whose response is basedon the ordering of pixels’ values under the filter mask Median Filter

Replaces the value of the pixel by the median of the intensitylevels in the neighborhood of the pixel

It’s capable of reducing noise levels with considerably lessblurring than linear smoothing filters

It’s particularly effective on unipolar or bipolar impulsivenoise

24

^

( s,t ) Sxyf ( x, y ) median g( s,t )

Restoration in the Presence of Noise Only

Order-Statistic Filters Maximum Filter

Replaces the value of the pixel by the maximum of theintensity levels in the neighborhood of the pixel

It is useful in finding the brightest points in the image and inreducing pepper noise

Minimum Filter Replaces the value of the pixel by the minimum of the

intensity levels in the neighborhood of the pixel

It is useful in finding the darkest points in the image and inreducing salt noise25

^

( s,t ) Sxyf ( x,y ) maximum g( s,t )

^

( s,t ) Sxyf ( x, y ) minimum g( s,t )

Restoration in the Presence of Noise Only

Order-Statistic Filters Midpoint Filter

It works best for randomly distributed noise such asGaussian and uniform noise

Alpha-Trimmed Mean Filter Compute the mean intensity of the trimmed values for the

pixels in the neighborhood. Trimming is performed bydeleting d/2 lowest and d/2 highest intensity values

It is useful in situations involving multiple types of noise suchas a combination of impulse and Gaussian noise26

12

^

( s,t ) S ( s,t ) Sxy xyf ( x, y ) maximum g( s,t ) minimum g( s,t )

1^r

( s,t ) Sxy

f ( x, y ) g ( s,t )mn d

Restoration in the Presence of Noise Only

Example 5.7. Order-Statistic Filters Recursive median filtering

Recursive median filtering is perfect for impulse noise. However, recursion filtering may lead to blurring 27

Image corrupted

with pepper noise

3x3 Median filtering 2nd pass

3x3 Median filtering1st pass

3x3 Median filtering 3rd pass

Restoration in the Presence of Noise Only

Example 5.8. Order-Statistic Filters Min and Max Filtering

28

Image corrupted

with pepper noise

Image corrupted with white

noise

Image filtered

with 3x3 min filter

Image filtered

with 3x3 max filter

Restoration in the Presence of Noise Only

Example 5.9. Order-Statistic Filters

29

Image corrupted with uniform and salt-and-pepper noise

Filtered with 5x5 Arithmetic Mean Filter

Filtered with 5x5 Geometric Mean Filter

Filtered with 5x5 Median Filter

Filtered with 5x5 Alpha-trimmed mean Filter (d=5)

Restoration in the Presence of Noise Only

Adaptive Filters The filters discussed so far are applied to the image

without considering the fact that image characteristics mayvary from one location to another

In what follow, we discuss a new class of filters calledadaptive filters

In such filters, the filtering operation is modified from onelocation to another based on some local measures, such asthe mean and variance of the pixel neighborhood

Adaptive filters are usually much better but at the expenseof increased filter complexity

30

Adaptive , Local Noise Reduction Filter The mean intensity and variance are two statistical

measures that are widely used because of their closelyrelated relation with the appearance of the image

The mean gives the average intensity in the region whilevariance is a measure of contrast

The filter under discussion uses four different values toperform filtering in a certain neighborhood

g(x,y) : the value of the pixel (x,y) in the noisy image

: the variance of the noise corrupting f(x,y) : the local variance for the pixels in Sxy mL : the local mean for the pixels in Sxy

2ησ

Restoration in the Presence of Noise Only

31

2Lσ

Adaptive , Local Noise Reduction Filter Assumptions of Operation

If is zero, the filter should return the value of g(x,y). This is thetrivial case in which there is no noise in the image

If the local variance is high relative to , the filter should return avalue close to g(x,y) since high local variance is usually associatedwith edges and these have to be preserved

If the two variances are equal, the filter should return the meanvalue of the pixels in Sxy . This situation correspond to the casewhen the local area has similar properties as the overall image, andlocal noise is to be removed by averaging.

Based on these assumptions, we can define this adaptive filter as

Restoration in the Presence of Noise Only

32

2ησ

2

2

^ ηL

L

σf ( x, y ) g( x, y ) ( g( x,y ) m )

σ

2ησ

Example 5.10. Adaptive , Local Noise Reduction Filter

Restoration in the Presence of Noise Only

33

Image corrupted

with Gaussian

noise (mean = 0, variance

1000)

Filtered with 7x7

Arithmetic Mean Filter

Filtered with 7x7

Geometric Mean Filter

Filtered with 7x7 Adaptive

Filter

Adaptive Median Filter

PAGE 332 in the textbook

Restoration in the Presence of Noise Only

34

Periodic noise appears as concentrated bursts of energyin the frequency domain at locations corresponding tothe periodic frequencies

Thus, periodic noise can be analyzed and filteredeffectively using frequency domain filtering

Use selective filtering Band pass/reject filters Notch pass/reject filters

Periodic Noise Reduction By Frequency Domain Filtering

35

Bandreject Filters One principle application of bandreject filters is in removing noise

components whose locations are approximately known

A good example is an image that is corrupted by periodic noisethat can be approximated as two-dimensional sinusoidal functions

We know that the Fourier transform of a sine consists of twoimpulses that are mirror images of each other about the origin ofthe transform

We may use ideal, Butterworth, or Gaussian bandreject filters

Periodic Noise Reduction By Frequency Domain Filtering

36

Example 5.11. Bandreject Filters

Periodic Noise Reduction By Frequency Domain Filtering

37

Image corrupted with Sinusoidal Noise

Magnitude of Spectrum

Butterworth Bandreject Filter Filtered image

Notch Filters Notch filters perform filtering by rejecting frequencies in a

predefined neighborhoods about the center of the frequency rectangle

Due to the symmetry of the Fourier transform, notch filters should appear symmetric about the origin

We may use ideal, Butterworth, or Gaussian notch filters

Notch filters can be of arbitrary shape

Periodic Noise Reduction By Frequency Domain Filtering

38

Example 5.12. Notch-reject Filters

Periodic Noise Reduction By Frequency Domain Filtering

39

Image Corrupted by Sinusoidal Noise Magnitude of Spectrum

Notch-reject Filter along the vertical axis

Filtered image

Estimation by Image Observation Assume that a degraded image is available without any

knowledge about the degradation function, which is assumed tobe linear, position-invariant

One way to estimate the degradation function is to gatherinformation from the image itself (like a blurred image)

For example, we can look at small patch of the image where thenoise is minimum and then process the patch to arrive atpleasant result

From the degraded image and the processed image (theestimate of original) we compute

Then we can generalize to find H(u,v) with the samecharacteristics of Hs(u,v)

Estimation of Degradation Function

40

patchpatch ^G ( u ,v )

H ( u ,v )

F( u ,v )

Estimation by Experimentation It is possible if the imaging system used to acquire the

degraded image is available Imaging is repeated with different system settings until

almost the same degraded image is obtained Using such settings, the system is used to image an impulse

(a small dot of light that is as bright as possible) to obtainthe impulse response of the degradation

In the frequency domain, we can compute the degradationfunction H(u,v) from the degraded image by

where A is the impulse strength

Estimation of Degradation Function

41

G( u,v )H( u,v )

A

Estimation by Experimentation

Estimation of Degradation Function

42

An impulse of light (magnified)

Imaged impulse withdegradation

G( u,v )H( u,v )

A

Estimation by Modeling It is based on finding a mathematical expression for the

degradation function Finding such mathematical expression can be through

some assumptions such as physical characteristics of theenvironment during the imaging process

deriving the mathematical model from basic principles For example, a model proposed to model atmospheric

turbulence is

where k is a constant that depends on the nature of theturbulence with higher values indicating higher turbulenceeffects

Estimation of Degradation Function

43

2 2 5 6/k( u v )H( u,v ) e

Estimation by Modeling Estimation of blurring degradation based on linear

uniform motion Assume that the image function undergoes a planer motion and

that x0(t) and y0(t) are the time-dependant components ofmotion in the x and y directions

The total exposure at any point of the recording medium isobtained by integrating the instantaneous exposure over thetime interval of imaging event

If T is the duration of exposure , it follows that

Estimation of Degradation Function

44

0 0

0

T

g( x, y ) f ( x x ( t ), y y ( t ))dt

Estimation by Modeling Estimation of blurring degradation based on linear

uniform motion The Fourier transform of g(x,y) is given by

Reversing the order of integration

Estimation of Degradation Function

45

2

20 0

0

j π( ux vy )

Tj π( ux vy )

G( u,v ) g( x, y )e dxdy

= f x x ( t ), y y ( t ) dt e dxdy

20 00

Tj π( ux vy )G( u,v ) = f x x ( t ), y y ( t ) e dxdy dt

Estimation by Modeling Estimation of blurring degradation based on linear

uniform motion The term inside the outer brackets from the previous slide is

the Fourier transform of the displaced function f[x-x0(t),y-y0(t)]

If x0(t) and y0(t) are known, the degradation function can bedirectly found

Estimation of Degradation Function

46

0

0

Tj2π(ux (t)+vy (t))- 0 0

Tj2π(ux (t)+vy (t))- 0 0

G( u,v ) = F( u,v ) e dt

= F(u,v) e dt

= F(u,v)H(u,v)

Example 5.13. Estimation by Modeling Estimation of blurring degradation based on linear

uniform motion Let the image undergoes a uniform motion in the x direction

only with x0(t) = at/T, then the degradation function is given by

If we also have a uniform motion in the y direction also by y0(t) = bt/T, then the degradation function is

Estimation of Degradation Function

47

jπuaTH ( u ,v ) = sin( πua)eπua

jπ ( ua vb )TH ( u ,v ) = sin( π ( ua bv ))eπ ( ua vb )

Example 5.13. Continued Estimation of blurring degradation based on linear

uniform motion

Example of blurring by the function on the previous slide with a = b = 0.1 and T = 1

Estimation of Degradation Function

48

Related Matlab Functions

imfilter

medfilt2

lpfilter

brfilter

nrfilter

generateNoiseImage

spatialFiltering

49