1

1

Chapter 5Image Restoration

Chapter 5Image Restoration

Objectives:To improve a given image in some predefined senseHow MATLAB and IPT models degradation phenomena and formulate restoration solutions.

Image restoration is an objective process while image enhancement is subjective.

2

A Model of the Image Degradation/Restoration Process

The degradation process is modeled as a degradation function that together with an additive noise term, operates on an input image f(x,y) to produce a degraded image g(x,y):

Given g(x,y), some knowledge about the degradation function H, and some knowledge about the additive noise term , the objective of restoration is to obtain an estimate, , of the original image as close as possible to the original input image.

),()],([),( yxyxfHyxg η+=

),( yxη),(ˆ yxf

2

3

Reverse engineering process

4

A Model of the Image Degradation/Restoration ProcessIf H is a linear, spatially invariant process, it can be shown that the degraded image is given in the spatial domain by:

Where h(x,y) is the spatial representation of the degradation function and “*” indicates convolution. We can also write its frequency domain equivalent as:

Where all terms in capital letter refer to Fourier transform of corresponding terms.The degradation function H(u,v) is called the optical transform function (OTF), and the h(x,y) is called the point spread function. MATLAB provides conversion functions:otf2psf and psf2otf.

),(),(),(),( yxyxfyxhyxg η+∗=

),(),(),(),( vuNvuFvuHvuG +=

3

5

A Model of the Image Degradation/Restoration ProcessBecause the degradation due to a linear, spatially invariant degradation function, H, can be modeled as convolution, sometimes the degradation process is referred to as “convolving the image with a PSF or OTF”. Similarly the restoration process is sometimes referred to as deconvolution.

We first deal with cases where H is assumed to be an identity operator, so it does not have any effect on the process. This way we deal only with degradation noise. Later we get H involve too.

6

Noise ModelsAbility to simulate the behavior and effects of noise is crucial to image restoration.We will look at two noise models: • Noise in the spatial domain (described by the noise probability density function), and• Noise in the frequency domain, described by various Fourier properties of the noise.

MATLAB uses the function imnoise to corrupt an image with noise. This function has the basic syntax:

g = imnoise(f, type, parameters)

Where f is the input image, and type and parameters will be described later.

4

7

Adding Noise with Function imnoiseFunction imnoise converts the input image to class doublein the range [0, 1] before adding noise to it.

This is very important to remember. For example, to add Gaussian noise of mean 64 and variance 400 to a uint8image, we scale the mean to 64/255 and the variance to 400/2552 for input into imnoise. Below is a summary of the syntax for this function (see Page 143 for more information)

g = imnoise(f, ‘gaussian’, m, var)g = imnoise(f, ‘localvar’, V)g = imnoise(f, ‘localvar’, image_intensity, var)g = imnoise(f, ‘salt & pepper’, d)g = imnoise(f, ‘speckle’, var)g = imnoise(f, ‘poisson’)

8

Example:On the left the original image. On the right:g = imnoise(f, 'gaussian', 127/255, 0);

Note that I have set the variance to 0, that is practically un-realistic. But for the sake of this example, we will let that go.

5

9

Example:On the left: g = imnoise(f, 'gaussian', 54/255, 200);On the right: g = imnoise(f, 'gaussian', 0, 200);

10

Generating Spatial Random Noise with a Specified DistributionOften, we wish to generate noise of types and parameters other than the ones listed on previous page. In such cases, spatial noise are generated using random numbers, characterized by probability density function (PDF) or by the corresponding cumulative distribution function (CDF).

Two of the functions used in MATLAB to generate random numbers are:rand – to generate uniform random numbers, andrandn – to generate normal (Gaussian) random numbers.

6

11

Example:Assume that we have a random number generator that generates numbers, w, uniformly in the [0 1] range. Suppose we wish to generate random numbers with a Rayleigh CDF, which is defined as:

<≥−=

−−

azazezF

baz

z for 0for 1)(

/)( 2

To find z we solve the equation:

OrSince the square root term is nonnegative, we are assured that no values of z less than a are generated.In MATALB: R = a + sqrt(b*log(1 – rand(M, N) ));

we baz =− −− /)(2

1)1ln( wbaz −+=

12

Function imnoise2 generates random numbers having CDFs shown in the table below. They all use rand as: rand(M,N)

7

13

imnoise vs. imnoise2The imnoise function generates a 1D noise array, imnoise2will generate an M-by-N noise array.imnoise scales the noise array to [0 1], imnoise2 does not scale at all and will produce the noise pattern itself. The user specifies the parameters directly.Note: The salt-and-pepper noise has three values:

0 corresponding to pepper noise1 corresponding to salt noise, and0.5 corresponding to no noise.

You can copy the imnoise2 function from the course web page.

14

r = imnoise2('gaussian', 100000, 1, 0, 1);p = hist(r, 50);bar(p)

0 10 20 30 40 50 600

1000

2000

3000

4000

5000

6000

7000

8000

8

15

r = imnoise2('gaussian', 256, 256, 0, 255);p = hist(r, 50);bar(p)

16

0 10 20 30 40 50 600

500

1000

1500

2000

2500

r = imnoise2(‘uniform', 100000, 1, 0, 1);p = hist(r, 50);bar(p)

9

17

0 10 20 30 40 50 600

1000

2000

3000

4000

5000

6000

7000

8000

9000

r = imnoise2(‘lognormal', 100000, 1, 1, 0.25);p = hist(r, 50);bar(p)

18

0 10 20 30 40 50 600

1000

2000

3000

4000

5000

6000

r = imnoise2(‘rayleigh', 100000, 1, 1, 0.25);p = hist(r, 50);bar(p)

10

19

0 10 20 30 40 50 600

0.5

1

1.5

2

2.5x 10

4

r = imnoise2(‘exponential', 100000, 1, 1, 1);p = hist(r, 50);bar(p) Does not matter

20

0 10 20 30 40 50 600

1000

2000

3000

4000

5000

6000

7000

8000

9000

r = imnoise2(‘erlang', 100000, 1, 2, 5);p = hist(r, 50);bar(p)

11

21

0 10 20 30 40 50 600

1

2

3

4

5

6

7

8

9

10x 10

4

r = imnoise2(‘salt & pepper', 100000, 1, 0.05, 0.05);p = hist(r, 50);bar(p) If r(x,y) = 0, black

If r(x,y) = 1, whiteIf r(x,y) = 0.5 none

22

Periodic NoiseThis kind of noise usually arises from electrical and/or electromagnetical interference during image acquisition. These kinds of noise are spatially dependent noise. The periodic noise is handled in an image by filtering in the frequency domain.Our model for a periodic noise is:

Where A is amplitude, u0 and v0 determine the sinusoidal frequencies with respect to the x- and y-axis, respectively, and Bx and By are phase displacements with respect to the origin. The M-by-N DFT of the equation is:

]/)(2/)(2[( ),( 00 NByvMBxuSinAyxr yx +++= ππ

)],()(),()[(2

),( 00/2

00/2 00 vvuuevvuueAjvuR NBvjMBuj yx −−−++= δδ ππ

12

23

This statement shows a pair of complex conjugate impulses located at (u+u0 , v+v0) and (u-u0 , v-v0), respectively.A MATLAB function (imnoise3) accepts an arbitrary number of impulse locations (frequency coordinates), each with its own amplitude, frequencies, and phase displacement parameters, and computes r(x, y) as the sum of sinusoids of the form shown in the previous page.The function also outputs the Fourier transform of the sum of sinusoides, R(u,v) and the spectrum of R(u,v). The sine waves are generated from the given impulse location information via the inverse DFT.Only one pair of coordinates is required to define the location of an impulse.The program generates the conjugate symmetric impulses.

24

C = [0 64; 0 128; 32 32; 64 0; 128 0; -32 32];[r, R, S] = imnoise3(512, 512, C);imshow(S, []);

C is k-by-2 matrix containing K pairs of frequency domain coordinates (u,v)r is the noise pattern of size M-by-N

R is the Fourier transform of r

S is the spectrum of R

13

25

u

v

[0,0] [32,0][-32,0]

26

figure, imshow(r, [])

14

27

C = [0 32; 0 64; 16 16; 32 0; 64 0; -16 16];[r, R, S] = imnoise3(512, 512, C);imshow(S, []);

C = [0 64; 0 128; 32 32; 64 0; 128 0; -32 32];

28

figure, imshow(r, [])

15

29

C = [6 32; -2 2];[r, R, S] = imnoise3(512, 512, C);imshow(S, []);

30

figure, imshow(r, [])

16

31

C = [6 32; -2 2];A = [1 5];[r, R, S] = imnoise3(512, 512, C, A);imshow(S, []);

32

figure, imshow(r, [])

17

33

Estimating Noise ParameterThe parameters of periodic noise typically are estimated by analyzing the Fourier spectrum of the image. Periodic noise tends to produce frequency spikes that often can be detected even by visual inspection.In the case of noise in the spatial domain, the parameters of the PDF may be known partially from sensor specifications. However, it is often necessary to estimate them from sample images. In general, the relationships between the mean, m, and variance, , of the noise, and the parameters a and brequired to completely specify the noise PDFs of interest (see Table 5.1).Problem: Estimating mean and variance from the sample image(s) and then using them to solve for a and b.

σ

34

Estimating Noise Parameter – cont.Let zi be a discrete random variable that denotes intensity levels in an image. Note that a random number generator usually produces numbers in the range [0 1]. You need to multiply that with the Max intensity value to get the intensity. Assume p(zi), I = 0, 1, 2, …, L-1, be the corresponding normalized histogram, where L is the number of possible intensity values. The central moments (moments around the mean) is defined as:

Where n is the moment order, and m is the mean:

Note that histogram is normalized, so sum of all p’s is 1.

∑−

=

−=1

0

)()(L

ii

nin zpmzµ

∑−

=

=1

0

)(L

iii zpzm

18

35

Estimating Noise Parameter – cont.From the first equation on the previous page, we can determine that = 1, and = 0. Do you know why?and:

Is the variance. We only go this far up (second component).

Function statmoments computes the mean and central moments up to order n, and returns them in row vector v.statmoments ignores these two moments and instead lets v(1) = m and v(k) = for k = 2,3, …, n.

∑−

=

−=1

0

22 )()(

L

iii zpmzµ

0µ 1µ

kµ

36

Example:Consider this 4x4 image and compute the first three central moments.

41020108440

10104102010020

∑−

=

−=1

0

)()(L

ii

nin zpmzµ

19

37

Example:Consider this 4x4 image and computed the first three central moments.

m = 9p0 = 2/16 , p4 = 4/16 , p8 = 1/16, p10 = 6/16, p20 = 3/16

41020108440

10104102010020

∑−

=

−=1

0

)()(L

ii

nin zpmzµ

10 =µ

01 =µ

=2µ

38

Estimating Noise Parameter – cont.In MATLAB: [u , unv] = statmoments(p, n)Where p is the histogram vector and n is the number of moments to compute. p must be 2q for unitq images.Output vector u contains the normalized moments based on values of the random variable that have been scaled to the range [0, 1]. All the moments are also in the same range. Vector unv contains the same moments as v, but computed with the data in its original range of values.

Example: If length(p) = 256 and v(1) = 0.5, then unv(1) would have the value 127.5, which is half of the range [0 255].

20

39

Estimating Noise Parameter – cont.Sometimes the noise parameter must be estimated directly from a given noisy image or set of images. In such cases we select part of the image that is as featureless as possible to emphasize the primary noise as much as possible. To select region of interest (ROI) in MATLAB, we can use roipoly function, which generates a polygonal ROI:

B = roipoly(f, c, r)Where f is the image of interest, and c and r are vectors of corresponding column and row coordinate of the vertices of the polygon.B is a binary image the same size as f with 0’s outside the region of interest and 1’s inside. It is used as a mask to limit operations to within the region of interest.

40

Estimating Noise Parameter – cont.We can also set the ROI interactively:

B = roipoly(f)Which displays the image f on the screen and allows the user specify the polygon using the mouse.Please see the help on this function to learn about other ways we can run it.

21

41

Example: Original Image

42

[B, c, r] = roipoly(f)

22

43

Histogram of the ROI

X = imnoise2(‘gaussian’, npix, 1, 147, 20)

So the noise seem to look like a Gaussian. So the best estimate for this noise is Gaussian.

Histogram of the image

44

Example:

23

45

[B, c, r] = roipoly(f);

Used the mouse toselect this area

46

c =

12.000013.000059.000012.0000

r =

184222219184

Y-X coordinates of the polygon (ROI)

npix =

880

Total number of pixels in that area:

24

470 50 100 150 200 250 3000

100

200

300

400

500

600

700

[p, npix] = histroi(f, c, r);

figure, bar(p, 1);

Histogram of the ROI.Is there a standard noise model that produces noise like this.

48

Restoration in the presence of Noise Only – Spatial FilteringIn this case our model will become (only degradation is noise):The method of choice for reduction of noise in this case is spatial filtering. The linear filters are implemented using the imfilter. The median, max, and min filters are non-linear, order-statistic filters. The median filter can be implemented using medfilt2. The max and min filters are implemented using ordfilt2. The spfilt function performs filtering in the spatial domain with any of the filters listed on the next page. A function called imlincomb computes the linear combination of the inputs.

),(),(),( yxyxfyxg η+=

25

49

50

Example:Given g let’s find the max filtered image of g.Assuming a 3-by-3 mask. We need to pad theimage first.

=

8228824664646482

g

=

000000082280082460064640064820000000

g Step 1:

=

640820000

m

Max is 8 in this 3x3 mask, so we represent the center pixel 2 with 8.

26

51

Step 2:Move right

=

464482000

m

Max is 8 in this 3x3 mask, so we represent the center pixel 8 with 8.

Step 3:Move right

=

646648000

m

Max is 8 in this 3x3 mask, so we represent the center pixel 4 with 8.

Step 4:Move right

=

004064000

m

Max is 6 in this 3x3 mask, so we represent the center pixel 6 with 6.

Step 5:DownLeft most

=

460640820

m

Max is 8 in this 3x3 mask, so we represent the center pixel 4 with 8.

52

We continue this until all elements are processed.

>> f = spfilt(g, 'max', 3, 3)

f =

8 8 8 68 8 8 88 8 8 88 8 8 8

Check the font colors in previous pages to see where everything has come from.

27

53

Example:[M, N] = size(f);R = imnoise2(‘salt & pepper’, M, N, 0.1, 0);c = find(R == 0);gp = f;gp(c) = 0;

Image corrupted with pepper noisewith probability 0.1

Pepper: Probability of 0.1

54

Image corrupted with salt noise with probability 0.1

Example:[M, N] = size(f);R = imnoise2(‘salt & pepper’, M, N, 0 0.1);c = find(R == 1);gs = f;gs(c) = 255;

Salt: Probability of 0.1

28

55

One good way to filter the pepper noise is to use a contraharmonic filter with a positive value of Q. fp = spfilt(gp, ‘chmean’, 3, 3, 1.5);Why?

∑

∑

∈

∈

+

=

xy

xy

Sts

Q

Sts

Q

tsg

tsgyxf

),(

),(

1

),(

),(),(ˆ

Try with Q = 2 for:

=

41266104482

g

Apply mask g over the entire image

56

Similarly, to filter the salt noise we can use a contraharmonic filter with a negative value of Q. fp = spfilt(gs, ‘chmean’, 3, 3, -1.5);Why?

∑

∑

∈

∈

+

=

xy

xy

Sts

Q

Sts

Q

tsg

tsgyxf

),(

),(

1

),(

),(),(ˆ

Try with Q = -2 for:

=

41266104482

g

29

57

Similar results can be obtained with the min or maxfilters.

fmax = spfilt(gp, ‘max’, 3, 3); fsmin = spfilt(gs, ‘min’, 3, 3);

Filter size

58

ExampleApply the min and max filter on the following image data that is corrupted with salt & pepper. Note that this is a 3 bit data, so the highest intensity (salt white) is 7.

=

0717110450623340

g

What is the percentage of salt, assuming there was no pure whitepixel before?What is the percentage of pepper, assuming there was no pure black pixel before?

30

59

Adaptive Spatial FiltersThe filters discussed in the previous few pages are applied to an image without regard for how image characteristics vary from one location to another.

In some cases, the result can be improved if the filters are capable of adapting their behavior depending on the characteristics of the image in the area that is being filtered.

One example of such filters is the adaptive median filter. As before let Sxy denotes a subimage centered at location (x, y) in the image being processed.

60

Algorithm for Adaptive Median FilterLet zmin = minimum intensity value in Sxy

zmax = maximum intensity value in Sxyzmed = median intensity value in Sxyzxy = intensity value at coordinate (x,y)

Algorithm is implemented in 2 levels:Level A: If zmin < zmed < zmax, go to level B

Else increase the window size (ODD)If window size Smax, repeat level AElse output zmed

Level B: If zmin < zxy < zmax, output zxyElse output zmed

Where Smax is the max allowed size of the adaptive filter window

≤

31

61

Example:Given g let’s find the max filtered image of g. Assuming our mask S3x3. We assume a Smax 3x3 mask. Replicate pad.

=

882288882288882466664644664822664822

gStep 1:

=

644822822

m

{2 2 2 2 4 4 6 8 8} , zmin= 2, zmax = 8, zmed = 4, and zxy = 2

Level A: If zmin < zmed < zmax, go to level B TrueLevel B: If zmin < zxy < zmax, output zxy False

Else output zmed This one

So we represent 2 with 4

62

Step 2:

=

464482482

m

{2 2 4 4 4 4 6 8 8} , zmin= 2, zmax = 8, zmed = 4, and zxy = 8

Level A: If zmin < zmed < zmax, go to level B TrueLevel B: If zmin < zxy < zmax, output zxy False

Else output zmed This one

So we represent 8 with 4

=

882288882288882466664644664822664822

g

32

63

Continue with this procedure until all are processed.

>> fa = adpmedian(g, 3)

fa =

4 4 6 64 6 4 66 4 4 66 2 2 8

=

0717110450623340

g

64

Adaptive Spatial FiltersFunction adpmedian will implement the adaptive median filter.

f = adpmedian(g, Smax)

Where g is the image to be filtered and Smax is the maximum allowable size of the adaptive filter window.

33

65

Example

g = imnoise(f, ‘salt & pepper’, 0.25);

This corrupts

the image with

salt and pepper noise,

25% salt +

0.05 pepper)

Note default is 0.05for a and b

66

f1 = medfilt2(g, [7 7], ‘symmetric’);

This seems to be

free of noise but

is a bit blurred and

Distorted.

34

67

f1 = adpmedian(g, 7);

This image seems

to be free of noise

and less blurred.

68

Periodic Noise Reduction by Frequency Domain FilteringThe periodic noise manifests itself as impulse-like bursts that are often visible in the Fourier spectrum. The principal approach for removing these components is via notch filtering. The transfer function of a Butterworth notch filter of order n is given by:

Where: D1(u,v) = [(u - M/2 – u0)2 + (v - N/2 – v0)2]1/2And D2(u,v) = [(u - M/2 + u0)2 + (v - N/2 + v0)2]1/2Where (u0, v0) and by symmetry (-u0, -v0) are locations of the notches and D0 is the measure of their radius. Note: filter is specified with respect to the center of the frequency rectangle.

n

vuDvuDD

vuH

+

=

),(),(1

1),(

21

20

35

69

Modeling the Degradation FunctionOne of the principal degradations encountered in image restoration is image blur. Blur that occurs with the scene and sensor at rest with respect to each other can be modeled by spatial or frequency domain lowpass filters. Another cause of blur is due to uniform linear motion between the sensor and scene during image acquisition. Image blur can be modeled with:

PSF = fspecial(‘motion’ , len, theta)This created a point spread function (PSF) that approximates the effects of linear motion of a camera by len pixels. theta is the angle in degrees with respect to the positive horizontal axis in a counter-clockwise direction.

9 and 0 defaults

70

Modeling the Degradation FunctionThe, function imfilter can be used to create a degraded image with a PSF that is either known or is computed using the procedure on the previous page:

g = imfilter(f, PSF, ‘circular’);circular reduces boarder effectsFinally, the degraded image is created by adding noise. This noise is created using one of the techniques we described at the beginning of the chapter.

g = g + noise;Where noise is a random noise image of the same size as g.

36

71

Example f = checkerboard(8);

72

PSF = fspecial(‘motion’, 7, 45);gb = imfilter(f, PSF, ‘circular’);

37

73

0 0 0 00 0 0 00 0 0 0.03760 0 0.0376 0.12830 0.0376 0.1283 0.03760.0145 0.1283 0.0376 00 0.0145 0 0

0 0.0145 00.0376 0.1283 0.01450.1283 0.0376 00.0376 0 00 0 00 0 00 0 0

PSF =

74

noise = imnoise(zeros(size(f)), ‘gaussian’, 0, 0.001);

38

75

gb = gb + noise;

Why can’t we see the noise in this image?

76

Direct Inverse FilteringThe simplest form to restore a degraded image is to find an estimate as:

and then find an estimate of the image by taking the inverse Fourier transform. This is called inverse filtering.

),(),(),(

vuHvuGvuF =

)

),(),(),(),(

vuHvuNvuFvuF +=

)

What are the problems you will run into here?

Typical approach is to form the ratio shown at the top and then limit the frequencies “near” the origin when you compute the inverse.

Why frequencies “near” the origin?

39

77

Wiener FilteringWiener filtering is one of the best image restoration approach. This approach seeks an estimate of f that minimizes the statistical error function:

Where E is the expected value operator and f is the undegraded image. The solution to this expression in frequency domain is:

})ˆ{( 22 ffEe −=

image undegraded theof spectrumpower the|),(| ),(noise theof spectrumpower the|),(| ),(

),( of conjugatecomplex the ),( ),(),( |),(|

functionn degradatio the ),(

),(),(/),(|),(|

|),(|),(

1),(ˆ

2

2

*

*2

2

2

====

==

=

+=

vuFvuSvuNvuS

vuHvuHvuHvuHvuH

vuHwhere

vuGvuSvuSvuH

vuHvuH

vuF

f

f

η

η

),(/),( vuSvuS fηIs called noise-to-signal ratio

78

),(/),( vuSvuS fη

Wiener Filtering

The noise-to-signal ratio becomes zero when the noise power spectrum becomes zero. See this:

Two other quantities can play a role in this model. One is the average noise power:

And the average image power:

Where M and N denote the vertical and horizontal sizes of the image and noise arrays, respectively. We can define a ratio R as: that we will use in place of noise-to-

signal ratio on previous page.

∑∑=u v

A vuSMN),(1 ηη

∑∑=u v

fA vuSMNf ),(1

A

A

fR η=

40

79

),(/|),(|

|),(|),(

1),(ˆ 22

vuGfvuH

vuHvuH

vuFAA

+

=η

Wiener Filtering

We get a new model:

Which is called parametric Wiener filter.

This filter is implemented in MATLAB as:

fr = deconvwnr(g, PSF)

This assumes that the noise-to-signal ratio is zero. This one:

fr = deconvwnr(g, PSF, NSPR)

Assumes that the noise-to-signal power ratio is known, either as a constant or as an array. The parametric version:

fr = deconvwnr(g, PSF, NACORR, FACORR)

Assumes the autocorrelation functions, NACORR and FACORR of the noise and undegrated image are known.

80

Wiener Filtering ExamplePreviously we obtained this image using:

g = g + noise;

41

81

Wiener Filtering Example – cont.Weiner filtering of degraded image g,fr1 = deconvwnr(g, PSF);This assumed noise-to-signal ratio was 0.

82

Sn = abs(fft2(noise)).^2; % noise power spectrumnA = sum(Sn(:) )/prod(size(noise)); % noise average powerSf = abs(fft2(f)).^2; % image power spectrumfA = sum(Sf(:))/prod(size(f)); % image average powerR = nA/fA; /% ratio

To restore the image we use R for noise-to-signal ratio in the Wiener filtering:fr2 = deconvwnr(g, PSF, R);

This produces the image on the next page.

),(/|),(|

|),(|),(

1),(ˆ 22

vuGfvuH

vuHvuH

vuFAA

+

=η

Compute:Wiener Filtering Example

42

83

Wiener Filtering ExampleCompare this image and the first one we obtained on Page 78.

84

Now we will use autocorrelation functions in the restoration.NCORR = fftshift(real(ifft2(Sn))); ICORR = fftshift(real(ifft2(Sf)));fr3 = deconvwnr(g, PSF, NCORR, ICORR);

The result is very close to the original image, but some noise is still evident.

Note, here we knew the noise function and we were able to estimate correct parameters.

43

85

Constrained Least Squares (Regularized) FilteringThe definition of 2-D discrete convolution is:

The degradation model in matrix form is: We want to find the minimum of a criterion function:

The frequency domain solution to this problem is given as:

Where is a parameter that must be adjusted so that the constraint is satisfied and P(u,v) is the Fourier transform of the function:

∑∑−

=

−

=

−−=∗1

0

1

0),(),(1),(),(

M

m

N

nnymxhnmf

MNyxfyxh

),(|),(||),(|

),(),(ˆ 22*

vuGvuPvuH

vuHvuF

+

=γ

−=

010141010

),( yxP

η Hf g +=

∑∑−

=

−

=

∇=1

0

1

0

22 constraint thesubject to )],([M

x

N

yyxfC

22 ||η||||fHg|| =− ˆ

γ

This method is based on the Laplacian Transform

86

Constrained Least Squares (Regularized) Filtering

In MATLAB, the Constrained least squares filtering is done by function deconvreg as:

fr = deconvreg(g, PSF, NOISEPOWER, RANGE)

Where g is the corrupted image, fr is the restored image, NOISEPOWER is proportional to , and RANGE is the range values where the algorithm is limited to look for a solution for .

The default range is [10-9 , 109]. A good starting estimate for NOISEPOWER is , where M and N are dimensions of the image and the parameters inside the brackets are the noise varianceand noise squared mean.

2||||ηη

][ 22 ηησ mMN +

44

87

ExamplePreviously we obtained this image using:

g = g + noise;

88

fr = deconvreg(g, PSF, 4);

The initial estimate of NOISEPOWER is (64)2[0.001 – 0] ~ 4.

45

89

fr = deconvreg(g, PSF, 0.4, [1e-7 1e7]);

The result is not as good as that we obtained with the Wiener’s method. It is obvious why. When we used that method we knew the noise and image spectra.

90

Iterative Nonlinear Restoration Using the Lucy-Richardson AlgorithmThe previous three methods are all linear. They also are “direct” in the sense that once the restoration filter is specified, the solution is obtained via one application of the filter.The non-linear iterative techniques have been gaining acceptance. They usually yield better result than the linear methods. Perhaps the only drawback to using this methods is that their behavior is not always predictable and that they require significant computational resources.One of the non-linear techniques discussed in the text is Lucy-Richardson (L-R) algorithm.

46

91

The L-R AlgorithmThis algorithm is based on a maximum-likelihood formulation in which the image is modeled with Poisson statistics. Maximizing the likelihood function of the model yields an equation that is satisfied when the following iteration converges.

As before “*” denotes convolution, is the undegraded image, g is the degraded image, and h is the spatial representation of the degraded function. Note that the subscript refers to the iteration number. As in any iterative approach, the question is when to stop the iteration. We usually stop the iteration when the result is somewhat acceptable.

∗∗−−=+ ),(ˆ),(

),(),(),(ˆ),(ˆ 1 yxfyxhyxgyxhyxfyxfk

kk

f̂

92

The L-R AlgorithmIn MATLAB the L-R algorithm is implemented by function deconvlucy:

fr = deconvlucy(g, PSF, NUMIT, DAMPAR, WEIGHT)Where fr is the restored image, g is the degraded image, PSF is the point spread function, NUMIT is the number of iterations (default is 10). DAMPAR denotes the threshold deviation of the resulting image from image g and WEIGHT is an array of same size as g that assigns weight to each pixel to reflect its quality.

f̂

47

93

L-R ExampleOriginal image: f = checkerboard(8)

94

Create a 7x7 PSF with standard deviation of 10PSF = fspecial(‘guassian’, 7, 10);Then blur the image using PDF and add to it Gaussian noise of zero mean and standard deviation of 0.01: SD = 0.01;

g = imnoise(imfilter(f, PSF), ‘gaussian’, 0, SD^2);

48

95

DAMPAR = 10*SD;LIM = ceil(size(PSF, 1)/2);WEIGHT = zeros(size(g));WEIGHT(LIM + 1:end – LIM, LIM + 1:end – LIM) = 1;WEIGHT is a 64x64 array with a border of 0’s 4 pixels wide and the rest all 1’s.NUMIT = 5; % Number of iterationsfr = deconvlucy(g, PSF, NUMIT, DAMPAR, WEIGHT);

96

NUMIT = 10; % Number of iterations

fr = deconvlucy(g, PSF, NUMIT, DAMPAR, WEIGHT);

10 iterations 20 iterations

49

97

With 100 iterations we didn’t get much better results. It however costs much more computing time.The thin black border is the result of 0’s in array WEIGHT.

98

Blind DeconvolutionA difficulty in image restoration is finding a suitable PSF. The image restorations that are not based on specific knowledge of the PSF are called blind deconvolution algorithms.One approach is based on maximum-likelihood estimation (MLE), which is basically an optimization strategy used to estimate quantities corrupted by random noise. Using this method the optimization problem is solved iterativelywith specified constraints. Once converged, the specific f(x,y) and h(x,y) that result in the maximum are restored image and the PSF.The Toolbox in MATLAB performs the blind deconvolution via function deconvblind:[fr, PSFe] = deconvblind(g, INITPSF);Where g is the degraded image, INITPSF is an initial estimate of the PSF, PSFe is the final computed estimate of this function, and fr is the image restored using the estimated PSF.

50

99

Blind Deconvolution – cont.The algorithm used to obtain the restored image is the L-R iterative restoration algorithm. The PSF estimation is strongly affected by the size of its initial guess, and less by its values.A more complex version of the deconvblind function:

[fr, PSFe] = deconvblind(g, INITPSF, NUMIT, DAMPAR, WEIGHT]

Where NUMIT, DAMPAR, and WEIGHT are as described in the L-R.

100

ExampleThe following PSF was generated using:PSF = fspecial(‘guassian’, 7, 10);Imshow(pixeldup(PSF, 73), [ ] );

51

101

We want to estimate that PSF. INITPSF = ones(size(PSF));NUMIT = 5; %number of iterations[fr, PSFe] = deconvblind(g, INITPSF, NUMIT, DAMPAR, WEIGHT);imshow(pixeldup(PSFe, 73), []);

Where:DAMPAR = 10*SD;

For SD = 0.01;And LIM = ceil(size(PSF, 1)/2);WEIGHT = zeros(size(g));

WEIGHT(LIM + 1:end – LIM, LIM + 1:end – LIM) = 1;WEIGHT is a 64x64 array with a border of 0’s 4 pixels wide and the rest all 1’s.

102

10 iterations 20 iterations

52

103

Image RegistrationImage registration methods seek to align two images of the same scene. Some of these images may have been taken at different times with different instruments. Or these images may have been taken by the same instruments under different circumstances. For examplesatellite images. In any case, we quantitative analyses are required to fix the geometric disturbances that are created by differences in camera angle, distance and orientation, sensor resolution, shift in subject position, and other factors.The MATLAB Toolbox supports the control points (tie points) which are a subset of pixels whose locations in the two images are known or can be selected interactively. In MATLAB, we can use cp2tform to fit a specified type of spatial transformation to the control points using least square technique.

104

tform = cp2tform(inputpoints, basepoints, ‘projective’);

53

105

Example:Original image, f Disturbed image, g

The control point coordinates are: (83, 81), (450, 56), (43, 293), (249, 392), and (436, 442). The corresponding control point locations in g are: (68, 66), (375, 47), (42, 286), (275, 434), and (523, 532)

106

basepoints = [83 81; 450 56; 43 293; 249 392; 436 442];inputpoints = [68 66; 375 47; 42 286; 275 434; 523 532];tform = cp2tform(inputpoints, basepoints, ‘projective’);gp = imtransform(g, tform, ‘XData’, [1 502], ‘Ydata’, [1 502]);

Result in:

54

107

>> s = 0.8000;>> theta = pi/6; % 30 degree>> T = [s*cos(theta) s*sin(theta) 0; -s*sin(theta) s*cos(theta) 0;

0 0 1]>> tform = maketform('affine', T);>> g = imtransform(f, tform);>> imshow(g)

Original, f Transformed, g

108

>> cpselect(g,f)

55

109

Then select the pair points of match.

110

Then use the File Tab to store the points:

>> tform = cp2tform(input_points, base_points, 'projective');>> gp = imtransform(g, tform);>> imshow(gp)

Result: