Image Restoration. CONTENT Introduction and Overview n(r,c) – additive noise function – Noise...

62
Image Restoration
  • date post

    19-Dec-2015
  • Category

    Documents

  • view

    224
  • download

    1

Transcript of Image Restoration. CONTENT Introduction and Overview n(r,c) – additive noise function – Noise...

Page 1: Image Restoration. CONTENT Introduction and Overview n(r,c) – additive noise function – Noise Models Gaussian Salt-and-pepper Uniform Rayleigh – Noise.

Image Restoration

Page 2: Image Restoration. CONTENT Introduction and Overview n(r,c) – additive noise function – Noise Models Gaussian Salt-and-pepper Uniform Rayleigh – Noise.

CONTENT• Introduction and Overview• n(r,c) – additive noise function

– Noise Models• Gaussian• Salt-and-pepper• Uniform• Rayleigh

– Noise Removal using Spatial Filters• Order filters• Mean filters

• h(r,c) - degradation Function– Geometric Transforms

Page 3: Image Restoration. CONTENT Introduction and Overview n(r,c) – additive noise function – Noise Models Gaussian Salt-and-pepper Uniform Rayleigh – Noise.

Introduction

• IR – process of finding an approximation to the degradation process and finding the appropriate inverse process to estimate the original image

• IR differ IE – it used a mathematical model for image degradation

Page 4: Image Restoration. CONTENT Introduction and Overview n(r,c) – additive noise function – Noise Models Gaussian Salt-and-pepper Uniform Rayleigh – Noise.

Overview • Types of degradation :-

– Blurring caused by motion or atmospheric disturbance

– Geometric distortion caused by imperfect lenses– Superimposed interference patterns caused by

mechanical systems– Spatial quantization– Noise from electronic sources

h(r,c) = degradation function

n(r,c) = additive noise function

Page 5: Image Restoration. CONTENT Introduction and Overview n(r,c) – additive noise function – Noise Models Gaussian Salt-and-pepper Uniform Rayleigh – Noise.
Page 6: Image Restoration. CONTENT Introduction and Overview n(r,c) – additive noise function – Noise Models Gaussian Salt-and-pepper Uniform Rayleigh – Noise.

IR process (fig 9.1-1)• We see that sample degraded images and knowledge of

the image acquisition process are inputs to development of a degradation model

• After the model has been developed, the next step is the formulation of the inverse process

• This inverse degradation process is then applied to the degraded image, d(r,c), which results in the output image, Î(r,c),

• The output image Î(r,c), is the restored image which represents an estimate of original image, I(r,c)

Page 7: Image Restoration. CONTENT Introduction and Overview n(r,c) – additive noise function – Noise Models Gaussian Salt-and-pepper Uniform Rayleigh – Noise.

• Once the estimated image has been created, any knowledge gained by observation & analysis of this image is used as additional input for further development of degradation model

• This process continues until satisfactory results are achieved

Page 8: Image Restoration. CONTENT Introduction and Overview n(r,c) – additive noise function – Noise Models Gaussian Salt-and-pepper Uniform Rayleigh – Noise.

System Model

• Degradation process model consists of 2 parts, the degradation function & the noise function

• General mode in spatial domain :d(r,c) = h(r,c) * I(r,c) + n(r,c)

where d(r,c) = degraded imageh(r,c) = degradation functionI(r,c) = original imagen(r,c) = additive noise function

Page 9: Image Restoration. CONTENT Introduction and Overview n(r,c) – additive noise function – Noise Models Gaussian Salt-and-pepper Uniform Rayleigh – Noise.

System Model (cont’d)

• Frequency domain :D(u,v) = H(u,v) * I(u,v) + N(u,v)

where D(u,v) = Fourier transform of the degraded image

H(u,v) = Fourier transform of the degradation function

I(u,v) = Fourier transform of the original image

N(u,v) = Fourier transform of the additive noise function

Page 10: Image Restoration. CONTENT Introduction and Overview n(r,c) – additive noise function – Noise Models Gaussian Salt-and-pepper Uniform Rayleigh – Noise.

Noise Models

• Any undesired information that contaminates an image• noise models is a random variable with a probability

density function (PDF) that describes its shape and distribution

• The actual distribution of noise in a specific image is the histogram of the noise

• Noise can be modeled with Gaussian (“normal”), uniform, salt-and-pepper (“impulse”), or Rayleigh distribution

Page 11: Image Restoration. CONTENT Introduction and Overview n(r,c) – additive noise function – Noise Models Gaussian Salt-and-pepper Uniform Rayleigh – Noise.

• Gaussian model – occur from electronic noise in image acquisition system– Most problematic with poor lighting conditions or vary

high temperatures– Also valid for film grain noise

• Salt-and-pepper noise (also called impulse noise, shot noise or spike noise) typically caused by malfunctioning pixel element in camera sensors, faulty memory locations, or timing errors in digitization process

Page 12: Image Restoration. CONTENT Introduction and Overview n(r,c) – additive noise function – Noise Models Gaussian Salt-and-pepper Uniform Rayleigh – Noise.

• Uniform noise is useful - it can be used to generate any other type of noise distribution, and is often used to degrade images for the evaluation of image restoration algo since provides the most unbiased or neutral noise model

Page 13: Image Restoration. CONTENT Introduction and Overview n(r,c) – additive noise function – Noise Models Gaussian Salt-and-pepper Uniform Rayleigh – Noise.
Page 14: Image Restoration. CONTENT Introduction and Overview n(r,c) – additive noise function – Noise Models Gaussian Salt-and-pepper Uniform Rayleigh – Noise.

Gaussian distribution 22 2/)(

22

1

mg

g eH

iancevar

deviation standard

(average)mean m

levelgray g

2

Page 15: Image Restoration. CONTENT Introduction and Overview n(r,c) – additive noise function – Noise Models Gaussian Salt-and-pepper Uniform Rayleigh – Noise.

• A bell-shapped• 70% of all values fall within the range from

one standard deviation (σ) below the mean (m) to one above

• About 95% fall within two standard deviations

Page 16: Image Restoration. CONTENT Introduction and Overview n(r,c) – additive noise function – Noise Models Gaussian Salt-and-pepper Uniform Rayleigh – Noise.

Uniform distribution

12

a)-(b variance

2

ba mean

elsewhere 0

bgafor , 1

2

abHUniform

• The gray level values of the noise are evenly distributed across a specific range

Page 17: Image Restoration. CONTENT Introduction and Overview n(r,c) – additive noise function – Noise Models Gaussian Salt-and-pepper Uniform Rayleigh – Noise.

Salt-and-pepper

• There are only 2 possible values, a and b, and the probability of each is typically less than 0.2 – with numbers greater than this the noise will swamp out the image

)pepper"("a gfor )salt"(" b gfor &

ABpapersaltH

Page 18: Image Restoration. CONTENT Introduction and Overview n(r,c) – additive noise function – Noise Models Gaussian Salt-and-pepper Uniform Rayleigh – Noise.

Rayleigh

4

) - (4 nce varia

4

mean

: where

2g /2

gRayleigh eH

Page 19: Image Restoration. CONTENT Introduction and Overview n(r,c) – additive noise function – Noise Models Gaussian Salt-and-pepper Uniform Rayleigh – Noise.

Original image without noise, and its histogram

Page 20: Image Restoration. CONTENT Introduction and Overview n(r,c) – additive noise function – Noise Models Gaussian Salt-and-pepper Uniform Rayleigh – Noise.

image with added Gaussian noise with mean = 0 and variance = 600, and its

histogram

Page 21: Image Restoration. CONTENT Introduction and Overview n(r,c) – additive noise function – Noise Models Gaussian Salt-and-pepper Uniform Rayleigh – Noise.

image with added uniform noise with mean = 0 and variance = 600, and its

histogram

Page 22: Image Restoration. CONTENT Introduction and Overview n(r,c) – additive noise function – Noise Models Gaussian Salt-and-pepper Uniform Rayleigh – Noise.

image with added salt-and-pepper noise with the probability of each 0.08, and its

histogram

Page 23: Image Restoration. CONTENT Introduction and Overview n(r,c) – additive noise function – Noise Models Gaussian Salt-and-pepper Uniform Rayleigh – Noise.

Noise Removal Using Spatial Filters

• Spatial filters can be effectively used to remove various types of noise

• Operate on small neighborhoods, 3x3 to 11x11• Will use the degradation model with the

assumption that h(r,c) causes no degradation where the only corruption to the image is caused by additive noise

Page 24: Image Restoration. CONTENT Introduction and Overview n(r,c) – additive noise function – Noise Models Gaussian Salt-and-pepper Uniform Rayleigh – Noise.

d(r,c) = I(r,c) + n(r,c)where

d(r,c) = degraded imageI(r,c) = original imagen(r,c) = additive noise function

Page 25: Image Restoration. CONTENT Introduction and Overview n(r,c) – additive noise function – Noise Models Gaussian Salt-and-pepper Uniform Rayleigh – Noise.

• Two primary categories; order filters and mean filters

• Order filters – implemented by arranging the neighborhood pixels in order from smallest to largest gray level value, and using this ordering to select the “correct” value

• Mean filters determine, in one sense or another, an average value

Page 26: Image Restoration. CONTENT Introduction and Overview n(r,c) – additive noise function – Noise Models Gaussian Salt-and-pepper Uniform Rayleigh – Noise.

• Mean filters work best with Gaussian or uniform noise

• Order filters work best with salt-and-pepper, negative exponential, or Rayleigh noise

• Mean filters have disad of blurring the image edges, or details

• Order filters such as mean can be used to smooth images

Page 27: Image Restoration. CONTENT Introduction and Overview n(r,c) – additive noise function – Noise Models Gaussian Salt-and-pepper Uniform Rayleigh – Noise.

Order Filters

• Operate on small subimages, windows, and replace the center pixel value (similar to convolution process)

• Given an N x N wondow, W, the pixel values can be ordered as follows

valueslevel)(gray Intensity theare ,.......,I,I,I

where

......,

2321

2321

N

N

I

IIII

Page 28: Image Restoration. CONTENT Introduction and Overview n(r,c) – additive noise function – Noise Models Gaussian Salt-and-pepper Uniform Rayleigh – Noise.

• (85, 88, 95, 100, 104, 104, 110, 110, 114)• Min = 85, Med = 104, max = 114 (will be replaced

at the center value)• Median filter is most useful• Max & min filters can eliminate salt or pepper

noise

85 88 95

104 104 100

114 110 110

Page 29: Image Restoration. CONTENT Introduction and Overview n(r,c) – additive noise function – Noise Models Gaussian Salt-and-pepper Uniform Rayleigh – Noise.

a) Image with added salt-and-pepper noise, the probability for salt = probability for pepper = 0.10, b) after median

filtering with a 3x3 window, all the noise is not removed

a) b)

Page 30: Image Restoration. CONTENT Introduction and Overview n(r,c) – additive noise function – Noise Models Gaussian Salt-and-pepper Uniform Rayleigh – Noise.

c) after median filtering with a 5x5 window, all the noise is removed, but the image is blurry acquiring the “painted”

effect

c)

Page 31: Image Restoration. CONTENT Introduction and Overview n(r,c) – additive noise function – Noise Models Gaussian Salt-and-pepper Uniform Rayleigh – Noise.
Page 32: Image Restoration. CONTENT Introduction and Overview n(r,c) – additive noise function – Noise Models Gaussian Salt-and-pepper Uniform Rayleigh – Noise.
Page 33: Image Restoration. CONTENT Introduction and Overview n(r,c) – additive noise function – Noise Models Gaussian Salt-and-pepper Uniform Rayleigh – Noise.

• Two order filters are midpoint and alpha-trimmed mean filters – both order and mean filters since they rely on ordering the pixels values, but are then calculated by an averaging process

• Midpoint filter – the average of max & min within the window;

• Most useful for Gaussian & uniform noise2

II Midpoint

......,set Ordered

2N1

2321

NIIII

Page 34: Image Restoration. CONTENT Introduction and Overview n(r,c) – additive noise function – Noise Models Gaussian Salt-and-pepper Uniform Rayleigh – Noise.
Page 35: Image Restoration. CONTENT Introduction and Overview n(r,c) – additive noise function – Noise Models Gaussian Salt-and-pepper Uniform Rayleigh – Noise.

• Alpha-trimmed mean is the average of pixel values within the window, but with some of the endpoint ranked excluded

• Useful for images containing multiple types of noise, Gaussian and salt-and-pepper noise

where T is the number of pixel values excluded at each end of the ordered set, and can range from 0 to (N2 – 1)/2

TN

Tii

N

ITN

IIII2

12

2321

2

1 mean trimmed-Alpha

......,set Ordered

Page 36: Image Restoration. CONTENT Introduction and Overview n(r,c) – additive noise function – Noise Models Gaussian Salt-and-pepper Uniform Rayleigh – Noise.

• Alpha-trimmed mean filter ranges from a mean to median filter, depending on the value selected for the T parameter

Page 37: Image Restoration. CONTENT Introduction and Overview n(r,c) – additive noise function – Noise Models Gaussian Salt-and-pepper Uniform Rayleigh – Noise.

Exercise Apply the following filters to the 3 x 3 subimages

below, and find the output for each: (a) median, (b) maximum, (c) minimum, (d) midpoint, (e) alpha-trimmed mean with T = 2.

9109

111212

101110

Page 38: Image Restoration. CONTENT Introduction and Overview n(r,c) – additive noise function – Noise Models Gaussian Salt-and-pepper Uniform Rayleigh – Noise.

Figure 9.3-5 Alpha-Trimmed Mean. This filter can vary between a mean filter and a median filter. a) Image with added noise: zero-mean Gaussian noise with a variance of 200, and salt-

and-pepper noise with probability of each = 0.03, b) result of alpha-trimmed mean filter, mask size = 3x3, T = 1, c) result of alpha-trimmed mean filter, mask size = 3x3, T = 2, d) result of alpha-

trimmed mean filter, mask size = 3x3, T = 4. As the T parameter increases the filter becomes more like a median filter, so becomes more effective at removing the salt-and pepper noise.

a) b)

c) d)

Page 39: Image Restoration. CONTENT Introduction and Overview n(r,c) – additive noise function – Noise Models Gaussian Salt-and-pepper Uniform Rayleigh – Noise.

Mean Filters• Function by finding some form of an average within the

NxN window, using sliding window concept to process entire image

• The most basic – arithmetic mean filter which finds the arithmetic average of pixel values ;

where N2 = the number of pixels in the NxN window, W

• Smooths out local variations & work best with Gaussian, gamma and uniform noise

Wcr

crdN ),(

2),(

1 mean Arithmetic

Page 40: Image Restoration. CONTENT Introduction and Overview n(r,c) – additive noise function – Noise Models Gaussian Salt-and-pepper Uniform Rayleigh – Noise.

• Contra-harmonic mean filter works well for images containing salt OR pepper type noise, depending on the filter order, R:

where W is the NxN window under consideration

• Negative values of R, eliminates salt-type noise• Positive values, eliminates pepper-type noise

Wcr

Wcr

),(

R

),(

1R

c)d(r,

c)d(r,

mean harmonic-Contra

Page 41: Image Restoration. CONTENT Introduction and Overview n(r,c) – additive noise function – Noise Models Gaussian Salt-and-pepper Uniform Rayleigh – Noise.

• Geometric mean filter works best with Gaussian noise, & retains detail information better than an arithmetic mean filter

• Defined as the product of pixel values within window, raised to the 1/N2 power:

Wcr

N

),(

12c)I(r, mean Geometric

Page 42: Image Restoration. CONTENT Introduction and Overview n(r,c) – additive noise function – Noise Models Gaussian Salt-and-pepper Uniform Rayleigh – Noise.

• Harmonic mean filter also fails with pepper noise but works well for salt noise;

• Retaining detail information better than the arithmetic mean filter

Wcr ),(

2

c)d(r,1

N HMean Harmonic

Page 43: Image Restoration. CONTENT Introduction and Overview n(r,c) – additive noise function – Noise Models Gaussian Salt-and-pepper Uniform Rayleigh – Noise.

• Yp mean filter is defined as follows:

p

Wcr

P

p N

crdY

/1

),(2

),(mean

Page 44: Image Restoration. CONTENT Introduction and Overview n(r,c) – additive noise function – Noise Models Gaussian Salt-and-pepper Uniform Rayleigh – Noise.

Exercise Apply the following filters to the 3 x 3 subimages

below, and find the output for each: (a) arithmetic mean, (b) contra-harmonic mean with R= -2, (c) contra-harmonic mean with R=+2, (d) geometric mean, (e) harmonic mean, (f) Yp mean with P = -1, (g) Yp mean with P = +2.

9109

111212

101110

Page 45: Image Restoration. CONTENT Introduction and Overview n(r,c) – additive noise function – Noise Models Gaussian Salt-and-pepper Uniform Rayleigh – Noise.

The Degradation Function

• Either spatially-invariant or spatially-variant• Spatially-invariant degradation affects all

pixels in the image same– Eg, poor lens focus and camera motion

• Spatially-variant degradation depend on spatial location & more difficult to model– Eg, imperfects in a lens or object motion

Page 46: Image Restoration. CONTENT Introduction and Overview n(r,c) – additive noise function – Noise Models Gaussian Salt-and-pepper Uniform Rayleigh – Noise.

Point Spread Function

• d(r,c) = I(r,c) * h(r,c) where * denotes the convolution process

• h(r,c) is called point spread function (PSF), or blur function

• PSF of a linear, spatially-invariant(shift invariant) system can be empirically determined by imaging a single point of light

Page 47: Image Restoration. CONTENT Introduction and Overview n(r,c) – additive noise function – Noise Models Gaussian Salt-and-pepper Uniform Rayleigh – Noise.

Estimation of the Degradation Function

• Estimated primarily by combinations of 1. Image analysis2. Experimentation3. Mathematical modeling

• Image analysis : examine a known point or line in an image, and estimate the PSF by measuring the width and distribution of known feature in blurred image

Page 48: Image Restoration. CONTENT Introduction and Overview n(r,c) – additive noise function – Noise Models Gaussian Salt-and-pepper Uniform Rayleigh – Noise.

• Experimentation : 1. PSF can be found by imaging a point of light2. A more reliable method is to use sinusoidal

inputs at many different spatial frequencies to find the H(u,v)

• Mathematical modeling examples:1. The motion blur model2. Atmospheric turbulence degradation model

used in astronomy and remote sensing

Page 49: Image Restoration. CONTENT Introduction and Overview n(r,c) – additive noise function – Noise Models Gaussian Salt-and-pepper Uniform Rayleigh – Noise.
Page 50: Image Restoration. CONTENT Introduction and Overview n(r,c) – additive noise function – Noise Models Gaussian Salt-and-pepper Uniform Rayleigh – Noise.

Geometric Transforms

• Spatially-variant• Images that have been spatially, or geometrically,

distorted• Used to modify the location of pixel values within

an image, typically to correct images that have been spatially warped

• Often referred as rubber-sheet transforms - image is modeled as a sheet of rubber and stretched and shrunk

Page 51: Image Restoration. CONTENT Introduction and Overview n(r,c) – additive noise function – Noise Models Gaussian Salt-and-pepper Uniform Rayleigh – Noise.

• Because of defective optics in image acquisition system, distortion in image display devices, or 2D imaging of 3D surfaces

• This methods are used in map making, image registration, image morphing, and other applications requiring spatial modification

• Simplest – translate, rotate, zoom & shrink• More sophisticated – 1) spatial transform & 2) gray level

interpolation

Input Image Spatial Transform

Spatial Transform

Gray Level Interpolation

Gray Level Interpolation Output Image

Page 52: Image Restoration. CONTENT Introduction and Overview n(r,c) – additive noise function – Noise Models Gaussian Salt-and-pepper Uniform Rayleigh – Noise.

Spatial Transforms

• Used to map the input image location to a location in the output image; it defines how the pixel values in output image are to be arranged

Page 53: Image Restoration. CONTENT Introduction and Overview n(r,c) – additive noise function – Noise Models Gaussian Salt-and-pepper Uniform Rayleigh – Noise.

• The original, undistorted image, I(r,c), and distorted (or degraded) image is

• Primary idea is to find a mathematical model for the geometric distortion process –and apply the inverse process to find restored image

I(r,c)

Geometric Distortion

)ˆ,ˆ( crd),(ˆˆ

),(ˆˆ

crCc

crRr

)ˆ,ˆ( crd

image distorted for the coordinatecolumn thedefines ),,(ˆ ˆ

image distorted for the coordinate row thedefines ),,(ˆ ˆ

crCc

crRr

),(ˆ and ),(ˆ crCcrR

Page 54: Image Restoration. CONTENT Introduction and Overview n(r,c) – additive noise function – Noise Models Gaussian Salt-and-pepper Uniform Rayleigh – Noise.

• Different equations for different portions of the image

• To determine the necessary equations, need to identify a set of points in the original image that match points in the distorted image

• These sets of points are called tiepoints, used to define the equations ),(ˆ and ),(ˆ crCcrR

Page 55: Image Restoration. CONTENT Introduction and Overview n(r,c) – additive noise function – Noise Models Gaussian Salt-and-pepper Uniform Rayleigh – Noise.

• Method to restore a geometrically distorted image consists of 3 steps:1. Define quadriterals (4 sided polygons) with known,

or best-guessed tiepoints for the entire image2. Find the equations for each set of

tiepoints,3. Remap all the pixels within each quadrilateral

subimage using the equations corresponding to those tiepoints

),(ˆ and ),(ˆ crCcrR

Page 56: Image Restoration. CONTENT Introduction and Overview n(r,c) – additive noise function – Noise Models Gaussian Salt-and-pepper Uniform Rayleigh – Noise.
Page 57: Image Restoration. CONTENT Introduction and Overview n(r,c) – additive noise function – Noise Models Gaussian Salt-and-pepper Uniform Rayleigh – Noise.

1. 2 images are divided into subimages, defined by tiepoints (fig. 9.6.3 a&b)

2. Using bilinear model for the mapping equations, these 4 points to generate the equations :

3. Involves application of the mapping equations, , to all the (r,c) pairs

ckrckckrkcrC

rkrckckrkcrR

ˆ),(ˆ

ˆ),(ˆ

8765

4321

),(ˆ and ),(ˆ crCcrR

Page 58: Image Restoration. CONTENT Introduction and Overview n(r,c) – additive noise function – Noise Models Gaussian Salt-and-pepper Uniform Rayleigh – Noise.

Exercise

• Example 9.6.1 & 9.6.2• The difficulty in above example arises when

we try to determine the value of d(41.4, 20.6)– Since digital images are defined only at integer

values for Î(r,c) as an estimate to the original image I(r,c) to represent the restored image

Page 59: Image Restoration. CONTENT Introduction and Overview n(r,c) – additive noise function – Noise Models Gaussian Salt-and-pepper Uniform Rayleigh – Noise.
Page 60: Image Restoration. CONTENT Introduction and Overview n(r,c) – additive noise function – Noise Models Gaussian Salt-and-pepper Uniform Rayleigh – Noise.

Gray Level Interpolation• The simplest – nearest neighbor method, where the pixel is

assigned the value of the closest pixel in the distorted image– Î(2,3) is set to the value of d(41,21), the row and column values

determined by rounding – Easy to implement and computationally fast

• More advance is to interpolate the value– More computationally extensive but more visually pleasing results– Easiest - neighborhood average. Provide smoother object edges

but slightly blurry

Page 61: Image Restoration. CONTENT Introduction and Overview n(r,c) – additive noise function – Noise Models Gaussian Salt-and-pepper Uniform Rayleigh – Noise.

Gray Level Interpolation• Better results- uses bilinear interpolation with the

equation:where = the gray level interpolating

equation• Example 9.6.3 & 9.6.4

4321 ˆˆˆˆ)ˆ,ˆ( kcrkckrkcrg

)ˆ,ˆ( crg

Page 62: Image Restoration. CONTENT Introduction and Overview n(r,c) – additive noise function – Noise Models Gaussian Salt-and-pepper Uniform Rayleigh – Noise.