Chapter 3: Image Restoration

Geometric Transforms

Overview

• Geometric transforms are used to modify the location of pixel values within an image.– Typically used to correct images that have

been spatially distorted.

• Often referred to as rubber-sheet transforms, because the image is modeled as a sheet of rubber that can be stretched and shrunk.

Overview

• Spatial distortion can be caused by:– Defective optics in an image acquisition

system.– Distortion in image display devices.– 2-D imaging of 3-D surface.

• Geometric transforms are also used for other applications that require spatial modification.– Example: Image morphing and image warping.

Overview

• The simplest image transforms (translate, rotate, zoom, shrink) have already been discussed in Chapter 2.– These only move pixels around and do

not really distort the image.• The more sophisticated geometric

transforms require 2 steps:– Spatial transform– Gray-level interpolation

Overview

• The spatial transform provides the location of the output pixel.

• Gray-level interpolation is required because pixel row and column coordinates provided by the spatial transform may not be integers.

Spatial Transforms

• Used to map the input image location to a location in the output image.

• Involves finding the equations that relate the coordinate of the pixels in input image to the coordinate of the pixels in output image.

),(ˆ crRr ),(ˆ crCc

Defines the row coordinate for the distorted image

Defines the column coordinate for the distorted image

Spatial Transforms

• The primary idea here is to find the equations for R(r,c) and C(r,c) and then apply the inverse process to find the restored image.

• The distortion that occurs may be different at different parts of the image.– Therefore, different parts of the image

may require different equations.

Spatial Transforms

• To determine the necessary equations, we need to identify a set of points in the original image that matches points in the distorted image.– These set of points are called tiepoints.

• The form of these equations is typically bilinear, although higher order polynomials can also be used.

Spatial Transforms

• The method to restore a geometrically distorted image consists of three steps:– Define quadrilaterals (four-sided

polygons) with known or best-guessed tiepoints for the entire image.

– Find the equation R(r,c) and C(r,c) for each set of tiepoints.

– Remap all the pixels within each quadrilateral subimage using the equation corresponding to those tie points.

Spatial Transforms

Spatial Transforms

Spatial Transforms

• The equations for R(r,c) and C(r,c) are generated as follows.

• The ki values are constants to be determined by solving the eight simultaneous equations.

• For example:

ckrckckrkcrC

rkrckckrkcrR

ˆ),(

ˆ),(

8765

4321

Spatial Transforms

(86, 43)

(70, 170)

(170,55)(190, 170)

(86, 86) (86, 171)

(171,86) (171, 171)

Spatial Transforms

170)171)(171()171()171(

190)171)(171()171()171(

55)86)(171()86()171(

170)86)(171()86()171(

170)171)(86()171()86(

70)171)(86()171()86(

43)86)(86()86()86(

86)86)(86()86()86(

8765

4321

8765

4321

8765

4321

8765

4321

kkkk

kkkk

kkkk

kkkk

kkkk

kkkk

kkkk

kkkk

Spatial Transforms

• After we have got all the ki values, then we can formulate the equations for R(r,c) and C(r,c).

• The equations can then be used to find the pixels values in the output image by mapping them to their corresponding pixels in the input/distorted image.

• Example: Assume that we have the following mapping equations.

Spatial Transforms

crccrcrC

rrccrcrR

ˆ0211),(

ˆ2335),(

To find I(2,3), substitute (r,c)=(2,3) in to the above equations. We find

170)3)(2(2)3(1)2(1),(

392)3)(2(3)3(3)2(5),(

crC

crR

Now, we let I(2,3) = d(39,17)

Spatial Transforms

• However, in practice, the ki values are not likely to be integers. For example:

crccrcrC

rrccrcrR

ˆ04.216.1),(

ˆ4.25.335.4),(

To find I(2,3), substitute (r,c)=(2,3) in to the above equations. We find

6.200)3)(2(4.2)3(1)2(6.1),(

4.414.2)3)(2(5.3)3(3)2(5.4),(

crC

crR

Now, we let I(2,3) = d(41.4,20.6)

Spatial Transforms

• The difficulty in this example arises when we try to determine the value of d(41.4,20.6).– Digital images are defined only at the

integer values for (r,c).

• Therefore, gray level interpolation must be performed to determine the pixel values for such intermediate location.

Gray-Level Interpolation

• The simplest method for gray-level interpolation is the nearest neighbor method.– The pixel is assigned the value of the closest

pixel in the distorted image.

• In the previous example, the four neighbor pixels are d(41,20), d(41,21), d(42,20), d(42,21).– Therefore, the value of I(2,3) is set to the value

of d(41,21).

Gray-Level Interpolation

Gray-Level Interpolation

• This method has the advantage of being easy to implement and computationally fast, but it does not provide optimal results.

• To obtain a more visually pleasing result, we can implement other more advance methods.

• The easiest of such methods is the neighborhood average method.

Gray-Level Interpolation

• This method uses the four surrounding values in the distorted image to estimate the desired value.– In the previous example, the value of

I(2,3) is set to the value of the average of d(41,20), d(41,21), d(42,20), d(42,21).

• This can be done one dimensionally using adjacent rows or columns, or two dimensionally using all four neighbors.

Gray-Level Interpolation

• Another more advance technique is the method called bilinear interpolation.

• It is done with the following equation:

• The four unknown constants (k1 to k4) are found by using the four surrounding points.– Note that these constants are different from

the ones used in the spatial mapping equation.

4321 ˆ,ˆˆˆ)ˆ,ˆ( kcrkckrkcrg

Gray-Level Interpolation

• Example: Supposed that we are looking for the gray level for d(1.3,2.6). The four surrounding points as follows:

• Then we define the following four simultaneous equations, and solve for the 4 unknowns:

48)3,2(,44)2,2(,55)3,1(,50)2,1()ˆ,ˆ( ddddcrd

Gray-Level Interpolation

4321

4321

4321

4321

)3)(2()3()2(48

)2)(2()2()2(44

)3)(1()3()1(55

)2)(1()2()1(50

kkkk

kkkk

kkkk

kkkk

Solving these simultaneous equations give us:

44ˆˆˆ6ˆ4)ˆ,ˆ(

44,1,6,4 4321

crcrcrg

kkkk

Gray-Level Interpolation

To find d(1.3,2.6), substitute, )ˆ,ˆ( cr = (1.3, 2,6)in the above equations. We will find:

5102.5144)6.2)(3.1()6.2(6)3.1(4)6.2,3.1( g

• Results of different methods:– Nearest neighbor: Can cause blocky effect at

the edges.– Neighborhood average: Smoother edges, but

image becomes a bit blurred.– Bilinear interpolation: Optimal result.

Gray-Level Interpolation

Gray-Level Interpolation

Gray-Level Interpolation

Nearest Neighbor Method

Gray-Level Interpolation

Neighborhood averagemethod

Bilinear Interpolation