CHAPTER 3 IMAGE RESOLUTION...
Transcript of CHAPTER 3 IMAGE RESOLUTION...
51
CHAPTER 3
IMAGE RESOLUTION ENHANCEMENT
This chapter discusses the three traditional interpolation techniques:
bilinear, nearest neighbor and bicubic interpolation and the proposed discrete
wavelet transform based resolution enhancement and gradient image. The
basis of all these techniques is wavelet transform.
3.1 INTRODUCTION
One of the major problems in images used in research is their
resolution. The unprocessed high resolution spatial data and low resolution
spectral data of satellite images are related with each other. Therefore, spatial
and spectral resolution enhancement of satellite images is desirable. The main
focus of this research is to improve the classification accuracy of noisy
images. In order to improve the accuracy of satellite images efficiently,
preservation of edges and contour information is very important.
Images are being analyzed and processed to obtain the enhanced
resolution and classified result. Interpolation is one of the techniques
generally used for image resolution enhancement. This is mainly used to
increase the number of pixels in a digital image. Interpolation has been widely
used in many image processing applications such as facial reconstruction
introduced by Yi-bo et al (2007), multiple description coding developed by
Rener et al (2008), feature extraction, image denoising and super resolution.
Image resolution enhancement in the wavelet domain is a relatively new
52
research topic and recently many new algorithms have been proposed by
Gupta and Rajiv (2007).
3.2 IMAGE INTERPOLATION
The interpolation of an image aims at estimating intermediate pixels
between the known pixel values in the available low resolution image. The
image interpolation process is nothing but the image synthesis operation. This
process is performed row by row and then column by column.
f(xk-1) f(xk) F(xk+1) f(xk+2)
(a) f(xk)
1/2 ½ 1/2 ½
(b) g(xn)
g(xn)g(xn+1)
11
(c) l(xk) l(xk-1) l(xk) L(xk+1) l(xk+2)
Figure 3.1 Interpolation of signal (a) original data sequence (b) down
sampled version of original data sequence and (c)
interpolated data sequence
53
The discrete sequence )( kxf of length N as shown in Figure 3.1
(a) and this sequence is filtered and down sampled by 2, thereby getting
another sequence )( nxg of length 2/N as shown in Figure 3.1 (b). The
interpolation process aims at estimating a sequence )( kxl of length N as
shown in Figure 3.1 (c), which is as close as possible to the original discrete
sequence )( kxf .
3.2.1 Nearest Neighbour Interpolation
Nearest neighbour interpolation is the simplest interpolation
scheme. The basis function associated with nearest neighbour interpolation is
given by Equation (3.1) as follows
xx
xx
2/122/12/11
2/10)(0
(3.1)
The basis for this scheme is interpolating and it is a pixel repetition
process.
3.2.2 Bilinear Interpolation
The bilinear interpolation has a large popularity due to its simplicity
of implementation. The basis function used in bilinear interpolation is given
by Equation (3.2) as follows
||101||||1
)(1
xxx
x (3.2)
54
3.2.3 Bicubic Interpolation
Another one method which is significantly effective in signal is the
bicubic interpolation. The bicubic interpolation basis function is interpolating
and can be expressed in Equation (3.3)
2||14||8||5||1||01||)3(||)2(
)(23
23
xxxxxxx
x (3.3)
where is an optimization parameter. It may be adaptive from point to point
depending on the signal local activity levels.
This chapter discusses the following resolution enhancement techniques:
Discrete wavelet transform
Stationary wavelet transform
WZP-CS based image resolution enhancement
DT-CWT based image resolution enhancement
Directional wavelet transform
Image resolution enhancement using SWT and DWT
The proposed DWT based interpolation technique
3.3 DISCRETE WAVELET TRANSFORM
Turgay and Huseyin (2009) discussed the 1-D discrete wavelet
transform is applied along the rows of the image rst, and then along the
columns to produce 2-D decomposition of image. Discrete wavelet transform
decomposes an image into four sub-bands namely low-low, low-high, high-
55
InputImage
Low pass filter
High pass filter
2
2
Low pass filter
Low pass filter
High pass filter
High pass filter
2
2
2
2
LL
LH
HL
HH
low and high-high. These four sub-bands can also be used to generate the
original image. The LL sub-band consists of illumination information, where
as the remaining sub-bands contain the information of edges. The
manipulation of these sub-bands gives the improved image i.e., the
enhancement in resolution. Figure 3.2 shows the block diagram of discrete
wavelet transform filter bank of level 1, to generate different sub-band
frequency images. An example of DWT sub-band images are shown in
Figure 3.3 to Figure 3.6.
Figure 3.2 DWT filter bank of level 1
Figure 3.3 LL sub-band image
56
Figure 3.4 LH sub-band image
Figure 3.5 HL sub-band image
Figure 3.6 HH sub-band image
57
3.4 STATIONARY WAVELET TRANSFORM
Hasan and Gholamreza (2011) discussed the wavelet transform can
also provide a scale-based decomposition. The wavelet transform of an image
typically consists of a large number of small coefficients i.e., it contains little
information and a small number of large coefficients i.e., it contains
significant information. Thus each wavelet coefficient is in two states namely
significant and insignificant.
For discrete time signals, discrete wavelet transform is implemented
by filtering the input signal with a low-pass filter and a high-pass filter and
down sampling the outputs by a factor of 2 as shown in Figure 3.2. Applying
the same decomposition to the low pass channel output yields a two-level
wavelet transform; such schemes can be iterated in a dyadic way to generate a
multilevel decomposition. The synthesis of the signal is obtained with a
scheme symmetrical to that of the analysis stage, i.e., by upsampling the
coefficients of the decomposition and by low-pass and high-pass filtering. It
can be shown that if the down sampler removes from the analysis stage and
the up samplers removes from the synthesis stage, then perfect reconstruction
can still be achieved.
Filter jH is interpolated by putting )12( 1j zeros between each of
the coefficient of the original filter 0H , so does for jL . The decrease of
bandwidth is accomplished by zeros padding of the filters instead of down
sampling of wavelet coefficients. This decomposition is called as Stationary
Wavelet Transform (SWT) or undecimated wavelet transform shown in
Figure 3.7.
58
LLj
Lj
Hj
HHj+1
HL j+1
LH j+1
LL j+1
Lj
Hj
2
2
2
2
Lj
Hj
Figure 3.7 Stationary wavelet transform
3.5 WZP-CS BASED IMAGE RESOLUTION ENHANCEMENT
Temizel and Vlachos (2005) discussed the two important steps of
Wavelet domain Zero Padding – Cycle Spinning (WZP-CS) algorithm as
follows:
1) An initial approximation of unknown high resolution image is
generated using wavelet domain zero padding.
2) Next, the cycle-spinning method is used to manage the
following tasks:
Using the high resolution image in step (1), a number of low
resolution images are generated by spatial shifting, wavelet
transforming and discarding the high frequency sub-bands.
The high resolution images are obtained by applying the WZP
processing to all those low resolution images.
59
InputImage
WZP
Shift DWT WZP Shift
Shift DWT WZP Shift
Shift DWT WZP Shift
.
.
.
OutputImage
The final high resolution image is reconstructed by re-aligning
and averaging these intermediated high resolution images.
Figure 3.8 shows the block diagram of the WZP-CS based
image resolution enhancement.
Figure 3.8 WZP-CS based image resolution enhancement
3.6 DT-CWT BASED IMAGE RESOLUTION ENHANCEMENT
Hasan and Gholamreza (2011a) discussed the Dual Tree-Complex
Wavelet Transform (DT-CWT) is used to decompose an input low resolution
image into different sub-bands. Then, the high-frequency sub-band images
and the input image are interpolated. The combination of all these sub-bands
is used to generate a new high-resolution image. The resolution enhancement
is achieved by using directional selectivity provided by the complex wavelet
transform. The sharpness of the high frequency details are contributed by six
different directions of high frequency sub-bands. Figure 3.9 shows details of
DT-CWT based image resolution enhancement technique, where the
enlargement factor through the resolution enhancement is .
60
InputImage
Low frequency sub-band images
High frequency sub-band images
Interpolationwith factor
Interpolated Highfrequency sub-band images
Interpolationwith factor /2
IDT-CWT
DT-CWT
Highresolution
output image
Figure 3.9 DT-CWT based image resolution enhancement
3.7 REGULARITY PRESERVING IMAGE INTERPOLATION
Conventional interpolation methods work in the time domain.
Turgay and Huseyin (2009) discussed in regularity preserving image
interpolation technique, the low-pass output of the wavelet analysis is
considered as the image to be interpolated. The original image can be given as
a single wavelet synthesis stage along with the high frequency sub-bands to
produce an image interpolated by a factor of two in both vertical and
horizontal directions. The formation of unknown high-frequency sub-bands is
required in the regularity-preserving interpolation strategy.
There are mainly two steps that are carried out to obtain the
unknown high-frequency sub-bands separately.
In the first step, the correlation across scales is identified in
each row edge with significant correlation. Then, near these
edges the rate of decay of the wavelet coefficients is
extrapolated to approximate the high-frequency sub-band
required to re-synthesize a row of twice the original size.
61
Input Image
Undecimated L-levelwavelet transform
Locate features withcorrelation
Extrapolated featureposition
Create newsub-band
Extrapolate magnitudedecay
2
2
G(z)
H(z)
Interpolatedimage
In second step, the same procedure as in first step is then
applied to each column of the row-interpolated image.
Figure 3.10 shows the block diagram of interpolation system
for 1-D row and column signals. )(zH and )(zG are analysis
filters.
Figure 3.10 Block diagram of interpolation signals
3.8 EDGE DIRECTED INTERPOLATION
An edge directed interpolation is a combination of bilinear and
covariance-based adaptive interpolation. This is used to reduce the overall
computational complexity. Conventional linear interpolation schemes such as
bilinear and bicubic interpolations are based on space-invariant models. These
are not able to capture the information around edges and produce interpolated
images with blurred edges and annoying artifacts. Geometric regularity is
very much essential for the visual quality of a natural image such as the
sharpness of edges and the freedom from artifacts.
62
Without loss of generality, assume that the low resolution image
jiX , of size WH directly comes from WH 22 i.e. jiji XY ,2,2 .
While using this edge preserving interpolation technique, interpolate
the interlacing lattice 12,12 jiY from the lattice jiji XY ,2,2 .
The hybrid approach used in edge directed interpolation technique is
covariance-based interpolation. This is applied to pixels near an edge and for
pixels in smooth regions (non-edge pixels), use simple bilinear interpolation.
Based on the observation that this approach is benefited for edge pixels.
3.9 IMAGE RESOLUTION ENHANCEMENT USING DWT AND
SWT
The main loss of interpolation technique in image resolution
enhancement is its high frequency components (i.e., edges), which is due to
the smoothing effect caused by interpolation. In order to increase the quality
of the resolution enhanced image, preserving the edges is essential. In this
technique, to preserve the high frequency components of the image DWT has
been employed.
Hasan et al (2011) discussed the resolution enhancement using
DWT and SWT uses bicubic interpolation with enlargement factor of two of
the high frequency sub-band images. While performing downsampling in
each of the DWT sub-bands causes information loss in the respective sub-
bands. So as to minimize this loss SWT is employed. The interpolated high
frequency sub-bands and the SWT high frequency sub-bands have the equal
size which means that can be added with each other. The new corrected high
frequency sub-bands can be interpolated further for higher enhancement. Also
it is well-known that in the wavelet transform domain, the low resolution
image is obtained by lowpass filtering of the high resolution image.
63
Figure 3.11 Image resolution enhancement using DWT and SWT
Figure 3.11 illustrates the block diagram of the image resolution
enhancement using DWT and SWT technique. By interpolating input image
by three, high frequency sub-bands by 2 and in the intermediate and final
interpolation stages respectively, and then by applying IDWT, as illustrated in
InputImage
DWT
SWT
LL
LH
HL
HH
LL
LH
HL
HH
InterpolatedLH
+
+
+
InterpolatedHH
InterpolatedHL
IDWT
High Resolution Image
64
Figure 3.11. The resolution enhanced image will contain sharper edges than
the interpolated image. This is due to the interpolation of high frequency
components and using the corrections obtained by adding high frequency sub-
bands of SWT of the input image. This will preserve more high frequency
components after the interpolation.
3.10 THE PROPOSED DWT BASED INTERPOLATION
TECHNIQUE
Resolution enhancement is a very important technique in satellite
image processing that aims to enhance the visual appearance of the image.
DWT has been employed to preserve the high frequency components of the
image. This work proposes DWT based interpolation technique for satellite
image resolution enhancement in high frequency sub-band images and the
denoised image. The final resolution enhanced image has been obtained by
inverse discrete wavelet transform. In order to obtain a sharper image that
preserves the edge information, interpolation technique is used. This
technique approximates the high frequency sub-band by subtracting the
interpolated LL sub-band from the denoised image. The proposed DWT based
interpolation technique consists of three main steps as follows:
(i) Decompose the input image into four frequency sub-bands
namely, LL, LH, HL and HH.
(ii) Find the difference between input image i.e., denoised image
and the low frequency sub-band image.
65
(iii) The result obtained based on DWT based interpolation is by
interpolating the high frequency sub-band by two and
performing the IDWT using half of the interpolation factor.
The LL sub-band without quantization is used as input for this
proposed technique. The interpolation technique uses this low frequency sub-
band image, which contains little information than the denoised image.
Therefore, the low resolution image is interpolated with the half of the
interpolation factor, to interpolate the high frequency sub-band.
The difference between the low resolution denoised image and the
interpolated LL sub-band image is a high frequency component. This
estimation is calculated by interpolating the high frequency sub-band by two
and performing IDWT using half of the interpolation factor. The additional
step proposed, that is, adding the difference image with the high frequency
components, generates sharper resolution enhanced image.
3.11 EXPERIMENTAL RESULTS AND DISCUSSIONS
The proposed work uses the DWT based interpolation technique to
enhance the resolution of the denoised image. The performance of this
proposed technique is compared with stationary wavelet transform. The
quantitative performance is measured using PSNR and it can be improved in
the resolution enhanced image compared to the denoised image. The
comparison results of SWT and DWT based interpolation technique are
shown in Table 3.1 and Figure 3.12 shows the graphical representation of
performance comparison of SWT and the proposed technique.
66
Table 3.1 Performance comparison between SWT and DWT based
interpolation technique
Sl.No
Region TitleStationary WaveletTransform - PSNR
(dB)
Proposed DWT basedInterpolation Technique
- PSNR (dB)
1 Kochi 32.62 33.15
2 Kanyakumari 32.76 36.58
3 Kolkata 32.91 35.39
4 Visakhapatnam 32.28 35.16
5 Sydney 32.49 35.00
Figure 3.12 Graphical representation of performance of SWT and DWT
based interpolation technique
67
(a) LL sub-band (b) LH sub-band
(c) HL sub-band (d) HH sub-band
Figure 3.13 Experimental results of (a) LL sub-band, (b) LH sub-band,
(c) HL sub-band and (d) HH sub-band images of Kochi
region
68
(a) Stationary wavelet transform
(b) DWT based interpolation technique
Figure 3.14 Experimental results of (a) SWT and (b) DWT based
interpolation technique resolution enhanced images of
Kochi region
Figure 3.13 shows the different sub-bands of wavelet coefficient
namely (a) LL sub-band, (b) LH sub-band, (c) HL sub-band and (d) HH sub-
band images of Kochi region and Figure 3.14 depicts (a) SWT and (b) DWT
based interpolation technique resolution enhanced images of Kochi region.
69
(a) LL sub-band (b) LH sub-band
(c) HL sub-band (d) HH sub-band
Figure 3.15 Experimental results of (a) LL sub-band, (b) LH sub-band,
(c) HL sub-band and (d) HH sub-band images of
Kanyakumari region
70
(a) Stationary wavelet transform
(b) DWT based interpolation technique
Figure 3.16 Experimental results of (a) SWT and (b) DWT based
interpolation technique resolution enhanced images of
Kanyakumari region
71
Figure 3.15 shows the different four sub-band images of
Kanyakumari region and Figure 3.16 depicts (a) SWT and (b) DWT based
interpolation technique resolution enhanced images of Kanyakumari region.
(a) LL sub-band (b) LH sub-band
(c) HL sub-band (d) HH sub-band
Figure 3.17 Experimental results of (a) LL sub-band, (b) LH sub-band,
(c) HL sub-band and (d) HH sub-band images of Kolkata
region
72
(a) Stationary wavelet transform
(b) DWT based interpolation technique
Figure 3.18 Experimental results of (a) SWT and (b) DWT based
interpolation technique resolution enhanced images of
Kolkata region
Figure 3.17 shows the wavelet coefficient sub-bands namely (a)
low-low (b) low-high (c) high-low and (d) high-high images of Kolkata
region and Figure 3.18 depicts (a) SWT and (b) DWT based interpolation
technique resolution enhanced images of Kolkata region.
73
(a) LL sub-band (b) LH sub-band
(c) HL sub-band (d) HH sub-band
Figure 3.19 Experimental results of (a) LL sub-band, (b) LH sub-band,
(c) HL sub-band and (d) HH sub-band images of
Visakhapatnam region
74
(a) Stationary wavelet transform
(b) DWT based interpolation technique
Figure 3.20 Experimental results of (a) SWT and (b) DWT based
interpolation technique resolution enhanced images of
Visakhapatnam region
Figure 3.19 shows the different sub-band images of Visakhapatnam
region and Figure 3.20 depicts (a) SWT and (b) proposed technique resolution
enhanced images of Visakhapatnam region.
75
(a) LL sub-band (b) LH sub-band
(c) HL sub-band (d) HH sub-band
Figure 3.21 Experimental results of (a) LL sub-band, (b) LH sub-band,
(c) HL sub-band and (d) HH sub-band images of Sydney
region
76
(a) Stationary wavelet transform
(b) DWT based interpolation technique
Figure 3.22 Experimental results of (a) SWT and (b) DWT based
interpolation technique resolution enhanced images of
Sydney region
77
Figure 3.21 shows the different sub-bands of wavelet coefficient
namely (a) LL sub-band (b) LH sub-band (c) HL sub-band and (d) HH sub-
band images of Sydney region and Figure 3.22 depicts (a) SWT and (b) DWT
based interpolation technique resolution enhanced images of Sydney region.
In order to evaluate the quantitative performance of DWT based
interpolation technique, the well known Barbara image is taken into account
to determine the performance of the proposed technique.
Table 3.2 shows the performance of resolution enhanced images.
While comparing the performance of SWT with DWT based interpolation
technique the PSNR value of the Barbara image is improved from 37.20dB to
41.49dB. Figure 3.23 shows the different sub-bands of Barbara image and
Figure 3.24 shows SWT and the proposed resolution enhanced images.
Table 3.2 Performance comparison of SWT and DWT based
interpolation technique resolution enhanced image -
Barbara
Image Title
Resolution Enhancement Techniques
Stationary WaveletTransform - PSNR
(dB)
Proposed DWT basedInterpolation Technique
- PSNR (dB)
Barbara 37.20 41.49
78
(a) LL sub-band (b) LH sub-band
(c) HL sub-band (d) HH sub-band
Figure 3.23 Experimental results of (a) LL sub-band, (b) LH sub-band,
(c) HL sub-band and (d) HH sub-band images of Barbara
79
(a) Stationary wavelet transform
(b) DWT based interpolation technique
Figure 3.24 Experimental results of (a) SWT and (b) DWT based
interpolation technique resolution enhanced images of
Barbara
In order to prove the preservation of edges using the proposed
technique, the zoomed out results of resolution enhanced images are shown in
Figure 3.25.
80
Region Name DWT based
Resolution Enhanced
Image
Zoomed out Result
(a) Kochi
(b) Kanyakumari
(c) Kolkata
(d) Visakhapatnam
(e) Sydney
Figure 3.25 Zoomed out results of resolution enhanced images
81
From the experimental results it reveals that newly developed image
resolution enhancement technique preserves the image edge information with
rich textures.
3.12 APPLICATIONS
Image Enhancement has contributed for research in a variety of
fields. Some of the application areas are listed below
In forensics enhancement is used for identification, gathering
of evidence and surveillance. Images obtained from
fingerprint detection and crime scene investigations are
enhanced to help in identification of culprits and protection of
victims.
In atmospheric science enhancement is used to reduce the
effects of haze, fog, mist and turbulent weather for
meteorological observations. It helps in detecting the exact
shape and structure of remote objects. Satellite images
undergo the restoration of images and enhancement to remove
noise.
In oceanography the study of images reveals interesting
features of water flow, sediment concentration, oil spill
detection, geomorphology and bathymetric patterns. These
features are more clearly observable using satellite images that
are digitally enhanced to overcome the problem of moving
targets, deficiency of light and obscure surroundings.
Virtual restoration of historic paintings and artifacts often
employ the techniques of enhancement in order to reduce
stains and crevices. Colour contrast enhancement, sharpening
and brightening are just some of the techniques used to make
82
the images vivid. Enhancement is a powerful tool for restorers
who can make informed decisions by viewing the results of
restoring a painting beforehand.
Medical imaging uses enhancement techniques for removing
noise and sharpening details to improve the visual
representation of the image. Since miniature details play a
critical role in diagnosis and treatment of disease, it is crucial
to highlight important features while displaying medical
images.
Numerous other fields including law enforcement,
microbiology, biomedicine, bacteriology, climatology,
meteorology, etc., benefit from various enhancement
techniques. These benefits are not limited to professional
studies and businesses but extend to the common users who
employ enhancement to cosmetically enhance and correct
their images.
3.13 SUMMARY
This chapter works for enhancing the quality of the image and
applies discrete wavelet transform which is followed by interpolation based
resolution enhancement to obtain a resolution enhanced image. This is done
to extract detailed information from the image and add it to the output image
of the Inverse Discrete Wavelet Transform in order to get a highly robust
resolution enhanced image. These proposed techniques have been tested on
landsat remote sensing images, where there PSNR and visual results show the
efficiency of the proposed techniques over the conventional resolution
enhancement techniques. In order to perform the image classification, feature
extraction is the important step. The chapter 4 describes the texture feature
extraction techniques used for classification.