[IEEE 2010 IEEE International Conference on Technologies for Homeland Security (HST) - Waltham, MA,...

7
Image Fusion Using the Parameterized Logarithmic Dual Tree Complex Wavelet Transform Shahan Nercessian, Karen Panetta Department of Electrical and Computer Engineering Tufts University Medford, MA, USA [email protected], [email protected] Sos Agaian Department of Electrical and Computer Engineering University of Texas at San Antonio San Antonio, TX, USA [email protected] Abstract—Image fusion combines multiple images into a single image containing the relevant information from each of the original source images. This paper introduces a new Parameterized Logarithmic Dual Tree Complex Wavelet Transform (PL-DT-CWT) and its application for image fusion. The new transform combines the Dual Tree Complex Wavelet Transform (DT-CWT) with the Parameterized Logarithmic Image Processing (PLIP) model, a nonlinear image processing framework for processing images. Experimental results via computer simulations illustrate the improved performance of the proposed algorithms by both qualitative and quantitative means. Keywords-image fusion; parameterized logarithmic image processing; dual tree complex wavelet transform I. INTRODUCTION Image fusion algorithms combine multiple source images into a single image while retaining the important features from each of the original images [1]. The original source images may differ in resolution, instrument modality, or image capture technique [2]. The aim of an image fusion algorithm is to integrate redundant and complementary information obtained from the source images in order to form a new image which provides a better description of the scene for human or machine perception [3]. As a result, image fusion plays a crucial role for computer vision and robotics systems, as fusion results can be used to aid further processing steps for a given task. Therefore, image fusion techniques are practical and essential for many applications, including multi-spectral remote sensing [4] and surveillance for homeland security [5]. For example, infrared images provide information of intruder or potential threat objects, while visible light images provide high- resolution structural information. Therefore, the use of image fusion is particularly practical for land and maritime border control human observers or automated threat detection systems. As fusion results can affect the results of additional processing performed by vision systems, the overall quality of the image fusion process is of paramount importance. The most basic of image fusion approaches include spatial domain techniques using simple averaging, Principal Component Analysis [6], and the Intensity-Hue-Saturation transformation [7]. However, such methods do not incorporate aspects of the human visual system in their formulation. It is well known that the human visual system is particularly sensitive to edges at their various scales [8]. Motivated by this fact, many multi-scale image fusion techniques have been proposed in order to yield more visually accurate fusion results. These approaches decompose image signals into low-pass and high-pass coefficients via a forward transform, fuse low-pass and high-pass coefficients according to specific fusion rules, and perform an inverse transform to yield the final fusion result. The use of different fusion rules for low-pass and high- pass coefficients provides a means of yielding fusion results which are in accordance to the human visual system. The most common of multi-resolution decomposition schemes for image fusion has been the wavelet transform [2]. Particularly, wavelet-based image fusion algorithms using the Discrete Wavelet Transform (DWT) [9], Stationary Wavelet Transform (SWT) [10], and Dual Tree Complex Wavelet Transform (DT-CWT) [5] have been proposed. The advantage of the DT-CWT relative to the DWT and SWT lies in its added directionality and its balance between overcompleteness and near shift invariance [11]. As a result, image fusion algorithms using the DT-CWT have been shown to be able to outperform the other wavelet-based approaches. The mentioned image fusion approaches are implemented using standard arithmetic operators. Conversely, the Logarithmic Image Processing (LIP) model was proposed to provide a nonlinear framework for visualizing images [12]. The LIP model views images in terms of their graytone functions, which are interpreted as absorption filters. It processes graytone functions using a new arithmetic which replaces standard arithmetical operators. The resulting set of arithmetic operators can be used to process images based on a physically relevant image formation model. As a result, image enhancement [13], edge detection [14], and image restoration algorithms utilizing LIP model have yielded better results. The Parameterized Logarithmic Image Processing (PLIP) model is a generalization of the LIP model which attempts to overcome shortcomings of the LIP model, namely the loss of information inherent to the framework [15]. Adaptations of edge detection [16] and image enhancement algorithms [17] using the PLIP model have demonstrated the improved performance achieved by the parameterized framework. In this paper, new image fusion algorithms using a new Parameterized Logarithmic Dual Tree Complex Wavelet Transform (PL-DT-CWT) are introduced. The new transform 978-1-4244-6048-9/10/$26.00 ©2010 IEEE 296

Transcript of [IEEE 2010 IEEE International Conference on Technologies for Homeland Security (HST) - Waltham, MA,...

Page 1: [IEEE 2010 IEEE International Conference on Technologies for Homeland Security (HST) - Waltham, MA, USA (2010.11.8-2010.11.10)] 2010 IEEE International Conference on Technologies for

Image Fusion Using the Parameterized Logarithmic Dual Tree Complex Wavelet Transform

Shahan Nercessian, Karen Panetta Department of Electrical and Computer Engineering

Tufts University Medford, MA, USA

[email protected], [email protected]

Sos Agaian Department of Electrical and Computer Engineering

University of Texas at San Antonio San Antonio, TX, USA

[email protected]

Abstract—Image fusion combines multiple images into a single image containing the relevant information from each of the original source images. This paper introduces a new Parameterized Logarithmic Dual Tree Complex Wavelet Transform (PL-DT-CWT) and its application for image fusion. The new transform combines the Dual Tree Complex Wavelet Transform (DT-CWT) with the Parameterized Logarithmic Image Processing (PLIP) model, a nonlinear image processing framework for processing images. Experimental results via computer simulations illustrate the improved performance of the proposed algorithms by both qualitative and quantitative means.

Keywords-image fusion; parameterized logarithmic image processing; dual tree complex wavelet transform

I. INTRODUCTION Image fusion algorithms combine multiple source images

into a single image while retaining the important features from each of the original images [1]. The original source images may differ in resolution, instrument modality, or image capture technique [2]. The aim of an image fusion algorithm is to integrate redundant and complementary information obtained from the source images in order to form a new image which provides a better description of the scene for human or machine perception [3]. As a result, image fusion plays a crucial role for computer vision and robotics systems, as fusion results can be used to aid further processing steps for a given task. Therefore, image fusion techniques are practical and essential for many applications, including multi-spectral remote sensing [4] and surveillance for homeland security [5]. For example, infrared images provide information of intruder or potential threat objects, while visible light images provide high-resolution structural information. Therefore, the use of image fusion is particularly practical for land and maritime border control human observers or automated threat detection systems. As fusion results can affect the results of additional processing performed by vision systems, the overall quality of the image fusion process is of paramount importance.

The most basic of image fusion approaches include spatial domain techniques using simple averaging, Principal Component Analysis [6], and the Intensity-Hue-Saturation transformation [7]. However, such methods do not incorporate aspects of the human visual system in their formulation. It is well known that the human visual system is particularly

sensitive to edges at their various scales [8]. Motivated by this fact, many multi-scale image fusion techniques have been proposed in order to yield more visually accurate fusion results. These approaches decompose image signals into low-pass and high-pass coefficients via a forward transform, fuse low-pass and high-pass coefficients according to specific fusion rules, and perform an inverse transform to yield the final fusion result. The use of different fusion rules for low-pass and high-pass coefficients provides a means of yielding fusion results which are in accordance to the human visual system.

The most common of multi-resolution decomposition schemes for image fusion has been the wavelet transform [2]. Particularly, wavelet-based image fusion algorithms using the Discrete Wavelet Transform (DWT) [9], Stationary Wavelet Transform (SWT) [10], and Dual Tree Complex Wavelet Transform (DT-CWT) [5] have been proposed. The advantage of the DT-CWT relative to the DWT and SWT lies in its added directionality and its balance between overcompleteness and near shift invariance [11]. As a result, image fusion algorithms using the DT-CWT have been shown to be able to outperform the other wavelet-based approaches.

The mentioned image fusion approaches are implemented using standard arithmetic operators. Conversely, the Logarithmic Image Processing (LIP) model was proposed to provide a nonlinear framework for visualizing images [12]. The LIP model views images in terms of their graytone functions, which are interpreted as absorption filters. It processes graytone functions using a new arithmetic which replaces standard arithmetical operators. The resulting set of arithmetic operators can be used to process images based on a physically relevant image formation model. As a result, image enhancement [13], edge detection [14], and image restoration algorithms utilizing LIP model have yielded better results. The Parameterized Logarithmic Image Processing (PLIP) model is a generalization of the LIP model which attempts to overcome shortcomings of the LIP model, namely the loss of information inherent to the framework [15]. Adaptations of edge detection [16] and image enhancement algorithms [17] using the PLIP model have demonstrated the improved performance achieved by the parameterized framework.

In this paper, new image fusion algorithms using a new Parameterized Logarithmic Dual Tree Complex Wavelet Transform (PL-DT-CWT) are introduced. The new transform

978-1-4244-6048-9/10/$26.00 ©2010 IEEE 296

Page 2: [IEEE 2010 IEEE International Conference on Technologies for Homeland Security (HST) - Waltham, MA, USA (2010.11.8-2010.11.10)] 2010 IEEE International Conference on Technologies for

leverages on the benefits of the DT-CWT and the PLIP to yield superior image fusion results. The rationale and effect of the added parameterization to the transform is analyzed. Experimental results via computer simulation illustrate the improved performance of the proposed algorithms by both qualitative and quantitative means. For quantitative assessment of image fusion results, the Piella and Heijmans quality metric [18] is used.

The remainder of this paper is organized as follows: Section 2 provides a brief overview of the DT-CWT and image fusion using the DT-CWT. Section 3 provides background information about the LIP and PLIP models. Section 4 subsequently introduces the new PL-DT-CWT and new image fusion schemes using the PL-DT-CWT. Section 5 describes the quality metric used for quantitative analysis. Section 6 compares the proposed image fusion algorithms with existing standards via computer simulations. Section 7 draws conclusions based on the presented experimental results.

II. IMAGE FUSION USING THE DUAL TREE COMPLEX WAVELET TRANSFORM

A. Dual Tree Complex Wavelet Transform The DWT is able to provide perfect reconstruction while

using critical sampling. However, it is shift variant and due to its critical sampling, the alteration of transform coefficients may introduce artifacts in the processed result. As it is desired that artifacts not be introduced due to the fusion process, fusion algorithms using the SWT were consequently formulated [10]. The SWT is a shift-invariant, overcomplete wavelet transform which attempts to reduce artifact effects of the DWT by upsampling analysis filters rather than downsampling approximation images at each level of decomposition. Despite the fact that image fusion algorithms using the SWT generally yield more visually pleasing fusion results than those using the DWT, the SWT is computationally more expensive as a result of its redundancy. Moreover, no added directionality is achieved relative to the DWT.

The Fourier transform represents signals as a weighted sum of complex sinusoids, where each complex sinusoid forms an analytic signal given by

)sin()cos( tjte tj ωωω += (1)

The high directionality and shift-invariance properties of the Fourier transform provide the rationale for the formulation of a similarly defined complex wavelet given as

)()()( tjtt irc ψψψ += (2)

which forms as close to an analytic signal as possible while also achieving the perfect reconstruction needed to process signals using wavelets. The DT-CWT [11] accomplishes this by using two filter banks and thus two bases. Given two filter banks {h0(n), h1(n)} and {g0(n), g1(n)}, the 2D DT-CWT calculates 4 DWTs in parallel, Fhh, Fgg, Fgh, and Fhg, where Fgh denotes a wavelet transform yielded using filters gi(n) along the rows and filters hi(n) along columns. As each DWT generates 3 sub-bands per decomposition level, the 12 sub-bands from the DWTs can be reorganized to form 6 complex, approximately analytic sub-bands. Consequent analysis shows that these complex sub-bands are oriented at ±15°, ±45°, and ±75°. Therefore, the DT-CWT has added directionality relative to the DWT and SWT, is nearly shift invariant, and is less redundant relative to the SWT. The filters used by the DT-CWT which approximately satisfy the desired conditions of perfect reconstruction and analyticity are the biorthogonal and q-shift filters described in [11]. At each decomposition level n, the 2D DT-CWT generates a real low-pass image )(

0ny and 6

complex high-pass sub-images )(niy , i = 1, ..., 6.

B. Image fusion decision rules using the DT-CWT An image fusion algorithm for two input images using the

DT-CWT is illustrated in Fig. 1. The input source images are transformed using the 2D DT-CWT. One fusion rule is used to fuse the approximation coefficients at the highest decomposition level. A second fusion rule is used to fuse the detail coefficients at each decomposition level. The resulting inverse transform yields the final fused result.

The approximation coefficients at the highest level of decomposition are most commonly fused via uniform averaging [19]. Given )(

0,1

NIy and )(

0,2

NIy , the approximation

coefficient sub-bands of images I1 and I2, respectively, at the highest decomposition level N, the approximation coefficients for the fused image F at the highest level of decomposition is given by

2

)(0,

)(0,)(

0,21

NI

NIN

F

yyy

+= (3)

Figure 1. Image fusion using the DT-CWT

297

Page 3: [IEEE 2010 IEEE International Conference on Technologies for Homeland Security (HST) - Waltham, MA, USA (2010.11.8-2010.11.10)] 2010 IEEE International Conference on Technologies for

Conversely, detail coefficients correspond to salient features such as edges, lines, or regions. Therefore, fusion rules for detail coefficients at each decomposition level can be formulated with respect to the human visual system in order to preserve the salient features of the original source images. Of the many fusion rules which have been proposed in the literature, two common detail coefficient fusion rules are:

1) Absolute maximum fusion rule: The absolute maximum fusion rule [19] selects the coefficient in each sub-band with greatest magnitude. For each of the i high-pass sub-bands at each level of decomposition n, the detail coefficients for the fused image F are determined by

( )( ) ( ) ( )( ) ( ) ( )⎪⎩

⎪⎨⎧

>=

lkylkylky

lkylkylkylky

niI

niI

niI

niI

niI

niIn

iF,,,

,,,,

)(,

)(,

)(,

)(,

)(,

)(,)(

,

212

211 (4)

2) Burt’s fusion rule: Burt’s fusion rule combines detail coefficients based on an activity and match measure [20]. The activity measure for each wxw local window of each sub-band is calculated for each source image, given as

( ) ( )( )

∑∈ΔΔ

Δ+Δ+=Wlk

niI

niI llkkylka

,

2)(,

)(, ,, (5)

The local match measure of each sub-band measures the correlation of each sub-band between the two source images, and is given as

( )( )

( )( )

( ) ( )lkalka

llkkyllkky

lkmn

iIn

iI

niI

Wlk

niI

niII ,,

,,2

,)(,

)(,

)(,

,

)(,

)(,,

21

21

21 +

Δ+Δ+Δ+Δ+

=∑

∈ΔΔ (6)

Comparing the match measure to a threshold T determines if detail coefficients are to be is combined by simple selection or by weighted averaging. The associated weights for fusion are given by

( )

( ) ( ) ( )( ) ( ) ( )

( )( ) ( ) ( )

( )( ) ( ) ( )

⎪⎪⎪⎪⎪

⎪⎪⎪⎪⎪

≤>⎟⎟

⎜⎜

−−

>>⎟⎟

⎜⎜

−+

≤≤

>≤

=

lkalkaTlkmT

lkm

lkalkaTlkmT

lkm

lkalkaTlkm

lkalkaTlkm

lk

niI

niI

niII

niII

niI

niI

niII

niII

niI

niI

niII

niI

niI

niII

ni

,,,,1

,1

21

21

,,,,1

,1

21

21

,,,,0

,,,,1

,

)(,

)(,

)(,,

)(,,

)(,

)(,

)(,,

)(,,

)(,

)(,

)(,,

)(,

)(,

)(,,

)(

2121

21

2121

21

2121

2121

λ

(7)

and for each of the i high-pass sub-bands at each level of decomposition n, the detail coefficients for the fused image F are determined by

( ) ( ) ( ) ( )( ) ( )lkylklkylklky niI

ni

niI

ni

niF ,,1,,, )(

,)()(

,)()(

, 21λλ −+= (8)

III. NONLINEAR IMAGE PROCESSING FRAMEWORKS

A. Logarithmic Image Processing Model The LIP model [12] processes images as absorption filters.

These absorption filters are represented as graytone functions which resemble image negatives. The graytone function g of an image I is calculated by

( ) ( )lkIMlkg ,, −= (9)

where M is the maximum value of the range of I. For the case of 8-bit images, M = 256. The LIP is characterized by its isomorphic transformation, defined as

( ) ⎟⎠

⎞⎜⎝

⎛ −−=Mg

Mg 1lnϕ (10)

( ) ⎥⎦

⎤⎢⎣

⎡⎟⎠

⎞⎜⎝

⎛−−−=−

Mg

Mg exp11ϕ (11)

The isomorphic transformation attempts to mathematically emulate the physical model observed when images are interpreted as absorption filters. LIP primitives, namely addition, subtraction, and scalar multiplication are consequently defined for graytone functions g1 and g2 and scalar constant c by

( ) ( )( )

Mgg

gggggg 212121

121 −+=+=+ −

+ ϕϕϕ (12)

( ) ( )( )2

2121

121 gM

ggMgggg

−−

=−=− −− ϕϕϕ (13)

( )( )c

Mg

MMgcgc ⎟⎟⎠

⎞⎜⎜⎝

⎛−−==× −

×1

11

1 1ϕϕ (14)

Given a graytone function g, the LIP wavelet transform [21] can be computed by directly making use of the isomorphic transformation by

( ) ( )( )( )gWgW ϕϕ 1−Δ = (15)

where W is a type of wavelet transform (e.g. DWT, SWT, etc.) with a given wavelet filter bank. The inverse wavelet transform is also easily arrived at using the isomorphic transformation by

( )( ) ( )( )( )( )gWWgWW ϕϕ 111 −−Δ

−Δ = (16)

298

Page 4: [IEEE 2010 IEEE International Conference on Technologies for Homeland Security (HST) - Waltham, MA, USA (2010.11.8-2010.11.10)] 2010 IEEE International Conference on Technologies for

B. Parameterized Logarithmic Image Processing Model The PLIP [15] model generalizes the isomorphic transformation which defines the LIP model by

⎟⎟⎠

⎞⎜⎜⎝

⎛−⋅−=

γγϕ

gg 1ln)(~ (17)

⎥⎦

⎤⎢⎣

⎡⎟⎟⎠

⎞⎜⎜⎝

⎛ −−=−

γγϕ

gg exp1)(~ 1 (18)

where γ is an image dependent parameter. Practically, γ ≥ M for positive γ and can also take on any negative value. It is apparent that when γ = M, the LIP model is yielded. It can also be easily shown that

aaa == −

∞→∞→)(~lim)(~lim 1ϕϕ

γγ (19)

Therefore, the PLIP approaches standard linear processing of graytone functions as |γ| approaches infinity. Depending on the nature of the algorithm, an algorithm which utilizes standard linear processing operators is in fact simply an instance of an algorithm using the PLIP model with γ = ∞. For positive γ, the PLIP model physically provides a balance between the standard linear processing model and the LIP model in order to minimize information loss while also retaining non-linear, logarithmic functionality according to a physical model. Negative values of γ, conversely, the model artificially injects added information. The resulting PLIP primitives based on the parameterized isomorphic transformation follow as

( ) ( )( )

γϕϕϕ 21

21211

21~~~~ gg

gggggg −+=+=⊕ −

(20)

( ) ( )( )2

2121

121

~~~~ggg

gggg−−

=−=Θ −

γγϕϕϕ (21)

( )( )cg

gcgc ⎟⎟⎠

⎞⎜⎜⎝

⎛−−==⊗ −

γγγϕϕ 1

11

1 1~~~ (22)

Consequently, forward and inverse PLIP wavelet transform can be calculated by

( ) ( )( )( )gWgW ϕϕ ~~~ 1−Δ = (23)

( )( ) ( )( )( )( )gWWgWW Δ−−

Δ−

Δ =~~~~~ 111 ϕϕ (24)

IV. IMAGE FUSION USING THE PARAMETERIZED LOGARITHMIC DUAL TREE COMPLEX WAVELET TRANSFORM

A. Parameterized Logarithmic Dual Tree Complex Wavelet Transform The PL-DT-CWT at decomposition level n is computed by

directly making use of (16) and (17). The PL-DT-CWT for a graytone function g at a scale n is generated by

( ) ( )( )( ))(0

1)(0

~~~~~ nn yDTCWTyWTCDT ϕϕ Δ−

Δ = (25)

where )0(0

~y = g. Similarly, the inverse procedure begins from the approximation coefficients at the high decomposition level N, with each synthesis level reconstructing approximation coefficients by

( )( ) ( )( )( )( ))(0

11)(0

1 ~~~~~~~ nn yWTCDTDTCWTyWTCDTWTCDT Δ−−

Δ−

Δ = ϕϕ (26)

Figure 2 illustrates the analysis and synthesis stages using the PL-DT-CWT. As the PL-DT-CWT essentially makes use of the standard DT-CWT with added preprocessing and postprocessing in the form of the isomorphic transformation calculations, it can be computed with minimal added computation cost.

B. Image fusion decision rules using the PL-DT-CWT If simple averaging is the fusion rule used to fuse average

approximation coefficients at the highest level of decomposition, it must also be adapted according to the PLIP model. For )(

0,1

~ NIy and )(

0,2

~ NIy , the approximation coefficient sub-

bands of images I1 and I2, respectively, at the highest decomposition level N yielded using the PL-DT-CWT, the approximation coefficients for the fused image F at the highest level of decomposition is given by

Figure 2. PL-DT-CWT analysis and synthesis

299

Page 5: [IEEE 2010 IEEE International Conference on Technologies for Homeland Security (HST) - Waltham, MA, USA (2010.11.8-2010.11.10)] 2010 IEEE International Conference on Technologies for

( ))(0,

)(0,

)(0, 21

~~~21~ N

IN

IN

F yyy ⊕⊗= (27)

The fusion rules mentioned for detail coefficients at each decomposition level are appropriately adapted via the PLIP isomorphic transformation. An analysis of (20) and (27) yields a naïve interpretation of the effect of γ on fusion results. Practically, γ can be interpreted as a brightness parameter, where negative values of γ yield brighter fusion results and positive values of γ yield darker fusion results. Note that this is accomplished while also maintaining the fusion identity that the fusion of identical source images is the source image itself. However, the impact of the parameterization is not limited to this observation as γ impacts the wavelet decomposition as well as the detail coefficient fusion rule.

V. QUANTITATIVE IMAGE FUSION QUALITY ASSESSMENT To assess the quality of fusion results quantitatively, the

Piella and Heijmans image fusion quality metric [18] is used. This metric has been developed based on the fact that a fusion result should somehow relate back to the original source images. The structural similarity measured using Bovik’s quality index [22] is used to relate the fused result to its original source images. The quality index Q0 proposed by Bovik to measure the similarity between two sequences x and y is given by

22220

22

yx

yx

yx

yx

yx

xyQσσ

σσ

μμ

μμ

σσ

σ

+⋅

+⋅= (28)

where σx and σy are the sample standard deviations of x and y, respectively, σxy is the sample covariance of x and y, and µx and µ are the sample means of x and y, respectively. For two images I and F, a sliding window technique is utilized to calculate the quality index Q0(I, F|w) for local wxw windows. The average of these quality indexes is used to measure the similarity between I and F, and is given by

( ) ( )wFIQ

WFIQ

Ww

|,1, 00 ∑∈

= (29)

The measure ranges from 0 to 1, with two identical images yielding a Q0 equal to 1. Defining s(I|w) as the saliency, and in the case of their particular implementation, the variance of the image I in local window w, the quality of the fused result can be assessed by first calculating local weights λ(w) for the source images I1 and I2, given by

( ) ( )

( ) ( )wIswIswIs

w||

|

21

1

+=λ

(30)

and then calculating the fusion quality index Q(I1,I2,F) for the fused result F by

( ) ( ) ( ) ( )( ) ( )( )∑

−+=Ww

wFIQwwFIQwW

FIIQ |,1|,1,, 201021 λλ (31)

The metric assesses fusion quality by calculating the local quality indexes between the fused image and the two source images, and weighting them according to the local saliency between the source images. To better reflect the human visual system, another weight is added to give more weight to regions in which the saliency of the source images is greater. Defining the overall saliency of a window C(w) by

( ) ( ) ( )( )wIswIswC |,|max 21= (32)

The weighted fusion quality index QW(I1,I2,F) is given by

( ) ( ) ( ) ( ) ( )( ) ( )( )∑∈

−+=Ww

w wFIQwwFIQwwcFIIQ |,1|,,, 201021 λλ (33)

where

( ) ( )( )∑

=

Ww

wCwCwc

'

' (34)

For the resulting measure, higher quality indexes imply better fusion results.

VI. EXPERIMENTAL RESULTS The effectiveness of the proposed image fusion algorithms

using the PL-DT-CWT is illustrated by comparing them with image fusion algorithms using the standard DT-CWT. Image fusion results using the standard DWT and standard SWT are also included for reference purposes and as an experimental means of showing the overall improvement achieved by the PL-DT-CWT approach. The algorithms were tested over a range of different image classes which may be particularly interest for homeland security applications, including visible/infrared image pairs for helicopter navigation and surveillance. There are many factors which influence wavelet-based image fusion results, including the type of wavelet transform, the number of decomposition levels, the choice of filter bank, and the fusion rule used to fuse coefficients at each scale. This paper emphasizes the transform which is used while keeping all other factors constant. In these experimental results, 3-level decompositions are performed for all methods, and both the absolute maximum and Burt’s detail coefficient fusion rules are examined. For the DWT and SWT approaches, biorthogonal 2.2 filters are used. For the DT-CWT and PL-DT-CWT approaches, biorthogonal Antonini filters are used for the first stage and q-shift 6-tap filters are used for the remaining stages of the decomposition. The fusion results are compared quantitatively by first normalizing source images and fused results to the range 0-255, and then using the Piella and Heijmans image fusion quality metric QW with w = 7. This metric is used to determine the optimal parameter value for γ,

300

Page 6: [IEEE 2010 IEEE International Conference on Technologies for Homeland Security (HST) - Waltham, MA, USA (2010.11.8-2010.11.10)] 2010 IEEE International Conference on Technologies for

with the resulting fused image thereby taken as the result using the PL-DT-CWT.

A portion of these image fusion results are shown in Fig. 3 and 4. Qualitatively, it is seen that the image fusion algorithms using the PL-DT-CWT outperform standard approaches, yielding fusion results with better contrast. In Fig. 3, the proposed image fusion algorithms are able to better capture the terrain information from Fig. 3a and the road information from Fig. 3b in the fused result. Similarly, in Fig. 4, the contrast between the trees in Figure 4and the person is more visually pleasing in the fusion result using the proposed PL-DT-CWT. These observations are reflected by the quality metric values in Tables 1 and 2. In all cases, fusion algorithms using the PL-DT-CWT outperformed the other approaches quantitatively.

VII. CONCLUSIONS In this paper, the DT-CWT was combined with the PLIP

model, yielding the new PL-DT-CWT for image analysis. Consequently, image fusion algorithms for the DT-CWT were reformulated for the PL-DT-CWT, yielding a new class of image fusion algorithms. The new image fusion algorithms using the DT-CWT were compared to image fusion algorithms using the DWT, SWT, and DT-CWT. Experimental results showed that the image fusion algorithms using the PL-DT-CWT yielded superior image fusion results both qualitatively and quantitatively.

ACKNOWLEDGMENT The authors would like to thank Dr. Alex Toet of the TNO

Human Factors Research Institute and Dr. Oliver Rockinger for kindly providing the registered images used for computer simulations, and Dr. Nick Kingsbury for kindly contributing the DT-CWT toolbox for MATLAB.

TABLE I. QUALITY METRIC VALUES FOR “NAVIGATION”

Fusion rule Decomposition

Absolute maximum Burt’s method

DWT 0.7363 0.7333

SWT 0.7460 0.7446

DT-CWT 0.7605 0.7589

PL-DT-CWT 0.7863 0.7781

TABLE II. QUALITY METRIC VALUES FOR “TREES”

Fusion rule Decomposition

Absolute maximum Burt’s method

DWT 0.7576 0.7239

SWT 0.7746 0.7604

DT-CWT 0.7715 0.7452

PL-DT-CWT 0.8009 0.7854

REFERENCES [1] M. Kumar and S. Dass, “A total variation-based algorithm for pixel-

level image fusion,” IEEE Trans. on Image Proc., vol. 18, no. 9, pp. 2137-2143, Sept. 2009.

[2] P. Hill, N. Canagarajah, D. Bull, “Image fusion using complex wavelets,” British Machine Vision Conference, pp. 487-496, 2002.

[3] Z. Zhang and R. Blum, “A categorization of multiscale-decomposition-based image fusion schemes with a performance study for a digital camera application,” Proc. IEEE, vol. 87, no. 8, pp. 1315–1326, Aug. 1999.

[4] S. Zhang, P. Wang, X. Chen, and X. Zhang, “A new method for multi-source remote sensing image fusion,” IEEE Inter. Geoscience and Remote Sensing Symposium, vol. 6, pp. 3948-3951, Nov. 2005.

[5] Z.X. Qiong, G. Z. Sheng, Y.H. Zhao, “Dynamic Infrared and Visible Image Sequence Fusion Based on DT-CWT using GGD,” Inter. Conf. on Comp. Sci. and Info. Tech., pp. 875-878, Sept. 2008.

[6] P.S. Chavez and A.Y. Kwarteng, “Extracting spectral contrast in Landsat Thematic Mapper image data using selective component analysis,” Photogrammetric Engineering and Remote Sensing, vol. 55, no. 3, pp. 339-348, 1989.

[7] S. Daneshvar, H. Ghassemian, “MRI and PET image fusion by combining IHS and retina-inspired models,” Information Fusion, vol. 11, no. 2, pp. 114-123, April 2010.

[8] M. Tabb and N. Ahuja, “Multiscale Image Segmentation by Integrated Edge and Region Detection,” IEEE Trans. on Image Proc., vol. 6, no. 5, May 1997.

[9] S.G. Mallat, “A theory for multiresolution signal decomposition: the wavelet representation,” IEEE. Trans. Pattern Anal. Machine Intell., vol. 11, pp. 674-693, July 1989.

[10] O. Rockinger, “Image Sequence Fusion Using a Shift-Invariant Wavelet Transform,” IEEE International Conference on Image Processing, pp. 288-291, 1997

[11] W. Selesnick, R.G. Baraniuk, and N.G. Kingsbury, “The Dual-Tree Complex Wavelet Transform,” IEEE Sig. Proc. Mag., vol. 22, no. 6, pp. 123-141, Nov. 2005.

[12] M. Jourlin and J.C. Pinoli, “Logarithmic image processing: the mathematical and physical framework for the representation and processing of transmitted images,” Advances in Imaging and Electron Physics, vol. 115, pp. 126-196, 2001.

[13] G. Deng, L.W. Cahill, and G.R. Tobin, “The study of logarithmic image processing model and its application to image enhancement,” IEEE Trans. on Image Proc., vol. 18, pp. 1135-1140, 2009.

[14] C. Vertan, A. Oprea, C. Florea, and L. Florea, “A pseudo-logarithmic image processing framework for edge detection,” Advanced Concepts for Intelligent Vision, vol. 5259, pp. 637-644, 2008.

[15] K. Panetta, E. Wharton, S. Agaian, “Human Visual System Based Image Enhancement and Logarithmic Contrast Measure,” Trans. on Systems, Man, and Cybernetics, Part B: Cybernetics, vol. 38, no. 1, pp. 174-188, Feb. 2008.

[16] E. Wharton, K. Panetta, and S. Agaian, “Logarithmic edge detection with applications,” IEEE Inter. Conf. on Systems, Man, and Cybernetics, pp. 3346-3351, Oct. 2007.

[17] E. Wharton, S. Agaian, and K. Panetta, “Comparative study of logarithmic enhancement algorithms with performance measure,” SPIE Electronic Imaging: Algorithms and Systems, Neural Networks, and Machine Learning, vol. 6064, 2006.

[18] G. Piella, H. Heijmans, “A new quality metric for image fusion,” International Conference on Image Processing, vol. 2, 2003.

[19] G. Piella, “A general framework for multiresolution image fusion: from pixels to regions,” Information Fusion. Vol. 4, pp. 259-280.

[20] P.J. Burt and R.J. Kolczynski, “Enhanced image capture through fusion,” International conference on Computer Vision, pp. 173-182, 1993.

[21] G. Courbebaisse, F. Trunde, and M. Journlin, “Wavelet transform and LIP model,” Image Anal. Stereo., vol. 21, no. 2, pp. 121-125, June 2002.

[22] Z. Wang and A.C. Bovik, “A universal image quality index,” IEEE Signal Processing letters, vol. 9, no. 3, pp. 81-84.

301

Page 7: [IEEE 2010 IEEE International Conference on Technologies for Homeland Security (HST) - Waltham, MA, USA (2010.11.8-2010.11.10)] 2010 IEEE International Conference on Technologies for

(a) (b)

(c) (d)

(e) (f)

(g) (h)

(i) (j)

Figure 3. (a)(b) Original “navigation” source images, fusion results using DWT and (c) absolute maximum, (d) Burt’s fusion rules, SWT and (e) absolute maximum (f) Burt’s fusion rules

DT-CWT and (g) absolute maximum and (h) Burt’s fusion rules, PL-DT-CWT with (i) absolute maximum and (j) Burt’s fusion rules

(a) (b)

(c) (d)

(e) (f)

(g) (h)

(i) (j)

Figure 4. (a)(b) Original “trees” source images, fusion results using DWT and (c) absolute maximum, (d) Burt’s fusion rules,

SWT and (e) absolute maximum (f) Burt’s fusion rules DT-CWT and (g) absolute maximum and (h) Burt’s fusion rules,

PL-DT-CWT with (i) absolute maximum and (j) Burt’s fusion rules

302