Transform Domain Measure of Enhancement – TDME – for Security Imaging Applications

Arash Samani, Karen Panetta Department of Electrical and Computer Engineering

Tufts University Medford, MA, USA

arash.samani@tufts.edu, karen@ece.tufts.edu

Sos Agaian Department of Electrical and Computer Engineering

University of Texas at San Antonio San Antonio, TX, USA

sos.agaian@utsa.edu

Abstract— Today, many security applications rely on imaging sensors. However, the quality of the captured image is highly susceptible to environmental lighting conditions such as poor or non-uniform illumination. Security imaging systems rely on efficient real-time image enhancement. For autonomous systems, determining and producing the best visually enhanced image output as perceived by the human visual system remains a challenge. In this paper, we present a metric for autonomous evaluation that will enable security systems to automatically determine the best human visual quality image. To achieve this, we have established a “no parameter no reference” metric that can determine the best visually pleasing image. The metric is algorithm independent such that it can be utilized for a diversity of enhancement algorithms. We present our DCT transform domain measure of enhancement (TDME). Unlike spatial domain measure of enhancement methods, the proposed measure is independent of image attributes and does not require any parameter selections to operate. The proposed measure is applicable to compressed and non-compressed images and can be used as an enhancement metric in conjunction with many different image enhancement methods.

Keywords- measure of enhancement; image contrast; image quality; Discrete Cosine Transform (DCT); compressed images; JPEG; imaging security system

I. INTRODUCTION Quantitatively assessing the improvement in image quality

after applying an enhancement process is difficult to measure. A processed image can be said to be enhanced over the original image, if it allows the observer to better perceive the desirable information in the image. However, the perception of the image’s improvement with respect to the human eye is difficult to qualify. Furthermore, there is no universal measure which can specify the both objective and subjective validity of the enhancement method [1]. There have been different definitions of an adequate measure of performance based on contrast [2] [3] [4] [5], with the purpose to provide feedback to the image enhancement processes in order to find the optimum enhancement parameters. Contrast based measure of enhancement methods can be categorized as either, spatial or transform based domain measures. The spatial domain measures are calculated based on luminance of pixels in different parts of an image, but the transform domain measures

work on the Discrete Cosine Transform (DCT), Discrete Fourier Transform (DFT), or Discrete Wavelet Transform (DWT) of the image [1].

Most spatial domain measures are derivatives of the Weber-Fechner Law, Michelson contrast measure [6], or Contrast Ratio (CR), using statistical analysis to better evaluate the image contrast enhancement. The EME, EMEE, AME, AMEE, LogAME and LogAMEE are examples of such spatial domain measures [1] [7] [8]. The performance of spatial domain measures are highly dependent on image attributes, therefore the choice of a measure of enhancement method could affect the results of image enhancement algorithms. To choose the proper spatial domain measure of enhancement, either a visual evaluation, or an automated process should be performed on an image prior to enhancement to determine the image attributes and properties. These properties such as periodic patterns, texture, randomness, uniform background, single or multiple targets, etc. would affect the measures outcome. If a measure is not designed to handle a specific image attribute properly, that measure won’t provide a good metric for the contrast enhancement for that specific image.

The second category of measure of image enhancement is the transform domain measures. These measures are calculated by analyzing the transform of the image (DCT, DFT, DWT, etc.) [2] [9] [10] [11]. Moving away from the measures that operate in spatial domain would provide some advantages. Firstly, most transform domain processes (DCT, DFT, DWT, etc.) are less dependent on image attributes. An image attribute that is present at a certain part of an image would be scrambled in the transform of image based on its frequency contents [12]. Secondly, these measures reduce the process time for the enhancement systems that operate in transform domains since the image transform has already been computed during the enhancement process [1] [2] [10] [11] [13]. Thirdly, when using a time domain measure for enhancement systems that operate in a transform domain, one needs to perform an inverse transform in order to evaluate the enhancement. In most discrete/digital systems these transforms and the inverse operations would produce artifacts in the image; hence the employed measure would be exposed to these artifacts [12].

978-1-4799-1535-4/13/$31.00 ©2013 IEEE 265

II. CONCEPT

A. Basic Implementation for Non-Compressed Images Our proposed TDME measure (Transform Domain

Measure of Enhancement) is based on the idea that enhancing the contrast of an image would create more high frequency content in the enhanced image compared to the original image. In other words, if considering a block of k-by-k in the lower frequency section of the image, after contrast enhancement some of the energy from this region will be transferred to the higher frequency region of the 2D spectrum as explained in Fig. 1. In Fig. 2, (a) shows the DCT transform of an image and (b) shows the DCT transform after image was enhanced using unsharp-masking [12], which demonstrates the spread of energy to the higher frequency region of the image spectrum.

Figure 1. The enhancement process causes energy transfer from low to high

frequency region of the image DCT (higher order coefficients)

Figure 2. (a) DCT Transform of original image, (b) DCT Transform of

enhanced image using unsharp masking; (b) shows more spread of energy in higher frequency region

The process starts by taking a 2-D Discrete Cosine Transform of a grayscale image. A sliding block of k-by-k pixels (k=1, 2,…, minimum of rows and columns) is used to separate the low frequency and the high frequency region of the image. The average magnitude of DCT coefficients for the high frequency region normalized by the average magnitude of DCT coefficients for the entire image would calculate the TDME measure for the current sliding k-by-k block.

1 2 ∑ ∑ ,1 ∑ ∑ ,11 1

where the numerator denotes the average of magnitude in the high frequency region and the denominator shows the average magnitude over the entire 2-D spectrum. The overall TDME measure is the average of TDME’s for each sliding k-by-k block.

1min , , 2

B. Compressed Images In the case of any DCT-based image compression standard,

such as JPEG, JPEG 2, and H.261, the TDME measure as introduced earlier would face a challenge of high frequency artifacts that are created as the results of the compression process. In a JPEG system, the encoder divides the original image into non-overlapping 8x8-pixel blocks (tiles) and the compression process (transform and encoding) is performed for each block separately. This lossy compression process changes the quantization of DCT coefficients in order to lower the bitrate required to encode the coefficients [12]. The compressed image would show discontinuity in image luminance at the edges of these tiles, which would translate to high frequency contents in DCT domain.

The TDME measure as described earlier would interpret these high frequency artifacts as detailed image contents or enhancement in contrast and return erroneous assessment of image contrast. To prevent this effect, we divide the image into non-overlapping m-by-m-pixel blocks (tiles) and performed the 2D DCT transform over each m-by-m block. The TDME is measured for each block and then averaged over the entire image. In most compression methods such as JPEG, JPEG 2 and H.261, m=8.

(a)

(b)

266

Figure 3. TDME generator algorithm process

For an m-by-m block-n, the TDMEn is calculated as follow: 1 1

12 2 ∑ ∑ ,1112 ∑ ∑ ,1111

11 3

1N N 4

where N is the number of non-overlapping tiles (m-by-m-pixel blocks) in the image.

In our experiments we used 8x8-pixel blocks (m=8) for each TDMEn to match the tiling that was used in compression methods in images from LIVE Image Quality Assessment Database [14].

The computation of the global TDME by averaging TDME for the non-overlapping tiles is deemed necessary for compressed images. We observed in our tests that TDME returns an accurate measure of contrast enhancement when it was applied to a sequence of images that were iteratively distorted with different distortion methods, as shown in the next section. Our tests were compared to the Mean Opinion Score, MOS, provided by the LIVE Image Quality Assessment Database [14]. Although it is not necessary to divide non-compressed images into 8x8 non-overlapping sub-images when using the basic TDME, we choose to apply a single unified implementation of TDME that is suitable for all image formats. With this approach, the TDME’s result agreed with MOS for iteratively distorted images in both non-compressed and compressed formats.

C. Comparison with Tang’s contrast enhancement method Since the JPEG compression process is operated on non-

overlapping blocks of 8x8 pixels of the image [12], it is

necessary to study the contrast enhancement on each block separately for the transform domain measures due to the high frequency content introduced at the edges between the tiles. These high frequency contents are not a result of having high contrast in the image, but are resulting artifacts of compressing each block separately and creating discontinuity between the tiles. Tang et al. proposed a DCT based image enhancement algorithm which uses a contrast measure to scale the contrast in an original image [10] [11]. They follow a similar concept to compute the DCT of each 8x8 block separately and compute a 15-element array contrast based on the DCT coefficients. Although Tang et al. didn’t directly propose a measure of image enhancement in [10] [11], for our work we derived a process to use an intermediate step of their contrast enhancement method to construct such a measure. We used a reverse process to measure the contrast enhancement as the ratio of the contrast arrays in each 8x8 block between the original image and the enhanced image then averaged over all the blocks in entire image.

There is a similarity between our proposed Transform Domain Measure of Enhancement and the measure derived by Tang [10] in dividing the image into 8x8 non-overlapping blocks and using DCT coefficients to compute a contrast array for each block, however, the use of DCT coefficient to calculate contrast array in our method is fundamentally different. The main difference is that in Tang’s measure all the coefficients are equally weighted, but in our proposed TDME the higher order coefficients (higher frequency contents) weigh more in the final measure. In an 8x8 block, the DCT magnitude associated with the first coefficient (k=1) is added once to TDMEn and the 2nd coefficient for k=2 is added twice and so on. Here are some of the differences between the two methods:

Tang’s method uses 15 radial frequencies (diagonal in the 8x8 block) and TDME uses 8 Cartesian frequencies therefore it yields to less computations.

Our proposed measure is normalized and it always returns a value between 0 and 1. This prevents saturation of the parameter in simulations or the implementation in image enhancement devices.

Tang’s method requires a reference image, therefore in that method the original image has to be processed again with every enhanced version of the test image (computationally extensive) or a copy of the contrast array should be kept in the memory for comparison (memory extensive). In our proposed method, we assign a global measure value to each image which makes it more suitable for real world implementations.

In Tang’s method all the frequencies are scaled equally, but in our proposed measure we put more emphasis on higher frequency contents.

In security applications it is desired to have real time evaluation of enhanced images and algorithms that utilizes less system resources, memory, or process time. Our TDME measure of image enhancement is a no-reference, no-parameter measure which makes it optimal for real-time autonomous security imaging applications.

Image Divide image into blocks of m‐by‐m

pixels

Find the DCT coefficients for each block

Create a segment of K‐by‐K pixels in each block, K=1:m‐1

Add the DCT magnitude outside of K‐by‐K segment

normalized by the total DCT magnitude of the block

Average the magnitude ratio for all the segments to get the TDME for the m‐by‐m

block

Average TDME over all the m‐by‐m blocks in the image for

the overall TDME of the image

267

(a) Original Image

(b) MOS=4.423, TDME=0.3206, Tang=0.7004

(c) MOS=3.308, TDME=0.2223, Tang=0.6042

(d) MOS=2.462, TDME=0.1672, Tang=0.5276

(e) MOS=1.462, TDME=0.1489, Tang=0.4251

(f) MOS=1.115, TDME=0.1421, Tang=0.4326

Figure 4. Image distortion using Gaussian blur, (a) original image, (b) to (f) distorted images

III. EXPERIMENTAL RESULTS We tested our proposed TDME measure against images

from the LIVE Image Quality Assessment Database [14] and compared the TDME results with MOS. A similar comparison was made for the measure derived from Tang’s contrast enhancement method [10]. In most cases, our proposed measure returned comparable or better results and showed better correlation with MOS. All measures and MOS were normalized to provide a fair comparison. All the color images were converted to grayscale prior to evaluation of image contrast measure.

A. Non-Compressed Image Fig. 4 shows an example of an image from [14] with

different degrees of distortion using Gaussian blur. Fig. 5 shows how TDME measure monotonically decreases as the image contrast reduces due to applying Gaussian blur, which agrees with MOS.

Figure 5. The TDME measure monotonically decreases as the image contrast

reduces by applying Gaussian blur to images in Fig. 4

268

(a) Original Image

(b) MOS=4.577, TDME=0.6655, Tang=0.9483

(c) MOS=4.154, TDME=0.4615, Tang=0.7282

(d) MOS=3.192, TDME=0.3869, Tang=0.7644

(e) MOS=2.346, TDME=0.1982, Tang=0.791

(f) MOS=1.192, TDME=0.1531, Tang=0.5985

Figure 6. Image distortion using JPEG2000 compression, (a) the original image, (b-f) the compression is increasing to distort the image quality

B. Compressed Image This example shows an image from [14] that was

distorted using JPEG2000 compression. In Fig. 6, compression increases from (b) to (f). As shown in Fig. 7, the evaluation from our proposed measure, TDME, agrees with MOS of the test images and the graph shows that TDME is monotonically decreasing for images b to f (indices 1 to 5). In comparison, Tang’s method does not agree with the visual assessment of test images.

C. Comparison with MOS In this section we show a comparison with MOS along

several images from [14]. In Fig. 8 each data point from the graph is the correlation of the measure of enhancement (TDME or Tang’s) with MOS for a five-image sample set. The closer the values are to 1, means more the measure agrees with MOS. On a single graph for each sample set, as shown in Fig. 5 and Fig. 7, the slope between two sample points indicates the amount of distortion added from one image to the next. If the slope between two samples in the measure is close to the slope between two samples of MOS, it means the measure closely correlates to MOS.

Figure 7. Shows by increasing the image compression (Fig. 6 b-f) the TDME measure monotonically decreases, thus agreeing with MOS.

269

Figure 8. Shows the correlation between the measures of enhancement and MOS over different sample sets verifying that TDME highly correlates to

MOS.

IV. OBSERVATIONS We evaluated the TDME measure against a variety of test

images with different image attributes (texture, randomness, uniform background, periodic patterns, etc.) from the LIVE Image Quality Assessment Database [14]. As shown in Fig. 8, our TDME measure displays a good correlation with the mean opinion score regardless of the image attributes. It also, on average, performs better than the other commonly known transform domain measure derived from Tang’s contrast enhancement method. Unlike the spatial domain measures, the performance of TDME proved to be independent of image attributes. The TDME measure was also successfully tested on compressed and raw images against different sources of distortion such as different compression methods (JPEG, JPEG2000, LAR), Gaussian blurriness, white noise, etc.

V. CONCOLUSION Most security systems require enhancement of low

quality images, especially for improving contrast in underexposed images. Almost all enhancement processes require parameter selection and optimization, thus a reliable measure of enhancement would provide feedback for the

enhancement process to help select and optimize such parameters.

In this article we introduced a DCT domain based contrast enhancement measure TDME, which is independent of the image attributes unlike the spatial domain based measures. This method does not require a reference, nor has parameters to select or optimize, which makes it suitable for real time autonomous security system imaging applications.

REFERENCES [1] S. S. Agaian, K. Panetta, and A. M. Grigoryan, "Transform-based

image enhancement algorithms with performance measure," Ieee Transactions on Image Processing, vol. 10, pp. 367-382, Mar 2001.

[2] S. S. Agaian, B. Silver, and K. A. Panetta, "Transform coefficient histogram-based image enhancement algorithms using contrast entropy," Ieee Transactions on Image Processing, vol. 16, pp. 741-758, Mar 2007.

[3] A. B. a. A. L. Negrate., "Contrast Enhancement Technique Based on Local Detection of Edges," Computer Visualization of Graphic Image Processes, vol. 46, pp. 162-274, 1989.

[4] J. M. P. J. K. Kim, K. S. Song, and H. W. Park. , "Adaptive Mammographic Image Enhancement using First Derivative and Local Statistics," IEEE Tran. Medical Imaging, vol. 16, pp. 495-502, 1997.

[5] R. B. P. W. M. Morrow, R. M. Rangayyan, and J. E. L. Desautels., "Region-Based Contrast Enhancement of Mammograms," IEEE Tran. Medical Imaging, vol. 11, pp. 392 - 406, 1992.

[6] A. A. Michelson, "Studies in Optics," Chicago, IL: Univ. Chigaco Press, 1927.

[7] E. Wharton, K. Panetta, S. Agaian, and Ieee, "Human Visual System Based Similarity Metrics," in 2008 Ieee International Conference on Systems, Man and Cybernetics, ed, 2008, pp. 685-690.

[8] K. A. Panetta, E. J. Wharton, and S. S. Agaian, "Human visual system-based image enhancement and logarithmic contrast measure," Ieee Transactions on Systems Man and Cybernetics Part B-Cybernetics, vol. 38, pp. 174-188, Feb 2008.

[9] E. Peli, "Contrast In Complex Images," Journal of the Optical Society of America a-Optics Image Science and Vision, vol. 7, pp. 2032-2040, Oct 1990.

[10] J. S. Tang, E. Peli, and S. Acton, "Image enhancement using a contrast measure in the compressed domain," Ieee Signal Processing Letters, vol. 10, pp. 289-292, Oct 2003.

[11] Q. J. T. Sun, "A new contrast measure based image enhancement algorithm in the DCT domain " presented at the IEEE International Conference on Systems, Man and Cybernetics, 2003.

[12] R. Gonzalez, "Digital Image Processing," 2007. [13] S. Aghagolzadeh and O. K. Ersoy, "Transform Image-Enhancement,"

Optical Engineering, vol. 31, pp. 614-626, Mar 1992. [14] U. Laboratory for Image & Video Engineering ECE, "Subjective

Image Quality Assessment - LIVE Image Quality Assessment Database."

270