DC coefficient restoration technique and its application to image coding

F.-W. Tse, W.-K. Cham and J.Z. Liu

Abstract: In the traditional approach of block transform image coding, a large number of bits are allocated to the DC coefficients. A technique called DC coefficient restoration (DCCR) has been proposed to further improve the compression ability of block transform image coding by not transmitting the DC coefficients but estimating them from the transmitted AC coefficients. Images thus generated, however, have inherent errors that degrade the image visual quality. In the paper, a global estimation DCCR scheme is proposed that can eliminate the inherent errors. The scheme estimates all the DC coefficients of the blocks simultaneously by minimising the sum of the energy of all the edge difference vectors of the image. The performance of the global estimation DCCR is evaluated using a mathematical model and experiments. Fast algorithms are also developed for efficient implementation of the proposed scheme.

1 Introduction

Block transform image coding is one of the common lossy image coding methods. It is used in JPEG and MPEG coding standards. The rationale of transform coding is that a unitary transform is used to decorrelate the image pixel information for efficient quantisation and bit allocation on the transform coefficients individually. In JPEG, for exam- ple, the image is divided into blocks and then transformed into transform coefficients using the unitary transform DCT. The transform coefficients are quantised using uniform quantisers and then encoded using entropy coding such as Huffman coding or arithmetic coding. A large number of bits are used to represent the DC coeffi- cients because significant information is collected in the DC coefficients which are believed to be decorrelated from the AC coefficients by the unitary transform.

Let y ( p , q ) be the transform coefficient in a block. For the extreme case where the block size equals the image size, there is only one DC coefficient y(0, 0) which represents the DC component of the image. When the block size is reduced the DC coefficients increase in number and represent the DC component as well as some low-frequency components. For a zero-mean image, the correlation of the DC coefficient and any AC coefficient, i.e. E[y(O, O)y(p, q)] , p # 0 or q # 0, is normally close to zero. Cham and Clarke [ l ] found that for a block size significantly less than the image size, the DC coefficients and AC coefficients are related despite their small correla- tion. They found that DC coefficients can be restored using a criterion called minimum edge difference (MED) and three causal schemes were proposed [ l ] to restore the DC

0 IEE, 2002 IEE Proceedings online no. 20020613 DOI: 10.1049/ip-vis:20020613 Paper first received 18th June 2001 and in revised form 3 1 st May 2002 The authors are with the Department of Electronic Engineering, The Chinese University of Hong Kong, Hong Kong

272

coefficients using the MED criterion. However, all three schemes produce visual artefacts in the restored images due to accumulation of errors in the DC coefficients. Later causal schemes were proposed to replace the simple DPCM predictor to predict DC coefficients in JPEG image coding [2]. As bits are allocated to code the prediction error, so the error accumulation artefacts are eliminated. Experimental results show that the MED criterion is more capable of exploiting the interblock redundancy of the DC coefficients and results in better compression performance than baseline JPEG.

In this paper, a global estimation scheme is proposed that can restore, in most cases, at least 80% of the DC coefficients without visual artefacts. As a result, we may not need to transmit or store these DC coefficients and so can increase the compression ability of a block-based image coding system such as JPEG. While there is a more advanced coding standard JPEG2000, today the most widely used image compression standard is still JPEG. High-resolution images generated by digital cameras are normally represented in JPEG form. The proposed algorithm can help to further reduce the size of these JPEG images while keeping image visual quality unchanged. It is similar to post-processing in that they both exploit the residue image redundancy or image properties to achieve higher coding efficiency. Postprocessing aims to improve the visual quality, and sometimes the PSNR as well, while keeping the data size unchanged. The proposed method aims to reduce the data size while keeping the visual quality unchanged.

2 DC coefficient restoration using MED criterion

2.1 Minimum edge difference criterion Suppose that an original image having the size Nln x N2n pixels is divided into N1 x N2 blocks with each block having n x n pixels. Let X i , j be the n x n square matrix representing the pixel values of the (i,j)th block where 1 5 i 5 NI and 1 5 j 5 N2. In block transform coding, such as baseline JPEG [3], each block is transformed into the

IEE Proc.-Vis. Imoge Signal Process.. Vol. 149, No. 5, October 2002

transform domain x,, by the two-dimensional separable unitary transform as Y, , = TXl,,T< where T is the n x n unitary transform matrix, such as the DCT. The superscript T means transpose. Let al ,J be the DC coefficient of the (i, j)th block, which can be expressed as

R l =

where xj, ,;(p, q) is the ( p , q)th element of the matrix Xi, i , 1 ( p , q I n.

Let U;, , be the n x n matrix representing the pixel values ofthe (i,j)th block whose DC level is zero. The matrix Uj, is called the AC component of the (i,j)th block. Let u;, ,(p, q ) be the ( p , q)th element of U;, j . Thusxi,Jp, q) = u i , J p , p ) +

Define the edge difference vector d l , i , j between the ai,, j ln.

adjacent blocks Xj,j-l and X;., as

0 ' . .. -1 0

where 5 I, ,,,(k) = ul,,- I (k, n ) - ul, J(k, 1). Similarly the edge difference vector d2, 1 , , between the adjacent blocks Xl- I, and Xl ,J is defined as

R2 =

where t2, ,,,(k) = ~ ~ - 1 , j (n , k ) - u,,,( I , k). DC coefficients of an image can be restored from its AC component if an image does not have very high activity [I]. Restoration is possible because most of the blocks satisfy a criterion called minimum edge difference (MED). The MED criter- ion is used to estimate the DC coefficient of each block from the AC component by minimising the sum of the energy of the edge difference vectors between adjacent blocks. Mathematically, it suggests that the value of

tends to minimise Ild~,;, ;112 + lld~,;,jIl2 + lld~,~,,,+ I 1 1 2 + Ild2,;+ l,.i)12, where 1 1 0 1 ) is the L2 norm. The MED criterion is based on the fact that the block transitions are usually smooth in natural images. Thus the sum of the energy of the edge difference vectors along the block boundaries is usually small.

In block transform image coding, the redundancy iden- tified by the MED criterion has been used to recover blocks corrupted by channel errors by error concealment techni- ques [4, 51. In applications where channel errors are eliminated by network protocols or advanced error correction schemes, we can exploit this redundancy to achieve higher compression. DC coefficients are chosen to be discarded and then restored because they occupy the most bits among all coefficients, which therefore facilitates the highest data compression.

0 . . ' . -1 0

2.2 Global estimation scheme Previous schemes using the MED criterion estimate the DC coefficients sequentially. As a result, estimation errors of the DC coefficients accumulate during the progress of the estimation and the restored images suffer from observable blocking effect. To solve the error accumulation problem, a global estimation scheme is proposed to estimate all the DC coefficients of the blocks simultaneously by minimis- ing the sum of the energy of all the edge difference vectors of the image given by

By differentiation with respect to {a;, ,} , the minimisation problem becomes a system of linear equations with N I N2 linear equations which can be written in the form

Sa = b (1)

where a = [al, la1,2 ... aN,,N,]T is the vector of DC coeffi: cients to be restored, b is formed by t{I ,*),; ,Jk) as b = [bi,ibi,2 ... ~ N , , N > ] < where

b;,j I? n

hb(i , j - 1 ) C t ~ , ~ , j ( k ) + d b ( i - 1, j ) C 52,;.j(k) k= I k= I

n

- h&/ -t 1) C Cl,j.,+I(k) - 6,G + 1 ,A k= I

and db(i,j) = 1 if 1 5 i I N l and 1 9 j 5 N 2 , 0 otherwise. S is an N 1 x N , block tridiagonal matrix given by

r R 1 -I 0 " ' 0 1

0 -I Rl

where

O 1 r 2 -1 o ...

0 . . . -l 0 -1 -'1 2 I- .' O 1 r 3 - 1 0 " '

1 0 I.. . ' - l 0 -1 -'1 3

I is the identity matrix, and R1, R 2 , I E DtN2 N2.

The MED criterion may fail at some locations in images, such as the location where the activity around the block boundary is high or there is a step edge along the block boundary. A block selection scheme is suggested to identify blocks whose DC coefficients cannot be estimated accu- rately by the global estimation scheme due to the failure of the MED criterion at these block locations. The quantised DC coefficients of these selected blocks are transmitted to the receiver to estimate the other DC coefficients as the constraints. Mathematically, the block selection scheme modifies the system of linear equations (1) to

Shah = b6 (3)

where Sh is formed from S by crossing out thep(k, 1 )th row and p(k , 11th column simultaneously from S for each (k,l)eIC, where I C C { I , 2, . . . , Nl} x ( 1 , 2, . . . , N2) is the set of indices of the selected blocks whose original DC coefficients are selected for transmission, and p(k, I ) = N2(k - I ) + 1. a h is formed from a by crossing out the p($ 1 )th row of a for each (k, 1 ) E IC. b h is formed from b by adding ak, I to the elements h k , 1-1, b k , [ + I , bnpl, I and b k t l , 1 and then crossing out the p(k , 1 )th row, for each (k, 1 ) E IC. The locations of the selected DC coefficients are

273 IEE Psm-Vis. hinge Signor Proce.s.s, Vol. 149, No. 5, October 2002

transmitted to the receiver using an N 1 x N2 bitmap which is entropy compressed before the transmission.

In block-based transform coding, a blocking effect is usually observable in images compressed at low bit rates because of the severe quantisation of both AC and DC transform coefficients. From the above discussion, one can see that the MED criterion is based on the smooth characteristic of natural images. The MED criterion esti- mates the DC coefficients such that the resulting image is smooth across the block boundaries. Therefore the proposed DCCR scheme can reduce the blocking effect due to large DC coefficient quantisation error during DC coefficient estimation. Simulation results are presented in Section 6.

3 DC coefficient information in block transform image coding

The performance of DCCR is affected by the block size, the block selection of the small portion of DC coefficients for the estimation of other DC coefficients, the adjacent element correlation of the image, and the amount of quantisation error in the AC coefficients. In this Section, we examine these factors in detail.

3. I The effectiveness of DCCR depends on the block size. For simplicity, the square block size n is chosen to be a positive integral power of 2. Let Xi:] be one of the n x n blocks. The superscript indicates the block size explicitly. Smaller blocks can be obtained by dividing the block X!") into four blocks of size ( n / 2 ) x (n/2) , which are X$?<),2j-l, X8!2>,2j, X g ! $ L l , and X$#, respectively, as illustrated in Fig. 1. Denote $](p, q ) as the ( p , q)th transform coeffi- cient of the (i,j)th block, and &] as the DC coefficient of the (i,j)th block. Note that the superscript indicates the block size. If the transform Tis the Walsh transform, it can easily be shown that

DCCR at various block sizes

Equation (4) shows that the DC coefficient u!:) and the three lowest frequency AC coefficients ylr)( 1 , 2), #(2, 1) and y!r)(2, 2) of the n x n block transform coefficients by Walsh transform can be determined from the DC coeffi- cients a2,-1,21-1, ( n / 2 ) uZ?),~,, and &$' of the ( n / 2 ) x (n /2 ) blocks. As a result, the DC coefficient and the three lowest frequency AC coefficients of each n x n

n n

nl2

block of an image can be restored by DCCR with the block size (n /2) x ( n / 2 ) pixels. In other words, under the Walsh transform, the redundancy in the DC coefficient and the three lowest frequency AC coefficients of each n x n block can be removed and then recovered by DCCR with the block size (n /2) x (n /2) pixels.

The original test image is shown in Fig. 2, and Fig. 3 shows the DCCR restored images at various block sizes. DCCR is performed without block selection and the AC coefficients are not quantised. For block sizes 16 x 16 and 8 x 8, the visual quality of the restored images is similar to that of the original image. For block sizes 4 x 4 and 2 x 2, there are obvious blocking effects and contrast reduction in the restored images.

Fig. 4 shows the results of DCCR restored images from 20% of the total number of the DC coefficients selected. In comparison with the results in Fig. 3, the decoded images are significantly improved in both visual quality and PSNR. The blocking effect and contrast reduction of Fig. 3 has disappeared in Fig. 4, even for block size 2 x 2.

3.2 Block selection The experiment above shows the effectiveness of block selection. A large number of DC coefficients can be predicted accurately from a small number of DC coeffi- cients and AC coefficients. One of the major issues is the determination of the number of DC coefficients to be selected. However, the relation between the number of DC coefficients selected and the quality of DCCR restored images is too complex to formulate. It depends on the nature of the images and also the method of block selection used. Experiments were performed to determine a suitable amount (%) of DC coefficients to be retained. Fig. 5 shows the PSNRs of DCCR restored images from different numbers of DC coefficients at different bit rates using the test image 'Lena'. The block size is 8 x 8 pixels. The coding and block selection algorithms used are proposed in Sections 4 and 5. As shown in Fig. 5, more selected DC coefficients result in higher PSNRs of the restored images in general. However, the graph shows the little further PSNR improvement when selecting more than 20% DC coefficients because the side information becomes more

Fig. 2 Original image 'Lena'

214 IEE Proc.-Vis. Image Signal Process., Vol. 149. No. 5, October 2002

a b

C d

Fig. 3 a 16 x 16 (PSNR=33.48 dB) b 8 x 8 (PSNR = 30.28 dB) c 4 x 4 (PSNR = 25.74 dB) d 2 x 2 (PSNR = 20.62 dB)

DCCR vesults ut various block sizes

significant as the number of DC coefficients selected increases. Fig. 6 shows the images restored by DCCR using various numbers of original DC coefficients at 0.5 bits/pixel. There is no observable difference between the images restored at levels from 20% to 40% of the total number of DC coefficients, while the image restored at 10% slightly inferior in visual quality. The results indicate that the visual quality improvement becomes less signifi- cant after a sufficient number of DC coefficients are selected. This is important because the computational complexity of block selection increases as the number of DC coefficients selected increases. It is suggested that selecting 20% of the total number of DC coefficients is a good balance between restored image quality and compu- tational complexity of the coding algorithm.

3.3 on DCCR In this Section, we analyse the prediction ability of DCCR using a stochastic image model. An ensemble of images

Effect of quantisation and image correlation

IEE Proc -V,s lmuge Signul Process, Vol 149, No 5, October 2002

can be modelled using a simple zero-mean stationary discrete random field having a separable covariance func- tion, nix, between two image pixels at ( p , q ) and (Y, s), which is given by [6]

R,(p, 4; Y, s) = p”’-r’p’q-%; ( 5 )

where p is the correlation coefficient and g; is the variance of the image pixels. For most natural images, p 2 0.9. The previous analysis given in [2] evaluates the prediction ability of DCCR using MED criterion to predict the DC coefficient d i , of the (i,j)th block, given the original DC coefficients ai,j-l and a i p l , and the AC components of three blocks ( i , j - l), (i - 1, j ) and ( i , j ) . For simplicity, the authors ignored all quantisation errors in the transform coefficients and found that the prediction error variance of the MED-predicted DC coefficient is

4 M E D = ai, j - ~i . j>’l

275

C

a b

d

Fig. 4 a 16 x 16 (PSNR=44.30dB) b 8 x 8 (PSNR = 43.43 dB) c 4 x 4 (PSNR = 42.47 dB) d2 x 2 (PSNR=41.80dB)

DCCR results froni 20% of original DC coefficients at various block sizes

0.2 0.4 0.6 0.8 1 .o

bit rate, bitsipixel

Fig. 5 PSNRs (in dB) of the DCCR restored image ‘Lenu’,from various amounts of selected DC coeflcients at different bit rates (in bitslpixel)

216

Here the block size is assumed to be n x n pixels. The performance of DCCR can be expressed using the normalised error variance (NEV), which is defined as tidMED = fs;MED/& where o& is the variance of the original DC coefficient.

In practice, DCCR is performed using quantised trans- form coefficients. We have proved [7 ] that the NEV of the estimated DC coefficient by DCCR from the quantised AC coefficients using MED criterion can be expressed as

where o$ is the increase in error variance of the predicted DC coefficient due to quantisation, which can be expressed as

where e is the quantiser performance factor, which depends on the probability density function of the transform coeffi- cients [8]. R is a parameter used in designing the quantiser

IEE Proc.-Vis. Imrige Signal Process., Vol. 149, No. 5, October 2002

a b

C

All images are coded at 0.5 bits/pixel a 10%; b 20%; c 30%; d 40%

d

Fig. 6 DCCR results.fiom various amounts (99) of' original DC coejficients

for quantising the transform coefficient according to the optimal bit allocation assumption. C T ( p ) is a function which depends on the unitary transform T and the correlation coefficient p only. Moreover, C,(p) 2 0, and lim,,+ , C,(p) = 0. The proof can be found in [7].

Suppose that the transform T is DCT whose transform coefficients are usually modelled using a Laplacian distri- bution, and we have i2=4.5 [SI. Fig. 7 shows a graph plotting (7) at R=0.3, 0.5, 0.7, and 1.0 bit per pixel with respect to the correlation coefficient p . This figure also shows the performance of MED prediction without quan- tisation consideration and the first-order DPCM DC predic- tor used in JPEG without quantisation error in terms of the NEV derived in [2]. Note that R is a quantiser design parameters. As R increases, the total number of bits allocated to the AC coefficients also increases and hence the AC coefficients are coded more accurately. In other words, as R increases, the DCCR prediction is performed from more accurate AC coefficients.

The plot shows that quantisation increases the prediction error. When the correlation coefficient p approaches 1, all the error variances approach zero as expected. Moreover, when the correlation coefficient p is sufficiently large, the performance of MED predictions, even with quantisation

J E E Proc.-Vis. Image Sipol Process.. Vol. 149, No. 5, October 2002

JPEG with quantisation at 0.3 bpp JPEG with quantisation at 0.5 bpp JPEG with quantisation at 0.7 bpp JPEG with quantisation at 1 .O bpp MED without quantisation MED with quantisation at 0.3 bpp MED with quantisation at 0.5 bpp MED with quantisation at 0.7 bpp

..

0.5 0.6 0.7 0.8 0.9 1 .o correlation coefficient p

Fig. 7 Normalised error variances for various predictors

211

errors, is better than that of the DPCM predictor used in JPEG. For most natural images, p is about 0.9. We may conclude that MED prediction, even with quantisation of the transform coefficients, can predict the DC coefficients with smaller error variances. It is shown in Section 6 that DCCR based on the MED criterion favourably operates at low bit rate with visually better decoded images.

4 Global estimation using PCG method

In this Section, we describe the fast algorithm for DCCR using a conjugate gradient (CG) method. CG is an iterative method achieved by constructing the approximate solution in the Krylov subspace using the Ritz-Galerkin approach. The advantages of using the CG method are [9]: (i) it does not require estimation of parameters; (ii) it takes advan- tages of the distribution of the eigenvalues of the iteration operator; and (iii) it requires fewer restrictions on the system matrix for optimal behaviour. The only restriction on the system matrix S is that it is symmetric positive definite. It is believed heuristically that CG converges very fast if the spectral radius KQ x 1. Preconditioned conju- gate gradient (PCG) is a modification of CG in which a preconditioner M is chosen such that K(M-'S) is small or the eigenvalues of M-IS are clustered [ l o ] . For efficient calculation, M should be effectively sparse and so its inverse can easily be computed.

The system matrix Sb in (3) is a block tridiagonal matrix, which can be explicitly written as

(9)

where the ith row of the blocks represents the system of linear equations formed by the ith row of the unknown DC coefficients. We can then apply incomplete Cholesky factorisation on s b to find an approximate matrix M as the preconditioner:

A , = D , (10)

where Ai is a sparse approximation of A i ' . Using Cholesky factorisation of A i , Ai=LiLL where L j is a lower triangular matrix, we obtain a preconditioner M having the form

M =

X

where W , = EjLr-

278

Several preconditioners have been investigated in [l I ] and MINV(1) was the best one among them, and is given by

A , ( j , j ) = ( D , - E,B(AL1l , 1W?)(j , j) N2

- AL'i(j, I) (15) / = I

I # b J * l l

Here A ( i , j ) is the (i,j)th element of the matrix A . B(A, p ) is the banded matrix of the 2p + 1 main diagonals of A . The MINV( 1) preconditioner uses the property that the condition number of M-'SI, is improved if M is chosen such that each row sum is equal to that of sb [12] . The overall computation involves (1 /2)(N2 - 2)(N, - 1) + 10(N2 - 1 ) flops and 2N2 storage for each A , [ l l ] .

5 Block selection algorithm

The block selection scheme is used to select a portion of the original DC coefficients at the locations where the MED criterion fails. In this section, a fast algorithm for block selection is described.

5.1 Greedy algorithm approach Suppose that 0 5 n, 5 NINz blocks are to be selected. Let IC be the set of indices of the n, selected blocks. Since the DC coefficients in a are arranged in lexicographical order, we label the DC coefficients ai,, as a(k), where k = N2i +j, 1 5 k l N 1 N 2 . We have expressed the block selection scheme using (3) in Section 2.2 by taking the selected DC coefficients out of vector u in (1) to form ab. Alter- natively, we may let the selected DC coefficients in vector a be zero, and consequently let the corresponding row and column in S be ze_ro, and modif vecLor b to 6'. Define the selection matrix P E niNIN2 N1d as P(i , j ) = 1 if i = j and i IC, 0 otherwise. Then the ideal block selection solution is to find the set IC such that

(PSP)(Pa) = Pb' and llpa - Paorig 11 is minimum

where aorig is the vector of original DC coefficients and 6' is calculated according to IC. b' is the vector calculated from b by adding ai,,i to the elements bj, ,- , , bj, ,+ and b;,,, and then setting the Mh row to zero, for each (i,j) E IC. Obviously, the minimisation is found from

bj- l ,

( Np) possible solutions. We propose a greedy algorithm approach to solve this problem, as given in algorithm 1 . In algorithm I , C(.,.,.) represents the numerical method used to solve the system of linear equations, such as the PCG method discussed in Section 4. For example, given A and 6, C(A, x, b) finds the value of x such that A x = b . Most of the computations come from step 3 of algorithm 1 . Although the PCG method is used, the speed of algorithm 1 is still far from suitable for practical applications.

Algorithm 1: The block selection algorithm based on a greedy algorithm. - Step 1. Initialise P = I , IC = 4, and b' = b. Step 2 . while less than n, blocks are selected, i.e. Card(lC) < n, do

IEE Proc.-Vis. h u g e Signul Process., Vol. 149, No. 5, October 2002

Step 3 . Step 4. maximised. Step 5 . K+Ku$). Step 6. Step 7. end while

Solve-C(PSP, Pa, kb’) for Pa. Find k=value of k for which la(k) - ao“g((k)I is

Update P and 6’ according to the new K.

5.2 Alternative interpretation We first write down the equations in the system o f linear equations (l),

n

Subtracting a;‘? from both sides, it can be shown that

where e j , j=a j , , - a;‘? is the error of the estimated DC coefficient.

As a result, the DCCR problem can be rewritten in terms of the estimation errors of the DC coefficients as

Se = d (18) where e = [e1,1e1,2 ... eN,,NZ]Tis the vector of the estimation errors and d is the column vector having the components E$= ~ d l , ;,,(k) and E$= ld2, ;,;(k). The selection of the DC coefficient of the (i,j)th block is equivalent .to setting the corresponding error ei,/ to zero. Using (18), the block selection algorithm 1 can be modified to algorithm 2. Note that the update of the vector d’ in line 6 of algorithm 2 is similar to that of b’ in algorithm 1.

Algorithm 2: Modified block selection algorithm. Step 1. Initialise P = I , K = 4, and d’ = d. Step 2. while less than n, blocks are selected, Le. Card( K ) < n,s do Step 3. Solve C(PSP, Pe, Pd’) for Pe. Step 4. Find E = value of k for which le(k)l is maximised. Step 5 . K ~ K ujk} . Step 6. Update P and d‘ according to the new K. Step 7. end while

Equation (18) can be considered as a discrete version of the continuous two-dimensional Poisson’s equation V2e(x, y ) = d(x, y) , if one uses the centred finite difference approximation subjected to the Neumann boundary condi- tion having zero normal derivative along the boundary [13]. The continuous Poisson’s equation is assumed to be solved in the unit square boundary [0, 11 x [0, I]. The discretisation is performed with mesh spacing

1 N2 - 1

and Ay = ~

1 Ax=-- N l - 1

It is well known that the solution e(x, y ) can be expressed in terms of the linear combination of the Fourier components elnpxelnyy, where i = d(--I) [ 141. For ease of analysis, we use the zero Dirichlet boundary condition instead of the Neumann boundary condition.

IEE Pvoc -VIA Image Signal Procesd , Viol 149, No 5, October 2002

5.3 Fast block selection algorithm In algorithm 2, the major computation is solving the system of linear equations using the algorithm C(.,.,.) in line 3. Instead of obtaining a complete solution in line 3 of algorithm 2, one may use the approximate solution of e such that it is accurate enough for the block selection step in line 4 of algorithm 2. We choose the simple Gauss- Seidel iteration method [15, 161 and analyse its iterations for the discrete Poisson’s equation problem. The following analysis is inspired by Briggs’ book [17].

When studying stationary linear iterations, it is sufficient to work with the homogeneous linear system Se=O and use an arbitrary initial guess to start the iteration scheme. The reason for doing this is that the exact solution e = 0 is known and the error in an approximation e‘ is simply -e‘. Note that the solution of the corresponding continuous Poisson’s equation is elnpxelxqy. Thus the solution of Se = d is also the linear combination of the discrete vectors wp,q = [w I , I w l ,2 . . . wN, ,,I: where wk I = elnpkheinq‘AY, for 1 5 k i N l - 2 and 15 15 N2 - 2. Note that kand Zare the sampling indices introduced by the continuous to discrete transformation while p and q are the harmonic orders. Large p and q mean high-frequency harmonics. For simpli- city, we assume the image has square dimensions, thus N 1 = N2 = N and Ax = Ay = 1 / ( N - 1) = h, and therefore we have wk,l= elnpk’?elnqlh for I 5 k, 1s N - 2. In fact, w ~ , ~ for 1 ~ p , q 5 N - 2 are the eigenvectors of S.

Fig. 8 shows the iteration errors Ile(k)ll, using the harmo- nics p = q = { 1,3,6) as the initial guesses with N = 33. It indicates that the high-frequency harmonics (large p and q) converge faster than the low-frequency harmonics.

The Gauss-Seidel iteration step can be explicitly written as

Denote the iteration error of the element after the kth iteration as E$:]. Therefore E!:] = -e::]. Suppose that the initial guess of the Gauss-Seidel iteration is the ( p , q)th harmonic. Then after k iterations, e!,”’ can be expressed as A ( Q e i W 1.J p( i - 1) + d i - 1)) where A!:] ’is the amplitude of the harmonic after k iterations and AI:] = 1. It can be deduced that the ratio of the iteration error after a single Gauss-Seidel iteration is given by [ 181

1.0 r\

0.4 I\\ 0 20 40 60 80 100

iterations

Fig. 8 number of iterations

Iterative errors for d@rent harmonics plotted against the

279

I 0 0.2 0.4 0.6 0.8 1 .o

lPhl

Absolute iteration error ratios of a single Gauss-Seiciel Fig. 9 iteration ojdifferent harnzonics with p = q

Moreover, if Iphl, lghl >; then u i , j s i . Therefore, the iteration errors of the high-frequency harmonics decay to 0 and converge to the solution quickly after a few Gauss- Seidel iterations. Fig. 9 plots the absolute iteration error ratios uf, after a single Gauss-Seidel iteration of different harmonics with p = q . It shows that the harmonics with high frequencies IphI = 1qhl 2 i decay more quickly than those with low frequencies IphI = lqhl 5 i .

From the above analysis of the Gauss-Seidel iteration, we propose the fast block selection algorithm shown as algorithm 3.

Algorithm 3: Proposed block selection algorithm. Step 1. Formulate the equation Sa = b as Se = d . Step 2 . Solve C(x, e, d) for e. Step 3. Initialise P = I , I C = q 5 , and d’=d. Step 4. while less than I Z , ~ blocks are selected, i.e. Card(K) < iz.? do Step 5 . Obtain an updated value-of e by applying a single Gauss-Stidel iteration to (pSp)(Pe) = Pd’. Step 6. Find = value of k for which le(k)l is maximised. Step 7. I C t K u J k } . Step s. Update P and d’ according to the new IC. Step 9. end while

There are several changes in algorithm 3 in comparison wit! the original algorithm 1 : (i) the problem is formulated as Se = d, instead of solving Sa = b directly (line 1); (ii) a complete solution to Se = d is solved before selection of the blocks (line 2); (iii) a single Gauss-Seidel iteration is used in the proposed algorithm instead of a complete solution found as in the original algorithm prior to the block

selection step (line 5 ) . The last change greatly reduces the computation to select the required number of blocks.

An explanation of the proposed algorithm follows. We know that the errors of the estimated DC coefficients of the unselected blocks should be small. Thus the harmonics used to represent that error vector e mostly have low frequencies. When applying an iteration scheme, such as the Gauss-Seidel iteration, to the system of linear equa- tions Se = d, most of the high-frequency harmonics will decay quickly except those that are necessary to represent the estimation errors due to the vector d. Thus a few Gauss-Seidel iterations are sufficient to separate out the set of DC coefficients having large estimation errors. Therefore the block selection step can be applied without solving the whole problem completely. Note that, when a block is selected, the system of linear equations is changed and some high-frequency components may be added. Therefore the Gauss-Seidel iteration is applied after a block is selected prior to the next block selection so that the unnecessary high-frequency. harmonics introduced during block selection are reduced. To reduce the amount of computation, only a single Gauss-Seidel iteration is applied between two successive block selections. The adequacy of the single Gauss-Seidel iteration is verified through the experiment described in Section 6. The step (line 2) for finding a complete solution at the very begin- ning of the proposed algorithm is used to reduce the amplitudes of the unnecessary harmonics and their inter- ference in deciding the selection in the block selection loop. A fast algorithm such as the PCG approach discussed in Section 4 is a low-cost method to achieve this goal.

6 Simulation results

In JPEG, the DC coefficients and the AC coefficients are encoded separately. Therefore, one can simply ignore the DC coefficients and encode only the AC coefficients. In decoding, the DC coefficients can be restored by the DCCR scheme. Moreover, DCCR can be applied to the JPEG encoded images by simply removing the DC encoded bit stream in the encoded images.

We have implemented a codec based on the JPEG image file syntax. The bitmap representing the retained DC coefficients is compressed using adaptive arithmetic coding [19]. The selected DC coefficients are first quan- tised, and then embedded in the encoded image file along with the compressed bitmap using the APPO marker.

We employ the generalised block-edge impairment metric (GBIM) proposed in [20] to quantify the degree of blocking effect in decoded images. GBIM is based on a human visual system model. Higher GBIM values mean a more severe blocking effect. Table 1 shows the GBIMs of

Table 1: GBlMs of the images decoded by JPEG and DCCR at different bit rates (in bits/pixel)

JPEG DCCR with JPEG

Image 0.25 0.5 0.7 0.25 0.5 0.7

Lena 3.408 1.957 1.668 2.625 1.776 1.568

Peppers 3.537 2.040 1.770 2.741 1.858 1.650

Sailboat 3.586 2.101 1.768 2.882 1.896 1.652

Airplane 3.124 1.841 1.578 2.535 1.668 1.480

Goldhill 3.113 1.895 1.628 2.699 1.756 1.532

280 IEE Proc.-Vis. Image Sigiial Pvocess., Vol. 149. No. 5. Octobes 2002

34 -

0.2 0.3 04 0.5 0.6 07

bit rate, bitslpixel

Fig. 10 and DCCR at dtfeuent bit rates (in bitslpixel)

PSNRs (in dB) of the image Zena’decoded bjj JPEG

images decoded by JPEG and DCCR. The results indicate that DCCR reduces the blocking effect of the decoded images significantly, especially at low bit rates.

Fig. 10 shows the PSNRs of the decoded test image ‘Lena’ encoded by JPEG and by a DCCR that retains 20% of DC coefficients at different bit rates. Results from other test images are summarised in Table 2. At high bit rates, JPEG coded images have higher PSNRs than those coded with DCCR. This is because in DCCR most of the bits allocated to the DC coefficients are removed, and therefore the PSNRs of images decoded by DCCR are inferior. However, at low bit rates, the PSNRs of the images decoded by DCCR are higher than those decoded by JPEG. It is thus deduced that the a priori smoothing model inherited by the MED criterion in DCCR recovers the information lost in quantisation.

The ability of the MED criterion to reduce the blocking affect can be observed from the decoded image results shown in Fig. 11. At low bit rates, the JPEG decoded image suffers from severe blocking effect due to the coarse quantisation in the transform coefficients. The DCCR decoded image, on the other hand, has smoother block boundaries. They are illustrated in the zoomed parts of the test images shown in Fig. 12. In fact, at high bit rates, the images decoded by DCCR are visually similar to those decoded by JPEG, although they have lower PSNR than those decoded by JPEG.

It has been found that the decoding time of DCCR by PCG numerical algorithm using the preconditioner M W ( 1) is about 50% of the JPEG image decoding time [l I]. On the encoder side, the major computation overhead is in the block selection step. Table 3 lists the times used in block

a

b

Fig. 11 Image Zena’ decoded by JPEG and DCCR at 0.25 bitslpixel a JPEG b DCCR

Table 2: PSNRs (in dB) of the images decoded by JPEG and DCCR at different bit rates (in bits/pixel)

JPEG DCCR with JPEG

Image 0.25 0.5 0.7 0.25 0.5 0.7

Lena 31.52 34.88 36.31 31.75 34.54 35.78

Peppers 31.16 34.01 35.13 31.49 33.82 34.81

Sailboat 27.25 30.26 31.62 27.37 29.93 31.10 Airplane 30.36 34.38 36.17 30.48 34.09 35.72

Goldhill 29.23 31.70 32.98 29.17 31.39 32.51

IEE Proc.-Vis. Image Signal Process., Vol. 149, No. 5, October 2002 28 I

a

b

Fig. 12 (shoulder) using JPEG and DCCR at 0.25 bitslpixel a P E G b DCCR

Zoomed parts of the decoded images of image ‘Lena’

selection by algorithms 1 and 3 discussed in Section 5 . They are measured from implementations running in a SUN UltraSparc 1 workstation. They show that the fast algorithm proposed in Section 5 has significant improve- ment in computation time over the original algorithm 1. Moreover, the quality of the images decoded by the fast algorithm is comparable with those decoded by the original algorithm 1.

7 Conclusions

A DC coefficient restoration (DCCR) technique has been evaluated using a stochastic Markov random field image

Table 3: Computation time of block selection schemes and PSNRs of the restored images

Algorithm 1 Algorithm 3

Image Time, s PSNR, dB Time, s PSNR, dB

Lena 51.58 36.69 2.22 36.70

Peppers 47.52 35.61 2.22 35.62

Sailboat 55.50 33.12 2.22 33.12

Airplane 58.85 37.34 2.19 37.29

Goldhill 53.09 34.64 2.22 34.64

model. It is shown that the DCCR using the MED criterion is better able to estimate the DC coefficients from the AC coefficients than the DPCM prediction, even in the presence of quantisation errors. Fast algorithms are designed for both DC coefficient estimation and selection, and they are used in a DCCR application to image coding. These algorithms are then incorporated into JPEG. Experi- ments show that the proposed coding algorithms can reduce the blocking effect and result in better quality of decoded image, especially under low bit rate conditions.

8 References

1 CHAM, W.K., and CLARKE, R.J.: ‘DC coefficient restoration in trans- form image coding’, IEE Proc. F, Commun. Radar Signal Process., 1984,131, pp. 709-713

2 SEE, C.T., LO, K.T., and CHAM, W.K.: ‘Efficient encoding of DC coefficients in transform coding of images using P E G scheme’. Proceedings of ISCAS’91, 1991, Singapore, vol. I, pp. 404407

3 PENNEBAKER, W., and MITCHELL, J.: ‘JPEG still image compres- sion standard’ (Van Nostrand Reinhold 1993)

4 PARK, J.W., and LEE, S.U.: ‘Recovery ofcorrupted image data based on the NURBS Interpolation’, IEEE Trans. Circuits Syst. Video Technol., 1999,9, pp. 1003-1008

5 SUN, H., and KWOK, W.: ‘Concealment of damaged block tranform coded images using projections onto convex sets’, IEEE Trans. Image Process., 1995, 4, pp. 4 7 0 4 7 1

6 JAIN, A.K.: ‘Fundamentals of digital image processing’ (Prentice Hall, 1989)

7 TSE, F.W.: ‘Analysis and design of coefficient restoration in image coding’. PhD thesis, Department of Electronic Engineering, The Chinese University of Hong Kong, June 2000

8 JAYANT, N.S., and NOLL, P.: ‘Digital coding of waveforms: principles and applications to speech and video’ (Prentice-Hall, 1984)

9 CONCUS, P., GOLUB, G.H., and O’LEARY, D.P.: ‘A generalized conjugate gradient method for the numerical solution of elliptic partial differential equations’, in BUNCH, J.R., and ROSE, D.J. (Eds.): ‘Sparse matrix computations’ (Academic Press, New York, 1976)

10 CONCUS, P., GOLUB, G.H., and MEURANT, G.: ‘Block precondition- ing for the conjugate gradient method’, SIAMJ Sci. Stat Comput., 1985,

11 TSE, F.W., and CHAM, W.K.: ‘Numerical methods for DC coefficient restoration’. Proceedings of 1997 Int. Conf. on Information, Commu- nications and Signal Processing, 1997, Singapore, vol. 2, pp. 965-969

12 DUPONT, T., KENDALL, R.P., and RACHFORD, H.: ‘An approximate factorization procedure for solving self adjoint elliptic difference equa- tions’, SIAMJ Numer. Anal., 1968, 5 , pp. 559-573

13 STRAUSS, W.A.: ‘Partial differential equations: an introduction’ (John Wiley & Sons, 1992)

14 BOAS, M.L.: ‘Mathematical methods in the physical sciences’ (John Wiley & Sons, 1983, 2nd edn.)

15 VARGA, R.: ‘Iterative analysis’ (Prentice-Hall, 1962) 16 YOUNG, D.: ‘Iterative solution of large linear systems’ (Academic

6, pp. 220-252

Press. 1971 \ 17 BRiGdS, W.L.: ‘A multigrid tutorial’ (SIAM, 1987) 18 BIRKHOFF, G., and LYNCH, R.E.: ‘Numerical solution of elliptic

problems’ (SIAM, 1984)

for data comoression’. Commun. ACM. 1987. 30. oo. 520-540 19 WITTEN, I.H., NEAL, R.N., and CLEARLY, J.G.: ‘Arithmetic coding

20 WU, H.R., and W E N , M.: ‘A generalized bldik-edge impairment metric for video coding’, IEEE Signal Process. Lett., 1997, 4, pp. 317-320

282 IEE Proc.-Vis. Image Signal Process., Vol. 14Y, No. 5. October 2002