1333 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 4. NO.9, SEPTEMBER [995

REFERENCES

[1] A. Macovski, Medical Imaging Systems. Englewood Cliffs, NJ: Prentice-Hall, 1983.

[2] C. L. Chan, A. K. Katsagge1os, aod A. V. Sahakian, "[mage sequence filtering in quantum-limited noise with applications to low-dose Auo­roscopy," IEEE Trans. Med. Imag., vol. 12, pp. 610--621, Sept. 1993.

[3] R. E. Sequeira, J. A. Gubner, and B. E. A. Saleh, "Image detection under low-level illumination," IEEE Trans. Image Processing, vol. 2. pp. 18-26, Jan. 1993.

[4] D. T. Kuan, A. A. Sawchuk, T. C. Strand, and P. Chavel, "Adaptive noise smoothing filter for images with signal-dependent noise," IEEE Trans. Pal/ern Anal. Machine Inlell., vol. 7, pp. 653--665, Mar. 1985.

[5] M. Unser and M. Eden, "Maximum-likelihood estimation of linear signal parameters for Poisson processes," IEEE Trans. Acousl., Speech, Signal Processing, vol. 36, pp. 942-945, June 1988.

[6] B. Picinbono and P. Duvaut, "Optimal linear-quadratic systems for detection and estimation," IEEE Trans. Inform Theory, vol. 34, pp. 304-311, Mar. 1988.

[7] S. S. Jiang and A. A. Sawchuk, "Noise updating repeated Wiener filter and other adaptive noise smoothing filters using local image statistics," Appl. Opl., vol. 25, pp. 2326-2337, July 1986.

[8] C. M. Lo and A. A. Sawchuk, "Nonlinear restoration of filtered images with Poisson noise," in Proc. SPlE, vol. 207, 1979, pp. 84-95.

[9] A. Papoulis, Probability, Random Variables, and Stochaslic Processes, 2nd ed. New York: McGraw-Hili, 1984.

llO] T. Hebert and R. Leahy, "A generalized EM algorithm for 3-D Bayesian reconstlUction from Poisson data using Gibbs pliors," IEEE Trans. Med. Imag., vol. 8, pp. 194-202, June 1989.

[II] B. Picinbono, "Higher-order statistical sigoal processing with Volterra filters," in Proc. Workshop on Higher Order Spectral Analysis, Vail, Colorado, June 1989, pp. 62--67.

[12] T. Koh and E. J. Powers, "Second-order Volterra filtering and its application to oonlinear system identification," IEEE Trans. Acous/., Speech, Signal Processing, vol. 33, pp. 1445-1455, Dec. 1985.

[13] B. Friedlander and B. Porat, "Asymptotically optimal estimation of MA and ARMA parameters of non-Gaussian processes from high-order moments," IEEE Trans. AUioma/. Con/I'. , vol. 35, pp. 27-35, Jao. 1990.

[14] C. L. Chan, A. K. Katsaggelos, and A. V. Sahakian, "Recursive locally linear MMSE motion-compensated image sequence filtering under quantum-limited conditions," 1. Visual Commul1. Image Rep., vol. 4, pp. 349-363, 1993.

[[ 5] L. M. Garth and Y. Bresler, "00 the inferiority of higher-order detec­tion io narrowband processing," in Proc. ICASSP, vol. IV, 1993, pp. 208-211.

[16] N. L. Johnson and S. Kotz, Discre/e Distributions. New York: Wiley­Interscience, 1969.

Image Classification Using Spectral and Spatial Information Based on MRF Models

Tatsuya Yamazaki and Denis Gingras

Abstract-A new criterion for classifying multispectral remote sensing images or textured images by using spectral and spatial information is proposed. The images are modeled with a hierarchical Markov Random Field (MRF) model that consists of the observed intensity process and the hidden class label process. The class labels are estimated according to the maximum a posterWri (MAP) criterion, but some reasonable approximations are used to reduce the computational load. A stepwise classification algorithm is derived and is confirmed by simulation and experimental results.

I. INTRODUCTION

Multispectral remote sensing data contain spatial as well as spectral information, and the utilization of both kinds of infOlmation should enable us to classify images more accurately than we can with conventional pixelwise techniques. There are two kinds of spatial information available. The first is a priori knowledge that the areas of classified regions tend to be rather large in comparison with pixel size. This homogeneity assumption seems reasonable because the adjacent pixels in a remote sensing satellite image are apt to have similar gray levels because the image is taken from such a far point. The second kind of spatial information is the class-dependent texture property that constitutes a group of pixels. This texture property defines the correlation between the gray level of one pixel and those of its neighbors. The correlation is considered to be dependent on a classified region.

Zhang et al. [5] improved the classi fication of multispectral data by using the homogeneity assumption in addition to spectral information: image segmentation was performed using the maximum a posteriori (MAP) estimation and the MRF was introduced to implement the ho­mogeneity assumption. Jeon and Landgrebe [6] have also developed a classification method utilizing spatial homogeneity information with MRF models within the framework of MAP estimation and have even dealt with temporal information Neither of these working groups, however, have used the texture property. Some class of MRF model is efficient to express textures, therefore it is desirable to include a criterion by the spatial texture information into MAP estimation.

This correspondence therefore proposes a new criterion for classi­fying multispectral remote sensing data or textured images. It consists of three factors: the class-dependent spectral statistical properties, the spatial homogeneity of the classified image, and the class­dependent texture property. This correspondence also gives a stepwise classification algorithm implementing this criterion.

ManusClipt received September 21,1993; revised December 19,1994. This work was supported by the National Optics Institute, Ste-Foy, Canada, and by the Japan Science and Tecbnology Fund of Canada. The associate editor coordinating the review of this paper and approving it for publication was Prof. Rama Chellappa.

T. Yamazaki is with the Kansai Advanced Research Center. Communica­tions Research Laboratory, Ministry of Posts and Telecommunications, 588-2 Iwaoka, Iwaoka-cho, Nishi-ku, Kobe 651-24 Japan.

D. Gingras is with the National Optics Institute, Sainte-Foy, (Quebec), GIP 4N8, Canada.

fEEE Log Number 9413726.

1057-7149/95$04.00 © 1995 IEEE

1334 IEEE TRANSACTIONS ON [MAGE PROCESSING, VOL. 4, NO.9, SEPTEMBER 1995

TABLE I TRUE MLL MODEL PARAMETERS <P

{3] {32 {3, {3.

class I 0.1 0.1 0.21 0.21

class 2 1.0 0.05 0.05 0.05

class 3 0.05 01 0.05 0.05

class 4 005 005 01 0.05

class 5 005 005 0.05 0.1

~3

D

~I--oo-~I

Fig. 1. MLL model parameters.

II. PROBLEM STATEMENT

We assume that a classified image x and observed data :ti are realizations of stochastic processes X and Y, respectively. Multi­spectral data y = {y I, ... ,:til,}, which are observed through !\' bands, are supposed to be acquired on a finite rectangular lattice 1'1' = {(i, j) : 1 :::; i :::; IV" 1 :::; j ::; IVy}. The set :ti' = {vi'l,"', Y~'xNy}' A· = 1,"', !\', denotes the data taken at the kth

wavelength, where 1/:; E {O"", G - 1} and G is the number of observable gray levels. It is also possible to describe the multispectral data with :ti = b,j : 1 ::; i :::; N.,,1 ::; j ::; Ny}, where

:tiij = CI/L,"" y!)} is a feature vector observed at the ijth pixel. Our goal is the optimal classified image x' = {.r 1l, ... , ,J; N x Ny }

based on the observed data y. Each site of the segmented image is to be assigned into one of M classes; that is, .r ij E {1, 2, ... , M}, where 1\1 is the number of classes and assumed to be known. This optimization is executed from the viewpoint of the maximum a posteriori (MAP) estimation.

III. IMAGE MODEL

Let us take notice of only one image taken at the kth wavelength, k:ti , which is one textured image and modeled with a hierarchical

MRF model. That is, we use a special case of a pairwise interaction (PWI) model. [1] for the region process X and a multi-level logistic (MLL) model [3] for the texture process. The fundamentals of MRF's can be found in the literature (for example, [2]).

According to Besag [1], the local conditional density function (LCDF) of the PWI model for Xij with a class m is given as

(I)

I Oboe~od dolo y. I

l I o";mALo 0~O) I

STEP I l

I STEP II l

I convuged imAge zit I

l I I

l

STEP III

I I

I eatimaLe xV;1 1 l; ad p = p + 1.

I

I converged image i = %111. I Fig. 2. Stepwise classification algorithm.

where [>'ij(l) is the number of the ijth pixel's neighbors having class 1 and ~,. is a parameter to be estimated.

On the other hand, the LCDF of the MLL model for y~ with a gray-level 9 is

P(yt = y I !J~;J )

= _1_ - Lc EC, ,e""i, ",;"ej VC;j(pairv"" )(v;'j=g,<J.') (2)e Z'J

where <P = {;3" : n = 1. .. ·,4} and the MLL model parameters ;3" (/I = 1, ... ,4) are assigned as shown in Fig. J. The potentials for pairwise cliques of type II are defined as

'f k k l Yij = '!/"ij (3)oth.el·u'ise

where .IJ~iJ is the gray level of the pair-partner pixel. Although the Gauss MRF model [4] is also suitable for representing

the statistical properties as well as the spatial correlations and can deal with continuous values, we use the MLL model because it is easier to estimate the statistical and the spatial parameters separately than to estimate them together in a model. And because the observable value at the ijth pixel at the kth wavelength, Y:j, is assumed be a discrete one, the MLL model is useful enough in our experiments.

IEEE TRANSACTIONS ON fMAGE PROCESSING. VOL. 4. NO.9. SEPTEMBER 1995 l335

Fig. 3. True classified image x for the computer simulation.

Fig. 4. One of the observed textured images (y I ) generated on the computer. This image consists of five classes filled with class-dependent textures and has added multi-Gaussian noise.

Suppose that the spectral correlation is a pixelwise multi-Gaussian distribution conditioned with the class. Then the LCDF for the ijth pixel coming from class Tn is

P(Y,j I :I'ij = III)

1 eXl~[-~(Y"-/L )1'B- I ('/I.'_/L )](27i')/</2(c!et. E l )1/2' 2 m III IIII) .11) m

(4)

where fL,,. = {fi~, : k = J, ... , 1\'} and B m = {B~: : k,l = 1,' ... [\'} are a mean vector and a covariance matrix defined on

Fig. 5. Classified image x\2// as a result of STEP III: error rate 2.25 %.

Fig. 6. Image classified by using only one observed image shown in Fig. 4: error rate 7.09 %.

class m. To simplify the description, we set e = {fL,n' Em : m. = J,···.M}.

IV. CLASSIFICATION CRITERION

Given the observed data y and the estimated parameter set tP {-y, 4>, 8}, our purpose is to find the optimal classification x· that maximizes the posterior distribution P(XIY, tP). That is

x' = arg[max P(X = x I y, tP)]. (5)x

The number of candidates for x, however, is enonnous: AI Nx x Ny. To avoid this computational impossibility, we use an iterative approach

1336 IEEE TRANSACTIONS ON lMAGE PROCESS1NG, VOL 4, NO.9, SEPTEMBER 1995

Fig. 7. Observed remote sensing image yl.

where the previous classification obtained at the pth iteration, say x(p), is assumed to be known. Then, using Bayes rule, the new

classification for the ijth pixel at the (p+l )tb iteration is assigned as

X (P+I) - aro'[lnax P(',' . I x(p) .l»]'ij - b ,.: ., 'J ll/(i.j)'Y' '" . (6)

oj

Scanning over the lotal sites, we get the new classification at the (p+l)tb iteration X(I'+I) = {:l'~j+l)}. Since it is easily shown that

the posterior probability for x(l'+ I) never decreases compared with

the one for x(/'), the suboptimal classification i can be obtained as

the convergence of iteration. This method was originally proposed

by Besag [I] and is called the iterated conditional modes (rCM)

algorithm.

Hence, we approximate the right-hand side of (6) as

P(Xij I x~}(i,j),y,tli)

(7)

Fig. 8. Classified image X}~)I resulting from STEP III.

Taking logarithms of (7), substituting each LCDF's, and neglecting constant terms, we obtain the following quantity Q( '1, <1>. e) as a criterion for a class of the ijth pixel to maximize:

(8)

A stepwise classification algorithm (Fig. 2) is designed to im­plement the criterion described by (8). It uses only the pixelwise spectral information in the first step, and because the pixel wise classified image is too coarse for estimation of the MLL parameters, the homogeneity assumption is needed to get a better segmented image in the second step. After the MLL parameter estimates are obtained, a more accurate classification is executed in the third step. rn the second and third steps, the model parameter estimation and the image classification are executed iteratively and adaptively.

IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 4, NO.9, SEPTEMBER 1995 l337

Fig. 9. Image classified using only the one-wavelength lmage shown in Eg. 7

The model parameters are estimated by means of the conventional maximum likelihood (ML) estimation and the maximum pseudo­likelihood (MPL) estimation. See [I] for more details on MPL estimation.

V. EXPERIMENTAL RESULTS

First, the classification algorithms were tested on simulated four­wavelength textured images generated on a computer. The 128 x 128 true classified image shown in Fig. 3 was generated by the Gibbs sampler (GS) [2] algorithm according to the PWI model with the parameter r = 1.5 and consists of five classes (regions). After filling each c.lass with a class-dependent texture (speckled, horizontal, vertical, oblique, or contra-oblique) and adding a class­dependent multi-Gaussian noise, we obtained four observed images y = {yl, ... , y4}. One, 1/, is shown in Fig. 4. Table I lists the MLL model parameters used to generate the textures, and Table II lists the statistical prope11ies (mean vectors and covariance matrices) for each class.

1~1

1~2

/'3

1',

1'5

TABLE II TRUE STATISTICAL PARAMETERS e

102.865

102.803

102.859

102.851

107.686

107.686

107.683

107.688

112.654

112654

112.672

112.673

117.338

117.329

117.353

117330

122.559

122568

122565

122.568

T

T

T

T

T

D,

D2

D3

D,

D5

6.098 5.483 5.533 5.536

5882 5.490 5.474

5.906 5.525

5.796

6.072 5807 5788 5.767

6.177 5.809 5791

6.219 5.783

6303

6.618 6.059 6.074 6.056

6.490 6.047 6.028

6.436 6.057

6.319

6.422 6.149 6.156 6.239

6.524 6.148 6252

6.611 6.267

6908

7.266 6.460 6.471 6664

6.706 6.228 6.429

6.601 6.415

6821

When the proposed stepwise classification procedure was driven with the observed images y and the initial statistical parameters e\O) estimated from the sample images, the final classified image x\~~) shown in Fig. 5 was obtained. Its error rate l is only 2.25 %. The iteration terminated when there was no more new updating for x.

It might seem easy to classify each region using the mean values, but the problem we deal with here is not so straightforward. The Gaussian noise and the properties of textures in each region make it difficult to obtain a neatly (and correctly) classified image such as Fig. 3 from Fig, 4. In addition, this computer simulation is useful in assessing how effectively the algorithm works because the true classified image exists.

To compare multispectral and single-spectral classifications, we made a classification using only one observed image shown in Fig. 4. The resultant single-spectral c.lassified image (Fig. 6) clearly has more misclassifications (the error rate is 7.09 %) around the regional boundaries than does the multispectral one.

Another experimental image classification with fixed parameter estimation instead of simultaneous parameter estimation showed that although the fixed parameters accelerate the convergence of the

I The error rate is defined as the ratio of the number of pixels in the classified image that differ from the lrue image to the number of total pixels.

1338 IEEE TRANSACTIONS ON lJ"lAGE PROCESSfNG, VOL. 4, NO.9, SEPTEMBER 1995

TABLE III ESTIMATED MLL MODEL P,~RAMETERS FOR X)~)I

lit {32 {3J {3,

wavelength 1 class 1 0822 0.694 0736 0.716

class 2 0.772 0.814 0753 0.755

class 3 0782 0.742 0.797 0813

class 4 1032 0.860 0713 0.737

cla" 5 0.687 0.982 0.745 0.782

wavelength 2 class 1 1007 0,559 0.575 0.588

class 2 1.114 0,633 0545 0,575

class 3 1065 0,575 0.528 0.591

class 4 1.016 0.944 0.605 0683

class 5 1044 0.556 0539 0,692

wavelength 3 class 1 1.082 0.486 0.578 0610

class 2 0998 0.702 0657 0675

class 3 1.015 0.572 0667 0.665

class 4 1063 0.894 0.667 0,714

class 5 0.930 0.595 0,738 0.666

wavelength 4 class 1 1301 0.558 0391 0.486

class 2 0.994 0897 0.811 0.779

class 3 0.941 0.937 0.777 0,796

class 4 0962 0.957 0794 0.684

class 5 0.983 0.878 0,810 0.843

algorithm, the resultant error rate (6.44 %) is about three times as much as that obtained with the simultaneous estimation,

Second, the classification algorithms were applied to real remote sensing data. Experimental data y =: {yl, . .. ,yl} were acquired by Landsat through four spectral bands and yl is shown in Fig. 7. These 298 x 384 images are expected to be classified into five categories: water (class one), conifers (class two), deciduous (class three), open fields (class four), and epidemics (class five). In the classified images, a pseudocolor representation is used: class one, black, class two, red, class three, green, class four, yellow, and class five, violet.

Driving the stepwise classification procedure with the observed images y and the initial statistical parameters e\O) estimated from

the sample image data, we obtained the final classified image X)~)1 shown in Fig. 8. In this experiment, we skipped STEP II because xl

was good enough for estimating the MLL model parameters directly, and the updating of less than one percent of pixels of the whole image was set as the convergence standard. As the final estimated parameters, we obtained li~), =: 0.811, iI>\~)J listed in Table III, and

e\~1 listed in Table IV. To compare multispectral and single-spectral classifications, we

made classifications using only images observed each at a single wavelength, One such classified image using yl is shown in Fig. 9 and clearly includes misclassifications. Moreover it proved difficult to obtain a unified single image combining all images, each classified

1-'1

1'2

?'3

I~,

fL S

TABLE IV ESTIMf\TED STATISTICAL PARAMETERS FOR X\~)1

62.488

17.962

13.355

8723

67.318

24.736

19.985

52.151

68.308

25.418

20.463

75465

75.067

31.216

28.382

73.526

69.415

26456

21.079

100.913

T

B,

T

B 2

T

D3

T

B,

39.237 25.430

19.702

43458

30962

54.894

21.986

31964

29.073

T

Ds

3459 1.284

2.097

1400

1:374

2.285

295.283

3717

6.096

3552

86669

2.020 0.413 0.478 0.566

1.023 0.468 0843

1.378 1039

3.857

3508 1.498 2.098 6.160

1.862 1.799 6.781

3569 7.050

54.435

3.352 0.865 1.272 4.253

1058 0827 3733

2.105 3076

71.023

at a single wavelength, because the classifications differed for each wavelength.

VI. CONCLUSION

In tbis correspondence, a new method based on the MAP esti­mation criterion is proposed for the classification of multispectral remote sensing or textured images. Tbe images are modeled witb a hierarcbical MRF: the PWI model for tbe hidden class label process and the MLL model for the observable texture process. The spectral information at each pixel is modeled witb a multi-Gaussian distribution. The criterion consists of I) the pixelwise dependency of the data on each class, 2) the homogeneity assumption of class label process, and 3) tbe texture property dependent on each class. A stepwise classification procedure has been developed to implement the criterion, The information that tbe procedure uses increases gradually, and the classification results are improved step by step. The model parameters are estimated iteratively and adaptively.

The proposed algoritbm is applied to both the textured images generated on the computer and to real remote sensing data. Although images classified according to the new criterion are significantly better than these obtained using the conventional pixelwise classification, tbe proposed procedure is computationally intensive because of tbe estimation of a number of parameters, especially for multispectral

1339 IEEE TRANSf\CTIONS ON [MAGE PROCESSrNG, VOL. 4. NO 9, SEPTEMBER 1995

remote sensing data. The MRF local property, however, makes it is [3] C. S Won and H. Derin, "Unsupervised segmentation of noisy and

possible to distribute the load on parallel processors. textured images using markov random fields," COl1/jJut. Visiol/ Graph. Image Processing, vol. 54, pp. 308-328, July 1992.

[4) B. S. Manjunath, T. Simchony, and R. ChelJappa, "Stochastic and deterministic networks for texture segmentation." IEEE Trans. Acoust.,

REFERENCES Speech. Signal Processing, vol. 38, pp. 1039-1049. June 1990. [5] M. C. Zhang, R. M. Haralick, and J. B. Campbell, "Multispcctral image

[IJ J. E. Besag. "On the statistical analysis of dirty pictures." I Royal Statist. context classification using stochastic relax.ation," IEEE Tr{lns. Syst.. Soc. E, vol. 48. no. 3, pp. 259-302, 1986. Mall, Cybem.. vol. 20, pp. 128-140, Jan./Feb. 1990.

[2] S. Geman and D. Geman, "Stocbastic relaxation, Gibbs distributions, [6] B. Jeon and D. A. Landgrebe, "Classification with spatio-temporal and the Bayesian restoration of images," IEEE Tmns. Pat/ern Mull. interpixel class dependency contexts," IEEE Tmns. Geosci. Remote Machine II/tell., vol. PAMI-6, pp. 721-741, Nov. 1984 Sensing, vol. 30, pp. 663-{j72, July 1992.