Signal Processing 17 (1989) 213-226 213 Elsevier Science Publishers B.V.

E D G E D E T E C T I O N A N D R E S T O R A T I O N O F N O I S Y I M A G E S BY T H E

E X P E C T A T I O N - M A X I M I Z A T I O N A L G O R I T H M

M a s a h i r o T A N A K A

Junior College Course of Economics, Shiga University, Hikone 522, Japan

Tohru K A T A Y A M A

Department of Applied Mathematics and Physics, Faculty of Engineering, Kyoto University, Kyoto 606, Japan

Received 21 March 1988 Revised 21 September 1988

Abstract. This paper presents an approach to edge detection and restoration for noisy images by using a 1-D state-space model subject to a Bernoulli-Gaussian noise. A method of identifying unknown parameters and Bernoulli sequence of the model is developed by applying the EM algorithm. An edge detection algorithm based on the estimate of Bernoulli sequence is also derived. Simulation studies using real images are included to show the effectiveness of the present method.

Zusammenfassung. Ein Ansatz zur Detektion und Widerherstellung von Kanten in einem verrauschten Bild wird vorgestellt. Er macht yon einem eindimensionalen Zustandsraum-Modell Gebrauch. Fiir die Beschreibung der Bildkanten wird zun/ichst angenommen, da~ das Systemrauschen als wei[3e Impulsstfrung mit Gau,g'scher Summenverteilung zu beschreiben ist. Ein Verfahren zur Identifikation der unbekannten Modell- und Kanten-grf~n wird entwickelt, das mit dem EM-Algorithmus arbeitet. Hierdurch werden Sch/itzer in einfacher Weise formelm/i~ig angegeben. Simulationen auf der Grundlage realer Bilder werden in die Untersuchungen einbezogen; so wird die Wirksamkeit der Methode gezeigt, die aufdem EM-Algorithmus beruht.

R6sum6. Cet article pr6sente une approch¢ de d6tection de contour et de restoration d'images brnit6es en utilisant un mod61e unidimensionnel d'espace d'6tat perturb6 par du bruit Gaussian-Bernoulli. Une m6thode d'identification des param~tres inconnus et de la s6quence de Bernoulli du module est d6velopp6e en applicant ralgorithme de maximisation d'esp6rance. Un algorithme de d6tection de contour bas6 sur I'estimation de la s6quence de Bernoulli est 6galement donn6. Des 6tudes de simulations sont incluses pour montrer l'efficacit6 de la m6thode propos6e.

Keywords. Edge detection, restoration, fixed-interval smoothing, identification, EM algorithm.

1. Introduction

For the res to ra t ion o f no isy images, much at ten-

t ion has been pa id to the d e v e l o p m e n t o f compu ta -

t iona l ly efficient a lgor i thms based on the K a l m a n

fil tering [ 5 , 7 , 1 2 , 13]; see also survey pape r s

[20, 22]. A n d the topics in image process ing inc lud-

ing res tora t ion , enhancemen t , edge de tec t ion have

been covered in [2, 16].

We see tha t the K a l m a n f i l ter -based l inear restor-

a t ion a lgor i thms m e n t i o n e d above are based on

the a s sumpt ion that the gray level o f the or ig inal

image is a s ample f rom a 2-D h o m o g e n e o u s ran-

d o m field. Hence the l inear res tora t ion techniques

that in t roduce a g loba l smoo th ing for noise reduc-

t ion necessar i ly induce the effect o f b lurr ing ,

e l imina t ing sharp b o u n d a r i e s that d iv ide the tex-

ture segments . This resul ts in a d e g r a d a t i o n o f the

vis ibi l i ty o f the res tored images. In this respect ,

Nahi and Hab ib i [14] have d e v e l o p e d a recurs ive

dec i s ion -d i r ec t ed a lgor i thm for the e n h a n c e m e n t

o f noisy images based on a non- l inea r s ta t is t ical

0165-1684/89/$3.50 © 1989, Elsevier Science Publishers B.V.

214 M. Tanaka, T. Katayama

model. Also, Ingle and Woods [3] and Woods [22] have presented a multi-model space-variant filter- ing algorithm for the estimation and detection of noisy images. Although these algorithms have shown considerable success, it is still needed to develop a method of identifying model parameters from the observed images, since they are usually not known a priori.

In this paper, we consider a 1-D state-space model for an image, in which the edges or the abrupt changes in the gray level of the original image are characterized in terms of the spiky sys- tem noise with a Gaussian sum distribution as in [21]. It is also assumed that the observed image is corrupted by an additive Gaussian white noise. For the identification of the state-space model, we apply the Expectation-Maximization (EM) algorithm [ 1 ], by which the Maximum- Likelihood (ML) estimates are recursively obtained with less computation. Since the EM algorithm for the state- space model requires the Kalman filter and the fixed-interval smoother [18, 19], the restored image is obtained simultaneously with the estimates of the model parameters.

In Section 2, we present the mathematical model of an image, where the state-space model and the probability density functions (PDFs) of the system and the observation noises are given. In Section 3, the EM algorithm is briefly explained, and the identification method of the system parameters and the edge detection scheme are developed. In Sec- tion 4, results of simulation experiments using real images are presented, including edge detection and restoration of noisy images.

2. Image model

Suppose the 1-D image x(k) is expressed by an AR(1) model, and the observed image y(k) is corrupted by an additive Gaussian white noise v(k). Thus we have the scalar state-space model

x(k) = f x ( k - 1)+ w(k), (2.1)

y(k) =x(k)+v(k), k= 1 ,2 , . . . , N, (2.2) Signal Processing

/ Edge detection and restoration

where x(k) denotes the gray level of the image at the kth pixel of a certain horizontal or vertical line and x(0) is Gaussian with mean Xo and variance Po. For real images, since the differences {x(k)- x(k-1)} are very small except for edges, {x(k)} is nearly a random walk, so t h a t f is close to unity. Thus we naturally observe that the magnitude of the system noise w(k) would be small for homogeneous parts of an image, and would be extremely large for edges. The preceding feature of an image does not meet the assumption that w (k) is Gaussian, although this assumption is com- monly adopted in linear estimation and the Kal- man filtering. In this respect, we assume that w(k) takes either wo(k) or wl(k), i.e., w(k) is expressed a s

w(k) = (1 - T(k))wo(k) + T(k)w~(k), (2.3)

where PDFs of wo(k) and wl(k) are

wo(k) ~ N(0, qo), (2.4)

w~(k) ~ N(O, q,), (2.5)

respectively, and where qo.~ q~. In (2.3), T(k) is a switching parameter of Bernoulli sequence with

Pr{T(k) = 0} = 1 -p , (2.6)

Pr{T(k) = 1} =p, (2.7)

where 0~<p < 1. A Bernoulli-Gaussian modeling related to (2.3) has already been used in deconvol- ution problems [6, 8, 10] and in state estimation with uncertain observations [4, 17]. Note that the PDF of w(k) is a Gaussian sum such that

w(k) ~ (1 -p)N(O, qo)+pN(O, ql). (2.8)

The observation noise is supposed to be Gaussian with

v(k) ~ N(0, r), (2.9)

where {w(k)} and {v(k)} are independent. Then we assume that the edge has occurred at k if T(k) = 1 because the absolute value of w~(k) can be very large.

If {T(k)} were known, w(k) would be reduced to a non-stationary Gaussian process with mean

M. Tanaka, T. Katayama / Edge detection and restoration

zero and variance q(k) where

q(k)=~qo,~ if y ( k ) = 0 , (2.10)

tq~, if y(k) = 1.

Hence the optimal Kalman filter or the fixed- interval smoother could be readily applied. But, since our problem includes the edge detection, a Bernoulli sequence {y(k)} must be estimated together with the state x(k).

It should be noted that the following discussion is easily extended to the n-dimensional state-space model

x(k) = Fx(k - 1)+ Gw(k), (2.11)

y(k) = Hx(k)+ v(k), (2.12)

where x(k) -- [ x l ( k ) , . . . , x~(k)] T, and

[01:1 [!] F = "- G = , , ,

f . . . . f~

H = [ 0 . . . 0 1]. (2.13)

Also it might be more appropriate to use 2-D image models [5, 7, 12] in the present situation. We observe however that our edge detection scheme cannot easily be incorporated into 2-D image models, since there are many types of edges for the 2-D case. Moreover, the identification of model parameters and a Bernoulli sequence will be more difficult for 2-D image models. Therefore, we employ a simple 1-D model of (2.1) and (2.2) based on a line-by-line scan [13] for developing a decision-directed restoration method.

V N = {v(1) , . . . , v(N)},

N y = { y ( 1 ) , . . . , y(N)}..

3. Identification of an image model

In this section, we apply the EM algorithm for the identification of the image model described in Section 2. For notational simplicity, we introduce the following:

0 = {f, qo, ql, r},

X N = { x ( 0 ) , . . . , x ( N ) } ,

yN = {.y(1), . . . , y(N)},

215

3.1. The EM algorithm

The EM algorithm [1] is originated from sim- plifying the numerical procedure of the ML estima- tion by utilizing the complete-data likelihood. Shumway [18] and Shumway and Stoffer [19] applied the EM algorithm to the identification of state-space models. In the standard state-space model, the complete likelihood function Lc(O) is defined as Lc(0) -- p ( yN, X N I 0). We first compute E{log Lc(O)[ yN, 0O)}, the conditional expectation of the log-likelihood function based on the current estimate 0 ") of 0 and all the observations yN (E step). Then we compute 0 "÷1) , the new estimate of 0 which gives the maximum of E{IogLc(O)[yN, o ~} (M step). By the EM algorithm, we can obtain simpler formulae for estimating parameters of the state-space model compared to the standard ML method by Newton- Raphson or some other nonlinear optimization.

In this subsection, we first consider the iden- tification of 0 at one horizontal or vertical line which includes several edge elements. We give the complete likelihood function of unknown para- meters 0 under the assumption that the state x N is known, and then derive the conditional expecta- tion of it based on the observations yN and the estimates 0 "), ?N,) obtained in the previous iter- ation. Then we derive the estimate of ?N by maximizing the conditional expectation

E{p(yN[ yN , x N , 0 ) [ yN, 0~0, y N ( i ) } .

Suppose that X N, yN, and yN are known and define the complete likelihood function of 0 as Lc( O) := p( yN, xN]o, yN). Then it is clear that ~

L~(O)=p(yNIx N, O, yN)p(XN[o, yN).

(3.1)

1 T h e f o l l o w i n g d i s c u s s i o n is a l s o v a l i d i f w e d e f i n e Lc(O ) := p(V N, xNIo, y N ) [ 1 9 ] , s i n c e

p(vN, xN[o, ,~N)=p(VN]o ' .yN)p(XNIo ' yN)

=p( yNIxN, O, yN)p(XN[o, V N) Vol. 17. No. 3. July 1989

2 1 6 M. Tanaka, T. Katayama / Edge detection and restoration

It is easily verified from (2.1) and (2.2) that

N p ( y N I x N , O, y N ) = l-I p(y(k)[x(k), 0),

k = l

(3.2)

p(X~l o, v ~ ) --p(x(O)) N

x I] p(x(k)[x(k-1) ,O,y(k)) , (3.3) k ~ l

where

p(y(k) lx(k), O) = ( 2 " t r ) - l / 2 r - ' / 2

x e x p { - ( y ( k ) - x(k))2/2r}, (3.4)

p( x( O ) ) = ( 2~r)-~/E pol/2

x e x p { - ( x ( 0 ) - Xo)2/2Po},

(3.5)

p(x(k)[x(k-1) , O, y(k))

= (2~r)-l/Eq-~/E(k)

x e x p { - ( x ( k ) - f x ( k - 1))2/2q(k)}. (3.6)

Note that q(k) in (3.6) is given by (2.10). Define l~(O):= log L¢(O). Then we obtain

-2lc(0) = log Po+ (x(0) -Xo)2/po

+ ~ log qo k e D o

+ ~. (x(k)+fx(k-1))Z/qo k c D o

+ Y~ log ql k E D !

+ 2 (x (k) - fx (k- 1))2/q, k ~ D 1

N N

+ Y, log r+ ~ (y(k)-x(k))2/r , k = l k = l

(3.7)

where Dj={k[y(k)=j,l<~k<~N}, j = 0 , 1 , and where the constants irrelevant to the following discussion are omitted.

Let 0 (i) and yN(1) be the estin~ates at the ith iteration. Then noting that f, qo, ql and r are unknown constants, we can easily obtain the condi- tional expectation of the complete log-likelihood

function (3.7) as (Appendix A)

J,(O) = E{-21c(O)l yN, 0(,), N(,)}

= log Po + [P(01 N) + (~(0l N) - Xo)2/Po]

Signal Processing

where

N

+ Y. log r k = l

N

+ r -1 k = l

{(y(k) - ~ ( k l N ) ) 2 + P(kIN)}

+ ~ log qo k c D o

+%1 ~. {A(k)_EfB(k)+f2A(k_ l ) } k c D o

+ Y, log ql k E D I

+ql ' • {A(k ) -2 fB(k )+f2A(k -1 )} , k E D !

(3.8)

A(k) = P(k lN) + ~2(k I N) , (3.9)

B(k) = P(k, k - 1IN) + ~ ( k l N ) ~ ( k - 1[ N), (3.10)

and where ~,(klN) , P(k lN) , and P(k, k - l l N ) are defined as

~(k[N)=E{x(k) IYN, O('), yN(')}, (3.11)

P(k[ N) = E{(x(k) - ~(kl N)) 2

[ y N O(i), yN(O}, (3.12)

P(k, k - l IN) -- E{(x(k) - ~ ( k [ N))

x ( x ( k - l) - ~ ( k - l IN))

[ y N , 0 ( 0 , N ( i ) } . (3.13)

It is easy to see that these quantities are obtained by the fixed-interval smoother constructed by the Kalman filter and the backward equations [11, 18, 19] given as follows:

Kalman filter (a) Time update

~(k[ k - 1) =f ( ' )~(k - l [ k - 1), (3.14)

M. Tanaka, T. Katayama / Edge detection and restoration

P ( k l k - 1) = ( f ( ' ) )2P(k- 1.l k - l)

+ (1 - y ( ' ) (k - 1))q(o ')

+y(')(k-1)q~ i).

(b) Kalman gain

K(k) = P(k lk - 1)(P(klk - 1) + r°)) -1.

(c) Measurement update

(d)

(3.15)

(3.16)

~ ( k l k ) = ~ ( k l k - 1 )

+ K ( k ) ( y ( k ) - ~ ( k l k - 1 ) ) , (3.17)

P ( k l k ) = P ( k l k - 1 ) - K ( k ) P ( k l k - 1 ) . (3.18)

Boundary conditions

~(0[0)=Xo, (3.19)

P(OlO)=Po. (3.20)

Backward equations (a) Smoothed estimate

~ ( k J N ) = ~(k[ k) + S(k)(~(k + I I N)

-~(k+llk)).

(b) Smoother gain

S(k) =f( ' )P(k[k)P-~(k + I I k).

(c)

(3.21)

(3.22)

Covariance equations

p(kIN ) = P(kIk ) + S2(k)(P(k + 1[ N)

-P(k+l]k)),

k = N - I , . . . , 0 . (3.23)

P(k,k- I IN)

= S(k - 1)P(klk) + S(k)S(k - 1)

x (P(k + 1, k i N ) - f " ) P ( k l k ) ) ,

k = N - 1 . . . . ,1. (3.24)

217

(d) Boundary condition

P ( N , N - I [ N )

= 9 # ' ) ( 1 - K ( N ) ) P ( N - 1 1 N - 1 ) . (3.25)

Next, to derive the estimate of yN, we consider the conditional probability of yS based on y N X N, and 0:

Pr(yN i yN, X N, 0)

p( y N [ x N ' yN, O)p(XN[yN, O) Pr(TN I 0)

- p(YN, XNIO )

(3.26)

From the relation (2.2), yN is irrelevant to yN if X N is known. Hence we find that

p ( Y ~ I X N , yN, O)=p(YNIXN, O). (3.27)

Therefore (3.26) is expressed as

p ( X ~ [y N, 0)ar(yN [ 0) pr(yN i yN, X N, O) - p ( x N I o )

(3.28) Now we define J2(O) as

j2(yN) = E{Pr(yN ] yN, X N, O)

where

×p(XNIO)IYN, 0<,), yN<,>}

= E { p ( X N I y ~, 0)

x Pr(yN I 0)1 yN, 00), yN<,)}

N

= ~ exp{-g(k, y(k))/2}Pr(y(k)), k = 1

(3.29)

g(k, y(k))

= (1 - y ( k ) ) [ l o g q(o°+{A(k)- 2fB(k)

+fZA(k - 1 )}/q¢o °]

+ y(k)[log q~)+ {A(k) - 2fB(k)

+f2A(k - 1)}/q~°]. (3.30)

Thus we can obtain new estimates 0 ~+l) by minimizing (3.8) and yNO+l) by maximizing (3.29) independently. In fact, differentiating Jl(O) with

Vol. 17. No. 3, July 1989

218 M. Tanaka, T. Katayama / Edge detection and restoration

respect to 0 yields (see Appendix B)

q~') E B(k)+q(o° ~ B(k) k ~ D o k ~ D l

q~') 2 A ( k - 1 ) + @ ') vZ A ( k - 1 ) k E D 0 k ~ D t

(3.31) q(o I+1)= ~ {A(k)-2f(~)B(k)

k c D O

+ (f(°)23(k - 1 )}/# (Do),

f ( i + l ) _

{A(k)-2f(~)B(k) k E D I

+ (f('))2A(k- 1)}/#(D1),

(3.32)

(3.33)

q~i+l) =

N

r('+')= 2 {(Y(k)-; (kIN))2+P(klN)}/N, k = l

(3.34)

where #(Dj) , j = 0 , 1 denote the number of the elements of set Dj, and/gj, j = 0, 1 are given as

O,={klT(k)=j,l<~k<~N}, j = 0 , 1. (3.35)

Maximizing J2(y N) with respect to y(k) yields

if exp{-g(k, 0)/2}(1 - p )

>exp{-g(k, 1)/2}p, otherwise. (3.36)

~/(i+,)(k ) =t 0' 1.1,

(iii) Give initial estimates 0 (°) and y(°)(k)= 0, k= 1 , . . . , N. Set i=0.

(iv) Repeat (a) to (c) until 0 (i+1~ and { y(i+~)( k ) } converge.

(a) Compute ; (k iN) , P(klN), and P ( k , k - l IN) by the fixed-interval smoother of (3.14)-(3.25).

(b) Compute A(k), B(k), k = 1, . . . , N by (3.9), (3.10).

(c) Compute the estimates 0 (i+~), y(i+~)(k) by (3.31)-(3.34), and (3.36).

(v) Define Yh(j, k):= ~(k), ;h(j, k):= ; (k iN) for every horizontal line j = 1 , . . . , N.

(vi) Define ~v(k,j):= ~(k), ;v(k,j):= ; (k l N) for every vertical line j = 1 , . . . , N.

(vii) Decide an edge has occurred at (k,j)th pixel if e(k,j)= "~h(k,j)GC/v(k,j) is unity, where G is the logical OR.

(viii) Adopt one of ;h(Lk), ;v(k,j) or ½(;h(£ k) + ;v(k,j)) as the restored image.

4. Example

In this section, we apply the edge detection and restoration algorithm developed in Section 3 to real images. "Girl image" which is used for the first experiment is shown in Fig. 1, where the size

3.2. Edge detection and restoration algorithm

We propose an algorithm for the edge detection and restoration of an image in this subsection. To obtain an image consisting of edges, we synthesize two binary images {Yh(', ')} and {Yv(',')}, esti- mates of Bernoulli sequences which are obtained by scanning the image along horizontal and vertical directions respectively. Both horizontal and ver- tical scannings are needed, since edge elements parallel to a coordinate axis are hard to be detected.

The algorithm is summarized as follows: (i) Normalize the image by subtracting the

mean value. (iO Pick a horizontal or vertical line to identify

parameters and assign an appropriate value to an edge probability p. Signa l P roces s ing

Fig. 1. Girl image.

M. Tanaka, T. Katayama / Edge detection and restoration

of the image is 256 x 256. From the histogram of the image in Fig. 2, we can see that the density has a long tail in the region of the low gray levels. Hence, in order to condense the low gray levels and to enlarge the high gray levels, we transform the gray levels as

Zij ~- 1 0 ( y l j - 2 0 6 ) / N " 1 0 0 (4.1)

The histogram of the transformed image by (4,1) is shown in Fig. 3.

We use, say, the 100th horizontal line to identify the image. Table 1 shows the identified parameters f, t~o, t~, ~ for nine cases where the initial values are given as f(o) = 0.9, q(o °) = 1.0, q~O) = 100.0, and r (°)= 20.0. The estimated parameters have conver: ged within the precision of three figures after 100 iterations in all the cases. The edges detected from the original images are shown in Figs. 4(a) and 4(b) where p=0 .1 and 0.15, respectively. The

219

corresponding parameter estimates are shown in Cases 1 and 2 of Table 1.

The edge detector by Roberts [15] is a familiar and simple operator which is defined by

8( i , j ) = (y( i , j ) - y( i - 1 , j - 1)) 2

+ ( y ( i - l , j ) - y ( i , j - 1 ) ) e. (4.2)

The edges are detected according to

f l (edge), if 8 ( / , j ) > T,

3,(i,j) = ~0 (no edge), otherwise, (4.3)

where T is the threshold to detect edges. Figure 4(c) is the detected edge by the Roberts' operator for the original image, where T was set at the 10 percentile of 8(i,j)'s. We observe that the edges of Figs. 4(a) and 4(b) are quite similar to those of Fig. 4(c).

~00 1000 1500 ZOO0 2500 3000 3500 4000 ~500 SO00

0<=> 4 7 S<-> 9 3

tO<=> 14 3 15<=> 19 34

75<=> 29 41 30<=> 34 132 u 3S<=> 39 95 , o . . . . . 11+ . . 4S<=> 49 206 n i t SO<=> S4 122 s t 55<=> $9 226 1888

63<=> 69 231 ¢e8¢ 70<-> 74 132 *¢ ?S<=> 79 300 s s t t m 8 80<=> 8+ 138 83 8S<s> 89 3+5 ¢ ¢ 8 8 e 8 90<"> 94 196 e e s

9S<z> 99 4~2 e s e e e e s e

100(=)10+ 562 l t l l l l l l l S e 10S<x>I09 3S0 8 8 m • e e e

110<t>116 76S I l l t l l I I S X l I l l I 115<=>119 1 0 5 6 t I S e 8 8 8 t t ¢ z z x 8 8 e I l S t e 120<=>12~ 1 3 4 8 u e n x e n x e z ¢ t x x s x u t z s s s e e 125<s>1~9 689 8 l l t l e e t t l e t 8 1 3 0 < t > 1 3 ~ 1348 I t l X t l e l t t S t l l e $ l l l X X X t l t e l t J 8 135<e>139 1803 * s s e x s e s e s e s e e e e e e t n e s . e • e t • e N * 8 ¢ * 1+0<=>1+4 1706 1 8 1 1 1 8 1 1 1 1 8 1 8 1 1 1 1 1 1 1 1 1 1 1 1 1 8 1 1 1 1 1 1 1 1 + 5 , 1 ) 1 + 9 18+6 l l l l l l l t e t ¿ l t l l l l l t t t S t l l t l t 1 8 1 1 1 1 1 1 150<=)15+ 1807 e t s o x t t s s t e s n t x t e J m z s t n s • e s s l s t $ t m t 1 5 5 < - > 1 5 9 1733 8 8 s e o x e x s e s s , s s e s u ¢ $ e s ¢ s s s t 8 8 8 ¢ ¢ ¢ 160<=>164 1738 t e , t z z s , J e s ¢ s s e ¢ t s t t s z s s s s , t s • n e t t 165<->169 2920 e t s n s e ¢ t 8 8 ¢ e e e s e s t x z t t t 8 8 8 s s s 8 8 1 s s t , s 8 8 e , s t s e s t s t e e e ¢ s e t n 170<:>174 19~,5 8 S l l S X l l l e l S t l e t e t I t e S t S S l t J l S t t t S t $ • • 1 7 5 < ' > 1 7 9 1916 t e t o x e l e x s e e e s e e ¢ t t x e e s t e e e ¢ • ¢ e s e s l e e s

180<8>18+ ~626 x s , s l x s s s e x x e z , e e , s s s x x H e s t e e e x s t s x s s • e e e e . . u s u s • 185<=>189 165S l s , n l x ¿ l S l l l l l l l S S l l e l l l l , l l e n l ~+0<I~I++ ~+++ I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I ~+S<l)~++ 30+~ I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I ~ 0 0 < I ) ~ 0 + +600 •••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••• +OS<I)~O+ PO~P •••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••• 2 1 0 " I > 2 1 + 3811 I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I 215"=>21+ 3330 I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I ++0<I)+~+ ~+3+ I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I 2 + ~ < I ) ~ + ~ + I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I I 230<1>23+ 1+86 I l n n l l n U l l l l l l l l l l l l l l l , I 2 3 5 ( 1 ) ~ 3 + 140~ l l l I l l l 8 1 8 1 l l l l l l l t l l l l l l l l l ~+0<1)+++ ~+l+ l l l l l l l l l l l l l l l l l l l l l l l l +++<13++9 9+5 Itlltttltlttltllll

~$5<=>2$9 0

Fig. 2. Histogram of Girl image.

Vol . 17, N o . 3, J u l y 1 9 8 9

220 M. Tanaka, T. Katayama / Edge detection and restoration

LEVEL 0<=> S<=>

10<=> 15<=> 20<=> 25<=> 50<=> 35<=> 40<=> 44 45< ,> 49 30<8> 34 35<=> 59 60<*> 64 65<=> 69 70<8> 74 75<=> 7 9

80<8> 8 * 85<8> 8 9

90<8> 94 95<s> 9 9

100< ->104 103<8>109 110<8>114 115<=>119 120<*>124 1~5< :>1?9 130<=>134 1 3 5 < 8 > 1 3 9

140<8>144 145<=>149 130<=>154 155<=>159 160< :>164 155<8>169 170< :>174 1 7 5 < = > 1 7 9

1 8 0 < = > 1 8 4

185<=>189 190< •>154 195<=>199 ?00<8>?04 ?05<=>?09 710< :>714 215<8>219 ?20<=>??8 ??5<8>??9 230<=>?34 235<1>739 ?40<8>?44 745<=>249 250< :>754 255< :>759

4 0 9 0

14 0 1 9 121 24 545 29 848 54 783 39 1180

1583 2625 3 3 5 1

3 3 5 7

3 5 4 0

2 6 9 3

3910 2 8 0 6

2 6 6 5

2~75 4303 492 t 6 7 1 5

4 3 7 9

246? ?446 1 4 0 6

1 4 9 3

1 1 9 5

1041 ?08 788 311

500 1000 1500 2000 2500 3000 3500 4000 4S80 3000 . . . . . . . . . ! . . . . . . . . . [ . . . . . . . . . ! . . . . . . . . . ! . . . . . . . . . ! . . . . . . . . . ! . . . . . . . . . ! . . . . . . . . . ! . . . . . . . . . ! . . . . . . . . .

t t

* t t t t t * t a * * . . = * t * * l t t t * * * t

* t * • ¢ • i i 1 1 1 1 1 1 i

I l l l l l l l l l t I l l l l l l l l 1 1 1

I 1 I I l l l l l l l l I I I 1 1 t I I I I I I 1 1 I I I I I

I I I I I I I l l I I I l l 1 1 1 1 1 ! t I t * I t l l l l l l l l l l l l l l l l l l l l l l i t I 1

I I I I I I t i l l I I I 1 1 1 I l l l l l l l l l l l l l l l l l l l l l l l l I ! ! 1 l l l l l l l l l l l l l l l l I ! 1 I l l

I l l l l l l l l l l l l l l l l l l l l l l l I l l l l l l l l l l l l l l l l * l l l l l I l l l l l l l l l l l l l I I I I I I I I I t

I I I I I I I I I I I I I I t l t l t l l l l l l l l t l l l l l l l l l l l l l t l t l l l l l l l l l l l l l l l l l l l l l l l l

I I I I I l l l l l I I I I l l l l l l l l l l l l l l l l l l t l l l l l l l l l l l l l l l l l l l

I l l l t l l l l l l l l l l l l l l l l l l l l l l l l l l l l l I I I I I I t l l l l l l l l l l l l l l l l t l I I t I I I I I I I I I I I 1 1 1 I I

I l l l l l l l l l l l l l t t l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l

! I I I I I l l l l l l l l l l l l l l 1 1 ! 1 1 I I I I l l l l l l l l l l l I I I 1 I I I I l l l l I

I I I I l l l t l I I I l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l t l l I l l l l l l l l l l l l l

~ * ~ * ~ * ~ t ~ * ~ t ~ * * ~ i * ~ * ~ * * ~ * ~ * ~ * * * ~ * ~ * *

t l t t l t t t t l l t t t t t l t t t l t t l l m t t t t l l l t t t l t l l t t l l t t l l l t t l l l t t l t t t t l l l l t l l l l t l t t t t l l t t l l l t t l t l l t l l l t t l

S 8 8 B e * t e e • t • • • t t • • • • l e t t t t t t S t • t • t e s e e t s e e t s s • t t t s e e t • 8 • • m t t t t t t s t t * • • • 8 • t t t t t t t t • • • t t t t t t t t t t • t • • 8 8

: : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : . . . . . . . . . . . . . . . . . . . . . . . . • l l l l l t t t • • l l l t t t t t l t t l t t t l l l t • • • • I t • l i l t • • • l t • t

• t J l t t * S t t J t t J t t t S t • t t S t C t t •

• t t t t S S t t • Z t Z S t • t t t t S S ~ e C t t t •

• t * S e Z e S S * 8 8 • S S t t t t ~ S S •

t t J t t t t t t t t t t t t t t t t •

8 • • • • t J t S t 8 ¢ 8 8

8 • • t t t J t J • t t t t t

t m t t *

Fig. 3. Histogram of the transformed image.

Table 1

Identified parameters

Case p r f qo ql

1 0.1 0 0.969 16.3 380.0 1.6

2 0.15 0 0.966 11.8 310.1 3.0

3 0 29.11 0.935 57.3 - - 18.2

4 0.1 29.11 0.958 19.3 481.2 32.4

5 0.15 29.11 0.953 11.8 344.2 36.2

6 0.3 29.11 0.912 18.7 251.3 41.7 7 a 0.3 29.11 0.937 11.6 186.3 33.3

8 0.5 29.11 0.930 8.6 97.8 31.4

9 0.3 97.03 0.929 10.6 118.7 101.2

a The 70th horizontal line is used for the identification.

Next, we formed a noisy image by adding a zero-mean Gaussian white noise to the trans- formed image of (4.1). Fig. 5 shows the noisy image where the SNR is 14.3 dB 2. Fig. 6 shows the

2 SNR = 10 loglo (variance of signal/variance of noise).

Signal Processing

restored images for p = 0, 0.3, 0.5, and the corre- sponding parameter estimates are depicted in Cases 3, 6, 8 in Table 1. For comparison, the restored image by Lee's algorithm [9] is shown in Fig. 7 and the edges detected by the present algorithm with p = 0.3 and by the Roberts' operator are shown in Fig. 8, where the thresnold T is set at 20 percentile. Also Tables 2 and 3 summarize the improvement ~7 in SNR, and CPU time. Note that ~ = 10 log]o (o.2/o.2), where o .2 and o.2 are the variances of the images before and after the restoration, respectively.

We observe that Fig. 6(a) is rather noisy com- pared with Fig. 6(b). This is due to the fact that for Case 3. (p = 0) the present algorithm has no strong smoothing capability, since the estimate is much smaller than the true value of r and the estimate qo is much larger than that for Case 6. For Case 6, however, the estimate F becomes quite

M. Tanaka, T. Katayama / Edge detection and restoration 221

a

.( .

q J J,~.

Fig. 4. Edge detection. (a) p =0.1, (b) p=0.15, (c) Roberts' operator.

larger than the true value, so that our algorithm has strong noise smoothing capability, bringing blurring effect into the restored image of Fig. 6(b). Hence Fig. 6(a) is superior to Fig. 6(b) in the visibility of minute or high-frequency parts as well as in the improvement in SNR. Further increasing the a priori probability to p = 0.5 (Case 8) yields a good estimate of r a n d hence the power of edge

detection will be increased. Thus the improvement in SNR for Case 8 gets better than for Case 6 as shown in Table 2, and the visibility of the minute parts of Fig. 6(c) becomes slightly better than that of Fig. 6(b). We see from Table 2 and 3 that the present algorithm is comparable to Lee's algorithm [9] in both the improvement in SNR and CPU time. Also, the visibility of Fig. 7 is closer to that

Vol. 17, N o 3, July 1989

222 M. Tanaka, T. Katayama / Edge detection and restoration

effectively in the restored image without blurring the edges.

An extension of the present approach to the 2-D NSHP image model [22] is a present topic of our research.

Acknowledgment

The authors are grateful to H. Sakai who intro- duced us to the EM algorithm, and to K. Kubo for his assistance in computer simulation. All the computations were carried out with FACOM M-780 at the Data Processing Center of Kyoto University.

Fig. 5. Noisy image.

of Figs. 6(b) and 6(c) rather than Fig. 6(a). Com- paring Figs. 8(a) and 8(b), we see that the edges obtained by the present method is bit fat but less noisy than the edge detection by the Roberts' operator, since the latter has no capability of noise smoothing.

Also, noise-free "Couple image" and the edges detected by the same procedure as above are shown in Fig. 9 where p = 0.1.

5. Conclusions

We have developed a method of edge detection and restoration for a noisy image by using a l-D state-space model. To characterize the edges of an image, the system noise is modeled by a combina- tion of two Gaussian white noises with different variances, where theii occurrence is controlled by a Bernoulli sequence. Unknown parameters and edges are identified by applying the EM algorithm, where estimators are expressed by simple for- mulae. By the simulation studies using real images, we show that edges can be detected fairly well when the additive noise is negligibly small. Although sharp edge detection is not obtained for highly noisy images, the noise was reduced Signal Processing

Appendix A. Derivation of (3.8)

Let x be a random variable, and ~ be a it- algebra. Let ~ : = E { x l ~ } be the conditional expectation of x with respect to ~. Then, for any constant a,

E{(x - a)2]~} = E{(x - .~ ÷ .~ - a)2]~}

= E { ( x - : ~ ) 2 l ~ } + ( . ~ - a ) 2, (A.1)

since )~-a is G-measurable. Using the relation (A.1), we can easily derive (3.8).

Appendix B. Derivation of (3.31)

Differentiating Jl(0) with respect to f yields

0Jl(0)~f 0f0 [ - k~Do log qo

+qo' ~ { A ( k ) - 2 f B ( k ) k~Do

+ f 2 A ( k - 1)}

+ ~ log ql keDt

+q~l ~ { A ( k ) - 2 f B ( k ) k~D!

+ f 2 A ( k - 1)}]

=2qo ~ [ - B ( k ) + f A ( k - 1 ) ] keDo

+2q~ -1 ~ [ - B ( k ) + f A ( k - 1 ) ] . k ~ Dl

(a.1)

M. Tanaka, T. Katayama Edge detection and restoration 223

Fig. 6. Restored images from noisy tmage. (a) p = 0, (b) p = 0.3, (c) p = 0.5.

Table 2

Improvement in SNR and CPU time by the present algorithm

Table 3

Improvement in SNR and CPU time by Lee's algorithm [9].

Case p -q (dB) CPU a (s)

3 0 3.23 2.002 Fig. 6(a) 6 0.3 2.34 2.003 Fig. 6(b) 8 0.5 2.72 1.994 Fig. 6(c)

Case Window a r/(dB) CPU (s)

1 3 x 3 2.35 0.730 2 5 × 5 3.27 1.605 Fig. 7 3 7 × 7 2.93 2.847

a7 × 7 window is recommended in [9]. a CPU includes the time for the parameter identification by the EM algorithm.

Vol. 17, No. 3, July 1989

224 M. Tanaka, T. Katayama / Edge detection and restoration

Fig. 7. Restored image by Lee's algorithm with 5 x 5 window [9].

• , ~ , - - ~ , = ~ ~ - ~

Signal Processing

Fig. 8. Edge detection from noisy image. (a) p = 0.3, (b) Roberts' operator.

M. Tanaka, T. Katayama / Edge detection and restoration 225

Fig. 9. Couple image. (a) original, (b) edge detection with p = 0.1.

Setting aJ~(O)/af= 0, we obtain the new estimate f ( i+ l ) as

f ( i+ l ) _

q]') Y~ B(k)+q~o') 2 B(k) k c D o k ~ D 1

q]" ~ A(k-1)+q~j ) Y~ A ( k - 1 ) k c D o k c D 1

(B.2)

We can obtain other parameters by the same

manner.

References

[1] A.P. Dempster, N.M. Laird and D.B. Rubin, "Maximum likelihood for incomplete data via the EM algorithm", J. Royal Star. Soc., Vol. B-39, 1977, pp. 1-38 (with dis- cussion).

[2] T.S. Huang, Two-Dimensional Digital Signal Processing I, Springer-Verlag, Berlin, 1981.

[3] V.K. Ingle and J.W. Woods, "Multiple model recursive estimation of images", Proc. Int. Conf. on Acoust., Speech and Signal Process., '79, Washington, D.C., April 1979, pp. 642-645.

[4] A.G. Jaffer and S.C. Gupta, "Recursive Bayesian estima- tion with uncertain observations", IEEE Trans. Inf. Theory, Vol. IT-17, No. 4, September 1971, pp. 614-616.

[5] A.K. Jain and J.R. Jain, "Partial differential equation and finite difference methods in image processing II--Image restoration", IEEE Trans. Automat. Control, Vol. AC-23, October 1978, pp. 813-834.

[6] T. Katayama and M. Tanaka, "Detection and estimation of a Bernoulli-Gauss process for linear discrete-time sys- tems", Int. J. Systems Sci., Vol. 17, No. 5, May 1986, pp. 687-702.

[7] I". Katayama, T. Hirai and K. Okamura, "A fast Kalman filter approach to restoration of blurred images", Signal Process., Vol. 14, No. 2, March 1988, pp. 165-175.

[8] J.J. Kormylo and J.M. Mendel, "Maximum likelihood detection and estimation of Bernoulli-Gaussian proces- ses", IEEE Trans. Inf. Theory, Vol. 1T-28, No. 3, May 1982, pp. 482-488.

[9] J.-S. Lee, "Digital image enhancement and noise filtering by use of local statistics", IEEE Trans. Pattern Anal. Machine lntelL, Vol. PAM1-2, No. 2, March 1980, pp. 165-168.

[10] A.K. Mahalanabis, S. Prasad and K.P. Mohandas, "Recursive decision directed estimation of reflection coefficients for seismic data deconvolution", Automatica, Vol. 18, No. 6, November 1982, pp. 721-726.

[11] P.S. Maybeck, Stochastic Models, Estimation, and Control, Vol. 2, Academic Press, 1979.

[12] M.S. Murphy and L.M. Silverman, "Image model rep- resentation and line-by-line recursive restoration", IEEE Trans. Automat. Control, Vol. AC-23, No. 5, October 1978, pp. 809-816.

[13] N. E. Nahi, "Role of recursive estimation in statistical image enhancement", Proc. IEEE, Vol. 60, No. 7, July 1972, pp. 734-738.

Vo 17, No. 3, July 1989

226 M. Tanaka, T. Katayama

[14] N.E. Nahi and A. Habibi, "Decision-directed recursive image enhancement", IEEE Trans. Circuits Syst., Vol. CAS-22, No. 2, March 1975, pp. 286-293.

[15] L.G. Roberts, ,'Machine perception of there dimensional solids", in: J.T. Tippett et al., eds., Optical and Electrv- Optical Information Processing, MIT Press, Cambridge, MA, 1965, pp. 159-197,

[16] A. Rosenfeld and A.C. Kak, Digital Picture Processing, Academic Press, 1976.

[17] Y. Sawaragi, T. Katayama and S. Fujishige, "Sequential state estimation with interrupted observations", Inform. Control, Vol. 21, No. 1, August 1972, pp. 56-71.

[18] R.H. Shumway, "Some applications of the EM algorithm to analyzing time series data", in: D. Brillinger et al., eds., Time Series Analysis of Irregularly Observed Data (Lecture Notes in Statistics), Springer-Verlag, 1984, pp. 290-324.

/ Edge detection and restoration

[19] R.H. Shumway and D.S. Stoffer, "An approach to time series smoothing and forecasting using the EM algorithm", J. Time Series Analysis, Vol. 3, No. 4, 1982, pp. 253-264.

[20] L.M. Silverman and F.J. Clara, "Recent results in recur- sire and nonlinear image restoration", in: A. Bensoussan and J.L. Lions, eds., Analysis and Optimization of Systems, (Lecture Notes in Control and Information Sciences, Vol. 28), Springer-Verlag, 1980, pp. 721-743.

[21] M. Tanaka and T. Katayama, "Robust Kalman filter for linear discrete time system with gaussian sum noises", lnt J. Systems Sci., Vol. 18, September 1987, pp. 1721-1731.

[22] J. Woods, "Two-dimensional Kalman filtering", in: T.S. Huang, ed., Two-Dimensional Signal Processing I, Sprin- ger-Verlag, Berlin, 1981, pp. 155-205.

Signal Processing