Detection and smoothing of edge contours in images by one-dimensional Kalman techniques

IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS, VOL. 20, NO. 1, JANUARY/FEBRUARY 1990 159

Detection and Smoothing of Edge Contours in Images by One-Dimensional

Kalman Techniques

Ahstrucf -An edge model in images is developed and the associated edge detection and contour determination algorithms are developed based on statistical signal processing approaches. An edge point along any scan direction is defined as a sufficient jump in the mean of a time series that is assumed to be white Gaussian around the edge point with the same variance on either side. An edge contour is defined as a sequence of such edge points, each point denoted by the two coordinates of the image plane. Each coordinate sequence is modeled as first order auto regressive Gauss- ian over and above a straight line sequence of arbitrary, finite slope and intercept. The first stage of our overall algorithm examines the time series along each row and each column, forms a window around each potential edge point, detects the location and estimates the variance of the error in the location of the edge point, by pattern recognition techniques. The second stage forms noisy edge contours by graph searching techniques. The third-stage smoothing of an edge contour is formulated as a well defined Kalman smoothing problem. It is shown that the parameters of the system and noise can all be estimated from the data itself. Experimental results on a simple image are discussed.

I. INTRODUCTION

DGE DETECTION and edge following are useful in E several areas of image processing, pattern recognition, robotics, and medical imaging. Use of the edge contour rather than most other preprocessed data helps in simpler representation, registration, and identification. Thus the problems of edge extraction and tracking have been widely addressed with proposed solutions ranging over simple thresholding, signal processing, and filtering techniques [1]-[4], definition of admissible edge profile functions [ 5 ] , time series analysis approach [6], [7], etc. Desirable re- quirements of a good edge detection and tracking scheme are low computation, ability to work with subpictures in sequential/parallel procedures, intuitive and analytical justification, and good and uniform accuracy over a wide variety of images. No such scheme is available mainly because of our inability to give a good mathematical

Manuscript received December 10, 1988; revised June 17, 1989. This paper was partially presented at the IEEE International Conference on Acoustics, Speech, and Signal Processing, New York, Apr. 1988.

G. R. Dattatreya is with the Computer Science Program, University of Texas at Dallas. MS MP 3.1, P.O. Box 830688, Richardson, Texas

L. N. Kana1 is with the Department of Computer Science, University

IEEE Log Number 8931979.

75083-0688.

of Maryland, College Park, MD 20742.

definition of edges in practical images. This paper de- scribes studies on the edge detection and tracking problem with an edge point defined as any abrupt change in the local average of gray levels.

Basseville [7] and Basseville et al. [6] study several such sequential detectors of change in mean for edge detection. They also use an edge following algorithm to smooth the detected edge points on successive scan lines. Let a time series { x, } change its mean from p o to - p o at some point k + 1 > 1. { x I } is considered to be the gray levels of pixels along a scan line and k , the edge point. The expected value of the cumulative sum, E{Cy=,x,} attains a maximum at n = k. If the variance of x, is relatively small, the cumulative series X;=lxI is expected to reach a maximum at n = k and fall off more or less monotonically. Since the cumulative sum can reach a maximum at some j not only due to a jump down in the mean but also due to random fluctuations, j is not taken to be an edge point unless and until the cumulative sum falls below the attained maximum by a predetermined quantity. This is the principle behind Hinkley’s algorithm used by Basseville [7]. If the sum does fall down by the required amount, an alarm indicating the detection of an edge point is given. The actual edge point would be a few pixels behind the alarm and this bias is also predetermined if the two mean values ( p o and - p o in this simple case) are known. Modifications for arbitrary but known means p1 and p 2 including positive jumps are trivial. When the means are unknown, the first mean is estimated from the first few samples and the detector looks for a jump in mean of at least & p o . But one problem then is that the bias on the alarm (to be subtracted from the alarm to get the actual edge point) is neither known nor constant. The solution we propose to this problem is a closer examination of the subseries, once an alarm is given. This results in an adaptive thresholding scheme for edge detection. The implementation is computationally efficient.

Edge tracking in images can be either combined with edge detection itself or achieved by a second level algorithm once edge points along scan lines are detected by any method. Basseville et al. [7] use a noisy straight-line model of edges with the vertical axis as the independent variable. Edges are assumed to flow downwards (possibly

001 8-9472/90/0100-0159$01 .OO 01 990 IEEE

160 lEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS, VOL. 20, NO. 1, JANUARY/FEBRUARY 1990

inclined to the vertical axis). The typical situation is that a set of edges have been followed (up to a certain horizontal level) as noisy straight-lines. The edge points (detected by the first level time series algorithm) along the next ordinate (horizontal scan line) are used to 1) continue the existing edges, 2) start some new edge/s, or 3) terminate some existing edges. Depending on the noise variance of individ- ual noisy straight-lines, windows of admissibility are defined. Decision rules are then used to filter, predict, start, or stop edge contours. This method is not suitable for horizontal or near horizontal edges. A general suggestion for tackling horizontal edges, by storing local averages of gray levels along successive scan lines, is given in [7].

Our viewpoint is that an edge should be followed and smoothed as it proceeds and should not be viewed as a function of a predetermined coordinate. To this end, we propose to form two time series for each edge contour. The two time series are the horizontal and vertical coordinates of the contour. Both these are modeled as first order autoregressive (AR) processes. The noise and the AR parameters can be estimated from the experimental data itself. The uncertainty in the line by line detected edge points is included in the smoothing model. With these estimated model parameters, a Kalman smoothed version of the data can be obtained as the final edge contour. This is computationally superior to two-dimensional (2-D) filtering whle not treating a contour as a one-dimensional (1-D) function. The size of the image poses no problem since smoothing is done only on the detected edge points.

These ideas are developed further to design computational procedures for edge detection and smoothing. Sec- tion I1 discusses line by line detection leading to the adaptive thresholding scheme. A contour extraction procedure is described in Section 111. Section IV discusses preprocessing techniques for preparing the data for smoothing. Kalman techniques for edge smoothing are given in Section V. Section VI points out some experimental observations and Section VI1 presents concluding re- marks.

11. LINE BY LINE EDGE DETECTION

The detection of a jump in the mean value of gray levels along a scan line is worked out starting from the following simple problem. A whte Gaussian time series { x i } with a constant variance U changes its mean from p1 to p2 soon after some K between i = 1; . ., N . What is the best estimate of the onset of change? Assuming that the onset is a random variable uniformly distributed between 1 and N , this problem is solved by maximizing the a posteriori probability conditioned on the observation. Then

P ( K = k lx , ; * - , x N ) N \ l

Maximizing this expression with respect to k is identical to minimizing

k N c (xi - P A 2 + C ( X i - P 2 ) * i =1 i = k + l

with respect to k and this simplifies to

The difference will reach an extremum soon after which the mean changes. The implication is that the cumulative sum should be compared with a linearly increasing thresh- old. While (1) is good for detecting the presence of a jump, it is inadequate for a good estimation of its location. We also require the varianFe of the estimate. If p2 is known, K may be estimated by k given by

L N

I = 1 r = r $ + l c ( X , - P l ) + c ( x 1 - 1 1 2 ) = 0 . (2)

Notice that this technique is similar to estimating the mean E [ Y ] of a random variable Y from N samples { y ! } by Cfll(y - E [ Y ] ) = 0. Equation (2) yields

N

C xr - N P ~

(3) L = r=l

P1- P 2

Taking expectations we find that E [ & ] = K showing that (4) is an unbiased estimate. It is shown as

Nu var [ L] =

(EL1 - P d 2 ' (4)

This expression seems to defy our intuition that the estimation variance should decrease with increasing window length N . Actually, when we know for sure that there is a jump (between known pl and p 2 ) somewhere in a window, a smaller window helps in avoiding large deviations.

In practice p1 and p2 are not known. However the user can specify (with some knowledge of the gray-level distri- bution) a minimum 8p such that any jump in the mean by an amount 6p or more should be viewed as an edge point. The p1 can be estimated along with u 2 from the first few pixels as the procedure examines the time series for a possible jump. The cumulative sum Exi is compared with two thresholds k ( p l k 6 p / 2 ) with p1 recursively estimated. The difference is monotonically decreasing (or increasing) as long as the mean does not jump. The instant the difference increases rather than continue to decrease (or decreases rather than continue to increase) is a candidate for a jump in mean to a value greater than p1 + 6 p (or less than p1 - 6p). However the change in the direction

DATTATREYA AND KANAL: DETECTION AND SMOOTHING OF EDGE CONTOURS IN IMAGES 161

(increase rather than decrease) of the expression

can be due to random fluctuations in {xi} and not due to a jump in the mean. Further, the actual jump location can be away from the present instant for the same reason and also since the jump in mean can be by an amount greater than and not exactly equal to 6p. The solution to this problem is adaptive thresholding. On encountering such a candidate jump, the following procedure is adopted. A positive jump is assumed. The modification for a negative jump is obvious.

Recursive estimation of pl is stopped to avoid cor- rupting it by samples with p2 as the mean. Cumulative sums are continued to be compared with thresholds k(pl + 6 p / 2 ) to confirm that the candidate did indeed signal a jump and was not due to local fluctuations. This is done by verifying that the difference, shown in (5) at the candidate instant, continues to be the minimum. If the difference reverts to decreasing further, the candidate is re- jected from being considered as signaling a lump. The delay for such an examination is not critical and can be preset. A smaller window is created with a few samples on either side of the candidate jump. The p2 is estimated over an interval confirmed by Step 2) to be beyond the jump point. Estimates p1 and p2 now available are used in the estimation scheme of (3). The accompanying variance is estimated by (4). The procedure is termed adaptive thresholding since p now being used is different from pl + 6p used in the pilot detection scheme. Similar arguments and procedures hold for a negative jump in the mean. Practical considerations such as the number of samples in the window are discussed in Section VI.

111. CONTOUR EXTRACTION

The line by line detection procedure yields a set of pairs; each pair corresponds to the coordinates of an edge point. To smooth the edge contours, we first need to form the rough contours from the set of edge points. Any technique can be used for t h s purpose, prior to the application of smoothing algorithms of Sections IV and V. For the sake of completeness and for an understanding of the interac- tion required between the first stage detection and the final stage smoothing, we suggest the following graph theoretic procedure. The set of edge points S is maintained as a sparse matrix implemented as a linked list [8]. The S implicitly forms an undirected graph with vertices given by the edge points. Two vertices are adjacent (connected by a graph edge) if they are geometric neighbors of each other. Note that a pixel has at most eight neighbors in a digitized picture. As part of initial cleaning, all isolated vertices are removed. Let rl; . e, r,,, be all the vertices in S such that each r, has exactly one neighbor. The R is the list

(r1; a , rm). Let s1 be the first element in S (when S is searched in a row major order). In the following outline of an iterative computational procedure, R , S , and s1 are dynamic. The C is a list of c,; each c, itself is a list containing the edge points of one rough edge contour, to be smoothed later. The order of edge points in c, is the same as the order in which they form a contour.

BEGIN EXTRACT i = 0; open an empty list C while S is not empty do

i = i + 1; j = 1; open an empty list c,. if R is empty then c,/ = s1 else c,, = rl delete c,/ from S ; if c,/ E R then delete cfJ from R update s1 while cfJ has at least one neighbor do

N = (n,,. . -, n k ) , neighbors of c,,

delete n, from N append N to R delete cIJ from S ; IF c,, E R then delete c,, from R update s1 end (while)

j = j + l ; c , ,=n l

close the list c, = ( c1, end (while)

close the list C = ( c1; . . , c,,) end extract. This procedure performs a depth first search of the

graph formed by connecting neighbors of the edge points detected in the first stage. When the search procedure finds multiple paths, it continues along one and marks the rest to form distinct contours. The procedure can be modi- fied to be more efficient; however in images in which the edge points do not completely smear the image array, this procedure is quite efficient. The procedure does not look for closed contours; if an edge contour turns out to be broken, it will be treated as two separate contours. At the end, contours containing less than a prespecified number of edge points, say five, may be deleted from further consideration.

, elk); append c, to C

Iv. DATA PREPARATION FOR SMOOTHING

A contour obtained from the previous procedure is a sequence of pairs, each pair being an edge point in the 2-D image. Further, successive elements of the sequence are roughly the successive edge points along the unsmoothed contour. Consider the edge contour of a circle of about 14 pixels diameter to illustrate our modeling of the two time series, one for each coordinate. Fig. 1 shows the sequence of the edge points. We form two real numbered sequences, one for each coordinate. These sequences are shown in Fig. 2(a) and (b). We fit the best straight line for each sequence by a least-square fit and consider the residual sequences. These are shown in Figs. 3(a) and (b). The residual sequences resulting from a true edge contour are modeled as sample sequences from zero mean, first order autoregressive (AR) process. An examination of the sequences in

162

10

5 ’

IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS, VOL. 2 0 , NO. 1 , JANUARY/FEBRUARY 1990

-

. . . . . . . . .

I I

8 5 10

Fig. 1. Edge contour of circle of approximately fourteen pixels in diameter.

Figs. 3(a) and 3(b) provides intuitive justification of the model. Note that the sequences result from the edge contour of a high curvature full circle. The residuals resulting from a noisy-edge contour is modeled as AR sequences plus noise. We now show that the noisy-edge contour together with the noise parameter estimated in the line by line detection, and the contour extraction scheme are sufficient to apply the above time series model for smoothing. Let { y, } be a sequence of real numbers representing one coordinate of the edge points along a contour. This corresponds to one of the sequences in Figs. 2(a) and (b) in the example. This sequence is modeled as a noisy first order AR process over and above a straight line of arbitrary slope and intercept. The random variable r, is therefore given by

r, =a,+ a,i + X ( i ) + y ( i ) ( 6 )

where a, + a,i + X(i) is the true edge contour and y(i) is the noise due to the uncertainty in the line by line edge detection. The y(i) is assumed to be Gaussian with a variance p2 given by an averaged version of the estimated Nu2/1pl - p2I2. The true edge contour is a, + a,i plus the zero mean first order Gaussian X ( i ) given by

X ( i + l ) = + X ( i ) + w ( i ) (7)

with @ as the AR parameter and w(i ) , a zero mean white Gaussian sequence. This model for the edge contour is not as complicated as it appears at first. It is a signal (true-edge contour) plus noise (uncertainty in the first-level detection). The signal is modeled as a first order AR process. The AR process mean is a straight line of arbitrary inclination and hence the term a0 + a,i. The inclination is with respect to the axis of the index of the edge contour sequence and not with respect to one of the coordinate axes of the image. The parameters can all be estimated from the data itself. First, the straight line parameters a, and a, are computed bv a least sauare fit. If the seauence is from 1 to N . a, and

0 10 20

0 i0 20 38

(b) Fig. 2. (a) x-coordinate sequence and (b) y-coordinate sequence of

circle contour.

a, are given by N N N

~ i y i - a l ~ i 2 - a o ~ i = 0 i=l i = l i =1

N N

C y i - a l C i -a ,N=O. (8) i = l i = l

Once a, and a, are computed, a new zero mean sequence is formed as

Z ( i ) = - a,i - a,. (9)

Note that upper case letters denote random variables, and lower cases, their outcomes. The sequence { z ( i ) } is di- rectly amenable to smoothing. Then

Z( i ) = X ( i ) + y( i ) (10)

DATTATREYA AND KANAL: DETECTION AND SMOOTHING OF EDGE CONTOURS IN IMAGES

5 -

-5-

0

163

I I I the independence of certain random variable, we have

E [ z2( i )] = E [ x2 ( i )] + E [ y 2 ( i )] = E [ x'( i)] + p2, (12) . . . _ _

. . . . E [ Z ( i )Z( i +1)] = E [ X ( i ) X ( i +1)] + E [ y ( i ) y ( i +1)]

+ E [ x( i) y ( i + l)]

+ E [ x(i + l )y ( i ) ]

(13)

= + E [ X 2 ( i ) ] , (14)

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . = E [ X ( i) ~ ( i +I ) ] ,

E [ X ( i) X ( i + I)] = +E [ x'( i ) ] + E [ ~ ( i ) w ( i )]

. . . '

. . . and . .

E [ X 2 ( i + l)] =+'E[ X 2 ( i ) ] + E [ w'(i)]. (15) Manipulating (12)-(15), we have

10 ae 33

5 I

. . . .

0 v(0) = 0

(17) += E[Z2(i)]-p2 '

a2 = E [ w'(i)] = [ l - $21 E [ x,"] . and

(18) Equations (16)-(18) together with p2 computed from the first stage detection allow us to compute the required parameters from the data { ~ ( i ) } .

V. KALMAN SMOOTHING

Smoothing is a straightforward computational procedure once the parameters are computed. The data z(i) is first filtered from i = 1 to N and then back smoothed. The equations given are adopted from Sage and Melsa [9] for the present case

0 10 ae 30

(b)

best straight line. Fig. 3. (a) x-coordinate sequence and (b) y-coordinate sequence minus

is the noisy observation of the signal X(i) and

X ( i + l ) =+X(i)+w(i) (11)

is the signal generation model. All sequences are zero mean and Gaussian. The y ( i ) and w(i) are white. The required parameters are

1) p2 the variance of y ( i ) available from the uncertainty estimate in the first stage detection,

2) +, and 3) a2 the variance of w(i).

The sequence { z ( i ) } is the available data computed from { y , } . Taking second moments of (10) and (11) and noting

The filtered version of the data given by f(j) is

f (0) = 0 , (23)

f ( j ) = +f ( j - 1) + K ( j ) [ ( j ) - +f ( j - 111 3

j = l ; * * , N . (24)

The smoothed version is s ( k l N ) for the kth point of the sequence. The N signifies that the smoothed version takes into account all the data. Then

s(" = f ( N L (25)

+ V ( k ) v( k + I l k ) '

A ( k ) =

164 I 1 I t I KAXSAC 1IONS ON SYSTEMS. SlAN. A N D C Y B E R N t I ICs. Vol.. 20. NO. 1. JANC!ARY/I;EBRliARY 1990

( c ) (4 Fig. 4. ( a ) Original picture (b) Line-by-line edge detection result.

(c) Kalman \nioothcd edge contours using half estimated noise variancg. (d ) Kalman mioothcd edge contours using estimated noise variance.

Equations (26) and (27) are computed for k = N - 1: . .,1. The computation involved is simple and iterative. Several variables are introduced only for clarity. The memory and computation are proportional to the length of the edge contour and are not related to the size of the image. After smoothing, the mean straight line sequence a, + cllz should be added to obtain the smoothed sequence corresponding to one coordinate of an edge contour. This, done for the two sequences corresponding to each edge contour (a sequence each for the horizontal and vertical coordinates). yields the final edge points ordered in the same sequence as the contour. These points are real numbers and need to be rounded off to the nearest integers.

VI . EXPERIMENTAL RESUL I S ASD OBSERVATIONS

Fig. 4 shows the photographs of the data and results of our preliminary experiments. Fig. 4(a) shows the 64 x 64 original picture. Fig. 4(b) shows the result of the first stage line-by-line edge detection process. Figs. 4(c) and (d) show Kalman smoothed edge contours. The picture in Fig. 4(c) used Kalman smoothing with half of the estimate of the noise variance; that in Fig. 4(d), the estimate itself. The difference in outputs is minor; the smoothed contour at the bottom compares better with the original picture. Figs. 4(b)-(d) are bright spots of edge points in a dark background. In the photographs, the background is not per- fectly dark merely due to poor reproduction during en- largement is not a result of computer processing.

Several questions arise while attempting to implement the techniques developed in this paper to practical images. In some cases. the second level smoothing may not be necessary. I n the line-by-line detection, the horizontal and vertical procedures will yield many common edge points which should be simple to tackle. I f multiple paths in a

single contour are sufficiently separated, the procedure can confuse this for two edge contours. There is no way to recognize this and merge the two. The contour extraction problem as well as the smoothing problem are not computationally as hard as they may initially appear, since they work only on the detected edge points and not on the entire image.

The line-by-line detection needs a few parameters to be supplied initially.

1) The number of samples to be used for the initial estimation of p1: In experiments conducted, as few as three samples were found to be sufficient for the purpose. The delay: This is the number of points beyond a detected extremum in (3) required to confirm that the extremum is due to a jump in the mean and not due to local fluctuations. A delay of three to four was found adequate for the images experimented on. The minimum jump in the mean for it to be called an edge point: This depends mainly on the range of gray levels although the type of edges and noise can affect. Generally a sixth to a third of the range of gray levels (or the highest gray level) should lead to reasonable results.

2)

3 )

The variance u 2 of gray level is not required for line by line detection but only for the second level smoothing. To reduce computation, this variance and the associated uncertainty in the edge estimation N u 2 / l p , - p r l z can be estimated around a small number of edge points and averaged to be used as a constant over the smoothing operations. Note that N is used as the length of the window in the first-stage detection and as the number of edge points in an unsmoothed contour in the second stage. In order to maintain the uniformity of the first stage detection process over edges with directions ranging from horizontal to vertical, the line by line detection is sug- gested to be used along horizontal as well as vertical scan lines. In large images stores in magnetic tapes, accessing pixels at one coordinate point on several scan lines poses problems of large amounts of data movement. In such cases results of line by line detection along only one direction can be coupled with the second-level smoothing.

VII. CONCLUSION

The major concern of this paper has been the develop- ment of appropriate formulations and solutions to the problem of generating descriptions of smoothed edge contours. A two-phase statistical signal-processing technique is developed for this purpose. The problem of smoothing the edge points detected by time series techniques is formulated as smoothing a Gauss Markoff sequence observed with additive Gaussian noise. The estimation of the model parameters are worked out completely from the data itself. Computations involved are of much smaller complexity than the image size. Experimental results demonstrate the

DATTATREYA AND KANAL: DETECTION AND SMOOTHING OF EDGE CONTOURS IN IMAGES 165

suitability of our approach for images which are not flooded with edges. The second-level smoothing procedure is observed to have considerable effect in edge traclung. Several edge points were moved to new locations and several more got merged to produce smooth and less ambiguous contours.

REFERENCES

T. Peli and D. Malah, “A study of edge detection algorithms,” Comput. Graphics Imuge Processing, vol. 20, pp. 1-21, 1982. L. Davis, “A survey of edge detection techniques,” Comput. Graphics Image Processing, vol. 4, pp. 248-270, 1975. K. S. Shanmugam and C. Paul, “A fast-edge thinning operator,” IEEE Truns. Svst. Man Cybern., vol. SMC-12, pp. 561-572, July/Aug. 1982. K. S. Shanmugam, F. M. Dickey, and J. A. Green, “An optimal frequency domain filter for edge detection in digital pictures,” IEEE Trans. Pattern Anal. Machine Intell., vol. PAM-1, pp. 37-49, Jan. 1979. W. Groch. “Extraction of line shaped objects from aerial images using a special operator to analyze the profiles of functions,” Com- put. Graphics Imuge Processing, vol. 18, pp. 347-358, 1982.

[6] M. Basseville, B. Espiau, and J. Gasnier, “Edge detection using sequential methods for change in level-Part I: A sequential edge detection algorithm,’’ IEEE Trans. Acoust. Speech Signal Processing, vol. ASSP-29, pp. 24-31, Feb. 1981. M. Basseville, “Edge detection using sequential methods for change in level-Part 11: Sequential detection of change in mean,” IEEE Trans. Acoust. Speech Signal Proc., vol. ASSP-29, pp. 32-50, Feb. 1981. E. Horowitz and S. Sahni, Fundamentals of Data Structures in Puscal. A. P. Sage and J. L. Melsa, Estimation Theory with Applications in Communications and Control.

[7]

181

[9] Rockville, MD: Computer Science Press, 1984.

New York: McGraw-Hill, 1971.

G. R. Dattatreya (M87), for a photograph and biography please see page 157 of this TRANSACTIONS.

L. N. Kanal (S’50-A’54-M59-SM64-F‘72), for a photograph and biography please see page 157 of this TRANSACTIONS.

Detection and smoothing of edge contours in images by one-dimensional Kalman techniques

Documents

Transcript of Detection and smoothing of edge contours in images by one-dimensional Kalman techniques