Picture Segmentation by a Tree Traversal Algorithm

STEVEN L. HOROWITZ AND THEODOSIOS P A ' V L I D I S

Princeton University, Princeton, New Jersey

ABSTRACT. In the past, picture segmentation has been performed by merging small primitive regions or by recursively splitting the whole picture. This paper combines the two approaches with signifi- cant increase in processing speed while maintaining small memory requirements. The data structure is described in detail and examples of implementations are given.

KEY WORDS ̂ N~ PHRASES: picture segmentation, region detection, boundary identification, split- and-merge algorithms, chromosome outlines, human face feature outlines, landscape outlines, radiograph outlines

CR eATEGOm~S: 3.63, 5.13, 5.32

1. Introduction

An important problem in picture processing, pattern recognition, and scene analysis is the detection of objects from their background. For high contrast and noise free pictures, differentiation can be used successfully for edge detection and then objects can be identi- fied by an edge following algorithm. However, such a simple process fails for noisy pictures with fuzzy boundaries. This is also true when objects are defined by texture rather than brightness level [1, 2]. The major difficulties are the nonuniformity in the brightness level of objects (high frequency noise) and the loss of contrast (low frequency noise). For a given class of picture, one could use a bandpass filter followed by a thresholding operation. However, the design of these operators is nontrivial and may depend critically on the choice of parameters (e.g. the threshold level). This is particularly true for pic- tures where one must identify many objects, each at a different brightness level. These difficulties have led to the development of alternative techniques, which are usually of one of the following two types.

(1) Direct region detection. The algorithm of Brice and Fennema [2, 3] and subsequent exteusions of it [4] are typical of this kind. Picture elements (pixels) are merged se- quentially to form bigger regions on the basis of global considerations.

(2) Extelwled edge detection. The algorithms of Rosenfeld and Thurston [5, 6], Mar- telli [7], and Hueckel [8] are examples of this approach. A more complex scheme of dif- ferentiation is used in combination with edge following.

In both cases semantic information can be used to improve the results [9, 10]. A generalization of approaches of the first type can be made in terms of functional

approximation. If f(x, y) is the brightness function of a picture defined on a domain D, then we may attempt to divide D into the minimum number of regions where f(x, y) satisfies certain constraints (e.g. is approximately constant). Preliminary work has given encouraging results [11] in the sense that one can obtain large regions quickly

Copyright © 1976, Association for Computing Machinery, Inc. General permission to republish, but not for profit, all or part of this material is granted provided that ACM's copyright notice is given and that reference is made to the publication, to its date of issue, and to the fact that reprinting privileges were granted by permission of the Association for Computing Machinery. This work was supported by the National Science Foundation under Grant GK-36180. Author's address: Computer Science Laboratory, Department of Electrical Engineering, Princeton University, Princeton, NJ 08540.

Journal of the A~ochttion for Computing Machinery. Vol. 23, No. 2, Aprd 1976, pp. 368~388.

Picture Segmentation by a Tree Traversal Algorithm

(a)

369

FIG. 1.

(b)

Illustration of the use of (a) piecewise constant approximations and (b) piecewlse linear approxnnations for segmentation

without the use of semantics, which can be applied later. Two classes of algorithms have been developed. In one, segmentation is first performed along raster lines which are subsequently grouped into bigger regions [11, 12]. In the second, which is the subject of this paper, a direct two-dimensional segmentation is attempted.

In Section 2 we discuss this scheme in the context of other region detection approaches. A detailed comparison with some of the complex edge following schemes [5-8] will be the subject of future work.

As a quick illustration of how functional approximation achieves segmentation, con- sider the profile of a noisy picture shown in Figure 1. A piecewise constant approxima- tion (a) or a piecewise linear approximation (b) with a variable breakpoint can be used to detect the "edge" between the left part and the right part. The use of such approxi- mations for shape description is not new (see [13, ¥ol. ]I , Fig. 10-4]), but it is only recently that fast algorithms applicable to "irregular" data have been developed [14-16].

The effect of such processing is to eliminate high frequency noise without "smearing" boundaries (Figure 1 and also [15, Figs. 3-7]). If a piecewise constant approximation is used then the resulting segmentation usually consists of a few largexegions and a large number of small regions corresponding to transition areas. If necessary, these can be merged quickly by a small region elimination routine.

2. Region Detection Methods

Region detection methods can be divided into the following two types. (1) Merging or bottom-up. The picture is divided into a large number of small regions

(possibly coinciding with single pixels) which are then merged to form larger regions. The "phagocyte" algorithm of Brice and Fennema [3] is typical of this kind. Algorithms which scan a waveform sequentially from left to right to determine the longest interval such that an approximation is below a given tolerance [17] also belong to this class. There have been a number of merging criteria for different types of primitive regions which are discussed in the literature [1-3, 10, 11, 18-20].

(2) Splitting or top-down. The picture is successively divided into smaller and smaller regions until certain criteria are satisfied. There were very few applications of this scheme in picture processing until recently [21, 22], but it has been a well-known method for polygonal approximation of boundaries [2].

3 7 0 S. L. HOROWITZ AND T. PAVLIDIS

I t is possible to devise a mixed split-and.merge scheme, and this has been carried out in the case of waveforms and boundaries with encouraging results [16]. Extension of this strategy is proposed here for pictures. The advantages are (1) an increase in computa- tional speed, and (2) an additional degree of freedom allowing improvement in seg- mentation quMity, both without expanding memory requirements. Below is an abstract formulation of the problem.

Let X be the domain of a picture (e.g. a square). Let ](x, y) be the brightness function defined on X. A logical predicate P is defined on subsets S of X as follows:

{true if there exists a constant a such that If(x, y) - al _< e [see ftn. 1] P(S) = ~ for all points (x, y) E S,

(false otherwise, where e is a prescribed error tolerance.

segmentation of X is a partition of X into subsets S , , i = 1, • • • , m, for some m such teat :

(~) ~ x = U,"L, ~,, (b) S, NS , = ~ for a l l i # j , (c) P(S,) = true for all i, (d) P(S, O S,) = false for all i # j,

p~vided S~ and S, are adjacent in X. Note that the number m is not unique, nor must it be the minimum under which conditions (a ) - (d ) hold [23]. In many practical cases, however, imposing these conditions yields an m close to the minimum [16].

A merging scheme starts with a partition satisfying (c) and proceeds to fulfill (d); a splittinff scheme starts with a partition satisfying (d) and proceeds to fulfill (c). A ~lit-and-merge procedure begins with an arbitrary partition satisfying neither condition ~nd produces a partition satisfying both.

The above formulation can be expressed in a graph theoreticM framework by a tree whose nodes correspond to the subjects of X such that

(i) X is the root of the tree; (ii) the successors of the node corresponding to a subset S of X are the nodes cor-

responding to proper, disjoint, collectively exhaustive subsets of S; and (iii) the leaves of the tree represent the smallest subsets of X under consideration

(e.g. single pixels). ~igure 2 shows an example of such a segmentation tree construction. I t can be shown

that a segmentation corresponds to a node cutset which is the minimal set of nodes selaarating the root from all the leaves. The root R may be separated from a leaf L in one of three ways:

(i) R is in the cutset. (ii) L is in the eutset.

(iii) There exists one and only one node N in the outset such that R is an ancestor of N and L is a descendent of N. This implies that there is no other cutset closer to the root with nodes where P(S) is true. The terms top-down and bottom-up used before 9an be visualized in terms of moves along the segmentation tree and the corresponding alterations on the original node cutset.

Another operation, known as sidewise merging or grouping [11], is accomplished either by transforming the tree structure by combining leaves, or by partitioning the final cut- set !nto region equivalence classes, based on node maxima and minima with respect to the prescribed error tolerance as discussed earlier. Thus adjacent nodes with different predecessors and belonging to different levels may be merged or grouped together to form larger irregular subsets S (possibly oi genus greater than zero) provided P(S) holds true. Figure 3 shows an example of grouping and the resulting modifications in tree; structure.

t Equ iva l en t to [maxz if(x, y)) - rains (f(x, y))] ~. 2e.

Picture Segmentation by a Tree Traversal Algorithm

/ ,' / " ,

(Q)

FIG. 2.

4

II 12 13 21

221 222 231 232 311 312 313 321 322 3

(b)

(a) Example of a directed region segmentation; (b) tree represeutiug the above segmentation

SECOND ( ~ / ~ ) SPLIT MERGE 12 8 22=[,,,_,..._~ ~.

FIRST SPLIT

37t

II 12 21 22 II 12~22 21 I'1 1'2

FIG. 3. Illustration of the split-and-merge procedure

I t is obvious that this method can be generalized to other than piecewise constant approximations by redefining the predicate P ( S ) . For example, a segmentation by texture can be achieved by replacing the condition I f ( x , y) - a I ~ e with

where m and n are small numbers and F(u, v) is the Fourier Transform ranked in order of significant components.

For simplicity, all the discussion in this paper is limited to the piecewise consonant case. However, the algorithm is directly extendable to other criteria and all tha t i~

372 S. L. HOROWITZ AND T. PAVLIDIS

required is to replace the statements involved directly with the evaluation of P(S). Such statements are flagged in the sequence with the $ sign. In particular, a comparison to 2e should be replaced by an evaluation of P (S) , and the evaluation of maxima and minima by whatever calculations are needed for finding an approximation.

The choice of the parameter e is obviously very important. In essence, 2e is the tol- erated variation of brightness within a single region. I ts exact value depends on the class of pictures under consideration and, in particular, the brightness distribution over the scene. Probably the easiest way to estimate it is to try a few different values on a typical sample and then choose the one giving the best results. In general, the precise value is not too important (in contrast, say, to the choice of threshold). Our experience has been that e = 16 will give reasonable results for any pictures whose brightness varies from 0 to 127. For other ranges a proportionally scaled value should be used.

3. Computational Considerations

In most practical applications the computational efforts involved in splitting and merg- ing are not the same. Merging d subsets into a single set requires two groups of d com- parisons (updating the maximum and minimum values of f(x, y) on the domain of the subsets). On the other hand, splitting a single set into d subsets necessitates a reevalu- ation of the maximum and minimum off (x , y) on the domain of each of the d subsets. Let

cost of merging {S,, . . . , Sd} into So = C,~,

cost of splitting So into { S, , • • • , Sd} = C,.

Note that C= is independent of iS01 but dependent on d (i.e. Cm = 2.d.$) while C, is dependent on IS01 but independent of d (i.e. C, = 2-1S0].~), where ~ is the cost of a single comparison.

Common as the above model might be, there may be cases where a more symmetrical situation exists, especially in uniform approximation by high order polynomials; also, the dependence on size need not be linear [23].

The choice between a bottom-up or top-down procedure should depend on the level of the tree at which the majority of the nodes in the cutset corresponding to the final segmentation occurs. Asymptotically the top-down procedure is of order IXI IOgd IX[ while the bottom-up is of order ]XI, where d, as before, is also the node degree of the tree. The difference in asymptotic behavior explains the popularity of bottom-up methods in the literature as compared to top-down methods.

Theoretically and experimentally, savings in computation can be achieved if an inter- mediate level of the tree is chosen as a starting cutset. Such a choice should attempt to minimize the number of splits and merges needed to reach the final cutset, which usually encompasses many levels (some nodes close to the root and others close to or coinciding with leaves) due to the nonuniformity in the distribution of detail over a typical picture. Figure 4(a) illustrates the computational effort yielded by applying the two methods.

Fxo. 4.

f ~ SEGMENTATION TREES 4

(a ) (b) = CUTSET CORRESPONDING

TO FINAL SEGMENTATION

• m = NUMBER OF MERGES

dis = NUMBER OF SPLITS

Illustration of the computational economy achieved by the split-and-merge method

Picture Segmentation by a Tree Traversal Algorithm 373

If I, and lm are such initial levels for top-down and bot tom-up methods respectively, a combined split-and-merge procedure, s tar t ing at a middle level/o such tha t l,~ < /o < l , , can move in both directions with an overall economy as a result. Figure 4 (b) i l lustrates the smaller computational effort associated with a spht-and-merge approach. Tha t this is indeed the case has been verified for waveforms and boundaries [16, 23], and holds true for the picture examples presented in this paper.

For a given class of pictures a certain distr ibution of the number and sizes of regions is expected. This should determine the choice of the star t ing level, i.e. one where the block size is near the average expected size.

4. Implementation of Split-and-Merge

In order to process an N X N picture P (where N = 2z~), the concepts of "segmentation t ree" and "node cutset" can be utilized. The segmentation tree is implemented as follows. Each node corresponds to a square picture region segment (block). Leaves of the tree are one-by-one blocks (i.e. single pixels) at level log2N = l~, and all other nodes are defined recursively. A level I node b (where 0 <~ l < l~) which is located ( taken to mean the index of the upper left corner pixel of the corresponding block) at (x, y) has sides of length z = N / 2 ~ and has four successors with sides of length z/2 located a t (x, y) , (x + z/2, y), (x, y + z/2) , and (x -k z/2, y + z/2). Note tha t the above definition has the root node (level 0) located at (1, 1). This is, of course, the complete picture with sides of length N. Aside from specifying (Xk, Yk) and Zk, each node bk has associated values Mk and mk equal to the maximum and minimum of the brightness function f (x , y) on the corresponding block.

Rather than storing the complete tree in memory (necessitating ~x"~N 4' 5(4N ~ o ~ - - 0 "~

-- 1) /3 integers), only the cutset is stored as 5 arrays (xk, Yk, Zk, Mk , m~) of size N 2. This is achieved by a suitable encoding for the position of bk in the array given (xk , yk) which enables us to retrieve the tree structure, i.e. subsets of four nodes and their common predecessor. Init ial ization of the array at level l0 with size of the block sides so = N / 2 l° is performed as follows:

k 4-- 0; DOi = 1 T O N B Y s o ;

D O j = 1 T O N B Y s 0 ; k~-- k + 1; x~-- i ; yk4---3; zk~---so ; M ~ MAX(p,.~ :~ = x k , " " ,xk + so,J = y~, "'" ,yk + s0); $ mk~MIN(p , . , : i = z k , . . . , x ~ + s 0 , j = Y k , ' ' ' , Y k T s 0 ) ; $ END;

END,

Note tha t k contains the value Co = 4 l° after the initialization step (the number of nodes in the initial cutset) .

Now nodes bkl, bk2, bk~, bk4 have a common predecessor bk0 located at (xk0, y~0) if:

k l = 2t°.[(xko -- 1) -t- 21°(yko -- 1) ] /N -I- 1,

k2 = k l + 2 l°-z, k3 = k l + 2 2~°-1, k4 = k 3 + 2 z°-t,

for a given level l (where 0 < l _< 10). Figure 5 il lustrates the relationship between the five nodes and represents some of the associated variables.

Merging nodes bkl, bk2, bka, bk4 is a process tha t removes the four nodes from the cutset and replaces them with node bko (actual ly bkl updated) . In the following listing, s denotes the size of the block sides.

14-lo;84-8o; DO WHILE I > 0;

DO FOR ALL bkl 6 level l; IF z~l = zk~ = z~a = zk~ = s THEN DO,

374

]PIG. 5.

(xko.,Yko) Zko

OMko

bkO SPLIT L

S. L. HOI~OWITZ AND T. PAVLIDIS

(Xkl ,Ykl ) Zkl "Zk2

OMkl bkl bk2

bk3 bk4 • mkO omk4

LEVEL ~ LEVELJ÷I

Illustration of some parameters involved in the splitting or merging of a node

g ~-- MAX(Mkl, Mk~, Mu, Mk4); $ h ~ MIN(mkl, m,2, rn~3, m~i); $ I F a - h ~ 2eTHENDO; $

z,~ ~-- 0; z,~a ~-- 0; z~ <-- 0; z~l * - 2s; M~l ~ g; real ~ h; $ END;

END; END;

l ~ - l - 1; s~-2s; END;

The cutset still has Co nodes, but merged nodes either have been updated or have had z set to zero.

Splitting a node bko (actually bk~ before updating) is a process that removes the node from the cutset and replaces it with nodes bkl, b~ , bk3, b~4.

[see ftn. 2]

c<--Co; DOE1 = 1 TOc;

IF 0 < zkt _< so THEN DO; l ~ - l o ; s * - S o ; DO WHILE l < l~ ;

IF M~i - m,z > 2e THEN DO; k2~---c + 1; k ~ * - c + 2; k ~ - - - c + 3; z t l ¢-- 8 / 2 ; zk~ ¢ - zkt ; zt~ ¢-- zt~ ; Zk4 ~ Zk l ;

ZLg 4"-- X k l ; XL3 ~ " X k l "3 L Zk l ; ZL4 ~ Xk3 ;

Y ~ ~ ykl + Zkl ; yk* ~ Yzt ; Yk* ~-- yk2 ; [Mkl, "" , Mt~ and mr1 , .." , m~4 are calculated using values of p,.~ as demonstrated previously], l ~ - l + l; c ~ - c ~ 3 END;

ELSE l ~ 1N ; END;

END; END;

$ [see ftn. 2]

This yields the final eutset satisfying properties ( a ) - ( d ) in Section 2. I t is easy to see that both processes terminate. Merging can proceed until the outset

consists solely of the root node (e.g. a one-color picture), and splitting can proceed unti l the cutset consists of the 4 ~ = N ~ leaf nodes only (e.g. an alternating two-color picture). Furthermore, the two processes are mutually exclusive (i.e. all of the merging operations are, followed by all of the splitting operations without violating final cutset properties), since once four nodes have been merged they cannot be split. Also, nodes that have been split cannot be merged back to their original configuration, although merging with other adjacent nodes is possible through grouping, which is the third step in the processing sequence.

s As defined in Section 2; see ftn. 1.

Picture Segmentation by a Tree Traversal Algorithm 375

5. I~nplementation of Grouping

The split-and-merge step is followed by a grouping procedure in order to remove "arbi- t rary" region boundaries imposed by the "arbi trary" segmentation inherent in the data structure. For example, a natural region which was partitioned originally into two different blocks that are never merged will stay partitioned among the successors of these blocks in the tree. Grouping abandons the tree structure and examines adjacent unrelated blocks found in the final cutset. Block adjacency is obtained from a matrix A initialized as follows:

DO k ~- 1 TO c SUCH THAT z, ~ 0; DOz = x~TOxk~sk;

DOj = ykTOy~÷s , ; a,.3 *- k; END;

END; END;

Thus if a point p,.~ in the picture is a member of the block specified by node b~, the corresponding entry a,., in the adjacency matrix contains k. Grouping is then executed by a labeling algorithm [24]. A block is either unlabeled and unscanned (zk > 0) or labeled and unscanned (zk < 0 and bk is on the stack S), or labeled and scanned (z~ < 0 and bk is not on the stack S). Nodes that are labeled during the n th application of the algorithm are placed consecutively on the n th region list R . . Also the kth block b~ has the value of a new associated variable rk set to the region number n to which it is as- signed:

S ~-- empty; n ~-- 0; DOk = 1TOc;

IF z, > 0 THEN DO; zk ~- -- z, ; PUSH(b,) --~ S; u ~-- M,; v ~ ink; n ~ n + l , s i z e ~ 0 ; s u m ~ 0 ; DO WHILE S~ empty;

POP (b,) ~-- S; PUSH(b.) ~ R~ ; r, *- n; s i ~ ~ size~ + z? ; DOa = x, T O z , - - z , - 1;

DOE = y, T O y , - - z ,-- 1; SUM, ~-- SUM~ + p~.~; END;

END; DO FOR ALL b, E A SUCH THAT b~ @ b,; [see ftn. 3]

g ~-- MAX(u, M~); h ~-- MIN(v, m,); $ IF g - h < 2e THEN DO; $ [see ftn. 4]

PUSH(be) --~ S; u ~ g; v ~-- h; $ END;

END; END;

END; END;

Note that n contains the final number of regions, sizes contains the number of pixels, and sum,/size, is the average brightness level of the region specified by R , . Figure 6 shows an example of the grouping procedure outlined above. The resulting partit ioning of the cutset into equivalence classes determining the tree structure transformations is obviously a segmentation satisfying properties ( a ) - (d ) in Section 3.

I t should be pointed out that the results of the grouping process are order dependent, since when a block is added to a region the maximum and the minimum of the bright- ness in that region are updated. This is, of course, a common feature of all techniques

s ~ = "is adjacent to." ' e is as defined in Section 2 (ftn. 2).

376 S. L. HOROWITZ AND T. PAVLIDIS

I I 4 5

I 6 7 I0 II

f¢ [ ] 8 9 12 13

REGION I = ( I ,4 ,7 ,10) REGION U = (6 ,8 ,9 ,12 ) REGIONI]] = (2,3,5,11 ) REGION l~ • (13)

FIe. 6. Illustration of the grouping procedure on a representative picture section with corresponding adjacency matrix

which update the features of a region once new points are added to it. In practice, such order dependence does not create any serious problems (see examples).

After grouping, the picture has been divided into regions where the brightness level remains within the limits specified in Section 3 and adhered to in Section 4. However, such a partition still does not correspond necessarily to a normal visual impression of the picture. There are two reasons for this, both causing the presence of many more regions than "seen." The first is the existence of narrow transition zones between large regions. These zones are not perceived visually as separate entities; instead they are made components of the "real" regions. Second is the existence of high frequency noise. These artifacts are not eliminated by the original approximation using the minimum and maximum brightness level criteriort of Section 3. Both problems are well known and usually have been treated on the basis of semantic information [3, 9, 10]. Strong and Rosenfeld have presented a detailed discussion of this problem [25].

In practice, most of these "odd" regions are of very small size (quite often single plxels). This bimodal size distribution [12] suggests that significant improvements can be achieved if small regions are eliminated by combining them with the "nearest neighbor," defined to be the adjacent region with the smallest difference in average brightness. In addition, regions with very similar average brightnesses are merged without regard to size. An average value is used at this stage to overcome the constraints imposed by the minimax value. Also, average brightness is in closer agreement with visual impression and results in higher contrast. For a discussion of various error norms in this context, see [23].

In order to merge adjacent regions, a new data structure providing region adjacency information is needed. By utilizing the relations present in the block adjacency matrix A, the region lists R, and the block-region associations r, it is possible to create (in one pass) a region adjacency graph whose nodes correspond to regions and whose branches repre- sent the adjacency relationship. Let n be the number of regions and ~/= ~ - i d, , where d~ is the number of regions adjacent to region i (i.e. the degree of graph node i). Since

is much smaller than n in practice, the graph is sparse and should be represented in adjacency list form requiring order nd storage rather than in other forms requiring order n 2 space [24]. The complete graph need not be stored in memory. Instead, each adjacency list L is generated and accessed only once, thereby using at most max,(d,) locations.

The elimination algorithm proceeds as follows (es,~, is the largest region to be combined and ear= is the largest difference in average brightness to cause adjacent regions to be merged) :

DO i = 1 TO n; IF size, > 0 THEN DO; DO FOR ALL bk E R, ;

DO FOR ALL bj E A SUCH THAT b~ @ bk;

Picture Segmentation by a Tree Traversal Algorithm

0

¢+,J-~)O=

(t-~.j) 0

®~ ~--0 c,, i+.

0 0 0 (i+1 ,.I)

Fro. 7. Elementary search direc- tions during boundary tracing

', • ~ ~ o

i i

. - 4 5----o

o o o +

V~I~'I'EX

FIG. 8. Production of sing|e-pixe|-width boundaries

IF (r e ~ ~ A r~ q L) THEN PUSH (r~) --~ L; END;

END; avg ~-- MINleLIsumz/mze~ -- sum,/size,I, m ~-- INDEX (MIN~ L[sumdmze~ -- sum,/size,[) ; DO WHILE (size, ~ es,~ A avg ~ e~vt)"

size, ~ raze, + size~; SUM, ~-- sum+ ~ summ; R, ~ RdIR~; size~ ~ 0; DO FOR ALL b~ E R~;

rk ~-- i; DO FOR ALL b~ E A SUCH THAT b~ ~ bk;

IF (r~ ~ i A r~ ~ m/k r~ ¢ L) THEN PUSH (r~) --~ L; END;

avg +- MINi e n Jsumdsizez -- sum./size. [ ; m <-- INDEX (MINze L[sumdsizez -- sum,/size,[, END;

END; END;

END;

377

[see ftn. 5]

6. Boundary Extraction

One advantage of picture segmentation by direct part i t ioning into regions is tha t closed boundaries are readily determined from the region description. This is in contras t to region formation through edge detection, where extensive tracking is necessary, often yielding contours which are not closed (5-8]. All t ha t is necessary here is a list of regions specified by their upper r ight-hand corners (x , , y,) and a matr ix A such tha t if p,.~ belongs to region k then a,.~ = k.

Beginning with (x , , y,) , points in the boundary can be extracted by a simple tracing procedure. Only four directions are searched (see Figure 7), thereby enabling a symmetric boundary extension of one unit in two directions in order to avoid having boundaries of two-pixel width (see Figure 8) [12]. This is preferable to the scheme introduced by Brice and Fennema [3], which extends boundaries by ½ unit in 8 directions necessitating a new matrix of 4N 2.

As the extended boundaries are identified, points where three or four regions meet can be found, also in linear t ime (see Figure 8). Once these picture "vert ices" are de- termined, the boundaries are easily part i t ioned and can be approximated by piecewise polynomials for further da ta compaction or feature analysis [12, 16]. In the present im- plementation a simple editing has been performed by replacing linear sections of bound- aries by their endpoints. This reduces significantly the total number of points pr needed to specify the region boundaries.

+ II = "is concatenated with."

~ 8 s . L . HOROW1TZ AND T. PAVLIDIS

7. Examples of Implementation

The algorithms described in the previous section were implemented on an IBM 360/91 machine using the PL/1 optimizing and Fortran H compilers. Samples were 7-bit, 64X64 pictures (0 < f(x, y) < 127, 1, = 6, n ~ = 4096). However the actual number of gray levels could be smaller. In order to illustrate the generality of the method, a number of dissimilar subjects were used: human faces, chest x-rays, landscapes, and chromosome pictures. All were digitized using a TV camera connected to an HP 2116 minicomputer. The results are shown in Figures 9-12. In each case, (a) is the original produced by a line printer program [26], (b) is a CALCOMP plot of the boundaries, and (c) is the gray level histogram. Vertices (where three or four regions meet) are marked by little squares. On each original the listed parameters D and L indicate the minimum and maximum of the gray scale used by the line printer program. In each case, D and L were chosen near the extrema of the actual range of gray levels. The symbols used for each level are shown beneath the histogram.

Table I summarizes the statistics of the processing operations. 10, c0, and e are as defined in Section 3, e~v~ and e .... are as defined in Section 5, and the rest of the symbols have the following meanings:

M number of merge operations S number of split operations B number of nodes in the cutset after the split-and-merge step

R, number of regions after the grouping step Rj number of regions after the elimination step

P, P' number of extended boundary points before and after the compaction operation described in Section 6

Y number of vertices as defined in Section 6.

In one example the segmentation is rather simple (Figure 12) and one could achieve comparable results by thresholding. However, the choice of the threshold value is non- trivial [27], while the algorithms proposed require as input only error tolerances such as e, e~v~, and e ..... This can be chosen and remain invariant for large classes of pictures.

8. Discussion

A successful picture segmentation must take into account semantics. However, it is desirable to reduce the amount of data before that step by a general scheme like the one described in this paper. The examples of Section 7 show that one can go quite far in obtaining reasonable segmentations without any significant semantic information. The rather substantial programming effort for the efficient implementation of the initial segmentation algorithm need not be repeated as the subject of interest changes. Only a small number of parameters must be varied. The result is both data compaction and organization. Algorithms of high computational complexity can be used afterward on this reduced description.

Fl~ure i (sub3ect) I

9 (face)

~C (X-Ray)

{S£ene)

12 (Chrom.)

TABLE I. STATISTICS OF SEGMENTATION

%0

5

4

5

5

c le 0 I

1024 16

256 I 16 16

1024 I 9 9

1024 1 12

f B ;eavg eslze M S R l

18 16 164 151 985 148

16 34 21 217 49

15 164 1391949 186 i

12 16 223!i 80 5951115

f Rf

!13

ii

13

6

P P' V T*

1272 281 16 4.59

1008 62 19 ].45 l

]2101125 23 3.62 I

8001 192 4 3. 12

pzcture processzng tzme in seconds (~ 10%) - domlnated by I/O.

Picture Segmentation by a Tree Traversal Algorithm 379

~" ~ ~~' '

I

, =

N

.~

• . ~ e I

• ° 0 • ' . • i

to °.. o " ' i • . .0 • • ~ I I I I I I W E i ~ E E ~ m ~ m ~ ~ 0 0 ~II~

0 " ~ n R m a a R a I g O O o a o o o o m a ~ o i g o o o o o l o o m o m i ~ g o i o o l o l o o a

~ m ~ I O m m o R m g s g g a g ~ o g o g g g o o ~ o l m m o u m e o o g o o o l g Q g m i l l 0 0 1 • ~ E I I I I i O i i i m Immm m o e m e e o e a m n o o e o e m o e o e e e e e e

- ~ . . . . . . u w w . . I I S l . I I u H l . ~ l r - - u o u o o o . . . . . . . H.= II-- .IIIII.

"Do~ImImimoIO +++~o~Im

~KIIIIIIIIIIIIK: } ~ ...... I . . . i i HIeIII~M + II II It ~ + ÷~II * ~ ~ H m~N. ~ 0I~m÷ , ,

, - - o - - - - - - o - ~ : ~ ' : : I ' ~ . : " : . . . . . . . . . o . . . . . . . . . . . . . ~ A ~ " . . . . . . - * .

O::IIOI .... IOOIn,® .... 0 . . . . . . . . O:~::I . . . . . . . . . . . . . . ~ . . . . . . . . . . OI#

+IIOIiIIIIIOiOi ~ I I * * ~ I ~ B I Z Z k O + i i i + ~ , + + , I~IiI+

.*.

I I I I I I O I O O I I I I I I O l O l I I i i O i I i i O i I I O m o o I I I l I O O O l I O I O O i O i i i i ~ l " " °'~IOIIIIIIIIIIIIIOIIHiI~iIi I O I I I I I I O I I I D O I I I I O I O I i i i i

I I "I~IIIIIOIII i O i O i I O I i i i i i ~ I I I I I O I O I I I O I O O i O i O i O I i i i • , , , l . ~ o ® B i O O O i l g o i e e i H i i o i i e i i ~ i i i i i i i i i i i i i O i g e O g i i O i I

i i i I ~I *~ i i ~ . i ~ i i i i i i i i i i I O i i i i O i i i ~ i i I i i i i i i i i i i i i i i H i i O I O • • • • i. I .Ii~ " ° ° I I ~ I I I I I O I O O i O IIIIIIIIOIOIOIOOOOiiii

I • I~ i i I e , ~ i i i i . ~ " ' , ' + t N ~ Z ~ I I m i I D I O I I I O g I I I O O i I i i I i O I ~I~I ~ I f I ~ I ~ I I ~ e I e I ~ ® ~ K I ~ I I I I I I e I O O O I I I I I I I I O I I

I ~ • I ~ I I • • • * • I ~ , ~ • • I i I I . I I I ~ . 1 • • • | - I • . • ~ 1 I I I I * C • i o • I • I I . , I ! o I • I • • ¢ I • • O I • • o . . , . , • ~ ~ I I l l l t

O l I I I " I I " ° ÷ I I ÷ " ÷ + N ~ m l I I f I t I t " I I I I f ° I @ @ " ° I I I • " ° • • • e e I I I + ° ° + I I e , • t r i m

: : ~ e~ I I t , l l l l , , , I e l . i 1 . * 1 I I I • t , I , l l t ¢ . . 1 1 i i i i i , . , . l l , , @ e l ¢ . . ~ i i , , i i i ~ t i I . . I , o e , . . | i . . . , • . 1 . , . . . . . . I I H + H I I l I • o , . I i I • I " ' • o I " " ° " ° I • I e l I I i . • . . . i i i i . . . . . I " * " " ' " . . . . i i , o i ~ ~1 i l l . ,

I I I • I I I I I , , I . . . . I . , , , . , I I I I I I I m, + , . + . o I I I I I I U H I I I t I f l [ ~

3 8 0 s . L . HOROWITZ AND T. PAVLID1S

~ O

%%%%%"

# t t t ~ t t # l l i"

****mm

Picture Segmentation by a Tree Traversal Algorithm 381

T

-L tJ-m~ T

, I . .L T- L

L

L

iii!!!i iiiiiiiiiiiiiiiiii iiiiiiiiiiiiiiiiii!iiiii !!ii!!!! l i ~ l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l ~ O i l ÷ l ! l ~ { ~ { ~ = l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l ~ t l l ~ l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l I ~ l t ~ l l l n l l l l l l l l l l l l l o l o i i o i i i o i i • I I I I t l I I I I I H g I ' i ~® . . . . . . . " = = - ' o = = = = = - = o - = = = o - = = = = = . l l = l l l l l l l l l l l l l l l : = l l = ~ : ~ . : ~ : l l l l l l l l l l l l l l l l l l O I I I I I I l l l l l l l l l l l l l l l l l l l l l l l l l O l l l l l ~ I "

N I O I K I I O O O I N O O O I I I I O O O O O I I I O I I N I O I N I ~ O N i O I R i i O i I i I R ~ I ~ O ~ I ~ : l l l l l i m l O I l l l l l l l l l l l l m l l O l l l t l l l l l l l l l l l n l l l l l n l l l l l l ~ = l l ~ l t + ! " . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . q ! , . ; : .

~ ~ t II H ~ II II II = 0 ~ = $ 0 I ( ~ = ~ II ~ I tl H + i ii [i .o ii H ii . ii ÷ t ~ ~ + ~ ii t ii it i I i i , I " : 1 : i I#l # I "

O ~ H H H H I I ~ H ~ ÷ I I t t ÷ H t I I H I I ÷ t "`11 | I I I l l ' , ÷ + 1 1 ÷ 1 1 I l l ÷ t tl • ~ ? I I ~ : : . . . . . . . . , . . . . . . . . . . . . . . . . . . . . . . . . . . . . ' . . . . . . . . . . . . I l l , l l l I N H I ÷ H ~ ~ t1111 t t l t , . l t l l , l t l l l I I I I I I ~ 1 ~ ' ' ÷ * " . I I I I * 1 1 t

+ ÷ . . . * . . . 1 1 1 6 ~ l l l ~ l l l ~ , . l l t l l l i i i i i i i • # # ~ l l II ~ I ~ H M ÷ i + ~ ii el # ~ ~ ;1"~ ÷ ÷ I " ' l l II II " I ~ . . . . II II I ' '11 "'11 ÷ I I II "* I l l l l I e

÷ + t ÷ # ÷ + t l l ÷ o . I II I I I I I t 1 1 + 1 1 . . 0 ~ . o l i . 1 1 t

~ : ~ = = : ~ : ~ = ~ = = : ~ : ; ~ : = : = : ~ : ~ g ~ ? l : . . . . . . . . ,,~ . . . . . . . ' , . ' . . , i . o . i i

• . . o. " o, ~ l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l I = ~ = ~ O ~ l l ÷ i + ~ ÷ . . ÷ l l ÷ l l ÷ U l l , I ~ I I I I I I l I I I I I l I I I I I I i i i i i i i t l l i i i i i O l i i l i = ~ = = ~ O i i H ~ + I ÷ ~ t ÷ ~ ÷ + , . I . ~ l i l l l l l l l l l l l l l l l l l l l l l l l l l U l l l l l l t l l l l l ~ l l ~ O l l f f ~ l l ~ ÷ H + ~ + t . , l l I

= = l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l : : ~ : ~ ; ~ : ~ , , , ÷ ~

~ l l l l l l l l l l l l l l l l l l l t l l O i l l l l O I i ~11 = = ~ . . . . . . " ' " " . . . . . . " " - - - - - - - - - . . . l l l l l l l l l ' " " . . . . • . . . . ® " " " ~ * { ~ l ~ l l l l l l l l l l l I I I l l l l I I l I I I I = ~ H ÷

I I I I I I I l l l l l l l l l l l l l l l l l l l l l i l t l l l l l l ~ f f l

~ ~ ~ , ~ k ~ S H B B B I ~ I D I D D B H K B D B D D B D R O B O i H H i O O i K I B G ~ k ~ Z k l ~ m ~ k l ~ S ~ k k ~ % ~ k g K K m l l O U B l m m l m m l i O m a O l m U i l l N i f l I I ~ m ~ l ~ m ~ s ~ ~ k ~ k k S ~ S ~ G U K i a K i B B S B k k K l l n l l i ~ l l B

?

.= O

g

5

O

~r2

382 S . L . HOROWITZ AND T. PAVLIDIS

=1; MO

* * *0

* * * i °

* * * . . o

* * * * * * * * * + . e * * * * * * * * * *

. . . * * * * * * * H t l

. . . . * * ~ , ~

***************** *********************** *********************** ***********************

********************************* ***************************

************************* ***************IQ

*******io m

~mm ~mO

*ma *mO *mO

*gg

*gO

*mO .gg - -o

* ga l l * t g * Og

*g i l l qllll qllU

* O0

Picture Segmentation by a Tree Traversal Algorithm 383

• " ~ " ' ~ ! t t O ° Oa t e • O~ • O D I J O O t • m " " " " • • • O t e I , t # , e a O • e I

O . . IO IO oo l , • • • • I • , t • # t l i t I , . . , I o I • I I ! I I I I a I I a g t • I I I i I i I l I ] I I I I # I I I I i I l I a t t , , m , o t . . ! ~

• I I i t t • : • • ~ ~ I "4 'V' • 1 I ~ iii ~1 | . ¢ "~ I I IF • , , I . , I ,,I. e I I , . + . . . . I I , , l I I I IM +

• • . + aI4IO + I-I II l/~'ll ~ "I" III ~ N ~ 41 II ÷ H + + N~/}I-I II I-,I I N~ ~I~ iI ~.iN H I/~.~

:': . . . . . , . ' • , . , I : , ' . ' , ' . , ' . I . . . . . . . . . . . . . . . . : + + " ' . . . . . . . ' * * " . . . . . . , * + , , , , - , , * ~

. . . . I I I I I I I I + I-.-~ ~ I-.~ I I I I I ~ ~ M ÷ i I . k i i . v i i . i . H i i • I ~ I i i I ~ I ~ I I I I I I I 1311 I llIIIll NII~ii . . . . . . . . . . , ' . : ' . . : ~ + . ~ . , = = ~ . ~ ® . . . . , ;,'t, ', . . . . . . , , ~ , , . . . . . . + * + . . . . . . . . .÷,~.+"~"~

• • , I , ~ I I , I I , ' I , . I I , ~ 1 " I " , ' I I I I I I I I I ÷ ' " I I " " ' * I I ' I I " " ' l " ~ I~ l i l I I . I I - I I , . I I I - I ~ - , I ~

. . . . . . . . . ' . . . . ' . . . . . . ' ~ ; ' " " ' ; ' " : ,I ' . . . . . . . . . . . . . . . . . . . . . . o . . . . . . , . . . . . • I . ~ I ~ I I , . I , . I , . I • , , i I I l l i l - . ~++ l l I I I I l 3 1 ~ i I I , . l . - I # . I f ; i l ~ i • + I I . I - . ~ " I I " I I I " I " I " I ~ ' l l ' l I I I I I I ' l l : ~÷+ l l l l I - ~+ l l +~ l l I I I I : ; I I ' ~ IU÷~ e l I I "m@l l÷ l , =~ l • , . I , - ÷M~

I .M@IMI I I . . I I I i 11 111 e I • • • ~ ~ ~ I I ÷ ~ i... i ~.... i I.-.. ÷ i l... i ~ @ + . . I + @ • I ~ I - . I ~ IIII~Illll , I I ÷ l l ÷ l . . . . I l tM I I , . I ~ e I I . , ~ + I + + + " + • • • I + l I I I , . . . # . . + i I ~ i i . i t@Mm l . - l l l ~ i ~ . . . . i i i I , I I O ~ I i E ~ z ~ I I i ~ l l i i i i . . i i i ÷ . i I . . . i i ~ i i i i i i i I i I , I 11114 -11 - I - . . . . i ÷ * l . l l . l . ~ l l l . . . l

• I • • I I I I ,, I ,. II L ~ I ~ I I I I I I I I I I I I I I I I I I I I I ~ D l l I I i I I I I I i . i i I i I i i

0 , I I ~ . I , l I

I I I I • I I + I-4 ,.I ~ I I l l I l l I I I l l l I I I I I I I I I I I I I I ~ l ~ I I : I ~ l . ~ l l l i ~ i ~ II ~ I I ~ I ~ I I ~ O ~

I • e l I . . I 1 . ~ I I , , I I I I i I I i i i * . i , . i . , . . . . I • • I : J i l l . ~ I ~ M , , + I O H~ . .~OHOi MO , I l l 1 . . I . . . . . I , . I I I I I I I , , I . . . . U I I i I e l l l N l , . , I I +OHI I 11 . . I 1~ . , I I 11 . +

I I I I I I . . . . I I I I . , I , . I . . . . . . . . . . . . • I • l l O l l ~ I ~ • I I I , , I I I , I . . I I I ~++ .~ I I . . , I I , I I . I I I , I I I . . I , , I . . . . . , , • I I . , , , I I . ~ . IF I ~ ] I I I I I I I I I I + I . I . I i i 411~d~ i ~O • - I I I I . . . . . , . . . . . . . . . I • el , , , + ¢ * , , , , i i ~ . I . ~4 . . . . I I ~ J I l M . , i i i ÷ i i 11 i i i i m ~ 3 ~ ~

' . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . " ' " ' ' I t , , , , l i l ~ " ' l . . . . . . i i l . . . . l . . . . . . . . i i I l . . . . I

• , I I I "I" I . . , . . ~ - . I . . , I I I , I , , i i i i I

......................................... ! !iiiiiliiii!iiiiiliiiiiiiilii! I I ii • i i i l I i i I ....... I I l., . I it i i., I,, I i ' ~ I . . . . . , . . , l l I . . l i , I I I I I . . . . . . I + , , . i . I

• . . . . I I l l l . l . . l l l l i . . i i . . . . I i i • . • • • • , , • . . . . . . . . , • • * , I I I I I . . I I . * I I I I U

• , - , I I I I I I I , , . I . . I - ~ , I , I I . . . . 11~ . - I I I , , I I I I " l ~ I I I l I l l I I l I l l l l I l l l l l l l l l I I i l • I I I , , I I I I I # . . . . I II I i , I . . . . . I I I I I I I I I I ÷ , . I I I , , • I . # l l ~ I I I l ~ I I l l I I l l I I I I I I I I I I I • I " I I I I I I , . I I I I

• ,l .. ,t .... ,, ,I "I I ~ , I I .... II IIII II ,-II,,I ,,I I "'I I I II÷ l l l l ~ I ~ I I l l I l l I I I l l l l l I I l I l l I • " • "'+ I • I II II .... I II I I • • I I I I I l l I ~ l l l l I l l l l l l l l I I l l I I I I

" : ' " ' . , . . . . . , , ,' ":I "' . I . " . . ' , ' 7 ' . . ~ ' " ' " ", ~ , ' " ' " ~ " ~, , - , .m , 0 , . , .u ~ 0 ; : ~ , 0 , , 0 I , , 0 , . . . . 0 , I I o I ~ I I O O I I O m i i a i i i i I i i i i i i I i V ; , " /A . . . . . . . . . . . . . . . . . . . . . . . . . . -----------.-==..o

I - , - I I I I • " I II II I II I -, I II I I I I l l l I I I I ~ I I I I l l I I I I I I I I I I ..............................................

t I I I I I i I I • • • . , I I I . . . . . . I I . . . . . . ,,..,,.,., .I,' "'''::~ ":':'" • I I " " I I . . . . . . . ' " l l , . l l , . l l , . , l . I I # • I • • I • : I , i " I I I , , I I I ~ " I I I . . . . i I I I I I I I I I I , . I I | I • I I I I I • I • I ,m I • • • I I . . . . I I I I I I I I I I ÷ , , I I I

P

N

o

]

I

=

v

,.9 oo

384 S. I.,. HOROWIT~ AND T. PAVLIDIS

II

****e~m

Picture Segmentation by a Tree Traversal Algorithm 3 8 5

+ tl I I + 41. + + ~ .1~. + i l + .11. + + .11- I I + + + ÷ ÷ + I - . 4 ~ l i l l l l l l t ' ~ X ~ O ~ ~ - I ~ l i "11. + 4. Or . . . . f I . . ,11 i o ~ . , i i , I ~ : ~ i , + I I • + + + i i + i + + @ + l - ~ i ~ ~ H i l + + + i~-4+ @ + i i H ~ J i = i i I t i = ~ I E E ~ Z O It II II i , , I I • o l ! " i i |1

t t , 0 l • • l I , ! • I I I ~ l l ~ I I ~ I I ~ l I I + M + + + + M ~ I I ~ I ~ I I ~ I I I I + I I , . I | , . I + . . . . I I , I I I I I I I I I I I I ~ + ~ l l l : I l l l l l I I I I l I t g l I I I l I S , 1 1 - 1 i l + .k.l~ t i l + + + + + l E I I I I t C ~ I l I l I l O ~ t : 4 - . . . . . . I It . . 1 . . . . I I I . . . . I | + I I I I . • . l l l . l ~ t ,

. . . . . . . . . . . . . O . . . . . . . . . . . . . . . . . . . . . . . I~ t , I . . . . . . ~ , l l t . . . . . . . . . . . . . . . . * + I

. . . . . " . . . . . . . ° . . . . . . . . . . . . . . ° . . . . . . . . . . . . . . . . . . i,.! i i i l '+' I c e + ® i I I I I I I I I I I l l l U l : M l l p O I - - ~ + t t + l ' i l l i t O I l I l l l l C l i l c : l l : ~ l " l + ,4- l i l i " 1 1 II II II • • I I I I I I I l l I I ~ I I O H + +I ~ - ~ l l I I I I l l l II I I +I I I -,II I t ", •

l l , - i t I,-t+il, I,-,ll,,-+ i i l l < l l l l l l l l i l l l l l t l , , - i l - - 4 4 1 I I ~ t ~ z Y ~ l l l l l l i m , l k + O l l l l l l l l < N I . - + iI , - + , , . l l I I 1 , I I I I o, 11 I I + II l o t I I ~ l [~ l l l i l l l l l l l l l l l l l l ~ l l ~ t i 3 t / ) I I • 1,1+ ~l, t . + 1 - - i i i i i ~ l l l l ~ l ~ t . ' l i i I . - i O I I l i ~ , , 3 1 4 1 + t + I I I l t l l l O , i , i l l i I I I I I I I . ,11 i l I • I t I . . . . i i * , 4 t i , , , , l l l l i l l l l l < 3 D i + t • 11-4 I1 # t • ~ I I O O ~ C I ~ I t l I I o i l l l l m m I ~ t 1" -4OI I I I I I I l :~ t . r ' l I I I I + II , , +0 I I I .+ I I 1, I 1, I I I .0 I I ~ O i l l l l l l l ~ i t l l l l l ~ l l I I N l - l l l l - i t I t I I I I I I I H I I l - i O i l l l l l l l E I l i I I b ' ~ # + I I I I I I I I I I I I I + II II , ' 1 , , I I . . . . I I + *11 l l ~ l l l l I I I l I l ~ 1 1 1 I O ~ + i t t i - 4 t - O ~ l # t i - ~ l l l - l C f l H H ~ - 4 I I I l I I l i t ~ i I l l , ~ l l t ~ l - 4 + ~ l l l l l l l l ~ + I I I I I II II . . . . I t I I I I I I II I l l + O I I I I I t l l l l l l l l l l l l l + I O l l ~ l l l l l + i t ~ - + N I , - , I N l t - , I N I , - i l , - + i , - + l - - t l , - l a , . l " l ~ l l l l l ~ l i t t l : : l l l # N O I I < I I I H II II I I II I I I I I I • I " . . . . t 4 t 1 1 1 1 1 1 + l l t i i l l ~ i l l - - ~ l l t I,-,+.il I l l - - 4+ - ,4 t t l ~ l+ ,~ l l l , - 41 - , l l l H l l ~ l O l / ~ l " , l O l ~ l t ) t + , l l l : ~ l l ~ l l l l : ; + + i l I I " , - , . . . . . . . I t I I I i l l - - I ~ t l l l t l l + , . , " 1 1 114t I I I I I - l l - + ' t i t i ' - I I I I I I+ '+ l " i l l I - ' l l l I I I I ~ ' ~ l " l l - 4 ~ l C I I I l l - - I + + 4 1 ' O I I I I 4 < H i , i t + i l I I I I I I . . . . . . . . / " 1 " 1 I t I I l l < l ' l t " " " i . " ~ I " 1 " l t . , , l i -~ l i I -+, l t , l l l , , -4 i , -+~ /~ l l i - , , l ' i t l ' ~ l , - - I I /~ l l t - , i l - - : l < i ) l l : ~ l t ~ l l + l l + # I I I I I l i " t l l I I i l i l U I I II I I ,I . . . . . II I " 1 , . , l " + l / ~ l O l i I " " l t

I-4 k - - l # 41 l . . i k.4 H l--li--li,.~l i..pl..4 m ~]i i~I) X,~ lip I I I I I ~ I m I..-,4,11+ if+ ~ + ~ I I I I ~ ' 3 ~ t l II + + II II II II II .ll. + II • • . * I , . . + I . , , , + I . , + I " • - I I ,i, I I I I I I t ' 1 I

~ I - ~ I - ~ I . 4 N . 4 1 - ¢ H N I I I I N I . - ~ I M I I I I ~ O ~ I ~ I I I i ~ I N + ~ I I ~ + + I I + + + I I II I o , I I I I I I I I I + , , + 1 . . I I . . . . I i I I l l • . II+~41-.41 l - - l l i l . . ~ l i l l l l i l , l l l l l S ~ 3 1 1 1 1 I l l i ' l l l i 3 I I I l i l l l l I E H + l i l l i i l l + l l I I + I I I I + + I I . . . . . . . . . . II I . . . . I I ~ I I ~ - I I • • , I ! •

l + ~ , ~ . ~ . l l . ~ l l ~ m + ® ~ 3 m m ~ i l l : l l l l l l i l l l ~ + . l . + i + l l + + l l l l l l l l l l l l l l , , l l I # . . . . . . l l , - i , . • i o , l l l l l l i + i i - + ~ + i . , 4 i k . 4 1 / ~ l i l - - i l - 4 1 - + l l C , ~ z i l l x ~ l - + ~ - l l - l i . . - ~ i i I I I i l l i # + + I I + I + l l , i . . i - + II II II II II . . I I . , I I I l , . t , . . . . . , 0 . , , + • I I i ~ • •

i . - , 4 ~ . - , 4 1 1 1 . . . , i + i . . . . l l l l . . ~ l i 4 1 iI l l r ~ l i . - 4 1 . . , , l ~ . i l - ~ l i ~ l l l ~ i . . i t i . i + + + 4 , 1 1 1 i i i II l l , l l Ii i i . , I I II , , I I I . . . . I + I , , I . . . . I I I + + • + , I II

' - ' " . . . . . . " " " * * * ~ " ~ * + + " " ~ + + + + + * + . . . . ,," :: ; ' : : :: ~," '~ '.'. :. : . . : '. ~ "" 7 . . : I . . . . . . . . . ,, . . . . ,, . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ++ ,, . . . . . . . ,, ,, ,, ,, I : . , ~ I # + , - t i H I I H + - - i l i l ' - i ~ l l l l l - ' l l - - I ' l ' + i l l ' - ' , i l ' - I t t + + ' i ' l l t t + + + + + l l + l l l l l l . . . . . . I I I i o . II I 1 . . , I I 1 1 . , 1 1 1 1 I I . | I I O , M

l i # l . - i ~ - , l l i p . i + i l l i . ~ l i ~ , 4 1 1 1 - , l i l l - 4 1 - , i + i i k l - 4 ~ - ~ l ~ - - i + X i + l l + l l i l i i i i i i . i , + 4 . + + i i iI . . . . . . i i . . . . I • I I I f i l l , . i i , i . I 1 1 1 i~c~31111 i H l - - , l i . l i l l H l l m i - 4 H l i m n l - . l i . ~ i 4 . + i - , l l i . 4 1 i l l , * 4 . 1 1 + + I I + 1 4 I I + @ + + , , I I • . . . . . . + . . . . . . I i i I I I + I I I V . l l l

+ - - I H t t t t i l i l l H l i H i - , 1 4 1 - , i l l l l i H 4 1 , + . l t i l i l l l l n l l l l + + + . . . . + l l + l l . . . . I I , , l l . . . . II * I + , I I I ' I • I I I , , i l l l l l i k-4 t H i - i ~ t--41-,l ,i- t- l t-~i ~+.1 iI+ iI+ i l, ~l, i,-+l+-,ik..I t l. + @ @ i i ~ ii i i I i i i ii + + + +1 ii ii Ill II + i i I • + , , . + + I l + i . , . l l t l

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ~ ' , : : i . . . . . l ' l ' ', I . . . . ~ " 4 II i~.~k.4.11 .II, k-,41--1 ~, l i . .4m~Vp 41+ i l i l , i , .~ i - -pik ,M+ + 41~ -ll i l 4 . + .I~ II .I. II II ~i. 4 , I U II +o II + , II * . I , , I II • ++ . i + • m ,,1. O . l u l l l l - 4 N # N t 4 + t + l - . l l - - l . l l , i l + i l t # I ,If ~ll. + ~k ,i. xit + II I I II I I I I + # I I I I I I l , I I II , l i I I I I t I I I I I I I I I I I I I I II l " M O l ~ I ]E~ t t f # ~ i n . l l . l + . l t N l - . l t f x l l f + @ , l f t t . l l ~ 4 1 , + l l + + t +11 i i i l l i , l t 4 - . . . . i I . . . . i i i i . + , l l i I i I i I I IE~ I I I l l o o Z l l 3 C l ( i ~ l k ~

~ . ~ t H I I I M I t + # H + # # + + + + t + + II + @ t + + + II I I + I I + I I I I . . . . I i * . l l . , l l l I I I I I I I l * i i I + l l I I I l l l l l l M ~ i l , , i , i 41 41+ i l . ~ l i ~ + .i . 4. 41+ i # + + + + 41 i i ~ + i i i i + i i + + + + i i . , i i i i i @ i i i i . . . . I I l . o l i I i e l + • i i i + ~I~ l l l l l l l i n k i i i i i

i..4 ~...,~t ~-.i i...,! + i i i + i..-4 i , i i . . , i . . l l .,I. ~ll I I . . , k 41. II 4 . 4k -i, i+ .i. 4k II . . . . II It II . + + . . . . II I . . . . II . . . . . • • i I • " i l I IIII IIII I , .~ III"~ '~i~ ~II . 4 + i . ! i l l ~ - 4 1 - - 1 i + ~ k . l l l l i + . 4 ~ - l i l , l . . ~ @ i i + l l l l l l . i + i + + o * t l l l l l + i i + . i . l l + l I . . . . . . I I . o 1 * * I • * + I * I I l ~ l l ~ ' l i l l l i l l l l l l ~ + i ~ l l + + i i.~i~.-.4.1 i l , + + + + i -4 + if. + @ + ~i. ~iI, i l + + II II + + * . II II II II II @ + .i- . . . . I I # + . I . o l i I # I I I • + I O i l q l l l l 3 1 " ~ I ~ I I .U. + + .II+ I I i l 41 I I ,ik . i + + + . l l i-.~l,il~ II II + + + + + l i . o II + . l l II + II . * II II , , n , . U i • 1 1 I I . . . . . . i i i i ; ] l l I I l I l l f ~ ' l i i m l l l l l +,-4++-iP.i , l ,+, l .+~l. i -- i+l l , , t . l l i + , ~ . + + t + l l I I i 1 + ~ l l + l l i l U I I I l , , + i l . . . . . . l , , l l i l i l . I , I I , , I I , i , , I C : l l l i l . q ~ , , + + l l i - - l t t , Z + l - - t l t , l . + i t 4 t , l i , ÷ + l l ÷ i t + H + t t + l l i l l i + + + , l l . . . . . . i l l l + l l l i , , , I . . . . i l I I "~ , , , , , i , , I I ; l ~ t l l : ~ . , l . I I I !

, + + , l + l l + + l t r - , l + - ' + + + + - - I I ' l ' l l ' l l ' + l . + + + + ' l ¢ + l l + l l . . . . + i l l i i l i l 0 , 1 i l l l l , , l l i , ' , . . . . . . I l l , i • • , t , , i i z I ~ l ) O , , , l l l l t + i l , - + + + t , t . l - I ÷ , M . + l l l i + < l + + + + + i l ~ i + + l l l l i l l l i i , l l ' l " . . . . I . , , l l i i . . . . I I ' ' , 1 i , , + 1 ~ , i , + ' - t I O i i , I + 1 + 1 1 1 , i , + + + t + + ~- I . I , + + + + + ' i l+ ÷ ~ + i l l i li i i I I I I . . . . I I I I l i , , l l i I I I I 1 + I i • . + , • + 1 I I i i i i ,011 - , • , t + 4 1 1 1 , , i , + + i t , l t + + + l - I + i l + + + + + + + + + l l ' ÷ l l l i l l l l + + , , l l . . . . . . I I I I . . . . I + 1 I . . . . I I I • . i e l • , I + . t • . i i + O I ! ~ i X < I + + t l - , , l + l i + + + l i + i l , t l , # l l + + i i + + + l i l l i i l l l l l l l l i l - , I ! . . . . . . . . I , , • + 1 . . . . I . I I I I I I * l I I I ~ i l ~ l l I l ~ l ~ , + + l l + 4 i . + l l + + l l + + . l , ' i i + + l i + ÷ + + i l l l , , I , , l l i i 0 , 1 t , i - - I I I I I . . . . * 1 • + , , I , , , ~ i I I i , I I t I i -~-~,~11 , # ' l t i l + i ' + - ~ + t t l ' + + + + + + ' l ' + , , ' k l i l i ÷ i l 0 , i I i . . . . . . I I , i , , , l l l l * 1 " i , , I , i i I ~ t + . i , i i , , t

ii I , , ,11 i I l i • , i • i I i , i , i I . i • • I I • I I I ' I I + + . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . : : : : L : : : ' . : ' . 7 " : I : : ' ~ ' I ' I . . . . i ii ,i, ,11, + ,i+ ii + ii .i, t t + + ,@ ii i i + i i t l, ,i, i i i I . . . . ~1 i+ ~ 1 i i i I # +1 I •

, , t . + + , l - + ~ , . l + ~ l l + ÷ ii + - k i i I I I I + + l i I 1 . 0 1 • . , 0 1 ~ .~ • ~+ , t + + l l ÷ i + l l i l l l + l l l l ' l - i l l l i l l l , - I I + l l . . . . . . I . . . . I I I . i i I I I I I i i i • • . + . i I i # • i • i i ~

O

.!

O

:m

,.C:I

i l I

v ?

o o

g

3 8 6 S. L. HOROWITZ AND T. PAVLIDIS

I . I m i d M I

**,° u, ! * ! : ,

l i e (I , 1 ~ * t l

"1t ,1t II

t1~ t ~-I II t t II II ,11. t II I 1 - - ~ t t I 1 ÷

v V ~ *

t t 2~11 t t Z II ' - . ~

* * ~111 * * 1 1 1 1 1 1

~ ,11- t u r n

* * I l m l

* * m i l l * * I I I I I * * I l l l

* * llmlll

* I l l l l

* i m m * IIHm . ~ i I D * ~ i I D * I I ~ mltlm m o

* m i d t~llm

* I I H D

*11911

: | II-o

Picture Segmentation by a Tree Traversal Algorithm 387

One potent ial problem in all "global" picture processing schemes is the necessity to keep the whole picture in fast memory during the computations. In this paper there was no problem because the picture resolution was low (64X64). However this usually is not sufficient for many applications, including radiography. The present scheme allows the introduction of "paging" in such cases, Thus a 512×512 picture can be divided into sixty-four 64X64 pictures which can be processed independently up to and including the elimination step. Then the regions so obtained can be used as input to a second pass of the grouping algorithm. Since the number of regions is well below tha t of pixels, the process can be quite economical in terms of memory requirements.

The results of the segmentation are well suited for syntactic pa t te rn recognition, es- pecially after polygonal or polynomial approximations of the region boundaries [28, 29].

REFERENCES

1. ROSENFELD, A. Picture Processing by Computer. Academic Press, New York, 1969. 2. DUDA, R.O., AND HART, P.E. Pattern Classification and Scene Analys~s. Wiley, New York,

1973. 3. BRICE, C.R., AND F~NNEMA, C.L. Scene analysis using regions Artif. Intel. J. 1, 3 (Fall 1970),

205--226. 4. BAJCSY, R. Computer identification of visual surfaces. Comput. Graphics and Image Processzn#

2 (1973), 118-130. 5. ROSZNFELD, A., ANn THURSTON, M. Edge and curve detection for visual scene analysis. IEEE

Trans. Computers C-SO ((May 1971), 562-569. 6. ROSSNFSLD, A., TRURSTON, M., AND LZz, Y.H. Edge and curve detection: Further experi-

ments. IEEE Trans. Computers C-21 (July 1972), 677-715. 7. MARTELI,I, A. Edge detection using heuristic methods. Comput Graphics and Image Processing

I (1972), 169--182. 8. HUECKEL, M.J. A local visual operator which recognizes edges and lines. J. ACM ~0, 4 (Oct.

1973), 634---647. 9. KSLLY, M.D. Edge detection in pictures by computer using planning. In Machine Intelhgence,

Vol. 6, B. Meltzer and D. Michie, Eds., American Elsevier, New York, 1971, pp. 397--409 10. YAKIMOVSKV, Y. Scene analysis using a semantic base for region growing. CS-73-380, Comput.

Sci. Dep., Stanford U., Stanford, Calif., June 1973. 11. PAVLIDIS, T. Segmentation of pictures and maps through functional approximation Comput.

Graphics and Image Processing 1 (1972), 360--372. 12. FENG, H.Y., AND PAVLIDIS, T. The generation of polygonal outlines of objects from grey

level pictures. 1EEE Trans. Circuits and Systems CAS-22 (May 1975), 427--439. 13. RICE, J.R. The Approximation of Functions, Vols. 1, 11. Addison-Wesley, Reading, Mass.,

1964 and 1969. 14. PAVLIDIS, T., AND MAIKA, A.P. Uniform piecewise polynomial approximation with variable

joints. J. Approx. Theory 12 (Sept. 1974), 61-69. 15. PAVLIDIS, T. Waveform segmentation through functional approximation. IEEE Trans. Com-

puters C-~ (July 1973), 689--697. 16. PAVLIDXS, T., AND HOROWXTZ, S.L. Segmentation of plane curves IEEE Trans. Computers

EC-28 (Aug. 1974), 860-870. 17. PAVL~DIS, T. Linguistic analysis of waveforms. In Software Engineering, J. Tou, Ed., Aca-

demic Press, New York, 1971, pp. 203-205. 18. MEIssL, W.S. Computer-Oriented Approaches to Pattern Recognition. Academic Press, New

York, 1972. 19. STRONG, J.P., AND ROSENFELD, A. A region coloring technique for scene analysis Comm.

ACM 6, 4 (April 1973), 237-246. 20. HARLOW, C.A., AND EISESRSlS, S.A. The analysis of radiographic images. IEEE Trans. Com-

puters C-2~ (July 1973), 678-689. 21. ROB~RTSON, T.V., SWAIN, P.H., AND FU, K.S. Multispectral image partitioning. TR-EE 73-26,

Purdue U., Lafayette, Ind., Aug. 1973. 22. KLINOER, A. Data structures and pattern recognition. Proc. First Int. Joint Conf. ovf Pattern

Recognition, Washington, D.C., Oct. 1973, pp. 497-498 (available from IEEE, New York). 23. PAwIvIs, T. The use of algorithms of piecewise approximations for picture processing. Mathe-

matical Software II Conf., Purdue U., Lafayette, Ind., May 1974. Also Tech. Rep. No. 152, Comput. Sci. Lab., Princeton U., Princeton, N.J., May 1974. Submitted to a Technical Journal.

388 S. L. HOROWITZ AND T. PAVLIDIS

24. STEXeU~Z, K. An Introduction to Diecretc Systems. Wiley, New York, 1974. 25. STRONG, J.P., ANn ROSENF~LD, A. A region coloring technique for scene analysis. Comm.

ACM 16, 4 (April 1973), 237-246. 26. HI~NnERSON, P., AND TANIMOTO, S. Considerations for efficient picture output via lineprinter.

Comput. Graphics and Image Processing $ (Dec 1974), 327-335. 27. NAeEL, R.N., ANn ROSENFELD, A. Steps towards handwritten signature verification. Proc.

First Int. Joint Conf. on Pattern Recognition, Washington, D.C., Oct. 1973, pp. 59--66 (avail- able from IEEE, New York).

28. CreES, Y.P., ASP Fu, K.S. Recognition of X-ray picture patterns. IEEE Trans. Systems, Man, and Cybernetics 8MC-.4 (March 1974), 145-156.

29. FENQ, H.Y.F., ANn PAVLIDIS, T. Decomposition of polygons into simpler components: Feature generation for syntactic pattern recognition, lEE Trans. Computers C-$~ (June 1975), 636--650.

RECEXVED AUGUST 1974; REVXSED APRIL 1975

Jourmd of the Amodation for Computing Machinery, Vol. 23, No. 2, April 1976.