IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. PAMI-4, NO. 6, NOVEMBER 1982

Stable Matching Between a Hand Structureand an Object Silhouette

JEAN-DANIEL BOISSONNAT

Abstract-A method is proposed which determines the possible waysto grasp an object, defined by its silhouette. This method is quitegeneral and versatile: the geometry of the object is arbitrary and alarge class of grippers is allowed; no a priori information is needed,except the parameters of the gripper, and so the method applies, inparticular, to all the automatic prehension problems when the objectand/or its orientation are unknown. Suitable segmentation and param-etrization of the silhouette yields an explicit solution and a very fastand simple algorithm.

Index Terms-Automatic prehension, machine vision, robotics, shapedescription, silhouette.

I. INTRODUCTIONNE of the main purposes of robotics is to increase theversatility of the industrial robots. In particular, in the

field of automatic prehension, a lot of work has been done todefine versatile robot hands [1]-[3], suitable 3-D sensors[41-[6], or general control laws allowing any robot to graspany object. This paper is mainly concerned with this lastpoint.When choosing a control law, the first question to be an-

swered is: is the object to be grasped in an arbitrary way orwith a required orientation? If a given orientation is required,a model of the object is needed, and we have to solve threeproblems which are far from trivial: the definition of a repre-sentation of the object; the definition of the possible graspsand eventually their sorting according to stability, accessibility,etc.; and the estimation of the object pose by matching thesensed data and the model.The definition of a 3-D model of an object is an open

problem, and although a lot of work has been done (see, forgeneral references, [7] and [8]), most of these methods arecostly and rather suited, in the case of complex objects, totheir visualization than to the fast automatic actions. This isthe reason why most of the methods which try to define inan automatic way the possible grasps of an object were severelylimited due to the complexity of the geometrical calculations.These limitations concern the shape of the object, the shapeof the gripper, and the characteristics of the suitable grasps.Lozano-Perez [9] considers only a few types of grasps definedby elementary volumes (parallelepipeds and cylinders). Wing-ham's approach [10] is more general, but the grasps arerestricted to parallel faces. Laugier [11] associates geometri-

Manuscript received October 13, 1981.The author is with the Institut National d'Informatique et d'Auto-

matique (INRIA), 78153 Le Chesnay C6dex, France.

cal reasoning and artificial intelligence techniques to findpartial solutions to the problem. All of them consider two-fingered grippers. The last step in automatic prehension usinga model is to estimate the 3-D object pose. The problem isquite complex, and available solutions concern only particularcases: simple 3-D objects [13], isolated objects in a stableposition [12], [14], [15], and partially occluded 2-D objects[16], [17].When the objects are complex, like most foundary part, in

an arbitrary pose, possibly occluded by others, the approachesbased on an object model are much too costly to satisfyindustrial requirements. In that case, a two-step solution canbe proposed: first pick up the object without any prerequisiteorientation, and then put it on a flat surface where it can berecognized and handled by one of the previously mentionedtechniques. Several methods have been proposed which aresuitable for different classes of objects. Nonvisual systemswith the aid of proximity sensors [18], [19] are very economi-cal, relatively simple logics, but are limited to a few geometricshapes like cylinders or cuboids. Other systems use a TVcamera. Kelley et al. [20] isolate an area with homogeneoustexture corresponding to an approximately flat region, andplace a vacuum gripper there. Hanafusa and Asada [21]compute the location of their elastic fingers system from visualinformation of two-dimensional object silhouettes. The SIRCHsystem [22] also uses the two-dimensional silhouette of theobject, and selects a suitable hand among three differentfunctions. Many of the recognition problems associated withindustry are two-dimensional in nature, even though three-dimensional objects are to be handled, but there are caseswhere a three-dimensional silhouette (i.e., a curve in Euclidianthree-space) is needed. Systems have been proposed to extractsuch three-dimensional edge information from images [23] orfrom range data [6] by stereo comparison.This paper describes a general method to find the stable

possible grasps of a gripper along an object silhouette. 2-D or3-D silhouettes are allowed, and a large class of grippers can beconsidered. The approach is based on a local analysis of apolygonal approximation of the silhouette. We show that thebehavior of the gripper along the silhouette reduces to thestudy of the behavior of the gripper on five elementary partscalled in the following "primitives." An appropriate parame-trization of the silhouette and of the gripper is chosen, moreefficient than the sequence of the Cartesian coordinates of itsvertices, which yields an explicit solution. A first result isestablished for an isolated finger and a planar silhouette in

0162-8828/82/1 100-0603$00.75 © 1982 IEEE

603

604 IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. PAMI-4, NO. 6, NOVEMBER 1982

//////

7777'

t .LZIftZd'ZLyLL~//> /

(a) (b) (c)

three concurrino fingers

Fig. 1. Some usual types of grippers.

Section III. This result is generalized to a class of grippers inSection IV.An algorithm, based on this approach, is proposed and ana-

lyzed in Section VI where some further results of practical use

are also included. Numerical results are available in SectionVII.

II. GENERALITIESThe silhouette is composed of one or several simple non-

crossing curves, corresponding to the visible parts of the objectcontour, oriented, as conventio'n, in such a way that the nor-

mal points towards the object (i.e., the external contour iscounterclockwise, the holes clockwise). These curves will beapproximated by polygonal lines (for an algorithm, see forexample, [24]), but the method is available for some otherrepresentations such as conics.The gripper is defined as an ordered set of fingers whose

relative positions and directions of tightening verify some

constraints. The fingers are supposed to be rectangles whoseprincipal axes are the optical axis and their direction oftightening. Some usual types of grippers are shown in Fig. 1.The figure is drawn in the plane normal to the optical axis.A position of the gripper along the silhouette is said to be

stable if, when tightening (i.e., when the fingers tend to movealong their directions of tightening), it does not leave the ob-ject. The position is said to be feasible if the intersection be-tween the fingers and the object is reduced to points belongingto the silhouette.A stable and feasible position of the gripper is called a grasp.

III. CLASSIFICATION OF THE STABLE POSITIONS OF A

FINGER ALONG THE SILHOUETTELet S be a planar polygonal line, i.e., a sequence (S1,.*. , Sn)

of points of the silhouette defined by their coordinates, andlet be a finger of length L.The following study is based on an appropriate segmentation

of S which is quite a natural idea for -D can only touch a partof S. 'To restrict our study to the cases of practical interest,we assume that the gripper fits the object well, and that thepolygonal approximation of the silhouette is correct. Moreprecisely, we make the following hypothesis: around eachvertex Si = Vi n Vi+I of the silhouette S, there is a free space

of radius L whose intersection with S and the sector (Vi Vi+1)exterior to the object in the neighborhood of Si can onlycontain points belonging to, at most, three connected seg-

ments of the polygonal line S not forming a triangle.Under that hypothesis, the different configurations that a

circular window of radius L, moving along the silhouette, can

(d) (e)

I

(f) (g)

Fig. 2. Local views of the silhouette.

V

VI VV

Fig. 3. Definition of the elementary primitives.

see, are shown in Fig. 2. They are grouped according to therelative angles between the segments.From the point of view of stable positioning, the behaviors

of those configurations may have some similarity: (a) isstraight and the finger 3 must lie flat on it; (b) and (c) are

convex and allow a set of orientations for D; (d) and (e) are

concave and require T to enter the concavity and remain there;(f) and (g) combine the convex and concave properties.More precisely, we shall see in Section II that the study of all

of these configurations reduce to the study of five "elementaryprimitives" which will be investigated first.

A. Elementary Primitives

We give in this section the stable positions of on some

elementary' shapes which must be considered, at this stage, as

abstract' isolated entities. In particular, they are not neces-

sarily parts of the silhouette.The so-called "elementary primitives" are particular cases of

the configu'rations (a), (b), (d), (f), and (g) (see Fig. 2), de-fined by the following propositions and drawn in Fig. 3.

-1) A straight primitive (noted S) is a'n oriented segment Vwhose length is at'least L.2) A convex primitive (noted (p), respectively, a concave

two fingers three parallel fingers

BOISSONNAT: MATCHING BETWEEN IIAND STRUCTURE AND OBJECT SILHIOUETTE 60

n

L x

Fig. 4. Definition of the parameters of an oriented segment.

primitive (noted C), is the union of two connected orientedsegments V, V' such that (V, V') > 0, respectively, 0 <(V', V) < r/2 -, V I|| and V' < L.13) A left Z primitive (noted Z1), respectively, a right

primitive (noted Zr), is the union of three connected orientedsegments V, V', Vi such that V' + V" | AL, || Vj| =L,respectively, V+ V' AL, V" |= L -, (V, V' + V") = 0,

(V, V') > 0, (V", V') > 0, respectively (V, V') > 0, (V',V )>0.Positioning the finger JD on to a primitive requires three

conditions.1) D must be correctly oriented; the allowed orientations

are the angular sectors (V, V) for 5, (V, V') for C, C, and Z1,

and (V', V") for Zr.

2) JD must be correctly positioned laterally. A lateral posi-tion is correct when T lies on S, touches the convex vertex I =

V n V' of (), goes into the concativity of C, or when the leftG (respectively, right R) endpoint of JO goes into the con-

cavity of Z1 (respectively, Zr). For each type of primitive, theallowed lateral positions are worked out by expressing that theorthogonal projection on to T (noted 7r) of an oriented seg-

ment or of a point satisfy some condition. The conditions are:

for S, D C 7r(V); for 0, 7r(I)C ; for C,5I C 7r(V+ V'); fori,GC1rr(V' + V");and for Z, R E r(V+ V').3) S must lie in the half space delimited by ? not containing

the object.Simple expressions of these conditions can be obtained by

means of a suitable parameterization of the primitives. Weconsider, at first, the case of a straight primitive.Let {R} denote a reference frame whose origin is 0. We call

the parameters of an oriented segment V the four-tuple (1, a,

d, h) where I is the Euclidian norm of V, a is its orientation,and d and h are the coordinates of its middle M expressed inthe frame {r} deduced from {R} by the rotation (0, a). SeeFig. 4.From the point of view of stable positioning of D, a convex

primitive is characterized by some oriented segment, bisectorof (V, V') whose middle is I = V n V', and the width of theallowed orientations around the bisector. Therefore, we

choose, as parameters for 0, the parameters of the orientedsegment V/211 V|| + V'1211 V' I and the angle = (V, V')/2.

lWe take, for the angles, the unique determination between -7r and+Jr.

A concave primitive is characterized by the oriented segmentV+ V', the orientation of the bisector of (V', V), and thewidth of the allowed orientations around the bisector. Wechoose as parameters for C the parameters of V+ V', theorientation ,B of the bisector, and 0 = (V', V)/2.For the Z, (respectively, Zr) primitive, we choose as param-

eters the parameters of V' + V" (respectively, V + V'), and= (V, V')/2 (respectively, 0 = (V', V")/2).In each case, the parameters are the parameters of an oriented

segment in addition, eventually, with one or two angles. Tomake the sequel more readable, we shall use the general nota-tion Y = (1, a, d, h, (, 0) where 1, a, d, h are the parameters ofthe oriented segment, ,B is the mean allowed orientation, andf is the width of the allowed orientations around (3. Clearly,= a for S and , ,B = a + 4 for Z1, and , = a -q forZ,. More-

over, if there is no friction (see, otherwise, Section V), 0=0for S.To achieve the parametrization of the whole problem, we

also need the parameters of the finger D which are defined asthe parameters of its edge, oriented in such a way that it forms,with the direction of tightening, a direct frame. We set D =(L,O, 6, ).We now use such a parameterization to give an analytical

expression of the three conditions previously mentioned ofthe stable positioning of T on the primitives.In addition to the previous notations, we set

(h(t)) (h ) t(t)) (S H(t)) (O

where

/ cos (t) sin (t)R(t) =

-sin (t) cos (t)

The parameters (1, a + t, d(t), h(t)) are the parameters of theoriented segment deduced from the oriented segment (1, a,d, h) whose middle isM by the rotation (M, t).Straightforward calculations yield the following proposition

giving the conditions for the stable positioning of a finger onan elementary primitive.Proposition 1: The stable positions of the finger 1D on the

primitive 9i are given by the conditions

1) Io- 3lI<2) J6 + eL/2 - d(O - a)| X cos (O - a) - p3) t(rq) + tH(t71 )/2 = h(r?2)

where e, ¢, X, ,, qj, and 7l2 are defined in Table I.

B. Stable Positions ofa Finger Along a SilhouetteIn general, the stable positions of D along the silhouette do

not directly reduce to stable positions of T on elementaryprimitives. One reason is that we have not yet considered con-cave primitives whose angle is less than ir/2. Moreover, somepositions have been omitted or are not feasible. Omitted posi-tions may happen with configuration (e) of Fig. 2 where thestable positions, for a grip resting on two nonsuccessive seg-ments, are not stable positions on the concave primitives.Nonfeasible positions may happen with configurations (f)

605

606 IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. PAMI-4, NO. 6, NOVEMBER 1982

TABLE I

Primitive c xA 7X1 T2

8 0 ° ° 1 -L ° 0 - c2

0 0 0 0 - L ° 0 - ce2

I LAC 0 sign (o- a) 2 2 a - 0 °

1I 1 1 2 0° e - 0°2

2r -1 -1 0 a -6 0r ~~~~~~~2

S.,8

2 2 S

53

List of primitives

r

S1 (Sl'S2)' "32 =- (S2'S3)' '3 = (S3,S4), S4 (S5,S6), S' (S6,S7),S (71' 8) '7 (S8'.Sl)'V (S3,SP,S3`), V2=(SO,S6"S),, N=(S8, S 8), V4 =( Si Si S ),C' = ( S 2 S 2s2) I (S7 'S SI7'

vr(S4,S4,S5,S").

and (g) where some of the stable positions on the convexprimitive intersect with the object. On the other hand, theset of stable positions of -D on configuration (c) is the unionof the two nonintersecting sets of stable positions of D on thetwo convex primitives.The aim of this section is the study of the configurations

which can not be considered as the union of their convex orconcave elementary primitives. They are called the "general-ized primitives," and are defined by the following propositions.

1) The first generalized concave primitive (noted 9,) is aconcave primitive (V, V') such that u = (V', V)>>r/2, | V|and V' < L/sin u. Note that the limit value of the Euclidiannorms of V and V' is the minimal value which allows all thestable positions.2) The second generalized concave primitive (noted 9,,) is

the concatenation of two successive concave primitives (V, V')and (V', V") whose common oriented segment V' verifies|| V <L. . Note that the hypothesis of Section III impliesthat (V", V)> 0.3) A generalized left 2 primitive (noted §1), respectively,

right Z primitive (noted er), is the concatenation of a convexand a concave, respectively, a concave and a convex, elemen-tary primitive (V, V') and (V', V"), not forming a 2 ele-mentary primitive, and such that V'| <AL and || V' + Vff l =L, respectively, || V + V' jj L. Note that the hypothesis onthe silhouette implies that II VI = II V' + V" II = L.These definitions and the hypothesis on the silhouette E yield

a unique segmentation of E into elementary and generalizedprimitives (an elementary primitive is, now, an elementaryprimitive not belonging to a generalized primitive). An exam-ple of such a segmentation is given in Fig. 5.The second result about generalized primitives is that we can

replace each of them by one or two elementary primitiveswithout changing the set of solutions for the positioning of S.Before settling this result, we introduce a realistic constraint.

In order to guarantee the positioning of the finger, we restrictthe set of positions of D on an elementary or a generalizedprimitive to the positions such that, when opening the gripper,D does not meet any point of this primitive.Thus, the first generalized primitive is equivalent, from the

point of view of stable positioning, to the elementary concaveprimitive whose parameters are those of the oriented segmentV+ V', the orientation ,B of the bisector of (V', !V), and (p =7r- (V', V). So we can replace the q, primitive by an elemen-

Fig. 5. Segmentation of a silhouette.

5i+l S;~~~~~~~~~~~~~~~~~

Si+1

(b)

Fig. 6. Equivalence of GI and elementary primitives.

tary concave primitive without changing the set of stablepositions.An analogous result is available for the second generalized

concave primitive g, = (V, V', V") which is equivalent tothe elementary or to the first generalized concave primitive(accordingly as (V", V) is smaller or greater than 7r/2) builton the oriented segments V and V" lengthened to the inter-section of their supports.The case of the generalized Z primitives is a bit more com-

plex. We give the equivalence relations for a 91 primitive; thecase of a gr primitive can be immediately deduced. Let g=(SSiS Si )-see Fig. 6. We distinguish two cases.

1) (SiS;i, SiS )> r [Fig. 6(a)]: Let Ti be the symmetricpoint of S7 with respect to Si. The 91 primitive is equivalentto the two elementary primitives 0 = (SjSS7') and =

(TiSiSi+ I Si ).2) (SiSi, SiS;) < rr [Fig. 6(b)]: Let Ti be the intersection

between the supports of S'Si and Si+IS. The gi primitive isequivalent to the two elementary primitives C built on thepoints (S;TiS;!) and 21 = (S;SiSi+, Se).The classification of the stable positions of -D along the

silhouette proceeds -from these rules of equivalence andProposition 1. A position of the finger T is a stable andfeasible position along the silhouette iff it is a stable andfeasible position on one elementary primitive.2This yields the main result of this section, expressed in the

following propositi-on.Proposition 2: The sets IJi of the stable positions of D onto

the different elementary primitives (natural or created by theabove rules) constitute a partition of the set J1 of the stable

2In fact, a few limit positions (one for each intersection of twoprimitives) may be taken into account twice. This is without practicalimportance, but can be avoided by excluding the last endpoint of eachprimitive.

(a)

s8

,Sj

T;

BOISSONNAT: MATCHING BETWEEN HAND STRUCTURE AND OBJECT SILHOUETTE

positions of -D onto the silhouette, i.e., UiIli -V 1I and ¶Ti ntl = 0 for every i, j(i # j).

IV. STABLE POSITIONING OF A GRIPPER ALONGTHE SILHOUETTE

The last section studied the stable positioning of an isolatedfinger; we are now concerned with the case of the differentfingers of a gripper defined by their geometrical characteristicsand the constraints between them.Most of the grippers can be defined by two kinds of linear

constraints, the first ones specifying the relationships betweenthe orientation of the fingers, and the second ones the rela-tionships between their directions of tightening.As examples, we give the constraints defining the three

grippers of Fig. 1.Two-Fingered Gripper: 02 - 01 = T, 81 + 62 = O, t1 + t2 < 0.Gripper with Three Parallel Fingers: 02 - 01 = 03 - 01 = ,

61 +8 2 = C1,, 1 +8 3 = C2 where C1 and C2 are constant.Eventually, tj + t2 < °, t2 + 63 < °-Gripper with Three Concuring Fingers: 02 - 01 = 03 -

02=01 -03 =2ir/3,81 + 62 +63 =0-By definition (see Section II), a grasp is a stable and feasible

position of the gripper. Equivalently, it is a set of positions,one for each finger, which are stable, feasible (in the sense ofSection III), and which satisfy the constraints of the gripper.This definition and Proposition 2 of the previous section

imply that a grasp associates p elementary primitives if thegripper has v fingers. The problem is now to find the condi-tions under which p primitives constitute a grasp and, if so,the possible positions for the gripper. These conditions areobtained by writing the conditions of Proposition 1 for eachfinger, and then reducing the number of variables owing to theconstraints of the gripper. The results for a two-fingeredgripper are given in the following proposition; the completeproof is given in the Appendix.By convention, we associate the finger D= (L, Oi, 8 , ti)

with the primitive Ti = (li, yig di, hi, j3i, Oi) for i = 1, 2.In addition to the notations of Proposition 1, we set

(hd) (hd) + ( )(h2) with a=a2 +fr- 0

() (1/2)+12a)(2D

61 + e2=

2/1=111 +II2

A_ = (d_ X)2 + (h _p)2 - (_- eL)2

A+ = (d +X)2 + (h + p)2 - (p +eL)2

tiI = sup (tan (-0j + i- ot, + (i - 1) 7r))i

i

Proposition 3: Two primitives constitute a grasp iff1) t' = sup (t' , t' ) < t" = inf (t"', t"')2) X> IdI whenh +p |ip+Lei3) h<0.The allowed positions for the gripper are defined byt' <tan (01 - a,)<t"sup ((- l)I1 di(0i - oi) - Xi(0i - ti) + pi)i

< 6 1 < inf ((- 1)'-' d(Oi °ti) + )Xi(Oi °ti Pi

+(X i)qI2 = hi(r?2i) (i = 1, 2)

and the constraints of the gripper.A simple case of this proposition occurs when one primitive

(say T, ) is a straight primitive. In that case, the direction oftightening must be normal to P1. Proposition 3 reduces to thefollowing corollary.Corollary: The straight primitive 9, and the primitive T2

constitute a grasp iff1) II.<02 with0= + 7r 2

2) Id, + d2() - e2L/21|< 11 - L/2 +X2 cos a! witha= a2 +- at13) h, +h2(a)<0.The allowed positions for the gripper are defined by 01 = 1

81 - dl(ao + 7r)jI(l1 -L)/2, =hl,2=h2(01'+r)if9-2is a straight or a convex primitive, t2(a1) + t2H(a1)/2 = h2 ifY2 is a concave or a Z primitive.Due to Proposition 2 which claims that we just have to con-

sider elementary primitives, Proposition 3 gives all the possiblegrasps of a two-fingered gripper along the silhouette. More-over, Proposition 3 gives a simple, complete, and explicitsolution of the problem, requiring only a few tests on theparameters of the primitives to know if two primitives consti-tute a grasp and, if so, the allowed positions for the gripper.This result is quite general, and can be extended to the class

of grippers defined by linear constraints between the orienta-tions and the directions of tightening of the fingers. See theAppendix for the cases of the two three-fingered grippers ofFig. 1.

V. IMPLEMENTATION

A. Complexity

An algorithm looking for grasps along a silhouette in theprevious manner must achieve successively

1) the segmentation of the silhouette into primitives2) the parameterization of these primitives3) the tests of Propositions 3 and 4 (or the analogous

propositions in another case of gripper) and the calculation ofthe position of the gripper.Although ali the operations involved are very simple, if there

are n primitives, we may have to compute 0(nP) tests for a

gripper with p fingers. In some applications, this may becostly, but substantial reduction of computational time can

be gained when sorting the primitives with respect to theirorientations. In the case of almost convex silhouettes, a more

precise result can be found.Proposition 5: The complexity of the algorithm can be

reduced to 0(n log n) expected time in the case of almostconvex silhouettes.

I, _-(h - p)(d- X)+(p- eL) /t2 (h-p)2 - (_- eL)2

,, _ -(h + p) (d + X) - (p + eL) J\/t2 (h + p)2 (11 + EL)2

607

IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. PAMI-4, NO. 6, NOVEMBER 1982

Proof: The proof is stated in the case of a two-fingeredgripper. The situation is quite similar in the other cases. Agrasp associates two primitives £P and T2 whose orientationsmust verify

-01 - 02 +f31 +r.-<-2 <01 +02 +f1 +iT

This condition follows from part 1) of Proposition 3.Finding the smallest (3i greater than -01 - 02 + f1 + 7r and

the greatest f3i smaller than k1 + 02 + 1 + rr can be computedin 0 (log n) time if the (3i have been previously sorted.

If the number of concave primitives is much smaller thanthe number of convex primitives, the sum of the interiorangles of the contour gives 4 i7r. Thus, the mean number oftests (i.e., the number of (2 satisfying the inequalities above)depends mainly on (31 + ir- (32. Therefore, if the (i are uni-formly distributed, the complexity of the algorithm is thecomplexity of the sorting algorithm O(n log n). O

B. FrictionNo mention has been made of some friction between the

gripper and the object. If there is any, only the case of astraight primitive must be modified. The complete solutionneeds to define left and right primitives accordingly as theleft or the right extremity of the finger lays on the primitive.A rough solution losing a few positions is to consider E as aconcave primitive whose parameter ' is equal to the frictioncoefficient. For both cases, the general form of Proposition1 remains valid and so does the method and the followingresults.

C Three-Dimensional SilhouettesBy 3-D silhouette, we mean a curve in 3-D space. If, as

previously mentioned (Section II), the fingers are rectanglesparallel to the axis of the sensor which acquires the silhouette(supposed to be vertical), the following proposition gives thesupplementary condition to have a grasp.Let di be the set of altitudes of the contact points between

a finger Si and the silhouette. To assure the stable positioningof the gripper along the silhouette we need

1) to assure the stable positioning, in the horizontal plane,of the projections of the gripper and of the silhouette; and2) the intersection of the di not to be empty.One notices that, in the case of a 3-D silhouette, we just

have one more test, in addition to the tests of Proposition 3.

D. Sorting of the GraspsThe algorithm gives all the grasps. Generally, not all the

grasps are needed, and we just want "good" grasps. So a ques-tion of practical importance is the sorting of the grasps, and,therefore, the choice of criterion.We have to notice, at first, that the different types T of

grasp are not equally as good. In the case of a two-fingeredgripper, the different types of configurations are shown inFig. 7.The value of T orders the types of grasps with* respect to

their a priori stability. A grasp with a large value of T is pre-ferred to grasps with smaller T.

T = 2

T = 4 T = 5 T = 6

Fig. 7. The types of grasp (the Z primitives are accounted for by theconvex ones).

TABLE IITHE GRASPS OF EXAMPLE I

T tan AO A6 Criterion

(a) 6 0.17 0.13 6.30(b) 4 0 2.12 6.12(c) 3 0 0.22 3.22(d) 2 0.43 0.12 2.55(e) 2 0.26 0.15 2.41

For each type T of grasp, the stability of the grasp can bemeasured by the tolerances AO and A8 around the medianvalues 0 and 6 of the grasp (these tolerances are given, withoutany more calculations, by Proposition 3) and by the lengthd of the set of the common altitudes between the primitives.Thus, a good criterion to order the grasps is of the form

pT+ua/O +rAb +v(3.If not all the grasps are required, this criterion can be used

to stop the algorithm when a good grasp is found. Moreover,we can fasten the algorithm by looking, at first, for a priorigood grasps (i.e., with a high value of T).

VI. EXPERIMENTAL STUDY

An implementation of this algorithm was written in Fortran.The program asks for the coordinates of the points of thesilhouette, for the parameters of the gripper, and for thefriction coefficient. The outputs are, for each grasp, thepositions of the fingers and the value of the criterion. Onlythe configurations whose T is greater than 1 are considered.One shape used for experiments was the silhouette of Fig. 5.

In this example, we assume that the friction coefficient iszero. Five grasps are found by the algorithm and sorted ac-cording to the criterion T + tan (AO) + A5. Numerical resultsare shown in Table II and Fig. 8.Another example has been performed using real data. The

silhouette is that of a visible part of a workpiece lying in abin (the workpiece with a star in Fig. 9), provided by meansof a 3-D range data sensor [6]. The friction coefficient is 0.1.The actual criterion used to sort the grasps contains a term

about the altitudes and some heuristics that prevent the work-piece from being dropped or staying in the bin, which arebased on the relative position of the grasp and the estimatedmoments of inertia of the workpiece. However, to make theresults more readable, the same criterion as before is used here

608

f.

BOISSONNAT: MATCHING BETWEEN HAND STRUCTURE AND OBJECT SILHOUETTE

(b)

TABLE III

T tan AO As Criterion

(a) 6 0.13 0.04 6.17(b) 6 0.10 0.02 6.12(c) 6 0.06 0.02 6.08(d) 4 0.12 0.38 4.50(e) 4 0.13 0.25 4.38(f) 4 0.14 0.24 4.38

/\\I

(d)

(e)

Fig. 8. The grasps of Example 1. The dotted lines represent the freespace between the two fmgers of the gripper.

Fig. 10. The grasps of Example 2.

Fig. 9. View of the bin.

to sort the grasps.

Fig. 10.The best ones are shown in Table III and

VII. CONCLUSIONThe stable matching of a hand structure and an object

silhouette has been studied by means of a local analysis whichlooks under each finger of the gripper. The local analysis is

worked out using some natural segmentation of the polygonalapproximation of the silhouette into primitives and a suitableparametrization of these primitives. The result is the completeexplicit list of all the possible grasps. These grasps, in the caseof a gripper with p fingers, are defined by testing p-tuples ofprimitives. The number of tests for each p-tuple is very lowand, moreover, the complexity of the algorithm can, in a num-

ber of cases, be reduced to the complexity of a sorting algo-rithm, leading to little computation time.The method is general and versatile, and can be applied to a

large class of grippers and to arbitrary objects which need notto be modeled. The method gives a complete solution forplanar objects. In the case of three-dimensional objects, themethod is suitable every time a precise orientation of theobject is not required, and then we save a representation of theobject which is often a quite hard and time-consuming task.Moreover, the proposed method promises to apply to true

three-dimensional objects. The primitives might be triangles

(a)

(c)

609

,.f\Vo

L-..

610 IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. PAMI-4, NO. 6, NOVEMBER 1982

(instead of straight primitives in the two-dimensional case)and the different configurations formed by one triangle andits adjacent triangles, owing to their relative positions.

APPENDIX IPROOF OF PROPOSITION 3

A. Case ofa Two-Fingered GripperTwo elementary primitives Ti = (li, x1, di, hi, pii,qi) (i = 1, 2)

constitute a grasp iff there exist 8i, Oi, ti(i= 1, 2) satisfyingthe conditions of Proposition 1 and the constraints of thegripper:

1) IOi -oil < i2) Isi + ecL12 - d(Oi - ai) Xi cos (0i - ai) - pi3) t(7li) + tiH(q1li)/2 = h(n2i)4) 02 - 01 =-T

5) 61+62 =0

6) 1+ 2 <0.

Those inequalities are solved in three steps.Step 1: Let t=tan (01 - a,); using 4), 1)

lent to

t, t <t' (Al)where

t -sup(tan( + o°t,+ (i-1)and

t' =inf (tan(Qi + i3- a1 + (i - 1) )).

Step 2:3 Using 5) and noticing that cos (ai - 0i) > 0 (forfi < ir/2 and Xi cos (zi- Oi) - pi must be positive), we cansee that 2) has solutions iff there exists t satisfying

Id +ht-eL l+t2 <X+ pt- l+t2 (A2)where d, h, c, X, p, and,u are defined by

with oa= °O2 + Ir- Ot1(h/ (h1/ (a)

C1 +C2

2

P = 1 + P2 -

Equation (A2) is an inequality of degree 2 whose general solu-tion is deduced from Appendix II.In the case of a two-fingered gripper, condition 3) implies

that we can assume that h ± p <0 (see below). If we ex-

clude the case of two convex or 2 primitives which does notyield a priori good grasps (see Section V-D), the solution isquite simple. Inequality (A2) has a solution iff

becomes equiva-

where

A_(X- d)2 + (p - h)2 - (pj- eL)2

A+=(X+d)2 +(p +h)2 - (1+eL)2

t -(X- d) (p - h) + (g- eL) V[7(p - h)2 - (i - eL)2

" _ -(X+ d) (p + h) - (Ij + eL)(p +h)2 - +L2

and

if h+,u.>|P+eL|, X>Idl.Under those conditions, the solutions are given by

t2 2t.t2.

(AS)

(A6)Step 3: Due to the fact that the primitives cannot cross one

another and that Xi <L, 1 + t2 has the same sign as h. So,

t1 + t2 <0 is equivalent to h < 0. (A7)

Moreover, we can choose, as first primitive P1, the primitivewhose associated "support" (oo, 13, d, , h 1 ) does not cross theassociated oriented segment (12, 12, d2, h2) of the otherprimitive, and so h - p and h + p, which have the same sign ash, are negative.Conclusion: Conditions (Al) and (A3)-(A7) are conditions

for T1 and Y2 to make a grasp. The allowed positions for thegripper are given by (Al), (A6), 2), 3), and the constraints ofthe gripper. El

B. Case ofa Gripper with Three Parallel FingersIn that case, the conditions are given by conditions I)-3)

above for i = 1, 2, 3 and the constraints of the gripper:

4) 02-01 =03-01 =rr;5) 61 +62 =C, 51 +63 =C'6) t 1 +t2 < 0, t1'+t3 < 0.

Step 1: Using the same notation t = tan (01 - a1), the condi-tion is identical to the condition (Al) of part A.Step 2: We can consider the gripper as the association of

two grippers formed, respectively, by the two fingers 1-2and 1-3. Therefore, condition 2) has solutions iff there existst satisfying the two inequalities

{jd+ht- (C+ eL/2)AI +t2] +rt-L 1+t2 (A8)fd'+h't- (C'+e'L/2) l +t2I< l'+r't- L 1+t2

(A9)where, in addition to the previous notations,

(h)=')h + (O) (h3) with ot = ° +1+r-0a3

A_ 1>0 A+>0 (A3) /l' /1112 \ 13/2\I<~~~ ~ ~ ~ ~~~~iI J+

t ~~~, tIt ~~~(A4) \r \0 .1 \0

3We exclude here the trivial case where one of the primitives is a

straight primitive.

I el+_ 3=

2

BOISSONNAT: MATCHING BETWEEN HAND STRUCTURE AND OBJECT SILHOUETTE

Inequalities (A8) and (A9) are in the form of inequality(A2) of part A. The general solution is deduced fromAppendix II.Step 3: By arguments similar to those of part A, we can see

that the conditions 2j + 2 < 0 and t2 + 3 < 0 are equivalentto h < 0 and h' < 0.

C. Case ofa Gripper with Three Concurring Fingers

The conditions are given by the previous conditions I)-3)and the constraints of the gripper:

4) 02 - 01 =03 - 02 =01 - 03 =2ir/35) 61 +62 +63 =0-

Step 1: Let t = tan (01 - a ); the condition is very similar toinequality (Al), i.e., t4 < t < t' where

t =sup (tan (-Oi+Ii- t + 2 ))

t' = inf (an (i+p >

Step 2: We can choose arbitrarily two directions of tighten-ing (say, for instance, 62 and 63). Thus, condition 2) has asolution iff there exists t satisfying

Ld+ht- C 2Sl+rt- L (AIO)

where d, h, c, 1. and r are defined by

(r h() h2 h3

27r 47ra -=02 +, a =+a a3 a1 +3 3

(1) 11i/2) +()(12)2 +R()(1312)

e=61 +62 +E3-

Inequality (AlO) is in form of inequality (A2), and so, theresults of Appendix II apply.Conclusion. In both cases of three-fingered grippers, we have

explicit solutions to the problem, as for the two-fingeredgripper. In each case, the solution is obtained by solvinginequalities of degree two whose general solution is given inAppendix II.

APPENDIX II

SOLUTION OF THE INEQUALITY D +Ht > U 1 +t

Let A = U2(D2 + H,2 -U2) and t+ = (-DH ±,A)(U2) ifH2 $ U2 and t± = (U2 - D2)/2DH ifH2 = U2.

ifA U0, the inequality is true for every t if U < 0, and fornone if U> 0.

If IH I <.-UandD > 0, the inequality is true for every t.If IH I. U and D -< 0, the inequality is never true.The other cases are given in Table IV.

TABLE IV

U H t

>0 >U [t+, +o[-U A< U [t+,t-j61-UI]-0, t- I

<0 >-U[t, +-[U <<-U ]-o, t+] U [t-, + [<U ] _O, t+4

ACKNOWLEDGMENT

Helpful discussions with L. Baratchart and F. Germain are

gratefully acknowledged.

REFERENCES

[1] A. Rovetta, I. Franchetti, and P. Vicentini, "On a general pre-hension multipurpose system," presented at the 9th Int. Symp.Ind. Robots, Washington, DC, 1979.

[2] F. Skinner, "Multiple prehension hands for assembly robots,"presented at the 5th Int. Symp. Ind. Robots, Chicago, IL, 1975.

[3] F. R. E. Crossley and F. G. Umholtz, "Design of a three-fingeredhand mechanism and machinery theory," vol| 12, no. 1, 1977.

[4] C. J. Agin, "Sequential strategies for finding and identifyingobjects with a plane of light," SRI Int. 9th Rep. on MachineIntell. Res. Applied to Indust. Automation, Aug. 1979.

[51 P. M. Will and K. S. Pennington, "Grid coding: A novel techniquefor image processing," Proc. IEEE, vol. 60, June 1972.

[6] J. D. Boissonnat and F. Germain, "A new approach to the prob-lem of acquiring randomly oriented workpieces out of a bin,"presented at the 7th Int. Joint Conf Artificial Inteli., Van-couver, B.C., Canada, Aug. 198 1.

[71 D. M. Ballard and C. M. Brown, "Computer vision," 1982, to bepublished.

[81 R. Bajcsy, Workshop on thc Representation of Threc-DimensionalObjects, Univ. Pennsylvania, Philadelphia, May 1979.

[9] T. Lozano-Perez, "The design of a mechanical assembly system,"M.I.T. Al Lab., Rep. Al-TR 397, Dec. 1976.

[10] M. Wingham, "Planning how to grasp objects in a clutteredenvironment," M.Phil. thesis, Univ. Edinburgh, Edinburgh, Scot-land, 1977.

[11] C. Laugier, "Determination automatique de prises d'objets 'apartir d'indices morphologiques locaux," presented at theCongrese AFCET Reconnaissance des Formes et IntelligenceArtificielle, Nancy, France, 1981.

[12] J. F. Challine, M. Clerget, and F. Germaln, "Deux aspects de larobotique: Perception et ordonnancement," RAIRO Auto-matique, vol. 11, no. 3, 1977.

[13] N. Y. Chen, J. R. Birk, and R. B. Keley, "Estimating workpiecepose using the feature points method," IEEE Trans. Automat.Contr., vol. AC-25, Dec. 1980.

[14] M. Baird, "Image segmentation technique for locating automotiveparts on belt conveyor's," in Proc. 5th It. Joint Conf ArtificialIntell., Cambridge, MA, Aug. 1977, pp. 694-695.

[15] C. Rosen, R. Duda, J. Kremers, D. Nitzan and W. Park, "Explora-tory research in advanced automation-Third report," StanfordRes. Inst., Menlo Park, CA, Dec. 1974.

[16] W. Perkins, "A model-based vision system for industrial parts,"IEEE Trans. Comput., vol. C-27, pp. 126-143, Feb. 1978.

[17] B. Bhanu and 0. D. Faugeras, "Recognition of occluded two-dimensional objects," Helsinki, 1981.

[18] B. Fspiau. and J. Y. Catros, "Use of optical reflectance sensorsin robotics applications," IEEE Trans. Syst., Man, Cybern., vol.SMC-10, Dec. 1980.

[19] A. T. Witwicki, "A method of non-positioned workpieces taking,"in Proc. 9th Symp. Id. Robots, Washington, DC, 1979, pp.489-505.

[20] J. R. Birk, R. B. Keliey, and H. A. S. Martins, "An orientingrobot for feeding workpieces stored in bins," IEEE Trans. Syst.,Man, Comput.,vol.SMC-11,Feb. 1981.

611

IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. PAMI-4, NO. 6, NOVEMBER 1982

[211 H. Hanafusa and M. Asada, "Stable prehension by a robot handwith elastic fingers," presented at the 7th Symp. Ind. Robots,Tokyo, Japan, 1977.

[221 W. B. Heginbotham, C. J. Page, and A. Pugh, "Robot research atthe University of Nottingham," in Proc. 4th Symp. Ind. Robots,1974, pp. 53-64.

[23] D. J. Burr and R. T. Chien, "A system for stereo computervision with geometric models," in Proc. 5th rnt. Joint. Conf.ArtificialIntell., Cambridge, MA, Aug. 1977, p. 583.

[24] T. Pavlidis, Structural Pattern Recognition. New York: Springer-Verlag, 1977.

Jean-Daniel Boissonnat was born in Nice,France, on May 18, 1953. He graduated fromthe Ecole Superieure d'Electricite in 1976and received the Docteur-Ingenieur degree inautomatic control from the University ofRennes, France, in 1979.Since then, he has been working at the

Institut National d'Informatique et d'Auto-matique (INRIA), Le Chesnay, France. Hiscurrent research interests are in 3-D modeling,computational geometry, and related fields.

Convex Digital SolidsCHUL E. KIM AND AZRIEL ROSENFELD, FELLOW, IEEE

Abstract-A definition of convexity of digital solids is introduced.Then it is proved that a digital solid is convex if and only if it has thechordal triangle property. Other geometric properties which character-ize convex digital regions are shown to be only necessary, but notsufficient, conditions for a digital solid to be convex. An efficientalgorithm that determines whether or not a digital solid is convex ispresented.

Index Terms-Chordal triangle property, chord property, digital con-vexity, digital solid, efficient algorithm, half-cell expansion, semidigitalpoint.

I. INTRODUCTIOND ISCRETE geometry has applications in image processing,

pattern recognition, and other areas. Before an image ofan object is processed by a computer, it is digitized to yield afinite subset of digital points called a digital image. Then, var-ious types of operations, such as template matching and geo-metric property measurement, are performed on sets of digitalpoints.Convexity is an important and useful geometric property.

Since digital image processing has been restricted almost en-tirely to two-dimensional images, digital convexity has beenstudied only for digital regions, that is, 8-connected [10]finite subsets of digital points in the plane. Sklansky in [13]defined a digital region to be convex if and only if there is aconvex region whose image (under digitization) is the digitalregion. He then showed that a digital region is convex if and

Manuscript received September 16, 1980; revised April 16, 1982.This work was supported by the U.S. Air Force Office of ScientificResearch under Grant AFOSR-77-3271.C. E. Kim was with the Department of Computer Science, University

of Maryland, College Park, MD 20742. He is now with the Departmentof Computer Science, Washington State University, Pullman, WA99164.A. Rosenfeld is with the Computer Vision Laboratory, Computer

Science Center, University of Maryland, College Park, MD 20742.

only if its minimum-perimeter polygon is convex [13], [14].A digital region R is said to have the line property if everydigital point on the line segment between any two points ofRis a point of R. A digital version of continuous-domain con-vexity defined by Minsky and Papert [7] is that a digitalregion is convex if and only if it has the line property. How-ever, no useful results regarding convex digital regions havebeen derived from this definition. Let d1, d2 be two pointsof R and let R' be the union of all cells (grid squares) whosecenters are points of R. Then P(R; dl, d2) denotes the areabounded by the line segment djd2 and the boundary of R'.R is said to have the area property if for any two points-dl,d2 of R, every digital point of P(R; dl, d2) is a point of R.In [2], a digital region is defined to be convex if and only ifit has the area property. Moreover, it is shown in [2], [4]that the above three definitions are equivalent, the definitionof Sklansky having been slightly modified.In [11 ] Rosenfeld introduced the chord property to charac-

terize digital straight lines. The chord property and the areaproperty turned out to be equivalent [5] -. Furthermore, itwas shown in [3], [4] that a digital region is convex if andonly if any two points of the region may be connected by adigital straight line segment in the region.These results lead us to believe that convexity of digital

regions is now well defined and understood. Because of therecent growing interest in three-dimensional image processing[1], [6], [9], [16], it seems essential to develop a theory ofthree-dimensional discrete geometry.In this paper we introduce a definition of convexity of digi-

tal solids, which is an extension of that of Sklansky. Then, weshow that a digital solid is convex if, and only if, it has thechordal triangle property. However, this characterization ofconvexity does not lend itself to the development of an ef-ficient algorithm to recognize convex digital solids. Thus,convexity of digital solids is further characterized in terms

0162-8828/82/1 100-0612$00.75 © 1982 IEEE

612