Cooperative Multi-Robot Box-pushing in a Cluttered Environment

Ezra Federico Parra-Gonzalez and Jose Gabriel Ramırez-TorresCentro de investigacion y estudios avanzados del IPN

Laboratorio de tecnologıas de la informacionCarretera a Monterrey KM 6, Ciudad Victoria, Tamaulipas Mexico

{eparra,grtorres}@tamps.cinvestav.mx

Abstract

The multi-Robot box-pushing problem in cluttered envi-ronments has demonstrated to be a very complex problemwith multiple practical applications. In this document wepresent a new strategy to solve it, inspired in the wave-front algorithm which it also includes some pertinent mod-ifications to obtain trajectories that facilitate the box dis-placement for non-holonomic mobile robots. The proposedmethod obtains its benefits by the reduction of route dis-tances, reducing the direction changes in the routes, and bysearching the best pushing points for robots.

1. Introduction

Researchers generally agree that multi-robot systemshave several advantages over single-robot systems [1].The most common motivations to develope multi robotsystem solutions are that: 1) the task complexity is too highfor a single robot to accomplish; 2) the task is inherentlydistributed; 3) building several resource-bounded robotsis easier than having a single powerful robot; 4) multiplerobots can solve problems faster using parallelism; and 5)the introduction of multiple robots increases robustnessthrough redundancy.

The difficulty arises from the multiple forces applied onthe box (friction, directions and strength of contact forcesapplied by robots, weight, etc.). This task gets even moredifficult due to the dynamics of the entities in interaction(robots, obstacles, object to push, etc.).

Box-pushing is related to the well-known “pianomover’s problem” (Schwartz and Shari 1983) and can besummarized as follows: We have a number N of mobilerobots that must drive a box B towards a target area T. Thedisplacement of B requires at least the cooperation of a

critical number Nc of mobile robots.

The rest of this paper is organized as follows: Section 2presents a brief review of related published works. The fulldescription of the proposed method is presented in section 3along with some numerical results. Finally, some conclud-ing remarks and future works ideas are given in sections 4and 5.

2 Related work

Many researchers have dealt with this topic area; few ofthese projects have been demonstrated on physical robotsystems. This research area has a large number of practicalapplications that makes it particularly interesting to bestudied. In this section we briefly report some of the mostoutstanding strategies proposed up to the moment.

Communication in multi-robot teams has been exten-sively studied since the inception of distributed roboticsresearch. Yamada [8] reports that, independently of theconfiguration of the system (centralized or decentralized),the communication needs can be omitted and replaced bythe application of behavioral mechanisms based on localinformation. On the other hand, Munoz [5] reports howthe communication can considerably improve the behaviorof the multi-robot systems, increasing the coordination, thecooperation, the collisions handling and finally opening thepossibility of implementing the client-server schemes tothe involved robots.

Some studies address the problem by the well-known“Object closure” strategy. Wang [7] propose the manipu-lation by surrounding the object with series of robots, then,the position of the object can be controlled by the positionof each robot that enclose and push the object. Never-theless, this theory bases its operations on an importantnumber of holonomic robots.

Electronics, Robotics and Automotive Mechanics Conference 2008

978-0-7695-3320-9/08 $25.00 © 2008 IEEE

DOI 10.1109/CERMA.2008.49

514

One of the most studied strategies is the well-known“Pusher-Watcher” approach [3] [2]: A robot observes(watcher) the movement of the object and controls theoperations of the team (pushers) that manipulate the box.

Recent multi-robot strategies make use of the modelknown as “swarm intelligence”. Li [4] experiments usingcommunities of homogenous robots. These self-organizedsystems are based on decentralized and collective behav-iors.

Another recent strategy is the reinforcement learning.Wang [6] implements a variant including a mechanismof decision based on a Markov process known as “Q-Learning”. The main concern about this technique is re-lated to the storage capacities and high demand of processcapabilities.

3 Proposed method

The purpose of the present work is to obtain a new strat-egy to solve the box-pushing problem; the strategy has beendeveloped to obtain compatible results with non-holonomicrobots. The characteristics of the solution are listed below.

• Find the shortest trajectories.

• Look for trajectories with the smaller number of robotreconfigurations (from here we will define reconfigu-rations as the change of position of the robots aroundthe object in order to obtain a suitable position to pushthe object).

• To give preference to certain box rotations, in order toimprove the pushing points.

• With N mobile robots, look for the best robot coordi-nation in order to solve the problem in the minimumtime.

3.1 Space representation

For this strategy it is necessary to obtain the configura-tion space representation (C-space). This representation isobtained from a convex polygonal description of the 2Denvironment, where each obstacle is represented by a set oflinear constraint as follows:

aix + biy ≤ di

i = 1...Pk , where Pk is the number of faces of object Kand for each (ai, bi); ai �= 0 or bi �= 0

The box is also represented as a set of convex polygons.

3.1.1 Finding intersections between polygons (colli-sions)

We need a method to verify wether there is an intersectionbetween 2 given polygons or not. This can be achieved ifwe find at least a segment of a line which satisfies both setsof constraints.

The general concept to make this evaluation is asfollows:

Given a line, defined by a point x0 and a vector −→m0, wecan find the intersection between this line and a set of linearconstraints by finding the intersection of the line with eachsingle constraint and use them to determine the limits of thesegment.

3.1.2 Intersection between a line and a polygon

We will define the line in the following way:

−→x = −→x0 + λ−→m0, λεR

The restriction (from the set of restrictions that definethe polygon) that is going to be evaluated with the line isrepresented in this way:

−→ni ·−→x = di

By solving λ from these equations we can obtaindistance between the point x0 and the intersection.

−→n · (−→x0 + λ−→m0) = d

−→n · −→x0 + −→n ·λ−→m0 = d

−→n · λ−→m0 = d −−→n · −→x0

λ(−→n · −→m0) = d −−→n · −→x0

λ = d−−→n ·−→x0(−→n ·−→m0)

where −→n · −→m0 �= 0

The sign of −→ni · −→m0 indicates which side of the lineverifies the constraint, thus if λ is a max value or min valueof the segment.

To verify the intersection between two polygons we usethe previous method in this way.

Given 2 polygons:

• nAix ≤ dAi

• nBix ≤ dBi

515

Figure 1. Obtaining a segment of a line thatsatisfies the polygon’s restrictions.

Verify for each single constraint nix ≤ di if there is asegment on the line nix = di which verifies both sets ofconstraints at the same time.

• Rotate the restriction to be evaluated to obtain its di-rector vector.−→m0 = R · −→n0

• Calculate a point in the line; in this case we decide tofind the point that is through the perpendicular vectorthat joins the origin with the −→m0 vector.−→x0 = (−→d0 · −→n0x,

−→d0 · −→n0y)

3.2 Building the C-space

Assuming we have the mathematical representation ofall the involved objects and the capacity to identify if anintersection exist between two polygons, the following stepis the formal construction of the C-space. The C-space is arepresentation of all the configurations of the object (box)in the 2D space.

The configuration of the box is defined by its position(x,y) and orientation (theta). Thus the C-space representa-tion is a 3D matrix, where each cell determine if the config-uration is collision-free or not.

In the next paragraph, we will make a detailed descrip-tion of the steps we follow to construct the C-space.

The first step in the construction of C-space, is toestablish the resolution of the three-dimensional matrix [X][Y] [C]. Y is related to the amount of rows obtained fromthe division of the horizontal testing area by the defined

Figure 2. Graphical representation of the C-space when the box is rotated 75 degrees;the positions where the box is not in collisionare drawn.

horizontal resolution. In the same manner X correspondsto the columns, and the number is obtained by dividing thevertical testing area between the defined vertical resolution.Finally, C that is related to the amount of layers, is obtaineddividing the 360 degrees of rotation by the values of therotation increase. In the shown examples, the matrixrepresents an area that goes from -16 up to 16 in the axis ofthe X’s, -16 up to 16 in the axis of Y’s and 72 layers withan increase of 5 degrees between each one.

The final pace is to verify for each cell if a collision existsbetween the object and, at least, one of the obstacles of theenvironment. This step is made by testing each cell [x][y][c]of the matrix by moving the object to the (x,y) position androtate (c * resolution of rotation) degrees. Finally we applythe function to find collisions to verify if the configuration iscollision-free ([x][y][c] = 0) or if a collision exists ([x][y][c] = 1) with any obstacle.

3.3 Path generation

In this section we are presenting the way we can obtainan optimal trajectory for the box. In the next subsection thewave front algorithm will be described, and also the pro-posed heuristic to find a semi-optimal box trajectory will beshowed.

3.4 Wave front algorithm

The wave front algorithm allows us to calculate theshortest route between two points in a cluttered envi-ronment. The only previous requirement is to have an

516

Figure 3. Graphical representation of the tra-jectory obtained without any heuristic to se-lect the neighbors.

occupation matrix that represents the points in which theobject is or not in collision with other objects. In this case,we will use the C-space matrix that was constructed inthe previous steps. This algorithm works basically in thefollowing form.

Starting from an occupation matrix in which the freepoints are identified with infinite value and the occupiedpoints with negative values (-1), the first step is to mark adestiny point with a zero value.

The second step is to validate the neighboring squares,in this case: north, south, east and west for one layer. If wewish to construct a wave front that can move along a three-dimensional matrix, it is necessary to add the validation ofthe points of superior layers (up), and of course the pointsof inferior layers too (down). If the neighboring squares arefree (they are marked with an infinite value) marks themwith the value of the actual square plus 1. Needless to saythat the neighboring squares that are occupied (marked with-1) will not be marked. This operation is recursively appliedon the neighbors, until all cells susceptible of change (thosewhich are not isolated) have been marked.

3.5 Trajectory computation based on the wavefront algorithm

Once the wave front matrix has been constructed,computing the trajectory is as simple as to start from achosen point and follow the less value neighbor squares, upto the goal.

In figure 3 we show the obtained trajectory from a sim-ple implementation in which the order of the neighboringevaluation is defined in the next order: north, south, east,west, up and down.

3.6 Trajectory computation with reduction of re-configurations

It is clear that working with non specialized mobilerobots (in pushing task) it is necessary, in some cases,to modify the position of those with respect to the box(reconfiguration) to be able to obtain the suitable pushingposition. In this work we use an heuristic of the neighborselection which reduces the amount of robot reconfigura-tions throughout the trajectory.

The reduction of robots reconfigurations throughout thetrajectory allows in first instance to guarantee an importantreduction of time, simply because the change of pushingpoints of the robots can become a process that needssome time to be completed. Another aspect that can beimproved when the robots reconfigurations are reducedis the increase of the precision of the movements. Thecontinuous movements of the robots around the box canstep up the possibilities of having collisions among themand of course, the increase of the odometry errors by theconstant movement of the robots can affect the precision ofthe pushes.

The proposed heuristic for reconfigurations reduction isbased on the conventional wave front algorithm, with theinclusion of the following variations:

Once the wavefront matrix is computed, the processstarts from a valid chosen point and validates all the neigh-bors (north, south, east, west, up and down). If at least 2neighbors had an smaller value than the value of the startingpoint, the following step would be to measure the reach ofall the possible routes following the direction of the lessvalue neighbors. For instance, if we start from a definedpoint with a value of 99 and it has only two neighborswith smaller values, west with a value of 98 and up with avalue also of 98, the following step is to evaluate and countthe decreasing squares in the up and west directions. Thecounting will end when the decrement of the neighborsvalues finishes. Finally we have to compare the number ofsquares in both directions and follow the one that had thegreatest value.

The measurement process will be repeated many timesto the end of a route, or until a route leads to the destinypoint.

517

Figure 4. Graphical representation of the tra-jectory obtained with the proposed heuristicto reduce the reconfigurations.

Figure 4 shows the obtained trajectory after imple-menting the proposed heuristic. If we compare it with thetrajectory shown in figure 3, we can verify that the changesof the box direction are reduced, therefore, the number ofreconfigurations of robots is also reduced.

It is important to emphasize that while we are reducingthe robot reconfigurations we are diminishing the move-ments of the box too, this reduction of movements is equalto have greater continuity routes.

3.7 Trajectory computation including layer vali-dation and reconfiguration reduction.

Once we managed to obtain a greater continuity route,the following aspect to improve is the robots pushingpoints. In this document we propose a method to privilegecertain box rotations (0, 90, 180 and 270 degrees). Withthis method we try to improve the pushing points of therobots. This improvement will help to obtain a greaterprecision in the pushing task.

It is possible to improve the pushing points if themovements north, south, east and west are made in certainbox rotations, being thus, a method that privileges the useof certain box rotations will help to obtain routes withbetter pushing points.

In this work we use a heuristic to privilege certain layers(box rotation) in the building trajectory process. In thefollowing lines, we describe the heuristic that is used:

Figure 5. Graphical representation of the tra-jectory obtained with the proposed heuristicto reduce the reconfigurations and improvethe pushed points by the layers validation.

In the previous section we suggested that the neighborsreach measure was made at the beginning of the processand whenever it was arrived at the end of a route segment.In order to be able to privilege the use of certain layers ofthe C-space, it is recommended to measure again not onlywhen arriving at the end of route segment, but also when weidentify that we are evaluating the neighbor of a square thatbelongs to one of those layers that are trying to privilege.This action throws two results:

1. In spite of following a specified direction, we can mea-sure again searching new directions among some spec-ified layers, of course this action increases the proba-bilities of finding a new route but now using the desiredlayer.

2. The fact of placing new measurement points canclearly raise (sensibly) the quantity of robot reconfigu-rations.

In figure 5 we can see the heuristic to reduce reconfigu-rations and the layer validation applied.

3.8 Finding pushing points.

Obtaining the pushing points for each sub route is a veryimportant task since it will allow us to identify the pointsof the object that can be pushed in each sub trajectory anddiscard the drag points. The manner to calculate them isthe following one:

518

Figure 6. Valid pushing points for some stepsof the route.

• Obtain the vector that defines the box movement dv,for example: for the North direction the vector is (0,1),for the south direction the vector is (0, -1). In the caseof the rotations it is only necessary to define if it isclockwise or not, since the points for the rotations canbe previously defined.

• For each point of the object, the method obtains theprojection of the vector dv in the two object restric-tions that in their intersection define the point.

• If at least one of the 2 projections is negative, the pointis valid for pushing; on the other hand, if both are zeroor positive, the point is not valid for pushing.

4 Future work

In future works the pushing will be used to determine(by an optimization strategy) the best combination of robotsand pushing points in each sub trajectory to diminish theamount of reconfigurations of the N robots. Also a moduleto find the advantageous robots location is in developmentto place robots which are not participating in an interval ofpushing in a favorable position for another sub trajectory.The purpose is to continue reducing reconfiguration timesand to obtain a coordination strategy capable of concludingthe task in an efficient way.

5 Discussion and conclusions

In this paper, we have developed a strategy to solve theBox-pushing problem that incorporates characteristics to

improve the movement of objects in cluttered environmentsby a team of non-holonomics mobile robots. As it can beseen in the partial results, the proposed changes for thecomputation of the object trajectory improve the reductionof robots reconfigurations by the application of an alter-native heuristic based on the wavefront algorithm. Thereduction of reconfigurations is directly bound up with thereduction of necessary time to complete the task. Anotherpoint that throws good results was the improvement ofpushing points by applying an heuristic to evaluate andprivilege some layers of the C-space while the trajectoryis being constructed; this action is directly related to theprecision of the task.

This research was partialy funded by project number51623 from ”Fondo Mixto Conacyt-Gobierno del Estado deTamaulipas”

References

[1] L. E. Parker and F. Tang. Building multirobot coalitionsthrough automated task solution synthesis. Proceedings ofthe IEEE, 94(7):1289–1305, 2006.

[2] B. P. Gerkey and M. J. Mataric. Pusher-watcher: An approachto fault-tolerant tightly-coupled robot coordination. Roboticsand Automation, 2002. Proceedings. ICRA ’02. IEEE Inter-national Conference on, 1:464–469, 2002.

[3] K. Kovac, I. Zivkovic, and B. Dalbelo Basic. Simulation ofmulti-robot reinforcement learning for box-pushing problem.Electrotechnical Conference, 2004. MELECON 2004. Pro-ceedings of the 12th IEEE Mediterranean, 2:603– 606, 2004.

[4] Y. Li and X. Chen. Modeling and simulation of a swarm ofrobots for box-pushing task. 12th Mediterranean Conferenceon Control and Automation, Kusadasi, Aydin, Turkey, 2004.

[5] A. Munoz Melendez and A. Drogoul. Analyzing multi-robotbox-pushing. Avances en la Ciencia de la Computacion.Memoria de los Talleres del Quinto Encuentro Internacionalde Computacion. Universidad de Colima, pages 530–539,2004.

[6] Y. Wang and C. W. de Silva. Multi-robot box-pushing single-agent q-learning vs team q-learning. Intelligent Robots andSystems, 2006 IEEE/RSJ International Conference on, pages3694–3699, October 2006.

[7] Z. Wang and V. Kumar. Object closure and manipulation bymultiple cooperating mobile robots. Robotics and Automa-tion, 2002. Proceedings. ICRA ’02. IEEE International Con-ference on, 1:394–399, 2002.

[8] S. Yamada and J. Saito. Adaptive action selection withouthexplicit communication for multi-robot box-pushing. Sys-tems, Man and Cybernetics, Part C: Applications and Re-views, IEEE Transactions on, 31(3):398–404, 2001.

519