Enhanced Mars Rover Navigation TechniquesRichard Volpe Tara Estlin Sharon Laubach

Clark Olson J. (Bob) BalaramJet Propulsion Laboratory

California Institute of TechnologyPasadena, California 91109

Abstract

Robust navigation through rocky terrain by small mobilerobots is important for maximizing science return from up-coming missions to Mars. We are addressing this problemat multiple levels through the development of intelligent se-quencing, sensor constrained path planning, natural terrainvisual localization, and real-time state estimation. Each ofthese techniques will be described, and their complementaryaspects discussed. Experimental results are provided from im-plementation on our Mars rover prototype operating in real-istic scenarios.

1 Introduction

Launching in 2003 and again in 2005, NASA’s Mars SampleReturn (MSR) spacecraft will place two science rovers on thesurface of the planet. Each rover will be the primary sampleacquisition system for the mission, venturing out from the lan-der to core rocks and place instruments against them. Missionconstraints will restrict these surface operations to less than 90days, with communication only twice per day. Therefore, au-tonomous operations are required to image, navigate, sample,and return to the lander. Obviously, the complexity and ac-curacy of these autonomous operations directly influence theamount of science operations that will be performed.

For this reason, it has been the objective of the Long RangeScience Rover research project to introduce new functionalityand features into Mars rovers to enable greater science returnfrom upcoming missions. This paper reports on advances infour areas of rover systems:dynamic sequence generation,autonomous path planning, visual localization, and state esti-mation.

On-board planning with dynamic sequence generation al-lows much higher level commands to be provided by groundcontrollers, while increasing the optimality and robustnessof rover operations on the surface. For instance, during thePathfinder Mission, the Sojourner rover [7] was provided ex-tremely detailed sequences on a daily basis, which fatiguedoperators, and disallowed contingency operations when theflow of execution was non-nominal. Contrary to this, wehave been experimenting with on-board replanning which canchange the execution of daily activities based on unanticipatedvariations in quantities such as position, terrain, power, andtime.

For the longer traverses required of upcoming missions, au-tonomous path planning is desirable since operators will notbe able to see three-dimensional terrain features out to themore distant goal locations. Whereas Sojourner drove a to-tal of 84 meters during its entire mission, MSR rovers willbe capable of driving this distance in a single day. But stereoimagery of the terrain provided to operators will only havean envelope of 20 meters at best resolution. Therefore, thepath planning advances described here, will allow the rover tobe its own operator. It can image the terrain from periscopiccameras, select a path through the terrain to the edge of theeffective stereo range, and repeat the process until the goal isachieved.

Visual localization is a technique for using the changingview imaged by the rover to accurately determine its changein position in the environment. It utilizes the same terrainimagery as path planning, but for the purpose of monitoringthe apparent motion of three dimensional ground features af-ter the rover has completed a move. In this way, the on-boardposition estimate of the rover can be updated to compensatefor errors caused by wheel slippage or rock bumping. OnPathfinder, this localization functionality was performed man-ually by operators viewing Sojourner from the fixed positionlander cameras, restricting the update to once a day and per-mitting operations only within the stereo envelope of the lan-der. In contrast, the terrain-based localization described herecan be applied to many forms of landerless operations: incre-mental long traverses, local operations within range of a priorstereo panorama, localization in descent imagery, and closedchain rover moves with estimate smoothing.

In addition to periodic visual localization of the rover, wehave developed real-time position and heading estimation us-ing all other sensors on the vehicle: angular rate, accelerom-eter, sun sensor, and mobility system linkage (rocker-bogey)configuration. This technique moves far beyond the simpledead-reckoning of Sojourner, and improves upon our previousadvances in position estimation with sun sensing [11]. Theresults aid navigation during path execution, provide betterinput to localization, and replace both in visually featurelessterrain such as sand dunes.

In the next section we describe a typical scenario in whichthese techniques are employed for a rover long traverse. Thisis followed by sections describing each of the techniques ingreater detail. All have been implemented in our research sys-tem, Rocky 7 [12], and experimental validation is in progress.

1

Figure 1: Aerial view of the Arroyo Seco test site.

2 Long Range Traverse Scenario

To better understand the utility and sequence of operation forthese techniques, this section describes a typical long rangetraverse scenario. Figure 1 shows an aerial view of the ArroyoSeco next to JPL, where we conduct tests requiring traverseslonger than our MarsYard outdoor test facility. This view sim-ulates imagery that will be obtained during lander descent infuture Mars missions. It is assumed that the actual landinglocation is contained in the scene (which is 100 meters, topto bottom). Therefore, from this view operators and scientistsmay select goal locations that are outside of the stereo rangingenvelope of the rover cameras.

We have conducted previous tests in the upper portion ofthe image, where the goal location, though distant, was visiblefrom the start position of the rover. Ongoing tests use the largehill in the center of the image to obscure the goal from a startlocation in the lower left corner of the image.

As an aid to the rover, an intermediate waypoint may beselected above and to the left of the hill. In addition, sciencetargets may be specified along the way. When such a list isprovided to the sequence planner, it will generate a set of sub-goals for the path planner.

Before moving toward each subgoal, mast stereo imageryis obtained and processed to form a partial panorama and ele-vation map of the terrain in the desired direction, as shown inFigure 2. The elevation map is used to select a path throughthe local terrain, out to a distance of approximately five me-ters in the direction of the subgoal. After a path is determined,the rover moves along it using hazard cameras to avoid previ-ously unseen obstacles, while position and heading estimationare used to maintain its course.

Once the envelope of the previous panorama is reached, therover raises the mast again for more imaging. First, it looksback from its new vantage point to perform a localization op-eration. Then, it images forward in the direction of the currentsubgoal, repeating the path planning process. If panoramicimagery cannot discern a path around a large obstacle, therover will move to the obstacle and begin to follow its bound-ary until a straight line path is possible.

After reaching the subgoal, the sequence planner providesthe rover with the next subgoal. Modification of this subgoalis possible, and some targets may be discarded if constraints

dictate. Based on the sequencer’s decision, path planning pro-ceeds to the next subgoal it is provided. Details of this se-quencer are provided next.

3 Planning and Sequencing

To sequence and prioritize activities during the rover tra-verse, we have employed a dynamic on-board planning sys-tem named CASPER (Continuous Activity Scheduling, Plan-ning, Execution, and Replanning) [2, 3]. CASPER uses tech-niques from Artificial Intelligence planning and schedulingto generate rover command sequences and to dynamicallymodify those sequences in response to changing operatingcontext. Generated sequences satisfy the input goals whileobeying each of the rover’s resource constraints and opera-tions rules. Through dynamic planning, CASPER can au-tonomously modify planned activities when unexpected or un-predictable events occur.

Plans are produced for the rover by utilizing an iterative re-pair algorithm [13] which classifies plan conflicts and resolvesthem by performing one or more plan modifications. Con-flicts occur when a plan constraint has been violated, wherethis constraint could be temporal (e.g., a science activity mustoccur at a certain time) or involve a rover resource (e.g., therover has a limited amount of power) or state (e.g., the rovermust be at a certain position). If orbital or descent imageryis available to provide global information, CASPER uses atangent graph path planner [4] to estimate traversal lengthsand to determine intermediate waypoints that are needed tonavigate around any known obstacles. In addition, optimiza-tion heuristics can be utilized within CASPER to improve planquality. For the scenario discussed below, heuristics based onthe Traveling Salesman Problem were used to reduce overalltraversal distance and execution time.

As the rover moves sequentially through the plan, CASPERmonitors feedback from the rover sensors and control systems.Examples of this information are: command execution sta-tus, current position estimate and its uncertainty, and currentpower reserves. From these updates, new conflicts and/or op-portunities may arise, requiring the planner to re-plan in or-der to accommodate the unexpected events. For example, anupdate may cause an immediate plan change. If the wheelslippage has caused the position estimate uncertainty to growtoo large, the planner can command the rover to stop and per-form localization earlier than originally scheduled. An updatemay also affect activities scheduled much later in the plan. Ifa particular traversal has used more battery power than ex-pected, the planner may need to discard one of the remainingscience goals. Plan updates can also cause a re-ordering ofplan activities.

Figure 3 shows an example scenario in a map, where darkshapes represent obstacles known a priori (e.g. from landerdescent imagery). In this case, the initial plan for the traversewill bring the rover to an unexpected obstacle near the firstgoal, represented as a lightly shaded shape. Circumnavigationaround this obstacle will move the rover closer to other goals,triggering CASPER to recognize the situation and re-plan to

2

Figure 2: Steps of on-board terrain sensing: panoramic mosaic view from rover mast stereo imager, composite of heightsextracted from stereo views, and terrain elevation map.

Re-planned path

Original planned path

Figure 3: After encountering a previously unknown obstacleshown in light grey, CASPER replans the sequence of targets.

visit the closest goal first. We are currently evaluating this andsimilar scenarios experimentally.

4 Path Planning

While moving to global waypoints selected by CASPER, itis assumed that previously undetected obstacles may be en-countered. Therefore, to navigate to the target waypoint, wehave developed an iterative, sensor-based path planner calledRoverBug[5, 6]. It was developed specifically for vehicleswith a limited field-of-view (FOV), and constraints on com-putational resources. It comprises two operational modes,motion-to-goalandboundary-following, which interact to en-sure global convergence. In addition, a “virtual” submodeof boundary-followingimproves efficiency and handles thelimited FOV of the vehicle’s sensors: stereo camera pairs,mounted on the chassis and on a 1.2 m mast. The visibleregion ranged by each stereo pair is roughly a wedge of the

ground plane, with limited downrange radiusR and half-angle�. For the chassis(R;�) = (1:5m; 45�), and for the mast(7m; 15�).

The RoverBug motion planner identifies the minimal num-ber of sensor scans needed to proceed at each step, whilespecifying the area to scan and avoiding unnecessary rovermotion. The planner uses a streamlined local model of con-vex polygons enveloping detected obstacles. This is renewedat every step to avoid global map maintenance issues. Otherthan recorded points and parameters, no information is passedbetween steps. However, the algorithm does require good lo-calization to track the goal position and to determine whetherthe rover has executed a loop around an obstacle. Therefore,the planner has been paired with an on-board localization al-gorithm described in Section 5.

The RoverBug algorithm relies upon the construction of alocal tangent graph within the visible wedge. The tangentgraph consists of all line segments in freespace connecting theinitial position, the goal, and all obstacle vertices, such that thesegments are tangent to any obstacles they encounter. A linelis tangentto an obstacleO at a vertexx iff in a neighborhoodof x, the interior ofO lies entirely on one side ofl [4]. Thelocal tangent graph(LTG) is the tangent graph restricted tothe visible region.

Typically, motion-to-goalis the dominant behavior. Its ob-jective is monotonic progress toward the goal,T , along theLTG. After executing the resultant subpath from a givenwedge,motion-to-goalbegins anew. This cycle repeats untileither the rover reaches the goal, or no clear path toT existswithin the visible region. If the planner detects that the rovercannot make forward progress through the current wedge, therover must skirt an obstacle to reach the goal. In this case,RoverBug then switches to itsboundary-followingmode.

Upon detecting the obstacleO, it is clear that the rover mustcircumvent it in order to resume progress towardT . The ob-jective of theboundary-followingmode is to skirt the bound-ary of the obstacle, finding shortcuts where possible. Uponfirst detecting the obstacle, and on each subsequent imag-ing step along the obstacle boundary, the algorithm “virtuallyslides” along the boundary using “gaze control”, avoiding un-necessary motion toward the obstacle while progressing as faras is locally possible around the border.Boundary-following

3

Figure 4: Experimental results from a multi-step run usingRoverbug in the JPL MarsYard.

continues until the rover either completes a loop, in whichcaseT is unreachable and the algorithm halts, or the locallyvisible region contains a new subpath toward the goal. In thelatter case, the mode switches back tomotion-to-goal.

It can be shown that with the two simple operational modesof motion-to-goalandboundary-following, the RoverBug al-gorithm is guaranteed to reach the goal or halt if the goal isunreachable [5]. The algorithm completes in finite time, iscorrect, and produces locally optimal paths (that is, shortestlength paths given local knowledge of obstacles). Further-more, RoverBug deals with the limited FOV of typical all-terrain rovers in a manner which is made efficient by the useof autonomous gaze control to reduce the need to sense andstore data.

Experimental data from the use of RoverBug on Rocky 7 isshown in Figure 4. The path begins in the lower right cornerof the image, toward a goal approximately 21 m distant, in theupper left. Each wedge depicts a rangemap produced frommast imagery, and extends roughly 5m from the imaging po-sition. The obstacles are marked by a black convex hull, anda grey silhouette. Each subpath ends with an apparent side-step in the path, which is a position correction. At these spots,rover localization is performed and the result is used to updatethe current estimate of the rover position. This localizationtechnique is described next.

5 Localization

Automatic visual localization is performed by Rocky 7 in or-der to correct errors that have accumulated in the rover posi-tion during traverses, as described in Section 6. It is accom-plished by imaging the terrain near the rover and comparingit to a previously generated elevation map. The imaging di-rection is guided by on-board analysis of the previous map.Both terrain maps are generated using stereo vision on-boardthe rover, as shown in Figure 2.

The problem is formulated in terms of maximum-likelihood

estimation, using the distances from the occupied cells in thelocal map to their closest occupied cells in the global map asmeasurements. In order to accurately model the sensor un-certainty, we use a probability density function for the mea-surements that is the weighted sum of two terms representingthe cases where the measurement is an inlier (in the sense thatthe terrain position under consideration in the local map alsoexists in the global map) or an outlier [9].

The robot position is determined by a multi-resolutionsearch strategy which is guaranteed to locate the optimal po-sition in the discretized search space [10]. The pose space isfirst discretized at the same resolution as the occupancy gridsso that neighboring positions in the pose space move the rel-ative positions of the grids by one grid cell. We then test thenominal position of the robot given by dead-reckoning so thatwe have an initial position and likelihood to compare against.Next, the pose space is divided into rectilinear cells. Each cellis tested to determine whether it could contain a position thatis better than the best position found so far. Cells that cannotbe pruned are divided into smaller cells, which are examinedrecursively. When a cell is reached that contains a single po-sition in the discretized pose space, then this position is testedexplicitly.

The uncertainty in the localization is estimated in terms ofboth the variance of the estimated positions and the proba-bility that a qualitative failure has occurred. Since the likeli-hood function measures the probability that each position inthe pose space is the actual robot position, the uncertainty inthe localization is measured by the rate at which the likeli-hood function falls off from the peak. In addition, subpixellocalization is performed in the discretized pose space by fit-ting a surface to the peak which occurs at the most likely robotposition.

Prior to performing localization, the rover analyzes the ter-rain in the map generated at the initial rover position in orderto select alocalization target. This target is the position in theterrain that the rover images from its final position in orderto generate a match against the global map. It is important toselect a location that has very distinctive terrain to enable thelocalization to be performed with the smallest uncertainty.

The localization target is determined by estimating theamount of error present in the map computed at the initialrover position as well as the amount of error that would begenerated by imaging the terrain from the final rover position.These errors are encoded in a probability map of the terrainexpected to be seen from the final rover position. Each cellin this map contains an estimate of the probability that thecell will be seen as occupied by the rover. By treating thisprobability map as a terrain map and comparing it to the mapgenerated at the initial rover position, we can predict the un-certainty that will occur in the localization for any target thatthe rover may view for use in terrain matching. The locationwith the lowest predicted uncertainty is selected as the local-ization target.

Figure 5 shows an example of a target selected usingRocky 7. The rover was initially located at the bottom cornerof the map facing to the right. Six stereo pairs were used tobuild an elevation map of the terrain between nine and twelve

4

Figure 5: Example of target selection for localization.

o’clock, out to a distance of five meters. Given the goal lo-cation three meters in front of the rover, a localization targetat eleven o’clock was selected automatically. After moving tothe goal location, the rover imaged the target again, establish-ing its new position and replacing the on-board estimate withthe corrected position. In this example the initial estimate er-ror was made large by using odometry only. Typically, thislocalization procedure results in decreasing the error to oneone-percent of the distance traveled.

6 State Estimation

In between visual localization operations, non-visual methodsof state estimation are called for. Such methods are importantin their own right since they serve as backup to vision-basedmethods in regions of low visual content and also allow bet-ter initialization of the vision methods. This section describesa method to improve the precision of the odometry estimateby using the full kinematics of the rocker-bogey mechanismsof the rover as it traverses undulating, bumpy terrain. TheKalman filtering framework adopted also provides a naturalBayesian means of combining any visually based motion es-timates into the full state estimate.

6.1 Rover Model For Estimation

The process model used in the filter is chosen so that the sen-sor data is used as an input to drive the process equation. Thisavoids the difficulty of modeling the detailed process dynam-ics. Since details are discussed elsewhere [1], we only out-line some features of the rover contact and kinematics modelshere.

The contact point vector is modeled very simply as a set ofone parameter contacts about the equator of each wheel. Inreality there is an additional off-equatorial coordinate for thecontact point at each wheel, a contact rotation angle, and twoparameters that describe the point on the ground [8]. Howeverthe one parameter model suffices to capture and couple therotational and translational velocities.

Instead of attempting to formulate an explicit inverse kine-matics relation for the articulated rover, we choose to embedthe easily established forward kinematics within aconstraintthat is treated as ameasurementin the filter. This exploits the

0 2 4 6 8 10−10

0

10

20

ξ1

(deg

) ESTSIM

0 2 4 6 8 10−10

−5

0

5

10

ξ2

(deg

) ESTSIM

0 2 4 6 8 10−10

−5

0

5

10

t (s)

ξ3

(deg

) ESTSIM

0 2 4 6 8 10−20

−10

0

10

ξ4

(deg

) ESTSIM

0 2 4 6 8 10−10

−5

0

5

ξ5

(deg

) ESTSIM

0 2 4 6 8 10−5

0

5

10

t (s)

ξ6

(deg

) ESTSIM

Figure 6: Simulation results showing true values and esti-mates for wheel contact positions.

ability of the Kalman filter to perform the appropriate least-squares averaging of the action of each kinematic chain in therover. Each such forward kinematic chain has a componentdefined by sequence of links joining the rover frame to eachwheel contact point, and a component given by the slip be-tween the wheel and the ground.

We introduce the notion of a slip measurement or con-straint, that defines the relative 6-DOF motion of the contactframe on the wheel with respect to the ground. This slip isa function of the vehicle configuration, the 6-DOF vehiclevelocity, the wheel-to-ground contact point location, and thejoint rates associated with the kinematic chain emanating fromthe rover frame to the contact point.

The slip constraint measurement can be decomposed into aknown deterministic component and a component that is onlyknown in a statistical sense. The deterministic component ofthe slip, indicated by a non-zero nominal value of the slip,is used to capture the effects of a known steering action or aknown average slip rate over different kinds of terrain. Thestatistically modeled component is due to wheel-ground in-teraction at each individual wheel. This is a function of thecontroller (compliant or stiff) being used, and the terrain cur-vature. We have selected a simple un-correlated slip modelfor our early implementations, with the covariance strengthsdetermined by experiments.

6.2 Estimation Experiments

Figure 6 shows the results of a simulation for a rover mov-ing over undulating terrain. Such simulation experiments arevaluable since ground-truth is easily established. Note thatthe tracking of the contact points is quite accurate, althoughwith some lag. By comparing the performance of the kine-matic estimator with one based upon dead-reckoning, the im-proved performance is apparent. Specifically, the averagedwheel odometry obtained by integrating the speed (as indi-cated at each wheel) results in a2% error over the actual dis-tance traveled (even when compensated for the instantaneousvehicle pitch angle). Using the kinematic estimator, the error

5

0 5 10 15 20 25−10

−5

0

5

ξ1

(de

g)

0 5 10 15 20 25−2

−1

0

1

ξ2

(de

g)

0 5 10 15 20 25−1

0

1

2

t (s)

ξ3

(de

g)

0 5 10 15 20 25−20

−10

0

10

ξ4

(de

g)

0 5 10 15 20 25−10

0

10

20

ξ5

(de

g)

0 5 10 15 20 25−20

−10

0

10

t (s)

ξ6

(de

g)

Figure 7: Experimental results from Rocky 7 showing the es-timate for wheel contact angles.

in distance traveled is then estimated to within the much im-proved value of0:3%. This is due to the corrections for theeffects of the nonlinear internal kinematics and the variationsin contact angles at all of the wheels.

We have also conducted experiments with Rocky 7 travers-ing an obstacle on the right-side of the vehicle. Estimatedcontact states are shown in Figure 7. Note that the contact an-gle variations are quite large under the right wheels as wouldbe expected by the traversal of those wheels over the obstacle.Since the final configuration of the rover is such that the right-side bogey wheels are in the middle of traversing the obstacle,the corresponding contact points are significantly displacedfrom zero at the end of the motion. However, the contact pointfor the right front wheel returns to near zero as it proceeds onlevel ground after climbing over the obstacle. As expectedthe wheels on the left side of the vehicle experience very lit-tle change in contact angles. The estimated position values arewithin 1 cm of the ground-truth data and the estimated contactangles are within 5 degrees.

7 Summary

This paper has provided an overview of four new techniquesdeveloped to enhance the functionality of autonomous longrange Mars rover navigation: intelligent sequencing, sensorconstrained path planning, natural terrain visual localization,and Kalman Filter state estimation. While integration andtests continue, initial results indicate a dramatic improvementin both knowledge of the rover’s position in the terrain andnavigation decision making based on that knowledge.

8 Acknowledgments

Large systems like ours require the support of many peoplebeyond the authors. We would like to recognize the follow-ing individuals for their invaluable contribution to the workdescribed in this paper: Richard Petras, Darren Mutz, GregRabideau, Mark Maimone, and Samad Hayati.

The research described in this paper was carried out by theJet Propulsion Laboratory, California Institute of Technology,under a contract with the National Aeronautics and Space Ad-ministration. Reference herein to any specific commercialproduct, process, or service by trade name, trademark, manu-facturer, or otherwise, does not constitute or imply its endorse-ment by the United States Government or the Jet PropulsionLaboratory, California Institute of Technology.

References[1] J. Balaram. Kinematic State Estimation for a Mars Rover.

Robotica, Special Issue on Intelligent Autonomous Vehicles,Accepted for publication, 1999.

[2] S. Chien, R. Knight, R. Sherwood, and G. Rabideau. IntegratedPlanning and Execution for Autonomous Spacecraft. InIEEEAerospace Conference, Aspen CO, March 1999.

[3] T. Estlin, G. Rabideau, D. Mutz, and S. Chien. Using Con-tinuous Planning Techniques to Coordinate Multiple Rovers.In IJCAI Workshop on Scheduling and Planning, Stockholm,Sweden, August 1999.

[4] J. Latombe.Robot Motion Planning. Kluwer Academic Pub-lishers, Boston, 1991.

[5] S. Laubach.Theory and Experiments in Autonomous Sensor-Based Motion Planning with Applications for Flight PlanetaryMicrorovers. PhD thesis, California Institute of Technology,May 1999.

[6] S. Laubach and J. Burdick. An Autonomous Sensor-BasedPath-Planner for Planetary Microrovers. InIEEE InternationalConference on Robotics and Automation, Detroit MI, 1999.

[7] J. Matijevic et al. Characterization of the Martian SurfaceDeposits by the Mars Pathfinder Rover, Sojourner.Science,278:1765–1768, December 5 1997.

[8] D. Montana. The kinematics of contact and grasp.Interna-tional Journal of Robotics Research, 7(3):17–32, 1988.

[9] C. Olson. Subpixel Localization and Uncertainty EstimationUsing Occupancy Grids. InIEEE International Conference onRobotics and Automation, pages 1987–1992, Detroit, Michi-gan, May 1999.

[10] C. Olson and L. Matthies. Maximum-likelihood Rover Local-ization by Matching Range Maps. InIEEE International Con-ference on Robotics and Automation, pages 272–277, Leuven,Belgium, May 1998.

[11] R. Volpe. Navigation Results from Desert Field Tests of theRocky 7 Mars Rover Prototype.International Journal ofRobotics Research, 18(7), 1999.

[12] R. Volpe et al. Rocky 7: A Next Generation Mars Rover Pro-totype.Journal of Advanced Robotics, 11(4):341–358, 1997.

[13] M. Zweben, B. Daun, E. Davis, and M. Deale. Scheduling andRescheduling with Iterative Repair. In J. Zweben and M. Fox,editors,Intelligent Scheduling, pages 241–256. Morgan Kauf-man, 1994.

6