Motion Planning:Motion Planning:A Journey of Robots, Digital Actors, Molecules and A Journey of Robots, Digital Actors, Molecules and

Other ArtifactsOther Artifacts

Jean-Claude LatombeJean-Claude Latombe

Computer Science DepartmentComputer Science Department

Stanford UniversityStanford University

My Research InterestsMy Research Interests

Autonomous agents that sense, plan, and act in real and/or virtual worlds

Algorithms and systems for representing, capturing, planning, controlling, and rendering motions of physical objects

Applications:

– Manufacturing

– Mobile robots

– Computational biology

– Computer-assisted surgery

– Digital actors

Goal of Motion PlanningGoal of Motion Planning

Compute Compute motion strategiesmotion strategies, e.g.:, e.g.:

– geometric paths geometric paths

– time-parameterized trajectoriestime-parameterized trajectories

– sequence of sensor-based motion commandssequence of sensor-based motion commands

To achieve To achieve high-level goals, high-level goals, e.g.:e.g.:

– go to A without colliding with obstaclesgo to A without colliding with obstacles

– assemble product Passemble product P

– build map of environment Ebuild map of environment E

– find object Ofind object O

Goal of Motion PlanningGoal of Motion Planning

Compute Compute motion strategiesmotion strategies, e.g.:, e.g.:

– geometric paths geometric paths

– time-parameterized trajectoriestime-parameterized trajectories

– sequence of sensor-based motion commandssequence of sensor-based motion commands

To achieve To achieve high-level goals, high-level goals, e.g.:e.g.:

– go to A without colliding with obstaclesgo to A without colliding with obstacles

– assemble product Passemble product P

– build map of environment Ebuild map of environment E

– find object Ofind object O

Goal of Motion PlanningGoal of Motion Planning

Compute Compute motion strategiesmotion strategies, e.g.:, e.g.:

– geometric paths geometric paths

– time-parameterized trajectoriestime-parameterized trajectories

– sequence of sensor-based motion commandssequence of sensor-based motion commands

To achieve To achieve high-level goals, high-level goals, e.g.:e.g.:

– go to A without colliding with obstaclesgo to A without colliding with obstacles

– assemble product Passemble product P

– build map of environment Ebuild map of environment E

– find object Ofind object O

ExamplesExamples

Is It Easy?Is It Easy?

Basic ProblemBasic Problem

Statement:Statement: Compute a collision-free path for a rigid or Compute a collision-free path for a rigid or articulated object (the robot) among static obstaclesarticulated object (the robot) among static obstacles

Inputs:Inputs:– Geometry of robot and obstaclesGeometry of robot and obstacles– Kinematics of robot (degrees of freedom)Kinematics of robot (degrees of freedom)– Initial and goal robot configurations (placements)Initial and goal robot configurations (placements)

Outputs:Outputs:– Continuous sequence of collision-free robot configurations Continuous sequence of collision-free robot configurations

connecting the initial and goal configurationsconnecting the initial and goal configurations

Example with Rigid ObjectExample with Rigid Object

Example with Articulated ObjectExample with Articulated Object

Extensions to the Basic ProblemExtensions to the Basic Problem

Moving obstaclesMoving obstacles

Multiple robotsMultiple robots

Movable objectsMovable objects

Deformable objectsDeformable objects

Goal is to gather data by Goal is to gather data by sensingsensing

Nonholonomic constraintsNonholonomic constraints

Dynamic constraintsDynamic constraints

Optimal planningOptimal planning

Uncertainty in control and Uncertainty in control and sensingsensing

Application: Design for ManufacturingApplication: Design for Manufacturing

General ElectricGeneral Electric General MotorsGeneral Motors

General MotorsGeneral Motors

Application: Robot Programming and PlacementApplication: Robot Programming and Placement

David Hsu’s PhDDavid Hsu’s PhD

Application: Checking Building CodeApplication: Checking Building Code

Charles Han’s PhDCharles Han’s PhD

Application: Generation of Instruction SheetsApplication: Generation of Instruction Sheets

Application: Model Construction by Mobile Application: Model Construction by Mobile RobotRobot

Hector Gonzalez’s PhDHector Gonzalez’s PhD

Application: Graphic Animation of Digital ActorsApplication: Graphic Animation of Digital Actors

James Kuffner’s PhDJames Kuffner’s PhD

Application: Computer-Assisted Surgical PlanningApplication: Computer-Assisted Surgical Planning

Rhea Tombropoulos’s PhDRhea Tombropoulos’s PhD

Joel Brown’s PhDJoel Brown’s PhD

Application: Prediction of Molecular MotionsApplication: Prediction of Molecular Motions

Amit Singh’s PhDAmit Singh’s PhD

Motion in Configuration Space Motion in Configuration Space

q2

q1

q3

q0

qn

q4

Q(t)

)(

)(

)(0

tq

tq

tQ

n

Tt ,0

Parts DOFL 19 68H 51 118

Disc Robot in 2-D WorkspaceDisc Robot in 2-D Workspace

Rigid Robot Translating in 2-DRigid Robot Translating in 2-D

CB = B A = {b - a | a in A, b in B}CB = B A = {b - a | a in A, b in B}

Rigid Robot Translating and Rotating in 2-DRigid Robot Translating and Rotating in 2-D

C-Obstacle for Articulated RobotC-Obstacle for Articulated Robot

Other Representation ConceptsOther Representation Concepts

State space (configuration x velocity)State space (configuration x velocity)

Configuration/state x time spaceConfiguration/state x time space

Composite configuration/state spacesComposite configuration/state spaces

Stability regions in configuration/state spacesStability regions in configuration/state spaces

Visibility regions in configuration/state spacesVisibility regions in configuration/state spaces

Etc …Etc …

Motion Planning as a Computational ProblemMotion Planning as a Computational Problem

Goal: Goal: Compute the connectivity of a space (e.g., the Compute the connectivity of a space (e.g., the collision-free subset of configuration space)collision-free subset of configuration space)

High computational complexity: High computational complexity: Typically requires time exponential in an input Typically requires time exponential in an input parameter, e.g., the number of degrees of freedom, the parameter, e.g., the number of degrees of freedom, the number of moving obstacles, …number of moving obstacles, …

Two main algorithmic approaches:Two main algorithmic approaches:

– Planning by random samplingPlanning by random sampling– Planning by computing criticalitiesPlanning by computing criticalities

Motion Planning as a Computational ProblemMotion Planning as a Computational Problem

Goal: Characterize the connectivity of a space (e.g., Goal: Characterize the connectivity of a space (e.g., the collision-free subset of configuration space)the collision-free subset of configuration space)

High computational complexity: Requires time High computational complexity: Requires time exponential in number of degrees of freedom, or exponential in number of degrees of freedom, or number of moving obstacles, or etc…number of moving obstacles, or etc…

Two main algorithmic approaches:Two main algorithmic approaches:

– Planning by random samplingPlanning by random sampling

– Planning by extracting criticalitiesPlanning by extracting criticalities

Motion Planning as a Computational ProblemMotion Planning as a Computational Problem

Goal: Characterize the connectivity of a space (e.g., Goal: Characterize the connectivity of a space (e.g., the collision-free subset of configuration space)the collision-free subset of configuration space)

High computational complexity: Requires time High computational complexity: Requires time exponential in number of degrees of freedom, or exponential in number of degrees of freedom, or number of moving obstacles, or etc…number of moving obstacles, or etc…

Two main algorithmic approaches:Two main algorithmic approaches:

– Planning by random samplingPlanning by random sampling

– Planning by extracting criticalitiesPlanning by extracting criticalities

Principle of Randomized PlanningPrinciple of Randomized Planning

free space

qqbb

qqgg

milestone

[Kavraki, Svetska, Latombe,Overmars, 95][Kavraki, Svetska, Latombe,Overmars, 95]

(Probabilistic Roadmap)(Probabilistic Roadmap)

Why Does it Work?Why Does it Work?[Kavraki, Latombe, Motwani, Raghavan, 95]

In Theory, a PRM Planner …In Theory, a PRM Planner …

Is Is probabilistically completeprobabilistically complete, i.e., whenever a , i.e., whenever a solution exists, the probability that it finds one solution exists, the probability that it finds one tends toward 1 as the number N of milestones tends toward 1 as the number N of milestones increasesincreases

Under rather general hypotheses, the rate of Under rather general hypotheses, the rate of convergence is exponential in the number N of convergence is exponential in the number N of milestones, i.e.:milestones, i.e.:

Prob[failure] ~ exp(-N)Prob[failure] ~ exp(-N)

In practice, PRM Planners …In practice, PRM Planners …

Are fastAre fast Deal effectively with many-dof robotsDeal effectively with many-dof robots Are easy to implementAre easy to implement Have solved complex problemsHave solved complex problems

Example 1: Planning of Manipulation MotionsExample 1: Planning of Manipulation Motions

ReachReach

GrabGrab

TransferTransfer

ReleaseRelease

ReturnReturn

Example 1: Planning of Manipulation MotionsExample 1: Planning of Manipulation Motions

Example 2: Air-Cushioned RobotExample 2: Air-Cushioned Robot

air bearingair bearing

gaz tankgaz tank

air thrustersair thrustersobstacles

robotrobot

(Aerospace Robotics Lab)(Aerospace Robotics Lab)

Total duration : 40 secTotal duration : 40 sec

Example 3: Radiosurgical PlanningExample 3: Radiosurgical Planning

Cyberknife (Neurosurgery Dept., Stanford, Cyberknife (Neurosurgery Dept., Stanford, Accuray) Accuray)

Surgeon Specifies Dose ConstraintsSurgeon Specifies Dose Constraints

Critical

Tumor

Fall-off of Dose Around the Tumor

Dose to theTumor Region

Dose to the Critical Region

Fall-off of Dosein the Critical Region

Beam Selection AlgorithmBeam Selection Algorithm Place points uniformly at random on the Place points uniformly at random on the

surface of the tumorsurface of the tumor Pick beam orientations at random at Pick beam orientations at random at

these pointsthese points

Beam Selection AlgorithmBeam Selection Algorithm Place points uniformly at random on the Place points uniformly at random on the

surface of the tumorsurface of the tumor Pick beam orientations at random at Pick beam orientations at random at

these pointsthese points

Compute Beam WeightsCompute Beam Weights

• 2000 < Tumor < 22002000 < B2 + B4 < 22002000 < B4 < 22002000 < B3 + B4 < 22002000 < B3 < 22002000 < B1 + B3 + B4 < 22002000 < B1 + B4 < 22002000 < B1 + B2 + B4 < 22002000 < B1 < 22002000 < B1 + B2 < 2200

• 0 < Critical < 5000 < B2 < 500

T

C

B1

B2

B3

B4

T

Sample CaseSample Case

Linac plan80% Isodose surface

CARABEAMER’s plan80% Isodose surface

Sample CaseSample Case

50% Isodose Surface

80% Isodose Surface

Linac plan CARABEAMER’s plan

Example 4: Indoor Map Building by RobotExample 4: Indoor Map Building by Robot

Next-Best View StrategyNext-Best View Strategy

Computing Next Sensing PositionComputing Next Sensing Position Sample the free edges of the visited region at random. For each sample point, compute the subset of visited region from which this point is visible and sample Sample the free edges of the visited region at random. For each sample point, compute the subset of visited region from which this point is visible and sample

this subset at random. this subset at random. >> Set of candidate positions >> Set of candidate positions qq

Select “best” candidate Select “best” candidate qq based on following criteria: based on following criteria:

– overlap of visible environment edges (to ensure reliable alignment)overlap of visible environment edges (to ensure reliable alignment)– amount of potential new space visible from amount of potential new space visible from qq– length of path to go to length of path to go to qq

Map Construction ExampleMap Construction Example

1122

44 66

Robotics Lab MapRobotics Lab Map

45m45m

Example 5: Digital Actor with Vision SensingExample 5: Digital Actor with Vision Sensing

Example 5: Digital Actor with Vision SensingExample 5: Digital Actor with Vision Sensing

Example 6: Predicting Molecule Docking MotionsExample 6: Predicting Molecule Docking Motions

Future Work: Minimally Invasive Surgey Future Work: Minimally Invasive Surgey Amidst Soft Tissue StructuresAmidst Soft Tissue Structures

Future Work: Autonomous Interactive CharactersFuture Work: Autonomous Interactive Characters

A Bug’s Life (Pixar/Disney) Toy Story (Pixar/Disney)

Tomb Raider 3 (Eidos Interactive) Final Fantasy VIII (SquareOne)The Legend of Zelda (Nintendo)

Antz (Dreamworks)

Future Work: Protein FoldingFuture Work: Protein Folding

Summary/ConclusionSummary/Conclusion

Over the last decade there has been considerable progress Over the last decade there has been considerable progress in motion planning techniques and their applicationin motion planning techniques and their application

While motion planning originated in robotics, the areas of While motion planning originated in robotics, the areas of application are now very diverse: product design, application are now very diverse: product design, manufacturing, graphic animation, video games, biology, manufacturing, graphic animation, video games, biology, etc…etc…

There are orders of magnitude more processors embedded There are orders of magnitude more processors embedded in physical devices (cars, planes, surgical instruments, in physical devices (cars, planes, surgical instruments, etc) than desktop computers, and the gap is still growing. etc) than desktop computers, and the gap is still growing. The interest in modeling and computing the motion of The interest in modeling and computing the motion of physical objects will continue to grow.physical objects will continue to grow.