State-Space Planning Dr. Héctor Muñoz-Avila Sources: Ch. 3 Appendix A Slides from Dana Nau’s...

State-Space Planning

Dr. Héctor Muñoz-Avila

Sources:• Ch. 3• Appendix A•Slides from Dana Nau’s lecture

Reminder: Some Graph Search Algorithms (I)

G = (V,E)

• Breadth-first search: Use a queue (FIFO policy) to control order of visits• State-firsts search: Use an stack (FILA policy) to control order of visits• Best-first search: minimize an objective function f(): e.g., distance. Djikstra

• Greedy search: select best local f() and “never look back” • Hill-climbing search: like greedy search but every node is a solution

• Iterative deepening: exhaustively explore up to depth i

Vertices

Edges: set of pairs of vertices (v,v’) A

CB

D

F

G

E

Z1020

16

46

8 24

20

69

State-Space Planning

AC

B A B C A CB

CBA

BA

C

BAC

B CA

CAB

ACB

BCA

A BC

AB

C

ABC

• Search is performed in the state space

• State space•V: set of states•E: set of transitions (s, (s,a))

• But: computed graph is a subset of state space

•Remember the number of states for DWR states?

• Search modes:• Forward• Backward

Nondeterministic Search•The oracle definition: given a choice between possible transitions, there is a device, called the oracle, that always chooses the transition that leads to a state satisfying the goals

•Alternative definition (book): parallel computers

“current state”

...

...

...

…Oracle chooses this transition because it leads to a favorable state

Forward Search

take c3

move r1

take c2 …

…

goals+(g) s and goals–(g) s =

satisfied

Soundness and Completeness

• A planning algorithm A can be sound, complete, and admissible• Soundness:

Deterministic: Given a problem and A returns P failure then P is a solution for the problem

Nondeterministic: Every plan returned by A is a solution• Completeness:

Deterministic: Given a solvable problem then A returns P failure

Nondeterministic: Given a solvable problem then A returns a solution

• Admissible: If there is a measure of optimality, and a solvable problem is given, then A returns an optimal solution

• Result: Forward-search is sound and complete Proof: …

Reminder: Plans and Solutions

• Plan: any sequence of actions = a1, a2, …, an such thateach ai is a ground instance of an operator in O

• The plan is a solution for P=(O,s0,g) if it is executable and achieves g i.e., if there are states s0, s1, …, sn such that

• (s0,a1) = s1

• (s1,a2) = s2

• …

• (sn–1,an) = sn

• sn satisfies g

Dana Nau: Lecture slides for Automated PlanningLicensed under the Creative Commons Attribution-NonCommercial-ShareAlike License: http://creativecommons.org/licenses/by-nc-sa/2.0/

Example of a deterministic variant of Forward-search that is not sound?

Deterministic Implementations• Some deterministic implementations

of forward search: breadth-first search depth-first search best-first search (e.g., A*) greedy search

• Breadth-first and best-first search are sound and complete But they usually aren’t practical because they require too much memory Memory requirement is exponential in the length of the solution

• In practice, more likely to use depth-first search or greedy search Worst-case memory requirement is linear in the length of the solution In general, sound but not complete

• But classical planning has only finitely many states

• Thus, can make depth-first search complete by doing loop-checking

s0

s1

s2

s3

a1

a2

a3

s4

s5

sg

a4

a5 …


So why they are typically not used?

How can we make it complete?

Another domain: The Blocks World• Infinitely wide table, finite number of children’s blocks• Ignore where a block is located on the table• A block can sit on the table or on another block• Want to move blocks from one configuration to another

e.g.,

initial state goal

• Can be expressed as a special case of DWRBut the usual formulation is simpler

c

a

bc

a b e

d

Classical Representation: Symbols

• Constant symbols:The blocks: a, b, c, d, e

• Predicates:ontable(x) - block x is on the tableon(x,y) - block x is on block yclear(x) - block x has nothing on itholding(x) - the robot hand is holding block xhandempty - the robot hand isn’t holding

anything

c

a b e

d


unstack(x,y) Precond: on(x,y), clear(x), handempty Effects: ~on(x,y), ~clear(x), ~handempty,

holding(x), clear(y)

stack(x,y) Precond: holding(x), clear(y) Effects: ~holding(x), ~clear(y),

on(x,y), clear(x), handempty

pickup(x) Precond: ontable(x), clear(x), handempty Effects: ~ontable(x), ~clear(x),

~handempty, holding(x)

putdown(x) Precond: holding(x) Effects: ~holding(x), ontable(x),

clear(x), handempty

Classical Operators c

a b

ca b

c

a b

c

ab

c

a bDana Nau: Lecture slides for Automated PlanningLicensed under the Creative Commons Attribution-NonCommercial-ShareAlike License: http://creativecommons.org/licenses/by-nc-sa/2.0/

Branching Factor of Forward Search

• Forward search can have a very large branching factorE.g., many applicable actions that don’t progress toward

goal• Why this is bad:

Deterministic implementations can waste time trying lots of irrelevant actions

• Need a good heuristic function and/or pruning procedureChad and Hai are going to explain how to do this

a3

a1

a2

…a1 a2 a50a3

initial state goal

Backward Search• For forward search, we started at the initial state and

computed state transitionsnew state = (s,a)

• For backward search, we start at the goal and compute inverse state transitionsnew set of subgoals = –1(g,a)

• To define -1(g,a), must first define relevance:An action a is relevant for a goal g if

• a makes at least one of g’s literals true– g effects(a) ≠

• a does not make any of g’s literals false– g+ effects–(a) = and g– effects+(a) =


How do we write this formally?

Inverse State Transitions

• If a is relevant for g, then –1(g,a) = (g – effects+(a)) precond(a)

• Otherwise –1(g,a) is undefined

• Example: suppose thatg = {on(b1,b2), on(b2,b3)}a = stack(b1,b2)

• What is –1(g,a)?• What is –1(g,a) = {s : (s,a) satisfies g}?

g0

g1

g2

g3

a1

a2

a3

g4

g5

s0

a4

a5


How about using –1(g,a) instead of –1(g,a)?

Efficiency of Backward Search

• Backward search can also have a very large branching factorE.g., many relevant actions that don’t regress toward the

initial state• As before, deterministic implementations can waste lots of

time trying all of them

a3

a1

a2

…a1 a2 a50a3

initial state goal

Compared with forward search?


Lifting

• Can reduce the branching factor of backward search if we partially instantiate the operators this is called lifting

q(a)foo(a,y)

p(a,y)

q(a)

foo(x,y)precond: p(x,y)effects: q(x)

foo(a,a)

foo(a,b)

foo(a,c)

…

p(a,a)

p(a,b)

p(a,c)


Lifted Backward Search

• More complicated than Backward-search Have to keep track of what substitutions were performed

• But it has a much smaller branching factor


The Search Space is Still Too Large

• Lifted-backward-search generates a smaller search space than Backward-search, but it still can be quite large Suppose a, b, and c are independent, d must precede all of

them, and d cannot be executed We’ll try all possible orderings of a, b, and c before realizing

there is no solution More about this in Chapter 5 (Plan-Space Planning)

c

b

a

goal

a b

b a

b a

a c

b c

c b

d

d

d

d

d

dDana Nau: Lecture slides for Automated PlanningLicensed under the Creative Commons Attribution-NonCommercial-ShareAlike License: http://creativecommons.org/licenses/by-nc-sa/2.0/

STRIPS• π the empty plan• do a modified backward search from g

instead of -1(s,a), each new set of subgoals is just precond(a)

whenever you find an action that’s executable in the current state, then go forward on the current search path as far as possible, executing actions and appending them to π

repeat until all goals are satisfied

g

g1

g2

g3

a1

a2

a3

g4

g5

g3

a4

a5

current search path

a6

π = a6, a4s = ((s0,a6),a4)

g6

a3

satisfied in s0


The Sussman Anomaly

Initial state goal

• On this problem, STRIPS can’t produce an irredundant solution

c

a b c

a

b


The Register Assignment Problem

• State-variable formulation:

Initial state: {value(r1)=3, value(r2)=5, value(r3)=0}

Goal: {value(r1)=5, value(r2)=3}

Operator: assign(r,v,r’,v’)

precond: value(r)=v, value(r’)=v’

effects: value(r)=v’

• STRIPS cannot solve this problem at all


How to Fix?

• Several ways:

Do something other than state-space search• e.g., Chapters 5–8

Use forward or backward state-space search, with domain-specific knowledge to prune the search space

• Can solve both problems quite easily this way• Example: block stacking using forward search


Domain-Specific Knowledge• A blocks-world planning problem P = (O,s0,g) is solvable

if s0 and g satisfy some simple consistency conditions

• g should not mention any blocks not mentioned in s0

• a block cannot be on two other blocks at once

• etc.

– Can check these in time O(n log n)

• If P is solvable, can easily construct a solution of length O(2m), where m is the number of blocksMove all blocks to the table, then build up stacks from the

bottom

• Can do this in time O(n)

• With additional domain-specific knowledge can do even better …

Additional Domain-Specific Knowledge• A block x needs to be moved if any of the following is

true:s contains ontable(x) and g contains on(x,y)s contains on(x,y) and g contains ontable(x)s contains on(x,y) and g contains on(x,z) for some y≠zs contains on(x,y) and y needs to be moved

initial state goal

e

d

d

ba

c c

a

b


Axi

oms

Domain-Specific Algorithmloop

if there is a clear block x such that x needs to be moved and x can be moved to a place where it won’t need to be

movedthen move x to that place

else if there is a clear block x such that x needs to be movedthen move x to the table

else if the goal is satisfiedthen return the plan

else return failurerepeat

initial state goal

e

d

d

ba

c c

a

b


Easily Solves the Sussman Anomalyloop

if there is a clear block x such that x needs to be moved and x can be moved to a place where it won’t need to be

movedthen move x to that place

else if there is a clear block x such that x needs to be movedthen move x to the table

else if the goal is satisfiedthen return the plan

else return failurerepeat

initial state goal

ba

c

c

a

b

Properties

• The block-stacking algorithm:

Sound, complete, guaranteed to terminate

Runs in time O(n3)• Can be modified to run in time O(n)

Often finds optimal (shortest) solutionsBut sometimes only near-optimal (Exercise 4.22 in the

book)• PLAN LENGTH for the blocks world is NP-complete


An Actual Application: FEAR.

• F.E.A.R.: First Encounter Assault Recon• Here is a video clip• Behavior of opposing NPCs is modeled through STRIPS

operatorsAttackTake cover…

• Why?

AttackFromCover

Gotot AttackDrawt

Gotot

Flipt

Lightt

Lightt

Reloadt

Otherwise you would need too build an FSM for all possible behaviours in advance

http://www.youtube.com/watch?v=YnCTcZXo4zI&mode=related&search=

FSM vs AI Planning

Pla

nn

ing

Op

erat

ors

•PatrolPreconditions: No MonsterEffects: patrolled

•FightPreconditions: Monster in sightEffects: No Monster

Patrol Fight

Monster In Sight

No Monster

FSM:

A resulting plan:

Patrolpatrolled

Fight

No MonsterMonster in sightNeither is more powerful than the other one

But Planning Gives More Flexibility

• “Separates implementation from data” --- Orkin

reasoning knowledge

Pla

nn

ing

Op

erat

ors

•PatrolPreconditions: No MonsterEffects: patrolled

•FightPreconditions: Monster in sightEffects: No Monster…

Many potential plans:

PatrolFightPatrolFightPatrolFightPatrolFight

PatrolFight

…

If conditions in the state change making the current plan unfeasible: replan!

State-Space Planning Dr. Héctor Muñoz-Avila Sources: Ch. 3 Appendix A Slides from Dana Nau’s...

Documents

Transcript of State-Space Planning Dr. Héctor Muñoz-Avila Sources: Ch. 3 Appendix A Slides from Dana Nau’s...