Siddharth Srivastava, Neil Immerman, Shlomo Zilberstein University of Massachusetts Amherst.
Challenges in Finding Generalized Plans · 2020-06-20 · Siddharth Srivastava Joint work with Neil...
Transcript of Challenges in Finding Generalized Plans · 2020-06-20 · Siddharth Srivastava Joint work with Neil...
Universal Challenges Our Framework Algorithm Results
Challenges in Finding Generalized Plans
Siddharth SrivastavaJoint work with Neil Immerman and Shlomo Zilberstein
University of Massachusetts, Amherst
ICAPS 2009 Workshop onGeneralized Planning: Macros, Loops, Domain Control
September 20th, 2009
Siddharth Srivastava Challenges in Finding Generalized Plans
Universal Challenges Our Framework Algorithm Results
Generalized Planning
Plans or planning structures that “work in many situations”
- Triangle Tables [Fikes et al., 1972]
- Case Based Planning [Hammond, 1986]
- Explanation Based Planning[Minton et al., 1989, Shavlik, 1990]
- Contingent Planning
- Learning domain specific planners from examples[Winner and Veloso, 2003]; Planning with loops[Levesque, 2005];
Siddharth Srivastava Challenges in Finding Generalized Plans
Universal Challenges Our Framework Algorithm Results
Overview
Universal Challenges
Our Framework
Generalized Planning with Sensing Actions
Results
Siddharth Srivastava Challenges in Finding Generalized Plans
Universal Challenges Our Framework Algorithm Results
Examples of Generalized Plans
Classical PlansmvToTable(b3), mvToTable(b2), mvToTable(b1)
More General:“Unstack”:while(∃b: topmost(b)∧¬ onTable(b)) {mvToTable(b)}
Still More General:FF, SATPLAN, SGPLAN, . . .
Common fundamental problem (Generalized Planning):Find a function G (a generalized plan):
G : Problem instance→ sequence of actions
What makes us prefer one over another?
Siddharth Srivastava Challenges in Finding Generalized Plans
Universal Challenges Our Framework Algorithm Results
Examples of Generalized Plans
Classical PlansmvToTable(b3), mvToTable(b2), mvToTable(b1)
More General:“Unstack”:while(∃b: topmost(b)∧¬ onTable(b)) {mvToTable(b)}
Still More General:FF, SATPLAN, SGPLAN, . . .
Common fundamental problem (Generalized Planning):Find a function G (a generalized plan):
G : Problem instance→ sequence of actions
What makes us prefer one over another?
Siddharth Srivastava Challenges in Finding Generalized Plans
Universal Challenges Our Framework Algorithm Results
Examples of Generalized Plans
Classical PlansmvToTable(b3), mvToTable(b2), mvToTable(b1)
More General:“Unstack”:while(∃b: topmost(b)∧¬ onTable(b)) {mvToTable(b)}
Still More General:FF, SATPLAN, SGPLAN, . . .
Common fundamental problem (Generalized Planning):Find a function G (a generalized plan):
G : Problem instance→ sequence of actions
What makes us prefer one over another?
Siddharth Srivastava Challenges in Finding Generalized Plans
Universal Challenges Our Framework Algorithm Results
Examples of Generalized Plans
Classical PlansmvToTable(b3), mvToTable(b2), mvToTable(b1)
More General:“Unstack”:while(∃b: topmost(b)∧¬ onTable(b)) {mvToTable(b)}
Still More General:FF, SATPLAN, SGPLAN, . . .
Common fundamental problem (Generalized Planning):Find a function G (a generalized plan):
G : Problem instance→ sequence of actions
What makes us prefer one over another?Siddharth Srivastava Challenges in Finding Generalized Plans
Universal Challenges Our Framework Algorithm Results
Challenges for Any Approach to Generalized Planning
1 Applicability Test2 Cost of Instantiation3 Domain Coverage4 Quality of instantiated plans5 Complexity of creating generalized plans
Siddharth Srivastava Challenges in Finding Generalized Plans
Universal Challenges Our Framework Algorithm Results
Applicability Test
G : Problem instanceplan instantiation−−−−−−−−−−→ a1, . . . an
One approach: simulated execution.Cost of instantiation will be wasted if G cannot solve it.
NavigateGrids /*Start at bottom left*/
repeatwhile ¬rightmost do
mvR()endmvU()while ¬leftmost do
mvL()endmvU()
until atgoal
G
S
Siddharth Srivastava Challenges in Finding Generalized Plans
Universal Challenges Our Framework Algorithm Results
Applicability Test
G : Problem instanceplan instantiation−−−−−−−−−−→ a1, . . . an
One approach: simulated execution.Cost of instantiation will be wasted if G cannot solve it.
NavigateGrids /*Start at bottom left*/
repeatwhile ¬rightmost do
mvR()endmvU()while ¬leftmost do
mvL()endmvU()
until atgoal
G
S
Siddharth Srivastava Challenges in Finding Generalized Plans
Universal Challenges Our Framework Algorithm Results
Applicability Test (ctd.)
Historically not common: not required for very general(FF) or very simple plans (a1, . . . an).
Computed generalized plans typically have a limitedapplicability.More of a problem with compact representations (loops).
Simulated execution may not even terminate!!
Ideal applicability test: linear in the size of the problem
Siddharth Srivastava Challenges in Finding Generalized Plans
Universal Challenges Our Framework Algorithm Results
Cost of Plan Instantiation
G : Problem instanceplan instantiation−−−−−−−−−−−→ a1, . . . an
Makes generalized plans like “unstack” (O(n)) moredesirable than classical planners (O(exp(n))).
In hindsight: low COI = one of the main motivationsbehind this field.
Siddharth Srivastava Challenges in Finding Generalized Plans
Universal Challenges Our Framework Algorithm Results
Domain Coverage
The set/fraction of solvable problems solved by a generalizedplan.
Historically one of the most measured attributes.
Trade-offs with cost of instantiation.
Siddharth Srivastava Challenges in Finding Generalized Plans
Universal Challenges Our Framework Algorithm Results
Quality of Instantiated Plans
The computational cost (makespan/number of actions/timeetc.) of executing the instantiated plan.
Satisficing, optimal generalized plans.
Trade-offs with domain coverage and cost of instantiation.
Siddharth Srivastava Challenges in Finding Generalized Plans
Universal Challenges Our Framework Algorithm Results
Complexity of Creating Generalized Plans
Serious problems with applicability test, instantiation:
Loop termination, progress
Siddharth Srivastava Challenges in Finding Generalized Plans
Universal Challenges Our Framework Algorithm Results
Complexity of Creating Generalized Plans
glass paper
Test Type
Pick Next Pick Next
collect In PaperContainer collect In GlassContainercollect In PaperContainer
Serious problems with applicability test, instantiation:Loop termination, progress
Siddharth Srivastava Challenges in Finding Generalized Plans
Universal Challenges Our Framework Algorithm Results
Our Objective
Compute algorithm-like “generalized” plans.Low cost of instantiationEfficient applicability testsEfficient generation of generalized plans
Need to determine progress and termination.
Siddharth Srivastava Challenges in Finding Generalized Plans
Universal Challenges Our Framework Algorithm Results
Concrete States as Logical Structures
V = {object1, bin1, isGlass1, isPaper1, in2, empty1, collected1, forGlass1, forPaper1}
[[object(2)]] =1
[[isGlass(2)]] = 1
[[bin(1)]] = 1
[[in(2,1)]] = 1
in
bin
isGlass
object
1
2
S1
S2
in
bin
isPaper
object
1
2
[[object(2)]] =1
[[isPaper(2)]] = 1
[[bin(1)]] = 1
[[in(2,1)]] = 1
Siddharth Srivastava Challenges in Finding Generalized Plans
Universal Challenges Our Framework Algorithm Results
Example: The Collect Action
Collect(o,c)
object(o) ∧ container(c) ∧ (isGlass(o)↔ forGlass(c)) ∧∃b(bin(b) ∧ in(o, b) ∧ robotAt(b))
in′(u, v) := (in(u, v) ∧ u 6= o) ∨(¬in(u, v) ∧ u = o ∧ v = c)
empty′(u) := (empty(u) ∧ u 6= c) ∨ in(o, u)collected′(u) := collected(u) ∨ o = u
c container, forPaper, empty
objectin
bin
isPaper
b
o
b c container, forPaper
object, collected
isPaper
o
in
bin, empty
Collect(o,c)
Siddharth Srivastava Challenges in Finding Generalized Plans
Universal Challenges Our Framework Algorithm Results
Abstraction Using 3-Valued Logic
in
bin
isGlass
object
1
2
in
bin
isPaper isGlass
object
1
2
= 1/2
in
bin
isPaper
object
1
2
Use 3−Valued logic to abstract as:
TVLA: [Sagiv et al., 2002]
Siddharth Srivastava Challenges in Finding Generalized Plans
Universal Challenges Our Framework Algorithm Results
Abstraction Using 3-Valued Logic
in
bin
isGlass
object
1
2
in
bin
isPaper isGlass
object
1
2
in
bin
isPaper
object
1
2
Focus and coerce w.r.t {isPaper(x)}
Objects are either paper or glass
Integrity Constraint:
Implementation of “sensing” actions
Siddharth Srivastava Challenges in Finding Generalized Plans
Universal Challenges Our Framework Algorithm Results
Abstraction Using 3-Valued Logic
U
V
in
bin
isPaper isGlass
object
1
2
in
bin
isPaper isGlass
object
3
4
in
bin
object
isGlassisPaper
in
bin
isPaper isGlass
object2n
2n−1
ConcretizationCanonical Abstraction
Integrity Constraint:
Each bin has a unique object
= "summary" element
Siddharth Srivastava Challenges in Finding Generalized Plans
Universal Challenges Our Framework Algorithm Results
Abstraction Using 3-Valued Logic: Summary
TVLA [Sagiv et al., 2002]: Three Valued Logic Analysis
Abstraction predicates: unary predicates.
Element’s role = set of abstraction predicates satisfied
Collapse elements of a role into summary elements.
Use integrity constraints to retreive concrete states.
Siddharth Srivastava Challenges in Finding Generalized Plans
Universal Challenges Our Framework Algorithm Results
Action Application on Belief States
Make structures precise by creating possible cases: focus(automatic)
Apply action
Rolei
φ
S0
Rolei Role
iRolei
Rolei
φφ
S SS1 2 3
Focus
Siddharth Srivastava Challenges in Finding Generalized Plans
Universal Challenges Our Framework Algorithm Results
Action Application on Belief States
Make structures precise by creating possible cases: focus(automatic)
Apply action
S2
1S2
1
3S
3S3
S3
2S2
a1S
1S1
2S1
Focus Update Blur
Siddharth Srivastava Challenges in Finding Generalized Plans
Universal Challenges Our Framework Algorithm Results
Action Branches and Plan Preconditions
Branches solve only some members of abstract structuresMay be classifiable, e.g #R{S} > 1
Extended-LL domains: all branches are classifiable
Subtract role-count changes to obtain preconditions atstart.
Generalize to simple loops, nested loops due to shortcutsand sensing actions.
Siddharth Srivastava Challenges in Finding Generalized Plans
Universal Challenges Our Framework Algorithm Results
Plan Generalization
Use abstract structures to recognize loop invariants in example concrete plans.
Example Execution
goToNextBin()S 0#
S 1# senseType() S 2
# preProc−Paper() S 3# collectPaper()
S 4#
2 objects of each type collected;
2 bins remaining
Find Loops
S 0goToNextBin() senseType()
S 2preProc−Paper()
S 3S 1
goToNextBin() S 6senseType()S 5
S 7
S 8
collectPaper()
Developed for completely observable settings [Srivastava et al., 2008]
Siddharth Srivastava Challenges in Finding Generalized Plans
Universal Challenges Our Framework Algorithm Results
Merging Generalized Plans
Plan for Unhandled StructurepreProc−Glass() collectGlass() goToNextBin()
S 7#
S 9#
S 10#
S 11#
Generalize and Merge
S 0goToNextBin() senseType()
S 2preProc−Paper()
S 3S 1S 5 goToNextBin()
S 8
S 6
S 7 S 9preProc−Glass() collectGlass()
S 10S 12
collectPaper()
senseType()
senseType()
goToNextBin()
A single plan may not explore all possibilities.
Construct problem instances from unsolved belief states.
Solve them using classical planners.
Siddharth Srivastava Challenges in Finding Generalized Plans
Universal Challenges Our Framework Algorithm Results
Example Resultsp0 = ‖{paper, collected}‖; pc0 = ‖{empty,container,forPaper}‖;g0, gc0 : similar for glass; b0 = ‖{bin}‖
Loop 1goToNextBin()
senseType()
apply−PaperPreProc(obj)
senseType()
apply−PaperPreProc(obj)
collect−PaperCont(obj)
collect−Paper−Cont(obj)
paper
paper
Precons: pc0 = l1; b0 = l1
Solves 1 out of 2n
Loops 1 & 2goToNextBin()
senseType()
apply−PaperPreProc(obj)
apply−GlassPreProc(obj)
senseType()
apply−PaperPreProc(obj)
collect−PaperCont(obj)
goToNextBin()
collect−Glass−Cont(obj)
collect−Paper−Cont(obj)
paper
2
Precons:pc0 = l1; gc0 = l2; b0 = l1 + l2
2n−1 + 1 out of every 2n
Siddharth Srivastava Challenges in Finding Generalized Plans
Universal Challenges Our Framework Algorithm Results
Merging Generalized Plans: Algorithm
Input: Existing plan Π, eg trace tracei
Output: Extension of Πif Π = ∅ then1
Π← tracei2return Π3
endmpΠ, mpt ← findMergePoint(Π, tracei, bpΠ, bpt)4repeat5
if mpΠ found then6bpΠ, bpt ← findBranchPoint(Π, tracei, mpΠ, mpt)7
endif bpΠ found then8
mpΠ, mpt ← findMergePoint(Π, tracei, bpΠ, bpt)9addEdges(Π, tracei, bpt, mpt, mpΠ, bpΠ)10
enduntil new bpΠ or mpΠ not foundreturn Π11
Algorithm 1: ARANDA-Merge
Siddharth Srivastava Challenges in Finding Generalized Plans
Universal Challenges Our Framework Algorithm Results
Addressing the Challenges
Cost of testing applicability: independent of the size of theproblem.
Cost of instantiation: linear, or better with role-lists
Domain Coverage can increase exponentially with newexamples
Complexity of creating generalized plan: O(s · n2eg) to find
loops, O(s · neg) for preconditions.
Siddharth Srivastava Challenges in Finding Generalized Plans
Universal Challenges Our Framework Algorithm Results
Conclusions
Clear formal framework for algorithmic plans, avoidingintractability of automated program synthesis.
Approach for learning generalized conditional plans withnested loops by composition of simple linear plans.
Efficient methods for computation of measures of progressand preconditions.
Siddharth Srivastava Challenges in Finding Generalized Plans
Universal Challenges Our Framework Algorithm Results
Transport Domain
Siddharth Srivastava Challenges in Finding Generalized Plans
Universal Challenges Our Framework Algorithm Results
Transport Domain: Results
m0 = ‖{monitor, atD2}‖; s0 = ‖{server, atD1}‖
Loop 1
load(server, T2)
mv(T2, D3)
load(server, T1)
unload(T2) mv(T2, L)
mv(T2, D2)
mv(T1, L) unload(T1)
mv(T1,D1)
load(LCD, T2)
mv(T2,L)
server present
Precons: mo = l1; s0 = l1
Loops 1 & 2
load(server, T2)
mv(T2, D3)
load(server, T1)
unload(T2) mv(T2, L)
mv(T2, D2)
mv(T1, L) unload(T1)
mv(T1,D1)
load(LCD, T2)
mv(T2,L)load(server,T2)mv(T2, D1)
mv(T2,L)
server lost
Precons:m0 = l1; so = l1 + k1
Siddharth Srivastava Challenges in Finding Generalized Plans
Universal Challenges Our Framework Algorithm Results
Example Results: Domain Coverage
0
0.2
0.4
0.6
0.8
1
0 5 10 15 20 25 30 35
Cov
erag
e (D
)
Max Number of Bins (n)
D(n>=8) = 1
D(n>=8) = 0.5
D(n>=8) = 0.25
CFF-soln7Gen(Eg 1)
Gen(Eg 1+2)Gen(Eg 1..3)Gen(Eg 1..4)
Dπ(n) = |Sπ(n)|/|T (n)|
Siddharth Srivastava Challenges in Finding Generalized Plans
Universal Challenges Our Framework Algorithm Results
Related Work
Plans with Loops[Winner and Veloso, 2007]: no preconditions or sensingactions, but use partial ordering.[Levesque, 2005]: single planning parameter, limitedpreconditions.[Cimatti et al., 2003]: “hard” loops.
Planning with unknown quantities:[Milch et al., 2005]: action operators not provided.
Siddharth Srivastava Challenges in Finding Generalized Plans
Universal Challenges Our Framework Algorithm Results
References I
Cimatti, A., Pistore, M., Roveri, M., and Traverso, P. (2003).Weak, strong, and strong cyclic planning via symbolic modelchecking.Artif. Intell., 147(1-2):35–84.
Fikes, R., Hart, P., and Nilsson, N. (1972).Learning and Executing Generalized Robot Plans.Technical report, AI Center, SRI International.
Hammond, K. (1986).CHEF: A Model of Case-Based Planning.In Proceedings of AAAI-86, pages 267–271.
Levesque, H. J. (2005).Planning with loops.In Proc. of IJCAI, pages 509–515.
Siddharth Srivastava Challenges in Finding Generalized Plans
Universal Challenges Our Framework Algorithm Results
References II
Milch, B., Marthi, B., Russell, S. J., Sontag, D., Ong, D. L.,and Kolobov, A. (2005).Blog: Probabilistic models with unknown objects.In Proc. of IJCAI, pages 1352–1359.
Minton, S., Carbonell, J. G., Knoblock, C. A., Kuokka, D. R.,and Etzioni, O. (1989).Explanation-based Learning: A Problem SolvingPerspective.Artif. Intell., 40(1-3).
Sagiv, M., Reps, T., and Wilhelm, R. (2002).Parametric shape analysis via 3-valued logic.ACM Transactions on Programming Languages and Systems,24(3):217–298.
Siddharth Srivastava Challenges in Finding Generalized Plans
Universal Challenges Our Framework Algorithm Results
References III
Shavlik, J. W. (1990).Acquiring recursive and iterative concepts withexplanation-based learning.Machine Learning, 5:39–40.
Srivastava, S., Immerman, N., and Zilberstein, S. (2008).Learning generalized plans using abstract counting.In Proc. of AAAI, pages 991–997.
Winner, E. and Veloso, M. (2003).Distill: Learning domain-specific planners by example.In Proc. of ICML, pages 800–807.
Siddharth Srivastava Challenges in Finding Generalized Plans
Universal Challenges Our Framework Algorithm Results
References IV
Winner, E. and Veloso, M. (2007).LoopDISTILL: Learning domain-specific planners fromexample plans.In Workshop on AI Planning and Learning, ICAPS.
Siddharth Srivastava Challenges in Finding Generalized Plans