Temporal Plan Execution: Dynamic Scheduling and

Simple Temporal Networks

1

Brian C. Williams

16.412J/6.834J

December 2nd, 2002

Model-based

Programming&

Execution

Task-Decompositio

nExecution

Reactive Task Expansion

ProjectiveTask Expansion

Goals

Dynamic Scheduling and Task Dispatch

TaskDispatch

Temporal Plan

CommandsObservations

Modes

Goals and Environment Constraints

TemporalNetworkSolver

TemporalPlanner

Model-based

Programming&

Execution

Task-Decompositio

nExecution

Reactive Task Expansion

ProjectiveTask Expansion

Goals

Dynamic Scheduling and Task Dispatch

TaskDispatch

Temporal Plan

CommandsObservations

Modes

Goals and Environment Constraints

TemporalNetworkSolver

TemporalPlanner

Outline

• Temporal Representation– Qualitative temporal relations– Metric constraints– Temporal Constraint Networks– Temporal Plans

• Temporal Reasoning forPlanning and Scheduling

• Flexible Execution throughDynamic Scheduling

Y

Qualitative Temporal Constraints(Allen 83)

• x before y

• x meets y

• x overlaps y

• x during y

• x starts y

• x finishes y

• x equals y

X Y

X Y

X Y

YX

YX

Y X

X

• y after x• y met-by x• y overlapped-by x• y contains x• y started-by x• y finished-by x• y equals x

Example: Deep Space One Remote Agent Experiment

Max_ThrustIdle Idle

Poke

Timer

Attitude

Accum

SEP Action

SEP_Segment

Th_Seg

contained_by

equals equalsmeets

meets

contained_by

Start_Up Start_UpShut_Down Shut_Down

Thr_Boundary

Thrust ThrustThrustThrustStandby Standby Standby

Th_Sega Th_Seg Th_SegIdle_Seg Idle_Seg

Accum_NO_Thr Accum_ThrAccum_Thr Accum_ThrThr_Boundary

contained_by

CP(Ips_Tvc) CP(Ips_Tvc) CP(Ips_Tvc)

contained_by

Th_Seg

Qualitative Temporal ConstraintsMaybe Expressed as Inequalities

(Vilain, Kautz 86)

• x before y X+ < Y-

• x meets y X+ = Y-

• x overlaps y (Y- < X+) & (X- < Y+) • x during y (Y- < X-) & (X+ < Y+) • x starts y (X- = Y-) & (X+ < Y+) • x finishes y (X- < Y-) & (X+ = Y+) • x equals y (X- = Y-) & (X+ = Y+)

Inequalities may be expressed as binary interval relations:X+ - Y- < [-inf, 0]

Metric Constraints

• Going to the store takes at least 10 minutes and at most 30 minutes.→ 10 < [T+(store) – T-(store)] < 30

• Bread should be eaten within a day of baking.→ 0 < [T+(baking) – T-(eating)] < 1 day

• Inequalities, X+ < Y- , may be expressed as binary interval relations:→ - inf < [X+ - Y-] < 0

Metric Time: Quantitative Temporal Constraint Networks

(Dechter, Meiri, Pearl 91)

• A set of time points Xi at which events occur.

• Unary constraints

(a0 < Xi < b0 ) or (a1 < Xi < b1 ) or . . .

• Binary constraints

(a0 < Xj - Xi < b0 ) or (a1 < Xj - Xi < b1 ) or . . .

TCSP Are Visualized UsingDirected Constraint Graphs

1 3

42

0[10,20]

[30,40][60,inf]

[10,20]

[20,30][40,50]

[60,70]

Simple Temporal Networks(Dechter, Meiri, Pearl 91)

Simple Temporal Networks:

• A set of time points Xi at which events occur.

• Unary constraints(a0 < Xi < b0 ) or (a1 < Xi < b1 ) or . . .

• Binary constraints (a0 < Xj - Xi < b0 ) or (a1 < Xj - Xi < b1 ) or . . .

Sufficient to represent:• most Allen relations • simple metric constraints

Sufficient to represent:• most Allen relations • simple metric constraints

Can’t represent:• Disjoint tokensCan’t represent:• Disjoint tokens

Simple Temporal Network

• Tij = (aij Xi - Xj bij)

1 3

42

0[10,20]

[30,40][60,inf]

[10,20]

[20,30][40,50]

[60,70]

A Completed Plan Forms an STNA Completed Plan Forms an STN

Outline

• Temporal Representation• Temporal Reasoning for

Planning and Scheduling– TCSP Queries– Induced Constraints– Plan Consistency– Static Scheduling

• Flexible Execution throughDynamic Scheduling

TCSP Queries(Dechter, Meiri, Pearl, AIJ91)

• Is the TCSP consistent?

• What are the feasible times for each Xi?

• What are the feasible durations between each Xi and Xj?

• What is a consistent set of times?

• What are the earliest possible times?

• What are the latest possible times?

TCSP Queries(Dechter, Meiri, Pearl, AIJ91)

• Is the TCSP consistent? Planning

• What are the feasible times for each Xi?

• What are the feasible durations between each Xi and Xj?

• What is a consistent set of times? Scheduling

• What are the earliest possible times? Scheduling

• What are the latest possible times?

To Query an STN Map to aDistance Graph Gd = < V,Ed >

70

1 3

42

020

50

-10

40

-30

20 -10

-40-60

1 3

42

0[10,20] [30,40]

[10,20]

[40,50]

[60,70]

Tij = (aij Xj - Xi bij)Xj - Xi bij

Xi - Xj - aij

Edge encodes an upper bound on distance to target from source.

Gd Induces Constraints

• Path constraint: i0 =i, i1 = . . ., ik = j

→ Conjoined path constraints result in the shortest path as bound:

where dij is the shortest path from i to j

X j Xi ai j 1 ,i jj1

k

X j Xi dij

0 1 2 3 4

0 0 20 50 30 70

1 -10 0 40 20 60

2 -40 -30 0 -10 30

3 -20 -10 20 0 50

4 -60 -50 -20 -40 0

d-graph

Shortest Paths of Gd

70

1 2

43

020

50

-10

40

-30

20 -10

-40-60

STN Minimum Network

0 1 2 3 4

0 [0] [10,20] [40,50] [20,30] [60,70]

1 [-20,-10] [0] [30,40] [10,20] [50,60]

2 [-50,-40] [-40,-30] [0] [-20,-10] [20,30]

3 [-30,-20] [-20,-10] [10,20] [0] [40,50]

4 [-70,-60] [-60,-50] [-30,-20] [-50,-40] [0]

0 1 2 3 4

0 0 20 50 30 70

1 -10 0 40 20 60

2 -40 -30 0 -10 30

3 -20 -10 20 0 50

4 -60 -50 -20 -40 0

d-graph STN minimum network

Conjoined Paths are Computed using All Pairs Shortest Path

(e.g., Floyd-Warshall’s algorithm )

1. for i := 1 to n do dii 0;

2. for i, j := 1 to n do dij aij;

3. for k := 1 to n do

4. for i, j := 1 to n do

5. dij min{dij, dik + dkj};

ik

j

Testing Plan Consistency

0 1 2 3 4

0 0 20 50 30 70

1 -10 0 40 20 60

2 -40 -30 0 -10 30

3 -20 -10 20 0 50

4 -60 -50 -20 -40 0

d-graph70

1 2

43

020

50

-10

40

-30

20 -10

-40-60

No negative cycles: -5 > TA – TA = 0

Scheduling:Feasible Values

0 1 2 3 4

0 0 20 50 30 70

1 -10 0 40 20 60

2 -40 -30 0 -10 30

3 -20 -10 20 0 50

4 -60 -50 -20 -40 0

d-graph

• X1 in [10, 20]

• X2 in [40, 50]

• X3 in [20, 30]

• X4 in [60, 70]

Latest Times

Earliest Times

Scheduling without Search: Solution by Decomposition

0 1 2 3 4

0 0 20 50 30 70

1 -10 0 40 20 60

2 -40 -30 0 -10 30

3 -20 -10 20 0 50

4 -60 -50 -20 -40 0

d-graph

• Select value for 1

15 [10,20]

Scheduling without Search: Solution by Decomposition

0 1 2 3 4

0 0 20 50 30 70

1 -10 0 40 20 60

2 -40 -30 0 -10 30

3 -20 -10 20 0 50

4 -60 -50 -20 -40 0

d-graph

• Select value for 1

15 [10,20]

Solution by Decomposition

0 1 2 3 4

0 0 20 50 30 70

1 -10 0 40 20 60

2 -40 -30 0 -10 30

3 -20 -10 20 0 50

4 -60 -50 -20 -40 0

d-graph

• Select value for 2, consistent with 1

45 [40,50], 15+[30,40]

• Select value for 1

15

Solution by Decomposition

0 1 2 3 4

0 0 20 50 30 70

1 -10 0 40 20 60

2 -40 -30 0 -10 30

3 -20 -10 20 0 50

4 -60 -50 -20 -40 0

d-graph

• Select value for 2, consistent with 1

45 [45,50]

• Select value for 1

15

Solution by Decomposition

0 1 2 3 4

0 0 20 50 30 70

1 -10 0 40 20 60

2 -40 -30 0 -10 30

3 -20 -10 20 0 50

4 -60 -50 -20 -40 0

d-graph

• Select value for 2, consistent with 1

45 [45,50]

• Select value for 1

15

Solution by Decomposition

0 1 2 3 4

0 0 20 50 30 70

1 -10 0 40 20 60

2 -40 -30 0 -10 30

3 -20 -10 20 0 50

4 -60 -50 -20 -40 0

d-graph

• Select value for 2, consistent with 1

45

• Select value for 1

15

• Select value for 3, consistent with 1 & 2

30 [20,30], 15+[10,20],45+[-20,-10]

Solution by Decomposition

0 1 2 3 4

0 0 20 50 30 70

1 -10 0 40 20 60

2 -40 -30 0 -10 30

3 -20 -10 20 0 50

4 -60 -50 -20 -40 0

d-graph

• Select value for 2, consistent with 1

45

• Select value for 1

15

• Select value for 3, consistent with 1 & 2

30 [25,30]

Solution by Decomposition

0 1 2 3 4

0 0 20 50 30 70

1 -10 0 40 20 60

2 -40 -30 0 -10 30

3 -20 -10 20 0 50

4 -60 -50 -20 -40 0

d-graph

• Select value for 2, consistent with 1

45

• Select value for 1

15

• Select value for 3, consistent with 1 & 2

30 [25,30]

Solution by Decomposition

0 1 2 3 4

0 0 20 50 30 70

1 -10 0 40 20 60

2 -40 -30 0 -10 30

3 -20 -10 20 0 50

4 -60 -50 -20 -40 0

d-graph • Select value for 4, consistent with 1,2 & 3O(N2)

• Select value for 2, consistent with 1

45

• Select value for 1

15

• Select value for 3, consistent with 1 & 2

30

Outline

• Temporal Representation

• Temporal Reasoning forPlanning and Scheduling

• Flexible Execution throughDynamic Scheduling

Model-based

Programming&

Execution

Task-Decompositio

nExecution

Reactive Task Expansion

ProjectiveTask Expansion

Goals

Dynamic Scheduling and Task Dispatch

TaskDispatch

Temporal Plan

CommandsObservations

Modes

Goals and Environment Constraints

TemporalNetworkSolver

TemporalPlanner

Executing Flexible Temporal Plans [Muscettola, Morris, Pell et al.]

EXECUTIVE

CONTROLLED SYSTEM

Handling delays and fluctuations in task duration:• Least commitment temporal plans leave room to adapt

• flexible execution adapts through dynamic scheduling• Assigns time to event when executed.

Issues in Flexible Execution

1. How do we minimize execution latency?

2. How do we schedule at execution time?

Time Propagation Can Be Costly

EXECUTIVE

CONTROLLED SYSTEM

EXECUTIVE

CONTROLLED SYSTEM

Time Propagation Can Be Costly

EXECUTIVE

CONTROLLED SYSTEM

Time Propagation Can Be Costly

EXECUTIVE

CONTROLLED SYSTEM

Time Propagation Can Be Costly

EXECUTIVE

CONTROLLED SYSTEM

Time Propagation Can Be Costly

EXECUTIVE

CONTROLLED SYSTEM

Time Propagation Can Be Costly

Issues in Flexible Execution

1. How do we minimize execution latency? Propagate through a small set of

neighboring constraints.

2. How do we schedule at execution time?

EXECUTIVE

CONTROLLED SYSTEM

Compile to Efficient Network

EXECUTIVE

CONTROLLED SYSTEM

Compile to Efficient Network

EXECUTIVE

CONTROLLED SYSTEM

Compile to Efficient Network

Issues in Flexible Execution

1. How do we minimize execution latency? Propagate through a small set of

neighboring constraints.

2. How do we schedule at execution time?

Issues in Flexible Execution

1. How do we minimize execution latency? Propagate through a small set of

neighboring constraints.

2. How do we schedule at execution time? Through decomposition?

Dynamic Scheduling by Decomposition

A

C

B

D

[0,10] [-2,2]

[-1,1][0,10]

Simple Example

A

C

B

D0

0

10

10 2

-2

-1

1

Equivalent Distance Graph Representation

A

C

B

D0

0

10

10 2

-2

-1

1

Equivalent Distance Graph Representation

A

C

B

D0

0

10

10 2

-2

-1

1

11

-2

1-1

Equivalent All Pairs Shortest Paths (APSP) Representation

• Compute APSP graph• Decomposition enables assignment without search

Dynamic Scheduling by Decomposition

Assignment by Decomposition

A

C

B

D0

0

10

10 2

-2

-1

1

11

-2

1-1t = 0

<0,10>

<0,10>

<2,11>

A

C

B

D0

0

10

10 2

-2

-1

1

11

-2

1-1

t = 3

<0,10>

<0,0> <2,11>

Assignment by Decomposition

A

C

B

D0

0

10

10 2

-2

-1

1

11

-2

1-1

<2,2>

<0,0> <4,4>

But C now has to be executed at t =2, which is already in the past!

Solution:

• Assignments must monotonically increase in value.

Execute first, all APSP neighbors with negative delays.

Assignment by Decomposition

t = 3

Dispatching Execution Controller

Execute an event when enabled and active

• Enabled - APSP Predecessors are completed – Predecessor – a destination of a negative edge that

starts at event.

• Active - Current time within bound of task.

Initially:• E = Time points w/o predecessors• S = {}Repeat:1. Wait until current_time has advanced st

a. Some TP in E is activeb. All time points in E are still enabled.

2. Set TP’s execution time to current_time.3. Add TP to S.4. Propagate time of execution to TP’s APSP immediate neighbors.5. Add to A, all immediate neighbors that became enabled.

a. TPx enabled if all negative edges starting at TPx have their destination in S.

Dispatching Execution Controller

EXECUTIVE

Propagation is Focused

• Propagate forward along positive edges to tighten upper bounds.

– forward prop along negative edges is useless.

• Propagate backward along negative edges to tighten lower bounds.

–Backward prop along positive edges useless.

Propagation Example

A

C

B

D0

-1

10

92

-2

-1

1

-2

-11

11

<0,0>

S = {A}

Propagation Example

A

C

B

D0

-1

10

92

-2

-1

1

-2

-11

11

<0,0>

<1,10>

S = {A}

Propagation Example

A

C

B

D0

-1

10

92

-2

-1

1

-2

-11

11

<0,0>

<1,10>

<0,9>

S = {A}

Propagation Example

A

C

B

D0

-1

10

92

-2

-1

1

-2

-11

11

<0,0>

<1,10>

<0,9>

<2,11>

S = {A}

Propagation Example

A

C

B

D0

-1

10

92

-2

-1

1

-2

-11

11

<0,0>

<1,10>

<0,9>

<2,11>

S = {A}

E = { C }

A

C

B

D0

0

10

10 2

-2

-1

1

11

-2

1-1

t = 3

<0,10>

<0,0> <2,11>

Execution time is O(n)

• worst case

• best case

Filtering:

• some edges are redundant

• remove redundant edges

Reducing Execution Latency

C

A B

150

-10010080

-50 -20

• Edge Dominance– Eliminate edge that is redundant due to the

triangle inequality AB + BC = AC

C

A B

150

-10010080

-50 -20

• Edge Dominance– Eliminate edge that is redundant due to the

triangle inequality AB + BC = AC

C

A B

150

-10080

-50 -20

• Edge Dominance– Eliminate edge that is redundant due to the

triangle inequality AB + BC = AC

C

A B

150

-10080

-50

• Edge Dominance– Eliminate edge that is redundant due to the

triangle inequality AB + BC = AC

An Example of Edge Filtering

• Start off with the APSP network

A

C

BD

0

-110

92

-2

-11

-2

-11

11

An Example of Edge Filtering

• Start at A-B-C triangle

A

C

BD

0

-110

92

-2

-11

-2

-11

11

An Example of Edge Filtering

• Look at B-D-C triangle

A

C

BD

0

-1

2

-2

-11

-2

-1

11

9

1

An Example of Edge Filtering

• Look at B-D-C triangle

A

C

BD

0

-1

2

-2

-11

-1

11

9

1

An Example of Edge Filtering

• Look at D-A-B triangle

A

C

BD

0

-1

2

-11

-1

11

9

1

An Example of Edge Filtering

• Look at D-A-C triangle

A

C

BD

0

2

-11

-1

11

9

1

An Example of Edge Filtering

• Look at B-C-D triangle

A

C

BD

0

2

-11

-1

9

1

An Example of Edge Filtering

• Look at B-C-D triangle

A

C

BD

0

-11

-1

9

1

An Example of Edge Filtering

• Resulting network has less edges than the original

A C B D0 -1

1

-1

9 1

A

C

B

D0

0

10

10 2

-2

-1

1

Avoiding Intermediate Graph Explosion

Problem:

• APSP consumes O(n2) space.

Solution:

• Interleave process of APSP construction with edge elimination– Never have to build whole APSP graph

Model-based

Programming&

Execution

Task-Decompositio

nExecution

Reactive Task Expansion

ProjectiveTask Expansion

Goals

Dynamic Scheduling and Task Dispatch

TaskDispatch

Temporal Plan

CommandsObservations

Modes

Goals and Environment Constraints

TemporalNetworkSolver

TemporalPlanner