Speciﬁcation of Dynamic Strategy Switchingali.cmi.ac.in/icla2009/slides/jan07/lasi/1500.pdf ·...

Specification of Dynamic Strategy Switching

Soumya PaulJoint work with R. Ramanujam and S. Simon

The Institute of Mathematical SciencesTaramani, Chennai - 600 113

January 8, 2009

Soumya PaulJoint work with R. Ramanujam and S. Simon (The Institute of Mathematical Sciences Taramani,Specification of Dynamic Strategy Switching January 8, 2009 1 / 64

Motivation


Motivation

• Should I bowl a short pitch delivery?


Motivation


• Should I bowl a slower ball?


Motivation



• Should I bowl to the off or on his legs?


Motivation



• Should I bowl to the off or on his legs?

• If I bowl a wrong ’un, will it be too predictable?


Motivation[2]


Motivation[2]

• Should I attack or defend?


Motivation[2]


• If he bowls to my legs, should I pelt him for a boundary?


Motivation[2]



• Or should I just score a single so that he doesn’t change his line?


Motivation[2]



• Or should I just score a single so that he doesn’t change his line?

• If I score too many runs, will he be taken off the attack?


Motivation[3]

• (goodlength-legs followed by short-off . . .) or (short-legs followed bygoodlength-off . . .) or . . .. In addition bowl a slower delivery now andagain.


Motivation[3]

• (goodlength-legs followed by short-off . . .) or (short-legs followed bygoodlength-off . . .) or . . .. In addition bowl a slower delivery now andagain.

• (goodlength-defend, short-single, . . .) or (goodlength-single,short-defend, . . .) or . . .. Hit a boundary now and again.


Motivation[4]

• Strategies are structured.


Motivation[4]


• Complex strategies are built from simpler strategies.


Motivation[4]


• Complex strategies are built from simpler strategies.

• A player doesn’t decide on an exact strategy beforehand but maychange her strategy dynamically as the game progresses depending onthe outcome so far.


Formalising

• N = 1, 2, . . . , n is the set of players.


Formalising


• For each i ∈ N, Ai is a finite set of actions. A = A1 × . . .× An arethe action tuples.


Formalising



• Arena G = (W ,→,w0).


Formalising



• Arena G = (W ,→,w0).

• At any position w each player i chooses an action ai ∈ Ai .


Formalising



• Arena G = (W ,→,w0).

• At any position w each player i chooses an action ai ∈ Ai .Thisdefines an edge in the arena and the play moves along this edge to anew position.


Formalising



• Arena G = (W ,→,w0).


• Thus a play is just a sequence ρ ∈ Aω.


Formalising



• Arena G = (W ,→,w0).


• Thus a play is just a sequence ρ ∈ Aω.

• The tree unfolding of G is called TG .


Example

N = 1, 2,A1 = a, b,A2 = c , d,G :

w0

(b,c),(b,d)

''(a,c),(a,d)

((w1

(b,c),(b,d)

gg (a,c),(a,d)vv


Example[2]

Let t1 = (a, c), t2 = (a, d), t3 = (b, c), t4 = (b, d). The tree unfolding TG

of G is:

ǫ

qqddddddddddddddddddddddd

vvnnnnnn

((PPPPPP

--ZZZZZZZZZZZZZZZZZZZZZZZ

t1wwnnnnnn

~~~~ @@

t2wwnnnnnn

~~~~ @@

t3~~~~ @

@

''PPPPPP t4~~~~ @

@

''PPPPPP

t1 t2 t3 t4 t1 t2 t3 t4 t1 t2 t3 t4 t1 t2 t3 t4

......

......


Strategies

• A strategy for player i is a partial function:

σ : TG Ai

from the nodes of the tree unfolding TG of the arena G (histories) toher action set.


Strategies


σ : TG Ai


• If σ is not defined for some history, she may play any action there.


Strategies


σ : TG Ai



• Σi denotes the set of all strategies of player i .


Strategies


σ : TG Ai



• Σi denotes the set of all strategies of player i .

• A strategy σ may be viewed as a subtree T σ

G of TG .


Example

Let σ be the strategy of player 1 which is undefined at the empty historybut prescribes her to play the action a for all subsequent histories. ThenT σ

G looks like:


Example

Let σ be the strategy of player 1 which is undefined at the empty historybut prescribes her to play the action a for all subsequent histories. ThenT σ

G looks like:

ǫ

rrfffffffffffffffffff

xxxx

##FFF

F

,,XXXXXXXXXXXXXXXXXXX

(a, c)

yytttt

(a, d)

yytttt

(b, c)

%%JJJ

J(b, d)

%%JJJ

J

(a, c) (a, d) (a, c) (a, d) (a, c) (a, d) (a, c) (a, d)

......

......


Strategy Specifications

• Players change/compose/form strategies based on certain observableproperties of the game.




• P is a countable set of propositions that talk about the observables inthe game.





• V : W → 2P is a valuation function.





• V : W → 2P is a valuation function.

• V may be lifted to TG , V : TG → 2P in the usual way.


Strategy Specifications[2]

An observable property of the game may be of the form:




Ψ ::= p ∈ P




Ψ ::= p ∈ P | ψ1 ∨ ψ2




Ψ ::= p ∈ P | ψ1 ∨ ψ2 | ¬ψ




Ψ ::= p ∈ P | ψ1 ∨ ψ2 | ¬ψ | ⊖ ψ




Ψ ::= p ∈ P | ψ1 ∨ ψ2 | ¬ψ | ⊖ ψ | ψ1Sψ2




Ψ ::= p ∈ P | ψ1 ∨ ψ2 | ¬ψ | ⊖ ψ | ψ1Sψ2

The truth of a formula at a node t = a1 . . . ak of the game tree is definedas:




Ψ ::= p ∈ P | ψ1 ∨ ψ2 | ¬ψ | ⊖ ψ | ψ1Sψ2


• TG , t |= p iff p ∈ V (t).




Ψ ::= p ∈ P | ψ1 ∨ ψ2 | ¬ψ | ⊖ ψ | ψ1Sψ2


• TG , t |= p iff p ∈ V (t).

• TG , t |= ψ1 ∨ ψ2 iff TG |= ψ1 or TG |= ψ2.




Ψ ::= p ∈ P | ψ1 ∨ ψ2 | ¬ψ | ⊖ ψ | ψ1Sψ2


• TG , t |= p iff p ∈ V (t).


• TG , t |= ¬ψ iff TG , t 6|= ψ.




Ψ ::= p ∈ P | ψ1 ∨ ψ2 | ¬ψ | ⊖ ψ | ψ1Sψ2


• TG , t |= p iff p ∈ V (t).


• TG , t |= ¬ψ iff TG , t 6|= ψ.

• TG , t |= ⊖ψ iff k > 0 and TG , tk−1 |= ψ.



A strategy of player i can be of the form:




Πi ::= σ ∈ Σi




Πi ::= σ ∈ Σi | π1 ∪ π2




Πi ::= σ ∈ Σi | π1 ∪ π2 | π1 ∩ π2




Πi ::= σ ∈ Σi | π1 ∪ π2 | π1 ∩ π2 | π1aπ2




Πi ::= σ ∈ Σi | π1 ∪ π2 | π1 ∩ π2 | π1aπ2 | (π1 + π2)




Πi ::= σ ∈ Σi | π1 ∪ π2 | π1 ∩ π2 | π1aπ2 | (π1 + π2) | ψ?π

where ψ ∈ Ψ.





where ψ ∈ Ψ.Intuitively:






• π1 ∪ π2 means that the player plays according to the strategy π1 orthe strategy π2.






• π1 ∪ π2 means that the player plays according to the strategy π1 orthe strategy π2.

• π1 ∩ π2 means that if at a history t ∈ TG , π1 is defined then theplayer plays according to π1; else if π2 is defined at t then the playerplays according to π2.



• π1aπ2 means that the player plays according to the strategy π1 and

then after some history, switches to playing according to π2. Theposition at which she makes the switch is not fixed in advance.





• (π1 + π2) says that at every point, the player can choose to followeither π1 or π2.






• ψ?π says at every history, the player tests if the property ψ holds ofthat history. If it does then she plays according to π.






• ψ?π says at every history, the player tests if the property ψ holds ofthat history. If it does then she plays according to π.

For a specification π for player i , let [[π]]G ⊆ Σi be the strategies (partialfunctions), it denotes.


Examples

• Σbowler = σshort , σgood , σoutside−off , σlegsP = p(short,single), p(short,boundary ), . . . , p(legs,sixer), . . .

¬3- (p(good,sixer) ∧ p(legs,sixer))?(σshort + σgood + σoutside−off + σlegs) ∪3- (p(good,sixer ) ∧ p(legs,sixer))?(σshort + σoutside−off )


Examples

• Σbowler = σshort , σgood , σoutside−off , σlegsP = p(short,single), p(short,boundary ), . . . , p(legs,sixer), . . .

¬3- (p(good,sixer) ∧ p(legs,sixer))?(σshort + σgood + σoutside−off + σlegs) ∪3- (p(good,sixer ) ∧ p(legs,sixer))?(σshort + σoutside−off )

• Σbowler = σ5, σ2, . . .

σ5aσ2


Bounded Memory Strategies

A strategy σ is said to be bounded memory if there exists:




• A finite set M, the memory of the strategy.





• mI ∈ M, the initial memory.






• A function δ : A × M → M, the memory update.







• A function g : A × M → A, the action update.







• A function g : A × M → A, the action update.

such that when a1 . . . ak−1 is a play and the sequence m0,m1, . . . ,mk isdetermined by m0 = mI and mi+1 = δ(ai−1,mi ) thenσ(a1 . . . ak−1) = g(ak−1,mk).


Finite State Transducer

A finite state transducer FST over input alphabet A and output alphabetAi is a tuple A = (Q,→, I , f ) such that




• Q is a finite set (of states).





• →: Q × A → 2Q is the transition function.






• I ⊆ Q is the set of initial states.






• I ⊆ Q is the set of initial states.

• f : Q → Ai is the output function.


FST for Bounded Memory Strategy

Given a bounded memory strategy σ for player i we can construct an FSTAσ = (Q,→, I , f ) over A and Ai such that the output of the transducercorrectly reflects whatever the strategy σ prescribes.




• Q = M × A ∪ ǫ × Ai × W .




• Q = M × A ∪ ǫ × Ai × W .

• →: Q × A → 2Q such that if for any (m, a, a,w) ∈ Q, δ(a,m) =

m′ and g(a,m′) = a′ then (m, a, a,w)a′→ (m′, a′, a′,w ′) such that

a′(i) = a and wa′→ w ′. If g is not defined at (a,m′) then

(m, a, a,w)a′→ (m′, a′, a′,w ′) for all a′ ∈ Ai and w

a′→ w ′.




• Q = M × A ∪ ǫ × Ai × W .





a′→ w ′.

• I = (mI , ǫ, a,w0) | a = g(ǫ,mI ) if defined, else a ∈ Ai.




• Q = M × A ∪ ǫ × Ai × W .





a′→ w ′.

• I = (mI , ǫ, a,w0) | a = g(ǫ,mI ) if defined, else a ∈ Ai.

• f : Q → Ai such that f ((m, a, a,w)) = a.


Language of an FST

• A run χ of an fst A = (Q,→, I , f , λ) on a (total) strategy µ is alabelling of the nodes of strategy tree T µ

G with the states of Q suchthat the transitions of A are respected. That is, if there is an edgefrom node from a1 . . . ak to a1 . . . ak+1 in T µ

G thenχ(a1 . . . ak) ∈→ (χ(a1 . . . ak), ak+1).


Language of an FST




• A strategy µ is said to be accepted by A if there exists a run χ of Aon µ such that ∀t = a1 . . . ak ∈ T µ

G , a(i) = f (χ(t)).


Language of an FST




• A strategy µ is said to be accepted by A if there exists a run χ of Aon µ such that ∀t = a1 . . . ak ∈ T µ

G , a(i) = f (χ(t)).

• The language of A, L(A) is defined to be the set of all strategies thatare accepted by it.


Transducer Lemma

Lemma

Given game arena G, a player i ∈ N and a strategy specification π ∈ Πi ,

there is a transducer Aπ such that for all µ ∈ Ωi , µ ∈ [[π]]G iff µ ∈ L(Aπ).


Transducer Lemma

Lemma



Proof Sketch:By induction on π


Transducer Lemma

Lemma



Proof Sketch:By induction on π

• For σ ∈ Σi we have an FST Aσ from the construction above.


Proof Sketch

• π1 ∪ π2:

Aπ1 · · ·

Aπ2 · · ·


Proof Sketch[2]

• π1 ∩ π2:

Aπ1 · · ·

Aπ2 · · ·


Proof Sketch[3]

• π1aπ2:

Aπ1 · · ·

Aπ2 · · ·


Proof Sketch[4]

• π1 + π2:

Aπ1 · · ·

Aπ2 · · ·


Proof Sketch[5]

• ψ?π′:

Aπ′ · · ·

MCS(CL(ψ)) · · ·


Stability

• A strategy is switch-free if it does not have any of the a,+ or theψ?π′ constructs.


Stability


• Given a strategy π ∈ Πi of player i , let the set of substrategies of πbe Sπ.


Stability


• Given a strategy π ∈ Πi of player i , let the set of substrategies of πbe Sπ.

• Let SF (Sπ) be the set of switch-free strategies of Sπ.


Stability[2]

Let π ∈ Πi be a strategy of player i and Aπ = (Q,→, I , f , λ) be the FSTfor π. Let GAπ = (W ′,→′,w ′

0) where


Stability[2]


0) where

• W ′ = W × Q


Stability[2]


0) where

• W ′ = W × Q

• →′⊆ W ′ × W ′ such that (w1, q1)a

→′ (w2, q2) iff w1a→ w2, q

a→ q2

and f (q1) = a(i)


Stability[2]


0) where

• W ′ = W × Q

• →′⊆ W ′ × W ′ such that (w1, q1)a

→′ (w2, q2) iff w1a→ w2, q

a→ q2

and f (q1) = a(i)

• w ′0 = w0 × I


Stability[3]

G


Stability[3]

GAπ


Results

Theorem

Given a game arena G = (W ,→,w0) with a valuation V : W → 2P , a

subarena R of G and strategy specifications π1, . . . , πn for players 1 to n,

the question, Does the game eventually settle down to R is decidable.


Proof

• Construct

Gπ = (· · · ((GAπ1)Aπ2 · · · )Aπn) = (Wπ,→π,wπ)


Proof

• Construct


• Let F ⊆ Gπ = (W ′,→′) such thatW ′ = (w , q1, . . . , qn) | w ∈ R , q1 ∈ Qπ1, . . . , qn ∈ Qπn Let→′=→π ∩(W ′ × W ′).


Proof

• Construct



• Check if F is a maximal connected component in Gπ. If so proceed,else output a ‘NO’.


Proof

• Construct



• Check if F is a maximal connected component in Gπ. If so proceed,else output a ‘NO’.

• Check if all paths starting at w ′ ∈ wπ reach F . If so, output a ‘YES’,otherwise, output a ‘No’.


Proof[2]

Gπ


Proof[2]

F


Proof[2]

F c


Results[2]

Theorem

Given a game arena G = (W ,→,w0) with a valuation V : W → 2P , a

subarena R of G and strategy specifications π1, . . . , πn for players 1 to n,

the question, If the game converges to R, does the strategy of player i

become eventually stable with respect to switching is decidable.


Proof

• Check if the game eventually settles down to R .


Proof


• For all initial nodes w ′ ∈ wπ repeat:


Proof


• For all initial nodes w ′ ∈ wπ repeat:Let w⋆

π = (w⋆, q⋆1 , . . . , q

⋆n) be the state of F which is first reachable

from w ′. Let GR = (R ,→ R × R ,w⋆). For each π′ ∈ SF (Sπi)

construct

G ′π = (· · · (· · · ((GAπ1)Aπ2 · · · )Aπ′)· · · )Aπn)

and check if the entire graph is a connected component. If it is notthe case, output a ‘NO’ and halt. Else repeat.


Proof


• For all initial nodes w ′ ∈ wπ repeat:Let w⋆

π = (w⋆, q⋆1 , . . . , q

⋆n) be the state of F which is first reachable

from w ′. Let GR = (R ,→ R × R ,w⋆). For each π′ ∈ SF (Sπi)

construct

G ′π = (· · · (· · · ((GAπ1)Aπ2 · · · )Aπ′)· · · )Aπn)

and check if the entire graph is a connected component. If it is notthe case, output a ‘NO’ and halt. Else repeat.

• Output a ‘YES’.


Complexity

• Size of Aπ is O(2|π|).


Complexity


• The size of the restricted graph |GAπ| is O(|G | · 2|π|).


Complexity



• Checking for the maximal connected components of a graph can bedone is time polynomial in the size of the graph.


Complexity



• Checking for the maximal connected components of a graph can bedone is time polynomial in the size of the graph.

• Thus the running time of the decision procedures is O(|G | · 2|π|).


Extensions

• Restriction of choice.


Extensions

• Restriction of choice.

• Imposing neighbourhood structures on the players.


Thanks


Speciﬁcation of Dynamic Strategy Switchingali.cmi.ac.in/icla2009/slides/jan07/lasi/1500.pdf ·...

Documents

Transcript of Speciﬁcation of Dynamic Strategy Switchingali.cmi.ac.in/icla2009/slides/jan07/lasi/1500.pdf ·...