From Graph Models to Game Models

Tom Henzinger EPFL

Graph Models of Systems

vertices = states

edges = transitions

paths = behaviors

graph

Extended Graph Models

MULTIPLE ACTORS:

game graph

LIVENESS: -automaton

PROBABILITIES: Markov decision process

stochastic game

regular game

Graphs vs. Games

a

baa b

a

Games model Open Systems

Two players: environment / controller / input vs.

system / plant / output

Multiple players: processes / components / agents

Stochastic players: nature / randomized algorithms

-synthesis [Church, Rabin, Ramadge/Wonham, Pnueli/Rosner]

-receptiveness [Dill, Abadi/Lamport]

-scheduling [Sifakis et al.]

-reasoning about system components [Kupferman/Vardi et al.]

-early error detection [deAlfaro/H/Mang]

-model-based testing [Gurevich et al.]

-interface compatibility [deAlfaro/H]

-program repair [Bloem et al.]

-etc.

Applications of Graph Games

Example

P1:

init x := 0

loop

choice | x := x+1 mod 2| x := 0

end choice

end loop

S1: (x = y )

P2:

init y := 0

loop

choice | y := x | y := x+1 mod 2

end choice

end loop

S2: ( y = 0 )

Graph Questions

8 ( x = y )

9 ( x = y )

CTL

Graph Questions

8 ( x = y )

9 ( x = y )00

10 11

01

X

CTL

Zero-Sum Game Questions

hhP1ii ( x = y )

hhP2ii ( y = 0 )

ATL [Alur/H/Kupferman]

Zero-Sum Game Questions

hhP1ii ( x = y )

hhP2ii ( y = 0 )

00

00 00

10

10 10

01

01 01

11

1111ATL [Alur/H/Kupferman]

Zero-Sum Game Questions

hhP1ii ( x = y )

hhP2ii ( y = 0 )

00

00 00

10

10 10

01

01 01

11

1111ATL [Alur/H/Kupferman]

X

Zero-Sum Game Questions

hhP1ii ( x = y )

hhP2ii ( y = 0 )

00

00 00

10

10 10

01

01 01

11

1111ATL [Alur/H/Kupferman]

X

Nonzero-Sum Game Questions

hhP1ii ( x = y )

hhP2ii ( y = 0 )

00

00 00

10

10 10

01

01 01

11

1111

Secure equilibra [Chatterjee/H/Jurdzinski]

Nonzero-Sum Game Questions

hhP1ii ( x = y )

hhP2ii ( y = 0 )

00

00 00

10

10 10

01

01 01

11

1111

Secure equilibra [Chatterjee/H/Jurdzinski]

Winning Conditions

Qualitative: -regular (safety; Buchi; parity)

Quantitative: max; lim sup; lim avg

Quantitative Game Questions

hhP1ii lim sup

hhP1ii lim avg

4

2

2

0

2

0

0

4

3

Quantitative Game Questions

hhP1ii lim sup = 3

hhP1ii lim avg

4

2

2

0

2

0

0

4

3

Quantitative Game Questions

hhP1ii lim sup = 3

hhP1ii lim avg = 1

4

2

2

0

2

0

0

4

3

Many Open Problems

Buchi (lim sup) games in subquadratic time ?

Parity (lim avg) games in polynomial time ??

Solving Games by Value Iteration

Generalization of the -calculus: computing fixpoints of transfer functions (pre; post).

Generalization of dynamic programming: iterative optimization.

q

Region R: Q ! V

q’

R(q’)

Solving Games by Value Iteration

Generalization of the -calculus: computing fixpoints of transfer functions (pre; post).

Generalization of dynamic programming: iterative optimization.

q

Region R: Q ! V

q’

R(q’)

R(q) := pre(R(q’))

Q states transition labels : Q Q

transition function

= [ Q ! {0,1} ] regions with V = B

9pre:

q 9pre(R) iff ( ) (q,) R

8pre:

q 8pre(R) iff ( ) (q,) R

Graph

a cb

Graph

9 c = ( X) ( c Ç 9pre(X) )

a cb

Graph

9 c = ( X) ( c Ç 9pre(X) )

a cb

Graph

9 c = ( X) ( c Ç 9pre(X) )

a cb

Graph

9 c = ( X) ( c Ç 9pre(X) )

8 c = ( X) ( c Ç 8pre(X) )

Q1, Q2 states ( Q = Q1 [ Q2 ) transition labels : Q Q

transition function

= [ Q ! {0,1} ] regions with V = B

1pre:

q 1pre(R) iff q 2 Q1 Æ ( ) (q,) R or q 2 Q2 Æ (8 2 )

(q,) 2 R

2pre:

q 2pre(R) iff q 2 Q1 Æ (8 ) (q,) R or q 2 Q2 Æ (9 2 ) (q,) 2 R

Turn-based Game

c

Turn-based Game

a b

c

Turn-based Game

a b

hh1ii c = ( X) ( c Ç 1pre(X) )

c

Turn-based Game

a b

hh1ii c = ( X) ( c Ç 1pre(X) )

c

Turn-based Game

a b

hh1ii c = ( X) ( c Ç 1pre(X) )

hh2ii c = ( X) ( c Ç 2pre(X) )

c

Turn-based Game

a b

hh1ii c = ( X) ( c Ç 1pre(X) )

hh2ii c = ( X) ( c Ç 2pre(X) )

c

Turn-based Game

a b

hh1ii c = ( X) ( c Ç 1pre(X) )

hh2ii c = ( X) ( c Ç 2pre(X) )

Q1, Q2 states ( Q = Q1 [ Q2 ) transition labels : Q N £ Q transition function

= [ Q ! N ] regions with V = N

1pre:

1pre(R)(q) = (max ) max( 1(q,), R(2(q,)) ) if q 2 Q1 (min 2 ) max( 1(q,), R(2(q,)) ) if q 2 Q2

2pre:

2pre(R)(q) = (min ) max( 1(q,), R((q,)) ) if q 2 Q1 (max 2 ) max( 1(q,), R(2(q,)) ) if q 2 Q2

Quantitative Game

c

Quantitative Game

a b0

1

2

5

3

c

Quantitative Game

a b

hh1ii 0 = ( X) max( 0, 1pre(X) )

0

1

2

5

3

0 0 0

c

Quantitative Game

a b

hh1ii 0 = ( X) max( 0, 1pre(X) )

0

1

2

5

3

1 0 0

c

Quantitative Game

a b

hh1ii 0 = ( X) max( 0, 1pre(X) )

0

1

2

5

3

1 2 0

c

Quantitative Game

a b

hh1ii 0 = ( X) max( 0, 1pre(X) )

0

1

2

5

3

2 2 0

Q states 1, 2 moves of both players : Q 1 2 Q transition function

= [ Q ! {0,1} ] regions with V = B

1pre:

q 1pre(R) iff (1 1) (2 2) (q,1,2) R

2pre:

q 2pre(R) iff (2 2 ) (1 1) (q,1,2) R

Concurrent Game

a cb

1,1 1,2

2,1 2,2

1,1 1,2 2,2

2,1

Concurrent Game

a cb

1,1 1,2

2,1 2,2

1,1 1,2 2,2

2,1

Concurrent Game

hh2ii c = ( X) ( c Ç 2pre(X) )

a cb

1,1 1,2

2,1 2,2

1,1 1,2 2,2

2,1

Concurrent Game

hh2ii c = ( X) ( c Ç 2pre(X) )

a cb

1,1 1,2

2,1 2,2

1,1 1,2 2,2

2,1

Concurrent Game

hh2ii c = ( X) ( c Ç 2pre(X) )

Pr(1): 0.5 Pr(2): 0.5

Q states 1, 2 moves of both players : Q 1 2 Dist(Q) probabilistic transition function

= [ Q ! [0,1] ] regions with V = [0,1]

1pre:

1pre(R)(q) = (sup 1 1 ) (inf 2 2) R((q,1,2))

2pre:

2pre(R)(q) = (sup 2 2) (inf 1 1) R((q,1,2))

Stochastic Game

[deAlfaro/Majumdar]

a cb

1

1

2

2Pl.1Pl.2

a: 0.6 b: 0.4

a: 0.1 b: 0.9

a: 0.5 b: 0.5

a: 0.2 b: 0.8

1

1

2

2Pl.1Pl.2

a: 0.0 c: 1.0

a: 0.7 b: 0.3

a: 0.0 c: 1.0

a: 0.0 b: 1.0

Stochastic Game

a cb

1

1

2

2Pl.1Pl.2

a: 0.6 b: 0.4

a: 0.1 b: 0.9

a: 0.5 b: 0.5

a: 0.2 b: 0.8

1

1

2

2Pl.1Pl.2

a: 0.0 c: 1.0

a: 0.7 b: 0.3

a: 0.0 c: 1.0

a: 0.0 b: 1.0

Stochastic Game

hh1ii c = ( X) max( c, 1pre(X) )

0

10

a cb

1

1

2

2Pl.1Pl.2

a: 0.6 b: 0.4

a: 0.1 b: 0.9

a: 0.5 b: 0.5

a: 0.2 b: 0.8

1

1

2

2Pl.1Pl.2

a: 0.0 c: 1.0

a: 0.7 b: 0.3

a: 0.0 c: 1.0

a: 0.0 b: 1.0

Stochastic Game

hh1ii c = ( X) max( c, 1pre(X) )

0

11

a cb

1

1

2

2Pl.1Pl.2

a: 0.6 b: 0.4

a: 0.1 b: 0.9

a: 0.5 b: 0.5

a: 0.2 b: 0.8

1

1

2

2Pl.1Pl.2

a: 0.0 c: 1.0

a: 0.7 b: 0.3

a: 0.0 c: 1.0

a: 0.0 b: 1.0

Stochastic Game

hh1ii c = ( X) max( c, 1pre(X) )

0.8

11

a cb

1

1

2

2Pl.1Pl.2

a: 0.6 b: 0.4

a: 0.1 b: 0.9

a: 0.5 b: 0.5

a: 0.2 b: 0.8

1

1

2

2Pl.1Pl.2

a: 0.0 c: 1.0

a: 0.7 b: 0.3

a: 0.0 c: 1.0

a: 0.0 b: 1.0

Stochastic Game

hh1ii c = ( X) max( c, 1pre(X) )

0.96

11

a cb

1

1

2

2Pl.1Pl.2

a: 0.6 b: 0.4

a: 0.1 b: 0.9

a: 0.5 b: 0.5

a: 0.2 b: 0.8

1

1

2

2Pl.1Pl.2

a: 0.0 c: 1.0

a: 0.7 b: 0.3

a: 0.0 c: 1.0

a: 0.0 b: 1.0

Stochastic Game

hh1ii c = ( X) max( c, 1pre(X) )

limit 1

11

Solving Games by Value Iteration

Safety: Buchi: Parity: …Many open questions:

How do different evaluation orders compare? How fast do these algorithms converge? When are they

optimal?

Q control locations transition labels S program statements : Q S £ Q transition function P

predicates

= [ Q ! 2P ] regions with V = 2P

9pre:

p 9pre(R)(q) iff ( ) ( wp[(q,)] R(2(q,)) ) p )

Predicate Abstraction for Programs

Graph-based (finite-carrier) systems:

Q = Bm = boolean formulas [e.g. BDDs]

pre = (9 x 2 B)

Timed and hybrid systems:

Q = Bm £ Rn

= formulas of (Q,·,+) [e.g. polyhedral sets]pre = (9 x 2 Q)

Beyond Graphs as Finite Carrier Sets

Summary

Model checking is a very special (boolean) case of graph-based optimization problems.

It can be generalized to solve much more general questions that involve multiple players, quantitative resources, probabilistic transitions, and continuous state spaces.

The theory and practice of this is still wide open …