K. Rustan M. LeinoRiSE, Joint work with:Peter Müller (ETH Zurich)Jan Smans (KU Leuven)

Special thanks to Mike BarnettVMCAI, Madrid, Spain, 18 January 2010

Verifying Concurrent Programs with Chalice

Concurrent programsInterleaving of thread executionsUnbounded number of: threads, locks, …We need some basis for doing the reasoning

A way of thinking!

ChaliceExperimental language with focus on:

Shared-memory concurrencyStatic verification

Key featuresMemory access governed by a model of permissionsSharing via locks with monitor invariantsCopy-free non-blocking channelsDeadlock checking, dynamic lock re-ordering

Other featuresClasses; Mutual exclusion and readers/writers locks; Fractional permissions; Two-state monitor invariants; Asynchronous method calls; Memory leak checking; Logic predicates and functions; Ghost and prophecy variables

Dealing with memory (the heap)Access to a memory location requires

permissionPermissions are held by activation recordsSyntax for talking about permission to y: acc(y)

Incdemo

Transfer of permissionsmethod Main(){

var c := new Counter;call c.Inc();

}

method Inc()requires acc(y);ensures acc(y);

{y := y + 1;

}

acc(c.y)

The two halves of a callcall == fork + join

is semantically like

… but is compiled to more efficient code

call x,y := o.M(E, F);

fork tk := o.M(E, F);join x,y := tk;

Parallel Incdemo

Passing permissions to threadsclass XYZ {var x: int;var y: int;var z: int;

method Main(){var c := new

XYZ;fork c.A();fork c.B();

}

…}

method A()requires acc(x);

{x := x + 1;

}

method B()requires acc(y) &&

acc(z);{

y := y + z;}

Read permissionsacc(y) write permission to yrd(y) read permission to y

At any one time, at most one thread can have write permission to a location

Passing permissions to threadsclass Fib {var x: int;var y: int;var z: int;

method Main(){var c := new

Fib;fork c.A();fork c.B();

}

…}

method A()requires rd(x) && acc(y)

{y := x + 21;

}

method B()requires rd(x) && acc(z)

{z := x + 34;

}

Fractional permissionsacc(y) 100% permission to yacc(y, p) p% permission to yrd(y) read permission to yWrite access requires 100%Read access requires >0%

= +

acc(y) acc(y,69) acc(y,31)

rd(y) acc(y,)

Shared stateWhat if two threads want write access to the same location?

method A() …{

y := y + 21;}

method B() …{

y := y + 34;}

class Fib {var y: int;method Main(){var c := new

Fib;fork c.A();fork c.B();

}

…}

acc(c.y) ?

Monitorsmethod A() …{

acquire this;y := y + 21;release this;

}

method B() …{

acquire this;y := y + 34;release this;

}

class Fib {var y: int;

invariant acc(y);method Main(){var c := new

Fib;share c;fork c.A();fork c.B();

}

…}

acc(c.y)

acc(y)

Locks and permissionsThe concepts

holding a lock, andhaving permissions

are orthogonal to one anotherIn particular:

Holding a lock does not imply any right to read or modify shared variables

Their connection is:Acquiring a lock obtains some permissionsReleasing a lock gives up some permissions

Monitor invariantsLike other specifications, monitors can hold both permissions and conditionsExample: invariant acc(y) && 0 ≤ y

acc(y)

Owicki Gries counter

[Chalice encoding by Bart Jacobs]

demo

Abstractionclass MyClass {var x,y: int;predicate Valid { acc(c.x) && acc(c.y) && x ≤ y } …

}

Abstractionclass MyClass {var x,y: int;predicate Valid { acc(c.x) && acc(c.y) && x ≤ y }

method New() returns (c: MyClass)ensures c.Valid;

{…

}

method Mutate()requires c.Valid;ensures c.Valid;

{…

}}

Abstractionclass MyClass {var x,y: int;predicate Valid { acc(c.x) && acc(c.y) && x ≤ y }method New() returns (c: MyClass)ensures c.Valid;

{var c := new MyClass { x := 3, y := 5 };fold c.Valid;

}method Mutate()requires c.Valid;ensures c.Valid;

{unfold c.Valid;c.y := c.y + 3;fold c.Valid;

}}

acc(c.x)acc(c.y)x ≤ y

c.Valid

Abstractionclass MyClass {var x,y: int;predicate Valid { acc(c.x) && acc(c.y) && x ≤ y }method New() returns (c: MyClass)ensures c.Valid;

{var c := new MyClass { x := 3, y := 5 };fold c.Valid;

}method Mutate()requires c.Valid;ensures c.Valid;

{unfold c.Valid;c.y := c.y + 3;fold c.Valid;

}}

acc(c.x)acc(c.y)x ≤ yc.Valid c.Valid

Channelschannel Ch(c: Cell, z: int) where acc(c.y) && c.y ≤ z;

Channelschannel Ch(c: Cell, z: int) where acc(c.y) && c.y ≤ z;

class Cell {var x,y: int;

method Producer(ch: Ch){var c := new C { x := 0, y := 0 };send ch(c, 5);

}

method Consumer(ch: Ch){receive c,z := ch;…

}}

acc(c.y)acc(c.x)

Channelschannel Ch(c: Cell, z: int) where acc(c.y) && c.y ≤ z;

class Cell {var x,y: int;

method Producer(ch: Ch){var c := new C { x := 0, y := 0 };send ch(c, 5);

}

method Consumer(ch: Ch){receive c,z := ch;…

}}

acc(c.y)

c.y ≤ z

Preventing deadlocks

Deadlocks are prevented by making sure no such cycle can ever occur

The program partially order locksThe program is checked to acquire locks in strict ascending order

A deadlock is the situation where a nonempty set (cycle) of threads each waits for a resource (e.g., lock) that is held by another thread in the set

Wait orderWait order is a dense partial order(Mu, <<) with a bottom element << is the strict version of <<The wait level of an object o is stored in a mutable ghost field o.muAccessing o.mu requires appropriate permissions, as for other fields

Example: Avoiding deadlocks

With these preconditions, both methods verifyThe conjunction of the preconditions is false, so the methods can never be invoked at the same time

method M()

{acquire a;acquire b;…

}

method N()

{acquire b;acquire a;…

}

requires rd(a.mu);requires rd(b.mu);

requires rd(a.mu)requires rd(b.mu)requires waitlevel <<

b.mu;requires b.mu << a.mu;

requires waitlevel << a.mu;requires a.mu << b.mu;

Setting the wait orderRecall, the wait level of an object o is stored in the ghost field o.muInitially, the .mu field is The .mu field is set by the share statement:

picks some wait level strictly betweenL and H, and sets o.mu to that levelProvided L << H and neither denotes an extreme element, such a wait level exists, since the order is denseChanging o.mu requires acc(o.mu), as usual

share o between L and H;

Formal semantics of ChaliceGiven as translation to Boogie

Chalice

Z3

Boogie

Boogie is an intermediate verification language

Encoding the heapChalice: o.fBoogie: Heap[ o, f ]

where Heap is declared to be a map fromobjects and field names to values

To encode permissions, use another map: Mask

Encoding a callcall M()

=Exhale[[ P ]]; Inhale[[ Q ]]

method M()requires P;ensures Q;

Exhale and InhaleDefined by structural inductionFor expression P without permission predicatesExhale P ≡ assert PInhale P ≡ assume P

Exhale acc(o.f, p) ≡ assert p ≤ Mask[o,f];Mask[o,f] := Mask[o,f] – p;

Inhale acc(o.f, p) ≡if (Mask[o,f] == 0) { havoc

Heap[o,f]; }Mask[o,f] := Mask[o,f] + p;

Example

call M()=assert 100 ≤ Mask[this,y];Mask[this,y] := Mask[this,y] – 100;oldH := Heap;if (Mask[this,y] = 0) { havoc Heap[this,y]; }Mask[this,y] := Mask[this,y] + 100;assume Heap[this,y] = oldH[this,y] * oldH [this,y];

class Cell {var y: int;method Square()requires acc(y);ensures acc(y) && y =

old(y*y);

Boogie translation demodemo

An alternative to old

Logical constant:Let the verifier figure out K!

method Square(c: Cell)requires acc(c.y);ensures acc(c.y) && c.y ==

old(c.y*c.y);method Square(c: Cell, ghost K: int)requires acc(c.y) && K == c.y;ensures acc(c.y) && c.y == K*K;

call Square(c, c.y);

call Square(c, *);

Square only reads c.y

method Square(c: Cell) returns (r: int)requires acc(c.y, ε);ensures acc(c.y, ε) && r == c.y*c.y;

method Square(c: Cell) returns (r: int)requires acc(c.y);ensures acc(c.y) && r == c.y*c.y;

method Square(c: Cell, ghost K: perm)returns (r: int)

requires acc(c.y, K);ensures acc(c.y, K) && r == c.y*c.y;

ε loses procedural abstraction

Language questions

A better notation? rd(c.y)? Does that pick one K for each method activation?Can these 3 kinds of “parameters” (real, ghost, permission) be treated more uniformly?Which kinds should be indicated explicitly by the programmer and which should be figured out by the compiler?

method Square(c: Cell, ghost K: perm)returns (r: int)

requires acc(c.y, K);ensures acc(c.y, K) && r == c.y*c.y;

Chalice summaryPermissions guide what memory locations are allowed to be accessedActivation records can hold permissionsPermissions can also be stored in various “boxes” (monitors, predicates, channels)Permissions can be transferred between activation records and boxesLocks grant mutually exclusive access to monitors

Try it for yourselfChalice (and Boogie) available as open source:http://boogie.codeplex.com

Tutorial and other papers available from:http://research.microsoft.com/~leino