Chair of Software Engineering

Software Verification

Stephan van Staden

Lecture 10: Model Checking

2

Overview

The big picture Example to be verified Ingredients of a model checking system The state explosion problem Applications of model checking SLAM

3

The big picture Application domain: mostly concurrent, reactive

systems

Verification of particular properties

Verify that a system satisfies a property:1. Model the system in the model checker’s

description language. Call this model M2. Express the property to be verified in the

specification language of the model checker. Call this formula ϕ

3. Run the model checker to show that M satisfies ϕ, i.e. M ⊨ ϕ

Automatic for finite-state models

4

Example2 processes execute in parallel

Each undergoes transitions n → r → c → n →… where n denotes “not in critical section” r denotes “requesting to enter critical section” c denotes “in critical section”

Requirements: Safety: Only one process may execute critical

section code at any point Liveness: Whenever a process requests to enter

its critical section, it will eventually be allowed to do so

Non-blocking: A process can always request to enter its critical section

5

Example (cont)We write a program P to fulfill these requirements. But is it really doing its job?

We construct a model M for P where M captures the relevant behavior of P:

n1n2

r1n2

r1r2

r1c2c1r2

n1r2

n1c2c1n2

s2

s5

s7s4

s1

s0

s3 s6

6

Example (cont)Based on a definition of when a model satisfies a property, we can determine whether M satisfies P’s required properties

Example: M satisfies the safety requirement if no state reachable from s0 (including s0) is labeled c1c2. Thus our M satisfies P’s safety requirement

We can formalize the idea

Note: the conclusion that P satisfies these requirements depends on the (unverified) assumption that M is a faithful representation of all the relevant aspects of P

7

Ingredients of a model checking system

Models, specifications, a satisfaction relation, a satisfaction checking algorithm.

Tasks:1. Define what models are2. Define what specifications are3. Define when a model satisfies a specification4. Devise an efficient algorithm to decide satisfaction

8

1. Define what models areMany formalisms, e.g. Kripke models, LTSs, …

A transition system M = (S, →, L) is a set of states S with a transition relation → (a binary relation on S), such that every s ϵ S has some s’ ϵ S with s → s’, and a labeling function L: S → Ρ(Atoms), where Atoms is a set of atomic descriptions.

Our example: a transition system with S = {s0, s1, s2, s3, s4, s5, s6, s7} → = {(s0,s1), (s0,s5), (s1,s2), (s1,s3), (s4, s5), …} L(s0) = {n1,n2}, L(s1) = {r1,n2}, L(s3) = {r1,r2},

… where Atoms = {n1, r1, c1, n2, r2, c2}

Define a concrete syntax in which models can be described

9

2. Define what specifications areThey describe properties of models

Temporal logics (e.g. LTL, CTL) are often used to specify properties of concurrent, reactive systems.

Syntax of LTL formulas:ϕ ::= true | false | p | (¬ϕ) | (ϕ ˄ ϕ) | (ϕ v ϕ) | (ϕ → ϕ) | (X ϕ) | (G ϕ) | (F ϕ) | (ϕ U ϕ) | (ϕ W ϕ) | (ϕ R ϕ)

Example specifications: G ¬(c1 ˄ c2), G (r1 → F c1) and G (r2 → F c2)

Define a concrete syntax for specifications

10

3. Define the satisfaction relationFor a path π = s1→s2→... in a model M & LTL formula ϕ:π ⊨ trueπ ⊨ falseπ ⊨ p iff p ϵ L(s1)π ⊨ ¬ϕ iff π ⊨ ϕπ ⊨ ϕ1 ˄ ϕ2 iff π ⊨ ϕ1 and π ⊨ ϕ2

π ⊨ ϕ1 v ϕ2 iff π ⊨ ϕ1 or π ⊨ ϕ2

π ⊨ ϕ1 → ϕ2 iff π ⊨ ϕ2 whenever π ⊨ ϕ1 π ⊨ X ϕ iff π2 ⊨ ϕ (πi = si→si+1→...)π ⊨ G ϕ iff for all i ≥ 1, πi ⊨ ϕ π ⊨ F ϕ iff there is some i ≥ 1 such that πi ⊨ ϕ π ⊨ ϕ1 U ϕ2 iff there is some i ≥ 1 such that πi ⊨ ϕ2 and for all 1 ≤ j < i, πj ⊨ ϕ1

/

/

neXtGloballyFutureUntilWeak-untilRelease

11

3. Define the satisfaction relation (cont.)π ⊨ ϕ1 W ϕ2 iff either π ⊨ ϕ1 U ϕ2 or for all i ≥ 1, πi ⊨ ϕ1 π ⊨ ϕ1 R ϕ2 iff either there is some i ≥ 1 such that πi ⊨ ϕ1 and for all 1 ≤ j ≤ i we have πj ⊨ ϕ2, or for all k ≥ 1, πk ⊨ ϕ2

Suppose M = (S, →, L) is a model, s ϵ S and ϕ is an LTL formula. M,s ⊨ ϕ iff for every path π of M starting at s we have π ⊨ ϕ

Thus we are able to prove for our example that: M,s0 ⊨ G ¬(c1 ˄ c2) (the safety property holds). M,s0 ⊨ G (r1 → F c1) (the liveness property does not

hold – a counterexample path is s0→s1→s3→s7→s1→s3→s7→...).

The non-blocking property is not expressible in LTL.

/

12

4. Provide a checking algorithmDevise an efficient algorithm to decide M ⊨ ϕ

Typically without user interaction

Approaches: semantic, automata, tableau, …

Provide a counterexample when M ⊨ ϕ. The counterexample provides a clue to what is wrong: The system might be incorrect The model might be too abstract (in need of

refinement) The specification might not be the intended one

/

13

The state explosion problemThe number of states grow exponentially with the number of system components

For our example: 3 processes execute in parallel, 4 processes, etc.

Techniques to make state explosion less severe: Abstraction Induction Partial order reduction …

14

ApplicationsVerification of specific properties of: Hardware circuits Communication protocols Control software Embedded systems Device drivers (e.g. SLAM project of Microsoft) …

Real-world problems: “1020 states and beyond”

Useful: 2007 Turing Award (Clarke, Emerson and Sifakis)

We now turn to SLAM

15

SLAM introductionFaulty drivers often responsible for OS crashes

Verifying proper driver API usageE.g. a lock is never released without being first acquired

Prevent

16

SLAM introduction (cont)Input: a C program P (the device driver code) and temporal safety properties expressed as a SLIC specification ϕ

Output (if terminates): P ⊨ ϕ or P ⊨ ϕ/

17

SLIC property specificationA SLIC spec ϕ is a state machine

Example: state { enum {Unlocked = 0, Locked

= 1} state = Unlocked;

}

KeAcquireSpinLock.return { if (state==Locked) abort; else state = Locked;}

KeReleaseSpinLock.return { if (state==Unlocked) abort; else state = Unlocked;}

Unlocked Locked

Error

Rel Acq

Acq

Rel

18

Translating SLIC spec to C

state { enum {Unlocked = 0, Locked

= 1} state = Unlocked;

}

KeAcquireSpinLock.return { if (state==Locked) abort; else state = Locked;}

KeReleaseSpinLock.return { if (state==Unlocked) abort; else state = Unlocked;}

enum {Unlocked = 0, Locked = 1}

state = Unlocked;

void slic_abort() { SLIC_ERROR: ;}

void KeAcquireSpinLock_return { if (state==Locked) slic_abort(); else state = Locked;}

void KeReleaseSpinLock_return {

if (state==Unlocked) slic_abort();

else state = Unlocked;}

19

The device driver code P

do { KeAcquireSpinLock(); nPacketsOld = nPackets; if (request) {

request = request->Next;KeReleaseSpinLock();nPackets++;

}} while (nPackets !=

nPacketsOld);

KeReleaseSpinLock();

20

The instrumented program P’Compose P with C version of ϕ to obtain an instrumented P’. If SLIC_ERROR is reachable in P’, then P ⊨ ϕ else P ⊨ ϕ.Example P’:

/

enum {Unlocked = 0, Locked = 1}

state = Unlocked;

void slic_abort() { SLIC_ERROR: ;}

void KeAcquireSpinLock_return { if (state==Locked) slic_abort(); else state = Locked;}

void KeReleaseSpinLock_return {

if (state==Unlocked) slic_abort();

else state = Unlocked;}

do { KeAcquireSpinLock();A: KeAcquireSpinLock_return(); nPacketsOld = nPackets; if (request) {

request = request->Next; KeReleaseSpinLock();

B: KeReleaseSpinLock_return(); nPackets++;

} } while (nPacketsOld !=

nPackets); KeReleaseSpinLock();C: KeReleaseSpinLock_return();

21

The SLAM verification processIterative process to determine whether SLIC_ERROR is reachable in P’:

prog. P’prog. P

SLIC rule

boolean program

pathpredicates

slic

c2bp

bebop

newton

22

SLAM iterationLet E0 be the predicates present in the conditionals of ϕ.

For iteration i:1. Apply c2bp to construct the boolean program BP(P’,Ei)2. Apply bebop to check if there is a path pi in BP(P’,Ei) that

reaches SLIC_ERROR. If bebop determines that SLIC_ERROR is not reachable, then ϕ is valid in P and the algorithm terminates

3. If there such a path pi, then newton checks whether pi is feasible in P’. Two possible outcomes:

1. “yes”: ϕ has been invalidated in P and the algorithm terminates with counterexample pi

2. “no”: newton finds a set of predicates Fi that explain the infeasibility of pi in P’

4. Let Ei+1 := Ei ∪ Fi and i := i + 1 and proceed to the next iteration

23

Some observationsThe original satisfaction problem has been translated into a boolean program reachability problem.

For every statement s in P’ and every predicate e ϵ Ei, c2bp determines the effect of s on e to build the boolean program.

24

Iteration 0E0 = {state = Locked, state = Unlocked}

BP(P’,E0):decl {state=Unlocked},

{state=Locked};{state=Unlocked} := T;{state=Locked} := F;

void slic_abort() { SLIC_ERROR: skip;}

void KeAcquireSpinLock_return { if ({state=Locked}) slic_abort(); else { {state=Locked} := T;

{state=Unlocked} := F; }}

void KeReleaseSpinLock_return { if ({state=Unlocked}) slic_abort(); else { {state = Unlocked} := T;

{state = Locked} := F; }}

do { skip;A: KeAcquireSpinLock_return(); skip; if (*) {

skip; skip;

B: KeReleaseSpinLock_return(); skip;

} } while (*); skip;C: KeReleaseSpinLock_return();

25

Iteration 0 (cont)bebop provides path p0 = A→A→SLIC_ERROR

newton determines that p0 is infeasible in P’ and returns F0 = {nPacketsOld = nPackets} as explanation – it must be both true and false on p0: do {

KeAcquireSpinLock();A: KeAcquireSpinLock_return(); nPacketsOld = nPackets; if (request) {

request = request->Next; KeReleaseSpinLock();

B: KeReleaseSpinLock_return(); nPackets++;

} } while (nPacketsOld !=

nPackets); KeReleaseSpinLock();C: KeReleaseSpinLock_return();

26

Iteration 1E1 = {state = Locked, state = Unlocked, nPacketsOld = nPackets}

BP(P’,E1):

decl {state=Unlocked}, {state=Locked};

{state=Unlocked} := T;{state=Locked} := F;

void slic_abort() { SLIC_ERROR: skip;}

void KeAcquireSpinLock_return { if ({state=Locked}) slic_abort(); else { {state=Locked} := T;

{state=Unlocked} := F; }}

void KeReleaseSpinLock_return { if ({state=Unlocked}) slic_abort(); else { {state = Unlocked} := T;

{state = Locked} := F; }}

do { skip;A: KeAcquireSpinLock_return(); b := T; if (*) {

skip; skip;

B: KeReleaseSpinLock_return(); b := F;

} } while (!b); skip;C: KeReleaseSpinLock_return();

27

Iteration 1 (cont)bebop establishes that SLIC_ERROR is unreachable in BP(P’,E1)

SLAM concludes that P ⊨ ϕ

28

SourcesLogic in Computer Science – Modelling and Reasoning about Systems, 2nd edition, Michael Huth & Mark Ryan, Cambridge University Press, 2004.

Automatically Validating Temporal Safety Properties of Interfaces, T. Ball & S. K. Rajmani, SPIN 2001

Tutorial on "Software Model Checking with SLAM" at PLDI 2003