CS-E3220 Declarative Progamming

Model-Checking and Validation

Jussi Rintanen

Department of Computer ScienceAalto University

November 6, 2019

This Lecture

Model-checking in Temporal logics

SAT-Based Bounded Model-Checking (BMC)

Abstraction

Counterexample-guided abstraction refinement (CEGAR)

Model-CheckingTest if a CTL formula is true in a transition system

AFfinalstateAGEFprogress

Test if an LTL formula is true in a transition system

GF(green1)G(¬(green1 ∧ red1) ∧ ¬(green1 ∧ yellow1) ∧ ¬(yellow1 ∧ red1))G(green1 → F (red1))G(red1 → (red1Uyellow1))G¬(green1 ∧ green2)

Model-Checking

Model-checking: test M |= φ for given model M and formula φApplications:

Computer-aided verification (LTL, CTL, CTL∗)Advanced databases: query evaluation (description logics)

Easier than testing logical consequence, but can be hard

logic logical consequence model-checkingpropositional logic co-NP-complete P-completePDL EXP-complete P-completeCTL EXP-complete P-completeCTL∗ 2-EXP-complete PSPACE-completeLTL PSPACE-complete PSPACE-complete

Why Model-Checking?

Deductive verification: system models as formulas, theorem-proving

Poor scalability of theorem-provers −→ model-checking preferable

Practical handling of transition system models with hundreds ofthousands of statesSince 1990ies, model-checking without explicitly represented system−→ symbolic model-checking with OBDD & SAT:

OBDD-based model-checking for CTLSAT-based model-checking for LTL

Model-Checking for Propositional Logic

fun MC(And(p,q),M) = boolAnd(MC (p,M),MC (q,M))

| MC(Or(p,q),M) = boolOr(MC (p,M),MC (q,M))

| MC(Neg p,M) = boolNot (p,M)

| MC(Atom a,M) = M(a)

fun boolAnd(true,true) = true

| boolAnd _ = false

fun boolOr(false,false) = false

| boolOr _ = true

fun boolNot false = true

| boolNot true = false

M is the model: mapping from propositional variables to { true, false }

This algorithm runs in linear time if M is constant time (e.g. arrayindexed by propositional variables)

Model-Checking a Modal Logic with One Modality

1 Label each world w with L(w) = {x ∈ X |M |=w x}2 Consider all subformulas φ′ of φ in the order of increasing length.

For every world w , do L(w) := L(w) ∪ {φ′} ifφ′ = ψ1 ∧ ψ2 and ψ1 ∈ L(w) and ψ2 ∈ L(w), orφ′ = ψ1 ∨ ψ2 and ψ1 ∈ L(w) or ψ2 ∈ L(w), orφ′ = ¬ψ1 and ψ1 6∈ L(w), orφ′ = �ψ and ψ ∈ L(w ′) for all w ′ such that wRw ′, orφ′ = ♦ψ and ψ ∈ L(w ′) for some w ′ such that wRw ′.

The runtime of this procedure is O(|φ| × |W |)

Model-Checking for Temporal Logics

LTL, CTL, CTL∗ talk about infinite paths, even when the transitionsystem is finiteP-time model-checking for PDL and CTL:

Label the computation graph with all subformulas φ0 of φTruth in a node a function of truth of subformulas in the node and itssuccessors

PSPACE-hard model-checking for LTL and CTL∗:Need to consider exponentially many exponentially long paths

CTL Model-Checking

For every w ∈ W , L(w) := {x ∈ X |M |=w x}Consider all subformulas φ′ of φ in the order of increasing length.For every world w , do L(w) := L(w) ∪ {φ′} if

φ′ = ψ1 ∧ ψ2 and ψ1 ∈ L(w) and ψ2 ∈ L(w), orφ′ = ψ1 ∨ ψ2 and ψ1 ∈ L(w) or ψ2 ∈ L(w), orφ′ = ¬ψ1 and ψ1 6∈ L(w).

Formulas E (φUψ) and EGφ handled in the following slides.All other operators can be reduced to the above:

EFφ ≡ E (>Uφ)AFϕ ≡ ¬EG¬ϕAGϕ ≡ ¬E (>U¬ϕ)AXϕ ≡ ¬EX¬ϕ

A(ϕUψ) ≡ ¬E (¬ψU(¬ϕ ∧ ¬ψ)) ∧ ¬EG¬ψ

CTL Model-Checking: Labeling for E (φUψ)

For every world w , updateL(w) := L(w) ∪ {E (φUψ)} if

ψ ∈ L(w) or

φ ∈ L(w) and E (φUψ) ∈ L(w ′) forsome w ′ such that wRw ′

until no more updates possible.

Example: E (aUb)

w1 w2

w3

w4

w5

b

a

aa

Systematic algorithm given on the next slide...

CTL Model-Checking: Labeling for E (φUψ)

For every world w , updateL(w) := L(w) ∪ {E (φUψ)} if

ψ ∈ L(w) or

φ ∈ L(w) and E (φUψ) ∈ L(w ′) forsome w ′ such that wRw ′

until no more updates possible.

Example: E (aUb)

w1 w2

w3

w4

w5

bE (aUb)

a

aa

Systematic algorithm given on the next slide...

CTL Model-Checking: Labeling for E (φUψ)

For every world w , updateL(w) := L(w) ∪ {E (φUψ)} if

ψ ∈ L(w) or

φ ∈ L(w) and E (φUψ) ∈ L(w ′) forsome w ′ such that wRw ′

until no more updates possible.

Example: E (aUb)

w1 w2

w3

w4

w5

bE (aUb)

aE (aUb)

aa

Systematic algorithm given on the next slide...

CTL Model-Checking: Labeling for E (φUψ)

For every world w , updateL(w) := L(w) ∪ {E (φUψ)} if

ψ ∈ L(w) or

φ ∈ L(w) and E (φUψ) ∈ L(w ′) forsome w ′ such that wRw ′

until no more updates possible.

Example: E (aUb)

w1 w2

w3

w4

w5

bE (aUb)

aE (aUb)

aE (aUb)

aE (aUb)

Systematic algorithm given on the next slide...

CTL Model-Checking: Labeling for E (φUψ)

Procedure EU(α,β)T := {w ∈ W |β ∈ L(w)};For Each w ∈ T Do L(w) := L(w) ∪ {E (αUβ)};While T 6= ∅ Do

take any w ∈ T ;T := T\{w};For Each t such that tRw DoIf α ∈ L(t) and E (αUβ) 6∈ L(t) Then

L(t) := L(t) ∪ {E (αUβ)};T := T ∪ {t};

End IfEnd For

End While

CTL Model-Checking: Labeling for EGφ

Let Wφ = {w ∈ W |φ ∈ L(w)}Let G = 〈Wφ,R ∩ (Wφ ×Wφ)〉Find strongly connected components of G

For each SCC C such that |C | > 1 orwRw for some w ∈ C , do L(w) :=L(w) ∪ {EGφ} for all w ∈ C

For each w ∈ Wφ, if EGφ ∈ L(w ′) forsome w ′ such that wRw ′ then L(w) :=L(w) ∪ {EGφ}

w1 w2

w3

w4

w5

w6

w7

w8 w9

w10 w11

a a

a

a

a

a

a

a a

Systematic algorithm given on the next slide...

CTL Model-Checking: Labeling for EGφ

Let Wφ = {w ∈ W |φ ∈ L(w)}Let G = 〈Wφ,R ∩ (Wφ ×Wφ)〉Find strongly connected components of G

For each SCC C such that |C | > 1 orwRw for some w ∈ C , do L(w) :=L(w) ∪ {EGφ} for all w ∈ C

For each w ∈ Wφ, if EGφ ∈ L(w ′) forsome w ′ such that wRw ′ then L(w) :=L(w) ∪ {EGφ}

w1 w2

w3

w4

w5

w6

w7

w8 w9

w10 w11

a a

a

a

a

a

a

a a

Systematic algorithm given on the next slide...

CTL Model-Checking: Labeling for EGφ

Let Wφ = {w ∈ W |φ ∈ L(w)}Let G = 〈Wφ,R ∩ (Wφ ×Wφ)〉Find strongly connected components of G

For each SCC C such that |C | > 1 orwRw for some w ∈ C , do L(w) :=L(w) ∪ {EGφ} for all w ∈ C

For each w ∈ Wφ, if EGφ ∈ L(w ′) forsome w ′ such that wRw ′ then L(w) :=L(w) ∪ {EGφ}

w1 w2

w3

w4

w5

w6

w7

w8 w9

w10 w11

a a

a

a

a

a

a

a a

Systematic algorithm given on the next slide...

CTL Model-Checking: Labeling for EGφ

Let Wφ = {w ∈ W |φ ∈ L(w)}Let G = 〈Wφ,R ∩ (Wφ ×Wφ)〉Find strongly connected components of G

For each SCC C such that |C | > 1 orwRw for some w ∈ C , do L(w) :=L(w) ∪ {EGφ} for all w ∈ C

For each w ∈ Wφ, if EGφ ∈ L(w ′) forsome w ′ such that wRw ′ then L(w) :=L(w) ∪ {EGφ}

w1 w2

w3

w4

w5

w6

w7

w8 w9

w10 w11

a a

a

a

a

a

a

a a

EGa

EGa

EGa

EGa

Systematic algorithm given on the next slide...

CTL Model-Checking: Labeling for EGφ

Let Wφ = {w ∈ W |φ ∈ L(w)}Let G = 〈Wφ,R ∩ (Wφ ×Wφ)〉Find strongly connected components of G

For each SCC C such that |C | > 1 orwRw for some w ∈ C , do L(w) :=L(w) ∪ {EGφ} for all w ∈ C

For each w ∈ Wφ, if EGφ ∈ L(w ′) forsome w ′ such that wRw ′ then L(w) :=L(w) ∪ {EGφ}

w1 w2

w3

w4

w5

w6

w7

w8 w9

w10 w11

a a

a

a

a

a

a

a a

EGa

EGa

EGa

EGa

EGa

EGa

Systematic algorithm given on the next slide...

CTL Model-Checking: Labeling for EGφProcedure EG(α)S ′ := {w ∈ W |α ∈ L(w)};SCC := {C |C is a SCC of 〈S ′,R ∩ (S ′ × S ′)〉, |C | ≥ 1 or there is w ∈ C with wRw};T := {w ∈ C |C ∈ SCC};For Each w ∈ T Do L(w) := L(w) ∪ {EGα};While T 6= ∅ Do

take any w ∈ T ;T := T\{w};For Each t ∈ S ′ such that tRw DoIf EGα 6∈ L(t) Then

L(t) := L(t) ∪ {EGα};T := T ∪ {t};

End IfEnd For

End While

Bounded Model-Checking for LTL

SAT-based LTL model-checking solves two exponential problems atthe same time:

Find a path in an exponential-size transition graphTest that the LTL formula is satisfied on that path

Large size of transition systems masks the PSPACE-hardness of LTLmodel-checking

LTL model-checking popular since 1999, due to very goodscalability of SAT solvers

Use of CTL model-checking decreasing, due to transition systemsbeing very large

Bounded Model-Checking for LTL

s0 s1 s2 s3 s4 s5

Model-checking limited by size of computation graphs (1M, 1B)

Symbolic methods (OBDD, SAT) can relax this limitation

Biere et al. (1999) reduced LTL model-checking to SAT

Consider paths of the form s0, s1, . . . ,︷ ︸︸ ︷sl , . . . , sk ,

︷ ︸︸ ︷sl , . . . , sk , sl , . . .

The segment sl , . . . , sk repeats an infinite number of times

Encode this path with a loop as a propositional formula

Bounded Model-Checking for LTL

s0 s1 sl−1 sl sl+1 sk−1 sk

[[x ]]l ,ki = x@i [[Xφ]]l ,ki = [[φ]]l ,ksucc(i) Here:succ(i) = i + 1 if i < ksucc(k) = l

[[¬x ]]l ,ki =¬x@i [[Gφ]]l ,ki =∧k

j=min(l ,i)[[φ]]l ,kj[[φ1 ∨ φ2]]l ,ki = [[φ1]]l ,ki ∨ [[φ2]]l ,ki [[Fφ]]l ,ki =

∨kj=min(l ,i)[[φ]]l ,kj

[[φ1 ∧ φ2]]l ,ki = [[φ1]]l ,ki ∧ [[φ2]]l ,ki [[φUψ]]l ,ki =∨ ∨k

j=i([[ψ]]l ,kj ∧∧j−1

z=i [[φ]]l ,kz )∨i−1j=l ([[ψ]]l ,kj ∧

∧j−1z=l [[φ]]l ,kz ∧

∧kz=i [[φ]]l ,kz )

[[φRψ]]l ,ki = similar

φ is true on some path (representable as a (k , l)-loop) iff[[φ]]l ,k0 ∧ I ∧ T [X @1/X ′] ∧ · · · ∧ T [X @(k − 1)/X ,X @k/X ′] ∧ T [X @k/X ,X @l/X ′] issatisfiable

Bounded Model-Checking for LTL

s0 s1 s2 s3 s4

[[XGa]]2,40 = a@1 ∧ a@2 ∧ a@3 ∧ a@4

[[aUb]]2,41 = b@1∨(b@2 ∧ a@1)∨(b@3 ∧ a@1 ∧ a@2)∨(b@4 ∧ a@1 ∧ a@2 ∧ a@3)

Bounded Model-Checking for LTL

2-D search: Test formulas for (k , l)-loops with increasing k , l

1-D search: Encode choice of l in formulas for fixed k , increase kStrengths:

Very effective in finding faulty behaviors (for “low” values of k)Applicable to far bigger systems than OBDDs

Weaknesses:No general method for deciding when to stop increasing kHence: Often not a practical method for proving correctness

Abstraction in Verification

Correctness of a system can sometimes be determined from anabstraction of a system

Ignore some aspects of the system modelAbstracted system has more possible behaviors than the original systemIf all executions of abstracted systems have given property, then so do thoseof the original system

Reasoning about the abstract system can be far easierFewer state variables −→ number of abstract states smaller

Applications:Software verification (conventional program code)sequential circuits (CPUs)others

How to Abstract a System Model

Abstracting a state = distinctions between states eliminatedAbstracting state variables = distinctions between values eliminated

Examples of abstracting state variablesintegers x , y by fx ,y : Z× Z→ {0, 1} such that

fx ,y(x , y) =

{1 if x > y0 otherwise

Boolean b by fb(v) = 0 (constant function: variable eliminated)

weekday d by fd(Sat) = 0, fd(Sun) = 0, fd(d) = 1 otherwise

How to Abstract a System ModelAbstracting states: s mapped to s ′ = f (s) by mapping values of eachvariable or subset of variables to its abstraction:

state variables X partitioned to X1, . . . ,Xn

abstraction functions are f1, . . . , fnabstract state variables are y1, . . . , ynvalue s ′(yi) in the abstracted state is obtained by fi :

s ′(yi) = fi(x i1, . . . , x

iji)

where Xi = {x i1, . . . , x

iji}

Equivalence relation ≡f induced by f : s1 ≡f s2 iff f (s1) = f (s2)

A Simple Special Class of Abstractions

We next only consider abstracting Boolean variables one by oneBoolean state variables x are abstracted either

by fx(v) = 0 (eliminating the variable), orfx(v) = v (retaining the variable)

A subset of state variables is eliminated, others remain intact

How to Abstract a System Model

Simple abstraction: eliminate some state variable(s)

Eliminate x from formula φ: generate ∃x .φ↔ φ[>/x ] ∨ φ[⊥/x ]Eliminate x from effect e:

Replace all assignments x := B by εReplace eITE(φ, e1, e2) by eITE(∃x .φ, e ′1, e

′2) where e ′i is ei with x eliminated

This allows eliminating x from any transition rule (φ, e)

Example

Load object in truck at A:(TatA∧OatA, (OatA := 0; OinT := 1))

Unload object from truck at B:(TatB∧OinT, (OinT := 0; OatB := 1))

Move truck from A to B:(TatA, (TatA := 0;TatB := 1))

Possible states AA, BA, AT, BT, BB AA

ATBT

BB

BA

load

A to B

unlo

ad

A to B

Example, AbstractedAbstract away both TatA and TatB:

Load object in truck at A:(OatA, (OatA := 0; OinT := 1))

Unload object from truck at B:(OinT, (OinT := 0; OatB := 1))

Move truck from A to B: (>, ε)

Possible states:

abstract concrete-B BB-T AT, BT-A AA, BA

-A

-T

-B

load

unlo

ad

Arrows in the abstraction are a superset of thearrows in the original system

Properties of Abstractions

Let T be a transition system and T ′ its abstraction (w.r.t vars Z ).Let φ be a formula and φ′ = ∃Z .φ.

Theorems:

If the path in T has length n, then the path in T ′ has length ≤ n

If there is path to φ in T , then there is a path to φ′ in T ′

Corollary:

If there is no path to φ′ in T ′, then there is no path to φ in T

Properties of Abstractions: Spurious Paths

If there is path p to φ′ in T ′, then there might be no path to φ in T .Here p is called a spurious path.Core question in model-checking with abstractions: What to dowith spurious paths?

We want to test if LTL formula φ can be true on a pathThere is such a path in the abstracted systemDoes this path have a counterpart in the concrete system?

Reachability Checking with Abstraction

Test if any state satisfying φ is reachable:

1 Build initial abstraction of transition system (not abstracting φ)2 Test if φ is reachable in the abstracted system.3 If φ not reachable in abstraction, also not reachable in unabstracted

system. Stop.4 Let t1, . . . , tn be the transition sequence reaching φ5 If t1, . . . , tn is a transition sequence also in the original system, stop.6 Generate a new (less abstract) abstraction.7 Go to step 2.

Counterexample-Guided Abstraction Refinement

A spurious path has been found

How is the abstraction refined?

1 Given t1, . . . , tn, identify the maximal prefix t1, . . . , ti that isexecutable in the original transition system

2 ti+1 is not possible in the original system, only in the abstraction3 Consider the corresponding state sequences s ′0, . . . , s

′i and s0, . . . , si

4 Transition ti+1 is possible in s ′i , but not in si5 This difference suggests how the abstraction must be changed!

Counterexample-Guided Abstraction RefinementThe (spurious) plan to transport object from A to B:

1 load object in vehicle in location A2 unload object from vehicle in location B

The state sequences:

action state original system abstract systemTatA TatB OatA OinT OatB TatA TatB OatA OinT OatB

load s0 1 0 1 0 0 - - 1 0 0unload s1 1 0 0 1 0 - - 0 1 0

s2 NA NA NA NA NA - - 0 0 1

In the original system, unload’s precondition TatB∧ OinT is false!Refine the abstraction: include the state variable TatB.

Example, Abstraction Refined

Load object in truck at A:(OatA, (OatA := 0; OinT := 1))

Unload object from truck at B:(TatB∧OinT, (OinT := 0; OatB := 1))

Move truck from A to B:(>, (TatB := 1))

Possible states -A, BA, -T, BT, BB -A

-TBT

BB

BA

load

A to B

unlo

ad

A to B

Counterexample-Guided Abstraction Refinement

CEGAR applicable to all leading methodsExplicit state reachability analysis (e.g. breadth-first search)Explicit state model-checkingOBDD-based reachability & model-checkingSAT-based reachability & model-checking

Critical in SW model-checking, where abstraction is unavoidable(due to the typically infinite state space)