Abstraction for Falsification

Abstraction for FalsificationAbstraction for Falsification

Thomas Ball

Orna Kupferman

Greta Yorsh

Microsoft Research, Redmond, US

Hebrew University, Jerusalem, Israel

Tel Aviv University, Israel

CAV’05

Abstraction for VerificationAbstraction for Verification

• Goal: prove properties

• Sound abstraction for verification– properties of abstract system hold for

corresponding concrete system

: C A– if abstract state a satisfies property P then all

concrete states represented by a satisfy P





: C A a A if a P

then c C . (c)=a c P





: C A a A if a P


FalsificationFalsification

detect errors



• Sound abstraction for verification– errors of the abstract system exist in


: C A a A if a P



falsificationdetect errors



• Sound abstraction for verification– errors of the abstract system exist in


: C A a A if a P



falsificationdetect errors

c C . (c)=a c P

MotivationMotivation

• An abstraction that is sound for falsification need not be sound for verification.

• Existing frameworks for abstraction for verification – Modal Transition System (MTS)– MTS, PKS,KMTS - equivalent in expressive

power [ Godefroid,Jagadessan – VMCAI’03 ]

– can be too restrictive for falsification

Main ResultsMain Results

• New framework for abstraction – Ternary Modal Transition System (TMTS)– TMTS is stronger than MTS– Semantics of -calculus for TMTS

• Weak reachability– TMTS with parameterized transitions gives

tighter underapproximation– TMTS with assume-guarantee transitions for

complete reasoning

may

Modal Transition SystemsModal Transition Systems

underapproximation

overapproximation

Concrete Abstract

a

a’

total

a

a’

must

c. (c) = a c’ . (c’) = a’ c c’

MAY(a,a’)MAY(a,a’)

MUST+(a,a’)MUST+(a,a’)

MUSTMUST––(a,a’)(a,a’)

c, c’ . c c’ (c) = a (c’) = a’

(existential abstraction)

must may

underapproximation

c’. (c’) = a’ c. (c) = a c c’onto

a

a’

must

[ T. Ball - FMCO’04 ]

must maymust+ and must– are incomparable

TMTS strictly more expressive than MTSTMTS strictly more expressive than MTS

MTS • may and must+ transitions• precision preorder is logically characterized by PML

::= p | AX | |

TMTS• may, must+ and must– transitions• precision preorder is logically characterized by full-PML

::= p | AX | AY | |

• full-PML is strictly more expressive than PML [Pinter,Wolper - PODC’84] [Kupferman,Pnueli - LICS’95]

TMTS: what does it buy us?TMTS: what does it buy us?

• Verifying specifications with past operators

• Reasoning about specifications in falsification setting– must+ for verification and must- for falsification

• Tighter weak reachability in abstract system – combine must+ and must- along the path

Semantics of Semantics of -calculus for TMTS-calculus for TMTS

: C A• (C, c1)

• [ (A, a1) ] - the value of the -calculus formula in state a1 of TMTS A

• [ (A, a) ] = T – for all concrete state c with (c) = a, (C, c)

• [ (A, a) ] = T – there exists a concrete state c with (c) = a and (C, c)

• [ (A, a) ] = F– for all concrete state c with (c) = a, (C, c)

• [ (A, a) ] = F

– there exists a concrete state c with (c) = a and (C, c) • [ (A, a) ] = M

– there exist concrete states c and c’ such that

(c) = (c’) = a and (C, c) and (C, c’) • [ (A, a) ] =


Information Information LatticeLattice

T F

Truth Truth LatticeLattice

T

F


T F


T F

M

T

F

F

T

M


• [ (A, a) 1 2 ]

• [ (A, a) EX ]• [ (A, a) ]

[ (A, a) 1 2 ] =

[ (A, a) 1 ] # [ (A, a) 2 ]

6-valued Semantics of 6-valued Semantics of 11 22

# F F M T T

F F F F F F F

F F F F F F F

M F F ? F M F

T F F F ? T

T F F M ? T

F F F


# F F M T T

F F F F F F F

F F F F F F F

M F F ? F M F

T F F F ? T

T F F M T T

F F F


# F F M T T

F F F F F F F

F F F F F F F

M F F ? F M F

T F F F T

T F F M T T

F F F



T F


T F

M

T

F

F

T

M

# F F M T T

F F F F F F F

F F F F F F F

M F F ? F M F

T F F F T

T F F M T T

F F F


# F F M T T

F F F F F F F

F F F F F F F

M F F F F M F

T F F F T

T F F M T T

F F F


[ (A, a) EX ] =

Semantics of EXSemantics of EX

F if for all a’, if may(a,a’) then [(A, a’) ] = F

T if exists a’ s.t. must+(a,a’) and [(A,a’) ] = T

T if exists a’ s.t. must–(a,a’) and [(A,a’) ] T

otherwise

c’

a EX = T

a’

must–

= T

c

• [ (A, a) EX ] = T

• exists a’ s.t. must–(a,a’) and [(A,a’) ] = T

• exists c’ such that (c’)=a’ and c’ • for all c’ with (c’)=a’ there is c

with (c)=a such that cc’

if [ (A, a) EX ] = T then there exists c with (c) = a and c EX

EX

Semantics of Semantics of

• The semantics of PML operators is monotonic

– Least fixpoint operator can be computed by iterations from F is the usual way:

– [(A,a) Z . (Z) ] = [ (A, a) *(F) ]

• The 6-valued semantics is at least as precise as the standard 3-valued semantics of -calculus for MTS

• [(A,a) ] = – 3-valued abstraction refinement of must+ transitions

[Shoham,Grumberg – CAV’03] adapt for must-

• Hypermust transitions– [Larsen,Xinxin-LICS‘90] [Shoham,Grumberg – CAV’04]– adapt for must– – MTS with hypermust+ is incomparable with TMTS

EX(x>6) T EX(x>6) F EX(x>6) = T


EX(x>6) = ?

must –

x = 7x = 10

may

x > 6

x > 6

x:=x–3

7 8 9 ...

7 8 9 ...


• The 6-valued semantics is at least as precise as the standard 3-valued semantics of -calculus for MTS

• [(A,a) ] = – 3-valued abstraction refinement of must+ transitions

[Shoham,Grumberg – CAV’03] adapt for must-

• Hypermust transitions– [Larsen,Xinxin-LICS‘90] [Shoham,Grumberg – CAV’04]– adapt for must– – MTS with hypermust+ is incomparable with TMTS

Weak ReachabilityWeak Reachability

• a’ is weakly-reachable from a c, c’ . (c)=a (c’)=a’ c * c’

c

c’ a’

ainitial state

error state

error trace

Related to testing

L1: TF L0: FT L0: FF


L4: FT L4: FFL4: TF

x<6 x>7 (x=6)(x=7)

may

may must– must–

must–must–

L0: if x<6 then

L1: x:= x + 3

L2: if x > 7 then

L3: x :=x – 3

L4:

Predicates: (x < 6) (x > 7)

ExampleExample



L4: FT L4: FFL4: TF

x<6 x>7 (x=6)(x=7)

may

may must– must–

must–must–

L0: if x<6 then

L1: x:= x + 3

L2: if x > 7 then

L3: x :=x – 3

L4:


ExampleExample

x = 5

Underapproximation of Underapproximation of Weak ReachabilityWeak Reachability

• if [must+]*(a,a’) then a’ is weakly reachable from a

• Arbitrary combinations of must+ and must– transitions do not preserve weak reachability

• Find a tighter underapproximation of weak-reachability



L4: FT L4: FFL4: TF

x<6 x>7 (x=6)(x=7)

may

may must– must–

must–must–

L0: if x<6 then

L1: x:= x + 3

L2: if x > 7 then

L3: x :=x – 3

L4:


ExampleExample

must – ?must + ?

x = 9

x = 6

x = 5

x = 2

Underapproximation of Underapproximation of Weak ReachabilityWeak Reachability

• if [must+]*(a,a’) then a’ is weakly reachable from a

• Arbitrary combinations of must+ and must– transitions do not preserve weak reachability

• Find a tighter underapproximation of weak-reachability

ObservationsObservations

• a3 is weakly reachable from a1

if there exists a2 such that

must–(a1,a2) and must+(a2,a3)

• Onto nature of must– is preserved by [must-]*

• Total nature of must+ is preserved by [must+]*

a3

must+

a1

a2

must–

[T.Ball – FMCO’04]

UnderapproximationUnderapproximation

If there exists a1, a2, a3 such that

[must–]*(a1,a2) and

[must+]*(a2,a3)

then a3 is weakly-reachable from a1

a3

[must+]*

a1

a2

[must–]*

[T.Ball – FMCO’04]



L4: FT L4: FFL4: TF

x<6 x>7 (x=6)(x=7)

may

may must– must–

must–must–

L0: if x<6 then

L1: x:= x + 3

L2: if x > 7 then

L3: x :=x – 3

L4:


ExampleExample

a

a’

( total from a? )MUST+ ?MUST+ ?

( onto a’ ?)MUSTMUST– – ??

NONO

NONO

MAYMAY

Parameterized TransitionsParameterized Transitions

a

a’

must+()

total from

c. (c) = a c c’ . (c’) = a’ c c’

MUST+(MUST+())

Parameterized TransitionsParameterized Transitions

a

a’

must–()

MUSTMUST–(–())

c’. (c’) = a’ c’ c. (c) = a c c’

onto

if is TRUE then must+() is must+ and must–() is must–

ObservationObservation



– must–(1)(a1,a2)

– must+(2) (a2,a3)

– 1 2 a2 is satisfiable

a3

must+(2)

a1

a2

must–(1)

12

ObservationObservation



– must–(1)(a1,a2)

– must+(2) (a2,a3)

– 1 2 a2 is satisfiable

• Strongest parameters 1 and 2

a3

a1

a2

must–(1)

12

must+(2)

a

a’

s

MUST+ ( WP(s,a’) )MUST+ ( WP(s,a’) )

Strongest ParametersStrongest Parameters

Generated automatically as part of the construction of TMTS

c. (c) = a c c’ . (c’) = a’ c c’

if must+() then a ( WP(s,a’))

a

a’

s

MUSTMUST– – ( SP (s,a) )( SP (s,a) )

c’. (c’) = a’ c’ c. (c) = a c c’

if must–() then a ( SP(s,a))



L4: FT L4: FFL4: TF

x<6 x>7 (x=6)(x=7)

may

may must– must–

must–must–

L0: if x<6 then

L1: x:= x + 3

L2: if x > 7 then

L3: x :=x – 3

L4:


ExampleExample

SP(x:=x+3, x<6) = x < 9

WP(x:=x-3, x<6) = x < 9



L4: FT L4: FFL4: TF

x<6 x>7 (x=6)(x=7)

must– must–

must–must–

L0: if x<6 then

L1: x:= x + 3

L2: if x > 7 then

L3: x :=x – 3

L4:


ExampleExample

SP(x:=x+3, x<6) = x < 9

WP(x:=x-3, x<6) = x < 9

must–(x<9)

must+(x<9)

must– (x < 9)

must+ (x < 9)

Tighter UnderapproximationTighter Underapproximation

If there exists a1,...,a5 s.t.

[must–]*(a1,a2)

must–(1)(a2,a3)

must+(2) (a3,a4)

[must+]*(a4,a5)

1 2 a3 is satisfiable

then a5 is weakly-reachable from a1

a4

a2

a3

12

a5

a1

must+(2)

must–(1)

[must+]*

[must–]*

Complete Reasoning Complete Reasoning

– a’ is reachable by a certain sequence of abstract transitions from a

– a’ is weakly-reachable from a

• Assume-guarantee transitions– another type of parameterized transitions:

<> must+ <’>

a

a’

<>must+<‘ > c. (c) = a c

c’ . (c’) = a’ c’ ’ c c’

< < > MUST+ > MUST+ < < ’ ’ >>

Assume-Guarantee TransitionsAssume-Guarantee Transitions

’

Which and ’ predicates do we need?

’

a

a’

c’. (c’) = a’ c’ ’

c . (c) = a c c c’

< < > MUST> MUST–– < < ’ > ’ >

<>must–<‘ >

The idea...The idea...

33

3 3

is satisfiable

a4

a2

a3

a5

a1

s1

s2

s3

s4

<1>must– <2>

<2>must– <3>

1 = a1

2 = SP(s1, 1) a2

3 = SP(s2, 2) a3

<4>must+ < 5>

<3>must+ < 4>

3 = WP(s3,4) a3

4 = WP(s4,5) a4

5 = a5

Assume-guarantee transitionsAssume-guarantee transitions

• Complete Reasoning about Weak Reachability– a’ is reachable by a certain sequence of

assume-guarantee transitions from a– a’ is weakly-reachable from a

• Finding right parameters ~ computing loop invariants

Weak Reachability: SummaryWeak Reachability: Summary

[must–] * [must+]*must–(1) must+(2)

[must–] * [must+]*

• Previous work [T.Ball – FMCO’04]:

• Parameterized transitions

• Assume-guarantee transitions – complete reasoning

ApplicationsApplications

• Falsification of properties in CTL, LTL

• Abstraction-guided test generation– tighter underapproximation of weakly-

reachable states improves coverage of the generated tests

– example of QuickSort’s partition function

SummarySummary

• Ternary Modal Transition System (TMTS)– onto and total must transitions– full-PML logical characterizes precision

preorder on TMTS

• 6-valued semantics of -calculus for TMTS

• Tighten underapproximation of weak reachability with parameterized transitions– completeness result using assume-guarantee

transitions

Abstraction for Falsification

Documents

Transcript of Abstraction for Falsification