Finite State Verification for Software Systems

Finite State Verification for Software Systems

Lori A. ClarkeUniversity of Massachusetts

Laboratory for Advanced Software Engineering Researchhttp://laser.cs.umass.edu/

Other contributors: G. Avrunin, J. Cobleigh, M. Dwyer,

G. Naumovich, and L. Osterweil

Testing can:

Uncover failures Show specifications are not met for

specific test cases Be an indication of overall reliability

cannot: Prove that a program will/will not

behave in a particular way

Distributed Systems Better performance, better flexibilty,

but there is a cost distributed systems are more difficult

to test than sequential systems number of execution paths can grow

exponentially with the number of tasks Testing can not even demonstrate that a

system works on the selected/executed test data

Formal Verification: An Alternative to Testing

Theorem Proving Prove properties about all possible executions Use mathematical reasoning Difficult and error prone

Finite State Verification Prove properties about all possible executions Reason about a finite abstract, model of the

system not as powerful as theorem proving

Almost a totally automated process

Finite State Verification: State of the Art Reachability Based Approaches

Symbolic Model Checking -- SPIN or SMV Exponential worst-case bound

Integer Linear Constraints INCA Exponential worst-case bound

Data Flow Analysis FLAVERS Polynomial worst-case bound

Property

System

Property Translator

SystemTranslator

ReasoningEngine

TFG

ConsistentFSA

Ada, Java, C++, Jovial

Inconsistent+

counter example

Event alphabet

Architecture of FLAVERS

FLAVERS: Where is the magic?

System Model Primarily event-based, not state-based Reason about sequences of events Does not enumerate all reachable states

Includes all “relevant” executions Usually over-approximates relevant executable

behaviors Model is conservative wrt a property

Incrementally refine model as needed add state information that is demonstrated to be needed

3

4

2

9

T1 T2

5

7

8

1 61,6

<0,0> 1,6

<1,0> 1,6

<1,1> 1,6

<0,1>

2,6 <1,0>

2,6 <0,1>

2,6 <1,1>

2,6 <0,1>

start

......... ...

Enumerating the State Space

3

4

2

9

T1 T2

5

7

1,6

2,6

5,9

3,6

4

8

1,71 6

2,7 1,8

3,7 2,8

3,8

e1

e2

e3

e1

e1e2

e3

Enumerating the event space

e1e2

e2

3

4

2

9

T1 T2

5

7

8

1 6e1

e2

e3

4

9

T1 T2

5

8

1e1

e2

e3

FLAVERS model of the system

Modeling the System: FLAVERS Automatically creates the program

model from the source code Instead of the state space, explicitly

represents interleaved execution via edges Compute MHP

Optimize the representation Model only events of interest

Conservative with respect to a property

Some comments on the model Works well if the events of interest are

relatively sparse in the overall system E.g., method invocations, task interaction Inter-procedural analysis done via in-lining

An imprecise, but conservative, representation Too imprecise for deadlock

Seems to greatly reduce the analysis cost...

Disclaimer

Property

System

Property Translator

SystemTranslator

ReasoningEngine

TFG

ConsistentFSA


Inconsistent+

counter example

Event alphabet

Architecture of FLAVERS

Example Property FSA

The elevator always closes its doors before moving

close,open,move

0

1

openclose

2

move

closemove

open

Reasoning engine: State Propagation States of the property are propagated

through the TFG The property is proved if only accepting

(non-accepting) states are contained in the final node of the TFG overall complexity is O(N2S)

N is the size of the model of the system S is the number of states guaranteed to terminate

Elevator Controller

void main(){…

1,2,4: if (elevatorStopped){...

3: openDoors();}...


7: closeDoors();}

9: moveToNextFloor();}

1: if

3: open

7: close

5: if

9: move

7,9

Elevator Controller

Worklist: 3, 5,

1: if

3: open

7: close

5: if

9: move

close

0

1

openclose

2move

closemove

open

Property

openmove

<0>

<1>

<0,1>

<0>

<0,2>

7,9

Elevator Controller

void main(){…


3: openDoors();}...


7: closeDoors();}


Worklist: 3, 5,

1: if

3: open

7: close

5: if

9: move

close

0

1

openclose

2move

closemove

open

Property

openmove

<0>

<1>

<0,1>

<0>

<0,2>

Conservative Analysis if a property is found to be valid, it

is definitely true if a property is found to be invalid

An invalid trace may correspond to an error

The invalid traces do not correspond to executable behavior

Property

System

Property/Constraint Translator

SystemTranslator

ReasoningEngine

TFG

ConsistentFSA


Inconsistent+

counter example

Event alphabet

Revised Architecture of FLAVERS

Constraints

Elevator Controller

void main(){…


3: openDoors();}...


7: closeDoors();}


1: if

3: open

7: close

5: if

9: move

6: S==true 8: S==false

4: S==false2: S==true

Example constraint for a boolean variable

== is a predicate

= is assignment

viol

S==trueS=true

S==trueS=true

S==true

S==falseS=false

S==false

S==trueS=true

S==falseS=false

S==falseS=false

S=false

S=true

true false

unknown

Worklist: 2, 4, 3,

6 ,8,5,

State Propagation

1: if

3: open

7: close

5: if

9: move



<0,0>

<0,1>

<1,1>

<0,2>

<1,1>,<0,2>

<1,1>,<0,v>

0

21

viol

S==true S==false

S==true

S==true

S==false

S==false

S==falseS==true

Constraint

close

0

1

openclose

2move

closemove

open

Property

openmove

Worklist: 2, 4, 3,

6 ,8,5,

7,9

State Propagation

1: if

3: open

7: close

5: if

9: move



<0,0>

<0,1>

<1,1>

<0,2>

<1,1>,<0,2>

<1,1>,<0,v>

0

21

viol

S==true S==false

S==true

S==true

S==false

S==false

S==falseS==true

Constraint

close

0

1

openclose

2move

closemove

open

Property

openmove

Worklist: 2, 4, 3, 5, 6, 8, 7Worklist: 2, 4, 3, 5, 6, 8, 7, 9

State Propagation

1: if

3: open

7: close

5: if

9: move



<0,0>

<0,1>

<1,1>

<0,2>

<1,1>,<0,2>

<1,1>,<0,v>

<0,1>

<1,v>,<0,2>

<0,1>,<0,2>

0

21

viol

S==true S==false

S==true

S==true

S==false

S==false

S==falseS==true

Constraint

close

0

1

openclose

2move

closemove

open

Property

openmove

Constraints Used to add semantic information Can model many different kinds of

information Variables or flow of control Missing components Environment Users

Can provide automated assistance

Evaluating FLAVERS

Experimental Evaluation Case studies:

Communication Protocols: 3-way handshake and alternating bit

demonstrated the importance of modeling the environment

Architectural Design written in Wright demonstrated importance of doing analysis early

Industrial demonstrations: SAIC: ADS Command Instruction Sets, HLA Northrop: B2 Jovial systems MCC: Quest project, C++ systems

Need a paradigm shift! Software systems are increasingly used

in critical applications Medicine Transportation Communication Finance

Cannot continue to do business as usual Testing as the prime validation technique is

costly and ineffective

New life to an old vision Significant validation throughout the

software lifecycle: architectural design low level design coding debugging Maintenance

Finite state verification is an extremely general approach

Early fault detection Requirement specs ==> properties Design specs ==> properties

properties X system representations ==> early feedback

Future Plans Difficult to predict performance

For any FSV technique For FLAVERS: Adding constraints increases

the worst-case bound, but may (or may not) improve performance

Need more automated support Analysis to help select additional

information to be modeled Abstraction techniques for modeling

Future Work Explore alternative state propagation

algorithms E.g., Initial verification vs re-verification

Improve counter-example generation Short paths User guidance

Better support for specifying properties Build on property pattern specifications

FSA templates combined with Disciplined Natural Language

Action may occur zero times.

FSA template:

action response

action

response

DNL template:

response or(action,response)

Multiple occurrences of action eventually result in only one occurrence of response.

Response cannot occur before the first action occurs.

(action,response)

repetition phrase


The Repetition phrase describes whether or not the property is repeatable.

This property is repeatable.

This property is not repeatable.

action

repetition phrase



Response Property


response

response

Competed FSA template…completed FSA

…and a DNL template.…and its DNL representation of your property.

action

action

response


Multiple occurrences of action eventually result in only one occurrence of response.

Response cannot occur before the first action occurs.

(action,response)


action


Multiple occurrences of action eventually result in only one occurrence of response. Action may occur zero times. Response cannot occur before the first action occurs. This property is repeatable.

Instantiated Response Property

Future Work Software composition techniques

Contractual composition Integration with testing Better approaches for dealing with

dynamism Full lifecycle support Support for Java Experimental Evaluation

Conclusions

FLAVERS is effective at verifying a wide range of interesting properties

Very general approach that can be applied to any event flow model

Much easier to use than theorem proving based techniques Can be mostly automated Hides the “formal” in formal methods

Is it easy enough?

Finite State Verification for Software Systems

Documents

Transcript of Finite State Verification for Software Systems