Apply Program Model Checking Qian Ling Oct. 28, 2004 Bell Labs Research China.

Lucent TechnologiesLucent TechnologiesBell Labs InnovationsBell Labs Innovations

Apply Program Model CheckingApply Program Model Checking

Qian LingQian Ling

Oct. 28, 2004Oct. 28, 2004

Bell Labs Research ChinaBell Labs Research China

2


AgendaAgenda

• MotivationMotivation• Program Model CheckingProgram Model Checking• Program Model Checking ToolsProgram Model Checking Tools

– The Spin ToolThe Spin Tool– The Verisoft ToolThe Verisoft Tool– The SLAM ToolThe SLAM Tool

• Use Spin tool to check TTS ServerUse Spin tool to check TTS Server• Future WorkFuture Work

3


MotivationMotivation

• Testing of concurrent programsTesting of concurrent programs– VMS TTS Server ProgramVMS TTS Server Program– OS Adapter ProjectOS Adapter Project– POSIX thread impl. on VxWorks 5.4POSIX thread impl. on VxWorks 5.4

• Progress in Program Model CheckingProgress in Program Model Checking– New tech and tools developedNew tech and tools developed

4


AgendaAgenda




5


SW Quality Control TechSW Quality Control Tech

• Software TestingSoftware Testing– Unit/integration/system testUnit/integration/system test– Functional test, Conformance testFunctional test, Conformance test– Stress testStress test– API test, etcAPI test, etc

• Code ReviewCode Review– Peer ReviewPeer Review– Design ReviewDesign Review

• Program Model CheckingProgram Model Checking– Formal MethodFormal Method– Software VerificationSoftware Verification

• Software Quality AssuranceSoftware Quality Assurance

6


Software TestingSoftware Testing

Unit A

{.......}

Unit B

{........}

Unit C

{...........}Unit TestIntegration

Test

System Test“Sequential, Stand-alone”

7


Code ReviewCode Review

do{ if (select(FD_SETSIZE, &read_fds, NULL, NULL, &TimeOut) != -1) { if (FD_ISSET(sock, &read_fds)) { amount = read(sock, buf, TTS_RECEIVE_BUFFER_LENGTH);

..............

else { /*Timeout waiting for socket*/ iRet = TTS_ERROR_TIMEOUT; break; }

} while ((amount >= 0) && (!bAllDone));

>

“Static”

8


Facing Concurrent & Facing Concurrent & Distributed SoftwareDistributed Software

It is not too surprising that thebehavior of even non-buggy distributed applications can easilydefy our human reasoning skills.

Gerard J. Holzmann

9


Facing Concurrent & Facing Concurrent & Distributed Software, Distributed Software, contcont

• DeadlockDeadlock• Livelock, starvationLivelock, starvation• UnderspecificationUnderspecification

– unexpected reception of messagesunexpected reception of messages

• OverspecificationOverspecification– Dead codeDead code

• Violations of constraintsViolations of constraints– Buffer overrunsBuffer overruns– Array bounds violationsArray bounds violations

• Assumptions about speedAssumptions about speed– Logical correctness vs.real-time performanceLogical correctness vs.real-time performance

Really Hard IssueReally Hard Issue

10


What is Model Checking?What is Model Checking?

• [Clarke & Emerson 1981][Clarke & Emerson 1981] “ “Model checking is an automated technique that, given a Model checking is an automated technique that, given a

finite-state model of a system and a logical property, finite-state model of a system and a logical property, systematically checks whether this property holds for (a systematically checks whether this property holds for (a given initial state in) that model.”given initial state in) that model.”

• Model checking tools automatically Model checking tools automatically verify whether verify whether M |= φ M |= φ HoldsHolds

where M is a (finite-state) model of a system and where M is a (finite-state) model of a system and property φ is stated in some formal notationproperty φ is stated in some formal notation

• Problem: Problem: state space explosionstate space explosion• SPINSPIN is one of the most powerful model is one of the most powerful model

checkerchecker

11


Classic v.s. Modern MCClassic v.s. Modern MC

WaterfallSW Lifecycle

Process

12


Classic Model CheckingClassic Model Checking

13


Modern Model CheckingModern Model Checking

• Abstraction: To fight with explosionAbstraction: To fight with explosion• Focus on Modern Model CheckingFocus on Modern Model Checking

14


AgendaAgenda




15


Tools for program model Tools for program model CheckingChecking

• SPINSPIN– Gerard J. Holzmann (Bell-Labs), 1991~ 1991~– Proving the correctness of process interaction, Proving the correctness of process interaction,

asynchronized controlasynchronized control

• VerisoftVerisoft– Patrice Godefroid (Bell Labs), 1996~Patrice Godefroid (Bell Labs), 1996~– Runtime analyses for multi-process programsRuntime analyses for multi-process programs

• SLAMSLAM– Thomas Ball (MS Research), 1999~Thomas Ball (MS Research), 1999~– Static analyses for “API Usage Rules” in programsStatic analyses for “API Usage Rules” in programs

16


Tools for program model Tools for program model CheckingChecking, cont, cont

• EraserEraser– Stefan Savage (University of Washington) & DEC, 1997~Stefan Savage (University of Washington) & DEC, 1997~– Dynamic detecting data-race in lock-based multithread Dynamic detecting data-race in lock-based multithread

programsprograms

• FLAVERSFLAVERS– Matthew B. Dwyer, L. A. Clarke (University of Kansas Matthew B. Dwyer, L. A. Clarke (University of Kansas

State/Massachusetts Amherst) &DoD, DAAH, NSF, State/Massachusetts Amherst) &DoD, DAAH, NSF, IBM… , 1999~IBM… , 1999~

– Check if all executions adhere to specification, data flow Check if all executions adhere to specification, data flow analysis, stackanalysis, stack

• SMVSMV– CMU, 1992~CMU, 1992~– Symbolic model checkerSymbolic model checker

• ……

17


Tool ComparisonTool Comparison

• Modeling LanguagesModeling Languages• Static Analyze v.s. DynamicStatic Analyze v.s. Dynamic• Explicit State space?Explicit State space?• Scope of verificationScope of verification

– Safety onlySafety only– Acceptation cycle, etcAcceptation cycle, etc

• Stages in developmentStages in development– Design phaseDesign phase– Code ReadyCode Ready– Incremental developmentIncremental development

18


AgendaAgenda


– The Spin Tool The Spin Tool – The Verisoft ToolThe Verisoft Tool– The SLAM ToolThe SLAM Tool


19


The SPIN Model CheckerThe SPIN Model Checker

• SPIN = SPIN = SSimple imple PPromela romela InInterpreterterpreter– is a tool for analysing the logical is a tool for analysing the logical

consistency of concurrent systems, consistency of concurrent systems, specifically of data communication specifically of data communication protocols.protocols.

– Concurrent systems are described in the Concurrent systems are described in the modelling language called Promela.modelling language called Promela.

– state-of-the-art model checkerstate-of-the-art model checker

20


PromelaPromela

• Promela Promela = = ProProtocol/tocol/ProProcess cess MeMeta ta LaLanguagenguage– allows for the dynamic creation of concurrent allows for the dynamic creation of concurrent

processes.processes.– communication via message channels can be communication via message channels can be

defined to bedefined to be• synchronous (i.e. rendezvous), orsynchronous (i.e. rendezvous), or• asynchronous (i.e. buffered).asynchronous (i.e. buffered).

– resembles the programming language Cresembles the programming language C– specification language to model finite-state specification language to model finite-state

systemssystems

21


Modern Model Checking with Modern Model Checking with SPINSPIN

Program Source Code(in C/C++,Java, LISP...)

PREMELA Model

Program Errors

Mapping,Abstraction

Verification

Specification

LTL(Linear Temporal Logic)

Formulate

22


Foundation of SPINFoundation of SPIN

• Based on Based on automata theoretic modelautomata theoretic model– System ModelSystem Model

• Translate process tmpl to automatonTranslate process tmpl to automaton• State space=product of all automataState space=product of all automataGlobal behavior modeled by an automatonGlobal behavior modeled by an automaton

– SpecificationSpecification• Correctness requirement in LTL formulaCorrectness requirement in LTL formula• Convert LTL to Büchi automatonConvert LTL to Büchi automaton

– VerificationVerification• global automatonglobal automaton× × Büchi automatonBüchi automaton

– (Accepted language = empty) (Accepted language = empty) not satisfiednot satisfied– (Accepted language ‡ empty) (Accepted language ‡ empty) satisfied behaviorssatisfied behaviors

• SPIN formalize the erroneous behaviorsSPIN formalize the erroneous behaviors– Prove such behavior are impossibleProve such behavior are impossible– Provide detail matched behaviorsProvide detail matched behaviors

Vardi & Wolper 83Vardi & Wolper 83

23


Algorithms of SPINAlgorithms of SPIN

• Nested Depth-First Search Nested Depth-First Search (Cycles detection)(Cycles detection)– 11stst search: from initial state, find accepting states search: from initial state, find accepting states– 22ndnd search (nested): from all accepting states search (nested): from all accepting states

• LTL to Büchi automatonLTL to Büchi automaton– Normalize LTL formulasNormalize LTL formulasrecursive conversionrecursive conversion– Make use of Promela condition statement Make use of Promela condition statement

• Partial Order ReductionPartial Order Reduction– Represent classes of execution sequence with one single Represent classes of execution sequence with one single

path, staticallypath, statically• Memory ManagementMemory Management

– State Compression State Compression (60%~80% reduction)(60%~80% reduction)• Separate local state (small) and global stateSeparate local state (small) and global state• Global state use index to refer to local stateGlobal state use index to refer to local state

– Bit-State Hashing Bit-State Hashing (Improve problem coverage (Improve problem coverage 1)1)• 2 bits are used to store a reachable state2 bits are used to store a reachable state• 2 bits: F1(S1) = M[m1], F2(S2)=M[m2]2 bits: F1(S1) = M[m1], F2(S2)=M[m2]• Collision problemCollision problem

Problem coverage = M / (N*S)

M=2^m: memory space in bytes

N=2^n: true # of Reachable states

S=2^s: bytes for each state

24


Debug with SPINDebug with SPIN

• Correctness Requirement:Correctness Requirement:– Inline assertionsInline assertions

• assertassert(…)(…)– End-state label / deadlockEnd-state label / deadlock

• endendxxxxxx::– Progress-state label / livelockProgress-state label / livelock

• progressprogressxxxxxx::– Acceptance-state labelAcceptance-state label

• acceptacceptxxxxxx::– LTL propertiesLTL properties

• Make use of LTL specificationMake use of LTL specification– Never-claimsNever-claims

• never {…}never {…}

Commonly Used

Commonly Used

25


Extensions to SPIN (1)Extensions to SPIN (1)

• dSpin = dSpin = ddynamic ynamic SPINSPIN– R. Iosif & R. Sisto, Kanas State University, 1999~R. Iosif & R. Sisto, Kanas State University, 1999~– Solve Limitations in SPIN, Solve Limitations in SPIN,

• Object creation & deletionObject creation & deletion• Pointer and reference variablesPointer and reference variables• Polymorphic function callsPolymorphic function calls

– SolutionsSolutions• Memory managementMemory management

– Dynamic memory allocationDynamic memory allocation– Reference mechanismReference mechanism

• Functional managementFunctional management– Function declaration, call and referenceFunction declaration, call and reference– Local scopingLocal scoping

26



• FeaVer/Modex = FeaVer/Modex = ModModel el ExExtractor for Ctractor for C– Gerard J. Holzmann, Bell Labs, 1998~Gerard J. Holzmann, Bell Labs, 1998~– Extract Promela model from ANSI CExtract Promela model from ANSI C– 3 phases:3 phases:

• Parsing: Parsing: C C Full Parse Tree Full Parse Tree• Interpretation: Interpretation: Tabled AbstractionTabled Abstraction• OptimizationOptimization

– Applications:Applications:• Lucent PathStar™ Access ServerLucent PathStar™ Access Server

27



• JPF1/JPF2 = JPF1/JPF2 = JJava ava PPath ath FFinderinder– Automated Software Engineering Group, NASAAutomated Software Engineering Group, NASA– JPF1, since 1998JPF1, since 1998

• A translator from Java 1.0 to PromelaA translator from Java 1.0 to Promela• Suitable for multi-threaded Java programsSuitable for multi-threaded Java programs• ApplicationsApplications

– Find deadlocks in Game ServerFind deadlocks in Game Server– Analyze Remote AgentAnalyze Remote Agent

– JPF2, since 2000JPF2, since 2000• Work on Java Byte Code, 100% Java supportWork on Java Byte Code, 100% Java support• Garbage CollectionGarbage Collection• Static analysis dependencies for partial-order reductionStatic analysis dependencies for partial-order reduction• Contain runtime race-cond analysis by “Eraser” AlgorithmContain runtime race-cond analysis by “Eraser” Algorithm• Use Bandera to calculate slicesUse Bandera to calculate slices

28



• Bandera ToolsetBandera Toolset– Santos Laboratory, since 2000Santos Laboratory, since 2000– source code source code FSM model checker, SPIN/SMV FSM model checker, SPIN/SMV

• Analyze, Analyze, • Abstraction Abstraction • TransformationTransformation

– More than just a SPIN extension!More than just a SPIN extension!

29


AgendaAgenda




30


VeriSoftVeriSoft

• A tool for systematically testing A tool for systematically testing concurrent reactive softwareconcurrent reactive software

• Stateless search, for:Stateless search, for:– DeadlocksDeadlocks– LivelocksLivelocks– Assertion violationsAssertion violations– DivergenceDivergence

• Replay Error ScenariosReplay Error Scenarios

• Applications:Applications:– 4ESS Heart-Beat Monitor 4ESS Heart-Beat Monitor

debugging and unit debugging and unit testingtesting

– WaveStar 40G R4 WaveStar 40G R4 integration and system integration and system testingtesting

– 7R/E PacketStar Gateway 7R/E PacketStar Gateway Solutions Feature Server Solutions Feature Server unit testingunit testing

– CDMA Call Processing CDMA Call Processing Library testingLibrary testing

VeriSoft

BA C

System Processes

deadlock

VeriSoft

BA C

System Processes

VeriSoft

BA C

System Processes

deadlockdeadlock

31


AgendaAgenda




32


SLAMSLAM

• Check Temporal safety property in sequential Check Temporal safety property in sequential C programsC programs

• Based on Boolean ProgramBased on Boolean Program• Solutions:Solutions:

– Preparation:Preparation:• Model conterexample in SLICModel conterexample in SLIC• C C BP (C2BP) BP (C2BP)

– Verification:Verification:• BEBOP: perform reachability analysis of BPBEBOP: perform reachability analysis of BP• NEWTON: refine BP by discover additional predictsNEWTON: refine BP by discover additional predicts

– Error in BP Error in BP Error in P Error in P

• ApplicationsApplications– API in Windows XP device driversAPI in Windows XP device drivers

33


AgendaAgenda




34


VMS TTS ServerVMS TTS Server

• Multi-threadedMulti-threaded– 1 server thread1 server thread– 1~10 tts working threads1~10 tts working threads– 1~10 IBM viavoice synthesis threads1~10 IBM viavoice synthesis threads

• Asynchronzed Message QAsynchronzed Message Q– server server working threads working threads

• Synchronized Synchronized – synthesis threads synthesis threads working threads working threads

IBM V_V Engine

working thread 1

working thread 10

Socket ServerThread

socketreq

...........

35


VMS TTS ServerVMS TTS Server, cont, cont

• A Sequence DiagramA Sequence Diagram

36


PurposePurpose

• Check safety propertiesCheck safety properties– potential deadlockspotential deadlocks– Unexpected end statesUnexpected end states– Unreachable codeUnreachable code

• Check if it works for 10 V V instancesCheck if it works for 10 V V instances– Currently we have only 1 instanceCurrently we have only 1 instance

• Check about the Msg LossCheck about the Msg Loss

37


Result in SimulationResult in Simulation

• No Error found in program (simulation)No Error found in program (simulation)– 1 bug found in promela model1 bug found in promela model

• Deadlock in server threadDeadlock in server thread• Forget to add lock when access globals in abstr.Forget to add lock when access globals in abstr.

– There are un-reached codeThere are un-reached code– Abstraction is not a easy workAbstraction is not a easy work

• Programs works for 10 threadsPrograms works for 10 threads• There are message lossThere are message loss

– Under high stressUnder high stress• Every thread are full loadedEvery thread are full loaded

– Increase # of buffers Increase # of buffers fewer loss fewer loss– Increase # of instance Increase # of instance fewer loss fewer loss– Decrease synthesis delayDecrease synthesis delay

• As for performanceAs for performance– Simulation is not trustableSimulation is not trustable

38


Result in PANResult in PAN

39


Restrictions, LessonsRestrictions, Lessons

• No Timer in PromelaNo Timer in Promela– Use loop to simulate a timerUse loop to simulate a timer– Different behavior in simulation & PANDifferent behavior in simulation & PAN

• Hard to locate an errorHard to locate an error– Too many messagesToo many messages– Generally, subtle errors occur after quite a Generally, subtle errors occur after quite a

long timelong time

• Platform dependant APIs are hard to be Platform dependant APIs are hard to be abstractedabstracted– What is the size of the Win32 thread What is the size of the Win32 thread

message queue?message queue?

40


AgendaAgenda




41


Future WorkFuture Work

• More applicationsMore applications– OS Adapter ProjectOS Adapter Project– POSIX threadPOSIX thread– ……

• Tool enhancementsTool enhancements– Locate bug easier ?Locate bug easier ?– Timer ?Timer ?

• Automata Automata LTL ? LTL ?

42


Reference SiteReference Site

• SPINSPIN– http://spinroot.com/spin/whatispin.htmlhttp://spinroot.com/spin/whatispin.html

• VerisoftVerisoft– http://cm.bell-labs.com/who/god/verisoft/http://cm.bell-labs.com/who/god/verisoft/

• SLAMSLAM– http://research.microsoft.com/slam/http://research.microsoft.com/slam/


Suggestions!Suggestions!

Thanks! &Thanks! &

Apply Program Model Checking Qian Ling Oct. 28, 2004 Bell Labs Research China.

Documents

Transcript of Apply Program Model Checking Qian Ling Oct. 28, 2004 Bell Labs Research China.