Automatic Verification of Computer Programs
description
Transcript of Automatic Verification of Computer Programs
![Page 1: Automatic Verification of Computer Programs](https://reader035.fdocuments.us/reader035/viewer/2022062310/568166fa550346895ddb5d0f/html5/thumbnails/1.jpg)
Chair of Software Engineering
Automatic Verificationof Computer Programs
these slides contain advanced material and are optional
![Page 2: Automatic Verification of Computer Programs](https://reader035.fdocuments.us/reader035/viewer/2022062310/568166fa550346895ddb5d0f/html5/thumbnails/2.jpg)
2
What is verification
• Check correctness of the implementation given the specification
• Static verification– Check correctness without executing the program– E.g. static type systems, theorem provers
• Dynamic verification– Check correctness by executing the program– E.g. unit tests, automatic testing
• Automatic verification– Push-button verification
![Page 3: Automatic Verification of Computer Programs](https://reader035.fdocuments.us/reader035/viewer/2022062310/568166fa550346895ddb5d0f/html5/thumbnails/3.jpg)
3
Overview
• Verification is just one part of the process• All parts can (in theory) be automated
Specification &Implementation
Dynamic Verification
Static Verification
Fault Correction
Specification – Verification – Correction
![Page 4: Automatic Verification of Computer Programs](https://reader035.fdocuments.us/reader035/viewer/2022062310/568166fa550346895ddb5d0f/html5/thumbnails/4.jpg)
4
How to get the specification
• Need machine-readable specification for automatic verification (not just comments)
• Different variants:– Eiffel‘s „Design by Contract“
• Built-in contracts– .Net 4.0 „Code Contracts“
• Contracts implemented as a library– JML „Java Modeling Language“
• Dialect of Java featuring contracts as special comments– D „Contracts”
• Evolved from C++, built-in contracts
Specification – Verification – Correction
![Page 5: Automatic Verification of Computer Programs](https://reader035.fdocuments.us/reader035/viewer/2022062310/568166fa550346895ddb5d0f/html5/thumbnails/5.jpg)
5
Contracts in different languages
Specification – Verification – Correction
deposit (amount: INTEGER) require amount >= 0 do balance := balance + amount ensure balance = old balance + amount end
public void deposit(int amount){ Contract.Requires(amount >= 0); Contract.Ensures(balance == Contract.OldValue<int>(balance) + amount); balance += amount;}/*@
requires amount >= 0; ensures balance == \old(balance)+amount@*/public void deposit(int amount) { balance += amount}
function deposit(int amount)__in { assert(amount >= 0); int oldb = balance; }__out { assert(bal == oldb + amount); }__body { balance += amount}
D
CodeContracts
JML
Eiffel
![Page 6: Automatic Verification of Computer Programs](https://reader035.fdocuments.us/reader035/viewer/2022062310/568166fa550346895ddb5d0f/html5/thumbnails/6.jpg)
6
Writing full specifications
• Writing expressive specification is difficult• Specifying full effect of routines
– Describing what changes– Describing what does not change (frame condition)
Specification – Verification – Correction
put (v: G; i: INTEGER) require lower <= i and i <= upper ensure item (i) = v across lower |..| upper as j all j /= i implies item (j) = old item (j) end modifies area
old not allowed in across expressionmodifies not
expressible in Eiffel
![Page 7: Automatic Verification of Computer Programs](https://reader035.fdocuments.us/reader035/viewer/2022062310/568166fa550346895ddb5d0f/html5/thumbnails/7.jpg)
7
MML and EiffelBase2
• Model-based contracts use mathematical notions for expressing full specifications
Specification – Verification – Correction
put (v: G; i: INTEGER) -- Replace value at `i'. note modify: map require has_index (i) do at (i).put (v) ensure map |=| old map.updated (i, v) end
map: MML_MAP [INTEGER, G] -- Map of keys to values. note status: specification do create Result across Current as it loop Result := Result.updated (it.key, it.item) end end
note model: mapclass V_ARRAY [G]...end
![Page 8: Automatic Verification of Computer Programs](https://reader035.fdocuments.us/reader035/viewer/2022062310/568166fa550346895ddb5d0f/html5/thumbnails/8.jpg)
8
Contract inference
• Generate contracts based on implementation• Dynamic contract inference– Infer contracts based on program runs
• Static contract inference– Infer contracts without running the program
Specification – Verification – Correction
![Page 9: Automatic Verification of Computer Programs](https://reader035.fdocuments.us/reader035/viewer/2022062310/568166fa550346895ddb5d0f/html5/thumbnails/9.jpg)
9
Dynamic contract inference
• Location invariant – a property that always holds at a given point in the program
• Dynamic invariant inference – detecting location invariants from values observed during execution
• For pre- and postcondition inference, select routine entry and exit as program points
...x := 0... x = 0
Specification – Verification – Correction
![Page 10: Automatic Verification of Computer Programs](https://reader035.fdocuments.us/reader035/viewer/2022062310/568166fa550346895ddb5d0f/html5/thumbnails/10.jpg)
10
DAIKON example
• Uses templates for inferred contracts, e.g.
• Program point: ACCOUNT.deposit::ENTER• Variables of interest: balance, amount• Invariants:
balance = const
balance >= const
amount = const
amount >= const
balance = amount
• Samples
balance 0 amount 10
balance 10 amount 20
balance 30 amount 1
x = const x >= const x = y
0
0
10
101
Specification – Verification – Correction
![Page 11: Automatic Verification of Computer Programs](https://reader035.fdocuments.us/reader035/viewer/2022062310/568166fa550346895ddb5d0f/html5/thumbnails/11.jpg)
11
Static contract inference
• Infer precondition from postcondition/body– Weakest precondition calculus
• Infer loop invariants from postcondition– Generate mutations from postcondition
Specification – Verification – Correction
bubble_sort (a: ARRAY [T]) require a.count > 0 ensure sorted (a) permutation (a, old a)
from i := n until i = 1invariant 1 <= i <= n sorted (a[i+1..n]) permutation (a, old a)loop -- move the largest element -- in 1..i to position iend
Static analysis of program
Mutation from postcondition
Directly from postcondition
![Page 12: Automatic Verification of Computer Programs](https://reader035.fdocuments.us/reader035/viewer/2022062310/568166fa550346895ddb5d0f/html5/thumbnails/12.jpg)
12
Dynamic verification
• Check that program satisfies its specification by executing the program
• Manual– Write unit tests (xUnit framework)– Execute program and click around
• Automatic– Random testing
Specification – Verification – Correction
![Page 13: Automatic Verification of Computer Programs](https://reader035.fdocuments.us/reader035/viewer/2022062310/568166fa550346895ddb5d0f/html5/thumbnails/13.jpg)
13
Automatic testing with contracts
• Select routine under test• Precondition used for input validation– Test is valid if it passes precondition
• Postcondition used as test oracle– Test is successful if it passes postcondition
Specification – Verification – Correction
![Page 14: Automatic Verification of Computer Programs](https://reader035.fdocuments.us/reader035/viewer/2022062310/568166fa550346895ddb5d0f/html5/thumbnails/14.jpg)
14
Automatic testing with contracts
deposit (v: INTEGER) require v > 0 do balance := balance + v ensure balance = old balance + v end
Test Execution
Test Input
Test Oracle
Successful FailedPrecondition Test valid Test invalidBody (see postcondition) Error in programPostcondition Test succesful Error in program
![Page 15: Automatic Verification of Computer Programs](https://reader035.fdocuments.us/reader035/viewer/2022062310/568166fa550346895ddb5d0f/html5/thumbnails/15.jpg)
15
Random testing
• Create random objects– Call random creation procedure– Call random commands– For arguments, generate random input
• Basic types– Random numbers– Interesting values: max_value, 1, 0, -1, …
Specification – Verification – Correction
![Page 16: Automatic Verification of Computer Programs](https://reader035.fdocuments.us/reader035/viewer/2022062310/568166fa550346895ddb5d0f/html5/thumbnails/16.jpg)
16
AutoTest
• Basic operation:– Record sequence of calls made to create objects– Call routine under test with different objects– If execution is ok, this is a successful test case– If a postcondition is violated, this is a failing test case
• Improve test case generation– Smarter input selection
(e.g. use static analysis to select objects)– Test case minimization (removing unnecessary calls)– Build object pool– …
Specification – Verification – Correction
![Page 17: Automatic Verification of Computer Programs](https://reader035.fdocuments.us/reader035/viewer/2022062310/568166fa550346895ddb5d0f/html5/thumbnails/17.jpg)
17
Static verification
• Need a model of the programming language– What is the effect of an instruction
• Translate program to a mathematical representation
• Use an automatic or interactive theorem prover to check that specification is satisfied in every possible execution
Specification – Verification – Correction
![Page 18: Automatic Verification of Computer Programs](https://reader035.fdocuments.us/reader035/viewer/2022062310/568166fa550346895ddb5d0f/html5/thumbnails/18.jpg)
18
AutoProof process
• Translates AST from EiffelStudio to Boogie• Uses Boogie verifier to check Boogie files• Traces verification errors back to Eiffel source
EiffelStudio AutoProof Boogie
EiffelAST
BoogieFile
BoogieErrors
EiffelErrors
Specification – Verification – Correction
![Page 19: Automatic Verification of Computer Programs](https://reader035.fdocuments.us/reader035/viewer/2022062310/568166fa550346895ddb5d0f/html5/thumbnails/19.jpg)
19
AutoProof translation
implementation APPLICATION.make { var a;entry: havoc a; assume (a!= Void) && (!Heap[a, $allocated]); Heap[a, $allocated] := true; Heap[a, $type] := ACCOUNT; call create.ACCOUNT.make(a); assert Heap[a, ACCOUNT.balance] = 0;}
make local a: ACCOUNT do create a.make check a.balance = 0 end end
Specification – Verification – Correction
![Page 20: Automatic Verification of Computer Programs](https://reader035.fdocuments.us/reader035/viewer/2022062310/568166fa550346895ddb5d0f/html5/thumbnails/20.jpg)
20
Automatic Fault Correction
• Build a test suite– Manual or automatic
• Find and localize faults– Failing test cases– Static analysis
• Try fixes– Apply fix templates with random code changes
• Validate fixes– Run test suite again, now all tests have to pass
Specification – Verification – Correction
![Page 21: Automatic Verification of Computer Programs](https://reader035.fdocuments.us/reader035/viewer/2022062310/568166fa550346895ddb5d0f/html5/thumbnails/21.jpg)
21
AutoFix: model-based localization
• Abstract state as boolean queries• Find differences between passing and failing tests
Specification – Verification – Correction
move_item (v: G) -- from TWO_WAY_SORTED_SET. -- Move `v' to the left of cursor. require v /= Void ; has (v) local idx: INTEGER ; found: BOOLEAN do idx := index from start until found or after loop found := (v = item) if not found then forth end end remove go_i_th (idx) put_left (v) end
not is_emptynot beforenot afternot isfirst1not is_empty
not before not after isfirst2not is_empty
not beforenot after sorted3 not is_empty
beforenot after1not is_empty
beforenot aftersorted2Invar. from failing
not is_emptybeforenot after…
Invar. from passing
not is_emptynot beforenot after…
…
0 1 count-1 count count+1
![Page 22: Automatic Verification of Computer Programs](https://reader035.fdocuments.us/reader035/viewer/2022062310/568166fa550346895ddb5d0f/html5/thumbnails/22.jpg)
22Specification – Verification – Correction
AutoFix: instantiating fixes
• Fix schema for common fixes
if fail_condition thenfixing_action
elseoriginal_instruction
end
if fail_condition thenfixing_action
endoriginal_instruction
if before then forthendput_left(v)
Instantiatemove_item (v: G) require v /= Void ; has (v) local idx: INTEGER ; found: BOOLEAN do idx := index from start until found or after loop found := (v = item) if not found then forth end end remove go_i_th (idx) put_left (v) end
![Page 23: Automatic Verification of Computer Programs](https://reader035.fdocuments.us/reader035/viewer/2022062310/568166fa550346895ddb5d0f/html5/thumbnails/23.jpg)
23
Demo
• AutoTest• AutoProof• AutoFix
Specification – Verification – Correction
![Page 24: Automatic Verification of Computer Programs](https://reader035.fdocuments.us/reader035/viewer/2022062310/568166fa550346895ddb5d0f/html5/thumbnails/24.jpg)
24
Eiffel Verification Environment (EVE)
• Research branch of EiffelStudio• Integrates most tools developed by us– AutoTest (dynamic verification)– AutoProof (static verification)– AutoFix (fault correction)– AutoInfer (dynamic contract inference)– MultiStar (static verification)– AliasAnalysis (static analysis)
• Other tools currently not integrated– CITADEL (dynamic contract inference)– gin-pink (static loop invariant inference)
![Page 25: Automatic Verification of Computer Programs](https://reader035.fdocuments.us/reader035/viewer/2022062310/568166fa550346895ddb5d0f/html5/thumbnails/25.jpg)
25
Putting It All Together
EVE AutoTestAutoProof MultiStar
ElementStatically Verified
AutoFix
Manual Proof
CITADEL AutoInfer
Manual Fixes
Element Dynamically
Verified
proof failed
tests ok
no fix found
fix found
tests failed
tests ok
proof ok no new contracts
new contracts
proof ok
AliasAnalysis gin-pink
static inference
![Page 26: Automatic Verification of Computer Programs](https://reader035.fdocuments.us/reader035/viewer/2022062310/568166fa550346895ddb5d0f/html5/thumbnails/26.jpg)
26
References• EVE: Eiffel Verification Environment
http://se.inf.ethz.ch/research/eve/• AutoTest, AutoProof, AutoFix, CITADEL, …
http://se.inf.ethz.ch/research/• CodeContracts
http://research.microsoft.com/en-us/projects/contracts/• Java Modeling Language (JML)
http://www.cs.ucf.edu/~leavens/JML/• D Programming Language
http://dlang.org/• Daikon
http://groups.csail.mit.edu/pag/daikon/• Boogie Verifier
http://boogie.codeplex.com/