1 Program verification -- 096229 Ofer Strichman Room 412 [email protected] Office hours: after...
-
Upload
prosper-scott -
Category
Documents
-
view
227 -
download
2
Transcript of 1 Program verification -- 096229 Ofer Strichman Room 412 [email protected] Office hours: after...
![Page 2: 1 Program verification -- 096229 Ofer Strichman Room 412 ofers@ie.technion.ac.il Office hours: after class.](https://reader036.fdocuments.us/reader036/viewer/2022062300/56649cf65503460f949c5184/html5/thumbnails/2.jpg)
2
Agenda for the first class
Up to slide 36: what is this course about ?
Slide 37: why not take this course Slide 38-39: why take this course Slide 40: requirements
![Page 3: 1 Program verification -- 096229 Ofer Strichman Room 412 ofers@ie.technion.ac.il Office hours: after class.](https://reader036.fdocuments.us/reader036/viewer/2022062300/56649cf65503460f949c5184/html5/thumbnails/3.jpg)
3
The goal – reliable software systems
Importance of reliable software – almost needless to mention.
A nice collection of famous bugs:http://en.wikipedia.org/wiki/Software_bugs
![Page 4: 1 Program verification -- 096229 Ofer Strichman Room 412 ofers@ie.technion.ac.il Office hours: after class.](https://reader036.fdocuments.us/reader036/viewer/2022062300/56649cf65503460f949c5184/html5/thumbnails/4.jpg)
4
The goal – reliable software systems
Here are some famous ones: ESA Ariane 5 Flight 501 self-destruction 40
seconds after takeoff (June 4, 1996). A conversion from 64-bit floating point to 16 bit
integer with a value larger than possible with Arian 4. The overflow caused a hardware trap.
The 2003 North America blackout was triggered by a local outage that went undetetected.
A race condition in General Electric’s monitoring software prevented an alarm.
![Page 5: 1 Program verification -- 096229 Ofer Strichman Room 412 ofers@ie.technion.ac.il Office hours: after class.](https://reader036.fdocuments.us/reader036/viewer/2022062300/56649cf65503460f949c5184/html5/thumbnails/5.jpg)
5
The goal – reliable software systems
Here are some famous ones: The MIM-104 Patriot bug, which resulted in the
deaths of 28 Americans in Dharan, Saudi Arabia (February 25, 1991).
The radar classifies detections according to a trajectory model it builds in real time. (“in x mSec the missile should be in position y”)
Due to rounding of the clock values, it accumulates inaccuracies. After several hours this inaccuracy is critical.
The Pentium bug Incorrect floating-point division. Cost Intel ~ $400,000,000
![Page 6: 1 Program verification -- 096229 Ofer Strichman Room 412 ofers@ie.technion.ac.il Office hours: after class.](https://reader036.fdocuments.us/reader036/viewer/2022062300/56649cf65503460f949c5184/html5/thumbnails/6.jpg)
6
The goal – reliable software systems
Not unusual to have more than 50% of resources allocated to testing
Testing and verification are (becoming) the bottleneck of development
![Page 7: 1 Program verification -- 096229 Ofer Strichman Room 412 ofers@ie.technion.ac.il Office hours: after class.](https://reader036.fdocuments.us/reader036/viewer/2022062300/56649cf65503460f949c5184/html5/thumbnails/7.jpg)
bEgInSlIdE
7
Remarks by Bill Gates17th Annual ACM Conference on Object-Oriented
Programming, Seattle, Washington, November 8, 2002
“… When you look at a big commercial software
company like Microsoft, there's actually as much
testing that goes in as development. We have as
many testers as we have developers. Testers
basically test all the time, and developers basically
are involved in the testing process about half the
time…
![Page 8: 1 Program verification -- 096229 Ofer Strichman Room 412 ofers@ie.technion.ac.il Office hours: after class.](https://reader036.fdocuments.us/reader036/viewer/2022062300/56649cf65503460f949c5184/html5/thumbnails/8.jpg)
bEgInSlIdE
8
Remarks by Bill Gates17th Annual ACM Conference on Object-Oriented
Programming, Seattle, Washington, November 8, 2002
“… We've probably changed the industry we're in.
We're not in the software industry; we're in the
testing industry, and writing the software is the
thing that keeps us busy doing all that testing.”
“…The test cases are unbelievably expensive; in
fact, there's more lines of code in the test harness
than there is in the program itself. Often that's a
ratio of about three to one.”
![Page 9: 1 Program verification -- 096229 Ofer Strichman Room 412 ofers@ie.technion.ac.il Office hours: after class.](https://reader036.fdocuments.us/reader036/viewer/2022062300/56649cf65503460f949c5184/html5/thumbnails/9.jpg)
9
The goal – reliable software systems
Quality dilemma: quality / features / time More efficient methods for test and
verification are always needed No ‘silver-bullet’ when it comes to testing. Formal verification is on the rise...
![Page 10: 1 Program verification -- 096229 Ofer Strichman Room 412 ofers@ie.technion.ac.il Office hours: after class.](https://reader036.fdocuments.us/reader036/viewer/2022062300/56649cf65503460f949c5184/html5/thumbnails/10.jpg)
10
Formal methods
Formal = based on rigorous mathematical logic concepts. It is ‘machine-readable’, and hence can be
used by a verification algorithm. Once we formally specify what we expect
from the program, we can try to prove that the program satisfies the specification.
![Page 11: 1 Program verification -- 096229 Ofer Strichman Room 412 ofers@ie.technion.ac.il Office hours: after class.](https://reader036.fdocuments.us/reader036/viewer/2022062300/56649cf65503460f949c5184/html5/thumbnails/11.jpg)
11
Local assertions: at this point in the program always x > y. How is it different than writing ‘assert ( x > y);’ ?
A more global property: The array bound is never exceeded
Temporal properties:If a request is sent, acknowledgement is eventually received
Specification can be: Informal, textual, visual...
![Page 12: 1 Program verification -- 096229 Ofer Strichman Room 412 ofers@ie.technion.ac.il Office hours: after class.](https://reader036.fdocuments.us/reader036/viewer/2022062300/56649cf65503460f949c5184/html5/thumbnails/12.jpg)
12
... or it can be formal
(1· x · 5 U x=7) Æ G (x¸ 0)
(Translated from: The value of x will be between 1 and 5, until at some point it will become 7, and it will always (“Globally”) be positive.)
1· x · 5
x=7
x¸ 0
![Page 13: 1 Program verification -- 096229 Ofer Strichman Room 412 ofers@ie.technion.ac.il Office hours: after class.](https://reader036.fdocuments.us/reader036/viewer/2022062300/56649cf65503460f949c5184/html5/thumbnails/13.jpg)
13
A system can be general and messy...
struct node *copy_node(struct node *f) {
static int count = 0;
struct node *temp;
count ++;
if (f == NULL) return NULL;
if ((temp = (struct node *) malloc (sizeof (struct node))) == NULL)
fprintf(stderr,"Out of memory");
temp -> tag = f -> tag;
temp -> name = strdup(f -> name);
temp -> left = copy_node(f -> left);
temp -> right = copy_node(f -> right);
temp -> cnf_id = f -> cnf_id;
return temp;
}
![Page 14: 1 Program verification -- 096229 Ofer Strichman Room 412 ofers@ie.technion.ac.il Office hours: after class.](https://reader036.fdocuments.us/reader036/viewer/2022062300/56649cf65503460f949c5184/html5/thumbnails/14.jpg)
14
...or its underlying algorithm can be modeled
E.g., as a finite-state machine
s1 s3s2pull
release
release
extended malfunction
![Page 15: 1 Program verification -- 096229 Ofer Strichman Room 412 ofers@ie.technion.ac.il Office hours: after class.](https://reader036.fdocuments.us/reader036/viewer/2022062300/56649cf65503460f949c5184/html5/thumbnails/15.jpg)
15
The program can be tested...
Try inputs x1 = 2, x2 = 25, str = ‘llk’ x1 = 0, x2 = 31, str = ‘aaa’ ...
... As good as the set of input test-cases ... and as good as the reference system
Manual inspection Regression testing Specification-based testing
For simple, non-critical code, this is the easiest and most immediate option.
![Page 16: 1 Program verification -- 096229 Ofer Strichman Room 412 ofers@ie.technion.ac.il Office hours: after class.](https://reader036.fdocuments.us/reader036/viewer/2022062300/56649cf65503460f949c5184/html5/thumbnails/16.jpg)
16
...or can be formally verified
Given the formal specification G(:malfunction) // always not
malfunctioning
And the system model
Run a model-checker program to formally verify that the system satisfies the specification.
s1 s3s2pull
release
releaseextended malfunction
![Page 17: 1 Program verification -- 096229 Ofer Strichman Room 412 ofers@ie.technion.ac.il Office hours: after class.](https://reader036.fdocuments.us/reader036/viewer/2022062300/56649cf65503460f949c5184/html5/thumbnails/17.jpg)
bEgInSlIdE
17
Remarks by Bill Gates17th Annual ACM Conference on Object-Oriented
Programming, Seattle, Washington, November 8, 2002
“We also went back and say, OK, what is the state-of-the-art in terms of being able to prove programs, prove that a program behaves in a certain way? This is the kind of problem that has been out there for many decades...
...When I dropped out of Harvard, this was an interesting problem that was being worked on, and actually at the time I thought, well, if I go and start a company it will be too bad, I'll miss some of the big breakthroughs in proving software programs because I'll be off writing payroll checks. “
![Page 18: 1 Program verification -- 096229 Ofer Strichman Room 412 ofers@ie.technion.ac.il Office hours: after class.](https://reader036.fdocuments.us/reader036/viewer/2022062300/56649cf65503460f949c5184/html5/thumbnails/18.jpg)
bEgInSlIdE
18
Remarks by Bill Gates17th Annual ACM Conference on Object-Oriented
Programming, Seattle, Washington, November 8, 2002
...and it turns out I didn't miss all that much in terms of rapid progress which now has become, I'd say, in a sense, a very urgent problem. And although a full general solution to that is in some ways unachievable, for many very interesting properties this idea of proof has come a long way in helping us quite substantially just in the last year.
We call the system that does this kind of proof -- a model-checking system.
![Page 19: 1 Program verification -- 096229 Ofer Strichman Room 412 ofers@ie.technion.ac.il Office hours: after class.](https://reader036.fdocuments.us/reader036/viewer/2022062300/56649cf65503460f949c5184/html5/thumbnails/19.jpg)
19
Model-checking
Input: A finite state machine M A temporal property P P
Output: Does M satisfy P ?
M
P = “never in state 11”
11
0001 10
![Page 20: 1 Program verification -- 096229 Ofer Strichman Room 412 ofers@ie.technion.ac.il Office hours: after class.](https://reader036.fdocuments.us/reader036/viewer/2022062300/56649cf65503460f949c5184/html5/thumbnails/20.jpg)
20
Model-checking
Nondeterminism is used to model systems with inputs
Model-checking is exponential in the number of inputs
M
11
00 1001 Minputs
outp
uts=
![Page 21: 1 Program verification -- 096229 Ofer Strichman Room 412 ofers@ie.technion.ac.il Office hours: after class.](https://reader036.fdocuments.us/reader036/viewer/2022062300/56649cf65503460f949c5184/html5/thumbnails/21.jpg)
21
Model-checking
The model M defines a set of behaviors (traces) L(M)
The model P defines a set of allowed behaviors L(P)
M
11
00 1001
P
Never “11”
11
![Page 22: 1 Program verification -- 096229 Ofer Strichman Room 412 ofers@ie.technion.ac.il Office hours: after class.](https://reader036.fdocuments.us/reader036/viewer/2022062300/56649cf65503460f949c5184/html5/thumbnails/22.jpg)
22
The model checking problem: L(M) µ L(P)
Q: What if this problem is too hard in practice?
Model-checking as language-inclusion
L(M)
L(P)error
?
![Page 23: 1 Program verification -- 096229 Ofer Strichman Room 412 ofers@ie.technion.ac.il Office hours: after class.](https://reader036.fdocuments.us/reader036/viewer/2022062300/56649cf65503460f949c5184/html5/thumbnails/23.jpg)
23
Program verification
Bill: Prove that the program is correct !
MS-employee: let me quote Carl Popper’s answer about verification vs. falsification in science. Science does not progress by finding eternal
truths... ... only by refuting previous theories.
![Page 24: 1 Program verification -- 096229 Ofer Strichman Room 412 ofers@ie.technion.ac.il Office hours: after class.](https://reader036.fdocuments.us/reader036/viewer/2022062300/56649cf65503460f949c5184/html5/thumbnails/24.jpg)
24
Program verification
Bill: Ok, So just prove that the program satisfies the specification !
MS-employee : ok ... but even if I prove that the system
satisfies the specification, there can still be an error in the proof process or the proof tool that I use...
... and there is no guarantee that the specification fully specifies what we consider as ‘correct’.
![Page 25: 1 Program verification -- 096229 Ofer Strichman Room 412 ofers@ie.technion.ac.il Office hours: after class.](https://reader036.fdocuments.us/reader036/viewer/2022062300/56649cf65503460f949c5184/html5/thumbnails/25.jpg)
25
Program verification
Bill: Ok, fine, so just increase the probability that the program is correct !
MS-employee : Well, boss, if you want this process to be
automatic, please provide me a model of the program, and a formal (machine-readable) specification...
... Such that checking the model with respect to the specification is a decidable problem.
![Page 26: 1 Program verification -- 096229 Ofer Strichman Room 412 ofers@ie.technion.ac.il Office hours: after class.](https://reader036.fdocuments.us/reader036/viewer/2022062300/56649cf65503460f949c5184/html5/thumbnails/26.jpg)
26
Program verification
Bill: Will it be better than testing ? MS-employee: when it is possible, it is
much better.
![Page 27: 1 Program verification -- 096229 Ofer Strichman Room 412 ofers@ie.technion.ac.il Office hours: after class.](https://reader036.fdocuments.us/reader036/viewer/2022062300/56649cf65503460f949c5184/html5/thumbnails/27.jpg)
27
Testing / Formal Verification
A very crude dichotomy:
In practice:Many types of testing, Many types of formal verification.
Testing
Correct with respect to the set of test inputs, and reference system
Easy to perform
Dynamic
![Page 28: 1 Program verification -- 096229 Ofer Strichman Room 412 ofers@ie.technion.ac.il Office hours: after class.](https://reader036.fdocuments.us/reader036/viewer/2022062300/56649cf65503460f949c5184/html5/thumbnails/28.jpg)
28
Testing / Formal Verification
TestingFormal Verification
Correct with respect to the set of test inputs, and reference system
Correct with respect to all inputs, with respect to a formal specification
Easy to performDecidability problems, Computational problems ,
DynamicStatic
A very crude dichotomy:
In practice:Many types of testing, Many types of formal verification.
![Page 29: 1 Program verification -- 096229 Ofer Strichman Room 412 ofers@ie.technion.ac.il Office hours: after class.](https://reader036.fdocuments.us/reader036/viewer/2022062300/56649cf65503460f949c5184/html5/thumbnails/29.jpg)
29
Testing + formal-verification
Bill: is there a mid-way ? MS-employee: Yes: Specify formally, verify
informally
This is called run-time verification... The idea is to write a small program that
monitors the tested program It checks during run-time that the tested
program satisfies a given (Formal) specification.
![Page 30: 1 Program verification -- 096229 Ofer Strichman Room 412 ofers@ie.technion.ac.il Office hours: after class.](https://reader036.fdocuments.us/reader036/viewer/2022062300/56649cf65503460f949c5184/html5/thumbnails/30.jpg)
30
Where do we start?
Bill: ok, where do we start ? MS-employee: with some basics of Logics Bill: Why Logic ? MS-employee: I will show you an example.
![Page 31: 1 Program verification -- 096229 Ofer Strichman Room 412 ofers@ie.technion.ac.il Office hours: after class.](https://reader036.fdocuments.us/reader036/viewer/2022062300/56649cf65503460f949c5184/html5/thumbnails/31.jpg)
31
Reminder: some Boolean connectives
Æ and Ç or : not ! implies $ equal
A propositional formula: :(a Ç (b ! (c Æ : a)))
![Page 32: 1 Program verification -- 096229 Ofer Strichman Room 412 ofers@ie.technion.ac.il Office hours: after class.](https://reader036.fdocuments.us/reader036/viewer/2022062300/56649cf65503460f949c5184/html5/thumbnails/32.jpg)
32
Program transformation
if(!a && !b) h();
else if(!a) g();
else f();
if(a) f();
else if(b) g();
else h();
*
if(a) f();
else {
if(!b) h();
else g(); }
(
How can we check that these two versions are equivalent?
+
if(!a) {
if(!b) h();
else g(); }
else f();
![Page 33: 1 Program verification -- 096229 Ofer Strichman Room 412 ofers@ie.technion.ac.il Office hours: after class.](https://reader036.fdocuments.us/reader036/viewer/2022062300/56649cf65503460f949c5184/html5/thumbnails/33.jpg)
33
Code transformation example – cont’d
Represent procedures as independent Boolean variablesoriginal := optimized :=
if :a Æ b then h if a then f
else if :a then g else if b then g
else f else h
Transform if-then-else chains into Boolean formulas
transform(if x then y else z) ≡ (x Æ y) Ç (:x Æ z)
Check equivalence of boolean formulaetransform(original) , transform(optimized)
![Page 34: 1 Program verification -- 096229 Ofer Strichman Room 412 ofers@ie.technion.ac.il Office hours: after class.](https://reader036.fdocuments.us/reader036/viewer/2022062300/56649cf65503460f949c5184/html5/thumbnails/34.jpg)
34
Code transformation example – cont’d
original ≡ if :a Æ :b then h else if :a then g else h
≡ (:a Æ :b) Æ h Ç :(:a Æ :b) Æ if :a then g else f
≡ (:a Æ :b) Æ h Ç :(:a Æ :b) Æ (:a Æ g Ç a Æ f )
optimized ≡ if a then f else if b then g else h
≡ (a Æ f ) Ç ( :a Æ if b then g else h )
≡ ( a Æ f ) Ç ( :a Æ (b Æ g Ç :bÆ h) )
Verification Condition : ((:a Æ :b) Æ h Ç :(:a Æ :b) Æ (:a Æ g Ç a Æ f )) $
(a Æ f ) Ç (:a Æ (b Æ g Ç :b Æ h))
The question: is valid ?
![Page 35: 1 Program verification -- 096229 Ofer Strichman Room 412 ofers@ie.technion.ac.il Office hours: after class.](https://reader036.fdocuments.us/reader036/viewer/2022062300/56649cf65503460f949c5184/html5/thumbnails/35.jpg)
35
Not just software...
ÆÆ
Ç
ÇÇ
Prove the equivalence of two circuits:
![Page 36: 1 Program verification -- 096229 Ofer Strichman Room 412 ofers@ie.technion.ac.il Office hours: after class.](https://reader036.fdocuments.us/reader036/viewer/2022062300/56649cf65503460f949c5184/html5/thumbnails/36.jpg)
36
So...
We will learn: A brief introduction to Propositional Logic
Incl. General logic jargon such as: validity, satisfiability, soundness, completeness,...
In a later stage in the course we will also learn algorithms for checking whether a given formula is valid/satisfiable
Temporal Logic, and its ability to formally specify systems
![Page 37: 1 Program verification -- 096229 Ofer Strichman Room 412 ofers@ie.technion.ac.il Office hours: after class.](https://reader036.fdocuments.us/reader036/viewer/2022062300/56649cf65503460f949c5184/html5/thumbnails/37.jpg)
37
So...
... and Specification-based testing (specifically: Run-
time verification) Other types of testing
... And also Modeling of systems Formal verification of systems against formal
specification with model-checking. More... if we have the time
![Page 38: 1 Program verification -- 096229 Ofer Strichman Room 412 ofers@ie.technion.ac.il Office hours: after class.](https://reader036.fdocuments.us/reader036/viewer/2022062300/56649cf65503460f949c5184/html5/thumbnails/38.jpg)
38
Why not take this course
In the end of the course you will NOT be able to automatically verify the big software you wrote in C
No one can, (completely) automatically. After your Ph.D, you might be able to do it with some
manual guidance.
On the bright side ... with certain restrictions, or compromises on what it means to ‘verify’, you might be able to verify even large C programs.
![Page 39: 1 Program verification -- 096229 Ofer Strichman Room 412 ofers@ie.technion.ac.il Office hours: after class.](https://reader036.fdocuments.us/reader036/viewer/2022062300/56649cf65503460f949c5184/html5/thumbnails/39.jpg)
39
Why take this course
There are some restricted type of programs that can be verified automatically.
Finite-state programs (e.g. protocols) and more. The ‘killer-application’: hardware circuits.
There is an industry behind it, incl. in Haifa. Developers of verification tools: IBM, Intel. Users of such tools: IBM, Intel, Melanox, Freescale, Verisity
(now Cadence), Galileo (now Marvel)...
Background in hardware is not necessary at all.
Most developers & users in this field do not have it.
![Page 40: 1 Program verification -- 096229 Ofer Strichman Room 412 ofers@ie.technion.ac.il Office hours: after class.](https://reader036.fdocuments.us/reader036/viewer/2022062300/56649cf65503460f949c5184/html5/thumbnails/40.jpg)
40
Why take this course (cont’d)
Very different than other courses you took...
Combines Theory (logic, automata-theory) Algorithms Acquaintance with type of programs and
systems you have probably not seen before.
Not difficult... Only assumes ‘mathematical maturity’,
not specific knowledge.
![Page 41: 1 Program verification -- 096229 Ofer Strichman Room 412 ofers@ie.technion.ac.il Office hours: after class.](https://reader036.fdocuments.us/reader036/viewer/2022062300/56649cf65503460f949c5184/html5/thumbnails/41.jpg)
41
Requirements
~5 homework assignments 2 exams, 3-questions each (1.5 hours).