Inferring and checking system rules by static analysis William R Wright.

Inferring and checking system rules by static analysis

William R Wright

CS 297: Security and Programming Languages2

Material reviewed…

Checking System Rules Using System-Specific, Programmer-Written Compiler Extensions. Dawson Engler, et al. OSDI 2000.

RacerX: Effective, Static Detection of Race Conditions and Deadlocks. Dawson Engler, et al. SOSP 2003.

Bugs as Deviant Behavior: A General Approach to Inferring Errors in Systems Code. OSDI 2000.


System Rules

Systems follow their own unique set of design based “correctness” rules

Such rules go beyond “no dereferencing of NULL pointers


Example – System rule

Temporal ordering: b must always follow a


Checking of such rules

These rules are often unchecked.

For example, suppose I am required to issue conn.close() but forget to do so.

Code compiles even though I broke a system “rule”.


How to check system wide rules?

Although compilers observe general language semantics, they are ignorant of the rules unique to systems.

Static analysis can apply systems rules.


Checking systems Rules

ESC enables some checking via annotations (even automating annotations via Houdini).

Vault – very manual.

SLAM – labeled “promising” - next presentation!


Proposed method

The proposed method complements those efforts.

Goal is to extract ad hoc rules from source code, requiring minimal effort.

Also, to provide an extensible framework to define and check systems rules.


Meta-Level compilation (MC)

One may define a systems rule via metal, a “high level, state-machine” language.


Details about metal

A rule defined in metal is called a State Machine (SM).

Once so defined, we compile the rule(s) with mcc, a metal compiler, and dynamically link the result into xgcc (based on GNU gcc).


Details about metal (cont’d)

When compiling the code be analyzed, xgcc outputs errors based on deviations from the metal rules.

Notice that modifications to source are unnecessary. If one there is a bug fix, one can easily recompile with the compiler of choice.


Sample Metal rule templates

With metal one may define systems rules such as:– “Never/always do X”– “Always do X before/after Y”– “In situation X, do (not do) Y”– “In situation X, do Y rather than Z”


Example – metal rule(from Deviant paper)


Example: …more problem code (Extensions paper)


Example (cont’d)

metal rule can find the bug - 6 lines of code– (Extensions, Fig. 1, Section 3.1)– Finds deviations by looking fo

Functions to look for– Disable interrupts: cli()– Re-enable interrupts: sti() or

restore_flags(flags) [restores to original interrupt status when paired with save_flags(flags)]


Inferring deviant behavior

Suppose you were just hired to get uptime from 98% to 99.999% on an airline reservation system with 5,000,000 lines of code.

You know little about the system. Those who do ask you to help with the debugging.

The newspaper reports that “Software problems” caused your employer’s latest disaster.


Inferring deviant behavior (cont’d)

The programming team has already used their knowledge of the system and as much static analysis as they could come up with.

What if you could automate an examination of the source that results in a set of Metal rules that reflect the ad hoc (undocumented) behavior that the system should follow?


Inference Method

Assume that action taken by code is there to accomplish something.

Divide actions into inferences about programmer “beliefs”.


MUST beliefs

Beliefs then go in two categories:

– MUST beliefs:

if (p==null){System.out.println(

“The pointer is” + p + “.”);

}Programmer expresses belief that p is null inside the block,

then contradicts the belief.


MAY beliefs

– MAY beliefs: back to original example

– Since we see stmt.close after stmt.executeQuery, maybe this is a system rule.


Beliefs – discovering

One may derive possible MAY beliefs by:

– Traversing the program, observing all actions that happen in tandem.

– Assuming that they MUST happen in that way.– In a second pass, applying those assumptions.– Initiate a statistical analysis of the results (errors).


Statistical analysis

One may quickly rule out many MAY beliefs when finding that they are rarely or infrequently followed. These are “coincidences”, not MUST beliefs.


Statistical analysis - detail

Sort the errors by the z statistic:

Which essentially measures the degree to which a MAY belief is supported by its incidence (contradictions accounted for) in the code sample.

One examines errors from the most likely to the least, stopping when the effort becomes counterproductive.


Results

Coverity, commercialized these methods Look at http://scan.coverity.com/ “Number of defects Fixed (since 3/6/2006):

6131. Targeted gcc, Samba, Linux-2.6, Perl, PHP,

OpenSSL, and apparently plenty of “private” code.

Inferring and checking system rules by static analysis William R Wright.

Documents

Transcript of Inferring and checking system rules by static analysis William R Wright.