ITLTracer: Runtime Verification of Properties expressed in...

ITLTracer: Runtime Verification of Properties expressedin ITL

Dr. Helge Janicke ([email protected])

De Montfort University

Version: 906913f9b356(September 30, 2010)

[email protected] (DMU) ITLTracer 906913f9b356 1 / 29

[email protected]

Introduction

Runtime Verification

Definition (Runtime Verification)

Combines formal verification and Program Execution.

Detects faults by passively observing system behaviour during normaloperation.

Validates that the system under test (SuT) satisfies formallyexpressed requirements (e.g. in Interval Temporal Logic (ITL)).

Pros

Testing in the real world!

No (little) overhead duringdevelopment.

Reactive Systems.

Cons

Failure Recovery.

Performance overhead.

Not a substitute for rigoroustesting.


Introduction

Runtime Verification in Context

When is RuntimeVerification useful?

Information onlyavailable at run-time.

Dependent onenvironment (reactivesystem).

(security) criticalsystem.


Introduction

Problem Statement

Currently we use AnaTempura for Runtime Verification.

AnaTempura uses information obtained from the system via assertionpoints as input to Tempura which computes a trace. Properties arethen evaluated as state invariants1.

Tempura does not allow properties to contain negation of formulae,non-deterministic choice or implication, as these are not executable.(David’s Functional Programming implementation may be analternative!)

Properties, such as SANTA Policies, use some of these, which madetheir encoding cumbersome and lead to restrictions in the language.

1This including references to next or previous [email protected] (DMU) ITLTracer 906913f9b356 4 / 29

Introduction

Contribution and Applications

Semantics based on partitions instead of states.This allows for a more efficient evaluation of formulae containingsequential compositions (Chop).

Efficient encoding of partitions.Traces are stored in a compacted form saving space and access time.

Prototype of a Runtime Monitor (pure Java).See www.tech.dmu.ac.uk/~heljanic

Applications include:

Runtime Verification, where temporal assertions are needed.

SANTA Policy Enforcement.

Formal Log-File/ Attack analysis.

...


www.tech.dmu.ac.uk/~heljanic

Framework Overview

Motivation

Observation

Most programs are sequential, or contain large parts that are executedsequentially.

In a sequential program only few variables can change at a time(typically only one).

Depending on the interpretation of states this means that only veryfew variables are changed from one state to the next.

Storing all variable/value mappings for each state can be expensiveand it is more sensible to store only the changes.

This is what partitions do ...


Semantics

Definition (Partitioning)

Let a partitioning partx(σ) for a variable x ∈ Var be a nonempty, finitesequence of tuples (l0, r0), (l1, r1), ..., (ln, rn) where:

l0 = 0 start at 0

rn = |σ| end with length of sigma

∀0≤i≤n · li ≤ ri order

∀0≤i<n · ri + 1 = li+1 progression

∀0≤i≤n · ∀li≤t≤ri · σl(x) = σt(x) constant mapping

Example


Semantics



l0 = 0 start at 0





Example

grouping of states


Semantics



l0 = 0 start at 0





Example

partitioning


Semantics



l0 = 0 start at 0





Example

alternative partitioning


Semantics

Definition (Partition)

Let δx map from any given partition in the partition interval of partx(σ)and variable x ∈ Var to the set of values (Val).

δx : partx(σ) 7→ Val

such that: ∀l≤t≤r · δx(l , r) = σt(x)

Definition (Unrolling)

We define unrol(δx) as a function:

unrol(δx) = λi ·

if (l0 ≤ i ≤ r0 → δx(l0, r0)

[] (l1 ≤ i ≤ r1 → δx(l1, r1)

...

[] (lk ≤ i ≤ rk → δx(lk , rk)

fi


Semantics

Definition (Equivalence)

Two partitions are equivalent δx ≈ δ′x iff unrol(δx) = unrol(δ′x)

δx(0, 1) = 0

δx(2, 4) = 1

δx(5, 5) = 0

δ′x(0, 1) = 0

δ′x(2, 3) = 1

δ′x(4, 4) = 1

δ′x(5, 5) = 0

The partitions contain sufficient information to reconstruct the underlyingσ via unrolling. Equivalence with respect to a variable x is established bychecking if both underlying intervals σ and σ′ are identical w.r.t. thevariable x .


Semantics

Definition

Let the partition interval δ be defined as a tuple of all partitions.

δ = 〈δx0, δx1, ..., δxn〉

where xi ∈ Var . With the restriction that all partitions are of equal length:

∀xi∈Var · len(δx0) = len(δxi )

where len(δx) = r xn is the right index of the last tuple in the partitioning.We also use len(δ) to denote the length of the partition interval.

We refer to ∆x(σ) as the set of all possible partitionings for the variable x .Correspondingly to ∆ for the set of all possible partition intervals.


Semantics

The syntax below:

e ::= g(e1, ..., en) | fin v | ©v | v | µf ::= p(e1, ..., en) | f1 ; f2 | f1

∗ | skip | f1 ∧ f2 | ¬ f1


Semantics

Semantics of Expressions

The semantics of expressions is defined by induction over the syntax:

E [[v ]](δ) = δv (0, r v0 )

E [[g(e1, ..., en)]](δ) = g(E [[e1]](δ), ..., E [[en]](δ))

E [[ fin v ]](δ) = δv (lvn , rvn )

E [[©v ]](δ) =

χ(Val) if r vn = 0

δv (0, r v0 ) if r v0 > 0

δv (1, r v1 ) otherwise

E [[µ]](δ) = µ

Where lvi and r vi are the left element and the right element of the i ’s tuplein the partition partv (σ).


Semantics

Semantics of Formulae

The semantics of formulae is defined by induction over the syntax:

δ � f1 ; f2 iff ∃δ′ ∈ ∆ · δ′ ≈ δ and δ′ is aligned. And

δ′[0, k] � f1 and δ′[k , |δ′|] � f2 for some k , 0 ≤ k ≤ |δ′|δ � f1 ∧ f2 iff δ � f1 and δ � f2

δ � ¬ f iff σ 2 f

δ � skip iff len(δ) equal to 1

δ � empty iff len(δ) equal to 0

δ � p(e1, ..., en) iff p(E [[e1]](δ), ..., E [[en]](δ))

δ � f ∗ iff δ � empty or δ � f ∗ ; f

The complexity of the chop is now reduced from being proportional to theinterval length len(δ) to the number of partitions |δ|.


Tool-Support

Tool-Support

Pure Java implementation

javacc for parsing and rewriting.

Java interface as well as stdin option.

Evaluation of properties on demand.


Tool-Support

Keeping a TraceVariables

States

...

...

0 1 2 3 4 5

x

y

z

0 0 1 1 1 0

1 1 0 0 0 0

0 0 0 0 0 1

Figure: Trace

Variables

...

x

y

z

0@0 1@2 0@5

1@0 0@2

0@0 1@5

Figure: Partitioned Trace

Instead of recording the values of variables in each state only the changesare recorded. Implicitly a variable keeps its value if it is not changed. Theimplementation keeps record of rn the interval length an thus only needsto store the li values of the tuples together with the value as the all otherri ’s can be computed.


Tool-Support

UML Classdiagram: Trace Recording

DataStore<T>- state:long = 0L- variables:HashMap<String,VariableStore<T»+ add(id:String, value:T)+ advanceState(delta:int)+ get(state:long, id:String) : T+ getInv(state:long, id:String, inv:long[]):long[]+ getState():long

VariableStore<T>- data:List<DataEntry<T»- lastpivot:long+ (final) varid:String+ VariableStore(varid: String)+ add(state:long, value:T)+ get(state:Long):T+ getDataEntries():List<DataEntry<T»+ getEntry(state:Long):DataEntry<T>+ getEntryIndex(state:long):int+ getInv(state:long,inv:long[]):long[]

DataEntry+ (final) state : long+ (final) value : Object+ DataEntry(state: long, value: Object)

uk.ac.dmu.heljanic.satchk

0..n

containsID -> Variablemappings

containsstate -> value

mappings

Checker

Figure: Class Diagram


Tool-Support

Unrolling and Partitions

The unrol function is implemented by the methodget(state:long):T which returns the value for a given state byidentifying the corresponding partition using an adapted binary search.

The function uses a cursor, so that repeated calls for close statesresult in constant algorithmic complexity.

The function getInv(state:long, inv:long[]):long[]

implements the partitioning. For any state it returns thecorresponding partition element.

Data is recorded using the add(state:long, value:T) functionthat records the new variable value for the state.

The class DataStore collates all Variable stores and handles theidentifier mappings. It also is responsible for keeping the overallinterval length.


Tool-Support

Alignment

When evaluating a “chop”, the partitions must be aligned.The alignments are computed “on the fly”, ie. no restructuring of therecorded trace takes place. This is achieved through the efficientimplementation of the unrol function.Only those partitions of variables that are evaluated as part of the“chop” are aligned. The partitioning of other variables cannot affectthe outcome.

Example

true ; (x = 1 ∧ y = 3)[email protected] (DMU) ITLTracer 906913f9b356 19 / 29

Tool-Support

Tool Interface

ITLTracer can be used interactively, via the stdin/stdout interface. Thisallows ITLTracer to trace almost any program and is also useful for testingand verification of recorded traces (e.g. log analysis)

Starting the tool

$ java uk.ac.dmu.heljanic.itltracer.ITLTracer

This is ITLTracer $Revision: 906913f9b356 $ (c) 2010 H. Janicke

itlt:ESCAPE <escapecode> ... set the escape code for assertions.

default = itlt:

escape [x=v] [@s] ... records value v for variable x in state s.

Omitting x=v advance only the state to s.

Omitting @s records for the last set state.

itlt:EXIT ... quit.

itlt:PRINTSIGMA ... prints the recorded trace (sigma).

itlt:PRINTDELTA ... prints the recorded trace (delta).

itlt:CHECK <property> ... checks the <property> on the trace.


Examples

Recording a Trace

To record a trace, we set the escape sequence thatis recognised by ITLTracer (default “itlt:”). Anystring received on stdin that is prefixed by theescape sequence will be interpreted as an assertionpoint.

The example on the right is a sequence thatconstructs the interval used in the previousexamples.

Illegal inputs

Attempting to record a variable with a differentvalue in the same state or attempts to reduce thecurrent state will result in an exception.

Exampleitlt:ESCAPE

x=0

y=1

@2

x=1

y=2

y=3 @4

x=0 @5

y=7


Examples

Displaying Trace

ITLTrace can also output the trace tostdout in two formats:

itlt:PRINTDELTA This command

will output the partition interval δ.

itlt:PRINTSIGMA This commandwill reconstruct the underlying intervalσ using the unrol function.

Example (itlt:PRINTSIGMA)

SIGMA=

0 y= 1 x= 0

1 y= 1 x= 0

2 y= 2 x= 1

3 y= 2 x= 1

4 y= 3 x= 1

5 y= 7 x= 0

Example(itlt:PRINTDELTA)

DELTA=

DELTA(y)

(0, 1) -> 1

(2, 3) -> 2

(4, 4) -> 3

(5, 5) -> 7

DELTA(x)

(0, 1) -> 0

(2, 4) -> 1

(5, 5) -> 0


Examples

Checking a Property

ITLTracer can check a property using the check command:

itlt:CHECK some suffix {x=1 and y=2}ITLTracer only processes the primitive syntax and derived operators suchas “some suffix” (viz. ♦) are expanded during the parsing:

Example (Rewriting ♦f with true ; f (prefix notation))

rewriting DIAMOND node

IN = Diamond(And(Eq(Id[x], ILit[1]), Eq(Id[y], ILit[2])))

OUT = Chop(BLit[true], And(Eq(Id[x], ILit[1]), Eq(Id[y], ILit[2])))

parsed successfully

Once reduced to primitive operators the property can be evaluated againstthe trace.


Examples

Checking a Property (step by step)DELTA=

DELTA(y)

(0, 1) -> 1

(2, 3) -> 2

(4, 4) -> 3

(5, 5) -> 7

DELTA(x)

(0, 1) -> 0

(2, 4) -> 1

(5, 5) -> 0

Processing: l=0 r=5 f=Chop(BLit[true], And(Eq(Id[x], ILit[1]), Eq(Id[y], ILit[2])))

chop: k=5Processing: l=5 r=5 f=And(Eq(Id[x], ILit[1]), Eq(Id[y], ILit[2]))

Processing: l=5 r=5 f=Eq(Id[x], ILit[1])

Processing: l=5 r=5 f=Id[x]

return= 0 : [5, 5, 5, 5]

Processing: l=5 r=5 f=ILit[1]

return= 1 : [0, 5, 5, 5]

return= false : [5, 5, 5, 5]

return= false : [5, 5, 5, 5]

chop: k=4Processing: l=4 r=5 f=And(Eq(Id[x], ILit[1]), Eq(Id[y], ILit[2]))

Processing: l=4 r=5 f=Eq(Id[x], ILit[1])

Processing: l=4 r=5 f=Id[x]

return= 1 : [2, 4, 4, 5]

Processing: l=4 r=5 f=ILit[1]

return= 1 : [0, 5, 4, 5]

return= true : [2, 4, 4, 5]

Processing: l=4 r=5 f=Eq(Id[y], ILit[2])...

Evaluation is backwards and uses left and right indicies instead of partitionindicies, to avoid repartitioning. In this case a k in the partition (2,3)evaluates to true.


Conclusion and Future Work

Conclusions

ITLTracer checks ITL properties against a (recorded or streamed)system trace.

Overcomes (executable) restriction of AnaTempura, viz. a wider rangeof properties can be used.

Partitions can reduce algorithmic complexity, as evaluation of chop isO(|δ|) rather than O(|σ|).

Checks can be initiated by the target program. The checking ofproperties can be reduced to specific assertion points within thetarget program.

Prototype implementation is available atwww.tech.dmu.ac.uk/~heljanic


www.tech.dmu.ac.uk/~heljanic


Applications

Runtime Verification, where temporal properties are important.

Policy Based Management: History Based Access Control Policies canbe enforced. (Without refinement to executable form!)

Compliance Checking:

Lean Engineering, Standard Operations.Management Applications: Business Process Monitoring.Service Oriented Computing: Contract Monitoring.

Incident Response and Forensic Analysis: Checking for knownmalicious behaviours (properties) against log files (system trace)

Intrusion Detection Systems: Checking for known behaviour patterns(properties) in network traffic or access patterns (trace).

Trust in Collaborative Systems: Establishing trust based onbenevolent and malicious behaviour patterns.

Archaeology (Properties of Data Analysis Processes).



Future Work (I)

Reuse results from previous checks:

σ = σ0...σn σ′ = σ0...σnσn+1

Reuse known prefix results.

σ � f ; true =⇒ σ′ � f ; true

σ 2 f ; true and σ′ � f =⇒ σ′ � f ; true

σ 2 f ; true and σ′ 2 f =⇒ σ′ 2 f ; true

Reuse suffix results (state formulae only)

σ � true ; w =⇒ σ′ � true ; w

σ 2 true ; w and σn+1 � w =⇒ σ′ � true ; w

σ 2 true ; w and σn+1 2 w =⇒ σ′ 2 true ; w

By annotating the formulae with the last evaluation result andreference to δ (including the computed partitioning).



Future Work (II)

Extend supported Data Types (All immutable Java Objects can besupported).

Allow query calls to stored objects in the trace (query means thatcalling the function is free of side-effects)

Locking mechanism: Properties require only certain variables andbased on previous result evaluations part of the trace becomesirrelevant (ie. σ � f ; true =⇒ σ′ � f ; true) therefore any futurequery will return true, thus the record of σ can be released (if noother property requires this part of the trace).

Temporal Formulae compute partitions based on invariants (currentlyexpressed on σ). This will have to be converted to the new partitionsemantics (δ).

testing, publications...



Questions?

Any Comments or Questions?

Dr. Helge Janicke Software Technology Research Laboratory

Faculty of Technology

E: [email protected] De Montfort University

P: ++44 (0) 116 257 7617 Bede Island 1.16

W: www.tech.dmu.ac.uk/~heljanic LE1 9BH Leicester, UK


ITLTracer: Runtime Verification of Properties expressed in...

Documents

Transcript of ITLTracer: Runtime Verification of Properties expressed in...