Path-aware Time-triggered Runtime Verification

Samaneh Navabpour1, Borzoo Bonakdarpour2, Sebastian Fischmeister1

1Department of Electrical and Computer Engineering2School of Computer Science

University of Waterloo

Runtime Verification

2

Program

Observer

VerifierObservation

Report

Steering

Event-triggered

Runtime Verification Framework

Event-triggered Monitoring

• Jittery overhead

• Bursts of invocations of the observer

3

Undesirable transient overload situations in time-sensitive systems

1 B. Bonakdarpour, S. Navabpour, and S. Fischmeister, “Sampling-based Runtime Verification”, 88-102, FM’11

Alternative Monitoring Approach

• Goals for the monitoring approach:

• Predictable monitoring

• Bounded overhead at each intervention

4

Candidate solution:Time-triggered Monitoring

Problem of Time-triggered Monitoring

5

L1 L2 L3 L4 L5 L6 L7 L8 L9 L10 L11 L12 L13 L14 L15 L16 L17

m

Program Execution

Monitor

L18 L19

Sample from monitor

Critical instruction

Sampling period = 2

m……

• Achieving sound state reconstruction1

1 B. Bonakdarpour, S. Navabpour, and S. Fischmeister, “Sampling-based Runtime Verification”, 88-102, FM’11

Longest Sampling Period (LSP)

6

C1

6

1 1

A2..4

4

D5

C2

7

1

C3

8

C4

9

1

1

B1

10

B2

11

1

1

LSP = 1

• Longest Sampling Period (LSP)1:• is the minimum shortest path between

two critical nodes

1. fib(int n) { 2. int i, Fnew, Fold, temp,ans; 3. Fnew = 1; Fold = 0; 4. i = 2; 5. while( i <= n ) { 6. temp = Fnew; 7.* Fnew = Fnew + Fold; 8.* Fold = temp; 9. i++; } 10.* ans = Fnew; 11. return ans;}

Problem:

redundant samples excessive overhead

1 B. Bonakdarpour, S. Navabpour, and S. Fischmeister, “Sampling-based Runtime Verification”, 88-102, FM’11

Cause of Redundant Sampling

• Using complete CFG to calculate LSP

7

C1

6

1 1

A2..4

4

D5

C2

7

1

C3

8

C4

9

1

1

B1

10

B2

11

1

1LSPpath1 = 1 LSPpath2 = 5

LSP = 1

optimal

LSP 6 samples

Path2

LSPpath2 1 samples

84% reduction in samples

path1

Not optimal

path2

Path-aware Time-triggered Monitoring

1. Predict execution path

2. Calculate LSP using only predicted path

8

C1

6

1 1

A2..4

4

D5

C2

7

1

C3

8

C4

9

1

1

B1

10

B2

11

1

1

Path-aware LSP (paLSP)

path2

LSP = 5

Path Prediction Function

1. Predict execution path

9

Path prediction function

Implement path prediction function using symbolic execution

Execution path 1

Path constraint 1

Execution path 2

Path constraint 2

… ….

Execution path n

Path constraint n

1. Symbolize inputs. 2. Create table. 3. Check path constraints.

.

.

.

Environment

Problem:

paLSP = LSP

Adaptive Path-aware Time-triggered Monitoring

• Hypothetical execution path:

10

C1

6

1 1

A2..4

4

D5

C2

7

1

C3

8

C4

9

1

1

B1

10

B2

11

1

1

LSP = 1 LSPpath3 = 1

region1 LSP= 5

region2 LSP= 1

18 samples

7 samples

60% reduction in samples

path3=<path2+path1>region1

region2

LSP Regions

• An LSP region is a set of subpaths of an execution path:

• the same paLSP

• each subpath is maximal

• Regionalization objectives:1. Reducing the number of LSP regions

2. Reducing the number of samples

3. Maintaining the absolute jitter of paLSP

11

Regionalization Algorithm

12

A

W1 W2 W3

B C D E F

A B C D E F

A B C D E F

A B C D E F

A B C D E F

General Regionalization

• Can have different regions for different subpaths:

13

A B C D

Path 1

A B F

Path 2

5 10 15

E5 1 2

LSP = 5 LSP = 10

LSP = 1

General Rationalization: each arc in the CFG resides in one and only one LSP

region

Tool Chain

14

Variables of interest

C program Symbolizer

LLVM(CFG creator)

KLEE(Symbolic Executor)

Regionalization

paLSPCalculator

TableCompresso

r

Assumptions

• Limited to programs handled by KLEE

• Program is sequential

• Program runs on a single processor

15

Handling KLEE Limitations

• Concretization:• Extract the instruction where concretization happens

• Find the node containing the instruction in CFG

• Append following sub-CFG to executed path

16

…...

…

…

…

concretization

Handling KLEE Limitations (cont’)

• Incomplete paths:• Extract the last executed instruction

• Find the node containing the last executed instruction in CFG

• Append following sub-CFG to executed path

17

…...

…

…

…

Last instruction

Reducing Table Size

18

Problem:• Program can have infinite execution paths.• A large table size results in large lookup

overhead at runtime.• Can not reduce monitoring overhead.

Reducing Table Size

• KLEE patch: • extracting unique paths:

• Table Compressor• Remove entries that do not improve LSP.

19

Path with loop sequence

Consecutive occurrences of

Reduce consecutive occurrences to

Tool Chain (cont’)

• Table Compressor:• Implication Reduction:

20

A C1

B1

E1 E2

B2 B3

ZC2 C3

D1 D2 D3 D4 D5 D6 D7 D8

10 10

1

10

10

10

20 20 20 20 20

5

5 5 5 5 5 5

55 5

paLSP = 1

PCpath1

PCpath2

PCpath3

paLSP PC LSP Regions

1

Experimental Settings

• We use programs from SNU benchmark

• We run the program and monitor on MCB1700 board with RTX OS

• Time-triggered monitoring modes:• Fixed-LSP

• Path-aware LSP

• Adaptive path-aware LSP ( )

• History1

21

1 B. Bonakdarpour, S. Navabpour, and S. Fischmeister, “Sampling-based Runtime Verification”, 88-102, FM’11

Experimental Settings (cont’)

• Metrics for evaluation:1. The values of the fixed LSP, paLSP, and adaptive paLSP

2. The number of redundant samples taken at run time by the monitor

3. The execution time of the monitored program. This value projects the amount of monitoring overhead

22

Values of paLSP and Adaptive paLSP

23

• paLSP increases sampling period 2.4 times• Adaptive paLSP increases sampling period 3.3 times

1. More paths with sparse critical instructions Better paLSP

2. More paths with small concentration of critical instructions Better Adaptive paLSP

Redundant Samples of paLSP and Adaptive paLSP

24

• paLSP decreases redundant samples by 44.8%• Adaptive paLSP decreases redundant samples by 64%

1. More paths with sparse critical instructions Less redundant samples by paLSP

2. More paths with small concentration of critical instructions Less redundant samples by Adaptive paLSP

Monitoring Overhead of paLSP and Adaptive paLSP

25

• paLSP reduces monitoring overhead by 34%• Adaptive paLSP reduces monitoring overhead by 51%

78% reduction in redundant samplesOverhead of adaptive paLSP more than paLSPNeed to keep low overhead for looking up the table at runtime.

Monitoring Overhead of paLSP and Adaptive paLSP with History

26

• 66% of paLSP+history has less overhead than event-triggered• 75% adaptive paLSP+history has less overhead than event-

triggered

Summary

• Sampling period must be devised based on execution path of the program (paLSP).

• Redundant samples can be further reduced when sampling period changes dynamically at runtime (adaptive paLSP).

• By merging history and paLSP or adaptive paLSP, we achieve a monitor suitable for time sensitive systems.• Predictable monitoring

• Bounded overhead

• Imposes less overhead than event-triggered

27

28

Thank you

Questions?