System Design Research Laboratory Model-based Testing and Monitoring for Hybrid Embedded Systems Li...

System Design Research Laboratory

Model-based Testing and Monitoring for Hybrid Embedded Systems

Li TanJesung Kim

Oleg SokolskyInsup Lee

University of PennsylvaniaNovember 2004


Outline1. Motivations and overview2. Model-based testing.

1. Generating model-based testers.3. Model-based monitoring

1. Synthesizing model-based monitors from logic specifications

2. Instrumenting models4. The case study on SONY AIBO dog

1. Design-level validation2. “on-board” validation

5. Conclusion


The Goal Goal: apply the state-of-the-art runtime

verification technique to model-based embedded system design and validation domain.

Requirements: Incorporating the rigid specification of the

system properties using temporal logics. Using the existing techniques in model-

based design to reduce the cost of migration.

Supporting both design-level and implementation-level validations.

The procedure can be fully automatically by a set of software tools.


Road Map

V.Implementation Level Validation

CodeGenerator

Self-testing and Self-monitoring Code

Model Simulator

IV. Design Level Validation

Self-testing and Self-monitoring Model

Coverage Criteria/

environment constrains

Tester model

I. CreateModel-basedTester

System model

Instrumented model

P2C instrumentatormPEDL

event definition

II. Instrumentthe Model

mMEDLspecification

M2C Synthesizer

Monitor model

III. SynthesizeModel-based Monitor

Five steps towards model-based self-testing and self-monitoring embedded systems


Embedded System Modeling with Hybrid Automata

A hybrid automaton A={S, V, T, G, W, D, I, s0} extends a EFSM {S, V, T, G, W, s0} with continuous behaviors

1. S is the set of modes2. V is the set of variables3. T=V£ V is the set of transitions4. G assigns each t 2 T a guard, a predicate

over V.5. W assigns each t 2 T an assignment for V

µ V.6. D assigns each s 2 S a set of differential

equations on V.7. I assigns each s 2 S an invariant, a

predicate over V.8. s0 2 S is the initial mode.


An example: Dog Head Controller

.

x 45?

x -45

|| 10?

|| >10?

46

10

x

x)( xkx

x

: The visibility of the ballx: The angle of the head: The angle of the ball

Hybrid automata can be composed concurrently. Hybrid automata can be composed hierarchically.

1. A mode can be a collection of submodes.

46

10

x

x


Model-based TesterA model-based tester emulates an environment that supplies a

test case Environment

/hardware constrains

Environment /hardware constrains

Environ. Model

+

Simulator

System Model

No

Test Trace

Yes

Tester Model

Determination

CoverageChecker

Coverage

Criteria

Coverage

Criteria


Runtime verification

MEDLspecification

ProgramInstrumentation

Java Programs

Runtime verification (monitoring) is to check the execution of a software program against its logic specification

Java Programs

+Filter

Monitoring script

Monitor

MEDL compiler

Computer

events


Model-based Runtime Verification

MEDLspecification

ModelInstrumentation

Monitor Synthesizer

Simulator

System Automaton

System Automaton Observer Monitoring Automaton

Instrumented Model


mMEDL: expressing your properties

1. MEDL (Meta Events Definition Language) is a linear interval temporal logic for specifying safety properties.

1. MEDL is initially introduced for monitoring Java programs in MaC (Monitor and Checking) System [KKL01]

2. Syntax1. Defined on conditions, events, and expressions.

C := [E, E) | : C | C && C | C||C | Q QE := e | start(C) | end(C) | E || E | E&&E | E when CQ := time(E) | c |Q } Q

Where e is primitive event, c is a constant, 2 {>, <, =}, and } 2 {*,/,+,-}

.


Semantics by example Informal Specification: if the dog loses the ball 50

seconds after the ball becomes visible, an alarm should be raised.

mMEDL script

Begin import event isVisible, isInvisible, track,lost;condition visible= [isVisible, isInvisible);event becameTruelost= lost when visible;alarm lostTrack=start (time(becameTruelost)-time(isVisible)>50);

End

isVisible, isInvisible, track, and lost are primitive events isVisible (isInvisible) occurs when >10 becomes true

(false). Lost (track) occurs when |-x| > 10 become true (false).

An alarm is an event which indicates the violation of the safety requirement.

.


Synthesizing model-based monitor

0: import events e1,e2, e3 1: condition C=[e1, e2) 2: alarm E= e1 when C

|| ||PÇ: NewEvent

T0: P=0\and NewEvent? T:=t, P:=1

NewEvent´ :(Ve1>T Ç Ve>>T Ç Ve3

>T)

“engine” automatonVC: current value of CC: last time when C changes

VE: last time when E occurs

Violation is detected if VE>0

Sequence of Primitive Events

P2

T0: P=2Æ VC=1 Æ Ve1 =t?VE:=t, P=0

Td: P=2Æ ~( VC=1 Æ Ve1 =t)? P=0

P1

T0: P=1ÆUp?VC:=1, C:=t, P:=2

T1: P=1ÆDown?VC:=0, C:=t, P:=2

Up´ (VC=0 Æ Ve1 =t)

Down´ (VC=1 Æ Ve2 =t)

Td: P=1Æ : Up Æ : Down?P:=2

Synthesis is modularized


mPEDL: define primitive events

Primitive events report the status change in the system model.

Primitive events are formally encoded in mPEDL (model-based Primitive Event Definition Language)

Events are defined as the changes on some predicate.

MonScr Dogexport event isVisible, isInvisible, lost, track;monobj int dog.vision;monobj int dog.ball_pan_delta;

event isVisible= start (dog.vision>10);event isInvisible= end (dog.vision>10);event lost = start (dog.ball_pan_delta>10);event track = end (dog.ball_pan_delta>10);

End


Model Instrumentation

x 45?

x -45

|| 10?

|| >10?

46

10

x

x)( xkx 46

10

x

x

1. Model Augmentation A observer automaton is concurrently composed with

the model. The structure of the model will not be changed.

10|| x10|| x 10|| 10||

|-x| >10?Vlost=t

1t

|-x|10?Vtrack=t

|| >10?VisVisible=t

||10?VisInvisible=t

Observer


Design-level validation The composition of instrumented model, tester, and monitor forms a self-validating Charon

model Tester supplies test trace during simulation

Model-based tester emulates the movement of a “virtual” ball by changing input variables and The occurrence of an event is indicated by the changes on the event variables during

simulation

(a) Alarm Detection


Design-level validation

(b) Primitive Events Emitted by Instrumented Model

(c) The simulation trace of monitor


Implemtation-level validationModel-based Tester

Model-based Monitor

System Model

Modular Code Generation

Monitor

Generated Code

Tester

Link as needed


ConclusionWe proposed a framework for testing and monitoring

model-based embedded systems. The property being verified is encoded in a

temporal logic. It doesn’t require the changes on existing model-

based design environment. The framework is for both design-level and

implementation-level validations. It produces a self-testing and self-monitoring

code for embedded system A set of tools are developed to automate the

process.


Thank you!


Computation overhead

The computational overhead of tester and monitor on simulator (1000 step @ 0.001 sec/step)

On SONY AiBo Dog, such slowdown doesn’t break real-time constrains. The controller program is a real-time system The slowdown is tolerated because the extra cost of tester and monitor can

be observed within each integration step.


Implementation-level validation

The alarm is used for calling the external functions to perform visible actions.

1. “play” function is called when an alarm lostTrack is detected

The space overhead of tester and monitor.


MEDL semanticsInterpreting MEDL on runs of hybrid automata, 1. A condition C maps a time period with true, false, or ?. If C

is,1. [E1, E2). C is true from event E1 to E2 (not included)

2. C1&&C2. C is true when C1 and C2 are both true.

3. C1||C2. C is true when either of C1 and C2 is true.

4. : C1. C is the dual of C1.

2. An event E maps a time instance with true or false. If E is,1. start(C). E occurs at the time C becomes true.2. end(C). E occurs at the time C becomes not true.3. E1||E2. E occurs when either E1 or E2 occurs.

4. E1&&E2. E occurs when both E1 and E2 occur.

5. E1 when C. E occurs when E1 occurs and C is true

3. Q is an expression. If Q is,1. time (E). Q’s value is the latest time E occurs.

2. Q1 } Q2. Q’s value is Q1 } Q2.

.


Related works and future directions1. Related works: “platform-specific”

approach v.s. “general purpose” approaches.

1. NASA Ames: Java PathExplorer2. UPenn: MaC tools3. Works on synthesizing finite

model/test-oracle from the formal specification [DilRam96,GiaHav01]

2. Ongoing and future researches1. Optimizing monitor-synthesizing

algorithm.2. Approximating general linear temporal

property as monitoring automaton.3. Testing and monitoring the code from

conventional sources.


Computation overhead

The computational overhead of tester and monitor on simulator (1000 step @ 0.001 sec/step)

On SONY AiBo Dog, such slowdown doesn’t break real-time constrains. The controller program is a real-time system The slowdown is tolerated because the extra cost of tester and monitor can

be observed within each integration step.


Itemized space overhead

The space overhead of tester and monitor.

System Design Research Laboratory Model-based Testing and Monitoring for Hybrid Embedded Systems Li...

Documents

Transcript of System Design Research Laboratory Model-based Testing and Monitoring for Hybrid Embedded Systems Li...