Testing and Monitoring at Penn

Testing and Monitoring Model-based Generated Program

Li Tan, Jesung Kim, and Insup Lee

July, 2003

Testing and Monitoring at Penn

Outline1. Motivations2. Overview of our methodology3. Creating tester4. Generating monitor5. Case Study and Performance Issues6. Related Work and Future direction

Testing and Monitoring at Penn

MotivationsChallenges of testing and monitoring an implementation

with respect to its model.

1. Testing and monitoring integrated hardware/software platforms, e.g., embedded systems.

1. The architecture of platforms are different from general-purpose computing environments: Industrial processors, real-time operation systems.

2. Limited resources.3. Testing and monitoring tasks should be

executed in real systems.

2. Reducing the development cost of monitoring/testing platform.

3. Generating tester and monitor from the model and its requirement specification.

Testing and Monitoring at Penn

Our goalBuilding self-testing and self-monitoring executable

programs. Given,

A model-based code generator for hybrid automata.

Targeted platform: Sony robotic dog, a typical embedded system.

Yield, A miniaturized tester and monitor directly

works on the targeted platform. The ability to synthesize concrete tester and

monitor from the high-level specification. The path

Modeling testing and monitoring tasks. Generating tester and monitor from the

models.

Testing and Monitoring at Penn

Overview of our approachModel monitoring and testing jobs ……

Generating monitors and testers from models!

Testing and Monitoring at Penn

Modeling testing taskTesting task is modeled as a deterministic hybrid

automata.Textual

Description of Testing Task

Textual Description of Testing Task

Testing automata Testing automata

System ModelSystem Model+

Simulator/ Coverage CheckerNot meet requirement

Refine test automataCode Generator

Embedded Tester

Testing and Monitoring at Penn

Testing Sony Dog: requirements

1. Testing platform:2. Testing requirements can be a combination of

coverage criteria and system properties. 1. (Coverage-based Testing) Testing should

cover all the modes in system model (mode coverage).

2. (Specification-based Testing) Dog should not lose the track of ball if ball is visible. 1. Practically, the dog will lose track of the

ball if the ball moves too fast, 1. What is the threshold of ball’s speed?2. What is the reaction of the dog?

Testing and Monitoring at Penn

Modeling requirements2. Translate to a two-mode hybrid automata,

3. Simulator/Coverage Checker is used to refine testing automata and check the required coverage criteria

1. What value of a, b, c, d will make testing meet the requirement?

2. Simulation/coverage checker will execute the models on the simulation level and check the coverage.

Vision=false

Vision=truePosition of ball:

Pos=d ¢ sin(a t2+b t+c)

Testing and Monitoring at Penn

Modeling MonitorMonitoring task is modeled as a deterministic hybrid

automata.

Textual Description of

System properties

Textual Description of

System properties

Deterministc Time

automata

Deterministc Time

automata

Hybrid Automata

Hybrid Automata

Code GeneratorMonitor

Testing and Monitoring at Penn

Monitoring Sony Dog System specification:

1. Dog should not lose the track of ball if ball is visible.1. Dog doesn’t lost the ball: |ball-head| < 10.2. Dog should been given a fair chance to make its efforts: |ball-head|<10

five seconds after the ball is visible. Monitoring automaton is a deterministic timed automaton.

Vision indicates visibility of the ball, and 10 is the threshold of a “visible” ball

Testing and Monitoring at Penn

Synthesizing monitor Translate to hybrid automaton

Time is handled as linear differential equation

Generating monitor from the hybrid model.

Testing and Monitoring at Penn

Put it together: a case study

Monitoring automaton

Monitoring automaton

System ModelSystem Model Testing automaton

Testing automaton

Modular compilation

Monitor Generated Code Tester

Link as needed

Testing and Monitoring at Penn

Performance Issues1. Testing and monitoring can be simulated on model

level.

2. The space overhead of tester and monitor.

Testing and Monitoring at Penn

ConclusionsWe proposed a framework of generating tester and

monitor via code generation:1. It allows the rapid-prototyping of testing and

monitoring programs. 1. Monitor and tester tasks are specified in the high-

level modelling language.2. Tester and monitor can be re-targeted to a different

platform for which code generation process is available.

2. Testing and monitoring may be preformed both on simulation level and implementation level.

1. Tester and monitor are executable on targeted platform.

3. Resource saving, 1. Each tester and/or monitor is a customer job. 2. (for modularity compilation) tester and monitor can

be linked and applied as needed

Testing and Monitoring at Penn

Related works and future directions1. Related works: “platform-specific”

approach v.s. “general purpose” approaches.

1. NASA Ames: Java PathExplorer2. UPenn: MaC tools3. Works on synthesizing finite model/test-oracle

from the formal specification [DilRam96,GiaHav01]

2. Ongoing and future researches1. Incorporating existing specification language

to the framework.1. MEDL to CHARON translator

2. Approximating general linear temporal property as monitoring automaton.

3. Testing and monitoring the code from conventional sources.