Model-based Testing of Hybrid Systems Michiel van Osch IPA Spring Days on Testing 19 April – 21...
-
date post
19-Dec-2015 -
Category
Documents
-
view
217 -
download
1
Transcript of Model-based Testing of Hybrid Systems Michiel van Osch IPA Spring Days on Testing 19 April – 21...
Model-based Testing of Hybrid Systems
Michiel van Osch
IPA Spring Days on Testing19 April – 21 April 2006
2
Motivation• Hybrid Systems
• Testing might be expensive, dangerous, or resources might be limited
• Discrete and real-time model-based testing does not test the continuous aspects of the system
3
Content
• Refresh– Model-based Testing– Discrete Conformance & ioco Tests– Timed Conformance & tioco Tests
• Hybrid Systems• Hybrid Input-output Conformance• Hybrid Input-output Conformance Tests• Conclusions
– Results– Limitations and Future Work
4
Automated Model Based Testing
model
SUT
IUTconforms to
model
pass fail
test
tool
testgeneration
tool
test execution
toolIUT passes tests
IUT conf model
soundexhaustiv
e
5
Input-output Conformance
Impl. ioco Spec. iff for all traces α:
out(Impl. after α) out(Spec. after α)
6
Test-case Generation and Execution
1. Terminate with verdict pass
2. Select an input from the specification and apply it to the implementation
3. Observe an output or a timeout from the implementation and check if it is allowed according to the specification
7
Example
s0s1 s4
s2
s3
?Button2
!Tea
?Button1
?Button2
?Button1
!XLCoffee
!Coffee t0
t1
?Button1
t3
?Button2
Pass
!XLCoffee
t2
!
FailFail Fail
!Coffee !Tea!
FailFail Fail
!Coffee !Tea!XLCoffee
8
Timed Input-output conformances
• Different views: time as observable event versus time as parameter of the (output) actions
s0s3!tick !tick !tick !tick
!XLCoffee
Vs.
s0s3!XLCoffee(4)
9
Timed Input-output Conformance (Krichen)
Impl. tioco Spec. iff for all traces α:
out(Impl. after α) elapse(Impl. after α)
out(Spec. after α) elapse(Spec. after α)
10
Timed Test-case Generation and execution
1. Terminate with verdict pass
2. Select an input from the specification and apply it to the implementation
3. Observe an output or time passing from the implementation and check if it is allowed according to the specification
11
Example Test-case Generation
t0
t1
?Button1
t2
!tick
Fail Fail
t3
?Button2
!tick
Pass
!XLCoffee
Fail Fail
Fail Fail
!tick
!tick
!tick
Fail
!tick
s0s1 s4
s2
s3
?Button2
!Tea
?Button1
?Button2
?Button1
!XLCoffee
!Coffee
!tick !tick !tick
!tick !tick !tick !tick
!tick !tick
!tick !tick
13
Content
• Refresh– Model-based Testing– Discrete Conformance & ioco Tests– Timed Conformance & tioco Tests
• Hybrid Systems• Hybrid Input-output Conformance• Hybrid Input-output Conformance Tests• Conclusions
– Results– Limitations and Future Work
14
Hybrid Systems
• Practically:– Discrete behavior plus continuous behavior– Continuous behavior can be input observed through sensors
or output generated by actuators
• Mathematically– Discrete actions plus flow of continuous variables
(trajectories)– Variables can be input variables and output variables
15
Hybrid Systems
Time
Coffee
0
?Button1 ?Button2
s0s1
s2
s3
?Button1
?Button2
?Button1
Coffee’ = 4 cl/sec.
Coffee’ = 3 cl/sec.
Coffee’ = 0 Coffee’ = 0
16
Hybrid Systems
s0s1
s2
s3
?Button1
?Button2
?Button1
Water’ = 4 cl/sec.Coffee’ = Water’
Water’ = 3 cl/sec.Coffee’ = Water’
Water’ = 0Coffee’ = 0
Water’ = 0Coffee’ = 0
Water
17
Content
• Refresh– Model-based Testing– Discrete Conformance & ioco Tests– Timed Conformance & tioco Tests
• Hybrid Systems• Hybrid Input-output Conformance• Hybrid Input-output Conformance Tests• Conclusions
– Results– Limitations and Future Work
18
The Idea
• Besides for the output actions, also for every reachable state the set of trajectories allowed by the implementation is a subset of the set of trajectories allowed by the specification
• No quiescence because continuous output always occurs
19
Hybrid Conformance (continuous output only)
Impl. is input-output conform a Spec. iff for all traces α:
out(Impl. after α) out(Spec. after α) and
traj(Impl. after α) traj(Spec. after α)
s0s1
s2
s3
?Button1
?Button2
?Button1
Coffee’ = 4
Coffee’ = 3
Coffee’ = 0 Coffee’ = 0
20
Hybrid Conformance Issues
• The continuous output directly depends on the continuous input, therefore we have to define our relation on both input and output trajectories.
• The implementation is input enabled (for both discrete behavior and continuous behavior), but we do not want the specification to be input complete.
21
Hybrid Conformance (continuous input plus output)
s0s1
s2
s3
?Button1
?Button2
?Button1
Water’ = 4Coffee’ = Water’
Water’ = 3Coffee’ = Water’
Water’ = 0Coffee’ = 0
Water’ = 0Coffee’ = 0
infilter(traj(Impl. after α), traj(Spec. after α)) is the set of trajectories allowed by the implementation, with respect to the input trajectories allowed by the specification
Water’ = 3
Coffee’ = 3
Water’ = 2
Coffee’ = 2
22
Hybrid Input-output Conformance
Impl. hioco Spec. iff for all traces α:
out(Impl. after α) out(Spec. after α) and
infilter(traj(Impl. after α), traj(Spec. after α)) traj(Spec. after α)
23
Hybrid Conformance (continuous input plus output)
s0s1
s2
s3
?Button1
?Button2
?Button1
Water’ = 4Coffee’ = Water’
Water’ = 3Coffee’ = Water’
Water’ = 0Coffee’ = 0
Water’ = 0Coffee’ = 0
s4 s5!”NoWater”
Water’ =0Coffee’ = Water’
Water’ = 0Coffee’ = Water’
Coffee’ = 0Water = 0
out(Impl. after α) out(Spec. after α) and
infilter(traj(Impl. after α), traj(Spec. after α)) traj(Spec. after α) {if the implementation cannot do output action “!NoWater”}
{“NoWater”} and
In state s4:
24
Hybrid Conformance(continuous input plus output)
If there is a trajectory from state s then
out(s) = {o in O| s →} {ξ}else
out(s) = {o in O| s →}
o
o
out(Impl. after α) out(Spec. after α) and
infilter(traj(Impl. after α), traj(Spec. after α)) traj(Spec. after α) {if the implementation cannot do output action !”noWater”}
{ξ} {“NoWater”} and
In state s4:
25
Hybrid Input-output Conformance
Impl. hioco Spec. iff for all traces α:
out(Impl. after α) out(Spec. after α) and
infilter(traj(Impl. after α), traj(Spec. after α)) traj(Spec. after α)
26
Content
• Refresh– Model-based Testing– Discrete Conformance & ioco Tests– Timed Conformance & tioco Tests
• Hybrid Systems• Hybrid Input-output Conformance• Hybrid Input-output Conformance Tests• Conclusions
– Results– Limitations and Future Work
27
Hybrid Test-case
• Tree like structure• Two terminal states: pass and fail• Deterministic for actions• Strongly time deterministic for trajectories
28
Hybrid Test-case Generation
1. Terminate with verdict pass
2. Select an input from the specification and apply it to the implementation
s0s1?Button1
Water’ = 0Coffee’ = 0
Water’ = 0Coffee’ = 0
Specification
t0
t1
?Button1
Test-Case
29
Hybrid Test-case Generation
3. Select an input trajectory from the specification, apply it to the implementation and observe the output trajectory simultaneous, possibly interrupted by an output action.
Specification
s1Water’ = 0Coffee’ = 0
Test-Case
t1
t2
Water’ = 0Coffee’ = 0
Fail Fail
Fail Fail
!”Boilerlow”
30
Hybrid Test-case Generation
4. If an output action has to happen immediately according to the specification then observe an output action and check if it is allowed according to the specification or let time pass by selecting and applying and observing a trajectory
fail fail
s4 s5!”NoWater”
Coffee’ = 0Water = 0
Specification
t4
Pass
!”NoWater”
Test-Case
31
Example Test-case Generation
t0
t1
?Button1
t3
?Button2
Fail FailWater’ = 0Coffee’ = Water’
t4
t2 Fail Fail
Water’ = 0Coffee’ = 0
!”Boilerlow”
Fail Fail
Pass
!”NoWater”
fail fail
32
Content
• Refresh– Model-based Testing– Discrete Conformance & ioco Tests– Timed Conformance & tioco Tests
• Hybrid Systems• Hybrid Input-output Conformance• Hybrid Input-output Conformance Tests• Conclusions
– Results – Limitations and Future Work
33
Results
• A hybrid conformance theory• Sound and exhaustive• Natural extensions to other theories• Still, open issues …
34
Limitations and Future Work
• Continuous input depending on continuous output• Uncountable many states and trajectories• Taking into Practice
– Sampling– Test-selection– Inaccuracy / robustness– …