Feedback-Directed Random Test Generation Automatic Testing & Validation CSI5118 By Wan Bo.
CSI5118 W2001
description
Transcript of CSI5118 W2001
CSI5118 W2001
• Outline– Review Verification & Validation– Introduction to EFSM Models– Introduction to SDL
• e.g. EggTimer
– Principles of Validation & Verification
EFSM Models
• Extended Finite State Machines– Same as FSM’s– Enhanced features on transitions
• no inputs (spontaneous transitions)
• supports variables
• guards (or conditions)
• several outputs
• input/output parameter support (data)
FSM Models
• Example (FSM)
Input
State x y
s0 0,s1 1,s0
s1 1,s1 1,s2
s2 1,s0 0,s1
0
1 2
y/1
x/1
y/0
y/1x/1
x/0
Tabular Form:Each table entry shows what symbol is output, and which new state is to be adopted
Graph Form:The label “x/0” means that “when x is the input, 0 is the output.”
EFSM ModelFax Machine Example
I R
TE
i/RDY
d, g/DLNG, PAGE
to/TIME
s/ERRf/ERR
i/PAGE
-/TOK
to/TIME
s/ERR
d, g, f/DLNG, ERR
Notation UsedStatus = {I,R,T,E}I = IdleR = Ready to TransmitT = TransmittingE = Error
Input events = {I,d,g,s,to,f}i = insert paged = dial destination numberg = press “Go” buttons = press “Stop” buttonto = timeout (5 seconds elapsed) f = failure of the attempted or pending action
Output Responses = {TOK, TIME,PAGE, DLNG, ERR, RDY}TOK = “Transmission Ok”TIME = (Time of day)PAGE = “Transmitting page”DLNG = “Dialing”ERR = “Error”RDY = “Document Ready”
EFSM ModelFax Machine Example
I R
TE
i/RDY
d, g/DLNG, PAGE
to/TIME
s/ERRf/ERR
i/PAGE
-/TOK
to/TIME
s/ERR
d, g, f/DLNG, ERR
Or...Transition n (d,g,f/DLNG, ERR)
FROM: ‘R’WHEN: ‘d’ then ‘g’ then ‘f’ACTIONS: ‘DLNG’ then ‘ERR’ TO: ‘E’
...
EFSM Model with Variables
Partial Model of a Stack
Empty Normal
Full
push
push push
pop
pop
pop
push
FSM
Empty Normal
Full
[SC>1] push (SS=1)
[SC=1]push
(SS=1)
[SS+1=SC]push
(SS=SC)
[SS+1<SC]push
(SS=SS+1)
EFSM
SC = Stack CapacitySS = Stack Size
As an exercise, do the ‘pop’ interactionfor this EFSM
SDL• Specification and Description
Language– Initially built for communicating
systems– Divides the specification in
• Environment
• System(s)SD T rw /a/insite/users/isales/sdl/E ggB oiler/egg_ boiler.sdt
rw /a/insite/users/isales/sdl/E ggB oiler/
A nalysis M odel
U sed F iles
SD L System Structu re
E ggB oiler [unconnected]
D ecT im e rw D ecT im e.spdD isplaying rw D isplaying.sprI n terfacing rw I n terfacing.sprT im ing rw T im ing.spr
EggT im er R W E ggB oilerC ontroller.sbk
EggT im erSystem R W E ggT im erSystem .ssy
T T C N T est Specification
O ther D ocum ents
The Environment Border
SDL
• A Whole system
SD T rw /a/insite/users/isales/sdl/E ggB oiler/egg_ boiler.sdt
rw /a/insite/users/isales/sdl/E ggB oiler/
A nalysis M odel
U sed F iles
SD L System Structu re
E ggB oiler [unconnected]
D ecT im e rw D ecT im e.spdD isplaying rw D isplaying.sprI n terfacing rw I n terfacing.sprT im ing rw T im ing.spr
EggT im er R W E ggB oilerC ontroller.sbk
EggT im erSystem R W E ggT im erSystem .ssy
T T C N T est Specification
O ther D ocum ents
SystemBlock
Process
Procedure
Inside an SDL System
S ys tem E g g T im erS ys tem 1(1)
S IG N A L O neM inuteB tn, S tartB tn , R esetS to p B tn;
S IG N A LL IS T B utto ns = O neM inuteB tn, S tartB tn , R esetS to p B tn;
S IG N A L D isp lay(M inuteD ig it, S eco nd D ig it1, S eco nd D ig it2), B ell;
S IG N A LL IS T To U ser = D isp lay, B ell;
S IG N A L S tartB o iling , S to p B o iling ;
S Y N TY P E M inuteD ig it = Integ er C O N S TA N TS 0:5E N D S Y N TY P E ;
S Y N TY P E S eco nd D ig it1 = Integ er C O N S TA N TS 0:5E N D S Y N TY P E ;
S Y N TY P E S eco nd D ig it2 = Integ er C O N S TA N TS 0:9E N D S Y N TY P E ;
E g g T im er
E g g B o iler
U serInterface(To U ser)
(B utto ns )
InternalS tartB o iling ,S to p B o iling
Channel (and channel name)System name
Signals (Messages)Declarations
Signalist (set of messages)
Newtype definitions
SDL System Structure
E g g T im er
E g g B o iler
U serInterface(To U ser)
(B utto ns )
InternalS tartB o iling ,S to p B o iling
Signals are shown in square brackets,separated by a comma
System blocks
Channel name
Signallists are declared like signals,but enclosed in parenthesis
Environment of the system
The EggTimer SDL Block
B lo ck E g g T im er 1(1)
S IG N A L IncM inute, S tartT im er, R esetT im er;
S IG N A L D isp layT im e(M inuteD ig it,S eco nd D ig it1,S eco nd D ig it2), R ing TheB ell;
D ecT im e
Interfac ing (1,1)
T im ing (1,1) D isp laying (1,1)
U serInterface
InterfaceIn
(B utto ns )
U serInterface
To T im er
IncM inute,S tartT im er,R esetT im er
In terfaceO ut
D isp lay,B ell
In ternalTo B o iler
S tartB o iling ,S to p B o iling
To D isp lay
D isp layT im e,R ing TheB ell
Connection points (system channel names)
Procedure declaration Process declarations
General declarations (signals, types, vars)
routes
Signals
The EggTimer SDL Block Structure
B lo ck E ggT im er 1(1)
Interfac ing (1,1)
D ecT im e
Tim ing (1,1) D isp laying (1,1)
U serInterface
InterfaceIn(B utto ns )
U serInterface
To T im erIncM inute,S tartT im er,R esetT im er
InterfaceO ut
D isp lay,B ell
InternalTo B o iler
S tartB o iling ,S to p B o iling
To D isp lay
D isp layT im e,R ing TheB ell
ProcessesAt least 1 instance and at most1 instance of this process
Procedure Reference
Displaying sends to the environmentand receives from Timingetc...
SDL Process exampleP ro cess T im ing 1(1)
D C L m M inuteD ig it := 0, s1 S eco nd D ig it1 := 0, s2 S eco nd D ig it2 := 0;
T im er O neS eco nd ;
S etting T im e
IncM inute
m < 5
m := m + 1
D isp layT im e(m ,s1,s2)
S etting T im e
S tartT im er
S et(N O W + 1,O neS eco nd )
S tartB o iling
T im ing
O neS eco nd
D ecT im e(m ,s1,s2)
D isp layT im e(m ,s1,s2)
m = 0 ands1 = 0 and
s2 = 0
S to p B o iling
R ing TheB ell
S etting T im e
S et(N O W + 1,O neS eco nd )
-
R esetT im er
m := 0;s1 := 0;s2 := 0;
R eset(O neS eco nd )
S etting T im e
R esetT im er
m := 0;s1 := 0;s2 := 0;
S etting T im e
true
false
true false
State
Initial stateInitial transition
Declarations
Input
Output
If
Procedure call
Task
Timer Input(or timeout)
SDL Declaration example
D C L m M inuteD ig it := 0, s1 S eco nd D ig it1 := 0, s2 S eco nd D ig it2 := 0;
T im er O neS eco nd ;
To say your declaring something
Identifier(instance name)
Instance type
InitializationTimer declaration
Timer Identifier
SDL Transition exampleS etting T im e
IncM inute
m < 5
m := m + 1
D isp layT im e(m ,s1,s2)
S etting T im e
true
false
FROM
WHEN
ACTIONS
SEND
TO
Local variable
Parameters to send
SDL Tasks
m := m + 1
S et(N O W + 1,O neS eco nd )
D ecT im e(m ,s1,s2)
m := 0;s1 := 0;s2 := 0;
m := 0;s1 := 0;s2 := 0;
Incrementing or attributing a value
Making a procedure call
Setting the timer
Set([NOW+]<Duration>, <TimerName>)<variable> := <expression>[; <another task]
SDL Procedures(and operators)
;F PA R IN /O U T m M inuteD ig it, IN /O U T s1 S eco nd D ig it1, IN /O U T s2 S eco nd D ig it2;
P ro ced ure D ecT im e 1(1)
s2 = 0
s1 = 0
m = 0
s1 := 5;s2 := 9;
m := m -1;
s2 := 9;s1 := s1 - 1;
s2 := s2 - 1;
true
true
truefalse
false
false
Formal Parameters
Param modifiername
and type
ProcedureStart
Procedure return
If first ‘seconds’digit is 0 thengo down, elsego left
Modifiers:IN - Parameter is an Input parameterOUT - Parameter is an Output parameterIN/OUT - Parameter is both IN and OUT
SDL Message Handling
Each process•has one and only one message queue•sends (by default) a message to the closest receiver (if the process itself is capable of receiving such message, it will send to itself)•the default sending procedure can be modified by
•VIA:
•TO:
<Message_name> VIA <channel_name>
<Message_name> TO <process_id>
T im ing (1,1)
To T im erIncM inute,S tartT im er,R esetT im er
To D isp lay
D isp layT im e,R ingTheB ell
SDL Message Handling
This process queue has four possible messages
•IncMinute•StartTimer•ResetTimer•OneSecond
The first three are external messages where OneSecond is an internal message, indicating when the clock has decreased one second.
There could be several other messages comingfrom other processes or other blocks
Principles of Validation
i) Design– User-centred view
• scenarios
• use-cases
• MSCs
• functional, blackbox, providing a service (or NOT), end to end
– guided execution in SDL high-level design
– refined design, re-run validation scenarios
Next i:Tr. Start
SDL Guided execution• Using the Navigator in a SDL
Validator tool.
Up 1
Next 1:Signal:
OneMinuteBtn
Next 2:Signal: StartBtn
Next 2:Signal:
ResetStopBtn
Next 1:Tr. Start
...
Next 1:...
Input: DisplayTimeSender: Timing:1Parameter(s):1,0,0
By clicking on theNext, we go derivinga scenario by followingthe state-space tree
SDL Guided Execution(in MSC form)
M S C Valid ato rTrace
env_ 0
Interfac ing
Interfac ing _ 1
T im ing
T im ing _ 2
D isp laying
D isp laying _ 3
Id le
S etting T im e
D isp laying
O neM inuteB tn
IncM inute
S etting T im er
D isp layT im e
1, 0, 0
S etting T im e
D isp lay
1, 0, 0
D isp laying
Valid ato r traceg enerated b yS D T Valid ato r 2.3
Functional MSC for the previous EggTimer scenario
M S C Validato rTrace
E nviro nm ent E g g T im erS ys tem
O neM inuteB tn
D isp lay
1, 0, 0
Valid ato r traceg enerated b yS D T Valid ato r 2.3
And executing this MSC in the Validator Tool,by clicking on “Verify MSC” and selectingthe proper file, we obtain (as the system has been correctly specified)
1 report
1 MSC Verification
MSC ValidatorTraceverifiedDepth: 7
Principles of Validationii) Test
• functional user interface tests in TTCN
S1
S2 S3
c a
b
a
b+
Test NameEvent a Event b pass
Tree and TabularCombined Notation
•TTCN standard for formal test case descriptions•Facilitate sharing of tests•Eliminates Ambiguities•UNIX-based TTCN Workbench toolkit•TTCN used for X.25, ISDN, CCS7, and many other protocols•All PDU parameters specified•All tests are formally assigned verdicts
Principles of Validationii) Test (cont’d)
• basic syntax & semantics of TTCN– TREE and TABLE Combined Notation
Test Case Dynamic Behaviour
Test Case Name:Group:Purpose:Default:
Behavior Constr. VerLabel Description Reference dict Remarks
A!X A?Y A!J A?K Pass
A?Z Fail Wrong Resp.
A?Otherwise Inc. Test body not reached
Extended Comments:
Principles of Verification
i) Design– State-space search
Process Chart
Transition Chart
State Space
-transitions
reduced state space
Principles of Verification
i) Design (cont’d)– Possible outcomes
• Unspecified reception
• queue overflow
• deadlock
• infinite livelock (no real progress)
– Random walk
Principles of Verification
ii) Test– Conformance Testing (TTCN)
• preamble
• test step
• verification sequence
• postamble
Principles of VerificationStructure of a Transition Test Case
for Conformance Testing
PREAMBLE: Initializationto Preamble State
TEST BODY: Application ofTest Stimulus and Confirmation of
System Response
VERIFICATION: Confirmationof Post-Transition State
POSTAMBLE: Reset to StableState for Next Test
Principles of Verification
Represent S1 S3 S4a/X b/Y
As a sequence of user actions and system responses, starting in state 1 and finishing in state 4.
In TTCN
+Preamble(S1) !a ?X !b ?Y +Verify(S4) PASS * FAIL * Inconclusive