Post on 30-Jul-2020
OOT-T3C © 2006 M. Winter
OO-Testing meets TTCN-3 1
Prof. Dr. Mario Winter FH Köln
Object-Oriented Testing meets TTCN-3
TTCN-3 User Conference 2006The Evolution of TTCN-3
31 May- 2 June 2006, Berlin, Germany
Slide 2 Prof. Dr. Mario Winter, FH KölnTTCN3UC Berlin 2006
Object-Oriented Testing meets TTCN-3
Why do we test?
Why is testing OO-SW difficult?Specification and testing of OO-SW
The UML Testing ProfileTTCN-3 cases for OO
Man (imperfect)
God (perfect)
Software(faulty)
Specification(correct (?))
Program(misconduct)
Testcase(temptation)
OOT-T3C © 2006 M. Winter
OO-Testing meets TTCN-3 2
Slide 3 Prof. Dr. Mario Winter, FH KölnTTCN3UC Berlin 2006
Object-Oriented Testing meets TTCN-3
Overview
• Why is testing object-oriented software difficult?• Specification and testing of object-oriented software• The UML Testing Profile • TTCN-3 cases for OO
Slide 4 Prof. Dr. Mario Winter, FH KölnTTCN3UC Berlin 2006
Object-Oriented Testing meets TTCN-3
which fails observably
Failure
Why do we test?
• Errare humanum est! (to err is human)• Even with formal specification and
program generation ...– Spec could be wrong– Generator doesn‘t work as expected– Platform doesn‘t work as expected– ...
• Testing checks that the Software– ... does what it should do and ...– ... doesn‘t what it shouldn‘t do
• But always remember that ...– ... „testing can only show the presence
of faults (i.e. bugs), not their absence“!!! [E.W. Djikstra]
Fault
in some program
Error
of a person
Why is testing OO-SW difficult?Specification and testing of OO-SW
The UML Testing ProfileTTCN-3 cases for OO
OOT-T3C © 2006 M. Winter
OO-Testing meets TTCN-3 3
Slide 5 Prof. Dr. Mario Winter, FH KölnTTCN3UC Berlin 2006
Object-Oriented Testing meets TTCN-3
TestLevel
When do we Test?
Test cases based on work artifacts (documents)
Requirements
Construction
Analysis
Architecture
Design
Coding
Unit Test
Integration Test
System Test
AcceptanceTest
Valid
ation
&
Verif
icatio
n
Why is testing OO-SW difficult?Specification and testing of OO-SW
The UML Testing ProfileTTCN-3 cases for OO
Analysis and design
Implementation andexecution
Evaluation of test exit criteria
Post testing activities
Start
End
Planning and control
Test process[ISTQB Certified Tester – Foundation Level Syllabus 2005]
Slide 6 Prof. Dr. Mario Winter, FH KölnTTCN3UC Berlin 2006
Object-Oriented Testing meets TTCN-3
How do we Test?
• Static Testing– No executables necessary– Automated source code analysis– Manual reviews and inspections
• Dynamic Testing– Executables necessary– Test cases with input and expected behaviour/output– Test harness for unit and integration testing needed
Processor
Test ObjectObservationStimulation
Why is testing OO-SW difficult?Specification and testing of OO-SW
The UML Testing ProfileTTCN-3 cases for OO
OOT-T3C © 2006 M. Winter
OO-Testing meets TTCN-3 4
Slide 7 Prof. Dr. Mario Winter, FH KölnTTCN3UC Berlin 2006
Object-Oriented Testing meets TTCN-3
Black box testing vs. white box testing
Input parameter/state
Specification needed!
Output parameter/state
Black Box Testing
Test object
Input parameter/state
Specification and realization (sources)needed!
Output parameter/state
White Box Testing
Test object
Why is testing OO-SW difficult?Specification and testing of OO-SW
The UML Testing ProfileTTCN-3 cases for OO
Slide 8 Prof. Dr. Mario Winter, FH KölnTTCN3UC Berlin 2006
Object-Oriented Testing meets TTCN-3
Test driver
Test output
Compare &report
Test case 1 Test case 2 ... Test case m
Why is dynamic testing difficult?
Why is testing OO-SW difficult?Specification and testing of OO-SW
The UML Testing ProfileTTCN-3 cases for OO
Stubs, Mocks
Stub 1
Stub 2
...
Stub n
Run time environment, analysis tools, simulators, monitors
Test object
Point of Control
Point ofObservation
Processor
OOT-T3C © 2006 M. Winter
OO-Testing meets TTCN-3 5
Slide 9 Prof. Dr. Mario Winter, FH KölnTTCN3UC Berlin 2006
Object-Oriented Testing meets TTCN-3
Why is object-oriented testing even more difficult?
• Object-Orientation = “Normal programming” + Encapsulation + Inheritance + Polymorphism + Late Binding
• Encapsulation– Bundling of data with the operations that manipulate that data– Classes often hide encapsulated information by presenting only public interfaces
• Inheritance– Forms new classes using already defined classes– Generalization: is-a relationship, captures hierarchical relationship between classes
• Polymorphism – Ability of object (references) belonging to different classes to respond to calls of
operations of the same name, each one according to a class-specific behavior– Programmer (and program) doe not have to know the exact classes of objects in
advance, so behavior can be implemented in different ways• Late binding (aka. dynamic binding)
– At run time, the machine (“system”) decides which method to execute– Methods searched upwards the inheritance hierarchy
Why is testing OO-SW difficult?Specification and testing of OO-SW
The UML Testing ProfileTTCN-3 cases for OO
Slide 10 Prof. Dr. Mario Winter, FH KölnTTCN3UC Berlin 2006
Object-Oriented Testing meets TTCN-3
Example: Inheritance or Generalization?Superclass
Subclass
Rectangle
- side1- side2
+ setSide1()+ setSide2()+ getArea()
Square
- side1- side2
+ setSide1()+ setSide2()+ getArea()
SomeClass
- ...
+ ...
11setSide1(10)
: SomeClass myR:Rectanglefoo()
setSide2(20)
A := getArea()
A = 200OK!
setSide1(10)
: SomeClass myR : Squarefoo()
setSide2(20)
A := getArea()
A = 100?A = 200?OOPS!
{ inv: side1 = side2 }
myR
OOT-T3C © 2006 M. Winter
OO-Testing meets TTCN-3 6
Slide 11 Prof. Dr. Mario Winter, FH KölnTTCN3UC Berlin 2006
Object-Oriented Testing meets TTCN-3
Hierarchical Inheritance Testing (HIT)
Create new test casesExecute new test cases
Create new test casesExecute new test cases
Repeat test cases of superclass
White-box test cases
Create new test casesExecute new test cases
New defined operations
Complement test casesExecute test cases of superclass and complementedtest cases
Redefined operations(Pure generalization)
Execute test cases of superclassInherited operations
Black box test casesOperations of subclass
Slide 12 Prof. Dr. Mario Winter, FH KölnTTCN3UC Berlin 2006
Object-Oriented Testing meets TTCN-3
Where are we now?
• Why is testing object-oriented software difficult?• Specification and testing of object-oriented software• The UML Testing Profile • TTCN-3 cases for OO
OOT-T3C © 2006 M. Winter
OO-Testing meets TTCN-3 7
Slide 13 Prof. Dr. Mario Winter, FH KölnTTCN3UC Berlin 2006
Object-Oriented Testing meets TTCN-3
Input parameters/State
Specification needed!
Output parameters/State
Black Box Testing
Class Under Test (CUT)
Black box testing of classes
• Goal – Validation of the class interface, i.e. its public operations (and members)
• Value1. Conformance of the class realization w.r.t. its specification2. Robustness of the class
• Needs – Assertions, i.e. pre- and postconditions for each public operation of the class under test
(CUT) together with its invariant• Result
– Reusable and extendable class test cases which validate the classes interface conformance and its robustness
Why is testing OO-SW difficult?Specification and testing of OO-SW
The UML Testing ProfileTTCN-3 cases for OO
Slide 14 Prof. Dr. Mario Winter, FH KölnTTCN3UC Berlin 2006
Object-Oriented Testing meets TTCN-3
How to specify interfaces? Design by contract (B. Meyer)
• Client uses suppliers (public) operations – Precondition = clients price– Postcondition = suppliers obligation– Invariant = “eternal” assumptions
• Contract– Iff the client satisfies the precondition, then the
supplier guarantees the postcondition!– Client and supplier satisfy their invariants
before and after each (public) operation execution
• A contract is a prescription, not only a description!
Contract
Client Supplier
Inva
riant
Inva
riant
If you promise to call me with the precondition satisfied thenI, in return, promise to deliver a final state in which the
postcondition (and my invariant) is satisfied
Why is testing OO-SW difficult?Specification and testing of OO-SW
The UML Testing ProfileTTCN-3 cases for OO
OOT-T3C © 2006 M. Winter
OO-Testing meets TTCN-3 8
Slide 15 Prof. Dr. Mario Winter, FH KölnTTCN3UC Berlin 2006
Object-Oriented Testing meets TTCN-3
Specification of interfaces with assertions
• Precondition (of an operation)– States the properties that must hold whenever the operation is called.– Refers to input-parameters of the operation und the state of the component
• Postcondition (of an operation)– States the properties that the operation guarantees when it returns (assuming its
precondition was satisfied) – Refers to output-parameters of the operation und the state of the component
• Both preconditions and postconditions describe properties of individual operations –but often there are more general properties
• Invariant (of the class)– Expresses global properties of all instances of a class, which must be preserved by all
operations– Refers to the state / instant variables of the class– Must be satisfied before and after each execution of each (public) operation
Why is testing OO-SW difficult?Specification and testing of OO-SW
The UML Testing ProfileTTCN-3 cases for OO
Slide 16 Prof. Dr. Mario Winter, FH KölnTTCN3UC Berlin 2006
Object-Oriented Testing meets TTCN-3
Example: BoundedStack (Stack with bounded capacity)
2 ← size1
MAXSIZE
class BoundedStackState preserving operations
size():integer; // Number of elementsMAXSIZE(): integer; // Maximum count of elementstop():Object; // Pointer to topmost element
State changing operationsBoundedStack(maxSize: integer);// Constructor~BoundedStack(); // Destructorpush(element: Object); // Stack element on toppop(); // Removes topmost element
class BoundedStackState preserving operations
size():integer; // Number of elementsMAXSIZE(): integer; // Maximum count of elementstop():Object; // Pointer to topmost element
State changing operationsBoundedStack(maxSize: integer);// Constructor~BoundedStack(); // Destructorpush(element: Object); // Stack element on toppop(); // Removes topmost element
push(element: Object)3 ← size21pop()
top():Object
Why is testing OO-SW difficult?Specification and testing of OO-SW
The UML Testing ProfileTTCN-3 cases for OO
OOT-T3C © 2006 M. Winter
OO-Testing meets TTCN-3 9
Slide 17 Prof. Dr. Mario Winter, FH KölnTTCN3UC Berlin 2006
Object-Oriented Testing meets TTCN-3
Why is testing OO-SW difficult?Specification and testing of OO-SW
The UML Testing ProfileTTCN-3 cases for OO
Contract of BoundedStack class BoundedStack {
/** invariant@ self.size() >= 0 AND self.size() <= self.MAXSIZE() */
public BoundedStack (Integer maxSize){/** pre@ maxSize > 0 *//** post@ self.MAXSIZE() = maxSize@pre */}public void push (Object item) throws FullStackException {/** pre@ self.size() < self.MAXSIZE() *//** post@ self.size() = self.size()@pre+1 AND self.top() = item@pre */}public Object top () throws EmptyStackException {/** pre@ self.size() > 0 *//** post@ return != null */}public void pop () throws EmptyStackException {/** pre@ self.size() > 0 *//** post@ self.size() = self.size()@pre - 1 */}public Collection all () {/** pre@ true *//** post@ (self.size() > 0 implies return.size() = self.size()) AND
(self.size() = 0 implies return = null)*/}
}
class BoundedStack {
/** invariant@ self.size() >= 0 AND self.size() <= self.MAXSIZE() */
public BoundedStack (Integer maxSize){/** pre@ maxSize > 0 *//** post@ self.MAXSIZE() = maxSize@pre */}public void push (Object item) throws FullStackException {/** pre@ self.size() < self.MAXSIZE() *//** post@ self.size() = self.size()@pre+1 AND self.top() = item@pre */}public Object top () throws EmptyStackException {/** pre@ self.size() > 0 *//** post@ return != null */}public void pop () throws EmptyStackException {/** pre@ self.size() > 0 *//** post@ self.size() = self.size()@pre - 1 */}public Collection all () {/** pre@ true *//** post@ (self.size() > 0 implies return.size() = self.size()) AND
(self.size() = 0 implies return = null)*/}
}
Slide 18 Prof. Dr. Mario Winter, FH KölnTTCN3UC Berlin 2006
Object-Oriented Testing meets TTCN-3
Contract-based test cases for BoundedStack
Context ExtendedStack A B C D E F invariant@ this.size() >= 0 AND TRUE TRUE TRUE TRUE TRUE TRUE invariant@ this.size() <= this.MAXSIZE TRUE TRUE TRUE TRUE TRUE TRUE ExtendedStack() pre@ true TRUE ExtendedStack()post@ this.size() = 0 AND TRUE ExtendedStack()post@ this.MAXSIZE = maxSize TRUE push() pre@ this.size() < this.MAXSIZE TRUE TRUE push() post@ this.size()=this.size()@pre+1 - TRUE top() pre@ this.size() > 0 FALSE TRUE TRUE top() post@ return != null - - TRUE pop() pre@ this.size() > 0 FALSE TRUE TRUE TRUE pop() @post this.size() = this.size()@pre – 1
- - - TRUE
all() pre@ true TRUE TRUE TRUE TRUE TRUE TRUE all() post@ this.size() > 0 AND - - - - FALSE TRUE all() post@ return.size() = this.size() OR - - - - dc TRUE all() post@ not this.size() > 0 AND - - - - TRUE FALSE all() post@ return = null - - - - TRUE dc
Context BoundedStackinvariant@ self.size() >= 0 AND
self.size() <= self.MAXSIZE()BoundedStack() pre@ maxSize > 0BoundedStack()post@ self.size() = 0 AND
self.MAXSIZE() = maxSizepush() pre@ self.size() < self.MAXSIZE()push() post@ self.size()=self.size()@pre+1top() pre@ self.size() > 0top() post@ return != nullpop() pre@ self.size() > 0pop() @post self.size() = self.size()@pre – 1all() pre@ trueall() post@ self.size() > 0 AND
( return.size() = self.size() ORnot self.size() > 0 ANDreturn = null )
Don‘t CareDon‘t Care
Conformance testing: precondition satisfiedRobustness testing: precondition not satisfiedTest oracle: postcondition satisfied
Conformance testing: precondition satisfiedRobustness testing: precondition not satisfiedTest oracle: postcondition satisfied
Why is testing OO-SW difficult?Specification and testing of OO-SW
The UML Testing ProfileTTCN-3 cases for OO
OOT-T3C © 2006 M. Winter
OO-Testing meets TTCN-3 10
Slide 19 Prof. Dr. Mario Winter, FH KölnTTCN3UC Berlin 2006
Object-Oriented Testing meets TTCN-3
State-based specification for BoundedStack
empty filled fullpush
push[ size < MAX-1]pop[ size > 1 ] push
pop[ size = 1 ]pop
push[ size = MAX-1]
~Stapel
Stapel
final
initaltop top
Three (five) states:empty: size() = 0;filled: 0 < size() < MAXSIZE();full: size() = MAXSIZE();initial: before construction;final: after destruction;
Three (five) states:empty: size() = 0;filled: 0 < size() < MAXSIZE();full: size() = MAXSIZE();initial: before construction;final: after destruction;
3 ← size21
MAXSIZE
Why is testing OO-SW difficult?Specification and testing of OO-SW
The UML Testing ProfileTTCN-3 cases for OO
Slide 20 Prof. Dr. Mario Winter, FH KölnTTCN3UC Berlin 2006
Object-Oriented Testing meets TTCN-3
Life cycle-based test cases (aka. state based)
filled
emptypush
filledpush
full
push
empty
pop
filledpop
filledpop
fullpush
final
~Stapel
initialStapel
filledtop
fulltop
errortop
error
~Stapel
error
~Stapel
errorpop
empty filled fullpush
push[ size < MAX-1]pop[ size > 1 ] push
pop[ size = 1 ]pop
push[ size = MAX-1]
~Stapel
Stapel
final
initaltop top
Three (five) states:empty: size() = 0;filled: 0 < size() < MAXSIZE();full: size() = MAXSIZE();initial: before construction;final: after destruction;
Three (five) states:empty: size() = 0;filled: 0 < size() < MAXSIZE();full: size() = MAXSIZE();initial: before construction;final: after destruction;
3 ← size21
MAXSIZE
Why is testing OO-SW difficult?Specification and testing of OO-SW
The UML Testing ProfileTTCN-3 cases for OO
OOT-T3C © 2006 M. Winter
OO-Testing meets TTCN-3 11
Slide 21 Prof. Dr. Mario Winter, FH KölnTTCN3UC Berlin 2006
Object-Oriented Testing meets TTCN-3
Testing operation calls and late binding
Am1(){myB.m2();}
B1m2(){...}
B2m2(){...}
m2(aB: B){...}
A2A1
myBB
Operating object (as „client"), parameter object und server object classes:How many different combinations of classes and states?Operating object (as „client"), parameter object und server object classes:How many different combinations of classes and states?
OpUTOpUT
CallCall
Parameter classParameter classClient classClient class
Server classServer class
Two states SA1 and SA2Two states SA1 and SA2
Three States SB1, SB2 and SB3
Three States SB1, SB2 and SB3
(3 C * 2 S) * (3 C * 3 S) * (3 C * 3 S) = 486 possibilities!(3 C * 2 S) * (3 C * 3 S) * (3 C * 3 S) = 486 possibilities!
Why is testing OO-SW difficult?Specification and testing of OO-SW
The UML Testing ProfileTTCN-3 cases for OO
Slide 22 Prof. Dr. Mario Winter, FH KölnTTCN3UC Berlin 2006
Object-Oriented Testing meets TTCN-3
Idea: Not all combinations, but all pairwise combinations• Orthogonal arrays (OA)• 2 x 3 classes with 2 resp. 3 states!• 6 factors with cardinality 2 resp 3
Am1(){myB.m2();}
B1m2(){...}
B2m2(){...}
m2(aB: B){...}
A2A1
myBB
A = 1, A1 = 2, A2 = 3, ZA1 = 1, ZA2 = 2B = 1, B1 = 2, B2 = 3, ZB1 = 1, ZB2 = 2, ZB3 = 3
A = 1, A1 = 2, A2 = 3, ZA1 = 1, ZA2 = 2B = 1, B1 = 2, B2 = 3, ZB1 = 1, ZB2 = 2, ZB3 = 3
OA18(21x37)
Nr ClientState
ClientClass
ParameterClass
Paramete-State
ServerClass
ServerState
1 1 1 1 1 1 1 1 1
2 1 1 2 2 2 2 2 2
3 1 1 3 3 3 3 3 3
4 1 2 1 1 2 2 3 3
5 1 2 2 2 3 3 1 1
6 1 2 3 3 1 1 2 2
7 1 3 1 2 1 3 2 3
8 1 3 2 3 2 1 3 1
9 1 3 3 1 3 2 1 2
10 2 1 1 3 3 2 2 1
11 2 1 2 1 1 3 3 2
12 2 1 3 2 2 1 1 3
13 2 2 1 2 3 1 3 2
14 2 2 2 3 1 2 1 3
15 2 2 3 1 2 3 2 1
16 2 3 1 3 2 3 1 2
17 2 3 2 1 3 1 2 3
18 2 3 3 2 1 2 3 1
Why is testing OO-SW difficult?Specification and testing of OO-SW
The UML Testing ProfileTTCN-3 cases for OO
OOT-T3C © 2006 M. Winter
OO-Testing meets TTCN-3 12
Slide 23 Prof. Dr. Mario Winter, FH KölnTTCN3UC Berlin 2006
Object-Oriented Testing meets TTCN-3
Create and execute test cases
• Testscripts – TTCN-3 – Parameterized – Executable
:A {ZA1} :B {ZB1}myB
:B {ZB1}aB
A = 1, A1 = 2, A2 = 3, ZA1 = 1, ZA2 = 2B = 1, B1 = 2, B2 = 3, ZB1 = 1, ZB2 = 2, ZB3 = 3A = 1, A1 = 2, A2 = 3, ZA1 = 1, ZA2 = 2B = 1, B1 = 2, B2 = 3, ZB1 = 1, ZB2 = 2, ZB3 = 3
module Test of Class A {modulepar {
B_Type theB_Par;B_Type anotherB_Par;
}external const float TestExecutionTime;control {var verdicttype testCaseVerdict := none; testCaseVerdict :=
execute(connectmyB(theB) ,TestExecutionTime);if (testCaseVerdict == pass) {
testCaseVerdict := execute(m1(anotherB) ,TestExecutionTime);
}}
}
module Test of Class A {modulepar {
B_Type theB_Par;B_Type anotherB_Par;
}external const float TestExecutionTime;control {var verdicttype testCaseVerdict := none; testCaseVerdict :=
execute(connectmyB(theB) ,TestExecutionTime);if (testCaseVerdict == pass) {
testCaseVerdict := execute(m1(anotherB) ,TestExecutionTime);
}}
}
Nr ClientState
ClientClass
ParameterClass
Paramete-State
ServerClass
ServerState
1 1 1 1 1 1 1 1 1
Why is testing OO-SW difficult?Specification and testing of OO-SW
The UML Testing ProfileTTCN-3 cases for OO
Slide 24 Prof. Dr. Mario Winter, FH KölnTTCN3UC Berlin 2006
Object-Oriented Testing meets TTCN-3
Specification and testing OO-SW
All messages, all nodes, allbranches, ..., all paths
UML sequence diagr.
All actions, all transitions, n-Paths, ..., all paths
UML activity diagramDiagram with formalsemantics
All messages, all linksUML communication diagram
Structure based
All functionsAPIInformal (natural language)All clausesObject-Z, VDMFormal theory
Normal flow, all alternate flows, ..., all flows
UML use case diagram
Informal (natural language)Functionbased
(declarative/ operational)
All messages, all nodes, all branches, ..., all paths
SDL MSC,UML sequence diagram
Message basedInteraction
based(operational)
All states, all transitions, n-Paths, ..., all paths
UML state chartDiagram with formal semantics
State based (operational)
All conditions, MC/DC, ...UML-OCLFormal languageContract based
(declarative)
Method / Coverage CriteriaConcrete SyntaxTypeSpecification
Why is testing OO-SW difficult?Specification and testing of OO-SW
The UML Testing ProfileTTCN-3 cases for OO
OOT-T3C © 2006 M. Winter
OO-Testing meets TTCN-3 13
Slide 25 Prof. Dr. Mario Winter, FH KölnTTCN3UC Berlin 2006
Object-Oriented Testing meets TTCN-3
Where are we now?
• Why is testing object-oriented software difficult?• Specification and testing of object-oriented software• The UML Testing Profile• TTCN-3 cases for OO
Slide 26 Prof. Dr. Mario Winter, FH KölnTTCN3UC Berlin 2006
Object-Oriented Testing meets TTCN-3
Example: Automated Teller Machine (ATM)
System Under Test(SUT)ATM
Main Test Component
(MTC)
Bank User
Parallel TestComponent
(PTC)
Bank ServerUser
InterfaceNetworkInterface
Test Coordination
Test System
Why is testing OO-SW difficult? Specification and testing of OO-SW
The UML Testing ProfileTTCN-3 cases for OO
NetworkInterface
UserInterface
Bank ServerBank User at ATM
Ina Schieferdecker: The UML 2.0 Testing Profile. Proc. TAV 19, Erlangen, 2003
OOT-T3C © 2006 M. Winter
OO-Testing meets TTCN-3 14
Slide 27 Prof. Dr. Mario Winter, FH KölnTTCN3UC Berlin 2006
Object-Oriented Testing meets TTCN-3
ATM system test scenario
Menu selection
Verify Amount& write ec-card
Take ec-card
Card readerEjectmoney
Take
& Eject ec-card
amount
emptyMoney
Input
Why is testing OO-SW difficult? Specification and testing of OO-SW
The UML Testing ProfileTTCN-3 cases for OO
Identification
Slide 28 Prof. Dr. Mario Winter, FH KölnTTCN3UC Berlin 2006
Object-Oriented Testing meets TTCN-3
UTP: ATM system test class diagram
ATM«import»
ATMTest
«testSuite»ATMSuite
-verdict : Verdict-amount : IMoney-targetBank : SwiftId-targetAccount : String-sourceAccount : String
«testCase» +validWiring() : Verdict«testCase» +invalidPIN() : Verdict«testCase» -authorizeCard() : Verdict
*
-accounts«testComponent»BankEmulator
IBank
«interface»IAccount
-pinOk : Boolean-enteredPIN : String-message : String-t1 : Timer
«testComponent »HWEmulator
hwComIATM
IHardware
Test packageTest packageTest componentTest component
MiscellaneousMiscellaneousTest suite
with test casesTest suite
with test cases
Why is testing OO-SW difficult? Specification and testing of OO-SW
The UML Testing ProfileTTCN-3 cases for OO
OOT-T3C © 2006 M. Winter
OO-Testing meets TTCN-3 15
Slide 29 Prof. Dr. Mario Winter, FH KölnTTCN3UC Berlin 2006
Object-Oriented Testing meets TTCN-3
UTP: ATM system test case behavioral specificationsd invalidPIN
HardwareEmulator
«sut»atm
storeCardData(current)
display(”Enter PIN”)
isPinCorrect(invalidPIN)
«validationAction»pass
BankEmulator
{readOnly} Integer invalidPIN; { current.isPinCorrect(invalidPIN) == false }
isPinCorrect(invalidPIN)
display(”Invalid PIN”)
display(”Enter PIN again”)
isPinCorrect : falseisPinCorrect : false
t1(2.0)
t1
{0 .. 3}
SUT and Test Component Lifelines
SUT and Test Component Lifelines
StimulusStimulus
ObservationObservation
Timing annotations
Timing annotations
Arbitrated verdict
Arbitrated verdict
Data partitionData partition
Why is testing OO-SW difficult? Specification and testing of OO-SW
The UML Testing ProfileTTCN-3 cases for OO
Slide 30 Prof. Dr. Mario Winter, FH KölnTTCN3UC Berlin 2006
Object-Oriented Testing meets TTCN-3
Where are we now?
• Why is testing object-oriented software difficult? • Specification and testing of object-oriented software• The UML Testing Profile • TTCN-3 cases for OO
OOT-T3C © 2006 M. Winter
OO-Testing meets TTCN-3 16
Slide 31 Prof. Dr. Mario Winter, FH KölnTTCN3UC Berlin 2006
Object-Oriented Testing meets TTCN-3
TTCN-3 – The standard
Why is testing OO-SW difficult? Specification and testing of OO-SW
The UML Testing Profile TTCN-3 cases for OO
Presentation Format n
Graphical Format
Tabular Format
Other Types & Values 2
IDL
ASN.1 Types & Values
Other Types & Values n
Presentation Format nUML Testing Profile
…
C++
msc mi_synch1_conc1
mtc ISAP1 MSAP2
TTCN-3CoreLanguage
:testcase myTestcase () runs on MTCType system TSIType{ mydefault := activate (OtherwiseFail);
verdict.set(pass);:connect(PTC_ISAP1:CP_ISAP1,mtc:CP_ISAP1);:map(PTC_ISAP1:ISAP1, system:TSI_ISAP1);:PTC_ISAP1.start(func_PTC_ISAP1());PTC_MSAP2.start(func_PTC_MSAP2());Synchronization(); all component.done;log(”Correct Termination”);
}:
Slide 32 Prof. Dr. Mario Winter, FH KölnTTCN3UC Berlin 2006
Object-Oriented Testing meets TTCN-3
Eclipse Test & Performance Tools Platform (TPTP)
• Object oriented testing with TTCN-3• Open source top level project of the Eclipse foundation• Includes the HYADES TTCN-3 tool set
– TTCN-3 Core Language Editor– TTCN-3 Compiler
• TPTP is divided into four projects. – TPTP Platform – Monitoring Tools– Testing Tools– Tracing and Profiling Tools
• But– TTCN-3 integration with least common denominator principle – Many powerful TTCN-3 concepts were left out!
http://www.eclipse.org/tptp
Why is testing OO-SW difficult? Specification and testing of OO-SW
The UML Testing Profile TTCN-3 cases for OO
OOT-T3C © 2006 M. Winter
OO-Testing meets TTCN-3 17
Slide 33 Prof. Dr. Mario Winter, FH KölnTTCN3UC Berlin 2006
Object-Oriented Testing meets TTCN-3
Using C/C++ with TTCN-3
• Draft ETSI ES 201 873-10 Methods for Testing and Specification (MTS); The Testing and Test Control Notation version Part 10: Using C/C++ with TTCN-3 (lead author: Jens Grabowski)
Why is testing OO-SW difficult? Specification and testing of OO-SW
The UML Testing Profile TTCN-3 cases for OO
Slide 34 Prof. Dr. Mario Winter, FH KölnTTCN3UC Berlin 2006
Object-Oriented Testing meets TTCN-3
What did we reach with TTCN-3?
• OO-Testing = Message Sequences = TTCN-3• Excellent notation for black-box test scripts
– Modular– Reusable
• Excellent tool support for TTCN-3 specification and execution– Commercial and open source– Different domains (communication, automotive, web, ...) well covered
Why is testing OO-SW difficult? Specification and testing of OO-SW
The UML Testing Profile TTCN-3 cases for OO
OOT-T3C © 2006 M. Winter
OO-Testing meets TTCN-3 18
Slide 35 Prof. Dr. Mario Winter, FH KölnTTCN3UC Berlin 2006
Object-Oriented Testing meets TTCN-3
What is still missing regarding OO-Testing?
• Subtyping for TTCN-3 structures– Data structures– Modules– Test Cases– ...
• Full UML testing profile binding– Subclassing (i.e. generalization) – Tool Support w.r.t. full UML support
• Java Binding („Using Java with TTCN-3“)
• More wishes?
Why is testing OO-SW difficult? Specification and testing of OO-SW
The UML Testing Profile TTCN-3 cases for OO
Rectangle
- side1- side2
+ setSide1()+ setSide2()+ getArea()
Rectangle
- side1- side2
+ setSide1()+ setSide2()+ getArea()
Square
- side1- side2
+ setSide1()+ setSide2()+ getArea()
Square
- side1- side2
+ setSide1()+ setSide2()+ getArea()
Slide 36 Prof. Dr. Mario Winter, FH KölnTTCN3UC Berlin 2006
Object-Oriented Testing meets TTCN-3
The Future: MDA and MBT
UML(EDOC)
SystemsourceArchitecture
UTP/TTCN-3
Testcode
UTP/TTCN-3
Why is testing OO-SW difficult? Specification and testing of OO-SW
The UML Testing ProfileTTCN-3 cases for OO
ModelDrivenArchitecture
ModelBasedTesting
OOT-T3C © 2006 M. Winter
OO-Testing meets TTCN-3 19
Slide 37 Prof. Dr. Mario Winter, FH KölnTTCN3UC Berlin 2006
Object-Oriented Testing meets TTCN-3
Summary
• Why is testing object-oriented software difficult?– Inheritance– Polymorphism, late binding
• Specification and testing of object-oriented software– Unified Modeling Language– Testing techniques
• The UML Testing Profile– Black box testing notation– ATM case study
• TTCN-3 cases for OO– Eclipse TPTP– Using C/C++ with TTCN-3
Slide 38 Prof. Dr. Mario Winter, FH KölnTTCN3UC Berlin 2006
Object-Oriented Testing meets TTCN-3
Program (Object-Oriented)
Testcase(TTCN-3)