Object Oriented TestingObject Oriented Testing

Ali Kamandi

(kamandi@ce.sharif.edu)

Sharif University of Technology

2

Object Oriented Testing Object Oriented Testing Find the greatest possible number of errors with a

manageable amount of effort applied over a realistic time span.

The nature of OO programs changes both testing strategy and testing tactics.

Do we need less efforts for testing because of greater reuse of design patterns?

Answer is no! Binder argues that more testing is needed to obtain high reliability in OO systems, since each reuse is a new context of usage.

3

Testing Object-Oriented Applications:Testing Object-Oriented Applications:Why is it Different?Why is it Different?

No sequential procedural executions

No functional decomposition

No structure charts to design integration testing

Iterative O-O development and its impact on testing and integration strategies

4

ObjectivesObjectives

To cover the strategies and tools associated with object oriented testing Analysis and Design Testing

Unit/Class Tests

Integration Tests

Validation Tests

System Tests

To discuss test plans and execution for projects

5

Object-Oriented TestingObject-Oriented Testing

When should testing begin? Analysis and Design:

Testing begins by evaluating the OOA and OOD models How do we test OOA models (requirements and use cases)? How do we test OOD models (class and sequence diagrams)? Structured walk-throughs, prototypes Formal reviews of correctness, completeness and consistency

Programming: How does OO make testing different from procedural programming? Concept of a ‘unit’ broadens due to class encapsulation Integration focuses on classes and their execution across a ‘thread’

or in the context of a use case scenario Validation may still use conventional black box methods

6

Criteria for Completion of TestingCriteria for Completion of Testing

When are we done testing? (Are we there yet?) How to answer this question is still a research question

1. One view: testing is never done… the burden simply shifts from the developer to the customer

2. Or: testing is done when you run out of time or money

3. Or use a statistical model: Assume that errors decay logarithmically with testing time Measure the number of errors in a unit period Fit these measurements to a logarithmic curve Can then say: “with our experimentally valid statistical model we

have done sufficient testing to say that with 95% confidence the probability of 1000 CPU hours of failure free operation is at least 0.995”

7

Strategic IssuesStrategic Issues

Issues to address for a successful software testing strategy: Specify product requirements long before testing commences

For example: portability, maintainability, usabilityDo so in a manner that is unambiguous and quantifiable

Understand the users of the software, with use cases

Develop a testing plan that emphasizes “rapid cycle testing” Get quick feedback from a series of small incremental tests

Build robust software that is designed to test itself Use assertions, exception handling and automated testing tools (Junit).

Conduct formal technical reviews to assess test strategy & test cases “Who watches the watchers?”

8

Testing OOA and OOD Models (1)Testing OOA and OOD Models (1)

The review of OO analysis and design models is especially useful because the same semantic constructs (e.g., classes, attributes, operations, messages) appear at the analysis, design, and code level.

Therefore, a problem in the definition of class attributes that is uncovered during analysis will circumvent side effects that might occur if the problem were not discovered until design or code (or even the next iteration of analysis).

By fixing the number of attributes of a class during the first iteration of OOA, the following problems may be avoided:

Creation of unnecessary subclasses. Incorrect class relationships. Improper behavior of the system or its classes.

If the error is not uncovered during analysis and propagated further more efforts needed during design or coding stages.

9

Testing OOA and OOD Models (2)Testing OOA and OOD Models (2)

Analysis and design models cannot be tested in the conventional sense, because they cannot be executed.

Formal technical review can be used to examine the correctness and consistency of both analysis and design models.

Correctness:

Syntax: Each model is reviewed to ensure that proper modeling conventions have been maintained.

Semantic: Must be judged based on the model’s conformance to the real world problem domain by domain experts.

Consistency:

May be judged by considering the relationship among entities in the model.

Each class and its connections to other classes should be examined.

The Class-responsibility-collaboration model and object-relationship diagram can be used.

10

Testing ModelsTesting Models

CriteriaCorrectness

Completeness

Consistency

Early informal models are tested informally

The criteria should be interpretive in the context of iterative incremental approach

11

Model Testing ApproachesModel Testing Approaches

Testing by comparisoncompares each model to its predecessor or to previous

forms of the model

Testing by inspectionuses checklists to make sure that the model meets certain

criteria

Testing by verificationfollows certain steps to assure completeness and

consistency of one part of the model with another

12

Examples of Analysis and Design Examples of Analysis and Design Models to be TestedModels to be Tested

CRC cards

Class specifications

Use cases

State-Transition ModelsState transition diagrams for classes, clusters, and subsystems

Object networkMessage sequence between methods in classes

Transaction-Flow Models

13

Testing the Class ModelTesting the Class Model

1. Revisit the Use Cases, CRC cards and UML class model. Check that all collaborations are properly represented. Inspect the description of each CRC index card to determine if a delegated responsibility is part of the collaborator’s definition Example: in a point of sale system. A read credit card responsibility of a credit sale class is accomplished if

satisfied by a credit card collaborator

2. Invert connections to ensure that each collaborator asked for a service is receiving requests from a reasonable source Example: a credit card being asked for a purchase amount

Have you tested your analysis and design? If not, who will do it?

3. These steps are applied iteratively to each class and through are applied iteratively to each class and through each evolution of the OOA model.each evolution of the OOA model.

14

Testing OO CodeTesting OO Code

Class/Unit testsClass/Unit tests IntegrationIntegrationteststests

ValidationValidationteststests

SystemSystemteststests

15

Requirement Definition

Preliminary Design

Detailed Design

Coding

Integration Testing

Unit Testing

Systems Testing

The Structured Testing PyramidThe Structured Testing Pyramid

16

Use Case Analysis

Class Hierarchy Design

Method Design

Method Coding

Cluster (Integration) Testing

Method Testing

Use Case Testing

Class Testing

The OO Testing PyramidThe OO Testing Pyramid

Test OOD

Test OOA

Test OOD

17

Unit TestUnit Test

What is a unit?A single, cohesive function?

A function whose code fits on one page?

The amount of code that can be written in 4 to 40 hours?

Code that is assigned to one person?

We can no longer test a single operation in isolation but rather as part of a class.

In object-oriented programs, a unit is a method within a class.

Smallest testable unit is the encapsulated class

18

Class Testing ProcessClass Testing Process

classclassto beto be

testedtested

test casestest cases

resultsresults

softwaresoftwareengineerengineer

How to test?How to test?

Why a Why a loop?loop?

19

Methods for Generating Test CasesMethods for Generating Test CasesFor Unit Testing For Unit Testing

Statement coverage

Graph basedBranch coverage

Condition coverage

Path coverage

All unit testing methods are also applicable to testing methods within a class.

20

Class Test Case DesignClass Test Case Design

Berard [BER93,94] proposes the following approach:

1. Identify each test case uniquely - Associate test case explicitly with the class and/or method to be tested

2. State the purpose of the test

3. Each test case should contain:

a. list of specified states for the object that is to be tested

b. A list of messages and operations that will be exercised as a consequence of the test

c. A list of exceptions that may occur as the object is tested

d. A list of external conditions for setup (i.e., changes in the environment external to the software that must exist in order to properly conduct the test)

e. Supplementary information that will aid in understanding or implementing the test

Automated unit testing tools facilitate these requirements

21

Challenges of Class TestingChallenges of Class Testing OO class is the target for test case design. Encapsulation:

Difficult to obtain a snapshot of a class without building extra methods which display the classes’ state

Inheritance and polymorphism: Each new context of use (subclass) requires re-testing because a method

may be implemented differently (polymorphism). Other unaltered methods within the subclass may use the redefined method

and need to be tested

How is class testing different from conventional testing?- Conventional testing focuses on input-process-output, whereas class testing

focuses on each method, then designing sequences of methods to exercise states of a class

White box tests: Basis path, condition, data flow and loop tests can all apply to individual

methods, but don’t test interactions between methods

22

Notes on OO Testing 1.Notes on OO Testing 1.

Applicability of conventional test case design:

Although white-box testing is useful, the efforts might be redirected to tests at a class level.

Use cases can provide useful input in the design of black-box and state-based tests.

Fault-based testing: (Tests have high likelihood of uncovering faults)

Its planning begins with the analysis model.

If real faults in an OO system are perceived to be implausible then this approach is really no better than any random testing technique.

If the analysis and design models can provide insight into what is likely to go wrong, then fault-based testing can find significant numbers of errors with relatively low efforts.

23

Notes on OO Testing 2.Notes on OO Testing 2.

Impact of OO programming on Testing: When an operation is invoked, it may be hard to tell exactly what code

gets exercised. It can be hard to determine the exact type or class of a parameter. OO operations are smaller, more time needed for integration. So

integration faults become more plausible. In 3 large scale studies: 50% of methods are less than 2 C++ statements or 4

Smalltalk statements (Law of Demeter connection)

Scenario-based test design:Concentrates on what the user does, not what the product does.Capturing the tasks via use-cases that the user has to perform.

Testing surface structure and deep structureSurface structure: externally observable structure (end-user).Deep structure: internal technical details of an OO program.

24

OOT Methods: Class Level 1OOT Methods: Class Level 1 Random testing

identify operations applicable to a class

define constraints on their use

identify a minimum test sequence

an operation sequence that defines the minimum life history of the class (object)

generate a variety of random (but valid) test sequences

exercise other (more complex) class instance life histories

Example:

Class: Account

Operations: open, setup, deposit, withdraw, balance, summarize, creditlimit, close.

1. Open – setup – deposit – withdraw – close

2. Open – setup – deposit –[deposit | withdraw | balance | summarize] * – withdraw – close. Generate random test sequences using this template

25

OOT Methods: Class Level 2OOT Methods: Class Level 2 Partition Testing

reduces the number of test cases required to test a class in much the same way as equivalence partitioning for conventional software

state-based partitioning

categorize and test operations based on their ability to change the state of a class (e.g.: deposit, withdraw)

attribute-based partitioning

categorize and test operations based on the attributes that they use ( e.g.: creditlimit attribute)

category-based partitioning

categorize and test operations based on the generic function each performs. (e.g.: (Init OP: open , setup) (Comp. OP: deposit, withdraw) (queries: balance, summarize, creditlimit) (Termination OP: close))

26

What Methods to Test Within ClassesWhat Methods to Test Within Classes

New methods: defined in the class under test and not inherited or overloaded by methods in a superclass: Complete testing

Inherited methods: defined in a superclass of the class under test: Retest only if the methods interacts with new or redefined method.

Redefined methods: defined in a superclass of but redefined in the class under test : complete Retest reusing tests from the superclass.

27

Class Testing TechniquesClass Testing Techniques

In addition to testing methods within a class (either glass box or black box), the following three techniques can be used to perform functional testing for the class:State-Transition testing

Transaction-Flow testing

Exception testing

28

State-Transition TestingState-Transition Testing

A state-transition model describes the different states and transitions of a class in the context of its position in the inheritance hierarchy.

(OMT dynamic model)

The state of an object is the combination of all the attribute values and objects that the object contains.

An object may transition from a state to state as a result of an event, which may yield an action.

29

ExampleExample

Prospect

Retired-member Life-member

MemberReceive application

Establish-membership

Receive cancellation

Receive cancellation5-years anniversary

30

State-Transition TestingState-Transition Testing

Create test cases corresponding to each transition path that represent a full object life cycle

Make sure each transition is exercised at least once.

31

[2] Integration Testing[2] Integration Testing

OO does not have a hierarchical control structure so conventional top-down and bottom-up integration tests have little meaning

Integrating operations one at a time into a class is often impossible because of the direct and indirect interactions of the components that make up the class.

Integration applied three different incremental strategies: Thread-based testing: integrates classes required to respond to one

input or event

Use-based testing: integrates classes required by one use case

Cluster testing: integrates classes required to demonstrate one collaboration

- What integration testing strategies will you use?

32

Cluster (Integration) Testing:Cluster (Integration) Testing:Why is it Different?Why is it Different?

Event-Driven execution

No functional decomposition

The sequence of execution is not known

Integration testing must be driven by how object will behave dynamically

Object composition introduces a new dimension of integration testing

33

Random Integration TestingRandom Integration Testing

Multiple Class Random Testing1. For each client class, use the list of class methods to

generate a series of random test sequences. Methods will send messages to other server classes.

2. For each message that is generated, determine the collaborating class and the corresponding method in the server object.

3. For each method in the server object (that has been invoked by messages sent from the client object), determine the messages that it transmits

4. For each of the messages, determine the next level of methods that are invoked and incorporate these into the test sequence

34

Cluster (Integration) TestingCluster (Integration) Testing A cluster is a collection of classes (possibly from different

systems) cooperating with each other via messaging.

A cluster specification should include methods from each class that will be accessed

Cluster testing focuses on the interaction among the instances of the classes in the cluster

It assumes that each class has been tested individually

Cluster testing is considered a second level of integration testing

35

Methods for Forming ClustersMethods for Forming Clusters

Function-based clusteringBased on requirements and use cases

Difficult to perform if requirements were not available during the design phase

Subject-based clusteringBased on subject areas that need to test separately

Project Schedule-based clustering

Contract-based clustering

36

Types of Errors Found During Types of Errors Found During Integration TestingIntegration Testing

Messaging errors:Failure to meet a requirement, i.e., no method to send or

receive a message

Incompatible method and message in sender and receiver

Incorrect event timing between object actions

Incorrect instantiation or destruction of objects

37

Types of Errors Found During Types of Errors Found During Integration Testing Integration Testing (cont.)(cont.)

User interface errors:A given sequence of user actions does not have the

expected effect on the component.

The timing of events received from the user results in incorrect functioning of the component

38

Techniques for Object-Oriented Techniques for Object-Oriented Integration TestingIntegration Testing

Message Quiescence

Event Quiescence

39

Message QuiescenceMessage Quiescence

A Method/Message path (MM-Path) is a sequence of method executions linked by messages.

An MM-Path starts with a method and ends when it reaches a method that does not issue a message of its own, i.e., reaches a message Quiescence.

40

Event QuiescenceEvent Quiescence

An input port event followed by a set of MM-Paths, and terminated by an output event is called Atomic System Function (ASF).

41

meth1

meth3meth2

Class 1

INPUT PORT EVENT A

B

ASF INPUT PORT EVENT

OUTPUT PORT EVENTA

meth1 meth3

meth2

meth2

meth1

B

ASF OUTPUT PORT EVENTClass 2

Class 3

MM-Path

Message

1

2

3

42

ATM PIN EntryATM PIN Entry

Customer enters card (event)

Screen requesting PIN entry is displayed

An interleaved sequence of digit key toughs with audible and visual feedback

Possibility of cancellation by customer

System disposition (valid PIN or card retained)

43

BANKCARDSLOT

SECURITY

SCREEN

Keypad

SpecialKeypad

NumKeypad

44

getKeyEvents

parseKeyEvent

showMessage

pinForPan

checkPin

Screen

memberCard

ValidateCard

CardSlot

Bank

NumKeypadSecurity

Customer inserts card

ASF Starts here

Message is displayed

ASF ends here

Key pushers

45

[3] Validation Testing[3] Validation Testing

Are we building the right product?

Validation succeeds when software functions in a manner that can be reasonably expected by the customer.

Focus on user-visible actions and user-recognizable outputs

Details of class connections disappear at this level

Apply: Use-case scenarios from the software requirements spec

Black-box testing to create a deficiency list

Acceptance tests through alpha (at developer’s site) and beta (at customer’s site) testing with actual customers

How will you validate your software product?

46

[4] System Testing[4] System Testing

Software may be part of a larger system. This often leads to “finger pointing” by other system dev teams

Finger pointing defence:1. Design error-handling paths that test external information

2. Conduct a series of tests that simulate bad data

3. Record the results of tests to use as evidence

Types of System Testing: Recovery testing: how well and quickly does the system recover

from faults Security testing: verify that protection mechanisms built into the

system will protect from unauthorized access (hackers, disgruntled employees, fraudsters)

Stress testing: place abnormal load on the system Performance testing: investigate the run-time performance within

the context of an integrated system

47

System TestingSystem Testing

All rules and methods of traditional systems testing are also applicable applicable to object-oriented.

A thread can be “re-defined” as a sequence of method executions linked by messages in the object network.

Use cases provide a more precise mechanism for writing test cases.

48

ExampleExample

Use Case # 1 of the Telephone Banking System:

Task: Making a balance transfer

User Class: Current customer

Frequency: one per week per customer

User Action System Response

User dials the number System plays greeting and ask for account number

User enters invalid account number System informs user and ask for account number again

User enter a valid account number System asks for PIN#

49

Automated TestingAutomated Testing

Junit at Junit.org

CPPUnit on SourceForge.net,

NUnit for .Net

Rational Robot

50

OO Testing MethodologyOO Testing Methodology

Jorgensen and Erickson propose 5 levels:

A Method Unit testing

Message Quiescence Integration

Event Quiescence Integration

Thread testing System

Thread interaction System

51

Stages of Test [RUP] Stages of Test [RUP]

Developer Testing

Unit Test

Integration Test

System Test

Acceptance Test

52

Test Planning [RUP]Test Planning [RUP]

What iteration you are you in?

What stage of test (unit test, integration test, system test) you are performing?

Types of test (functional, stress, volume, performance, usability, distribution, and so on).

Evaluation criteria used (code-based test coverage, requirements-based test coverage, number of defects, mean time between failure, and so on.)

Testing techniques used (manual and automated)

53

Completion Criteria [RUP]Completion Criteria [RUP]

54

Manual/Automated Testing [RUP]Manual/Automated Testing [RUP]

55

The END