1

CS 432 Object-Oriented Analysis and Design

Week 7Testing

2

Software Testing

Testing is the process of exercising aTesting is the process of exercising aprogram with the specific intent of findingprogram with the specific intent of findingerrors prior to delivery to the end user.errors prior to delivery to the end user.

“You can’t test in quality. If it’s not there before you begintesting, it won’t be there when you’re finished testing.”

3

Testing

Testing is a process of identifying defects Develop test cases and test data

A test case is a formal description of A starting state One or more events to which the software must

respond The expected response or ending state

Test data is a set of starting states and events used to test a module, group of modules, or entire system

4

Testing discipline activities

5

Testing is Use Case Driven

Terms associated with testing: Verification – are we building the system right? Validation – are we building the right product?

Testing the Use Case Model is a validation task that determines whether the customer’s needs are being met by the application.

Testing of remaining models is a verification task that determines whether the application correctly satisfies the requirements specified in the Use Case Model

Use-Case Model

Design Model

Implem Model

Test Model«trace»

XTest Case

7

Testing Workflow

Defines the primary activities required to develop the Test Model: Develop Test Plan

Outline the testing strategies to be used including model tests,

Estimating the resources required to complete the testing activities,

Scheduling the testing as part of the overall project plan.

8

Design Model Tests Determine strategies for testing a model’s

correctness, Determine strategies for testing a model’s

completeness, Determine strategies for testing a model’s

consistency, Design Test

Identify and describe test cases for each build Identify and structure test procedures specifying

how to perform or carry out the test cases that test the executable application components and their integration,

Testing Workflow

9

Implement Test – automate test procedures by creating test components that define executable components that automate all or part of the test procedures, if possible

Perform Tests – execute the tests cases as part of: an iteration release or implementation build including:

Unit testing is the white-box technique in which programmers verify that the software they implemented satisfies its design requirements (see Section 6.3)

Regression testing verifies that changes to one part of an application do not break other previous working parts/components

of the application,

Testing Workflow

10

Integration testing verifies the communication and cooperation among major system components that have been previously unit tested,

User acceptance testing tests whether the users can actually use the system and the difficulties they encounter when doing so (often a Cognitive Psychology issue),

Stress testing ensures the system can handle significant loads, e.g., 5,000 simultaneous users.

Performance testing ensures the system meets performance objectives, e.g., response within 3 seconds.

Evaluate Test – evaluate the results of the tests and the testing strategy.

Testing Workflow

11

Testing Categories

black-box testing techniques use only functional knowledge of an entity being tested. E.g. only knowledge on use cases, or interfaces

white-box testing techniques utilize knowledge of the internal control structure logic of the entity being tested. Behavioral diagrams capture some aspect of

the internal behavior of how an entity realizes the functionality associated with a use case.

12

Models – How to Test

In general, develop use cases that address these issues: correctness – tests both the syntactic and

semantic accuracy of a model. From a syntactic perspective, the model must be

tested to verify that the various UML elements used within the model are used correctly.

From a semantic perspective, the elements within the model must accurately correspond to the reality being represented by the model.

13

completeness – tests whether a model is complete by verifying whether any required elements are missing from the model. This is typically accomplished by determining whether the model can appropriately handle scenarios developed for this purpose.

consistency – tests whether the elements in a model are used in a conflicting way. It also verifies that the elements of a model don’t contradict with the elements of other models on which it depends.

Models – How to Test

14

Use Case Model

The testing of the Use-Case Model must consider the following issues: Correctness – does each use case accurately

represent a requirement? Completeness – do the use cases represent all of

the functionality needed for a satisfactory product Consistency – generally this requires examining

extension and included use cases to ensure that their relation to other use cases is consistent. Naturally, it’s possible to declare two use cases that contradict each other, but this is rare in practice.

15

Analysis Model Testing

Testing the analysis model amounts to determining whether the application being modeled correctly interprets the application domain:

Correctness – the description of the domain concepts are accurate; the algorithms will produce the expected results. The concepts and algorithms cover the use cases

Completeness – the concepts are sufficient to cover the scope of the content specified. Sufficient detail is given to describe concepts to the required depth. Experts agree with the attributes and behaviors assigned to each class.

Consistency – Model elements should be consistent with the business’s definitions and meanings. Where there are multiple ways to represent a concept of action, those ways should be modeled equivalently.

16

Design Model Testing

Testing the design model is conceptually similar to testing the analysis model, but also requires addressing the following issues:

Correctness – Each class accurately implements the semantics of an interface. Classes corresponding to interfaces must implement the interface

Completeness – classes are defined for each interface in the architecture. Preconditions for method use are appropriate specified. Post-conditions and error conditions are specified.

Consistency – The behaviors in the interface of each class provides either a single way to accomplish a task or, of there are multiple ways, they provide the same behavior, but with different preconditions

17

Testing an object-oriented design presents some additional challenges not typically found in the testing of procedure programming designs:

Interfaces: Classes implement interfaces and it’s necessary to ensure that the class correctly implements the functionality required by its interface.

Inheritance: The use of inheritance also introduces coupling problems in which a change to one class results in a change to all of its subclasses. This requires ensuring that the change that is now being inherited is appropriate for all of the subclasses. Delegation can often alleviate many of the issues associated with inheritance.

Delegation: The delegation of tasks to other objects, though similar to the use of modules in procedure programming languages, also presents its own set of issues. In a class, the delegation is encapsulated behind a public interface, hence its necessary to ensure that the implementation of the interface is still correct when the class to which the functionality has been delegated is changed.

Design Model Testing

18

Implementation Model Testing

Testing the implementation model focuses on verifying that the implemented code satisfies the design in the design model (and the requirements of the use-case model).

Correctness – The executable components of the system correctly build (e.g., compile and deploy) and adhere to the design model.

Completeness – the executable components provide all of the functionality specified in the design model.

Consistency – The executable components of the system are appropriately integrated in a way that provides the functionality specified by the use-case requirements

19

Testing Framework

The object-oriented concepts you have learned in this class can easily be used to develop a simple testing framework for any application you are building.

The execution of such tests will be automatically performed every night as part of the most recent build process.

20

Example – Test Method

#nextId() : int+find(in id : int) : Account+save(in account : Account) : void

AccountCatalog

#nextId() : int+verifyTest() : boolean+find(in id : int) : Account+save(in account : Account) : void

AccountCatalog

With Method:

Example – Test Handler

+verifyTest() : boolean+assert(in method : String, in target : Object, in parameters : Object[], in value : Object) : boolean

#log : FileInputStream

TestHandlerImpl

+verifyTest() : boolean

«interface»TestHandler

«implement»

•Create new class that handles the common testing behaviors•This can be done with Java, C# but not C++• Specifies that the execution of the given message should result in a returned value equal to the given object

verifyTest() - AccountCatalog

public verifyTest() { // Get a newly created unique account id int id = nextId();

// Create a new account object with this id. Account account = new Account(id);

// Save the newly created account object save(account); // Object[] parameters = new Object[1]; parameters[0] = new Integer(id);

// Assert that the execution of the find method // on the current AccountCatalog object (this) // with the single id parameter, should return // a value equal to the account object. If not, // assert will log an appropriate error message // to the log file. assert(“find”, this, parameters, account);

23

General Testing Methodologies

24

Figure 13-3: Test types and detected defects

25

Unit Testing

The process of testing individual methods, classes, or components before they are integrated with other software

Two methods for isolated testing of units Driver

Simulates the behavior of a method that sends a message to the method being tested

Stub Simulates the behavior of a method that has not yet

been written

26

Unit Testing

modulemoduleto beto betestedtested

test casestest cases

resultsresults

softwaresoftwareengineerengineer

27

Unit Testing

interface interface

local data structureslocal data structures

boundary conditionsboundary conditions

independent pathsindependent paths

error handling pathserror handling paths

modulemoduleto beto betestedtested

test casestest cases

28

Unit Test Environment

ModuleModule

stubstub stubstub

driverdriver

RESULTSRESULTS

interface interface

local data structureslocal data structures

boundary conditionsboundary conditions

independent pathsindependent paths

error handling pathserror handling paths

test casestest cases

29

Test Cases

Test cases should uncover errors such as: Comparison of different data types Incorrect logical operators or precedence Expectation of equality when precision error

makes equality unlikely Incorrect comparison of variables Improper or nonexistent loop termination Failure to exit when divergent iteration is

encountered Improperly modified loop variables

30

Integration Testing

Evaluates the behavior of a group of methods or classes Identifies interface compatibility, unexpected

parameter values or state interaction, and run-time exceptions

System test Integration test of the behavior of an entire

system or independent subsystem Build and smoke test

System test performed daily or several times a week

31

Integration Testing Strategies

Options:Options:•• the “big bang” approachthe “big bang” approach•• an incremental construction strategyan incremental construction strategy

32

Top Down Integration

top module is tested with top module is tested with stubsstubs

stubs are replaced one at stubs are replaced one at a time, "depth first"a time, "depth first"

as new modules are integrated, as new modules are integrated, some subset of tests is re-runsome subset of tests is re-run

AA

BB

CC

DD EE

FF GG

Main disadvantage -> is the need for stubs and attendant testing difficulties that can be associated with them. Should test majorcontrol functions early to help with this.

33

Bottom-Up Integration

drivers are replaced one at a drivers are replaced one at a time, "depth first"time, "depth first"

worker modules are grouped into worker modules are grouped into builds and integratedbuilds and integrated

AA

BB

CC

DD EE

FF GG

clustercluster

Major disadvantage -> the program as an entity does not exist until the last module is added.

34

Usability Testing

Determines whether a method, class, subsystem, or system meets user requirements

Performance test Determines whether a system or subsystem

can meet time-based performance criteria Response time specifies the desired or maximum

allowable time limit for software responses to queries and updates

Throughput specifies the desired or minimum number of queries and transactions that must be processed per minute or hour

35

User Acceptance Testing

Determines whether the system fulfills user requirements

Involves the end users

Acceptance testing is a very formal activity in most development projects

36

Object-Oriented Testing

begins by evaluating the correctness and consistency of the OOA and OOD models

testing strategy changes the concept of the ‘unit’ broadens due to

encapsulation integration focuses on classes and their execution

across a ‘thread’ or in the context of a usage scenario validation uses conventional black box methods

test case design draws on conventional methods, but also encompasses special features

37

Who Tests Software?

Programmers Unit testing Testing buddies can test other’s programmer’s

code Users

Usability and acceptance testing Volunteers are frequently used to test beta

versions Quality assurance personnel

All testing types except unit and acceptance Develop test plans and identify needed changes

38

Debugging: A Diagnostic Process

39

The Debugging Process

test casestest cases

resultsresults

DebuggingDebugging

suspectedsuspectedcausescauses

identifiedidentifiedcausescauses

correctionscorrections

regressionregressionteststests

new testnew testcasescases

40

Debugging Effort

time requiredtime requiredto diagnose theto diagnose thesymptom andsymptom anddetermine thedetermine thecausecause

time requiredtime requiredto correct the errorto correct the errorand conductand conductregression testsregression tests

41

Symptoms & Causes

symptomsymptom

causecause

symptom and cause may be symptom and cause may be geographically separated geographically separated

symptom may disappear when symptom may disappear when another problem is fixedanother problem is fixed

cause may be due to a cause may be due to a combination of non-errors combination of non-errors

cause may be due to a system cause may be due to a system or compiler erroror compiler error

cause may be due to cause may be due to assumptions that everyone assumptions that everyone believesbelieves

symptom may be intermittentsymptom may be intermittent

42

Consequences of Bugs

damagedamage

mildmildannoyingannoying

disturbingdisturbingseriousserious

extremeextremecatastrophiccatastrophic

infectiousinfectious

Bug TypeBug Type

Bug Categories:Bug Categories: function-related bugs, function-related bugs, system-related bugs, data bugs, coding bugs, system-related bugs, data bugs, coding bugs, design bugs, documentation bugs, standards design bugs, documentation bugs, standards violations, etc.violations, etc.

43

Debugging Techniques

brute force / testingbrute force / testing-- run-time traces,output statements-- run-time traces,output statements

backtrackingbacktracking-- start where error is found and work-- start where error is found and work

backwardsbackwards

inductioninduction -- cause elimination, binary partitioning-- cause elimination, binary partitioning

deduction deduction

44

Debugging: Final Thoughts

Don't run off half-cocked, Don't run off half-cocked, thinkthink about the about the symptom you're seeing.symptom you're seeing.

Use toolsUse tools (e.g., dynamic debugger) to gain (e.g., dynamic debugger) to gain more insight.more insight.

If at an impasse, If at an impasse, get helpget help from someone else.from someone else.

Be absolutely sure to Be absolutely sure to conduct regression testsconduct regression tests when you do "fix" the bug.when you do "fix" the bug.

1.1.

2.2.

3.3.

4.4.

45

Configuration and Change Management

Controls the complexity associated with testing and supporting a system through multiple development and operational versions

Integrally related to project management, implementation, testing, and deployment activities

Change control procedures are typically developed in the first iteration before development

Need for formal procedures depends on size and cohesiveness of project

46

Figure 13-7 Configuration and change management discipline activities

47

Versioning

Alpha version Test version that is incomplete but ready for

some level of rigorous integration or usability testing

Beta Test version that is stable enough to be tested

by end users for an extended period of time Production version

System version that is formally distributed to users or made operational for long-term use

Maintenance release System update that provides bug fixes and

small changes to existing features

48

Submitting Change Requests and Error Reports

Typical change control procedures include Standard change request forms

Completed by a user or system owner

Review of requests by a change control committee Assess impact on system, security, and budget

Extensive planning for design and implementation

Bugs reports are often reported separately because of the need for an immediate fix

49

Figure 13-11: A sample change request form

50

Figure 13-12: A sample change review form

51

Planning and Managing Testing

Testing activities must be distributed throughout the project Unit and integration testing occur whenever

software is developed, acquired, or combined with other software

Usability testing occurs whenever requirements or design decisions need to be evaluated

User acceptance tests are conducted as a final validation of the requirements, design, and implementation activities

52

Development Order

Input, process, output (IPO) development Implements input modules first, process

modules next, and output modules last Important user interfaces are developed early

Top-down Implements top-level modules first There is always a working version of the

program Bottom-up

Implements low-level detailed modules first Programmers can be put to work immediately

53

Framework Development

Foundation classes Object framework that covers most or all of

the domain and data access layer classes Reused in many parts of the systems and

across applications Whenever possible, developers choose

use cases for early iterations that rely on many foundation classes Testing early finds bugs before dependent

code is developed

54

Direct Deployment

Installs a new system, quickly makes it operational, and immediately turns off any overlapping systems

Advantages Simplicity

Disadvantages Risk of system unavailability

Used when a new system is not replacing an old system and/or downtime can be tolerated

55

Direct Deployment and Cutover

56

Parallel Deployment

Operates both old and new systems for an extended time period

Advantages Relatively low risk of system failure

Disadvantage Cost to operate both systems

Used for mission-critical applications Partial parallel deployment can be

implemented with increased risk of undetected errors

57

Figure 13-24Parallel deployment and operation

58

Phased Deployment

Installs a new system and makes it operational in a series of steps or phases

Advantages Reduced risk

Disadvantages Increased complexity

Useful when a system is large, complex, and composed of relatively independent subsystems

59

Figure 13-25Phased deployment with direct cutover and parallel operation

60

Personnel Issues

New system deployment places significant demands on personnel

Temporary and contract personnel may be hired to increase manpower, especially during a parallel deployment System operators Personnel with experience in hardware or

software deployment and configuration Employee productivity decreases

temporarily with a new system due to the learning curve