1

Software Testing and Quality Assurance

Lecture 14 - Planning for Testing (Chapter 3, A Practical Guide to Testing Object-

Oriented Software)

2

Lecture Outline To learn how to analyze the risks

associated with verifying the required functionality.

Adequacy of test cases.

3

Risk analysis ―A tool for testing Risk analysis is part of planning and

development effort. A risk ―anything that threatens the

successful achievement of a project’s goals

A risk is an event that has some probability of happening and, if it occurs, there will be some loss (down time, financial).

4

Risk analysis ―A tool for testing

Fundamental principle for risk-based testing Test most heavily those portions of the

system that pose the highest risks to the project to ensure that the most harmful faults are identified.

5

Risk analysis ―A tool for testing: risk types

Project risks: include managerial and environmental risks (e.g. insufficient supply of qualified personnel).

Business risks: are associated with domain-related concepts. This type of risk is related to the functionality of the program (e.g. changes on the health insurance policy).

6

Risk analysis ―A tool for testing: risk types

Technical risks: include some implementation concepts (e.g. the quality of the code).

7

Risk analysis ―A tool for testing: risk analysis Risk analysis ―is a procedure for identifying

risks and for identifying ways to prevent problems from becoming real.

The output of risk analysis is a list of identified risks in the order of the level of risk that can be used to allocate limited resources and

to prioritize decisions.

8

Risk analysis ―A tool for testing: risk analysis Risks in OO software projects are unique to

the architectural features: Complex interactions among objects Complex behavior associated with a class

specification Changing or evolving project requirement Complexity of a class measured by the size of its

specification The number of relationships a class has with other

classes

9

Risk analysis ―A tool for testing: risk analysis (cont...)

Source of risks: For system testing, the various uses of the

system are prioritized based on the importance to the user and proper operation of the system.

Risks are also associated with the programming language and development tools that are being used to implement the software (strong typed vs. weak typed languages).

10

Risk analysis ―A tool for testing: risk analysis (cont...)

Conducting the analysis: Risk analysis technique includes three

tasks: Identify the risks that each use case poses to

the development effort (classify as low, medium and high –may be also very high)

Quantify the risk Produce a ranked list of use cases

11

Risk analysis example Brickles example

You may combine the frequency and the criticality to determine which should be tested more heavily.

Use Risk Level

Frequency

Criticality

Scenario

Wins Medium Low High Player wins game

Loses

Medium High Low Player loses game

12

Risk analysis example Risk level:

Conservative strategy: select the higher value.

Averaging strategy: select the value between the two values.

13

A testing process: planning issues Issues that must be addressed to give a

basic shape to the test process: Planning issues:

Testing the models. Class testing, interaction testing, system

testing, regression testing.

14

A testing process: dimensions of software testing Who performs the testing?

Developers, independent testers or combination of the two.

Developers may exchange code and test each other’s code.

Which pieces will be tested? Test nothing, test a sample, test

everything, or just the ones associated with high risks.

15

A testing process: dimensions of software testing

When will testing be performed? Test every day, test components as they

are developed, test all components together at the end, at special milestones.

How will testing be performed? Knowledge of specification only,

knowledge of specification and implementation.

16

A testing process: dimensions of software testing (cont...)

How much testing is adequate? No testing, exhaustive testing (consider the

expected lifetime of the software), is the software life-critical,

Coverage: is a measure of how completely a test suite exercises the capabilities of a piece of software (other measures: if every LOC executed at least once, the number of requirement that is checked by the test suite).

17

A testing process: adequacy of test cases

Adequacy of test cases: test the software enough to be reasonably sure that the software works as it is supposed to.

18

A testing process: adequacy of test cases

Adequacy can be measured based on coverage: Coverage based on the requirements: based on

what the software is supposed to do ―how many of the requirements called out in the specification are tested

Coverage based on the code: based on how the software actually works ―how much of the software was executed as a result of running test suites

19

A testing process: adequacy of test cases (cont...)

Functional testing (specification-based or black box testing): test cases are constructed based solely on the

software’s specification and not on how the software is implemented.

Structural testing (implementation-based or white box testing): test cases are constructed based on the code

that implements the software.

20

A testing process: adequacy of test cases (cont...)

Use some combination of both approaches.

21

A testing process: adequacy of test cases― risk analysis (cont...)

Risk analysis in the testing process is applied to determine the level of details and amount of time to dedicate to testing a component.

A reasonable scale of increasing risk for components is: Prototype components Production components Library components Framework components

Ris

k in

crea

ses

22

Key points A risk ―anything that threatens the

successful achievement of a project’s goals Risk analysis technique includes three tasks:

identify, quantify the risk and produce a ranked list of use cases.

Adequacy of test cases: test the software enough to be reasonably sure that the software works as it is supposed to.