Testing -- Part II

Testing

The role of testing is to: Locate errors that can then be fixed to

produce a more reliable product Design tests that systematically uncover

different classes of errors

The purpose of testing is to:

Remove as many errors as possible from the product

protecting the user from loss of time, effort, and money

protecting the reputation of the software developer in the marketplace

Testing

Recall Testing cannot show the absence of errors;

it can only show that errors are present! As much as 50% of the development effort

will be expended in testing

Purpose of testing phase to affirm the quality of the product and indirectly

the process to find and eliminate errors to validate that the software solves the original

problem to demonstrate that all specified functions are

performed by the product to estimate the operational reliability of the system

Testing

In general testing is "bottom-up"

But-

Programming is "top-down"

Testing

1. Test small units in all possible ways to detect any errors

2. Combine them into units, testing both their detailed structure and their function

3. Concentrate on how software responds to the typical kind of operations that the user will require.

Phases of Testing1. Unit testing

2. Integration testing

3. System testing

4. Regression testing

Testing: Error categories

a) Those that merit immediate attention (they crash the system).

b) Errors that need to be corrected before testing is complete but can be ignored during the immediate testing schedule.

c) there may be errors that may be acceptable to the user when compared to the cost of correcting them at the moment--errors put off until future releases.

d) errors that are non-reproducible

Testing = Error Location

The importance of removing an error is proportional to its severity, the frequency with which it will occur, and the degree to which the customer will be aware of it.

Regression testing Testing that occurs after software

modifications are made. When "corrections" are made the software

must subjected to all of the tests it has passed to this point.

WHY??

Regression testing can be very painful!

Why?

Basic testing strategies Structural testing – focuses on a UNIT of

code. (Unit testing) Functional testing – focuses on the

interaction/interface between code components. (Integration testing)

Pragmatic testing – System testing• Mimics the real world• Tests the system as a whole.

Structural testing Idea is to check all logical paths through a

module Commonly used during unit tests View the software in terms of its detailed

design and attempt to test it so that we certify the correct behavior of every statement, every decision or every path through the code

Testing

Every statement, every decision or every path ---

EVERY!

???

Structural Testing is

- Very detail focused

- Program-based testing

- Tests for conformity to the detailed design

- Microscopic view

Structural Testing:

Test cases (data) are needed that test: all logical conditions for both true and

false values loops should be executed over their

entire range of values

Structural Testing:

Test cases (data) are needed that test: validity of internal data structures must be

verified everything should be checked at the

boundaries loops need to be tested at every execution,

especially if there is a local variable that affects anything inside the loop

Structural Testing is never perfect!

Structural testing cannot test all possible paths for all combinations of logic, because even a simple procedure there are too many paths -- still each paths should be executed at least one

Functional testing

Focus is on INTERFACE between the units Observe properties of a software until it

conforms to specifications Determine whether the overall purpose has

been satisfied Utilized during the integration tests

Functional testing Macroscopic view Focuses on functions specified for the

product. Are functions implemented satisfactorily?

Specification-based testing Tests the product for conformity at the

specification level.

Functional testing

How is this done?

A set of inputs is supplied and the outputs are to be verified against the results expected under the terms of the specification

Pragmatic testing Commonly used during systems testing Attempts to model the actual usage that the

software will encounter after it is released t the customer

Focuses on usability and reliability rather than universal correctness

General Goals of testing Detection and elimination of errors Prevention of additional errors during

software modification Validation of the product (Are we building the

right product?)

Release of product on the scheduled date

Goals may be in conflict

a) if we resolve to correct every error that testing uncovers, minor errors may lead to contract penalties or a lost market opportunity

b) bowing to the schedule and hurrying testing to the point that we fail to detect and correct serious errors may incur other contract penalties or loss of reputation.

We need predetermined goals for the product being tested:

How dependable must the product be?

medical software word processing software scheduling software

We need predetermined goals for the product being tested:

How important is it to meet the schedule?

Testing

Software must be tested in a realistic setting

Software must be carefully and extensively tested in the same way that users are expected to use it

Alpha testing Alpha testing occurs when the software has

been written for a specific "client." These systems are usually "revised" or

"adapted" as part of the testing process until the client agrees that the system is an acceptable implementation of the system requirements.

Beta Testing: Broad Market Appeal

Beta Testing involves delivering a system to a number of potential customers who agree to use the system.

AND

Report problems to developers After exposing the system to real-time real-use operation.

Beta Testing

The System is then "adapted" and either beta tested again or put out for general sale.