New_2_strategies.ppt

Software Testing

Testing Strategies

Software Testing Strategies

● First software quality assurance tools applied to control the software product's quality before its shipment

● November 1994 survey by Perry 24 % of project development budget was for testing 32 % of project management budget was for testing 27 % of project time was schedule for testing (actually they allocate 45 % of their schedule time for

testing)

Outlines

● 2.1 Introduction Testing Objectives

● 2.2 Software Testing Strategies● 2.3 Software Test Classifications● 2.4 White Box Testing● 2.5 Black Box Testing

Objective

● Explain testing objectives● Discuss the difference between the various testing

strategies● Describe the concept of black box and white box

testing

Outlines



Definition Revisited

● Myers Testing is the process of executing a program with

intention of finding errors● Paul Jorgensen

Testing is obviously concerned with errors, faults, failures and incidents. A test is the act of exercising software with test cases with an objective of Finding failure and Demonstrate correct execution

● ISO Technical operation that consist of the determination of

one or more characteristics of a given product, process or service according to a specified procedure

What is the objectives of Software Testing?

Direct Objectives

● To identify and reveal as many errors as possible in the tested software

● To bring the tested software, after correction of the identified errors and retesting to an acceptable level of quality

● To perform the required tests efficiency and effectively, within budgetary and scheduling limitation

Indirect Objective

● To compile a record of software errors for use in error prevention (by corrective and preventive actions)

Outlines



Stages of Testing

● Module or unit testing.● Integration testing,● Function testing.● Performance testing.● Acceptance testing.● Installation testing.

Testing Strategies

● We began by 'testing-in-the-small' and move toward 'testing -in-the-large'

● For conventional software The module (component) is our initial focus Integration of module follows

● For OO software OO class that include attributes and operations and

implies communication and collaboration

Unit Testing

• Program reviews.• Formal verification.• Testing the program itself.

– black box and white box testing.

Black Box or White Box?

• Maximum # of logic paths - determine if white box testing is possible.

• Nature of input data.• Amount of computation involved.• Complexity of algorithms.

Unit Testing Details

• Interfaces tested for proper information flow.• Local data are examined to ensure that integrity is

maintained.• Boundary conditions are tested.• Basis path testing should be used.• All error handling paths should be tested.• Drivers and/or stubs need to be developed to test

incomplete software.

Generating Test Data

• Ideally want to test every permutation of valid and invalid inputs

• Equivalence partitioning it often required to reduce to infinite test case sets– Every possible input belongs to one of the equivalence

classes.– No input belongs to more than one class.– Each point is representative of class.

Integration Testing Strategies

Test methodologies may vary but two basic testing strategies applied:

● Test the software in its entirety “Big Bang Testing”

● Test the software in modules - Unit Tests, Integration Tests, Systems Tests “Incremental testing”

● Two strategies for incremental testing Bottom - up testing (test harness). Top - down testing (stubs).

● Sandwich Testing

Top-Down Integration Testing

• Main program used as a test driver and stubs are substitutes for components directly subordinate to it.

• Subordinate stubs are replaced one at a time with real components (following the depth-first or breadth-first approach).

• Tests are conducted as each component is integrated.• On completion of each set of tests and other stub is replaced with a

real component.• Regression testing may be used to ensure that new errors not

introduced.

Top Down Integration

top module is tested with top module is tested with stubsstubs

stubs are replaced one at stubs are replaced one at a time, "depth first"a time, "depth first"

as new modules are integrated, as new modules are integrated, some subset of tests is re-runsome subset of tests is re-run

AA

BB

CC

DD EE

FF GG

● Stub is a piece of code emulating a called function

Top down – Advantages & Disadvantages● Top-down integration supports fault isolation;● Major design flaws show up early. Modules of a

product can be divided into two groups: The logic modules and operational modules. By coding and testing the logic modules before the

operational modules, top-down integration will exploit any major design faults early in the development process

● The main disadvantage of top-down integration is that potentially reusable modules may not be adequately tested.

Bottom-Up Integration Testing

• Low level components are combined in clusters that perform a specific software function.

• A driver (control program) is written to coordinate test case input and output.

• The cluster is tested.• Drivers are removed and clusters are combined moving

upward in the program structure.

Bottom-Up Integration

drivers are replaced one at a drivers are replaced one at a time, "depth first"time, "depth first"

worker modules are grouped into worker modules are grouped into builds and integratedbuilds and integrated

AA

BB

CC

DD EE

FF GG

clustercluster

● a driver is a piece of code emulating a calling function

Bottom-up – Advantages & Disadvantages● The operational modules are thoroughly tested

when using a bottom-up strategy. It also provides fault-isolation, as does top-down integration.

● Major design faults will be left undetected until late in the development, since the logic module are integrated last.

● This may result in large costs in redesigning and recoding substantial portions of the project.

Sandwich Testing

● Combine the two so as to capitalize upon their strengths and minimize their weaknesses

● neither top-down nor bottom-up inplemenetation/integration is suitable for all the modules, the solution is to partition them

Sandwich Testing

Top modules areTop modules aretested with stubstested with stubs

Worker modules are grouped into Worker modules are grouped into builds and integratedbuilds and integrated

AA

BB

CC

DD EE

FF GG

clustercluster

Outlines



Software Test Classification

● Software test may be classified according to the testing concept or to the requirement classification

According to Testing Concept

● What concept to test? Output

● Output is used to archive an acceptable level of quality Structural of the software

● Internal structural and calculations involved is included for satisfactory testing

● Two Classes have been developed Black box testing

● Identify bug according to software malfunctioning● Functionality testing

White box testing● Examine internal calculation paths in order to identify bug● Structural testing

According to Requirement

● The test id carried out to ensure full coverage of the respective requirement Operation

● Correctness, Reliability, Efficiency, Integrity, Usability Revision

● Maintainability, flexibility, testability Transition

● Portability, reusabilily, interoperability● From the requirement we can define the test

classification● White box and black box can be used implemented

respectively

Test Classification according to Requirement

Correctness 1.1 Output correctness tests1.2 Documentation tests1.3 Availability tests1.4 Data processing & calculations correctness tests 1.5 Software qualification tests

Reliability 2. Reliability testsEfficiency 3. Stress test (load tests and durability tests)Integrity 4. Software system security testsUsability 5.1 Training usability tests

5.2 Operational usability testsMaintainability 6. Maintainability tests

Flexibility 7. Flexibility tests Testability 8. Testability testsPortability 9. Portability testsRe usability 10. Re usability testsInteroperability 11.1 Software interoperability tests

11.2 Equipment interoperability test

Black box testingTesting that ignores the internal mechanism of the

system or component and focuses solely on the outputs in response to selected inputs and execution conditions

Testing conducted to evaluate the compliance of a system or component with specified functional requirements

White box testing Testing that takes into account the internal

mechanism of a system or component

IEEE definitions

Outlines



White Box Testing

● White Box Testing requires verification of every program statement and comment

● White box testing enables performance of Data processing and calculation correctness tests

● Every computation operation must be examined Software qualification tests,

● Software code (including comments) Maintainability test

● Failure causes detection, support adaption, software improvement

Reusability test● Reusable for future software packages

Data processing and calculation correctness tests● Checking the operation by each test cases

Different path Which path is going to be tested?

● Two approaches Path coverage

● Path coverage of a test is measured by the percentage of all possible program paths included in planned testing.

Line coverage● Line coverage of a test is measured by the percentage of

program code lines included in planned testing.

Path coverage

● Different path is created by condition statements IF-THEN-ELSE DO-WHILE DO-UNTIL

● Consider a simple module with 10 conditional statement allowing two options each Creates 210 different path = 1024 path Examples:

● Taximeter

Taxi meter1.Minimal fare: $2. This fare covers the distance

traveled up to 1000 yards and waiting time (stopping for traffic lights or traffic jams, etc.) of up to 3 minutes.

2. For every additional 250 yards or part of it: 25 cents.

3. For every additional 2 minutes of stopping or waiting or part thereof: 20 cents.

4. One suitcase: 0 change; each additional suitcase: $1.

5. Night supplement: 25%, effective for journeys between 21.00 and 06.00.

● Regular clients are entitled to a 10% discount and are not charged the night supplement.

S ≤ 1S >1

Yes

WT ≤ 3WT > 3

D ≤ 1000

1Charge the minimal fare

2Distance

5Waiting time

14Night

journey?

11Regular client?

3

6

4

12 13

16

15

17Print receipt.

8No.of suitcases9 10

D > 1000

7

No

NoYes

Flow Chart

Examples

● 24 different paths may be indicated● In order to archive full path coverage of the

software module, we have to prepare at least 24 test cases

● We need to find a minimum number of paths to cover all line of code Line coverage

3

6

9

12

5

2

1

8

11

15

4

17

7

16

10

1314

R1

R2

R3

R5R4

R6

Flow Graph

3

6

9

12

5

2

1

8

11

15

4

17

7

16

10

1314

R1

R2

R3

R5R4

R6

The Minimum number of paths

Minimun 3 path is required

Advantages of White box TestingAdvantages: * Direct determination of software correctness as expressed in

the processing paths, including algorithms. * Allows performance of line coverage follow up. * Ascertains quality of coding work and its adherence to coding

standards.Disadvantages : * The vast resources utilized, much above those required for

black box testing of the same software package. * The inability to test software performance in terms of

availability (response time), reliability, load durability, etc.

Outlines



Test Classification according to Requirement

Correctness 1.1 Output correctness tests1.2 Documentation tests1.3 Availability tests1.4 Data processing & calculations correctness tests 1.5 Software qualification tests

Reliability 2. Reliability testsEfficiency 3. Stress test (load tests and durability tests)Integrity 4. Software system security testsUsability 5.1 Training usability tests

5.2 Operational usability testsMaintainability 6. Maintainability tests

Flexibility 7. Flexibility tests Testability 8. Testability testsPortability 9. Portability testsRe usability 10. Re usability testsInteroperability 11.1 Software interoperability tests

11.2 Equipment interoperability test

Black Box Testing

● Apart from correctness tests(those 2), maintainability and re usability, most of other testing classes are unique to black box testing Explain the importance of black box testing

● However, due to the special characteristic of each testing strategy, black box testing cannot automatically substitute for white box testing

Black-Box Testing

● Black-box testing is testing from a functional or behavioral perspective to ensure a program meets its specification

● Testing usually conducted without knowledge of software implementation - system treated as a “black box”

● Black-box test design techniques include: equivalence partitioning, boundary value analysis, cause-effect graphing, random testing

How much testing is adequate?

● Completely validating IEEE 754 floating-point division requires 264 test-cases!

float divide(float x, float y)

● From practical and economic perspectives, exhaustive testing is usually not possible Which software pieces should we test? Which test cases should we choose?

Equivalence class partitioning (EC)

● A black box method aimed at increasing the efficiency of testing and, at the same time, improving coverage of potential error conditions.

Boundary Value Analysis

● Based on experience / heuristics Testing boundary conditions of equivalence classes is more

effective● Choose input boundary values as equivalence classes

representatives● Choose inputs that invoke output boundary values● Examples:

(0, 10] ⇒ validate using 0, 1, 2, 9, 10, 11 Read up to 5 elements ⇒ validate reading 0, 1, 4, 5, 6 elements

BVA as an equivalence partitioning extension● Choose one (or more) arbitrary value(s) in each

equivalence class● Choose valid values exactly on lower and upper

boundaries of equivalence class● Choose invalid values immediately below and

above each boundary (if applicable)

Equivalence Class Partitioning (EC)

● An equivalence class (EC) is a set of input variable values that produce the same output results or that are processed identically.

● EC boundaries are defined by a single numeric or alphabetic value, a group of numeric or alphabetic values, a range of values, and so on.

● An EC that contains only valid states is defined as a "valid EC," whereas an EC that contains only invalid states is defined as the "invalid EC."

● In cases where a program's input is provided by several variables, valid and invalid ECs should be defined for each variable.

Equivalence Class Partitioning (EC)

According to the equivalence class partitioning method:

● Each valid EC and each invalid EC are included in at least one test case.

● Definition of test cases is done separately for the valid and invalid ECs.

Equivalence Class Partitioning (EC)● In defining a test case for the valid ECs, we try to

cover as many as possible “new” ECs in that same test case.

● In defining invalid ECs, we must assign one test case to each “new” invalid EC, as a test case that includes more than one invalid EC may not allow the tester to distinguish between the program’s separate reactions to each of the invalid ECs.

● Test cases are added as long as there are uncovered ECs.

Example – Ticket Price

● Ticket price depends on four variables Day (weekday, weekend) Visitor's status (OT = one time, M = members) Entry Hour (6 – 19, 19.01 – 24) Visitor's age (up to 16, 16.01 – 60, 60.01 – 120)

$5.50$5.50$12$12$4$4$8$8Age: 60.01-120.00

$8$7$18$15$6$5$17$10Age 16.01-60.00

$4$3.50$9$7.50$3$2.50$6$5Age: 0.00-16.00

19.01-

24.00

6.00-19.00

19.01-24.00

6.00-19.00

19.01-24.00

6.00-19.00

19.01-24.00

6.00-19.00

Entry hour

MemMemOtOtMemMemOtOtVisitor’s status

Mon., Tue., Wed., Thu, Fri. Sat., Sun.Day

Entrance Ticket Price table

Invalid visitor’s age150.17.55OtMon.15

Invalid visitor’s ageTTR7.55OtMon.14

Invalid entry [email protected]

Invalid entry hour8.44.40OtMon.12

Invalid visitor’s status8.47.5588Mon.11

Invalid day8.47.55OtMox.10Invalid ECs

$5.50120.024.00MemSat.9

$5.5060.0124.00MemSat.8

$860.019.01MemSat.7

$816.0119.01MemSat.6

$3.5016.019.00MemSat.5

$3.500.06.00MemSat.4

$5.5065.022.44MemSat.3

$842.720.44MemSat.2

$58.47.55OtMon.1Valid ECs

Test case resultVisitor’s age

Entry hour

Visitor’s status

Day of

week

Test case no.

Test case type

Test cases – The Ticket price

Advantage of Black Box Testing

Advantages:● Allows us to carry out the majority of testing

classes, most of which can be implemented solely by black box tests, i.e. load tests and availability tests.

● For testing classes that can be carried out by both white and black box tests, black box testing requires fewer resources.

Disadvantages of Black Box TestingDisadvantages:● Possibility that coincidental aggregation of several

errors will produce the correct response for a test case, and prevent error detection.

● Absence of control of line coverage. There is no easy way to specify the parameters of the test cases required to improve coverage.

● Impossibility of testing the quality of coding and its strict adherence to the coding standards.

New_2_strategies.ppt

Documents

Transcript of New_2_strategies.ppt