Unit Testing

CSSE 376, Software Quality Assurance

Rose-Hulman Institute of Technology

March 27, 2007

2

Outline

Role of unit testingTesting strategies

Black box methodsWhite box methods

3

Role of Unit Testing

Assure minimum quality of units before integration into system

Focus attention on relatively small units

Marks end of development step

4

Testing versus Debugging

Testing Debugging

planned unplanned

results are recorded

unrecorded

may be done by non-developers

always done by developers

look for errors look for causes of errors

5

Limits of Testing

Testing can never demonstrate the absence of errors

Exhaustive testing is infeasibleTesting may be done imperfectly

6

Strategies for Unit Testing

Black boxuse only specification of programtest implementation against its

specification

White boxuse structure or other properties of a

program to generate tests

7

Black Box Methods

Equivalence partitioningBoundary value analysis

8

White Box Methods

9

White Box Methods

Coveragestatementbranchpath

CyclomaticDataflowMutationError Based

10

Coverage Methods

Statementexecute each statement

Branchexecute each branch

Pathexecute each path

11

Statement Coverage

Execute each statement in the program

Considered minimum criterion for most unit testing

May be difficult to achieve for error cases

12

Example Program

1: if (a < 0) {

2: return 0 }

3: r = 0;

4: c = a;

5: while (c > 0) {

6: r = r + b;

7: c = c - 1; }

8: return r;

13

Statement Tests

a = 3, b = 4executes 1, 3, 4, 5, 6, 7, 5, 6, 7, 5, 6, 7, 5,

8

a = -3, b = 2executes 1, 2

14

Branch Coverage

Execute each branch of the program at least once

Differs from statement coverage only for "if" statements without "else"s and case statements without default cases.

15

Dataflow TestingMore than branch coverage, but less

than path coverageUses information about references to

variables to select paths

16

Definitions and UsesDefining node

input statementlhs of assignment

Usage nodeoutput statementrhs of assignmentcontrol statement

17

Suspicious PathsVariable is defined (set to a new

value) but never referencedVariable is referenced but never

definedVariable is defined twice before it is

used

18

DU PathsDefinition-use (du) path wrt V:

a path containing a defining node for V at the beginning a usage node for V at the end, and no def's or use's in between

DU testing: execute each du-path of each variable

19

Example Program1: if (a < 0) {

2: return 0 }

3: r = 0;

4: c = a;

5: while (c > 0) {

6: r = r + b;

7: c = c - 1; }

8: return r;

20

Example DU Paths(corrected 3/28/07)

Def (c) = {4, 7}Use (c) = {5, 7}Def (r) = {3, 6}Use (r) = {6, 8}du-paths for c:

4 - 5, 7 – 5

du-paths for r:3 - 4 - 5 - 6, 6 - 7 - 5 - 6, 3 - 4 - 5 - 8,

6 - 7 - 5 - 8

21

Test Cases for DU Paths (corrected 3/28/07)

a = 21 - 3 - 4 - 5 - 6 - 7 - 5 - 6 - 7 - 5 - 8

Covers du-paths:4 - 5, 7 - 5, 3 - 4 - 5 - 6, 6 - 7 - 5 - 6

22

Cartoon of the Day

23

Mutation Testing(As Applied to White-Box)

Path testing exercises the control of a program, not the computations along the paths

Most programs under test are "almost" right

24

Mutation Method1. Pre-process program to generate

mutants2. Execute all the mutants and the

original program3. If a mutant differs from the original,

then it is "killed"4. If all mutants are killed, then test set

is adequate.

25

Mutation Metaphor

26

Example Programfunction square( x: integer): integer;

begin

square := x * x

end

27

Example Mutantfunction square( x: integer): integer;

begin

square := x + x

end

28

Executing MutantsTest set {2} does

not kill the mutantTest set {2, 4} does

kill the mutant, and it reveals the flaw in the program

29

Which Mutants?Examples:

Off by one errors (i+1, i, i-1)

Different variable (i, j, k)

30

Assumptions ofMutation Testing

Competent Programmer: The perfect program is very close to the program under test

Oracle Hypothesis: You know the output for any input

Continuity: Every complicated mistake can be caught by looking for simple mistakes.

31

Problems with Mutation Very expensive

each test runs the whole program many mutants for every program

Some mutants fail to halt May be difficult to detect when

a mutant is really just an equivalent program

Not reliable: may not try the right mutants