White-Box Testing
description
Transcript of White-Box Testing
White-Box Testing
Pfleeger, S. Software Engineering Theory and Practice 2nd Edition. Prentice Hall, 2001.Ghezzi, C. et al., Fundamentals of Software Engineering. Prentice Hall, 2002.Pressman, R., Software Engineering A Practitioner’s Approach, Mc Graw Hill, 2005.Hutchinson, Marnie, Software Testing Fundamentals, Wiley, 2003.
0310
Coverage
• Statement
• Branch
• Condition
• Path
• Def-use
• Others
Statement Coverage
• Every statement gets executed at least once
• Every node in the CFG gets visited at least once
Example (from Hutchinson, Software Testing Fundamentals, Wiley, 2003)
2
3
4
7
5
1
8
9
6
2
3
4
7
5
1
8
9
6
Examples: number of paths needed
for Statement Coverage? (from Hutchinson, Software Testing Fundamentals, Wiley, 2003)
2
3
4
7
5
1
8
9
6
2
3
4
7
5
1
8
9
6
Examples: number of paths needed
for Statement Coverage? (from Hutchinson, Software Testing Fundamentals, Wiley, 2003)
2
3
4
7
5
1
8
9
6
2
3
4
7
5
1
8
9
6
4 1
Branch coverage
• Every decision is made true and false• Every edge in a CFG of the program gets
traversed at least once• Also known as
– Decision coverage– All edge coverage– Basis path coverage
• Branch is finer than statement• C1 is finer than C2 if
T1C1 T2C2 T2 T1
Examples: number of paths needed
for Branch Coverage? (from Hutchinson, Software Testing Fundamentals, Wiley, 2003)
2
3
4
7
5
1
8
9
6
2
3
4
7
5
1
8
9
6
Examples: number of paths needed
for Branch Coverage? (from Hutchinson, Software Testing Fundamentals, Wiley, 2003)
2
3
4
7
5
1
8
9
6
2
3
4
7
5
1
8
9
6
5 2
Condition coverage
• Every complex condition is made true and false by every possible combination– E.G., (x and y)
• x = true, y = true• x=false, y=true• x = true, y= false• x =false, y = false
• There are lots of different types of condition coverage: Condition, multiple condition, condition/decision, modified condition/decision (MCDC), …I’m only covering the simplest.
• Condition coverage is not finer than branch coverage – There are pathological cases where you can achieve Condition and not
Branch– Under most circumstances, achieving Condition achieves Branch
Condition Coverage: CFGs• One way to determine the number of paths is to break the compound
conditional into atomic conditionals• Suppose you were writing the CFG for the assembly language
implementation of the control construct If (A AND B) then
CEndif
(short circuit eval) (no short circuit eval) LD A LD A ; in general, lots of BZ :endif LAND B ; code for A and
B LD B BZ :endif BZ :endif JSR C JSR C :endif nop:endif nop
AND Condition
1. read (a,b); 2. if (a == 0 && b == 0) then {3. c a + b }4. else c a * b
paths:1, 2A,2B,3, J1, 2A, 2B, 4, J1, 2A, 4, J
2A
2B
Join
3
4
1
OR Condition
1. read (a,b); 2. if (a == 0 || b == 0) then }3. c a + b;4. while( c < 100)5. c a + b;}
Paths:1, 2A, 3, 4, 5, 4 … J1, 2A, 3, 4, J1, 2A, 2B, 3, 4, 5, 4, … J1,2A, 2B, J
2A
3
Join
4
2B
5
1
Path
• A path is a sequence of statements
• A path is sequence of branches
• A path is a sequence of edges
Examples: number of paths needed
for Path Coverage? (from Hutchinson, Software Testing Fundamentals, Wiley, 2003)
2
3
4
7
5
1
8
9
6
2
3
4
7
5
1
8
9
6
Examples: number of paths needed
for Path Coverage? (from Hutchinson, Software Testing Fundamentals, Wiley, 2003)
2
3
4
7
5
1
8
9
6
2
3
4
7
5
1
8
9
6
5 16
Path Coverage-1• Every distinct path through code
is executed at least once
• Example1. read (x)2. read (z)3. if x 0 then begin4. y x * z;5. x z end6. else print ‘Invalid’ 7. if y > 1 then8. print y9. else print ‘Invalid’
• Test Paths:1, 2, 3, 4, 5, J1, 7, 8, J21, 2, 3, 4, 5, J1, 7, 9, J21, 2, 3, 6, J1, 7, 8, J2,1, 2, 3, 6, J1, 7, 9, J2
1,2,3
4,5
Join1
6
7
8
Join2
9
Counting Paths
• It is not feasible to calculate the total number of paths
Linearly independent paths
• It is feasible to calculate the number of linearly independent paths
• The number of linearly independent paths is the number of end-to-end paths required to touch every path segment at least once
• A linearly independent path introduces at least one new set of process statements or a new condition
Independent Paths• Consider the following encoding:
– Label each edge in the CFG with a positive integer
– Describe a path using a vector where each element of the vector is the number of times the edge at that index is traversed:
A
D
1
B C
2
43
• The path ABD can be written as
[1, 0, 1, 0]
Independent Paths• Consider the following encoding:
– Label each edge in the CFG with a positive integer
– Describe a path using a vector where each element of the vector is the number of times the edge at that index is traversed:
• Paths can be combined by adding or subtracting these path vectors
• A basis set is a set of linearly independent paths• Each path in the basis set can not be formed as a
combination of other paths in the basis set
Basis Set• A basis set:
– p13: [1,0,1,0]– p14: [1,0,0,1]– p23: [0,1,1,0]
• The path 24 can be constructed by p23+p14-p13[0,1,1,0] + [1,0,0,1] – [1,0,1,0] = [0,1,0,1]
A
D
1
B
2
43
• p13, p24 is not a basis set– p13: [1,0,1,0]
– p24: [0,1,0,1]
– (No way to construct [1,0,0,1])
• Another basis set:– p13: [1,0,1,0]
– p14: [1,0,0,1]
– p23: [0,1,0,1]
Examples: number of linearly independent
paths? (from Hutchinson, Software Testing Fundamentals, Wiley, 2003)
2
3
4
7
5
1
8
9
6
2
3
4
7
5
1
8
9
6
Examples: number of linearly independent
paths? (from Hutchinson, Software Testing Fundamentals, Wiley, 2003)
2
3
4
7
5
1
8
9
6
2
3
4
7
5
1
8
9
6
5 5
Cyclomatic Complexity• Software metric for the logical complexity of a
program.• Defines the number of independent paths in the
basis set of a program• Provides the upper bound for the number of tests
that must be conducted to ensure that all statements been have executed at least once
• For Edges (E) and Nodes (N) V(G) = E – N + 2
Independent Path Coverage• Basis set does not yield minimal
test set• Example
1. read (x)2. read (z)3. if x 0 then begin4. y x * z;5. x z end6. else print ‘Invalid’ 7. if y > 1 then8. print y9. else print ‘Invalid’
• Cyclomatic complexity: 3• Test Paths:
1, 2, 3, 4, 5, J1, 7, 8, J21, 2, 3, 4, 5, J1, 7, 9, J21, 2, 3, 6, J1, 7, 8, J2,
1,2,3
4,5
Join1
6
7
8
Join2
9
Def-Use Coverage• Def-use coverage: every path
from every definition of every variable to every use of that definition is exercised in some test.
• Example1. read (x)2. read (z)3. if x 0 then begin4. y x * z;5. x z end6. else print ‘Invalid’ 7. if y > 1 then8. print y9. else print ‘Invalid’
1,2,3
4,5
Join
6
7
8
Join
9
Def: x, zUse: x
Def: y, xUse: x, z
Use: none
Use: y
Use: yUse: none
Test Path: 1, 2, 3, 4, 5, 7, 8, J
Strength of Coverage
Statement
Branch
Def-Use
Path
Arrows point from weaker to stronger coverage.
Stronger coverage requires more test cases.
Condition
Groups:
A
B
D
B1D2D1
E
C
A is read(x, y, z)
B is if ((x>10) && (y<20))
D is if (z >10)
E is repeat until g = 5
Show me different minimal test sets that satisfy the 4 criteria
Statement:
A
B
D
B1D2D1
E
C
A is read(x, y, z)
B is if ((x>10) && (y<20))
D is if (z >10)
E is repeat until g = 5
•A,B,B1,E C: x(A) = 0; g (E) =5
•A, B,D,D1,E C: x(A)=11, y(A) = 11, z(A)=11, g(E)=5
•A, B, D, D2, E, C: x(A)=11, y(A) = 11, z(A)=9, g(E)=5
Branch:
A
B
D
B1D2D1
E
C
A is read(x, y, z)
B is if ((x>10) && (y<20))
D is if (z >10)
E is repeat until g = 5
•A,B,B1,E C: x(A) = 0; g (E) =5
•A, B,D,D1,E C: x(A)=11, y(A) = 11, z(A)=11, g(E)=5
•A, B, D, D2, E, C: x(A)=11, y(A) = 11, z(A)=9, g(E)=5
•A,B,B1,E,A …: x(A) = 0; g (E) =4
Condition:
A
B
D
B1D2D1
E
C
A is read(x, y, z)
B is if ((x>10) && (y<20))
D is if (z >10)
E is repeat until g = 5
•A,B,B1,E C: x(A) = 0; g (E) =5
•A,B,B1,E C: x(A) = 11; y(A) = 22; g (E) =5
•A,B,B1,E C: x(A) = 0; y(A) = 22; g (E) =5
•A, B,D,D1,E C: x(A)=11, y(A) = 11, z(A)=11, g(E)=5
•A, B, D, D2, E, C: x(A)=11, y(A) = 11, z(A)=9, g(E)=5
•A,B,B1,E,A …: x(A) = 0; g (E) =4
Independent Paths:
A
B
D
B1D2D1
E
C
A is read(x, y, z)
B is if ((x>10) && (y<20))
D is if (z >10)
E is repeat until g = 5
•A,B,B1,E C: x(A) = 0; g (E) =5
•A, B,D,D1,E C: x(A)=11, y(A) = 11, z(A)=11, g(E)=5
•A, B, D, D2, E, C: x(A)=11, y(A) = 11, z(A)=9, g(E)=5
•A,B,B1,E,A,B,B1,E,C: x(A) = 0; g (E) =4