Some Testing Techniques (enhanced 6 + 4) Course Software Testing & Verification 2014/15 Wishnu...
-
Upload
doris-burke -
Category
Documents
-
view
214 -
download
1
Transcript of Some Testing Techniques (enhanced 6 + 4) Course Software Testing & Verification 2014/15 Wishnu...
![Page 1: Some Testing Techniques (enhanced 6 + 4) Course Software Testing & Verification 2014/15 Wishnu Prasetya.](https://reader030.fdocuments.us/reader030/viewer/2022032523/56649d8e5503460f94a7730d/html5/thumbnails/1.jpg)
Some Testing Techniques(enhanced 6 + 4)
Course Software Testing & Verification2014/15
Wishnu Prasetya
![Page 2: Some Testing Techniques (enhanced 6 + 4) Course Software Testing & Verification 2014/15 Wishnu Prasetya.](https://reader030.fdocuments.us/reader030/viewer/2022032523/56649d8e5503460f94a7730d/html5/thumbnails/2.jpg)
Plan
• Unit testing tool• Mocking (Sec. 6.2.1) enhanced• Black box testing Input Partitioning (Ch 4).• Regression (6.1) enhanced
2
![Page 3: Some Testing Techniques (enhanced 6 + 4) Course Software Testing & Verification 2014/15 Wishnu Prasetya.](https://reader030.fdocuments.us/reader030/viewer/2022032523/56649d8e5503460f94a7730d/html5/thumbnails/3.jpg)
Unit Testing
• Making sure that the units are correct.• Invest in unit testing! Debugging an error at
the system-test level is much more costly than at the unit level.
• Note: so-called “unit testing tool” can often also be used to facilitate integration and system testing.
3
![Page 4: Some Testing Techniques (enhanced 6 + 4) Course Software Testing & Verification 2014/15 Wishnu Prasetya.](https://reader030.fdocuments.us/reader030/viewer/2022032523/56649d8e5503460f94a7730d/html5/thumbnails/4.jpg)
But what is a “unit” ?
• In principle, you decide. “Units” are just something you can compose to build something bigger. Possibilities: function/method , or class as unit.
• However, different types of units may have types of interactions and complexity, thus requiring different approaches– a function’s behavior depends only on its parameters; does
not do any side effect.– procedure depends-on and affects params– method: params, instance vars, static vars– class: is a collection of interacting methods
4
![Page 5: Some Testing Techniques (enhanced 6 + 4) Course Software Testing & Verification 2014/15 Wishnu Prasetya.](https://reader030.fdocuments.us/reader030/viewer/2022032523/56649d8e5503460f94a7730d/html5/thumbnails/5.jpg)
Unit testing in C#
• You need at least Visual Studio Professional; and for code coverage feedback you need at least Premium.
• Check these tutorials/docs (Visual Studio 2010):– Walkthrough: Creating and Running Unit Tests– Walkthrough: Run Tests and View Code Coverage– Walkthrough: Using the Command-line Test Utility– API Reference for Testing Tools for Visual Studio, in
particular Microsoft.VisualStudio.TestTools.UnitTesting, containg classes like
• Assert, CollectionAssert, ...
• In this lecture we will just go through the concepts5
![Page 6: Some Testing Techniques (enhanced 6 + 4) Course Software Testing & Verification 2014/15 Wishnu Prasetya.](https://reader030.fdocuments.us/reader030/viewer/2022032523/56649d8e5503460f94a7730d/html5/thumbnails/6.jpg)
The structure of a solution with “test projects”
6
class Thermometer private double valprivate double scaleprivate double offsetpublic Thermometer(double s, double o)public double value()public double warmUp(double v) public double coolDown(double v)
A test project is a just a project in your solution that contains your test-classes.
![Page 7: Some Testing Techniques (enhanced 6 + 4) Course Software Testing & Verification 2014/15 Wishnu Prasetya.](https://reader030.fdocuments.us/reader030/viewer/2022032523/56649d8e5503460f94a7730d/html5/thumbnails/7.jpg)
The structure of a “test project”
• A solution may contain multiple projects; it may thus contain multiple test projects.
• A test project is used to group related test classes.• You decide what “related” means; e.g. you may want
to put all test-cases for the class Thermometer in its own test project.
• A test class is used to group related test method.• A test method does the actual testing work. It may
encode a single test-case, or multiple test-cases. You decide.
7
![Page 8: Some Testing Techniques (enhanced 6 + 4) Course Software Testing & Verification 2014/15 Wishnu Prasetya.](https://reader030.fdocuments.us/reader030/viewer/2022032523/56649d8e5503460f94a7730d/html5/thumbnails/8.jpg)
Test Class and Test Method
8
[TestClass()]public class ThermometerTest { private TestContext testContextInstance; //[ClassInitialize()] //public static void MyClassInitialize(...) ... //[ClassCleanup()] //public static void MyClassCleanup() ... //[TestInitialize()] //public void MyTestInitialize() ... //[TestCleanup()] //public void MyTestCleanup() ...
[TestMethod()] public void valueTest1() ... [TestMethod()] public void valueTest2() ....}
public void valueTest1() { target = new Thermometer(1,0); double expected = - 273.15 ; double actual = target.value(); Assert.AreEqual(expected, actual);}
Be careful when comparing floating numbers, you may have to take imprecision into account, e.g. use this instead:
AreEqual(expected,actual,delta,”...”)
![Page 9: Some Testing Techniques (enhanced 6 + 4) Course Software Testing & Verification 2014/15 Wishnu Prasetya.](https://reader030.fdocuments.us/reader030/viewer/2022032523/56649d8e5503460f94a7730d/html5/thumbnails/9.jpg)
Positive and Negative Test
• Positive test: test the program on its normal parameters’ range.
• But can we afford to assume that the program is always called in its normal range? Else do Negative test: test that the program beyond its normal range.
• E.g. when unit testing a method, to test if it throws the right kind of exceptions.
9
![Page 10: Some Testing Techniques (enhanced 6 + 4) Course Software Testing & Verification 2014/15 Wishnu Prasetya.](https://reader030.fdocuments.us/reader030/viewer/2022032523/56649d8e5503460f94a7730d/html5/thumbnails/10.jpg)
Inspecting Test Result
10
![Page 11: Some Testing Techniques (enhanced 6 + 4) Course Software Testing & Verification 2014/15 Wishnu Prasetya.](https://reader030.fdocuments.us/reader030/viewer/2022032523/56649d8e5503460f94a7730d/html5/thumbnails/11.jpg)
Inspecting Coverage
11
![Page 12: Some Testing Techniques (enhanced 6 + 4) Course Software Testing & Verification 2014/15 Wishnu Prasetya.](https://reader030.fdocuments.us/reader030/viewer/2022032523/56649d8e5503460f94a7730d/html5/thumbnails/12.jpg)
Finding the source of an error: use a debugger!
12
• Add break points; execution is stopped at every BP.
• You can proceed to the next BP, or execute one step at a time: step-into, step-over, step-out.
• VisualStudio uses IntelliTrace logging you can even inspect previous BPs.
![Page 13: Some Testing Techniques (enhanced 6 + 4) Course Software Testing & Verification 2014/15 Wishnu Prasetya.](https://reader030.fdocuments.us/reader030/viewer/2022032523/56649d8e5503460f94a7730d/html5/thumbnails/13.jpg)
The Debug class
• Debug.Print(“therm. created”)• Debug.Assert(t.scale() > 0) to check for
properties that should hold in your program.• Will be removed if you build with debug off
(for release).• Check the doc of System.Diagnostics.Debug
13
![Page 14: Some Testing Techniques (enhanced 6 + 4) Course Software Testing & Verification 2014/15 Wishnu Prasetya.](https://reader030.fdocuments.us/reader030/viewer/2022032523/56649d8e5503460f94a7730d/html5/thumbnails/14.jpg)
Test Oraclesome patterns will return later
• Full: Assert. IsTrue( t == -273.15)• Partial: Assert.IsTrue( t -274) • Property-based : Assert.IsTrue(value_postcond(T))
14
public void valueTest1() { T = new Thermometer(0) Assert.IsEqual(T.value(), - 273.15)}
An oracle specifies your expectation on the program’s responses.
More costly to maintain, e.g. if you change the intended behavior of the program.
![Page 15: Some Testing Techniques (enhanced 6 + 4) Course Software Testing & Verification 2014/15 Wishnu Prasetya.](https://reader030.fdocuments.us/reader030/viewer/2022032523/56649d8e5503460f94a7730d/html5/thumbnails/15.jpg)
Discussion: propose test casesin particular the oracles...
15
reverse(a) { N = a.length if (N 1) return for (int i=0; i< N/2 ; i++) swap(a,i, N-1-i)}
incomeTax(i) { if (i18218) return 0.023 * i t = 419 if (i32738) return t + 0.138 * (i – 18218) t += 1568 if (i54367) return t + 0.42 * (i – 32738)}
Property-based testing fits nicely for reverse, but not for incomeTax; for the latter we’ll have to fall back to conrete-value oracles, which unfortunately tend to be more costly to maintain.
![Page 16: Some Testing Techniques (enhanced 6 + 4) Course Software Testing & Verification 2014/15 Wishnu Prasetya.](https://reader030.fdocuments.us/reader030/viewer/2022032523/56649d8e5503460f94a7730d/html5/thumbnails/16.jpg)
Discussion: the Oracle Problem (6.5)
• Every test-case needs an oracle; how to construct it!? always a big problem!
• Using concrete values as oracles is often powerful, but potentially expensive to maintain.
• Using “properties” on the other hand has the problem that it can be hard to write a complete yet simple property capturing correctness.
• A pragmatic middle way could be redundancy-based testing.
16
![Page 17: Some Testing Techniques (enhanced 6 + 4) Course Software Testing & Verification 2014/15 Wishnu Prasetya.](https://reader030.fdocuments.us/reader030/viewer/2022032523/56649d8e5503460f94a7730d/html5/thumbnails/17.jpg)
(Unit) Testing a class
• Many classes have methods that interact with each other (e.g. as in Stack). How to test these interactions?
• How to specify (and test) these interactions? Options:– class invariant– Abstract Data Type (ADT)– Finite State Machine (FSM)
17
![Page 18: Some Testing Techniques (enhanced 6 + 4) Course Software Testing & Verification 2014/15 Wishnu Prasetya.](https://reader030.fdocuments.us/reader030/viewer/2022032523/56649d8e5503460f94a7730d/html5/thumbnails/18.jpg)
Specifying with class invariant
• Regardless the interaction with other methods, each method of a class C has to keep the state of its target object consistent.
• Express this with a class invariant, e.g. – this.T >= -273.15– this.saldo >= 0– forall x in persons, typeOf(x) is Employee
• Class invariant cannot express a constraint over the interaction itself.
18
![Page 19: Some Testing Techniques (enhanced 6 + 4) Course Software Testing & Verification 2014/15 Wishnu Prasetya.](https://reader030.fdocuments.us/reader030/viewer/2022032523/56649d8e5503460f94a7730d/html5/thumbnails/19.jpg)
Formulate and test the class invariant
19
Stack<T>private Object[] contentprivate int toppublic Stack()push(T x)T pop()bool classinv() { return 0top && top<content.length } }
Example test-cases, check class-inv after :1. call to the constructor2. constructor ; push(x)3. constructor ; push(x) ; push()4. constructor ; push(x) ; pop()5. some random sequence of
push and pop
![Page 20: Some Testing Techniques (enhanced 6 + 4) Course Software Testing & Verification 2014/15 Wishnu Prasetya.](https://reader030.fdocuments.us/reader030/viewer/2022032523/56649d8e5503460f94a7730d/html5/thumbnails/20.jpg)
Specifying a class as an ADT
• An Abstract Data Type (ADT) is a model of a (stateful) data structure. The data structure is modeled abstractly by only describing a set of operations (without exposing the actual state).
• The semantic is described in terms of “logical properties” (also called the ADT’s axioms) over those operations.
20
![Page 21: Some Testing Techniques (enhanced 6 + 4) Course Software Testing & Verification 2014/15 Wishnu Prasetya.](https://reader030.fdocuments.us/reader030/viewer/2022032523/56649d8e5503460f94a7730d/html5/thumbnails/21.jpg)
Example : stack
21
Stack<T>bool isEmpty()push(T x)T pop()
Stack axioms :
• For all x,s : s.push(x) ; y = s.pop() ; assert (y==x )
• For all x and s : s.push(x) ; assert ( s.isEmpty())
• For all x and s.isEmpty() : s.push(x) ; s.pop() assert (s.isEmpty())
Depending of the offered operations, it may be hard/not
possible to get a complete axiomatization.
![Page 22: Some Testing Techniques (enhanced 6 + 4) Course Software Testing & Verification 2014/15 Wishnu Prasetya.](https://reader030.fdocuments.us/reader030/viewer/2022032523/56649d8e5503460f94a7730d/html5/thumbnails/22.jpg)
Test each ADT’s axiom
22
For all x,s : s.push(x) ; y = s.pop ; assert (y==x )
For example, three test cases :
1. empty s2. non-empty s3. s already contains x
![Page 23: Some Testing Techniques (enhanced 6 + 4) Course Software Testing & Verification 2014/15 Wishnu Prasetya.](https://reader030.fdocuments.us/reader030/viewer/2022032523/56649d8e5503460f94a7730d/html5/thumbnails/23.jpg)
Specifying a class with a finite state machine (FSM) (2.5.1, 2.5.2)
23
open()
close()
write()
File:
• Specifies which sequences of operations are valid• Can be combined with class-inv: test that after every valid
sequence the class-inv hold. • You can re-apply graph-based coverage concepts we had before
FSM can also come from your UML models.
![Page 24: Some Testing Techniques (enhanced 6 + 4) Course Software Testing & Verification 2014/15 Wishnu Prasetya.](https://reader030.fdocuments.us/reader030/viewer/2022032523/56649d8e5503460f94a7730d/html5/thumbnails/24.jpg)
In a larger project....
• You want to test your class Heater ; it uses Thermometer which is not ready yet!
• We can opt to use a mock Thermometer. A mock of a program P:– has the same interface as P– only implement a very small subset of P’s behavior– fully under your control
• Analogously we have the concept of mock object. • Make mocks yourself e.g. exploiting inheritance, or use
a mocking tool.
24
![Page 25: Some Testing Techniques (enhanced 6 + 4) Course Software Testing & Verification 2014/15 Wishnu Prasetya.](https://reader030.fdocuments.us/reader030/viewer/2022032523/56649d8e5503460f94a7730d/html5/thumbnails/25.jpg)
Mocking with Moq
25
test1() { Heater heater = new Heater() var mock = new Mock<IThermometer>() mock.Setup(t => t.value()).Returns(-275.15) heater.thermometer = mock.object heater.limit = 303.0 heater.check() Assert.IsFalse(heater.active)}
interface IThermometer double value()double warmUp(double v)
class Heaterdouble limitbool activepublic check() { if (thermometer.value() >= limit) active = false}
thermometer
![Page 26: Some Testing Techniques (enhanced 6 + 4) Course Software Testing & Verification 2014/15 Wishnu Prasetya.](https://reader030.fdocuments.us/reader030/viewer/2022032523/56649d8e5503460f94a7730d/html5/thumbnails/26.jpg)
Mocking with Moq(google it for more info!)
26
var mock = new Mock<IThermometer>() mock.Setup(t => t.value()).Returns(303.00001)
mock.Setup(t => t.warmUp(0)).Returns(0)
mock.Setup(t => t.warmUp(It.IsInRange <double>(-10, 10, Range.Inclusive)) .Returns(0)
mock.Setup(t => t.warmUp (It.IsAny<double>())) .Returns((double s) => s + 273.15)
Many more mock-functionalities in Moq.But well, mocking can be tedious. E.g. what to do when your Heater wants to call warmUp in an iteration?
![Page 27: Some Testing Techniques (enhanced 6 + 4) Course Software Testing & Verification 2014/15 Wishnu Prasetya.](https://reader030.fdocuments.us/reader030/viewer/2022032523/56649d8e5503460f94a7730d/html5/thumbnails/27.jpg)
Beyond unit testing, what if we want to abstract away from the source code ?
• Or you simply don’t have access to it• (Def 1.26) White box testing : common at the
unit-testing level• (Def 1.25) Black box testing: common at the
system-testing level. Approaches :– Partition-based testing– Model-based testing
27
![Page 28: Some Testing Techniques (enhanced 6 + 4) Course Software Testing & Verification 2014/15 Wishnu Prasetya.](https://reader030.fdocuments.us/reader030/viewer/2022032523/56649d8e5503460f94a7730d/html5/thumbnails/28.jpg)
Partitioning the inputs
• Based on “your best understanding” of save’s semantic.• Terminology: characteristic, block. The domain of a
characteristic is divided into disjoint blocks; the union of these blocks must cover the entire domain of the characteristic.
• Assumption : values of the same block are “equivalent”28
save(String fname, Object o)
fname : (A) existing file (B) non-existing file
o : (P) null (Q) non-null serializable (R) non-null non-serializable
![Page 29: Some Testing Techniques (enhanced 6 + 4) Course Software Testing & Verification 2014/15 Wishnu Prasetya.](https://reader030.fdocuments.us/reader030/viewer/2022032523/56649d8e5503460f94a7730d/html5/thumbnails/29.jpg)
So, what input values to choose?
• (C4.23, ALL) All combinations must be tested. |T| = (i: 0i<k: Bi) ; does not scale up.
• (C4.24, EACH CHOICE) Each block must be tested. |T| = (max i: 0i<k: Bi) ; usually too weak.
29
save(String fname, Object o)
fname : (A) existing file (B) non-existing file
o : (P) null (Q) non-null serializable (R) non-null non-serializable
![Page 30: Some Testing Techniques (enhanced 6 + 4) Course Software Testing & Verification 2014/15 Wishnu Prasetya.](https://reader030.fdocuments.us/reader030/viewer/2022032523/56649d8e5503460f94a7730d/html5/thumbnails/30.jpg)
t-wise coverage
• (C4.25, pair-wise coverage). Each pair of blocks (from different characteristics) must be tested.
• (C4.26, t-wise coverage). Generalization of pair-wise.• Obviously stronger than EACH CHOICE, and still
scalable. • Problem: we just blindly combine; no semantical
awareness. 30
AB
PQ
XY
T : (A,P,X) , (A,Q,Y) , ... more?
![Page 31: Some Testing Techniques (enhanced 6 + 4) Course Software Testing & Verification 2014/15 Wishnu Prasetya.](https://reader030.fdocuments.us/reader030/viewer/2022032523/56649d8e5503460f94a7730d/html5/thumbnails/31.jpg)
Adding a bit of semantic
• (C4.27, Base Choice Coverage, BCC) Decide a single base test t0. Make more tests by each time removing one block from t0, and forming combinations with all remaining blocks (of the same characteristics).
|T| = 1 + (i : 0i<k : Bi - 1) 31
AB
PQ
XYZ
Example: t0 = (A,P,X), generates these additional test requirements : (B,P,X) (A,Q,X) (A,P,Y) (A,P,Z)
![Page 32: Some Testing Techniques (enhanced 6 + 4) Course Software Testing & Verification 2014/15 Wishnu Prasetya.](https://reader030.fdocuments.us/reader030/viewer/2022032523/56649d8e5503460f94a7730d/html5/thumbnails/32.jpg)
Or, more bits of semantics
• (C4.28, Multiple Base Choices). For each characteristic we decide at least one base block. Then decide a set of base tests; each only include base blocks. For each base test, generate more tests by each time removing one base block, and forming combinations with remaining non-base blocks.|T| at most M + (i : 0i<k : M*(Bi - mi)) 32
AB
PQ
XYZ
Example, base tests = (A,P,X), (A,P,Y)(A,P,X) generates (B,P,X) (A,Q,X) (A,P,Z)
(A,P,Y) generates (B,P,Y) (A,Q,Y) (A,P,Z)
![Page 33: Some Testing Techniques (enhanced 6 + 4) Course Software Testing & Verification 2014/15 Wishnu Prasetya.](https://reader030.fdocuments.us/reader030/viewer/2022032523/56649d8e5503460f94a7730d/html5/thumbnails/33.jpg)
Example-2, MBCC
• (C4.28, Multiple Base Choices). For each characteristic we decide at least one base block. Then decide a set of base tests; each only include base blocks. For each base test, generate more tests by each time removing one base block, and forming combinations with remaining non-base blocks. 33
ABC
PQR
XYZ
Bold : base blocksChosen base tests = (A,P,X), (A,Q,Y)These produce these additional test requirements:
(B,P,X)(C,P,X)
(B,Q,Y)(C,Q,Y)
(A,R,X)
(A,R,Y)
(A,P,Z)
(A,Q,Z)• base-blocks are not cross-combined
except as in the base tests.• non-base blocks are not cross-
combined with each other.
![Page 34: Some Testing Techniques (enhanced 6 + 4) Course Software Testing & Verification 2014/15 Wishnu Prasetya.](https://reader030.fdocuments.us/reader030/viewer/2022032523/56649d8e5503460f94a7730d/html5/thumbnails/34.jpg)
Constraints, to exclude non-sensical cases
• Example:– combo (A,P,Y) is not allowed.– if P is selected, then X must also be selected.
• Solvable: pair-wise coverage + (A,P,Y) is not allowed.• Can be unsolvable, e.g. pair-wise coverage + (A,P) is
not allowed.• General problem: given a coverage criterion C and a
set of constraints, find a test set T satisfying both. • In general the problem is not trivial to solve.
34
![Page 35: Some Testing Techniques (enhanced 6 + 4) Course Software Testing & Verification 2014/15 Wishnu Prasetya.](https://reader030.fdocuments.us/reader030/viewer/2022032523/56649d8e5503460f94a7730d/html5/thumbnails/35.jpg)
Overview of partition-based coverage
35
EACH CHOICE
ALL
t-Wise Multiple Base Choice Coverage
Pair-Wise Base Choice Coverage
![Page 36: Some Testing Techniques (enhanced 6 + 4) Course Software Testing & Verification 2014/15 Wishnu Prasetya.](https://reader030.fdocuments.us/reader030/viewer/2022032523/56649d8e5503460f94a7730d/html5/thumbnails/36.jpg)
Model-based testing
• We already seen that use an FSM as a model of the system you test
• Such an FSM specifies which sequences of interactions are valid, you can use this information to guide your testing.
36
open()
close()
write()
File:
![Page 37: Some Testing Techniques (enhanced 6 + 4) Course Software Testing & Verification 2014/15 Wishnu Prasetya.](https://reader030.fdocuments.us/reader030/viewer/2022032523/56649d8e5503460f94a7730d/html5/thumbnails/37.jpg)
Model-based testing
• We can add further details, e.g. system-invariants, or by decorating the states in the FSM with predicates that are supposed to hold there; provided the system allows such predicates to be inspected (in the black-box setting the system under test will have limited observability)
37
open()
close()
write()
File:
locked() locked()
![Page 38: Some Testing Techniques (enhanced 6 + 4) Course Software Testing & Verification 2014/15 Wishnu Prasetya.](https://reader030.fdocuments.us/reader030/viewer/2022032523/56649d8e5503460f94a7730d/html5/thumbnails/38.jpg)
Regression Test
• To test that a new modification in your program does not break old functionalities. To be efficient, people typically reuse existing test sets.
• Usually applied for system-testing, where the problem is considered as more urgent. Challenge: very time consuming (hours/days!).
38
![Page 39: Some Testing Techniques (enhanced 6 + 4) Course Software Testing & Verification 2014/15 Wishnu Prasetya.](https://reader030.fdocuments.us/reader030/viewer/2022032523/56649d8e5503460f94a7730d/html5/thumbnails/39.jpg)
Some concepts first...• Test Selection Problem: suppose P has been modified
to P’. Let T be the test set used on P. Choose a subset T’T to test P’.
• Obviously, exclude obsolete test cases: those that can’t execute or whose oracles no longer reflect P’ semantic. Let’s assume: we can identify them.
• You want the selection to be safe : T’ includes all test-cases in T that will execute differently on P’.
• Only attractive if the cost of calculating T’ + executing T’ is less than simply re-executing the whole T.
39
![Page 40: Some Testing Techniques (enhanced 6 + 4) Course Software Testing & Verification 2014/15 Wishnu Prasetya.](https://reader030.fdocuments.us/reader030/viewer/2022032523/56649d8e5503460f94a7730d/html5/thumbnails/40.jpg)
Idea: select those that pass through modified code
• If m is the only method in P that changes, the obvious strategy is to select only test-cases that pass through m.
• Better: only select test-cases that pass m’s “modified” branch.
40
m(x) { if (d) y = x+1 else y=0 }
m(x) { if (d) y = x-1 else y=0 }
(orginal)
(modified)
![Page 41: Some Testing Techniques (enhanced 6 + 4) Course Software Testing & Verification 2014/15 Wishnu Prasetya.](https://reader030.fdocuments.us/reader030/viewer/2022032523/56649d8e5503460f94a7730d/html5/thumbnails/41.jpg)
Corner case
• Both branches of the first if are modified. Using the previous strategy means that we have to select all test-cases that pass m. Yet we see that the paths [d, e, stmt] and [ d, e] present in both old and new m; so there is actually no need to select them.
41
m(x) { if (d) y = x+1 ; if (e) stmt }
m(x) { if (d) { y = x+1 ; if (e) stmt ; u = 0 } else if (e) stmt }
![Page 42: Some Testing Techniques (enhanced 6 + 4) Course Software Testing & Verification 2014/15 Wishnu Prasetya.](https://reader030.fdocuments.us/reader030/viewer/2022032523/56649d8e5503460f94a7730d/html5/thumbnails/42.jpg)
Looking at it abstractly with CFG
42
m(x) { if (d) y = x+1 ; if (e) stmt }
m(x) { if (d) { y = x+1 ; if (e) stmt ; u = 0 } else if (e) stmt }
dy=x+1
e stmt
u=0
end
estmt
dy=x+1
e stmt
end
Notice that [d, e, stmt, end] and [d, e, end] appear in both, and equivalent.
![Page 43: Some Testing Techniques (enhanced 6 + 4) Course Software Testing & Verification 2014/15 Wishnu Prasetya.](https://reader030.fdocuments.us/reader030/viewer/2022032523/56649d8e5503460f94a7730d/html5/thumbnails/43.jpg)
Some concepts• We assume: P is deterministic. each test-case
always generate the same test path.• Let p and p’ be the test-paths of a test-case t when
executed on P and P’; t is modification traversing if not(p p’). let’s select modification traversing test-cases.
• p p’ if they have the same length, and for each i, pi p’i the latter means they contain the same sequence of instructions.
• So far this is not helpful, because such a selection strategy requires us to first execute t on P’. Then it is not attractive anymore!
43
![Page 44: Some Testing Techniques (enhanced 6 + 4) Course Software Testing & Verification 2014/15 Wishnu Prasetya.](https://reader030.fdocuments.us/reader030/viewer/2022032523/56649d8e5503460f94a7730d/html5/thumbnails/44.jpg)
“Intersection” Graph
• First, extend CFG so that branches are labelled by the corresponding decision value (e.g. T/F for if-branch). Label non-branch edge with some constant value.
• Each node of G’’ is a pair (u,u’). Then G’’ is defined like this :– The pair of initial nodes (s0,s0’) G’’. – If (u,u’)G’’, and uu’, and uv is an edge in G, and u’v’ and edge in G’
both with the same label, then (u,u’) (v,v’) should be an edge in G’’.44
a
c
end
b
G : a
c
end
b
G’ :
c
d
G’’ = G G’: a,a
c,c
end,end
b,bc,c
end,d
![Page 45: Some Testing Techniques (enhanced 6 + 4) Course Software Testing & Verification 2014/15 Wishnu Prasetya.](https://reader030.fdocuments.us/reader030/viewer/2022032523/56649d8e5503460f94a7730d/html5/thumbnails/45.jpg)
“Intersection” Graph
• Each path p in G’’ describes how a path in G would be executed on G’ if the same decisions are taken along the way. Note that this is calculated without re-executing any test-case on P’.
• Any path in G’’ ends either in a proper exit node (green), or in a pair (u,u’) where not uv (red). This would be the first time a test-case would hit a modified code when re-executed on P’.
• The old test-cases are assumed to have been instrumented, so that we know which nodes/edges in G it traversed. 45
a
c
end
b
G : a
c
end
b
G’ :
c
d
G’’ = G G’: a,a
c,c
end,end
b,bc,c
end,d
![Page 46: Some Testing Techniques (enhanced 6 + 4) Course Software Testing & Verification 2014/15 Wishnu Prasetya.](https://reader030.fdocuments.us/reader030/viewer/2022032523/56649d8e5503460f94a7730d/html5/thumbnails/46.jpg)
Selection Algorithm
• Select test-cases that pass [a,b,c,end] in G a bit expensive• (Safe but not minimalistic) Select test-cases that pass a node u in G that is
part of a red-node in G’’. same problem as before, it will select also select [a,c,end] which is not modification traversal.
• (Rothermel-Harold, 1997) Select test-cases that pass an edge e in G that in G’’ leads to a red-node in G’’. actually the same problem.
46
a
c
end
b
G : a
c
end
b
G’ :
c
d
G’’ = G G’: a,a
c,c
end,end
b,bc,c
end,d
![Page 47: Some Testing Techniques (enhanced 6 + 4) Course Software Testing & Verification 2014/15 Wishnu Prasetya.](https://reader030.fdocuments.us/reader030/viewer/2022032523/56649d8e5503460f94a7730d/html5/thumbnails/47.jpg)
Selection Algorithm
• (Ball algorithm,1998) Partition G’’ nodes to those can can reach a green-node (G partition), and those that cannot (NG partition). Look at edges in G’’ that cross these partitions (so, from G to NG).
A test path p is modification traversing if and only if it passes through a crossing edge (as meant above). use this as the selection criterion.
47
a
c
end
b
G : a
c
end
b
G’ :
c
d
G’’ = G G’: a,a
c,cend,end
b,b c,c
end,d
NG-partition
G-partition