Mutation Testing with PIT (Booster 2014, 2014-MAR-13)
-
Upload
filip-van-laenen -
Category
Technology
-
view
696 -
download
4
description
Transcript of Mutation Testing with PIT (Booster 2014, 2014-MAR-13)
![Page 1: Mutation Testing with PIT (Booster 2014, 2014-MAR-13)](https://reader031.fdocuments.us/reader031/viewer/2022020115/5481ceedb4af9f7d0d8b45f8/html5/thumbnails/1.jpg)
© Computas AS 11.03.14
Mutation Testing with PIT
Filip van LaenenBooster 20142014-03-13
![Page 2: Mutation Testing with PIT (Booster 2014, 2014-MAR-13)](https://reader031.fdocuments.us/reader031/viewer/2022020115/5481ceedb4af9f7d0d8b45f8/html5/thumbnails/2.jpg)
2 © Computas AS 11.03.14
Agenda
• Basics of mutation testing• Relation to other testing techniques• Example• Mutation testing techniques• Mutation testing tools• Personal experiences and recommendations
![Page 3: Mutation Testing with PIT (Booster 2014, 2014-MAR-13)](https://reader031.fdocuments.us/reader031/viewer/2022020115/5481ceedb4af9f7d0d8b45f8/html5/thumbnails/3.jpg)
3 © Computas AS 11.03.14
Basics ofMutation Testing
![Page 4: Mutation Testing with PIT (Booster 2014, 2014-MAR-13)](https://reader031.fdocuments.us/reader031/viewer/2022020115/5481ceedb4af9f7d0d8b45f8/html5/thumbnails/4.jpg)
4 © Computas AS 11.03.14
Mutation Testing in a Nutshell
Seeking The Summoner @ The Daily WTFhttp://thedailywtf.com/Articles/Seeking-The-Summoner.aspx
![Page 5: Mutation Testing with PIT (Booster 2014, 2014-MAR-13)](https://reader031.fdocuments.us/reader031/viewer/2022020115/5481ceedb4af9f7d0d8b45f8/html5/thumbnails/5.jpg)
5 © Computas AS 11.03.14
Mutation Testing in a Nutshell (cont'd)
![Page 6: Mutation Testing with PIT (Booster 2014, 2014-MAR-13)](https://reader031.fdocuments.us/reader031/viewer/2022020115/5481ceedb4af9f7d0d8b45f8/html5/thumbnails/6.jpg)
6 © Computas AS 11.03.14
Mutation Testing in a Nutshell (cont'd)
![Page 7: Mutation Testing with PIT (Booster 2014, 2014-MAR-13)](https://reader031.fdocuments.us/reader031/viewer/2022020115/5481ceedb4af9f7d0d8b45f8/html5/thumbnails/7.jpg)
7 © Computas AS 11.03.14
Mutation Testing in a Nutshell (cont'd)
• Unit tests guard the source code• But who guards the guardians?
• Do the unit tests cover all source code?• Lines?• Branches?• Paths?
• Do the unit tests test the right things?
Mutation testing tests the tests!
![Page 8: Mutation Testing with PIT (Booster 2014, 2014-MAR-13)](https://reader031.fdocuments.us/reader031/viewer/2022020115/5481ceedb4af9f7d0d8b45f8/html5/thumbnails/8.jpg)
8 © Computas AS 11.03.14
Mutation Testing in a Nutshell (cont'd)
![Page 9: Mutation Testing with PIT (Booster 2014, 2014-MAR-13)](https://reader031.fdocuments.us/reader031/viewer/2022020115/5481ceedb4af9f7d0d8b45f8/html5/thumbnails/9.jpg)
9 © Computas AS 11.03.14
Mutation Testing in a Nutshell (cont'd)
![Page 10: Mutation Testing with PIT (Booster 2014, 2014-MAR-13)](https://reader031.fdocuments.us/reader031/viewer/2022020115/5481ceedb4af9f7d0d8b45f8/html5/thumbnails/10.jpg)
10 © Computas AS 11.03.14
Mutation Testing in a Nutshell (cont'd)
![Page 11: Mutation Testing with PIT (Booster 2014, 2014-MAR-13)](https://reader031.fdocuments.us/reader031/viewer/2022020115/5481ceedb4af9f7d0d8b45f8/html5/thumbnails/11.jpg)
11 © Computas AS 11.03.14
Mutation Testing in a Nutshell (cont'd)
![Page 12: Mutation Testing with PIT (Booster 2014, 2014-MAR-13)](https://reader031.fdocuments.us/reader031/viewer/2022020115/5481ceedb4af9f7d0d8b45f8/html5/thumbnails/12.jpg)
12 © Computas AS 11.03.14
Mutation Testing in a Nutshell (cont'd)
![Page 13: Mutation Testing with PIT (Booster 2014, 2014-MAR-13)](https://reader031.fdocuments.us/reader031/viewer/2022020115/5481ceedb4af9f7d0d8b45f8/html5/thumbnails/13.jpg)
13 © Computas AS 11.03.14
Mutation Testing in a Nutshell (cont'd)
![Page 14: Mutation Testing with PIT (Booster 2014, 2014-MAR-13)](https://reader031.fdocuments.us/reader031/viewer/2022020115/5481ceedb4af9f7d0d8b45f8/html5/thumbnails/14.jpg)
14 © Computas AS 11.03.14
Mutation Testing in a Nutshell (cont'd)
int max(int a, int b) { return (a < b) ? b : a;}
int max(int a, int b) { return (a <= b) ? b : a;}
![Page 15: Mutation Testing with PIT (Booster 2014, 2014-MAR-13)](https://reader031.fdocuments.us/reader031/viewer/2022020115/5481ceedb4af9f7d0d8b45f8/html5/thumbnails/15.jpg)
15 © Computas AS 11.03.14
Mutation Testing in a Nutshell (cont'd)
![Page 16: Mutation Testing with PIT (Booster 2014, 2014-MAR-13)](https://reader031.fdocuments.us/reader031/viewer/2022020115/5481ceedb4af9f7d0d8b45f8/html5/thumbnails/16.jpg)
“
16 © Computas AS 11.03.14
Does Mutation Testing Work?
In practice, if the software contains a fault, there will usually be a set of
mutants that can only be killed by a test case that also detects that fault.
Geist et. al., “Estimation and Enhancement of Real-time Software Reliability through Mutation Analysis,” 1992
![Page 17: Mutation Testing with PIT (Booster 2014, 2014-MAR-13)](https://reader031.fdocuments.us/reader031/viewer/2022020115/5481ceedb4af9f7d0d8b45f8/html5/thumbnails/17.jpg)
“
17 © Computas AS 11.03.14
Does Mutation Testing Work? (cont'd)
Complex faults are coupled to simple faults in such a way that a test data set
that detects all simple faults in a program will detect most complex faults.
K. Wah, “Fault Coupling in Finite Bijective Functions,” 1995
![Page 18: Mutation Testing with PIT (Booster 2014, 2014-MAR-13)](https://reader031.fdocuments.us/reader031/viewer/2022020115/5481ceedb4af9f7d0d8b45f8/html5/thumbnails/18.jpg)
18 © Computas AS 11.03.14
Does Mutation Testing Work? (cont'd)
• “Generated mutants are similar to real faults.”• Andrews, Briand, Labiche, ICSE 2005
• “Mutation testing is more powerful than statement or branch coverage.”• Walsh, Ph.D. Thesis, State University of New York at
Binghampton, 1985• “Mutation testing is superior to data flow
coverage criteria.”• Frankl, Weiss, Hu, Journal of Systems and Software,
1997
![Page 19: Mutation Testing with PIT (Booster 2014, 2014-MAR-13)](https://reader031.fdocuments.us/reader031/viewer/2022020115/5481ceedb4af9f7d0d8b45f8/html5/thumbnails/19.jpg)
19 © Computas AS 11.03.14
Relation to OtherTesting Techniques
![Page 20: Mutation Testing with PIT (Booster 2014, 2014-MAR-13)](https://reader031.fdocuments.us/reader031/viewer/2022020115/5481ceedb4af9f7d0d8b45f8/html5/thumbnails/20.jpg)
20 © Computas AS 11.03.14
Relation to Other Testing Techniques
• Unit tests• Test-Driven Development (TDD)• Test coverage• Static code analysis• Fuzz testing
![Page 21: Mutation Testing with PIT (Booster 2014, 2014-MAR-13)](https://reader031.fdocuments.us/reader031/viewer/2022020115/5481ceedb4af9f7d0d8b45f8/html5/thumbnails/21.jpg)
21 © Computas AS 11.03.14
Relation to Other Testing Techniques
![Page 22: Mutation Testing with PIT (Booster 2014, 2014-MAR-13)](https://reader031.fdocuments.us/reader031/viewer/2022020115/5481ceedb4af9f7d0d8b45f8/html5/thumbnails/22.jpg)
22 © Computas AS 11.03.14
Is Mutation Testing New?
• R. Lipton, “Fault Diagnosis of Computer Programs,” 1971
• R. Lipton et. al., “Hints on Test Data Selection: Help for the Practicing Programmer,” 1978
• Historical obstacles:• No unit testing• No TDD• Inefficient implementation
• Hence time-consuming• No integration with IDEs
![Page 23: Mutation Testing with PIT (Booster 2014, 2014-MAR-13)](https://reader031.fdocuments.us/reader031/viewer/2022020115/5481ceedb4af9f7d0d8b45f8/html5/thumbnails/23.jpg)
23 © Computas AS 11.03.14
Practical Exampleof Mutation Testing
![Page 24: Mutation Testing with PIT (Booster 2014, 2014-MAR-13)](https://reader031.fdocuments.us/reader031/viewer/2022020115/5481ceedb4af9f7d0d8b45f8/html5/thumbnails/24.jpg)
24 © Computas AS 11.03.14
Code Along…
https://github.com/computas-fvl/booster2014
git clone \https://github.com/computas-fvl/booster2014.git
mvn clean site:site \ org.pitest:pitest-maven:mutationCoverage
![Page 25: Mutation Testing with PIT (Booster 2014, 2014-MAR-13)](https://reader031.fdocuments.us/reader031/viewer/2022020115/5481ceedb4af9f7d0d8b45f8/html5/thumbnails/25.jpg)
25 © Computas AS 11.03.14
Mutation TestingTechniques
![Page 26: Mutation Testing with PIT (Booster 2014, 2014-MAR-13)](https://reader031.fdocuments.us/reader031/viewer/2022020115/5481ceedb4af9f7d0d8b45f8/html5/thumbnails/26.jpg)
26 © Computas AS 11.03.14
Mutation Testing Techniques
• Three aspects:• Mutation injection• Mutation types• Unit test selection per mutant
• Key properties:• Efficiency• Performance
![Page 27: Mutation Testing with PIT (Booster 2014, 2014-MAR-13)](https://reader031.fdocuments.us/reader031/viewer/2022020115/5481ceedb4af9f7d0d8b45f8/html5/thumbnails/27.jpg)
27 © Computas AS 11.03.14
Mutation Injection
• Source code mutation• Binary code mutation• Caveats:
• De-mutation• Compilation errors• Invalid binary code
![Page 28: Mutation Testing with PIT (Booster 2014, 2014-MAR-13)](https://reader031.fdocuments.us/reader031/viewer/2022020115/5481ceedb4af9f7d0d8b45f8/html5/thumbnails/28.jpg)
28 © Computas AS 11.03.14
Mutation Types
• Some mutations never change behaviour• Constants reused by unit tests• Log messages
• Some mutations can change behaviour• Switching between < and ≠• Switching between < and ≤
• Some mutations always change behaviour• Switching between < and ≥
![Page 29: Mutation Testing with PIT (Booster 2014, 2014-MAR-13)](https://reader031.fdocuments.us/reader031/viewer/2022020115/5481ceedb4af9f7d0d8b45f8/html5/thumbnails/29.jpg)
29 © Computas AS 11.03.14
Mutation Types (cont'd)
int max(int a, int b) { return (a < b) ? b : a;}
int max(int a, int b) { return (a <= b) ? b : a;}
int max(int a, int b) { return (a >= b) ? b : a;}
![Page 30: Mutation Testing with PIT (Booster 2014, 2014-MAR-13)](https://reader031.fdocuments.us/reader031/viewer/2022020115/5481ceedb4af9f7d0d8b45f8/html5/thumbnails/30.jpg)
30 © Computas AS 11.03.14
Mutation Types (cont'd)
for (int i = 0; i < 10; i++) …
for (int i = 0; i != 10; i++) …
for (int i = 0; i >= 10; i++) …
![Page 31: Mutation Testing with PIT (Booster 2014, 2014-MAR-13)](https://reader031.fdocuments.us/reader031/viewer/2022020115/5481ceedb4af9f7d0d8b45f8/html5/thumbnails/31.jpg)
31 © Computas AS 11.03.14
Mutation Types Guaranteed to Change Behaviour
• Negation of the comparison• Switching between = and ≠• Switching between < and ≥• Switching between > and ≤
• Negation of boolean conditions• Adding a ¬, ! or ~
• Shortcutting boolean conditions• Replacement with True or False
*
![Page 32: Mutation Testing with PIT (Booster 2014, 2014-MAR-13)](https://reader031.fdocuments.us/reader031/viewer/2022020115/5481ceedb4af9f7d0d8b45f8/html5/thumbnails/32.jpg)
32 © Computas AS 11.03.14
Unit Test Selection
• Goal: find the unit test that “kills” the mutant• Selection aids:
• Hints• Name/package matching• Code coverage tools• Automatic learning• Other heuristics
![Page 33: Mutation Testing with PIT (Booster 2014, 2014-MAR-13)](https://reader031.fdocuments.us/reader031/viewer/2022020115/5481ceedb4af9f7d0d8b45f8/html5/thumbnails/33.jpg)
33 © Computas AS 11.03.14
Unit Test Selection (cont'd)
• System:• 50 classes• 20 unit tests per class• 1 ms per unit test• Unit testing time: 50 × 20 × 1ms = 1s
• 10 mutants per class:• Brute-force: 10 × 50 × 1s = 6m 20s• Educated: 10 × 50 × 20 × 1ms = 10s
![Page 34: Mutation Testing with PIT (Booster 2014, 2014-MAR-13)](https://reader031.fdocuments.us/reader031/viewer/2022020115/5481ceedb4af9f7d0d8b45f8/html5/thumbnails/34.jpg)
34 © Computas AS 11.03.14
Unit Test Selection (cont'd)
• System:• 500 classes• 20 unit tests per class• 1 ms per unit test• Unit testing time: 500 × 20 × 1ms = 10s
• 10 mutants per class:• Brute-force: 10 × 500 × 10s = 13h 53m 20s• Educated: 10 × 500 × 20 × 1ms = 1m 40s
![Page 35: Mutation Testing with PIT (Booster 2014, 2014-MAR-13)](https://reader031.fdocuments.us/reader031/viewer/2022020115/5481ceedb4af9f7d0d8b45f8/html5/thumbnails/35.jpg)
35 © Computas AS 11.03.14
Complexity
• f: Number of function points• φ: Number of function points per class, ≥ 1• τ: Number of unit tests per function point, ≥ 1• μ: Number of mutants per function point, ≥ 1
• Brute-force: (f × τ) × (f × μ) = τ × μ × f²• Educated force: τ × μ × φ × f• Ideal: τ × μ × f
![Page 36: Mutation Testing with PIT (Booster 2014, 2014-MAR-13)](https://reader031.fdocuments.us/reader031/viewer/2022020115/5481ceedb4af9f7d0d8b45f8/html5/thumbnails/36.jpg)
36 © Computas AS 11.03.14
Complexity (cont'd)
• c: Number of classes• φ: Number of function points per class, ≥ 1• τ: Number of unit tests per function point, ≥ 1• μ: Number of mutants per function point, ≥ 1
• Brute-force: (f × τ) × (f × μ) = τ × μ × φ² × c²• Educated force: τ × μ × φ² × c• Ideal: τ × μ × φ × c
![Page 37: Mutation Testing with PIT (Booster 2014, 2014-MAR-13)](https://reader031.fdocuments.us/reader031/viewer/2022020115/5481ceedb4af9f7d0d8b45f8/html5/thumbnails/37.jpg)
37 © Computas AS 11.03.14
Loops
for (int i = 0; i < 10; i++) …
for (int i = 0; i < 10; i--) …
![Page 38: Mutation Testing with PIT (Booster 2014, 2014-MAR-13)](https://reader031.fdocuments.us/reader031/viewer/2022020115/5481ceedb4af9f7d0d8b45f8/html5/thumbnails/38.jpg)
38 © Computas AS 11.03.14
Infinite Loops
• Terminate mutants that take too long to run• What's “too long”?
• Ruins the performance• Can be hard to predict
![Page 39: Mutation Testing with PIT (Booster 2014, 2014-MAR-13)](https://reader031.fdocuments.us/reader031/viewer/2022020115/5481ceedb4af9f7d0d8b45f8/html5/thumbnails/39.jpg)
39 © Computas AS 11.03.14
Other Problems
• Recursion:• Stack overflows• Out of memory exceptions
• Syntax errors• Segmentation faults
![Page 40: Mutation Testing with PIT (Booster 2014, 2014-MAR-13)](https://reader031.fdocuments.us/reader031/viewer/2022020115/5481ceedb4af9f7d0d8b45f8/html5/thumbnails/40.jpg)
40 © Computas AS 11.03.14
Mutation TestingTools
![Page 41: Mutation Testing with PIT (Booster 2014, 2014-MAR-13)](https://reader031.fdocuments.us/reader031/viewer/2022020115/5481ceedb4af9f7d0d8b45f8/html5/thumbnails/41.jpg)
41 © Computas AS 11.03.14
Mutation Testing Tools
• Java:• PIT• Jester (Nester, Pester)• Jumble
• Ruby:• Mutant• Heckle
• C#:• NinjaTurtles
![Page 42: Mutation Testing with PIT (Booster 2014, 2014-MAR-13)](https://reader031.fdocuments.us/reader031/viewer/2022020115/5481ceedb4af9f7d0d8b45f8/html5/thumbnails/42.jpg)
42 © Computas AS 11.03.14
PIT
• Java• Junit & TestNG• Maven or command-line• Operates on byte code• Large set of mutators
• Also possibly equivalent mutators available• Highly configurable• Sensible defaults
• http://pitest.org/
![Page 43: Mutation Testing with PIT (Booster 2014, 2014-MAR-13)](https://reader031.fdocuments.us/reader031/viewer/2022020115/5481ceedb4af9f7d0d8b45f8/html5/thumbnails/43.jpg)
43 © Computas AS 11.03.14
PIT Mutators
• Conditionals Boundary• Negate Conditionals• Remove Conditionals*• Math• Increments• Invert Negatives• Inline Constant*• Return Values• Void Method Calls• Non Void Method Calls*• Constructor Calls*
![Page 44: Mutation Testing with PIT (Booster 2014, 2014-MAR-13)](https://reader031.fdocuments.us/reader031/viewer/2022020115/5481ceedb4af9f7d0d8b45f8/html5/thumbnails/44.jpg)
44 © Computas AS 11.03.14
PIT Sample Report
![Page 45: Mutation Testing with PIT (Booster 2014, 2014-MAR-13)](https://reader031.fdocuments.us/reader031/viewer/2022020115/5481ceedb4af9f7d0d8b45f8/html5/thumbnails/45.jpg)
45 © Computas AS 11.03.14
Jester
• Java• JUnit• Usually run from the command-line
• Grester for Maven2• Operates on source code• Runs all unit tests on all classes• Reporting and documentation could be better
• http://jester.sourceforge.net/• http://sourceforge.net/projects/grester/
![Page 46: Mutation Testing with PIT (Booster 2014, 2014-MAR-13)](https://reader031.fdocuments.us/reader031/viewer/2022020115/5481ceedb4af9f7d0d8b45f8/html5/thumbnails/46.jpg)
46 © Computas AS 11.03.14
Jester Sample Report Overview
![Page 47: Mutation Testing with PIT (Booster 2014, 2014-MAR-13)](https://reader031.fdocuments.us/reader031/viewer/2022020115/5481ceedb4af9f7d0d8b45f8/html5/thumbnails/47.jpg)
47 © Computas AS 11.03.14
Jester Sample Detailed Report
![Page 48: Mutation Testing with PIT (Booster 2014, 2014-MAR-13)](https://reader031.fdocuments.us/reader031/viewer/2022020115/5481ceedb4af9f7d0d8b45f8/html5/thumbnails/48.jpg)
48 © Computas AS 11.03.14
Pester and Nester
• Pester• Jester for Python• PyUnit
• Nester• Port of Jester for C#• NUnit• Integrated with Visual Studio• But outdated…• http://nester.sourceforge.net/
![Page 49: Mutation Testing with PIT (Booster 2014, 2014-MAR-13)](https://reader031.fdocuments.us/reader031/viewer/2022020115/5481ceedb4af9f7d0d8b45f8/html5/thumbnails/49.jpg)
49 © Computas AS 11.03.14
Nester Sample Report
![Page 50: Mutation Testing with PIT (Booster 2014, 2014-MAR-13)](https://reader031.fdocuments.us/reader031/viewer/2022020115/5481ceedb4af9f7d0d8b45f8/html5/thumbnails/50.jpg)
50 © Computas AS 11.03.14
Jumble
• Java• JUnit• Run from the command-line• Operates on byte code• Runs unit tests on a class• Reporting and documentation could be better• Claims to be faster than Jester
• http://jumble.sourceforge.net/index.html
![Page 51: Mutation Testing with PIT (Booster 2014, 2014-MAR-13)](https://reader031.fdocuments.us/reader031/viewer/2022020115/5481ceedb4af9f7d0d8b45f8/html5/thumbnails/51.jpg)
51 © Computas AS 11.03.14
Jumble Sample Report
Mutating FooTests: FooTestMutation points = 12, unit test time limit 2.02s..M FAIL: Foo:31: negated conditionalM FAIL: Foo:33: negated conditionalM FAIL: Foo:34: - -> +M FAIL: Foo:35: negated conditional......Score: 67%
![Page 52: Mutation Testing with PIT (Booster 2014, 2014-MAR-13)](https://reader031.fdocuments.us/reader031/viewer/2022020115/5481ceedb4af9f7d0d8b45f8/html5/thumbnails/52.jpg)
52 © Computas AS 11.03.14
Mutant
• Ruby 1.9 and 2.0• rspec2• Usually run from the command-line• Good to-the-point reporting• Good performance• OK documentation• Active project
• https://github.com/mbj/mutant
![Page 53: Mutation Testing with PIT (Booster 2014, 2014-MAR-13)](https://reader031.fdocuments.us/reader031/viewer/2022020115/5481ceedb4af9f7d0d8b45f8/html5/thumbnails/53.jpg)
53 © Computas AS 11.03.14
Heckle
• Ruby 1.8• Doesn't work on Ruby 1.9
• Test::Unit and rSpec• Usually run from the command-line• Runs a set of unit tests on a class or a method• Good to-the-point reporting• Good performance• Virtually no documentation
• http://rubyforge.org/projects/seattlerb/• http://docs.seattlerb.org/heckle/
![Page 54: Mutation Testing with PIT (Booster 2014, 2014-MAR-13)](https://reader031.fdocuments.us/reader031/viewer/2022020115/5481ceedb4af9f7d0d8b45f8/html5/thumbnails/54.jpg)
54 © Computas AS 11.03.14
Heckle Mutations
• Booleans• Numbers• Strings• Symbols• Ranges• Regexes• Branches (if, while, unless, until)
![Page 55: Mutation Testing with PIT (Booster 2014, 2014-MAR-13)](https://reader031.fdocuments.us/reader031/viewer/2022020115/5481ceedb4af9f7d0d8b45f8/html5/thumbnails/55.jpg)
55 © Computas AS 11.03.14
NinjaTurtles
• .Net:• C#• Visual Basic/VB.NET• Any other .NET language code
• Doesn't seem much alive
• http://www.mutation-testing.net/
![Page 56: Mutation Testing with PIT (Booster 2014, 2014-MAR-13)](https://reader031.fdocuments.us/reader031/viewer/2022020115/5481ceedb4af9f7d0d8b45f8/html5/thumbnails/56.jpg)
56 © Computas AS 11.03.14
NinjaTurtles Mutations
• Sequence point deletion• Arithmetic operator substitution (*, /, +, -, %)• Bitwise operator substitution (&, |, ^)• Branch substituion (condition, always and never
branch)• Conditional boundary substition (< and <=, >
and >=)• Substitution of reads from variables,
parameters and fields of the same type• Substitution of writes to variables of the same
type
![Page 57: Mutation Testing with PIT (Booster 2014, 2014-MAR-13)](https://reader031.fdocuments.us/reader031/viewer/2022020115/5481ceedb4af9f7d0d8b45f8/html5/thumbnails/57.jpg)
57 © Computas AS 11.03.14
Personal Experiencesand Recommendations
![Page 58: Mutation Testing with PIT (Booster 2014, 2014-MAR-13)](https://reader031.fdocuments.us/reader031/viewer/2022020115/5481ceedb4af9f7d0d8b45f8/html5/thumbnails/58.jpg)
58 © Computas AS 11.03.14
Experiences and Recommendations
• Use mutation testing from day 1• Start on a small code base
• Keep number of unit tests per class low• Have small classes
• Select a good tool• Configurable• Flexible• One that can output the mutant
![Page 59: Mutation Testing with PIT (Booster 2014, 2014-MAR-13)](https://reader031.fdocuments.us/reader031/viewer/2022020115/5481ceedb4af9f7d0d8b45f8/html5/thumbnails/59.jpg)
59 © Computas AS 11.03.14
Experiences and Recommendations (cont'd)
• Believe the tool• Or try to proof that the tool is wrong
• Fix the problem• Don't turn mutation testing off
• Embrace the “more than 100%” test coverage• Path coverage• Less code• More unit tests• More intelligent unit tests
![Page 60: Mutation Testing with PIT (Booster 2014, 2014-MAR-13)](https://reader031.fdocuments.us/reader031/viewer/2022020115/5481ceedb4af9f7d0d8b45f8/html5/thumbnails/60.jpg)
60 © Computas AS 11.03.14
Improvements
![Page 61: Mutation Testing with PIT (Booster 2014, 2014-MAR-13)](https://reader031.fdocuments.us/reader031/viewer/2022020115/5481ceedb4af9f7d0d8b45f8/html5/thumbnails/61.jpg)
61 © Computas AS 11.03.14
Improvements
• Integration with more unit testing frameworks• Better unit test–source code mapping
• Better heuristics• Parallellisation• Better reporting• IDE integration• Building tool integration
![Page 62: Mutation Testing with PIT (Booster 2014, 2014-MAR-13)](https://reader031.fdocuments.us/reader031/viewer/2022020115/5481ceedb4af9f7d0d8b45f8/html5/thumbnails/62.jpg)
62 © Computas AS 11.03.14
Questions?
Computas ASLysaker Torg 45, pb 482N-1327 LysakerNORWAY
Tel +47-67 83 10 00Fax +47-67 83 10 01Org.nr: NO 986 352 325 MVAwww.computas.com
Contact:
[email protected] @filipvanlaenen