DejaVOO: A Regression Testing Tool for Java Software
1
1. Regression Testing • Performed on new versions of the software to provide confidence that: – the changed parts behave as intended – the unchanged parts are not adversely affected by the modifications • Can be very expensive, especially in some contexts – daily re-testing after nightly builds – frequent re-testing during development – for the extreme programming paradigm DejaVOO: A Regression Testing Tool for Java Software * Alessandro Orso, Donglin Liang, Mary Jean Harrold and Manas Tungare — Georgia Institute of Technology http://www.cc.gatech.edu/aristotle/ 2. Regression Test Selection 3. Potential Savings 4. Our Safe Regression Test Selection Algorithm: Overview a. Construct graph representations for P and P’ P’ Version of P Program P T T-T’ T’ T’ A safe algorithm • requires certain assumptions for safety • ensures that T’ contains all test cases in T that can expose a fault in P’ – same execution environment – deterministic test runs – … 12 9 6 3 Time to select T’ Time to run T’ 12 9 6 3 Time to run T Time to select T’ + Time to run T’ < Time to run T Savings if() doC doA if() doB doA G G’ 1 2 1 2 c. Differentiate G, G’ to find dangerous entities if() doC doA if() doB doA G G’ 1 2 1 2 Dangerous Edge b. Associate test cases in T with entities in P tc1 tc2 tc3 en1 X en2 X X TC En d. Select test cases based on dangerous entities tc1 tc2 tc3 en1 X en2 X X TC En 5. Our Safe Regression Test Selection Algorithm: Characteristics • It handles Java object-oriented features resulting in simple components and complex interactions – inheritance – polymorphism and dynamic binding – exception handling • It handles incomplete programs that play an important role in object-oriented software – components – applications using libraries (e.g., AWT) – subsystems JVM: Java Virtual Machine Profiler: JVM Profiling Interface JABA: Java Architecture for Bytecode Analysis 6. DejaVOO: An Implementation of our Regression Test Selection Algorithm for Java Software 7. Empirical Studies: Subjects • Siena: Event notification system (185 methods), 7 versions, 138 test cases, 70% method coverage • Jedit: Text editor (3495 methods), 11 versions, 189 test cases, 75% method coverage • Jmeter: Web-application test tool (109 methods), 5 versions, 50 test cases, 67% method coverage • RegExp: Regular-expression library (168 methods), 10 versions, 66 test cases, 46% method coverage 0 10 20 30 40 50 60 70 80 90 100 % Test Cases Selected Siena V1-V7 Jedit V1-V11 Jmeter V1-V5 RegExp V1-V9 Program Name Sizes of Selected Subsets of T Prototype GUI for DejaVOO • It shows a SeeSoft view of the source code of P • It highlights the lines of code that have been modified • It displays the test cases to be rerun • It lets the user selectively shows which test cases traverse which parts of the code and vice-versa * This work was supported in part by a grant from Boeing Aerospace Corporation to Georgia Tech, by National Science Foundation award CCR-9707792 to Ohio State University, and awards CCR-9988294, CCR-0096321, and EIA-0196145 to Georgia Tech, and by the State of Georgia to Georgia Tech under the Yamacraw Mission. JVM Profiler JABA Graph Traversal Test Selection Tool Events JIG for P' JIG for P T P Dangerous Entities T’ Coverage Matrix Execute P / Record coverage Identify dangerous entities P’ P Architecture of DejaVOO
-
Upload
manas-tungare -
Category
Business
-
view
2.294 -
download
1
description
Transcript of DejaVOO: A Regression Testing Tool for Java Software
1. Regression Testing• Performed on new versions of the software to provide confidence that:
– the changed parts behave as intended– the unchanged parts are not adversely affected by the modifications
• Can be very expensive, especially in some contexts– daily re-testing after nightly builds– frequent re-testing during development– for the extreme programming paradigm
DejaVOO: A Regression Testing Tool for Java Software*
Alessandro Orso, Donglin Liang, Mary Jean Harrold and Manas Tungare — Georgia Institute of Technologyhttp://www.cc.gatech.edu/aristotle/
2. Regression Test Selection
3. Potential Savings
4. Our Safe Regression Test Selection Algorithm: Overview
a. Construct graph representations for P and P’
P’ Versionof P
ProgramP
T
T-T’
T’
T’
A safe algorithm• requires certain assumptions for safety • ensures that T’ contains all test cases in T that can expose a fault in P’
– same execution environment– deterministic test runs– …
12
9
6
3
Time toselect T’
Time torun T’
12
9
6
3Time to run T
Time to select T’ + Time to run T’ < Time to run T Savings
if()
doCdoA
if()
doBdoA
G G’
1 2 1 2
c. Differentiate G, G’ to find dangerous entities
if()
doCdoA
if()
doBdoA
G G’
1 2 1 2
Dangerous Edge
b. Associate test cases in T with entities in P
tc1 tc2 tc3
en1 X
en2 X X
TCEn
d. Select test cases based on dangerous entities
tc1 tc2 tc3
en1 X
en2 X X
TCEn
5. Our Safe Regression Test Selection Algorithm: Characteristics• It handles Java object-oriented features resulting in simple components and complex interactions
– inheritance– polymorphism and dynamic binding– exception handling
• It handles incomplete programs that play an important role in object-oriented software– components– applications using libraries (e.g., AWT)– subsystems
JVM: Java Virtual MachineProfiler: JVM Profiling InterfaceJABA: Java Architecture for Bytecode Analysis
6. DejaVOO: An Implementation of our Regression Test Selection Algorithm for Java Software
7. Empirical Studies: Subjects
• Siena: Event notification system (185 methods), 7 versions, 138 test cases, 70% method coverage• Jedit: Text editor (3495 methods), 11 versions, 189 test cases, 75% method coverage• Jmeter: Web-application test tool (109 methods), 5 versions, 50 test cases, 67% method coverage• RegExp: Regular-expression library (168 methods), 10 versions, 66 test cases, 46% method coverage
0102030405060708090
100
% T
est
Cas
es S
elec
ted
Siena V1-V7
Jedit V1-V11
Jmeter V1-V5
RegExp V1-V9
Program Name
Sizes of Selected Subsets of T
Prototype GUI for DejaVOO• It shows a SeeSoft view of the source code of P • It highlights the lines of code that have been modified• It displays the test cases to be rerun• It lets the user selectively shows which test cases traverse which parts of the code and vice-versa
* This work was supported in part by a grant from Boeing Aerospace Corporation to Georgia Tech, by National Science Foundation award CCR-9707792 to Ohio State University, and awards CCR-9988294, CCR-0096321, and EIA-0196145 to Georgia Tech, and by the State of Georgia to Georgia Tech under the Yamacraw Mission.
JVM Profiler
JABAGraph
Traversal
TestSelection
Tool
Events
JIG for P'JIG for P
T
P
Dangerous
Entities
T’
CoverageMatrix
Execute P / Record coverage
Identify dangerous entities
P’
P
Architecture of DejaVOO
