Using Structural Information and User Feedback to Improve...
48
When and How Using Structural Information to Improve IR-based Traceability Recovery Annibale Panichella 1 , Collin McMillan 2 , Evan Moritz 3 , Davide Palmieri 4 , Rocco Oliveto 4 , Denys Poshyvaniyk 3 , Andrea De Lucia 1 1 Software Engineering Lab , University of Salerno, Italy 2 University of Notre Dame, Notre Dame, USA 3 The College of William and Mary, Williamsburg, USA 2 University of Molise, Pesche (IS), Italiy
Transcript of Using Structural Information and User Feedback to Improve...
When and How Using
Structural Information to
Improve IR-based Traceability
Recovery
Annibale Panichella1, Collin McMillan 2, Evan Moritz 3, Davide Palmieri 4, Rocco Oliveto 4, Denys Poshyvaniyk 3, Andrea De Lucia 1
1 Software Engineering Lab , University of Salerno, Italy 2 University of Notre Dame, Notre Dame, USA
3 The College of William and Mary, Williamsburg, USA 2 University of Molise, Pesche (IS), Italiy
Imapct Analysis
Requirements tracing
Program Comprehension
Traceability information is still not
commonplace in software projects!
Information Retrieval
IR-based Traceability Recovery
Use Case Insert Laboratory Data
Description The user inserts the data of a
specific laboratory
Events 1. The user opens the Laboratory
GUI
2. The user inserts the
laboratory data
.
.
. /* *This class implements the GUI for
managing laboratories data */
public class GUI Laboratory Data {
private jFrame window;
private jButton insert;
...
public GUI Laboratory Data(){
window = ne JFrame();
insert = new JButton();
...
}
...
}
GUILaboratoryData.java
IR-based Traceability Recovery
Use Case Insert Laboratory Data
Description The user inserts the data of a
specific laboratory
Events 1. The user opens the Laboratory
GUI
2. The user inserts the
laboratory data
.
.
. /* *This class implements the GUI for
managing laboratories data */
public class GUI Laboratory Data {
private jFrame window;
private jButton insert;
...
public GUI Laboratory Data(){
window = new JFrame();
insert = new JButton();
...
}
...
}
GUILaboratoryData.java
IR-based Traceability Recovery
Use Case Insert Laboratory Data
Description The user inserts the data of a
specific laboratory
Events 1. The user opens the Laboratory
GUI
2. The user inserts the
laboratory data
.
.
. /* *This class implements the GUI for
managing laboratories data */
public class GUI Laboratory Data {
private jFrame window;
private jButton insert;
...
public GUI Laboratory Data(){
window = new JFrame();
insert = new JButton();
...
}
...
}
GUILaboratoryData.java
Similarity 42%
IR-based Traceability Recovery
Use Case Insert Laboratory Data
Description The user inserts the data of a
specific laboratory
Events 1. The user opens the Laboratory
GUI
2. The user inserts the
laboratory data
.
.
. /* *This class implements the GUI for
managing laboratories data */
public class GUI Laboratory Data {
private jFrame window;
private jButton insert;
...
public GUI Laboratory Data(){
window = new JFrame();
insert = new JButton();
...
}
...
}
GUILaboratoryData.java
True Link
IR-based Traceability Recovery
Source_1 Target_2 95.4%
Source_3 Target_4 92.1%
Source_1 Target_1 85.6%
Source_2 Target_2 83.2%
Source_3 Target_3 81.2%
Source_1 Target_3 79.0%
Source_3 Target_2 77.5%
Source_2 Target_4 64.3%
Source_2 Target_3 53.2%
Target
Artifacts
IR Engine
Source
Artifacts
Candidate Links
Source_1 Target_2 95.4%
Source_3 Target_4 92.1%
Source_1 Target_1 85.6%
Source_2 Target_2 83.2%
Source_3 Target_3 81.2%
Source_1 Target_3 79.0%
Source_3 Target_2 77.5%
Source_2 Target_4 64.3%
Source_2 Target_3 53.2%
IR-based Traceability Recovery
Target
Artifacts
IR Engine
Source
Artifacts
Candidate Links
Structural Information and Traceability
Combining Textual and
Structural Analysis of Software
Artifacts for Traceability Link
Recovery - Collin McMillan, Denys
Poshyvanyk, Meghan Revelle TEFSE 2009
/* *This class implements the GUI for
managing laboratories data */
public class GUILaboratoryData {
private jFrame window;
private jButton insert;
...
public GUILaboratoryData(){
window = ne JFrame();
insert = new JButton();
...
}
...
}
Structural Information and Traceability
Use Case Insert Laboratory Data
Description The user inserts the data of a
specific laboratory
Events 1. The user opens the Laboratory
GUI
2. The user inserts the
laboratory data
.
.
.
GUILaboratoryData.java
linked
public class Laboratory{
private String name;
private String position;
...
public void setName(String pName){
this.name=pName;
}
...
}
Laboratory.java
/* *This class implements the GUI for
managing laboratories data */
public class GUILaboratoryData {
private jFrame window;
private jButton insert;
...
public GUILaboratoryData(){
window = ne JFrame();
insert = new JButton();
...
}
...
}
Use Case Insert Laboratory Data
Description The user inserts the data of a
specific laboratory
Events 1. The user opens the Laboratory
GUI
2. The user inserts the
laboratory data
.
.
.
GUILaboratoryData.java
linked
linked
Structural Information and Traceability
/* *This class implements the GUI for
managing laboratories data */
public class GUILaboratoryData {
private jFrame window;
private jButton insert;
...
public GUILaboratoryData(){
window = ne JFrame();
insert = new JButton();
...
}
...
}
GUILaboratoryData.java
linked
Use Case Insert Laboratory Data
Description The user inserts the data of a
specific laboratory
Events 1. The user opens the Laboratory
GUI
2. The user inserts the
laboratory data
.
.
.
public class Laboratory{
private String name;
private String position;
...
public void setName(String pName){
this.name=pName;
}
...
}
Laboratory.java
linked
Structural Information and Traceability
/* *This class implements the GUI for
managing laboratories data */
public class GUILaboratoryData {
private jFrame window;
private jButton insert;
...
public GUILaboratoryData(){
window = ne JFrame();
insert = new JButton();
...
}
...
}
GUILaboratoryData.java
linked
Use Case Insert Laboratory Data
Description The user inserts the data of a
specific laboratory
Events 1. The user opens the Laboratory
GUI
2. The user inserts the
laboratory data
.
.
.
public class Laboratory{
private String name;
private String position;
...
public void setName(String pName){
this.name=pName;
}
...
}
Laboratory.java
linked
Structural Information and Traceability
Structural dependecy
/* *This class implements the GUI for
managing laboratories data */
public class GUILaboratoryData {
private jFrame window;
private jButton insert;
...
public GUILaboratoryData(){
window = ne JFrame();
insert = new JButton();
...
}
...
}
Use Case Insert Laboratory Data
Description The user inserts the data of a
specific laboratory
Events 1. The user opens the Laboratory
GUI
2. The user inserts the
laboratory data
.
.
.
GUILaboratoryData.java
Structural dependecy
Structural Information and Traceability
public class Laboratory{
private String name;
private String position;
...
public void setName(String pName){
this.name=pName;
}
...
}
Laboratory.java
linked
linked
Transitivity
/* *This class implements the GUI for
managing laboratories data */
public class GUILaboratoryData {
private jFrame window;
private jButton insert;
...
public GUILaboratoryData(){
window = ne JFrame();
insert = new JButton();
...
}
...
}
Use Case Insert Laboratory Data
Description The user inserts the data of a
specific laboratory
Events 1. The user opens the Laboratory
GUI
2. The user inserts the
laboratory data
.
.
.
GUILaboratoryData.java
Structural dependecy
Structural Information and Traceability
public class Laboratory{
private String name;
private String position;
...
public void setName(String pName){
this.name=pName;
}
...
}
Laboratory.java
linked
linked
Transitivity
Open Issues
1) The choice of the bonus value is cricual • Different systems require different bonus
• Different IR methods require different bonus
Open Issues
1) The choice of the bonus value is cricual • Different systems require different bonus
• Different IR methods require different bonus
2) When applying the bonus? . The transitivity property does not always hold
Example
/* *This class implements the GUI for
managing laboratories data */
public class GUIDoctorData {
private jFrame window;
private jButton insert;
...
public GUIDoctorData(){
window = ne JFrame();
insert = new JButton();
...
}
...
}
Use Case Insert Laboratory Data
Description The user inserts the data of a
specific laboratory
Events 1. The user opens the Laboratory
GUI
2. The user inserts the
laboratory data
.
.
.
GUIDoctorData.java
Not
linked
Example
/* *This class implements the GUI for
managing laboratories data */
public class GUILaboratoryData {
private jFrame window;
private jButton insert;
...
public GUILaboratoryData(){
window = ne JFrame();
insert = new JButton();
...
}
...
}
Use Case Insert Laboratory Data
Description The user inserts the data of a
specific laboratory
Events 1. The user opens the Laboratory
GUI
2. The user inserts the
laboratory data
.
.
.
GUIDoctorData.java
Not
linked
Example
/* *This class implements the GUI for
managing laboratories data */
public class GUILaboratoryData {
private jFrame window;
private jButton insert;
...
public GUILaboratoryData(){
window = ne JFrame();
insert = new JButton();
...
}
...
}
Use Case Insert Laboratory Data
Description The user inserts the data of a
specific laboratory
Events 1. The user opens the Laboratory
GUI
2. The user inserts the
laboratory data
.
.
.
GUIDoctorData.java
public class Authorization{
public void setAuthorization(Doctor
pDoctor, Laboratory pLab){
...
}
...
}
Autorization.java
Not linked
Not
linked
Example
/* *This class implements the GUI for
managing laboratories data */
public class GUILaboratoryData {
private jFrame window;
private jButton insert;
...
public GUILaboratoryData(){
window = ne JFrame();
insert = new JButton();
...
}
...
}
Use Case Insert Laboratory Data
Description The user inserts the data of a
specific laboratory
Events 1. The user opens the Laboratory
GUI
2. The user inserts the
laboratory data
.
.
.
GUILaboratoryData.java
Structural dependecy
public class LaboratoryAuthorization{
public void setAuthorization(Doctor
pDoctor){
...
}
...
}
LaboratoryAuthorization.java
Not linked
Not
linked
Example
/* *This class implements the GUI for
managing laboratories data */
public class GUILaboratoryData {
private jFrame window;
private jButton insert;
...
public GUILaboratoryData(){
window = ne JFrame();
insert = new JButton();
...
}
...
}
Use Case Insert Laboratory Data
Description The user inserts the data of a
specific laboratory
Events 1. The user opens the Laboratory
GUI
2. The user inserts the
laboratory data
.
.
.
GUILaboratoryData.java
Structural dependecy
public class LaboratoryAuthorization{
public void setAuthorization(Doctor
pDoctor){
...
}
...
}
LaboratoryAuthorization.java
Not linked
Not
linked
Example
/* *This class implements the GUI for
managing laboratories data */
public class GUILaboratoryData {
private jFrame window;
private jButton insert;
...
public GUILaboratoryData(){
window = ne JFrame();
insert = new JButton();
...
}
...
}
Use Case Insert Laboratory Data
Description The user inserts the data of a
specific laboratory
Events 1. The user opens the Laboratory
GUI
2. The user inserts the
laboratory data
.
.
.
GUILaboratoryData.java
Structural dependecy
public class LaboratoryAuthorization{
public void setAuthorization(Doctor
pDoctor){
...
}
...
}
Laboratory.java
Not linked
linked
Structural information is not
always useful
Approach 1: Optimistic Combination (O-CSTI)
Source_1 Class_2 95.4%
Source_3 Class_4 92.1%
Source_1 Class_1 85.6%
Source_2 Class_2 83.2%
Source_3 Class_3 81.2%
Source_1 Class_3 79.0%
Source_3 Class_2 77.5%
Source_2 Class_4 64.3%
Source_2 Class_3 53.2%
Code
Classes
IR Engine
Source
Artifacts
Candidate Links
Step 1: traditional IR process
Approach 1: Optimistic Combination (O-CSTI)
Source_1 Class_2 95.4%
Source_3 Class_4 92.1%
Source_1 Class_1 85.6%
Source_2 Class_2 83.2%
Source_3 Class_3 81.2%
Source_1 Class_3 79.0%
Source_3 Class_2 77.5%
Source_2 Class_4 64.3%
Source_3 Class_4 53.2%
Candidate Links
Class_2
Class_1
Class_3
Class_4
+ Bonus
+ Bonus
Step 2: applying bonus to all the links
Approach 1: Optimistic Combination (O-CSTI)
Source_1 Class_2 95.4%
Source_3 Class_4 92.1%
Source_1 Class_1 85.6%
Source_2 Class_2 83.2%
Source_3 Class_3 81.2%
Source_1 Class_3 79.0%
Source_3 Class_2 77.5%
Source_2 Class_4 64.3%
Source_3 Class_4 53.2%
Candidate Links
Step 2: applying bonus to all the links
Class_2
Class_1
Class_3
Class_4
+ Bonus
+ Bonus
+ Bonus
Approach 1: Optimistic Combination (O-CSTI)
Source_1 Class_2 95.4%
Source_3 Class_4 92.1%
Source_1 Class_1 85.6%
Source_2 Class_2 83.2%
Source_3 Class_3 81.2%
Source_1 Class_3 79.0%
Source_3 Class_2 77.5%
Source_2 Class_4 64.3%
Source_3 Class_4 53.2%
Candidate Links
Step 2: applying bonus to all the links
Class_2
Class_1
Class_3
Class_4
+ Bonus
+ 2Bonus
+ Bonus
+ Bonus
Approach 1: Optimistic Combination (O-CSTI)
End?
YES
NO
Apply the bonus to all the links
Tradidional IR process
The user judges the i-th link
Re-order the list
Approach 2: User Driven Combination (U-CSTI)
Approach 2: User Driven Combination (U-CSTI)
Approach 2: User Driven Combination (U-CSTI)
Tradidional IR process
The user judges the i-th link
if correct
Apply bonus No bonus
Re-order the list
End?
YES NO
YES
NO
Adaptive bonus
Sim(Source1, Class1) = Sim(Source1, Class1) + δ * Sim(Source1, Class1)
Adaptive bonus
Sim(Source1, Class1) = Sim(Source1, Class1) + δ * Sim(Source1, Class1)
yvariabilitlistranked2
)min()max(
SimSim
Source_1 Class_2 95.4%
Source_3 Class_4 92.1%
Source_1 Class_1 85.6%
Source_2 Class_2 83.2%
Source_3 Class_3 81.2%
Source_1 Class_3 79.0%
Source_3 Class_2 77.5%
Source_2 Class_4 64.3%
Source_2 Class_3 53.2%
Candidate Links
max(Sim)
min(Sim)
vari
abili
ty
Implementation
We provide the experiments and datasets for download at
http://www.cs.wm.edu/semeru/data/csmr13/
TraceLab Components
1) Adaptive Bonus
2) Optimistic Comibination
(O-CSTI)
3) User Driven Combination
(U-CSTI)
4) Different IR Methods:
- Vector Space Model
- Jensen-Shannon
Implementation
We provide the experiments and datasets for download at
http://www.cs.wm.edu/semeru/data/csmr13/
TraceLab Components
1) Adaptive Bonus
2) Optimistic Comibination
(O-CSTI)
3) User Driven Combination
(U-CSTI)
4) Different IR Methods:
- Vector Space Model
- Jensen-Shannon
IR
method
Empirical Evaluation
Context
We compared three IR-based processes:
1) IR process alone
2) O-CSTI (optimistic combination)
3) U-CSTI (user driven combination)
Results
Vector Space Model
Results
Jensen-Shannon Divergence
Results
Tracing Use Cases onto Code Classes on SMOS
Vector Space Model
Results
Tracing Use Cases onto Code Classes on SMOS
Vector Space Model Jensen-Shannon Divergence
Optimality of the Adaptive Bonus
Adaptive Bonus vs Fixed Bonus
- We used different fixed bonus values
Optimality of the Adaptive Bonus
EasyClinic UC-CC with VSM EasyClinic TC-CC with JS
Adaptive Bonus vs Fixed Bonus
- We used different fixed bonus values
Conclusions
![CSMR 2012csmr2012.sed.hu/sites/csmr2012.sed.hu/files/CSMR2012-Program-w… · CSMR 2012 – Szeged, Hungary [1] Welcome from the Conference Chairs The Conference on Software Maintenance](https://static.fdocuments.us/doc/165x107/5fb4d8dc78cbd370594b3e71/csmr-csmr-2012-a-szeged-hungary-1-welcome-from-the-conference-chairs-the-conference.jpg)
