130404 fehmi jaafar - on the relationship between program evolution and fault-proneness -- an...

17
Fehmi Fehmi Fehmi Fehmi Jaafar Jaafar Jaafar Jaafar , Yann , Yann , Yann , Yann- - -Gaël Gaël Gaël Gaël Guéhéneuc Guéhéneuc Guéhéneuc Guéhéneuc, , , S S Sy y yl l lv v vi i ie e e H H Ha a am m me e el l l, , , a a an n nd d d Bram Adams Bram Adams Bram Adams Bram Adams Université de Montréal École Polytechnique de Montréal Quebec Canada 1

description

Macro-co-changes, bugs, software evolution, patterns

Transcript of 130404 fehmi jaafar - on the relationship between program evolution and fault-proneness -- an...

Page 1: 130404   fehmi jaafar - on the relationship between program evolution and fault-proneness -- an empirical study

FehmiFehmiFehmiFehmi JaafarJaafarJaafarJaafar, Yann, Yann, Yann, Yann----Gaël Gaël Gaël Gaël GuéhéneucGuéhéneucGuéhéneucGuéhéneuc,,,, SSSSyyyyllllvvvviiiieeee HHHHaaaammmmeeeellll,,,, aaaannnndddd Bram AdamsBram AdamsBram AdamsBram Adams

Université de MontréalÉcole Polytechnique de Montréal

Quebec Canada

1

Page 2: 130404   fehmi jaafar - on the relationship between program evolution and fault-proneness -- an empirical study

1.1.1.1. IntroductionIntroductionIntroductionIntroduction

2.2.2.2. RelatedRelatedRelatedRelated WorkWorkWorkWork

3.3.3.3. ProblemProblemProblemProblem StatementStatementStatementStatement

4.4.4.4. EmpiricalEmpiricalEmpiricalEmpirical StudyStudyStudyStudy4.4.4.4. EmpiricalEmpiricalEmpiricalEmpirical StudyStudyStudyStudy

5.5.5.5. ResearchResearchResearchResearch QuestionsQuestionsQuestionsQuestions

6.6.6.6. ResultsResultsResultsResults

7.7.7.7. OngoingOngoingOngoingOngoing WorkWorkWorkWork

8.8.8.8. ConclusionConclusionConclusionConclusion

2

Page 3: 130404   fehmi jaafar - on the relationship between program evolution and fault-proneness -- an empirical study

Programs evolve continuously, requiring constant maintenance and development.

Features are added and faults are fixed. Programs become more complex over

time and thus, harder to maintain.

This decay could be detected by measuring the instability of the program, a poor

code quality and a high fault rates..

3

Page 4: 130404   fehmi jaafar - on the relationship between program evolution and fault-proneness -- an empirical study

Ostrand et al. [3] found that 20% of classes contains 80% of faults. At the same

Change-Log Approaches[1] use process metrics extracted from the versioning

system, assuming that recently or frequently changed classes are the most

probable source of faults.

Code-Metrics approaches[2] use source code metrics, assuming that complex

or larger classes are more fault-prone.

Ostrand et al. [3] found that 20% of classes contains 80% of faults. At the same

time, these 20% of classes accounted for 50% of the source code.

Assuming that all classes are considered to have the same likelihood for fault-

proneness is not realistic.

Not all classes are there to last forever, some are meant for experimentation, so

it could be expected that they have more faults.

4

[1] A. E. Hassan, “Predicting faults using the complexity of code changes,” in Proceedings of the 31st International Conference on Software Engineering, 2009.[2] R. Moser, W. Pedrycz, and G. Succi, “A comparative analysis of the efficiency of change metrics and static code attributes for defect prediction,” in Proceedings of the 30th international conference on Software engineering, 2008[3] T. Ostrand, E. Weyuker, and R. Bell, “Predicting the location and number of faults in large software systems,” Software Engineering, IEEE Transactions, 2005.

Page 5: 130404   fehmi jaafar - on the relationship between program evolution and fault-proneness -- an empirical study

Most previous fault prediction approaches were proposed to analyse fault-

proneness using complexity code metrics and-or change metrics.

Classes that exhibit similar evolution profiles, are considered as co-evolved

classes, may have interdependencies among them.

However, it is not clear how classes with similare evolution behavior are linked

with faults. Indeed, evolution studies out there did not link different evolution with faults. Indeed, evolution studies out there did not link different evolution

behavior to faults.

How we can relate the evolution of classes in object-oriented programs with

fault-proneness.

5

Page 6: 130404   fehmi jaafar - on the relationship between program evolution and fault-proneness -- an empirical study

In JFreeChart, we find that ChartPanel.java and CombinedDomainXYPlot.java were

introduced, changed and renamed in the same versions but in different periods and by

different developers.different developers.

The bugID195003710 reported “ a bug either in ChartPanel or CombinedDomainXYPlot

when trying to zoom in/out on the range axis”.

6

Page 7: 130404   fehmi jaafar - on the relationship between program evolution and fault-proneness -- an empirical study

We classify classes according to their class-profiles. We report three types of class

evolution:

Short-lived classes: They have a very short lifetime, i.e., they exist only during one

version of the program.

Persistent classes: They never disappear after their first introduction into the

program.

7

program.

Transient classes: They appear and disappear many times during the program

lifetime.

Co-evolved classes: They have the same evolution profile and are related by static

relationships.

Page 8: 130404   fehmi jaafar - on the relationship between program evolution and fault-proneness -- an empirical study

Given several versions of a program, we extracted their class diagrams using

an existing tool PADL [3].

8

[3] Y.-G. Guéhéneuc and G. Antoniol, “DeMIMA: A multilayered framework for design pattern identification,” Transactions on Software Engineering (TSE), vol. 34, no. 5, pp. 667–684, 2008.[4] S. Hassaine, Yann-Ga¨el, S. Hamel, and A. Giuliano, “Advise: Architectural decay in software evolution,” in Proc. 16th European Conference on Software Maintenance and Reengineering, 2012.[5] F. Jaafar, Y. Guéhéneuc, S. Hamel, and G. Antoniol, “An exploratory study of macro co-changes,” in Working Conference on Reverse Engineering (WCRE). IEEE, 2011,

We identified class renamings, class changes, and fault fixing using two

previous approaches: ADvISE [4] and Macocha [5].

We created the set of class-profiles that describes the evolution of each class in

the program.

Page 9: 130404   fehmi jaafar - on the relationship between program evolution and fault-proneness -- an empirical study

9

Page 10: 130404   fehmi jaafar - on the relationship between program evolution and fault-proneness -- an empirical study

RQ1: What is the relation between class lifetime and fault-proneness?We group classes according to their profiles through the program lifespan, by

considering the renaming, refactoring, and structural changes of classes, to determine

how class lifetime are related to fault-proneness.

10

RQ2 :What is the relation between class co-evolution and fault-proneness?We check if the proportion of faults fixed by maintaining co-evolved classes are

significantly more than faults fixed using not co-evolved classes.

Page 11: 130404   fehmi jaafar - on the relationship between program evolution and fault-proneness -- an empirical study

HRQ10: There is no statistically significant difference between proportions of

faults carried by Persistent, Shortlived, and Transient classes in ArgoUML,

We use Fisher’s exact test and the Chi-Square test

to check two hypothesis. We also compute the odds

ratio [20] that indicates the likelihood for an event to

occur.

11

HRQ20: There is no statistically significant difference between proportions of

faults involving co-evolved classes or not co-evolved classes in the three

programs.

faults carried by Persistent, Shortlived, and Transient classes in ArgoUML,

JFreeChart, and XercesJ.

Page 12: 130404   fehmi jaafar - on the relationship between program evolution and fault-proneness -- an empirical study

12

Page 13: 130404   fehmi jaafar - on the relationship between program evolution and fault-proneness -- an empirical study

13

Page 14: 130404   fehmi jaafar - on the relationship between program evolution and fault-proneness -- an empirical study

14

Page 15: 130404   fehmi jaafar - on the relationship between program evolution and fault-proneness -- an empirical study

We found that Persistent classes are significantly less fault-prone than Short-

lived and Transient classes.

Faults fixed by maintaining co-evolved classes are significantly more than

faults fixed using not co-evolved classes.

Special attention must be given to these entities to keep the design intact

during program evolution because they could have a negative impact on the

fault-proneness of the program.

15

Page 16: 130404   fehmi jaafar - on the relationship between program evolution and fault-proneness -- an empirical study

Relating class lifetime and change-proneness.

16

Using these results to improve faults detection tools.

Prevent change in co-evolved classes.

Identifying the lifetime followed by classes belonged to design motifs such as design

patterns and anti-patterns.

Page 17: 130404   fehmi jaafar - on the relationship between program evolution and fault-proneness -- an empirical study

We provide empirical evidence of the relationships between class

evolution and fault proneness.

We showed that Persistent classes are significantly less fault-prone than

other classes and that faults fixed by maintaining co-evolved classes are

significantly more than faults fixed using not co-evolved classes.

17

We provide a basis for future research to understand the causes and the

eventual consequences of these findings.