Analysis of CK Metrics

“Empirical Analysis of Object-OrientedDesign Metrics for PredictingHigh and Low Severity Faults”Yuming Zhou and Hareton Leung, Member, IEEE Computer Society

IEEE TRANSACTIONS ON SOFTWARE ENGINEERING, VOL. 32, NO. 10, OCTOBER 2006

Renaat Verbruggen

Contribution of Paper 1

This paper makes a number of contributions.

First, using a publicly available data set(NASA) , they present new evidence indicating an association between OO design metrics and fault-proneness, thereby providing valuable data in an important area for which limited experimental data is available.

Renaat Verbruggen

Contribution of Paper 2

Secondly, they validate the association

between OO design metrics and fault-proneness of classes across fault severity. To the best of their knowledge, in spite of its importance, there has been no such previous research.

Renaat Verbruggen

Contribution of Paper 3

Thirdly, on the methodological front, the fault-

proneness of classes is analysed using not just the familiar method of logistical regression but also applied machine learning techniques.

Renaat Verbruggen

Main findings

This study makes two main findings. First, most of the OO design metrics in this study are

statistically related to the fault-proneness of classes across fault severity.

Second, the predictive abilities of these metrics strongly

depend on the severity of faults. More specifically, thedesign metrics are better predictors of low severity

faults inclasses than of high severity faults.

Renaat Verbruggen

Renaat Verbruggen

Descriptive Statistics of the Classes

Renaat Verbruggen

Validation Results of the Hypotheses

Renaat Verbruggen

Conclusions 1

The CBO, WMC, RFC, and LCOM metrics arestatistically significant across fault severity, whileDIT is not significant for any fault severity. NOC isstatistically significant with regard to ungraded/lowseverity faults, but in an inverse direction, i.e., a classwith a large NOC means a low fault-proneness. Thesignificance of NOC with regard to high severityfaults cannot be tested because of the quasi-completeseparation problem. However, the results of machinelearning methods indicate that the faultpronenessprediction usefulness of NOC with regardto high severity faults is poor and, in this, is similarto that of DIT.

Renaat Verbruggen

Conclusions 2

The fault-proneness prediction capabilities of thesemetrics differ greatly depending on the fault severityused. When applied to the classification of classes asfault-prone and not fault-prone in terms of ungraded/low severity faults, the logistic regressionand machine learning models based on these metricsachieve a performance comparable to previousstudies. When applied to fault-proneness rankingof classes in terms of low severity faults, the logisticregression model based on these metrics outperformsthe simple model based on class size. In bothcases, however, the prediction usefulness of thesemetrics with regard to high severity faults is limited.