Authors:Gabriel de Souza P. MoreiraRoberto Pepato MelladoDenis Ávila MontiniProf. Dr. Luiz Alberto Vieira DiasProf. Dr. Adilson Marques da Cunha

Abstract Product Software Metrics Software Quality Engineering Standards for SW Products Measurement and Analysis - CMMI Goal-Driven Measurement Process GQ(I)M Selected Metrics Continuous Integration Automatic Metrics Extraction Process Software Metrics DW Model Metrics Analysis Conclusions

The paper describes a framework for a software internal quality measurement program with automatic metrics extraction. This framework was successfully implemented in an Industrial Software Factory. That was possible through the implementation of a proposed Continuous Integration (CI) environment to periodically analyze source codes and extract metrics. These metrics were consolidated in a Data Warehouse, allowing On-line Analytical Processing (OLAP) and Key Performance Indicator (KPI) analysis with high-performance and user-friendly interface. The measurement program followed GQ(I)M paradigm for metrics selection to ensure that collected metrics are relevant from the Software Factory goals perspective. Finally, the Measurement and Analysis Process Area of the Capability Maturity Model integration - CMMi was used for measurement and analysis planning and implementation.

Measurements can be used by software engineers to help assessing the quality of technical work products and to assist in tactical decisions during a project [1].

To accomplish real-time quality assessment, engineers must use technical measures to evaluate quality in objective rather than in subjective ways. [1]

First generation: ISO/IEC 9126 [9]. Second generation: ISO/IEC 25000 (SQuaRE) [7]. Both ISO standard families are based on the model

described in the bellow premisses:

Internal metrics can be used as indicators that allow forecasting the system behavior during tests and operation phases and to prevent problems during the development process [9].

Measurement and Analysis (MA) is a supporting Process Area (PA) at the Maturity Level 2 of CMMI.

The purpose of MA is to develop and sustain a measurement capability that is used to support management information needs. [2]

The Goal-Driven Measurement Planning Process [3] proposes 10 steps for measurement planning.

1. Identify your business goals.2. Identify what you want to know or learn.3. Identify your subgoals.4. Identify the entities and attributes related to your subgoals.5. Formalize your measurement goals.6. Identify quantifiable questions and related indicators that you

will use to help achieve your measurement goals.7. Identify the data elements that you will collect to construct the

indicators that help answer your questions.8. Define measures to be used, and make these definitions

operational.9. Identify the actions that you will take to implement the

measures.10.Prepare a plan for implementing the measures.

GQ

I

M

Goal-Question-(Indicator)-Measure [3] was based on GQM paradigm from Basili [5].

GQM and its GQ(I)M extension are useful because they facilitate to identify not only precise required measures, but also the reasons why the data are being collected.

(*) .NET specific metrics (**) OO specific metrics

NbLinesOfCode – means the number of logical lines of code. *NbILInstructions – means the number of instructions in Intermediate

Language (IL) of .NET Platform. NbMethods – means the number of methods (abstract, virtual or non-

virtual, constructor, property or indexer) present in a type. NbFields – means the number of fields (field, enum, read only field or

constant) present in a type. Code Source Cyclomatic Complexity (CC) – means the number of

decisions that can be taken in a procedure. The CC for a type is defined as the sum of its methods CC.

*IL Cyclomatic Complexity (ILCC) – means the .NET Language independent cyclomatic complexity measure. It is computed from IL as 1 + the number of different offsets targeted by a jump/branch IL instruction.

TypeRank –TypeRank values are computed by applying the Google’s PageRank algorithm on the graph of type dependencies. A homothety of center 0.15 is applied to make it so that the average of TypeRank is 1.

Lack Of Cohesion Of Methods (LCOM) –Single Responsibility Principle states that a type should not have more than one reason to change. Such a class is said to be cohesive. A high LCOM value generally pinpoints a poorly cohesive class. There are several LCOM metrics. The LCOM takes its values in the range [0-1]

Lack of Cohesion Of Methods Henderson-Sellers (LCOM HS) –the LCOM of HS (HS stands for Henderson-Sellers) that takes its values in the range [0 to 2]. A LCOM HS value highest than 1 should be considered alarming.

**NbChildren –the number of children for a class is the number of sub-classes (whatever their positions in the sub branch of the inheritance tree). The number of children for an interface is the number of types that implement it.

**Depth of Inheritance Tree (DIT) – means the Depth of Inheritance Tree for a class or a structure as its number of base classes.

NbLinesOfComment – means the sum of the number of lines of comment that can be found in each of its partial definition.

PercentageComment – means this metric is calculated by using the following formula: PercentageComment = 100 * NbLinesOfComment / (NbLinesOfComment + NbLinesOfCode).

Afferent coupling at type level (Ca) – means the Afferent Coupling for a particular type is the number of types that depends directly on it.

Efferent coupling at type level (Ce) – means the Efferent Coupling for a particular type is the number of types it directly depends on.

Association Between Classes (ABC) – means the Association Between Classes metric for a particular class or structure is the number of members of others types it directly uses in the body of its methods

PercentageCoverage – means the percentage of code covered by unit tests.

The CI [4] is a software engineering practice where developers frequently integrate their work in a centralized repository.

As the work is integrated, an automated build is executed to discover integration errors as quick as possible.

Some new approaches can be added to this CI practice, including but not limited to: unit testing, code coverage, metrics extraction, static code analysis.

Cruise Control.Net for CI; Nant for automating build tasks; NUnit for unit testing; PartCover for extracting test code coverage; NDepend for extracting software metrics.

The implemented process for automatic metrics extraction in the case study is presented below.

XML

Source-Code Static Analysis(NDepend in CI)

Source Code

XML Processing

MDBMDB

ETL

MDBDW

OLAP

DW Model implemented in SQL Server Analysis Services 2008

OLAP analysis over a Software Metrics Cube

Example of SW Metrics Key Performance Indicators (KPI)

DIT CC LCM LCM HS ECMeasured 3 87 0.98 1.08 42Maximum (Desired) 5 50 1.00 2.00 50

Unit Tests Coverage in a project time frame

The main contributions of this research was obtained from the planning of a software product metrics program in a Industrial Software Factory and its implementation using a Continuous Integration environment for automatic metrics extraction from source codes.

GQ(I)M allowed to ensure during the planning that all collected metrics have matched measurement objectives.

The practices of Measurement and Analysis (MA) of CMMi level 2, was used to successfully plan the metrics program by considering which kind of analysis could be done after its implementation.

In a case study, software metrics were systematically collected from the source code and consolidated in a DW through the customization of a proposed CI environment strategy. This application has allowed software product analysis by using high-performance architecture, DW, and user-friendly interface with OLAP.

There were some created Indicators like KPI and statistical graphics that have also allowed a higher level of understanding of the project source code status and its evolution. At last, it was found that some KPI could also be used to compare Object-Oriented projects developed in the same platform.

[1] R. S. Pressman, “Software Engineering”, 6ª Edition, McGraw Hill, NY, 2006. [2] CMMI, Version 1.2 - CMMI-DEV, V1.2, CMU/SEI-2006-TR-008 - Improving processes for better

products, SEI - Carnegie Mellon University, Pittsburgh, 2006. [3] R. E. Park; W. B. Goethert; W. A. Florac, CMU/SEI-96-HB-002 - Goal-Driven Software Measurement -

A Guidebook. August 1996. [4] Fowler M. Continuous Integration. Available in

http://martinfowler.com/articles/continuousIntegration.html. Accessed at October 29, 2009. [5] Victor R. Basili, Gianluigi Caldeira, H.Dieter Rombach. The Goal Question Metric Approach. 5th

ACM-IEEE International Symposium on Empirical Software Engineering (ISESE ’06), Rio de Janeiro, 2006.

[6] NDepend Metrics Definition. Available in http://www.ndepend.com/metrics.aspx. Accessed at October 23, 2009.

[7] ISO/IEC 25000, Software Engineering – Software Product Quality Requirements and Evaluation (SQuaRE) - Guide to SQuaRE, 2005.

[8] ISO/IEC 25020, Software and System Engineering - Software Product Quality Requirements and Evaluation (SQuaRE) – Measurement Ref. Model and Guide, 2007.

[9] ISO/IEC 9126-3, Software Engineering – Product Quality - Part 3: Internal Metrics, Geneva, 2003. [10] Suryn, W.; Abran A.; and April A. ISO/IEC SQuaRE: The Second Generation of Standards for

Software Product Quality," 7th IASTED International Conference on Software Engineering and Applications, California, USA, 2003.]