Software metrics, measurement and analytical methods · Software metrics, measurement and...

Software metrics,measurement andanalytical methods

Alicia Marinache

Quality metrics

Objectives of qualitymeasurement

Requirements ofquality metrics

Classification ofsoftware qualitymetrics

Process metrics

Process Qualitymetrics

Process Timetablemetrics

Error removaleffectiveness metrics

Process productivitymetrics

Product metrics

HD quality metrics

HD productivity andeffectiveness metrics

Correctivemaintenance qualitymetrics

Limitations ofSoftware metrics

Summary

1/29

Software metrics, measurement andanalytical methods

SE 3S03 - Tutorial 2

Alicia Marinache

Department of Computer ScienceMcMaster University

Week of Jan 19, 2015

Acknowledgments: The material of these slides is based on [1]

(chapter 21)


Alicia Marinache

Quality metrics




Process metrics





Product metrics

HD quality metrics




Summary

2/29

Outline

Quality metricsObjectives of quality measurementRequirements of quality metricsClassification of software quality metrics

Process metricsProcess Quality metricsProcess Timetable metricsError removal effectiveness metricsProcess productivity metrics

Product metricsHD quality metricsHD productivity and effectiveness metricsCorrective maintenance quality metrics

Limitations of Software metrics


Alicia Marinache

Quality metrics




Process metrics





Product metrics

HD quality metrics




Summary

3/29

Objectives of quality measurement

I You can’t control what you can’t measure- Tom DeMarco

I Quality measure (IEEE, 1991)I A quantitative measure of the degree to wich an item

possesses a quality attributeI A function interpreted as the degree to witch the

software possesses a quality attribute

I Metrics as QA tool


Alicia Marinache

Quality metrics




Process metrics





Product metrics

HD quality metrics




Summary

4/29


I Facilitate management control as well as planning andexecution of managerial interventions

I Deviations of actual functional performance fromplanned performance

I Deviations of actual timetable and budget performancefrom planned performance

I Identify situations that require or enable development ormaintenance process improvement

I Accumulation and analysis of metrics informationregarding the performance of teams, units


Alicia Marinache

Quality metrics




Process metrics





Product metrics

HD quality metrics




Summary

5/29


@ Scott Adams, Dilbert.com


Alicia Marinache

Quality metrics




Process metrics





Product metrics

HD quality metrics




Summary

6/29

Software quality metrics - requirements

I General requirementsI RelevantI ValidI ReliableI ComprehensiveI Mutually exclusive

I Operative requirementsI Easy and simpleI Does not require independent data collectionI Immune to biased interventions


Alicia Marinache

Quality metrics




Process metrics





Product metrics

HD quality metrics




Summary

7/29

Classification of software quality metrics

I By software life cycle phasesI Process metricsI Product metrics

I By what they measureI QualityI TimetableI EffectivenessI Productivity

I ExamplesI KLOC: dependendent on language, tool, programmer

experienceI Function points: measure of resources required, based

on requirements


Alicia Marinache

Quality metrics




Process metrics





Product metrics

HD quality metrics




Summary

8/29

Process Quality metrics

I Error density: based on software volume (KLOC, FPs)and errors counted (number or wighted number)

I Error severity: detect situations when # of severe errorsare increasing, while # of errors are generally decreasing

I NotationsI NCE: # of code errors detected in the software code by

code inspections and testingI NDE: # of development (design and code) errorsI WCE, WDE: weigthed code / development errorsI NFP: # of function points


Alicia Marinache

Quality metrics




Process metrics





Product metrics

HD quality metrics




Summary

9/29

Process Quality metrics - Examples

Error severity class Relative weight

Low severity 1Medium severity 3High severity 9

ESC NCE RW Weighted errorsa b c d = b x c

Low severity 42 1 42Medium severity 17 3 51High severity 11 9 99

Total 70 - 192


Alicia Marinache

Quality metrics




Process metrics





Product metrics

HD quality metrics




Summary

10/29

Process Quality metrics - ExamplesCode Name Formula

CED Code Error Density CED =NCE

KLOC

DED Dev. Error Density DED =NDE

KLOC

WCED Weighted Code Error Density WCED =WCE

KLOC

WDED Weighted Dev. Error Density WDED =WDE

KLOC

WCEF Weighted Code Errors perFunction point

WCEF =WCE

NFP

WDEF Weighted Dev. Errors perFunction point

WDEF =WDE

NFP

ASCE Average Severity of Code Er-rors

ASCE =WCE

NCE

ASDE Average Severity of Dev. Er-rors

ASDE =WDE

NDE


Alicia Marinache

Quality metrics




Process metrics





Product metrics

HD quality metrics




Summary

11/29

Process Quality metrics - Examples

Measures and metrics CED WCED

NCE 70 -WCE - 192KLOC 40 40

CED (NCE/KLOC) 1.75 -WCED (WCE/KLOC) - 4.8


Alicia Marinache

Quality metrics




Process metrics





Product metrics

HD quality metrics




Summary

12/29

Process Timetable metrics

Code Name Calculation formula

TTO Time Table Observance TTO =MSOT

MS

ADMC Average Delay of MilestoneCompletion

ADMC =TCDAM

MS

Notes:

I MSOT: # of milestones completed on time

I MS: # of total milestones

I TCDAM: # Total Completion Delays for All Milestones.Milestones completed on time = ”0” delays.

I TTO and ADMC can be measured throughout thedevelopment process


Alicia Marinache

Quality metrics




Process metrics





Product metrics

HD quality metrics




Summary

13/29

Error removal effectiveness metrics


DERE Development Errors Re-moval Effectiveness

DERE =NDE

NDE + NYF

DWERE Development WeightedErrors Removal Effective-ness

DWERE =WDE

WDE + WYF

Notes:

I NYF: # of software failures detected during 1 year ofmaintenance service

I WYF: wighted # of software failures detected during 1 yearof maintenance service

I These 2 metrics can only be measured after the software isoperational for a period of time


Alicia Marinache

Quality metrics




Process metrics





Product metrics

HD quality metrics




Summary

14/29

Process productivity metricsCode Name Calculation formula

DevP Development Productivity DevP =DevH

KLOC

FDevP Function point DevelopmentProductivity

FDevP =DevH

NFP

CRe Code Reuse CRe =ReKLOC

KLOC

DocRe Documentation Reuse DocRe =ReDoc

NDoc

Notes:

I DevH: total working hours

I ReKLOC: # of thousands of reused lines of code

I ReDoc: # of reused pages of documentation

I NDoc: # of pages of documentation


Alicia Marinache

Quality metrics




Process metrics





Product metrics

HD quality metrics




Summary

15/29

Product metrics

I Help desk services: instructing customers on thesoftware usage; solution of implementation issues.

I HD quality metrics

I HD productivity and effectiveness metrics

I Corrective maintenance services: correction of softwarefailures identified by customers/users

I Corrective maintenance quality metrics

I Corrective maintenance productivity and effectivenessmetrics


Alicia Marinache

Quality metrics




Process metrics





Product metrics

HD quality metrics




Summary

16/29

HD quality metrics

I HD calls density metrics

I HD issues severity metrics

I HD success metrics

Notes:

I NHYC = # of HD calls during a year of service

I KLMC = thousands of lines of maintained software code

I WHYC = weighted HD calls received during one year ofservice

I NMFP = # of function points to be maintained

I NHYOT = number of HD calls per year completed ontime during one year of service


Alicia Marinache

Quality metrics




Process metrics





Product metrics

HD quality metrics




Summary

17/29

HD quality metrics


HDD HD calls Density HDD =NHYC

KLMC

WHDD Weighted HD calls Density WHDD =WHYC

KLMC

WHDF Weighted HD calls per Func-tion Point

WHDF =WHYC

NMFP

ASHC Average Severity of HD Calls ASHC =WHYC

NHYC

HDS HD Service Success HDS =NHYOT

NHYC


Alicia Marinache

Quality metrics




Process metrics





Product metrics

HD quality metrics




Summary

18/29

HD productivity and effectiveness metrics


HDP HD Productivity HDP =HDYH

KLMC

FHDP Function point HD Productiv-ity

FHDP =HDYH

NMFP

HDE HD Effectiveness HDE =HDYH

NHYC

Notes:

I HDYH = total yearly working hours invested in HD servicingof the software system


Alicia Marinache

Quality metrics




Process metrics





Product metrics

HD quality metrics




Summary

19/29

Corrective maintenance quality metrics

I Software system failures density metrics: demand forcorrective maintenance

I Software system failures severity metrics: severity ofsoftware system failures

I Failures of maintenance services metrics: maintenanceservices not on time, or correction failed

I Software system availability metrics: disturbancescaused to customer due to full or partial unavailability

Notes:

I NYF/WYF = / weighted # of software failures detected during a year

of maintenance service

I KLMC = thousands of lines of maintained software code

I NMFP = # of function points designated for the maintained software

I RepYF = # of repeated software failure calls


Alicia Marinache

Quality metrics




Process metrics





Product metrics

HD quality metrics




Summary

20/29

Corrective maintenance quality metrics


SSFD Software System Failure Den-sity

SSFD =NYF

KLMC

WSSFD Weighted Software SystemFailure Density

WSSFD =WYF

KLMC

WSSFF Weighted Software SystemFailure per Function point

WSSFF =WYF

NMFP

ASSSF Average Severity of SoftwareSystem Failures

ASSSF =WYF

NYF

MRepF Maintenance Repeated repairFailure

MRepF =RepYF

NYF


Alicia Marinache

Quality metrics




Process metrics





Product metrics

HD quality metrics




Summary

21/29

Software system availability metrics


FA Full Availability FA =NYSerH − NYFH

NYSerH

VitA Vital Availability VitA =NYSerH − NYVitFH

NYSerH

TUA Total Unavailability TUA =NYTFH

NYSerH

I NYSerH: # of hours software system is in service during one year

I NYFH / NYVitFH: # of hours where at least one (vital) functionfailed during one year

I NYTFH: # of hours of total failure during one year

I

I 1-TUA ≥ VitA ≥ FA


Alicia Marinache

Quality metrics




Process metrics





Product metrics

HD quality metrics




Summary

21/29

Software system availability metrics


FA Full Availability FA =NYSerH − NYFH

NYSerH

VitA Vital Availability VitA =NYSerH − NYVitFH

NYSerH

TUA Total Unavailability TUA =NYTFH

NYSerH

I NYSerH: # of hours software system is in service during one year

I NYFH / NYVitFH: # of hours where at least one (vital) functionfailed during one year

I NYTFH: # of hours of total failure during one year

I NYFH ≥ NYVitFH ≥ NYTFH

I 1-TUA ≥ VitA ≥ FA


Alicia Marinache

Quality metrics




Process metrics





Product metrics

HD quality metrics




Summary

22/29


I Budget constraints

I Human factors

I Data validity

I Metrics: low validity and limited comprehensiveness

I Weak metrics:

I Process:

KLOC, NDE, NCE

I Product: KLMC, NHYC, NYF


Alicia Marinache

Quality metrics




Process metrics





Product metrics

HD quality metrics




Summary

22/29



I Human factors

I Data validity


I Weak metrics:

I Process: KLOC, NDE, NCE

I Product:

KLMC, NHYC, NYF


Alicia Marinache

Quality metrics




Process metrics





Product metrics

HD quality metrics




Summary

22/29



I Human factors

I Data validity


I Weak metrics:

I Process: KLOC, NDE, NCE

I Product: KLMC, NHYC, NYF


Alicia Marinache

Quality metrics




Process metrics





Product metrics

HD quality metrics




Summary

23/29

Process & Product metrics - influence factors

I Process

1. Programming style & language

(KLOC)2. Volume of comments (KLOC)3. Software complexity (KLOC, NCE)4. Percentage of reused code (NDE, NCE)5. Thoroughness of design review and testing teams (NCE)6. Reporting style (NCE, NDE)

I Product

1. Quality of software, doc (NYF, NHYC)2. Programming style, documentation volume (KLMC)3. Software complexity (NYF)4. Percentage of reused code (NYF)5. # of installations, # of users (NHYC, NYF)


Alicia Marinache

Quality metrics




Process metrics





Product metrics

HD quality metrics




Summary

23/29


I Process

1. Programming style & language (KLOC)2. Volume of comments (KLOC)3. Software complexity

(KLOC, NCE)4. Percentage of reused code (NDE, NCE)5. Thoroughness of design review and testing teams (NCE)6. Reporting style (NCE, NDE)

I Product



Alicia Marinache

Quality metrics




Process metrics





Product metrics

HD quality metrics




Summary

23/29


I Process

1. Programming style & language (KLOC)2. Volume of comments (KLOC)3. Software complexity (KLOC, NCE)4. Percentage of reused code

(NDE, NCE)5. Thoroughness of design review and testing teams (NCE)6. Reporting style (NCE, NDE)

I Product



Alicia Marinache

Quality metrics




Process metrics





Product metrics

HD quality metrics




Summary

23/29


I Process

1. Programming style & language (KLOC)2. Volume of comments (KLOC)3. Software complexity (KLOC, NCE)4. Percentage of reused code (NDE, NCE)5. Thoroughness of design review and testing teams

(NCE)6. Reporting style (NCE, NDE)

I Product



Alicia Marinache

Quality metrics




Process metrics





Product metrics

HD quality metrics




Summary

23/29


I Process

1. Programming style & language (KLOC)2. Volume of comments (KLOC)3. Software complexity (KLOC, NCE)4. Percentage of reused code (NDE, NCE)5. Thoroughness of design review and testing teams (NCE)6. Reporting style

(NCE, NDE)

I Product



Alicia Marinache

Quality metrics




Process metrics





Product metrics

HD quality metrics




Summary

23/29


I Process

1. Programming style & language (KLOC)2. Volume of comments (KLOC)3. Software complexity (KLOC, NCE)4. Percentage of reused code (NDE, NCE)5. Thoroughness of design review and testing teams (NCE)6. Reporting style (NCE, NDE)

I Product



Alicia Marinache

Quality metrics




Process metrics





Product metrics

HD quality metrics




Summary

24/29

Example - Function point method

I Measures project size by functionality

I Independent of language / development toolI Three stages

1. Compute crude function points (CFP)2. Compute the relative complexity adjustment factor

(RCAF)3. Compute the number of function points (FP):

FP = CFP x (0.65 + 0.01 x RCAF)

Language Average LOC

C 128

C++ / Java 64

VB 32

Power Builder 16

SQL 12


Alicia Marinache

Quality metrics




Process metrics





Product metrics

HD quality metrics




Summary

25/29

Example - Function point method, Stage 1

Crude function points computation


Alicia Marinache

Quality metrics




Process metrics





Product metrics

HD quality metrics




Summary

26/29

Example - Function point method, Stage 2

Relative complexity adjustment factor computation


Alicia Marinache

Quality metrics




Process metrics





Product metrics

HD quality metrics




Summary

27/29

Function point - advantages and disadvantages

I Advantages

I Estimates can be prepared during the pre-project phaseI High reliability: independent of development tools or

languages

I Disadvantages

I Needs detailed requirements specificationsI Needs an experienced FP teamI Requires lots of evaluationsI It is not comprehensive: most successful in data

processing systems


Alicia Marinache

Quality metrics




Process metrics





Product metrics

HD quality metrics




Summary

28/29

Summary

I Software quality metrics are implemented to supportcontrol of software development projects and to deliverdata to be further analysed by QA teams.

I Metrics ca be calculated from direct measurements ofvarious attributes. Ensure the measurements ad metricsare relevant, valid, reliable, comprehensive, mutuallyexclusive, as well as easy to collect and understand, andobjective.

I There are 2 main ways to measure the size of a softwareproject: KLOC and FPs. Each have advantages anddisadvantages.


Alicia Marinache

Appendix

References

29/29

References I

D. Galin.Software Quality AssuranceFrom theory to implementation.Addison Wesley, 1998.

R. Patton.Software Testing.Sams Publishing, 2006.

Software metrics, measurement and analytical methods · Software metrics, measurement and...

Documents

Transcript of Software metrics, measurement and analytical methods · Software metrics, measurement and...