Software Metrics

Objectives

• Understand why measurement is important

• Understand the basic metrics approaches used in industry

• Know how and how to extract and apply relevant metrics

Software Metrics Motivation

• Quantitative tool to manage risk in software projects

• Software metrics has always been driven by two key objectives:

– software cost/resource estimation

– software quality control and estimation

• Measuring ‘size’ and measuring ‘defects’ is central

IBM Space Shuttle Software Metrics Program (1)

Software Release Number

Defects perKLOC

Pre-release

Post-release


95% high

expected

95% low

Actual

Predicted total error rate trend (errors per KLOC)

Onboard flight software releases

0

2

4

6

8

10

12

14

1 3 5 7 8A 8C 8F


0

5

10

15

20

25

8B 8C 8D 20

Onboard flight software failuresoccurring per base system

Basic operational increment

How Software Metrics have evolved

A Brief History of Software Metrics

• First book (Gilb 1976) but LOC-based measures since 1960’s

• LOC used as surrogate for different notions of software size

• Drawbacks of LOC as size measure led to complexity metrics and function point metrics

• Further metrics for design level, and for different language paradigms

The Enduring LOC Measure• LOC: Number of Lines Of Code• The simplest and most widely used measure of

program size. Easy to compute and automate• Used (as normalising measure) for

– effort/cost estimation (Effort = f(LOC))– quality assessment/estimation (defects/LOC))– productivity assessment (LOC/effort)

• Alternative (similar) measures– KLOC: Thousands of Lines Of Code– KDSI: Thousands of Delivered Source Instructions– NCLOC: Non-Comment Lines of Code– Number of Characters or Number of Bytes

LOC metric: how many LOC here?with TEXT_IO; use TEXT_IO;procedure Main is

--This program copies characters from an input--file to an output file. Termination occurs--either when all characters are copied or--when a NULL character is input

Nullchar, Eof: exception ;Char: CHA RACTER;Input_file, Output_file, Console: FILE_TYPE;

Beginloop

Open (FILE => Input_file, MODE => IN_FILE,NAME => “CharsIn”);

Open (FILE => Output_file, MODE =>OUT_FILE,NAME => “CharOut”);

Get (Input_file, Char);if END_OF_FILE (Input_file) then

raise Eof;elseif Char = ASCII.NUL then

raise Nullchar;else

Put(Output_file, Char);end if;

end loop ; exception

when Eof => Put (Console, “no null characters”);when Nullchar => Put (Console, “null terminator”);

end Main

Problems with LOC type measures

• No standard definition

• Measures length of programs rather than size

• Wrongly used as a surrogate for: – effort

– complexity

– functionality

• Fails to take account of redundancy and reuse

• Cannot be used comparatively for different types of programming languages

• Only available at the end of the development life-cycle

The ‘defect density’ measure: an important health warning

defect density = number of defects found system size (KLOC)

Defect density is used as a de-facto measure of software quality

Simple COCOMO Effort Prediction

effort = a (size)b

effort = person monthssize = KDSI (predicted)

a,b constants depending on type of system:

‘organic’: a = 2.4 b = 1.05‘semi-detached’: a = 3.0 b = 1.12‘embedded’: a = 3.6 b = 1.2

COCOMO Development Time Prediction

time = a (effort)b

effort = person monthstime = development time (months)

a,b constants depending on type of system:

‘organic’: a = 2.5 b = 0.32‘semi-detached’: a = 2.5 b = 0.35‘embedded’: a = 2.5 b = 0.38

Regression Based Cost Modelling

10,000

1000

100

10

log a

1K 10K 100K 1000K 10000K

log E (Effort)

log S(Size)

Slope b

log E = log a + b * log S

E = a * S b

Software size attributes• length the physical size of the product

• functionality measures the functions supplied by the product to the user

• complexity – Problem complexity

– Algorithmic complexity

– Structural complexity

– Cognitive complexity

The search for more discriminating metrics

• Capture cognitive complexity

• Capture structural complexity

• Capture functionality (or functional complexity)

• Language independent

• Can be extracted at early life-cycle phases

Halstead’s Software Science MetricsA program P is a collection of tokens, classified aseither operators or operands.

n1 = number of unique operatorsn2 = number of unique operandsN1 = total occurrences of operatorsN2 = total occurrences of operands

Length of P is N = N1+N2 Vocabulary of P is n = n1+n2

Theory: Estimate of N is N = n1 log n1 + n2 log n2

Theory: Effort required to generate P is

Theory: Time required to program P is T=E/18 seconds

E = n1 N2 N log n2n2

(elementary mental discriminations)

McCabe’s Cyclomatic Complexity Metric v

If G is the control flowgraph of program Pand G has e edges (arcs) and n nodes

v(P) = e-n+2

v(P) is the number of linearly independent paths in G

here e = 16 n =13 v(P) = 5

McCabe proposed: v(P)<10 for each module P

More simply, if d is the number ofdecision nodes in G then

v(P) = d+1

Flowgraph based measures

• Many software measures are based on a flowgraph model of a program

• Most such measures can be automatically computed once the flowgraph ‘decomposition’ is known

• The notion of flowgraph decomposition provides a rigorous, generalised theory of structured programming

• There are tools for computing flowgraph decomposition

Albrecht’s Function PointsCount the number of:

External inputsExternal outputsExternal inquiriesExternal filesInternal files

giving each a ‘weighting factor’

The Unadjusted Function Count (UFC) is the sum ofall these weighted scores

To get the Adjusted Function Count (FP), multiplyby a Technical Complexity Factor (TCF)

FP = UFC x TCF

Function Points: ExampleSpell-Checker Spec: The checker accepts as input a document file and anoptional personal dictionary file. The checker lists all words not containedin either of these files. The user can query the number of words processedand the number of spelling errors found at any stage during processing

A = # external inputs = 2, B =# external outputs = 3, C = # inquiries = 2, D = # external files = 2, E = # internal files = 1

Assuming average complexity in each case

SpellingChecker

User User

errors found enquiry

words processes enquiry

Document file

Personal dictionary

# words processed message

# errors message

report on misspelt wordswords

UFC = 4A + 5B + 4C +10D + 7E = 58

Dictionary

Function Points: Applications

• Used extensively as a ‘size’ measure in preference to LOC

• Examples:

Productivity

Quality

Effort prediction

FPPerson months effort

DefectsFP

E=f(FP)

Function Points and Program Size

AssemblerCAlgolCOBOLFORTRANPascalRPGPL/1MODULA-2PROLOGLISPBASIC4 GL DatabaseAPLSMALLTALKQuery languagesSpreadsheet languages

3201501061061069180807164646440322116

6

Language Source Statements per FP

Current State-of-the-Art

• Company-wide measurement programmes

• Benchmarking

• SEI’s CMM process assessment

• Metrics tools as standard in IDEs

• Measurement theory as a unifying framework

• International software measurement standards

ISO 9126 Software Product Evaluation Standard

Quality characteristics and guidelines for their use

Chosen characteristics are:

• Functionality

• Reliability

• Usability

• Efficiency

• Maintainability

• Portability

Software Metrics:From Goals to Actions

Goals

Measures

Data

Facts/trends

Decisions

Actions

Goal Question Metric (GQM)

• There should be a clearly-defined need for every measurement.

• From the goals, generate questions whose answers will tell you if the goals are met.

• From the questions, suggest measurements that can help to answer the questions.

The Metrics Plan (for each goal)

• WHYWHY metrics can address the goal• WHAT WHAT metrics will be collected, how they will be

defined, and how they will be analyzed• WHO WHO will do the collecting, who will do the analyzing,

and who will see the results• HOWHOW it will be done - what tools, techniques and

practices will be used to support metrics collection and analysis

• WHEN WHEN in the process and how often the metrics will be collected and analyzed

• WHERE WHERE the data will be stored

Summary

• ‘Software metrics’ driven by two objectives:– cost/effort estimation

– quality assessment and prediction

• All common metrics traceable to above objectives

• It is easy to extract code metrics, but they need to fit a pre-defined plan

Software Metrics

Documents

Transcript of Software Metrics