Metrics

3 flavors3 flavors

Software metrics Agile Project metrics Organizational metrics

before we get to them….before we get to them….

a warning (or three)a warning (or three)

how many of you have an in car how many of you have an in car GPS system?GPS system?

metrics satisfy our metrics satisfy our inquisitive natureinquisitive nature

and can easily become a and can easily become a distractiondistraction

steering with numbers is the wrong steering with numbers is the wrong metaphormetaphor

metrics are your guide and can help you metrics are your guide and can help you make decisionsmake decisions

make sure your metrics support the decisions make sure your metrics support the decisions you need to make…you need to make…

1. Construct metrics for root cause 1. Construct metrics for root cause decision making…decision making…

Or use GQMOr use GQM

GOAL QUESTION METRICGOAL QUESTION METRIC

WERNER HEISENBERG (1901 - 1976)

‘‘The more precisely the The more precisely the POSITIONPOSITION is is determined, the less precisely the determined, the less precisely the MOMENTUM MOMENTUM is known’is known’

detrimentally affecting software detrimentally affecting software project teams ability to adapt to project teams ability to adapt to circumstance is a key mistake.circumstance is a key mistake.

every project team should be every project team should be ruthless in its adaption to situational ruthless in its adaption to situational circumstance.circumstance.

2. Don’t restrict a teams ability to 2. Don’t restrict a teams ability to maximize their effectiveness maximize their effectiveness through over zealous or bureaucratic through over zealous or bureaucratic reporting practices…reporting practices…

In order for metrics to be useful they In order for metrics to be useful they have to be accepted.have to be accepted.

3. don’t build 3. don’t build metrics that metrics that are too are too complicated complicated to to understand.understand.

So….So….

back to metrics…back to metrics…

3 flavors3 flavors

Software metrics Agile Project metrics Organizational metrics

The primary metric for agile The primary metric for agile software development is software development is whether software exists and whether software exists and is demonstrably suitable for is demonstrably suitable for use in its intended purpose.use in its intended purpose.

This key metric is tested by This key metric is tested by demonstration at the end of demonstration at the end of every iteration and release.every iteration and release.

Individual metricsIndividual metrics

Defects Complexity Conformance Coverage Object Orientation Volatility

Much more useful when correlated…Much more useful when correlated…

Goal: I’d like to reduce defects.Goal: I’d like to reduce defects.

Question: Where do defects come Question: Where do defects come from and why?from and why?

Panopticode Panopticode shows the hot spots shows the hot spots in code where there are in code where there are intersections between low test intersections between low test coverage, high volatility and high coverage, high volatility and high complexity.complexity.

In the end code metrics can help In the end code metrics can help remove uncertainty from remove uncertainty from delivery and build a system with delivery and build a system with a longer life span, lower cost of a longer life span, lower cost of change.change.

Metrics smorgasboardMetrics smorgasboard Velocity: Number of story points completed in any given iteration.

Number of story points started in a given iteration.Total Number of Story points remaining in the project. Total Number of 'new' story points in project.

Number of stories added to the project in a given iteration.Number of story points added to the project in a given iteration.

Total number of stories in project. Total number of story points in project.

Resources:

Planned man days available per iteration.Actual man days used per iteration.

Total man days accrued on project to date.Total man days remaining on budget.

Total Man days lost to non project work.

Financials:

Cost per iteration.Cost per release.

Total days worked on Project.Total days worked by resource type.Monthly spend.Total Monthly spend.

Total Cost to dateTotal Cost to complete.Estimated Cost at complete.

% Unit Test coverage.

Acceptance TestsTotal number of Acceptance tests per iteration.    Automated    Non Automated.    

Total Acceptance tests passing per iteration. Total acceptance tests failing per iteration.Total acceptance tests ignored per iteration.

Total automated Acceptance tests in project.Total non automated Acceptance tests in project.% Automated.

Defects (Bugs identified post development).Total number of defects per iteration.Total number of defects by severity.

Quality.

Unit TestsTotal number of unit tests per iteration.Total Unit tests passing per iteration.Total Unit tests failing per iteration. Total Unit tests ignored per iteration.

Uh oh.Uh oh.

We just broke golden rule #1.We just broke golden rule #1.

So, lets figure out what we need So, lets figure out what we need first…first…

Goal, Question, Metric.Goal, Question, Metric.

here are some common Goals, here are some common Goals, questions and examples of how questions and examples of how metrics support them…metrics support them…

Goal: I want predictability.Goal: I want predictability.

Q: Is my project team converging on Q: Is my project team converging on a solution that meets the outcomes a solution that meets the outcomes for my business?for my business?

Iteration

Total Effort remaining at start.

Work completed this iteration

Work Added this iteration.

Total effort remaining End Date.

Estimated Cost.

Actual Cost.

Accrued Costs.

Iteration 7-8 49 10 +0 39 01 Sept 34, 650 34,650 34,650

Iteration 9 39 12 +0 27 15 Sept 30,80030,800

65, 450

Iteration 10 27 10 +5 22 29 Sept 31,350

31,35096,800

Iteration 11 22 11 +0 11 13 Oct 31,900

31,900128,700

Iteration 12 11 11 +0 0 21 Oct 22,000

22,000150,700

100% of Outcomes met

Goal: I want to understand the Goal: I want to understand the capacity of the team and how it capacity of the team and how it changes.changes.

Q: Will defects affect the Q: Will defects affect the predictability of delivery?predictability of delivery?

Hypothesis: we can use yesterdays Hypothesis: we can use yesterdays weather to predict how much we can weather to predict how much we can get done next time.get done next time.

Velocity is simply the number of Velocity is simply the number of Story points delivered last time used Story points delivered last time used as a predictor for next time…as a predictor for next time…

Goal: I want to reduce waste in my Goal: I want to reduce waste in my process and deliver value to the process and deliver value to the customer as often as possible.customer as often as possible.

Q: Where are the bottle necks in my Q: Where are the bottle necks in my development process?development process?

Goal: I don’t want any nasty Goal: I don’t want any nasty surprises late in the project that will surprises late in the project that will affect the predictability of delivery.affect the predictability of delivery.

Q: Will defects affect the Q: Will defects affect the predictability of delivery?predictability of delivery?

0

10

20

30

40

50

60

70

80

90

Mon Tue Wed Thur Fri Mon Tue Wed Thur Fri Mon Tue Wed Thur Fri

Date

# o

f w

ork

ing

Acc

Tes

ts

Not workingWorking

At this point things can become less At this point things can become less quantitative and more qualitative…quantitative and more qualitative…

Goal: I want to build capabilities that Goal: I want to build capabilities that allow me to repeatably deliver high allow me to repeatably deliver high value products/services.value products/services.

Q: Are my teams developing Q: Are my teams developing repeatable capabilities?repeatable capabilities?

Pairing

TDD

ATDP

Cont Integration

Stand up’s

Retrospectives

Design Storming

Re-factoring

Goal: I’m a low risk business, if one Goal: I’m a low risk business, if one project fails this year I don’t want it project fails this year I don’t want it to sink the organization!to sink the organization!

Q: What is the risk profile of my Q: What is the risk profile of my project portfolio?project portfolio?

Potential Value

Risk

Cost

Goal: I want to maximize the value Goal: I want to maximize the value from my workforce ensuring smooth from my workforce ensuring smooth transition between projects….transition between projects….

Q: When will I need capacity?Q: When will I need capacity?

Be careful with this one…Be careful with this one…

Don’t break golden Don’t break golden rule number 2….rule number 2….

Goal: To understand if our strategy Goal: To understand if our strategy in the market place is working so in the market place is working so that we can adapt quickly. that we can adapt quickly.

Q: What do my customers think ?Q: What do my customers think ?

Fred ReichheldFred ReichheldLoyalty expertLoyalty expert

‘The ultimate ‘The ultimate question’question’(otherwise known (otherwise known as NPS)as NPS)

NNet et PPromoter romoter SScore: would you core: would you recommend us to a friend or recommend us to a friend or Colleague?Colleague?

P – D = NPSP – D = NPS

The difference between the The difference between the number of Promoters and number number of Promoters and number of Detractors = Net Promoter Score of Detractors = Net Promoter Score (NPS)(NPS)

Goal: To leverage our strategy to Goal: To leverage our strategy to make our enterprise successful.make our enterprise successful.

Q: Do our projects align with the Q: Do our projects align with the vision of our organization?vision of our organization?

A Starbucks on every corner

Inspire customers to enjoy the daily pleasure of Peet's Coffees & Teas by providing a distinctive, superior product, superior coffee and tea knowledge and superior service to every customer, every day.

Typical Key Performance indicators:

Financial Cash FlowROIFinancial ResultReturn on capital employedReturn on equity

Customer Delivery Performance to Customer - by DateDelivery Performance to Customer - by Quantity

Internal Business Processes Number of ActivitiesOpportunity Success Rate

Learning & Growth Investment RateIllness rate

The balanced score card.The balanced score card.

keep metrics simplekeep metrics simple

Figure out why you want to measure Figure out why you want to measure something before you figure out something before you figure out how.how.

GOAL > Question > MetricGOAL > Question > Metric

Everywhere is different . Build Everywhere is different . Build metrics that make sense for YOU.metrics that make sense for YOU.

Don’t restrict team process Don’t restrict team process adaptations, adapt reporting as adaptations, adapt reporting as requiredrequired