Post on 24-Jan-2015
description
Metrics
3 flavors3 flavors
Software metrics Agile Project metrics Organizational metrics
before we get to them….before we get to them….
a warning (or three)a warning (or three)
how many of you have an in car how many of you have an in car GPS system?GPS system?
metrics satisfy our metrics satisfy our inquisitive natureinquisitive nature
and can easily become a and can easily become a distractiondistraction
steering with numbers is the wrong steering with numbers is the wrong metaphormetaphor
metrics are your guide and can help you metrics are your guide and can help you make decisionsmake decisions
make sure your metrics support the decisions make sure your metrics support the decisions you need to make…you need to make…
1. Construct metrics for root cause 1. Construct metrics for root cause decision making…decision making…
Or use GQMOr use GQM
GOAL QUESTION METRICGOAL QUESTION METRIC
WERNER HEISENBERG (1901 - 1976)
‘‘The more precisely the The more precisely the POSITIONPOSITION is is determined, the less precisely the determined, the less precisely the MOMENTUM MOMENTUM is known’is known’
detrimentally affecting software detrimentally affecting software project teams ability to adapt to project teams ability to adapt to circumstance is a key mistake.circumstance is a key mistake.
every project team should be every project team should be ruthless in its adaption to situational ruthless in its adaption to situational circumstance.circumstance.
2. Don’t restrict a teams ability to 2. Don’t restrict a teams ability to maximize their effectiveness maximize their effectiveness through over zealous or bureaucratic through over zealous or bureaucratic reporting practices…reporting practices…
In order for metrics to be useful they In order for metrics to be useful they have to be accepted.have to be accepted.
3. don’t build 3. don’t build metrics that metrics that are too are too complicated complicated to to understand.understand.
So….So….
back to metrics…back to metrics…
3 flavors3 flavors
Software metrics Agile Project metrics Organizational metrics
The primary metric for agile The primary metric for agile software development is software development is whether software exists and whether software exists and is demonstrably suitable for is demonstrably suitable for use in its intended purpose.use in its intended purpose.
This key metric is tested by This key metric is tested by demonstration at the end of demonstration at the end of every iteration and release.every iteration and release.
Individual metricsIndividual metrics
Defects Complexity Conformance Coverage Object Orientation Volatility
Much more useful when correlated…Much more useful when correlated…
Goal: I’d like to reduce defects.Goal: I’d like to reduce defects.
Question: Where do defects come Question: Where do defects come from and why?from and why?
Panopticode Panopticode shows the hot spots shows the hot spots in code where there are in code where there are intersections between low test intersections between low test coverage, high volatility and high coverage, high volatility and high complexity.complexity.
In the end code metrics can help In the end code metrics can help remove uncertainty from remove uncertainty from delivery and build a system with delivery and build a system with a longer life span, lower cost of a longer life span, lower cost of change.change.
Metrics smorgasboardMetrics smorgasboard Velocity: Number of story points completed in any given iteration.
Number of story points started in a given iteration.Total Number of Story points remaining in the project. Total Number of 'new' story points in project.
Number of stories added to the project in a given iteration.Number of story points added to the project in a given iteration.
Total number of stories in project. Total number of story points in project.
Resources:
Planned man days available per iteration.Actual man days used per iteration.
Total man days accrued on project to date.Total man days remaining on budget.
Total Man days lost to non project work.
Financials:
Cost per iteration.Cost per release.
Total days worked on Project.Total days worked by resource type.Monthly spend.Total Monthly spend.
Total Cost to dateTotal Cost to complete.Estimated Cost at complete.
% Unit Test coverage.
Acceptance TestsTotal number of Acceptance tests per iteration. Automated Non Automated.
Total Acceptance tests passing per iteration. Total acceptance tests failing per iteration.Total acceptance tests ignored per iteration.
Total automated Acceptance tests in project.Total non automated Acceptance tests in project.% Automated.
Defects (Bugs identified post development).Total number of defects per iteration.Total number of defects by severity.
Quality.
Unit TestsTotal number of unit tests per iteration.Total Unit tests passing per iteration.Total Unit tests failing per iteration. Total Unit tests ignored per iteration.
Uh oh.Uh oh.
We just broke golden rule #1.We just broke golden rule #1.
So, lets figure out what we need So, lets figure out what we need first…first…
Goal, Question, Metric.Goal, Question, Metric.
here are some common Goals, here are some common Goals, questions and examples of how questions and examples of how metrics support them…metrics support them…
Goal: I want predictability.Goal: I want predictability.
Q: Is my project team converging on Q: Is my project team converging on a solution that meets the outcomes a solution that meets the outcomes for my business?for my business?
Iteration
Total Effort remaining at start.
Work completed this iteration
Work Added this iteration.
Total effort remaining End Date.
Estimated Cost.
Actual Cost.
Accrued Costs.
Iteration 7-8 49 10 +0 39 01 Sept 34, 650 34,650 34,650
Iteration 9 39 12 +0 27 15 Sept 30,80030,800
65, 450
Iteration 10 27 10 +5 22 29 Sept 31,350
31,35096,800
Iteration 11 22 11 +0 11 13 Oct 31,900
31,900128,700
Iteration 12 11 11 +0 0 21 Oct 22,000
22,000150,700
100% of Outcomes met
Goal: I want to understand the Goal: I want to understand the capacity of the team and how it capacity of the team and how it changes.changes.
Q: Will defects affect the Q: Will defects affect the predictability of delivery?predictability of delivery?
Hypothesis: we can use yesterdays Hypothesis: we can use yesterdays weather to predict how much we can weather to predict how much we can get done next time.get done next time.
Velocity is simply the number of Velocity is simply the number of Story points delivered last time used Story points delivered last time used as a predictor for next time…as a predictor for next time…
Goal: I want to reduce waste in my Goal: I want to reduce waste in my process and deliver value to the process and deliver value to the customer as often as possible.customer as often as possible.
Q: Where are the bottle necks in my Q: Where are the bottle necks in my development process?development process?
Goal: I don’t want any nasty Goal: I don’t want any nasty surprises late in the project that will surprises late in the project that will affect the predictability of delivery.affect the predictability of delivery.
Q: Will defects affect the Q: Will defects affect the predictability of delivery?predictability of delivery?
0
10
20
30
40
50
60
70
80
90
Mon Tue Wed Thur Fri Mon Tue Wed Thur Fri Mon Tue Wed Thur Fri
Date
# o
f w
ork
ing
Acc
Tes
ts
Not workingWorking
At this point things can become less At this point things can become less quantitative and more qualitative…quantitative and more qualitative…
Goal: I want to build capabilities that Goal: I want to build capabilities that allow me to repeatably deliver high allow me to repeatably deliver high value products/services.value products/services.
Q: Are my teams developing Q: Are my teams developing repeatable capabilities?repeatable capabilities?
Pairing
TDD
ATDP
Cont Integration
Stand up’s
Retrospectives
Design Storming
Re-factoring
Goal: I’m a low risk business, if one Goal: I’m a low risk business, if one project fails this year I don’t want it project fails this year I don’t want it to sink the organization!to sink the organization!
Q: What is the risk profile of my Q: What is the risk profile of my project portfolio?project portfolio?
Potential Value
Risk
Cost
Goal: I want to maximize the value Goal: I want to maximize the value from my workforce ensuring smooth from my workforce ensuring smooth transition between projects….transition between projects….
Q: When will I need capacity?Q: When will I need capacity?
Be careful with this one…Be careful with this one…
Don’t break golden Don’t break golden rule number 2….rule number 2….
Goal: To understand if our strategy Goal: To understand if our strategy in the market place is working so in the market place is working so that we can adapt quickly. that we can adapt quickly.
Q: What do my customers think ?Q: What do my customers think ?
Fred ReichheldFred ReichheldLoyalty expertLoyalty expert
‘The ultimate ‘The ultimate question’question’(otherwise known (otherwise known as NPS)as NPS)
NNet et PPromoter romoter SScore: would you core: would you recommend us to a friend or recommend us to a friend or Colleague?Colleague?
P – D = NPSP – D = NPS
The difference between the The difference between the number of Promoters and number number of Promoters and number of Detractors = Net Promoter Score of Detractors = Net Promoter Score (NPS)(NPS)
Goal: To leverage our strategy to Goal: To leverage our strategy to make our enterprise successful.make our enterprise successful.
Q: Do our projects align with the Q: Do our projects align with the vision of our organization?vision of our organization?
A Starbucks on every corner
Inspire customers to enjoy the daily pleasure of Peet's Coffees & Teas by providing a distinctive, superior product, superior coffee and tea knowledge and superior service to every customer, every day.
Typical Key Performance indicators:
Financial Cash FlowROIFinancial ResultReturn on capital employedReturn on equity
Customer Delivery Performance to Customer - by DateDelivery Performance to Customer - by Quantity
Internal Business Processes Number of ActivitiesOpportunity Success Rate
Learning & Growth Investment RateIllness rate
The balanced score card.The balanced score card.
keep metrics simplekeep metrics simple
Figure out why you want to measure Figure out why you want to measure something before you figure out something before you figure out how.how.
GOAL > Question > MetricGOAL > Question > Metric
Everywhere is different . Build Everywhere is different . Build metrics that make sense for YOU.metrics that make sense for YOU.
Don’t restrict team process Don’t restrict team process adaptations, adapt reporting as adaptations, adapt reporting as requiredrequired