Agile Metrics

SergeKovaleff @gmail.com

.blogspot.com

linkedin.com/in/

facebook.com/

What is a Metric?

Process Improvement Cycle

PDCA (plan-do-check-act) by Dr. W. Edwards Deming DMAIC (Define-Measure-Analyze-Improve-Control) in Six Sigma

Improve

Process

Define Process

Execute

Process

Control ProcessMeasure

Process

A lot of Metrics different ...

• Business metrics – RTF (Running Tested Features) – Earned Business Value (EBV) – Net Present Value (NPV) – Internal Rate of Return (IRR) – Return of Investment (ROI)

• Code Metrics – Cyclomatic complexity – Best practices violation – Coding standards violation – Possible bugs – Code duplication – Code coverage – Dead code – Test Quality

• Design Metrics – Code dependencies

• Incoming (Affering Coupling) • Outgoing (Efferent Coupling)

– Abstractness • Number of abstract clases and interfaces • Number of concrete classes

• Process metrics – Agile practice maturity – Impediments cleared per iteration – Impediments carried over the next iteration – User stories carried over the next iteration – User stories done per iteration – Defects carried over the next iteration – Defects carried over the next iteration – Team member loading – Velocity of development – Backlog size

• Automation Metrics – Code coverage – Number of builds per day – Time taken per build – Number of failed/success builds – Trends in code metrics

• Testing Metrics – Acceptance tests per story – Defects count per story – Test time to run – Tests run frequency – Manual tests per story – Automation percent – Time to fix tests

This slide is not for human reading

Thermometer measures a temperature of the thermometer

There is no spoon THE METRIC

The single metric to control • Quality • Speed • Productivity • Customer satisfaction • Etc.

… is still being looked for :)

Metrics improve productivity?

Myth

Myth: bigger velocity is better

Myth

WILL ACCURACY OF ESTIMATES

INCREASED

Myth

IF REQUIRE COMMITTED

= 100% DELIVERED?

Choosing a GOOD metric

Encourage target behaviour

Measure trend

Easy to collect and measure

Inspire useful discussions (what have learnt)

Gives periodic feedback

Can make conclusions

and fix the process

Measurement effect

Tell me how you will measure me and I’ll tell you how I will behave.

Eli Goldratt “The Goal” (2004)

Leading vs. Lagging

Leading Lagging

Leading vs. Lagging

Leading Lagging

What to measure?

• Productivity metrics

• Predictability metrics

• Quality metrics

• Value metrics

Predict

Quality

Value

Productivity

And now ...

The Metrics!

Productivity metrics Productivity

Velocity Leading

Productivity

Myth

Risk of Story Points inflation

Truth: Story points are relative

Work In Progress Leading

Productivity

Story Cycle Time

Leading

Productivity

Predictability metrics Predict

Sprint Burn Down Chart Leading

Predict

Release Burn Down Chart Leading

Predict

Burn Up Chart

Leading

Predict

ROI (Return-on-Investment)

• Backlog item - Business-Value • Value in $

Quality metrics

Technical Debt

• Backlog

– Inner vs. External quality

– Testing Automation

– Reengineering

– Maintainability

– Documentation

Leading

Quality

Running Automated Tests

• Amount of tests

– Functional

– Unit

• Code Coverage

• Must be increasing

• Tests must pass

Leading

Quality

Post Sprint Defect Arrival

Quality

Lagging

Post Release Defect Arrival

THE SAME

Lagging

Quality

... after global Release

The ONLY valid Quality metric Lagging

Quality

Value Metrics

Customer Satisfaction Survey

The best way to find out whether your customers are

satisfied is to ask them

Lagging

Value

How satisfied are you with your purchase? the service you received? with our company overall?

Basic satisfaction

Customer loyalty

How likely are you to buy from us again? recommend our product/service to others? recommend our company to others?

TotalDetractorsomotersPr

Customer Satisfaction Survey

Lagging

Value

Lagging

Value

How often: “So often, that get more information, But not so often that annoy”

Key Drivers of satisfaction: • Conflicting goals • What does Client feel important • Scatter diagram (Key Driver Chart)

Employee Satisfaction Survey Lagging

Value

Lagging

Value

Short Summary ∑® • Goal

– Why do you want to measure?

– What would be the result of the intrusion?

• Classification – Time

• Leading

• Lagging

– Subject • Productivity

– Story Cycle Time

• Predictability – Burn up Chart

• Quality – Tech Debt

• Value – Net promoters

Questions time

Links • http://agilebasecamp.org/

• http://blog.scrumtrek.ru/2010/01/wip-story-cycle-time.html

• http://blog.scrumtrek.ru/2010/01/velocity.html

• http://www.scrum.org.za/uploads/2009/09/measuring-for-results-2-small.pdf

• http://scrumorlando09.pbworks.com/Scrum-Metrics-and-Myths

• http://www.slideshare.net/petebehrens/measuring-agility-top-5-metrics-and-myths

• http://www.slideshare.net/alimenkou/agile-metrics-2725666

• http://management.about.com/od/competitiveinfo/a/CustomerSatSurv.htm

• http://en.wikipedia.org/wiki/Net_Promoter