Lean Agile Metrics And KPIs

37
Introduction to Lean Agile Metrics and KPIs Yuval Yeret Senior Lean/Agile Consultant @ Agilesparks [email protected] Mobile: 054-4802458

Transcript of Lean Agile Metrics And KPIs

Page 1: Lean Agile Metrics And KPIs

Introduction toLean Agile Metrics and KPIs

Yuval YeretSenior Lean/Agile Consultant @

[email protected] Mobile: 054-4802458

Page 2: Lean Agile Metrics And KPIs

Brief Intro

AgileSparks is a company that provides complete Agile/Scrum solutions and helps companies to improve project management, with emphasis on software and hardware development.

Myself – Lean/Agile Consultant/Coach, coming from R&D leadership background, specializing in scaling agile through Lean. Find more about me at http://www.linkedin.com/in/yuvalyeret, my google profile

Page 3: Lean Agile Metrics And KPIs

Taken from http://aplndc.pbworks.com/f/2009-05-Agile-Metrics.pdf

Page 4: Lean Agile Metrics And KPIs

Measuring/Managing Organization Performance

What are we here for?

Do Agile? Develop software? Generate most “Value for R&D

investment” in a sustainable way? $ for the shareholders?

So how do we perform in a way that aligns with that?

Page 5: Lean Agile Metrics And KPIs

Assumption – Agile drives Delivery engineAgility (1)

Better Software Development Better Business

(1) Proven in industry-wide research (Cutter, Forrester, QSM), various enterprise companies (Yahoo, Google, Nokia, Salesforce, etc.)

Page 6: Lean Agile Metrics And KPIs

Assumption – Agile drives Delivery engineSo we…

Measure Software Development To Predict Business performanceMeasure Agility To Predict Software Development Performance

(1) Proven in industry-wide research (Cutter, Forrester, QSM), various enterprise companies (Yahoo, Google, Nokia, Salesforce, etc.)

Page 7: Lean Agile Metrics And KPIs

Lagging/Leading KPIs

Which leads us to KPIs – Key Performance Indicators

There are two major types of KPIs: leading and lagging indicators. • Leading indicators measure activities

that have a significant effect on future performance.

• Lagging indicators, such as most financial KPIs, measure the output of past activity.

Page 8: Lean Agile Metrics And KPIs

Development Capabilities• Sprint

Velocity Variance/Tren

d• Feature Cycle

Time• Story Flow /

PCE• Quality Audit

Pass Rate• Testing

code coverage

• % tests automate

d• Agility Score

• Team Satisfaction

Survey

Product• Delivered/

Earned Business

Value• Running

Tested Features Velocity

• Technical Debt (Defects

Backlog)• Customer

Satisfaction survey

• Frequency of upgrades to latest in field

Schedule• EVA /

AgileEVM• Release

Burndown• Days per

Release / Release

Frequency• Stabilization

% of Release Schedule

• Due Date Performance (preferably in Dollar Days)

Financial• NPV

• Savings or

Revenue $ per

Release

Page 9: Lean Agile Metrics And KPIs

Leading Agile Indicators

Use with Care!

Page 10: Lean Agile Metrics And KPIs

Focus on being really READY and really DONE

Being READY READY reduces every category of work (defects,

rework, total work required, and process overhead) by almost 50%

160%/340% productivity improvement DONE DONE - “Build Quality into

Sprints” Fix problems fast Don’t save quality issues for later With the right infrastructure and mindset, IMPROVES

productivity

Based on Lean/TOC principles…

Page 11: Lean Agile Metrics And KPIs

Relevant Measures READY READY

Leading: • Depth of READY backlog for teams

Lagging: • Story Flow / Process Cycle Efficiency (PCE) – time story

actually worked on versus overall time story in processing• Actual Effort variance

DONE DONE Average Time to fix for defects (goal – hours-days) Average Time to fix for build failures (goal – minutes) Defects backlog across sprints (goal – 0) Development/Stabilization schedule ratio (~30% normally. 10% goal)

Page 12: Lean Agile Metrics And KPIs

READY Leads to Flow and Productivity

Page 13: Lean Agile Metrics And KPIs

Why focus on “Time to Fix Defects”

Because Fixing bugs earlier is cheaper Why?

• Early warning (broken/incompatible/conflicting) – faster pin-pointing of root cause and fixing

• Easy Revert and offline debugging – not in the critical path

Savings of ~10K$ for each bug found during sprinthttp://studios.thoughtworks.com/2007/5/10/continuous-integration-in-the-enterprise-with-cruisecontrol

Example from a product in an Enterprise Portfolio – 40 escaping defects translate into 400,000$ (multiply per product per release)

Page 14: Lean Agile Metrics And KPIs

Example Bug fix costs per Maturity

No Agil

e Eng

ineeri

ng P

ractic

es

Basic

Engine

ering

Prac

tices

Advan

ced E

ngine

ering

Prac

tices

0

40

80

120

0

40

80

120~120

~90

~40

TotalEscapingNon-functional Testing PeriodSystem TestingComponent Testing

Agile Maturity

Tota

l Cos

ts

Page 15: Lean Agile Metrics And KPIs

Faster/Cheaper Stabilization Period Integration problems are detected and fixed continuously

- no last minute hiatus before release dates Less cycles / retesting – Arriving to Packaging with

higher quality/readiness Product/PIL focus on Exploratory, Advanced scenarios

sooner, and can finish faster Better predictability allows squeezing in more

functionality until the last minute Smoother and more predictable flow – shorter stops Makes frequent release strategy more feasible

financially

Page 16: Lean Agile Metrics And KPIs

Stabilization Period implications

IT #1 S1

IT #1 S2

IT #1 S3

IT #2 S1

IT #2 S2

IT #2 S3

IT #3 S1

IT #3 S2

IT #3 S3

IT #4 S1

IT #4 S2

IT #4 S3

IT #5 S1

IT #5 S2

IT #5 S3

0

2000

4000

6000

8000

10000

12000

14000

16000

18000

Burnup w/ CI v2Burnup w/ CIBurnup

Tota

l SP

Accu

mul

ated

Page 17: Lean Agile Metrics And KPIs

Release Burnup with and without CI

IT #1 S1

IT #1 S2

IT #1 S3

IT #2 S1

IT #2 S2

IT #2 S3

IT #3 S1

IT #3 S2

IT #3 S3

IT #4 S1

IT #4 S2

IT #4 S3

IT #5 S1

IT #5 S2

IT #5 S3

0

2000

4000

6000

8000

10000

12000

14000

16000

18000

Burnup w/ CI v2Burnup w/ CIBurnup

Tota

l SP

Accu

mul

ated

ShorterStabilization

Page 18: Lean Agile Metrics And KPIs

Release Burnup with and without CI

IT #1 S1

IT #1 S2

IT #1 S3

IT #2 S1

IT #2 S2

IT #2 S3

IT #3 S1

IT #3 S2

IT #3 S3

IT #4 S1

IT #4 S2

IT #4 S3

IT #5 S1

IT #5 S2

IT #5 S3

0

2000

4000

6000

8000

10000

12000

14000

16000

18000

Burnup w/ CI v2Burnup w/ CIBurnup

Tota

l SP

Accu

mul

ated

ProductivityImprovement

Page 19: Lean Agile Metrics And KPIs

Measures for Eng Practices Maturity

% Time build is broken % Time build is broken

due to compilation errors Come on…

% Code Coverage By Unit Tests By Test Automation

Checkin early and often Checkin frequency Size of checkins (Smaller

better)

Page 20: Lean Agile Metrics And KPIs

Burndown + EVM

Page 21: Lean Agile Metrics And KPIs

Agile Earned Value Mgmt

Page 22: Lean Agile Metrics And KPIs

AgileEVM Metrics

Page 23: Lean Agile Metrics And KPIs

Earned Business Value

Allocate BV points per minimally marketable feature (MMF)

Track completion of BV points Complements AgileEVM

Page 24: Lean Agile Metrics And KPIs
Page 25: Lean Agile Metrics And KPIs

KPIs

So Agile Maturity Score as a KPI should

1. Be aligned with the goal of being a more Agile.

2. Be owned by each Product – who are accountable for the outcome of their efforts

3. Predict successful software development delivering best business value for the buck

4. Provide timely, actionable data, so users can intervene and improve performance BEFORE its too late

5. Focus on key high-value indicators6. Easy to understand – users should know

how to drive for improved performance7. Balanced with Velocity/Quality/Stakeholder

satisfaction/other. 8. Trigger positive outcomes in each group

working with it9. Standardized scale across company10.Targets/Objectives should be driven by

context/situation of each group11.Reinforce performance by loosely

connecting to incentive systems? Maybe just as a very low influence. Main driver for incentives should continue to be business/team performance meeting goals. Agile Maturity should drive that

12.Stay relevant. Focus on areas which still need work, Add more measures for future objectives when become relevant

Page 26: Lean Agile Metrics And KPIs

Be aligned with the goal of being a more

Agile company

Page 27: Lean Agile Metrics And KPIs

Be owned by each Product – who are accountable for the outcome of their efforts

Page 28: Lean Agile Metrics And KPIs

Predict successful software development

delivering best business value for the buck

Page 29: Lean Agile Metrics And KPIs

Provide timely, actionable data, so users can intervene and improve performance BEFORE its too late

Page 30: Lean Agile Metrics And KPIs

Focus on key high-value indicators

Page 31: Lean Agile Metrics And KPIs

Easy to understand – users should know how to drive for improved performance

Page 32: Lean Agile Metrics And KPIs

Balanced with Velocity/Quality/Stakeholder satisfaction/other.

Page 33: Lean Agile Metrics And KPIs

Trigger positive outcomes in each group working with it

Page 34: Lean Agile Metrics And KPIs

Standardized scale across company

Page 35: Lean Agile Metrics And KPIs

Targets/Objectives should be driven by context/situation of each group

Page 36: Lean Agile Metrics And KPIs

Reinforce performance by loosely connecting to incentive systems?

Page 37: Lean Agile Metrics And KPIs

Stay relevant. Focus on areas which still need work, Add more measures for future objectives when

become relevant