Agile Metrics Boom or Bane

www.unicomlearning.com

India Agile Week -201413-June-2014, Mumbai

Agile Metrics: Boon or Bane?

Suchitra Vasudevan

General Manger

Atos India Pvt. Ltd.

www.agileinbusiness.com

Samir Umrikar

Associate Manager


Agile Metrics

UNICOM PresentsIndia Agile Week-2014


Agile Metrics - Agenda


� To Measure or Not to Measure?

� Challenges faced

� Case study of project

� Industry best practices /

recommendations

� Q& A


Agile Metrics


Metric = “A measure or a combination of measures for quantitatively assessing, controlling or improving

a process, a product, a team”

Key Performance Indicator =“Quantifiable metrics which reflect the performance of an organization in

achieving its goals and objectives”

Diagnostic = “A metric used to diagnose and improve the processes that produce business value. What

it measures might not have a direct value to the customer.”


To Measure OR Not To Measure?


Why - To Measure

√Feedback & Decision-making

�For team performance

�Continuous improvement

�Motivate, Challenge

What - To Measure

√“Just enough” simple metrics

√“Business Value” to customer / Measure-Up

�Outcomes

�Trends, Ratios

How - To Measure

√“Good-enough” quality

�Storypoints, DoD, Burndown charts

�“Hard-to-game” metrics

When - To Measure

√Start / Stop

√Frequency, regular (iteration/sprint)

• Why - Not to Measure

√ For individual performance

√ Comparing projects against

√ Controlling teams

• What - Not to Measure

√ Outputs (efforts, variance from plan..)

√ Numbers (rather than trends)

√ Difficult/time-consuming

• How- Not to Measure

√ Organization level rather than project-level

√ Not set targets (use minimal)


Journey to Agile


Client was completely following waterfall /

iterative model till 2011

Agile / Scrum methodology started

implementing in project in early 2012 –

closely linked to agile Global Delivery

Platform with customization specific to

client

Deliver highest business value with

real working software to customer in

the shortest time.

Status:

80% Java projects around 25 projects,

170 team members following Agile


Challenges faced / Solutions applied – 1/2


▶ Team new to Agile

– Scrum Training for teams, 35 Certified Scrum Masters (CSM) & 2 PO

– Agile Club – Monthly meeting to share best practices (client, local)

– Agile Day – KXC for team with innovation games and lessons learned

� Distributed delivery and roles spread across locations

– Scrum Masters in each location working with local scrum teams Sometimes

appointed 2 Scrum masters for a sprint

– Backup of key roles PO and SM present at offshore

– For large teams - daily Scrum meeting at each location using Lync (taking

time-zone constraints), “Scrum of scrum” meeting between Scrum Masters

across locations

▶ Common estimation model, standard definition of story points

– Complexity guideline template with guidelines at project-level to define

story points

– No standard estimation model defined


Improvements : Lessons Learned from Agile


Good practices / Learnings

Experience & Knowledge

Customer Brainstorming QualityEmployees

• Agile Club – extended

with local experts

• Deputy of client Key roles

(PO, SM) in local

• Mandatory Demo with

PO in sprint

• Agile Day - KXC, Quiz

• Lessons Learned in

Retrospective meetings

• Customization of Agile

based on client context

⇒Pair-programming of

new and experienced

members

⇒Roles matched across

distributed teams

• Scrum training,

certification (35 Certified

Scrum Masters, 2 Product

Owner)

• Daily scrum meetings with

teams and “Scrum of

Scrum” weekly across

locations

• Definition of “Done”

standardized

• Continuous

Integration (SONAR

dashboard,

automated testing)

• Usage of Jira

dashboard


Case Study Project - Overview


▶ Tools used to mange SCRUM / Agile Practices : JIRA + Green Hopper Plug-in

▶ Scrum Team Size: 6 to 8

▶ Product Owner Role is performed by someone from client team. However there are 2 product owners now in local team.

▶ Scrum Master Role is performed by local / client

▶ Sprint Size: 2 to 4 weeks

▶ Product Owner makes Release and Sprint backlog in JIRA.

▶ Before starting a sprint, local team provides the capacity to client team considering public / personal holidays of local team members working on the sprint.

▶ Local team receive requirements for a given sprint (ppt, doc). client team also creates stories for each sprint in JIRA.


Case Study project -Agile Metrics Dashboard (1/3)


Velocity Ratio :- Objective is to Keep track of the

velocity w.r.t. Baseline Velocity and analyse

improvement.

Target >=0

Indicators

• > 0 : Positive Velocity improvement trend

• = 0 : Velocity as per baseline

• < 0 : Velocity not as per expectations; further

analysis and actions needed

Story Completion Ratio:- Objective is to Measure

the budget and timeliness of deliverables of sprint

Target >= 70%

Indicators

• 100% -> Deliverables exceeding expectations

• > 70% and < 100% -> Deliverables meeting

expectations

• < 70% -> Deliverables NOT meeting expectations,

need analysis and actions




Story Acceptance Ratio:- Objective is to

Measure Quality of deliverables of sprint

Target >= 90%

Indicators

• 100% -> Deliverables exceeding expectations >

90% and < 100% -> Deliverables meeting

expectations

• < 90% -> Deliverables NOT meeting expectations,

need analysis and actions

Average Cost Per Story Point:- Objective is to

Measure effort / Cost spent to complete a story

point. Ideally it needs to decrease over Sprints.

Target <= Agreed baseline

Indicators

• Reducing Trend is good indicator

• In case of increasing trend then analysis and

actions needed.




Quality completeness by SPRINT:- Objective is to

Measure Quality of overall sprint. It is based on

• Product Backlog Stories Written (INVEST

Model- Independent, Negotiable, Valuable,

Estimable & Testable)

• Sprint Organised

• Sprint Content Proposed

• Sprint Planned

• Sprint Prepared

• Sprint Executed

• Sprint Reviewed

• Sprint Retrospective Completed

Phase wise Dashboard is also used to analyze

Quality with in and across SPRINT.

It is Quality check questionnaire based on various

phases of Sprint life cycle.


Case Study project - SONAR Dashboard


Unit Test Success:- Objective is to Measure how

many test cases are passed. It is run by J Units and

it is delivered at the time of Sprint Review.

Unit Test Coverage:- Objective is to Measure how

much existing code is covered by the test cases.

The objective is to cover 50% of legacy code in

next one year.

Rule Compliance :- As client has its own

technology and platform. There are specific coding

rules that had been defined and continuously

updated by client and the developed code by local

teams is expected too follow at least 95% of the

rules.


Industry best practices / recommendations


Here is a quick list that you may consider using:

• Velocity Ratio, Acceleration Ratio

• Resource Utilization, Burndown charts

• Average Cost

• Adherence to doneness/acceptance criteria, checklists

(checklist item burnup), standards (PCI Standards)

• Test coverage (Code Coverage, Branch Coverage),

Running tested features

• Rule Compliance (wrt. Customer standards), Team

member peer reviews

Category Metric Quality Defect Count, Technical Debt, Faults-Slip-Through

Predictability Velocity, Running Automated Tests

Value Customer Satisfaction Survey, Business Value Delivered (ROI, EV

analysis)

Lean Lead Time, Work In Progress, Queues

Cost Average Cost Per Functions


Questions / Queries



Agile Metrics - Agenda


List of references

• http://www.slideshare.net

• http://www.scrumalliance.org/

• http://agile2009.agilealliance.org

• http://www.toolsjournal.com/agile-articles


Organized byUNICOM Trainings & Seminars Pvt. Ltd.

[email protected]

Suchitra Vasudevan [email protected] [email protected]

Agile Metrics Boom or Bane

Documents

Transcript of Agile Metrics Boom or Bane