Predictability at Axial Venwise Forecasting Summit – January 17, 2014 // Axial HQ

Why Optimize for Predictability?

Better coordination Both within and across teams.

No more “managing expectations” Means more time managing people

It will actually make you faster. Predictability v. Speed is a false choice

Prerequisites

Relatively Stable Priorities/Goals One top priority for the whole company. One key product/strategy/hypothesis for each team to focus on. A small number of goals with clearly defined and measurable results for each team. Company priority, team focus and team goals cannot change too frequently.

Project Contributor Focus Minimal (and ideally no) interruptions. Work-In-Progress constraints (ideally constrained to doing 1 thing). A big enough team to allow for the above 2.

A Process and Supporting Tools The steps that take a project from start to finish needs to be (at least loosely) defined. Progress through these steps needs to be tracked somehow (using a tool).

Relatively Stable Priorities/Goals

One top priority for the whole company. Reduce paying Member churn through a unified company-wide focus on Member success.

One key product/strategy/hypothesis for each team to focus on. Team Engagement – Increased adoption of key products at higher frequencies lowers churn. Team Responsiveness – Members responding to each other at higher rates in less time lowers churn.

Goals with clearly defined and measurable results for each team. Objective: Predictably Ship Value to Our Members Key Result: Bat .714 (ship ‘L’ value iteration in 11 of 14 weeks in Q4 2014) Key Result: Increase overall NPS by X%. (Engagement) Key Result: Increase network-wide response rate to Y%. (Responsiveness)

Company priority, team focus goals cannot change too frequently. Company and team goals revisited once per quarter. No change for the last 2 quarters in objectives, but we have changed key results.

Project Contributor Focus

Minimal Interruptions Dedicated “bug-buster” to remove defects as interrupts from the rest of the team. Avoid iteration order and staffing dependencies (use forecasting when you can’t avoid).

Work-In-Progress Constraints Do one thing at a time. Do it until it is done. Done means in production.

A Big Enough Team Invest liberally in support (DevOps, Test Eng, IT) to avoid blockages. Predictability is not at all achievable (or probably desirable) on teams of 2 or less.

Stack – The Axial Process

1.  Break project into iterations"2.  Estimate value relative to other iterations (S/M/L)"3.  Estimate effort relative to other iterations (S/M/L)"4.  Stack iterations for “value arbitrage”"

•  L value/S effort first, M/S second, L/M third"•  Then S/S, M/M, L/L"

5.  Throw away iterations that aren’t worth it (S/L, etc.)."6.  Team figures out how to staff iteration #1"7.  Iteration #1 gets broken into execution tasks"8.  Programmer who is actually going to do the

execution task estimates effort relative to other execution tasks (S/M/L)."

9.  Work for 2 weeks."10. Repeat."

Estimating You think, if we could just get this right …

People Are Very Bad Estimators

Spend as little time estimating as possible Estimation is guessing, people are bad at guessing on an absolute basis. Only ask questions like … is this small similar to that small?

Estimates should be done by the person doing the work People are even worse at guessing on behalf of others. Managers are the worst at guessing on behalf of their reports. Collaboration on estimates helps nothing, my small and your small are different. Don’t play planning poker, it is a catastrophic waste of time, it is hurting your business.

Compensate for bad estimates with great forecasting I promise this will make sense by the end.

Great Forecasting Evidence-Based Scheduling + Stack

Tooling – The Important Stuff

Capture Data Passively If you require active time-tracking, people will forget, and you’ll have bad inputs

Only Care About the Big Picture Track when it starts, when the implementer can pick up the next item, and when it’s done. For Axial, this is: Working, Ready for Testing, Released.

Don’t forget about Vacations Keep a vacation calendar, and use it to adjust your schedules. Don’t adjust for sick days, that will get factored in.

JIRA + stackpm Demo

First We Compute Evidence … We need aggregates, and individual deliverables by size, by engineer

… We care about Lead Times Lead times measured in days, don’t count weekends or vacations

Then we take the data from a project Here we need to know the engineer, size, order and any dependencies

Use the Past to Simulate the Future Just like Joel, http://www.joelonsoftware.com/items/2007/10/26.html

0%

20%

40%

60%

80%

100%

10/18/13 10/25/13 11/1/13 11/8/13 11/15/13 11/22/13 11/29/13

You’ll See Disaster Coming … Cumulative Distribution of Email Digest Ship Dates Simulation

50% 75% 98%

They said 10/20

It shipped 10/31

And know who’s struggling …

0%

25%

50%

75%

100%

125%

Small Medium Large

Time to QA Co-Efficient of Variation (σ/μ) by Engineer by Size

Which will make you more predictable 08/02: Company Profiles 08/09: Company Search 08/16: Miss 08/23: Miss 08/30: Multi-Select on Recipients 09/06: New Opportunity Workflow 09/06: Call from Search 09/13: Company Profile Tombstones 09/13: News on Homepage 09/20: Recipients Folders 09/20: Transaction View/Create 09/27: Search Messaging w/Quotas 10/04: Notes on Recipients Page 10/11: Custom Opportunity Teasers 10/18: Received Opportunity Digest

10/25: Daily/Weekly Pursuit Digest 11/01: Weekly Performance Digest 11/08: Miss 11/15: TP Redesign v1 11/22: Public Opportunity Creation 11/29: Search from Everywhere 12/06: TP Row Redesign 12/13: Member Mail Addresses 12/20: Announcements from NAF 12/20: TP Pursue/Decline w/Msg 12/27: TP Notes 12/27: TP Details 12/27: Announcements on Co. Profiles 01/03: Response Rate Badges 01/08: Miss

And Speed You Up Time to first member-facing release in days, by initiative

0  

20  

40  

60  

80  

100  

120  

Started Stack Here

Thanks – Stay Tuned for StackPM What do you do?