Revisiting cost of change(and other things on my mind in 2009)

David J. Andersondja@djandersonassociates.com

USC CSSE Annual Research Review 2009

Lifecycle throughput & capacity

In this example, the developers have 2.5x greater capacity than the analysts. Analysis and customer acceptance (same resources) is the bottleneck. System testing has 1.5x greater capacity than the system throughput.

Example lifecycle with throughput capacity (approx) Function Points per unit of time (e.g. month)

Development is seldom the bottleneck

• Teaching Agile Management class over 3 years <10% of participants report development as their lifecycle/workflow bottleneck

• Alistair Cockburn has pointed out that rework in non-bottleneck stations is free and can be used as a process strategy

Cockburn, Alistair, “Spending efficiency to go faster”, Humans & Technology Report, March 2008, http://alistair.cockburn.us/%22Spending%22+Efficiency+to+Go+Faster

Cockburn, Alistair, “Two case studies motivating efficiency as a ‘spendable’ resource”, ICAM 2005 International Conference on Agility, Helsinki July 27–28, 2005http://alistair.cockburn.us/Two+Case+Studies+Motivating+Efficiency+as+a+%22Spendable%22+Quantity

Possible Conclusions

• The cost of change is unique to each project and value-stream and its resource allocation

• Research is required to discover the cost of change / rework framework for calculating cost of change in a specific workflow

• Question: How does throughput capacity change over project lifetime? Does the bottleneck move?

CYCLE TIME RELATIONSHIP TO QUALITY

Development cycle time has a non-linear relationship to quality

WIP is directly related to Lead Time and Quality

Device Management Ike II Cumulative Flow

020406080

100120140160180200220240

Time

Fe

atu

res

Inventory Started Designed Coded Complete

WIP

Lead Time

3 month lead time led to a >30x reduction in quality compared to 1 week lead time

Project B Cumulative Flow

0

25

50

75

100

125

150

175

9-O

ct

23-O

ct

6-Nov

20-Nov

4-Dec

18-Dec

1-Jan

15-Jan

29-Jan

12-Feb

26-Feb

11-M

ar

Time

Fe

atu

res

Inventory Started Designed Coded Complete

Lead Time

What I believe?...

time

Def

ect i

nser

tion

rate

linear

~5-10 days

Research Opportunities

• Is there existing data that can be analyzed to compare with this empirical result?

• Does the data correlate?• Can guidance be derived from the shape of

the resultant curve?...– Iteration lengths should not exceed 2 weeks– WIP and multi-tasking should be limited to

manage cycle time below 10 (working) days

REAL OPTION THEORYReal Options is Dead! Long live real options!

Real Options are not very useful

• Huchzermeier & Loch 1998 [INSEAD, Project Management under Risk]

• 5 types of uncertainty– Market payoff– Quality– Budget– Requirements– Schedule

Observations derived from Huchzermeier & Loch 1998

• Market payoff & budget uncertainty increase option valuations

• Quality uncertainty decreases option valuations. => Options are more valuable in high quality conditions

• Requirements uncertainty reduces options value. => In uncertain requirements environments lots of cheap/free options are desirable. With higher certainty, fewer higher value options are optimal

• Option value under schedule uncertainty depends on market payoff function

Conclusion

• Real Options valuation is only useful and possible in low uncertainty projects undertaken by high maturity (CMMI ML4/5 equiv.) organizations

• => Real Options are of no practical use in software engineering applications

SALVAGING VALUE IN THE WRECKAGE OF REAL OPTIONS

Real Options is Dead! Long live real options!

Abandon analogous mapping financial options but leverage some concepts

• Expiry Date• Liquidity• Optimal Exercise point?• Underlying asset?• Broker/market maker?• Valuation – determined as not useful

There is value in having options and making optimal decisions under uncertainty

• Understand goals• Create options• Define “last responsible moment” (decision

point at option expiry date)• Push back decision point as late as possible• Maximize information gathering before

decisionRef: Matts, Chris, “Last Responsible Moment”, Agile Journal, Jan 2009, http://www.agilejournal.com/component/content/article/884-last-responsible-moment-

Ref: Matts, Chris, “Last Responsible Moment”, Agile Journal, Jan 2009, http://www.agilejournal.com/component/content/article/884-last-responsible-moment-

Liquidity

• Liquidity is ability to transfer assets from one form to another – ability to exercise an option

• Liquidity is an aggregate of– Temporal elements– Work-in-progress (inventory)– Personnel (resources)

• Constraint personnel liquidity is ability to learn– (investment in) training will increase option liquidity

Liquidity guidance

• Generalists provide options liquidity (in comparison to specialists)– Rule: assign a specialist to a suitable task, hold generalists

in reserve• Resource availability affects liquidity

– Rule: assign a non-instant availability resource to non-urgent tasks, hold instant availability resources in reserve

• Cross-training, resource rotation generates option liquidity– E.g. Pair programming good, promiscuous pairing better

Ref: Belshee, Arlo, “Promiscuous Pairing and the Beginner’s Mind”, Proceedings of Agile Development Conference 2005, IEEE Computer Society

Liquidity & Task AllocationSkill A Skill B Skill C Skill D

Resource 1(Generalist)

Resource 2

Resource 3

Resource 4(Specialist)

Task requiring skill A

Who should we allocate the task in order to maintain option

liquidity?

If we assign a generalist…Skill A Skill B Skill C Skill D

Resource 1(Generalist)

Resource 2

Resource 3

Resource 4(Specialist)

3 Options

Liquidity 6

But if we assign a specialist…Skill A Skill B Skill C Skill D

Resource 1(Generalist)

Resource 2

Resource 3

Resource 4(Specialist) 4 Options

Liquidity 9

Some General Observations• Options liquidity is a key tool in project risk management• IID more liquidity than Waterfall (iterations)• Agile generally better than IID (generalist workforce)• 2 week Sprint Scrum than 4• 1 week iteration XP better than 2 weeks• Kanban/Pull system better than XP/Scrum (lower WIP, risk

based pull prioritization)• Marginal utility supplied by specialists can be traded

against liquidity from generalists• Efficiency (waste versus value-add) can be traded for

liquidity• Slack provides liquidity => line should not be balanced (i.e.

Lean manufacturing “balance the line” reduces liquidity)Ref: http://finance.groups.yahoo.com/group/real_options_discussion/message/369

Research questions?

• Can we measure/quantify liquidity?• Can we match required liquidity level to risk

profile (and/or uncertainty/variability in domain)?

• Can we provide (objective) guidance to managers on how much liquidity they need and how to maintain it?

Other research questions?

• Can we calculate an optimal exercise point (decision point) without calculating a value (and cost of exercising) for an option?

• Can this be useful if options are not free (or near free) to acquire and easy/low risk to exercise?

• Can these techniques be applied to architecture and design options and decisions[Alternative Architecture Assessment, Decision Analysis and Resolution (DAR)]?

Contact Details

David J. Andersondja@djandersonassociates.comhttp://www.agilemanagement.net/Tel: 206.201.2717Skype: agilemanagerhttp://www.linkedin.com/in/agilemanagement