7 Signs of Bad Project Estimates - And How You Can Fix It
-
Upload
officework-software-llc -
Category
Business
-
view
7.506 -
download
2
description
Transcript of 7 Signs of Bad Project Estimates - And How You Can Fix It
Kolinger Associates, LLC 2010 All Rights Reserved
7 Signs You Have a Bad Project Estimate(And what to do about it)
January 20th, 2010
Kolinger Associates, LLC 2010 All Rights Reserved
Today’s PresentationEstimating project cost and schedule is a serious challenge to our companies.
Common symptoms include projects that refuse to end, projects cancelled, people overworked, missing a window of opportunity for a new product, and overruns causing an inability to fund other important initiatives.
Tools may help - but this presentation is primarily about the ideas and concepts necessary for improvement.
Kolinger Associates, LLC 2010 All Rights Reserved
Overview• This presentation considers factors that are common to most but some
specific references to software development estimation.• Me: 25+ years, mostly in large corporations. Developed software,
consulted, trained, managed – and recovered - scores of projects. • Founded OfficeWork Software in 2005.
Kolinger Associates, LLC 2010 All Rights Reserved
Bad Estimates Very Common
• 67% of all projects under-planned by 290%• 1% of projects meet time, budget, and scope goals• 60% of all projects over 50k Lines of Code cancelled before
completion.• Common gripe: “<<your organization>> does not deliver”
A “Bad Estimate” working definition: Money got spent and the required result was not achieved in time.
A few sundry findings:
(82.3% of all statistics are made up on the spot. Just kidding.)
Kolinger Associates, LLC 2010 All Rights Reserved
The Saga1. Sufficient assets are not allocated, and there is not enough time.
2. The schedule (if there is one) starts to slip, one day at a time.
3. The project manager suddenly realizes the slippage and seeks a culprit.
4. People from various departments involved accuse people from other departments of delaying the project. More time is wasted in finger-pointing.
5. To make up time, the project manager decides to “crash” the project by applying more assets/resources to all activities that are currently being performed.
6. Everyone scrambles to crash his job, and people are infuriated to find out that they are either further behind or finished and their part is not yet needed. Interest wanes as people chafe under the new delays. The project is over-budget due to all the crashing, and the whole thing is either:
– a. on-time but shoddy, – b. well-done but late, – c. both shoddy and late, or – d. abandoned.
Dr. Robert Graham – Project Management as if People Mattered
Kolinger Associates, LLC 2010 All Rights Reserved
QuestionThink of a recent project, that went over budget, was late, or fell short of the original scope.
– Why did that happen? – What could you have done about it?
(Write down your answer)
Kolinger Associates, LLC 2010 All Rights Reserved
Test Your Estimation IQ
Answer: Genetic Mutation!
Question: Why do IT Project Managers estimate at the drop of a hat?
• Have Project• Will Estimate• No Requirements Needed!
Actual Project Manager DNA !
Warning: Do not ride in the elevator with your project sponsor! (Hint: You might rattle off an estimate!)
Gopal Kapur, Center for Project Management
Kolinger Associates, LLC 2010 All Rights Reserved
Question?
1. How many of you have managers who pressure for cost and date estimates before requirements are clear?
2. How many of you are those managers?3. What are you doing to encourage better
estimating?4. How is it going?
Kolinger Associates, LLC 2010 All Rights Reserved
Let’s Estimate!The Project (Input):Web-based information system. (Legacy system already exists; let’s just make it web-based)
The Estimate (Output):
– 12 months– 4.4 FTE– 5,600 hours– $400,000
“We will have it for you in 12 months”
The Estimation Process(Magic wand)
Kolinger Associates, LLC 2010 All Rights Reserved
What to look forSo, how can you recognize the bad estimate…before you get bit?
12 months4.4 FTE5,600 hours$400,000
Kolinger Associates, LLC 2010 All Rights Reserved
1. When You Hear ‘Done’ More Than 4 Times.
•Project ‘success’ is meeting expectations. Period. •Common finding:
– Everyone ‘knows’– Nobody agrees
•Define the project – in writing - one-pager: – Description: (What, where, when, how much)– Desired Results: (Benefits that will be achieved)– In Scope / Out of Scope – Assumptions
•De-babelize –Use language everyone understands•Sign-off from the sponsor and each stakeholder is critical.
(Harder than it sounds)
So, what the heck does ‘done’ look like?Look for a clear, written definition of the end-result.
Kolinger Associates, LLC 2010 All Rights Reserved
2. When you Hear, “What is the Plan?”
• The plan includes, but is more than the project schedule• The plan includes the project definition, the people, a WBS (Work Breakdown
by deliverable), estimates, resource assignments• Milestones and activities are connected (or linked) so that a critical path can be
identified and managed. • Connections are critical as they indicate hand-offs from one person to another.• Each deliverable has an owner• Each person on the team is associated with a deliverable• Have an independent group review the plan. (Find bug on paper vs.
production).
The estimate is based on the plan. Look for the plan with evidence of traceability to the project definition and make sure everyone is aware.
If you ever hear people saying, “I don’t know what the plan is”, take action!!
Your job as project manager is to keep everyone honest to the project’s schedule, scope, and budget. Even if the estimate were ‘perfect’ you can render it useless through a poor plan.
Kolinger Associates, LLC 2010 All Rights Reserved
Clue: Missing Dependencies
In this case, ‘date’ was the primary goal
Kolinger Associates, LLC 2010 All Rights Reserved
View resource graph and Gantt
Kolinger Associates, LLC 2010 All Rights Reserved
Review usage in excel
Kolinger Associates, LLC 2010 All Rights Reserved
3. When You Have a Single Number Estimate
Two critical lessons:1. Projects are bigger than they appear.2. The road to project hell is paved with single number estimates.
Best Case Worst CaseMost Likely
Look for a schedule based on appropriate estimate ranges
Gopal Kapur, Center for Project Management
Software is “different”
Kolinger Associates, LLC 2010 All Rights Reserved
Estimating Confidence in Software Delivery
Observed Limits of Estimate Accuracy
0.67x
0.8x1.0x
0.5x
0.25x
2x
4x
1.5x
1.25x
Initialproduct
definition
Approvedproduct
definition
Requirementsspecification
Productdesign
specification
Detaileddesign
specification
Productcomplete
Project cost(effort and size)
Projectschedule
0.85x
0.9x1.0x
0.8x
0.6x
1.25x
1.6x
1.15x
1.1x
Com
plet
ion
Underestimated
Overestimated
+ 400 %
- 400%
Exact 0%
Effort Duration
Best WorstMost Likely
10% Chance
50% Chance
90% Chance
Sizing
Rough
Fair
B. Boehm – SW Engineering Economics
Kolinger Associates, LLC 2010 All Rights Reserved
4. When It Has Not Been Done Before Look for an estimate that factors your capability
Project management is all about making promises and keeping promises.
Promises are more likely to be kept if:
• Made voluntarily (you agree)
• It’s your commitment (your data)
• it is according to capability (i.e., you have proven ability)
Include Risk Management (Project Management for Adults)
To maintain a viable estimate constantly manage project adversity and team capability
Stan Rifkin – Master Systems
Tom DeMarco - ASG
Robert Charette - ITHABI
Kolinger Associates, LLC 2010 All Rights Reserved
5. When You Can’t See Evidence of Historical Data
• The estimate should consider a reasonable work week for resources based upon historical data for individuals and the organization.
• Examples:– 60% productivity or 24 hours/week/person– Individual characteristics
• Technical skill factor• Business knowledge factor
– Careful use of overtime– Defect and rework rates
Look for an estimate that uses historical performance data as a reference
Only lunch breaks are 100% productive
Kolinger Associates, LLC 2010 All Rights Reserved
Efficiency
Kolinger Associates, LLC 2010 All Rights Reserved
Question
• What percent of a programmer’s time is spent coding?
• How is this factored into the estimate and schedule?
Kolinger Associates, LLC 2010 All Rights Reserved
What do Programmers Do?
Kolinger Associates, LLC 2010 All Rights Reserved
Look for a Repeatable Estimating Process
Historical data is only valuable (normalized) in the context of a repeatable process
Kolinger Associates, LLC 2010 All Rights Reserved
Estimation Methods
• Formula• Activity-based• Analogy
Kolinger Associates, LLC 2010 All Rights Reserved
How to Estimate(Kapur “Flinch” Method)
$850B …. For PHASE 1 !
Kolinger Associates, LLC 2010 All Rights Reserved
Balancing the Estimate
Your Goal: Commitment according to capability!
Kolinger Associates, LLC 2010 All Rights Reserved
Project Difficulty vs. Management Capability
Project: Web-Enabled Business Information System
Kolinger Associates, LLC 2010 All Rights Reserved
Historical Data: Example of Estimating Maturity
Observed Limits of Estimate Accuracy
0.67x
0.8x1.0x
0.5x
0.25x
2x
4x
1.5x
1.25x
Initialproduct
definition
Approvedproduct
definition
Requirementsspecification
Productdesign
specification
Detaileddesign
specification
Productcomplete
Project cost(effort and size)
Projectschedule
0.85x
0.9x1.0x
0.8x
0.6x
1.25x
1.6x
1.15x
1.1x
6. C
ompl
etio
n
Underestimated
Overestimated
Idea / OP
10/1/02~ 392% Variance
FTE 4.4
H/L Requirements
9/1/03~ 317% Variance
FTE 4.4
Most Code Complete
3/1/04~ 33% Variance
FTE 13.4
- Effort / Budget
- Schedule
LegendProject Complete
8/2/04
FTE 17.9Requirements Complete *1
2/1/04~ 80%Variance
FTE 10.0Footnote: * 1. Independent Estimate at point of most requirements complete
+ 400 %
- 400%
Exact 0%
Effort Duration
Kolinger Associates, LLC 2010 All Rights Reserved
Estimation Accuracy Commercial Web System
An Example of Estimating [Im]Maturity
Project Milestone
OP / Idea (Forecast)
H/L ReqDone
(Forecast)
Most Code Complete (Forecast)
Project Complete (Actual)
Date of Estimate 10/1/02 9/1/03 3/1/04 8/1/04 FinalEffort Hours 5,600 unknown ~ 16,000 25,000Cost 402,000$ 475,000$ 1,484,429$ 1,979,238$ Cost Variance 392% 317% 33% 0%FTE 4.4 4.4 13.4 17.9FTE Variance 311% 311% 33% 0%
Kolinger Associates, LLC 2010 All Rights Reserved
Where did the money go?
How much waste? ~60% Re-work!
$560k $620k
$280k
$100k$20k
Pay to take bugs out.
Pay to put bugs in.
Kolinger Associates, LLC 2010 All Rights Reserved
Quality Results Defects Found Pre and Post Implementation
1. Defects Discovered in Testing (after “Code Freeze”):
Code defects: 977 (70%)Missing / Incorrect Requirements: 394 (30%)Total: 1,371
2. Defects Discovered by the User in Production:
First 90 days: 136
Kolinger Associates, LLC 2010 All Rights Reserved
Better Estimation via Better Control of Quality Comparing Defect Removal Results of Two Projects
Case A Case B
Activity (Dollars) (Person Hours) Defects (Dollars) (Person Hours) Defects
Top-Level Design Review $2,875 115 50 $0 0 0
Detailed Design Review $17,500 700 140 $0 0 0
Code Inspection
$42,500 1,700 110 $0 0 0
Unit Test $19,250 770 50 $38,000 1,500 125
Integration Test
$62,250 2,490 100 $107,205 4,290 190
Total $128,625 5,775 450 (90%) $145,250 5,810 315
Maintenance $325,000 13,000 50 $797,500 31,900 185
Lifecycle Total $453,625 18,755 500 $942,750 37,710 500
Reference: Capers Jones, SPR
Cheap
Expensive
Kolinger Associates, LLC 2010 All Rights Reserved
Question – Crash Schedule?
• What happens when we compress the schedule?
• Good to know when devising a plan that will work – and some exec asks for it sooner than you can deliver.
• You need to know the minimum development time.
Kolinger Associates, LLC 2010 All Rights Reserved
Crashing the Schedule
Kolinger Associates, LLC 2010 All Rights Reserved
Effects of Schedule Compression - 1
100 KLOC
11
$1.5 MM
23 months
2.7 days
Project Size:
Team Size:
Cost
Duration
MTTD:
Optimized Compressed
QSM
Before
Can we go faster with more
people?
What’s the effect of a 15% schedule reduction?
Kolinger Associates, LLC 2010 All Rights Reserved
Effects of Schedule Compression - 2
100 KLOC
11
$1.5 MM
23 months
2.7 days
Project Size:
Team Size:
Cost
Duration
MTTD:
Same
24
$2.7 MM
20 months
1 day
Optimized Compressed
QSM
AfterBeforeWhat’s the effect of a 15% schedule reduction?
Worth it? Really?
Kolinger Associates, LLC 2010 All Rights Reserved
Effects of Schedule Compression - 3 Quality Considerations
Def
ects
/ M
ista
kes
Time
23 Months20 Months
• An individual’s learning curve must be accounted for in a shorter schedule. • There is a minimum development time!
A B
Kolinger Associates, LLC 2010 All Rights Reserved
Software Development Effort - 9 ProjectsIntenseIntense Schedule PressureSchedule Pressure
CostSchedule
Kolinger Associates, LLC 2010 All Rights Reserved
Deadline Effect
Kolinger Associates, LLC 2010 All Rights Reserved
Improving Process Reduces Costs
For a typical software system, consisting of 500K Lines of Code:
Stan Rifkin, Master SystemsLarry Putnam, QSMRay Dion, Raytheon
SEILevel
IIIIIIIVV
Effort(P. Mos.)
16,3626,4681,876
866342
Defects(Number)
25,0699,9092,8741,326
524
Total Cost(Mill. $)
163.364.718.8
8.73.4
Significant savings as a result of achieving Level 2
Most of these improvements come from better project management!
Kolinger Associates, LLC 2010 All Rights Reserved
Right-sizing ... or Dumb-sizing?
Mark Blaxill, The Fallacy of the Quick Overhead Fix, July 1991 Harvard Business Review
Bureaucratic Robust
Two Competitors, Same Products, Same Sales Revenues, Different Cultures
Fixers63%
Process Improvement
13%
Core Management
24%Fixers
29%
Process Improvement40%
Core Management
31%
Total Employees = 152 Total Employees = 80
Kolinger Associates, LLC 2010 All Rights Reserved
Main Build – Estimated Duration
Kolinger Associates, LLC 2010 All Rights Reserved
Main Build – Estimated Duration
Kolinger Associates, LLC 2010 All Rights Reserved
Main Build Time - 9 Projects
ESLOC (Thousands)
Industry Average
Months D
urationHistorical DataHistorical Data
Kolinger Associates, LLC 2010 All Rights Reserved
Main Build Effort - 9 ProjectsPerson M
onths
ESLOC (Thousands)
Historical DataHistorical Data
Kolinger Associates, LLC 2010 All Rights Reserved
Project Staffing Comparisons
0.0
5.0
10.0
15.0
20.0
25.0
30.0
35.0
A B C D E F G H I J K
Projects from ABC Division
Benchmark Peak Staff
Company Peak StaffProjects from XYZ Division
61%
108%
77%
64% -11%
180%
-25%-8.3%
0%
0%0%
Percent by which this project exceeds the typical business systems average staffing
Kolinger Associates, LLC 2010 All Rights Reserved
A Business Case for Process Improvement at Company X
SEI
Level
Calendar Time
(Mos)
Effort
(PMos)
Defects Total Cost (1,000s)
Discovered Shipped Median Lowest Highest
I
II
III
IV
V
18
11
9
7.5
5
67
16
9
5
2.5
297
71
40
22
11
11
3
1
1
0
614
148
82
44
23
202
109
58
32
5
20901
196
105
57
31
50 KLOC, Productivity Index (PI)=15, MBI=0.2
Kolinger Associates, LLC 2010 All Rights Reserved
Benchmark Power
One company could save $7 million in a portfolio of 14 projects by making the right improvements in their management approach. In addition, this portfolio of projects would finish 3.5 months sooner and with a much higher reliability/quality. This remarkable analysis is made possible by using the high-performance benchmark database and SLIM toolset.
Kolinger Associates, LLC 2010 All Rights Reserved
Know your capability !!!
Commitments must be tempered with known capability!
Question:
• For your current project, where would you look for a mismatch of commitment and capability?
• How should you respond?
Project Difficulty Management Capability
Kolinger Associates, LLC 2010 All Rights Reserved
6. When You Have to Micro-ManagePeople - Place them where strengths are leveraged, and their weaknesses really don’t matter.
– People aren’t fungible – Consider:
• 85% feel they are in the wrong job • 10% are in their dream job• 95% confess they lie on the job
– Misplaced people will disrupt– Fear repels learning and improvement– Focus on developing strengths not fixing weaknesses– Company slogans and performance reviews demoralize
“It’s time for performance review!”
“Malicious Compliance” – be careful what you ask for! You might get it!
Kolinger Associates, LLC 2010 All Rights Reserved
Kolinger Associates, LLC 2010 All Rights Reserved
7. When You Hear Any Answer That Starts With, “Well …”
• A ‘perfect’ estimate is valid as long as you manage to the plan.• Avoid use of ‘percent complete.’
– Remember the “90:90 Rule of Project Tracking.”• The first 90% of the project takes 90% of the work.• The last 10% takes the other 90%!
• Do not be bamboozled in project reviews! Verify:– 1. Do we have the right people?– 2. Do we have a good plan?– 3. Are we going to make it?
Look for effective project reviews and the signs of being off-track
Kolinger Associates, LLC 2010 All Rights Reserved
“Relax – We’re 90 Percent Complete!”
Traditional software progress measures do not provide visibilitySoftware Engineering Economics, Boehm – pp 608
Kolinger Associates, LLC 2010 All Rights Reserved
7 Signs – A Review1. When You Hear the Word ‘Done’ 4 Times.So, what the heck does ‘done’ look like?
2. When You Hear, “What is the Plan?”Look for the plan with evidence of traceability to the project definition and make sure everyone is aware.
3. When You Have a Single Number EstimateLook for a schedule based on appropriate estimate ranges
4. When It Has Not Been Done BeforeLook for an estimate that factors in data of your organization’s capability
5. When You Can’t See Evidence of Historical DataLook for an estimate that uses historical performance data as a reference
6. When You Have to Micro-ManageGet the right people in the right place
7. When You Hear Any Answer That Starts With, “Well …” Look for effective project reviews and the signs of being off-track
Kolinger Associates, LLC 2010 All Rights Reserved
Conclusion
• Radical improvement is possible• Information is not enough• Needed:
1. Invest in knowing your capability2. Learn to make commitments according to capability3. An outside perspective may help
You must protect your company from a bad estimate
Kolinger Associates, LLC 2010 All Rights Reserved
Acknowledgements
• Gopal Kapur – Center for Project Management• Dr. Stan Rifkin – Master Systems• Dr. Robert N. Charette – ITABHI Corporation• Dr. Barry Boehm - USC Center for Software Engineering • Dr. Gerald Chester – Strategies@Work• Dr. Michael Deutch – Hughes Aerospace• Dr. Robert Graham - SMC• Tom DeMarco – Atlantic Systems Guild• Larry Putnam, Sr. – Quantitative Software Management• Dan Swaigen – PM Connect• Gordon Landies – GL Ventures• John Maher, CPA – Maher Accountancy• Dennis Peacocke – SCS• Alan Lashbrook – AT&T• Ron Linski – Alliance Business Consulting• Betsy Guthrie - Autodesk
My thanks to the following individuals who have made important contributions to the body of knowledge presented in this document.
Kolinger Associates, LLC 2010 All Rights Reserved
About Us
• Process assessment and development• Planning and Estimating Training• People Assessment• Rescue Off-Track Projects• Tools Training and Implementation• Project Reviews
Contact 415-246-7264 – [email protected] - www.kolinger.net
Kolinger Associates provides solutions and advice for better estimating and managing large projects. With our help you will …
Spend your money a little differently … and get a much better result.