Post on 01-Jan-2016
© ISBSG International Software Benchmarking Standards GroupInternational Software Benchmarking Standards Group
Practical Software Project Estimation
Black Art or Science?
A Workshop
© ISBSG International Software Benchmarking Standards GroupInternational Software Benchmarking Standards Group
Acknowledgements
Tony Rollo SMS - UK
Terry Wright Multi Media Vic
Michael Stringer SAGE Technology
David Cleary Charismatek
Software Engineering Australia
© ISBSG International Software Benchmarking Standards GroupInternational Software Benchmarking Standards Group
Definitions
• ISBSG = International Software Benchmarking Standards Group
• FP = Function Point
• PDR = Project Delivery Rate
= Hours per FP
• PWE = Project Work Effort
© ISBSG International Software Benchmarking Standards GroupInternational Software Benchmarking Standards Group
What we will cover
• Introduction - the track record
• Factors affecting productivity
• Estimates – how accurate• Using a project history repository for:
– Measuring
– Estimating
Plus:• ISBSG Summary
© ISBSG International Software Benchmarking Standards GroupInternational Software Benchmarking Standards Group
What is our software delivery track record?
• Only 26% of software projects are successfulOnly 26% of software projects are successful
• 74% ended up in varying degrees of trouble74% ended up in varying degrees of trouble
• 32% of projects are terminated before delivering anything (average over-run = 87%)
• Impact of poor estimates:Impact of poor estimates:
– Missed delivery dates – loss of businessMissed delivery dates – loss of business
– Resources wasted on failing projects Resources wasted on failing projects
– Cancelled projects – money spent – no business valueCancelled projects – money spent – no business value
– Business case for IT investment invalidated Business case for IT investment invalidated
© ISBSG International Software Benchmarking Standards GroupInternational Software Benchmarking Standards Group
74% failures
26% of software 26% of software projects are successfulprojects are successful
Projects estimated using formal toolsProjects estimated using formal toolsand methods are twice as likely to succeedand methods are twice as likely to succeed
The Cost of FailureThe Cost of Failure
Impact:Impact:
Late delivery = Late delivery = loss of businessloss of business
CancelledCancelled project =project =
Failing project Failing project = lost opportunity= lost opportunity
Excess cost =Excess cost =Invalid investmentInvalid investment
© ISBSG International Software Benchmarking Standards GroupInternational Software Benchmarking Standards Group
Other Industries can measure
• Building Construction Industry – Metric = Cost per Metre of floor space to build
• Building Engineer’s estimate– Building Function = Office Accom.
– Location = CBD
– Height = 20 floors
– Cost per Square Metre = $1400 to $2300
– Variables
• Theatrettes, Gym, computer rooms, facade
• Quality
• BUT only 2:1 price variation (in SE we are 10:1)
© ISBSG International Software Benchmarking Standards GroupInternational Software Benchmarking Standards Group
All ‘mature’ industries can measure their productivity
…..why can’t we do it in software engineering?
© ISBSG International Software Benchmarking Standards GroupInternational Software Benchmarking Standards Group
Estimation ApproachesMacro estimation
• Equation Use – useful early ball parkEquation Use – useful early ball park– Based on a depth of historical dataBased on a depth of historical data– Imprecise for accurate estimatesImprecise for accurate estimates
• Comparison – provides a good guideComparison – provides a good guide– Based on representative experienceBased on representative experience– Experience MUST be relevantExperience MUST be relevant
• Analogy – objective, repeatableAnalogy – objective, repeatable– Based on past project attributesBased on past project attributes– Difficulty finding suitable past projectsDifficulty finding suitable past projects
© ISBSG International Software Benchmarking Standards GroupInternational Software Benchmarking Standards Group
Estimation ApproachesMicro- Estimation
Detailed and specific to the projectDetailed and specific to the projectMay overlook activities or itemsMay overlook activities or items
Work BreakdownProject broken downProject broken downinto components or tasksinto components or tasks
Each component or taskEach component or taskis separately estimatedis separately estimated
Results are aggregated to produce Results are aggregated to produce an estimate of the wholean estimate of the whole
© ISBSG International Software Benchmarking Standards GroupInternational Software Benchmarking Standards Group
Important Notes
• A formal project risk assessment is essential prior to estimating
• Always apply your own knowledge and experience to adjust estimates
© ISBSG International Software Benchmarking Standards GroupInternational Software Benchmarking Standards Group
Do’s and Don’tsNever rely on a single estimateNever rely on a single estimate
Use cross and sanity checksUse cross and sanity checks
Never give a single estimateNever give a single estimateGive most likely, least and greatestGive most likely, least and greatest
If Macro and Micro estimates vary:If Macro and Micro estimates vary:by more than 10-15%by more than 10-15%identify why and rework the estimatesidentify why and rework the estimates
© ISBSG International Software Benchmarking Standards GroupInternational Software Benchmarking Standards Group
Factors affecting ProductivityBased upon Statistical AnalysisBased upon Statistical Analysis• The most influential Factors are:The most influential Factors are:
– Primary Programming LanguagePrimary Programming Language– Platform MF, MR, PCPlatform MF, MR, PC– Team SizeTeam Size
• For detailed analysis – ISBSG For detailed analysis – ISBSG Benchmark R6Benchmark R6
• Consider these most influential Consider these most influential factors when estimatingfactors when estimating
Factors
© ISBSG International Software Benchmarking Standards GroupInternational Software Benchmarking Standards Group
Computer LanguagesOne analysis provided by ISBSG is the productivity of individual languages. It is useful to look at them with regard to the platform type
Mainframe languages tend to have a wide range of productivity figures
Mid range languages usually have a narrowrange of productivity figures
PC languages usually exhibit the highest productivity though for small projects
© ISBSG International Software Benchmarking Standards GroupInternational Software Benchmarking Standards Group
Team SizeA software project is a team activity
Team work is an important aspect
Large teams are more difficultto manage and communication is usually more time consuming
Teams of less than five give higher productivity
© ISBSG International Software Benchmarking Standards GroupInternational Software Benchmarking Standards Group
Accuracy of Estimates
The most useful estimates are Effort & Duration
ISBSG Data shows:
5
50
Met both estimates
Met one estimate
Met neither
Good estimates tended to be for smaller projects with short durations
Poor estimates for large projects, for new development and for client server,mid range, portable projects, lots of users
© ISBSG International Software Benchmarking Standards GroupInternational Software Benchmarking Standards Group
Estimation AccuracyEstimates based on Function Point sizing are producing the most accurate estimates
for delivery date, effort & cost
25
2550Use Work
Breakdown only
Use FunctionPoints only
Use Both Techniques
Estimation tools are used in about 10%
Management directivesets delivery in 15%
© ISBSG International Software Benchmarking Standards GroupInternational Software Benchmarking Standards Group
Accurate Estimation
Projects using only workBreakdown
Projects using both techniques
Severely underestimate cost
Get it right or overestimate slightly
Conclusion?
© ISBSG International Software Benchmarking Standards GroupInternational Software Benchmarking Standards Group
Project Size
Macro-estimation techniques require the
functional size of the proposed project.
© ISBSG International Software Benchmarking Standards GroupInternational Software Benchmarking Standards Group
Panic!
• I don’t count function points, should I leave now?
• No!
• You can “cheat” by using the known relationships between FP components
• If you have a logical data model, you can derive an estimated size
© ISBSG International Software Benchmarking Standards GroupInternational Software Benchmarking Standards Group
FP Breakdown
Input FPs33.5%
Output FPs23.5%
Enquiry FPs16.0%
File FPs22.1%
Interface FPs5.0%
New Developments
© ISBSG International Software Benchmarking Standards GroupInternational Software Benchmarking Standards Group
Using these component relationships
Use the number of Logical Files or the number of Inputs as the base for estimating size.
Outputs, Queries and Interface Files are too variable to use early in the lifecycle.
© ISBSG International Software Benchmarking Standards GroupInternational Software Benchmarking Standards Group
Estimating Size• 15 internal logical files (from a logical data
model)• 7.4 function points per logical file (median
from ISBSG)• 15 x 7.4 = 111 function points• Logical files ~ 22.1% of a project (from
ISBSG chart)
• 111FPs x 100/22.1 = 502FP Project Size
© ISBSG International Software Benchmarking Standards GroupInternational Software Benchmarking Standards Group
The Rule of the “Thirties”
Various organisations have come up with a rule of thumb of “thirty something” logical files equalling one unadjusted function point of total project size. The range is between 31 to 35. This lines up with our example: 15 x 34 = 510FP project size
© ISBSG International Software Benchmarking Standards GroupInternational Software Benchmarking Standards Group
Project History DataProductivity Rate is best derived from your own projects because of the large numberof attributes which influence productivity
Use ISBSG data for project history:
Industry wide data is useful when you have no relevant ‘experience’
Or as a Sanity check
© ISBSG International Software Benchmarking Standards GroupInternational Software Benchmarking Standards Group
ISBSG Software Project Database
• Project data for >1,200 projects
• Probably represents top 20% of industry
• Primarily MIS Applications• Data from 20 countries• All (but 5) since 1990, 50%
since 1998
© ISBSG International Software Benchmarking Standards GroupInternational Software Benchmarking Standards Group
Sanity ChecksOf Completeness: Low High Project
ILF 20% 24%
EIF 3% 12%
EI 26% 39%
EO 22% 24%
EQ 12% 14% 14%10%
30%
42%4%
Is anythingmissing?
Of the estimate
Is your project estimatevery different from ISBSG data for similarprojects ?
See the Benchmark Release 6
© ISBSG International Software Benchmarking Standards GroupInternational Software Benchmarking Standards Group
Estimation Using Equations
• Used to produce an initial ‘ball park’
• Based on regression equations• ISBSG Practical Project Estimation toolkit
– Has details in appendix
– Basic spreadsheet regression tool
• ISBSG Release 7 CD has an estimation tool
© ISBSG International Software Benchmarking Standards GroupInternational Software Benchmarking Standards Group
Using equations
• Regression equation tables are available for:– Productivity (person hours per function point)– Effort (person hours)– Duration (elapsed hours)– Speed of delivery (function points delivered per
elapsed calendar month)
© ISBSG International Software Benchmarking Standards GroupInternational Software Benchmarking Standards Group
Estimation - Equation approach
• Establish the project size
• Establish key attributes (eg. language, platform, team size)
• Select the formula
• Look up the equation tables
• Select the appropriate table values
• Do the calculation
© ISBSG International Software Benchmarking Standards GroupInternational Software Benchmarking Standards Group
Equation example
Ok Let’s have a go:
• Software project:
– Mainframe platform
– Size of 460 function points
© ISBSG International Software Benchmarking Standards GroupInternational Software Benchmarking Standards Group
Equation example - using tables
• Project Delivery Rate PDRRE =
14.35 Size –0.072 = 14.35 460 –0.072 = 9.2 hours per function point
• Project Work Effort PWERE =
14.35 Size 0.928 = 14.35 460 0.928 = 4,245.1 hours
© ISBSG International Software Benchmarking Standards GroupInternational Software Benchmarking Standards Group
Equation example - using ISBSG tools
Two tools available:– Simple spreadsheet regression tool (ISBSG
Tool Kit)
– Reality Checker (ISBSG Data CD R7)
© ISBSG International Software Benchmarking Standards GroupInternational Software Benchmarking Standards Group
Warning !
• It is very important to treat estimates obtained from regression equations as “ball park” only.
© ISBSG International Software Benchmarking Standards GroupInternational Software Benchmarking Standards Group
Estimation - Using comparison
• Comparison based estimation involves selecting a group of similar completed projects from a project database, then using the average of the median of the effort values.
© ISBSG International Software Benchmarking Standards GroupInternational Software Benchmarking Standards Group
Estimation Using ComparisonComparison based estimation:• Define the platform and identify that subset of
ISBSG data • Define the other attributes – language, team size
etc.• For each attribute obtain the median value • Determine the average of the medians for PDR
and delivery rate• Calculate the effort and duration• The result is your estimate
© ISBSG International Software Benchmarking Standards GroupInternational Software Benchmarking Standards Group
Example table - comparison estimation
Attribute Median Median
PDR delivery
speed
hrs/FP FPperMth
Language -Cobol 15.2 51.2
Business type Financial 8.9 44.5
© ISBSG International Software Benchmarking Standards GroupInternational Software Benchmarking Standards Group
Example comparison estimate
• Project Delivery Rate PDRAC = average of category median project delivery rates = 12 hours per fp
• Project Work Effort PWEAC =
PDRAC Functional Size= 12 460 =
5,520 hours
© ISBSG International Software Benchmarking Standards GroupInternational Software Benchmarking Standards Group
Example comparison estimate using ISBSG tool
• Use spreadsheet comparison tool on the ISBSG Toolkit CD
© ISBSG International Software Benchmarking Standards GroupInternational Software Benchmarking Standards Group
Using analogy
• Analogy estimation involves selecting from a project database, one completed project that closely matches the attributes of your planned project. Then use this ‘analogue’ as the basis of your estimate.
© ISBSG International Software Benchmarking Standards GroupInternational Software Benchmarking Standards Group
Analogy process
• Establish the attributes of your planned project• Search a repository of completed projects
If a suitable analogue is available:• Use the effort from the analogue as your base• Compare each attribute and adjust accordingly
© ISBSG International Software Benchmarking Standards GroupInternational Software Benchmarking Standards Group
Estimation by Analogy
Using Project Attributes:– Find a project that matches
If there is no project:– Eliminate one or more attributes as required
If there are several projects:– Then add more attributes if known– Or use averages of the multiple projects
Estimate from the projects attributes: – Speed of delivery, effort, duration etc
© ISBSG International Software Benchmarking Standards GroupInternational Software Benchmarking Standards Group
Simple example
New Project’s attributes:
• Size 540FP (filter on 250-1000)
• Mid Range
• New Development
• MIS
• 3GL
• C++ (Primary Programming Language)
© ISBSG International Software Benchmarking Standards GroupInternational Software Benchmarking Standards Group
Analogue Example Results
A project with the following values:
• 391FP
• 5,793hrs
• 14.8 hours per FP
• 13 Months
• 30.1 function points per month
© ISBSG International Software Benchmarking Standards GroupInternational Software Benchmarking Standards Group
Estimating from the chosen Analogue
Take the actual size of the new project: 540FPs
X the Analogue’s PDR of 14.8hrs per FP
= 7,992hrs project work effort
divide by the Analogue’s speed of delivery: 30.1FPs per month
= 17.9 months duration
© ISBSG International Software Benchmarking Standards GroupInternational Software Benchmarking Standards Group
Caution !
• The best analogues will normally be found in your own project repository
• The fewer the shared attributes between the analogue project and the target project, the more careful you must be.
© ISBSG International Software Benchmarking Standards GroupInternational Software Benchmarking Standards Group
Caution !
• Better project estimates are obtained by using a combination of work breakdown and macro-estimation techniques.
• There is no “silver bullet” for project estimation. You must apply your knowledge and experience to fine tune any estimate derived using macro-estimation.
© ISBSG International Software Benchmarking Standards GroupInternational Software Benchmarking Standards Group
The ISBSG Repository helps with Project Estimating
• Good estimating requires a measurement history.• Some teams have no history.• Many teams are going into new areas. • ISBSG data provides a metrics history.• ISBSG provides regression equations• ISBSG provides project delivery rate tables• Data can be used to build estimating frameworks.
© ISBSG International Software Benchmarking Standards GroupInternational Software Benchmarking Standards Group
Estimating Summary
• There are good techniques and tools available for software estimation
• Don’t rely on one approach or technique
• Always do a risk assessment first
• Always apply your own knowledge and experience
© ISBSG International Software Benchmarking Standards GroupInternational Software Benchmarking Standards Group
Summary - ISBSG Strengths
• Not profit motivated
• Provides Project Benchmarking
• Allows direct access to the data
• Allows networking
• Broad representation of IT– technologies, organisation types, geography
• International based standard
© ISBSG International Software Benchmarking Standards GroupInternational Software Benchmarking Standards Group
Your Choice
Black Art
or
Science ?