Test Estimation in Practice

Post on 13-Dec-2014

130 views 1 download

description

Anyone who has ever attempted to estimate software testing effort realizes just how difficult the task can be. The number of factors that can affect the estimate is virtually unlimited. The key to good estimates is to understand the primary variables, compare them to known standards, and normalize the estimates based on their differences. This is easy to say but difficult to accomplish because estimates are frequently required even when very little is known about the project and what is known is constantly changing. Throw in a healthy dose of politics and a bit of wishful thinking and estimation can become a nightmare. Rob Sabourin provides a foundation for anyone who must estimate software testing work effort. Learn about the test team’s and tester’s roles in estimation and measurement, and how to estimate in the face of uncertainty. Analysts, developers, leads, test managers, testers, and QA personnel can all benefit from this tutorial.

Transcript of Test Estimation in Practice

TR Half-day Tutorials

5/6/2014 1:00:00 PM

Test Estimation in Practice

Presented by:

Rob Sabourin

AmiBug.com

Brought to you by:

340 Corporate Way, Suite 300, Orange Park, FL 32073

888-268-8770 ∙ 904-278-0524 ∙ sqeinfo@sqe.com ∙ www.sqe.com

Rob Sabourin AmiBug.com

Rob Sabourin, P. Eng., has more than thirty years of management experience leading teams of software development professionals. A well-respected member of the software engineering community, Rob has managed, trained, mentored, and coached hundreds of top professionals in the field. He frequently speaks at conferences and writes on software engineering, SQA, testing, management, and internationalization. Rob wrote I am a Bug!, the popular software testing children's book; works as an adjunct professor of software engineering at McGill University; and serves as the principle consultant (and president/janitor) of AmiBug.Com, Inc. Contact Rob at Contact Rob at rsabourin@amibug.com.

IntroductionTEST Estimation IN PRACTICE

Course timingCourse timing

Electronic

devices

Electronic

devices

Smoking

Smoking

MealsMeals

Facilities

Facilities

Breaks

Breaks

Administrivia

6© 2012 SQE Training V3.0

Introductions

Name

CompanyNameBusiness

EnvironmentHardwareSoftware

ExperienceTestingManagement

Personal Course Objective(s)

Name

CompanyNameBusiness

EnvironmentHardwareSoftware

ExperienceTestingManagement

Personal Course Objective(s)

7© 2012 SQE Training V3.0

Course Agenda

1. Estimation2. Estimation Techniques

8© 2012 SQE Training V3.0

1estimation

Estimation

Estimate: – A tentative evaluation or rough calculation – A preliminary calculation of the cost of a

project – A judgment based upon one’s impressions;

opinion —The American Heritage Dictionary

It is very difficult to make a vigorous, plausible, and job-risking estimate that is derived by no quantitative method, supported by little data and certified chiefly by hunches of the managers.

— Fred Brooks10© 2012 SQE Training V3.0

The best estimatesThe best estimates

Represent the collective wisdom of practitioners (testers, developers, users, etc.) and have their buy-in

Provide specific, detailed catalogs of the costs, resources, tasks, and people involved

Present, for each activity estimated, the most likely cost, effort, and duration

Estimation: the creation of an approximate target for costs and completion dates Estimation: the creation of an approximate target for costs and completion dates

Test Estimation

11© 2012 SQE Training V3.0

Factors that can influence cost, effort, and duration include:Factors that can influence cost, effort, and duration include:

Required level of quality of the system

Size of the system to be tested

Historical data

Process factors (process maturity, etc.)

Material factors (tools, data, etc.)

People factors (skills, experience, managers, etc.)

Test Estimation (cont.)

12© 2012 SQE Training V3.0

Test Estimation (cont.)

• Delivery of estimates should include justification

• Negotiation and re-work of estimates is normal

• Final estimates represent a balance of organizational and project goals in the areas of quality, schedule, budget, and features

13© 2012 SQE Training V3.0

How Good an Estimator Are You?Low Estimate High Estimate

___________ to ____________

.

.

.

© Steve McConnell, Software Estimation, Microsoft Press, 2006

Surface temperature of the sun

Latitude of ShanghaI

Area of the Asian continent

Birth year of Alexander the Great

Total value of US currency in 2004

Total volume of the Great Lakes

Worldwide box office receipts of Titanic

Total length of Pacific Ocean coastline

Heaviest blue whale ever recorded

# of books published in US since 1776

14© 2012 SQE Training V3.0

How Good an Estimator Are You?• Surface temperature of the sun 10,0000 F / 6,000 C

• Latitude of ShanghaI 31° 10′ N

• Area of the Asian continent 17,212,000 sq mi

• Birth year of Alexander the Great 356 BC

• Total value of US currency in 2004 $719.9 Billion

• Total volume of the Great Lakes 5,500 cu mi

• Worldwide box office receipts of Titanic $1.835 billion

• Total length of Pacific Ocean coastline 84,300 miles

• Heaviest blue whale ever recorded 380,000 pounds

• Number of books published in US since 1776 22 million

15© 2012 SQE Training V3.0

How Good Is Our Industry (at Estimating)?

• Tata: 62% of projects fail to meet schedule

49% have budget overruns• Moiokken and Jorgensen: 30-40%

overruns

16© 2012 SQE Training V3.0

Class Discussion

Why is estimating not done well?

Your top five reasons:

1) Too many variables____________________

2) ____________________________________

5) ____________________________________

6) ____________________________________

5) ____________________________________

17© 2012 SQE Training V3.0

Why Estimates Are Inaccurate ― Part I

• Lack of estimating experience• Lack of historical data on which to base estimates• Lack of systematic estimation process, sound techniques, or

models suited to the project• Failure to include essential activities and products within the

scope of the estimates• Unrealistic expectations or assumptions• Failure to recognize and address the uncertainty inherent in

project estimates

Practical Software Measurement

Addison-Wesley, 2001

18© 2012 SQE Training V3.0

Why Estimates Are Inaccurate ― Part II

• Lack of education and training• Confusing the target with the estimate• Hope-based planning• Inability to communicate and support estimates• Incomplete, changing, and creeping requirements• Quality surprises (test and re-fix)

—adapted from Linda M. Laird

The Limitations of Estimation

19© 2012 SQE Training V3.0

Bohem’s Cone of Uncertainty

20© 2012 SQE Training V3.0

NHC Track Forecast Cone

21© 2012 SQE Training V3.0

“Testing” Track Forecast Cone

Best ca

se

Expected case

Worst case

T i m e

Tas

ks(or why it is important to constantly re-estimate)

22© 2012 SQE Training V3.0

What would have to happen to deliver this in four weeks?What would have to happen to deliver this in four weeks?

What should the estimate have been?What should the estimate have been?

The Fantasy Factor

Weeks

Today

0 1 2 3 4 5 6 7 8 9

1st 3rd2nd

23© 2012 SQE Training V3.0

Time

Time

SizeSize

Quality

Quality

Resources

Resources

Estimation

1, 2, 3, or 4 Variables + Many Modifiers:

If it’s not variable, then it’s fixed.

24© 2012 SQE Training V3.0

Time vs. Resources

=

25© 2012 SQE Training V3.0

2Estimation Techniques

Test Estimation Techniques ― Examples

• Intuition and guess• Work-breakdown-structures• Three-point estimates• Company standards and norms• % of project effort or staffing• Industry averages and predictive models (e.g., FP, TPA )• Team estimation sessions

– Wideband Delphi– Story point sizing– Poker estimation– T-shirt sizing

28© 2012 SQE Training V3.0

What do intuition and guess imply?What do intuition and guess imply?

No formal estimation process

Based on the experience of the team member(s) usually without the benefit of recorded metrics

Intuition and Guess

29© 2012 SQE Training V3.0

When would we use this technique?When would we use this technique?

Experienced staff members

Little or no recorded metrics available

Great accuracy is not required

Same type of project we’ve done in the past

Or at the other extreme, we don’t have a clue (often called a WAG or SWAG)

Intuition and Guess (cont.)

30© 2012 SQE Training V3.0

Project LevelProject Level

Phase LevelPhase Level

Task LevelTask Level

Phase LevelPhase Level

Task LevelTask Level

------

Task LevelTask Level

------

------

------

Task LevelTask Level

------

Phase LevelPhase Level

Task LevelTask Level

Phase LevelPhase Level

Task LevelTask Level

Work Breakdown Structure

31© 2012 SQE Training V3.0

When would we use this technique?When would we use this technique?

When the test team lacks credibility with their estimates

When we are doing an estimate for a project that is dissimilar to projects we have done in the past

When the project is too large to get our “arms around it” (e.g., it is too big to easily see the big picture)

Work Breakdown Structure (cont.)

32© 2012 SQE Training V3.0

When would we not use this technique?When would we not use this technique?

When the project has lots of turbulence!

Work Breakdown Structure (cont.)

33© 2012 SQE Training V3.0

Approach:Approach:

Develop goal

Break goal into deliverables

Break deliverables into activities

Decompose work into small manageable components that are of sufficient detail to allow you to make estimates of time and/or resources

Work Breakdown Structure ― Top Down

34© 2012 SQE Training V3.0

Work Breakdown Structure (cont.)

Testing for Completeness

• Status of a task is measurable• Start and end events are clearly defined• Further breakdown makes no sense• Each activity has a deliverable• Time or resources for an activity can be estimated• Duration of activity is within acceptable limits• Each activity is independent• Each activity forms a unique package of work that could

be outsourced

35© 2012 SQE Training V3.0

Work Breakdown Structure (cont.)

• Level of detail guidelines:– No single activity or group of activities should

exceed 80 hours of effort*– No activity or group of activities should exceed

the reporting period (e.g., if the reporting period is weekly, no activity should exceed one week)

– “If it makes sense” rule: The duration of each activity must pass the common sense rule

36© 2012 SQE Training V3.0

Three–Point Estimates

• Three-point estimates is an analytical technique using three cost or duration estimates:– Optimistic– Most likely – Pessimistic

• This technique is applied to improve the accuracy of the estimates of cost or duration

• Three-point estimation techniques can be employed with other estimating techniques such as Delphi

37© 2012 SQE Training V3.0

% of Project Effort or Staffing

• Based on the premise that there is a predictable correlation between development effort/time and test effort/time

• The preferred/“dictated” method of estimation in some organizations

• Sometimes associated with developer/tester ratios

• One method recognized by the ISTQB

38© 2012 SQE Training V3.0

When Would We Use This Technique?When Would We Use This Technique?An organization is producing more or less the same type of product on an ongoing basis

The organization (development and testing) uses consistent practices and has a stable staff

A crude estimate is needed

It is mandated

It works

% of Project Effort or Staffing

39© 2012 SQE Training V3.0

Issues:Issues:

We are basing our estimates on development estimates which are probably suspect

Some products that are relatively easy to develop may be difficult to test and vice versa

This technique may not take into account test automation, etc.

This method may not take the stated quality goals into account

Test Estimates Based on Development Estimates

40© 2012 SQE Training V3.0

Class DiscussionWhat is the ideal developer/tester ratio?

= ?

41© 2012 SQE Training V3.0

McConnell Developer to Tester Ratio

Environment Relative Importance Ratio(Developer:Tester Ratio)

Business Systems 3:1 to 20:1

Shrink Wrap 1:1 to 5:1

Scientific Projects 5:1 to 20:1

Systems Projects 1:1 to 5:1

Safety Critical Software 5:1 to 1:2

Microsoft Windows 2000 1:2

NASA Space Shuttle Flight Control Software 1:10

42© 2012 SQE Training V3.0

Test Point Analysis

• A method with the possibility to perform a technology-independent measurement of the test size of an information system on the basis of a function point analysis, and to use this measurement as a basis for a productivity measurement, an estimate of the required resources, and project management

Software Testing: A Guide to the TMap ApproachMartin Pol, Ruud Teunissen, Erik van Veenendaal

43© 2012 SQE Training V3.0

Test Point Analysis (cont.)

For more details on TPA,look in this book.

44© 2012 SQE Training V3.0

Principles of Test Point Analysis

• TPA only considers measurable quality characteristics that fall within the range of system or acceptance testing

• TPA is (in principle) analyst independent• TPA depends on the availability of function points• Sufficient knowledge of the test team is a precondition• TPA estimates are based on the assumption that, on average,

one complete re-test will be conducted

45© 2012 SQE Training V3.0

Team Estimates

• Teams often produce better estimates than individuals (See The Wisdom of Crowds)

• Team estimates can facilitate buy-in and commitment from the entire team

• Almost all estimating methods can be done using teams

46© 2012 SQE Training V3.0

The Wisdom of Crowds

Four elements required to form a wise crowd:

• Diversity of opinion• Independence• Decentralization• Aggregation

47© 2012 SQE Training V3.0

Test Estimation Sessions ― Wideband Delphi

48© 2012 SQE Training V3.0

Test Estimation Sessions ― Wideband Delphi

• Developed by Barry Boehm and John Farquhar in the 1970s*

• A variant of the existing Delphi method but with more interaction

• Popularized in Boehm’s book Software Engineering Economics in 1981

• Similar to the Platinum Poker method used in some agile projects

49© 2012 SQE Training V3.0

When would we use this technique?When would we use this technique?

Estimation for a new product or technology

To develop an “order of magnitude” estimate

Requirements are shaky or incomplete

Multiple disciplines are involved

Wideband Delphi

50© 2012 SQE Training V3.0

Test Estimation Sessions — Wideband Delphi

Steps from Barry Boehm’s book:

1. Coordinator presents each expert with a specification and estimation form

2. Experts discuss issues in group meeting3. Experts fill out form anonymously4. Coordinator distributes a summary5. In a group meeting experts discuss variances6. Experts fill out forms again for as many rounds as

necessary

51© 2012 SQE Training V3.0

Wideband Delphi Process Flow

Assemble estimates

and assumptions

Estimation meeting

Individual preparation

Planning and kickoff

meeting

Re-estimate

Acceptable

Done

No Yes

Adapted from Karl Wiegers

52© 2012 SQE Training V3.0

Sample Estimation Form — Wideband Delphi

Task Estimate #1 Estimate #2 Estimate #3 Estimate #4 Final

Change

Total

53© 2012 SQE Training V3.0

Discussion Items:Discussion Items:

What are story points?● Measure of size/complexity, not time● Relative unit of measure

Is there any relation to function points or test points?

Estimating Using Story Points ― Agile

54© 2012 SQE Training V3.0

Estimating Using Story Points ― Agile

55© 2012 SQE Training V3.0

Estimating Using Story Points ― Agile

• Story points are used for long term or release planning and tracking

• Point-estimated stories along with team velocity can be used to provide rough release-level scheduling and project progress

• Since story points are relative size indicators, a two-point story is always twice as big as a one-point story

56© 2012 SQE Training V3.0

Why Use Story Points?

• Cheaper• Allow us to change our minds as new

information becomes available• Don’t take a lot of time• Foster collaboration• Consistent• Provide credibility• Can use Planning Poker Cards with story points

57© 2012 SQE Training V3.0

Planning Poker

58© 2012 SQE Training V3.0

Planning Poker

• Based on the Wideband Delphi method espoused by Barry Boehm and others in the 60s and 70s (and Boehm’s work was possibly based on earlier works dating to the 40s)

• Most commonly used in agile software development

• A study published by the IEEE in April 2007 indicated that Planning Poker achieved less optimistic and more accurate estimates than those obtained through mechanical combinations of individual estimates

59© 2012 SQE Training V3.0

Planning Poker Rules

1. Form a group of no more than ten estimators and a moderator. The product owner is usually present but cannot estimate

2. Each participant gets a deck of cards. The decks frequently use a doubling sequence (½, 1, 2, 4, 8, 16, 32…) or the Fibonacci sequence (1, 2, 3, 5, 8, 13, 21, 34, 55, 89, 144…) or a “modified” Fibonacci sequence such as (0, 1, 2, 3, 5, 8, 13, 20, 40, 100)

3. The moderator reads one user story at a time to the team

4. The product owner answers questions about the story

60© 2012 SQE Training V3.0

Planning Poker Rules (cont.)

5. Each estimator privately selects a card from his or her deck representing their estimate of the “size” of the user story

6. When everyone is ready, all of the cards are flipped over at the same time

7. If there is a consensus, the results are recorded, and the team moves on to the next story

8. If the estimates vary widely, the owners of the high and low estimates defend their positions to the rest of the team

61© 2012 SQE Training V3.0

Planning Poker Rules (cont.)

9. The group briefly debates the arguments

10. Repeat from step 5 until the estimates converge

11. Repeat for all stories

62© 2012 SQE Training V3.0

Planning Poker Rules ― Tips

1. Those who actually could do the work are the ones who vote

2. Managers don’t vote

3. When there is a tie in the voting between two sizes that are consecutive, just pick the larger size and move on

4. Stop implementation discussions before they go too deep

5. Use an “I need a break” card

63© 2012 SQE Training V3.0

Planning Poker Rules ― Tips (cont.)

6. Use a timer to limit discussions

7. If consensus cannot be reached by the end of the third round of voting, pick the largest size and move on

8. Have the person creating the user stories meet with QA and Development leads prior to playing poker to answer the most obvious questions

9. Remember the baseline

10.Have fun!

64© 2012 SQE Training V3.0

T-Shirt Sizing

An estimation technique, usually used in agile projects, that uses relative sizing based upon a limited number of values (e.g., t-shirt sizes): S, M, L, XL

Steps:– Make S, M, L, XL cards– Stories are read one at a time– Each developer gives each story a t-shirt size– All developers raise their cards simultaneously– Discuss differences– Go back to step 3– Compile the stories into size buckets– Estimate the time to complete a S, M, L,XL

65© 2012 SQE Training V3.0

Hadden’s Size/Complexity Technique

Rita HaddenSmall Medium Large

Simple

Moderate

Complex

Complexity

Size

66© 2012 SQE Training V3.0

Sample Estimate

67© 2012 SQE Training V3.0

Test Objective Size Complexity Effort

1 to 3 1 to 3 Sessions 1 2 3to001 2 1 4 1 1 4 8to002 2 3 16 2 4 8 16to003 2 3 16 3 8 16 32to004 2 1 4to005 3 1 8to006 2 2 8to007 3 2 16Typical Usage Scenarios 2 2 8

total 80 sessions20 days

Siz

e

Complexity

Simple Defect Estimation Models

• Defect Detection Percentage (DDP)• Historical Extrapolation• “Bug Budgets”

68© 2012 SQE Training V3.0

Karl Wiegers’s Estimation Safety Tips

• A goal is not an estimate• The estimate you produce should be unrelated

to what you think the requester wants to hear• The correct answer to any request for an

estimate is “Let me get back to you on that”• Avoid giving single point estimates• Incorporate contingency buffers into estimates

69© 2012 SQE Training V3.0

Rick Craig’s Tips for Better Estimates

• Do it!• Collect metrics• Remember the “fantasy” factor• Don’t “pad” your estimates*• Don’t spend a ton of time• Estimates don’t have to be perfect

– Estimates are just estimates– They will change/constantly as you re-estimate– Remember planning risks and contingencies– Remember Brooke’s Law

• If the date is fixed, estimate something else• Use tools• Use ranges of value instead of discrete numbers

70© 2012 SQE Training V3.0

Robert Sabourin’s Tips for Better Estimates

• Consider estimating the amount of test coverage possible given a fixed amount of testing effort

• Present stakeholders with alternatives (Plan-a,Plan-b,Plan-9)

• Use ranges of values• Highlight business organizational and

technical factors which impact the estimate

• Using used envelopes to present rough estimates

71© 2012 SQE Training V3.0

Thank You

• On behalf of SQE Training, we appreciate your attendance at this course

• If you have additional training or consulting needs, please think of us

73© 2012 SQE Training V3.0

AppendixInstructor Material

Brainstorming Testing Estimates

• Min Max Missing– Identify participants– Hold kick off meeting– Individuals build list– Logging meeting– Estimate depth of testing– Estimate size in sessions– Identify new, missing, or

redundant ideas

75© 2012 SQE Training V3.0

BrainstormingTesting Ideas

• Min Max Missing– PMI Estimation– Given three values

● MIN = BEST CASE● MAX = WORST CASE● EXPECTED = NORMAL CASE● Estimate = (MIN + 4*EXPECTED + MAX)/6

76© 2012 SQE Training V3.0

Effort Estimation

ReadingSteve McConnell, Rapid Development: Taming Wild

Software Schedules. Chapter 8

77© 2012 SQE Training V3.0

Estimating Software Projects• Duration (on time)

– Schedule time– Delivery

• Quality (on quality)– Defect density– Defect arrival

• Cost (on budget)– Money

● Fixed● Variable

– Opportunity● What else could we be doing?

78© 2012 SQE Training V3.0

Estimating Software Projects

• Effort– Work– Manpower

• Staffing– Team definition– Skills, experiences

79© 2012 SQE Training V3.0

Estimation Process

McConnell Basic Steps:

1. Estimate size2. Estimate effort3. Estimate schedule

80© 2012 SQE Training V3.0

Estimation Process

1 - Estimate Size

81© 2012 SQE Training V3.0

Estimation Size

• SLOC– Source lines of code

• FP– Function points– Synthetic measure of software size

82© 2012 SQE Training V3.0

Estimation Size

• Software Tools– Calibration base on relevant experience– Combine several methods– Estimator (free!)

● www.construx.com

83© 2012 SQE Training V3.0

Estimation Size

• Analogy– Similar past projects– Relative to past experience– Must take into account

● Team● Technology● Complexity● Development process maturity

84© 2012 SQE Training V3.0

Estimating Size

• Algorithmic– Function points– Given requirements determine the number and

complexity of● Inputs● Outputs● Inquiries● Logical files● External interfaces

– Calculate and adjust based on “type of project/technology”

85© 2012 SQE Training V3.0

Function Points

Low Complexity Medium Complexity High Complexity

Program Characteristic count multiplier count multiplier count multiplier total

Inputs 6 3 2 4 3 6 44

Outputs 7 4 7 5 0 7 63

Inquiries 0 3 2 4 4 6 32

Logical Internal Files 5 7 2 10 3 15 100

External Interface Files 9 5 0 7 2 10 65

Unadjusted Function Point Total 304

Influence Multiplier 1.15

Adjusted Function Point Total 349.6

86© 2012 SQE Training V3.0

Function Points ― Influence Factors

1 Data communications How many communication facilities are there to aid in the transfer or exchange of information with the application or system?

2 Distributed data processing How are distributed data and processing functions handled?

3 Performance Did the user require response time or throughput?

4 Heavily used configuration How heavily used is the current hardware platform where the application will be executed?

5 Transaction rate How frequently are transactions executed daily, weekly, monthly, etc.?

6 On-line data entry What percentage of the information is entered on-line?

7 End-user efficiency Was the application designed for end-user efficiency?

87© 2012 SQE Training V3.0

Function Points ― Influence Factors

8 On-line update How many ILFs are updated by on-Llne transaction?

9 Complex processing Does the application have extensive logical or mathematical processing?

10 Reusability Was the application developed to meet one or many user’s (users’) needs?

11 Installation ease How difficult is conversion and installation?

12 Operational ease How effective and/or automated are start-up, back up, and recovery procedures?

13 Multiple sites Was the application specifically designed, developed, and supported to be installed at multiple sites for multiple organizations?

14 Facilitate change Was the application specifically designed, developed, and supported to facilitate change?

88© 2012 SQE Training V3.0

Function Points ― Influence Factors Score

Score Influence

0 Not present, or no influence

1 Incidental influence

2 Moderate influence

3 Average influence

4 Significant influence

5 Strong influence throughout

89© 2012 SQE Training V3.0

Function Points Influence Factors Score

INF = 0.65 + SCORE/100

90© 2012 SQE Training V3.0

Function Points ― Input

• Input– Number of screens, forms, dialogues, controls, or

messages through which an end user or another program adds, deletes, or changes data

91© 2012 SQE Training V3.0

Function Points ― Output

• Output– Screens, reports, graphs, or messages generated for

use by end users or other programs

92© 2012 SQE Training V3.0

Function Points ― Inquiries

• Inquiries– Direct access to data in database

93© 2012 SQE Training V3.0

Function Points ― Logical Files

• Logical Files– Major groups of end user data; could be a “file” or

“database table”

94© 2012 SQE Training V3.0

Function Points ― External Interfaces

• External Interface Files– Files controlled by other applications with which the

program interacts

95© 2012 SQE Training V3.0

Function Points ― Advantages

• Advantages include– Technology independent– Strong relationship to effort– Commonly used

• Disadvantages include– Requires specialized training to

compute

96© 2012 SQE Training V3.0

Estimation Process

2 - Estimate Effort

97© 2012 SQE Training V3.0

Effort Estimate

• Jones’s First-Order Estimation Practice

• Effort = (FP count) exponent

98© 2012 SQE Training V3.0

Jones’s First Order Exponents

Organization Capability

Kind of Software Best Average Worst

Systems 0.43 0.45 0.48

Business 0.41 0.43 0.46

Shrink Wrap 0.39 0.42 0.45

99© 2012 SQE Training V3.0

Estimation Process

3 - Estimate Schedule

100© 2012 SQE Training V3.0

Estimating Schedule

• Rule of Thumb (McConnell 8-1)

• Schedule = 3.0 * (effort) 1/3

101© 2012 SQE Training V3.0

Estimating Schedule

• Software Engineering Past Data– Example McConnell table 8-8, 8-9

● LOC vs. Schedule and Effort● Different project types

102© 2012 SQE Training V3.0

Estimation Convergence

Effort Range Schedule Range

Project Phase Optimistic Pessimistic Optimistic Pessimistic

Initial product definition 0.25 4.00 0.60 1.60

Approved product definition 0.50 2.00 0.80 1.25

Requirement specification 0.67 1.50 0.85 1.15

Product design specification 0.80 1.25 0.90 1.10

Detailed design specification 0.90 1.10 0.95 1.05

Product complete 1.00 1.00 1.00 1.00

103© 2012 SQE Training V3.0

Initialproductdefinition

Approvedproductdefinition

Requirementsspecification

Productdesignspecification

Detaileddesignspecification

Productcomplete

1.0×

0.25×

0.5×

1.5×

0.67×

1.25×

0.8×1.0×

0.6×

1.6×

1.25×

0.8×

1.15×

0.85×

1.1×

0.9×

Project Cost (effort and size) Project Schedule

Estimate ― Convergence Graph

104© 2012 SQE Training V3.0

More Relationships

Code Volume Approx. 100 LOC/FP, varies widely [Pressman, p. 94]

Schedule (FP)0.4Calendar months

Staffing (FP)/100Average – follows Raleigh curve

Effort Schedule * Staffing = (FP1.4)/150

105© 2012 SQE Training V3.0

Rules of Thumb

106© 2012 SQE Training V3.0

Rules of Thumb (cont.)

• Experience from several projects indicates that the amount of testing required in a project is proportional to the amount of development required

• Ratio depends on

– History– Organization, culture, structure– Projects– Teams

107© 2012 SQE Training V3.0

Rules of Thumb (cont.)

• Hadden’s Size/Complexity Technique

Small Medium Large

Simple

Moderate

Complex

Complexity

Size

108© 2012 SQE Training V3.0

Developer/Tester Ratios

109© 2012 SQE Training V3.0

Sample Estimate Project A

110© 2012 SQE Training V3.0

Test Objective Size Complexity Effort

1 to 3 1 to 3 Sessions 1 2 3to001 2 1 4 1 1 4 8to002 2 3 16 2 4 8 16to003 2 3 16 3 8 16 32to004 2 1 4to005 3 1 8to006 2 2 8to007 3 2 16Typical Usage Scenarios 2 2 8

total 80 sessions20 days

Siz

e

Complexity

Sample Estimate Project B

111© 2012 SQE Training V3.0

Test Objective Size Complexity Effort1 to 3 1 to 3 Sessions 1 2 3

to001 1 1 1 1 1 4 8to002 1 1 1 2 4 8 16to003 2 1 4 3 8 16 32to004 2 1 4to005 2 1 4to006 2 1 4to007 1 1 1to008 1 1 1to009 3 2 16to010 2 2 8to011 2 2 8to012 2 2 8to013 2 2 8to014 3 2 16to015 3 2 16to016 2 2 8Typical Usage Scenarios 2 2 8

total 116 sessions29 days

Complexity

Siz

e

Sample Estimate Project C

112© 2012 SQE Training V3.0

Test Objective Size Complexity Effort1 to 3 1 to 3 Sessions 1 2 3

to001 3 2 16 1 1 4 8to002 1 1 1 2 4 8 16to003 1 1 1 3 8 16 32to004 1 1 1to005 1 1 1to006 1 1 1to007 1 2 4to008 1 2 4to009 1 1 1to010 1 2 4to011 1 2 4to012 2 2 8to013 2 2 8to014 2 2 8to015 1 1 1to016 1 1 1to017 2 1 2to018 2 2 8to019 3 2 16Typical Usage Scenarios 2 2 8

total 98 sessions24.5 days

Complexity

Siz

e

Sample Estimate Project D

113© 2012 SQE Training V3.0

Test Objective Size Complexity Effort1 to 3 1 to 3 Sessions 1 2 3

t001 1 1 1 1 1 4 8t002 1 1 1 2 4 8 16t003 1 1 1 3 8 16 32t004 2 1 4t005 2 1 4t006 2 2 8t007 1 1 1t008 1 1 1t009 1 1 1t010 2 1 4t011 2 2 8t012 1 1 1t013 1 2 4t014 1 2 4t015 1 2 4t016 1 1 1t017 2 1 4t018 1 1 1t019 1 2 4t020 2 1 4t021 1 1 1t022 1 1 1t023 2 2 4t024 1 1 1t025 2 1 4t026 1 1 1t027 1 1 1t028 2 1 4t029 1 2 4t030 1 2 4t031 2 1 4t032 1 1 1Scenarios 2 2 8

total 99 sessions24.75 days

Complexity

Siz

e