LEADERSHIP

ESTIMATION

BENCHMARKING

SCOPE MANAGEMENT

1

Triangle benchmarking in practice

IT Confidence 2015 Florence,

19.10.2015

Pekka Forselius

LEADERSHIP

ESTIMATION

BENCHMARKING

SCOPE MANAGEMENT

Definitions related to benchmarking

benchmark: ”something that is used as a standard by

which other things can be judged or measured”

Source: Longman Dictionary of

contemporary English

2

benchmark: ”reference point against which

comparisons can be made” Source: ISO/IEC 29155-1 IT project

performance benchmarking framework

benchmarking: ”activity of comparing

objects of interest to each other or

against a benchmark to evaluate

charasteristic(s)” Source: ISO/IEC 29155-1

LEADERSHIP

ESTIMATION

BENCHMARKING

SCOPE MANAGEMENT

But before practice …

3

a brief look at background and

basic elements of Triangle

benchmarking

LEADERSHIP

ESTIMATION

BENCHMARKING

SCOPE MANAGEMENT

Project Management - What?

project management

consists of the ten

knowledge areas (source:

Project Management Body

of Knowledge, PMI)

… a project manager must

manage all ten areas at

same time

evaluating success is

complex and often (too)

challenging

4

LEADERSHIP

ESTIMATION

BENCHMARKING

SCOPE MANAGEMENT

A simplified approach

Actually, when we try to

manage a project to

generally agreed

success, we are allowed

to concentrate on the

three most important

knowledge areas (source: all

articles about ”Project Management

Triangle” and ”Iron Triangle”)

5

LEADERSHIP

ESTIMATION

BENCHMARKING

SCOPE MANAGEMENT

Project Management Triangle

6

Everybody

”KNOWS”

Project

Management

triangle, but

unfortunately

very few ICT

decision

makers

understand

its nature

correctly!

LEADERSHIP

ESTIMATION

BENCHMARKING

SCOPE MANAGEMENT

Observations about PM triangle

”If you can’t measure it, you cannot manage it!” –

For management purposes all three dimensions

need to be measured, and the measurement

principles shall be defined. Examples:

Cost = Supplier’s development cost from requirements

specification to ready to install.

Time = Duration of development in months, from the

same activities as above.

Scope = Quantity or size of the outcomes.

Shape and size of the triangle are not constant!

7

LEADERSHIP

ESTIMATION

BENCHMARKING

SCOPE MANAGEMENT

A new look at the PM triangle

Everybody wants more

outcomes with same

cost and time, i.e.

”higher triangle”, but

how to get it?

What are the important

elements of a triangle?

How to influence the

shape of a triangle?

8

LEADERSHIP

ESTIMATION

BENCHMARKING

SCOPE MANAGEMENT

Elements of a PM triangle

9

LEADERSHIP

ESTIMATION

BENCHMARKING

SCOPE MANAGEMENT

Examples of IT PM triangles

10

LEADERSHIP

ESTIMATION

BENCHMARKING

SCOPE MANAGEMENT

Business case, starting point

Early requirements from a feasibility

study, including business processes

Stakeholder analysis, including users

Scope statement, system overview

picture

Will another system be replaced?

Investment calculations, including

numbers of transactions, volumes, etc.

Rough budget and schedule 11

LEADERSHIP

ESTIMATION

BENCHMARKING

SCOPE MANAGEMENT

Amount of outcomes

Size of the software, estimated and

measured in Function Points

Methods: all ISO/IEC FSM standards,

e.g. FiSMA method (ISO/IEC

29881:2010) at all accuracy levels

Tools: FiSMA 1.1 Size Estimator and

Experience® Service

An independent Scope Manager

recommended (not necessary to be external)

12

LEADERSHIP

ESTIMATION

BENCHMARKING

SCOPE MANAGEMENT

Focus on quality too

Quality requirements MUST be

connected to functional requirements

Method: specify first the requirements

for the entire IT system, then for the

business processes, and for the lower

level FUR if needed.

Tools: e.g. FiSMA Quality

Requirements Analysis, ISO/IEC 9126

and 25010 (Software product quality

standards)

13

LEADERSHIP

ESTIMATION

BENCHMARKING

SCOPE MANAGEMENT

Capability of Developer Team

The better the capability of developer team, the

less re-work needed, and the more competitive

price they can propose!

Availability of resources and maturity of

development process are important!

The capability level depends also on skills and

experience of the developers, i.e. how well they

can:

• Read, question, communicate, and understand the functional

requirements

• Design the functions

• Provide the program code

• Test the outcome units and integrated components

• Prepare the installation of software.

14

NOTE! The

required developer

skills are all related

to SDLC –standard

(ISO/IEC 12207)

LEADERSHIP

ESTIMATION

BENCHMARKING

SCOPE MANAGEMENT

Capability of Product Owner

The better the capability of product owner, the

faster the delivery of software!

Availability of resources and maturity of

requirements management process are

important!

The capability level of product owner also

depends on how clearly its team can:

• Recognize and define all users

• Write all necessary user stories

• Specify terms and define ER model

• Draw business process charts

• Write and update use cases

• Define functional requirements

• Specify quality requirements

15

NOTE! The skills

required from the

product owner have

NOTHING to do with

management approach

(agile or not).

LEADERSHIP

ESTIMATION

BENCHMARKING

SCOPE MANAGEMENT

The most important metrics

To evaluate first the

reality of project plans,

and in the end the

success of the project

we need to measure:

Delivery speed =

h/b (FPs/month)

Unit price = a/h

(€/FP)

16

LEADERSHIP

ESTIMATION

BENCHMARKING

SCOPE MANAGEMENT

From theory to practice …

17

what data is needed, how to draw

a triangle from data elements, and

what kind of IT projects can be

compared by Triangle

benchmarking?

LEADERSHIP

ESTIMATION

BENCHMARKING

SCOPE MANAGEMENT

What data to collect?

18

• Project ID (+ connections to program and portfolio if needed)

• Basic classifiers

• Software size

• Total effort and/or cost

• Start date

• End date

• (Duration can be calculated based on the dates)

LEADERSHIP

ESTIMATION

BENCHMARKING

SCOPE MANAGEMENT

How to draw a standard triangle

1 unit = 200 FP

1 unit = 100 K€ = 1000 h

1 unit = 3 months

19

LEADERSHIP

ESTIMATION

BENCHMARKING

SCOPE MANAGEMENT

Be careful with your presentation!

20

LEADERSHIP

ESTIMATION

BENCHMARKING

SCOPE MANAGEMENT

Color coding for Triangle benchmarking

21

Traffic light analogy:

• Red is bad

• Yellow is normal

• Green is good

All combinations are possible

LEADERSHIP

ESTIMATION

BENCHMARKING

SCOPE MANAGEMENT

Scalability issues of Triangle benchmarking

The method is applicable

for comparing:

Single iterations (e.g.

retrospectives)

IT projects

IT development programs

The examples on the next

slides show that the

triangles work at all levels

22

Input for retrospectives, for

evaluating the iterations:

LEADERSHIP

ESTIMATION

BENCHMARKING

SCOPE MANAGEMENT

Example 1: ES40 UI+BL/4SUM Partners

23

LEADERSHIP

ESTIMATION

BENCHMARKING

SCOPE MANAGEMENT

Example 1: ES40 UI+BL program

24

LEADERSHIP

ESTIMATION

BENCHMARKING

SCOPE MANAGEMENT

Example 2: Valtimo/MoSH subprogram RTK

25

Six iterations:

LEADERSHIP

ESTIMATION

BENCHMARKING

SCOPE MANAGEMENT

Example 2: Valtimo/MoSH 8 subprograms

26

LEADERSHIP

ESTIMATION

BENCHMARKING

SCOPE MANAGEMENT

Example 2: Valtimo/MoSH program

27

LEADERSHIP

ESTIMATION

BENCHMARKING

SCOPE MANAGEMENT

Conclusions from Triangle benchmarking results

If you see red, don’t cry!

If you see green, don’t laugh!

Whatever you see, don’t become proud or ashamed!

IF you see red, pay all attention to triangle slopes:

you can only improve your capability (i.e. your skills,

availability and tools). Product owner can improve

the way to specify and manage requirements and

developer team can improve keeping their focus on

the user needs.

28

LEADERSHIP

ESTIMATION

BENCHMARKING

SCOPE MANAGEMENT

Thank you!

Pekka Forselius, MSc, MBA,

Certified Scope Manager, Immediate Past President of ISBSG, Senior Advisor at FiSMA

email: pekka.forselius@4sumpartners.com

see also www.4sumpartners.com, www.fisma.fi and www.isbsg.org

29