A balanced metrics set for software business

Protecting the irreplaceable | f-secure.com

Towo Toivola, Director, Quality of Operations, F-Secure Corporation

A Balanced Metrics Set for Software Business

This presentation is about how to

choose the right metrics, not about

how to implement them.

F-Secure Corporation

TowoProvocativeUnclearInteractive

LeanAgile

Quality15 SW years

Why do you want to measure things?This is often not clear.

To some people it seems measuring and having data has a value of its own.

Metrics are powerful!

..both in good and bad.

A fool with a powertool is a more dangerous fool.

Visibility!

© F-Secure ConfidentialApril 8, 20236

The Most Common Screw-ups with Metrics

• Be fond of metrics but understand little of them

• Measure whatever is easy to measure

• Produce a lot of data and leave it there

• Make them superimportant before you have measurement in place

• Measure everything!

• Measure just one thing and focus all attention to it

• Measure something because others are doing it

• Tie people’s salary to arbitrary metrics targets

• Manipulate metrics and results (or make others do that)

• Refuse to accept metrics results

• Can you think of more?

April 8, 20237

The whole point of metrics is the amount and quality of action taken based on them.

© F-Secure Confidential

8 © F-Secure Confidential2013-02-01

Metrics results

Analysis Reality

Report

Decide

Measure

Impact

Experts

Leaders

Doers

The whole point of metrics is the amount and quality of action taken based on them.

Take action


You Want Useful Metrics

To be useful means being..

• Practical (easy and cheap to measure)

• Overarching (holistic, the purpose more than just the potential means)

• Actionable, insightful (gives clues to ’Why’ and ’What to do about this’)

• Responsive to action (not too much lag)

• Accurate and trustworthy (you can rely on the information)

April 8, 202310

The Problem


Ease of measurement

Clueness

Holistic

LagAccuracy

Financials

Bug countWork hours

Lines of code

Development lead time

Value Throughput

Net Promoter

Score

Work time usage

efficiency

Utilization

Staff engagement

Work in progress

Staff engagement

Lag

Accuracy

Financials

Bug count

Work hours

Lines of code

Development lead

time

Value Throughpu

t

Net Promoter

Score

Work time

usage efficiency

Utilization

Work in progress

Insightful

Holistic Financials

Bug count

Work hours

Lines of code

Development lead time

Utilization

Staff engagemen

t Work in progress

Value Throughput

Net Promoter

Score

Work time

usage efficiency


TrustworthyYou need to get this right regardless of the chosen metrics

Thanks: thettablog.blogspot.fi

+

+

=

=+


Our Solution

A multi-level metrics system

April 8, 202316

Naturally.

Unbeatably:

Accurate

Holistic


Unfortunately:

Lagging

Not insightful

April 8, 202317

Since we develop software:- Bug count- Codebase- etc.

Accurate

Responsive

Easy to measure

Insightful


But:

Very local,

means to an end only

April 8, 202318

To comply with the law:Work hoursAccurate

Responsive

Required


To care for our people and culture:Staff engagementHolistic

Easy to measure


But to measure operational excellence in software business we choose..

A multi-level metrics system

Slide version 2012-12-28

Lead Time (speed)

Amount of Days

Value Throughput (amount)

Flow of Value

Net Promoter Score(quality)Width of smile

Work In Progress (sunken investment)The unfinished things

We need to know.

How much does our BL

contribute to company success?

How our customers experienc

e our products?

How much customer

value do we produce?

We need to know.

1Q12 2Q12 3Q12 4Q12?

0

2

4

6

8

10

Product3Product2Product1

Value Throughput

[Bogus

data]

M€

1Q12 2Q12 3Q12 4Q12

05

10152025

Business3Business2Business1

Work in Progress

[Bogus

data]

M€

Slide version 2012-09-03

Lead Time

[Bogus

data]

Net Promoter Score

[Bogus

data]


Why?More insightful than financials

More responsive than financials

More holistic than typical low-level metrics

Accurate enough to be very useful

Practical enough to measure

They support each other

LTNPS

VT

WIP

€


So..

Financial KPI

Operational KPI

People KPI

Variety of lower-level precision metrics used locally


Thank you for your time.I hope we have interesting questions and time to discuss them.

Ease of measurement

CluenessHolisticity

LagAccuracy

FinancialsBug countWork hours

Lines of code

Development lead

time

Value Throughpu

t

Net Promoter

ScoreWork time

usage efficiency

Utilization

Staff engageme

nt

Work in

progress

A balanced metrics set for software business

Business

Transcript of A balanced metrics set for software business