...How to Develop Metrics That Matter

Presented to the Suncoast Chapter of IFMA

January 14, 2016

Presented by Bob Lambewww.linkedin.com/in/lambe

This document can be shared for non-commercial purposes.

All content not noted otherwise is (c) RAL Location Strategies.

Background...

WHY THE INTEREST IN

METRICS AND KPI’S?

Performance Evaluation

• Performance metrics and KPIs continue to

surface as a topic of discussion.

– We all have scorecards for our performance and that

of our staff/providers.

– We all are interested in measuring those things that

can make a difference.

We are sometimes not clear how

these 2 things are related...

Information Overload

Every day, facilities managers get information

from a multitude of systems:• Corporate finance systems

• Corporate HR systems

• Facility work order/CMMS

systems

• CAFM/CADD/BIM systems

• Project plans

• Equipment sensors

• Occupancy sensors

• Security systems

• Cameras

• Meter readings

• Spreadsheets

• Building Audits

• Equipment Alarms

• Building Automation

systems

• Energy Management

systems

• Email

• Text messages

• Online data services

• Industry publications

• Benchmarking reports

• Commissioning reports

• Social media

• Photographs

• Technical/design reports

• Regulations & standards

• Industry cost guides

• And more...

Metrics can help us cut thru the noise,

identify actions needed, and

detect things of interestThis multitude of data types is one of

the opportunity areas for “big data”

analysis in facilities management

http://www.theinvisiblegorilla.com/videos.html

42% of viewers do NOT notice

the gorilla when focused

on counting the number of

passes.

We often do not notice unexpected things

if we are focused on something else.

Even Simple Metrics Can Be Useful

- 500 1,000 1,500 2,000 2,500

FACILITIES SERVICES - General Request

ELECTRICAL - Light Out/Lamp Replacement

FACILITIES SERVICES - Move Workspaces

FACILITIES SERVICES - Conference Room Setup

HVAC - Too Hot/Too Cold

HVAC - General HVAC Repair

HVAC - General HVAC Request

PLUMBING - Fixture Not Working

PLUMBING - General Plumbing Repair

FACILITIES SERVICES - Move Misc Items

FACILITIES SERVICES - Copier Service Request

Number of WR (Avg 19,650/Year)

Request

Issue (

86 T

ypes A

vaila

ble

)

Top 10 WR Issues by Number of Requests

FY13 FY14 FY15

30% of

Demand

WO are

from these

6 Issues

FACILITY MANAGEMENT

METRICS THAT MATTER

From a White Paper of the Same Name...

Free download: https://ralstrategies.com/services-kpi.php

Let’s Start with the Basics...

TYPES OF METRICS

Metrics - Measurements

• Measurements are just

data:

– Cost

– Size

– Quantity

– Temperature

– Status

– Yes/No

– Time/Date stamp

Emergency work orders wrench time

last month

Total wrench time worked last month

Metrics - Indicators

• Indicators are typically total or

average Measurements:

– Total quantity work orders

– Average cost per work order

• Key Indicators are typically

relationships between Indicators,

and have a “target” range:

– % work orders completed by month

Metrics – Lagging (Results)

• Some metrics report past results

• Best used to report actual

performance and historical trends

– Number of emergency repairs as % of

total equipment work orders

– Hot/cold calls per building occupant

– Energy use by building by month

Metrics – Leading (Predictive)

• Some indicators used to suggest likely future

performance

– Projected equipment downtime based on condition index at

last service date

– Projected energy use by building based on weather forecast

• Some “real time” indicators used as proxy for leading

indicators

– Increase in hot/cold calls as measure of HVAC equipment

needing service

Can Organize Types in 2x2 Matrix

Results Indicator

Tells you what you have done

Performance Indicator

Tells you what to do

Key Results Indicator

Provides perspective on how you

have performed

Key Performance Indicator

Tells you what to do to increase

performance

• A future measurement is a forecast

• Forecast error is a metric, but it measures the accuracy

of the forecast and not the reality of the measurement

– Not a useful metric unless your business is forecasting!

We cannot measure the future

Useful Metrics Are Situational

Useful Metrics Change with Role

• CFO

– Facility Cost as Earnings per Share

• Facilities Director

– Operating Cost per Square Foot

• Building Manager

– Cost per Scheduled Work Order

• Building Engineer

– Which WO need to be done today?

Corporate Level

Strategic Level

Functional Level

A Matrix of Metrics?

More related to organizational objectives

More Key Indicators

More specific Results and Performance

Indicators

More immediate (time sensitive)

Less

functionally specific

Functionally specific

Functional RoleO

rganiz

ational Level

CREATING BETTER METRICS

Goodhart’s Law

"When a measure becomes a target, it

ceases to be a good measure."

The Example of the Russian Nail Factory

When the government created targets of output for the

factory that were measured in weight, the result was that

the factory produced a small number of very heavy nails.

When the target was changed to the amount of nails the

factory produced a massive amount of only tiny nails.

Source: various internet sites, originally a joke or uban legend?

Image source: http://lesswrong.com/lw/1ws/the_importance_of_goodharts_law/

• One challenge with metrics is that we often

confuse process and performance, mixing up:

– Goals / objectives,

– Events / milestones,

– Activities,

– Critical Success Factors (CSF), and

– Key Performance Indicators (KPI).

• Goal: Safe workplace

• Event: Regular safety meetings

• Activity: Audits, Investigations

• CSF: Safety oriented culture

• KPI: Lost Time/OSHA Incidents

Establish KPI’s using KPQ’s

• Often we start with the data we have

and then figure out how to use it.

• It is better to start with the questions

that need to be answered to

accomplish our strategic objectives

– Our KPIs will be the answers

– Start with your organization’s scorecard

Strategic Objective

KPQKPI

KPI

KPQ

KPI

KPI

KPI

Example KPQ’s

• Do our facilities provide a safe and productive

environment for customers and employees?

• How well do we react to spikes in service requests?

• Is our facility cost as low as it can be without

jeopardizing the building condition?

• Do we have qualified staff ready and willing to fill

vacancies when needed?

• Do our buildings perform (energy) as well as they

should?

• What sustainability investments would provide the

largest benefit per cost?

Design KPI’s To Answer The KPQ’s

A good KPI....

• Helps to answer one or more KPQ.

• Is based on relevant, available data.

• Provides actionable information for

the intended user.

• Is available on the required frequency.

Strategic Objective

KPQKPI

KPI

KPQ

KPI

KPI

KPI

KPI Design Considerations:

1. Link to Strategy

2. Definition

3. Calculation

4. Purpose

5. Data Sources

6. Future Targets

Images source: http://en.wikipedia.org/wiki/Cockpit

We need to continue

improving our ability to

understand key

performance indicators

to effectively “pilot” our

organizations

Presentation of KPI’s can be Important

Airplanes are constantly off course, but

metrics help them reach desired destination.

Presentation Methods

Scorecards Dashboards Reports

Typically used for

tracking higher-level

KPIs on multiple topics

Typically real time info

on Indicators or KPIs;

Trend may be important

Typically used for

complicated lagging

metrics and analysis

Format Helps

Use Icons for Multiple KPIs

Stoplights are one way to

simplify large data sets.

• Use of graphics, even just color,

can help users more easily

understand the metric

Ineffective KPI Graphic

Does not assist the user in

understanding if action is required

More Effective KPI Graphic

Easy to understand current

position and desired position.

BAD METRICS

Dysfunctional Consequences and

Other Things That Can Go Wrong

Avoid Bad Metrics with

“Inversion Mental Model”

It is not enough to think about difficult problems one way. You need to think about

them forwards and backwards. "Indeed, many problems can’t be solved forward."

says Charlie Munger, the business partner of Warren Buffett and Vice Chairman at

Berkshire Hathaway.*

• Simply stated, applying the inversion mental model is to

start by avoiding bad solutions - avoiding stupidity is

easier than seeking brilliance.

• Here are 8 questions you can ask to determine if you

have a bad metric....

* Source: https://www.farnamstreetblog.com/2013/10/inversion/

Avoiding Bad Metrics

1. Is it Understandable?

2. Does it Reward the

Wrong Behavior?

3. Does it Conflict With

Another Metric?

4. What Does it Measure

(Mixed)?

5. Does It Matter

(To User)?

6. Does it Show Significant

Changes?

7. Does it Use Good Data?

8. Does it Need to Be

Measured?

What About Big Data?

It’s in the media, and

headed our way...

Big data in facilities:

• Energy use

• Various types of

sensors

• Everything else...

Images source: http://www.ey.com/GL/en/Services/Advisory/EY-big-data-big-opportunities-big-challenges

Some BI and Big Data Systems

And many more...

For more info...

www.linkedin.com/in/lambe StrategicFacilityGuide.com