Measurement for improvement

Mike Davidge

22

3

“Measurement is for improvement not judgement.”

D. Berwick

Measurement for improvement

4

Model for Improvement

5

7 Steps to measurement

1 Decide aim

2 Choose measures

3 Define measures

6 Review measures

5 Analyse & present

7 Repeat

steps 4-6

4 Collect data

6

Measures checklist – a handy reminder

Section 1– Rationale– Definitions– Data required– Goals

Section 2– Collect– Analyse– Review

7

Clarifying aim is crucial

1 Decide aim

2 Choose measures

3 Define measures

6 Review measures

5 Analyse & present

7 Repeat

steps 4-6

4 Collect data

Take the LIFT test. Would you be able to describe your aim in a

couple of sentences?

Take 5 minutes to agree your aim

8

Choosing the right measures

1 Decide aim

2 Choose measures

3 Define measures

6 Review measures

5 Analyse & present

7 Repeat

steps 4-6

4 Collect data

9

Types of measure

Outcome measures– Reflect the impact on the patient– For example: unplanned return to ITU or crash calls.

Process measures– Reflect the way you work– For example: % compliance with Sepsis 6 bundle.

Balancing measures– reflect unintended consequences– For example: if you have implemented changes to reduce your post

operative length of stay, you also want to know what is happening to your post operative readmission rate. If this has increased then you might want to question whether, on balance, you are right to continue with the changes or not.

10

AIM PRIMARY DRIVERS

SECONDARY DRIVERS

To improve recognition and

timely management of

patients identified with sepsis in ED and CDU by

achieving 90% compliance with evidence based

therapy (SEPSIS 6) by March 2013

1

2

33

Early identification of patients with

possible sepsis in ED, CDU and

Wards

1

Ensure sepsis best management practices in ED, CDU and wards

2

Seamless transitions

• Timely triage• Timely notification to, and assessment

by nurse and doctor• Early and repeated lactate

measurements• Monitoring and communication of

progress• Early aggressive administration of IV

fluids• IV antibiotics administered within 1

hour• Blood cultures taken before IV

antibiotics are given• Education of sta in sepsis as time ff

critical illness• Effective communication between

ED , CDU and SCAS• E ective communication and ff

transition with in-patient wards• Patient shadowing and information

for patients

Driver diagram

11

Choose your measures

Use the driver diagram You have 5 minutes to

customise to your system Now take another 5

minutes to decide which drivers and change ideas to measure

Homework: Continue to discuss!

AIM PRIMARY DRIVERS

SECONDARY DRIVERS

To improve recognition and

timely management of

patients identified with sepsis in ED and CDU by

achieving 90% compliance with evidence based

therapy (SEPSIS 6) by March 2013

1

33

Early identification of patients with

possible sepsis in ED, CDU and

Wards

1

Ensure sepsis best management practices in ED, CDU and wards

2

Seamless transitions

• Timely triage• Timely notification to, and assessment

by nurse and doctor• Early and repeated lactate

measurements• Monitoring and communication of

progress• Early aggressive administration of IV

fluids• IV antibiotics administered within 1

hour• Blood cultures taken before IV

antibiotics are given• Education of staff in sepsis as time

critical illness• Effective communication between

ED , CDU and SCAS• Effective communication and

transition with in-patient wards• Patient shadowing and information

for patients

12

Definitions – an Achilles heel

1 Decide aim

2 Choose measures

3 Define measures

6 Review measures

5 Analyse & present

7 Repeat

steps 4-6

4 Collect data

13

Collect

Decisions, decisions What - All patients or a sample? Who – is collecting? Where – is the data located? How – hospital system or audit? When – Real time or retrospective?

What is your baseline?

1 Decide aim

2 Choose measures

3 Define measures

6 Review measures

5 Analyse & present

7 Repeat

steps 4-6

4 Collect data

14

Where to measure?

Start ? Decision Point ? Handover ? End ?

14

15

It’s not an add-on

Organise everything around value-added (front line) work processes

W Edwards DemingAll value-adding work is inherently local;

All improvement is inherently local; therefore,

As you implement a data collection system,

You mustn't destroy clinical productivity

Instead, you mustIntegrate data collection into workflow at the front line

16

Admission and Recognition Bundles

Detail actions that should take place on a regular, routine basis such as:

– Observations– Calculating and recording NEWS score– Querying sepsis if the score is high– Communicating NEWS and risk to whole team

The aim is to embed these actions and behaviours into normal everyday practice

So a high compliance with these bundles is demonstration that practice is based upon the best available evidence

17

Planning & testing your data collection

What, Who, When, Where and How?

You have 5 minutes to discuss your data collection plan

And decide your first small test of change (PDSA)

Homework: Run the PDSA

18

Analyse

“The type of presentation you use has a crucial effect on how you

react to data”

1 Decide aim

2 Choose measures

3 Define measures

6 Review measures

5 Analyse & present

7 Repeat

steps 4-6

4 Collect data

19

How we traditionally assess performance:2 point comparisons

Why has the number of crash calls gone up? Our service is getting worse. We need to do something!

What decision are you going to make?

Last quarter

This quarter Change

62 66 +6%

20

What’s a person’s normal body temperature?

21

In the real world, everything varies....How long does it take you to get to work?

How many patients did we admit today?

Is my temperature always the same?

How long does it take to take a patients BP?

22

“Data contains both signal and noise. To be

able to extract information, one must

separate the signal from the noise within

the data.”

Walter Shewhart

23

There are two types of variation

While every process displays variation: some processes display controlled

variation (common cause)– Stable pattern of variation = noise– constant causes/ “chance”

while others display uncontrolled variation– pattern changes over time = signal– special cause variation/“assignable” cause eg infection or

hypothermia

We should display data in a way that shows which is present

24

Revisiting Crash calls

2006Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec31 36 27 18 22 40 23 31 42 19 31 16

2007Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec

Last quarter

This quarter Change

62 66 +6%

2006Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec31 36 27 18 22 40 23 31 42 19 31 16

2007Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec32 32 54 22 51 17 24 11 27 24 26 16

2011

2012

25

As a run chart..

Complaints during 2006 to 2007

0

10

20

30

40

50

60

Jan

Feb

Mar

Apr

May Jun

Jul

Aug

Sep

Oct

Nov

Dec Jan

Feb

Mar

Apr

May Jun

Jul

Aug

Sep

Oct

Nov

Dec

2006 2007

26

Run charts

• Plot data in time order

• Calculate and display median as a line

• Analyse chart by studying how values

fall around median

Complaints during 2006 to 2007

0

10

20

30

40

50

60

Jan

Feb

Mar

Apr

May Ju

n

Jul

Aug

Sep

Oct

Nov

Dec Ja

n

Feb

Mar

Apr

May Ju

n

Jul

Aug

Sep

Oct

Nov

Dec

2006 2007

Number

27

10/1

0/20

11

10/1

1/20

11

10/1

2/20

11

10/0

1/20

12

10/0

2/20

12

10/0

3/20

12

10/0

4/20

12

10/0

5/20

12

10/0

6/20

12

10/0

7/20

12

10/0

8/20

12

10/0

9/20

12

0

20

40

60

80

100

120

Percentage of sepsis patients that receive sepsis six with one hour_x000d_NHH _x000d_from Oct 2011 to Oct 2012 - All Wards

Weeks

% c

ompl

ianc

e

Data from an in hours Outreach serviceDuring this time sepsis was a major contributor to death in 15% of cases

28

Annotate your charts!

29

Review measures

1 Decide aim

2 Choose measures

3 Define measures

6 Review measures

5 Analyse & present

7 Repeat

steps 4-6

4 Collect data

Where will the measures be reviewed?

When (how frequently) will we review them?

Question 1

Question 2

It is a waste of time collecting and analysing your data if

you don't take action on the results

30

Where do you put stuff you want everyone to know?

Why do we hide track and trigger scores at the foot of the bed and then audit them infrequently?

31Insert name of presentation on Master Slide

32

Putting Important Information In a Prominent Place

Communicates to the whole team, all the time

Quickly exposes where staff have difficulty with performing observations/calculating score

Promotes education and training for possible eventualities

33

7 Steps to measurement

You may not get it right first time! You may need several attempts to get it right for you

1 Decide aim

2 Choose measures

3 Define measures

6 Review measures

5 Analyse & present

7 Repeat

steps 4-6

4 Collect data

34

Next steps – the project plan

Measures– Do you have an agreed set of measures? If

not, how and when will you get them agreed?

Definitions– Who will complete measures checklists for all

remaining measures and by when?

Review meetings– Have you agreed when you will review your

measures? Set a date for the first meeting

Test your process– When are you going to follow the 7 step

process for your first measures?

1 Decide aim

2 Choose measures

3 Define measures

6 Review measures

5 Analyse & present

7 Repeat

steps 4-6

4 Collect data

Thank you and here’s to effective

measurement!