“Why clinical audit? Is this just

another quality improvement

tool or can it be transformative?”

Grant Phelps

Sydney August 25th 2014

…“it depends”

Declaration - My inbuilt bias

We work for the owners of the health care system

We have a professional obligation to leave the health

care system in a better state than we found it

Cost, Safety, Consumer focus

Improvement is all of our business

This is no longer optional

The core business of health care is the delivery and

receipt of clinical services.

Everything else is subservient to and supports that

“Our Health care systems are

perfectly designed to achieve the

results they achieve”Don Berwick

It’s a miracle it isn’t worse - our system survives on professionalism and goodwill

http://www.bloomberg.com/visual-data/best-and-worst/most-efficient-health-care-countries

We haven’t solved the safety problem yet

1

10

100

1,000

10,000

100,000

1 10 100 1,000 10,000 100,000 1,000,000 10,000,000

Number of encounters for each fatality

To

tal li

ves l

ost

per

year

REGULATEDDANGEROUS

(>1/1000)

ULTRA-SAFE

(<1/100K)Health Care

Mountain Climbing

Base Jumping

Driving

Chemical Manufacturing

Chartered Flights

Scheduled

Airlines

European Railroads

Nuclear Power

Note: both dimensions are logarithmic scales

c/o Dr. Rene Amalberti

Scheduled

Airlines

We’re still changing the paradigm from..

“what’s the matter?” healthcare to …

“what matters to you?” healthcare

The business of Health Care is…

The clinician patient

interaction

Supported by Management

Influenced by policy

This is where value is created, and destroyed

And where the real improvements are to be found

The transformation vision

Experience of Care

Improving

Population

health

Reducing

unit

cost of care

Berwick et al Health Affairs May/June

2008

Source: Airbus industries 2014

A transformed industry?

Fatal accidents vs number of flights

We know how to fix healthcare.

•Better care is systems care• Understanding clinical care in process terms

• systems so that clinicians can do it right first time

Better care is cheaper careThose saved resources can be used to do good

It is possible to close the quality gapClinicians driving clinical improvement through learning

Clinical

Microsystems

Patient and Community

Clinical Service deliveryTranslating management into clinical

outcomes

Receive care

Clinical PracticeClinical rules

Service design and structure

Patient involvement

Interpreters of careDoes care meet needs?

Improving healthcare delivery

= changing behaviours

Improving healthcare?

Kizer K. NEJM 2003

Health Affairs 2002

Health Affairs, 2002

www.cec.health.nsw.gov.au

Measuring

healthcare:

clinical audit

1916

End Result cards

Codman led thinking on

Measuring outcomes

Benefits of public reporting

Transparency as an enabler of improvement

Admitting error as prelude to improvement

Creating a Learning organisation

Clinical audit is largely a QC activity

Continuous

Improvement

Quality

Assurance

Quality Control

against a standard

Quality Improvement

– against ?

Clinical audit for most clinicians is…?

“the systematic review of elements of

clinical care against predetermined

criteria…”

It’s about quality control

Does it change behaviour?

Health.vic.gov.au

Individual Quality Control

adjusted colonoscopy completion rate 2007. data

presented in batches of 10 as % completion rate

70%

80%

90%

100%

1 3 5 7 9 11 13 15 17 19 21 23 25 27

completion mean target

Audit can tell us what we already know..

Registries and clinical improvement

Risk of death with after hours discharge from ICU

1 in 7 patients are discharged from ICU after 6pm

Relative risk of death

1.34 ( CI 1.30 – 1.38 p<0.0001)

(Source – ANZICS Core)

Now what do we do????

Clinical audit should be about QI…

“the systematic review of elements of clinical

care against predetermined criteria, with the

aim of identifying areas for improvement and

then developing, implementing and evaluating

strategies intended to achieve that

improvement

Health.vic.gov.au 2007

Clinical audit should help us transform the

system to this…

A care system which generates no needless…

• Deaths

• Pain

• Delays

• Helplessness

• Waste

What would a transformed

organization look like?

A place where collaboration thrives; where everyone collaborates to improve both care and the organization – where organizational values really mean something…

A place where the consumer is the business, in every way

A place where staff experience reward and joy in their work

Resources are sufficient

Transforming our thinking on quality

The right person

Makes the right choice for them

For the right reasons

To receive the right treatment

At the right price

By the right person

At the right time

In the right place

For the right outcome

VTE prophylaxis - Does feedback work?

Pre intervention

November 2007

Post

intervention

May 2008

Result

Numbers of cases

reviewed 236 300

High risk cases

(Defined risk

assessment completed)80% 83%

Prophylaxis (Target

60%) as per ANZ

Guidelines 34.9% 43.2%

8.3%

Improvement

but 16.8% below

target

Yates M., et al IMJ 2013

The effects of guideline dissemination

and implementation strategies

Across four cluster RCT’s of various educational

strategies, a median absolute improvement of

+8.1% (range +3.6% to +17%) compliance with

guidelines was observed.

The impact is limited and sustainability is tenuous

Grimshaw JM, Thomas RE, MacLennan G, et al. Effectiveness and efficiency of guideline

dissemination and implementation strategies. Health Technol Assess 2003

Cochrane review 2006

Audit and feedback can be effective in improving

professional practice. When it is effective, the effects

are generally small to moderate. The relative

effectiveness of audit and feedback is likely to be greater

when baseline adherence to recommended practice is

low and when feedback is delivered more intensively

Jamtvedt G., et al Cochrane Database of Systematic Reviews 2006

What do we know?

“audit and feedback will continue to be an

unreliable approach to quality improvement until we

learn how and when it works best. Conceptualising

audit and feedback within a theoretical framework

offers a way forward”

Foy R et al BMC Health Services Research 2005;5(1):50

What if you change the system?

Clinical Site Baseline

May 2008

(n=300)

Post NIMC

Change Jan

2009

(n=43)

Follow up

Audit

September200

9

(n=40)

Follow up

Audit

November

2009

(n=40)

Medicine 43.2% 83.7% 95% 82.5%

Surgery 60 % 78.4% 69% 67%

Yates M., et al IMJ 2013

Why isn't clinical audit more effective?

Lack of clear purpose – QI vs CQ

Data collection and management

resources

Lack of expertise in design and analysis

Lack of planning

Lack of clinical engagement and leadership

Poor professional culture and poor relationships within and between teams

Lack of integration with other activities ( within a clinical governance framework)

An inability to improve care processesww.health.vic.gov.au

Culture change to drive Improvement in health care

delivery

Clinical Risk / Safety as our tool

Retrospective

management level

Based on identified outcomes, which might be random.

Is really Quality Assurance in disguise.. Are we good enough?

Element of big stick

? Real culture change

Quality improvement as a tool

Prospective

Owned at Coal face level

Based on process improvement

Quality Improvement will reduce risk

How good can we be?

Success = culture change

What drove this

improvement?

WAASM - DVT

prophylaxis

MJA 21 Nov 2005

Bampton P., et al BMJ 2006;332;1320-1323

Understanding and improving the care process

Audit makes a difference to outcomes, but

improving care processes is the main game

Mortality Rate

6.04

3.65

0.71

8.57

0

1

2

3

4

5

6

7

8

9

2002 2003 2004 2005

Year

%

So…. We’ve moved on from ‘medical

audit’ to something that’s more about

understanding and improving systems?

And it that’s the case, shouldn’t we be

talking about suites of measures which

are based in the process of care and

which can tell us something about the

system and how to improve it?

High performing healthcare organisations

understand the why and the how ..….

Why an organization produces its services Who are the customers and what are the broader social needs the organization

fulfils?

How those services are produced What are the processes of daily work or, in other words, the means of production?

How the organization improves its services What are the improvement activities or means of improvement?

Batalden, P.B. and J. Mohr. 1997. Quality Management in Health Care 5(3): 1-12.

HBR 1995

The major enabler: Clinical Engagement

‘Full collaboration in relentless improvement’

Lee and Cosgrove; HBR June 2014

Engaging Clinicians… “Clinicians are the process”

CultureQuality based culture - “It’s all about the patient”

Leadership

Clinical leaders with ability to make decisions

Welcome bottom up leadership

Alignment

Sharing knowledge

Teamwork

Ensuring clinical teams drive quality through department levelAdapted from Kaplan & Norton “Balanced Scorecard” HBS press 1996

Engaging clinicians

Create shared goals

See clinicians as drivers of clinical processes (and

thus the source of cost and harm)

Leverage the personal commitment to quality

Self motivated and autonomous

Lack understanding of ‘system’.

Reinertsen J. et al “Engaging Physicians in a shared Quality agenda. IHI 2007

Necessary but

not sufficient?

Quality control

enabling quality

improvement

Summary

Clinical audit terminology needs to change

It is not an end in itself

It’s just a quality tool and by itself isn’t terribly

helpful if you’re interested in real improvement

Transformation requires much much more –

systems and systematic improvement is the main

game Clinical audit but one input

Clinical Engagement is essential in understanding

and changing any care process