Raters of the Lost Art – The Sequel The HoNOS/CA Challenge

Ossi Beck, John Dillon, Jason Monk

Queensland Mental Health Clinical Improvement Team

Clinical Systems, Collection and Performance Unit | Mental Health Alcohol and Other Drugs Branch

Queensland Health

The Holy Grail of Outcomes Collection

Quantity

We are getting one part of the equation right

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Proportion of outcome collection occasions where required clinical outcome measures were recorded

Queensland FYTD 2016/17

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

FY 2009/10 FY 2010/11 FY 2011/12 FY 2012/13 FY 2013/14 FY 2014/15 FY 2015/16 FY 2016/17

Proportion of outcome collection occasions where required clinical outcome measures were recorded

Queensland 2009/10 - 2016/17

Start Review End Target

But what about the quality?

Australian Mental Health Care Classification

PH

ASE

LEV

EL IN

FOR

MA

TIO

N

Consolidating Gain

Phase details

Assessment Only

Acute

Functional Gain

Intensive Extended

Unknown

PR

OD

UC

TS F

OR

CO

STIN

G

MENTAL HEALTH CLASSIFICATION

EPIS

OD

E LE

VEL

INFO

RM

ATI

ON

1

Establishment details

Consumer demographics

Diagnosis data (Principal and

Additional)

Mental health legal status

Episode commencement and end details

First episode descriptor*

END

CLA

SSES

COSTING

Provider details

Establishment details

Pro

visi

on

of

Serv

ice

Intervention

Preparation

Travel

Documentation

Observations

NOCC collection

Rep

ort

ed a

gain

st

Use

d t

o c

reat

e

Can

be

use

d t

o c

ost

Use

d t

o c

reat

e

Queensland Training Framework

Training Modality Audience Facilitator Resources

Pre-requisite for all

training sessionsIndividual – online All clinicians Online amhocn.org

Orientation Group – in-person New starters Clinical Educator and/or CIC

Orientation package

Clinician’s handbook Vol 1

Sharing Perspectives

Annual refreshers

Team – in-person All clinicians QMHCIT HoNOS Challenge

Individual – online All clinicians Online amhocn.org

Group – in-person All clinicians Clinical Educator and/or CICRefresher package

Sharing Perspectives

Experienced

clinicians changing

setting or target

population

Group or individually

– in-person

Clinicians moving to

another setting or

target population

CICOrientation package

Clinician’s handbook Vol 1

Train-the-trainer

(initial and

refresher)

Group – in-person Clinical Educators

and CIC’sQMHCIT Train-the-trainer package

“Them”

• 20 Mental Health Service Organisations

• 5,700 paid FTEs

• 1 Manager

• 3 Improvement Coordinators

• 3 Indicator Coordinators

“Us”

The HoNOS Challenge

• Annual event

• Refresher training

• Team building

– All in-scope teams are invited to

participate

– Two-Step process

• Rate individually

• Rate as a team

Process of the Challenge

Vignettes distributed to all in-scope teams

•CYMHS, Adult, 65+

Clinicians rate the vignette individually

•Using the glossary and reference to rating rules

Collective rating by team

•Reference to glossary and notes, robust debate and discussion

Team rating submitted via Survey Monkey TM

•One rating per team

Release of consensus ratings

• Including rationale for ratings

The Vignettes

• Need to be realistic and

challenging to foster

discussion and debate

• Themed, for example Drug

and Alcohol or CALD &

ATSI specific

Participation

About 300 teams are in

scope for the Challenge

2011 2012 2013 2014 2015 2016

14% 33% 34% 25% 34% 42%

0%

10%

20%

30%

40%

50%

- Higher participation rates for

CYMHS teams

- Lower participation rates for

inpatient units

2016 Results

Adult 6 17 36 12

Child & Youth 6 20 13 2

Older Persons 2 4 2 2

122 Teams participated in the Challenge (42%)Adult: 38%

CYMHS: 48%

65+: 40%

106 Teams were awarded a HoNOS medal (87%)Adult: 83%

CYMHS: 95%

65+: 80%

Plans for 2017

• Increase participation by inpatient units

– Targeted promotion

– Offer a setting specific vignette

• Include a Mental Health Phase of Care rating

• Increase overall participation to more than 50%

In Summary

Benefits

• Improves inter-rater reliability within

and across Mental Health teams

• Increases awareness of importance of

collecting outcome measures

• Creates “healthy” competition to

reinforce good practice and reward

excellence

• Assists with key performance

indicators (outcomes readiness and

change in consumer’s clinical

outcomes)

Limitations

• Lack of control over the actual rating

process

• non-mandatory activity preaching to

the converted?

• Dependant on supply of vignettes

15