1

Skye Pamela Barbic, PhD, OT

1Centre for Health Evaluation and Outcome Sciences

2Department of Occupational Science and Occupational Therapy3Faculty of Medicine, University of British Columbia, Vancouver, BC

4St. Paul’s Hospital, Vancouver, BC

Understanding the recovery needs of Canadians and how to measure them: what’s next?

Kelty Dennehy Mental Health Resource Center Breakfast EventJanuary 27th, 2017

2

Objectives1. To summarize a national study that assessed the recovery needs of Canadians.

2. To introduce a new measure for capturing the recovery needs of Canadians (PROM).

3. Discuss the opportunity to collaborate with patients, families, communities, and government to build sustainable, accessible, and coordinated care pathways that support the recovery of Canadians.

Canadian Study

PROM measure

EngagementMoving

forwards

3

Conflicts of interest

• I have no conflicts of interest to declare

4

Context: Mental Illness in Canada

• Affects approximately 6.7 million Canadians1,

• Living with a serious mental illness may shorten one’s lifespan by nearly 25 years. 3

• High rates of chronic illnesses as diabetes, obesity, heart ailments and respiratory diseases

• Vulnerability to homelessness, unemployment and alcohol consumption.

4

5

What is health?

7

What is Physical Health?

What is mental health?

9

10

12

What is an outcome?

What is measurement?

15

WAIST: YARD

ARM SPAN: FATHOM

HAND

Foot

Elbow-Fingertip

: Cubit

16

30,000 BC 3000 BC 2540 BC 1799

17

1960

18

MAD Science dept MAD £33, June 1957, Page 36

So far, the system has been adopted in Terra del Fuego, Afghanistan, and Southern

Rhodesia

MAD Science dept MAD #33, June 1957, Page 36

So far, the system has been adopted in Terra del Fuego, Afghanistan, and Southern

Rhodesia

1919

World FactBook, CIA 2016 Appendix G (weights and

measures)

At this time, only three countries-Burma, Liberia, and the US have NOT adopted the International System of Units as their official

weights and measures

20

Why is this a problem?

Measurement

Rating scales should be developed from clear construct definitions to ensure that a

substantive construct theory determines scale content.

a construct theory….

….is “the story we tell about what it means to move up and down the scale for a

variable of interest (eg. Temperature, reading ability, memory). Why is it, for example,

that items are ordered as they are on the item map? This story evolves as knowledge

increases regarding the construct” (p 308)

Stenner, A., Burdick, H., Sandford, E., and Burdick, D. How accurate are lexile text

measures? J Applied Measurement, 2006; 7: 307-322.

21

22

Physical SciencesHealth Sciences

22

23

HOW DO WE MEASURE OUTCOMES WE CANNOT SEE?

How do we create a common language of

measurement in health?

Can we achieve a common unit in

health?

24

The Personal Recovery Outcome Measurement Study

25

Context: Deinstitutionalization

25

26

Shift in mental health

26

27

RECOVERY

28

The Recovery Problem

• Single most targeted outcome in mental health

• Individual treatment

• Group level

• System reform

• Significant gap in the conceptual clarity of recovery and lack of valid measures that can capture the outcome.

29

THE PROM Study

• Global Objective:

• to assess the personal recovery needs of Canadians with severe mental illness who receive community outpatient mental health services.

• Specific objectives to:

• (i) describe the personal recovery profile of a Canadian outpatient sample, and

• (ii) measure the extent to which the full range of the recovery is covered by existing rating scales.

• (iii) develop a measure that is fit for purpose to cover the full range of recovery and can inform care.

How can you measure recovery?

30

31

Recovery Measures

3232

33

Profile Step 1 (n=982):

High levels

• Hope

• Motivation

• Sense of personal empowerment

• Goal Orientation

Low levels

• Connectedness

• Contribution to the community

• Feeling part of society

• Participation in employment (9%)

• Quality of Life

34

Time USEIF YOU TAKE ONE THING AWAY- TAKE THIS AWAY!

35

Time in structured roles: Toronto (n=224)

35

92

4543

10

29

<2 3 TO 5 6 TO 15 16-30 30+

Time in structured roles (hours)

80%

40%

36

Early look at Vancouver (n=658)

36

<2 3 TO 5 6 TO 15 16 TO 30 31+

Vancouver Time Use (hours/week)

83%

40%

37

REVISIT Mental Illness in Canada

• Affects approximately 6.7 million Canadians1,

• Results in reduced workplace productivity and over $50 billion in direct costs to the healthcare system.1,2

• Living with a serious mental illness may shorten one’s lifespan by nearly 25 years. 3

• High rates of chronic illnesses as diabetes, obesity, heart ailments and respiratory diseases

• Vulnerability to homelessness, unemployment and alcohol consumption.

37

38

“What if we did not discharge people to the community unless we knew they were connected to the community and had at least 20 hours/week of meaningful activity…it can’t be just

about medical stabilization”(said a certain awesome psychiatrist in BC working in a tertiary care setting)

(true story from August 2016)

39

Back to the PROM study

What did we learn about measuring recovery outcomes using self-report measures?

40

40

Recovery Measures

41

???

Natural order to items

Walk a mile

Run 5 miles

Run 10

miles

Run 20 miles

Run marathon

Run Boston

Marathon

Endurance

Item Difficulty

Personal ability

42

Recovery

A B C D E F

Recovery???

Item Difficulty

Personal ability (more recovery)

43

44

Recovery

A B C D E F

Recovery???

Item Difficulty

Personal ability (more recovery)

44

Item Difficulty

Personal ability (more recovery)

Recovery???

Recovery Assessment Scale

• Results in press

45

Recovery

Low

High

Recovery Assessment Scale

• Results in press

46

Recovery

Low

High

Illness Management and Recovery Scale

• Results in press

47

Low

High

Recovery

4848

49

Phase 2: PROM initial item set

• 40 item developed (2 focus groups, n=19)

• A-prioi hypothesis about the item order

• Tested on new sample (n=106 Community dwelling)

Recovery

50

5151

Too long

Measurement Scale

DisSD Neutral SAAgree

5252

4-6

1-3

Reco

very

SCORE 11/30

ADJ SCORE 11/30

ADJ2 SCORE 4/30

56

Recovery

57

•Cost

•Time to complete

•Readability

•How does it inform practice

•Face

•Content

•Criterion

•Divergent

•Convergent

•Cross-cultural

•Response Scale

•Test-retest

•Internal Consistency

•Person Separation

Reliability Validity

UtilityMeaning

Ruler for informing care?

58

Recovery

59

Where do we measure recovery?

60

Traditional

Decrease hospitalization

Community Tenure

Decrease Service Utilization

61

New Carrots?

• Citizenship

• Employment/School

• Participation/social

• The “right” housing

• Time use/ engagement

62

“We are able to talk to patients about where they are at, but also reflect on what our goals of services are and make plans to help clients get there”.

Our team now holds Friday morning rounds called “going to the PROM” to share findings from the patients and reflect on how our services can be oriented to meet the recovery needs of individuals who seek care at this clinic.”

63

Service level

“The PROM allowed the health authority to identify a

mismatch in our services…we offer a lot of support for

vulnerable folks in crisis, but many of our patients

highlighted the need for more opportunities for goal

setting, goal implementation, and achieving a higher

level of citizenship in their communities…we need to

strive towards having outcomes and services that are at

the higher end of the ruler.”

64

“I am able to walk in my doctor’s office with my recovery and say ‘Hey Doc, I am a 17 on the recovery ruler….I want to be a 23….let’s make a plan for how to get me there”

65

Future work

• Towards a conceptual and measurement model to capture the needs of youth living with mental illness in Canada (Barbic, Mathias):

• Identification of needs of youth

• Identification of measurement strategy to capture these needs

• Development of an item bank for outcomes

66

To receive a free copy of the PROM: please contactSkye.barbic@ubc.ca

@skye_barbic

Skye Barbic