Outcome-InformedEvidence-Based Practice

John G. Orme & Terri Combs-Orme

http://ormebook.com/

Pearson Education

Measuring and Monitoring Client Progress

“However beautiful the strategy, you should occasionally look at the results.”

Sir Winston Churchill

Contemporary Conceptual Definition of EBP

“…a process for making practice decisions in which practitioners integrate the best research evidence available with their practice expertise and with client attributes, values, preferences, and circumstances”

Rubin, 2008, p. 7

Steps in the EBP Process

Develop an answerable question Locate relevant evidence Critically analyze the evidence Combine evidence with client attributes,

values, preferences, and circumstances and with your practice expertise

Apply to practice Measure and monitor client outcomes,

and adjust intervention as needed

Limitations of EBP

What you might know How intervention

works when implemented under ideal conditions (i.e., efficacy)

How intervention works when implemented under routine practice conditions (i.e., effectiveness)

What you don’t know How intervention

works when you implement it with your particular client in your practice setting

Outcome-Informed Practice (OIP)

Practice in which you: Measure your client’s outcomes at regular,

frequent, pre-designated intervals, in a way that is sensitive to & respectful of client

Monitor these outcomes at regular, frequent, pre-designated intervals to determine if client is making satisfactory progress

Modify your intervention plan as needed along the way by using this practice-based evidence, in concert with evidence-based practice, to improve your client’s outcomes

Single-Case Designs

Family of designs characterized by: Systematic repeated measurement of a

client’s outcome(s) at regular, frequent, pre-designated intervals under different conditions (baseline and intervention)

Evaluation of outcomes over time & under different conditions in order to monitor client progress, identify intervention effects &, more generally, learn when, why, how, & extent to which client change occurs

Intervention Research vs. OIP

Intervention Research Usually initiated to

inform practice by developing generalized causal knowledge about interventions

Benefits to participants of secondary importance

Not tailored to individual participants

Specific informed consent requirements

OIP Primary purpose to

improve well-being of particular client

Tailored to emerging problems, goals, needs, characteristics, & circumstances of each particular client without generalization to other clients

Why Outcome-Informed Practice?

The Top Ten Reasons

1:To Obtain the Best Client Outcomes

Even empirically supported intervention may not work with a particular client

Many factors other than your intervention have effect on client outcomes Ongoing, relatively objective feedback to

the practitioner reduces deterioration & treatment dropout, improves overall outcome, & leads to fewer treatment sessions

Why may ESIs not work for this client? ESIs beneficial for average research

subject; some unchanged & some worse Research participants

often not representativeof your clients (e.g., race,ethnicity, sexualorientation)

ESIs may be difficult totransport to your practicesetting

Why may ESIs not work for this client? Specific elements of ESIs only one

ingredient in recipe that contributes to client success

Quality of therapeutic alliance influences client’s outcome whatever intervention you use

ESIs are templates that need to be customized to individual clients Tailored to personal, contextual & changing

situations causing & maintaining problems faced by a particular client

2: To Avoid Natural Biases

Practitioners tend to overestimate improvement & underestimate deterioration, in relation to client self-reports

Practitioners have much more confidence in their abilities to judge clients’ progress than is warranted by the data

Bias

Tendency to see and interpret information consistently with an emotional preference or preconceived expectation

Confirmation & Falsification

We seek information to confirm our biases.

We should seek information to falsify our conclusions

3: To Improve Decision-Making You’ll have more & better information

with which to make practice decisions How else would you know if what you’re

doing is working?

4: To Prevent Client Deterioration 5 to 10% of adult & 14 to 24% of child

clients deteriorate while receiving services

Practitioners find itespecially difficult to detectclient deterioration

Measuring & monitoringclient outcomes can reducerates of deterioration,partly by reducing rates ofdropout from treatment

5: To Bridge the Gaps in EBP

Evidence-based practice is place to start, but not sufficient: RCTs tell us whether interventions work with

the average client, not a particular client Clients have individual characteristics &

circumstances Methodologies underlying EBP by no means

perfect Empirically-supported interventions not

available for every client problem

6: To Improve Your Relationships with Your Clients Demonstrates your respect for your

client by giving your client an important voice

Demonstrates conscientiousness on your part & may enhance client’s confidence in you

7: To Enhance Your Development as a Practitioner Huge differences in client outcomes

among therapists, even using same intervention methods

Clinical practice withoutongoing feedback is likelearning archery whilewearing a blindfold; yourskills are unlikely toimprove if you cannot seewhere the arrow is landing

8: To Be Accountable

Monitoring client outcomes constantly & modifying intervention as needed provides important tool for achieving clients’ goals in as short a time as possible & conserving limited resources

Some argue that, at least for psychotherapy services, outcome-based accountability is coming—soon

9: To Meet Your Ethical Obligations

For example… NASW Code of Ethics, 5.02 Evaluation

and Research (a) Social workers should monitor and

evaluate policies, the implementation of programs, and practice interventions.

Can I practice without measuring and monitoring client outcomes? No All practitioners

measure and monitor client outcomes—the question is how best to do it

Instructor’s Manual

Free download from Pearson website Sample syllabus For each chapter:

Suggested discussion prompts Suggested chapter activities and

assignments; Essay questions; Additional resources (i.e., books, journal

articles, websites)

Companion Website

Ormebook.com For each chapter:

PowerPoint presentation. List of recent relevant published articles

and books for additional reading Internet resources Chapter tables and figures in Microsoft

Word

Companion Website (cont’d)

Chapter 2 Bibliography of evidence-based practice

texts Online resources for evidence-based

practices Single-case design bibliography

Chapter 5 Microsoft Word 2007 templates for

constructing single-case design graphs and instructions for using these templates (illustrated below)

Companion Website (cont’d)

Chapter 9 Excel program for scoring CES-D Excel program for scoring Hudson’s

scales Excel program for calculating reliable

change Word document describing how to

determine a clinical cutoff Word document describing how to

determine clinically significant change for Hudson’s scales

Companion Website (cont’d)

Coming soon… Crossword puzzles Flashcards Additional in-class and out-of-class

exercises Send us your ideas Contribute to the web page

Textbook

For each chapter: Critical thinking questions and practice

tests integrated with 2008 CSWE EPAS to assess student application of the core competencies

Complex, realistic case with session-by-session descriptions, monitoring data & graphs made with the Excel template

An Example Case

Eve

32-year-old HIV-positive client, hospital outpatient clinic

Referred to social worker for non-adherence to retroviral medication regimen

Lives with partner of 7 years & young daughter: not HIV-positive.

BFA in music & works occasionally playing piano in restaurants or bars

Client reports high stress & drug side-effects as problems with parenting & working

Searching the evidence

Social worker finds few evidence-based interventions for HIV+ women

Best option (with men) seems to be individually tailored intervention that focuses on eliminating client’s individual barriers to adherence (Martin et al., 2010)

Eve’s barriers: high levels of stress, forgets medications (does understand regimen)

Many evidence-based interventions for reducing stress

Constructing the Baseline

Course of the Intervention

Eve practices deep breathing & reports feeling better

Makes gradual progress in adherence, but not to 100% quickly enough

SW suggests several other evidence-based interventions to reduce stress, such as meditation, & involving partner, but Eve refuses

Finally turn to technology

The Pill Phone

 Provides visual/audible prompts to take medication

 Tracks/stores pill-taking records  Shows what most pills look like  Confirms dose was taken Displays potential side effects Now an iPhone app

The Course of Intervention

1 2 3 4 50

10

20

30

40

50

60

70

80

90

100

1

2

3

4

5

6

7

Week

Pe

rce

nta

ge

Ad

he

ren

ce

Ab

ility

to C

op

e w

ith S

tre

ss

Baseline

The Course of Intervention

1 2 3 4 50

10

20

30

40

50

60

70

80

90

100

1

2

3

4

5

6

7

Week

Pe

rce

nta

ge

Ad

he

ren

ce

Ab

ility

to C

op

e w

ith S

tre

ss

Baseline Monitoring & deep breath-ing

The Course of Intervention

1 2 3 4 50

10

20

30

40

50

60

70

80

90

100

1

2

3

4

5

6

7

Week

Pe

rce

nta

ge

Ad

he

ren

ce

Ab

ility

to C

op

e w

ith S

tre

ss

Baseline Monitoring & deep breath-ing

The Course of Intervention

1 2 3 4 50

10

20

30

40

50

60

70

80

90

100

1

2

3

4

5

6

7

Week

Pe

rce

nta

ge

Ad

he

ren

ce

Ab

ility

to C

op

e w

ith S

tre

ss

Baseline Monitoring & deep breath-ing

Monitoring, deep breath-ing & Pillphone

The Course of Intervention

1 2 3 4 50

10

20

30

40

50

60

70

80

90

100

1

2

3

4

5

6

7

Week

Pe

rce

nta

ge

Ad

he

ren

ce

Ab

ility

to C

op

e w

ith S

tre

ss

Baseline Monitoring & deep breath-ing

Monitoring, deep breath-ing & Pillphone

In this case…

Monitoring permitted early identification of serious problem & quick implementation of intervention

Graphing illustrated early that pace of change was insufficient

Graphs provided clear understanding of problem & motivation to client

Note focus on the client, not the intervention

Thank you