Good Bad Ugly Webinar

The Heart and Soul of Change: Becoming Better At What We Do

June 17-19 New Orleans

Featuring: Barry Duncan, Jacqueline Sparks, Makungu Akinyela, Michael Lambert, Lori Ashcraft, Bob Bohanske, John Murphy

Sponsored by the Louisiana Counseling AssociationSave the dates! Registration information forthcoming at

www.lacounseling.org and www.heartandsoulofchange.com

Learn:What Works and How to Deliver ItHow to Improve Your Outcomes and Accelerate Your DevelopmentHow to “Level” the Counseling ProcessHow to Join Forces with the Recovery and Social Justice Movements

The Good, the Bad, & the Ugly of Psychotherapy

Barry Duncan, Psy.D. Barry Duncan, Psy.D. www.heartandsoulofchange.comwww.heartandsoulofchange.com954.721.2981 954.721.2981 www.whatsrighwithyou.comwww.whatsrighwithyou.com

[email protected]@comcast.net

Consumer DrivenOutcomes Management

What’s UpThree Rationales

What’s UpThree Rationales

Solves inconsistency in outcomes among therapists & treatments; improves outcomes…period

Based in research about what works (the

common factors)Based on known

predictors of outcome

I love this work

Solves inconsistency in outcomes among therapists & treatments; improves outcomes…period

Based in research about what works (the

common factors)Based on known

predictors of outcome

I love this work

Consumer Driven Outcomes Management

Long Ago in a Galaxy Far Away


Long Ago in a Galaxy Far AwayBefore the validation studies

demonstrated the psychometricsBefore RCTs demonstrated the power of

feedback to improve outcomesBefore the science…There was the desire to give consumers a

voice in their own care, to bring them into the inner circle—that’s what started what you will hear about today

Before the validation studies demonstrated the psychometrics

Before RCTs demonstrated the power of feedback to improve outcomes

Before the science…There was the desire to give consumers a

voice in their own care, to bring them into the inner circle—that’s what started what you will hear about today

PsychotherapyThe Good…

PsychotherapyThe Good…

Study after study, Study after study, and studies of and studies of studies show the studies show the average treated average treated client is better off client is better off than 80% of the than 80% of the untreated sample.untreated sample.

Recent study:Recent study:2,000+ providers2,000+ providers6,000+ clients6,000+ clientsOutcomes Outcomes

equivalent to RCTs equivalent to RCTs for depression, for depression, specifically the specifically the TDCRP.TDCRP.

Recent study:Recent study:2,000+ providers2,000+ providers6,000+ clients6,000+ clientsOutcomes Outcomes

equivalent to RCTs equivalent to RCTs for depression, for depression, specifically the specifically the TDCRP.TDCRP.

Minami, T., Wampold, B., Serlin, R. Hamilton, E., Brown, J., Kircher, J. (2008). Benchmarking Minami, T., Wampold, B., Serlin, R. Hamilton, E., Brown, J., Kircher, J. (2008). Benchmarking the effectiveness of psychotherapy treatment for adult depression in a managed care environment. the effectiveness of psychotherapy treatment for adult depression in a managed care environment. Journal of Consulting and Clinical Psychology, 76(1), 116-24.Journal of Consulting and Clinical Psychology, 76(1), 116-24.

Therapy WorksAs Good As RCTsTherapy WorksAs Good As RCTs

PsychotherapyThe Bad…

PsychotherapyThe Bad…

Drop out Drop out rates rates average average 47%47%

Therapists Therapists vary… a lot vary… a lot

TDCRP: top third psychiatrists giving placebo bested bottom third giving meds; clients of best therapists improve 50% more and dropped out 50% less; medication useful for clients of more effective therapists and not for less

Wampold, B., & Brown, J. (2006). Estimating variability in outcomes attributable to therapists: A naturalistic study of outcomes in managed care. Journal of Consulting and Clinical Psychology, 73 (5), 914-923.

The BadIncredible Variation Among

Therapists

The BadIncredible Variation Among

Therapists

Client/Extratherapeutic Factors (87%)

Treatment Effects 13%

Feedback Effects 15-31%

Alliance Effects 38-54%

Model/Technique 8%

Model/Technique Delivered:Expectancy/AllegianceRationale/Ritual (General Effects) 30-?%

Therapist Effects 46-69%

And the UglyProviders Don’t Know

And the UglyProviders Don’t Know

20-70% range Therapists graded

their effectiveness, A+ to F—67% said A or better; none rated below average.

Therapists don’t know Therapists don’t know how effective they how effective they areare

20-70% range Therapists graded

their effectiveness, A+ to F—67% said A or better; none rated below average.

Therapists don’t know Therapists don’t know how effective they how effective they areare

Hansen, N., Lambert, M., Forman, E. (2002). The psychotherapy dose-response effect and its implications for treatment delivery services. Clinical Psychology: Science and Practice, 9, 329-343.

Sapyta, J., Riemer, M., & Bickman, L. Feedback to clinicians: Theory, research, and practice. Journal of Clinical Psychology: In Session, 61, 145-153

•Least effective therapists believe they are as helpful as most effective.

•In the absence of reliable information about outcome, what do we expect?

On Becoming A Better Therapist

More on Provider Differences


More on Provider Differences

Hiatt, D. & Hargrave, G. (1995). Characteristics of highly effective therapists in managed behavioral provider networks. Behavioral Healthcare Tomorrow, July August, 19-22

The Result:Consumer Confidence Troubling

The Result:Consumer Confidence Troubling

APA asked: “Is this an important reason why you might choose not to seek help?”

The highest % responses were lack of ins. (87%) & cost (81%).

The 3rd was a lack of confidence in the outcome of tx (77%).

Despite tx efficacy, DOs are a problem, many do not benefit, therapists vary, don’t know about effectiveness, & there is a crisis of confidence among consumers.

APA asked: “Is this an important reason why you might choose not to seek help?”

The highest % responses were lack of ins. (87%) & cost (81%).

The 3rd was a lack of confidence in the outcome of tx (77%).

Despite tx efficacy, DOs are a problem, many do not benefit, therapists vary, don’t know about effectiveness, & there is a crisis of confidence among consumers.

To The RescueConsumer Driven Outcomes

Management

To The RescueConsumer Driven Outcomes

Management Can alleviate these problems. Howard et al. (1996)

advocated for the systematic eval. of client response during treatment to “determine the appropriateness of the current tx…the need for further tx…[and] prompt a clinical consultation for patients who [were] not progressing at expected rates”

Can alleviate these problems. Howard et al. (1996)

advocated for the systematic eval. of client response during treatment to “determine the appropriateness of the current tx…the need for further tx…[and] prompt a clinical consultation for patients who [were] not progressing at expected rates”

Feedback and OutcomeLambert’s Five Trials

Feedback and OutcomeLambert’s Five Trials

All 5 sig. gains for feedback 22% of TAU at-risk cases

improved compared with 33% for feedback to therapists, 39% for feedback to therapists & clients, & 45% when supplemented with support tools

A strong case for routine measurement of outcome in everyday clinical practice

All 5 sig. gains for feedback 22% of TAU at-risk cases

improved compared with 33% for feedback to therapists, 39% for feedback to therapists & clients, & 45% when supplemented with support tools

A strong case for routine measurement of outcome in everyday clinical practice

Feedback v TAU; Over doubled effectiveness; ES: .50; 4 times # of clients reached clinically sig. change

FU: TAU-34.2% v. 18.4% Feedback sep./divorce rate

Feedback v TAU; Over doubled effectiveness; ES: .50; 4 times # of clients reached clinically sig. change

FU: TAU-34.2% v. 18.4% Feedback sep./divorce rate

Anker, M., Duncan, B., & Sparks, J. (2009). Using client feedback to improve couple therapy outcomes: A randomized clinical trial in a naturalistic setting. Journal of Consulting and Clinical Psychology, 77, 693-704.


Isn’t It Good, Norwegian Wood


Isn’t It Good, Norwegian Wood

Reese et al (in press, Psychotherapy)

First Independent Study

Reese et al (in press, Psychotherapy)

First Independent Study

N=148: Feedback group doubled controls (10.4 vs. 5.1 pts); ES: .48

Like Norway study, clients, regardless of risk status, benefit from continuous feedback

N=148: Feedback group doubled controls (10.4 vs. 5.1 pts); ES: .48

Like Norway study, clients, regardless of risk status, benefit from continuous feedback

Norway: 9 of 10 got better outcomes

Feedback raised effectiveness of the lower ones to their more successful colleagues.

Therapist in low effectiveness group became the BEST with feedback!

Norway: 9 of 10 got better outcomes

Feedback raised effectiveness of the lower ones to their more successful colleagues.

Therapist in low effectiveness group became the BEST with feedback!

Regarding Therapist Variability

Feedback Improves Outcomes

Regarding Therapist Variability

Feedback Improves Outcomes

Counselor's Outcomes(n=30 or more cases)

00.20.40.60.8

11.21.41.61.8

Counselor

Eff

ec

t s

ize

Mean Effect Size for all Cases

Miller, S.D., Duncan, B.L., Sorrell, R., & Brown, G.S. (February, 2005). The Partners Miller, S.D., Duncan, B.L., Sorrell, R., & Brown, G.S. (February, 2005). The Partners for Change Outcome Management System. for Change Outcome Management System. Journal of Clinical Psychology, 61Journal of Clinical Psychology, 61(2), 199-(2), 199-208.208.

Provider Variation Feedback Improves

Effectiveness

Provider Variation Feedback Improves

Effectiveness

Effects on EfficiencyCancellations, No Shows, LOS

Effects on EfficiencyCancellations, No Shows, LOS

Claude (2004) compared the ave. # of sessions, canc., no shows, and % of long-term cases before and after OM. Sample: 2130 closed cases seen in a public CMHC.

Ave. # of sessions dropped 40% (10 to 6) while outcomes improved by 7%; canc. and no show rates were reduced by 40% and 25%; and % of long term null cases diminished by 80% (10% to 2%).

An estimated savings of $489,600. Such cost savings did not come at the expense of client satisfaction with services—during the same period satisfaction rates improved significantly.

Claude (2004) compared the ave. # of sessions, canc., no shows, and % of long-term cases before and after OM. Sample: 2130 closed cases seen in a public CMHC.

Ave. # of sessions dropped 40% (10 to 6) while outcomes improved by 7%; canc. and no show rates were reduced by 40% and 25%; and % of long term null cases diminished by 80% (10% to 2%).

An estimated savings of $489,600. Such cost savings did not come at the expense of client satisfaction with services—during the same period satisfaction rates improved significantly.

http://www.clipartconnection.com/clipartconnection.com/showphoto.php?photo=12982&papass=&sort=1&thecat=167

Addiction Services: Retention Before and After

Addiction Services: Retention Before and After

Improved Completion Rates:50% or Less Before 82% After With Improved

Outcomes, Increased Satisfaction, and Decreased Complaints

Improved Completion Rates:50% or Less Before 82% After With Improved

Outcomes, Increased Satisfaction, and Decreased Complaints

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

Bohanske, R., & Franczak, M. (in press). Transforming public behavioral healthcare: A case example of consumer directed services, recovery, and the common factors. In B. Duncan et al. (Eds.) The Heart and Soul of Change: Delivering What Works, 2nd Ed. Washington, DC: American Psychological Association.

Arizona Length of Stay

Arizona Length of Stay

0

20

40

60

80

100

120

140

DAYS

1 2

Pre - Post CDOI

Out-Patient Adult (GMH/SA)

Series1

138

88


“Severely and Persistently Mentally Ill” Services in a CMHC

“Severely and Persistently Mentally Ill” Services in a CMHC

Reduced LOS in CM, therapy, & residential services by 72, 59, & 47% respectively

No shows & canc. reduced by 30%.

Reduced LOS in CM, therapy, & residential services by 72, 59, & 47% respectively

No shows & canc. reduced by 30%.






Model/Technique 8%



Successful V. Unsuccessful Providers

Focus on Strengths

Successful V. Unsuccessful Providers

Focus on StrengthsStudied videos of 120

sessions of 30 clients.Unsuccessful providers

focused on problems, neglected strengths.

Successful providers focused on strengths before moving to problems….

Studied videos of 120 sessions of 30 clients.

Unsuccessful providers focused on problems, neglected strengths.

Successful providers focused on strengths before moving to problems….

Gassman, D. & Grawe, K. (2006). General change mechanisms: The relation between problem activation and resource activation in successful and unsuccessful therapeutic interactions. Clinical Psychology and Psychotherapy, 13, 1-11.

http://www.alibaba.com/catalog/11387870/Hand_painted_Animail_Lion_Oil_Paintings.html


Bottom Line


Bottom LinePrivilege clients’ experience & rally their resources to the cause….what ORS is all about

Privilege clients’ experience & rally their resources to the cause….what ORS is all about

“The quality of the patient's participation . . . [emerges] as the most important determinant of outcome."

Orlinsky, D. E., Rønnestad, M. H., & Willutzki, U. (2004). Fifty years of process -outcome research: In M. J. Lambert (Ed.), Bergin and Garfield’s handbook of psychotherapy and behavior change (5th ed., pp. 307-390). New York: Wiley.

Therapists Variables that Predict Change

Therapists Variables that Predict Change

Therapists with the best results:

Emphasize the therapist-client relationship

Are better at the alliance across clients

Therapists with the best results:

Emphasize the therapist-client relationship

Are better at the alliance across clientsBaldwin, S. A., Wampold, B. E., & Imel, Z. E. (2007). Untangling the alliance-outcome correlation: Exploring the relative

importance of therapist and patient variability in the alliance. Journal of Consulting and Clinical Psychology, 75(6), 842-852.; Vocisano et al. (2004). Therapist variables that predict symptom change in psychotherapy with chronically depressed outpatients. Psychotherapy, 41(3), 255-265.





Model/Technique 8%



The Norway Feedback Project (n=500)

Therapist Variability and the Alliance

The Norway Feedback Project (n=500)

Therapist Variability and the AllianceTherapists accounted

for 9% of the variabilityAlliance at last session

accounted for 40% of that variability

Implication: Continuous alliance monitoring is essential; need to focus on the alliance as job 1

Therapists accounted for 9% of the variability

Alliance at last session accounted for 40% of that variability

Implication: Continuous alliance monitoring is essential; need to focus on the alliance as job 1

•Considered to be the most sophisticated comparative Considered to be the most sophisticated comparative clinical trial ever conducted:clinical trial ever conducted:

•Four approaches (CBT, IPT, Drug, Placebo).Four approaches (CBT, IPT, Drug, Placebo).

Elkin, I. Et al. (1989). The NIMH TDCRP: General effectiveness of treatments. Archives of General Psychiatry, 46, 971-82.

•No No differencedifference in outcome between approaches in outcome between approaches•The The client’s client’s rating of the alliance at the second session rating of the alliance at the second session the best predictor of outcome across conditions.the best predictor of outcome across conditions.

TDCRP The Alliance

TDCRP The Alliance

•The largest study ever conducted on the treatment of The largest study ever conducted on the treatment of problem drinking:problem drinking:

•Three different treatment approaches studied (CBT, 12-step, Three different treatment approaches studied (CBT, 12-step, and Motivational Interviewing).and Motivational Interviewing).

•NO NO difference in outcome between approaches.difference in outcome between approaches.

Project MATCH Group (1997). Matching alcoholism treatment to client heterogeneity. Project MATCH Group (1997). Matching alcoholism treatment to client heterogeneity. Journal of Studies on Alcohol, 58Journal of Studies on Alcohol, 58, 7-29., 7-29.Babor, T.F., & Del Boca, F.K. (eds.) (2003). Babor, T.F., & Del Boca, F.K. (eds.) (2003). Treatment matching in Alcoholism. Treatment matching in Alcoholism. Cambridge University Press: Cambridge, UK.Cambridge University Press: Cambridge, UK.Connors, G.J., & Carroll, K.M. (1997). The therapeutic alliance and its relationship to alcoholism treatment participation and Connors, G.J., & Carroll, K.M. (1997). The therapeutic alliance and its relationship to alcoholism treatment participation and outcome. outcome. Journal of Consulting and Clinical Psychology, 65Journal of Consulting and Clinical Psychology, 65(4), 588-98.(4), 588-98.

•The client’s rating of the therapeutic alliance the best The client’s rating of the therapeutic alliance the best predictor of:predictor of:

•Treatment participation;Treatment participation;•Drinking behavior during treatment;Drinking behavior during treatment;•Drinking at 12-month follow-up.Drinking at 12-month follow-up.

Project MATCH The Alliance

Project MATCH The Alliance

•600 Adolescents marijuana users:600 Adolescents marijuana users:•Significant co-morbidity (3-12 problems).Significant co-morbidity (3-12 problems).

•Two arms (dose, type) and one of Two arms (dose, type) and one of three types of treatment in each arm:three types of treatment in each arm:

•Dose arm: MET+CBT (5 wks), Dose arm: MET+CBT (5 wks), MET+CBT (12 wks), Family Support MET+CBT (12 wks), Family Support Network (12 wks)+MET+CBT;Network (12 wks)+MET+CBT;•Type arm: MET/CBT (5 wks), ACRT (12 Type arm: MET/CBT (5 wks), ACRT (12 weeks), MDFT (12 wks).weeks), MDFT (12 wks).

Cannabis Youth Treatment ProjectCannabis Youth Treatment ProjectThe AllianceThe Alliance

•Alliance predicted: Premature drop-out; Alliance predicted: Premature drop-out; Substance abuse post-treatment, and Substance abuse post-treatment, and cannabis use at 3 and 6 month follow-up.cannabis use at 3 and 6 month follow-up.

Dennis, M.,et al. (2004). The Cannabis Youth Treatment (CYT) Study: Main findings from two randomized trials. Journal of Substance Abuse Treatment, 27, 197-213.

http://www.growkind.com/home.html

The AllianceBottom LineThe Alliance

Bottom LineThe Alliance Matters—Big Time

TDCRP: alliance accounted for up to 21% of the variance…what the SRS is all about

The Alliance Matters—Big Time

TDCRP: alliance accounted for up to 21% of the variance…what the SRS is all about

The PredictorsWhat Really Predicts

Outcome?

The PredictorsWhat Really Predicts

Outcome?

Client variables:Demographics, etc

Diagnosis

Treatment Models& Techniques

•Client’sClient’s rating of the rating of the alliancealliance the best predictor of the best predictor of engagement and outcome.engagement and outcome.

•Client’sClient’s subjective subjective experience of change early experience of change early in the process the best in the process the best predictor of success for any predictor of success for any particular pairing.particular pairing.

Starting With Starting With the Predictorsthe PredictorsGifts to Improve OutcomesGifts to Improve Outcomes

Howard, K. et al. (1986). The dose-effect response in psychotherapy. American Psychologist, 41, 159-164.

A

B

•The bulk of change occurs sooner rather than latter

Who drops out?

Sooner Rather Than Later

Baldwin, S., Berkeljon, A., Atkins, D., Olsen, J., & Nielsen, S. (2009). Rates of change in naturalistic psychotherapy: Contrasting dose-effect and good-enough level models of change. Journal of Consulting and Clinical Psychology, 77(2), 203-211.

Some clients do take longer, but the mythology never dies

N=4676; 77% attended 8 or less, and 91% 12 or less

Note that even for the clients who take longer, change starts early…just is flatter

Cannabis Youth Treatment Project

Project MATCH

http://www.chestnut.org/LI/Posters/CYT_%20MF_APA.pdfBabor, T.F., & DelBoca, F.K. (eds.) (2003). Treatment Matching in Alcoholism. United Kingdom: Cambridge, 113.

Early change in treatment is a robust predictor of outcome and retention in treatment.Gotta measure outcome!

It’s A FactEarly Change is The Rule

But any measurement system that takes more than 5 minutes will never happen. Therapists can get really worked up over anything that adds paperwork, especially when they don’t see the clinical usefulness.

But any measurement system that takes more than 5 minutes will never happen. Therapists can get really worked up over anything that adds paperwork, especially when they don’t see the clinical usefulness.

Feasibility FirstLike I Need A Hole in the

Head

Feasibility FirstLike I Need A Hole in the

Head

When PacifiCare Introduced 30 Item Measure

When PacifiCare Introduced 30 Item Measure

Practitioners complained:

About the length & frequent administration

Cut into sessions & increased workload

Was intrusive Exec. Dir. of MPA said: “I have never seen such

negative reaction from providers.”

Practitioners complained:

About the length & frequent administration

Cut into sessions & increased workload

Was intrusive Exec. Dir. of MPA said: “I have never seen such

negative reaction from providers.”

Hanlon, P. (2005). PacifiCare screening tool, policies raise concerns. New England Psychologist, 13, 1,11-12.

•Scored to the Scored to the nearest nearest millimeter.millimeter.

•Add the four Add the four scales together scales together for the total score.for the total score.

•Give at the Give at the beginning of the beginning of the visit; Client visit; Client places a mark places a mark on the line.on the line.

•Each line 10 cm Each line 10 cm (100 mm) in (100 mm) in length.length.

•Reliable, valid, feasible

Becoming BetterMeasuring Outcome

Becoming BetterMeasuring Outcome

The Session Rating Scale Measuring the

Alliance

The Session Rating Scale Measuring the

Alliance

•Give at the end Give at the end of session;of session;

•Each line 10 cm Each line 10 cm in length;in length;

•Reliable, valid, Reliable, valid, feasiblefeasible

•Score in cm to Score in cm to the nearest mm;the nearest mm;

•Discuss with Discuss with client anytime client anytime total score falls total score falls belowbelow 36 36

A.S.I.S.T:A.S.I.S.T:Automated Scoring, Interpretation, and data Storage ToolAutomated Scoring, Interpretation, and data Storage Tool

•Give at the beginning of the visit

•Client clicks a mark on the line

•Each line 10 cm (100 mm) in length

•Scored to the nearest millimeter

•Four scales added for the total score

SRSSRS

•Give at the end of each session;

•Each line 10 cm in length;

•Scored to the nearest mm

•Discuss with client anytime total score falls below 36

Answers the Call Consumer-Centered, Recovery-

Based

Answers the Call Consumer-Centered, Recovery-

Based New Freedom Commission:

Care is consumer-centered, with providers working in full partnership with the consumers they serve to develop individualized plans of care.

SAMHSA and Partners: National Consensus Statement on Mental Health Recovery

PCOMS only system that partners with consumers

New Freedom Commission: Care is consumer-centered, with providers working in full partnership with the consumers they serve to develop individualized plans of care.

SAMHSA and Partners: National Consensus Statement on Mental Health Recovery

PCOMS only system that partners with consumers

Partners with consumers to tailor services based on response & fit, provides an early warning system

to prevent drop-outs & negative outcomes, & solves helper variability—feedback improves performance based on factors accounting for and predicting positive outcomes

Partners with consumers to tailor services based on response & fit, provides an early warning system

to prevent drop-outs & negative outcomes, & solves helper variability—feedback improves performance based on factors accounting for and predicting positive outcomes



Good Bad Ugly Webinar

Education

Transcript of Good Bad Ugly Webinar