Leslie Barfknecht, LCSW

What is FIT?

What works in Psychotherapy?

Why FIT?

The Measures

Next steps to implementing FIT

(FIT)

What is FIT?

An empirically supported, pantheoretical approach for evaluating and improving the quality and effectiveness of behavior health services. It involves routinely and formally soliciting feedback from clients regarding the therapeutic alliance and outcome of care and using the resulting information to inform and tailor service delivery. ~Scott Miller ICCE FIT Manuals

What is FIT?

PCOMS-Partners for Change Outcome Management System:

• Outcome Rating Scale (ORS)

• Session Rating Scale (SRS)

Outcome Rating Scale (ORS):

measures a person’s level of distress and functioning

Outcome Questionnaire-45.2 (OQ-45.2),

Youth Outcome Questionnaire (YOQ)

Child Outcome Rating Scale (CORS)

Individually (Personal well-being)

I----------------------------------------------------------------------I

Interpersonally

(Family, close relationships)

I----------------------------------------------------------------------I

Socially (Work, school, friendships)

I----------------------------------------------------------------------I

Overall

(General sense of well-being)

I----------------------------------------------------------------------I

Session Rating Scale (SRS):

Client’s perspective of the common factors/alliance

Working Alliance Inventory (WAI)

Helping Alliance Questionnaire-II (HAQ-II)

Relationship

I-------------------------------------------------------------------------I

Goals and Topics

I------------------------------------------------------------------------I

Approach or Method

I-------------------------------------------------------------------------I

Overall

I------------------------------------------------------------------------I

I felt heard, understood, and

respected.

I did not feel heard, understood, and

respected.

We worked on and talked about what I

wanted to work on and

talk about.

We did not work on or

talk about what I wanted to work on and

talk about.

Overall, today’s session was right for

me.

There was something missing in the session

today.

The therapist’s approach is a good fit

for me.

The therapist’s approach is not a good

fit for me.

Yes! The average treated client is better off than 80% of the untreated sample in most studies of therapy.

Treatment Effect Size

Psychotherapy .8-1.2

Bypass Surgery .8

Pharmacotherapy for arthritis .61

Fluoride to prevent tooth decay .25

Lipsey, M. W., & Wilson, D. B. (1993) Shadish, W. R. & Baldwin, S. A. (2002)

How effective are you compared to other

psychotherapists in the field with similar

training, education & experience?

On average, clinicians rated themselves in the 80th percentile

Less than 4% considered themselves average (50th percentile)

No one rated themselves below average

8% rated themselves lower than 75th percentile

25% rated their performance at the 90th% or higher compared to their peers

Walfish et al, (2012)

• Emphasis on specific treatment models

• Failure to address dropouts (not meaningfully engaged?)

• Failure to identify who is benefitting and who may be deteriorating while in care?

• Substantial variation in outcomes between clinicians with similar training and experience.

• Not knowing overall rate of effectiveness and tendency of average clinicians to overestimate.

• Effectiveness plateaus over time in the absence of concerted efforts to improve.

“Common Factors” founded by Saul Rosenzweig in 1936 when he published a paper which asserted that all models of therapy could be equally successful, due to competent therapists sharing common factors that aided their patients… also known as the “Dodo Bird Hypothesis”

Therapeutic Alliance

Agreement on relationship

Agreement on goals

Agreement on tasks Edward S. Bordin, (1979)

What Works in Therapy: Research on the Alliance

•Research on

the alliance

reflected in over

1100 research

findings.

Norcross, J. (2009). The Therapeutic Relationship. In B.

Duncan, S. Miller, B. Wampold, & M. Hubble (eds.). The

Heart and Soul of Change. Washington, D.C.: APA Press.

Means or

Methods

Goals,

Meaning or

Purpose

Client’s View of the

Therapeutic Relationship

Client

Preferences

The following statement was approved as policy of the American Psychological Association (APA) by the APA Council of Representatives during its August, 2005 meeting.

“Evidence-based practice in psychology (EBPP) is the integration of the best available research with

clinical expertise in the context of patient characteristics, culture, and preferences”

Expectancy, Placebo,

and Allegiance

Effects (4%)

Client/Extratherapeutic Factors

Treatment Effects

Model/Technique

Effects (1%)

Therapist Effects (4-9%)

Alliance Effects (5-8%)

The Therapeutic Factors

Client/Extratherapeutic Factors:

Independent of treatment and include: readiness for change, strengths, resources, level of functioning before treatment, social support systems, socioeconomic status, personal motivations and life events (Hubble et al., 2010)

It is estimated to account for 80-87% of variability in scores between treated and untreated clients (Wampold, 2001)

Treatment Effects (est.13-20%):

Therapist Effects: 4-9%

WHO provides the therapy is an important determinant of outcome

Some clinicians are more effective than others

“better” therapists form better therapeutic relationships with a broader range of clients

97% of the difference in outcome between therapists is accounted for by differences in forming therapeutic relationships

Brown et al., 2005

Luborsky et al., 1986

Wampold & Brown, 2005

Baldwin et al., 2007

Treatment Effects (est.13-20%):

Alliance Effects (5-8%):

The therapeutic relationship is the largest contributor to outcome in behavioral health services

Client level of engagement is the most potent predictor of change in therapy

Treatment Effects (est.13-20%):

Expectancy, Placebo & Allegiance Effects (4%):

Both the client and therapist’s expectations and beliefs about therapy and its potential effects

Client: hope and expectations about the healing properties of therapy; belief in the therapist & belief in the treatment provided (“placebo effect”)

Therapist: positive expectations, faith in therapy as a practice & allegiance to the approach and methods utilized

Treatment Effects (est.13-20%):

Model/Technique Effects (1%)

All therapies involve methods

The effect of these depends on the degree to which these methods fit with clients’ preferences and expectations

Models work best when they engage and inspire participants and provide a structure to therapy

Consumer: age, gender, diagnosis and previous treatment history

Clinician: Age, gender, years of experience, professional discipline,

degree, training, licensure, theoretical orientation, amount of supervision, personal therapy, specific or general competence & use of evidence based practices

Model/technique Matching therapy to diagnosis Adherence/fidelity/competence to a particular treatment approach

Wampold & Brown, 2005; Beutler et al., 2004; Hubble et al., 2010;

Nyman et al., 2010; Miller et al., 2007; Benish et al., 2008; Imel et al., 2008 Miller et al., 2008; Wampold et al., 1997; Wampold et al., 2002

Wampold et al., 2001; Duncan & Miller, 2005; Webb et al., 2010

Nearly 10,000 books on therapy have been published;

The number of treatment models has grown from 60 to over 400

Over 145 manualized treatments for 51 of the 397

possible DSM diagnostic groups 280 SAMHSA endorsed Evidence Based Interventions

on NREPP

Duncan, B., Miller, S., Wampold, B. & Hubble, M. (Eds.) (2009). The Heart and Soul of Change (2nd Ed.).

Washington, D.C.: APA Press.

Duration of therapy without positive change

Early client change

Consumer rating of the alliance (2)

Level of consumer engagement

Improvement in the alliance over the course of treatment

Client’s level of distress at the start of therapy (1)

Clinician allegiance to their choice of treatment approach

The debate…

Differences between two administrations of the same measurement tool must be attributable to the changes in the variable being measured.

Does the instrument accurately measure what it claims to measure?

Construct Validity:

whether a measure actually captures the construct it is intending to capture

E.g., does a well-being measure actually measure well-being or an alliance measure actually measure alliance

Concurrent Validity: how a well a scale is correlated with other well-established or documented scales

ORS and Outcome Questionnaire-45.2 (OQ-45.2) range between -.53 and -.74

SRS and other measures of alliance range between .48 and .63

Discriminate Validity: whether a measure can accurately differentiate between clinical groups and non-clinical groups (aka specificity)

ORS has very good specificity with OQ-45.2

The degree to which an instrument can be explained, completed and interpreted quickly and easily.

Any outcome and alliance measures taking in excess of 5 minutes are less likely to be used by clinicians and increase the likelihood of complaints by consumers of MH services (Duncan, Miller & Sparks, 2004)

Reliable & Valid + Feasibility = the instrument more useful in predicting, evaluating and guiding treatment than shorter (single factor, general distress) scales.

Consistent use increases validity of the data.

Clinical Cutoff

Midpoint between the average distress of: “functional” and “dysfunctional” population

Adults: 25

Reliable change

Amount of change being attributed to work being done in therapy and more than just measurement error (e.g., chance)

FIT is being used with broad and diverse groups of adults, youth and children in agencies and treatment settings around the world including:

Inpatient

Outpatient

Residential

Prison-based (mandated care)

Case Management Bohanske, B. & Franczak, M. (2009)

Duncan, B., Miller, S., Wampold, M., & Hubble, M. (2009)

Miller, S. , 2010 13 RCT’s involving 12374 clinically, culturally and economically diverse consumers: Routine outcome monitoring and feedback as

much as doubles the “effect size” (reliable and clinically significant change)

Decreases drop-out rates by as much as half (1/2)

Decreases deterioriation by 33% Reduces hospitalizations and shortened length of

stay by 66% Significantly reduced cost of care (non-feedback

groups increased)

Outcome Rating Scale

Session Rating Scale

Brief, client rated 4 item visual analogue scale

Measures the client’s experience of well-being

Designed and normed for adults and adolescents (13+)

Clinical cutoff = 25 Defines the boundary between normal and clinical range of

distress

Reference point for evaluating the severity of distress for a particular client/client sample

Individually (Personal well-being)

I----------------------------------------------------------------------I

Interpersonally

(Family, close relationships)

I----------------------------------------------------------------------I

Socially (Work, school, friendships)

I----------------------------------------------------------------------I

Overall

(General sense of well-being)

I----------------------------------------------------------------------I

Brief, client rated 4 item visual analogue scale

Measures the therapeutic alliance perceived by the client

Designed and normed for adults and adolescents (13+)

Alliance cutoff = 36

A score of or below 36 is considered cause for concern because fewer than 24% of cases score lower than 36

Miller & Duncan, (2004)

Relationship

I-------------------------------------------------------------------------I

Goals and Topics

I------------------------------------------------------------------------I

Approach or Method

I-------------------------------------------------------------------------I

Overall

I------------------------------------------------------------------------I

I felt heard, understood, and

respected.

I did not feel heard, understood, and

respected.

We worked on and talked about what I

wanted to work on and

talk about.

We did not work on or

talk about what I wanted to work on and

talk about.

Overall, today’s session was right for

me.

There was something missing in the session

today.

The therapist’s approach is a good fit

for me.

The therapist’s approach is not a good

fit for me.

Couples therapy

Family therapy

Children and Adolescents

Multi-site or multi-service agencies

SPMI

Mandated Clients

Primary objective:

Ensure services being delivered engage the client and are effective in each case

When alliance scores are low or progress is absent, slow, or uneven, supervision discussion explores:

The client’s goals for treatment

Different methods or levels of care

Additional services or providers

ICCE FIT Manuals

https://www.centerforclinicalexcellence.com/

http://scott-d-miller-ph-d.myshopify.com/collections/performance-metrics/products/performance-metrics-licenses-for-the-ors-and-srs

openFIT