New Wine in Old Bottles? New Wine in Old Bottles? Challenges in Implementing Challenges in Implementing New Practices in Old SystemsNew Practices in Old Systems

Peter S. Jensen, MDPeter S. Jensen, MD

Ruane Professor of Child Psychiatry, Ruane Professor of Child Psychiatry, Center for the Advancement of Children’s Mental Health Center for the Advancement of Children’s Mental Health

Columbia University College of Physicians and Surgeons Columbia University College of Physicians and Surgeons New York State Psychiatric InstituteNew York State Psychiatric Institute

The Problem: The Gaps Between The Problem: The Gaps Between What We Know vs. What we Do for What We Know vs. What we Do for

Children’s Mental HealthChildren’s Mental Health Unmet need as high now as 20 years ago Unmet need highest among minority youth System fragmentation major barrier to access (still) Only 1/3 to 1/5 of children with most severe needs get

MH services Knowledge about child mental health has greatly

increased over the past decade—esp. in neurogenesis, behavioral science, prevention, clinical treatments and services

National Averages of MH NeedNational Averages of MH Need

0.00%

1.00%

2.00%

3.00%

4.00%

5.00%

6.00%

7.00%

8.00%

9.00%

10.00%

Alaba

ma

Califo

rnia

Color

ado

Florid

a

Massa

chus

etts

Michiga

n

Minnes

ota

Mississ

ippi

New Je

rsey

New Y

ork

Texa

s

Was

hingt

on

Wisc

onsin

MH service need

National Average MH Need for Children at 6-17: 7.09%

Data Source: NSAF wave 1 and 2

Geographic Variations in Unmet NeedGeographic Variations in Unmet Need

Proportion of children with measured need who did not receive any MH care

0.0%10.0%20.0%30.0%40.0%50.0%60.0%70.0%80.0%90.0%

Alabam

a

Califo

rnia

Colora

do

Florida

Mas

sach

uset

ts

Mich

igan

Minn

esot

a

Miss

issipp

i

New Je

rsey

New Y

ork

Texas

Was

hingt

on

Wisc

onsin

Unmet MH Need Among Children 6-17: 64.7%

Most children with need do not receive any Most children with need do not receive any MH servicesMH services

64.7

65.2

64.3

63.6

64

64.4

64.8

65.2

65.6

6 to 17 6 to 11 12 to 17

Age groups

% N

o ca

re

Data from NSAF

Unmet need highest among Hispanic Unmet need highest among Hispanic childrenchildren

5969

7767

0102030405060708090

White African-American

Hispanic Other race

% N

o c

are

Data from NSAF

Qualitative insights from comparisons remain valid even if absolute numbers are biased

Non-traditional family structure not associated Non-traditional family structure not associated with more unmet needwith more unmet need

45

64 6570

010203040

50607080

No parents Single parent BlendedFamily

Twobiologicalparents

% N

o c

are

Data from NSAF

Extent of Mental Disorders InExtent of Mental Disorders InU.S. Children and AdolescentsU.S. Children and Adolescents

0

5

10

# of Children(Millions)

Depression Anxiety DBD ADHD Sz Autism/PDD

7.8%7.8%8.0%8.0%

5.6%5.6% 5.0%5.0%

1.0%1.0% 0.5%0.5%

Source: Office of the Surgeon General, andSource: Office of the Surgeon General, andNational Institute of Mental Health, 1999National Institute of Mental Health, 1999

The path to long-term negative outcomes for at-risk children The path to long-term negative outcomes for at-risk children and youthand youth

Risks of not meeting children’s Risks of not meeting children’s mental health needsmental health needs

Suicide School failure and dropout Injuries, hospitalization Chronic mental illness Drug and alcohol use Violence Divorce, family break-up Lifelong dependence on welfare

If children’s mental health needs go untreated, the risks are great:

Challenge: Psychotherapies as Provided in Routine Clinic Settings

Have Little to No Effect

Challenge: Psychotherapies as Provided in Routine Clinic Settings

Have Little to No Effect

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Smith &Glass,1977

Weisz etal., 1987

Weisz etal., 1995

Weisz etal, 1995

Mea

n E

ffec

t S

izes

Mea

n E

ffec

t S

izes

Weisz et al., 1995

Children & AdolescentsChildren & AdolescentsAdultsAdults

UniversityUniversity

Clinic settingsClinic settings

So what do we know about:So what do we know about:

…the obstacles to dissemination of proven approaches, and where are these obstacles located across the various levels of “the system?”

…what works, in terms of effective assessment and treatment interventions effective (vs. ineffective) service models (as well as

ineffective models) effective organizational strategies effective policies

…how to bring about change in parents, “providers,” policy-makers, organizations

…specific strategies and potential next steps that we can start to do now

Three Levels:

Child & Family Factors: e.g., Access & Acceptance

“Provider” (school, MH) Factors: e.g., Skills, Use of EB, Attitudes

Systemic and Societal Factors: e.g., Organizations, Funding Policies

Barriers vs. “Enhancers” to Delivery Barriers vs. “Enhancers” to Delivery of Effective MH Servicesof Effective MH Services

“Effective” Services

Efficacious Treatments

PARENT Example (MTA): PARENT Example (MTA): Would You Recommend this Would You Recommend this

Treatment to Another Parent? Treatment to Another Parent?

Medmgt Comb Beh

Not recommend 9% 3% 5%

Neutral 9% 1% 2%

Slightly Recommend 4% 2% 2%

Recommend 35% 15% 24%

Strongly recommend 43% 79% 67%

0.5

1

1.5

2

2.5

0 100 200 300 400

Assessment Point (Days)

Ave

rage

Sco

re

CC-NOMEDS

CC-MEDS

BEH

MED

COMB

Key Differences, MedMgt vs. CC:

Initial Titration

Dose

Dose Frequency

#Visits/year

Length of Visits

Contact w/schools

PROVIDER Example:PROVIDER Example:Treatment Effects on Inattention (teacher)Treatment Effects on Inattention (teacher)

(Community Controls Separated By Med Use)(Community Controls Separated By Med Use)

ORGANIZATIONAL Example: Glisson & Himmelgarn (1998) Parameter Estimates for Hypothesized Six-Variable Model

ServiceQuality

CountyDemographics

ServiceOutcomes

(problem levels)

-.13*

.12*

-.05-.24*

-.03

.02

-.36*

.01.06

-.20*

* p < .05

OrganizationalClimate

Interorganizatnl Services

Coordination

InterorganizationalRelationships

Organizational Impact on Organizational Impact on Children’s Mental HealthChildren’s Mental HealthOrganizational Impact on Organizational Impact on Children’s Mental HealthChildren’s Mental Health

The strongest predictor of child improvement was organizational climate (Glisson & Himmelgarn, 1998)

But organizational culture, not climate, explained variations in service quality (Glisson & James, 2002)

Relationship between organizational characteristics and effective implementation of new technologies can be identified, but rarely incorporated into studies of EBPs and their translation into practice

15,000 Hours: Rutter et al., impact of school environments on children’s outcomes

How Has Change Been Attempted? How Has Change Been Attempted? The Bad, the Good, and the UglyThe Bad, the Good, and the Ugly

Parent/Family Approaches Bad: Finger-wagging, blame, transfer, attrition Good: Engagement, empowerment Ugly: Current situation mostly reflects bad strategies

Provider (mental health, schools/clinics) Bad: CME, CEU, journals Good: Academic detailing, hands on, MC/II Ugly: Drug companies only using effective methods

System Bad: System of Care as the sole answer Good: MST, Wraparound, Co-location Ugly: Current fragmentation

The UglyThe Ugly

40-60% families may drop out of services before their formal completion (Kazdin et al., 1997)

Children from vulnerable populations are less likely to stay in treatment past the 1st session (Kazdin, 1993)

Factors related to drop-out: Stressors associated with treatment, treatment irrelevance, poor relationship with therapist (Kazdin et al., 1997)

Barriers to ParticipationBarriers to Participation

Triple threat: poverty, single parent status and stress

Concrete obstacles: time, transportation, child care, competing priorities

Attitudes about mental health, stigma Previous negative experiences with

mental health or institutions

M. McKay, 1999

How Has Change Been Attempted? How Has Change Been Attempted? The Bad:The Bad:

Parent/Family Approaches Bad:

Finger-wagging: “you should…” Blame: “The family is non-compliant and

dysfunctional” or “You need to do this, or else I can’t help you.”

Loss by transfer, attrition (“…maybe the family will move away or just stop coming…”)

Ugly: the current situation

The Good: The Good: Engagement InterventionsEngagement Interventions

Focused telephone procedures associated with increased initial show rates

Structural family therapy telephone engagement intervention associated with 50% decrease in initial no-show rates and a 24% decrease in premature terminations (Szapocznik, 1988; 1997)

M. McKay, 1999

The Good:Family Engagement Study

The Good:Family Engagement Study

0

20

40

60

80

100

120

Accepted 1st appt 2nd appt 3rd appt

% for first interview(n=33)

% for comparison(n=74)

M. McKay, 1999

Other Examples of “the Good” – the “5 Es”

Other Examples of “the Good” – the “5 Es”

Engage, Evaluate: Engage, Evaluate: Elicit concerns, Respond to emotions, Build rapport, Keep questions open-ended, Do not interrupt.

Elicit: Elicit: How does patient explain illness?

Educate: Educate: Tell, Ask: “I think you have…”, “What do you know about...?”, “Let me tell you more about…”, “Can you repeat what I said…?”

Enlist/Negotiate/Review: Enlist/Negotiate/Review: “Would you be willing to…?” Negotiate: “Why don’t we agree on…?”

Implications re: Parents/familiesImplications re: Parents/families

Participation rates can be increased by intensive engagement interventions that are tailored to specific populations

Collaboration, engagement, family input and choice, active problem solving are key

M. McKay, 1999

0.5

1

1.5

2

2.5

0 100 200 300 400

Assessment Point (Days)

Ave

rage

Sco

re

CC-NoMeds

CC-Meds

Beh

Med

Comb

Key Differences, MedMgt vs. CC:

Initial Titration

Dose

Dose Frequency

#Visits/year

Length of Visits

Contact w/schools

THE UGLY: Model vs. Typical Treatments for THE UGLY: Model vs. Typical Treatments for ADHDADHD

The Good and the Bad: Effectiveness of The Good and the Bad: Effectiveness of Interventions by Intervention Type Interventions by Intervention Type

05

101520253035

Positive Negative

Davis, 2000

No. of Interventions demonstrating positive or negative/inconclusive change

Implications re: Changing Provider Implications re: Changing Provider BehaviorsBehaviors

• Summary: Changing professional performance is complex - internal, external, and enabling factors• No “magic bullets” to change practice in all circumstances and settings (Oxman, 1995)• Multifaceted interventions targeting different barriers more likely effective than single

interventions (Davis, 1999)• Adult learning methods: learner-centered, active, relevant to needs, learn-work-learn • “readiness to change,” Prochaska & DiClemente, 1983• Little to no theory-based studies, yet are desperately needed due to excessive costs, lack of

progress in field• Consensus guideline approach necessary, but not sufficient. Lack of fit w/physicians’ mental

models

The Good and the Bad: The Good and the Bad: Systems of CareSystems of Care

Ideological commitment to integration of services, family involvement, cultural competence

CASSP--state mh child/adolescent services

Fort Bragg Demonstration Project Stark County Project

Comprehensive and coordinated range of services

Fort Bragg = $94 million, 5 year Demonstration funded by the Army

Quasi-experimental - Demonstration and control sites

Longitudinal - 7 waves of data collection Sample 1 = 984 families= “outcome

sample” Sample 2 = 8,813 families= “ service use

pop”

System of Care Studies: System of Care Studies: (Bickman et al.)(Bickman et al.)

0

2

4

6

8

10

FY88 FY89 FY90 FY91 FY92 FY93

Pre-Demonstration Demonstration Comparison

Start of Demonstration June 1, 1990

% S

erve

dThe Good: Demonstration of The Good: Demonstration of

Increased Access Increased Access

More than 3 times as many outpatient clients had only one visit at the comparison (24%) than at the demonstration (7%)

The Good: Fewer drop-outs The Good: Fewer drop-outs

0

10

20

30

40

50

Demonstration Comparison

At 6 months

% in

Con

tinuo

us C

are

The Good: Better continuity of care The Good: Better continuity of care (fewer breaks in care)(fewer breaks in care)

Most aspects of intake and assessment services

Most aspects of outpatient services Transition and discharge issues in inpatient

and outpatient services

The Good: Parent satisfaction The Good: Parent satisfaction greatergreater

Fewer reported system-level problems Greater quality of mental health services

available Better service system performance Better adherence to the goals of an ideal service

system

The Good: Demonstration Sites The Good: Demonstration Sites were rated as having:were rated as having:

1

2

3

4

5

Coordination Overall Coordination for MilitaryDependents

Ft. Bragg Ft. Campbell Ft. Stewart

High

Low

The Good? Service System The Good? Service System Coordination greaterCoordination greater

Child Behavior Checklist (CBCL & YABL) Youth Self Report (YSR & YASR) Vanderbilt Functioning Index (VFI) -parent

and youth versions Caregiver Strain Index (CSI) Family Assessment Device (FAD) Individualized Measures - most severe

subscale, presenting problem - parent and youth

The Bad: OutcomesThe Bad: Outcomes

45

50

55

60

65

0 1 2 3 4 5

Years from Intake

Comparison

Demonstration

The Bad: YSR No differencesThe Bad: YSR No differences

The Bad: Vanderbilt Functioning The Bad: Vanderbilt Functioning Index Shows No Differences Index Shows No Differences

0

5

10

15

20

25

0 1 2 3 4 5

Time in Years

VF

I Sc

ore

Demo-O Comp-O

Demo-M Comp-M

More of the The Bad: Average Cost Per More of the The Bad: Average Cost Per Treated Child Higher in DemonstrationTreated Child Higher in Demonstration

N = 8,813 ChildrenN = 8,813 Children

$0

$2

$4

$6

$8

$10

FY 93 Demonstration Period (over 3 years)

Demonstration Comparison

Ave

rage

Cos

t in

$100

Dol

lars

Demonstration increased access

Demonstration used less restrictive settings

Greater client satisfaction at Demonstration

But, no differences in clinical outcomes

Clients got better at both sites equally

Relapse was significant and unexplained

Costs significantly higher at Demonstration

Implications re: Systems ChangeImplications re: Systems Change

K. Hoagwood, 2003

Two major reviews of preventive intervention trials in past 3 years; 34 effective interventions cited by Greenberg et al, 1999, focused largely on parenting and school-delivered interventions

Reviews of school-based services (Rones & Hoagwood, 2000) identified 2 dozen effective programs targeting risk reduction and treatments

More than 1500 published clinical trials on outcomes of psychotherapies for youth

6 meta-analyses of their effects More than 300 published clinical trials on safety/efficacy of

psychotropic medications Approx 50 field trials of community-based services

The Good: Strength of the Evidence The Good: Strength of the Evidence on Prevention, Treatment, & Serviceson Prevention, Treatment, & Services

K. Hoagwood, 2003

The Good: Available Summaries of The Good: Available Summaries of Evidence-based InterventionsEvidence-based Interventions

Surgeon General’s Mental Health Report, 1999

Surgeon General’s Youth Violence Report, 2001

Surgeon General’s Report on Culture, Race & Ethnicity, 2002

Weisz & Jensen (1999) Mental Health Services Research

Burns, Hoagwood, Mrazek (2000) Child Clinical and Family Psychology Review

Burns & Hoagwood (2002) Eds. Evidence-based treatments for youth. Oxford University Press

Three Levels:

Child & Family Factors: e.g., Access & Acceptance

“Provider” (school, MH) Factors: e.g., Skills, Use of EB, Attitudes

Systemic and Societal Factors: e.g., Organizations, Funding Policies

Barriers vs. “Enhancers” to Delivery Barriers vs. “Enhancers” to Delivery of Effective MH Servicesof Effective MH Services

“Effective” Services

Efficacious Treatments

Redesign Attempts to Circumvent Multiple Levels Redesign Attempts to Circumvent Multiple Levels of Obstacles: Wraparound and MST: of Obstacles: Wraparound and MST: Common Common

CharacteristicsCharacteristics

Redesign Attempts to Circumvent Multiple Levels Redesign Attempts to Circumvent Multiple Levels of Obstacles: Wraparound and MST: of Obstacles: Wraparound and MST: Common Common

CharacteristicsCharacteristics

Comprehensive community-based interventionsfor severe emotional and behavioral disorders

System of care values

Provided at home, in schools, and neighborhoods

Operated within any human service sector

Developed and studied in the ‘real world’

Trainers and training materials developed

Outcomes monitored

Less expensive than residential care

Fidelity measures

Comprehensive community-based interventionsfor severe emotional and behavioral disorders

System of care values

Provided at home, in schools, and neighborhoods

Operated within any human service sector

Developed and studied in the ‘real world’

Trainers and training materials developed

Outcomes monitored

Less expensive than residential care

Fidelity measures

HOME BASED MODEL (vs. outpatient, inpatient)

Low case load (4-6 families/therapist) 24 hr./7 day availability of clinicians Target children at risk of placement Services provided to the family (&

individuals) Time limited (average 4 months)

Multi-systemic Therapy (MST) ModelMulti-systemic Therapy (MST) Model

In Comparison with Control Groups, MST: Improved family relations and functioning Increased school attendance Decreased adolescent psychiatric

symptoms Decreased adolescent substance use Decreased long-term rates of rearrest 25%

to 70%

Consistent MST clinical outcomesConsistent MST clinical outcomes

Missouri Delinquency ProjectMissouri Delinquency Project

MSTCompleters

MSTDropouts

IT Completers

IT Dropouts

Refusers0%

20%

40%

60%

80%

100%

120%

0 0.6 1.1 1.7 2.2 2.8 3.3 3.9 4.4 5

Years Past Treatment Termination

Per

cen

t o

f O

ffen

der

sN

ot

Re-

Arr

este

d

The Good with the Bad: Effective The Good with the Bad: Effective Community-based Services Cost $dollarsCommunity-based Services Cost $dollars

The Good with the Bad: Effective The Good with the Bad: Effective Community-based Services Cost $dollarsCommunity-based Services Cost $dollars

Intensive Case Management Intensive Case Management (including Wraparound)(including Wraparound)

Cost: $2,500 - 35,000 per Cost: $2,500 - 35,000 per yearyear

Multisystemic TherapyMultisystemic Therapy

Cost: $5,063 per yearCost: $5,063 per year

5 RCTs and 1 quasi-5 RCTs and 1 quasi-experimentalexperimental

•less restrictive placementsless restrictive placements

•some increased functioningsome increased functioning

7 RCTs and 1 quasi-7 RCTs and 1 quasi-

experimental experimental

•fewer arrestsfewer arrests

•fewer placementsfewer placements

•decreased aggressive decreased aggressive

behaviorbehavior

5 RCTs and 1 quasi-5 RCTs and 1 quasi-experimentalexperimental

•less restrictive placementsless restrictive placements

•some increased functioningsome increased functioning

7 RCTs and 1 quasi-7 RCTs and 1 quasi-

experimental experimental

•fewer arrestsfewer arrests

•fewer placementsfewer placements

•decreased aggressive decreased aggressive

behaviorbehavior

Teen Screen Study

REDESIGN: CO-LOCATION EFFECTS ON REFERRAL FOR TREATMENT IN A SCHOOL-

SCREENED POPULATION

Average vs. Focused Co-location

Referral

In need but not yet obtaining Rx 100% 100% 100%

Refuse referral 33% 19% 5%

Fail first visit 22% 25% 4%

First visit only 20% 17% 6%

> One visit 25% 38% 85%

But WHAT ELSEBut WHAT ELSE

goes into the service system?goes into the service system?

But WHAT ELSEBut WHAT ELSE

goes into the service system?goes into the service system?

The Good: Evidence-Based Outpatient TreatmentThe Good: Evidence-Based Outpatient Treatment

Well-Established Probably EfficaciousWell-Established Probably EfficaciousDEPRESSION

None Self-Control (children)Coping with Depression (adolescents)

ADHDBehavioral Parent Training Behavioral Management TrainingBehavioral Interventions in the Classroom Behavioral Modification in the Classroom

ANXIETYNone Cognitive-Behavioral

PhobiaParticipant Modeling Imaginal and In Vivo DesensitizationReinforced Practice Live and Filmed Modeling

DISRUPTIVE BEHAVIORLiving with Children Delinquency Prevention ProgramVideotape Modeling Parent-Child Interaction Therapy

Parent Training ProgramTime-Out Plus Signal Seat Treatment

Anger Coping TherapyProblem Solving Skills Training

Anger Control Training with Stress Inocul.Assertiveness TrainingMultisystemic TherapyRational-Emotive Therapy

PreschoolPreschool

AdolescentAdolescent

School AgeSchool Age

Source: Journal of Clinical Child Psychology, Volume 27, Number 2, 1998

The Good, the Bad, and the Ugly: Varying Evidence for Medications in Childhood Disorders

STRONGSTRONG

ADHDADHD StimulantsStimulantsTCAsTCAs

MODERATEMODERATE

WEAKWEAK

DEPRESSIONDEPRESSION SSRIsSSRIs

AUTISM AUTISM AntipsychoticsAntipsychotics

OCDOCD SSRIs, TCAsSSRIs, TCAs

ODD/CDODD/CD Antipsychotics, Mood stabilizers, Antipsychotics, Mood stabilizers, StimulantsStimulants

ANXIETYANXIETY SSRIsSSRIs

BIPOLAR BIPOLAR LithiumLithium

TOURETTE’S TOURETTE’S AntipsychoticsAntipsychotics

K. Hoagwood, 2003

The Good: Other Comprehensive Community-Based The Good: Other Comprehensive Community-Based InterventionsInterventions

The Good: Other Comprehensive Community-Based The Good: Other Comprehensive Community-Based InterventionsInterventions

Treatment Foster CareTreatment Foster Care

Family Education and SupportFamily Education and Support

MentoringMentoring

Respite Services Respite Services

Crisis ServicesCrisis Services

Treatment Foster CareTreatment Foster Care

Family Education and SupportFamily Education and Support

MentoringMentoring

Respite Services Respite Services

Crisis ServicesCrisis Services

4 RCTs4 RCTs•more rapid improvementmore rapid improvement•decreased aggressiondecreased aggression•better post-discharge outcomesbetter post-discharge outcomes

1 RCT1 RCT•increased knowledge and self-efficacyincreased knowledge and self-efficacy

1 RCT1 RCT•less substance use and aggression;less substance use and aggression;•better school, peer, and family better school, peer, and family

functioningfunctioning

1 quasi-experimental1 quasi-experimental•fewer placementsfewer placements•reduced family stressreduced family stress

0 controlled, 1 pre-post0 controlled, 1 pre-post•placement prevented in 60-90% of placement prevented in 60-90% of

casescases

4 RCTs4 RCTs•more rapid improvementmore rapid improvement•decreased aggressiondecreased aggression•better post-discharge outcomesbetter post-discharge outcomes

1 RCT1 RCT•increased knowledge and self-efficacyincreased knowledge and self-efficacy

1 RCT1 RCT•less substance use and aggression;less substance use and aggression;•better school, peer, and family better school, peer, and family

functioningfunctioning

1 quasi-experimental1 quasi-experimental•fewer placementsfewer placements•reduced family stressreduced family stress

0 controlled, 1 pre-post0 controlled, 1 pre-post•placement prevented in 60-90% of placement prevented in 60-90% of

casescases

The Ugly: Evidence for Institutionally-Based CareThe Ugly: Evidence for Institutionally-Based CareThe Ugly: Evidence for Institutionally-Based CareThe Ugly: Evidence for Institutionally-Based Care

HospitalHospital

ResidentialResidentialTreatmentTreatmentCenterCenter

Group HomeGroup Home

Partial Partial HospitalizationHospitalization

HospitalHospital

ResidentialResidentialTreatmentTreatmentCenterCenter

Group HomeGroup Home

Partial Partial HospitalizationHospitalization

3 randomized clinical trials 3 randomized clinical trials

•findings in favor of community comparison findings in favor of community comparison conditionsconditions

2 quasi-experimental study designs2 quasi-experimental study designs

•Project Re-Ed: gains versus untreatedProject Re-Ed: gains versus untreated

•Gains in residential treatment center were equalGains in residential treatment center were equalto treatment foster care (TFC @ one-half cost)to treatment foster care (TFC @ one-half cost)

2 quasi-experimental study designs2 quasi-experimental study designs

•mixed findings -- gains and mixed findings -- gains and deterioration (arrest rates)deterioration (arrest rates)

1 randomized clinical trial1 randomized clinical trial

•partial hospital versus wait-list controlspartial hospital versus wait-list controls

•benefits at 6 months for behavior symptoms,benefits at 6 months for behavior symptoms,and familyand family

3 randomized clinical trials 3 randomized clinical trials

•findings in favor of community comparison findings in favor of community comparison conditionsconditions

2 quasi-experimental study designs2 quasi-experimental study designs

•Project Re-Ed: gains versus untreatedProject Re-Ed: gains versus untreated

•Gains in residential treatment center were equalGains in residential treatment center were equalto treatment foster care (TFC @ one-half cost)to treatment foster care (TFC @ one-half cost)

2 quasi-experimental study designs2 quasi-experimental study designs

•mixed findings -- gains and mixed findings -- gains and deterioration (arrest rates)deterioration (arrest rates)

1 randomized clinical trial1 randomized clinical trial

•partial hospital versus wait-list controlspartial hospital versus wait-list controls

•benefits at 6 months for behavior symptoms,benefits at 6 months for behavior symptoms,and familyand family

DARE (5th and 6th grade curriculum) Gun Buyback programs Boot Camps Peer counseling programs Summer job programs (at risk youth) Home detention with electronic monitoring Wilderness / challenge programs Generic counseling (non-behavioral)

More Ugly: What doesn’t work More Ugly: What doesn’t work (Elliott, 2000)(Elliott, 2000)

The Good, the Bad, and the Ugly:The Good, the Bad, and the Ugly:

Variation in the Evidence BaseVariation in the Evidence Base

STRONG STRONG EVIDENCEEVIDENCE

Multisystemic TherapyMultisystemic TherapyIntensive Case Intensive Case ManagementManagementTreatment Foster CareTreatment Foster Care

MODERATE MODERATE EVIDENCE EVIDENCE

NEGATIVE, MIXED,NEGATIVE, MIXED, or NO EVIDENCEor NO EVIDENCE

Family Education and Family Education and SupportSupportMentoringMentoringPartial HospitalizationPartial HospitalizationRespite CareRespite Care

Psychiatric Hospital (Inpatient)Psychiatric Hospital (Inpatient)

Residential Treatment CenterResidential Treatment CenterGroup HomeGroup HomeCrisis InterventionCrisis Intervention

K. Hoagwood, 2003

Extra-Organizational Context(financial policies, methods of reimbursement, state policies)

Organizational Clinician Child & Family

Fit Fidelity Outcomesculture, climate, structure

Clinical care processes training, supervision, alliance

Stakeholder engagementshared understanding of problems and choice

Model for Effective Implementation of EBPsModel for Effective Implementation of EBPs

K. Hoagwood, 2003

Dissemination and Adoption of Dissemination and Adoption of New Interventions New Interventions

Dissemination and Adoption of Dissemination and Adoption of New Interventions New Interventions

Interpersonal contact

Organizational support

Persistent championship of the intervention

Adaptability of the intervention to local situations

Availability of credible evidence of success

Ongoing technical assistance, consultation

Interpersonal contact

Organizational support

Persistent championship of the intervention

Adaptability of the intervention to local situations

Availability of credible evidence of success

Ongoing technical assistance, consultation

Source: Backer, Liberman, & Kuehnel (1986) Dissemination and Adoption of Innovative Source: Backer, Liberman, & Kuehnel (1986) Dissemination and Adoption of Innovative Psychosocial Interventions. Psychosocial Interventions. Journal of Consulting and Clinical PsychologyJournal of Consulting and Clinical Psychology, 54:111-118; , 54:111-118; Jensen, Hoagwood, & Trickett (1997) From Ivory Towers to Earthen Trenches. Jensen, Hoagwood, & Trickett (1997) From Ivory Towers to Earthen Trenches. J AppliiedJ AppliiedDevelopmental PsychologyDevelopmental Psychology

Developing Consensus for Useful (and Developing Consensus for Useful (and Used) Knowledge: A Primer?Used) Knowledge: A Primer?

“Begin with the end in mind” – who must eventually apply the knowledge?

Identification of key stakeholders enablers/disseminators (policy makers, mavens, connectors), and

those with veto power Formation of key working group

Shared view and identification of the problem Consensus/agreed-upon knowledge: College of Cardinals, Rand

methodology, variations, etc. “Buy-in”, involvement in developing solution Identify who else is needed for implementation

Strategic plan (varies by situation) Tool development, dissemination

School Administrators – Local, State, National Policy makers (state, national) Taxpayers Key Administrators: NASDE, NASMHPD Technical assistance centers, vendors for services Federal agencies, (DOE, OSEP) Educational and MH researchers Professional groups: NASP, AACAP, APA, NASW, School

counselor associations Parents, Child Advocacy groups Teachers (regular and special ed), NEA, AFT

Stakeholders w/Vested Interests in Improving Stakeholders w/Vested Interests in Improving

MH Practices in SchoolsMH Practices in Schools

““To tell you the truth, I don’t have much hope for this broad-based coalition”To tell you the truth, I don’t have much hope for this broad-based coalition”

Civil rights of the individual child, of parents Mental health needs of the individual child To diagnose or not to diagnose: philosophical

differences, system responsibility differences Vested interests in current systems and methods Who is responsible? How to serve children who are identified? Ethical issues, confidentiality

Special Challenges in Reaching Consensus on Special Challenges in Reaching Consensus on

Assessment, Diagnosis, & InterventionsAssessment, Diagnosis, & Interventions

Mission disparity between schools and school-based mental health programs Schools may not want to talk about mental health issues

(need to fly in under the radar?) You have to go into the school and help solve the mission

of the school Lack of a clear mandate for schools to deal with mental

health treatment issues directly Inadequate training in mental health issues for teachers,

administrators and other school professionals

Problems Identified by Consensus GroupProblems Identified by Consensus Group

Lack of strong advocacy efforts for school-based mental health Professional organizations need to unite around this issue Mental health is most likely to get into schools through

advocacy at the local Lack of a sustaining fiscal base and infrastructure to

support school-based mental health programs Funding for school-based mental health comes from various

sources, difficult to financially sustain SBMH services Shared Agenda paper published by NASDMH and state planning

grant program as an example of how to build infrastructure

Problems Identified by Consensus GroupProblems Identified by Consensus Group

Challenges associated with disseminating evidence-based mental health programs in schools Need for user friendly materials, infrastructure to

support training, and ongoing training Planning and scheduling issues Fidelity to program Adaptation

Problems Identified by Consensus GroupProblems Identified by Consensus Group

Identify what is being done to support SBMH in terms of 1) national and state level SBMH coalitions; 2) services in place within the portfolios of various federal and state agencies; 3) existing school reform efforts; and 4) existing materials, toolkits, communication strategies, training, etc.

Establish clear expectations that all schools and all school-based personnel will address the social-emotional as well as academic development of students

Develop policy initiatives to improve communication, collaboration, cooperation, and coalition-forming among local, state and federal agencies and advocacy groups that are responsible for supporting the social-emotional well-being of school students

Fund research investigating the variables related to sustaining implementation of effective school-based mental health practices

Develop a research recommendation to OSEP and other agencies regarding funding research that investigates variables related to sustained implementation of effective practices

Consensus Recommendations: National LevelConsensus Recommendations: National Level

Promote collaboration in school mental health research between NIH, the Department of Education, and SAMSA.

Consolidate existing curricula (on social-emotional competencies) for teaching and models of training

Identify core content for all disciplines that can be infused into the existing curriculum relating to course work in the areas of teacher, principal, and school psychologist preparation, etc.

Build consensus among experts re: knowledge and skills educators need in order to promote S-E competencies in children.

Convene a group to explore potential funding sources for training initiatives Work through credentialing bodies like NKAPA to promote inclusion of S-E competencies

in standards for teacher and educator trainingDevelop an infrastructure and 5-year plan to keep summit group working together.

Convene subgroups on key issues such as fiscal resources, teacher education, and evaluation/assessment

Develop a consensus document and allow other organizations to sign off on it

Consensus Recommendations: National LevelConsensus Recommendations: National Level

Market the value of school-based mental health and evidence-based practice to state education departments and other agencies.

Develop a consolidated web clearinghouse and a 800 number that people can access to get information about evidence-based school mental health practices

Develop a report card of the 10 key indicators of best practice in the area of school mental health (support Learning First Alliance efforts in this area). This report card could be used by states to assess how well schools are doing in this area

Develop a state-to-state network to promote school-based mental health

Consensus Recommendations: State LevelConsensus Recommendations: State Level

Identify key change agents in local school districts – a leadership core

Teach and provide ongoing consultation to these change agents the process (outlined by the group) of introducing mental health services to a school

Develop a manual/tool kit for how to introduce mental health service to a school

Consensus Recommendations: Local LevelConsensus Recommendations: Local Level

Step 1

Determine the Need

• Identify important new advances

• Work w/family partners & professional

• Obtain consensus & commitment on proven, best practices

CACMH Approach: A 4-step CACMH Approach: A 4-step processprocess

Step 2

Share the Solutions

• Through strategic partnerships

- (Parents, providers, policy- makers, etc.)

• Traditional media outlets

Step 3

Put Science to Work!

• Prepare the tools

• Implement and train at key sites

Step 4

Test, Refine, Disseminate

• Evaluate test sites

• Roll-out nationally

• Spread the word - results fed back into Step 2

Education and Mental Health InitiativeEducation and Mental Health Initiative • Improvement in MH Assessments/Interventions

• Partnerships for Change with key stakeholders• Definition of “Best Practices”• Helping stakeholders to “own” and disseminate their

message and findings

But: 14 years from innovation development to application!

• Careful site selection; systematic technical assistance vs. UC

• Feedback into political & professional processes

Knowledge Development & Application: Knowledge Development & Application: Strategic IssuesStrategic Issues

• Leadership, therapeutic alliance factors critical at all levels of “the system”• All Policy is “personal”• Begin with the end in mind: Don’t set out to build a house no one can

afford or wants to live in.• The enemy of the good is the perfect: raise the floor, not the ceiling• Win-win strategies with other key players who otherwise have veto power• ”Buy-in” -- partnership, not ownership• Co-location and blending of resources• Facilitate increased MH quality based on political action and clinical-

ethical principles, not only on RCTs, or business models/economic incentives

• Strategic action committees, short- and long-term objectives