Barbara Sims

Debbie Egan

Dean L. Fixsen

Karen A. Blase

Michelle A. Duda

Using Implementation Frameworks to Identify Evidence Based Practices

2011 PBIS National Forum

Rosemont IL

October 27, 2011

– Lisbeth Schorr, 1993

Implementation

Successful programs do not contain the seeds of their own replication.

Implementation Gap

RESEARCH PRACTICEGAP

Implementation is defined as a specified set of activities designed to put into practice an activity or program of known dimensions.

IMPLEMENTATION

Implementation Gap

RESEARCH PRACTICEGAP

Why Focus on Implementation?

IMPLEMENTATION

“Students cannot benefit from interventions they do not experience.”

Implementation Science

Effective NOT Effective

Effective

NOT Effective

IMPLEMENTATION

INT

ER

VE

NT

ION Actual Benefits

(Institute of Medicine, 2000; 2001; 2009; New Freedom Commission on Mental Health, 2003; National Commission on Excellence in Education,1983; Department of Health and Human Services, 1999)

Inconsistent; Not Sustainable; Poor outcomes

Unpredictable or poor outcomes;

Poor outcomes; Sometimes harmful

from Mark Lipsey’s 2009 Meta-analytic overview of the primary factors that characterize effective juvenile offender interventions – “. . . in some analyses, the quality with which the intervention is implemented has been as strongly related to recidivism effects as the type of program, so much so that a well-implemented intervention of an inherently less efficacious type can outperform a more efficacious one that is poorly implemented.”

Who’s Accountable for Learning?

Student Practitioner System

Active Implementation

Letting “It” happen. . . Innovation occurs without intervention

Helping “It” happen. . . Interested innovators figure it out on their own

Making “It” happen. . . Active use of strategies to support the adoption

of the innovation

Active installation of supports for the implementation of the innovation

Based on Greenhalgh, Robert, MacFarlane, Bate, & Kyriakidou, 2004

Active Implementation Frameworks

Successful implementation on a useful scale requires. . . Purposeful matching of critical implementation

activities to the stage of the process – “STAGES OF IMPLEMENTATION”

Active use of implementation core components “best practices”– “IMPLEMENTATION DRIVERS”

Organized, expert assistance – “IMPLEMENTATION TEAMS”

A focus on continuous, purposeful improvement – “IMPROVEMENT PROCESSES”

Implementation Stages

Exploration

•Assess needs

•Examine innovations

•Examine Implementation

•Assess fit

2 - 4 Years

Installation

Acquire resources

Prepare organization

Prepare implementation

Prepare staff

Initial Implementation

Implementation drivers

Manage change

Data systems

Improvement cycles

Full Implementation

Implementation drivers

Implementation outcomes

Innovation outcomes

Standard practice

Exploration: The Big Picture

What happens during Exploration?

Formalize Team Structures

Determine Need and Identify Options

Assess “Fit” and Feasibility

Promote “Buy in” for the innovation and for implementation supports

Re-Assess

Formalize Team Structures

Who will be accountable on a day-to-day basis for ensuring this work is done?

How will State leadership be a part of this process to ensure that successes are operationalized and barriers are removed?

Determine Need and Identify Options

What do your current data suggest is the most critical or pivotal need?

What is the supporting research or evidence of the strategies you are considering?

Assess Fit and Feasibility

What are the priorities of your State?

What is your theory of change (i.e., logic model, blueprint, outcome map, etc.)?

How will you measure progress toward that goal at the SEA? At the LEA?

Who will do what differently at the SEA to impact that outcome? At the LEA?

Promote Buy-In

How will readiness be created at the SEA?

How will readiness be created at the LEA?

Re-Assess and Decide

What has emerged during Exploration that impacts your decision?

GUIDING IMPLEMENTATION QUESTIONS

EBP: 5 Point Rating Scale: High = 5; Medium = 3; Low = 1. Midpoints can be used and scored as a 2 or 4.

High Medium Low

Need

Fit

Resources Availability

Evidence

Readiness for Replication

Capacity to Implement

Total Score:

Need in the Educational Setting,Socially Significant Issues,Parent & Community Perceptions of Need,Objective Data indicating Need

Need

Fit

Fit with current - • Initiatives• RtI Implementation • School and District

Priorities• Organizational structures• Community Values

ResourceAvailability

Resources Curricula & Classroom Materials, IT requirements, Staffing,Training and PD,Data Systems,Coaching & Supervision,Administrative & system supports needed

EvidenceOutcomes – Is it worth it?Fidelity dataCost – effectiveness data Number of studiesPopulation similaritiesDiverse cultural groupsEfficacy or Effectiveness

Evidence

Assessing Evidence-Based Programs and Practices

ReadinessQualified purveyor Expert or TA availableMature sites to observe# of replicationsHow well is it operationalized?Are Imp Drivers operationalized?

Intervention Readiness for Replication

CapacityStaff meet minimum qualificationsAble to sustain Implementation Drivers • Financially• StructurallyBuy-in process operationalized• Educators • Administrators• Families

Capacity to Implement

© National Implementation Research Network- 2009

Implementation Team

Management Team

PractitionersChildren

Po

licy En

abled

P

ractice (PE

P)

Pra

ctic

e In

form

ed

Po

licy

(P

IP)

Sys

tem

C

han

ge

Org

aniz

ed,

“Exp

ert”

Ass

ista

nce Adaptive Challenges

• RFP methods• Service Silos• Salaries• Funding• Credentialing• Licensing• Time/ scheduling• Union contracts• Duplication• Fragmentation• Hiring criteria• Federal/ State laws

Practice-Policy Feedback Loops

Implementation

Review and synthesis of the implementation research and evaluation literature (1970 – 2004)

Multi-disciplinary

Multi-sector

Multi-national

www.scalingup.org