Scaling and Sustaining PBIS:State, District, School Roles

Rob HornerUniversity of Oregon

www.pbis.org

Goals• Current status and lessons learned from states

scaling and sustaining PBIS.

• Specific suggestions for state, district and school personnel implementing PBIS (or any MTSS)

• -----------------------------------------------------------------------

• School/ District Self-Assessment

• State/District (District Capacity Assessment)

Assumptions• Knowledgeable about PBIS

• Demonstrations of effective implementation

• Read to focus on sustainability, scaling and building PBIS at all three tiers.

Why SWPBIS?

•The fundamental purpose of SWPBIS is to make schools more effective learning environments.

Predictable

Consistent

Positive

Safe

Main Messages• PBIS is a foundation for the next generation of

education.Effective (academic, behavior)

Equitable (all students succeed) Efficient (time, cost)

School-wide Positive Behavioral

Interventions and Supports (SWPBIS)

• The social culture of a school matters.

• A continuum of supports that begins with the whole school and extends to intensive, wraparound support for individual students and their families.

• Effective practices with the systems needed for high fidelity and sustainability

• Multiple tiers of intensity

Experimental Research on SWPBIS

Bradshaw, C.P., Koth, C.W., Thornton, L.A., & Leaf, P.J. (2009). Altering school climate through school-wide Positive Behavioral Interventions and Supports: Findings from a group-randomized effectiveness trial. Prevention Science, 10(2), 100-115

Bradshaw, C.P., Koth, C.W., Bevans, K.B., Ialongo, N., & Leaf, P.J. (2008). The impact of school-wide Positive Behavioral Interventions and Supports (PBIS) on the organizational health of elementary schools. School Psychology Quarterly, 23(4), 462-473.

Bradshaw, C. P., Mitchell, M. M., & Leaf, P. J. (2010). Examining the effects of School-Wide Positive Behavioral Interventions and Supports on student outcomes: Results from a randomized controlled effectiveness trial in elementary schools. Journal of Positive Behavior Interventions, 12, 133-148.

Bradshaw, C.P., Reinke, W. M., Brown, L. D., Bevans, K.B., & Leaf, P.J. (2008). Implementation of school-wide Positive Behavioral Interventions and Supports (PBIS) in elementary schools: Observations from a randomized trial. Education & Treatment of Children, 31, 1-26.

Bradshaw, C., Waasdorp, T., Leaf. P., (in press). Effects of School-wide positive behavioral interventions and supports on child behavior problems and adjustment. Pediatrics.

Horner, R., Sugai, G., Smolkowski, K., Eber, L., Nakasato, J., Todd, A., & Esperanza, J., (2009). A randomized, wait-list controlled effectiveness trial assessing school-wide positive behavior support in elementary schools. Journal of Positive Behavior Interventions, 11, 133-145.

Horner, R. H., Sugai, G., & Anderson, C. M. (2010). Examining the evidence base for school-wide positive behavior support. Focus on Exceptionality, 42(8), 1-14.

Ross, S. W., Endrulat, N. R., & Horner, R. H. (2012). Adult outcomes of school-wide positive behavior support. Journal of Positive Behavioral Interventions. 14(2) 118-128.Waasdorp, T., Bradshaw, C., & Leaf , P., (2012) The Impact of Schoolwide Positive Behavioral Interventions and Supports on Bullying and Peer Rejection: A Randomized Controlled Effectiveness Trial. Archive of Pediatric Adolescent Medicine. 2012;166(2):149-156 Bradshaw, Pas, Goldweber, Rosenberg, & Leaf, 2012Freeman, J., Simonsen, B., McCoach D.B., Sugai, G., Lombardi, A., & Horner, ( submitted) Implementation Effects of School-wide Positive Behavior Interventions and Supports on Academic, Attendance, and Behavior Outcomes in High Schools.

SWPBIS Experimentally Related to:1. Reduction in problem behavior2. Increased academic performance3. Increased attendance4. Improved perception of safety5. Reduction in bullying behaviors

6. Improved organizational efficiency7. Reduction in staff turnover8. Increased perception of teacher efficacy

9. Improved Social Emotional competence

Current Status

• Multiple Models for Scaling

Leadership Team

FundingVisibility Political

Support

Training Coaching Evaluation

Local School Demonstrtions

Active Coordination

TechnicalExpertise

Policy

Implementation Science Frameworks

WHO

Teams

WHEN

Stages

HOW

Drivers

HOW

Cycles

WHAT

Interventions

© Fixsen & Blase, 2008

Performance Assessment (Fidelity)

Coaching

Training

Selection

Systems Intervention

Facilitative Administration

Decision Support Data System

ImplementationDrivers

Com

pete

ncy

Dri

vers

Com

pete

ncy

Dri

vers

Organization D

rivers

Organization D

rivers

LeadershipLeadership

Adaptive Technical

Successful Student Outcomes

Program/Initiative/Framework (e.g. RtI)

Implementation Stages

• Exploration• Installation• Initial Implementation• Full Implementation

Implementation occurs in stages:

Fixsen, Naoom, Blase, Friedman, & Wallace, 2005

2 – 3 Years

Stages of ImplementationFocus Stage Description

Exploration/ Adoption

Decision regarding commitment to adopting the program/practices and supporting successful implementation.

Installation Set up infrastructure so that successful implementation can take place and be supported. Establish team and data systems, conduct audit, develop plan.

Initial Implementation

Try out the practices, work out details, learn and improve before expanding to other contexts.

Full Implementation

Expand the program/practices to other locations, individuals, times- adjust from learning in initial implementation.

Continuous Improvement/ Regeneration

Make it easier, more efficient. Embed within current practices.

Work to do it right!

Work to do it

better!

Should we do it!

Steve Goodman

Scaling up School-wide Positive Behavioral Interventions and Supports:The Experiences of Seven States with Documented SuccessRob Horner, Don Kincaid, George Sugai, Tim Lewis, Lucille Eber, Susan Barrett, Celeste Rossetto Dickey, Mary Richter, Erin Sullivan, Cyndi Boezio, Nancy Johnson, (2014 ), JPBI

Exploration Installation Initial Imp Full Imp

Leadership Team

Funding

Visibility

Political Support

Policy

Training

Coaching

Expertise

Evaluation

Demos

Interviews and Data Reviews with

the PBIS implementers from

Seven States that had at least

500 schools using PBIS.

Exploration and Adoption

Installation Initial Implementation

Full Implementation

Innovation and sustainability

Leadership Team

(coordination)

Do you have a state leadership team?

If you do, how was your first leadership team developed?

Who were members?

Who supported/lead the team through the exploration process?

Was any sort of self-assessment completed (e.g. the PBIS Implementation Blueprint Assessment)?

What was the role of State agency personnel in the exploration phase?

What were critical issues that confronted the team as it began to install systems changes?

What were specific activities the team did to ensure success of the initial implementation efforts?

Did the team change personnel or functioning as the # of schools/districts increased?

What has the Leadership team done to insure sustainability?

In what areas is the State “innovating” and contributing to the research and practice of PBIS (e.g. linking PBIS with literacy or math)?

Descriptive Summary: Oregon

1995-96

1996-97

1997-98

1998-99

1999-00

2000-01

2001-02

2002-03

2003-04

2004-05

2005-06

2006-07

2007-08

2008-09

2009-10

0

100

200

300

400

500

600

700

Exploration / Installation / Initial Imp /Full Imp & Innovate

Descriptive Summary: Missouri

97-98

98-99

99-00

00-01

01-02

02-03

03-04

04-05

05-06

06-07

07-08

08-09

09-10

10-11

0

100

200

300

400

500

600

700

Exploration / Installation /Initial Imp / Full Imp & Innovate

Descriptive Summary: North

Carolina

00-01 01-02 02-03 03-04 04-05 05-06 06-07 07-08 08-09 09-100

100

200

300

400

500

600

700

800

900

1000

Exploration / Installation / Initial & Full Imp / Innovate

Descriptive Summary: Colorado

02-03 03-04 04-05 05-06 06-07 07-08 08-09 09-10 10-110

100

200

300

400

500

600

700

800

Exploration / Installation / Initial & Full Imp / Innovate

Descriptive Summary: Florida

01-02 02-03 03-04 04-05 05-06 06-07 07-08 08-09 09-100

200

400

600

800

1000

1200

Exploration/ Installation/ Initial Imp / Full Imp / Innovate

Descriptive Summary: Maryland

1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 20100

100

200

300

400

500

600

700

800

900

Exploration / Installation / Initial Imp / Full Imp / Innovate

Descriptive Summary: Illinois

98-99 99-00 00-01 01-02 02-03 03-04 04-05 05-06 06-07 07-08 08-09 09-10 10-110

200

400

600

800

1000

1200

1400

1600

Exploration / Installation / Initial Imp /Full Imp & Innovate

Lessons Learned• Multiple approaches to achieving scaled

implementation• Colorado: Started with Leadership Team• Illinois: Started with Leadership Advocates and built team only after

implementation expanded.• Missouri: Strong initial demonstrations led to strong state support

• All states began with small “demonstrations” that documented the feasibility and impact of SWPBIS.

• Only when states reached 100-200 demonstrations did scaling occur. Four core features needed for scaling:

• Administrative Leadership / Support/ Funding• Technical capacity (Local training, coaching, evaluation and behavioral

expertise)• Local Demonstrations of feasibility and impact (100-200)• Evaluation data system (to support continuous improvement)

• Essential role of Data: Fidelity data AND Outcome data

Lessons Learned• Scaling is NOT linear

• Sustained scaling requires continuous regeneration

• Both Bottom Up and Top Down

• Threats to Scaling:o Competing initiativeso The seductive lure of the “new idea”o Leadership turnovero Legislative mandateso Fiscal constraint

Regular Dissemination of

Fidelity and Impact data is the best

“protective factor” for threats to scaling

Lessons Learned• Scaling requires planned efficiency

o The unit cost of implementation must decrease as the number of adoptions increases.

• Shift from external trainers to within state/district trainers• Use local demonstrations as exemplars • Increased coaching capacity can decrease investment in

training• Improved “selection” of personnel decreases turnover

and development costs• Use existing professional development and evaluation

resources differently

• Basic Message: The implementation practices that are needed to establish initial exemplars may be different from the practices used to establish large scale adoption.

o Jennifer Coffey, 2008

Sustainability• Most educational innovations do not endure beyond 9

months

• The likelihood of sustaining is NOT related to effectiveness

• Achieving Sustainability is essential for cost effectiveness and scaling up to levels of social importance.

• The variables that affect sustaining implementation are not necessarily the variables that affect initial adoption.

(Latham, 1988)

Research on PBIS Sustainability

• Coffey & Horner, 2012• Exceptional Children, 78, 407-422.

• Sample: 285 schools with SET scores--------------------------------------------------------• What predicted INITIAL Adopt of PBIS

• What predicted SUSTAINED use of PBIS

A PBIS Sustainability Study

(Coffey & Horner, 2012)

Never met SET (<80%)

Met SET (≥80%)

Implementers vs. Non-implementers

SET Subscale SET Overall

Expectations Defined

Reward System

Responding to Violations

Management (team and admin)

District-Level Support

Monitoring and Decision-Making

Expectations Taught

Met and lost SET in

5 years

Sustained SET for ≥5

years

Sustainers vs. Non-sustainers

SET Subscale SET Overall

Expectations Defined

Responding to Violations

District-Level Support

Monitoring and Decision-Making

Expectations Taught

Reward SystemManagement

(team and admin)

Perceived Importance of Contextual Features for Sustainability of PBIS

• McIntosh, K., Predy, L., Upreti, G., Hume, A. E. & Mathews, S. (2014). Perceptions of contextual features related to implementation and sustainability of School-wide Positive Behavior Support. Journal of Positive behavior Interventions, 16, 29-41.

1. School administrators actively support PBIS

2. School administrators describes PBIS as a top priority for the school

3. A school administrator regularly attends and participates in PBIS team meetings

4. The PBIS school team is well organized and operates efficiently

5. The school administrators ensure that the PBIS team has regularly scheduled time to meet

Most Important Features for

Sustainability

• Recent studies on sustainability of PBSo Perceptions of critical features for sustainability

(McIntosh, Predy, Hume, Turri, & Mathews, in press)o Factors predicting sustainability

(McIntosh et al., in press)o Critical features of PBS systems

(Mathews, McIntosh, Frank, & May, under review)

Research on Sustainability

Building Capacity

• State

• District

• School

• Classroom Traditional Behavior/ Classroom Management

Initial PBIS

Current PBIS

SISEP and Scaling UP

Districts• Coherent District Policy

o Social behavior is a priority in district improvement plano District commitment to selecting practices that are evidence-basedo District process for aligning multiple initiatives.

• Evaluation Capacityo Data systems that inform decision-making and provide policy feedback

** Fidelity and Impact

• Recruitment, Hiring, Evaluationo “Preference will be given to individuals with knowledge and experience

in implementation of multi-tiered academic and behavior supports.”

WHAT

Interventions

HOW

Cycles

Team Based Implementation

WHO

Teams

Districts

• Annual Faculty/Staff Orientationo Defines PBIS as a priorityo Defines what to expect in a school using PBIS.o 30-60 min of annual orientation

• Professional Development (Training)o PD is always tied to core improvement goalso PD typically involves distributed training (multiple events)o PD is always linked to on-site coaching.o PD is always linked to fidelity measure

• Coaching

HOW

Drivers

Districts

• Annual Faculty/Staff Evaluationso Evaluations include assessment of “implementation of multi-tiered

systems of academic and behavior support.”

• Build Technical Capacity to Implement Tier II and Tier III supports

• -----------------------------------------------------------• Assessing District Capacity

o The District Capacity Assessment (DCA)

Team

1.1 Team Authority: Tier I planning team exists, and meets at least monthly to use implementation fidelity and student outcome data to manage universal, school-wide support systems.

School organizational chart

Tier I systems planning team meeting minutes

Fidelity tool(s)

0 = No team identified to manage and monitor Tier I supports

1 = PBIS Team in place but does review data regularly

2 = PBIS Team exists with the authority to implement at least Tier I PBIS, and has at least monthly meetings with access to fidelity and student outcome data to guide action planning

Feature Data Sources

Scoring Criteria

0 = Not implemented1 = Partially implemented2 = Fully implemented

Implementation

1.4 Behavioral Expectations: School has five or fewer positively stated behavioral expectations for student and staff behaviors (i.e., teaching matrix) defined and in place.

Staff handbook Student

handbook Walk through

reports

0 = No clearly stated behavioral expectations have been identified and posted

1 = Behavioral expectations identified but no matrix

2 = Five or fewer behavioral expectations exist that are positive, posted, and identified for specific settings

Feature Data Sources

Scoring Criteria

0 = Not implemented1 = Partially implemented2 = Fully implemented

Subscale and Items Action Who When Next Update1. Commitment and

Leadership 

       

1. Systems Alignment 

 

       

1. Action Planning   

       

1. Performance Feedback  

       

1. Selection  

       

1. Training  

       

1. Coaching  

       

1. Decision Support System  

       

1. Facilitative Administration

 

       

1. Systems Intervention  

       

Next Steps• State

o Use SCA to define state capacityo Invest in state leadership teamo Build scaling plan with both “bottom up” and “top down” strategies

• Fuchs et al., 2014

• Districto Use DCA to define district capacityo Use District Self-Assessment for targeted content areas.o Invest in Leadership Teamo Build capacity to Train, Coach, Evaluate, Sustaino Collect school “fidelity” data regularly (TFI)o Collect student outcome data regularly

• Tier I ODR• Tier II CICO points• Tier III % students “Progressing” ISIS