Using the Self Assessment to Expand and Sustain SW-PBS

Susan Barrettwww.pbis.org

www.pbismaryland.org

OutcomesBuild Action Plan Using Self Assessment

State, LEA, cluster???

1. Introduce Topic2. Discuss3. Action Plan

3

PBS Systems Implementation Logic

Leadership Team

Management Team

FundingMarketing

Visibility Political Support

Training Coaching Evaluation

Active Coordination

Local School Teams/Demonstrations

Braiding Initiatives

Implementers BlueprintSelf Assessment

“More like guidelines”Provides common language and framework

Outcomes Use revised self assessment to build local

infrastructure Discussion Next Steps to ensure we sustain and move forward

for expansion (more schools or “up the triangle”?)

Core

Sup

port

Prog

ram

: Pr

ovid

ed to

all,

inte

nded

to re

ach

mos

t.

Continuum of SupportsContinuum of Supports

The required resources to address the

problem increases

The need to enhance

environmental structures increases

The frequency for collecting

and acting upon

information increases

As the magnitude of the problem increases….

Leadership TeamLanguage is important

e.g. OISM, MISI- “Stayin Alive”Integration Teams? Who are the players?

Do you have folks who can assign dollars to a budget?

change policy like job descriptions, code of conduct?

Do you have Community Partners?Management team- to do the day to day

activities, planning, visiting schools, etc Roles and Responsibilities may change over

time depending on implementation phaseEstablish a Partnership Agreement

Local Coordinator IdentifiedImplementation Phase should determine FTE

Access to Ongoing Training and Technical Assistance – Support

Meeting with other coordinators is critical!!

Local Management TeamCreating protocols/standards (State v

Local)

FundingPartnership Agreements

Folks in charge have to understand 3-5 years, systems change

MD ExampleBlending InitiativesSocial Marketing

Economic Benefits, Serendipity( TN example)GrantsBe careful what you wish for…

Marketing and VisibilityWho are your stakeholders?Do you have a spokesperson?Using the data to create newsletters,

presentations, fact sheets, elevator business cards- important you can get access to what you need to make your case on the fly!!

www.pbismaryland.org

Colorado- Marketing exemplarBe Careful

10

Multiple levels of VisibilityState and Local Level: Presentations,

Trainings, Stakeholder meetings, Interagency efforts, (Transformation; Mental Health Integration; Wraparound)

Multiple Media: Visual, Face to Face, Written, Website

Multiple Audiences: School Administrations and Instructional Leaders; University staff; Legislators, Potential alternative funders; State and Local Political appointees; Juvenile Justice; Vendors in the System of Care; Parent and other advocacy organizations; Community Members

EvaluationWhat are your questions?Do you have the tools to answer?Can you get the answer quickly?Easy, Efficient, RelevantEconomic BenefitsBehaviorAchievementRegular Feedback to all Stakeholders- MD

example

PBIS Maryland Partner MeetingDecember 10, 2007

Management Team meets with Assistant Superintendent of Student Services and Special Education

7

18

30

5257

81

90

116110

310

18

47 47

64

82

113110

43%

56%60%

90%

82%79%

91%

97%100%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

0

20

40

60

80

100

120

140

1999 2000 2001 2002 2003 2004 2005 2006 2007

Trained and Implementing by Cohort

Trained Implementing % of Trained Implementing

NOTE: Retrained schools will be counted in the most recent retraining year only.

13

65

50

72

18

5

19

9

31

7

27

5

11

48

4

32

50

7 8

30

12

2

1412

1013

64

33

70

18

2

17

6

30

1

27

3

11

45

4

31

37

7 6

24

9

2

1311 10

100% 98%

66%

97%100%

40%

89%

67%

97%

14%

100%

60%

100%

94%

100%97%

74%

100%

75%80%

75%

100%

93% 92%

100%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

0

10

20

30

40

50

60

70

80

Alle

gany

Anne

Aru

ndel

Balti

mor

e Ci

ty

Balti

mor

e Co

.

Calv

ert

Caro

line

Carr

oll

Ceci

l

Char

les

Dor

ches

ter

Fred

eric

k

Gar

rett

Har

ford

How

ard

Kent

Mon

tgom

ery

Prin

ce G

eorg

es

Que

en A

nnes

Som

erse

t

Spec

ial

St. M

arys

Talb

ot

Was

hing

ton

Wic

omic

o

Wor

cest

er

Implementing/Inactive by LSS

Trained Implementing % of Trained Implementing

91% Rate of Return

0%10%20%30%40%50%60%70%80%90%

100%

1999 2000 2001 2002 2003 2004 2005 2006 2007

% of schools submitting data

30

60

97286

IPI Fall 2007

Preparation Initiation Implementation Maintenance

473

# of schools reporting:

Evaluation Question: Are schools in beginning or advanced stages of implementation?Data Source: Implementation Phases InventoryOverall, the IPI data suggest a relatively advanced level of self-reported implementation among the schools in Maryland

Evaluation Question: Are schools implementing School-wide PBIS?Data Source: Team Implementation Checklist (Target Criterion = 80%)Based on the total average of the 327 checklists submitted, school teams report that 78% of items are in place.

T eam Implementation C hec klis tS pring 2007

85.6 86.874.8 76.7 71.9

95.0

80.8 77.0 75.5

51.9

78.3

63.170.6

90.0 85.0 85.0 90.0100.0

83.394.0

72.5

0.010.020.030.040.050.060.070.080.090.0

100.0

Alle

gany

Ann

eA

rund

elB

altim

ore

City

Bal

timor

eC

o.

Cal

vert

Car

olin

e

Car

roll

Cha

rles

Fre

deric

k

Har

ford

How

ard

Mon

tgom

ery

Prin

ceG

eorg

esQ

ueen

Ann

es

Som

erse

t

Spe

cial

St.

Mar

ys

Tal

bot

Was

hing

ton

Wic

omic

o

Wor

cest

er

LS

S A

vera

ges

Evaluation Question: Are schools implementing SW-Positive Behavior Support?Data Source: SET (Objective Criterion = 80%)

-All regions met objective criterion-A pre-post comparison regional average shows a 69% increase.

Evaluation Question: Do 80% of the students in MD Elementary PBIS schools using SWIS receive 0 or 1 office referrals?Data Source: SWIS PBIS Elementary schools in Maryland using SWIS report that 92% of their students receive 0 or 1 office referral.

Evaluation Question: Where is the location of behavior problems in Middle Schools?Data Source: SWISThe majority of the problem behaviors reported occur in the classroom followed by hallways.

We have 500 schools that have…•Problem solving teams with admin support and• teacher buy- in•Established Universal or SW Practices•Behavior Support Coach•Local Facilitator or Coordinator•Action Plan with outcome measures•Tools to assess fidelity and outcomes•Sustainability mechanisms (ongoing training, recognition)•Data Facilitator

RecommendationsSW-PBS Framework and Logic can easily be link other major education initiatives

RtI, Character Ed, Drop Out Prevention, Bully prevention

SW-PBS can also be linked to other major mental health initiatives

School-based MH, Systems of Care

“We have a unique opportunity and responsibility to promote integration of services for students across a continuum that meets all students’ needs”

What is Coaching Capacity?Why Coach? presentationPersonnel & resources organized to facilitate,

assist, maintain, & adapt local school professional development implementation efforts

Coaching is set of responsibilities, actions, & activities

Self AssessmentRoles and ResponsibilitiesCan you get your data easily? www.pbismaryland.org

Skill Mastery Competency Level Data Practices Systems

Level One (Coaches): Fluency @ Universal

level ŅLeadÓ teams through

process (Direct)

Multiple data collection systems

Problem solving with Data

School-wide PBS essential features Effective Instruction Classroom management Rules to inform adoption of

practices Model school examples Basics of ABA PBS responses to problem

behavior

Meetings / Team roles Communication within building Consultation Targeted technical assistance Task analyze team generated universal

practices for implementation Codify practices into policy

Level Two (Coaches/Trainers): Established effective

school site(s) Fluency at Small

group and Individual student level

Work with school team chair (Indirect)

Train teams in universals

Research data collection tools (e.g., SET)

Direct observation data at individual student level

Advance ABA Functional Behavioral Assessment Social Skill instruction

Systematic (data-based) student identification for small group/ individual supports

Create/support student support team or process

Task analyze team generated small group/individual practices for implementation

Adapting universal systems to support generalization of small group /individual plans

Consult with other agencies/parents Codify practices into policy

Level Three (Coordinator): Work with multiple

schools Train teams universal

Š individual level Train coaches Establish district-wide

system

Evaluation of district-wide efforts

Data Šdecision rules to identify needed supports within/across schools

Link school needs (data-based) to district professional development

Effective professional development / training skills

Map district policy to essential features of PBS

Leadership team roles / meetings Communication across district Resource bank (ŅexpertsÓ & materials) Codify practices into policy

Coach Competencies

Big Ideas#1 Schoolwide systems need to be in place to

support all students prior to building secondary and tertiary systems, therefore need to focus on SW PBIS coaches (at the universal level) first!

#2. Prior Experience and EndorsementAttendance at previous introductory PBIS team

training sessionsCoaches experienced with school team

implementation team member

Supervisor endorsementDistrict agreements & support given

Big Idea #3Not enough to be expert in PBIS content knowledge, you also need

facilitation skillsCurriculum included in coaches PD: Using PBIS Team Implementation Checklist Preparing for Using the School-wide Data Management System Using Data to Guide Decisions Using the Tools Using Data to Guide Instruction: Common Area Routines and Practices Leading the Development of Teaching School-wide Behavior Expectations

Teaching Behavior Assessing Classroom Management Developing and Using Strategies for Generating Ideas Learning Walk Facilitation Skills Assessing Committee / Workgroups Year-end Evaluation Action Planning

#4On-site coaching:Clearly identify roles of External and Internal Coach,

Team Leader and Team MembersWho develops an agenda?

Feedback on agenda

Data analysis- Who prints reports? Feedback on data discussion

Action planningFeedback on:

Running a meeting Opening, staying with agenda, closing, action steps Guiding questions Problem solving Participant Timelines

#5Need to plan for sustainability and

capacity buildingOn-going professional development available

to cultivate new building coaches Turnover and burnout Continuous regeneration

CoachingFocus on Role and Functions- not personFTE/Job DescriptionDoes your boss know what you do?Internal v. ExternalCommunity of PracticeFundingBuy InStaff Turnover

CoachingData Facilitator- Do your schools know how to

use the tools/forms? ReadinessComputer ApplicationDecision Making

Primary v. Precision

“Executive Coaching”-

TrainingRegular Training CycleCurriculum- Illinois, MO, VA, ORTrainers- TOTFocus on outcomesDifferentiated InstructionReadinessFollow Up

Type of Skill to be TrainedSkinner (1974) distinguishes between two

types of knowledge. Knowing About: can describe variables that

influence a phenomenon. Example: Describe principles of reinforcement.

Knowing How: can perform effectively Example: Shape the behavior of another.

One form of knowing does not imply the other.

Final thoughtsStay close to school’s needsDo SETs, visits, ongoing feedback from

various roles etc.Other topics need to be addressed

SustainabilityScalingExpansion

Sustainability

Scaling Up the Triangle

Scaling UpDoes not simply equal more schools or every school

within a district/region/stateOutcome = increasing school’s adoption and

sustained use of evidence-based practices with integrity that lead to improved academic and social outcomes for students with accompanying organizational supports to allow replication

Recommendations on Promoting New Initiatives

New initiatives should be adopted with:1. Formal assessment of how they may or may not

connect with other initiatives

2. Documented evidence of effectiveness

3. Well defined and relevant outcome indicators

4. Mechanism for assessing and evaluating their fidelity of treatment

(Adelman & Taylor, 2003)

Research Findings on Scaling Up

(Fixsen, Naoom, Blase, Friedman, & Wallace, 2005, p. 70)

Best evidence documents what doesn’t work:

Information dissemination alone

Training by itself

Research Findings on Scaling Up

(Fixsen, Naoom, Blase, Friedman, & Wallace, 2005, p. 70)

What worksLong term, multi-level approaches

Skills-based training

Practice-based coaching

Practioner performance-feedback

Program evaluation

Facilitative administrative practices

Methods for systems intervention

Recommendations(Fixsen, Naoom, Blase, Friedman, & Wallace, 2005, p. 77)

•Develop partnerships with skilled researchers•Establish a community of practices at

implementation sites•Share lessons learned across functional purveyor

teams from different programs

Expansion

From 60 to 600: The Perfect Storm

Maryland 494 schools North Carolina 548 schools

Illinois 611 schools Colorado 405 schools

Florida 250 schools New York 322 schools

Michigan 181 schools Ohio 221 schools

New Mexico 130 schools West Virginia 215 schools

Oregon 229 schools Louisiana 285 schools

Missouri 183 schools Georgia 171 schools

5 Year action planExample