USD #305 PBIS Evaluation

28
MARY BETH GEORGE, USD 305 PBIS DISTRICT COORDINATOR USD #305 PBIS Evaluation

description

USD #305 PBIS Evaluation. Mary Beth George, USD 305 PBIS District Coordinator. District-wide Planning. Create vision for district Foundation and leadership Use of resources Internal expertise Ensure sustainability. ٭. SWPBS Organizational Logic. Visibility. Political Support. Funding. - PowerPoint PPT Presentation

Transcript of USD #305 PBIS Evaluation

Page 1: USD #305  PBIS Evaluation

MARY BETH GEORGE, USD 305 PBIS DISTRICT COORDINATOR

USD #305 PBIS Evaluation

Page 2: USD #305  PBIS Evaluation

District-wide Planning

Create vision for districtFoundation and leadershipUse of resourcesInternal expertiseEnsure sustainability

Page 3: USD #305  PBIS Evaluation

Leadership Team

FundingVisibility Political

Support

Training Coaching Evaluation

Active Coordination

Local School Teams/Demonstrations

SWPBS Organizational Logic

٭

Page 4: USD #305  PBIS Evaluation

District Coordinator

External Coaches

Secondary School Teams

SALINA SWPBS DISTRICT ORGANIZATION

External Coaches

External Coaches

School Coaches

School Coaches

School Coaches

School Team

School Team

School Team

Page 5: USD #305  PBIS Evaluation

Current District PBIS Team

District coordinatorDistrict and building administratorsProfessionals representing curriculum and instruction Safe and drug free representationProfessional development Bus system leadershipMental health professionalsFamily membersStudentsSpecial education cooperativeInteragency partners (court services, mental health,

developmental disability services, children and family services, etc.)

Page 6: USD #305  PBIS Evaluation

What Progress has the District Made in Implementing PBIS?

Training for all schools to use the School-wide Information System (SWIS) 3 schools at tier 2 9 schools at tier 1

District Coordinator meetings occur regularly Linda Benjamin has been shadowing Project Stay professionals to

learn more about how the SIT training systems at the district level may benefit

Individuals within the district are participating in the Kansas Institute for PBS Tier 3 individualize planning across home, school and community Tier 3 Interagency event in the works for spring, 2010

Page 7: USD #305  PBIS Evaluation

District-level Progress (Continued)

Three PBIS District Newsletters Board presentations on PBIS Visits from outside school districtsInteragency meetings started to begin building Tier

3 capacityDistrict 3 year action plan for PBIS is in place

Page 8: USD #305  PBIS Evaluation

District 3 Year PBIS Plan

Document in MTSS Notebook

Page 9: USD #305  PBIS Evaluation

Fidelity of Implementation: District Team Checklist

Page 10: USD #305  PBIS Evaluation

District Team Checklist

Document in MTSS District Notebook

Page 11: USD #305  PBIS Evaluation

Are Schools Implementing PBIS?

Self-Assessment Data EBS Survey Results Oregon School Safety Survey, Action Plans, Office Discipline Referral Data

Effective Behavior Support Survey Team Implementation Checklist Data Annual action plans for each school School-wide Information System (SWIS) Data

Average referrals per day per month By time By location By student By problem behavior

School-wide Evaluation Tool (SET) Outcome Data

Page 12: USD #305  PBIS Evaluation

Fidelity of Implementation: Team Implementation Checklist

Page 13: USD #305  PBIS Evaluation

Team Implementation Checklist

Document in MTSS District Team Notebook

Page 14: USD #305  PBIS Evaluation

EBS Survey Data Results for Schools

Page 15: USD #305  PBIS Evaluation
Page 16: USD #305  PBIS Evaluation

Self Assessment Summary

Team Implementation Checklist results show 8 out of the 12 school teams are at 80% overall implementation,

There is a 68% increase in the number of surveys completed by staff, students, parents and community members between 2008-2009 and 2009-2010 school years.

There was a 16.6% decrease in the perceived risk factors across the USD 305 schools between 2008-2009 and 2009-2010 school years.

There was a 3.3% increase in perceived protective factors across the USD 305 Schools between the 2008-2009 and 2009-2010 school years.

Page 17: USD #305  PBIS Evaluation

Training and Support

All 12 buildings are receiving implementation support

Coaches meetings occur monthly to support schools

External coaches are now completing the SET in schools they are not supporting directly

All 12 buildings are implementing school-wide PBS in varying degrees

Targeted and intensive individual positive behavior support is being implemented at Heusner and Stewart

Page 18: USD #305  PBIS Evaluation

Training and Technical Assistance

Tier 1 Training All buildings district wide 2008-09

Advanced Tier 1 Training All buildings district wide 2008-09 (Except CHS)

Tier 2 Training Central High School, Heusner Elementary School,

Stewart Elementary School 2008-09

Tier 3 On site Support Central High School Fall 2009

Page 19: USD #305  PBIS Evaluation
Page 20: USD #305  PBIS Evaluation

Why Create Inter-rater Agreement for the SET?

Avoid “we’re doing MTSS for behavior” Provide consistent responses to school teams Build capacity of district personnel to guide PBIS/MTSS

process Link district data to state PBS Kansas

Districts participating are working on plan to unify data

Consistency between all state district coordinators and SET implementers

Districts understand they are implementing PBIS---there is evidence via inter-rater agreement

Page 21: USD #305  PBIS Evaluation

Inter-rater Agreement on the School-wide Evaluation Tool (SET)

1 District Coordinator + 3 External Coaches Reliable with 1 Benchmark Person;

Proportion of Agreement: Formula = [Number of Agreements/ (Agreements + Disagreements)] x 100%

Inter-Rater Agreement on SET for Reliable Raters

# of Reliable Raters in District

SETs Completed Inter-rater

Process

Average Agreement Across

All SETs

Reliability RangeAcross

Features

4 3 SETs in USD305

97.63% 66.7% - 100%Across

Features

Inter-rater agreement is achieved if the person scores 3 SETS with 80% or higher agreement with the benchmark inter-rater reliability agreement person.

Page 22: USD #305  PBIS Evaluation

Internal vs. External Evaluation of the School-wide Evaluation Tool (SET): Example of First Steps

Inter-rater Agreement System at the Salina School District District PBS Coordinator and Lead Inter-rater (Shonda)

completes the SET and compares scores SET inter-rater agreement with External Coaches (complete) Starting together with Shonda for SET Continuing inter-rater agreement with Mary Beth (25% of all

SETs completed) External coaches obtain agreement with district coordinator Data on inter-rater agreement is reported at the district level

External Evaluation: Show SET data are consistent with People Outside the District

Invite an expert to review PBIS data and training systems and provide guidance

Establish inter-rater agreement with others outside the district Other PBS District Coordinators Shonda

Page 23: USD #305  PBIS Evaluation

SW-PBS Impact on Student Behavior

School #1

School #2 School #3

Page 24: USD #305  PBIS Evaluation

SW-PBS Impact on Student Behavior

School #5 School #6

School #7

Page 25: USD #305  PBIS Evaluation

Examples of Decreases in ODRs Reported by School Teams

School #7 implemented a teacher mentor program last spring and data shows that last years ODR’s were 593 to date and this year’s are 438 which is a 26% decrease in ODR’s.

School #6 implemented a program to reinforce their expectations taught and their ODR’s dropped from 96 to 58 to date which is a 39% decrease in ODR’s.

Page 26: USD #305  PBIS Evaluation

Faculty, Staff, Students, Families and Community Stakeholders

There is a 68% increase in the number of surveys completed by Staff, students, parents and community members between 2008-2009 and 2009-2010 school years.

There was a 16.6% decrease in the perceived risk factors across the USD305 schools between 2008-2009 and 2009-2010 school years.

There was a 3.3% increase in perceived protective factors across the USD305 Schools between the 2008-2009 and 2009-2010 school years.

Page 27: USD #305  PBIS Evaluation

Policy Impact

Integrating district team for academics and behavior

Website created to share information formally with community

Board meeting presentationsEvaluation plan for MTSS for Behavior

Draft document for review

Page 28: USD #305  PBIS Evaluation

District Capacity Building

2009-10 the district is working on identifying trainers who will continue providing Tier 1 training

A District tertiary team planning process is now in progress to

Identify how to work smarter not harder with existing school personnel who support students who engage in serious problem behavior

Identify trainers for Tiers 2 and 3 who can support Student Improvement Teams onsite

Discuss how to collaborate with Project Stay by partnering when individual plans are being facilitated

Set the stage for internal district curriculum to support PBIS training on an ongoing basis