Achieving Sustainable Improvements in Programs for Students with Emotional and Behavioral Disabilities

Bridget Walker PhD: Seattle University Walkerb@Seattleu.edu

Lisa Hoyt PhD: Renton Academy Lisa.hoyt@rentonschools.us

WAREA Conference 2013

Seattle, WA

!Emotional or Behavioral Disabilities!(EBD) = !EveryBody’s Dilemma!

December 16, 2008! 2!

EBD: Ongoing Issues

  Poor outcomes for students throughout the life span

  Inconsistent access to high quality, comprehensive programs (often lack key practices, supports, and experiences)

  Teachers and staff report being UNDER- prepared, supported, & trained

  Shortage of trained professionals & high turnover   Staff report lower job satisfaction and higher stress

than other teachers (Billingsley, Fall & Williams, 2006; Bradley, 2008; Wagner, et al., 2006; 2003)

PRACTICES

Supporting Staff Behavior

Supporting Decision Making

Supporting Student Behavior

Positive Behavior Support

OUTCOMES

Social Competence & Academic Achievement

Supporting Staff Behavior

Supporting Decision Making

Supporting Student Behavior

Improving Programs for

Students with EBD…

OUTCOMES

Social Competence & Academic Achievement

PRACTICES PROCESS

We would add….

THE PARTICIPATORY PROGRAM EVALUATION PROCESS

(Clancy, 2011)

“Once the system forms, capacities can keep developing.” (Wheatley, 1999)

Together you get the “WHOLE” picture

Outside Perspective: External Coach

Inside Perspective: Stakeholders

Participatory Program Evaluation and Improvement Planning

Two Tools to Support Successful School/Program Improvement Using Participatory Evaluation

  Self-Assessment and Program Review for Schoolwide Positive Behavior Supports (SAPR-PBIS).

  Designed to support leadership teams in implementing practices across the three tiers of SWPBIS initiative

  Available from Brookes Publishing

  Participatory Evaluation & Expert Review for Classrooms Serving Students with EBD (PEER-EBD)

  Designed to support teams in specialized programs serving students with EBD and related disabilities

  Piloted in several school districts in Seattle area

  Contact authors for more information

Both by Walker & Cheney (2001, 2007, 2012)

Key Components of an Effective Specialized Programs for Students with

EBD (PEER-EBD)

Program Philosophy &

Systems

Group Climate and Culture/

Process

Structure Individual Programing

(Tsai, 2012; Walker & Cheney 2001, 2006; Walker & Fecser 2001, 2007)

• Statement of Mission & Philosophical Framework • Ongoing Professional Development • Meaningful Family Involvement • Clear Referral & Reintegration Process

Systems & Philosophy

• Balanced Behavior Management • Effective Crisis Prevention and Intervention • Schedule/High Level of Engagement • Data-based Decision Making

Structure

• Group Meetings and Prosocial Skills Instruction • Rules, Rituals, Routines • Effective Instructional Style & Strategies • Positive Responses to Feelings and Needs

Climate & Group Process

• Meaningful Overall Assessment (including transition) • Academic Accommodations & Modifications • Useful and Effective FBA & BIP • Wraparound Supports as appropriate • Culturally Relevant Supports and Interventions • Effective Therapeutic Supports

Individual Programming: Builds

Academic and Behavioral Competence

Reliability and Validity of PEER-EBD (Tsai, 2011)

  Strong internal consistency in each Domain/Component (alphas =.88-.94)   Solid content validity based on literature review and

panel of 32 national experts in field of EBD

  Factor Analysis confirms the fit across all four factors

  Strong face validity: Used in a number districts throughout the Puget Sound region to effectively improve programs for students with EBD and related disabilities.

Team Action Planning/Implementation/Repeat Reflect on Reviewers input & develops

annual action plan Implement plan and repeat PEER-EBD

process annually

External Review/Observations/Recommendations Visits classes, document review, and

interviews to develop ratings Charts team & reviewer data and develops

recommendations

Full Program Team Meeting and Rating Share individual scores and establish

consensus ratings Share experiences, perspectives, needs &

build + interdependence

Individual Rating

Individuals rate the program using PEER-EBD Practices

Steps in PEER-EBD Process

The Double Loop Learning Process (Argyris & Schon, 1996; Morgan, 1997)

Systems & structure to scan

and anticipate changes

Compare information to

current operational

norms

Challenge current

operating norms in self-reflective

process

Recognize, share and use collected

information to guide the next

steps and future direction

Adjust /Re-align as needed

Ongoing data gathering and analyses

Interpersonal Dynamics and… the way we always have done it…

A critical element that teams are reluctant to face..

An Ongoing Process

We are a group with both individual and group goals

Safety First

No Discounting

Give and receive honest feedback

No grudges-Let it go

Full Value Contract

5

4 3 2

1

(Project Adventure, 1988)

PEER-EBD is a Team Support and Development Process   Most supervision is done on an individual basis but

services for students with EBD are mostly TEAM based   Focuses on team interaction, knowledge, planning &

implementation

  Team is then more effective and inter-connected   “When teachers work in self-managing teams to develop

goals, curricula, instructional strategies, and staff development, students often achieve at higher levels” (Scribner, Sawyer, Watson, & Myers, 2007).

The team aspects of the PE process allow members of the group to share their unique perspectives, as well as to learn how their efforts fit together with others on the team..

Creates Cross Role/Function Dialogue

Scaffolds skillful discussion (learning to ask the right questions)

Promotes sharing of knowledge and transparency

Links benchmarks and EBP to broader context and daily experience

Builds positive interdependence

Data is discussed in the context of practice

Results Using PEER-EBD

 Highline School District (specialized classrooms in comprehensive schools)

 Renton School District (specialized classrooms in comprehensive schools) & Renton Academy (specialized school)

 Chaska Special Education Cooperative (specialized school serving multiple districts in Minnesota)

Highline School District Demographics

Number of Classrooms for

Students with EBD

15 Programs 7 elementary school classrooms

3 middle school classrooms 5 high school classrooms

Average Number of Students Served in

EBD Programs

197 Students

Number of Staff Serving Students

with EBD

102 Special Education Teachers Paraprofessionals

Administration, Related Service Support Staff

Ethnicity of Students

Hispanic Asian Black,

American Indian/ Alaskan Native White

Mixed Race

17.6% 3.8% 22.9% 6.1% 45% 4.6%

0.0

0.5

1.0

1.5

2.0

2.5

3.0

3.5

4.0

4.5

5.0

Foundation/Philosophy Class Structure Climate/Group Process Individualized Programming

Rat

ings

Components

SAPR-EBD Three Year Results

Reviewer2008

Team 2008

Reviewer 2009

Team 2009

Reviewer 2010

Team 2010

0.0

0.5

1.0

1.5

2.0

2.5

3.0

3.5

4.0

4.5

5.0

Foundation/Philosophy Class Structure Climate/Group Process Individualized Programming

Rat

ings

Domains

Highline PEER-EBD Three Year Results

Reviewer2008

Team 2008

Reviewer 2009

Team 2009

Reviewer 2010

Team 2010

CHANGES IN STUDENT DISCIPLINE RATES

Other Outcomes using PEER-EBD in Highline

•  76% of staff rated highly satisfied with overall impact of PE process

•  90% of staff indicated strong impact on student behavioral performance

•  Significant increase in staff retention •  More than 50% decrease in requests for

AES •  Increase in student graduation rates (24%

increase from pre-intervention rate of 0)

Changes based on PEER-EBD Process   Created mission/philosophy statement   Reorganized district services and supports   Closed one classroom   Increased levels of targeted professional

development   Implemented weekly program team meetings and

district wide PLCs monthly   Data-based decision making at multiple levels,

student, classroom and district was implemented   Paraprofessionals integrated more deeply into

instruction and therapeutic support   Regular classroom group meetings implemented

(e.g. goal setting, problem solving, positives)   Social skills taught daily integrated into program   Emphasis on group instruction in academics and

social skills

Renton School District Demographics

Number of Classrooms for

Students with EBD

9 Programs 3 elementary school classrooms

3 middle school classrooms 3 high school classrooms

Average Number of Students Served in

EBD Programs

75 Students

Number of Staff Serving Students

with EBD

43 Special Education Teachers

Paraprofessionals Administration

Related Service Support Staff

Ethnicity of Students

Hispanic Asian Black,

American Indian/ Alaskan Native White

Mixed Race

16% 2% 38% 2% 34% 8%

0.0

0.5

1.0

1.5

2.0

2.5

3.0

3.5

4.0

4.5

5.0

Component 1: Program

Foundations

Component 2: Program Structure

Component 3: Group Process

Component 4: Individual Supports

Renton School District EBD Classrooms 2013 Scores Includes 2 New Programs in

Comprehensive Schools

Reviewer 2010

Team 2010

Reviewer 2013

Team 2013

Changes in Renton EBD Programs based on PEER-EBD

  Expanded specialized EBD program classrooms into comprehensive schools

 Opened inclusion programs for every grade level

 Adopted a district wide philosophical framework: Therapeutic Re-EDucation (Hobbs, 1960)

  Invested in comprehensive training for ALL EBD program staff

 Closed one classroom  Hired Mental Health Specialists for every

program

Renton Academy

Renton Academy’s Mission Renton Academy is a team of specifically trained teacher/counselors

who are passionate and committed to creating an environment where learning abounds inside and outside the classroom. This effort is made complete by our students, their families, and the surrounding community. The Re-ED philosophy drives this venture, to build on every student’s strengths and help them become a contributing and participating member of our society.

Renton Academy is a public K-12 therapeutic school for students with EBD in the Renton, WA school district that is founded in the Re-EDucation philosophical framework.

0.0

0.5

1.0

1.5

2.0

2.5

3.0

3.5

4.0

4.5

5.0

Component 1: Program

Philosophy

Component 2: Program Structure

Component 3: Group Process

Component 4: Individualized

Supports

Renton Academy PEER-EBD 2009-2013

Reviewer 09/10

Team 09/10

Reviewer 12/13

Team 12/13

0.0

0.5

1.0

1.5

2.0

2.5

3.0

3.5

4.0

4.5

5.0

Renton Academy PEER-EBD 2012-13

Reviewer 09/10

Team 09/10

Reviewer 12/13

Team 12/13

Positive Growth   50% Reduction of Physical Interventions

  66% reduction of In-School Suspension

  Significant increase in students meeting standard on state math assessment.

 Double digit increases in students meeting standard on state academic assessments in reading and science.

RA Changes Based on PEER-EBD Process

  More appropriate referral process into the most restrictive program

  Monthly data check-in meetings for students   Professional Development in FBA and directed BIPs

and connection of these to classroom interventions   Incorporation of increased wrap around supports   All teacher highly qualified in every academic subject

they teach   Increase in academic rigor and professional

development around instruction   Professional goals around developing a restraint free

school

CSEC Regional Program Demographics

Number of Classrooms for

Students with EBD

7 classrooms 4 high school classrooms (Passages)

3 elementary classrooms (Steppingstones)

Average Number of Students Served in

EBD Programs

50 Students from 12 member districts in the region

(suburban/rural south of Minneapolis)

Number of Staff Serving Students

with EBD

26 Special Education Teachers Paraprofessionals

Administration, Related Service Support Staff

Ethnicity of Students

Hispanic Asian Black

American Indian/ Alaskan Native White

12% 2% 4% 4% 82%

CSEC (Minnesota) Year 1- Baseline

Changes Based on PEER-EBD Results

  Invested in comprehensive training for all staff  Developed a program-wide philosophical

framework to guide program systems & operations

  Restructured the schedule for the school day and how support staff are scheduled throughout program

  Reviewed academic and behavioral approaches, shifting towards positive behavior interventions, supports, and increased academic rigor

“The constant challenge is to design a daily program so engaging, so varied and new, yet orderly and stable… so meshed with the growth of the child’s mind, so rich in human interchange… filled with success in matters small and large, so unconcerned with failure, so appreciative of individuality and of common purpose.. so joyous, so aware… that the disturbed child finds him or herself immediately committed to a new way of living at once more satisfying to him or herself, and more satisfactory to the people in his/her life…”

-  Nicholas Hobbs, Ph.D. the architect of Re-EDucation. From the book, The Troubled and Troubling Child, 1982

Remember “In growing up, a child (and adult) should know some joy in each day and look forward to some joyous event for the morrow.”

- Nicholas Hobbs

We wish you joy in pursuing this important work!

Thank You!

For more information contact: Bridget Walker: walkerb@seattleu.edu Lisa Hoyt: lisa.hoyt@rentonschools.us

References   Cousins, J.B. & Earl, L.M. (2004). Participatory Evaluation in Education: Studies in Evaluation

Use and Organizational Learning. London, England: Routledge Falmer.

  Fullan, M. (2007). The New Meaning of Educational Change (4th ed.) New York: Teachers College Press.

  Hobbs, N. (1982). The Troubled and Troubling Child. Hoboken, NJ: Jossey-Bass (a division of Wiley publications)

  King, J. & Stevahn, L. (2013). Interactive Evaluation in Practice: Mastering the Interpersonal Dynamics of Program Evaluation. Los Angeles, CA: Sage Publications

  Patton, M. (2008). Utilization-Focused Evaluation 4th Edition. Thousand Oaks, CA: Sage.

  Tsai, S.F., Cheney, D. & Walker, B. (in review). Preliminary psychometrics of the Participatory Evaluation and Expert Review for Classrooms Serving Students with Emotional/Behavioral Disabilities (PEER-EBD). Behavior Disorders.

  Walker B. & Cheney, D. (2012). The Self-Assessment and Program Review for Schools Implementing SWPBIS. Baltimore, MD: Brookes Publishing.

  Walker, B. & Cheney, D. (2007). Participatory Evaluation and Expert Review for Programs Serving Students with EBD. Seattle, WA: Author

  Walker, B., Clancy, M., Tsai, S. & Cheney, D. (in review). Bridging Research to Practice: Implementing Meaningful Program Evaluation and Improvement to Better Serve Students with EBD. Beyond Behavior.