Evaluating District-Wide Professional Learning to Build Capacity for RTI Michelle Avila Bolling,...

Evaluating District-Wide Professional Learning to Build Capacity for RTI

Michelle Avila Bolling, Ed.S.Lisa Sirian Stear, Ph.D.

Sharon Rinks, Psy.D.Danny Hicks, Ed.S.Cindy Jenik, Ph.D.

Goals for Today’s Session

• Establish the context • Project conception• System support • Format for training• Enabling and disabling factors• Outcome data• Future plans • Questions• Explore the CD of materials

Carroll County

• West central GA• On GA/AL state line• 50 miles from

Atlanta• 503 square miles• Population 109,965

(2006 census)

• Cost of living 93% of U.S avg.

District Enrollment

• 23rd largest school system in Georgia• 14,800 students in 2008-2009• Pre-K through 12th grade• Ethnicity:

– 68% White– 18% African-American– 7% Other

Personnel

• 2,225 total staff members– 1,261 certified– 964 classified

• 99.25% “Highly Qualified” teachers• 62% of certified staff members hold masters degree or

higher• Psychological Services

– 4 doctoral, 4 educational specialist – 3 licensed, 5 certified– APPIC training program

Schools

• 11 elementary schools• 6 middle schools• 6 high schools• 3 ninth grade academies• 1 alternative school• 1 technical education center• 2 psychoeducational centers

Project Timeline • Spring 2007

– Georgia mandated RTI to begin Fall 2007

• Summer 2007 – District RTI Manual introduced

to principals – Every school provided a copy

• Spring/Summer 2008 – Conceptualized the RTI Institute – Created training materials

• Fall 2008– Conducted training

• Spring 2009– School teams redelivered

• Fall 2009/Spring 2010 – Evaluated progress

The Problem

• Frustration from implementation or lack thereof

• Need for training others • Lack of district resources and funds• Lack of state-level guidance• Confusion regarding the 4-tier pyramid

The Problem: Referral Data

0

50

100

150

200

250

300

2006-07

2007-08

2008-09

Number of Completed Referrals

Number ofCompletedReferrals

Manual Released Training Occurred

Project Conception

• Seeking System Support – Met with our supervisor– Met with the district PD coordinator– Mapped out a blueprint for training– Arranged for leadership module

Project Conception

• District Enabling Support– Extra work days over the summer to develop

modules– Time commitment– Extra support from graduate student interns and

practicum students– Provided materials, meeting space, and snacks– Paid for substitutes for teacher participants

Leadership Module

• Leadership and administrative support• Important for any innovation and essential if you want it to

work• Administrators control resources

– Time – Personnel – Materials/resources

• Administrators outline duties and expectations for personnel• Endorsement in the face of possible resistance to change

Unless commitment is made, there are only promises and hopes... but no plans.

- Peter Drucker

The Professional Learning Model

• Voluntary participation • Distributed learning over time• Meaningful choices• Opportunities for practice• Authentic cases • Learning communities/teaming• Responsive to participants’ needs• Tailored feedback

Selecting Team Members

• Voluntary participation• Administrative representation• 3-4 other individuals

– Curriculum specialists– Grade-level teacher leaders– Counselors – Psychologists – Graduation coaches– Special education teachers

• All team members had to commit to completion of the entire training.

I start with the premise that the function of leadership is

to produce more leaders, not more followers.

- Ralph Nader

Overview Module Agenda

• Overview of training • Establish a common language• Self-Assessment of readiness• Discuss goals and action planning• Gather questions and confusions

Reading Module Agenda• Process the Application Activity • Evidence-based RTI practices in reading • Practice progress monitoring administration

– Oral Reading Fluency– Maze

• Progress monitoring & data entry– Practice data entry– Review decision making for each case

• Case studies• Discuss Application Activity

Interventions

• Five areas – Phonemic Awareness– Phonics– Vocabulary – Fluency – Comprehension

Interactive Table of Contents

Writing Module Agenda

• RTI implementation status reports• Process the Application Activity• Activating prior knowledge – RTI Bingo• Evidence-based RTI practices in writing

– Universal screening– Intervention– Progress monitoring

• Practice progress monitoring scoring • Case studies• Discuss Application Activity

Interventions

• Six Areas– Beginning Writing– Handwriting– Spelling– Editing– Planning– Fluency

• Comprehensive Strategies

Math Module Agenda

• Round robin problem solving for RTI• Evidence-based RTI practices in math

– Universal screening– Intervention– Intervention fidelity – Progress monitoring

• Establishing goals challenge activity• Math case studies• Discuss Application Activity

Interventions

• Whole Group Strategies for Tiers I & II• Six Areas

– Number sense– Arithmetic– Fluency – Problem Solving – Reasoning – Mnemonic Strategies

Behavior Module Agenda

• School-wide Positive Behavior Supports (Tier I)• Evidence-based RTI practices in behavior

management – Universal screening– Intervention– Functional behavioral assessment

• Behavior intervention planning– Progress monitoring

• Discuss application activity

Interventions

• Class or School Interventions for Tiers I & II• Four Areas

– Antecedent Modifications– Replacement Behaviors– Consequence Modifications– Self-Monitoring

Wrap-Up Agenda

• SARTII• Case Presentations• Forms and Documents• Planning• Discussion

Researching the Program

Participant PerceptionsResearcher Perceptions

School Readiness RatingsRubric Evaluation of Permanent Products

Team Interview Data

Course Participants

• Total Number of Participants Who Completed Course N=58 – Elementary Participants N=36– Middle & High Participants N=22

Outcome Data – Self-Report Ratings

0

10

20

30

40

50

60

Percentage of Respondents

Novice Competent Expert

Self-Report Ratings Level of Understanding

Pre-Training

Post-Training

Prior to training, only 20.6% of participants reported at least a competent level of understanding of RTI. Post-training, 89.7% reported a competent level of understanding. Significantly higher rates of understanding found (t=-15.625, p<.001)


010

2030

4050

60



Self-Report Ratings Ability to Apply RTI Skills

Pre-Training

Post-Training

Prior to training, only 15.5% of participants felt competent to implement RTI. After the training, 82.8% of participants felt competent. Significantly higher rates of skills reported (t=-18.188, p<.001)


010

2030

4050

60



Self-Report Ratings Ability to Act as a Resource

Pre-Training

Post-Training

Prior to training, only 13.8% of participants felt competent to act as a resource to others.After the training, 81% felt competent.Significantly higher rates reported (t=-15.975, p<.001)


05

1015202530354045

Percentage

StronglyDisgree

Neutral StronglyAgree

RTI is a realistic initiative.

Pre-Training

Post-Training


0

10

20

30

40

50

Percentage

StronglyDisgree


My school implements RTI well.

Pre-Training

Post-Training


0

10

20

30

40

50

60

70

Percentage

StronglyDisgree


RTI will help our children be more successful.

Pre-Training

Post-Training

Participants reported significantly higher agreement that the RTI model will help children in their school be more successful (t=-8.045, p=.016).

Outcome Data – Redelivery (2/2009)

Format of RedeliveryPercentage of Schools

School-wide whole-group training 31%Grade-level team trainings 77%Individual consultation 31%Posting on intranet or distributing handouts

31%

Other: Weekly RTI support meeting 7%

• 87% of schools (13/15) did some form of redelivery• One additional school had 100% participation

Researcher PerceptionsEnabling Factors

• District & principal support• Group discussion opportunities• Application activities• Case studies – saw improvement• Consistent team membership• Thoughtful selection of team membership • Collaborative nature of the team• Technology support• Voluntary participants• Flexibility of implementation

Researcher PerceptionsDisabling Factors

• Participants were “voluntold”• Teams had limited administrative support• Resistance to change/inflexible structures• Too much content, too little time• Competing district initiatives• Location of training• Lack of resources and information for high school

students (only 1 of 5 high schools completed training)

• Flexibility of implementation – looking for district-wide guidance

Outcome Data - SARTII

Self-Assessment of RTI Implementation (SARTII)*

Status Ratings:Not Started (The activity occurs 0% of the time)Just Started (The activity occurs 1 to 24% of the time)In Progress (The activity occurs 25 to 74% of the time)Achieved (The activity occurs 75 to 100% of the time)Maintaining (continues to occur 75 to 100% of the time)

*Adapted from the IL-ASPIRE SAPSI v. 1.6 Center for School Evaluation, Intervention and Training (CSEIT) Loyola University Chicago, Florida

Problem Solving/Response to Intervention Project (SAPSI Fall, 2007), & North Carolina Comprehensive Assessment of Need (NCCAN)

Outcome Data – SARTIIConsensus

2.462.83

3.25

0

1

2

3

4

5

8/25/2008 11/19/2008 2/10/2010

An RTI Team is established and represents the roles of an administrator, facilitator, data manager, content specialist, parent,

and representative teachers.

Maintaining

Achieved

In Progress

Just Started

Not Started

Outcome Data – SARTIIInfrastructure

Maintaining

Achieved

In Progress

Just Started

Not Started

Outcome Data – SARTIIImplementation

Maintaining

Achieved

In Progress

Just Started

Not Started

Outcome Data – SARTII

• Growth over time in all areas, except…– Perception of district level support initially

increased, but has since decreased– Change in personnel – Funding

• Less growth in involving parents• Schools reported collecting data on consensus

• Unsure of the means of data collection – survey?

Referral Data

050

100

150

200250

300

350

400

2006-07

2007-08

2008-09

Number of Completed Referrals

Number ofCompletedReferrals

Manual Released Training Occurred

Referral Data

0

20

40

60

80

100

120

2006-07 2007-08 2008-09

Referral Data by Type

EBD

SDD

SLD

SLI

Rubric Ratings Over TimeRubric Ratings Over Time

16.00 16.36

22.77

35.23

0.00

5.00

10.00

15.00

20.00

25.00

30.00

35.00

40.00

Pre-Rating Post-Rating

Poin

ts E

arne

d

Nonparticipant Schools

Participant Schools

Though the participant schools began at a higher level of competency, their rate of growth significantly exceeded that of nonparticipating schools, F(1,23)=14.81 , p<.005.

Rubric Analysis

• Strengths – Universal screening– Tier III intervention plans

• Form is present• The target is identified • Baseline is appropriate• Clearly identified

appropriate interventions• Interventionist identified• Results reported

• Weaknesses– Tier II intervention plans

• Form• Baseline data• Progress monitoring plan

– Goal Setting (Tiers II & III)

– Sticking to review dates– Fidelity review– Documentation of

referral for evaluation– Intervention Review

Team – Rubric range 0-56

Interview Data: Team Members

• Assistant Principal• Counselor• School Psychologist• Special Education Teacher(s)• Regular Education Teacher(s)• EIP Teacher(s)

Interview Data How did everyone in your school become knowledgeable about the RTI process?

• RTI Institute– Redelivered training

• To the entire faculty and staff • In smaller groups

– “On-the-job training… learn while you work the process!”

• School-based RTI teams (composed of reps from each grade)– Monthly grade-level meetings

• Information provided through emails and online

Interview DataHow on board with RTI would you consider

your faculty and staff?

• “I’m not sure that they like it, but they’re doing it”(1st grade teacher)

• More on board with certain tiers than others• Consensus is increasing

– As they see the process work– By finding ways to make the whole process more “doable” and, thus,

acceptable– By requiring staff members to present students who were in need of

help to the RTI committee (e.g., their “two lowest in the class”)• Ways consensus could be improved

– If there were explicit rules from the county about what to do– If there was more accountability; administration could help with this

Interview Data Are the right personnel involved with RTI?

• Yes– Every grade-level/subject area is represented – Every “area” (e.g., counselors, EIP teachers, psychologists) – Diverse membership helps in the problem-solving process – Previous years’ teachers provide history and context– Decisions can be made because the right people are at the

meeting• No

– Not all schools have a regular education teacher from each grade-level involved with RTI

Interview DataDescribe Any Changes You’ve Seen in How Your School

Looks at Tier I Data.

• Data are accessible to teachers• Data are available in a timely manner• “We now look at data from more than one program,

including benchmarks, STAR, GRASP, and DIBELS, at regular intervals and move students up to Tier II when needed.” (5th grade teacher)

• “Progress monitoring data is used to direct and reform instruction.” (Assistant Principal)

Interview Data Describe Any Changes You’ve Seen in How Your School

Looks at Tier II/III Data.

• Don’t just go by teacher opinion anymore• Meetings occur at regular intervals • Data are analyzed at each meeting• Modifications to goals are made as needed• Interventions are modified/changed if they are not

working• “We have more tools to use as interventions so that

we have better data” (4th grade teacher)

Interview DataWhat Noticeable Changes in Classroom Practices Have

There Been Since RTI Was Implemented? • Frequent progress monitoring• More interventions to address specific deficits• More creative use of time- finding time during the

day for intervention• Altered schedules to create an extra class period • Teachers provide intervention in their areas of

strength • Increased awareness of behavioral difficulties

– collecting behavioral data– figuring out the reasons behind behavior – more aware of and involved in creating BIP’s

Interview DataWhat keeps the RTI process from functioning optimally

at your school?

• Not enough…– Time (also, time management issues)– Money– Manpower– Resources or identified strategies– People who have knowledge and/or experience

with the RTI process– Staff development time/day– Accountability

Interview DataWhat Keeps the RTI Process from Functioning Optimally

at Your School?

• Student absences/transiency• Group management issues

– individuals in each group progress at different rates which often results in the need for changes in groups and creation of new groups

• The time it takes to get a student into tier III or IV, – especially for students who have significant issues

• Lack of increased/intensive intervention once the student moves into Tier III

Interview Data What Would Help Your School Do RTI Better?

• More…– Time for the 3 P’s: planning, providing interventions, and

progress monitoring! – Money: to purchase resources/interventions and to fund

interventionists– Manpower: people to provide and monitor interventions – Experience in actually implementing the process– Intervention resources/updated intervention bank– Staff development: days and trainings– Psychologist involvement: time to observe, test, and offer

suggestions, support, and interventions

Interview Data What Would Help Your School Do RTI Better?

• Better support system overall– A person at each school whose sole responsibility is RTI– A person at each grade level who has the time to be

primarily responsible for RTI at that grade level• Better communication

– about available resources– with other schools in the county– with school systems who are doing RTI well

• Expanded use of certain programs (e.g., Orton-Gillingham for groups in all grade-levels)

• Explicit guidance from the county office

Moving Forward

• Plans – Continue to support

school efforts– Share our assessment

results with the District– Establish a collaborative

relationship with new district personnel

• Obstacles– Balancing individual

teacher needs with collective goals of the school/district

– Nonparticipating teams are at a disadvantage

– Consensus in some schools has been hard to establish

– New district personnel assigned to RTI

Exploring the CD

Questions

CONTACT [email protected]@carrollcountyschools.comSharon.Rinks@[email protected]

Evaluating District-Wide Professional Learning to Build Capacity for RTI Michelle Avila Bolling,...

Documents

Transcript of Evaluating District-Wide Professional Learning to Build Capacity for RTI Michelle Avila Bolling,...