FidelityLeadership

Exploring

Housekeeping

Sign In Sheets at each table Materials Wi Fi Access Timelines Breaks Lunch Restroom Location Turn cell phones to manner mode Other?

Essential Components

Research-Based Curriculum and Instruction On-going Assessment Collaborative Teaming Data-based Decision Making and Problem Solving

Fidelity of Implementation On-going Training and Staff Development Community and Family Involvement

Leadership

Outcomes for the day

Fidelity Understand what fidelity to the core program

means Discover ways to monitor fidelity

Leadership Roles and responsibilities of leaders within RtI

Fidelity….

strict observance of promises, duties adherence to fact or detail accuracy, exactness

Fidelity of Implementation Also known as treatment fidelity:

The degree to which plan was implemented as designed and intended

Why? Without, we don’t know if: Good results are actually due to the intervention Poor results are actually due to intervention, or to some

modifications We must know if it was the intervention that failed, or if it was

not followed/implemented properly Finally, we cannot assume the treatment will work with others if

we are not sure it was the treatment that worked in the first place

Why is fidelity important?

Even with high quality professional development variations in implementation occur

Programs are only as good as the quality of implementation. When quality varies student outcomes will be affected.

A main distinguishing factor of RtI is its presumed ability to rule out “instructional casualties”

Fidelity of Implementation

Treatment integrity is often assumed, rather than assessed

Outcomes cannot be attributed to the intervention unless one measures the extent to which the intervention plan was implemented

trust the system and the data understand accountability clearly articulate roles and responsibilities evaluate continuous improvement document student improvement

Fidelity of implementation of the RtI process

allows school staff and parents to:

In the RTI process, four activities are assessed on an ongoing basis to document fidelity:

The RtI process implementation fidelity

Prevention fidelity (Tier 1, Core Instruction or Positive Behavior Supports)

Intervention fidelity (Tier 2 & 3 small group and individual)

Assessment fidelity

Fidelity to the RtI Process

Collaborative teaming Data-based decision making Problem solving On-going professional development Curriculum & Instruction On-going assessments Community & family involvement Leadership

RtI Process

Collaborative Teaming Composition of the team(s)—WHO

Leadership/Core Problem Solving/Student

Roles for members Facilitator, recorder, timer, records, data, etc.

Scheduled meetings—WHEN Same day, same time

On-going dialogue concerning kids

RtI Process

Data-based Decision Making Validity/reliability of assessment data Role to ensure the team is interpreting the data

correctly Role to ensure assessments are aligned to

student needs and instruction Multiple measure may be needed Diagnostic—using to answer “why?” List of available assessment Students familiar with probe format

RtI Process

Problem Solving Process Define Analyze Develop & Implement Evaluate

RtI Process

On-going Training/Professional Development New team members

Plan to facilitate RtI mentor on staff

Staff Meetings Early release times PIR days Training on RtI process and programs

RtI Process

Community and Family Involvement Inform and educate

School handbook Newsletters Newspaper Benchmarking data (all) and meeting summaries and data (some)

Participation Leadership team Problem solving team Procedures/policies Home study component

Understanding = Support

RtI Process

Leadership Involved and an active member Informed (curriculum & instruction) Money Training Staffing Foster collaboration

Instruction

Research-based instructional practices Active engagement Corrective procedures Mastery Teaching/Direct Instruction Positive feedback

Planned, quality instructional opportunities Differentiated Instruction Multiple Intelligences Multi-modality Grouping Bloom’s Taxonomy

Prevention Fidelity

Research based core curriculum Best practices in instructional delivery Establish a school-wide positive behavior

support system

When implemented with integrity, that is as planned, there is sufficient evidence that

there will be positive results for the vast majority of students.

The effectiveness of any particular program is worthless if the program is

not implemented as intended.

A fidelity example…

Instruction Fidelity

Implementation fidelity of academic and behavioral programs is essential to establishing the reliability of a student’s response to intervention.

We cannot know for certain that students have a poor response to a program unless we can document that the program was implemented, and implemented as planned.

Drift

A phenomenon associated with implementation of any plan, program, assessment, or treatment is “drift”, or the unintentional subtle changes to a plan over time.

Content/CurriculumCore Curriculum Research-based reading, math, behavior program Components of the core program

Agreed upon by staff Ex. Worksheets, leveled readers, decodable books, etc.

Aligned to state & local standards Categorize critical, important, fluff Vertical Alignment for scope and sequence

Planned learning activities are targeted and specific to student need within the content

Scripted programs are followed

How Will The Program Be Used?

Core Curriculum The “base” program, designed to teach all components of

reading/math/behavior (5 areas of reading, math Focal Points, universal behavior expectations)

Supplemental Programs and materials designed to support the core by

teaching specific skills

Intervention Programs and materials designed to provide intensive support

for students performing below grade level

Developing Structuresto Improve Program Fidelity

1. Learn the Program Content—the “What” Delivery—the “How”

2. Observe3. Teach the Program4. Be Observed (checklists)

5. Refine6. Repeat—this is on-going

1. Learning the Program: ContentOrganization of the Program Scope and Sequence

High Priority Skills - are they quickly apparent and truly important skills? Can you easily identify what the activity is intended to teach?

Rate of introduction - How often are new skills introduced? Cumulative review - How frequently are skills reviewed? Supporting resources – aligned to objectives for instruction Mastery-Based (achieve 85%) or cover the content

Placement tests End of unit tests

1. Learning the Program: Content

Can the lesson be taught in the 90 minute reading block, or are there more activities than you can complete? What stays and what goes?

Does the program prioritize the activities for you? Or is prioritizing left up to you?

1. Learning the Program: Delivery

Reviews of reading curricula indicate that core programs vary widely in the quality of guidelines for instructional delivery.

Instructional practices1. Explicit instruction2. Demonstrate skills and strategies3. Guided practice4. Monitor independent practice5. Provide corrective feedback

Program DeliveryGeneral Features of Instruction

Delivery Adequate Enhancements

Explicit Instruction Yes No

Demonstrate Yes No

Guide Yes No

Practice Yes No

Corrective Feedback

Yes No

2. Observe

Classroom Observations Coach Trainer or consultant Teacher in your building Visit another school with the same

core program

Observe Instruction What does it look like and sounds like?

Is there? Active engagement of students Teacher demonstration, guided practice, and

independent practice Clear academic and behavioral expectations with

positive feedback Monitoring students’ understanding, corrective

feedback and review Independent work that is connected to the program Program implementation with fidelity (…with

enhancements as needed)

3. Teach

Develop comfort and fluency with the materials.

Find out how the program works with your students.

Practice implementation quality with enhancements as needed

Coaching/Peer support

Teaching Considerations

Are my students ready for this lesson? Are my students engaged and motivated? Why am I doing this lesson? Will this make my students a better reader? Am I using my formative assessment data?

4. Be Observed

Observing is hard work, being observed is even harder.

Make sure the observer knows your program and your students AND how long you have used the program.

Decide on 1-2 goals for the observation High quality fidelity of implementation is an

on-going process. This is not an evaluation.

Observation Feedback Describe what you saw. Tell what instruction

looked like and sounded like. e.g., “You gave every student a chance to …”

“I noticed you demonstrated the new…before students practiced it.”

“You built background knowledge by explaining…”

Avoid adjectives

5. Refine Use observation feedback to evaluate your program

implementation. Discuss how your students are performing on unit tests

and other assessment measures. Specific, objective feedback related to how well they

are learning the content and skills. Decide what enhancements or revisions you could make

the program, based on your students’ progress. Leave with one thing to work on.

5. Refine

Team Meetings Troubleshooting implementation issues Sharing grade-level resources for meeting the

needs of the lowest performing students Calibration checks for fidelity

Principals and Implementation

Working knowledge of adopted programs Regular communication with coaches and

teachers Facilitating fidelity without evaluation Active presence in classroom 4 point checklist for understanding instruction

A General Observation Checklist

General Features of Instruction Hi Med Low

Are students engaged?

Does teacher demonstrate, guide, and give independent practice?

Does teacher give clear expectations with positive feedback?

Does teacher monitor understanding and give corrective feedback?

Taking the Long View of Program Implementation

Adopting and implementing a new program is the beginning of a cycle of change.

Change is doable when we set small goals that are achievable.

Change and refine your implementation one step at a time. Resist the temptation to layer programs.

Work on implementation quality and enhancing instruction to meet the needs of your students.

Developing Structuresto Improve Program Fidelity

1. Learn the Program Content—the “What” Delivery—the “How”

2. Observe3. Teach the Program4. Be Observed (checklists)

5. Refine6. Repeat—this is on-going

Classroom Visits

Walk Throughs

Classroom Visits: Procedures

Student focus Length of time

Brief, 5-7 minutes May change over time

Data-driven focus Determine factors for

assessment Pace of instruction Curriculum components

No surprises! Shared expectations Shared components Shared process

Develop Implementation Plan

District: Teacher (Optional):

School: Grade:

Date: Period:

Team Member(s)

I. STUDENT BEHAVIORS

Degree of Evidence Component

1 2 3 4 N/A • Students are actively engaged with concepts relevant to the content of the lesson. • Students are able to explain what they’re learning. • Students have multiple opportunities to demonstrate mastery through varied and relevant, rigorous activities.

• Students are engaged in appropriate academic and social behaviors.

Comments II. TEACHER BEHAVIOR

Degree of Evidence Component

1 2 3 4 N/A • Teacher demonstrates a solid grasp of the content. • Teacher delivers instruction aligned to a rigorous learning objective. • Teacher demonstrates strong pedagogical skills, balancing direct instruction with modeling, facilitating, and/or coaching students as appropriate.

Comments III. FORMATIVE ASSESSMENT

Degree of Evidence Component

1 2 3 4 N/A • Teacher uses a variety of formative assessments of the daily learning objective. • Teacher adjusts instruction based on checks for understanding. • Teacher provides positive error correction, modeling, and practice opportunities. Comments

Classroom Walk-Through Observation Form

Team Debriefing

After each classroom visit… Review Classroom Observation form Identify preliminary findings Two to three minutes only Come to a consensus Identify data trends

Does the data reveal a

need for change?

Assess

Data

Fidelity Checksstrict observance of promises, duties Without fidelity checks,

decisions can be based on data that is no longer reliable as an outcome of the process.

accuracy; exactness

Student’s response to programming can only be considered as reliable data when programming is documented as having been implemented as planned.

How to Assess Fidelity Self-Reporting

Person doing intervention program can rate the degree to which each instructional component was implemented

These are easy and may actually serve as a prompt Drawbacks: Not as objective; social desirability effect; more

paperwork

Can use an weekly “Activity Log” (self report) Fidelity observations Handouts—Fidelity checks

Which Kind and How Often?

Try to use the most accurate, yet convenient The more often, the better

Some aspects of self-reporting should occur daily, or even with each intervention process

Direct observation by another individual: Principal, Intervention Specialist, Instructional

Coach, other Daily check-in recommended (is this feasible).

Fidelity - Time

Core—180 school days Days per week for core Time can be a factor over the course of a year

Instructional minutes per school day

Core = 90 minutes Strategic = 90 + 15-30 Intensive = 90 + 30-90

Fidelity - Time

Reading block composition Reading or Language Arts Spelling

Instruction vs. practice

10/3 10/4 10/5 10/6 10/7

PA PA PA PA PA

PH PH PH PH PH

V V V V V

Fl Fl Fl Fl Fl

Comp Comp Comp Comp Comp

         

10/10 10/11 10/12 10/13 10/14

PA PA PA PA PA

PH PH PH PH PH

V V V V V

Fl Fl Fl Fl Fl

Comp Comp Comp Comp Comp

10/17 10/18 10/19    

PA PA PA    

PH PH PH    

V V V    

Fl Fl Fl    

Comp Comp Comp    

10/24 10/25 10/26 10/27 10/28

PA PA PA PA PA

PH PH PH PH PH

V V V V V

Fl Fl Fl Fl Fl

Comp Comp Comp Comp Comp

Fidelity - Time

Benchmarking Assessments—scheduled 3 times per year (September, January, May)

Leadership and Problem Solving Team meetings scheduled regularly (monthly, weekly, biweekly, 3/qtr, etc.)

Progress Monitoring—weekly, biweekly, monthly

Fidelity - Assessments

Assessment Benchmark, Progress Monitoring, Diagnostic Delivery and scoring--consistency Use the script for ALL probes Practice probing and discuss scoring—

longitudinal consistency.

Data generated by assessments can only be as reliable as the extent to which the assessmentsare implemented

In a consistent and standardized way. Without measuring the fidelity of assessment

implementation, student responses cannot be evaluated with any reliability.

Student test results depend upon assessments being implemented and scored correctly

Formative evaluation is a KEY component of RtI

Next Steps:

What are your next steps for fidelity? Priorities:

Day to day quality instruction can have a direct impact on student achievement. Is it happening?

Consistent, planned, quality instructional opportunities Student need—targeted instruction and assessment

Leadership

Lessons Learned from the Front Line of RTI Implementation in Montana Schools RTI has developed differently at each site, decisions based upon local

culture, resources, and needs

Strong leadership is required

Administrators need specific professional development prior to systems change commitment.

Process can be vulnerable when:• school leadership changes• school staff changes• there is a large input of new students• When new content is added (e.g. Math)

“Time” is the biggest threat to full RTI implementation.

Beebe-Frankenberger, MASP Fall 2007

RTI Implementation: Most Frequently Experienced Issues In Need of Solutions

Time for RTI Team Meetings Scheduling (TIME)

Instructional blocks Tier 2 & Tier 3 interventions Ongoing assessment

Professional development for new curricular programs, interventions, and assessments

Increase communication and collaboration among teachers and staff; shared responsibility

Resistance to sharing students Time to train people

And…………TIME, TIME, TIME

Beebe-Frankenberger, MASP Fall 2007

Local Decisions: Strong Leadership and Planning is Critical

Curriculum and interventions Staffing (instruction, intervention, assessment) Types of specific assessments Differentiated classroom instruction vs. instructional grouping

(walk-to-read) RTI Team composition and function(s) How data is reviewed Forms (e.g. procedural, parent notification) Parent involvement Community involvement

Beebe-Frankenberger, MASP Fall 2007

RTI Leadership Team

Principal Instructional Coach Special Education Teacher(s) General Education Teacher(s) Paraprofessionals School Counselor School Psychologist Parent Secretary Other???

Identify and Define RolesJob Descriptions/Expectations

For all staff, including:• Administrators• Teachers• Specialized Personnel (counselors, speech, etc.)• Para educators• Lunchroom Staff• Bus Drivers• Maintenance• EVERYONE!!!!!

Team Leadership in RTI RTI Leadership Team is responsible for co-

planning with the building leader for RTI Implementation: Consensus and infrastructure building Implementation and sustainability Make collaborative decisions about essential components

based upon local culture, resources, and needs Provide the infrastructure for RTI Implementation and

eventually, sustainability Redeploy fiscal resources to provide continuum of funding

Instructional Leader-Principal

Responsibilities: Know the program(s) Model and teach effective instructional

practices. Fidelity Principal is the building instructional leader

Responsibilities/decisions made as the principal; others as a member of a team.

Principal’s Critical Decisions:Making TIME Building Leaders schedule TIME into the daily,

weekly and yearly calendar for: Instructional blocks (90 min reading, 60-90 math) Intervention blocks (enrichment, “freeze time” Ongoing, frequent professional development Regularly scheduled RTI Leadership Team meetings Regularly schedule RTI Problem Solving Team meetings Benchmarking, progress monitoring, diagnostic assessments Whole staff meetings for collaboration, teaching strategies,

and sharing success. Orienting new staff to RTI

Principal’s Critical Decisions: Finding Resources

Building leaders work with districts, school boards, state education departments to access fiscal resources to fund: Ongoing professional development Staffing Substitute teachers to relieve teachers to engage in RtI

collaborative meetings and professional development activities

Purchase evidence-based core curriculum, interventions, assessments

Leadership in RtI

Essential to success Collaborative model And…here are tips on Celebrating Your

Success!..........

Top 10 Ways to Succeed with RTI

10. START SMALL… don’t over commit!

11. Invest the resources to know what you are doing.

12. Be supportive of failure – no one fails alone.

13. Promote it, don’t sell it.

14. Let innovation spread naturally. If it is better than what came before it,

it will be naturally reinforced.

5. Change how you think

6. Be in for the long haul, not the short burn

7. Give away credit

8. Support knowledgeable leaders

9. Work smarter, not harder!

Source: David Tilly, 2005

For next time…

Fidelity Community/Family Education/Involvement

Try to send one communication to parents concerning what your school is doing to implement RtI.

Leadership