Minnesota Early Indicator and Response System (MEIRS) Training Today Session III: Initial...

Minnesota Early Indicator and Response System (MEIRS) Training

Today Session III:

Initial Implementation/Continuous Improvement

Upcoming

Session IV: Reflect/Sustain/Celebrate!

• Welcome & Introductions• Quick Review of Key Points MEIRS Sessions I & II• Updates & Progress • Questions & Challenges• Initial Implementation and Measuring Fidelity

• These sessions will not provide “the answer,” but will build your capacity to use the MEIRS process to address your school community needs

• We will be sharing with and learning from each other – thank you for your active participation!

education.state.mn.us 2

Agenda

Review: What is MEIRS?Minnesota Early Indicator and Response System

A Tool For Use by Interested Schools/Districts - Voluntary

Two Components

1. An early warning data system to screen for students in grades 6 and grade 9 who are at risk of not completing high school in four years

2. A corresponding process to a) review data for students at risk of not completing high school in four years, b) select, c) provide and d) monitor supports to help get students back on track to graduation

MEIRS Session I: Introduction and Overview• World’ Best Workforce – Graduation Goal• Key predictors of drop out and definitions of MEIRS

variables• Overview: Seven Step Early Warning System

Implementation Process• Establishing and using a team to review MEIRS

data and determine supports/interventions• Accessing the MEIRS secure report• How to read the report and interpret the data

Where Have We Been?



Seven Step Early Warning System Implementation Process

Data Driven Process (Nat’l High School Center, 2010)

STEP 1Establish Team Roles

STEP 7 Evaluate & Refine Monitor & Adjust

STEP 2 Provide Orientation

STEP 3 Run MEIRs Report

STEP 4 Review & Interpret Data

STEP 5 Dig Deeper

STEP 6 Select, Implement & MonitorInterventions

MEIRS Session II: Reviewing Your Data & Exploring a Response System• Brief overview of stages of implementation• Moving into the data – types of data• Data mining – conversations about what the data tell

us (root cause analysis, forming hypotheses, 5 whys, whys, setting goals, determining focus)

• Using a multi-tiered/leveled system to support students on the path to graduation (universal, targeted and intensive)

• Selecting a usable intervention(s) – “Hexagon Tool”• Homework: Select a useable intervention for Session III

Where Have We Been?


A Systematic Approach


Developing a Response System to Support Student Engagement

ALL(Layered not Leveled)

SOME

FEW

Tertiary Prevention – Tier 3 – Few • Individualized practices of increased

intensity for students who show minimal response to secondary prevention (individual level)

Secondary Prevention – Tier 2 – Some • Evidence-based practices of moderate

intensity that address the learning or behavior of most students at risk (group level)

Universal/Primary Prevention – Tier 1 – All• High quality core academic and behavioral

instruction and engagement strategies for all students

Updates & Progress• Please have a spokesperson provide a brief

informal update on progress to date in using the MEIRS data (team meetings; what did the data tell you; how have you used the data; what interventions are you considering implementing or strengthening)

Questions & Challenges• What questions do you have and what

challenges are you facing - brainstorm possible solutions

Where We are Today!


Today, we will talk about…• Installation and Initial Implementation Activities• Measuring fidelity of implementation

– Is the intervention being delivered in the way that it was intended – as developed?

• Developing and using practice profiles– Defining what it should look and sound like if the

intervention is being delivered as intended

• Continuous improvement cycles (PDSA)

Have You Selected a Usable Intervention?


Implementation Stages

Exploration InstallationInitial

ImplementationFull

Implementation

2-4 Years for School-Based Implementation

• Assess Needs.• Assess Fit and

Feasibility.• Identify Structural

and Functional Changes.

• Promote “Buy-In.”

• Structural Changes Made.

• Define and Initiate Training.

• Develop Coaching Plans.

• Evaluate Readiness of Data Systems.

• Surviving the Awkward Stage!

• Manage Change.• Utilize Data

Systems to Measure Progress.

• Use Improvement Cycles.

• Maintain and Improve Skills.

• Change or Adjust Policies to Support Work.

• Use Data Systems that are Reliable and Efficient.

A usable intervention is teachable, learnable, doable, and can be readily assessed in practice• Includes a clear description of the philosophy, research and

principles that underlie the intervention.• Includes inclusion and exclusion criteria about the population

most likely to benefit.• Describes essential functions and features. • Includes operational definitions of each core component in

terms that can be taught, learned, and done in practice.• Includes performance assessments that can provide evidence

that the program is being used as intended and is resulting in the desired outcomes (fidelity).


What is a usable intervention? (practice, strategy, program)

• What is the evidence that suggests this will be effective?

Use resources such as information from national technical assistance centers; documents and manuals; experts; research

• What are the key features of the program or practice that must be in place?

Research findings, materials, manuals do not necessarily provide clarity around core elements of the intervention.

Current and new evidence-based practices, frameworks, programs will have a range of operational specificity.

Developing clarity around the “it” is critical.

• What are the implementation requirements for this intervention?

Questions to Consider and Address


Implementation Pre-Requisites

• After you:– Have identified student needs based on data– Narrowed the focus based on digging deeper into the

data– Looked for “best practices” to address the need– Selected the intervention, strategy or practice

• Then it’s time to:– Clearly identify and operationalize core features of the

intervention

• Practice Profiles– Help operationalize practice, program and its core

features

13

– Provide operational definitions of critical features or core components of the intervention

– Based on Hall and Hord’s (2006) Innovation Configurations

– See Active Implementation Hub (12 minute video) – https

://unc-fpg-cdi.adobeconnect.com/_a992899727/ai-lesson3/

“Practice profiles enable a program to be teachable, learnable, and doable in typical human service settings. In this lesson you will learn to: identify the purpose of a practice profile, identify critical areas for successful practice profile development, identify levels of performance for a NIRN practice profile, and employ the practice profile process to operationalize your program or intervention.”


Practice Profiles

https://unc-fpg-cdi.adobeconnect.com/_a992899727/ai-lesson3/




Core Component or Critical Feature

Contribution to the Desired Outcome

Expected/Proficient: Description of educator behavior

Developmental: Description of educator behavior

Needs Improvement: Description of educator behavior

For each component, include an explanation of how it contributes to the desired outcome

List indicators.Includes activities and describes behaviors that exemplify educators who are able to generalize required skills and abilities to wide range of settings and contexts; skills are used consistently and independently – skills are sustained over time while continuing to grow.

List indicators. Includes activities and describes behaviors that exemplify educators who are able to implement required skills and abilities but in a more limited range of contexts and setting – skills are used inconsistently or need coaching to complete or successfully apply particular skills for improvement in order to move into expected. This column helps to define the focus of coaching.

List indicators. Includes activities and describes behaviors that exemplify educators who are not yet able to implement required skills or abilities in context.

ENTER CORE COMPONENT OR CRITICAL FEATURE HERE

ENTER DESIRED OUTCOME HERE

ENTER INDICATOR(S) OF CORE COMPONENT



Practice Profile for: ENTER SCHOOL NAME HERE

Name of Instructional Strategy or Practice ENTER HERE Date: ENTER DATE HERE

Purpose/Rationale for using this instructional strategy or practice (By using this document, teachers will engage in…): ENTER HERE

Desired Outcome: (To raise student achievement by…): ENTER DESIRED OUTCOME HERE

Citation of Research used: ENTER CITATION HERE

15

Practice Profile – What does it look like?

• Critical features of the intervention are identified and articulated (listed in the left hand column of the chart)

• For each critical feature – “implementer” behaviors are described that meet… – Best practice and “gold standard”– Acceptable variations in practice “developmental”– Identified ineffective practices and undesirable

practices• Have you ever developed or helped to develop a

Practice Profile or Innovation Configuration?

Hall and Hord, 2010 Implementing Change: Patterns, Principles, and Potholes (3rd Edition) and Adapted from work of the Iowa Area Education Agency

16

• A process is followed that includes – Forming a task group – Identifying & creating clear understanding of the purpose and

use of the practice profile – Gathering prerequisite materials and resources about the

intervention to use in developing the content – Determining core features – Defining levels of proficiency – Describing behaviors to operationalize the core features at each

level of proficiency – Developing drafts to be reviewed and revised at multiple levels

How are Practice Profiles (PP) Developed?







Actionable Actionable

feedback provides information to the recipient about how to improve in order reach what they are trying to achieve.

- Educator comments on what went right and what did not work – so learners can respond- Feedback is specific &concrete (e.g., You really worked hard on that poster, the final draft includes much more detail than the first one!)- Feedback is useful (e.g., The next time you come into the classroom and I am not here, sit at the table, get out your materials and begin working on the posted assignment)

Educator comments on the students performance, but is not sufficiently descriptive or is judgmental in nature (e.g., You seem really bored)

Vague or nondescript feedback that does not provide information about how to improve (such as “good job”)

Transparent Goal-ReferencedUser-FriendlyTimelyOngoingConsistent

Practice Profile for: Best School EverName of Instructional Strategy or Practice Effective Feedback Date: 10.31.14

Purpose/Rationale for using this instructional strategy or practice (By using this document, teachers will engage in…): To describe key components of effective feedback so that it is delivered with consistency.

Desired Outcome: (To raise student achievement by…): Improve academic and behavioral performance and increase student results (engagement, participation, motivation, assignment completion)

Citation of Research used: G. Wiggins, Seven Keys to Effective Feedback (2012)

18

• Responding to Problem Behavior in Schools: The Behavior Education Program (D. Crone, R. Horner, & L. Hawken)

• MiBLSi Check In/Check Out: Fidelity of Implementation Scoring GuideMichigan's Integrated Behavior and Learning Support Initiative (MiBLSi) is a Mandated Activities Project (MAP), funded under the Individuals with Disabilities Education Act (IDEA) through the Michigan Department of Education, Office of Special Education.adapted from L. Hawkin (2006) University of Utah

Resources


What is it?• Tier II group oriented intervention• Designed for students a) with behavior problems observed across

multiple settings, b) who do not require more immediate interventions (for violent behavior infractions)

• Foundational knowledge/implementation of Tier I PBIS (Positive Behavioral Interventions and Supports) is considered critical to enhance effectiveness

• Requires a CICO Coordinator

Four components• Morning Check-in• Teacher feedback throughout the day (via point card)• Afternoon Check-out• Parent feedback

Check-In Check-Out (CICO)


• Check-in with CICO Coordinator in the morning• Student carries point cared based on school wide expectations• Student receives frequent and regular feedback on behavior from

adults throughout the day• Student reviews goals with CICO Coordinator at end of day (check-

out)• Student takes point card home for parent signature and positive

feedback

Measures of effectiveness and implementation: Number of students in program is tracked daily and over time; % of students meeting their goals; % of program steps implemented; number of behavioral referrals, out of school and in school suspensions

CICO Daily Activities







Morning Check -In

The Check-In provides increased positive adult contact, social skills learning and increased opportunities for feedback to achieve desired outcomes

- Greets student by name with positive affect (smile)- Collects progress report from previous day and provides effective specific feedback about comments on the report- Gives points (0 = did not bring report; 1 = brought without signature; 2 = brought with signature)- Prompts student to collect coupon or trade in points – if relevant - and return to homeroom- Records data on attendance and files progress report

Acknowledges student with low affect

Collects progress report, but does not provide specific feedback

Gives points and records points

Does not consistently record attendance data

Does not acknowledge student, reaches out for progress report – but does not look at it .

Says points will be given later or tomorrow

Interacts with student using sarcasm - such as “what were you thinking – you think it is OK to poke other students with your pencil?”

Teacher Feedback

Afternoon Check -Out

Practice Profile for: Excellent SchoolName of Instructional Strategy or Practice Check-In Check -Out Date: 10.31.14

Purpose/Rationale for using this instructional strategy or practice (By using this document, teachers will engage in…): To better define and increase consistent delivery of the Check-In Check-Out

Desired Outcome: (To raise student achievement by…): To raise student achievement by preventing escalation of disruptive behavior; decrease in and out of school suspensions and decrease occasions of future problem behaviorCitation of Research used: Everett, Sugai, Fallon, Simonsen, O’Keefe (2011) School-wide Tier II Interventions: Check-In Check-Out

22

Evaluation Question Data SourceP = permanent product;

I = Interview; O= Observation

1. Does the school employ a CICO coordinator whose job is to manage the CICO (10-15 hours per week allocated) (0 = No CICO Coordinator, 1 = CICO coordinator but less than 10 hours per week allocated, 2= CICO Coordinator, 10-15 hours per week allocated)

Interviews with IAdministrator & CICOCoordinator

1. Does the school budget contain an allocated amount of money to maintain the CICO? (e.g. money for reinforcers, Daily Progress Report forms, etc. (0 = No, 2 = Yes)

CICO Budget P / IInterviews

1. Do students who are referred to the CICO receive support within a week? (0 = more than 2 weeks between referral and CICO support, 1 = within 2 weeks, 2 = within a week)

Interview P / ICICO Referrals & CICO Start dates

1. Does the administrator serve on the CICO team or review CICO data on a regular basis? (0 = no, 1 = yes, but not consistently, 2 = yes)

Interview I

1. Do 90% of CICO team members state that the CICO system has been taught/reviewed on an annual basis? (0 = 0-50%, 1 = 51-89%, 2 = 90–100%)

Interview I

1. Do 90% of the students on the CICO check-in daily? (Randomly sample 3 days for recording)

2. (0 = 0-50%, 1 = 51-89%, 2 = 90–100%)

CICO recording form P

1. Do 90% of students on the CICO check-out daily? (Randomly sample 3 days for recording)

2. (0 = 0-50%, 1 = 51-89%, 2 = 90–100%)

CICO recording form P

1. Do 90% of students on the CICO report that they receive reinforcement (e.g. verbal, tangible) for meeting daily goals? (0 = 0-50%, 1 = 51-89%, 2 = 90–100%)

Interview students on CICO I

Check-In/Check-Out (CICO)Fidelity of Implementation Scoring Guide

23

What will “drive” change and help ensure effective implementation so that we get intended and desired outcomes?

• Selecting, training and coaching of who will implement the intervention with fidelity

• Measuring implementation fidelity (effort and quality)

• Measuring progress of students and impact

• Putting a data system in place for making adjustments

• Addressing barriers to implementation

• Aligning with system context, mission and vision

• Providing leadership to support the work at multiple levels

Considerations


“Discovering what works does not solve the problem of program effectiveness. Once models and best practices are identified, practitioners are faced with the challenge of implementing programs properly. A poorly implemented program can lead to failure as easily as a poorly designed one. “

(Mihlalic, Irwin, Fagan, Ballard, & Elliot, 2004)

“An evidence-based practice is one thing. Implementation of an evidence-based practice is a completely different thing.”

(Fixsen, 2008)


Remember…

Usability TestingImprovement Cycles

Plan

DoStudy

Act Plan

DoStudy

Act Plan

DoStudy

Act

Changing on purpose to support the new way of work!

Session IV: Reflect, Sustain, Celebrate!

Contacts and Resources:

Regional Center of Excellence Director

Cammy Lehr

Minnesota Department of Education

[email protected]

MEIRS Webpage (MDE)

http://education.state.mn.us/MDE/StuSuc/DropPrev/MEIRS/index.htm

Thank you for your participation!

mailto:[email protected]





http://implementation.fpg.unc.edu/about-the-ai-hub

The Active Implementation Hub

For More Information on Active Intentional Management of

Implementing Educational Practicessee…




Minnesota Early Indicator and Response System (MEIRS) Training Today Session III: Initial...

Documents

Transcript of Minnesota Early Indicator and Response System (MEIRS) Training Today Session III: Initial...