DATA-BASED DECISION MAKINGUsing Outcome and Fidelity Data with Individual Student Support Plans

Session E12

Kelsey R. Morris, EdD—University of OregonDiane LaMaster—Illinois State Technical Assistance Collaborative

•Session Overview• Progress monitoring student-specific, behavior support

plans

• Selecting and prioritizing target behavior(s) to monitor

• Implementing a measurement system

• Evaluating behavioral progress monitoring data to inform intervention decisions

•Goal• Plan and carry out data collection to monitor target

behavior

• Use graphed progress monitoring data to determine when intervention changes are needed

Maximizing Your Session Participation

• Where are you in your implementation of the concepts presented?• Exploration & Adoption

• Installation

• Initial Implementation

• Full Implementation

• What do you hope to learn?

• What new learning do you take away from the session?

• What will you do with your new learning?

Data-based Decision Making• Effective teams use data to document progress and outcomes, guide

decisions, and inform stakeholders (Boudett, City, & Murnane, 2006; Burke, 2010; Deno, 2005; Hill 2010; Newton, Algozzine, Algozzine, Horner, & Todd, 2011; Newton, Horner, Algozzine, Todd, & Algozzine, 2009; Pidgeon & Gregory, 2004; Renfro & Grieshaber, 2009)

• A critical predictor of sustained implementation of SWPBIS (Coffey & Horner, 2012; McIntosh et al., 2013)

• Fidelity and student outcome data are essential (Fixsen, Blase, Metz, & Van Dyke, 2013)

• Continues to be a struggle for schools (Dunn, Airola, Lo, & Garrison, 2013; Schildkamp, Ehren, & Lai, 2012; Telzrow, McNamara, & Hollinger, 2000)

• Advances in computer technology could provide efficient means for data management (Wayman, 2005)

Intensity of Supports• Essential Question

• Is the student successful with this level of support?

• Intensity is a two-way street.• Improved student outcomes are

the result of continually monitoring and modifying (as needed) instruction, interventions, and supports.

Essential Tier III Systems ElementsCoordinating Team

Tier III systems planning team meets regularly

Team has an identified leader

Membership represents behavioral expertise, administrative authority, intensive support expertise, knowledge about students, and knowledge about school operations

Student Support Team

A uniquely constructed team exists for each individual student support plan

Student support team is comprised of relevant stakeholders

Student support team exists to design, implement, monitor, and adapt the student-specific support plan

Data-based Decision Making

Outcome and fidelity data are reviewed by a student’s support team at least monthly

Data are used to modify the support plan to improve behavior outcomes and improve fidelity of implementation

Student Support PlanHypothesis Statement

• Operational description of problem behavior

• Identification of context where problem behavior is most likely

• Maintaining reinforcers (e.g., behavioral function) in the identified context

Comprehensive Support

• Teaching strategies

• Strategies for removing rewards for problem behavior

• Specific rewards for desired behavior

• Safety elements, as needed

Systematic Evaluation

• Process for assessing fidelity

• Process for assessing outcomes

• Action plan for implementation

Intensity of Assessment

Universal—primary prevention• Monthly

Secondary—small group, targeted• Weekly or twice monthly

Tertiary—individualized, intensive• Daily or multiple times per week

Progress Monitoring• Progress monitoring is the process of systematically

planning, collecting, and evaluating data to inform programming decisions.

• Helps determine intervention effectiveness

• Helps in the development of effective support plans

Planning Progress Monitoring• Selecting target behaviors is part of planning for behavioral

progress monitoring.

• Plan for data collection• Select target behavior(s) to monitor

• Choose method for monitoring the behavior(s)

• Create a plan for collecting data (e.g., schedule, person(s) responsible)

• Collect data

• Evaluate data to make decisions

Selecting Target Behaviors• Identify the target behavior(s) of concern

• What does it look like? When does it occur? What is the perceived motivation?

• Define the target behavior(s)• A clear definition allows us to collect more reliable data

• Can you see it? Can you measure it? Do you know what it is and is not?

• Prioritize the target behavior(s)• More feasible data collection

• More efficient data analysis

• More effective decision making because the most important behavior is the focus

Develop a Measurement Approach

Considerations:

• How often will data be collected?• Related to intensity of behavior and timelines for making

intervention decisions

• Where will data be collected? What context(s)?

• When will data be collected?

• Who will collect the data?• Consider when, where, and how the data will be collected

• When and how will the data be entered to allow for analysis?

Adult Behaviors Cause Student Change

Outcomes Fidelity

Monitoring and Evaluating Progress• Involves examining the progress monitoring data to

determine if the student is responding to the intervention and supports.

• Involves managing and organizing data to support summary and analysis.

Consider:• Do you have a data system that supports graphing?• Who will be responsible for entering the data?• Who will be responsible for generating graphs/reports?• Who will review/analyze the data?

SIMEOSystematic Information Management for Educational Outcomes

SIMEO II• Online data collection and graphing database system

used to assist PBIS tier III student/family teams with data-based decision making.

• Access to this data through a virtual connection • 24 hours a day, 7 days a week

• Password protected

• Graphing capability

SIMEO Tools and Data• Student support teams use SIMEO to/for:

• Engage students, families, & teachers

• Team development & team ownership

• Ensure student/family/teacher voice• Identifying true needs and prioritizing items

• Effective interventions• Serious use of strengths

• Natural supports

• Focus on needs vs. services

• Progress monitoring and sustainability

• System support buy in

SIMEO Tools at Tier III• Student Disposition Tool (SD-T)

• Complex FBA/BIP and Wraparound

• Education Information Tool (EL-T)• Complex FBA/BIP and Wraparound

• Home, School, Community Tool (HSC-T)• Wraparound and RENEW

• RENEW High School Youth Status Tool• RENEW H.S.

Home, School, Community Tool (HSC-T)

• 33 items to assist teams with capturing strengths and needs of the student/family across home, school and community

• Questions address life domain areas:• Safety/Medical basic needs

• Social relationships

• Emotional functioning

• Behavioral functioning

• Cultural/Spiritual

HSC-T

Milton’s HSC-THOME SCHOOL COMMUNITY

Student’s family needs to know he is safe. Student needs to learn skills to be independent at home and in the community.

Milton’s Student Disposition Tool (SD-T)At Risk of Failure in Home, School, Community Placements

Jude’s HSC-T

Utility: engage the family and identify needs at baseline

HOME SCHOOL COMMUNITY

Jude’s HSC-T Progress Monitoring

HOME SCHOOL COMMUNITY

Longitudinal Data

HOME SCHOOL COMMUNITY

Jude’s Student Disposition Tool (SD-T)

B T2 T3 T4 T5 T6 T7

ODR

16 13 20 1 6 1 5

ISS

2 2 4 0 0 0 0

Jude’s Education Information Tool

ISIS-SWISIndividual Student Information System

Individual Student Information System

• ISIS-SWIS is a decision system for students receiving more intensive, individualized supports for academic, social, or mental health services.

• Allows for:• Uploading and storing documentation

• Defining data collection measures

• Summarizing data for decision making

Advantages of ISIS-SWISEfficiency

Structured creation and maintenance of student files

One home for progress monitoring, goal setting, and decision making

Instantaneous access to data

Equity

Equal access to quality support plan management

Enabling of clear roles, responsibilities, and predictability

Quality

Supports compliance with federal procedures for Tier III support

Comprehensive student file for quality decision making

Documentation of progress and intervention history

Flexibility

Files and measures tailored to a student’s needs

ISIS-SWIS Main

ISIS-SWIS School-wide Reports

Student File: Brian Bender

Data Entry Aligned with Measures

Brian: Assignment Completion

Brian: Assignment Completion & Asking For Help

Carly: Rate of Disruption & Staff Fidelity

ISIS-SWIS Study• The Effects of Self-delivered Performance Feedback and

Impact Assessment via the Individualized Student Information System (ISIS-SWIS) on Behavior Support Plan Treatment Fidelity and Student Outcomes (Pinkelman, 2014)

• Study Conditions

• Researcher as ISIS-SWIS Faciliator• Provide ISIS-SWIS training

• Provide follow-up coaching and support

• Provide ongoing technical assistance

• Trained teachers as ISIS-SWIS Coordinators

• Trained educational assistants as ISIS-SWIS Users

• Held weekly meetings with teachers and educational assistants

Teachers as ISIS-SWIS Coordinators• Manages student educational program

• Manages student file in ISIS-SWIS

• Outcome• Teacher competent in using ISIS-SWIS

• Student file established and ready for use

EA as ISIS-SWIS User• Team member who needs access to student file

• Enter data into ISIS-SWIS• Self-monitor fidelity data

• Fidelity checklist

• Rating scale (0-5)

• Student outcome data• Percent of points earned

• Frequency of problem behavior

• Number of teacher directed tasks

Weekly Meetings• EA, teacher, and researcher

• To ensure EA was using ISIS-SWIS regularly• Praise for using specific features of ISIS-SWIS regularly

• Identify features not being used

• Model, practice, and give feedback for features not being used

• Agreement for use

• Review graphs for data-based decision making• Is the plan being implemented with fidelity?

• Is the plan effective at improving student behavior?

• Do any changes need to be made?

Results: Fidelity

Baseline ISIS-SWIS

Observed Fidelity

Per

cen

tage

BS

P C

omp

onen

ts

Results: Fidelity & Academic

Engagement

Baseline ISIS-SWIS

Academic Engagement

Per

cent

age

BSP

Com

pone

nts

Percent 10 s Intervals A

E

Observed Fidelity

Results: Fidelity & ProblemBehavior

Baseline ISIS-SWISP

erce

nta

ge B

SP

Com

pon

ents P

ercentage 10s In

tervals

PB

Observed Fidelity

Takeaways• Tier III progress monitoring should be individualized just

as the Tier III supports are individualized.

• Student outcome and implementation fidelity data are critical

• Data-based decision making continues to be difficult for schools (Dunn et al., 2013; Newton et al., 2012; Schildkamp et al., 2012; Telzrow et

al., 2000).

• A lack of efficient tools and systems to assist in data collection, organization, and summarization impedes the process.

DATA-BASED DECISION MAKINGUsing Outcome and Fidelity Data with Individual Student Support Plans

Session E12

Kelsey R. Morris, EdD—University of OregonDiane LaMaster—Illinois State Technical Assistance Center