Post on 18-Dec-2015
Using Data for Effective Team Meetings within PB4L
Rob HornerUniversity of Oregon
www.pbis.org
Goals• Overview of Team-Initiated Problem Solving (TIPS)
• Using data to define a problem with precision
• Building comprehensive solutions
• Using data to monitor, evaluate and adapt
Implement Solution with High Integrity
Identify Goal for Change
Identify Problemwith
Precision
Monitor Impactof Solution and
Compare against Goal
Make SummativeEvaluationDecision
MeetingFoundations
Team-Initiated Problem Solving (TIPS II) Model
IdentifySolution and
CreateImplementation
Plan with Contextual Fit
Collect and Use
Data
What, Who, When, Where, and Why, and How Often?
How do we want the problem to
change?
What are we going to do to bring about
desired change?
Did we implement with fidelity?
Has the problem been solved?
Maintain, Adapt, Stop?
8
Problem Solving• What is a problem?
o Any difference between what exists and what is desired.
• What its the difference between problems that are significant enough to warrant a solution and those that can be tolerated?o Social values o Academic expectationso Political and organizational expectations
TIPS II Training Manual (2014) www.uoecs.org
10
Problem Solving• Define Problem
o Identify problems earlyo Identify problems with PRECISION before building solutions
• Establish Goalo Where do we want to be
• Build Solutions that are comprehensive and locally relevant
• Prevent, teach, extinguish, reward, monitor• Select solutions based on local strengths, culture, organization
• Build Action Plan for Implementing SolutionsTIPS II Training Manual (2014) www.uoecs.org
11
Precise Problem Statement• How often is a behavior occurring?
• Where is the behavior most likely to occur?• What is the behavior of concern?• Who is doing the behavior?• When is the behavior most likely?
• Given Who, What, Where, When:
oWHY does the behavior keep happening in this context?
TIPS II Training Manual (2014) www.uoecs.org
Start with Primary Problem Statements
Look at the Big Picture. Then use data to refine the problem to a Precise Problem Statement.
Move to Precise Problem Statements
Office discipline referrals for 3rd graders are above national median for schools our size.
Referrals for physical aggression among third grade students from 11:30-12:30 in the cafeteria are occurring at 2-3 per day and increasing over time. It is believed that this is happening due to insufficient supervision and access to peer attention.
Precise Statement?
PreciseNot Precise
Indicates a difference between what is
happening and what is desired.
What, Who, Where, When, Why, and How
Often
Not Precise or Not Precise?
• Students on bus #43 are physically aggressive to gain peer attention at least 5 times per bus ride
• Three 5th grade boys are name calling and touching girls inappropriately during recess in an apparent attempt to obtain attention. This is occurring at least 5 times a week.
• Boys are engaging in sexual harassment.
Precise
Not Precise
Precise
Not Precise or Not Precise?• Darin uses inappropriate language with a high
frequency in the presence of both adults and other children. This is creating a sense of disrespect and incivility in the school.
• Students are being aggressive on the bus.
Not Precise
Not Precise
Not Precise or Not Precise?
Minor disrespect and disruption are occurring at a rate of 1.5 per day/month, ( increasing over time) and are most likely during the last 15 minutes of our block periods when students are engaged in independent seat work. This pattern is most common in 7th and 8th grades, involves many students, and appears to be maintained peer attention. Precise
Not Precise or Precise?
• Danielle is 9 years old and refuses to follow parental requests to pick up toys, help with dishes, or do homework. She will sulk and pout until we get frustrated and send her to her room.
• Danielle is 9 years old and 3-4 times per day refuses to follow parental requests to do minor tasks like pick up toys, help with dishes or do homework at home. She will sulk and pout. The behaviors are maintained by escape from non-preferred activities.
Not Precise
Precise
Your Turn to practicea. Make up a PRIMARY problem statement
b. Transform that description into a PRECISE problem statement.• Who• What• Where• When• Why• How Often
Write down Primary and Precise statements Be ready to share your hypothetical example with group
5 minutes
19
Using Data to Define Problems with
Precision
• Major ODRs per day per montho Levelo Trendo Pattern (peak months?)
• Compare with national median (or average)
• Compare with local norms
TIPS II Training Manual (2014) www.uoecs.org
SWIS summary 2013-14(Majors Only)
4858 schools; 2,476,989 students; 3,054,592 ODRsGrade Range Number of
SchoolsMean Enrollment per school
MeanODRs per 100 stud/ school day
Median ODRs per 100 per stud/ school day
25th PercentileODR/100/ school day
75th PercentileODR/100/ school day
K-6 3021 450 .31 (.40) .20 .10 .38
6-9 894 613 .48 (.67) .33 .17 .58
9-12 452 817 .57 (1.04) .36 .17 .69
PreK-8 273 418 .39 (.41) .27 .13 .51
PreK-12 76 341 .73 (1.41) .36 .16 .58
Monitoring Data;Major ODRs per day per
month
Although the months are different, an average referral per day per month count allows for true comparisons.
Average Referrals Per Day Per Month
Transforming Data into
Information• Tell the story that is present in your data.
• How often are problems occurring?• Is the trend increasing, decreasing, stable over time?• How does this level compare to last year?• Are there months with unusually high levels (peaks)?
• How do our patterns compare with others?• How do our patterns compare with prior years?• How do our patterns compare with what our students, families
and staff believe is appropriate/acceptable?
Do we have a Problem?• Review ODR per day per month
o What is the overall level?o What is the pattern?o What is typical?o What is possible?o What is needed?
• If there is a problem… then define with precision
Elementary School with 150 Students
24
Questions to Ask of the Data
What is happening?What is typical?
What is possible?What is needed?
What is the narrative?Questions to Ask
What is happening?
What is typical?
What is possible?
What is needed?
75th PercentileMedian25th Percentile
Elementary School Example
Median
Elementary School Example
Median
Middle School Example
Median
765 Students
High School Example
Median
Multi-Year Example
Median
Year OneYear Two
What When
Who
Why
Designing Effective Behavior Support
Where
Define Problems with precision
TIP
S I
I T
rain
ing
Man
ual (
2014
)
ww
w.u
oecs
.org
31
How Often
Primary Problem Statements• Data examined at the SW level or with just one graph allow for “primary”
problem statements only, such as – Our school’s…o Average referrals per day per month for All Referrals & Minors are above
the 75th percentile for 5 of 6 monthso Average referrals per day per month for Majors show an increasing trendo Referrals for Defiance totaled 77 instances of the last three monthso Referrals in Classrooms have been too high for the last two monthso Referrals for each month of this year are higher than for the corresponding
month of the previous year TIP
S I
I T
rain
ing
Man
ual (
2014
)
ww
w.u
oecs
.org
32
ImplementSolution withHigh Integrity
Identify Goalfor Change
Identify Problemwith
Precision
Monitor Impact ofSolution and
Compare AgainstGoal
Make SummativeEvaluationDecision
MeetingFoundations
Team-Initiated Problem Solving II(TIPS II) Model
Identify Solution and Create ImplementationPlan with Contextual
Fit
Collect and Use Data
What is the problem?Who? What? Where? When? Why?
TIP
S I
I T
rain
ing
Man
ual (
2014
)
ww
w.u
oecs
.org
33
SWIS Big 4 for October 1, 2011 through December 31, 2011
ClassroomPlaygroun
d
Where?
Questions:1. What location(s) are associated with the most
ODRs?2. Sort by “structured” settings and “non-
structured” settings(Classroom & Gym vs. Commons, Cafeteria,
Hall, Playground)
Where
Where
SWIS Big 4 for October 1, 2011 through December 31, 2011
Defiance
What Behavior(s)
Questions:1. Are most common behavior problems (a) Student-Student, or (b) Adult-Student related?2. Are problem behaviors MAJOR or MINOR or BOTH?
What Behavior(s)
What Behavior(s)
What Behavior(s)
Who
Questions:1. Are there many, a few, or one student associated
with the problem?2. Are there more students emerging?
Who
11:45-12:00
When?When?
Questions:1. Are problem behaviors more likely at some times
of the day?2. What is happening during periods when
problems are most likely?
When
When
When
When: Middle School
WHY?—the hardest question• What is perceived as maintaining the problem behavior?
• Always assess motivation AFTER you have defined:o Who?o What?o Where?
• You always ask WHY students misbehave in the context.
• Look for “primary” motivation if there are multiple possibilities.
Why?• Obtain positives
o Attention from peerso Attention from adultso Personal satisfactiono Physical objectso Preferred activity
• Avoid negativeso Avoid peerso Avoid adultso Avoid worko Avoid undesirable activitieso Avoid undesirable objects/food/ sounds/ etc
Don’t Get too Big
------------
------------
----
Revenge
Prestige
Power
Status
Happiness
Avoid Work
Why?Why
Why• Motivation for many elementary students engaging in disruption
in the classroom
Why• Motivation for middle school students’ non-compliance and
insubordination in the classroom
Implement Solution with High Integrity
Identify Goal for Change
Identify Problemwith
Precision
Monitor Impactof Solution and
Compare against Goal
Make SummativeEvaluationDecision
MeetingFoundations
Team-Initiated Problem Solving (TIPS II) Model
IdentifySolution and
CreateImplementation
Plan with Contextual Fit
Collect and Use
Data
Identify Solution and Create Implementation Plan with
Contextual FitWhat and When
Implement Solution with High Integrity
Identify Goal for Change
Identify Problemwith
Precision
Monitor Impactof Solution and
Compare against Goal
Make SummativeEvaluationDecision
MeetingFoundations
Team-Initiated Problem Solving (TIPS II) Model
IdentifySolution and
CreateImplementation
Plan with Contextual Fit
Collect and Use
Data
Identify Solution & Create Implementation
Plan with Contextual Fit Objectives
o Use solution option categories to brainstorm solution actions for your precise problem statement
o Define the scope of necessary solutions• SW, specific setting, grade/group/individual student
o Define an action plan for each solution action• Who does what by when
TIP
S I
I T
rain
ing
Man
ual (
2013
)
ww
w.u
oecs
.org
56
How to Build a Solution Plan• Use solution elements to guide discussion• Brainstorm options for solution elements• Select a set of actions (one plan) that
o Require the least amount of effort that will produce desired changeso Strengthen what you already do well o Fit with the precision statement and priority o Are efficient & feasibleo Are likely to have the desired impact
TIP
S I
I T
rain
ing
Man
ual (
2013
)
w
ww
.uoe
cs.o
rg
57
Solution Action Elements Solution Action Elements Defined
Prevent Focus on prevention first. How could we reduce the situations that lead to these behaviors?
Teach How do we ensure that students know what they SHOULD be doing when these situations arise?
Reward How do we ensure that appropriate behavior is recognized?
Extinguish How do we work to ensure that problem behavior is NOT being rewarded.
Correct How will you correct errors?
Safety Are additional safety precautions needed?
Solution Implementation Plan Elements
TIP
S I
I T
rain
ing
Man
ual (
2013
)
ww
w.u
oecs
.org
58
Solution Action Elements Possible Generic Solution Actions
Prevent What can we do to prevent the problem?
Adjust physical environment.Define & document expectations and routines.Assure consistent & clear communication with all staff.
Teach What do we need to teach to solve the problem?
Explicit instruction linked to school wide expectations.Teach what to do, how to do it and when to do it.Model respect .
Reward What can we do to reward appropriate behavior?
Strengthen existing school wide rewards.Include student preferences.Use function-based reinforcers
Extinguish What can we do to prevent the
problem behavior from being rewarded? Use ‘signal’ for asking person to ‘stop’.Teach others to ignore (turn away/look down) problem behavior.
Correct What will we do to provide corrective feedback?
Intervene early by using a neutral, respectful tone of voice. Label inappropriate behavior followed by what to do Follow SW discipline procedures
Safety Do we need additional safety precautions?
Separate student from others if he/she is unable to demonstrate self-control.Make sure adult supervision is available.
How much effort is needed to bring about desired change?
• When deciding how many solution elements to implement consider: o Priority for change
• Severity of the problem• Intensity of the problem• Frequency of the problem• Potential safety concerns
o Impact of solution implementation
• Confidence that solution implementation will make a differenceo Feasibility/ Availability of resources
• Professional development, support, time, tools• Existing skills and capacities of implementers• Is this doable? Is it the most efficient approach?
TIP
S I
I T
rain
ing
Man
ual (
2013
)
ww
w.u
oecs
.org
60
Is Solution Appropriate for the context?• Solution actions should fit the context
o Consider your school’s culture & expectationso Consider your school’s improvement plan goalso Consider the school calendaro Consider impact of solution implementation on staff work load &
classroom instruction schedules• Implementation time & effort need to be worth it by meeting the goal
o Consider the function of the problem behavior
TIP
S I
I T
rain
ing
Man
ual (
2013
)
ww
w.u
oecs
.org
61
Your Turn• Too many ODRs (3.4 per day) for aggression and fighting
during 3rd period recess by many boys (and some girls) wanting access to the playground equipment.
• Prevent? Extinction?• Teach? Correction?• Reward? Safety?
• Data System Needed?
Your Turn• Precise Problem:
• Solutiono Prevent Extinctiono Teach Correctiono Reward Safety
o Data System
Action PlanningWhat will be done Who will do it By When
Data Collection1. Did we implement the solution?2. Did the solution benefit students?
Teacher Completed Fidelity ScalesEstablish a fidelity check routine that relates to
Implementation• A 1-5 scale is used for questions • Up to 3 questions per week• Share scale as a poster in the faculty room, an electronic
survey, or paper and pencilWe agreed to provide “high-five greetings” to all students entering your class on time in the morning this week? How did we do?
1 2 3 4 5No Yes
How many days (we agreed to 4) during the week did you review with students the procedures for
passing in the hall?
1 2 3 4 5✓ ✓✓✓ ✓✓✓
Summary• Solve Problems• Build “meeting foundations” (roles, minutes,
systems)• Use data to define problems with precision
o Define problems early
• Use data to build solutions• Include “Action Plans” with each solution• Use data to ask (a) did we implement, and (b) did
the plan benefit students?