Using the Three-Tiered Logic to find the Academic ... Datapresentation09 final.pdf · to find the...
Transcript of Using the Three-Tiered Logic to find the Academic ... Datapresentation09 final.pdf · to find the...
Using the Three-Tiered Logic to find the Academic-Behavior ConnectionBehavior Connection
Melissa Leahy Ph DMelissa Leahy, Ph.D.Kimberly Muniz, M.A./CAS, NCSP
Carroll County Public SchoolsCarroll County Public SchoolsMaryland
Obj tiObjectives
Review why and how to use data for decision-making Provide examples of how CCPS schools use various Provide examples of how CCPS schools use various
forms of data to monitor the effectiveness of PBIS Provide an overview of a current social skills
interventionHighlight and demonstrate templates utilized to share g g p
discipline data with staff and PBS teamsHighlight and demonstrate templates to link behavior
and academic data
Carroll CountyCarroll County
Carroll County has 135,000 residentsCCPS i h 9th l h l i hCCPS is the 9th largest school system in the state 42 schools
28 000 t d t 28,000 students Rural/suburban demographics
CCPS began implementing PBS in 2001CCPS began implementing PBS in 2001 Started with 1 school
Status of PBS in CCPSStatus of PBS in CCPS
17 schools (41%) implement PBS (13 elementary, 4 middle)middle). 3 alternative programs also implement PBS
5 schools will attend new team training in July 5 schools will attend new team training in July 14 schools use SWIS for data collection
Ongoing coaching and support has been and will continue to be provided for all schools implementingcontinue to be provided for all schools implementing PBIS
Supporting Social Competence &Academic Achievement
OUTCOMES4 PBIS
El t
Supporting
Elements
SupportingStaff Behavior
pp gDecisionMaking
PRACTICESPRACTICES
SupportingStudent Behavior
D tData
IS NOT: A scary or “four letter”
IS: Powerful when used to A scary or “four letter”
word Should not intimidate us
Powerful when used to discuss discipline
Empowering when used Should not intimidate us Just numbers
Empowering when used by school teams
Reviewed frequently toReviewed frequently to determine areas of strength and weaknessg
Why Collect Discipline Data?Why Collect Discipline Data?
Decision makingWh t d i i d k ? What decisions do you make?
What data do you need to make these decisions?
Professional Accountability
Decisions made with data (information) are more likely to be (a) implemented, and (b) effective
Wh t D Y D With D t ?What Do You Do With Data?
Make Informed DecisionsLook At Trends Use Data To InformUse Data To Inform
Improving Decision MakingImproving Decision-Making
SolutionProblem SolutionFrom
To ProblemProblem
SolvingSolution
Information
F i t iFrom primary to precise
Primary statements are vague and leave us with more questions than answersmore questions than answers
Precise statements include information about 5 “Wh” questions:questions: What is the problem and how often is it happening? Where is it happening Where is it happening Who is engaging in the behavior? When is the problem most likely to occur?When is the problem most likely to occur? Why is the problem sustaining?
From primary to precise: p y pAn example
Primary statement: Precise statementTh 30 ODR f “There is too much
fighting at our school” There were 30 more ODRs for
aggression on the playground than last year, and these are most likely to occ r from 12 00 12 30 d ringto occur from 12:00-12:30 during fifth grade’s recess because there is a large number of students, and th i i l t d t ttithe aggression is related to getting access to the new playground equipment. “
St t P bl S l iSteps to Problem-Solving
Define the problem(s) Analyze the data
Define the outcomes and data sources for measuring the outcomesg Consider 2-3 options that might work Evaluate each option
Is it safe? Is it doable? Will it work? Which option will give us the smallest change for the biggest
outcome?outcome? Choose an option to try Determine the timeframe to evaluate effectiveness Evaluate effectiveness by using the data Evaluate effectiveness by using the data
Is it worth continuing? Try a different option? Re-define the problem?
Effecti e Data S stemsEffective Data Systems
The data are accurate and valid The data are very easy to collect (1% of staff time) The data are very easy to collect (1% of staff time)Data are presented in picture (graph) formatData are current (no more than 48 hours old)Data are current (no more than 48 hours old)Data are used for decision-making
The data must be available when decisions need to be made (weekly?)
Difference between data needs at a school building versus data needs for a district
The people who collect the data must see the information used for decision-making.
Interpreting Office Referral Data: Is there a problem?
Absolute level (depending on size of school) Middle High Schools (> 1 per day per 100) Middle, High Schools (> 1 per day per 100) Elementary Schools (> 1 per day per 250)
Trends Trends Peaks before breaks? Gradual increasing trend across year?g y
Compare levels to last year Improvement?p
D tData
What data do our schools collect in addition to the “Big 5”?
5 1 R ti f ti k t t f l5:1 Ratio of tickets to referrals
The research tells us that we should be giving 5 positives to each corrective responsepositives to each corrective response
How is that measured?How is that measured? Number of incentives (coupons, coins, etc.) versus the
number of referrals.number of referrals.
N b f RRR Ti k tNumber of RRR Tickets
Quarter K 1 2 3 4 5 TotalQuarter K 1 2 3 4 5 Total
One 306 289 278 236 110 193 1412
Two 678 526 423 278 147 191 2243
Overall 984 815 701 514 257 384 3655
Ratio of Tickets: Referrals
6.0
4.0
5.0
2.0
3.0
0.0
1.0
r r r y h il y e al
SeptemberOcto
ber
November
December
Janua
ryFeb
ruary
March April May
June
Total
Individual Student data:P M it iProgress Monitoring
Student Name: October 2008
94% 94%98%100%
80% 80% 80%
62%72%72%78%72%
70%
80%
90%
50%
30%
53%
40%
50%
60%
0% 0% 0% 0% 0% 0% 0% 0% 0% 0%10%
20%
30%
0%
1-Oct
2-Oct
3-Oct
6-Oct
7-Oct
8-Oct
9-Oct
10-Oct
13-Oct
14-Oct
15-Oct
16-Oct
17-Oct
20-Oct
21-Oct
22-Oct
23-Oct
24-Oct
27-Oct
28-Oct
29-Oct
30-Oct
31-Oct
Average
M thl D tMonthly Data
Student Name: Percentage of points earned
100
708090
100
42 45 46
30405060
4.810 5.7 9
0102030
0Aug/Sept October November December January February March
T i l f St d t R f lTriangle of Student Referrals
1-5%Intensive, Individual InterventionsIndividual StudentsAssessment based
6+ referrals1-5%
5-10% 5-10%
Assessment-basedHigh Intensity
Targeted Group InterventionsSome Students (at-risk)Hi h Effi i
2-5 referrals
80-90% 80-90%
High EfficiencyRapid Response
Universal InterventionsAll Students 0 1 referral80-90% 80-90%All Students Preventive, proactive
0-1 referral
Triangle of Student ReferralsTriangle of Student Referrals
1-5%
Intensive, Individual InterventionsIndividual StudentsAssessment-based
i h i6+ referrals1-5%
5-10% 5-10%
High Intensity
Targeted Group InterventionsSome Students (at-risk)High Efficiency
2-5 referrals
80 90% 80 90%
g yRapid Response
Universal InterventionsAll St d t 0 1 referral80-90% 80-90%All Students Preventive, proactive
0-1 referral
Triangle of Student Referrals2007-20082006-2007
2%2%
Students with 6 or more referrals
2006 2007
Students with 6 referrals
5% 7% Students with 2-5 referralsStudents with 2-5
referral
referrals
93% 91% Students with 0-1 referrals
Students with 93% referrals0-1 referrals
Triangle of Student Referrals:A t/S t b 2005
August/September 2005
07%
Intensive, Individual InterventionsIndividual Students A t b d 1-5%
Students with 2 or more referrals
07%
10 15%
03%
Assessment-based Intense, durable procedures
Targeted Group Interventions Some Students (at risk)
1 5% more referrals
Students with 1 referral
10-15%Some Students (at-risk) High Efficiency Rapid Response
80-90% 90% Universal Interventions
All Settings All Students, Preventive, proactive
Students with 0 referrals
Triangle of Student Referrals:April 2006
Th Actual dataTheoryIntensive, Individual InterventionsIndividual Students 1 5%
Students with 2 or more referrals
3%
10 15%
4%Assessment-basedIntense, durable procedures
Targeted Group InterventionsSome Students (at risk)
1-5% more referrals
Students with 1 referral
10-15%
93%
Some Students (at-risk)High EfficiencyRapid Response
U i l I t tiStudents with
80-90%93%Universal Interventions
All SettingsAll Students, Preventive, proactive
0 referrals
Cost-Benefit AnalysisyCO ST/BEN EFIT A N A LYSIS W O RK SH EET
School nam e
Enter in fo be low
R obert M oton
2640
12201420
6601000
1500
2000
2500
3000
N um ber of referrals N ovem ber 2005
132
E lem entary School
660305 355
0
500
1000
Last
Yea
r
This
Yea
r
Tim
eR
egai
ned
N um ber of referrals April 2006 61
T
66
Average # Average # of
Average # o f m inutes student is out o f class due to referral
223
11
2
3
4
5
Average # o f m inutes staff need to process referral
5
1
0
1
Last
Yea
r
This
Yea
r
CCPS: Cost Benefit of ExemplarCCPS: Cost Benefit of Exemplar Schools
Office Referral Reduction Across
4 PBIS schools= 2514 PBIS schools= 251If one Office Referral=15 minutes of administrator time,
then 251 x 15=3,765 minutes
63 hours or
8 d8 daysof administrator time recovered and reinvested.
CCPS Exemplar Schools 2008
It’s not just about behavior!j
STUDENT
Good Teaching Behavior Management
STUDENT ACHIEVEMENT
Good Teaching Behavior Management
Increasing District & State Competency and Capacity
Investing in Outcomes, Data, Practices, and Systems
How do we link behavior and o do e be a o a dacademic data?
Questions to ask:Wh i f i d d? What information do you need?
What types of data do you currently use?How often? Is it enough?
Wh t ld k it b tt ? What would make it better? What are your goals when you leave to return to
b ildi ?your building?
Tertiary Prevention:specialized & individualized
i f d i h
RTI:3-Tiered
strategies for students with continued failure~5% Prevention Model
Secondary Prevention:~15% supplementary strategies
for students who do not respond to primary
Primary Prevention:h l id l idschool-wide or class-wide
systems for all students and staff
~80% of Students
Current FocusComparing academic and behavior data
B l
Classroom Performance:
State-Wide Assessment:
Discipline:
1-5%Below
grade level 6+ referrals1-5%Basic
d li 5-10% 5-10%Approaching grade level
2-5 referralsBorderline
80-90% 80-90%On or above
grade level0-1 referral
Proficient or
Advanced
Third Grade Data
Comparing academic and behavior data
Academic: MSA Reading
6+ referralsDiscipline:
3%
6%
6+ referrals(5 students)
2-5 referrals
1-5%6% Basic
(3 students)
0 1 referral70% proficient24% Adv
93% 0-1 referral(82 students)94%
Proficient or 24% Adv
Advanced
D fi itiDefinitions
Behavior Math Reading
R dRed 6+ Major/Minor Referrals </= 69% Below
Y ll 2 5 M j /Mi R f l 70 79% A hYellow 2-5 Major/Minor Referrals 70-79% Approach
Green 0 1 Major/Minor Referral 80+% On/AboveGreen 0-1 Major/Minor Referral 80+% On/Above
A d i B h i C tiAcademic-Behavior Connection
Friendship Valley Elementary
810 167
3
90%100%
815
60%70%80%90%
90 826930%
40%50%60%
0%10%20%30%
0%Behavior Math Benchmark Reading
Grade Level DataGrade Level Data4th Grade: Data from Spring 2008
Student
TeacherSub GroupAtte
ndancene Refe
rrals
Rigby LevelBenchmark
MSA ReadingMSA M
ath
Yearly A
Disciplin
e RMay M
ath B MS M
* 97 4.2 79 2 299 5.0 91 2 3100 5.0 84 3 394 28.0 69 2 29 8.0 6998 6.1 98 3 396 28.0 80 2 294 5.5 96 2 298 6 1 93 3 398 6.1 93 3 394 47 2 289 87 2 2
4th Grade Data
4% 1% 17% 22% 11% 9%
55%
20%7%
0%14%
4% 1% 17% 22% 11% 9%
80%
100%
76%92% 83%
65%
59% 55%
40%
60%
31% 36%0%
20%
ttend
ance
Discipli
ne
Rigby
Math
MSA R
MSA M
Atte D
R i l D t ll tiRegional Data collection
Combining school-based data from: 3 elementary schoolsy 1 middle school 1 high school
A regional approach to PBISA regional approach to PBIS
In elementary school the shaping of future productive In elementary school the shaping of future productive citizens begins with teaching them respect, responsibility, and re-thinking skills.
In middle school, they build on responsible and respectful behaviors and expand their focus to include building relationships with adultsinclude building relationships with adults
In high school the goals of shaping future leaders continues with a focus on preparing students to enter higher education and/or the world of workhigher education and/or the world of work
…EVERYONE plays a role in the teaching process
In the Region:
Students know what is expected of them and choose to do so because they: Know what to do Have the skills to do it See the natural benefits for acting responsiblyg p y
Adults and students have more time to: Focus on relationships Focus on classroom instruction Focus on classroom instruction
There is an instructional approach to discipline Instances of problem behavior are opportunities to learn and
ti i l b h ipractice pro social behavior
El t D tElementary Data
2008 Reading Levels-1st Quarter25
15
20
dent
s
10
15
Num
ber of
Stu
de
5
N
01 2 3 4 5
Grade Approaching Below
Academic Behavior ConnectionElementary WM region
Behavior MSA Math MSA Reading Behavior MSA Math MSA Reading
9%
6% 11% 9%
87 % 91 %Pro/adv
89 %Pro/adv o/ado/ad
East Middle School is:
Growing the GreenGrowing the GreenTheory Maryland EMS
Growing the GreenGrowing the Green
6%2%5%
Theory Maryland EMS
9%
4%
15% 9%
8780% 92% 87
87%87%
87%N=3N=123
Middle school State assessment math
MSA MATH ADVANCED AND PROFICIENT
Middle school State assessment math scores
MSA MATH ADVANCED AND PROFICIENT
300
NTS
100
200
OF
STU
DEN
60#
O 6786 61.6 54.6 77.2 83.1 89.2
2004 2005 2006 2007 2008
TOTAL7 61.6 68.4 70.5 69.7 82.7
8 51.3 63.8 61.3 65.3 73
TOTAL 174 5 186 8 209 218 1 244 9
YEAR
TOTAL 174.5 186.8 209 218.1 244.9
A Social Skills Intervention:A Social Skills Intervention: Mentoring Program
15 middle school students were selected to participateparticipate
Referrals were based on staff concernsReferrals were based on staff concerns poor attendance, decrease in grades, high ODR’s,
or teacher observations
Students were paired with high school student mentors in their regionmentors in their region
The Social SkillsThe Social Skills Improvement Scale (SSIS)
The SSIS Performance Screening Guide will be d d t t i iused as a pre- and post-mentoring experience
measure It ill f b bl b h i i f kill It will focus on observable behaviors in four skill
areas: Pro social Behaviors Pro-social Behaviors Motivation to Learn Reading Skills Reading Skills Math Skills
S i l Skill I t tiSocial Skills Instruction
Social skills deficits taken from the SSIS screening tool indicate the areas of social skill focus for the mentors.
Examples include: Listening skills
A ki f h l Asking for help Paying attention to your work
Mentoring ProgramMentoring Program Outcomes
Data will highlight the impact of mentoring on the students academic performance and behavior.
Examples of data to be collected include: student discipline records, attendance, grades, SSIS
pre and post data, student attitude surveys, teacher checklists and anecdotal comments from bothchecklists, and anecdotal comments from both mentees and mentors.
D t d T M tiData and Team Meetings
How do we assist our pupil/student services teams to use this data for decision-making?
SST/P il S i F tSST/Pupil Services Format
St d t Att dBehavior R f l
Academic P fStudent Attendance Referrals Performance
Jason Smith Missed 20 days 10 Benchmark data
S llSally Jones Missed 10 days 5 Benchmark data
Ben Miller 2 0-1 Benchmark data
T l tTemplates
Excel data template
Student Behavior Data
Cost-Benefit Analysis Worksheet
Student Services Teaming template
RResources
www.pbismaryland.org www.pbis.orgp g www.swis.org
[email protected] [email protected]
“Without data, you’re just another person with an opinion”- Unknown