Using the Three-Tiered Logic to find the Academic-Behavior ConnectionBehavior Connection

Melissa Leahy Ph DMelissa Leahy, Ph.D.Kimberly Muniz, M.A./CAS, NCSP

Carroll County Public SchoolsCarroll County Public SchoolsMaryland

Obj tiObjectives

Review why and how to use data for decision-making Provide examples of how CCPS schools use various Provide examples of how CCPS schools use various

forms of data to monitor the effectiveness of PBIS Provide an overview of a current social skills

interventionHighlight and demonstrate templates utilized to share g g p

discipline data with staff and PBS teamsHighlight and demonstrate templates to link behavior

and academic data

Carroll CountyCarroll County

Carroll County has 135,000 residentsCCPS i h 9th l h l i hCCPS is the 9th largest school system in the state 42 schools

28 000 t d t 28,000 students Rural/suburban demographics

CCPS began implementing PBS in 2001CCPS began implementing PBS in 2001 Started with 1 school

Status of PBS in CCPSStatus of PBS in CCPS

17 schools (41%) implement PBS (13 elementary, 4 middle)middle). 3 alternative programs also implement PBS

5 schools will attend new team training in July 5 schools will attend new team training in July 14 schools use SWIS for data collection

Ongoing coaching and support has been and will continue to be provided for all schools implementingcontinue to be provided for all schools implementing PBIS

Supporting Social Competence &Academic Achievement

OUTCOMES4 PBIS

El t

Supporting

Elements

SupportingStaff Behavior

pp gDecisionMaking

PRACTICESPRACTICES

SupportingStudent Behavior

D tData

IS NOT: A scary or “four letter”

IS: Powerful when used to A scary or “four letter”

word Should not intimidate us

Powerful when used to discuss discipline

Empowering when used Should not intimidate us Just numbers

Empowering when used by school teams

Reviewed frequently toReviewed frequently to determine areas of strength and weaknessg

Why Collect Discipline Data?Why Collect Discipline Data?

Decision makingWh t d i i d k ? What decisions do you make?

What data do you need to make these decisions?

Professional Accountability

Decisions made with data (information) are more likely to be (a) implemented, and (b) effective

Wh t D Y D With D t ?What Do You Do With Data?

Make Informed DecisionsLook At Trends Use Data To InformUse Data To Inform

Improving Decision MakingImproving Decision-Making

SolutionProblem SolutionFrom

To ProblemProblem

SolvingSolution

Information

F i t iFrom primary to precise

Primary statements are vague and leave us with more questions than answersmore questions than answers

Precise statements include information about 5 “Wh” questions:questions: What is the problem and how often is it happening? Where is it happening Where is it happening Who is engaging in the behavior? When is the problem most likely to occur?When is the problem most likely to occur? Why is the problem sustaining?

From primary to precise: p y pAn example

Primary statement: Precise statementTh 30 ODR f “There is too much

fighting at our school” There were 30 more ODRs for

aggression on the playground than last year, and these are most likely to occ r from 12 00 12 30 d ringto occur from 12:00-12:30 during fifth grade’s recess because there is a large number of students, and th i i l t d t ttithe aggression is related to getting access to the new playground equipment. “

St t P bl S l iSteps to Problem-Solving

Define the problem(s) Analyze the data

Define the outcomes and data sources for measuring the outcomesg Consider 2-3 options that might work Evaluate each option

Is it safe? Is it doable? Will it work? Which option will give us the smallest change for the biggest

outcome?outcome? Choose an option to try Determine the timeframe to evaluate effectiveness Evaluate effectiveness by using the data Evaluate effectiveness by using the data

Is it worth continuing? Try a different option? Re-define the problem?

Effecti e Data S stemsEffective Data Systems

The data are accurate and valid The data are very easy to collect (1% of staff time) The data are very easy to collect (1% of staff time)Data are presented in picture (graph) formatData are current (no more than 48 hours old)Data are current (no more than 48 hours old)Data are used for decision-making

The data must be available when decisions need to be made (weekly?)

Difference between data needs at a school building versus data needs for a district

The people who collect the data must see the information used for decision-making.

Interpreting Office Referral Data: Is there a problem?

Absolute level (depending on size of school) Middle High Schools (> 1 per day per 100) Middle, High Schools (> 1 per day per 100) Elementary Schools (> 1 per day per 250)

Trends Trends Peaks before breaks? Gradual increasing trend across year?g y

Compare levels to last year Improvement?p

D tData

What data do our schools collect in addition to the “Big 5”?

5 1 R ti f ti k t t f l5:1 Ratio of tickets to referrals

The research tells us that we should be giving 5 positives to each corrective responsepositives to each corrective response

How is that measured?How is that measured? Number of incentives (coupons, coins, etc.) versus the

number of referrals.number of referrals.

N b f RRR Ti k tNumber of RRR Tickets

Quarter K 1 2 3 4 5 TotalQuarter K 1 2 3 4 5 Total

One 306 289 278 236 110 193 1412

Two 678 526 423 278 147 191 2243

Overall 984 815 701 514 257 384 3655

Ratio of Tickets: Referrals

6.0

4.0

5.0

2.0

3.0

0.0

1.0

r r r y h il y e al

SeptemberOcto

ber

November

December

Janua

ryFeb

ruary

March April May

June

Total

Individual Student data:P M it iProgress Monitoring

Student Name: October 2008

94% 94%98%100%

80% 80% 80%

62%72%72%78%72%

70%

80%

90%

50%

30%

53%

40%

50%

60%

0% 0% 0% 0% 0% 0% 0% 0% 0% 0%10%

20%

30%

0%

1-Oct

2-Oct

3-Oct

6-Oct

7-Oct

8-Oct

9-Oct

10-Oct

13-Oct

14-Oct

15-Oct

16-Oct

17-Oct

20-Oct

21-Oct

22-Oct

23-Oct

24-Oct

27-Oct

28-Oct

29-Oct

30-Oct

31-Oct

Average

M thl D tMonthly Data

Student Name: Percentage of points earned

100

708090

100

42 45 46

30405060

4.810 5.7 9

0102030

0Aug/Sept October November December January February March

T i l f St d t R f lTriangle of Student Referrals

1-5%Intensive, Individual InterventionsIndividual StudentsAssessment based

6+ referrals1-5%

5-10% 5-10%

Assessment-basedHigh Intensity

Targeted Group InterventionsSome Students (at-risk)Hi h Effi i

2-5 referrals

80-90% 80-90%

High EfficiencyRapid Response

Universal InterventionsAll Students 0 1 referral80-90% 80-90%All Students Preventive, proactive

0-1 referral

Triangle of Student ReferralsTriangle of Student Referrals

1-5%

Intensive, Individual InterventionsIndividual StudentsAssessment-based

i h i6+ referrals1-5%

5-10% 5-10%

High Intensity

Targeted Group InterventionsSome Students (at-risk)High Efficiency

2-5 referrals

80 90% 80 90%

g yRapid Response

Universal InterventionsAll St d t 0 1 referral80-90% 80-90%All Students Preventive, proactive

0-1 referral

Triangle of Student Referrals2007-20082006-2007

2%2%

Students with 6 or more referrals

2006 2007

Students with 6 referrals

5% 7% Students with 2-5 referralsStudents with 2-5

referral

referrals

93% 91% Students with 0-1 referrals

Students with 93% referrals0-1 referrals

Triangle of Student Referrals:A t/S t b 2005

August/September 2005

07%

Intensive, Individual InterventionsIndividual Students A t b d 1-5%

Students with 2 or more referrals

07%

10 15%

03%

Assessment-based Intense, durable procedures

Targeted Group Interventions Some Students (at risk)

1 5% more referrals

Students with 1 referral

10-15%Some Students (at-risk) High Efficiency Rapid Response

80-90% 90% Universal Interventions

All Settings All Students, Preventive, proactive

Students with 0 referrals

Triangle of Student Referrals:April 2006

Th Actual dataTheoryIntensive, Individual InterventionsIndividual Students 1 5%

Students with 2 or more referrals

3%

10 15%

4%Assessment-basedIntense, durable procedures

Targeted Group InterventionsSome Students (at risk)

1-5% more referrals

Students with 1 referral

10-15%

93%

Some Students (at-risk)High EfficiencyRapid Response

U i l I t tiStudents with

80-90%93%Universal Interventions

All SettingsAll Students, Preventive, proactive

0 referrals

Cost-Benefit AnalysisyCO ST/BEN EFIT A N A LYSIS W O RK SH EET

School nam e

Enter in fo be low

R obert M oton

2640

12201420

6601000

1500

2000

2500

3000

N um ber of referrals N ovem ber 2005

132

E lem entary School

660305 355

0

500

1000

Last

Yea

r

This

Yea

r

Tim

eR

egai

ned

N um ber of referrals April 2006 61

T

66

Average # Average # of

Average # o f m inutes student is out o f class due to referral

223

11

2

3

4

5

Average # o f m inutes staff need to process referral

5

1

0

1

Last

Yea

r

This

Yea

r

CCPS: Cost Benefit of ExemplarCCPS: Cost Benefit of Exemplar Schools

Office Referral Reduction Across

4 PBIS schools= 2514 PBIS schools= 251If one Office Referral=15 minutes of administrator time,

then 251 x 15=3,765 minutes

63 hours or

8 d8 daysof administrator time recovered and reinvested.

CCPS Exemplar Schools 2008

It’s not just about behavior!j

STUDENT

Good Teaching Behavior Management

STUDENT ACHIEVEMENT

Good Teaching Behavior Management

Increasing District & State Competency and Capacity

Investing in Outcomes, Data, Practices, and Systems

How do we link behavior and o do e be a o a dacademic data?

Questions to ask:Wh i f i d d? What information do you need?

What types of data do you currently use?How often? Is it enough?

Wh t ld k it b tt ? What would make it better? What are your goals when you leave to return to

b ildi ?your building?

Tertiary Prevention:specialized & individualized

i f d i h

RTI:3-Tiered

strategies for students with continued failure~5% Prevention Model

Secondary Prevention:~15% supplementary strategies

for students who do not respond to primary

Primary Prevention:h l id l idschool-wide or class-wide

systems for all students and staff

~80% of Students

Current FocusComparing academic and behavior data

B l

Classroom Performance:

State-Wide Assessment:

Discipline:

1-5%Below

grade level 6+ referrals1-5%Basic

d li 5-10% 5-10%Approaching grade level

2-5 referralsBorderline

80-90% 80-90%On or above

grade level0-1 referral

Proficient or

Advanced

Third Grade Data

Comparing academic and behavior data

Academic: MSA Reading

6+ referralsDiscipline:

3%

6%

6+ referrals(5 students)

2-5 referrals

1-5%6% Basic

(3 students)

0 1 referral70% proficient24% Adv

93% 0-1 referral(82 students)94%

Proficient or 24% Adv

Advanced

D fi itiDefinitions

Behavior Math Reading

R dRed 6+ Major/Minor Referrals </= 69% Below

Y ll 2 5 M j /Mi R f l 70 79% A hYellow 2-5 Major/Minor Referrals 70-79% Approach

Green 0 1 Major/Minor Referral 80+% On/AboveGreen 0-1 Major/Minor Referral 80+% On/Above

A d i B h i C tiAcademic-Behavior Connection

Friendship Valley Elementary

810 167

3

90%100%

815

60%70%80%90%

90 826930%

40%50%60%

0%10%20%30%

0%Behavior Math Benchmark Reading

Grade Level DataGrade Level Data4th Grade: Data from Spring 2008

Student

TeacherSub GroupAtte

ndancene Refe

rrals

Rigby LevelBenchmark

MSA ReadingMSA M

ath

Yearly A

Disciplin

e RMay M

ath B MS M

* 97 4.2 79 2 299 5.0 91 2 3100 5.0 84 3 394 28.0 69 2 29 8.0 6998 6.1 98 3 396 28.0 80 2 294 5.5 96 2 298 6 1 93 3 398 6.1 93 3 394 47 2 289 87 2 2

4th Grade Data

4% 1% 17% 22% 11% 9%

55%

20%7%

0%14%

4% 1% 17% 22% 11% 9%

80%

100%

76%92% 83%

65%

59% 55%

40%

60%

31% 36%0%

20%

ttend

ance

Discipli

ne

Rigby

Math

MSA R

MSA M

Atte D

R i l D t ll tiRegional Data collection

Combining school-based data from: 3 elementary schoolsy 1 middle school 1 high school

A regional approach to PBISA regional approach to PBIS

In elementary school the shaping of future productive In elementary school the shaping of future productive citizens begins with teaching them respect, responsibility, and re-thinking skills.

In middle school, they build on responsible and respectful behaviors and expand their focus to include building relationships with adultsinclude building relationships with adults

In high school the goals of shaping future leaders continues with a focus on preparing students to enter higher education and/or the world of workhigher education and/or the world of work

…EVERYONE plays a role in the teaching process

In the Region:

Students know what is expected of them and choose to do so because they: Know what to do Have the skills to do it See the natural benefits for acting responsiblyg p y

Adults and students have more time to: Focus on relationships Focus on classroom instruction Focus on classroom instruction

There is an instructional approach to discipline Instances of problem behavior are opportunities to learn and

ti i l b h ipractice pro social behavior

El t D tElementary Data

2008 Reading Levels-1st Quarter25

15

20

dent

s

10

15

Num

ber of

Stu

de

5

N

01 2 3 4 5

Grade Approaching Below

Academic Behavior ConnectionElementary WM region

Behavior MSA Math MSA Reading Behavior MSA Math MSA Reading

9%

6% 11% 9%

87 % 91 %Pro/adv

89 %Pro/adv o/ado/ad

East Middle School is:

Growing the GreenGrowing the GreenTheory Maryland EMS

Growing the GreenGrowing the Green

6%2%5%

Theory Maryland EMS

9%

4%

15% 9%

8780% 92% 87

87%87%

87%N=3N=123

Middle school State assessment math

MSA MATH ADVANCED AND PROFICIENT

Middle school State assessment math scores

MSA MATH ADVANCED AND PROFICIENT

300

NTS

100

200

OF

STU

DEN

60#

O 6786 61.6 54.6 77.2 83.1 89.2

2004 2005 2006 2007 2008

TOTAL7 61.6 68.4 70.5 69.7 82.7

8 51.3 63.8 61.3 65.3 73

TOTAL 174 5 186 8 209 218 1 244 9

YEAR

TOTAL 174.5 186.8 209 218.1 244.9

A Social Skills Intervention:A Social Skills Intervention: Mentoring Program

15 middle school students were selected to participateparticipate

Referrals were based on staff concernsReferrals were based on staff concerns poor attendance, decrease in grades, high ODR’s,

or teacher observations

Students were paired with high school student mentors in their regionmentors in their region

The Social SkillsThe Social Skills Improvement Scale (SSIS)

The SSIS Performance Screening Guide will be d d t t i iused as a pre- and post-mentoring experience

measure It ill f b bl b h i i f kill It will focus on observable behaviors in four skill

areas: Pro social Behaviors Pro-social Behaviors Motivation to Learn Reading Skills Reading Skills Math Skills

S i l Skill I t tiSocial Skills Instruction

Social skills deficits taken from the SSIS screening tool indicate the areas of social skill focus for the mentors.

Examples include: Listening skills

A ki f h l Asking for help Paying attention to your work

Mentoring ProgramMentoring Program Outcomes

Data will highlight the impact of mentoring on the students academic performance and behavior.

Examples of data to be collected include: student discipline records, attendance, grades, SSIS

pre and post data, student attitude surveys, teacher checklists and anecdotal comments from bothchecklists, and anecdotal comments from both mentees and mentors.

D t d T M tiData and Team Meetings

How do we assist our pupil/student services teams to use this data for decision-making?

SST/P il S i F tSST/Pupil Services Format

St d t Att dBehavior R f l

Academic P fStudent Attendance Referrals Performance

Jason Smith Missed 20 days 10 Benchmark data

S llSally Jones Missed 10 days 5 Benchmark data

Ben Miller 2 0-1 Benchmark data

T l tTemplates

Excel data template

Student Behavior Data

Cost-Benefit Analysis Worksheet

Student Services Teaming template

RResources

www.pbismaryland.org www.pbis.orgp g www.swis.org

mzleahy@k12.carr.org kamuniz@k12.carr.org

“Without data, you’re just another person with an opinion”- Unknown