Kathleen Lynne Lane, Ph.D., BCBA-D, University of Kansas

Universal Screening: A Look at Behavior Screening Tools in Tiered Systems of Support

Chicago, October 29, 2014

Kathleen Lynne Lane, Ph.D., BCBA-D, University of Kansas

Lisa Powers, Ph.D., St. Louis Special School District

Wendy Peia Oakes, Ph.D. Arizona State University

Agenda

• Comprehensive, Integrated, Three-tiered (CI3T) Models of Prevention

• The Importance of Systematic Screening• Using Screening Data ...

– implications for primary prevention efforts– implications for teachers– implications for student-based interventions at

Tier 2 and Tier 3

• Directions and Experiences from the Field

Goal: Reverse Harm Specialized Group Systems for Students At-Risk

Goal: Prevent Harm School/Classroom-Wide Systems for All Students, Staff, & Settings

Academic Behavioral Social

Comprehensive, Integrated, Three-Tier Model of Prevention (Lane, Kalberg, & Menzies, 2009)

Tertiary Prevention (Tier 3)

Secondary Prevention (Tier 2)

Primary Prevention (Tier 1)

≈

≈

≈

PBIS Framework

Validated Curricula

Lane & Oakes

Goal: Reduce HarmSpecialized Individual Systems for Students with High-Risk

Primary Intervention PlanStatement

Purpose Statement

School-Wide Expectations

1. 2. 3. *see Expectation Matrix

Area I: AcademicsResponsibilities

Students will:

Area II: BehaviorResponsibilities

Students will:

Area III: Social Skills Responsibilities

Students will:

Faculty and Staff will: Faculty and Staff will: Faculty and Staff will:

Parents will: Parents will: Parents will:

Administrators will: Administrators will: Administrators will:

Lane & Oakes 2012

Essential Components of Primary Prevention Efforts

Systematic ScreeningAcademic Behavior

Treatment Integrity

Social Validity

Measure Aug

Sept Oct Nov Dec Jan Feb March

April May

School Demographics

Student Demographic Information

Screening Measures

SRSS-IE

Student Outcome Measures - Academic

Student Outcome Measures - Behavior

Program Measures

Social Validity - PIRS

Schoolwide Evaluation Tool (SET)

CI3T Treatment Integrity

Lane & Oakes

See Lane, Menzies, Oakes, and Kalberg (2012)

WHAT SCREENING TOOLS ARE AVAILABLE?

SSBD Screening ProcessPool of Regular Classroom Students

TEACHER SCREENINGon Internalizing and Externalizing

Behavioral Dimensions

3 Highest Ranked Pupils on Externalizing and on Internalizing

Behavior Criteria

TEACHER RATINGon Critical Events Index and Combined

Frequency Index

Exceed Normative Criteria on CEI of CFI

DIRECT OBSERVATIONof Process Selected Pupils in

Classroom and on Playground

Exceed Normative Criteria on AET and PSB

PASS GATE 1

PASS GATE 2

PASS GATE 3

Pre-referral Intervention(s)Child may be referred to Child

Study Team

(Lane & Oakes, 2012)

Externalizing

1.44%

Winter 2007

(N=60)

Winter 2008

(N=69)

Winter 2009

(N=66)

Winter 2007

(N=60)

Winter 2008

(N=69)

Winter 2009

(N=66)

0

10

20

30

40

50

60

70

80

137 7

1713

6

47 62 5943

56

60

Nominated But Did Not Exceed Criteria

Exceeded Norma-tive Criteria

Screening Time Point

Num

ber

of S

tude

nts

InternalizingExternalizing

6.18% 3.50% 3.18% 8.90% 6.50% 2.73%% computed based on

total # students screened

Source. Lane, Menzies, Oakes, & Kalberg, 20120. Figure 2.2 WES Elementary Systematic Screening for Behavior Disorders (SSBD; Walker & Severson, 1992) results comparing the percentage of students nominated and exceeding normative criteria for both

externalizing and internalizing behavior disorders over a three year period.

SSBD Results – Winter 2007 through Winter 2009Risk Status of Nominated Students

Student Risk Screening Scale

(SRSS; Drummond, 1994)

Student Risk Screening Scale(Drummond, 1994)

The SRSS is 7-item mass screener used to identify students who are at risk for antisocial behavior.

Uses 4-point Likert-type scale never = 0, occasionally = 1, sometimes = 2, frequently = 3

Teachers evaluate each student on the following items- Steal - Low Academic Achievement- Lie, Cheat, Sneak - Negative Attitude- Behavior Problem - Aggressive Behavior- Peer Rejection

Student Risk is divided into 3 categoriesLow 0 – 3Moderate 4 – 8High 9 – 21 (SRSS; Drummond, 1994)

Student Risk Screening Scale(Drummond, 1994)

Lane & Oakes

Student Risk Screening ScaleMiddle School Fall 2004 - Fall 2011

Fall Screeners

n = 12

n = 20

n = 507

Per

cent

age

of S

tude

nts

N=534

N=502

N=454

N=476

N=477

N=470

N=524

N= 539

Lane & Oakes

Variable Risk

Low(n = 422)M (SD)

Moderate(n = 51)M (SD)

High(n = 12)M (SD)

Significance Testing

ODR 1.50 (2.85)

5.02 (5.32)

8.42 (7.01)

L<M<H

In-School Suspensions

0.08 (0.38)

0.35 (1.04)

1.71 (2.26)

L<M<H

GPA 3.35 (0.52)

2.63 (0.65)

2.32 (0.59)

L>M, HM=H

Course Failures 0.68 (1.50)

2.78 (3.46)

4.17 (3.49)

L<M, HM=H

SAMPLE DATA: SRSSMiddle School Study 1: Behavioral & Academic Characteristics of SRSS Risk Groups

(Lane, Parks, Kalberg, & Carter, 2007)

Lane & Oakes

Convergent Validity: SRSS-E7, SRSS-I5, & SRSS-IE12 with the SSBDTarget as Measured by the SSBD

Student Condition According to the

SSBD

SRSS-IE Comparison

ROC

With Condition

N

Without the

Condition N

Area Under

the Curve (AUC)

Internalizing 21 1026 SRSS-I5 .849 SRSS-IE12 .818 Externalizing 51 1026 SRSS-E7 .952 SRSS-IE12 .921

Lane, K. L., Oakes, W. P., Harris, P. J., Menzies, H. M., Cox, M. L., & Lambert, W. (2012) Initial evidence for the reliability and validity of the Student Risk Screening Scale for internalizing and externalizing behaviors at the elementary level. Behavioral Disorders, 37, 99-122.

Note. SSBD refers to the Systematic Screening for Behavior Disorders (Walker & Severson, 1992). SRSS-IE5 refers to the version with 5 times retained. SRSS-IE12 refers to the original 7 items from the SRSS developed by Drummond (1994) combined with the new five items constituting the SRSS-IE5. The SRSS-E7 refers to the original 7 items constituting the SRSS.

STUDENT RISK SCREENING SCALE-IETEACHER NAME

0 = Never

Steal

Lie, Cheat, Sneak

Behavior Problem

Peer Rejection

Low Academic Achievement

Negative Attitude

Aggressive Behavior

Emotionally Flat

Shy; Withdrawn

Sad; Depressed

Anxious

Obsessive-Compulsive Behavior

Lonely

Self-Inflicts Pain

1 = Occasionally

2 = Sometimes

3 = Frequently

Use the above scale to rate each item for each

student.

Student Name

(Lane, Oakes, Harris, Menzies, Cox, & Lambert, 2012)

Original SRSS-IE 1412 items retained for use at the elementary level14 items under development in middle and high schools

How do we score and interpret the SRSS-IE at the Elementary Level?

1. All scores will be automatically calculated.

2. SRSS scores are the sum of items 1 – 7 (range 0 – 21)

3. Internalizing scores are the sum of items 8-12 (range 0-15)

EXAMINING YOUR SCREENING DATA …

… implications for primary prevention efforts

… implications for teachers

… implications for student-based interventions

See Lane, Menzies, Bruhn, and Crnobori (2011)

Social Skills Improvement System – Performance Screening GuideSpring 2012 – Total School

Reading Skills Math Skills Prosocial Behavior

Motivation to Learn

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

43.35 47.96 56.1255.42

45.60 47.55 36.73 38.24

11.04 4.49 7.14 6.34Adequate progress Moderate DifficultiesSignificant Difficulties

Subscales

Perc

ent

of

Stu

dents

N = 54

N = 223

N = 212

n = 489 n = 490 n = 490 n = 489

N = 22

N = 233

N = 235

N = 35

N = 180

N = 275

N = 31

N = 187

N = 271

Student Risk Screening ScaleMiddle School Fall 2004 - Fall 2011

Fall Screeners

n = 12

n = 20

n = 507

Per

cent

age

of S

tude

nts

N=534

N=502

N=454

N=476

N=477

N=470

N=524

N= 539

Lane & Oakes

Examining Academic and Behavioral Data Elementary Level

Examining Academic and Behavioral Data Middle and High School Levels

Comprehensive, Integrative,Three-tiered (CI3T) Models of Support

Assess, Design, Implement, and

Evaluate

Basic Classroom ManagementEffective Instruction

Low Intensity Strategies

Behavior Contracts Self-Monitoring

- -Functional Assessment-Based

Interventions

Schoolwide Positive Behavior Support


Higher Intensity Strategies

Assessment

Low-Intensity Strategies for Academics and Behavior

Active Supervision

ProximityPacing

Appropriate use of Praise

Opportunities to Respond

Instructive Feedback

Incorporating Choice

Self-AssessmentHow am I doing with … basic classroom management strategies? Instructional considerations? Low-intensity strategies?

Consider a book study … Build school site capacity

Active Supervisio

n

Behavior Specific Praise

Increased OTRs

Choice









≈

≈

≈

PBIS Framework

Validated Curricula


Assess, Design, Implement, and

Evaluate





Interventions




Assessment

BASC2 – Behavior and Emotional Screening ScaleSpring 2012

Total Sixth Seventh Eighth0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

85.42 87.67 82.18 86.21

10.74 8.68 12.38 11.33

3.85 3.65 5.45 2.46Normal Elevated Extremely Elevated

Subgroup

Perc

en

t of

Stu

den

ts

N = 24

N = 67

N = 533

N = 624 n = 219 n = 202 n = 203

A Step-by-Step ProcessStep 1: Construct your assessment schedule

Step 2: Identify your secondary supportsExisting and new interventions

Step 3: Determine entry criteriaNomination, academic failure, behavior screening scores,

attendance data etc.

Step 4: Identify outcome measuresPre- and posttests, CBM, office discipline data, GPA etc.

Step 5: Identify exit criteriaReduction of discipline contacts, academic success, reduction of

truancies and absences etc.

Step 6: Consider additional needs

Intervention Grids

Procedures for Monitoring: Assessment Schedule

Aug Sept Oct Nov Dec Jan Feb Mar Apr May

School Demographics

Student Demographics X X X X X X X X X X

Student Outcome Academic Measures

Benchmarking - AIMSweb X X X

Report Card Course Failures X X X X

Student Outcome Behavior Measures

Screener - SRSS X X X

Discipline: ODR X X X X

Attendance (Tardies/ Unexcused Absences) X X X

Referrals

SPED and Support-TEAM X X X

Program Measures

Social Validity (PIRS) X X X

Schoolwide Evaluation Tool X

CI3T Treatment Integrity X

Looking at Data …Expanding Your Tool Kit

• What data do you already collect?

• What are the cut scores for each screening tool?

• Remember … It is a just a screener.

Examining Academic and Behavioral Data Elementary Level

Sample Secondary Intervention GridSuppor

tDescription

Schoolwide Data: Entry

Criteria

Data to Monitor Progress

Exit Criteria

Behavior Contract

A written agreement between two parties used to specify the contingent relationship between the completion of a behavior and access to or delivery of a specific reward. Contract may involve administrator, teacher, parent, and student.

Behavior: SRSS - mod to high riskAcademic: 2 or more missing assignments with in a grading period

Work completion, or other behavior addressed in contractTreatment IntegritySocial Validity

Successful Completion of behavior contract

Self-monitoring

Students will monitor and record their academic production (completion/ accuracy) and on-task behavior each day.

Students who score in the abnormal range for H and CP on the SDQ; course failure or at risk on CBM

Work completion and accuracy in the academic area of concern; passing gradesTreatment IntegritySocial Validity

Passing grade on the report card in the academic area of concern

Sample Secondary Intervention Grid

Lane, Kalberg, & Menzies (2009). pp. 131 - 137, Boxes 6.1 - 6.4









≈

≈

≈

PBIS Framework

Validated Curricula


Assess, Design, Implement, andEvaluate





Interventions




Assessment

Changes in Harry’s Behavior

4/27 4/28 4/29 4/30 5/5 5/10 5/13 5/14 5/17 5/18 5/19 5/20 5/21 5/24 5/25 5/26 5/27 5/280

10

20

30

40

50

60

70

80

90

100

Date of Session

Per

cen

tage

of

AE

T

Baseline 1 Baseline 2

Intervention 2

Intervention 1

Cox, M., Griffin, M. M., Hall, R., Oakes, W. P., & Lane, K. L. (2012). Using a functional assessment-based intervention to increase academic engaged time in an inclusive middle school setting. Beyond Behavior, 2, 44 – 54.

A LOOK TO THE FIELD…

A Statewide Partnership

The University of Kansas

Professional Development Learning Center

STL CI3T Training 2014-2015 - 41

LPSD MS HS CI3T Training 2014-2015 - 42


Goal: Prevent HarmSchool/Classroom-Wide Systems for All Students, Staff, & Settings


Comprehensive, Integrated, Three-Tiered Model of Prevention (Lane, Kalberg, & Menzies, 2009)



≈

≈

≈

PBIS Framework

Validated Curricula


Positive

Action


St. Louis CI3T Training

CI3T Training Series

1:Two-Hour After School

2: Full Day

3:Two-HourAfter School

4: Full Day 5:Two-HourAfter School

6: Full Day 0

11/13/14 12/12/14 1/14/15 2/25/15 4/7/15

5/6/15

Your school has selected a TEAM to attend the training this year. Only they are asked to attend.


Session 1: 2 hr

MTSS: CI3T

Model: An

Overview

Session 2: full day

Building the Prima

ry Prevention Plan

Session 3: 2 hr

How to

Monitor the Plan

Session 4: Full Day

Building

Tier 2 Suppo

rts

Session 5: 2 hr

Building

Tier 3 Suppo

rts

Session 6: Full Day

Prepare to Implement

HW

Share Screeners; Complete Assessment Schedule

HW

HW

Share revised MTSS: CI3T plan; Complete MTSS: CI3T Feedback Form

HW


To contribute important information to your school’s TEAM as they attend training and develop

your school’s CI3T Plan___________________________________

We invite your participation….

Specifically, TODAY– SESSS: Schoolwide Expectations

Survey for Specific Settings. Share your opinions about student behaviorsimportant for success at your school (15 min)

– Demo: Tell us about yourself – Complete the brief confidential demographic information form (5 min)


And….• Provide your opinion on the developing plan in

the SPRING– Primary Intervention Rating Scale (10 min)

Complete a confidential survey giving your opinions on the first complete draft of the plan

– Comprehensive Three-Tiered Prevention Plan Feedback form (10 min)

Complete a short feedback form on the revised and completed CI3T Plan

**You will receive e-mail links to these surveys**STL CI3T Training 2014-2015 -

46

CI3T: Ticket Examples

CI3T: Prima

ry Prevention

Session 1: Overview of CI3T Prevention ModelsSetting a PurposeEstablish team meetings and rolesSession 2:Mission and PurposeEstablish Roles and ResponsibilitiesProcedures for TeachingProcedures for ReinforcingReactive PlanSession 3:Procedures for MonitoringSession 4: Revise Primary Plan using Stakeholder feedbackPrepare presentation

CI3T: Secon

dary Prevention

Session 5:Overview of Teacher focused StrategiesOverview of Student Focused StrategiesUsing data to determineDraft the Secondary Intervention Grid based on existing supports

CI3T: Tertia

ry Prevention

Session 6:Final revisions of CI3T Plan based on stakeholder feedbackDraft Tertiary Prevention Intervention GridsDesign Implementation Manual and Plan for roll out to faculty, students, and parents

MTSS: CI3T Training Series

Additional Professional

Development on Specific Topics

Core Content Curriculum

Teacher Drive Supports: Instructional Techniques to Improve Students’ Motivation; General Classroom Management

Practices; Low Intensity Behavior Supports

Functional Assessment-based Interventions

Reading, Math, Writing Benchmarking and

Progress Monitoring Tools

Student Driven Interventions, Strategies, &

Practices

Check In - Check Out

Additional Tier 3 Supports

CI3

T T

eam

Tra

inin

g S

eque

nce

Implementation

Stages of Tier 2

and 3 within CI3T

Behavior Screening

Tools

Using School-wide

Data to Identify

Students for Tier 2 and

Tier 3 Supports

Using Instructional Techniques to Improve Students'

Motivation

Using Simple Strategies to

Improve Classroom Behavior

Using Self-Monitoring

Strategies to Improve

Academic Performance

Professional Development: A Collaborative Effort to Empower Public School Systems

Project Empower

September 12

October 7

November 21

January 30

March 5

Five 2-hour sessions held after school: 5-7pm

www.ksdetasn.org (Go to Calendar and Search Project Empower)

http://www.ksdetasn.org/

Recommendationsto Consider

Recommendation #1: Build Stakeholders’ Expertise

Recommendation #2: Develop the Structures to Sustain and Improve Practices

Recommendation #3: Conduct Screenings in a Responsible Fashion

Recommendation #4: Consider Legal Implications- know your state laws

(Lane & Oakes, 2012)

Moving Forward

Questions:

[email protected]@ssdmo.org

Thank

you!

mailto:[email protected]

mailto:[email protected]

Designing, Implementing, and Evaluating Comprehensive, Integrated, Three-Tiered (CI3T) Models of Prevention: Step by Step Guide (2014). A special issue of Preventing School Failure.

http://www.tandfonline.com/toc/vpsf20/current#.U4zbm6ROVD8

Lane, K. L., Menzies, H. M, Oakes, W. P., & Kalberg, J. R. (2012). Systematic screenings of behavior to support instruction: From preschool to high school. New York, NY: Guilford Press.

Recommended Resources



Lane, K. L., Menzies, H. M., Bruhn, A.L., & Crnobori, M. (2011). Managing Challenging Behaviors in Schools: Research-Based Strategies That Work. New York, NY: Guilford Press.

Lane, K. L., Kalberg, J. R. & Menzies, H. M. (2009). Developing Schoolwide Programs to Prevent and Manage Problem Behaviors: A Step-by-Step Approach. New York, NY: Guilford Press.

Recommended Resources

Kathleen Lynne Lane, Ph.D., BCBA-D, University of Kansas

Documents

Transcript of Kathleen Lynne Lane, Ph.D., BCBA-D, University of Kansas