AN EVALUATION OF SOCIAL SKILL INTERVENTION EFFECTS … · AN EVALUATION OF SOCIAL SKILL...

i

AN EVALUATION OF SOCIAL SKILL

INTERVENTION EFFECTS USING THE

SKILLSTREAMING CURRICULUM

By Katie Kolbe-Holden

BS Pennsylvania State University, 2005

MEd California University, 2008

A DISSERTATION

Submitted in Partial Fulfillment of the

Requirement for the Degree of

Doctor of Psychology

(in School Psychology)

The University of Southern Maine

June, 2017

Advisory Committee:

Rachel Brown, Ph.D., NCSP, Associate Professor, Emerita, Adviser

Christina Flanders, Psy.D., NCSP, Assistant Professor, Plymouth State University

Jamie Pratt, Psy.D., BCBA-D, Assistant Professor, University of Southern Maine

ProQuest Number:

All rights reserved

INFORMATION TO ALL USERSThe quality of this reproduction is dependent upon the quality of the copy submitted.

In the unlikely event that the author did not send a complete manuscriptand there are missing pages, these will be noted. Also, if material had to be removed,

a note will indicate the deletion.

ProQuest

Published by ProQuest LLC ( ). Copyright of the Dissertation is held by the Author.

All rights reserved.This work is protected against unauthorized copying under Title 17, United States Code

Microform Edition © ProQuest LLC.

ProQuest LLC.789 East Eisenhower Parkway

P.O. Box 1346Ann Arbor, MI 48106 - 1346

10607007

10607007

2017

ii

Copyright © 2017 Katie Kolbe-Holden

All Rights Reserved.

iii

LIBRARY RIGHTS STATEMENT

In presenting this dissertation, AN EVALUATION OF SOCIAL SKILL MAINTENANCE

USING THE SKILLSTREAMING CURRICULUM, in partial fulfillment of the requirements

for an advanced degree at the University of Southern Maine, I agree that the Library

shall make it freely available for inspection. I further agree that permission for copying,

as provided for by the Copyright Law of the United States (Title 17, U.S. Code), of this

Dissertation for scholarly purposes may be granted. It is understood that any copying or

publications of this Dissertation for financial gain shall not be allowed without my

written permission. I hereby grant permission to the University of Southern Maine

Library to use my Dissertation for Scholarly purposes.

Signature: Katie Kolbe-Holden Date: _______6/12/2017________

Katie Kolbe-Holden

iv

AN EVALUATION OF SOCIAL SKILL INTERVENTION EFFECTS

USING THE SKILLSTREAMING CURRICULUM,

Katie Kolbe-Holden, M. Ed.

Dissertation Advisor: Rachel Brown

An Abstract of the Dissertation Submitted in Partial Fulfillment of the Requirements for

the Degree of Doctor of Psychology (in School Psychology)

University of Southern Maine

May 2017

Social skill deficits have been associated with not only social problems, but also

academic underachievement and further mental health problems. This study

investigated whether social skills training using the Skillstreaming curriculum was

associated with generalization of the instructed skills in other environments in the same

school. In addition, the study evaluated whether reductions in office discipline

referrals (ODRs), and increases in prosocial skills were observed on program-specific

(e.g., Skillstreaming) and the Social, Academic, and Emotional Behavior Screening

Scale (SAEBRS). This study was developed to be part of the school’s multi-tier

system of support (MTSS) for students with academic and behavior difficulties. An

MTSS includes increasingly intensive instruction and intervention for students whose

universal screening data indicate a need for assistance in order to meet school district

or state learning goals.

v

Selected fifth graders attending an elementary school in the Northeast were

selected for this study based on their scores on the Social, Academic, and Emotional

Behavior Screening Scale (SAEBRS; FastBridge Learning, 2017). The students’

scores were confirmed by their classroom teachers as indicative of a need for Tier 2

social skills intervention. Those nominated students whose parents provided informed

consent and who themselves gave assent were enrolled in the study. The participants

then completed additional assessments to confirm intervention need and to identify

specific learning goals. The participants attended regular small group intervention

sessions utilizing Skillstreaming the Elementary School Child during their lunch

periods.

Intervention outcomes were evaluated using multiple direct observations of the

students’ behaviors across diverse settings, ODR counts for each participant, pre- and

post-intervention administrations of the Skillstreaming program checklist, and a post-

intervention administration of the SAEBRS. Social skill measures and ODR counts

will be conducted pre and post intervention. Results indicated that the intervention did

not result in meaningful changes in the participants’ social skills across the observed

settings. Nonetheless, teacher ratings from pre- and post-intervention indicated that

the students’ social skills did improve over the course of the intervention. The results

are discussed in relation to the challenges related to behavior intervention and

expectancy effects.

vi

ACKNOWLEDGMENTS

First, thank you to Dr. Rachel Brown for assisting me along the way with this

dissertation, continued support through the Psy. D. program and for guiding me through

my last couple of years of this program even with my major life change, my son Ellis.

Thank you to my committee members, Dr. Christina Flanders and Dr. Jamie Pratt, for

participating in this project and all of the reading that it entails. Also, thank you to

Christina for providing me with supervision, skills, and encouragement early in the

program.

I would also like to express gratitude to my internship supervisor Dr. Laurie

Brodeur for her wealth of knowledge and skills she was able to pass along to me. Also,

for her understanding throughout the school year as I navigated full time work with an

infant. Thank you to the school, colleagues, and the students who participated to make

this project possible.

I am grateful to my husband Ryan for supporting through my many years of

schooling and to my son Ellis for learning to sleep through the night at an early age,

which allowed me to complete all of this work! Last, thank you to my mom, my dad, and

mother-in-law for helping us get through all of the life changes that we have encountered

in a short few years.

vii

TABLE OF CONTENTS

LIST OF TABLES ........................................................................................................... viii

LIST OF FIGURES ........................................................................................................... ix

CHAPTER 1: INTRODUCTION & LITERATURE REVIEW ....................................... 10 Implications of Underdeveloped Social Skills .............................................................. 11 Evidence-Based Interventions for Social Skills ............................................................ 14

CHAPTER 2: METHOD .................................................................................................. 20 Participants .................................................................................................................... 20 Setting ........................................................................................................................... 20 Materials ........................................................................................................................ 20

Dependent measures ................................................................................................. 21 Procedures ..................................................................................................................... 23

CHAPTER 3: RESULTS .................................................................................................. 27 Treatment Integrity ....................................................................................................... 27 Students’ Weekly Self-Ratings ..................................................................................... 28 SkillStreaming Checklist Ratings .................................................................................. 29 Direct Behavior Ratings ................................................................................................ 29 Office Discipline Referrals ........................................................................................... 34 SAEBRS Scores ............................................................................................................ 35 Social Validity Results .................................................................................................. 36

CHAPTER 4: DISCUSSION ............................................................................................ 37 Limitations and Future Research .................................................................................. 42

CHAPTER 5: SUMMARY ............................................................................................... 46

REFERENCES .................................................................................................................. 47

Appendix A: Skillstreaming Checklists ............................................................................ 55

Appendix B: SAEBRS (Teacher) Items ............................................................................ 75

Appendix C: Sample DBR Items ...................................................................................... 76

Appendix D: ODR Tracking Form ................................................................................... 77

Appendix E: Social Validity Scale for Teachers and Students ......................................... 78

Appendix F: Treatment Integrity Checklist ...................................................................... 79

Appendix G: Sample Skillstreaming Schedule and Lesson .............................................. 82

Appendix H: Group Self-Rating Report ........................................................................... 86

BIOGRAPHY OF THE AUTHOR ................................................................................... 87

viii

LIST OF TABLES

Table 1. Tiered Approach to Social Skill Training………………..………………….…15

Table 2. Social Outcome Intervention and Effect Sizes……………………………..…..16

Table 3. Participants’ Sex, Age, Grade, and Behavior Needs……….…………………..27

Table 4. Intervention Attendance, Goals, and Actual Attendance……………….………28

Table 5. Student and Teacher Pre- and Post- Intervention Skill Rating…………………30

ix

LIST OF FIGURES

Figure 1. Student Self-Assessment of Skills…………………………………..…………28

Figure 2. DBR Ratings…………………………………………………………………..33

Figure 3. Office Discipline Referrals (ODR) by Month……………………………...….34

Figure 4. Overall SAEBRS Score and Social SAEBRS Scores……………………..…..35

10

CHAPTER 1: INTRODUCTION & LITERATURE REVIEW

Children's peer relationships have a significant effect on childhood experiences.

In fact, some researchers suggest that later psychosocial adjustment depends on these

relationships (La Greca & Santogrossi, 1980). Children and adults perceive those who

are not able to successfully negotiate peer relationships as different, which can lead to

psychosocial maladjustment. Students who are perceived as different from the

majority can become targets or victims of violence (Dwyer, Osher & Warger, 1998).

Psychologists and other mental health professionals have examined what could prevent

students from lashing out in violence. It has been hypothesized that social emotional

learning through social skill instruction can increase pro-social skills (Taub, 2001) thus

decreasing aggressive behaviors that are often seen in children who need remediation

in social cognitive skills (August, Egan, Realmuto & Hektner, 2003).

Many methods have been employed to teach children prosocial skills. Some of

those methods include reinforcement of appropriate behaviors, social skill training

programs, utilizing peers as part of teaching (Sasso, Melloy & Kavale, 1990) and

social emotional curricula. Social skills training and social emotional curricula teach

pro-social replacement behaviors for the inappropriate behaviors a child might be

displaying that interfere with his or her daily functioning in social situations (Nedim

Bal, 2015). Social emotional curricula provide primary prevention programs to teach

prosocial skills to students. One program that has shown promising effects is called

Second Step. Second Step is a Tier I, dual language (English and Spanish),

intervention to reduce aggression and promote social competencies (Frey, Miriam,

Hirschstein & Guzzo, 2000). Another program utilized as primary prevention to teach

prosocial skills is called Strong Kids by Kenneth Merrell (2007). This program is a

11

series of 12 lessons that focus on teaching social-emotional skills, coping strategies

and replacement behaviors for inappropriate behaviors (Gueldner & Merrell, 2011). A

third such program, the Quest for the Golden Rule, is a computer- based program that

coaches and facilitates social skill learning to navigate bullying situations (Rubin-

Vaughan, Pepler, Brown & Craig, 2011). Consistent with the response to intervention

system, when primary prevention programs are in place it is hypothesized that 80% of

the students will respond at the primary prevention level (Abbott & Wills, 2012).

Students who do not respond adequately may be included in a social skills training

program. Social skills training programs utilize modeling, instruction, rehearsal and

feedback to help students improve target social skills (Sasso et al., 1990).

Implications of Underdeveloped Social Skills

Children’s social skills develop in social environments through interactions

with peers and adults. Children with underdeveloped social skills often display mental

health problems such as internalizing or externalizing disorders (Albrecht, Mathur,

Jones & Alazemi, 2015). Internalizing disorders can include anxiety and depression.

Externalizing disorders include aggressive behaviors such as bullying, frequent

tantrums, and physical aggression. These children tend to have co-occurring disorders,

and as they grow older substance abuse can be a concern (Nedim Bal, 2015).

Students with underdeveloped social skills also are likely to experience academic

difficulties. Kavale and Mostert (2004) reported that social skill deficits were observed in

75% of students diagnosed with Specific Learning Disability (SLD). Stone and La Greca

(1990) found that peers often rated children with learning disabilities as having a low

social status. Children with underdeveloped social skills and difficulty with academics

tend to be viewed as non-conforming to school social norms (Stone & La Greca, 1990).

12

Conversely, children who conform with school social expectations perform better

academically (Wilson & Shulha, 1995). Children with positive social skills such as

cooperation and following instructions tend to have more positive interactions with

teachers and do better academically than peers who are lacking in these skills (Milsom &

Glanville, 2010). Such findings suggest that prosocial skills have positive effects on

academics.

Office discipline referrals (ODRs) are used in some schools to track children’s

behaviors. ODR data can shed light on which students are sent to the office and for

what reasons. Analysis of ODR data suggests that teachers often view students with

social skill deficits as “uncooperative and complicated” in the learning environment

(Wight & Chapparo, 2008, p. 257), which can lead teachers to send such students to

the principal's or counselor's office. When multiple office referrals have occurred, the

next step is often another referral for a special education evaluation. Not all such

referrals result in special education placement, but the process consumes many hours

of time that could be spent on instruction.

In order to consider alternatives to office or special education referrals, analysis

of the reasons for students' social skill difficulties is important. When children lack

prosocial skills to solve problems with peers they might use other skills such as

aggression (August et al., 2003). Aggression often leads to office referrals where an

administrator (e.g., principal) is likely to use more substantial discipline such as

suspension or expulsion. Notably, children who were rated by teachers and peers as

having low social skills were 12.5 times more likely to be disciplined by an

exclusionary measure, like suspension (Duran, Zhou, Frew, Kwok & Benz, 2013).

13

And, any exclusionary step (e.g., suspension or expulsion) removes the student from

instruction that could otherwise improve school performance.

A wide array of mental health disorders has been shown to be connected to

social skill deficits and peer difficulties. Attention Deficit Hyperactivity Disorder

(ADHD) is one of the disorders associated with social skills problems (American

Psychiatric Association, 2013). Children with ADHD tend to experience more peer

rejection and, even after intervention, often still have difficulty maintaining successful

relationships (Becker, Luebee & Langberg, 2012). Another disorder, Autism

Spectrum Disorder (ASD) has social skill deficits as the main component of the

disorder. Some of the deficits include lack of perspective taking, inability to imitate

others, and poor perception of nonverbal social cues. Children with ASD also have

difficulty with successful friendships (Ratcliffe, Wong, Dossetor & Hayes, 2015).

Other disorders that seem to correlate with difficulties in social situations include

anxiety (generalized and social) (White & Roberson-Nay, 2009), oppositional defiant

disorder (ODD) (Becker et al., 2012), and intellectual disabilities (Plavnick, Kaid &

MacFarland, 2015).

It is also worth noting that social skill deficits that continue later in life have

been linked to an increased risk for substance abuse (Nedim Bal, 2015). The good

news is that intervention remediating social skills has shown positive effects on

academics, behavior, and attitudes toward social situations (Whetstone, Gillmor &

Schuster, 2015). Social skills intervention is a step toward helping students with

observed social difficulties be successful. Thankfully, several evidence-based

interventions are available to improve students' social skills. Research indicates that

14

teaching social and emotional skills addresses competencies students need to navigate

relationships with others. Prior research suggests that when pro-social skills are

directly taught, aggressive behaviors may be decreased and constructive behaviors

may be increased (Frey et al., 2000).

Evidence-Based Interventions for Social Skills

Research has shown that a tiered approach to social skills instruction can be the

most effective (Albrecht et al., 2015). This approach can be utilized for academics as

well as social skills. Tier 1 is universal and provided for all students. It is considered

primary prevention and includes teaching basic skills to all students. Tier 2 builds

upon what the students learned in Tier 1, but is more intensive and explicit. Tier 3 is

the most intensive and includes instruction for those students who are not responding

to the instruction provided at Tiers 1 and 2 (Brown-Chidsey & Andren, 2013; Brown-

Chidsey & Bickford, 2016). Social skills can be addressed in all tiers of the three-

tiered approach. Table 1 shows social skill instruction approaches at each level of

tiered instruction.

Several school-based interventions have been shown through research to

improve social skills in children. The United States Department of Education’s What

Works Clearinghouse (WWC) provides information about evidenced-based

interventions for specific academic, behavioral and emotional needs. A search on the

WWC website in April 2016 found 20 interventions related to student behavior. Of

those, seven were related to social outcomes or social-emotional development for

elementary students. The specific outcomes of those seven programs were evaluated to

identify key practices most related to social skill improvement (see Table 2). Reviews

15

Table 1.

Tiered Approach to Social Skill Training

Tier Specific Program Description

Tier 1 Strong Kids (Frey et al., 2000) Second Step (Taub, 2001) Quest for the Golden Rule (RubinVaughan, 2011)

Social skills training at this level is a benefit to all students (Albrecht, et al., 2015). High quality social skill instruction is provided for all students and taught by a teacher, school counselor, school social worker or other school personnel in the classroom. Instruction includes defining, teaching and reinforcing positive behaviors (Yong & Cheney, 2013). Students are monitored for progress (Bayat, Mindes, & Covitt, 2010).

Tier 2

Behavior Education Program (Yong & Cheney, 2013). Check and Connect (Yong & Cheney, 2013). First Steps to Success (Yong & Cheney, 2013)

Skillstreaming (McGinnis, 2012)

Children may be chosen to participate in Tier 2 program if after Tier 1 instruction they are still having peer relationship problems. For social skills, students in Tier 2 may participate in small groups with a counselor, social worker, special educator or their teacher. Tier 2 should be consistently implemented procedures across 10-15% of the students and consistent with Tier 1 expectations (Yong & Cheney, 2013). Data are collected frequently, possibly on a weekly basis to determine if the more intensive intervention is effective (Bayat et al., 2010).

Tier 3 Functional Behavior Assessment with function-based interventions, (Gresham, 2015) Replacement Behavior Training (RBT) (Gresham, 2015)

Students who are not successful with Tier 2 intervention receive specialized instruction to remediate social skill deficits with frequent progress monitoring. Research and literature is lacking on how to address social skills at the Tier 3 level when other options have been exhausted (Gresham, 2015).

16

Table 2

Social Outcome Interventions and Effect Sizes

Intervention Delivery Method Grade-Levels Study Examined

Effect Sizes for Social-Related Outcomes

Coping Power Small Group 4-5 r=.21 on aggressive behavior reduction (Lochman, Wells, Qu, & Chen, 2012)

Early Risers Individual K-2 d=.40 social etiquette scores compared to control (August et al., 2003)

Fast Track: Elementary School

Individual K d=-.22 aggression scores (Conduct Problems Prevention Research, 1999)

First Steps to Success

Individual K-3 d=-ranging from .54-.87 for the intervention group (Walker, Seeley, Small, Severson, Graham, Feil, Serna, Golly, Forness, 2009)

Lovaas Model of Applied Behavior Analysis

Individual PK d=.80 imitation, .47 socialization (Sallows, & Graupner, 2005)

Social Skill Training

Whole Class PK d=.83 sharing (mean effect size of two groups, d=2.8 being in a group (mean of effect size of two groups) (Guglielmo, & Tryon, 2001).

The Incredible Years

Individual PK-2 d=range of .55-.77 related to parent/child interactions, child interactions with peers and teacher/child interactions (Webster-Stratton, Reid, & Hammond, 2004)

of Coping Power and Fast Track: Elementary School indicated that the available

research for these programs was all based on a small amount of evidence and did not

show large effects. Coping Power suggested a small effect size on reducing

aggressive social behaviors (r=.21) when compared with the control group (Lochman

et al., 2012).

Fast Track: Elementary School suggested a similarly small effect size on

reducing aggressive behavior at d=-.22 (Conduct Problems Prevention Research,

1999). The Lovaas Model of Behavior Applied Behavior Analysis produced a large

effect size, d=.80, related to imitation skills and a moderate effect size related to

17

socialization skills, d=.47, however, the sample was based on an atypical group of

young children aged 24-42 months with Autism (Sallows & Graupner, 2005).

First Step to Success, Early Risers, and The Incredible Years showed the most

promising effects on the social emotional development of students, but also were based

on a small amount of available evidence. First Step to Success involves three

components: (a) screening students for antisocial behaviors (i.e., aggression or

opposition), (b) an intervention phase, and (c) a parent/guardian training component.

Walker et al. (2009) found effect sizes ranging from .54 to .87 for the intervention

group. Although it showed promising results in reducing antisocial behaviors in young

children, the program requires a lot of resources, staffing and money (Walker,

Kavanaugh, Stiller, Golly, Severson & Feil, 1998). It is possible that some schools

might not be able to find resources to implement a program on this large of scale.

Early Risers is a multi-year program to improve self-regulation, social skills

and parent/child relationships (Bernat, August, Hektner & Bloomquist, 2007). WWC

analysis showed a medium to large amount of evidence, but had an effect size of only

d=.40 on social etiquette scores when compared to the control group (August et al.,

2003). A concern about Early Risers is the cost, which ranges from $1500-$2500 per

student (U.S. Department of Education, 2012). For schools, this intervention might not

be the best choice because most school interventions are short-term and lower cost.

The Incredible Years is a program similar to First Steps to Success in that there

is a parent-training component and it has been shown to reduce antisocial behaviors.

Effect sizes ranging from d= .55-.77 were found related to parent and child

interactions, peer interactions, and teacher and child interactions (Webster-Stratton et

18

al., 2004). The Incredible Years involves training teachers to implement a prevention

curriculum in the Tier 1 classroom (Webster-Stratton, Reid & Stoolmiller, 2008). The

last social skills program included on WWC was Social Skills Training. Social Skills

Training does not involve a packaged training; rather, it is a collection of behavior

intervention techniques like incidental teaching (U.S. Department of Education, 2013).

A study by Guglielmo & Tryon (2001) found a large effect size from Social Skills

Training on sharing skills (d=.83).

Another program social skills training program that was not listed in WWC is

Skillstreaming. This program was first published in the 1970s based on research

conducted by Goldstein and colleagues (McGinnis & Goldstein, 1997). It is a social

skills instruction package and is similar to the social skills training techniques

reviewed at the WWC. The program focuses on what behavioral steps to use in

specific social situations (Lerner & Mikami, 2012). Skillstreaming is cost effective at

about $300 for a program book, lesson book and skills cards (DE-PBS, 2015).

Programming can be as brief as two days depending on the skill (McGinnis &

Goldstein, 1997). Effect sizes for research utilizing Skillstreaming are generally in the

moderate to large range (Sheridan et al., 2011). Sheridan et al. (2011) found effect

sizes as strong as d=-2.67 for specific problem behavior reductions. Skillstreaming

can be utilized on an individual basis or in a small group (McGinnis, 2012), which fits

with Tier 2 social skills training intervention. Based on the available evidence, it

appears that Skillstreaming could be as effective as those reviewed on the WWC, and

perhaps be easier and more cost effective for use in schools.

19

In order to examine the effects of a specific social skills program on the social

behaviors of fifth graders, the following research questions were addressed in this

study:

1. Will fifth grade students who participate in small group Skillstreaming sessions

generalize skills in other structured and less structured settings as measured by

Direct Behavior Ratings?

2. Will students who participate in small group Skillstreaming sessions have

reductions in Office Discipline Referrals (ODRS) as shown in School Wide

Information System (SWIS) data?

3. Will students who participate in small group Skillstreaming sessions see

increased competency in social skills as measured by the Skillstreaming

checklist?

4. Will students who participate in small group Skillstreaming sessions see an

increase in prosocial skill self-ratings and teacher ratings as determined by the

Social, Academic, and Emotional Behavior Risk Screener (SAEBRS)?

20

CHAPTER 2: METHOD

This study utilized a multiple baseline across settings design to evaluate the

effects of the Skillstreaming program on students’ social skills.

Participants

The study consisted of four student participants. The participants in the study

included three fifth grade males and one fifth grade female. To be considered eligible

for this study, participants must have had four or more office discipline referrals and

at-risk scores on the social portion of the Social, Academic, and Emotional Behavior

Risk Screener (SAEBRS) screening assessment. Students were excluded from

consideration if it was reported that they were moving to another school, had current

involvement with the legal system, or had opted out of mental health services through

the school system. Parent permission was documented through written procedures. In

addition, all participants provided witnessed assent for participation. All data

collection was conducted following the approval of the University’s Institutional

Review Board (IRB), as well as parental permission, and student assent.

Setting

The study was carried out in a public school in the Northeast that contained

grades 3 through 5. The school had 247 students enrolled during the 2016-2017

academic year, of which, 62% were eligible for free or reduced price school lunch.

The intervention group sessions were held in the Student Support Center and cafeteria

during the students’ lunchtime with the Behavior Support Teacher as an observer for

30% of the lessons.

Materials

21

The Skillstreaming Elementary School Child, Third Edition program was used

for this study. The curriculum consisted of checklists (teacher, parent, and child),

teaching procedures for the skills, and 60 possible skills outlines. Six specific skills

were chosen based on the participants’ scores on the program’s pre-intervention

assessment. Students also completed weekly homework assignments related to the

skill for each week.

Dependent measures. The effects of the intervention were measured using

four methods. First, the Skillstreaming checklist (McGinnis, 2012) was completed by

teachers and parents prior to the group. From these data, specific skills in the

curriculum were selected for the group. Students also completed selected questions

from the Skillstreaming checklist related to the skills the group was to learn. Second,

in order to determine if the Checklist score changes were consistent with another social

skills metric, teachers completed the SAEBRS (Kilgus, Chafouleas, Riley-Tillman &

Von der Embse, 2013). A third measure utilized to evaluate whether the skills

generalized to other settings was systematic observation of the participants in

structured and unstructured settings. Structured settings consisted of the classroom

with regular education teachers or special education teachers, student support center,

and group sessions. Unstructured settings included lunch, recess, hallways, and

assemblies. The observer recorded participant behaviors using Direct Behavior

Ratings (DBR; FastBridge Learning, 2017). These scales rate the frequency of

specific student behavior from 1-10 on each observation. Finally, the effects were

measured in relation to changes in the participants’ office discipline referrals (ODRs)

utilizing School Wide Information Systems (SWIS) (Educational and Community

22

Supports, 2016). At the end of the intervention teachers and students completed a

social validity questionnaire created by the researcher.

Skillstreaming Checklist. The Skillstreaming checklists consist of 60 items on

each of three Likert scales for the parent(s), teacher, and student (McGinnis, 2012).

These are designed to be a pretest measure to assign students to Skillstreaming groups.

For the purposes of this study these checklists were also used as a posttest measure.

These measures can be found in Appendix A.

Social, Academic and Emotional Behavior Risk Screener. SAEBRS screens

students who might be at risk for social, emotional, and behavioral problems. The

teacher version is a 20-item survey of students' academic, emotional, and social skills

(Kilgus et al., 2013; see Appendix B). The SAEBRS is available in digital form

through FastBridge Learning. The SAEBRS was completed by the teachers pre-

intervention, during the intervention, and post-intervention.

Direct Behavior Ratings (DBR) are daily to weekly observations of student

behavior across settings. DBR are designed as a form of progress monitoring to be

used alongside interventions to improve behaviors (Chafouleas & RileyTillman, 2015).

DBR are available in digital form from FastBridge Learning. The DBR system is

designed to be customized to each student's specific behavior(s) of concern. Each

student's target behavior(s) were observed in structured and unstructured settings at

least two times per week to check for generalization of behavior(s) using a 10-point

Likert rating scale. An example of this measure can be found in Appendix C.

Office Discipline Referrals (ODR) in elementary school are commonly related

to peer-directed problems (Predy, McIntosh, & Frank, 2014), which can stem from

23

poor social skills. An example includes students interfering with classmates during

lessons that result in being sent to the office. ODRs were reviewed for each

participant for the entire school year when the intervention occurred to determine if

there had been a reduction in referrals. The ODR tracking sheet can be found in

Appendix D.

Social Validity Scale. In addition to the four measures of student behavior

change, a social validity questionnaire was distributed to the participating students as

well as their teachers after the intervention ended. The questionnaire includes items

related to whether the Skillstreaming intervention was easy or difficult, if it achieved

its goals, and whether respondents would recommend it for other students with similar

behavior difficulties. This measure can be found in Appendix E.

Student Self-Ratings. The participants were also asked to rate their own skills

on a weekly basis based on the skill for the week. This measure can be found in

Appendix F.

Procedures

The selection criteria utilized to choose students for participation included

recruiting students with more than three ODRs for the 2015-2016 school year. From

that group of students, students were chosen based on the lowest scores (i.e., highest

risk) on the social skills subscale of the SAEBRS that their teachers completed. Four

students were recruited and parental consent and student assent were obtained for each.

Teachers, students and parents were asked to complete the Skillstreaming checklist.

Baseline behavior rating data were obtained using DBR behavior ratings across

structured and unstructured settings at least two times per week for each student

24

(cafeteria, classroom, playground, and special class, [e.g., music]). The Skillstreaming

checklist was utilized to determine skill deficit areas and what lessons to provide

students during intervention. All student scores in each skill area were examined.

Categories were chosen based on all four students scoring an “Almost Never”

or “Seldom” in skill areas. All students began the group after the two weeks of baseline

observation. The researcher provided the intervention with an outside evaluator

checking for treatment integrity during 30% of the observation sessions (see Appendix

G). The intervention utilized the Skillstreaming lessons specific to the skills the

students needed. Based on the students’ pre-intervention data, four lessons regarding

the topic of feelings and two other lessons related to problem solving and relaxation

were planned. After two lessons related to feelings it was clear that the participants

knew this topic well and a different lesson on avoiding trouble, more pertinent to their

social issues, replaced the remaining feelings lesson. Examples of a typical group

session schedule and sample lessons can be found in Appendix H.

Progress was monitored by the DBR in one unstructured setting and one

structured setting per week per student during the intervention and for two weeks

following the intervention. Structured settings included classrooms, Skillstreaming

lunch group, and special classes; unstructured settings included hallways, the

playground, and the cafeteria. During the observations, the observer recorded a rating

for each student related to disruptive behaviors with peers and adults as well as

appropriate interactions. Disruptive behaviors included volume and tone of voice that

others found aversive, loud noises disrupting others, and interactions with peers

causing arguments or physical aggression. Appropriate social skills were defined as

25

the various skills learned in the group including showing concern for another’s feelings

through words or actions, avoiding situations with peers that are causing problems,

using skills to relax when tense or upset, and using problem solving skills. Each

student was rated on each of the two areas on a 1-10 scale with 1 being constant

disruptive behaviors or no appropriate interactions and 10 being no disruptive

behaviors or all appropriate interactions. The intervention length was 6 weeks with an

average of 2.5 points of contact between the interventionist and participants per week

for a total of 15 sessions. Each participant completed a portion of the Skillstreaming

checklist related to the planned lessons for each week of the intervention.

Prior to the intervention, students met with the researcher for a 20-minute

rapport building session. During this session, students set individual goals for group

attendance (i.e., how many sessions do you think you will attend?). If each student

met his or her own attendance goal, a group reward consisting of a celebration lunch

would be held. Students also planned a final celebration opportunity. During the

intervention phase, students met with the researcher for 20 minutes for fifteen sessions

to complete a lesson within the Skillstreaming curriculum during their lunch time.

Students ate their lunches while working in the group. They completed one lesson per

week for a total of six lessons. Each week consisted of discussion of skill steps,

modeling of the skill by the researcher, role play of the skill by the students and a

discussion of skill practice. Students received a skill card with skill steps to take to

home and were assigned “homework” to practice the skill during that week. Teachers

also received a copy of the skill card with skill steps. After being given a chance to

practice the new skills, students self-assessed skills on a Likert scale weekly. During

26

the intervention, students also had four opportunities to invite peers to a lunch table to

practice skills in the cafeteria. The participants all met their own attendance goals and

all participated in the final celebration.

At the end of the intervention the participants, parents, and teachers completed

a portion of the Skillstreaming checklist related to the skills addressed during the

intervention (Appendix A). In addition, the teachers completed the SAEBRS again at

the end of the intervention. Data were analyzed using a visual inspection of DBR

graphs from baseline, intervention, and maintenance phases. The graphed data were

analyzed in relation to the direction and trend of data points. The graphed data were

compared to each participant's score changes on the Skillstreaming checklist and

SAEBRS. Although not a pre-post group design, comparison of DBR and rating scale

changes was included for the purpose of examining the accuracy of the skill ratings in

relation to real-time observations of students in multiple school settings. In addition,

ODRs were reviewed pre- and post-intervention to determine if a reduction in ODRs

related to peer issues was a secondary outcome of improved social skills.

27

CHAPTER 3: RESULTS

Four students were recruited for the study and all four students’ parents

provided permission for the students to participate; the students provided their own

assent as well. Student demographic data related to sex, age, and overall behavior

needs are shown in Table 3.

Table 3.

Participants' sex, age, grade, and behavior skill needs

Sex Age Grade Behavior Skill Needs

M 11 5 Engages in disruptive behavior in the classroom and unstructured environments for peer attention

M 11 5 Engages in disruptive behavior in the classroom and unstructured environments for peer attention

M 10 5 Impulsive F 11 5 Often joins disruptive behaviors others are displaying

Treatment Integrity

The behavior support teacher conducted treatment integrity checks to measure

the group leader’s fidelity to the lesson plans. Treatment integrity checks occurred

randomly over seven of the intervention sessions. Scores on all integrity checks

reached 100% across all sessions.

At the start of the intervention, students selected personal goals for intervention

attendance. Attendance and goals for each of the four students are shown in Table 4.

Each student exceeded his or her personal goal.

28

Table 4.

Intervention attendance goals and actual attendance

Student Attendance Goal Attendance A 80% 93% B 73% 80% C 73% 100% D 80% 100%

Students’ Weekly Self-Ratings

Each student also rated his or her own progress each week on the skill of the

week. Figure 1 shows each student’s assessment of his or her own progress related to

how well he or she utilized the skill when encountering situations related to the skill.

This measure was meant to be a formative assessment to remind the students to be

aware of using the skills outside of the group setting. Skills were rated on a 1 through

5 Likert-type scale with 1 being “I never used the skill appropriately this week” to 5

being “I used the skill perfectly every time I encountered that situation this week.”

Figure 1. Student self-assessment of skills.

0 1 2 3 4 5 KnowingYourFeelings RecognizingAnother's…

Relaxing ProblemSolving

UsingSelfControl AvoidingTrouble

StudentA




StudentB




StudentC




StudentD

29

SkillStreaming Checklist Ratings

Teachers and students rated the target social skills prior to the intervention

beginning and at the end of the intervention group using the Skillstreaming Teacher

and Student Checklist (See Appendix A). The higher the point value, the higher

functioning the student was performing in those skills. These ratings are shown in

Table 5. All students evidenced slight improvements in teacher ratings scales in all

skills. Student self-ratings were more variable and showed minimal improvement to

no improvement in their self-assessment of skills.

Direct Behavior Ratings

Student behavior progress, as monitored by Direct Behavior Ratings (DBR) in

one unstructured setting and one structured setting per week, per student, provided

evidence of student skill changes. Structured settings included classrooms, the

intervention group, and special classes; unstructured settings included hallways, the

playground, and cafeteria. Two specific categories of behavior were observed for each

student. During the observations, the observer recorded a rating for each student

related to targeted disruptive behaviors as well as appropriate interactions with peers

and adults. Disruptive behaviors included (a) volume and tone of voice that others

found aversive, (b) making loud noises disrupting others, and (c) interactions with

peers causing arguments or physical aggression. Appropriate interactions were

defined as exhibiting the various skills learned in the intervention group including, (a)

showing concern for another’s feelings through words or actions, (b) leaving situations

with peers that start to be problematic (d) using skills to relax when tense or upset, (e)

using self-control skills, and (f) using problem solving skills.

30

Table 5. Student and teacher pre- and post-intervention skill ratings

Student Teacher Items Pre Post Pre Post

1. Knowing your feelings (recognizing ones’ own feelings)

Almost Never 0% 0% 75% 0% Seldom 25% 25% 25% 25% Sometimes 25% 25% 0% 50% Often/Almost Always 50% 50% 0% 25%

2. Recognizing Another's Feelings (being able to identify how others are feeling)


3. Using Self Control Skills (using strategies to control self)


4. Problem Solving Skills (choosing positive alternatives to aggression)


5. Relaxation Skills (can the student relax when tense/upset)


6. Avoiding Trouble (does the child avoid situations that may cause problems)

Almost Never NA 25% 50% 0% Seldom NA 25% 50% 50% Sometimes NA 25% 0% 25% Often/Almost Always NA 25% 0% 25%

Each student was rated on each of the two areas on a 1-10 scale with 1 being constant

disruptive behaviors or no appropriate interactions and 10 being no disruptive

31

behaviors or all appropriate interactions. Figure 2 shows data related to the disruptive

behavior and appropriate interactions for each student. During intervention, Student A

evidenced no identifiable change in any of the areas. It is important to note that this

student was observed by the classroom teacher and researcher to engage in

increasingly disruptive and inappropriate interactions when the observer was present.

Student B began the intervention with higher appropriate interactions and lower

disruptive behaviors than the other students. During the intervention this student

maintained appropriate interactions and limited disruptive behaviors in both settings

and evidenced no change upon visual inspection of the graph. Student C revealed

variable appropriate interactions in structured settings during all phases, but

demonstrated a drop in appropriate interactions in unstructured settings. Concurrent

with the decrease in appropriate behaviors, student C demonstrated an increase in

disruptive behaviors in unstructured settings, although disruptive behaviors were

variable in all phases. Student D's appropriate interactions were variable in both

structured and unstructured settings during all phases. Nonetheless, student D

exhibited a modest decrease in disruptive behaviors in unstructured settings by the end

of the study.

The students’ DBR data in one unstructured setting and one structured setting

per week, per student, provided some evidence of student skill changes. Structured

settings included classrooms, the intervention group, and special classes; unstructured

settings included hallways, the playground, and cafeteria. Two specific categories of

behavior were observed for each student. During the observations, the observer

recorded a rating for each student related to targeted disruptive behaviors as well as

32

appropriate interactions with peers and adults. Disruptive behaviors included (a)

volume and tone of voice that others found aversive, (b) making loud noises disrupting

others, and (c) interactions with peers causing arguments or physical aggression.

Appropriate behaviors included (a) showing concern for another’s feelings through

words or actions, (b) leaving situations with peers that start to be problematic (d) using

skills to relax when tense or upset, (e) using self-control skills, and (f) using problem

solving skills. Each student was rated on each of the two areas on a 1-10 scale with 1

being constant disruptive behaviors or no appropriate interactions and 10 being no

disruptive behaviors or all appropriate interactions.

Figure 2 shows data related to the disruptive behavior and appropriate

interactions for each student. During intervention, Student A evidenced no identifiable

change in any of the areas. It is important to note that this student was observed by the

classroom teacher and researcher to engage in increasingly disruptive and

inappropriate interactions when the observer was present. Student B began the

intervention with higher appropriate interactions and lower disruptive behaviors than

the other students. During the intervention this student maintained appropriate

interactions and limited disruptive behaviors in both settings and evidenced no change

upon visual inspection of the graph. Student C revealed variable appropriate

interactions in structured settings during all phases, but demonstrated a drop in

appropriate interactions in unstructured settings. Concurrent with the decrease in

appropriate behaviors, student C demonstrated an increase in disruptive behaviors in

unstructured settings, although disruptive behaviors were variable in all phases.

Student D's appropriate interactions were variable in both structured and unstructured

33

Figure 2. DBR Ratings.

34

settings during all phases. Nonetheless, student D exhibited a modest decrease in

disruptive behaviors in unstructured settings by the end of the study.

Office Discipline Referrals

Another dependent variable, office discipline referrals (ODR), showed the

number of times each student was sent to the principal's office during the months when

the study was conducted. The ODRs were analyzed to assess both overall trends as

well as those referrals related to issues with peers. Figure 3 shows the number of

referrals per month that each student received in all three phases. Through visual

inspection of the graph, the effects of the intervention seem to be negligible when

related to ODRs.

Figure 3. Participants’ Office Discipline Referrals (ODR) by month

0

0.5

1

1.5

2

2.5

3

3.5

4

4.5

Sep-16 Oct-16 Nov-16 Dec-16 Jan-17 Feb-17 Mar-17 Apr-17

StudentA StudentB StudentC StudentD

35

SAEBRS Scores

Teachers completed the SAEBRS for all students in the fall (prior to the

intervention), winter (during the intervention) and in the spring (after the intervention).

The participating students' SAEBRS data were assessed to determine if there were

changes that could be associated with the intervention. Figure 4 shows SAEBRS

ratings that teachers completed for the participants. Participants A, B, and D, saw an

overall increase in SAEBRS scores from baseline (fall). All participants saw an

increase in SAEBRS scores on the social portion from baseline, however, Student C’s

scores changed the least. This student’s spring SAEBRS social scale score was slightly

higher than fall, but the spring total score was lower than fall.

Figure 4. Overall Social Scale and Total SAEBRS scores

0 10 20 30 40 50 60 Spring(Total) Winter(Total)

Fall(Total)

Spring(Social) Winter(Social)

Fall(Social)

StudentA


Fall(Total)


Fall(Social)

StudentB


Fall(Total)


Fall(Social)

StudentC


Fall(Total)


Fall(Social)

StudentD

36

Social Validity Results

A social validity scale was administered to teachers and the student participants

at the end of the study. In response to the question about whether the group fit easily

into the school schedule 100% of the teacher respondents answered “Yes.” Only

75% of students said “Yes.” One participant indicated that the group was not long

enough in duration, which is why he felt that “Maybe” it fit into the school schedule.

When asked if the group achieved the goals that the individual thought the group

would achieve 50% of teachers said “Yes” and 50% said “Maybe.” Students answered

with 75% feeling that “Maybe” the group achieved goals that they thought it would

and 25% said “Yes.” The last question asked if respondents would recommend this

group for other students. All of the teachers indicated they would recommend the

group for other students. Fifty percent of students would recommend the group to

others, 25% said “Maybe” and 25% said “No” they would not recommend the group to

other students.

37

CHAPTER 4: DISCUSSION

Previous studies have shown an increase in prosocial behaviors during

intervention and shortly after the intervention through the use of Skillstreaming

(Sheridan et al., 2011). This study examined Skillstreaming as a Tier 2 small group

intervention with a social integration component for fifth graders with identified social

skill deficits. The results were mixed. Although the students appeared to make

minimal gains in relation to observed behaviors, their teachers reported improvements

in overall social skills in all but one case. Nonetheless, the general conclusion from the

observation data is that intervention showed minimal effects related to improved social

skills.

The first research question this study was seeking to answer was related to

whether students were able to display and generalize behaviors learned through small

group intervention in real world practice settings. Although students practiced

appropriate behaviors in the intervention sessions, their application in structured and

unstructured school settings showed little to no change during and after the

intervention. Indeed, some students displayed worsening disruptive behaviors by the

end of the study. The exception was subject D who displayed a slight reduction in

disruptive behavior in structured settings at the end of the study. This finding suggests

that these students did not transfer the skills learned during intervention to other

environments. The DBR data were anticipated to be consistent with score changes in

the Skillstreaming checklist and the SAEBRS data, however, the results did not show

this. Instead, the DBR data were inconsistent, while the checklist scores indicated

minimal gains and the SAEBRS data evidenced moderate gains for students.

38

The second area this study examined related to ODRs and whether students would

display a drop in ODRs if they participated in small group intervention related to social

skills. The students’ ODR numbers did not seem to be affected by the intervention.

Similar to the DBR data, this finding suggests that the students did not employ the skills

learned during intervention in their classrooms in ways that would result in fewer trips to

the principal’s office.

The third research question this study addressed was whether there would be

significant change in student and teacher ratings of behaviors according to the

Skillstreaming checklist. Both the student and teacher ratings indicated that the gains

were minimal. Interestingly, the teachers’ scores indicated larger gains from the

intervention than the student scores, suggesting that the teacher-perceived gains were

higher than student-perceived gains. Students seemed to think their skills had not

improved from the pre-intervention rating, but teachers were more likely to rate them

as having improved skills.

The last research question this study examined related to improvements in

prosocial skills as rated by the teachers through SAEBRS. Three of the four students

saw gains in their overall SAEBRS scores and on the social portion of the SAEBRS

scores. One student (C) evidenced a reduction in the overall SAEBRS score, but also

evidenced a slight improvement in the social score area. All of the students showed

improvement on the social subscale from fall to spring. However, students A and C

had winter social subscale scores that were lower than their fall ratings.

Prior research, related to social skill training, has evidenced improvements in

reduction of aggressive (Taub, 2001) and antisocial behaviors (Walker et. al, 1998) as

39

well as shown an increase in positive interactions (Milsom & Glanville, 2010). Other

studies have resulted in mixed efficacy, or shown success with students with specific

disabilities’ such as Autism, Intellectual Disability, and Schizophrenia. For example, a

study completed by Van der Stouwe, Asscher, Hoeve, Van der Laan, and Stams

(2016), found that a social skills training program designed for incarcerated

adolescents showed little to no effects related to social problem solving, but showed

small effects related to social perspective taking and impulsivity. Several studies also

support that social skills training has been effective for students with Autism. Radley,

Hanglein, and Arak (2016) found improvements in social functioning among preschool

students with Autism from baseline to maintenance as rated by teachers and parents on

a social skills checklist. Another study (Laugeson, Frankel, Mogil & Dillon, 2009)

revealed that students with Autism evidenced improvements, as indicated through

rating scales completed by parents and teachers, and maintained these skills at the 14-

week post intervention period. O’Handley, Ford, Radley, Helbig, and Wimberly

(2016) utilized behavioral skills training with participants with Intellectual Disability

and found skills were maintained at six-week follow ups reported by teacher ratings.

Individuals with Schizophrenia also evidenced increased prosocial skills as measured

by rating scales in response to a computer-assisted cognitive remediation program for

social skill training (Kurtz, Mueser, Thime, Corbera & Wexler, 2015).

The present study results differ from previous studies in that the students did

not make expected gains from this intervention. A notable difference with prior

research is that this study included typically-developing students not served through

special education. Only one student in this study had a mental health diagnosis

40

(ADHD). Also, students selected for this study were students who had higher rates of

ODRs compared to others within the school population. Prior studies where social

skill instruction has been successful have primarily involved students with Autism. As

Van der Stouwe et al. (2016) showed, social skills training may not be as effective

with students who are engaging in delinquent behaviors. Also notable in the current

study were several students who instigated inappropriate social behavior in others.

Skillstreaming delivered in a group format might not be the most effective intervention

for students with these type of needs.

It is worth noting that among the studies in which success was reported, most

of the success was shown through teacher rating scales. It is possible that this can be

attributed to teacher expectancy effects due to knowing that a student is participating

in a social skills group. Although students in this study did show improvement in the

SAEBRS data collected, this improvement could be related to teacher expectations

rather than actual behavior change. The teachers were aware the students were in the

group working on social skills and might have rated students higher at the end of the

study because they expected them to improve. Another factor as to why students were

rated higher on SAEBRS rating scales could have been that students mature

throughout their 5th grade year as they prepare for middle school and may show

improvement in this type of data naturally.

Similarly, the teachers’ Skillstreaming checklist scores showed more

improvement than the students’ ratings. This could be another manifestation of

expectancy effects on the teacher rating scales. There might also have been social

influence factors that affected the students’ self-ratings. When completing the

41

Skillstreaming checklist at both pre- and post-intervention, the students appeared to

copy each other’s answers. For example, when one student would say a frequency of

behavior, another student would record the same response. When reminded to

“think about how YOU really act in this situation” some students then changed their

answers. Given this apparent social influence on student answers, having the students

complete the rating scales individually, rather than in the group setting, might have

been more beneficial for collecting valid data related to students’ assessment of

themselves.

Compared to the SAEBRS and Skillstreaming scores, DBR data for students

showed no improvement or effects related to the social skill training. The DBR data

might be less likely to include expectancy effects because they were observations of

the students in various school environments, although in this case the observer was not

blind to the purpose of the study. Although the researcher, who was the intervention

leader as well, conducted the observations, the results did not appear to include

improvement expectations or influence student behaviors, with one exception. One

student was observed to become louder and more disruptive during observations in

structured settings. The teacher mentioned that it was not the best measure of his

behaviors. Using a trained observer who was blind to the intervention phase and

unfamiliar to the students might have reduced the likelihood of expectancy and

reactivity effects. The DBR data showed that students did not generalize the skills

from the intervention to other school environments. DBR was selected because it is a

form of data collection that can be used in the classroom by teachers, however the

intended target behaviors in this study may not have adequately matched the data

42

collected through DBR, which could account for lack of generalization. It is also

possible that the timing of the DBR observations influenced the resulting data. In the

early part of the intervention phase, students were observed before all of the target

behaviors have been taught. As a result, the students might not have acquired the

observed behaviors. Still, if this was the case, some improvement in the students’

behaviors would be expected after all skills were taught and observed, but this was not

the case.

The differences seen among the SAEBRS scores, the Skillstreaming checklist,

and DBR data show why researchers should be cautious when choosing sources of

data to measure change. Studies only using rating scales, and not direct observation

data, might not show a true picture of student progress. Direct observations such as

DBR ratings are currently the best method for evaluating true behavior change.

Limitations and Future Research

Several limitations exist with the current study. The first one related to the

observations conducted. More observations were initially scheduled. However, due to

student schedule variations they could not be completed. Often, identified students

were not in the activities that the researcher had scheduled to observe (i.e., recess).

Another problem related to the observations was that one student engaged in

increasingly disruptive behavior while being observed. Although not possible for this

study, having other trained observer would have been beneficial for DBR collection in

order to mitigate observer effects. DBR data also lacked interobserver agreement,

thus observation inaccuracies were possible.

43

Another limitation related to the intervention schedule. The students’ lunch

time was selected for the interventions so they would not miss instructional time. This

schedule proved difficult because students needed time to eat lunch and were a bit

resistant to standing up for role plays while trying to eat. The scheduled time was also

very limited because it took students time to obtain their lunches from the cafeteria and

then enter the room to participate in the group. Actual “work time” for group sessions

ended up being around 15 minutes. When homework was mentioned related to the

group, students responded with “We don’t get homework in lunch bunches.” The

observer changed the wording to “skill practice” as opposed to “homework” to

navigate this challenge. It seems that a more structured intervention time for social

skills intervention might have been more effective.

Future Skillstreaming research could focus on integrating lessons in all areas of

the school with regular coaching of skills learned in the small group. It seems that

intervention schedule needs to be more focused and not necessarily have lunch

interfere. Some participants and teachers did request a lengthened group intervention

(longer than 6 weeks). Another area to explore in relation to teacher expectations of

behavior change might be to have other staff collect data about perceived student skill

gains (i.e. specials teachers, cafeteria workers, education technicians, etc.). If such

data could be collected in a “blind” fashion so that the raters do not know if the

students are in an intervention, this information would help to illuminate the

differences in teacher ratings and observed behaviors.

Pairing Skillstreaming with other methods currently used in schools could also

be an area for future research. For example, mindfulness is currently being used in

44

many schools. If this concept of mindfulness is taught at the universal level, the

mindfulness techniques could be integrated into Skillstreaming lessons at the Tier 2

level to reinforce schoolwide expectations, and to help students utilize universal

strategies for specific areas of individual need.

Future research is needed in the area of social skill training and government

regulations related to education. The Every Student Succeeds Act (ESSA) is a recent

renewal of the previous Elementary and Secondary Education Act (2015). ESSA not

only puts emphasis on student success in testing, but also allows student success to be

measured with other variables (Mathis, & Trujillo, 2016). One of these variables

could be social emotional learning. Curricula like Skillstreaming could be used to

show student growth and potentially meet the requirement of the nonacademic area

mentioned in ESSA. Future research could focus on how curriculum related to social

skills can address this need along with the various tools that can monitor behavior

change in a normative manner.

Prior research suggests that Skillstreaming shows promise as an intervention

that can improve some students’ social-emotional skills. Anecdotal feedback gathered

throughout the intervention indicated that students enjoyed the group and teachers

looked positively upon the group because students were getting assistance and

intervention that is not always available in the classroom setting. Nonetheless, the

results indicated that the students’ observed behaviors across settings did not

demonstrate improvement on a par with the teachers’ SAEBRS ratings. This

incongruence suggests that methods for monitoring student behavior improvements

need more research. In addition, more research is needed to determine how

45

Skillstreaming can integrate with a schoolwide social-emotional learning curriculum

with strategies to promote generalization.

46

CHAPTER 5: SUMMARY

Prior studies have shown Skillstreaming to be effective for teaching selected

students prosocial skills. Most of the prior research included students with Autism,

Intellectual Disability, or incarcerated youth, with the best outcomes observed among

students with Autism and Intellectual Disability. In this study with typically-

developing fifth graders, Skillstreaming showed mixed generalization effects related to

a Tier 2 intervention. Similar to previous studies, teacher checklists and ratings of

students’ skills indicated social skill improvement. Nonetheless, direct behavior

observations showed negligible results with little to no improvement in student

behavior. ODR data also showed little to no progress related to the intervention. Even

with the minimal progress, most participants and teachers recommended this

intervention for other students. The findings of this study further endorse that

researchers should be cautious about basing results only on informant reports such as

checklists or rating scales. Additional research to confirm the current results and

identify the best methods for documenting student social skill improvement is

recommended.

47

REFERENCES

Abbott, M., & Wills, H. (2012). Improving the upside-down response-to-intervention

triangle with a systematic, effective elementary school reading team.

Preventing School Failure: Alternative Education for Children and Youth,

56(1), 37-46. doi:10.1080/1045988X.2011.555793

Albrecht, S. F., Mathur, S. R., Jones, R. E., & Alazemi, S. (2015). A school-wide

three-tiered program of social skills intervention: Results of a three-year cohort

study. Education & Treatment of Children, 38, 565-586.

American Psychiatric Association. (2013). Diagnostic and statistical manual of mental

disorders: DSM-5. Washington, D.C: Author.

August, G. J., Egan, E. A., Realmuto, G. M., & Hektner, J. M. (2003). Four years of the

early risers early-age-targeted preventive intervention: Effects on aggressive

children's peer relations. Behavior Therapy, 34, 453-470. doi:10.1016/S0005-

7894(03)80030-8

Bayat, M., Mindes, G., & Covitt, S. (2010). What does RTI (response to intervention)

look like in preschool? Early Childhood Education Journal, 37, 493-500.

doi:10.1007/s10643-010-0372-6

Becker, S. P., Luebbe, A. M., & Langberg, J. M. (2012). Co-occurring mental health

problems and peer functioning among youth with Attention-Deficit/Hyperactivity

disorder: A review and recommendations for future research. Clinical Child and

Family Psychology Review, 15, 279-302. doi:10.1007/s10567-012-0122-y

Brown-Chidsey, R., Andren, K. J., & Harrison, P. L. (2013). Assessment for intervention:

A problem-solving approach (2nd ed.). New York: The Guilford Press.

48

Chafouleas, S.M. & Riley-Tillman, T.C., (2015). Direct Behavior Ratings. University

of Connecticut.

Conduct Problems Prevention Research Group. (1999). Initial impact of the fast track

prevention trial for conduct problems: II. Classroom effects. Journal of

Consulting and Clinical Psychology, 67, 648–657.

Cooper J.O, Heron T.E, Heward W.L. (2007). Applied behavior analysis (2nd ed.) Upper

Saddle River, NJ: Pearson.

DE-PBS Project. (2015). Skill-building interventions: Externalizing behaviors. PBS

Support (PBS) Targeted Interventions. Author.

Duran, J. B., Zhou, Q., Frew, L. A., Kwok, O., & Benz, M. R. (2013). Disciplinary

exclusion and students with disabilities: The mediating role of social skills.

Journal of Disability Policy Studies, 24(1), 15-26.

Educational and Community Supports. (2016). PBIS apps. Retrieved from:

https://www.pbisapps.org/Pages/Default.aspx

Frey, K. S., Hirschstein, M. K., & Guzzo, B. A. (2000). Second step: Preventing

aggression by promoting social competence. Journal of Emotional and

Behavioral Disorders, 8, 102-112. doi:10.1177/106342660000800206

Gresham, F. (2015). Evidence-based social skills interventions for students at risk for

EBD. Remedial and Special Education, 36, 100-104.

doi:10.1177/0741932514556183

Gueldner, B., & Merrell, K. (2011). Evaluation of a social-emotional learning program in

conjunction with the exploratory application of performance feedback

49

incorporating motivational interviewing techniques. Journal of Educational and

Psychological Consultation, 21, 1-27. doi:10.1080/10474412.2010.522876

Guglielmo, H. M., & Tryon, G. S. (2001). Social skills training in an integrated

preschool program. School Psychology Quarterly, 16, 158–175

Horner, R. D., & Baer, D. M. (1978). Multiple-probe technique: A variation on the

multiple baseline. Journal of Applied Behavior Analysis, 11, 189–196.

Kavale, K. A., & Mostert, M. P. (2004). Social skills interventions for individuals with

learning disabilities. Learning Disability Quarterly, 27, 31–43.

Kilgus, S.P., Chafouleas, S. M., Riley-Tillman, T. C., & von der Embse, N.P. (2013).

Social, Academic, & Emotional Behavior Risk Screener (SAEBRS). Minneapolis,

MN: FastBridge Learning.

Kurtz, M. M., Mueser, K. T., Thime, W. R., Corbera, S., & Wexler, B. E. (2015). Social

skills training and computer-assisted cognitive remediation in schizophrenia.

Schizophrenia Research, 162(1-3), 35-41. doi:10.1016/j.schres.2015.01.020

La Greca, A. M., & Santogrossi, D. A. (1980). Social skills training with elementary

school students: A behavioral group approach. Journal of Consulting and

Clinical Psychology, 48, 220-227. doi:10.1037/0022-006X.48.2.220

Laugeson, E. A., Frankel, F., Mogil, C., & Dillon, A. R. (2009). Parent-assisted social

skills training to improve friendships in teens with autism spectrum disorders.

Journal of Autism and Developmental Disorders, 39, 596-606.

doi:10.1007/s10803-008-0664-5

50

Lerner, M. D., & Mikami, A. Y. (2012). A preliminary randomized controlled trial of

two social skills interventions for youth with high-functioning autism spectrum

disorders. Focus on Autism and Other Developmental Disabilities, 27, 147-157.

Lochman, J. E., Wells, K. C., Qu, L., & Chen, L. (2013). Three-year follow-up of coping

power intervention effects: Evidence of neighborhood moderation. Prevention

Science,14, 364-376. doi:10.1007/s11121-012-0295-0

Mathis, W.J. & Trujillo, T.M. (2016). Lessons from NCLB for the Every Student

Succeeds Act. Boulder, CO: National Education Policy Center. Retrieved from

http://nepc.colorado.edu/ publication/lessons-from-NCLB

McGinnis, E. (2012). Skillstreaming the elementary school child: A guide for teaching

prosocial skills. Champaign, Illinois: Research Press Publishers.

McGinnis, E. & Goldstein, A. (1997). Skillstreaming the elementary school child: New

strategies and perspectives for teaching prosocial skills. Champaign, Illinois:

Research Press Publishers.

Milsom, A., & Glanville, J. L. (2010). Factors mediating the relationship between social

skills and academic grades in a sample of students diagnosed with learning

disabilities or emotional disturbance. Remedial and Special Education, 31, 241-

251. doi:10.1177/0741932508327460

Nedim Bal, P. (2015). The effect of social skills training program on adolescents.

Journal of International Social Research, 8, 699-699.

doi:10.17719/jisr.20153913789

51

O’Handley, R. D., Ford, W. B., Radley, K. C., Helbig, K. A., & Wimberly, J. K. (2016).

Social skills training for adolescents with intellectual disabilities: A school-based

evaluation. Behavior Modification, 40, 541-567. doi:10.1177/0145445516629938

Plavnick, J. B., Kaid, T., & MacFarland, M. C. (2015). Effects of a school-based social

skills training program for adolescents with autism spectrum disorder and

intellectual disability. Journal of Autism and Developmental Disorders, 45,

2674-2690. doi:10.1007/s10803-015-2434-5

Predy, L., McIntosh, K., & Frank, J. L. (2014). Utility of number and type of office

discipline referrals in predicting chronic problem behavior in middle schools.

School Psychology Review, 43, 472-489.

Radley, K. C., Hanglein, J., & Arak, M. (2016). School-based social skills training for

preschool- age children with autism spectrum disorder. Autism, 20, 938-951.

doi:10.1177/1362361315617361

Ratcliffe, B., Wong, M., Dossetor, D., & Hayes, S. (2015). The association between

social skills and mental health in school-aged children with autism spectrum

disorder, with and without intellectual disability. Journal of Autism and

Developmental Disorders, 45, 2487-2496. doi:10.1007/s10803-015-2411-z

Rubin-Vaughan, A., Pepler, D., Brown, S., & Craig, W. (2011). Quest for the golden

rule: An effective social skills promotion and bullying prevention program.

Computers & Education, 56(1), 166-175. doi:10.1016/j.compedu.2010.08.009

Sallows, G. O., & Graupner, T. D. (2005). Intensive behavioral treatment for children

with Autism: Four-year outcome and predictors. American Journal on Mental

Retardation, 110, 417–438.

52

Sheridan, B. A., MacDonald, D. A., Donlon, M., Kuhn, B., McGovern, K., & Friedman,

H. (2011). Evaluation of a social skills program based on social learning theory,

implemented in a school setting. Psychological Reports, 108, 420-436.

Stone, W. L., & La Greca, A. M. (1990). The social status of children with learning

disabilities: A reexamination. Journal of Learning Disabilities, 23, 32-37.

doi:10.1177/002221949002300109

Taub, J., (2001). Evaluation of the Second Step violence prevention program at a rural

elementary school. School Psychology Review, 31, 186-200.

United States Department of Education. Institute of Education Sciences, National Center

for Education Evaluation and Regional Assistance. (n.d.). What works

clearinghouse. Retrieved from http://ies.ed.gov/ncee/wwc/findwhatworks.aspx

U.S. Department of Education, Institute of Education Sciences, What Works

Clearinghouse. (2012). Children classified as Having an Emotional Disturbance

intervention report: Early Risers. Retrieved from http://whatworks.ed.gov.

United States Department of Education, What Works Clearinghouse (2013). Social skills

training. Retrieved from

http://ies.ed.gov/ncee/wwc/pdf/intervention_reports/wwc_socialskills_020513.pdf

Van der Stouwe, T., Asscher, J. J., Hoeve, M., van der Laan, Peter H, & Stams, Geert Jan

J. M. (2016). Social skills training for juvenile delinquents: Post-treatment

changes. Journal of Experimental Criminology, 12, 515-536. doi:10.1007/s11292-

016-9262-2A

Walker, H. M., Seeley, J. R., Small, J., Severson, H. H., Graham, B. A., Feil, E. G.,

Serna, L., Golly, A.M. & Forness, S. R. (2009). A randomized controlled trial

53

of the first step to success early intervention: Demonstration of program

efficacy outcomes in a diverse, urban school district. Journal of Emotional and

Behavioral Disorders, 17, 197-212. doi:10.1177/1063426609341645

Walker, H. M., Kavanaugh, K., Stiller, B., Golly, A., Severson, H. H., & Feil, E. G.

(1998). First step to success: An early intervention approach for preventing school

antisocial behavior. Journal of Emotional and Behavioral Disorders, 6(2), 66-80.

doi:10.1177/106342669800600201

Webster-Stratton, C., Reid, M. J., & Hammond, M. (2004). Treating children with

early-onset conduct problems: Intervention outcomes for parent, child, and

teacher training. Journal of Clinical Child & Adolescent Psychology, 33, 105-

124. doi:10.1207/S15374424JCCP3301_11

Webster-Stratton, C., Reid, M. J., & Stoolmiller, M. (2008). Preventing conduct

problems and improving school readiness: Evaluation of the incredible years

teacher and child training programs in high-risk schools. Journal of Child

Psychology and Psychiatry, 49, 471-488. doi:10.1111/j.1469-

7610.2007.01861.x

Whetstone, P. J., Gillmor, S. C., & Schuster, J. G. (2015). Effects of a metacognitive

social skill intervention in a rural setting with at-risk adolescents. Rural Special

Education Quarterly, 34, 25-35.

White, S. W., & Roberson-Nay, R. (2009). Anxiety, social deficits, and loneliness in

youth with autism spectrum disorders. Journal of Autism and Developmental

Disorders, 39, 1006-1013. doi:10.1007/s10803-009-0713-8

54

Wight, M., & Chapparo, C. (2008). Social competence and learning difficulties: Teacher

perceptions. Australian Occupational Therapy Journal, 55, 256-265.

doi:10.1111/j.1440-1630.2007.00706.x

Wilson, R., & Shulha, D. (1995). The relationship of social skills to academic

achievement. Guidance & Counseling, 11, 8-11.

Yong, M., & Cheney, D. A. (2013). Essential features of tier 2 social–behavioral

interventions. Psychology in the Schools, 50, 844-861. doi:10.1002/pits.21

55

Appendix A: Skillstreaming Checklists

75

Appendix B: SAEBRS (Teacher) Items

76

Appendix C: Sample DBR Items

77

Appendix D: ODR Tracking Form

Student # of ODRS

preintervention (total)

# of ODRs related to peer

issues preintervention

# of ODRS postintervention

(total)

# of ODRs related to peer

issues postintervention

Student A

Student B

Student C

Student D

78

Appendix E: Social Validity Scale for Teachers and Students

1

(No)

2

(Maybe)

3

(Yes)

DidIfindthegroupeasytofitintoaschool

schedule?

DidthegroupachievethegoalsIthoughtit

would?

WouldIrecommendthegroupforother

students?

79

Appendix F: Treatment Integrity Checklist

Week.Session 1.1 2.1 3.1 4.1 5.1 6.1 Comments

Wastheskilldefinedbythegroupleader utilizingskillcardsorposterssothatallgroupmemberscouldunderstandtheskill?

Wastheskillmodeledforthegroupbythegroupleader?

Didstudentsroleplaytheskill?

Wasfeedbackprovidedbythegroupleaderfortheroleplay?

Wasthereaconclusiontothegroupwithareviewoftheskill?

Werethestudentsassigned homework relatedtotheskill?

80


Wastheskillreviewedwiththestudentspriortoenteringthesetting?

Werethestudentsgivenanopportunitytopracticetheirskillinoneofthefoursettings(classroom,recess,lunch,orspecial)?

Werethestudentsreinforcedforpracticingtheskill?

81


Wasareviewofthehomeworkcompleted?

Werethestudentsgivenfeedbackontheir performance relatedtotheskillforthatweek?

Werethestudentsreinforcedforcompletinghomework?

82

Appendix G: Sample Skillstreaming Schedule and Lesson

Monday Wednesday Friday

Week One Listening (Introduce Skill and Steps), Modeling, Role Play

Practice Day-Students will practice listening in the classroom with group leader observing and prompting if needed.

Integration Day- Skill practice in the cafeteria with rewards at table (card games, PBIS passes)

Week Two Asking for Help

(Introduce Skill and Steps), Modeling, Role Play

Practice Day-Students will practice asking for help in the Art/Music/PE classroom with group leader observing and prompting if needed


Week Three Saying Thank You


Practice Day-Students will practice saying thank you in the cafeteria with group leader observing and prompting if needed


Week Four Bringing Materials to Class


Practice Day-Students will practice brining materials to class with the group leader observing and prompting if needed


86

Appendix H: Group Self-Rating Report

87

BIOGRAPHY OF THE AUTHOR

Kathleen (Katie) Kolbe-Holden was born in Philipsburg, Pennsylvania on

November 13, 1982, where she resided for all of her childhood. She attended the

Pennsylvania State University and graduated in 2005 with a Bachelor of Science

degree in Human Development and Family Studies. She then attended the California

University of Pennsylvania and graduated with a Masters of Education in Counselor

Education in May 2008. After working a few years as a school counselor in

Pennsylvania, she moved to Colorado and worked as a school counselor and preschool

teacher. She moved to New Hampshire in 2012 and began working on her Doctor of

Psychology in School Psychology while working as a school counselor. Upon

graduation in August 2017, Katie plans to work as a school psychologist.

AN EVALUATION OF SOCIAL SKILL INTERVENTION EFFECTS … · AN EVALUATION OF SOCIAL SKILL...

Documents

Transcript of AN EVALUATION OF SOCIAL SKILL INTERVENTION EFFECTS … · AN EVALUATION OF SOCIAL SKILL...