SPEP™ Regional Information SessionApril 24, 2018

Bob Tomassini, Deputy Director, Juvenile Court Judges’ CommissionJeff Gregro, Deputy Chief, Berks County Juvenile Probation Office

Shawn Peck, Juvenile Justice System Improvement Specialist, Penn State EPISCenter

1The EPISCenter is a collaborative partnership between the Pennsylvania Commission on Crime and Delinquency (PCCD), the Pennsylvania Department of Human Services (DHS), and the Bennett Pierce Prevention Research Center, College of Health and Human Development, Penn State University, the EPISCenter is funded by DHS and PCCD. This resource was developed by the EPISCenter through PCCD grant VP-ST-24368

Overview

• Juvenile Justice System Enhancement Strategy (JJSES)

• SPEP™ 101: Introduction & Overview

• SPEP™ in Pennsylvania

• The SPEP™ Process

• Enhancing the Partnership between Juvenile Probation and Service Providers

• Roles & Expectations

• Panelists

• Questions & Answers

• Readiness Assessment & County Application

2

Juvenile Justice System Enhancement Strategy (JJSES)

3

Bob Tomassini, Deputy Director, Juvenile Court Judges’ Commission

JJSES FrameworkAchieving our Balanced and Restorative Justice Mission

4

SPEP™ 101: Introduction & Overview

5

Shawn Peck

Meta-analysis of a comprehensive collection of studies of interventions for juvenile offenders

• 700+ controlled studies

• Juveniles in programs aimed at reducing delinquency

• Focused on the programs’ effects on recidivism

• Recidivism defined as re-arrest within twelve months post intervention

• Published and unpublished studies from 1950 to 2014 in the U.S. and U.K.

• Programs were for juveniles from 12-21 years of age

6Howell, J.C., & Lipsey, M. W. (2012) Research-based guidelines for juvenile justice programs. Justice Research and Policy, (14) 1, p.1-18.

What is the Standardized Program Evaluation Protocol (SPEP™)

• An assessment of therapeutic services within a program

• The goal is to improve services for delinquent juveniles ultimately resulting in reduced recidivism

• Matches services to the SPEP™ service types that are supported by research

• Measures the effectiveness of a service based on:

o Youth risk level and aggressive/violent history*

o Program philosophy, and type

o Quality of service

o Amount of service

7

*Strongest predictor of recidivism identified in the meta-analysis.

Lipsey, M. W. (2009). The primary factors that characterize effective interventions with juvenile offenders: A meta-analytic overview. Victims and Offenders (4), 124-147.

• Locally developed intervention types are abundant in juvenile justice

• Research shows some types are effective at reducing recidivism

• Main disadvantage of locally developed programs is they… “lack the specificity that comes with the protocol for a model program and the associated training and support systems that are also often available from the developer” (p. 3).

Howell, J.C., & Lipsey, M. W. (2012) Research-based guidelines for juvenile justice programs. Justice Research and Policy, (14) 1, p.1-18.

Background

8

Arrest

Warn, Counsel & Dismiss

Diversion;Informal Probation

Probation

Out of Home Placement

Level of Supervision

InterventionPrograms

RecidivismOutcomes

Program A

Program B

Program C

Program D

Program E

Program F

U%

V%

W%

X%

Y%

Z%

TotalReoffense

Rate

Prev

entio

n Pr

ogra

ms

T%

Risk assessment& risk-baseddispositions Effective

programs

Needs assessment;match needsto program

Minimizereoffending

SPEP™ and Juvenile Justice

9

Method to Establish Evidence-Based Practice

• Model Program Implementation

• Evaluation of Local Programs

• New Approach: Meta-analysis of research on program based on service “type”

10

Key Findings from the Meta-analysis

• Philosophy Matters

• Comparable Impact

• Four Main Factors

• Score is Predictive

11Howell, J.C., & Lipsey, M. W. (2012) Research-based guidelines for juvenile justice programs. Justice Research and Policy, (14) 1, p.1-18.

Key Finding # 1: Philosophy Matters

12Howell, J.C., & Lipsey, M. W. (2012) Research-based guidelines for juvenile justice programs. Justice Research and Policy, (14) 1, p.1-18.

Therapeutic Services

Restorative

Restitution/Community Service

Mediation

Counseling

Individual

Mentoring

Family

Family Crisis

Group

Mixed

Skill Building

Behavior Management

Cognitive Behavioral Therapy

Social Skills Training

Challenge

Remedial Academic Program

Job Related Training

13

Restorative Services

Services aimed at repairing the harm caused by delinquent behavior

• Restitution/Community Service

• Mediation

14

CounselingServices emphasizing relationship between responsible adult and offender, family or others, in an attempt to influence feelings, cognitions and behavior

• Mentoring

• Individual counseling

• Family counseling

• Family crisis counseling

• Group counseling

• Mixed counseling

15

Skill BuildingServices aimed at providing instruction, practice, incentives, etc. to assist with behavior control and/or ability to participate in prosocial activity

• Behavioral management programs

• Cognitive behavioral therapy

• Social skills training

• Challenge programs

• Remedial academic program

• Job related training

16

Example: Cognitive-behavioral Therapy

Definition• Corrects faulty cognitions or

perceptions

• Provides skills to monitor and correct thought patterns and behaviors

• Focuses on relapse prevention

Programs• Aggression Replacement

Training (ART)®

• Thinking for a Change (T4C)

17

Positive Impact

HarmfulImpact 21 “homegrown”

4 MST4 FFT

29 Total

Key Finding # 2: Comparable Impact

18Lipsey, M. W., Howell, J. C., Kelly, M. R., Chapman, G., & Carver, D. (2010) Improving the Effectiveness of Juvenile Justice Programs: A New Perspective on Evidence-Based Practice. Georgetown University, Washington, D.C., USA.

Key Finding # 3: Four Main Factors

Four factors most strongly related to recidivism reduction:

• Youth risk level and aggressive/violent history*

• Program philosophy, and type

• Quality of service

• Amount of service

19

*Strongest predictor of recidivism identified in the meta-analysis.

Lipsey, M. W. (2009). The primary factors that characterize effective interventions with juvenile offenders: A meta-analytic overview. Victims and Offenders (4), 124-147.

Standardized Program Evaluation Protocol (SPEP™)for Services to Juvenile Offenders©

Recalibrated version, 2013PointsPossible

PointsReceived

Primary and Supplemental Service Types [Identified according to definitions derived from the research]Primary Service Type for Program Being Rated

Group 1 services (5 points) Group 4 services (25 points)Group 2 services (10 points) Group 5 services (30 points)Group 3 services (15 points)

30

Supplemental Service TypeQualifying supplemental service used: Yes (5 points) No (0 points)

5

Quality of Service Delivery[Determined from a systematic assessment of the relevant features of the provider and provider organization]Rated quality of services delivered:

Low (5 points)Medium (10 points)High (20 points)

20

Amount of Service[Determined from data for the qualifying group of service recipients]Duration [Target number of weeks specified for each service type]% of youth who received at least the target weeks of service:

0% (0 points) 60% (6 points)20% (2 points) 80% (8 points)40% (4 points) 99% (10 points)

10

Contact Hours [Target number of hours specified for each service type]% of youth who received at least the target hours of service:

0% (0 points) 60% (6 points)20% (2 points) 80% (8 points)40% (4 points) 99% (10 points)

10

Risk Level of Youth Served[Determined from risk ratings on a valid instrument for the qualifying group of service recipients]

% of youth with medium or high % of youth with high riskrisk scores (greater than low): scores (greater than moderate):

0% (0 points) 75% (7 points) 0% (0 points) 25% (8 points)30% (2 points) 85% (10 points) 15% (3 points) 30% (10 points)50% (5 points) 95% (12 points) 20% (5 points) 35% (13 points)

25

Provider’s Total SPEP™ Score 100 (Insert Score)

Service Type

Service Quality

Amount of Service

Juvenile Risk

20

Key Finding # 4: Score is Predictive

21

*Quality of service delivery not scored in this sample.

Howell, J.C., & Lipsey, M. W. (2012) Research-based guidelines for juvenile justice programs. Justice Research and Policy, (14) 1, p.1-18.

SPEP™ in Pennsylvania

22Acknowledgements: Information taken from SPEP Scoring and Program Certification Training, Gabrielle Lynn Chapman, Peabody Research Institute, Vanderbilt University, July 16, 2013.

Jeff Gregro

Expanding the SPEP™ Footprint in Juvenile Justice

• Original development siteso North Carolinao Arizona

• Juvenile Justice System Improvement Project (JJSIP)o Connecticuto Floridao Pennsylvania

• OJJDP Justice System Reform and Reinvestment Initiativeo Delawareo Iowao Milwaukee County

• Independent participantso Tennesseeo Georgia o Queensland, Australiao Oregon

23

The Juvenile Justice System Improvement Project

• Funded by Federal OJJDP

• The Center for Juvenile Justice Reform – Georgetown University

• Pennsylvania chosen as one of four states in the “Improving the Effectiveness of Juvenile Justice Programs” Project

• In 2011, Berks County served as project pilot site

• Two major areas of focus based on the work ofo Buddy Howell’s “Comprehensive Strategy” work

o Mark Lipsey’s “Standardized Program Evaluation Protocol”

24

Implementation of SPEP™ in PA

• 2011 – Berks County Pilot Site

• 2012 – EPISCenter funded by PCCD to oversee rollout• 2013 – Initial Expansion – Allegheny, Bucks, Dauphin, Lehigh

(with Vanderbilt University Support and Technical Assistance)• 2015 – McKean, Mercer, Luzerne, Lycoming, Venango, York

• 2017 – Lebanon County

• 2018 – Statewide Expansion

25

SPEP™ in Pennsylvania

26

JJSES Leadership Team

SPEP™ Advisory Group

Juvenile Probation

Service Providers

Learning Community

EPISCenter

Role of the Leadership Team in SPEP™

• Oversees the training and technical assistance for the Juvenile Justice System Enhancement Strategy

• Responds to recommendations made by the SPEP™ Advisory Group

• Determines county readiness for SPEP™

27

Role of the SPEP™ Advisory Group (SAG)

• Review of SPEP™ implementation challenges and successes

• Make recommendations to the JJSES Leadership Team on the SPEP™ project

• Reviews resources created by the EPISCenter staff and SPEP™ Learning Community

28

SPEP™ Learning Community (LC)

• Face-to-face meetings coordinated by the EPISCenter JJSIS team and includes all probation staff trained in SPEP™

• LC members contribute to the agenda by submitting questions and/or providing training in areas of expertise

29

EPISCenter Technical Assistance

• Support provided according to each county’s unique strengths and capacity

• Data collection and analysis

• Resource development (website, logic model, manuals, fact sheets, data templates, etc.)

• Oversight of SPEP™ training in Pennsylvania

30

The SPEP™ Process

31

Jeff Gregro & Shawn Peck

SPEP™ Lifecycle

32

Service Classification &Quality Interviews

Data Collection & Analysis (Duration,

Dosage & Risk)

Feedback Report Results (Scores &

Recommendations)

Performance Improvement Plan

Development

Performance Improvement Plan

ImplementationService

Type

Provider Delivery

SPEP™ Assessment

Probation/ Court Usage

What is a Program?• A package of services delivered by a provider

• Distinct from the sites or locations in which those services are delivered (e.g., day treatment, group home, afterschool program).

33

ABC Agency(service provider)

Group Counseling

(service)

Multisystemic Therapy (MST)

(service)

Circle of Courage(program)

What is a Service?

• A single, identifiable treatment modality or behavioral therapy received by juveniles

• There can be more than one service in a program

Example: a program where all youth receive

1. individual counseling

2. group counseling

3. social skills training

34

Service Classification Interview

• Provider shares information with SPEP™ Team

• Discussion of daily operations

• Review of program materials

• Match if possible, to the SPEP™ research-is it therapeutic?

• Full Program Profile is completed to describe all of the services offered

35

Quality of Service Delivery Interview

• Interviews may occur the same day as service classification or on another day

• Length of time will vary – approximately 1.5 hours

• Interviews are guided by the Quality Measures Checklist

• Findings and Performance Improvement Recommendations are incorporated into the Feedback Report

36

Quality Measures Checklist

Includes points received and an explanation on how the quality score was derived:

• Written Protocol

• Staff Training

• Staff Supervision

• Response to Drift

37

Amount of Service

Effects on recidivism associated with:• Duration (weeks) of service

• Dosage (face-to-face hours) of service

38

SPEP™ Cohort

• A group of youth that received a service during a specific timeframe (selected by provider and probation)

• Duration and dosage are calculated for each youth individually

• Minimum of 10 youth necessary to obtain a full SPEP™ score

• Must have YLS score in order to be counted

39

Feedback Report Summary & Recommendations

• Summarizes the findings of the service in light of recidivism reduction

• Identifies improvement recommendations in areas for:

o Service Type (addition of supplemental service)

o Quality of Service Delivery

o Service Amount

o Risk Level

40

Feedback Reports

Standardized Templates exist for various types of SPEP™ Assessments:

• Initial assessment (baseline)

• First reassessment

• Two or more reassessments

41

Performance Improvement Process

Utilize the Performance Improvement Guide and Template to:

o Address the areas identified during the SPEP™ process

o The SPEP™ Team and provider identifies the timeframe and method for improvements

o Identify the technical assistance and support necessary to implement improvements

o Monitor the progress and outcomes (quarterly progress calls)

42

Performance Improvement Plan Example

43http://www.episcenter.psu.edu/juvenile/appendix

Standardized Program Evaluation Protocol (SPEP) Performance Improvement Plan

Performance Improvement Process

44

6 months to 24 months

SPEP Reassessment

12 months to 24 months

Achievement of Performance Improvement goals, data collection with new cohorts

Modifications are made to services per Performance Improvement Plan

30 days to 12 monthsPerformance Improvement Plan Implemented Progress Update Meeting/Calls occur

1 to 30 daysProvider & Probation collaborate

on ways to improve servicesPerformance Improvement

Plan createdPerformance Improvement Plan

is reviewedProgress call timeframes are

established

Start: Feedback Report Meeting-Day 1Presentation and discussion of SPEP Feedback Report

Overview of Performance Improvement Concepts

Intro to SPEP Performance Improvement Plan

Establish timeframe for Performance Improvement Plan

Enhancing the Partnership between Juvenile Probation and

Service Providers

45

Jeff Gregro

The Partnership

• Juvenile probation departments and service providers are equal partners in the SPEP™ process

• The SPEP™ process brings both stakeholders to the table from pre-SPEP™ planning to SPEP™ reassessments

46

Service Type

Provider Delivery

SPEP™ Assessment

Probation/ Court Usage

Roles & Expectations

47

Jeff Gregro

Probation’s Role in the Partnership

• Planning

o Initiates communication plans for SPEP™ locally

o Engages key stakeholders for SPEP™ (courts, key JPO staff)

o Identifies and prioritizes service providers for SPEP™

• Preparation

o Helps prepare the service provider for the SPEP™ process

o Participates in the SPEP™ interviews, facilitates the conversation, provides probation perspective on program role in service matrix

o Administers the YLS and provides assessment to provider

o Contributes to or fully writes the Feedback Report

48

Probation’s Role in the Partnership - Continued• Policyo Incorporating SPEP™ into job descriptions, program descriptions,

and provider contractso Determine policies and procedures related to SPEP™o Communicating information and updates regarding the SPEP™

project and SPEP™ assessment results• Practice

o Acts as an equal and collaborative partner in performance improvement

o Understands performance improvement is shared ownershipo Consistent messaging about the initiative across various levels of

Juvenile Court/Probationo Active member of SPEP™ Learning Community

49

Service Provider Roles for SPEP™• Prepare their programs for the SPEP™ Process

o Inform key agency stakeholders on the SPEP™ Process prior to Pre-SPEP™ visit

o Assess data collection processes

• Facilitate the Pre-SPEP™ visit and tour

o Items for discussion typically include:

Understanding of the SPEP™ process and what it entails

Cohort information (adjudicated youth with risk scores are essential)

50

Service Provider Expectations for SPEP™

• Prepare for Service Classification and Quality Interviews

o Program descriptions

o Manuals

o Staff

o Youth

• Participates in the SPEP™ interviews

• Verifying that the information on Full Program Profile is correct

• Completes data collection spreadsheets and assists in retrieving any missing YLS scores

51

Panelists

Russ Carlino, Chief Probation Officer, Allegheny County

David Evrard, Assistant Chief Probation Officer, Allegheny County

Bill Shultz, Placement Liaison, Allegheny County

Brian Barnhart, Trainer, Allegheny County

52

Questions & Answers

53

Readiness Assessment & County Application

54

Readiness Assessment & County Application

• Applications are currently being accepted for the next SPEP training occurring in October, 2018 (Dates TBA) If interested, please complete and submit the Readiness Assessment and County Application by email to SPEP@episcenter.org by June 15th, 2018.

• Todays PowerPoint as well as the application materials can be located at http://www.episcenter.psu.edu/juvenile

55

Thank you!The EPISCenter is a collaborative partnership between the Pennsylvania Commission on Crime and Delinquency (PCCD), the Pennsylvania Department of Human Services (DHS), and the Bennett Pierce Prevention Research Center, College of Health and Human Development, Penn State University. The EPISCenter is funded by DHS and PCCD. This resource was developed by the EPISCenter through PCCD grant VP-ST-24368.

56

206 Towers Building, University Park, PA 16802Phone : (814) 863-2568 Email: EPISCenter@psu.edu

www.EPISCenter.org

/EPISCenterPSU @EPIS_Center

For more information about the SPEP™ process please contact:

speck@episcenter.orgjgregro@countyofberks.com