Regional Informational Session - EPISCenter · • Main disadvantage of locally developed programs...
Transcript of Regional Informational Session - EPISCenter · • Main disadvantage of locally developed programs...
SPEP™ Regional Information SessionApril 24, 2018
Bob Tomassini, Deputy Director, Juvenile Court Judges’ CommissionJeff Gregro, Deputy Chief, Berks County Juvenile Probation Office
Shawn Peck, Juvenile Justice System Improvement Specialist, Penn State EPISCenter
1The EPISCenter is a collaborative partnership between the Pennsylvania Commission on Crime and Delinquency (PCCD), the Pennsylvania Department of Human Services (DHS), and the Bennett Pierce Prevention Research Center, College of Health and Human Development, Penn State University, the EPISCenter is funded by DHS and PCCD. This resource was developed by the EPISCenter through PCCD grant VP-ST-24368
Overview
• Juvenile Justice System Enhancement Strategy (JJSES)
• SPEP™ 101: Introduction & Overview
• SPEP™ in Pennsylvania
• The SPEP™ Process
• Enhancing the Partnership between Juvenile Probation and Service Providers
• Roles & Expectations
• Panelists
• Questions & Answers
• Readiness Assessment & County Application
2
Juvenile Justice System Enhancement Strategy (JJSES)
3
Bob Tomassini, Deputy Director, Juvenile Court Judges’ Commission
JJSES FrameworkAchieving our Balanced and Restorative Justice Mission
4
SPEP™ 101: Introduction & Overview
5
Shawn Peck
Meta-analysis of a comprehensive collection of studies of interventions for juvenile offenders
• 700+ controlled studies
• Juveniles in programs aimed at reducing delinquency
• Focused on the programs’ effects on recidivism
• Recidivism defined as re-arrest within twelve months post intervention
• Published and unpublished studies from 1950 to 2014 in the U.S. and U.K.
• Programs were for juveniles from 12-21 years of age
6Howell, J.C., & Lipsey, M. W. (2012) Research-based guidelines for juvenile justice programs. Justice Research and Policy, (14) 1, p.1-18.
What is the Standardized Program Evaluation Protocol (SPEP™)
• An assessment of therapeutic services within a program
• The goal is to improve services for delinquent juveniles ultimately resulting in reduced recidivism
• Matches services to the SPEP™ service types that are supported by research
• Measures the effectiveness of a service based on:
o Youth risk level and aggressive/violent history*
o Program philosophy, and type
o Quality of service
o Amount of service
7
*Strongest predictor of recidivism identified in the meta-analysis.
Lipsey, M. W. (2009). The primary factors that characterize effective interventions with juvenile offenders: A meta-analytic overview. Victims and Offenders (4), 124-147.
• Locally developed intervention types are abundant in juvenile justice
• Research shows some types are effective at reducing recidivism
• Main disadvantage of locally developed programs is they… “lack the specificity that comes with the protocol for a model program and the associated training and support systems that are also often available from the developer” (p. 3).
Howell, J.C., & Lipsey, M. W. (2012) Research-based guidelines for juvenile justice programs. Justice Research and Policy, (14) 1, p.1-18.
Background
8
Arrest
Warn, Counsel & Dismiss
Diversion;Informal Probation
Probation
Out of Home Placement
Level of Supervision
InterventionPrograms
RecidivismOutcomes
Program A
Program B
Program C
Program D
Program E
Program F
U%
V%
W%
X%
Y%
Z%
TotalReoffense
Rate
Prev
entio
n Pr
ogra
ms
T%
Risk assessment& risk-baseddispositions Effective
programs
Needs assessment;match needsto program
Minimizereoffending
SPEP™ and Juvenile Justice
9
Method to Establish Evidence-Based Practice
• Model Program Implementation
• Evaluation of Local Programs
• New Approach: Meta-analysis of research on program based on service “type”
10
Key Findings from the Meta-analysis
• Philosophy Matters
• Comparable Impact
• Four Main Factors
• Score is Predictive
11Howell, J.C., & Lipsey, M. W. (2012) Research-based guidelines for juvenile justice programs. Justice Research and Policy, (14) 1, p.1-18.
Key Finding # 1: Philosophy Matters
12Howell, J.C., & Lipsey, M. W. (2012) Research-based guidelines for juvenile justice programs. Justice Research and Policy, (14) 1, p.1-18.
Therapeutic Services
Restorative
Restitution/Community Service
Mediation
Counseling
Individual
Mentoring
Family
Family Crisis
Group
Mixed
Skill Building
Behavior Management
Cognitive Behavioral Therapy
Social Skills Training
Challenge
Remedial Academic Program
Job Related Training
13
Restorative Services
Services aimed at repairing the harm caused by delinquent behavior
• Restitution/Community Service
• Mediation
14
CounselingServices emphasizing relationship between responsible adult and offender, family or others, in an attempt to influence feelings, cognitions and behavior
• Mentoring
• Individual counseling
• Family counseling
• Family crisis counseling
• Group counseling
• Mixed counseling
15
Skill BuildingServices aimed at providing instruction, practice, incentives, etc. to assist with behavior control and/or ability to participate in prosocial activity
• Behavioral management programs
• Cognitive behavioral therapy
• Social skills training
• Challenge programs
• Remedial academic program
• Job related training
16
Example: Cognitive-behavioral Therapy
Definition• Corrects faulty cognitions or
perceptions
• Provides skills to monitor and correct thought patterns and behaviors
• Focuses on relapse prevention
Programs• Aggression Replacement
Training (ART)®
• Thinking for a Change (T4C)
17
Positive Impact
HarmfulImpact 21 “homegrown”
4 MST4 FFT
29 Total
Key Finding # 2: Comparable Impact
18Lipsey, M. W., Howell, J. C., Kelly, M. R., Chapman, G., & Carver, D. (2010) Improving the Effectiveness of Juvenile Justice Programs: A New Perspective on Evidence-Based Practice. Georgetown University, Washington, D.C., USA.
Key Finding # 3: Four Main Factors
Four factors most strongly related to recidivism reduction:
• Youth risk level and aggressive/violent history*
• Program philosophy, and type
• Quality of service
• Amount of service
19
*Strongest predictor of recidivism identified in the meta-analysis.
Lipsey, M. W. (2009). The primary factors that characterize effective interventions with juvenile offenders: A meta-analytic overview. Victims and Offenders (4), 124-147.
Standardized Program Evaluation Protocol (SPEP™)for Services to Juvenile Offenders©
Recalibrated version, 2013PointsPossible
PointsReceived
Primary and Supplemental Service Types [Identified according to definitions derived from the research]Primary Service Type for Program Being Rated
Group 1 services (5 points) Group 4 services (25 points)Group 2 services (10 points) Group 5 services (30 points)Group 3 services (15 points)
30
Supplemental Service TypeQualifying supplemental service used: Yes (5 points) No (0 points)
5
Quality of Service Delivery[Determined from a systematic assessment of the relevant features of the provider and provider organization]Rated quality of services delivered:
Low (5 points)Medium (10 points)High (20 points)
20
Amount of Service[Determined from data for the qualifying group of service recipients]Duration [Target number of weeks specified for each service type]% of youth who received at least the target weeks of service:
0% (0 points) 60% (6 points)20% (2 points) 80% (8 points)40% (4 points) 99% (10 points)
10
Contact Hours [Target number of hours specified for each service type]% of youth who received at least the target hours of service:
0% (0 points) 60% (6 points)20% (2 points) 80% (8 points)40% (4 points) 99% (10 points)
10
Risk Level of Youth Served[Determined from risk ratings on a valid instrument for the qualifying group of service recipients]
% of youth with medium or high % of youth with high riskrisk scores (greater than low): scores (greater than moderate):
0% (0 points) 75% (7 points) 0% (0 points) 25% (8 points)30% (2 points) 85% (10 points) 15% (3 points) 30% (10 points)50% (5 points) 95% (12 points) 20% (5 points) 35% (13 points)
25
Provider’s Total SPEP™ Score 100 (Insert Score)
Service Type
Service Quality
Amount of Service
Juvenile Risk
20
Key Finding # 4: Score is Predictive
21
*Quality of service delivery not scored in this sample.
Howell, J.C., & Lipsey, M. W. (2012) Research-based guidelines for juvenile justice programs. Justice Research and Policy, (14) 1, p.1-18.
SPEP™ in Pennsylvania
22Acknowledgements: Information taken from SPEP Scoring and Program Certification Training, Gabrielle Lynn Chapman, Peabody Research Institute, Vanderbilt University, July 16, 2013.
Jeff Gregro
Expanding the SPEP™ Footprint in Juvenile Justice
• Original development siteso North Carolinao Arizona
• Juvenile Justice System Improvement Project (JJSIP)o Connecticuto Floridao Pennsylvania
• OJJDP Justice System Reform and Reinvestment Initiativeo Delawareo Iowao Milwaukee County
• Independent participantso Tennesseeo Georgia o Queensland, Australiao Oregon
23
The Juvenile Justice System Improvement Project
• Funded by Federal OJJDP
• The Center for Juvenile Justice Reform – Georgetown University
• Pennsylvania chosen as one of four states in the “Improving the Effectiveness of Juvenile Justice Programs” Project
• In 2011, Berks County served as project pilot site
• Two major areas of focus based on the work ofo Buddy Howell’s “Comprehensive Strategy” work
o Mark Lipsey’s “Standardized Program Evaluation Protocol”
24
Implementation of SPEP™ in PA
• 2011 – Berks County Pilot Site
• 2012 – EPISCenter funded by PCCD to oversee rollout• 2013 – Initial Expansion – Allegheny, Bucks, Dauphin, Lehigh
(with Vanderbilt University Support and Technical Assistance)• 2015 – McKean, Mercer, Luzerne, Lycoming, Venango, York
• 2017 – Lebanon County
• 2018 – Statewide Expansion
25
SPEP™ in Pennsylvania
26
JJSES Leadership Team
SPEP™ Advisory Group
Juvenile Probation
Service Providers
Learning Community
EPISCenter
Role of the Leadership Team in SPEP™
• Oversees the training and technical assistance for the Juvenile Justice System Enhancement Strategy
• Responds to recommendations made by the SPEP™ Advisory Group
• Determines county readiness for SPEP™
27
Role of the SPEP™ Advisory Group (SAG)
• Review of SPEP™ implementation challenges and successes
• Make recommendations to the JJSES Leadership Team on the SPEP™ project
• Reviews resources created by the EPISCenter staff and SPEP™ Learning Community
28
SPEP™ Learning Community (LC)
• Face-to-face meetings coordinated by the EPISCenter JJSIS team and includes all probation staff trained in SPEP™
• LC members contribute to the agenda by submitting questions and/or providing training in areas of expertise
29
EPISCenter Technical Assistance
• Support provided according to each county’s unique strengths and capacity
• Data collection and analysis
• Resource development (website, logic model, manuals, fact sheets, data templates, etc.)
• Oversight of SPEP™ training in Pennsylvania
30
The SPEP™ Process
31
Jeff Gregro & Shawn Peck
SPEP™ Lifecycle
32
Service Classification &Quality Interviews
Data Collection & Analysis (Duration,
Dosage & Risk)
Feedback Report Results (Scores &
Recommendations)
Performance Improvement Plan
Development
Performance Improvement Plan
ImplementationService
Type
Provider Delivery
SPEP™ Assessment
Probation/ Court Usage
What is a Program?• A package of services delivered by a provider
• Distinct from the sites or locations in which those services are delivered (e.g., day treatment, group home, afterschool program).
33
ABC Agency(service provider)
Group Counseling
(service)
Multisystemic Therapy (MST)
(service)
Circle of Courage(program)
What is a Service?
• A single, identifiable treatment modality or behavioral therapy received by juveniles
• There can be more than one service in a program
Example: a program where all youth receive
1. individual counseling
2. group counseling
3. social skills training
34
Service Classification Interview
• Provider shares information with SPEP™ Team
• Discussion of daily operations
• Review of program materials
• Match if possible, to the SPEP™ research-is it therapeutic?
• Full Program Profile is completed to describe all of the services offered
35
Quality of Service Delivery Interview
• Interviews may occur the same day as service classification or on another day
• Length of time will vary – approximately 1.5 hours
• Interviews are guided by the Quality Measures Checklist
• Findings and Performance Improvement Recommendations are incorporated into the Feedback Report
36
Quality Measures Checklist
Includes points received and an explanation on how the quality score was derived:
• Written Protocol
• Staff Training
• Staff Supervision
• Response to Drift
37
Amount of Service
Effects on recidivism associated with:• Duration (weeks) of service
• Dosage (face-to-face hours) of service
38
SPEP™ Cohort
• A group of youth that received a service during a specific timeframe (selected by provider and probation)
• Duration and dosage are calculated for each youth individually
• Minimum of 10 youth necessary to obtain a full SPEP™ score
• Must have YLS score in order to be counted
39
Feedback Report Summary & Recommendations
• Summarizes the findings of the service in light of recidivism reduction
• Identifies improvement recommendations in areas for:
o Service Type (addition of supplemental service)
o Quality of Service Delivery
o Service Amount
o Risk Level
40
Feedback Reports
Standardized Templates exist for various types of SPEP™ Assessments:
• Initial assessment (baseline)
• First reassessment
• Two or more reassessments
41
Performance Improvement Process
Utilize the Performance Improvement Guide and Template to:
o Address the areas identified during the SPEP™ process
o The SPEP™ Team and provider identifies the timeframe and method for improvements
o Identify the technical assistance and support necessary to implement improvements
o Monitor the progress and outcomes (quarterly progress calls)
42
Performance Improvement Plan Example
43http://www.episcenter.psu.edu/juvenile/appendix
Standardized Program Evaluation Protocol (SPEP) Performance Improvement Plan
Performance Improvement Process
44
6 months to 24 months
SPEP Reassessment
12 months to 24 months
Achievement of Performance Improvement goals, data collection with new cohorts
Modifications are made to services per Performance Improvement Plan
30 days to 12 monthsPerformance Improvement Plan Implemented Progress Update Meeting/Calls occur
1 to 30 daysProvider & Probation collaborate
on ways to improve servicesPerformance Improvement
Plan createdPerformance Improvement Plan
is reviewedProgress call timeframes are
established
Start: Feedback Report Meeting-Day 1Presentation and discussion of SPEP Feedback Report
Overview of Performance Improvement Concepts
Intro to SPEP Performance Improvement Plan
Establish timeframe for Performance Improvement Plan
Enhancing the Partnership between Juvenile Probation and
Service Providers
45
Jeff Gregro
The Partnership
• Juvenile probation departments and service providers are equal partners in the SPEP™ process
• The SPEP™ process brings both stakeholders to the table from pre-SPEP™ planning to SPEP™ reassessments
46
Service Type
Provider Delivery
SPEP™ Assessment
Probation/ Court Usage
Roles & Expectations
47
Jeff Gregro
Probation’s Role in the Partnership
• Planning
o Initiates communication plans for SPEP™ locally
o Engages key stakeholders for SPEP™ (courts, key JPO staff)
o Identifies and prioritizes service providers for SPEP™
• Preparation
o Helps prepare the service provider for the SPEP™ process
o Participates in the SPEP™ interviews, facilitates the conversation, provides probation perspective on program role in service matrix
o Administers the YLS and provides assessment to provider
o Contributes to or fully writes the Feedback Report
48
Probation’s Role in the Partnership - Continued• Policyo Incorporating SPEP™ into job descriptions, program descriptions,
and provider contractso Determine policies and procedures related to SPEP™o Communicating information and updates regarding the SPEP™
project and SPEP™ assessment results• Practice
o Acts as an equal and collaborative partner in performance improvement
o Understands performance improvement is shared ownershipo Consistent messaging about the initiative across various levels of
Juvenile Court/Probationo Active member of SPEP™ Learning Community
49
Service Provider Roles for SPEP™• Prepare their programs for the SPEP™ Process
o Inform key agency stakeholders on the SPEP™ Process prior to Pre-SPEP™ visit
o Assess data collection processes
• Facilitate the Pre-SPEP™ visit and tour
o Items for discussion typically include:
Understanding of the SPEP™ process and what it entails
Cohort information (adjudicated youth with risk scores are essential)
50
Service Provider Expectations for SPEP™
• Prepare for Service Classification and Quality Interviews
o Program descriptions
o Manuals
o Staff
o Youth
• Participates in the SPEP™ interviews
• Verifying that the information on Full Program Profile is correct
• Completes data collection spreadsheets and assists in retrieving any missing YLS scores
51
Panelists
Russ Carlino, Chief Probation Officer, Allegheny County
David Evrard, Assistant Chief Probation Officer, Allegheny County
Bill Shultz, Placement Liaison, Allegheny County
Brian Barnhart, Trainer, Allegheny County
52
Questions & Answers
53
Readiness Assessment & County Application
54
Readiness Assessment & County Application
• Applications are currently being accepted for the next SPEP training occurring in October, 2018 (Dates TBA) If interested, please complete and submit the Readiness Assessment and County Application by email to [email protected] by June 15th, 2018.
• Todays PowerPoint as well as the application materials can be located at http://www.episcenter.psu.edu/juvenile
55
Thank you!The EPISCenter is a collaborative partnership between the Pennsylvania Commission on Crime and Delinquency (PCCD), the Pennsylvania Department of Human Services (DHS), and the Bennett Pierce Prevention Research Center, College of Health and Human Development, Penn State University. The EPISCenter is funded by DHS and PCCD. This resource was developed by the EPISCenter through PCCD grant VP-ST-24368.
56
206 Towers Building, University Park, PA 16802Phone : (814) 863-2568 Email: [email protected]
www.EPISCenter.org
/EPISCenterPSU @EPIS_Center
For more information about the SPEP™ process please contact:
[email protected]@countyofberks.com