Planning for Performance Measurement: Development of the OJJDP Performance Measurement System
-
Upload
troy-lester -
Category
Documents
-
view
34 -
download
4
description
Transcript of Planning for Performance Measurement: Development of the OJJDP Performance Measurement System
OJJDP Performance Measurement Training 1
Planning for Performance Planning for Performance Measurement: Measurement:
Development of the OJJDP Development of the OJJDP Performance Measurement Performance Measurement
SystemSystem
Presenters:
Greg Thompson, OJJDPDennis Mondoro, OJJDP
Heidi Hsia, OJJDP
OJJDP Performance Measurement Training 2
Enhance planning for performance measurement (PM) at the State level
Review development of the Performance measurement system
Review logic models and measures
Incorporate PM into the RFP
Understand reporting guidelines
Day 1 ObjectivesDay 1 Objectives
OJJDP Performance Measurement Training 3
Overcoming challenges in implementing performance measurement and being able to train subgrantees to:
Identify data sources
Share information
Monitor and track effectivelyUnderstand GMS reporting system and how to do performance reporting on GMS
Day 2 ObjectivesDay 2 Objectives
OJJDP Performance Measurement Training 4
Governmentwide move toward accountability
Government Performance and Results Act
President’s Management Agenda
Office of Management and Budget
Program Assessment Rating Tool
Paperwork Reduction Act of 1995
The Federal ClimateThe Federal Climate
Programs need to show effectiveness to justify funding.
OJJDP Performance Measurement Training 5
Funding and Information FlowsFunding and Information Flows
$ $ $
OJJDP
State
Subgrantee
Inform
ationCongress
OJJDP Performance Measurement Training 6
Information ExampleInformation Example
OJJDP
State
Subgrantee
Congress
Subgrantee Subgrantee Subgrantee
State
OJJDP Performance Measurement Training 7
Development MethodologyDevelopment Methodology
1. Categorize Formula Grants Program Areas:• System Improvement• Core Requirements• Prevention • Intervention
2. Develop logic models.3. Define common indicators.4. Apply prevention and selected intervention
models to Title V
Analysis possible by or across Program Area
OJJDP Performance Measurement Training 8
Sources for Indicator DevelopmentSources for Indicator Development
1. Federal guidelines and best practices
2. Literature from the field
3. Examples from other agencies at the Federal, State, and local levels
4. JJ Specialist/Title V Coordinator input• Written feedback• Conference calls
OJJDP Performance Measurement Training 9
Planning for Performance Planning for Performance Measurement: Measurement:
Reviewing the Logic Models Reviewing the Logic Models and Program Areas and Program Areas
Presenters:
Marcia Cohen, DSG, Inc.
Heidi Hsia, OJJDP
OJJDP Performance Measurement Training 10
What Is Performance What Is Performance Measurement?Measurement?
It is directly related to program goals and objectives.
It measures progress quantitatively.It is not exhaustive.It provides a “temperature” reading—it may
not tell you everything you want to know but provides a quick and reliable gauge of selected results.
OJJDP Performance Measurement Training 11
Measurement vs. EvaluationMeasurement vs. Evaluation
Impact evaluations are broader and assess the overall or net effects—intended or unintended—of the program as a whole.*
Scope
Time
Process evaluation
Outcome evaluation
Impact evaluation
Performance Measurement
Eval
uatio
n
Outcome evaluations investigate whether the program causes demonstrable effects on specifically defined target outcomes.*
Process evaluations investigate the process of delivering the program, focusing primarily on inputs, activities, and outputs.*
* Evaluation definitions excerpt from: Trochim, William M. The Research Methods Knowledge Base, 2nd Edition. Internet WWW page, at URL: <http://trochim.human.cornell.edu/kb/index.htm> (version current as of Aug. 02, 2000).
Program Monitoring
OJJDP Performance Measurement Training 12
Outputs Versus OutcomesOutputs Versus Outcomes
Outputs are products of program implementation/activities.
Outcomes are benefits or changes as a result of the program. There are two types of outcomes:
Short-term: The first benefits or changes experienced and the ones most closely related to program outputs.
Long-term: Link a program’s short-term and long-term outcomes. Often they are changes in practice, policy, decision-making, or behavior.
OJJDP Performance Measurement Training 13
With regard to the Formula Grants and Title V performance measures, to summarize: Outputs are at the micro level and reflect program-level activity.
Outcomes are at the macro level and, when aggregated, will reflect Federal outcomes.
A good performance measurement system should be results oriented and focus on desired outcomes, less on outputs.
Outputs Versus Outcomes (cont’d.)Outputs Versus Outcomes (cont’d.)
OJJDP Performance Measurement Training 14
Outcome Measure DefinitionsOutcome Measure Definitions
Short-term: Occurs during the program or by the end of the program.
Long-term: Occurs 6 months to 1 year after program completion.
OJJDP Performance Measurement Training 15
Logic Model: TemplateLogic Model: Template
A graphic representation that clearly lays out
the logical relationships between the problem
to be addressed, program activities, outputs,
and outcomes.
Activities
Outcomes • Short term
• Long term
OutputsProblem
OJJDP Performance Measurement Training 16
Juvenile Delinquency
Overall Formula Grants and Title V Programs Logic Model
Problems
OUTCOMES
Key
= System-level indicator
= Program-level indicator
Outputs
Juvenile Justice System Improvement Program Areas:10, 19, 23, 31, 33
Core Requirements Program Areas:6, 8, 10, 17, 28
Activities
Increased system
capacity
Improved planning and development
Improved program quality
Improved monitoring of compliance
Implement processes
Implementing Title V programs for keeping at-risk youth from offending or first-time, nonserious youth out of the JJ system.Program Areas:3, 4, 9, 10, 11, 12, 13, 15, 16, 18, 20, 21, 22, 25, 26, 27, 32, 34
Implementing Formula Grants prevention and intervention programsProgram Areas:1, 2, 3, 4, 5, 7, 9, 10, 11, 12, 13, 14, 15, 16, 18, 20, 21, 22, 24, 25, 26, 27, 29, 30, 32, 33, 34
Improved youth outcomes
Improved policies, procedures, operations, staffing, service delivery
Short Term Long Term
Improved juvenile justice systems
Increased compliance with Core Requirements
Reduced recidivism
Increased accountability
Outcome measures are for illustrative purposes only and are not comprehensive. To see a comprehensive list of outcomes, refer to the individual program area logic models.
To support both State and local prevention and intervention efforts and juvenile justice systems improvements
Objectives
To improve juvenile justice systems by increasing compliance with the Core Requirements and increasing availability and types of prevention and intervention programs
Goals
OJJDP Performance Measurement Training 17
Title V programs are for keeping at-risk youth from offending or first-time, nonserious youth out of the JJ system.
Title V has 18 Prevention and Early Intervention Program Areas:
3, 4, 9, 10, 11, 12, 13, 15, 16, 18, 20, 21, 22, 25, 26, 27, 32, 34
Title V Program AreasTitle V Program Areas
OJJDP Performance Measurement Training 18
Three Formula Grants Program Area categories:
Prevention and Intervention Program Areas:1, 2, 3, 4, 5, 7, 9, 10, 11, 12, 13, 14, 15, 16, 18, 20, 21, 22, 24, 25, 26, 27, 29, 30, 32, 33, 34
JJ System Improvement Program Areas:10, 19, 23, 31, 33
Core Requirements Program Areas:6, 8, 10, 17, 28
Formula Grants Program Areas Formula Grants Program Areas
OJJDP Performance Measurement Training 19
Planning for Performance Planning for Performance Measurement: Measurement:
Reviewing the Performance Reviewing the Performance MeasuresMeasures
Presenters:
Gregory Thompson, OJJDPHeidi Hsia, OJJDP
Marcia Cohen, DSG, Inc.
OJJDP Performance Measurement Training 20
Reporting RequirementsReporting Requirements
Output: All mandatory (bold) and two non-mandatory measures.
Outcome: All mandatory (bold) and two non-mandatory measures (may be either short- term or long-term).*
*Except Core Requirements and JJ System Improvement require only one non-mandatory measure.
OJJDP Performance Measurement Training 21
Key to Logic Models and Grid ChartsKey to Logic Models and Grid Charts
Bold = Mandatory measure
* = Mandatory only for intervention programs
** = Mandatory only for prevention programs
+ = Mandatory if applicable
OJJDP Performance Measurement Training 22
Output Objective CategoriesOutput Objective Categories
Six types of output objectives:• Increase organizational/system capacity• Improve planning and development• Improve program activities• Improve program efficiency • Improve the monitoring of compliance
OJJDP Performance Measurement Training 23
Outcome Objective CategoriesOutcome Objective Categories
Nine types of outcome objectives:• Reduce delinquency• Increase accountability• Improve system effectiveness• Increase prosocial behaviors• Increase program support• Increase compliance with Core Requirements• Reduce disproportionate minority contact• Improve school climate• Improve the management of the State Juvenile Justice and Delinquency Prevention program
OJJDP Performance Measurement Training 24
Prevention and Intervention Prevention and Intervention Program Areas Program Areas
OJJDP Performance Measurement Training 25
Mandatory Mandatory OutputOutput Measures for Measures for Prevention and InterventionPrevention and Intervention Programs Programs
Title V or FG $ awarded for services:
• The amount of Formula Grants and Title V funds in whole dollars that are awarded for delinquency prevention services during the reporting period. Program records are the preferred data source.
OJJDP Performance Measurement Training 26
Mandatory Mandatory OutputOutput Measures for Measures for Prevention Prevention
and Interventionand Intervention Programs (cont’d) Programs (cont’d)
Number of program youths served:• An unduplicated count of the number of youths served
by the program during the reporting period. Definition of the number of youth served for a reporting period is the number of program youth carried over from previous reporting period, plus new admissions during the reporting period. In calculating the 3-year summary, the total number of youth served is the number of participants carried over from the year previous to the first fiscal year, plus all new admissions during the 3 reporting fiscal years. Program records are the preferred data source.
OJJDP Performance Measurement Training 27
Mandatory Mandatory OutcomeOutcome Measures Measures for for PreventionPrevention** Programs** Programs
Number and percent of program youths exhibiting desired change in targeted behaviors: • Substance use • School attendance • Antisocial behavior • Family relationships • Pregnancies
Note: Double ** denotes mandatory for prevention programs
OJJDP Performance Measurement Training 28
Mandatory Mandatory OutcomeOutcome Measures Measures for for InterventionIntervention* Programs* Programs
Number and percent of program youths who offend or reoffend:
• The number of program youths who were rearrested or seen at juvenile court for a new delinquent offense. Appropriate for any youth-serving program.
Note: Single * denotes mandatory for intervention programs
OJJDP Performance Measurement Training 29
Mandatory Mandatory OutcomeOutcome Measures for Measures for Prevention and InterventionPrevention and Intervention Programs Programs
Number and percent of youth completing program requirements:
• The number and percent of program youth who have successfully fulfilled all program obligations and requirements. Program requirements will vary by program but should be a predefined list of requirements or obligations that clients must meet prior to program completion. Program records are the preferred data source.
OJJDP Performance Measurement Training 30
Example: Example: Probation Program Area Probation Program Area
OJJDP Performance Measurement Training 31
Outcome Measure DefinitionsShort Term:Occurs during the program or by the end of the programLong Term:Occurs 6 months to 1 year after program completion
Key = System-level indicator
= Individual-level indicator = Objectives
Bold= Mandatory measure
Performance measures should report on those activities funded by Title II (Formula Grants) funds.
Long-Term Outcome ObjectivesReduce delinquencyIncrease accountability
Short-Term Outcome Objectives
Reduce delinquencyImprove prosocial behaviorsIncrease accountabilityIncrease program support
#/% OF PROGRAM YOUTH WHO OFFEND OR REOFFEND
#/% of program staff with increased knowledge of program area
#/% OF PROGRAM YOUTH EXHIBITING DESIRED CHANGE IN TARGETED BEHAVIORS (E.G., SUBSTANCE USE, SCHOOL ATTENDANCE, ANTISOCIAL BEHAVIOR, FAMILY RELATIONSHIPS, SOCIAL COMPETENCIES)
#/% OF PROGRAM YOUTH COMPLETING PROGRAM REQUIREMENTS
#/% of program youth charged with formal probation violation
#/% of program youth committed to correctional facility
OUTCOME MEASURES
SHORT-TERM
#/% of program youth committed to correctional facility
#/% OF PROGRAM YOUTH WHO OFFEND OR REOFFEND
LONG-TERM
Delinquency
To improve JJ systems by increasing compliance with the Core Requirements and increasing the availability and types of prevention and intervention programs
Sample Logic Model: PA 24—Probation
PROBLEM
Goals
INTERVENTION PROGRAM
(Deficiency or lack of adequate probation
programs)
To support both State and local
intervention efforts and JJ
system improvements
SUBPROBLEM(S)
Objectives
Output ObjectivesIncrease organizational capacityImprove planning & developmentImprove program qualityImprove program activitiesImprove program efficiency
OUTPUT MEASURES
Conduct PlanningActivities
andImplement Program(s)
Monitor Program(s)
ACTIVITIES
FG $ AWARDED FOR SERVICES
# of planning activities conducted
# of MOUs developed
# of program slots available
# of FTEs funded with FG $
#/% of program staff trained
# of hours of program staff training provided
Use of best practice model (Y/N)
# of program materials developed
# of program/agency policies or procedures, created, amended, or rescinded
# OF PROGRAM YOUTH SERVED
Average length of stay in program
# of service hours completed
OJJDP Performance Measurement Training 32
PROGRAM AREA 24. ProbationPROGRAM AREA 24. ProbationOUTPUTOUTPUT PERFORMANCE MEASURES PERFORMANCE MEASURES
Sample Grid Chart: PA 24—ProbationOutput Performance Measures
OJJDP Performance Measurement Training 33
Sample Grid Chart: PA 24—ProbationOutcome Performance Measures
OJJDP Performance Measurement Training 34
Example: Example: Delinquency Prevention Delinquency Prevention
Program Area Program Area
OJJDP Performance Measurement Training 35
Outcome Measure DefinitionsShort Term:Occurs during the program or by the end of the programLong Term:Occurs 6 months to 1 year after program completion
Key = System-level indicator
= Individual-level indicator = Objectives
Bold= Mandatory measure
Performance measures should report on those activities funded by Title II (Formula Grants) funds.
Delinquency
To improve JJ systems by increasing compliance with the Core Requirements and increasing the availability and types of prevention and intervention programs
Sample Logic Model: PA 09—Delinquency Prevention
PROBLEM
Goals
PREVENTION PROGRAM
(Deficiency or lack of adequate delinquency prevention programs)
To support both State and local
intervention efforts and JJ
system improvements
SUBPROBLEM(S)
Objectives
Output ObjectivesIncrease organizational capacityImprove planning & developmentImprove program qualityImprove program activitiesImprove program efficiency
OUTPUT MEASURES
Conduct PlanningActivities
andImplement Program(s)
Monitor Program(s)
ACTIVITIES
FG $ AWARDED FOR SERVICES
# of MOUs developed
# of planning activities conducted
# of program slots available
# of FTEs funded with FG or TV $
#/% of program staff trained
# of hours of program staff training provided
Use of best practice model (Y/N)
# of program materials developed
# of program/agency policies or procedures, created, amended, or rescinded
# OF PROGRAM YOUTH SERVED
#/% of parents served
Average length of stay in program
# of service hours completed
Long-Term Outcome ObjectivesImprove prosocial behaviors
Short-Term Outcome Objectives
Improve prosocial behaviorsIncrease program supportIncrease accountability
#/% of program staff with increased knowledge of program area
#/% OF PROGRAM YOUTH EXHIBITING DESIRED CHANGE INTARGETED BEHAVIORS (E.G., SUBSTANCE USE, SCHOOL ATTENDANCE, ANTISOCIAL BEHAVIOR, FAMILY RELATIONSHIPS, PREGNANCIES)
#/% OF PROGRAM YOUTH COMPLETING PROGRAM REQUIREMENTS
#/% of program families satisfied with program
OUTCOME MEASURES
SHORT-TERM
#/% OF PROGRAM YOUTH EXHIBITING DESIRED CHANGE INTARGETED BEHAVIORS (E.G., SUBSTANCE USE, SCHOOL ATTENDANCE, ANTISOCIAL BEHAVIOR, FAMILY RELATIONSHIPS, PREGNANCIES)
LONG-TERM
#/% of youth satisfied with program
OJJDP Performance Measurement Training 36
Sample Grid Chart: PA 09—Delinquency PreventionOutput Performance Measures
OJJDP Performance Measurement Training 37
Sample Grid Chart: PA 09—Delinquency PreventionOutcomes Performance Measures
OJJDP Performance Measurement Training 38
Core Requirements Core Requirements Program Areas Program Areas
OJJDP Performance Measurement Training 39
Mandatory Mandatory OutputOutput Measures for Measures forCore RequirementsCore Requirements Programs Programs
FG funds awarded:
• The amount of Formula Grants money in whole dollars that is awarded for the separation requirement during the reporting period. Program records are the preferred data source.
OJJDP Performance Measurement Training 40
Mandatory Mandatory OutputOutput Measures for Measures for Core RequirementsCore Requirements Programs (cont’d) Programs (cont’d)
Number of programs implemented:
• The number of programs created and/or implemented during the reporting period. Program records are the preferred data source.
OJJDP Performance Measurement Training 41
Mandatory Mandatory OutcomeOutcome Measure for Measure for Core RequirementsCore Requirements Programs Programs
Change in the number of violations
The change in the number of violations of the [sight and sound] requirement from the previous reporting period compared with the current reporting period. Sight and sound compliance is determined according to the definition in the OJJDP Guidance Manual for Monitoring Facilities Under the JJDP Act of 2002. The Annual Compliance Monitoring Report is the preferred data source.
OJJDP Performance Measurement Training 42
Example: Example: Separation Program AreaSeparation Program Area
OJJDP Performance Measurement Training 43
Outcome Measure DefinitionsReported Annually.
Key = System-level indicator
= Individual-level indicator = Objectives
Bold= Mandatory measure
Performance measures should report on those activities funded by Title II (Formula Grants) funds.
Delinquency
To improve JJ systems by increasing compliance with the Core Requirements and increasing the availability and types of prevention and intervention programs.
Sample Logic Model: PA 28—Separation
PROBLEM
Goals
Outcome ObjectivesIncrease compliance with core requirementsIncrease program support
OUTCOME MEASURES
#/% of staff with increased knowledge of program area (e.g., separation regulations)
CHANGE IN THE NUMBER OF VIOLATIONS OF SIGHT AND SOUND SEPARATIONS
OUTPUT MEASURESACTIVITIES
Output Objectives•Improve monitoring of compliance•Increase organizational capacity•Improve planning & development
Conduct Planning and
Oversight Activities
Implement Processes
and/or Programs
FG $ AWARDED FOR SEPARATION
# of MOUs developed
Needs assessment completed (Y/N)
#/% OF PROGRAMS IMPLEMENTED
# of transportation plans developed
# of shelter beds contracted
#/% of staff trained
# of hours of staff training provided
# of materials developed
# of program policies/procedures created, amended, or rescinded
Monitor Processes
and/or Programs
#/% of site visits conducted
# of facilities receiving TA
CORE REQUIREMENT(Ensuring/monitoring compliance with the
Separation Core Requirement)
To support both State and local prevention and
intervention efforts and JJ
system improvements
SUBPROBLEM(S)
Objectives
OJJDP Performance Measurement Training 44
Sample Grid Chart: PA 28—SeparationOutput Performance Measures
OJJDP Performance Measurement Training 45
Sample Grid Chart: PA 28—SeparationOutcome Performance Measures
OJJDP Performance Measurement Training 46
Example: Example: Compliance Monitoring Compliance Monitoring
Program AreaProgram Area
OJJDP Performance Measurement Training 47
Outcome Measure DefinitionsReported Annually.
Key = System-level indicator
= Individual-level indicator = Objectives
Bold= Mandatory measure
Performance measures should report on those activities funded by Title II (Formula Grants) funds.
Delinquency
To improve JJ systems by increasing compliance with the Core Requirements and increasing the availability and types of prevention and intervention programs
Sample Logic Model: PA 06—Compliance Monitoring
PROBLEM
Goals
CORE REQUIREMENT(Ensuring/monitoring
compliance with Section 223 (a)(14) of the JJDP Act of 2002)
To support both State and local prevention and
intervention efforts and JJ
system improvements
SUBPROBLEM(S)
Objectives
Output Objectives•Improve monitoring of compliance•Increase organizational capacity•Improve planning & development
OUTPUT MEASURES
Conduct Planning and
Oversight Activities
ACTIVITIES
FUNDS ALLOCATED TO ADHERE TO SECTION 223 (a)(14) OF THE JJDP ACT OF 2002
# of activities that address compliance with Section 223 (a)(14) of the JJDP Act of 2002
# of MOUs developed
#/% of program staff trained
# of hours of program staff training provided
# of program materials developed
# of program policies/procedures created, amended, or rescinded
# of facilities receiving technical assistance
Outcome ObjectivesImprove the management of the State JJDP ProgramIncrease program support
OUTCOME MEASURES
#/% of staff with increased knowledge of program area (e.g., compliance monitoring)
SUBMISSION OF COMPLETE ANNUAL MONITORING REPORT TO OJJDP
OJJDP Performance Measurement Training 48
Sample Grid Chart: PA 06—Compliance MonitoringOutput Performance Measures
OJJDP Performance Measurement Training 49
Sample Grid Chart: PA 06—Compliance MonitoringOutcome Performance Measures
OJJDP Performance Measurement Training 50
Sample Logic Model:PA 10 – Disproportionate Minority Contact
PROBLEM
Outcome Measure DefinitionsShort-Term:Occurs during the program or by the end of the programLong-Term:Occurs 6 months to 1 year after program completion
Key = system-level indicator
= individual-level indicator = objectives
*Bold = mandatory measure
+ = mandatory only if applicable (if not applicable, choose a different measure)
* = mandatory for intervention programs only
** = mandatory for prevention programs only
Performance Measures should report on those activities funded by Title II (Formula Grants) funds.
Delinquency
To improve JJ systems by increasing compliance with the Core Requirements and increasing the availability and types of prevention and intervention programs
Goals
SUBPROBLEM(S)CORE REQUIREMENT
(Ensuring compliance with the DMC Core Requirement)
To support both State and local prevention and
intervention efforts and JJ
system improvements
Objectives
INTERVENTION PROGRAM(Deficiency or lack of
adequate intervention to reduce DMC)
PREVENTION PROGRAM(Deficiency or lack of adequate prevention
programs to reduce DMC)
JUVENILEJUSTICE SYSTEM IMPROVEMENT
(Lack of effective activities associated with planning and administration of the Formula Grants Program)
Output ObjectivesImprove monitoring of complianceIncrease system capacityImprove program qualityImprove planning & development
OUTPUT MEASURES
Conduct PlanningActivities
&Implement Program(s)
and Processes
Monitor Program(s)
and/or Programs
ACTIVITIES
FG OR TV $ AWARDED FOR DMC AT THE STATE AND LOCAL LEVELS
# of FTEs funded with FG or TV $
# of activities conducted
+ # OF PROGRAMS IMPLEMENTED
#/% of program staff trained
# of hours of program staff training provided
# of program materials developed
# of program/agency policies or procedures, created, amended, or rescinded
# of objective decision-making tools developed
+ # OF PROGRAM YOUTH SERVED
Average length of stay in program
# of service hours completed
#/% of site/TA visits conducted
OUTCOME MEASURES
Outcome ObjectivesReduce DMCImprove system effectivenessReduce delinquency
Number of minority staff hired
#/% of recommendations implemented from assessment studies
Short-Term
+ % OF CONTACT POINTS REPORTING REDUCTION IN DISPROPORTIONALITY AT STATE LEVEL
Long-Term
+ # OF STATE AGENCIES WITH IMPROVED DATA COLLECTION SYSTEMS
#/% of program staff with increased knowledge of program area
+ * #/% OF PROGRAM YOUTH WHO OFFEND OR REOFFEND
+ ** #/% OF PROGRAM YOUTH EXHIBITING DESIRED CHANGE INTARGETED BEHAVIORS (E.G., SUBSTANCE USE, SCHOOL ATTENDANCE, FAMILY RELATIONSHIPS, AND ANTISOCIAL BEHAVIOR)
+ #/% OF PROGRAM YOUTH COMPLETING PROGRAM REQUIREMENTS
#/% of program families satisfied with program
#/% of program youth satisfied with program
+ % OF CONTACT POINTS REPORTING REDUCTION IN DISPROPORTIONALITY AT LOCAL LEVEL
#/% of state agencies reporting improved data collection systems
+ * #/% OF PROGRAM YOUTH WHO OFFEND OR REOFFEND
# of local agencies reporting improved data collection systems
Outcome ObjectivesReduce DMCImprove system effectivenessImprove prosocial behaviorsReduce delinquencyIncrease program support
OJJDP Performance Measurement Training 51
Sample Grid Chart: PA 10—Disproportionate Minority ContactOutput Performance Measures
OJJDP Performance Measurement Training 52
Sample Grid Chart: PA 10—Disproportionate Minority ContactOutcome Performance Measures
OJJDP Performance Measurement Training 53
Juvenile Justice System Juvenile Justice System Improvement Program AreaImprovement Program Area
OJJDP Performance Measurement Training 54
Mandatory Mandatory OutputOutput Measure for Measure for JJ System ImprovementJJ System Improvement Programs Programs
Number of grants funded with Formula Grants funds:
• The number of grants funded with Formula Grants funds during the reporting period.
OJJDP Performance Measurement Training 55
Mandatory* Mandatory* OutputOutput Measures for Measures for JJ System ImprovementJJ System Improvement Programs Programs
* If applicable (if not applicable, select another non-mandatory measure)
• Number of programs implemented• Number of program youth served• Number of subgrants awarded• Number of grant applications reviewed
and commented upon
OJJDP Performance Measurement Training 56
Mandatory* Mandatory* OutcomeOutcome Measures for Measures for JJ System ImprovementJJ System Improvement Programs Programs
* If applicable (if not applicable, select another non-mandatory measure)
• Number/percent of youth completing program • Number/percent of youth exhibiting desired
changed in targeted behaviors • Average length of time between initial appearance
and disposition• Number of programs funded directly in line with 3-
year Plan• Number of Plan recommendations implemented • Number of allegations of civil rights violations filed• Number/percent of court hearings attended by an
advocate
OJJDP Performance Measurement Training 57
Outcome Measure DefinitionAnnual Measures: Measures of system change are conducted every 12 months.
Key = system-level indicator
= individual-level indicator = objectives
BOLD= mandatory measure
* = mandatory if applicable
Performance Measures should report on those activities funded by Title II (Formula Grants) funds.
Delinquency
To improve JJ systems by increasing compliance with the Core Requirements and increasing the availability and types of prevention and intervention programs.
Sample Logic Model:PA 33 – Youth Advocacy
PROBLEM
Goals
SYSTEM IMPROVEMENT
(Need for increased youth protection and advocacy)
To support both State and local prevention and
intervention efforts and JJ
system improvements
SUBPROBLEM(S)
Objectives
INTERVENTION PROGRAM
(Deficiency or lack of youth advocacy programs)
Output Objectives•Increase organizational capacity•Improve planning & development•Improve program efficiency
OUTPUT MEASURES
Conduct Planning Activities
ACTIVITIES
# of program slots available
# of FTEs funded with FG $
#/% of program staff trained
# of hours of program staff training provided
# of hours of training provided
# of program materials developed
# of program/agency policies or procedures created, amended, or rescinded
# of advocates trained
# hours spent in advocacy activities
Implement Program and/or
Procedure
Monitor Program and/or Process
FG $ AWARDED FOR SERVICES
Number of investigations conducted
Number of planning activities conducted
*# OF YOUTH SERVED
FG $ per program youth served
# of youth referred
Average length of tenure per advocate
Average time from assignment of case to first meeting with program youth
OUTCOME MEASURES
Outcome ObjectivesImprove system effectivenessIncrease program supportImprove program activities
#/% of youth affected by civil rights violations
*# OF ALLEGATIONS OF CIVIL RIGHTS VIOLATIONS
*#/% OF COURT HEARINGS ATTENDED BY AN ADVOCATE
#/% of substantiated civil rights violations
Number of assignments lasting until case completion
#/% of program youth/families satisfied with process/program
#/% of JJ staff satisfied with process/program
#/% of advocates satisfied with process/program
#/% of program staff with increased knowledge of program area (e.g., youth development, child abuse/neglect, or delinquency)
OJJDP Performance Measurement Training 58
Sample Grid Chart: PA 33—Youth Advocacy Output Performance Measures
OJJDP Performance Measurement Training 59
Sample Grid Chart: PA 33—Youth Advocacy Outcome Performance Measures
OJJDP Performance Measurement Training 60
Example: Example: State Advisory Group (SAG) State Advisory Group (SAG)
Program AreaProgram Area
OJJDP Performance Measurement Training 61
Mandatory Mandatory OutputOutput Measures for Measures for SAGSAG Program Area Program Area
Number of grants funded with Formula Grants funds:
• The number of grants funded with Formula Grants funds during the reporting period.
OJJDP Performance Measurement Training 62
Mandatory Mandatory OutputOutput Measures for Measures for (SAG) Program Area(SAG) Program Area
Number of grant applications reviewed and commented on:
• The number of grant applications reviewed and commented on to guide the development of juvenile justice programming in the State.
OJJDP Performance Measurement Training 63
Mandatory Mandatory OutcomeOutcome Measure for Measure forSAGSAG Program Area Program Area
Number and percent of Plan recommendations implemented:
• Number and percent of SAG recommendations for the State Plan implemented during the reporting period. The preferred data source is
committee meeting minutes.
OJJDP Performance Measurement Training 64
Outcome Measure DefinitionsReported Annually.
Key = System-level indicator
= Individual-level indicator = Objectives
Bold= Mandatory measure
Performance measures should report on those activities funded by Title II (Formula Grants) funds.
Delinquency
To improve JJ systems by increasing compliance with the Core Requirements and increasing the availability and types of prevention and intervention programs
Sample Logic Model: PA 31—State Advisory Group (SAG) Allocation
PROBLEM
Goals
JUVENILE JUSTICE SYSTEMS
IMPROVEMENT(Need for effective
SAG activities)
To support both State and local prevention and
intervention efforts and JJ
system improvements
SUBPROBLEM(S)
Objectives
Output Objectives•Improve planning and development•Improve program efficiency•Improve program quality
OUTPUT MEASURES
Conduct Planning and
Oversight Activities
ACTIVITIES
# OF GRANTS FUNDED WITH FORMULA GRANTS FUNDS
# of SAG committee meetings held
# of SAG subcommittee meetings held
Annual Report submitted to the Governor
#/% of SAG members trained
# OF GRANT APPLICATIONS COMMENTED ON
#/% of activities/meetings that involve youth
# of units of local government consulted during implementation of the 3-Year Plan
#/% of programs using best practice model
% of SAG allocation used
# of SAG members contributing to plan or plan update
Outcome ObjectivesIncrease compliance with Core RequirementsIncrease program support
#/% of SAG members who show increased knowledge of the program areas (for which they have oversight)
OUTCOME MEASURES
# of FG-funded programs sustained after 3 years
#/% OF PLAN RECOMMENDATIONS IMPLEMENTED
OJJDP Performance Measurement Training 65
Sample Grid Chart: PA 31—State Advisory Group Output Performance Measures
OJJDP Performance Measurement Training 66
Sample Grid Chart: PA 31—State Advisory Group Outcome Performance Measures
OJJDP Performance Measurement Training 67
Formula Grants Formula Grants
Performance Measurement Performance Measurement Implementation TimelineImplementation Timeline
(First year) (First year)
April 1, 2005 Reporting period begins
Sept. 30, 2005 Reporting period ends
Dec. 30, 2005 Data report due to OJJDP
OJJDP Performance Measurement Training 68
Formula Grants Formula Grants
Performance Measurement Performance Measurement Implementation TimelineImplementation Timeline
(Second and Subsequent Years) (Second and Subsequent Years)
Oct. 1, 2005 Reporting period begins
Sept. 30, 2006 Reporting period ends
Dec. 30, 2006 Data report due to OJJDP
OJJDP Performance Measurement Training 69
Title V Performance MeasurementTitle V Performance MeasurementImplementation Timeline Implementation Timeline
(First year)(First year)
January 1, 2005 Reporting period begins
Sept. 30, 2005 Reporting period ends
Nov. 30, 2005 Data reports due to OJJDP
March 31, 2006 FY 2005 OJJDP Report to Congress due
OJJDP Performance Measurement Training 70
Title V Performance MeasurementTitle V Performance MeasurementImplementation TimelineImplementation Timeline
(Second and Subsequent Years)(Second and Subsequent Years)
October 1, 2005 Reporting period begins
Sept. 30, 2006 Reporting period ends
Nov. 30, 2006 Data reports due to OJJDP
March 31, 2007 FY 2006 OJJDP Report to Congress due
OJJDP Performance Measurement Training 71
Performance MeasurementPerformance MeasurementTools and SupportTools and Support
• OJJDP Web site:
www.dsgonline.com/performance_measures.htmwww.dsgonline.com/performance_measures.htm
• Training
• Technical assistance