Linking Performance Measures to Benchmarks in the Budget Process
description
Transcript of Linking Performance Measures to Benchmarks in the Budget Process
![Page 1: Linking Performance Measures to Benchmarks in the Budget Process](https://reader035.fdocuments.us/reader035/viewer/2022062321/56814189550346895dad7465/html5/thumbnails/1.jpg)
1
Linking Performance Measures to Benchmarks in the Budget Process
March-April 2002
Department of Administrative Services
Oregon Progress Board
www.econ.state.or.us/opb
![Page 2: Linking Performance Measures to Benchmarks in the Budget Process](https://reader035.fdocuments.us/reader035/viewer/2022062321/56814189550346895dad7465/html5/thumbnails/2.jpg)
2
Overview
• Why measure performance?
• Why Oregon Benchmarks?
• What makes a good performance measure?
• What is required in the budget process?
• Getting started
![Page 3: Linking Performance Measures to Benchmarks in the Budget Process](https://reader035.fdocuments.us/reader035/viewer/2022062321/56814189550346895dad7465/html5/thumbnails/3.jpg)
3
Handouts
• Logic models– Logic model worksheets (yellow)– Logic model examples (ochre)
• Submission forms– Links to Oregon Benchmarks (blue)– Performance Measure Data Summary (green)
• Evaluation forms– PM criteria worksheet (off-white)– Today’s training evaluations (purple)
![Page 4: Linking Performance Measures to Benchmarks in the Budget Process](https://reader035.fdocuments.us/reader035/viewer/2022062321/56814189550346895dad7465/html5/thumbnails/4.jpg)
4
Why measure performance?
It’s at the core of results-based management• Provides greater accountability
Is the ship on course?
• Fosters internal learning and improvementIs the ship running well?
AND…it has been required since 1993.
See Appendix B.
![Page 5: Linking Performance Measures to Benchmarks in the Budget Process](https://reader035.fdocuments.us/reader035/viewer/2022062321/56814189550346895dad7465/html5/thumbnails/5.jpg)
5
Why link to Oregon Benchmarks?They articulate Oregon’s hopes and expectations.
• “High-level outcomes” or measures of societal well-being.
• Beacons for the “ship” and the “fleet”.
• For budget, link only to those that relate to your core mission and goals (“primary linkages”).
![Page 6: Linking Performance Measures to Benchmarks in the Budget Process](https://reader035.fdocuments.us/reader035/viewer/2022062321/56814189550346895dad7465/html5/thumbnails/6.jpg)
6
Oregon has ninety benchmarks in three broad categories.
• Economy• Education• Civic Engagement• Social Support• Public Safety• Community Development• Environment
![Page 7: Linking Performance Measures to Benchmarks in the Budget Process](https://reader035.fdocuments.us/reader035/viewer/2022062321/56814189550346895dad7465/html5/thumbnails/7.jpg)
7
What happens if your agency does not link to an Oregon Benchmark?.
• That’s OK. You have two options:
– You may submit other high-level outcomes to gauge how Oregon is doing relative to your mission.
– Small agencies: if this is not feasible, you can “look up” to your mission and/or mandate.
• All high-level outcomes should pass the “so what” test. Do Oregonians care?
So what??
![Page 8: Linking Performance Measures to Benchmarks in the Budget Process](https://reader035.fdocuments.us/reader035/viewer/2022062321/56814189550346895dad7465/html5/thumbnails/8.jpg)
8
Logic models define the links.
Goal (generallyunmeasurable)
Performance Measures
Impact Intermediate Outcome Measures
Agency Inputs and Activities
OutputMeasures
High-level outcome(s)
(measurable)
![Page 9: Linking Performance Measures to Benchmarks in the Budget Process](https://reader035.fdocuments.us/reader035/viewer/2022062321/56814189550346895dad7465/html5/thumbnails/9.jpg)
(Increase) % of offenders with intake assessments
Output
“So That”
% of offenders engaged in work, training, education and/or treatment(is increased)
Intermediate Outcome
“So That”
% of offenders showing a measurable improvement in behavior and/or skill level (is increased)
Intermediate Outcome
“So That”
% of paroled offenders convicted of a new felony within three years (is decreased)
High-Level Outcome (Benchmark #61)
A logic model embeds a continuum of measures in a“so that” chain.
![Page 10: Linking Performance Measures to Benchmarks in the Budget Process](https://reader035.fdocuments.us/reader035/viewer/2022062321/56814189550346895dad7465/html5/thumbnails/10.jpg)
10
What makes a good performance measure?
BASIC criteria required for 2003-05Performance measures should:1. Use GASB* terms and definitions2. Gauge progress towards agency goals and
benchmarks or other high-level outcomes3. Focus on a few key indicators4. Have targets5. Be based on accurate and reliable data
*Governmental Accounting Standards Board
![Page 11: Linking Performance Measures to Benchmarks in the Budget Process](https://reader035.fdocuments.us/reader035/viewer/2022062321/56814189550346895dad7465/html5/thumbnails/11.jpg)
11
• OUTCOME = Result (the best kind of measure)– High-level (societal) = OBM#11, Per capita income– Intermediate = Average wage of agency job placements
• OUTPUT = Product or service (“widget”)– # of job placements per quarter
• INPUT = Time, money, material or demand– FTEs in the “Job Placement Unit”– Dollars allocated to the “Job Placement Unit”– Case load or number of complaints– INPUTS ARE NOT STAND-ALONE PERFORMANCE
MEASURES• EFFICIENCY = Input per output
– # of days required to process a job application
Basic criteria #1. Use GASB definitions
![Page 12: Linking Performance Measures to Benchmarks in the Budget Process](https://reader035.fdocuments.us/reader035/viewer/2022062321/56814189550346895dad7465/html5/thumbnails/12.jpg)
12
Two kinds of intermediate outcomes: chunks and stones
EXAMPLE: Benchmark #18, Ready to Learn1. A “chunk” of the population is measured
for the high-level outcome (HLO) % of children of served families who are
ready to learn (versus % of all children in the county who are ready to learn)
2. “Stepping stone” toward the HLO is measured.
• % of trained parents who read regularly to their children (reading to kids is a stepping stone to being ready to learn)
![Page 13: Linking Performance Measures to Benchmarks in the Budget Process](https://reader035.fdocuments.us/reader035/viewer/2022062321/56814189550346895dad7465/html5/thumbnails/13.jpg)
# of intake assessments completed
Output
% of offenders engaged in work, training, education and/or treatment
Intermediate Outcome
% of offenders showing a measurable improvement in behavior and/or skill level
Intermediate Outcome
% of paroled offenders convicted of a new felony within three years
High-Level Outcome (Benchmark #64)
Basic criteria #2. Measure progress towards agency goals and benchmarks
Goal to “reduce repeat offenders” is UNMEASURABLE
MEASURES gauge progress
![Page 14: Linking Performance Measures to Benchmarks in the Budget Process](https://reader035.fdocuments.us/reader035/viewer/2022062321/56814189550346895dad7465/html5/thumbnails/14.jpg)
14
Basic criteria #3. Focus on a few key measures.
• Represent the scope of agency responsibility
• Number 30 max (except for mega-agencies)
• Include the best measures for:– “Is the ship on course?”– “Is the ship running well?”
• Additional measures internal toyour agency can provide more detailed management information.
![Page 15: Linking Performance Measures to Benchmarks in the Budget Process](https://reader035.fdocuments.us/reader035/viewer/2022062321/56814189550346895dad7465/html5/thumbnails/15.jpg)
Agencies should decide how “high up” to gofor their key measures.
More agency influ
ence
More policy in
tent
Consider level of agency
INFLUENCE
# of intake assessments completed
Output
% of offenders engaged in work, training, education and/or treatment
Intermediate Outcome
% of offenders showing a measurable improvement in behavior and/or skill level
Intermediate Outcome
% of paroled offenders convicted of a new felony within three years
High-Level Outcome (Benchmark #64)
![Page 16: Linking Performance Measures to Benchmarks in the Budget Process](https://reader035.fdocuments.us/reader035/viewer/2022062321/56814189550346895dad7465/html5/thumbnails/16.jpg)
16
Basic criteria #4. Performance measures should have targets.
• TARGET = Desired level at any given point in time
• Should be ambitious but realistic• Target setting is an art and a science
based on – trend data
– comparisons
– expert opinion
• Targets not required until Jan. 2003
Recidivism now
Recidivism TARGET
![Page 17: Linking Performance Measures to Benchmarks in the Budget Process](https://reader035.fdocuments.us/reader035/viewer/2022062321/56814189550346895dad7465/html5/thumbnails/17.jpg)
17
Basic criteria #5. Accurate and reliable data.
• Without trustworthy data, the system is meaningless.
• Example: verifiable employment records are better than estimated job creation
• Each measure should have at least one data point, preferably several.
• Data should describe what is being measured.
![Page 18: Linking Performance Measures to Benchmarks in the Budget Process](https://reader035.fdocuments.us/reader035/viewer/2022062321/56814189550346895dad7465/html5/thumbnails/18.jpg)
18
Performance measure criteria ADVANCED = required for 2005-07 biennium
Performance measures should:6. Link to an organizational unit
7. Cover organizational outcomes like efficiency and customer satisfaction
8. Allow comparisonsMore training on Advanced Criteria later
![Page 19: Linking Performance Measures to Benchmarks in the Budget Process](https://reader035.fdocuments.us/reader035/viewer/2022062321/56814189550346895dad7465/html5/thumbnails/19.jpg)
Annual Performance Reports submitted to DAS/LFO. (Annually in September)
Submit Links to Oregon Benchmarks (March - August 2002)
TA & Training on Performance Measures
Budget Instructions
Comments & Measures
Accompany Governor’s Recommended Budget (November 2002)
See Guidelines pp.10 & 11
Budget Timeline for Performance Measures
(April – August 2002)Adjustments (Optional)
Performance Measure Data Summary to Ways & Means (January - June 2003)
Agencies adjust measures and targets per legislature (June 2003)
Criteria-based review(April – Aug. 2002)
![Page 20: Linking Performance Measures to Benchmarks in the Budget Process](https://reader035.fdocuments.us/reader035/viewer/2022062321/56814189550346895dad7465/html5/thumbnails/20.jpg)
20
Hypothetical example #1
Impact
AGENCY INPUT/ACTIVITYAward grants to local contractors to conduct “best practice” juvenile crime prevention programs (JCP).
INTERMEDIATE OUTCOMES % of juveniles in JCP programs with significantly mitigated risk
factors.
GOALReduce juvenile
crime.
HLOJuvenile Arrests
(OBM#61)
Agency Performance
Measures
OUTPUTS# grants awarded by county
# days of TA delivered by county
![Page 21: Linking Performance Measures to Benchmarks in the Budget Process](https://reader035.fdocuments.us/reader035/viewer/2022062321/56814189550346895dad7465/html5/thumbnails/21.jpg)
21
Hypothetical example #2
Impact
AGENCY INPUT/ACTIVITYAward grants to local contractors to design/deliver “best practice”
parent education classes.
INTERMEDIATE OUTCOMES % of children from participating (trained) families entering school
ready to learn.
GOAL Healthy, thriving
children.
HLO: % of kindergarteners ready
to learn (OBM#18)
Agency Performance
Measures
OUTPUTS# grants awarded by county.
“Best practice” guidelines done by
![Page 22: Linking Performance Measures to Benchmarks in the Budget Process](https://reader035.fdocuments.us/reader035/viewer/2022062321/56814189550346895dad7465/html5/thumbnails/22.jpg)
22
Hypothetical example #3
Impact
AGENCY INPUT/ACTIVITY Jointly sponsor, with cities,
regional educational events for private citizens every quarter.
INTERMEDIATE OUTCOMES % participating citizens with
improved understandingCustomer satisfaction ratings
GOAL: Citizen involvement (C.I.) in
land use planning
HLO: % of cities with neighborhood
organizations.
Agency Performance
Measures
OUTPUTS# citizens trained.
# C.I. guidelines distributed.
![Page 23: Linking Performance Measures to Benchmarks in the Budget Process](https://reader035.fdocuments.us/reader035/viewer/2022062321/56814189550346895dad7465/html5/thumbnails/23.jpg)
23
Related Oregon Benchmarks (OBMs) or High-Level Outcomes (HLOs):
% of cities with active neighborhood organizations
Agency Goal OBM#HLO#
Key Performance Measure
PM #
PM Since
New or Mod.?
2000 Valu
e
2005 Target
Lead Division or Unit (Optional)
Citizen involvement in land use planning
1 Percent of participants with improved understanding
Ag# - 1
2002
New 55%
70%
Communications
Pertinent Benchmark or High-level outcome(s):Links to Oregon Benchmarks Form
HLO 1 - Percent of cities with active neighborhood organizations.
![Page 24: Linking Performance Measures to Benchmarks in the Budget Process](https://reader035.fdocuments.us/reader035/viewer/2022062321/56814189550346895dad7465/html5/thumbnails/24.jpg)
24
Performance Measure Definition(numbered as shown below) Data Targets
1997 1998 1999 2000 2001 2000 2001 2002 2003 2004 2005
Agency # - 1
Agency # - 2
Agency # - 3
Agency # - 4
Agency # - 5
Agency # - 6
Agency # - 7
Agency # - 8
Performance Measure Data Summary (for Ways and Means)
55%62% 70%60% 65%Percent of participants with improved understanding
![Page 25: Linking Performance Measures to Benchmarks in the Budget Process](https://reader035.fdocuments.us/reader035/viewer/2022062321/56814189550346895dad7465/html5/thumbnails/25.jpg)
25
Helpful websites
Governmental Accounting Standards Boardwww.gasb.org GASB home pagehttp://accounting.rutgers.edu/raw/seagov/pmg/
National Center for Public Productivity, RutgersA Brief Guide to Performance Measurement in Local Government (1997)http://newark.rutgers.edu/~ncpp/cdgp/Manual.htm#man1
John F. Kennedy School of Government, HarvardAn Open Memorandum to Government Executives - Get Results Through Performance Management (2001)http://www.ksg.harvard.edu/visions/
![Page 26: Linking Performance Measures to Benchmarks in the Budget Process](https://reader035.fdocuments.us/reader035/viewer/2022062321/56814189550346895dad7465/html5/thumbnails/26.jpg)
26
Additional resources
• Book and reports– Measuring Up, Jonathan Walters (1998)
– The Reinventor’s Fieldbook, David Osborne and Peter Plastrik, Chapter 7 (2000)
– Making Results-Based State Government Work, The Urban Institute (2001)
• Oregon Progress Board– Technical Assistance
– Training
– Strategic Planning
![Page 27: Linking Performance Measures to Benchmarks in the Budget Process](https://reader035.fdocuments.us/reader035/viewer/2022062321/56814189550346895dad7465/html5/thumbnails/27.jpg)
27
George DunfordPerformance Measure Manager, DAS(503) [email protected]
Jeffrey L. TryensExecutive Director, Progress Board(503) [email protected]
Rita ConradSenior Policy Analyst, Progress Board(503) [email protected]
DAS/Oregon Progress Board