Perform Meas Part 1 - Tematic Contexpert...Australia, Canada, Finland and Sweden include performance...
38
Essay P art 1
Transcript of Perform Meas Part 1 - Tematic Contexpert...Australia, Canada, Finland and Sweden include performance...
Essay
Part 1
Training forFinance
Managers
2
PerformanceMeasures
3
Essay
SummaryThis essay examines the developmentand implementation of performancemeasurement at the local level ofgovernment. We will define perfor-mance measurement and describe thebenefits to a local government thatuses it to improve the odds of successin achieving its service goals. We willalso discuss the steps involved insetting up a performance measurementsystem and obstacles that may beencountered along the way.
DefinitionPERFORMANCE MEASUREMENT is an objec-tive and systematic process forcollecting, analyzing and using infor-mation to determine how efficientlyand effectively local governmentservices are being delivered andobjectives are being achieved. APERFORMANCE MEASUREMENT SYSTEM is agauge of what the organization feels isimportant and how well it is perform-ing. As a good measurement systemcan drive an organization in a positivedirection, a poor system can lead it faroff course. The lack of measurement isa major obstacle to improvement.
To measure a government programeffectively, three things must be done.First is to define the outcomes theprogram is intended to achieve. Secondis to measure the performance of theprogram in relation to the achievementof its intended outcomes. Thirdis to report the results tothe decision makerswho are in a positionto act on the informa-tion. A performancemeasurementsystem is simplydoing these threethings continuouslyand systematically asshown in the diagram.
Measuring Government Programs
Measure
Define
Report
Benefits of PerformanceMeasurement
“What gets measured, gets done.” “Wemanage what we measure.” “If youcan’t measure it, you can’t control,manage or improve it.” These arefamiliar management axioms whichillustrate the importance of perfor-mance measurement. An effectiveperformance measurement systemprovides the feedback to managers anddecision makers that is critical tocontinuous improvement of govern-ment services. It is a way of keepingscore on how various operations aredoing.
Properly developed and administered, aperformance measurement system canhave a number of benefits for a localgovernment.
• Policy Making and Oversight.Strengthens policy formulation byproviding decision makers with asound basis for asking questionabout service needs and perfor-mance and making resourcereallocation decisions whenneeded.
• Operational Direction. Providesmanagers with a more systematicway to detect operational strengthsand weaknesses and to exercisecontinuous program analysis,
Training forFinance
Managers
4
scheduling, evaluation and costcontrol.
• Accountability. Can help depart-ments and the whole organizationearn the trust of citizens by demon-strating a good return in service forrevenue received.
• Planning. Facilitates strategic andoperational planning by providinginformation needed to set goalsand objectives and plan programsto accomplish them.
• Management. Provides a basis forthe early identification of opera-tional deficiencies and a way toshow how efficiently resources arebeing used to provide services andachieve goals.
• Budgeting. Improves the budgetprocess by making possible moreobjective decisions about resourceallocation and redistribution, costreduction and investment ofsurplus funds.
• Contracting. Aids competitivecontracting by presenting well-documented cost and performancedata and monitoring contractperformance for service quality.
• Work Supervision. Useful in achiev-ing improved employeeperformance by providing anobjective basis for setting perfor-mance targets and providingfeedback and incentives.
Many countries have recognized thebenefits of performance measurement.Agency performance is measured in theUnited States, Great Britain, Canada,
Australia, New Zealand, Sweden,Finland, Norway, Denmark, Switzer-
land and the Netherlands. In theUnited States and New Zealand,department heads are held accountablethrough performance contracts, target-ing, appraisal and pay for performance.Norway, Switzerland and the Nether-lands stress accountability and control.Great Britain focuses on public report-ing of performance. The United States,Australia, Canada, Finland and Swedeninclude performance information in thebudget. Australia, the United Statesand Norway integrate performancemeasurement into their planning.
A New Role for theFinance Officer
In many parts of the world, financeofficers are taking the lead in efforts toimprove the performance of localgovernment activities. Not only do theyconduct fiscal audits of governmentagencies andgovernment-fundedprograms,they alsomeasure theperformanceof specificfunctions of operational units (localgovernment departments) to determineif intended outcomes are beingachieved.
A particularly noteworthy example ofthis role of the finance manager can befound in the United States where theauditor’s office in the city of Austin,Texas has been conducting perfor-mance audits of several social serviceprograms for youth. One of its audits ofa prenatal and early childhood educa-tion program found that the servicecontractor was doing a commendablejob of defining and measuring programoutcomes. The agency was able todemonstrate that an acceptably highpercentage of its clients gave birth tohealthy children and that the childrenleaving the program maintained orimproved their healthy status.
On the other hand, an audit by theauditor’s office of a drug free educationprogram for school-age children runjointly by the local school district and
PerformanceMeasures
5
Setting up a Performance Measurement SystemCareful planning and design are crucial to an effective performance measurementsystem. A systematic approach can simplify the development of the system andensure its acceptance and use. The approach we propose consists of ten stepswhich are shown below. Each of the steps is described in more detail in the pagesthat follow.
the police department produced noevidence that the program was achiev-ing its stated objectives. The academicperformance of program participantsand the incidence of drug use was nobetter than for a comparison group. Theaudit report recommended changing the
objectives of the program to reflect whatit was actually accomplishing: changedknowledge and attitudes about drug useand improved relations between partici-pating youth and the police. The auditwent on to recommend a methodologyfor assessing these outcomes.
2. Define the Purpose
1. Assess the Organization's Readiness
3. Prepare a Policy Statement
4. Develop a Work Plan
5. Initiate Orientation and Training
6. Select Service Areas to be Measured
7. Formulate a Mission Statement, Goals and Objectives
8. Identify Measures
9. Establish a Data Collection, Analysis and Reporting System
10. Monitor and Evaluate
1 2 3
4 5 6 7 8 9 10
11 12 13 14 15 16 17
18 19 20 21 22 23 24
25 26 28 29 30 3127
Improvements?
Problems?
Achievements?
Users?
Information?
Policy Statement
605
10
15
20
2530
35
40
45
50
55
Training forFinance
Managers
6
Step 1: Assess theOrganization’sReadiness
Success in imple-menting aperformancemeasurementsystem dependson the degree to
which an organiza-tion is ready.Readiness meanshaving the rightmix of people,materials, andsupplies in place.
There must be enough time to com-plete the process, and thoseresponsible for carrying out the pro-cess must be committed to followthrough with development and imple-mentation plans. Completion of theOrganizational Readiness Assessmentin Exhibit A can help you determine ifyour organization is ready for perfor-mance measurement.
Step 2: Define the Purpose
The purpose for develop-ing a performancemeasurement systemmust be clearlydefined. Is the objec-tive to improvedecision making,planning, manage-ment, budgeting,
1 2 3
4 5 6 7 8 9 10
11 12 13 14 15 16 17
18 19 20 21 22 23 24
25 26 28 29 30 3127
Improvements?
Problems?
Achievements?
Users?
Information?
Policy Statement
accountability or to achieve some otherpurpose? What are the organizationalproblems or conditions the process isintended to address? Who are thepotential users of the performance dataand their information requirements?Answering these questions will equipyou to design a system that is respon-sive to specific information needs. Andit will assist in building acceptance andsupport for the process and aims ofperformance measurement.
Step 3: Prepare a PolicyStatement
Considerable time and resources areinvolved in implementing performancemeasurement. Therefore, well definedpolicies are necessary to legitimize andguide theprocess.Here aresome of thematters thatshould beaddressed in apolicy state-ment onperformancemeasurement:
• Formallegislativeand management commitment tosupport the project with adequatefinancial and staff resources and touse the resulting information.
• Extent of employee and citizeninvolvement.
• Comprehensive or selected munici-pal service focus.
• Use of in-house staff or outsideconsultants.
• Reporting frequency and format.
• Coordination with financial andpersonnel systems.
A substantial number of “no” answers should suggest caution in proceeding with a perfor-mance measurement system because of the risk of failure. Instead, an effort should be madeto determine what needs to be done to change the “no” answers to “yes.” Several strategiesare available:
• Hold a workshop or retreat for elected officials and senior managers to explore thedevelopment and implementation of performance measurement.
• Conduct technical skill workshops on performance measurement.
• Provide information on the experience of municipalities that have successfully imple-mented performance measurement.
• Establish an employee-management committee to study the feasibility of implementing aperformance measurement system.
• Begin strategic planning that leads to implementation of a pilot performance measure-ment project.
EXHIBIT A—A Performance MeasurementOrganizational Readiness Assessment
Yes No Assessment Criteria
___ ___ 1. Elected officials are committed to performance measurement, willsupport the system philosophically and financially and will use theinformation in policy making.
___ ___ 2. Management is committed to performance measurement, willsupport the effort with adequate financial and staff resources andwill use the information in decision making and managing.
___ ___ 3. Employees and their organizations are receptive to performancemeasurement and will support the effort.
___ ___ 4. Elected officials and management work to be objective and tomake decisions based on facts.
___ ___ 5. Elected officials and management are prepared to take correctiveaction to resolve performance problems identified by performancemeasurement.
___ ___ 6. Elected officials and management work well together and have arelationship based on a high level of trust.
___ ___ 7. The municipality has staff trained in analysis and adequate ana-lytical capability.
___ ___ 8. The accounting, budgeting and information management systemsare capable of supporting performance measurement.
___ ___ 9. The municipality is committed to a strategic and operationalplanning process.
___ ___ 10. There are data collection and measurement efforts currently ineffect.
___ ___ 11. Elected officials and management support public reporting andcustomer feedback.
___ ___ 12. Elected officials and management are willing to implement newmanagement systems and techniques.
PerformanceMeasures
7Step 4: Develop a Work Plan
The fourth step is to develop a writtenwork plan covering project manage-ment, staffing, time table, budget,training, strategy and monitoringcriteria. A well-conceived work plan canprevent costly mistakes and delays andhelp to sell the idea of performancemeasurement to the organization’spolicy makers, managers and employ-ees. The work plan should designate theperson or office responsible for manag-ing the project. Strong interpersonalskills, sufficient authority to resolveconflicts and strong top managementsupport are important factors to con-sider in selecting the project manager.In addition, the manager should begiven adequate time to devote to theproject. A committee consisting ofservice custom-ers (i.e.,customers canbe employees ofother depart-ments of thelocal govern-ment as well asexternal usersof a service)who have aninterest in theservice and itsobjectivesshould beorganized toassist theproject manager. Adequate staff supportis important to the success of a project.The size of the staff will depend on thescope of the project. Personnel assignedshould be competent to assist in theselection of measures, collecting data,analysis and report design. Existingbudget, finance, computer and manage-ment analysis staff may be assignedproject responsibilities.
Outside consultants are another option.They may be used when skilled staff isnot available or the scope of the projectis government-wide. Consultants canprovide valuable assistance in suchtechnical areas as data collection,citizen surveys, form and report designand training. Experienced consultantshave measure data banks and extensive
information on the experience of otherlocal governments that have imple-mented performance measurement.They can also be helpful in trouble-shooting and benchmarking. Factors toconsider in selecting a consultantinclude relevant local government
experience, positive clientreferences and qualifiedstaff. Be careful aboutusing packaged consult-ant systems which maynot fit your needs. Mostimportant, do notbecome over dependenton the consultant.
Maximum staff participation isthe key to a successful system.Your work plan should spellout how you can involve youremployees and their workunits in the development ofthe system. Labor-managementor user committees and em-
ployee surveys are two notablemethods.
Another element of the work plan isthe timetable. A realistic schedule forcompleting the necessary tasks is anecessary project management tool.Establish milestones for each phase of
Training forFinance
Managers
8
Step 5: Initiate Orientation andTraining
Acceptance of perfor-mance measurement isdependent on the
involvement of key users and effectiveorientation and training. Involvementof elected officials, staff and employ-ees in the design and implementationof the system lends credibility, avoidsmisunderstanding and facilitates theireventual ownership of the system.Orientation should cover the purposeof the project, how the data will beused and the steps involved. Devoteparticular attention in the orientationto the possibility employees willassociate performance measurementwith layoffs, work standards andperformance appraisals. Be preparedto answer questions raised by employ-ees such as these familiar ones:
• Why are we doing this?
• What assurance can you give usthat the data will not be misused ormisinterpreted?
the implementation including start-up,objective setting, measure develop-ment, data collection and reporting.Allow sufficient time for fine tuningeach phase. Experience shows thatimplementation of performance mea-surement can take two to five yearsdepending on the scope of the project.Highly recommended is a large, wall-size representation of the work planthat can serve as an easy reference forthe project team when discussingphases of a project and the actualprogress of work in relation to theestablished time lines.
The work plan should also includea project budget. The budget
should include an estimateof costs for staff,
consultant services,training, data
collection, computerservices, surveys and
other supplies andservices. Forecast costs for
two orthree years and factor in anamount for unexpected contingencies.Be realistic in your budget estimates.Consider collecting cost informationfrom other local governments that haveimplemented measurement systems.
Training and orientation is anotherimportant element of the work plan.Outline the type of training to beprovided and identify those to betrained. Include a training scheduleand information on the qualificationsof trainers to be used. Describe theorientation to be used and howprogress will be communicated to thepolicy makers and staff.
The work plan should address thestrategy to be used in selecting activi-ties to be measured. For example, willthe project be city wide or more limitedin scope? A comprehensive city wideapproach can be expensive, timeconsuming and complex. Pilot projectsoffer the advantage of being moremanageable and less risky, less expen-sive and easier to manage. Such pilotsoffer an increased opportunity forsuccess. By introducing performancemeasurement on a smaller scale, thereis greater chance of a successful first
experience and an opportunity toperfect the methodology. Be sure theplan includes a description of thecriteria to be used in selecting activitiesto be measured.
Finally, the work plan should describehow the project will be monitored.Establish performance criteria bywhich to measure the success of theproject. Examples are better qualityinformation, employee and citizensatisfaction, cost savings and docu-mented service improvements. Outlinehow data validity and reliability will beverified. You will find that citizen andpolicy maker confidence in the systemwill be directly related to the quality ofyour monitoring efforts.
PerformanceMeasures
9• Will the data be used to set work
standards?
• If not for these purposes, how willthe data be used?
• What are the incentives for suc-cess? The penalties for failure?
• Will the data be used to appraiseperformance or adjust pay?
• Will the data be used to reduceresources? Eliminate jobs?
• Who will interpret results? Howwill disputes be resolved?
Employee concerns like these can beidentified through group meetings andopinion surveys. Distribution of aninformation brochure is an excellentform of information giving, but it is nosubstitute for question and answeropportunities.
Performance measurement trainingmust be structured to meet the needsof different users. Examples of trainingtopics are:
• Performance measurement con-cepts and terminology
• Steps involved
• Writing objectives
• Selecting measures
• Data collection, analysis andreporting
• Uses of data
• Barriers
• Benchmarking
• Experiences of other local govern-ments
The use of handouts, videos andexercises can enhance the learningvalue of the training. During imple-mentation, briefing sessions,newsletters and other feedback meth-ods can help to retain employeeinterest and support.
Step 6: Select Service Areas tobe Measured
Thoughtful selection of service areas forinitiating performance measurement iscritical to the success of a project. Somelocal government services are easier tomeasure than others. Street and vehiclemaintenance, water supply, seweragedisposal, solid waste collection, parksand recreation, transportation, policepatrol, and fire suppression are servicesthat are relative easy to quantify andmeasure. Soft services such as socialservices are more difficult to measurebecause results are unpredictable,difficult to quantify and take a long timeto achieve.
There is evidence that a particularservice area is a good candidate for aperformance measurement projectwhen the following criteria are met:
• The service is readily susceptible tomeasurement.
• Measurement data are currentlybeing collected, and there isorganizational experience in usingquantifiable approaches.
• Written objectives have beenprepared.
Training forFinance
Managers
10
• The staff is interested and support-ive.
• The service is labor intensive andinvolves significant resources andcommunity impact.
• Elected officials and citizens haveexpressed high interest.
• Existing data indicates serviceproblems and a need for improve-ment.
• A new service or program is beinginitiated.
• Other private, public or nonprofitorganizations provide the service.
• Services have been extensivelymeasured by other local govern-ments and comparable data isavailable.
• Substantial measurement informa-tion is available in the professionalliterature.
PerformanceMeasures
11
Reflection
Using the above and other criteria, identify services in your local government thatare readily susceptible to measurement and explain your rationale for the selection.
Services __________________________
__________________________________
__________________________________
__________________________________
Services __________________________
__________________________________
__________________________________
__________________________________
Services __________________________
__________________________________
__________________________________
__________________________________
Services __________________________
__________________________________
__________________________________
__________________________________
Services __________________________
__________________________________
__________________________________
__________________________________
Rationale _________________________
__________________________________
__________________________________
__________________________________
Rationale _________________________
__________________________________
__________________________________
__________________________________
Rationale _________________________
__________________________________
__________________________________
__________________________________
Rationale _________________________
__________________________________
__________________________________
__________________________________
Rationale _________________________
__________________________________
__________________________________
__________________________________
Training forFinance
Managers
12
Step 7: Formulate a MissionStatement, Goals andObjectives
Goals and objectives provide directionand can create enthusiasm for theprovision of high quality public ser-vices. Normally, goals and objectivesare anchored by an organization ordepartment’s mission statement. Takentogether, the three constitute a hierar-chy of purpose for organizational unitsas shown in the following diagram.
The MISSION STATEMENT is the foundationfor every government function orservice. Goals and objectives, on theother hand, are derived from themission of an organization or organiza-tional unit. As critical elements of aperformance measurement system, allthree work together to provide:
• A basis for programming decisionsby creating explicit expectations forperformance against which accom-plishments can be measured andevaluated.
• A long-range orientation forreviewing the allocation of re-sources to priority programs andprojects and their impact onalleviating major municipal prob-lems.
• A clear understandable documen-tation of the need for andcommitment to continuous im-provement.
• A basis for results-oriented budget-ing.
• A focus that encourages everyoneto work together.
Mission Statement
Such state-ments definewhy an agencyor programexists, what itdoes, why itdoes it and forwhom it doesit. Missionstatements describe the overall andunique purpose of the agency and itsservices and furnish a basis for settinggoals and objectives. Here are ex-amples of mission statements:
• The mission of the Fire Departmentis to provide fire protection, emer-gency medical service and otheremergency services to all citizensfor the purpose of protecting lives,property and the environment.
• The mission of the Finance Depart-ment is to provide accounting,budgeting, financial planning, debtand cash management and pur-chasing services to city staff andthe public for the purpose ofprotecting public assets and sup-porting the full range of servicesprovided by the municipality toachieve its objectives.
• The mission of the Parks andRecreation Department is to pro-vide diverse recreational andcultural services to all citizens inorder to assist their development.
• The Technical Services Departmentprovides engineering, maintenanceand operational services for publicimprovements and equipment tomunicipal agencies and the publicfor the purpose of enhancing thequality of life for residents, busi-nesses and visitors.
605
10
15
20
2530
35
40
45
50
55
PerformanceMeasures
13
Reflection
Draft a mission statement for your organizational unit or program by looking at its func-tioning from the customer’s point of view.
______________________________________________________________________________
______________________________________________________________________________
______________________________________________________________________________
______________________________________________________________________________
______________________________________________________________________________
______________________________________________________________________________
______________________________________________________________________________
______________________________________________________________________________
______________________________________________________________________________
______________________________________________________________________________
______________________________________________________________________________
______________________________________________________________________________
______________________________________________________________________________
Goals
A GOAL is a general pur-pose statementdescribing what alocal governmentfunctional unit wouldlike to accomplish in thefuture. It is a purposefulstatement that centers on communityconcerns and aspirations based uponcurrent knowledge and values. Goalstatements serve as the basis fordeveloping directly related, measurableand shorter-range objectives. Generally,goals are broad in scope, timeless,subjective and related to importantcommunity needs. Here are a fewexamples of local government goalstatements:
Sample Goal Statements• Police: The safety of all persons
and property of the municipality.
• Recreation: Opportunities forcitizens to relax, enjoy nature,exercise, socialize, learn orpractice artistic or athletic skillsand witness or participate incultural events.
• Transportation: A traffic systemfree of accidents, congestion andnegative environmental impacts.
• Finance: Timely and accuratefinancial support services forcouncil and agency decisionmaking.
Training forFinance
Managers
14
Sample Objective statements• Achieve by September 1 an average response time of four minutes to emer-
gency police calls and an average response time of seven minutes to urgentassistance police calls.
• Invest 95% of the municipality’s available daily cash balance in accordancewith the city’s investment policy.
• Maintain minute files so that they can be retrieved within five minutes 85%of the time at a cost of $13.39.
• Maintain all turf and ground cover areas such that turf is green, healthy andno more than 2 1/2 inches in height for 90% of the area and ground cover is90% weed free.
• Increase the level of dog licensing this year 7% to 800 dogs per year.
• Increase the overall customer service rating for the public works departmentsfrom 82% to 86% during the fiscal year.
• Maintain vehicle downtime to 5% during the next six months.
• Maintain the percentage of invoices paid within 30 days to 98% during thisfiscal year.
Objectives
Mission and goals are operationalizedthrough objectives which describe howto measure specific goal- and mission-related accomplishments. An OBJECTIVE
is a statement of desired or plannedaccomplishment that is measurablewithin a given time.
Objectives break broad goals intonarrower more operational terms andserve as the basis for developingperformance measures. Well-draftedobjective statements are:
• Results oriented. They focus onuseful results, not the process orhow to achieve them.
• Specific and measurable. Theydefine in quantitative and verifiableterms what is to be accomplished.
• Time specific. They predict whenthe results will be realized.
• Realistic and attainable. They canbe achieved within a reasonabletime and cost.
• Understandable and challenging.They can be understood by those
responsible for implementation andprovide motivation for successfulperformance.
• Relevant to the managementinformation system. They provide asubstantive basis for monitoringand evaluation and encourageregular review and revision.
605
10
15
20
2530
35
40
45
50
55
PerformanceMeasures
15
Checklist for preparing objective statements___ Establish mission statements and goals before formulating objectives
___ Be sure objectives relate to the goals and mission statements
___ Focus on a single issue per objective
___ Limit the number of objectives to a critical few (3-7)
___ Use an action verb-noun structure to write objectives
___ Allow for a margin of error and take note of adverse consequences andmultiple objectives such as improving the flow of traffic and reducingaccidents
___ Enable those affected by the objectives to participate in their determina-tion
___ Ensure that objectives at one level of the organization are compatiblewith those at a higher level
___ Be sure objectives are compatible with each other
___ State in a way that encourages the consideration of alternative ways ofachieving them
___ Continually review and revise to ensure they reflect planned actions andchanging conditions
___ Provide incentives for the accomplishment of the objectives
___ Consider the financial resources needed to achieve the objectives
___ Establish accountability for achieving the results
___ Make the objectives reasonable but challenging
___ Support the objectives with an action plan
Training forFinance
Managers
16
Reflection
Evaluate the following objectives against the criteria stated above for well-draftedobjective statements and rewrite them to meet the criteria as fully as possible.
To sweep every street once every twoweeks.
To correct all building code violationsduring this fiscal year.
To fill all supply requisitions in atimely manner.
To reduce complaints this year throughimproved training and better informa-tion transmittal.
__________________________________
__________________________________
__________________________________
__________________________________
__________________________________
__________________________________
__________________________________
__________________________________
__________________________________
__________________________________
__________________________________
__________________________________
__________________________________
__________________________________
__________________________________
__________________________________
__________________________________
__________________________________
__________________________________
__________________________________
__________________________________
__________________________________
__________________________________
PerformanceMeasures
17Step 8: Identify Measures
After you have prepared a missionstatement, goals and objectives you areready to identify performance measureswhich quantify the resources used,services provided, results and cost ofservices. PERFORMANCE MEASURES assistyou in determining the extent to whichobjectives are accomplished. You canestablish measures before setting
objectives in order to obtain a betterunderstanding of a service. But theinformation will not tell you anythingabout the desired level of performance.This is why specific objectives arerecommended as a foundation fordeveloping performance measures.
For our purposes, we will concentrateon four types of performance mea-sures:
• Input measures
• Output measures
• Outcome measures
• Efficiency measures
Outcome measurement can be chal-lenging because:
• Special data collection proceduressuch as surveys, trained observerratings and technology may berequired.
Types and Descriptions of Measures
INPUT Show the resources (financial, personnel, materials, equipment) used toprovide a service such as expenditures budgeted, employee hours, quan-tity of materials and equipment hours. Input measures are easy toidentify and are often found in budgets and management reports. Theydo not, however, tell you anything about what is being achieved with theresources.
OUTPUT Show the workload accomplished such as tons of garbage collected, acresof parks mowed, potholes repaired, licenses issued and building inspec-tions completed. These measures are commonly collected but indicatelittle about the quality and cost of the work. They are subject to a varietyof interpretations.
OUTCOME Report the results and quality of the service provided. These measuresdescribe the extent to which services are accomplishing their intendedmission, goals and objectives. Examples of outcome measures includecitizen and user satisfaction ratings, crime clearance rates, percentage ofstreet lights out of service, and percentage of roads in excellent or goodcondition. A special type of outcome measure, sometimes called a qualitymeasure, addresses standards which are important to customers such astimeliness, accuracy, availability, safety, courtesy, convenience, compli-ance with regulations and knowledgeable, friendly and responsive service.Exhibit B provides additional examples of outcome measures.
EFFICIENCY Show the cost of outputs and outcomes in terms of dollars or employee hoursper unit. Examples of efficiency measures include cost per ton of garbagecollected, potholes repaired, complaints handled, inspections completed,crimes cleared and employee hours per crime cleared. Efficiency measures,evaluated over time, provide evidence of productivity trends.
Training forFinance
Managers
18
Additional guidelines to help you select useful measures1. Develop multiple measures (input, output, outcome and efficiency) for the
same service and objective. A set of measures is necessary to give a completepicture of performance.
2. Balance the measures so that the effect of improving any one or two isweighed in relation to the impact on the others.
3. Select measures for which data are readily available.
4. Involve those who use and collect the data in the development of the mea-sures. They can identify factors which are not within their control and causeunanticipated or unwanted results.
5. Consider customer requirements.
6. Review and revise measures when the mission and objectives change and ifthey do not adequately measure.
7. Limit the number of measures to a vital few. Too many confuse users anddistract from key ones.
8. Obtain information on measures used by other local governments.
9. Select measures that collectively provide the most practical and useful infor-mation for critical activities which are essential to carrying out the coremission.
10. For knowledge based services, measure performance in terms of deadlinesand cost targets met, quantity of work produced, extent to which work mustbe revised or corrected and extent to which recommendations are accepted.
• Results may be difficult to measurebecause they cannot be quantified.Information may not be available,and the time for validation of manyoutcomes can be several years.These problems are common inmeasuring social, education,health, environmental and otherquality of life services and knowl-edge-based staff services such asengineering, planning, legal andmanagement.
• Outcomes may be subject todiffering interpretations and af-fected by uncontrollable factorssuch as weather, populationdensity, absence of crime witnessesand changes in demand.
• Outcomes may require the coordi-nated efforts of numerous public,private and nonprofit agencies.
• It is often difficult to link theresults achieved with the servicesprovided.
• Unintended adverse impacts ofservice are often not obvious. Forexample, an increase in law en-forcement efforts may result in anincrease in police harassmentcomplaints.
Applying the Criteria for DevelopingPerformance Measures
Take a minute, now, to apply thesecriteria. Exhibit B, as mentioned earlier,gives examples of performance measuresfrom several common operational areasof a local government. Select a couple ofexamples from the list that might beappropriate for use in your own localgovernment. Then, using the criteria inthe box on page 20, rate the degree towhich each proposed measure meetseach of the 13 using the following scaleand points.
High Medium Low5 points 3 points 1 point
Exhibit C—Performance Measures
PerformanceMeasures
19
1. Has a strategic plan been devel-oped, regularly reviewed andupdated?
2. Have mission statements, goals andresults oriented objectives beenprepared and linked to the strategicplan?
3. Have input, output, outcome andefficiency measures been estab-lished?
4. Are outputs and outcomes bench-marked against objectives,performance targets, standards andalternative providers?
5. Are performance reports preparedon a timely basis and do theyinclude comparative, variance andexplanatory information?
6. Does performance informationmeet the needs of stakeholders(elected officials, management,employees, public, other users) byproviding understandable, accurate,relevant and useful information?
EXHIBIT D—Evaluation Checklist, Performance Measurement System
7. Is the system regularly monitoredand audited to ensure data com-pleteness, accuracy andreliability?
8. What performance improvements(service delivery, customersatisfaction, cost reduction, etc.)have resulted from the system?
9. What actions have been taken toimprove and refine the system?
10. Is the system integrated with theplanning, budgeting, performancemanagement, financial manage-ment and managementinformation systems?
11. What percent of the municipalityis covered by the system?
12. How do elected officials, manag-ers, employees and customers ratethe system?
Worksheet can be used to tally thisinformation. Total the points for anoverall numerical score for eachmeasure. The highest possible score is65 and the lowest is 13.
Exhibit D— Evaluation Checklist for thePerformance Measurement System isalso provided.
Considerable information on perfor-mance measures is available in theliterature. The British Audit Commissionhas established a nationwide set ofmeasures and publishes annual reportscomparing performance among localauthorities and police forces. In theSlovak Republic, the Association forPublic Utility has developed basic indicesof member organizations for technicalservices including local communications,public green spaces, public lighting,waste management, funeral and cem-etery services. Resources available in theUnited States include:
• Governmental Accounting StandardBoard, Service Efforts and Accom-plishment Reporting: Its Time HasCome.
• Government Finance OfficersAssociation, The Use Of Perfor-mance Measures in City and CountyBudgets.
• International City/County Manage-ment Association, How EffectiveAre Your Community Services?
Efforts are underway in the UnitedStates to develop standard comparativeperformance measures that can be usedto compare service performance amongdifferent jurisdictions. The InternationalCity/County Management Association isworking with a consortium of 37 largelocal governments while The Innova-tions Groups is working with 120smaller local governments to developsimilar comparative data.
The International City/County Manage-ment Association has available astate-of-the-art, interactive, multimediatraining CD-ROM to help local officialsproduce customized performancemeasurement programs for specificdepartments or projects.
Exhibit C—Performance Measures Worksheet
Service
Objectives
Potential Measures
Selected Measures Data
Strengths
Weaknesses
Out
put
Out
com
e
Effic
ienc
y
Use
fuln
ess
Dat
a av
aila
bilit
y
Valid
ity
Und
erst
anda
bilit
y
Accu
racy
Rel
iabi
lity
Rel
evan
ce
Uni
quen
ess
Tim
elin
ess
Con
trol
labi
lity
Cos
t
Com
plet
enes
s
Com
para
bilit
y
Scor
e
____
____
____
___
____
____
____
___
____
____
____
___
Training forFinance
Managers
20
Criteria for Developing Performance Measures
1. Usefulness. Does the measure provide the users with information needed to makedecisions or take corrective action?
2. Availability of data. Is information for the measure readily available?
3. Validity. Does the measure relate to the service and its objectives? Does it measurethe resources used, output, outcome or cost of the service? Does the measure indi-cate whether there is a need to react to the results of the measure?
4. Clarity. Will the measure be understood by those who collect and use it?
5. Accuracy. Is it feasible to confirm the accuracy of the measure over time?
6. Reliability. If the measurement is repeated, will the results be identical?
7. Relevance to objectives. Does the measure apply to all or most objectives. Does itprovide information to evaluate performance?
8. Uniqueness. Does the measure provide information not covered by any othermeasure?
9. Timeliness. Does the measure provide information before decisions must be madeand in time for corrective action to be taken?
10. Controllability. Does the user have control over the performance of the service?
11. Cost. Will the cost of data collection and analysis exceed the benefits?
12. Completeness. Does the measure provide a complete picture of the service and itsobjectives?
13. Comparability. Can the measure be used for inter-period and inter-jurisdictional and othercomparisons?
Output measure ____________________
__________________________________
__________________________________
__________________________________
__________________________________
__________________________________
__________________________________
Outcome measure __________________
__________________________________
__________________________________
__________________________________
__________________________________
__________________________________
__________________________________
Efficiency measure _________________
__________________________________
__________________________________
__________________________________
__________________________________
__________________________________
__________________________________
__________________________________
__________________________________
__________________________________
Reflection
Select one of the objectives from the reflection exercise earlier in this essay anddevelop output, outcome and efficiency measures for the objective.
PerformanceMeasures
21Step 9: Establish a Data
Collection, Analysis andReporting System
Now that the performance measurementsystem framework has been laid, it istime to move into data collection,analysis and reporting. Availability ofinformation is a critical element ofperformance measurement. A decisionhas to be made about the kind of infor-mation needed to determineperformance results and problems andhow that information can be obtained.Collecting and analyzing information canbe difficult and expensive. Consequently,considerable attention should be given tothese activities. The focus should be ondata that is easily collected and updated,data that is complete, timely, accurate,consistent over time, practical andaffordable. Let’s look at these activitiesone at a time beginning with eight basicsteps in data collection.
What to DoStep 1. Identify the personnel
responsible for data collection.Usually data collection is the respon-sibility of operating agencies whilecompilation, dissemination andverification is the responsibility of acentral office under the chief execu-tive. Those involved in datacollection should be encouraged toparticipate in the selection of thedata and the design of the collectionprocedure.
Step 2. Prepare clearly written datadefinitions which will be easilyunderstood by the users of theinformation and are neither ambigu-ous nor misleading.
Step 3. Design preprinted datacollection forms. Use a standardformat which will simplify collectionand aid comparability. Considerreader reading level and possiblelanguage barriers.
Step 4. Draft instructions on how touse the collection forms and conduct
Tips for collecting information• Keep paperwork simple and to a minimum.
• Establish baseline data which reflects the current status and analyze to settargets.
• Emphasize accuracy, timeliness, completeness, consistency and comparability.
• Limit the number of revisions to the definitions and collection procedure whichundermine comparability and frustrate those collecting the data.
• Involve staff in designing the collection system.
• Collect data for the most important objectives.
• Be tolerant of early collection deficiencies.
Training forFinance
Managers
22
training using samples of completedforms. Determine the need toprovide multi-lingual instructions.Pretest the forms and instructions toidentify problems and ensure theyare simple and easy to use.
Step 5. Establish a schedule andfrequency for collecting the informa-tion. For best results, collect the dataon a continuing basis.
Step 6. Computerize data collectionas much as possible. Considercomputer requirements in designingthe collection forms. Computers andinformation processing technologygreatly facilitate organizing, process-ing and reporting data.
Step 7. Select the appropriate datacollection technique including reviewof records, interviews, question-
naires, surveys, trained observersand use of technology. Existingrecords such as budgets, workorders, time sheets, citizen surveys,complaints, requests for service, andprogram statistics are a source ofuseful information on workload,duration of time to respond orcomplete work, completion dates,service interruptions and customersatisfaction (see box below for moredetails on data collection tech-niques.)
Step 8. Periodically review andmonitor the collection procedures toreduce errors due to incorrectlyrecorded data, transposed digits andarithmetical errors. Verify accuracy,sampling methods and collectionprocedure. Document problems withthe collection process and work toresolve them.
Data Collection Techniques
Use of existing data An inventory of existing data can simplify the collec-tion process and avoid needless repetition orduplication of effort.
Citizen or customer surveys Citizen or customer surveys are a useful way ofcollecting service outcome and quality information.Such surveys also tell why service is not being used.Surveys are a source of information on satisfaction,perception of community problems and preferredfinancing methods. They can address services ingeneral or specifically. Decisions relating to surveymethods, i.e., telephone, mail or face to face inter-views, questionnaire design, and samplingtechniques, are an integral part of survey methodol-ogy. Care must be exercised to guarantee the integrityof the results. Surveys can be expensive, time con-suming, misleading, and embarrassing. In order tomaximize the benefits, consideration should be givento obtaining professional assistance from consultantsor universities.
Observation Trained observers are employees who conduct sys-tematic visual ratings of such conditions as roadcleanliness or rideability and park and facility mainte-nance. The ratings are based on pre-specified ratingcriteria including photos, checklists, precise writtendefinitions and a numerical grading system. Littlejudgment or interpretation is required from theobservers. Care must be taken to periodically replicatea small sample of observations to minimize bias.
Measurement technologies Equipment such as noise, air quality and road rough-ness meters can be used to provide measurement ofconditions.
PerformanceMeasures
23
Data Analysis
Meaningful performance measurementdepends on the quality of the dataanalysis. Raw data doesn’t tell youmuch. It is extremely important tounderstand what the data reveal and donot reveal. Analyzing whatwent right and what wentwrong will help youprepare an actionplan to correct orimprove perfor-mance. Considerableinsight can beobtained by subject-ing current performanceto a number of comparisonsincluding:
1. Comparisons with previouslyestablished objectives.
Such a comparison will show the extentto which the objectives have beenachieved; if the objectives are over-stated, unrealistic or too costly; if theobjectives are understated; if resourcesneed to be reallocated; or if there is aneed for improvement.
2. Comparisons with a past perfor-mance period (month, year).
This type of comparisonindicates trends andshows whether theperformance is improving,declining or staying thesame.
3. Comparisons withperformance targetssuch as plannedversus actual.
Performance targets can be establishedas part of the budget, personnel ap-praisal or management by objectiveprocess. Targets are based on dates,duration of time, quantity and level ofcustomer satisfaction.
4. Comparisons with technicallydeveloped standards.
Such standards may have been estab-lished by work measurement,professional organizations, standard
setting bodies, or other levels ofgovernment. Examples include air,water, vehicle maintenance and ac-counting standards.
5. Inter-jurisdictional comparisons.
These are comparisons with otherjurisdictions having similar demo-
graphic, social and economiccharacteristics and providing
comparable services.Problems associated
with such compari-son includeavailability of dataand different key
characteristics, levels ofservice, objectives and
climatic conditions. Ministriesoften collect comparative data onbudget expenditures (gross and percapita) and staffing levels.
6. Benchmarking.
This is a process of measuring yourorganization’s performance against best-in-the-class organizations to improveservices, operations or cost position.BENCHMARKING helps to identify betterresults and more efficient methods andto incorporate them into an improvementplan. Best management practices refer tothe processes, practices and systems
identified in public andprivate organizations thatare performed exception-ally well and are widelyrecognized as improvingan organization’s perfor-mance and efficiency inspecific areas. You canapply benchmarking to awide range of processes,
such as accounting, maintenance ser-vices, cash management, training andcrime control. Australia makes extensiveuse of benchmarks at the local level.Canada emphasizes best practices.
Especially important to data analysis isthe need to explain what the data andcomparisons reveal. Variances from thebase line, targets, standards, priorperiods, and best practices should behighlighted and explained. Numbers bythemselves are often difficult to under-stand and may not lead to conclusions.
Training forFinance
Managers
24
Benchmarking—A Seven Step Process1. Identify and document the process you want to benchmark and determine
current performance levels.
2. Research industry trends and literature on the subject. Substantialbenchmarking information is available.
3. Select appropriate organizations for comparison.
4. Collect data from the selected organizations through questionnaires andinterviews.
5. Compare and contrast processes and develop recommendations and anaction plan.
6. Make comparisons among geographical areas and customer client groupssuch as neighborhoods and age groups. These comparisons are very usefulin comparing street cleanliness, crime rates, park facilities, water qualityand road condition by area of the municipality and recreational services byage group.
7. Make comparisons with private and nonprofit organizations providingcomparable services. These comparisons can be the basis for making keydecisions about performing a service in house, contracting it or privatizingit. Examples of comparable services include custodial, data processing, debtcollection, payroll, road maintenance, solid waste collection and water andwastewater services.
If you cannot explain the variances, thedata may be of limited value. Remember,the data will tell what the performanceis, not why it occurred. Additional moredetailed analysis and program evaluationwill be needed to further diagnose theproblems and devise solutions.
Questions to Ask about theData
1. Do the data fit the measures?
2. Are the data too difficult to use?
3. Are the data too hard or expensiveto acquire?
4. Do the measures and/or objectivesneed to be changed?
5. Do the data meet the informationneeds of the users?
6. Is more detailed analysis requiredto interpret the data?
7. Are the data sufficient to explainvariances?
Guidelines for Analyzing theData
1. Develop an analysis checklist
2. Verify data accuracy
3. Establish benchmarks
4. Scrutinize for unintended conse-quences
5. Consider uncontrollable factorssuch as population, climate,economic conditions, terrain andclient characteristics
6. Explain variances
7. Analyze over a sufficient period oftime so fluctuations are considered
8. Disregard or replace data thatfluctuates for no apparent reasonor are inaccurate
9. Establish a baseline
10. Be careful of averages and biasedstatistics
PerformanceMeasures
2511. Recognize limitations in the data
when interpreting results
12. Expect mistakes, misinformationand misunderstandings
13. Encourage a dialog on conclusionsreached from the analysis andestablish a procedure for resolvingdisagreements
Reporting
Once data collection and analysishave been com-pleted,performancereports areprepared. Goodinformation isuseless if no oneunderstandswhat it means.Therefore, it isimportant to presentthe information inways that are easy tounderstand, permit comparison andmeet the user’s information needs andhelp them to draw meaningful conclu-sions. The reporting system shouldshow what was expected, what actu-ally occurred, and account for thedifference.
Report formats vary depending uponthe level of detail desired and theaudience. Program managers willrequire detailed information on inputs,outputs, outcomes and efficiency. Anexception format which only reportswhere actual is above or below ex-pected by a specific percentage oramount is more appropriate for seniormanagers and elected officials. Citizenreports can provide more summarizedinformation on outcome and efficiency,changes over the previous period andactual performance against expected.Reports should provide the reader witha reasonable overall picture of perfor-mance and assist them in assessingservice efforts and casts. Differentgroups of users should be consulted indesigning the report formats and on theusefulness of the reports.
Exemplary reports make extensive useof graphics including graphs, charts,
photos, maps and tables which displaycomparative data. Maps are valuable inpresenting information on such condi-tions as street cleanliness or crimerates by geographic area. Graphs andcharts clearly show trends. Graphics
and color are especially valuable incitizen performance reports whichoften take the form of newspapersupplements and newsletters.
Characteristics of EffectiveReports
1. Compare planned to actual results.
2. Describe the activity, mission andobjectives.
3. Show trends over previous periods.
4. Present a mix of key measures foreach major objective.
5. Highlight variances.
6. Explain in narrative form thereasons for the variances.
7. Describe the improvement orcorrective action to be taken.
In reporting, pay particular attention tothe narrative explanatory informationprovided with the reports. This infor-mation should explain the reasons forvariances, factors that affect the resultsand limitations in the quality of thedata. Factors over which the agencyhas no or little control should beidentified. Changes in data definitions,measures and collection proceduresshould be noted. Finally, proposedimprovement efforts should be de-scribed along with any obstacles toimprovement.
When you draft the explanatory sectionof the report, ask yourself these ques-tions:
Reports
A
B
C
Training forFinance
Managers
26
• Do the objectives continue to beappropriate?
• To what extent are the objectivesbeing achieved?
• Why is the performance improving,getting worse or remaining thesame?
• Is the service competitive?
• Could the service be better pro-vided in an alternative way?
• Should the service be retained,discontinued or modified?
• Are resources being optimallyutilized?
• Are customer service needs beingmet?
• What secondary benefits andunintended adverse consequencesare occurring?
Report frequency also needs to beaddressed and depends on the audi-ence. Monthly reports are usuallyneeded by operating personnel. More
frequent reporting may be requiredwhere backlogs or cost increases canoccur in a short period of time. Quar-terly or semi-annual reports aregenerally sufficient for senior managersand elected officials. Exhibit E—Performance Report is a worksheet thatcan be used to develop a report or canbe used as an internal report. Citizenreports are typically issued on anannual basis. It is important to issuereports promptly within a few daysfollowing the end of the reportingperiod in order to permit the user totake appropriate corrective action asneeded.
In addition to specific performancereports, performance data are usedextensively in budgets, strategic andoperational plans, performance audits,program evaluations, annual financialreports, annual reports, performanceappraisals, citizen newsletters, pressreleases and special managementreports. There is considerable merit tolinking performance data with financialdata in annual budgets and compre-hensive annual financial reports.
Exhibit E—Performance ReportDepartment Division / Program Date
Performance Objectives
Last Year This MonthMeasures
Target
Year-To-Date
Actual TargetVariance Actual Variance Target Actual Variance
Output
Outcome
Efficiency
Resources
Budget
Positions
Last Year This YearExplanation of Variances and Proposed Corrective Action
PerformanceMeasures
27
Reflection
Think about setting up a performance reporting system for one of your pro-grams. What kind of data should you track in relation to your objectives?
Checklist for Evaluating yourPerformance Reports
___ Is information presented in atimely, comprehensive, accu-rate and readable manner?
___ Is the report user friendly?
___ Does the report make imagina-tive use of graphics?
___ Do formats satisfy the infor-mation needs of differentusers?
___ Are variances highlighted?
___ Is narrative explanatoryinformation presented?
___ Is accountability pinpointed?
___ Is an improvement planprovided?
____________________________________________________________________
____________________________________________________________________
____________________________________________________________________
____________________________________________________________________
____________________________________________________________________
____________________________________________________________________
____________________________________________________________________
____________________________________________________________________
____________________________________________________________________
____________________________________________________________________
____________________________________________________________________
____________________________________________________________________
____________________________________________________________________
____________________________________________________________________
____________________________________________________________________
____________________________________________________________________
Training forFinance
Managers
28
Step 10: Monitor and Evaluate
The integrity of a performance mea-surement system is dependent on closemonitoring and fine tuning or possiblycorrective action. Performance mea-surement is a continuous and evolvingprocess which requires constantscrutiny of performance for evidenceof accuracy, completeness and useful-ness.
Independent verification of the dataenhances its credibility. Therefore,periodic independent audits of thesystem are as crucial as financial postaudits. In Great Britain the AuditCommission for Local Authoritiesmonitors local authorities program
performance through value-for-moneyaudits. Local governments in theUnited States utilize elected auditors,internal auditors and external indepen-dent auditors to conduct performanceaudits and to verify performance data.
Close monitoring provides for reworkingobjectives, measures, performancetargets, collection procedures, reportformats and improvement plans inresponse to problems which are identi-fied and changing conditions. Typicalproblems involve unrealistic objectives,measures which do not adequatelymeasure, inaccurate and incompletedata, reports which do not meet theneeds of the users and deficient im-provement plans. Competent andongoing evaluation prevents the processfrom being undermined by distortedand biased information.
The monitoring process should involvethe operating staff, report users andcustomers who should be frequentlycontacted to determine their level ofsatisfaction with the system. In assess-ing the system, use the mission, goaland objective statements, customerfeedback, past trends, comparative dataand spot checks of the collectionanalysis and reporting techniques.
Reflection
What kind of a system would youdevelop to evaluate the effectiveness ofperformance measurement in yourorganization?
__________________________________
__________________________________
__________________________________
__________________________________
__________________________________
__________________________________
__________________________________
__________________________________
__________________________________
__________________________________
__________________________________
__________________________________
Would this include external verifica-tion? How would you create conditionsto assure that evaluation data is usedto make necessary corrections in thesystem?
__________________________________
__________________________________
__________________________________
__________________________________
__________________________________
__________________________________
__________________________________
__________________________________
__________________________________
__________________________________
__________________________________
PerformanceMeasures
29Obstacles
Performance measure-ment can be a timeconsuming, costly anddifficult process. Severalobstacles are oftenencountered in imple-menting a performancemeasurement system.
Management resistance
Management may beconcerned that theelected officials, citizensand media will use theinformation to hold themaccountable for outcomes over whichthey have no control. Managers mayfear that the information will bemisinterpreted and used to assignblame and withdraw resources to cutcosts. Management may view perfor-mance measurement as a passing fador another “make work” exercise andsee no benefit in it. Resistance canproduce counterproductive behaviorsuch as withholding or falsifyinginformation which will have a detri-mental effect on data collection andreporting.
Management resistance can be reducedor eliminated through proper orienta-tion, training, extensive involvement indesign and implementation and incen-tives to use performance data. Thesesteps must be supported by adequateresources, technical assistance andcontinuous feedback on results. Perfor-mance measurement should beintroduced in work areas which aregenerally regarded as well managed. Aprocedure should be made available tothese work units for resolving disputesover data interpretation. Any evidencethat performance measurement hasimproved management, work practicesand relationship inside and outside thework area should be widely publicizedand celebrated.
Employee resistance
Employees are threatened by measure-ment. They perceive negativeconsequences and are skeptical aboutits use and value. Employees often
view measurementas an excuse fordownsizing,increasingworkload andimposing unrea-sonable workstandards orquotas and workrules.
Measurement issometimes viewedby employees asmanagement’sway of punishingthem because they
are not doing their jobs. Employeeresistance based on erroneous informa-tion of this kind can be overcome byorientation and training. Involvementof employees and their organizations inthe design and implementation andrewarding program participation canhelp to allay fears about job loss,standards and adverse effects. Informalmeetings and progress information inemployee newsletters will help keepemployees informed about the benefitsof measurement. Managers must monitorcontinuously to be sure the system isimplemented fairly, equitably andwithout any possibility of favoritism.
Political resistance
Elected officials are often more inter-ested in inputs than outcomes. Theymay resist measurement informationwhich may show that their favoriteprograms are not working. There isalso a fear that the information will beused in a negative way by politicalopponents. Furthermore, electedofficials may be concerned that neces-sary service improvements resultingfrom performance measurement resultsmay not be politically feasible. Electedofficials are also aware that measure-ment data may be manipulated by thestaff or used by citizens for purelypolitical reasons.
The political support of elected officialscan be won by involving them early inthe process. They need to be informedabout the benefits of performancemeasurement and how they can use
Training forFinance
Managers
30
the information to improve the qualityand the political correctness of thedecisions that must be made. Electedofficials should be asked about thekinds of information they want. Theyneed to receive measurement resultsand information on the status ofimprovement plans in a form that isunderstandable without being oversim-plified. And they need to be asked fromtime to time if the information they arereceiving is useful and they wantanything done differently.
Lack of resources.
A substantial obstacle may be the lackof analytical staff, time, funds andinformation technology to implement aperformance measurement system.Installation of a comprehensive systemcan take three to five years. Analyticaland information processing capabilitiesare essential. The cost will depend onwhether an in-house or consultantapproach is used, if training is neededand how quickly the system must beimplemented.
Resource obstacles can be reduced bytraining in-house staff, starting with apilot project and using as much avail-able data as possible. Do not reinventthe wheel! Learn from the experienceof others who have developed success-ful systems.
Data limitations
Lack of information can be a majorbarrier. Historical, cost, employee hour,comparative and benchmark data maynot be available or not suitably organ-ized. It is not uncommon for unit costdata to be unavailable at the activitylevel. Considerable cost may be in-volved in collecting the data.Objectives may not have been preparedor may be unrealistic and contradic-tory. On the other hand, there may betoo much data scattered over a varietyof sources which may not be easilyaccessible.
Data problems can be minimized bythe careful selection of services to bemeasured, use of available data andinitiating the system on a gradual andpilot basis. Modification of the ac-
counting system to collect data on anactivity and cost center basis may beuseful. The introduction of accrualaccounting cost analysis, cost findingand activity based costing will also behelpful. Work order systems are an-other method of accumulating relevantmeasurement data. Effective use ofcomputer technology will facilitate theprocess.
Unrealistic expectations
Performance measurement systemsfrequently fail because of unrealisticexpectations. Public officials viewmeasurement as a panacea. They areprone to expect the system to produceresults overnight and to solve all theirproblems. You must recognize thelimitations of performance measure-ment including these all too commonones:
• The outcome may not occur untillong after the measurement periodcausing managers to believe theyhave failed when success is justaround the corner.
• Measurement doesn’t indicate theextent to which reported outcomesare due to agency efforts ratherthan external factors.
• It is not possible to know the exactrelationship between a service andits impact on the community.
• Not all services are easily measur-able.
• The measures being used may notbe the most relevant measures ofthe accomplishment of an objec-tive.
• The objectives themselves may beunrealistic.
• Measurement may produce short-range benefits that actually canretard progress toward a desiredoutcome such as using up anexcessive amount of limited re-sources to get a quick solution to arelatively insignificant problem.
• Measurement may incorrectly beexpected to provide all the informa-
PerformanceMeasures
31tion needed to formulate andassess policy.
Performance measurement only identi-fies problems and when they occur. Itdoes not explain why performance is atthe level reported or how it can beimproved. The process of measurementis separate from the process of evalua-tion. No single measure will cover100% of the service outcome.
There is no bottom line measure thatindicates the relative benefit of aservice to society. To be successful,expectations must be reasonable andefforts made to be certain everyoneinvolved is aware of measurementlimitations. An analytical capability
Reflection
What obstacles do you anticipate in implementing performance measurement inyour organization and how would you overcome them?
Obstacle __________________________
__________________________________
__________________________________
__________________________________
Obstacle __________________________
__________________________________
__________________________________
__________________________________
Obstacle __________________________
__________________________________
__________________________________
__________________________________
Obstacle __________________________
__________________________________
__________________________________
__________________________________
How Overcome? ___________________
__________________________________
__________________________________
__________________________________
How Overcome? ___________________
__________________________________
__________________________________
__________________________________
How Overcome? ___________________
__________________________________
__________________________________
__________________________________
How Overcome? ___________________
__________________________________
__________________________________
__________________________________
must exist or be developed to assesswhy performance is improving, wors-ening or staying the same. Multiplemeasures should be developed, andobjectives, measures, and improvementefforts modified when appropriate.Realism dictates initiating performancemeasurement in areas conducive tomeasurement. Services that are ingreatest need of improvement in termsof cost, quality and timeliness may bethe best candidates. Practitioners of theart would do well to remember thatstrategy, human and organizationculture factors, not just techniques,shape the success of performancemeasurement.
Training forFinance
Managers
32
Key PointsShown below is valuable advice fromthe experience of public agencies thathave successfully implemented perfor-mance measurement:
• Obtain strong, visible and sus-tained support from the electedofficials and senior management.
• Involve elected officials, manage-ment staff, employees andcustomers in design and implemen-tation. Determine their informationneeds and keep them informed.Clearly define the purpose formeasuring performance.
• Do not reinvent the wheel. Use theextensive body of knowledge andexperience of others availablethrough literature review andpersonal contacts.
• Provide on-going orientation andtraining, technical assistance andadequate resources.
• Think long term. Performancemeasurement is a repetitive trialand error process requiring ad-equate time to develop expertiseand fine tune.
• Introduce gradually. Focus oneasily quantifiable activities whichare strategically important, havehigh customer value and representa source of serious adverse conse-quences if not performedefficiently.
• Develop measures linked to mis-sion statements, goals and serviceobjectives.
• Limit measures to a vital few, thoseessential as information sources fordecision making.
• Develop a set of measures includ-ing inputs, outputs, efficiency andoutcomes which are responsive tomultiple priorities and give acomplete picture of performance.
• Install user-friendly data collectionand reporting procedures andformats.
• Pinpoint accountability for perfor-mance results.
• Audit to ensure the performancedata is complete, accurate, validand credible.
• Publicize the results of perfor-mance measurement internally andexternally.
• Institutionalize performancemeasurement by integrating it withplanning, budgeting, financialreporting and with program man-agement and evaluation.
• Develop service improvement plansthat focus on continuous improve-ment over the long run andestablish performance targets.
• Establish a reporting system thatreflects user needs, providescomparisons, highlights variancesand includes narrative explana-tions.
• Measure individual and groupperformance and focus on short,intermediate and long term out-comes.
• Work toward a climate and organi-zation culture that is accepting ofchange and provide incentives andresources for managers to focus onresults.
• Do not expect miracles. Be patientand tolerant of mistakes. Anticipateconfusion, denial, disbelief andresistance.
PerformanceMeasures
33
A substantial number of “no” answers should suggest caution in proceeding with a perfor-mance measurement system because of the risk of failure. Instead, an effort should be madeto determine what needs to be done to change the “no” answers to “yes.” Several strategiesare available:
• Hold a workshop or retreat for elected officials and senior managers to explore thedevelopment and implementation of performance measurement.
• Conduct technical skill workshops on performance measurement.
• Provide information on the experience of municipalities that have successfully imple-mented performance measurement.
• Establish an employee-management committee to study the feasibility of implementing aperformance measurement system.
• Begin strategic planning that leads to implementation of a pilot performance measure-ment project.
EXHIBIT A—A Performance MeasurementOrganizational Readiness Assessment
Yes No Assessment Criteria
___ ___ 1. Elected officials are committed to performance measurement, willsupport the system philosophically and financially and will use theinformation in policy making.
___ ___ 2. Management is committed to performance measurement, willsupport the effort with adequate financial and staff resources andwill use the information in decision making and managing.
___ ___ 3. Employees and their organizations are receptive to performancemeasurement and will support the effort.
___ ___ 4. Elected officials and management work to be objective and tomake decisions based on facts.
___ ___ 5. Elected officials and management are prepared to take correctiveaction to resolve performance problems identified by performancemeasurement.
___ ___ 6. Elected officials and management work well together and have arelationship based on a high level of trust.
___ ___ 7. The municipality has staff trained in analysis and adequate ana-lytical capability.
___ ___ 8. The accounting, budgeting and information management systemsare capable of supporting performance measurement.
___ ___ 9. The municipality is committed to a strategic and operationalplanning process.
___ ___ 10. There are data collection and measurement efforts currently ineffect.
___ ___ 11. Elected officials and management support public reporting andcustomer feedback.
___ ___ 12. Elected officials and management are willing to implement newmanagement systems and techniques.
Training forFinance
Managers
34
Technical Services
• Road smoothness as measured by aroughness meter.
• Pavement distress indicatorsmeasured by visual surveys thatrelate to maintenance performance.
• Percentage of streets rated accept-ably clean.
• Average customer satisfactionrating.
• Percentage of days that environ-mental standards are met.
Parks and Recreation Services
• Percentage of users rating theservice as satisfactory or unsatis-factory.
• Percentage of citizens who livewithin/not within, 15/30 minutestravel time of a city park or recre-ation facility.
• Number and percent of acts ofvandalism repaired within 3 daysof receipt of written work order.
• Percent of customer surveys whichindicate that park users are satis-fied with park maintenance.
• Percent of acres of turf maintainedmeasured by visual surveys thatmeet established quality standards.
Fire Services
• Percentage of citizens ratingperformance satisfactory.
• Actual cost of fire losses.
• Average response tire under fiveminutes.
• Average time to control fires.
• Percentage of fires preventable byinspection or education.
EXHIBIT B—Illustrative Outcome Measures
Economic Development
• Number and percentage of firmsreceiving assistance that located injurisdiction and that felt thatassistance contributed to theirlocation decision.
• Amount of added tax revenuesrelating to assisted firms thatlocated in the jurisdiction.
• Percentage of loans that are cur-rently in default.
• Percentage of clients rating thelength of time of processing theirloan application as appropriate.
• Percentage of firms locating else-where for reasons over whichagency had some influence.
Finance Services
• Percentage of invoices paid within30 days.
• Percent of accounting transactionsprocessed with less than 5 percenterror rate.
• Percent of accounts receivable over30 days.
• Percent of monthly budget reportscompleted five days after close ofreporting period.
• Percent of departments ratingservice as excellent, good or poor.
Computer Services
• Percentage of time the system isdown.
• Percentage of time deadlines arenot met.
• Average turnaround time.
• Average time to respond to com-plaints.
• Percent of departments ratingservice as excellent, good or poor.
PerformanceMeasures
35
Exh
ibit
C—
Per
form
ance
Mea
sure
s W
orks
hee
t
Serv
ice
Obj
ectiv
es
Pot
entia
l Mea
sure
s
Sele
cted
Mea
sure
sD
ata
Stre
ngth
s
Wea
knes
ses
Output
Outcome
Efficiency
Usefulness
Data availability
Validity
Understandability
Accuracy
Reliability
Relevance
Uniqueness
Timeliness
Controllability
Cost
Completeness
Comparability
Score
_______________
_______________
_______________
Training forFinance
Managers
36
1. Has a strategic plan been devel-oped, regularly reviewed andupdated?
2. Have mission statements, goals andresults oriented objectives beenprepared and linked to the strategicplan?
3. Have input, output, outcome andefficiency measures been estab-lished?
4. Are outputs and outcomes bench-marked against objectives,performance targets, standards andalternative providers?
5. Are performance reports preparedon a timely basis and do theyinclude comparative, variance andexplanatory information?
6. Does performance informationmeet the needs of stakeholders(elected officials, management,employees, public, other users) byproviding understandable, accurate,relevant and useful information?
EXHIBIT D—Evaluation Checklist, Performance Measurement System
7. Is the system regularly monitoredand audited to ensure data com-pleteness, accuracy andreliability?
8. What performance improvements(service delivery, customersatisfaction, cost reduction, etc.)have resulted from the system?
9. What actions have been taken toimprove and refine the system?
10. Is the system integrated with theplanning, budgeting, performancemanagement, financial manage-ment and managementinformation systems?
11. What percent of the municipalityis covered by the system?
12. How do elected officials, manag-ers, employees and customers ratethe system?
PerformanceMeasures
37
Exh
ibit
E—
Per
form
ance
Rep
ort
Dep
artm
ent
Div
isio
n /
Pro
gram
Dat
e
Per
form
ance
Obj
ectiv
es
Last
Yea
rTh
is M
onth
Mea
sure
s
Targ
et
Year
-To-
Dat
e
Actu
alTa
rget
Vari
ance
Actu
alVa
rian
ceTa
rget
Actu
alVa
rian
ce
Out
put
Out
com
e
Effic
ienc
y
Res
ourc
es
Bud
get
Pos
ition
s
Last
Yea
rTh
is Y
ear
Expl
anat
ion
of V
aria
nces
and
Pro
pose
d C
orre
ctiv
e Ac
tion
Training forFinance
Managers
38
Glossary
BENCHMARKING A process of measuring your organization’s performanceagainst best-in-the-class organizations to improve services,operations or cost position.
EFFICIENCY MEASURES Show the costs of outputs and outcomes in terms of dollars oremployee hours per unit.
GOAL A general purpose statement describing what a local govern-ment functional unit would like to accomplish in the future.
INPUT MEASURES Show the resources (financial, personnel, materials, equip-ment) used to provide a service.
MISSION STATEMENT The foundation for every government function or service. Itdescribes the overall and unique purpose of the agency and itsservices and furnishes a basis for setting goals and objectives.
OBJECTIVES A statement of desired or planned accomplishment that ismeasurable within a given time.
OUTCOME MEASURES Reports the results and quality of services provided. Thesemeasures describe the extent to which services are accomplish-ing their intended mission, goals, and objectives.
OUTPUT MEASURES Show the workload accomplished.
PERFORMANCE MEASURES Quantify the resources used, services provided, results and costof services. Used to determine the extent to which objectivesare accomplished.
PERFORMANCE A coordinated method or plan for determining how efficientlyand effectively local government services are being deliveredand objectives being met.
MEASUREMENT SYSTEM
