The Riskiness of Public Sector Performance Measurement: A Review and Research Agenda

Financial Accountability & Management, 30(3), August 2014, 0267-4424

The Riskiness of Public SectorPerformance Measurement: AReview and Research Agenda

SURESH CUGANESAN, JAMES GUTHRIE AND VEDRAN VRANIC∗

Abstract: Public sector performance measurement (PM) practices can be risky. Itis imperative that research engages with the riskiness of PM and how this might bereduced. To help move towards a less risky state of public sector PM, where benefitsoutweigh negative potential, this paper reviews studies on PM risk to ascertain whatexisting research indicates about (a) the risks of PM (b) the conditions in whichthe risks of PM are more likely to manifest, and (c) the approaches that mitigatethe risks of PM. Based on this, the paper outlines a way forward for public sectorPM research. Overall the paper makes a number of contributions. First, it bringstogether a disparate and fragmented field in reviewing studies examining the risksof PM. Second, the paper reframes the research question that public sector PMrisk research and public sector PM studies more broadly should engage in. Third,specific suggestions are offered in terms of how future research might proceed ina coordinated manner, examining and informing practice where PM is part of anintegrated and enabling control system for strategising and managing in the publicsector, and where citizen-centric PM is utilised as part of multiple evaluation modesto more effectively support accountability to external constituents.

Keywords: risk, performance measurement, public sector

INTRODUCTION

Initiatives to introduce, refine and/or expand public sector performancemeasurement (PM) regimes show no signs of abating (Lapsley, 2008). WhilePM can deliver numerous benefits (Osborne and Gaebler, 1993), it also carries

∗The first author is from the University of Sydney Business School. The second author isfrom the Department of Accounting and Governance, Macquarie University; the Departmentof Management, University of Bologna; and Honorary Professor, the University of SydneyBusiness School. The third author is from the Faculty of Business and Enterprise, SwinburneUniversity of Technology. The authors gratefully acknowledge the helpful comments of theanonymous referees and the editors of the issue.

Address for Correspondence: Suresh Cuganesan, The University of Sydney Business School,University of Sydney, NSW, Australia.e-mail: [email protected]

C© 2014 John Wiley & Sons Ltd, 9600 Garsington Road,Oxford OX4 2DQ, UK and 350 Main Street, Malden, MA 02148, USA. 279

280 CUGANESAN, GUTHRIE AND VRANIC

with it significant potential for unintended negative consequences. PM practicescan be ‘risky’ and potentially a fatal remedy (Power, 2004a), causing scandalsand so called ‘perverse outcomes’ in the public sector.

As public sector research increasingly concerns itself with issues of risk itis imperative that we do not overlook the riskiness of PM and how this mightbe reduced.1 We argue this for two main reasons. First, examples of PM risksmanifesting in the public sector are many. Given this, it is reasonable to holdthe position that we as a research community have not done enough to provideinsight on how the riskiness of PM might be best managed. Second, issues of riskand PM can be linked, as Power (2004b) also notes: ‘the concept of risk is beingenrolled in a new focus on outcomes and performance’ (p.13, emphasis in original).Hence we argue that public sector research needs to consider how less risky PMpractices might be attained.

In achieving a less risky state of public sector PM we need to establish thecurrent state of knowledge before mapping a way forward. Thus we pose in thispaper the following questions: What does existing research tell us about (a) the risksof PM (b) the conditions in which the risks of PM are more likely to manifest, and (c) theapproaches that mitigate the risks of PM? Reflecting on the answers to these questions,we articulate an agenda for future research that, hopefully, moves us towardsa state where the benefits of public sector PM practices outweigh its negativepotential.

The paper is structured as follows. The next section explains our conceptualapproach to the review while the third section outlines our method. The fourthsection presents the results of the review of prior research organised intodifferent risk categories while in the fifth section we outline a research agendato guide the way forward. We conclude our paper by discussing both the paper’scontributions and its limitations.

REVIEWING THE RISKINESS OF PM IN THE PUBLIC SECTOR

Recent literature reviews either wholly or partly on the topic of public sector PMcall for multi-theoretical perspectives and diverse research methods (van Heldenet al., 2008; and Goddard, 2010), a focus on developing and evaluating newtechniques and studies that examine successful implementation and providingguidelines for PM practice (van Helden and Northcott, 2010; and Jackson, 2011).Collectively, these reviews suggest the type of objectives that PM researchshould pursue and the manner in which PM research should occur. We seekto complement these broad prescriptions with specific suggestions (see the fifthsection) for future research on the riskiness of PM.

Focusing specifically on PM and risk, we were interested in typologies ofPM risks developed by prior studies. A search revealed sporadic efforts withthe more notable of these comprising lists of thirteen dysfunctional effects(Bouckaert and Balk, 1991), eight unintended behavioural consequences (Smith,1995) and seven risks of PM (de Bruijn, 2002).2 Examining the accompanying

C© 2014 John Wiley & Sons Ltd

THE RISKINESS OF PUBLIC SECTOR PERFORMANCE MEASUREMENT 281

explanations to each category mentioned on these lists revealed significantoverlap in categories both across and within lists. Four distinct categoriesemerged from this comparison that related to the negative effects of PMidentified by these studies.

The first risk category relates to misalignment between PM systems andpublic sector strategies and goals (Bouckaert and Balk, 1995). This leads tomeasurement myopia at the expense of long-term outcomes, an emphasis onlocal goal achievement at the expense of broader global objectives, and a focus onmeasured performance dimensions to the detriment of unmeasured ones such assystem responsibility and important inter-organisational collaboration (Smith,1995; and de Bruijn, 2002). The second risk category comprises PM-inducedgaming, where behavioural changes occur to maximise reported performance atthe expense of, or without any corresponding increases in, actual performance(Bouckaert and Balk, 1995). Changes in behaviour around target thresholds,avoidance of ratcheting effects, switching effort and cherry-picking servicedelivery recipients are all examples of gaming (Smith, 1995; and de Bruijn,2002). Also included are data manipulation and the presentation of data to createperformance impressions that are more favourable than actual performancelevels (Bouckaert and Balk, 1995; and Smith, 1995).

A third risk category relates to PM restricting flexibility and blockinginnovation. Smith (1995) refers to this as ossification while de Bruijn (2002)notes that PM can reward the constant reproduction of existing services andcause the ignoring of change imperatives. The fourth risk category involvesdistorted or inaccurate representations of performance through PM. Thisderives from the general difficulty of measuring public service performancein quantitative fashion. Public services are complex and performance is multi-dimensional, making it difficult to measure performance in quantitative fashion(Bouckaert and Balk, 1995; and Smith, 1995). At the least it requires extensiveresourcing to be done well, and can cause distortions when comparing acrosspublic sector agencies (de Bruijn, 2002).

Hence a review of prior work establishing typologies of PM risks allows theidentification of four categories: misalignment between PM and public sectorgoals, PM-induced gaming, PM restricting flexibility and/or blocking innovation,and, PM systems misrepresenting performance. However, despite the presence ofthese typologies, their explicit utilisation and/or further development is missingin subsequent studies on the riskiness of PM. Furthermore, there is no suggestionthat the items proposed in these lists are collectively exhaustive. Thus we usethe four categories as broad sensitising concepts (Blumer, 1954) whilst alsoremaining open to other risk types examined by the literature that we review.In the next section we outline our method for reviewing prior research.

METHOD

For space reasons, the review of the literature is necessarily selective. First,those articles that featured PM as a central topic published from 2005 until



2012 in selected public administration and accounting journals were identified.This was achieved by searching on the words ‘performance measurement’ and‘public sector’ (for mainstream accounting journals only) within abstracts,topic descriptors and/or keywords.3 This search process identified 66 articles,excluding both the review articles discussed earlier and those articles thatmentioned PM in passing. A further 13 articles were excluded because theydid not consider the negative potential or risks of PM. Consequently, 53 articlesformed the basis for review. This was conducted in two phases.

Two authors independently read each article with the objective of identifyingthe risk issues that were discussed. An inductive approach was chosen becausean exhaustive set of risk categories could not be pre-determined. Both authorsdiscussed the risk issues that had been identified and whether these could begrouped together. This discussion revealed that the four PM risk categoriesreferred to in the previous section encapsulated a number of the risk issuesidentified. In addition, the inductive classification process revealed three furthercategories of risks.

The first implicates PM in accountability relations to external constituents,with reporting on performance and the preservation of ideals of integrity,democracy, justice and equity all important aspects of discharging accountabilityfor stewardship of public resources. In this, the risk is that PM systemsare unsupportive of accountability relations to these external stakeholders.The second risk is that PM systems – which can be costly to operate –are under-utilised for public sector decision making, causing any potentialbenefits from PM to remain unrealised.4 A third risk of PM practices is thatthey can cause adverse impacts on employee welfare within the public sectororganisation. These three risks of unsupportive PM for external accountability,under-utilised PM information and PM adversely impacting employee welfarewere added to the previously identified four, resulting in a total of seven PMrisk categories used to present and analyse the results of the review of priorresearch.

The two authors then independently classified the studies according to whichrisk category they examined, with papers examining multiple categories includedin each. A comparison of the categorisation results indicated a match in excessof ninety percent with the remaining differences resolved through subsequentdiscussion. An independent researcher was subsequently provided with a randomsample of fifteen papers and the risk category descriptions as presented in thepaper. This yielded coding results that matched the coding of the two authors forthe sample of papers. The results of this process are presented in the Appendix.5

The second phase pertained to our specific interest in what this literaturereveals about risks, the conditions where they are more likely to manifest andpossible mitigation approaches. The two authors involved in phase one then re-read each paper noting any identification of antecedent conditions to PM risksand/or proposed mitigation approaches, with the results of this phase comparedand discussed to achieve inter-subjective consensus. Explicit discussion of riskantecedents and/or mitigation approaches was required for a study to be coded



as engaging with these topics. We present the results in the next sectionhighlighting the seven risk categories.6

RESULTS

Risk Categories Examined

Across all 53 studies identified as examining issues of PM risk, 43 were classifiedas examining only one risk category, six examined two risk categories and fourstudies examined three risk categories. Thus most studies of PM risk focus onone risk category at a time, with the risk categories examined 67 times in total bythe group of 53 studies. Table 1 presents the quantitative coding results from theanalysis of what risks were the focus of PM research and the proportion of studiesthat examine antecedent conditions and potential mitigation approaches.

Examining the frequency counts in Table 1 indicates that prior research hasfocused on a ‘top tier’ of risks, comprising unsupportive PM systems for externalaccountability, under-utilised PM information and misalignment between PMand public sector goals. A second-tier of risks examined in the reviewed studiescomprised PM-induced gaming and PM misrepresenting performance. A last tierof risks comprised PM restricting flexibility and/or innovation, and PM adverselyimpacting employee welfare. These were considered by a few studies only, andoften observed as a by-product of other risks manifesting.

As Table 1 indicates, a significant proportion of studies across all riskcategories has moved beyond describing the risk itself to discussing and

Table 1

Quantitative Coding Results

No. of Studies No. (%) Examining No. (%) ProposingRisk Examining Antecedent MitigationCategory Category Conditions Approaches

Unsupportive PM systems forexternal accountability

16 10 (63%) 5 (31%)

Under-utilised PM information 16 14 (88%) 4 (25%)Misalignment between PM and

public sector goals13 9 (69%) 4 (31%)

PM-induced gaming 9 6 (67%) 6 (67%)PM systems misrepresenting

performance6 3 (50%) 2 (33%)

PM restricting flexibility and/orblocking innovation

4 4 (100%) 1 (25%)

PM adversely impacting employeewelfare

3 2 (67%) 0 (0%)

Total 67



explaining conditions that are likely to give rise to its presence or absence.However, with the exception of studies examining PM-induced gaming, theproportion of studies that explicitly discuss practical mitigation approaches tothese risks manifesting is far lower (see also van Helden and Northcott, 2010).Prior research’s identification of antecedent conditions and risk mitigationapproaches are respectively discussed in more detail in the next sections.

Antecedent Conditions to Risks

Table 2 presents qualitative results from the coding of antecedent conditionsto risks, as identified in previous studies. Given that a number of antecedentconditions are repeated across the various risk categories, the discussionbelow focuses on describing the properties of the more commonly identifiedantecedents.

A number of studies see PM risks as inevitable consequences of PM in thepublic sector. Complexity in defining the nature of public services and ‘goodperformance’ is seen as a key antecedent to the risk of unsupportive systems forexternal accountability (for example, Yang and Holzer, 2006). This coupled withthe problem of attributing public service outcomes to individual organisationsmake accurate representation of performance through PM a flawed task (vande Walle, 2009). Public service task complexity also makes precise indicators ofperformance difficult to achieve, leading to conditions that increase the riskof PM-induced gaming (Heinrich and Marschke, 2010). Other studies thatsee public sector PM risk as inevitable link PM to the underlying logic ofNPM reforms. These claim that the economic rationality and logic that infusesand dominates NPM displaces other important dimensions of public services;namely public service ethos, a focus on citizenship outcomes and innovation (forexample, Watkins and Arrington, 2007).

Political influences and pressures are another widely cited antecedent toa number of different risks. Political pressures that enrol PM in impressionmanagement efforts can cause PM design to be unsupportive for its rolein discharging accountability to external constituents (Chang, 2009) and cancause PM-induced gaming (Bevan and Hood, 2006). These pressures, as well asturbulent policy environments and pre-occupation with management fads cancause PM to be misaligned to public sector goals and organisational contexts(Milakovich, 2006; and Nomm and Liiv, 2012). Volatile political support for PMcan also cause under-utilisation of PM information (Yang and Hsieh, 2007; andHou, 2011), as can political and institutional pressures to conform to norms ofeconomic rationality, especially if decoupled or loosely coupled PM systems areimplemented (Johansson and Siverbo, 2009; and Torres et al., 2011).

Poor design and implementation are also seen as causing PM risks inthree main ways. First, there is often an exclusion of stakeholder involvementand both stakeholder and strategic outcomes in the design of PM systems.This can cause PM to be unsupportive for external accountability (Dubnic



Tab

le2

Qua

litat

ive

Cod

ing

Res

ults

–A

ntec

eden

tC

ondi

tion

s

Uns

uppo

rtiv

eM

isal

ignm

ent

PMR

estr

ictin

gPM

Adv

erse

lyPM

Sys

tem

sU

nder

-util

ised

Bet

wee

nPM

PM-

PMS

yste

ms

Fle

xibi

lity

Impa

ctin

gfo

rExt

erna

lPM

and

Publ

icIn

duce

dM

isre

pres

entin

gan

d/or

Blo

ckin

gE

mpl

oyee

Acc

ount

abili

tyIn

form

atio

nS

ecto

rGoa

lsG

amin

gPe

rform

ance

Inno

vatio

nW

elfa

re

•C

ompl

exit

yin

defi

ning

the

boun

dari

esof

netw

orke

dpu

blic

serv

ices

•Ir

reco

ncila

ble

stak

ehol

der

conf

licts

in:(

a)w

hat

gove

rnm

ent

shou

lddo

(b)

wha

tco

nsti

tute

sgo

odpe

rfor

man

cean

d(c

)ho

wpe

rfor

man

cesh

ould

bem

easu

red

•R

efor

ms

bias

edby

econ

omic

logi

c,di

spla

cing

othe

r

•V

olat

ilepo

litic

alsu

ppor

t•

Ado

ptio

nra

ther

than

use

seen

assu

ffic

ient

give

nin

stit

utio

nal

pres

sure

s•

Lac

kof

lead

ersh

ipsu

ppor

t•

Cul

ture

that

does

not

supp

ort

PM •In

adeq

uate

stak

ehol

der

incl

usio

nin

PMde

sign

and

impl

emen

tati

on•

Lac

kof

inte

grat

ion

wit

h

•Po

litic

alin

flue

nces

onPM

to(a

)m

anag

eim

pres

sion

sof

gove

rnm

ent

(b)

dem

onst

rate

shor

t-te

rmim

prov

emen

ts•

Imm

atur

ean

dtu

rbul

ent

polic

yen

viro

nmen

ts•

Path

depe

nden

cies

cons

trai

ning

outc

ome

orie

ntat

ion

inPM

syst

ems

•In

suff

icie

ntco

nsid

erat

ion

ofho

wou

tcom

esar

e

•H

igh

publ

icse

ctor

task

com

plex

ity

mak

esgo

odm

easu

res

diff

icul

tto

desi

gn•

Impr

essi

onan

dre

puta

tion

impe

rati

ves

•Fa

ilure

byth

eSt

ate

toau

dit

PMin

form

atio

ndu

eto

inte

rest

inm

aint

aini

ngill

usio

nof

PMac

cura

cy•

Poor

PMde

sign

com

pris

ing

(a)

too

few

mea

sure

s(b

)no

tm

easu

ring

impo

rtan

t

•C

ompl

exit

yin

defi

ning

the

boun

dari

esof

netw

orke

dpu

blic

serv

ices

•Ir

reco

ncila

ble

stak

ehol

der

conf

licts

in:(

a)w

hat

gove

rnm

ent

shou

lddo

(b)

wha

tco

nsti

tute

sgo

odpe

rfor

man

cean

d(c

)ho

wpe

rfor

man

cesh

ould

bem

easu

red

•D

iffi

cult

yin

attr

ibut

ion

ofou

tcom

esto

indi

vidu

al

•In

suff

icie

ntde

lega

tion

ofde

cisi

on-r

ight

s•

Top

-dow

nim

plem

enta

tion

proc

esse

s•

Ref

orm

sbi

ased

byec

onom

iclo

gic,

disp

laci

ngot

her

perf

orm

ance

dim

ensi

ons

•Po

orly

desi

gned

PMsy

stem

s•

Top

-dow

nim

plem

enta

tion

proc

esse

s•

Ref

orm

sbi

ased

byec

onom

iclo

gic,

disp

laci

ngot

her

perf

orm

ance

dim

ensi

ons



Tab

le2

(Con

tinu

ed)

Uns

uppo

rtiv

eM

isal

ignm

ent

PMR

estr

ictin

gPM

Adv

erse

lyPM

Sys

tem

sU

nder

-util

ised

Bet

wee

nPM

PM-

PMS

yste

ms

Fle

xibi

lity

Impa

ctin

gfo

rExt

erna

lPM

and

Publ

icIn

duce

dM

isre

pres

entin

gan

d/or

Blo

ckin

gE

mpl

oyee

Acc

ount

abili

tyIn

form

atio

nS

ecto

rGoa

lsG

amin

gPe

rform

ance

Inno

vatio

nW

elfa

re

perf

orm

ance

dim

ensi

ons

•Po

litic

alin

flue

nces

onPM

to(a

)m

anag

eim

pres

sion

sof

gove

rnm

ent

(b)

dem

onst

rate

shor

t-te

rmim

prov

emen

ts•

Abs

ence

ofci

tize

nin

volv

emen

tin

PMde

sign

and

impl

emen

tati

on•

Insu

ffic

ient

cons

ider

atio

nof

how

outc

omes

are

tobe

achi

eved

inPM

desi

gn

publ

icse

ctor

stra

tegi

sing

and

orga

nisi

ngpr

oces

ses

•In

suff

icie

ntem

ploy

eem

otiv

atio

nan

dbu

y-in

toim

port

ance

/use

ofPM

syst

em•

Lac

kof

trai

ning

and

reso

urce

s

tobe

achi

eved

inPM

desi

gn•

Top

dow

nim

plem

enta

tion

proc

esse

s•

Lac

kof

inte

grat

ion

wit

hpu

blic

sect

orst

rate

gisi

ngan

dor

gani

sing

proc

esse

s•

Insu

ffic

ient

capa

bilit

yan

dre

sour

ces

wit

hin

agen

cies

acti

viti

esan

dou

tcom

es(c

)fa

ilure

toad

apt

mea

sure

sov

erti

me

•In

suff

icie

ntpu

blic

serv

ice

mot

ivat

ion

•G

oald

iver

genc

ebe

twee

nse

rvic

ede

liver

ers

and

thos

ese

ekin

gto

cont

rol

agen

cies

and

esta

blis

hing

caus

alit

ydu

eto

publ

icse

rvic

esta

skco

mpl

exit

y



and Frederickson, 2010) and misaligned to public sector goals (Micheli andNeely, 2010). In turn, the failure to measure important activities can be seenas a licence for gaming (Greener, 2005). Second, insufficient organisationalconditions for PM either pre-exist, and are not altered sufficiently to supportdesign and implementation, or are not created to support the operation and useof PM systems. Studies that observe under-utilisation of PM information focuson this, highlighting inadequate leadership support, lack of an organisationalculture that is sufficiently ‘performance-focused’ (for example, Sanger, 2008; andMoynihan and Pandey, 2010), and little consideration of how PM systems willlink to the strategic and management processes of the public sector organisation(for example, Ammons and Riverbark, 2008; and Arnaboldi and Palermo, 2011).

Finally, employee factors can cause PM risks to manifest. Poor motivationand enthusiasm due to insufficient training and support can cause under-utilisation of PM information (Hou, 2011; and Taylor, 2011) while an absence ofskills and resources to adequately design PM systems can cause misalignment(Amirkhanyan, 2008 and 2010). Poor motivation also exacerbates goal diver-gence within public sector organisations and enhances the risk of PM inducedgaming (Bevan and Hood, 2006). The few studies that examine PM restrictingflexibility and innovation and causing adverse impacts on employee welfarehighlight the impact of top-down implementation processes (Greener, 2005;and Guven-Uslu and Conrad, 2011) and insufficient delegation of decision-rights(Moynihan, 2006).

Proposed Risk Mitigation Approaches

Qualitative results from the coding of mitigation approaches identified by priorresearch are presented in Table 3. These tend to be specific to particular riskcategories and, accordingly are discussed by the risk they relate to.

Making PM more citizen-centric is a central theme behind ideas to mitigatethe risk of unsupportive PM systems for external accountability. Suggestionsinclude citizen participation in the design of PM systems (Yang and Holzer,2006), a framework for including both intermediate and end citizen-centricoutcomes (Wichowsky and Moynihan, 2008) and a process by which strategiesare mapped to overall citizen outcomes (Micheli and Neely, 2010). At a moremicro-level, the greater use of subjective indicators that reflect perceptionsof service recipients is suggested (Shingler et al., 2008). Another approachinvolves the adjustment ex post of measured performance to reduce the effectsof factors outside the control of the public sector organisation. This removalof ‘noise’ from PM arguably makes accountability for performance clearerto external stakeholders (Barnow and Heinrich, 2010). A final idea involvesgreater utilisation of program and policy evaluation techniques for externalaccountability purposes (Yang and Holzer, 2006).

In relation to under-utilisation of PM information, numerous studies developexplanatory models for PM usage but only a handful of studies offer guidance for



Tab

le3

Qua

litat

ive

Cod

ing

Res

ults

–Pr

opos

edM

itig

atio

nA

ppro

ache

s

Uns

uppo

rtiv

eM

isal

ignm

ent

PMR

estr

ictin

gPM

Adv

erse

lyPM

Sys

tem

sU

nder

-util

ised

Bet

wee

nPM

PMS

yste

ms

Fle

xibi

lity

Impa

ctin

gfo

rExt

erna

lPM

and

Publ

icPM

-ind

uced

Mis

repr

esen

ting

and/

orB

lock

ing

Em

ploy

eeA

ccou

ntab

ility

Info

rmat

ion

Sec

torG

oals

Gam

ing

Perfo

rman

ceIn

nova

tion

Wel

fare

•In

volv

eci

tize

nsin

PMde

sign

•A

dopt

citi

zen

focu

sed

PMfr

amew

orks

that

incl

ude

mea

sure

sof

citi

zens

hip

outc

omes

and

utili

sesu

bjec

tive

mea

sure

s•

Adj

ust

expo

stfo

run

anti

cipa

ted

and/

orun

cont

rolla

ble

fact

ors

•U

tilis

epr

ogra

man

dpo

licy

eval

uati

onm

ore

inPM

•St

akeh

olde

r(e

lect

edof

fici

alan

dci

tize

n)in

volv

emen

tin

desi

gn•

Form

alis

elin

ksbe

twee

nPM

and

orga

nisa

tion

alst

rate

gy,g

oal

sett

ing

and

com

mun

icat

ion

proc

esse

s•

Bui

ldor

gani

sati

onal

com

mit

men

tth

roug

hle

ader

ship

and

afo

cus

onem

ploy

ees

•L

ink

PMto

stra

tegy

form

ulat

ion

and

plan

ning

proc

esse

s•

Con

duct

stra

tegy

map

ping

aspa

rtof

PMde

sign

•Im

plem

ent

PMw

ith

bott

om-u

pfe

edba

cklo

ops

•M

easu

rein

term

edia

teou

tcom

es•

Sele

ctiv

eim

plem

enta

tion

ofPM

pack

ages

•A

ddm

ore

mea

sure

sto

PMsy

stem

•A

ssig

nne

gati

vew

eigh

tsto

inco

mpl

ete

mea

sure

s•

Tes

tan

dch

ange

mea

sure

sif

gam

ing

isob

serv

edas

part

ofex

tend

edde

sign

and

impl

emen

tati

on•

Gen

erat

ein

trin

sic

publ

icse

rvic

em

otiv

atio

nto

offs

etga

min

g

•A

djus

tex

post

for

unan

tici

pate

dan

d/or

unco

ntro

llabl

efa

ctor

s•

Lon

ger

post

prog

ram

peri

ods

toev

alua

tecl

ient

/ser

vice

deliv

ery

outc

omes

•In

crea

sed

man

ager

ial

deci

sion

righ

ts

•N

oap

proa

ches

wer

eid

enti

fied

tosp

ecif

ical

lym

itig

ate

adve

rse

impa

cts

onem

ploy

eew

elfa

re



Tab

le3

(Con

tinu

ed)

Uns

uppo

rtiv

eM

isal

ignm

ent

PMR

estr

ictin

gPM

Adv

erse

lyPM

Sys

tem

sU

nder

-util

ised

Bet

wee

nPM

PMS

yste

ms

Fle

xibi

lity

Impa

ctin

gfo

rExt

erna

lPM

and

Publ

icPM

-ind

uced

Mis

repr

esen

ting

and/

orB

lock

ing

Em

ploy

eeA

ccou

ntab

ility

Info

rmat

ion

Sec

torG

oals

Gam

ing

Perfo

rman

ceIn

nova

tion

Wel

fare

•C

ondu

ctst

rate

gym

appi

ngas

part

ofPM

desi

gnw

ith

acco

unta

bilit

yto

publ

icde

fini

ngth

eov

eral

lou

tcom

esto

beac

hiev

ed•

Com

bine

subj

ecti

vew

ith

obje

ctiv

em

easu

res

•Pr

ovid

ene

cess

ary

trai

ning

and

supp

ort

•E

mbr

ace

loca

lsp

ecif

icit

yin

desi

gnan

dim

plem

enta

tion

•W

ork

toga

inpo

litic

alsu

ppor

t

for

grea

ter

cust

omis

atio

n•

Adj

ust

expo

stfo

rse

rvic

ere

cipi

ent

dem

ogra

phic

fact

ors

topr

even

tef

fort

swit

chin

gan

dch

erry

-pic

king

•In

trod

uce

mor

eex

ante

unce

rtai

nty

into

targ

ets

and

mea

sure

spec

ific

atio

ns•

Aud

itpe

rfor

man

ceac

hiev

emen

tsfo

rga

min

g•

Uti

lise

non-

PMm

echa

nism

sto

man

age



practising managers based on these, with the remainder focused on implicationsfor further research. Suggestions to mitigate the risk of under-utilisationfocus on the most influential explanatory factors. These include formalintegration of PM systems in organisational strategy and management processes,building organisational commitment to PM through leadership, harnessingemployee motivation and providing training and support. Greater stakeholderinvolvement in PM design across both internal and external environments is alsoprescribed.

Studies examining misaligned PM imply greater inclusion of indicators ofoutcomes and strategic objectives as remedies. However, less than a third ofstudies explicitly discussed approaches to mitigate the risk of misaligned PMsystems. The use of techniques such as strategy mapping coupled with bottom-up feedback loops during design and implementation (Micheli and Neely, 2010)and greater customisation of PM packages (Modell, 2009a) are proposed. Thedevelopment of individual indicators that capture and aggregate intermediateoutcomes (Cuganesan and Lacey, 2011) is also suggested.

Several prescriptions have been developed in prior studies to mitigate PM-induced gaming. Ex ante mechanisms comprise the inclusion of additionalindicators in PM systems (Kelman and Friedman, 2009) and the introductionof more uncertainty into PM system targets and measures on the basis that‘complete specification of targets and how performance will be measuredalmost invites reactive gaming’ (Bevan and Hood, 2006, p. 533). Other ex antemitigation approaches include the assigning of negative weights to incompletemeasures in composite performance measures to discourage PM-induced gaming(for more detail, see Eisenkorpf, 2009). Ex post modifications include the useof ‘adjusted performance measures’ that control for effort substitution and‘cherry-picking’ of service recipient populations (Barnow and Heinrich, 2010)and greater auditing of PM information (Bevan and Hood, 2006). Generally,these studies conclude that it is only through trial and error that employeeresponses to specific indicators and metrics become known. Thus extendeddesign and implementation approaches are suggested, comprising actual testingand refinement of PM based on their observed gaming effects (Heinrichand Marschke, 2010). A few authors also suggest the cultivation of employeemotivation and use of non-PM modes of managing as ways to reduce gaming(Bevan and Hood, 2006; and Kelman and Friedman, 2009). However thesesuggestions are not expanded upon.

Turning to the remaining risks, we note that prior studies offer a fewsuggestions only in the way of risk mitigation. Given that the antecedentconditions to the risk of misrepresenting performance through PM are seen asinherent in the public sector context, partial mitigation only is proposed throughadjustment ex post for unanticipated and/or uncontrollable factors and longerpost program periods to evaluate client/service delivery outcomes (Schochet andBurghardt, 2008). Mitigating the risk of PM restricting flexibility and/or blockinginnovation is proposed through increased managerial authority and expanded



decision rights to accompany PM systems implementation (Moynihan, 2006),while no suggestions to mitigate the risk of adverse impacts on employee welfarewere identified by the small group of studies examining this risk.

A WAY FORWARD: INTEGRATED AND ENABLING PM

Based on the results presented in the previous section we outline a way forwardfor how future research should engage with the topic of PM risk. First wediscuss reframing the research question of PM risk research and PM researchmore broadly, before focusing more specifically on the role of PM in strategisingand managing within the public sector and in external accountability relations.

Reframing the Research Question of Public Sector PM Studies

The results presented in the previous section highlight the fragmented andnon-cumulative nature of research engaging in the risks of PM. We offer twoobservations in relation to this. First, the majority of studies examine PM risks inisolation from one another, despite their inter-related nature. Indeed, as notedearlier, there is significant overlap in the antecedent conditions to the differentrisk categories. Yet there is little understanding of how a set of antecedentconditions might yield different risks that influence each other over time.Knowing that risk manifestation is possible or probable will be useful in terms ofidentifying ex ante the conditions where PM systems can be implemented withless risk mitigation, and situations where more risk mitigation and associatedinvestment of monetary resources, time and energy might be required. Suchunderstandings can be invaluable in the resource constrained environmentsthat characterise contemporary public services.

In addition, partial examinations of risk can lead to well-intentionedmitigation approaches that reduce one category of risks but create another.For example, adding greater uncertainty into PM system targets and measuresto mitigate gaming (as suggested by Bevan and Hood, 2006) may result inadverse impacts on employee welfare (as noted by ter Bogt and Scapens, 2012,in relation to the subjectivities of performance evaluation). Involving citizens inthe design of PM systems may result in mitigating the risk of unsupportive PMfor external accountability, but questions have been raised about the capacityfor alignment to public sector strategies and goals in this context (Yang andHolzer, 2006). Thus future research on PM risk has to engage more holisticallywith the multiple risks that can occur in revealing their inter-related nature, theantecedent conditions that are more likely to create PM risks and the mitigationapproaches that may ameliorate multiple risks simultaneously (see, for example,our discussion in the next sub-section on creating and sustaining PM systems asenabling).

Second, the prior research we reviewed also advances numerous mitigationstrategies, as do the authors of the risk typologies reviewed in the second section,



namely, Smith (1995) and de Bruijn (2002). However, there has been little or notesting and experimentation of these amongst the studies we review. As such,it is difficult to know which mitigation approaches are effective and the costsof implementing these. Thus, a more pragmatic approach to PM risk researchis required where the mitigation approaches developed by research to date aretested through future studies (see also van Helden and Northcott, 2010). Here, aone-sided focus on risk reduction and/or elimination that often manifests in priorstudies (for example, Smith, 1995; and de Bruijn, 2002) is to be avoided. Fullmitigation of risks is likely to be costly and perhaps elusive. Instead, risks need tobe sufficiently mitigated to allow ‘the going ahead with good enough measurement’(Van de Walle, 2009, p. 53, emphasis in original) where the benefits of particularPM systems outweigh the risks that they create. Thus future research on PMshould focus on the risks as well as benefits generated by PM systems so thatrequisite knowledge can be built up which allows design, implementation andoperation of ‘good enough PM systems’ in practice.

Hence, the central research question that future research on public sector PMshould embrace is: how to design, implement and operate PM systems that generate positiveconsequences which outweigh the associated risks? We provide specific guidance for howfuture research might engage in this question in two specific domains, PM instrategising and managing public services and PM in external accountabilityrelations. The next two sections deal with each of these in turn. In so doing, weconcentrate less on public-sector context and political influences as these factorsare arguably difficult to change and less oriented to our PM focus. Instead, wefocus more on PM design, implementation and use given this is typically withinthe influence of those in charge of public sector PM initiatives.

PM as Enabling Organisational Control Systems for Strategy and Management

Future research into public sector PM systems should theorise and investigatePM as only one element of the ‘control systems’ – comprising both formalsystems and informal levers – that are used to plan, allocate resources, monitorperformance and implement corrective actions (Chenhall, 2003). Importantly,control systems do not play a strategy implementation role only but facilitatelearning and adaptation as environments shift and uncertainties crystallise(Simons, 1995). Control systems research argues against the analysis of specificelements of controls as this can lead to spurious findings and possibly conflictingresults (Chenhall, 2003). This criticism is potentially applicable to public sectorPM research to date. Thus, future research on PM needs to conceive of PMsystems as part of integrated control systems, with PM investigated alongsidethe values and norms of the organisation, strategy formulation and planningprocesses, performance evaluation processes, intangible or tangible rewardmechanisms (to the extent that these exist) and the structures and proceduralcontrols that operate.



It would be unfair to suggest that PM research has ignored entirely these otherelements of organisational control systems. However, there is little uniformityin how they are examined. The role of shared values is recognised in somestudies (for example, Kelman and Friedman, 2009) while the role of structuresis emphasised in others (for example, Dubnic and Frederickson, 2010). Morebroadly, a reduced role for PM in managing has been suggested (see de Bruijn,2002; and Bevan and Hood, 2006), but little guidance is offered on how thismight be achieved. By adopting an integrated control systems approach to thestudy of PM systems, many of the organisational conditions that were cited bythe reviewed studies as antecedents to PM risks will be incorporated withinthe control system construct, thereby facilitating a more structured approachto their analysis and the inter-relationships between these other controls andPM. Over time, research along these lines has potential to provide insights intohow public sector managers might counter the risks of PM in strategising andmanaging at their organisations through utilising non-PM controls.

To aid public sector managers and practitioners, future research shouldalso consider how PM is considered by those being subject to its effects andspecifically, the extent to which PM is ‘enabling’. We use the term ‘enabling’here as outlined by Adler and Borys (1996) to conceptualise the objective forPM systems as being controls that motivate and support managers and lower-level employees. Enabling controls are characterised by transparency, flexibilityand allow employees to repair imperfectly designed controls in responding tolocal contingencies (Ahrens and Chapman, 2004). This is instead of controlsbeing restrictive, bureaucratic and coercive, often leading to negative results inorganisations (Adler and Borys, 1996). Wouters and Wilderom (2008) specificallyexamine PM systems, and find that a design and implementation process thatbuilds upon the existing PM experiences of employees and their professionalism,engages with external parties in developing notions of performance, and allowsexperimentation with measures and a certain amount of local autonomy,contributes to a more enabling PM.

Notions of staff involvement, and retaining flexibility in the PM system,are not entirely new (see Smith, 1995) – what is required is research thatengages with these issues in a structured, theorised and cumulative approach.Providing insights to practising managers on how to create and sustain enablingPM systems will also help to address a number of PM risks. In relation tothe risk of under-utilised PM, studies examining enabling PM illustrate howgreater employee engagement and use of PM information can be achieved(Wouters and Wilderom, 2008). In relation to gaming, the few studies that haveexamined gaming of PM in the public sector observe the difficulties of mitigatingthis risk. Furthermore, many of the approaches suggested arguably complicatePM systems (for example, greater target uncertainty, negative weights). Analternative approach is to focus on building enabling PM in practice whichoffers potential for greater engagement with PM by employees in terms of howPM can be useful in local contexts. It can also compensate for the inherent



incompleteness of PM. In relation to restricting innovation and employeeimpacts, developing enabling PM as part of a broader control system mayactually encourage adaptation and innovation rather than restrict it (Ahrens andChapman, 2004), while the explicit recognition and involvement of an employeeperspective may help to address the risk of adverse impacts on employee welfare.

In relation to how to proceed, we suggest that case studies are requiredthat adopt a processual focus on how PM is designed, implemented and usedover time in public sector organisations. Future research needs to engage withexamining how PM systems function in the day-to-day dynamics of public sectororganisation. Theoretical perspectives that facilitate greater engagement withmicro-processes and/or specific practices through which PM is enacted are likelyto be useful. The benefits of these perspectives are that they provide a means forresearchers to access the detail or micro-processes of how PM is used in publicsector strategising and managing and link these to risks and benefits. Theorisingand examining how PM systems operate as one element of an integrated andenabling control system will provide insights into how particular patterns ofcontrols provide different configurations of risks and benefits, hopefully movingus towards the overall objective of achieving a less risky PM in the public sectorwhere benefits outweigh negative potential. Broader empirical testing through,for example, cross-sectional surveys would be useful once this deep and richknowledge base is established.

Citizen-Oriented PM and Multiple Evaluation Modes for External Accountability

In the context of PM and external accountability relations, prior researchproposes frameworks and approaches that seek to include both citizen par-ticipation in PM design and citizen perspectives in performance evaluation (forexample, Wichowsky and Moynihan, 2008), yet there has been little testingof these. Furthermore, direct inclusion of citizen perspectives in PM designraises challenges on how to create consensus views that are relatively stableover time (van de Walle, 2009), mitigate the impact of ‘issues of the moment’on perceptions, and ensure that evaluations of performance are not unfairlyweighted towards the negative due to political discourse between opposingparties (Yang and Holzer, 2006). Thus more work needs to be done oninvestigating the challenges and effects of integrating citizen involvement andcitizen perspectives in creating citizen-oriented PM and whether alternativenon-PM approaches are required. Research is also required to examine howsuch PM systems then affect planning, prioritisation and resource allocationdecisions of public sector organisations and networks and the downstream effectson performance.

In addition, how integration of citizens in PM design, implementation anduse alters the goals of citizens and their perceptions toward government(including the frames they adopt when evaluating government performance)is an interesting area for future research to pursue. On these points, there



is evidence suggesting that external stakeholder and citizen participation inPM design is positively associated with positive PM effects such as perceivedreliability of indicators, usage of PM information, relevance and learning effects(Yang and Holzer 2006; Li and Tang, 2009; see also Ho, 2006; and Yang andHsieh, 2007). However, these consequences are at the organisational level, andthere has been less research on how involvement influences or affects citizensthemselves and their attitudes towards government.

As noted earlier in the paper, difficulties in reaching precise definitions of thenature of public services and good performance often leads to PM systems thatare unsuitable for external accountability purposes (Yang and Holzer, 2006).Hence, re-orienting the research agenda also requires consideration of scopeof study and definitions of performance. In relation to scope of study, futureresearch needs to move to greater recognition and examination of networkedgovernment and how public services are delivered through a network of public,private and not-for-profit agents (Dubnick and Frederickson, 2010). While somewould argue that demarcating the boundaries of public services is impossible (forexample, Van de Walle. 2009), we propose that multiple levels of examinationare required. This would then allow evaluation over time as to whether particularcombinations of intra-organisational program level, public sector organisationallevel or network level PM are more supportive of accountability relationshipswith external stakeholders.

In relation to definitions of performance, we see questions of how PMsupports external accountability as requiring deeper engagement with multipleevaluation foci and modes. Rather than single composite measurement systems(as discussed by Van de Walle, 2009), we see multiple indicators of value-for-money and economy, efficiency and effectiveness as being important foraccountability, with effectiveness measures comprising outcome indicators orproxies/ intermediate-level indicators as per Cuganesan and Lacey (2011) as wellas citizenship centric values such as integrity, democracy, participation, justiceand equity (Dubnick and Frederickson, 2010; and Moynihan et al., 2011).7

As part of this, future research needs to examine whether evaluation of citizenimpact should occur at the intra-organisational program level, public sectororganisational level, government portfolio level or inter-government publicpolicy level. Questions about whether we measure organisational or networkperformance are particularly important for the public sector (Cuganesan andLacey, 2011; and Moynihan et al., 2011). Alternative performance evaluationapproaches also require examination in comparison to PM. Specifically, researchthat tests the efficacy of PM versus policy and program evaluation methodsin evaluating outcome achievement in the public sector is required (seeYang and Holzer, 2006). Further research could also engage usefully in thequestion of how performance should be represented. Studies observe thataccountability to external constituents is occurring although PM do little in thisregard (see Dubnick and Frederickson, 2010) and may have reached its limits(Catasus and Gronlund, 2005). Increasingly, the private-sector is examining



how narrative formats can be used to complement numbers in reports onperformance, including recent initiatives such as integrated reporting (IIRC,2012). Examining how the use of narratives to contextualise and complementquantitative performance indicators influences the decisions and actions ofexternal constituents may allow the development of guidelines and betterpractices over time that mitigate the risk of excessive trust in numbers.

In closing this sub-section we suggest that action research approaches and/orexperimental designs need to feature much more than they have to date.Trialling the approaches proposed by prior research to create citizen-orientedPM perspectives and alternative evaluation modes in enabling accountability toexternal constituents is important if practical guidance is to be produced on thistopic.

CONCLUSION

Public sector researchers have become increasingly interested in the manage-ment of risk (Hood and Miller, 2009; and Lapsley, 2009). Noting that PM systemscan cause risk, we argue that a review of research on PM risks in the publicsector and an articulation of a way forward for future research is timely giventhe number of scandals, unanticipated consequences and so called ‘perverseoutcomes’ that continue to manifest in the public sector, and the concurrentand growing interest in risk within public sector research. We have attemptedto do this in this paper as a means of moving to a state of less risky public sectorPM where the benefits outweigh the negative potential. We acknowledge thatthere is no firm body of evidence that establishes unequivocally that situationswhere negative potential outweigh the benefits of PM are the status quo. Indeed,the absence of this evidence reinforces the imperative for future public sectorPM research to engage in our reframed central research question. However,we feel that a situation where risks outweigh the benefits is a reasonable startingpoint on the basis of the numerous anecdotal cases of PM risks manifesting andthe many research studies that evidence the manifestation of different PM risks.8

It is important to acknowledge the inevitable limitations of a review ofliterature. First, the literature review for reasons of space is unavoidablyselective. Second, some element of subjectivity is involved in categorisingresearch and relating these to particular risk categories. However, steps such asutilising multiple coders were taken to mitigate this. Third, the risk categorieswe use are inductively generated from the journal articles we review, and itis probable that new technological, political and socio-economic developmentsare creating additional PM risks that we have not considered. Continuedengagement with the concerns of those managing, delivering and evaluatingpublic services is important to mitigate this (van Helden and Northcott, 2010).Finally, the research directions outlined are necessarily the views of the authorsonly.



Notwithstanding this, we feel this study contributes in a number of ways.First, we bring together a disparate and fragmented field in reviewing studiesexamining the risks of PM. This forms a first step towards a more cohesive andcumulative examination of PM risks, their antecedent conditions and proposedmitigation approaches. Second, we reframe the research question for PM riskresearch and PM research more broadly. Here we argue that PM risks shouldnot be considered in isolation from one another, and that the focus should be oninvestigating and testing PM practices to identify how PM systems can be devel-oped and used where the overall benefits exceed any negative potential. Third,specific suggestions are offered in terms of how future research might proceedin a coordinated manner, examining and informing practice where PM is part ofan integrated and enabling control system for strategising and managing in thepublic sector, and where citizen-centric PM is utilised as part of multiple evalu-ation modes to more effectively support accountability to external constituents.

In closing, we argue that greater engagement with issues of PM risk isnecessary. Indeed, if we do not attend to this, then Lapsley’s (2008, p.86)disheartening proposition about the future of PM may well come to pass: ‘[that]the practice of performance measurement will continue to exhibit dysfunctionaland contentious effects which will not be resolved in the foreseeable future’.



APPENDIXPhase 1 Coding Results

Risk Category Study

Unsupportive PM systems forexternal accountability (n = 16)

Catasus and Gronlund (2005); Collier (2006);Yang and Holzer (2006); Watkins and Arrington(2007); Gregory and Lonti (2008); Shingler et al.(2008); Wichowsky and Moynihan (2008); Bracci(2009); Chang (2009); Taylor (2009); Barnowand Heinrich (2010); Dubnic and Frederickson(2010); Tooley et al. (2010); Micheli and Neely(2010); Moynihan et al. (2011); Nomm and Liiv,2012

Under-utilised PM information(n = 16)

Melkers and Willoughby (2005); Ho (2006);Marvel and Marvel (2007); Yang and Hsieh(2007); Ammons and Riverbark (2008); Sanger(2008); Johansson and Siverbo (2009); Moynihanand Pandey (2010); Arnaboldi and Palermo(2011); Guven-Uslu and Conrad (2011); Hoqueand Adams (2011); Ho (2011); Hou (2011);Taylor (2011); Torres et al. (2011); Rautiainen,and Jarvenpaa (2012)

Misalignment between PM andpublic sector goals (n = 13)

Wall (2005); Milakovich (2006); Modell et al.(2007); Amirkhanyan (2008); Chang (2009);Modell (2009a); Amirkhanyan (2010);Cuganesan and Lacey (2011); Micheli and Neely(2010); Poister (2010); Conrad and Uslu (2011);ter Bogt and Scapens (2012); Nomm and Liiv,(2012)

PM-induced gaming (n = 9) Greener (2005); Bevan and Hood (2006); Courtyand Marschke (2007); Eisenkopf (2009); Kelmanand Friedman (2009); Barnow and Heinrich(2010); Heinrich and Marschke (2010);Moynihan et al. (2011); Lacey et al. (2012)

PM systems misrepresentingperformance (n = 6)

Catasus and Gronlund (2005); Crotty et al.(2006); Schochet and Burghardt (2008); Van deWalle (2009); Barnow and Heinrich (2010);Moynihan et al. (2011)

PM restricting flexibility and/orblocking innovation (n = 4)

Greener (2005); Moynihan (2006); Bracci (2009);Guven-Uslu and Conrad (2011);

PM adversely impactingemployee welfare (n = 3)

Greener (2005); Bracci (2009); ter Bogt andScapens (2012)



NOTES

1 We use the term ‘riskiness of PM’ to refer to its potential for causing unintended negativeconsequences rather than a more general state of ‘uncertainty’. In this we are consistent witha number of definitions and meaning of the term ‘risk’ (Power, 2004a).

2 Bouckaert and Balk’s (1991) thirteen dysfunctional PM effects relate to misinterpretingperformance, gaming, PM driven behaviours that are not aligned to important goals andrestricting innovation. Smith (1995) discusses pre-occupation with narrow measured objectivesrather than broader organisational objectives and a short-term rather than long-term focus,gaming and data manipulation, PM misinterpretation and the inhibition of innovation.de Bruijn’s (2002) risks are that PM prompts game-playing, blocks innovation, leads togaming, causes an over-focus on PM, reduces public-sector collaboration, and punishes goodperformance through ratcheting.

3 The selected journals comprised (in alphabetical order); Accounting, Auditing and Accountability;Accounting, Organizations and Society; The British Accounting Review; Critical Perspectives on Accounting;European Accounting Review; Financial Accountability & Management; International Public ManagementJournal; Journal of Policy Analysis and Management; Journal of Public Administration Research andTheory; Management Accounting Research; Public Administration Review; Public Administration; PublicManagement Review; The Accounting Review.

4 We are aware that under-utilisation of PM information may be seen by some as a positiveoutcome rather than as a risk. Our response to this under-utilisation of PM means that muchof the benefits are unlikely to be realised, resulting in situations of excessive costs in termsof measurement and/or risks (such as those identified in the paper) exceeding any positivepotential.

5 Peripheral or passing references to a PM risk category were considered insufficient for thepurposes of our classification process.

6 We wish to note that these categories are not mutually exclusive and are likely to be inter-related. Also, we acknowledge that the categories presented are not exhaustive.

7 We acknowledge here the point made by one anonymous referee that classic representativedemocracy may be a cost-effective way to handle integration of citizenship perspectives inperformance evaluation of government, especially given the likely heterogeneity of goals andperceptions of citizens. Yet, we would like to see inclusion of citizenship centric measures suchas those referred to here to provide more frequent performance signals (vis-a-vis electoralcycles) from an important stakeholder perspective.

8 We do not wish to suggest that the research agenda we propose be discontinued upon reachinga point where benefits exceed negative potential. Rather, ongoing studies should examine howthis differential can be increased over time.

REFERENCES

Adler, P. and B. Borys (1996), ‘Two Types of Bureaucracy: Enabling and Coercive’, AdministrativeScience Quarterly, Vol. 4, No. 1, pp. 61–90.

Ahrens, T. and C. Chapman (2004), ‘Accounting for Flexibility and Efficiency: A Field Study ofManagement Control Systems in a Restaurant Chain’, Contemporary Accounting Research, Vol.21, No. 2, pp. 271–301.

Amirkhanyan, A.A. (2008), ‘Collaborative Performance Measurement: Examining and Explainingthe Prevalence of Collaboration in State and Local Government Contracts’, Journal of PublicAdministration Research and Theory, Vol. 19, No. 3, pp. 523–54.

——— (2010), ‘Monitoring across Sectors: Examining the Effect of Nonprofit and For-ProfitContractor Ownership on Performance Monitoring in State and Local Contracts’, PublicAdministration Review, Vol. 70, No. 5, pp. 742–55.

Ammons, D.N. and W.C. Rivenbark (2008), ‘Factors Influencing the Use of Performance Datato Improve Municipal Services: Evidence from the North Carolina Benchmarking Project’,Public Administration Review, Vol. 68, No. 2, pp. 304–18.

Arnaboldi, M. and T. Palermo (2011), Translating Ambiguous Reforms: Doing Better Next Time?’Management Accounting Research, Vol. 22, pp. 6–15.



Barnow, B.S. and C.J. Heinrich (2010), ‘One Standard Fits All? The Pros and Cons of PerformanceStandard Adjustments’, Public Administration Review (Jan/Feb), pp. 60–71.

Bevan, G. and C. Hood (2006), ‘What’s Measured is What Matters: Targets and Gaming in theEnglish Public Health Care System’, Public Administration, Vol. 84, No. 3, pp. 517–38.

Blumer, H. (1954), ‘What is Wrong with Social Theory?’ American Sociological Review, Vol. 18, pp.3–10.

Bogt, H.J. ter and R.W. Scapens (2012), ‘Performance Management in Universities: Effects of theTransition to More Quantitative Measurement Systems’, European Accounting Review, Vol. 21,No. 3, pp. 451–97.

Bouckaert, G. and W. Balk (1991),Public Productivity Measurement: Diseases and Cures’, PublicProductivity & Management Review, Vol. 15, No. 2, pp. 229–35.

Bracci, E. (2009), ‘Autonomy, Responsibility and Accountability in the Italian School System’,Critical Perspectives on Accounting, Vol. 20, No. 3, pp. 293–312.

Catasus, B. and A. Gronlund (2005), ‘More Peace For Less Money: Measurement andAccountability in the Swedish Armed Forces’, Financial Accountability & Management, Vol. 21,No. 4, pp. 467–84.

Chang, L-C. (2009), ‘The Impact of Political Interests upon the Formulation of PerformanceMeasurements: The NHS Star Rating System’, Financial Accountability & Management, Vol. 25,No. 2, pp. 145–65.

Chenhall, R. (2003), ’Management Control Systems Design within its Organizational Context:Findings from Contingency-Based Research and Directions for the Future’, AccountingOrganizations and Society, Vol. 28, pp. 127–68.

Collier, P.M. (2006), ‘Costing Police Services: The Politicization of Accounting’, Critical Perspectiveson Accounting, Vol. 17, pp. 57–86.

Conrad, L. and P.G. Uslu (2011), ‘Investigation of the Impact of "Payment by Results" onPerformance Measurement and Management in NHS Trusts’, Management Accounting Research,Vol. 22, pp. 46–55.

Courty, P. and G. Marschke (2007), ‘Making Government Accountable: Lessons from a FederalJob Training Program’, Public Administration Review, Vol. 67, No. 5, pp. 904–16.

Crotty, S.N., N.A. Theobald and J. Nicholson-Crotty (2006), ‘Disparate Measures: Public Managersand Performance-Measurement Strategies’, Public Administration Review, Vol. 66, No. 1, pp.101–13.

Cuganesan, S. and D. Lacey (2011), ‘Developments in Public Sector Performance Measurement:A Project on Producing Return on Investment Metrics for Law Enforcement’, FinancialAccountability & Management, Vol. 27, No. 4, pp.458–79.

de Bruijn, H. (2002), Performance Measurement in the Public Sector: Strategies to Cope with theRisks of Performance Measurement’, International Journal of Public Sector Management, Vol 15,No. 7, pp. 578–94.

Dubnick, M.J. and H.G. Frederickson (2010), ‘Accountable Agents: Federal PerformanceMeasurement and Third-Party Government’, Journal of Public Administration Research and Theory,Vol. 20, Suppl. 1, pp. i143-59.

Eisenkopf, G. (2009), ‘Negative Weights for Performance Measures’, International Public ManagementJournal, Vol. 12, No. 3, pp. 332–44.

Goddard, A. (2010), ‘Contemporary Public Sector Accounting Research – An InternationalComparison of Journal Papers’, The British Accounting Review, Vol. 42, pp. 75–87.

Greener, I. (2005), ‘Health Management as Strategic Behaviour: Managing Medics andPerformance in the NHS’, Public Management Review, Vol. 7, No. 1, pp. 95–110.

Gregory, R. and Z. Lonti (2008), ‘Chasing Shadows? Performance Measurement of Policy Advicein New Zealand Government Departments’, Public Administration, Vol. 86, No. 3, pp. 837–56.

Guven-Uslu, P. and L. Conrad (2011), ‘A Longitudinal Study of Change in the English NationalHealth Service’, Financial Accountability & Management, Vol. 27, No. 4, pp. 385–408.

Heinrich, C.J. and G. Marschke (2010), ‘Incentives and their Dynamics in Public SectorPerformance Management Systems’, Journal of Policy Analysis and Management, Vol. 29, No.1, pp. 183–208.

Helden G.J. van and D. Northcott (2010), ‘Examining the Practical Relevance of Public SectorManagement Accounting Research’, Financial Accountability & Management, Vol. 26, No. 2, pp.213–40.



Helden, G.J. van, H. Aardema, H. J. ter Bogt and T.L.C.M. Groot (2010), ‘Knowledge Creationfor Practice in Public Sector Management Accounting by Consultants and Academics:Preliminary Findings and Directions for Future Research’, Management Accounting Research,Vol. 21, No. 2, pp. 83–94.

Ho, A.T.-K. (2006), ‘Accounting for the Value of Performance Measurement from the Perspectiveof Midwestern Mayors’, Journal of Public Administration Research and Theory, Vol. 16, No. 2, pp.217–37.

——— (2011), ‘PBB in American Local Governments: It’s More than a Management Tool’, PublicAdministration Review, Vol. 71, No. 3, pp. 391–401.

Hood, C. and P. Miller (2009), ‘Public Service Risks: What is Distinctive and What is New?’ Riskand Public Services (ESRC Public Services Programme, Oxford).

Hou, Y., R.S. Lunsford, K.C. Sides and K.A. Jones (2011), ‘State Performance-Based Budgeting inBoom and Bust Years: An Analytical Framework and Survey of the States’, Public AdministrationReview, Vol. 71, No. 3, pp. 370–88.

International Integrated Reporting Council (2012), ‘Integrated Reporting – Proto-type of the International <IR> Framework’ (available at http://www.theiirc.org/wp-content/uploads/2012/11/23.11.12-Prototype-Final.pdf) (accessed 6 March, 2013).

Jackson, P. (2011), ‘Governance by Numbers: What Have We Learned Over the Past 30 Years’,Public Money and Management (January), pp.13–26.

Johansson, T. and S. Siverbo (2009), ‘Explaining the Utilization of Relative Performance Evaluationin Local Government: A Multi-Theoretical Study using Data from Sweden’, FinancialAccountability & Management, Vol. 25, No. 2, pp. 197–224.

Kelman, S. and J.N. Friedman (2009), ‘Performance Improvement and Performance Dysfunction:An Empirical Examination of Distortionary Impacts of the Emergency Room Wait-TimeTarget in the English National Health Service’, Journal of Public Administration Research andTheory, Vol. 19, pp. 917–46.

Lacey, D., S. Cuganesan, S. Goode and K. Jacobs (2012), ‘Celebrating Adversity: Inter-organizational Dependence and Public Sector Performance Reporting in the AustralianFederal Police’, Public Administration, Vol. 90, No. 2, pp. 393–411.

Lapsley, I. (2008), ‘The NPM Agenda: Back to the Future’, Financial Accountability & Management,Vol. 24, No. 1, pp. 77–96.

——— (2009), ‘New Public Management: The Cruellest Invention of the Human Spirit?’, Abacus,Vol. 45, No. 1, pp. 1–21.

Li, P. and G. Tang (2009), ‘Performance Measurement Design within its Organisational Context—Evidence from China’, Management Accounting Research, Vol. 20, pp. 193–207.

Marvel, M.K. and P.H. Marvel (2007), ‘Outsourcing Oversight: A Comparison of Monitoring forIn-House and Contracted Services’, Public Administration Review, Vol. 67, No. 3, pp. 521–30.

Melkers, J. and K. Willoughby (2005), ‘Models of Performance-Measurement Use in LocalGovernments: Understanding Budgeting, Communication, and Lasting Effects’, PublicAdministration Review, Vol, 65, No, 2, pp. 180–90.

Micheli, P. and A. Neely (2010), ‘Performance Measurement in the Public Sector in England:Searching for the Golden Thread’, Public Administration Review, Vol. 70, No. 4, pp. 591–600.

Milakovich, M.E. (2006), ‘Comparing Bush-Cheney and Clinton-Gore Performance ManagementStrategies: Are they more Alike than Different?’ Public Administration, Vol. 84, No. 2, pp.461–78.

Modell, S. (2009a), ‘Bundling Management Control Innovations: A Field Study of OrganisationalExperimenting with Total Quality Management and the Balanced Scorecard’, Accounting,Auditing and Accountability Journal, Vol. 22, No. 1, pp. 59–90.

——— (2009b), ‘Institutional Research on Performance Measurement and Management in thePublic Sector Accounting Literature: A Review and Assesssment’, Financial Accountability &Management, Vol. 25, No. 3, pp. 277–303.

———, K. Jacobs and F. Wiesel (2007), ‘A Process Return? Path Dependencies, Institutions andPerformance Management in Swedish Central Government’, Management Accounting Research,Vol. 18, No. 4, pp. 453–75.

Moynihan, D.P. (2006), ‘Managing for Results in State Government: Evaluating a Decade ofReform’, Public Administration Review, Vol. 66, No. 1, pp. 77–89.

——— and S. Pandey (2010), ‘The Big Question for Performance Management: Why Do ManagersUse Performance Information?’, Journal of Public Administration Research and Theory (March), pp.1–18.



Moynihan, D.P., S. Fernandez, S. Kim, K.M. LeRoux, S. J. Piotrowski, B. E. Wright and K. Yang(2011), ‘Performance Regimes Amidst Governance Complexity’, Journal of Public AdministrationResearch and Theory, Vol. 21 (suppl 1), pp. i141–55.

Nomm, K. and T. Randma-Liiv (2012), ‘Performance Measurement and Performance Informationin New Democracies’, Public Management Review, Vol. 14, No. 7, pp. 859–79.

Osborne, D. and T. Gaebler (1993), Reinventing Government: How the Entrepreneurial Spirit isTransforming the Public Sector (New York: Penguin).

Poister, T.H. (2010), ‘The Future of Strategic Planning in the Public Sector: Linking StrategicManagement and Performance’, Public Administration Review, Vol. 70, pp. s246–54.

Power, M.K. (2004a), The Risk Management of Everything (Demos, London).——— (2004b), ‘Counting, Control and Calculation: Reflections on Measuring and Management’,

Human Relations, Vol. 57, pp. 765–82.Rautiainen, A. and M. Jarvenpaa (2012), ‘Institutional Logics and Responses to Performance

Measurement Systems’, Financial Accountability & Management, Vol. 28, No. 2, pp. 164–88.Sanger, M.B. (2008), ‘From Measurement to Management: Breaking through the Barriers to State

and Local Performance – Special Issue’, Public Administration Review, Vol. 68, pp. 70–85.Simons, R. (1995), Levers of Control (Harvard University Press, Boston).Schochet, P.Z. and J.A. Burghardt (2008), ‘Do Job Corps Performance Measures Track Program

Impacts?’ Journal of Policy Analysis and Management, Vol. 27, No. 3, pp. 556–76.Shingler, J., M.E. Van Loon, T.R. Alter and J.C. Bridger (2008), ‘The Importance of Subjective

Data for Public Agency Performance Evaluation’ Public Administration Review, Vol. 68, No. 6,pp. 1101–11.

Smith, P. (1995), ‘On the Unintended Consequences of Publishing Performance Data in the PublicSector’, International Journal of Public Administration, Vol, 18, pp. 277–310.

Taylor, J. (2009), ‘Strengthening the Link Between Performance Measurement and DecisionMaking’, Public Administration, Vol. 87, No. 4, pp. 853–71.

——— (2011), ‘Factors Influencing the Use of Performance Information for Decision Making inAustralian State Agencies’, Public Administration, Vol. 89, No. 4, pp. 1316–34.

Tooley, S., J. Hooks and N. Basnan (2010), ‘Performance Reporting by Malaysian Local Authorities:Identifying Stakeholder Needs’, Financial Accountability & Management, Vol. 26, No. 2, pp. 103–33.

Torres, L., V. Pina and A. Yetano (2011), ‘Performance Measurement in Spanish LocalGovernments: A Cross-Case Comparison Study’, Public Administration, Vol. 89, No. 1, pp.1–29.

Van de Walle, S. (2009), ‘International Comparisons of Public Sector Performance: How to MoveAhead?’, Public Management Review, Vol. 11, No. 1, pp. 39–56.

Wall, A. (2005), ‘The Measurement and Management of Intellectual Capital in the Public Sector:Taking the Lead or Waiting for Direction?’, Public Management Review, Vol. 7, No. 2, pp.289–303.

Watkins, A.L. and C.E. Arrington (2007), ‘Accounting, New Public Management and AmericanPolitics: Theoretical Insights into the National Performance Review’, Critical Perspectives onAccounting, Vol. 18, pp. 33–58.

Wichowsky, A. and D.P. Moynihan (2008), ‘Measuring How Administration Shapes Citizenship:A Policy Feedback Perspective on Performance Management’, Public Administration Review,Vol.68, No. 5, pp. 908–20.

Wouters, M. and C. Wilderom (2008), ‘Developing Performance Measurement Systems as EnablingFormalization: A Longitudinal Field Study of a Logistics Department’, Accounting, Organizationsand Society, Vol. 33, pp. 488–516.

Yang, K. and M. Holzer (2006), ‘The Performance–Trust Link: Implications for PerformanceMeasurement’, Public Administration Review, Vol. 66, No. 1, pp. 114–26.

——— and J.Y. Hsieh (2007), ‘Managerial Effectiveness of Government Performance Measure-ment: Testing a Middle-Range Model’, Public Administration Review, Vol. 67, No. 5, pp. 861–79.


The Riskiness of Public Sector Performance Measurement: A Review and Research Agenda

Documents

Transcript of The Riskiness of Public Sector Performance Measurement: A Review and Research Agenda