Slide 1

Slide 2

LSE 400 Paradoxes of auditing and performance measurement Mike Power Department of Accounting March 20 th 1pm-2pm

Slide 3

Audit and performance measurement An age of performance? Performance measurement: the dream Measurement challenges Gaming Crowding Performance cultures

Slide 4

An age of performance? Do we live in an age of performance evaluation and measurement..?..this lecture will be evaluated and graded. and lecturers grade and evaluate students Are we now the how am I doing society? Everyone wants feedback continually? Self-help book industry to support performance of individuals in all manner of areas: sport, romance, jobs, diet etc Performance obsession at all levels: individuals, organizations and states (regulators)..

Slide 5

Slide 6

2014 REF result for Business and Management

Slide 7

Slide 8

Slide 9

Slide 10

Slide 11

An age of performance A myriad of performance metrics....cannot be a modern organization without producing data for self and external evaluation purposes. The focus on performance measurement is cultural and systemic.. ..but because there is so much focus on performance in all its forms. ..we are also often disappointed

Slide 12

Slide 13

Performance measurement: the dream Measuring performance. makes it transparent and auditable Can be used as a contractual basis for rewarding/controlling organizational actors.. to incentivise themto exert effort ..to achieve desirable/improved outcomes Implicit THERMOSTATIC MODEL OF CONTROL: performance stds actual performance feedback (bonus, reputation etc)

Slide 14

Performance measurement: the dream UK Public services 1980s/1990s Pressures to improve performance, make visible and comparable, across time and across entities Growth of financial and non-financial indicators as targets in performance contracts to. .enable comparison of performance discipline and incentivise hospitals and managers via star rating system (climate of fear?)

Slide 15

Measurement challenges What aspect of performance to measure? Inputs: e.g. cost, headcount etc (relatively easy to measure and audit; cost control, but effort often unobservable) Efficiency: how inputs result in outputs e.g. cost effectiveness of producing a car; staff-student ratios Outputs: shorter term, easier to measure and audit: revenues, exam results, units produced Outcomes: much harder to measure: longer term, more to do with values, less visible: excellent research; healthier nation; safer and happier society?

Slide 16

Measurement challenges Prisons: e.g. re-offending rates? Schools: e.g. literacy test results? Hospitals: target minimum A&E and elective operation waiting times? Universities: e.g. student satisfaction? Art gallery: e.g. school children visitors per annum? Companies: e.g. growth in long term shareholder value All seems reasonable? But data demands? Relation of outputs(targets) to desired outcomes/values most challenging measurement frontier

Slide 17

Performance universe InputsOutputsOutcomes Easy to measure ZONE OF MEASUREMENT (audit) ZONE OF NON-MEASUREMENT Hard to measure Easy to audit Hard to audit Relation to goals unclear Relation to goals clearer

Slide 18

Measurement challenges Performance measurement assumes: 1. What is measured is meaningful (proxy) measure of all relevant performance dimensions E.g. soviet problem: focus on production rather than quality Students may be satisfied but not pushed? 2. What is measured is in fact measurable; categorization and consistent data - e.g. what is a mortality rate 3. Organizational agents will not react by gaming system

Slide 19

Reacting and gaming People react to performance systems (Espeland & Sauder) `When a measure becomes a target, it ceases to be a good measure.' (Strathern, interpreting Goodharts law) Scientists produced fewer patents when patent made into target! Why? Hood and Bevan (2006): Despite apparent improvements in performance relative to indicators requirements create targets which can be gamed No incentive to discover gaming = audit deficit Opportunistic output distortion: Ambulance response times edited = creative accounting Trollies used as beds to meet 12hour rule for waiting for a bed Target is waiting time for new appointments, so make follow-up waiting times longer Mortality data: surgeon reluctance to work on high risk cases How might lecturers game student satisfaction scores?

Slide 20

Crowding Intrinsic and extrinsic motivation Mission-based organizations and services E.g. paying charity workers caused them to work fewer hours Should LSE reward academics for being good citizens? Motivational crowding (Frey and Jegen, 2001): more control of performance (extrinsic) can generate loss of intrinsic motivation, reduction in effort! Crowding can become pathological/deviant..focus on indicators at expense of performance

Slide 21

Effort/motivation Zone of deviance Zone of crowding Control via performance measures + - Zone of motivation

Slide 22

Slide 23

This culture [of the Trust] is characterised by introspection, lack of insight or sufficient self- criticism, rejection of external criticism, reliance on external praise and, above all, fear. I found evidence of the negative impact of fear, particularly of losing a job, from top to bottom of this organisation. Regrettably, some of the causes of that fear have arrived at the door of the Trust from elsewhere in the NHS organisation in the form of financial pressures and fiercely promoted targets. (Francis Report on Mid Staffordshire NHS Trust, January 2013, page 184)

Slide 24

Slide 25

Tesco warned it must address damaging company culture 24 September, 2014 | By Jennifer CreevyJennifer Creevy Tesco must address an oppressive company culture that is damaging its business, a source closely connected to the grocers leadership team has told Retail Week. Following the shock revelation that Tesco had overestimated its half-year profits by 250m, the source said that it is critical that a culture that contributed to the crisis is changed in order to safeguard the future of the retailer, the UKs biggest and the third largest in the world. Tesco, which disclosed on Monday an overstatement of profits related to the recognition of income Tesco receives from suppliers, has long had a reputation for tough dealing with suppliers harking back to the era of Sir Terry Leahy. But the well placed source told Retail Week that the culture that has developed in recent years is different to the normal cut-and-thrust of retail and supplier relationships. New Tesco chief executive Dave Lewis, only three weeks into the job, launched an investigation into the profit overstatement crisis over the weekend after an employee alerted the retailers general counsel of the issue.

Slide 26

Slide 27

There was an over-emphasis on short-term financial performance, reinforced by remuneration systems that tended to reward revenue generation rather than serving the interests of customers and clients. There was also in some parts of the Group a sense that senior management did not want to hear bad news and that employees should be capable of solving problems. This contributed to a reluctance to escalate issues of concern Extract from Salz Review of Barclays bank Culture

Slide 28

Performance cultures These examples have common features; Soviet Union productionism: neglect of quality have many western parallels! E.g. financial crisis profit was privatised as performance value at the expense of risk: risk was externalised and socialised Stories of performance decoupled from underlying deviant normality. Staffordshire NHS trust; star system to blame; compliance with performance crowded out care? Tescos/Barclays: aggressive sales targets crowded out clients. Formal audit fails because it consumes the measured story Informal audit i.e. whistleblowing stigmatised and dangerous

Slide 29

Performance cultures It right to measure where we can..track things that matter over time But examples show importance of performance culture...how measures interpreted and used is criticalvalues which are less visible in performance system .on their own PIs may incentivise wrong/pathological behaviour; damage organizational mission Design challenges: pay attention to data quality Think hard about what is being measured and why? Is the tail of audit wagging the dog of performance measurement (Power, 1997) Paradox: best performance measures not always most Accurate and precise; positive role of uncertainty in minimising gaming and crowding effects Cultivate performance values as foundation for measurement

Slide 30

Questions & References REFERENCES Bevan and Hood (2006). What is measured is what matters: targets and gaming in the English public health care system Public Administration 84(3):517538. Espeland and Sauder (2007) Rankings and reactivity: how public measures recreate social worlds. American Journal of Sociology 113(1):1-40. Frey and Jegen (2001). Motivational crowding theory. Journal of Economic Surveys 15(1):589-611. Power (1997). The audit society. Oxford University Press

Slide 31