Maurits Waardenburg Martijn Groenleer · 2 Managing the Performance of Public Sector...

1

Working Paper No. 1 December 2016

Maurits Waardenburg Martijn Groenleer Jorrit de Jong Bas Keijser

Managing the Performance of Public Sector Collaborations: A Literature Synthesis using Examples from the Law Enforcement Domain

2

Managing the Performance of Public Sector

Collaborations:

A Literature Synthesis using Examples from the Law

Enforcement Domain

Uitgesproken door Prof.dr. Martijn Groenleer als bijdrage aan de bijeenkomst ‘Brabant Swingstate’, georganiseerd door BrabantKennis en TiREG, 20 maart 2017, Den Bosch

The Tilburg Center for Regional Law and Governance (TiREG) is a joint venture of the Departments of Public Law and Politics & Public Administration of Tilburg University and is supported by the Province of Noord-Brabant. Besides legal and public administration scholars, the Center brings together researchers from a variety of disciplines and institutes (also beyond Tilburg University) to conduct research and develop education in the field of regional law and governance. It focuses on exploring, explaining and evaluating the novel interactions between government, businesses, citizens and knowledge institutions that develop at the level of the region in response to complex societal challenges. Tilburg University Tilburg Center for Regional Law and Governance (TiREG) Reitse Poort 1 (1st Floor) Prof. Cobbenhagenlaan 125 5037 DB Tilburg The Netherlands Website: www.tilburguniversity.edu/tireg Email: [email protected] Phone: 0031 13 466 2128 © 2017 The author & TiREG

http://www.tilburguniversity.edu/tireg

mailto:[email protected]

TiREG Working Paper No. 1, December 2016

3

Managing the Performance of Public Sector

Collaborations:

A Literature Synthesis using Examples from the Law

Enforcement Domain

Maurits Waardenburg1, Martijn Groenleer2, Jorrit de Jong3 & Bas Keijser4

Abstract: Public sector agencies increasingly seek to collaborate with other public and private

organizations. These collaborations operate outside of traditional organizational boundaries,

raising new questions regarding performance management. Using illustrative examples from the

law enforcement domain, this article aims to synthesize the main findings of the existing

literature on performance management in collaborative governance settings, highlight current

knowledge gaps and identify key ways forward for research and practice. As part of the synthesis,

we identify a range of performance management challenges that collaborations face, which we

cluster into three categories: substantive problem-solving challenges, collaborative process

challenges, and multi-relational accountability challenges. We place these challenges along the

collaborative process and performance management life cycles that collaborations go through

and describe ways to deal with them as discussed in the literature. Based on the synthesis, we

identify two knowledge gaps in dealing with the collaborative performance management

challenges: both the underlying causes of these challenges and the possible ways to overcome

them remain underexplored. Our synthesis results in two key avenues for further research: case

studies to refine our understanding of challenges’ underlying causes, and action research in

combination with field experiments to develop and test possible performance management

interventions.

Introduction

1 Ash Center for Democratic Governance and Innovation, Kennedy School of Government, Harvard University, United States & Tilburg Center for Regional Law and Governance (TiREG), Tilburg School of Law & Tilburg School of Politics and Public Administration, Tilburg University, The Netherlands 2 Tilburg Center for Regional Law and Governance (TiREG), Tilburg School of Law & Tilburg School of Politics and Public Administration, Tilburg University, The Netherlands & Faculty of Technology, Policy & Management, Delft University of Technology, The Netherlands, <[email protected]> 3 Ash Center for Democratic Governance and Innovation, Kennedy School of Government, Harvard University, United States, <[email protected]> 4 TNO Defence, Safety & Security, The Netherlands & Faculty of Technology, Policy & Management, Delft University of Technology, The Netherlands


4

In 2006, the Dutch Public Prosecution Service adopted a novel approach: to fight human

trafficking in the Netherlands more effectively, it intensified its collaboration with other parties.

The approach focused on working together with so-called facilitators, including hotels, cleaning

companies, landlords and abortion clinics, to reduce the opportunity for human trafficking

practices. If successful, the approach would raise significant barriers for human traffickers. Yet,

the approach’s performance hinged on the cooperation of a host of government agencies and

private sector parties. In its struggle to fight human trafficking collaboratively, the Public

Prosecutor’s Office faced a variety of performance management challenges: How to measure

whether their approach successfully reduced the multi-faceted phenomenon of human trafficking

and did not merely displace it? How to make the coordination amongst varied parties with

dispersed interests sustainable and direct it towards realizing societal effects? How to attribute

the performance to any one organization in the network to inform future capacity investments

and hold actors accountable?

Problem-oriented law enforcement collaborations

To tackle complex organized crime problems, like the human trafficking problem described

above, law enforcement agencies increasingly seek to collaborate, realizing that no individual

agency holds the key to cracking these problems alone (Van Dijk 1995; Levi and Maguire 2004).

Collaboration with other agencies allows law enforcers to draw on a range of means at the

collective disposal of various parties, thereby expanding their capacity to raise barriers for

criminal networks and decreasing their reliance on the criminal justice system (Garland 1996;

Rosenbaum 2002). Orienting these collaborations towards crime problems rather than individual

criminal incidents, forces law enforcers to focus on reaching outcomes, i.e., addressing these

problems, rather than on reaching outputs, e.g., increasing the number of arrests (Goldstein

1990; Moore and Braga 2003; Sparrow 2008).

In order to be effective, the performance of such problem-oriented law enforcement

collaborations must be managed differently than traditional function- or process-oriented

organizations (Klijn, Koppenjan, and Termeer 1995; Sparrow 2008). In the past, “hierarchical

government bureaucracy was the predominant organizational model used to deliver public

services and fulfill public policy goals” (Goldsmith and Eggers 2004, 7). This model focused on

controlling processes to ensure service delivery (Stoker 2006), such as ensuring that police

departments processed all incoming incidents correctly. In the new model of networked

governance, however, public value outcomes and democratic accountability have taken center

stage (Klijn and Koppenjan 2000).

This need to manage outcomes rather than outputs and account to a multitude of stakeholders

in the democratic arena, including citizens and partner organizations, raises new questions about


5

performance management unresolved by the literature on performance management in single

organizations (Heinrich 2002; Ryan and Walsh 2004; Moynihan et al. 2011). A key idea of new

public management has been disaggregation, splitting up large public sector hierarchies into

smaller units, with their own management structures and performance measures. Following this

idea, also law enforcers’ tasks have been clearly defined in the past. They had pre-set and readily

measurable performance targets and there was no ambiguity about to whom they were

accountable. Collaborative governance and public value management introduces new

uncertainties in law enforcement professionals’ work environment: They are asked to take

increasing discretion in deciding what is valuable to pursue given the problem at hand, and with

whom to engage in collaborative relationships, and they need to account for seemingly vague

public value outcomes to their parent organizations, partners, as well as citizens (Moore 1995;

Bingham, Nabatchi, and O’Leary 2005; O’Flynn 2007; Alford and Hughes 2008; Bryson, Crosby,

and Bloomberg 2014).

Research aim and questions

Of course, organized crime is just one of the ‘wicked’ societal issues that collaborations are

fighting today (Rittel and Webber 1973; Clarke and Stewart 1997). The aim of this article is not

to elaborate on problem-oriented law enforcement collaborations. It is to provide actionable

insights into the state-of-the-art literature on performance management in collaborative

governance settings more broadly, so as to help collaborators across sectors, including health,

education, food, water, energy or transportation, deal with emerging performance management

challenges.

To do this, the article summarizes and synthesizes the available literature in the field, applying it

to the law enforcement context to illustrate implications in practice. Furthermore, it identifies

prevailing knowledge gaps and points out why such gaps exist. Finally, it determines promising

research avenues based on these knowledge gaps to strengthen research in the field and broaden

its applicability to collaborative governance in general. Hence, we pose ourselves the following

questions:

1. What does the available literature tell us about the performance management

challenges collaborations experience in solving (wicked) problems and possible ways

to overcome these challenges?

2. Based on our review of the literature, what seem to be prevailing knowledge gaps in

the field?

3. How can these gaps be addressed and what are the implications in terms of future

research and practice?


6

Research strategy

In answering these questions, we followed a semi-structured literature synthesis process. We

conducted an extensive thematic search for relevant literature, using several academic databases

including Web of Science, Scopus and Google Scholar, and consulted relevant journals directly.

We used a list of search terms ordered by particular themes constructed collectively by the

researchers. The search terms included variations on the three broad topics of interest,

“collaborative governance”, “performance management in the public sector”, and “collaborative

performance management” to ensure we cast our nets widely and were as inclusive as possible.

Using this strategy, we consulted the titles and abstracts of more than 3,000 sources. From this,

we derived a long list of more than 300 academic articles and academic book chapters that

seemed scientifically up-to-date, and academically as well as practically relevant to one of the

three topics of interest.

Next, we examined the sources in the long list in more detail- including their introduction,

conclusions and main bodies. We then prioritized the inclusion of articles in this literature

synthesis based on three criteria:

1. thematic relevance, determined by the fit between the publications’ insights and our

research questions;

2. scientific impact, including number of citations and/or references by leading authors

in the field;

3. practical relevance, determined by our knowledge of the field and regular discussions

with academics as well as practitioners.

We incorporated the resultant first short list of articles and books in an outline of the literature

synthesis. In some cases, this led to the discovery of additional key references to be included in

the long list and, occasionally, the short list of references in this article.

Afterwards, we had several cycles of iteration between consecutive versions of outlines/drafts

and the long list of literature to ensure all relevant articles were incorporated. In addition, we

sent around consecutive versions of the outlines and drafts to other academic topic experts to

ensure literature they deemed relevant was incorporated as well. In some cases, this involved

pulling in new references that were not included in the original long list.

This elaborate process enabled us to make a synthesis, identify knowledge gaps and determine

promising research avenues relevant to both theory and practice.


7

Structure of this article

The remainder of the article is structured along the topics of interest, our synthesis across and

the implications of this synthesis. Section two gives an introduction to the life cycle of

collaborative governance as a heuristic device to detail the state-of-the-art knowledge about

collaborative governance. Section three introduces the public sector performance management

life cycle to detail the state-of-the-art-knowledge about performance management in the public

sector in a similar vein. Both of these sections contain visuals giving the reader a systematic

overview of the major literature’s influence on this field. Beyond providing this systematic

overview, the life cycle perspective also contributes to our understanding of the different stages

both collaborative governance and performance management go through and how these align.

The fourth section builds on the previous two sections by giving a synthesis of the current

understanding on emerging performance management challenges along the life cycles of

collaborative governance and public sector performance management as well as possible ways

to overcome these challenges. In doing so, it contributes to the literature by showing how these

performance management challenges are intimately tied with the life cycle of collaborations,

while highlighting current knowledge gaps inhibiting our full understanding of how collaborations

can increase their effectiveness. Section five discusses the implications of this synthesis and

provides an insight into further research avenues that are relevant to both theory and practice.

We will use the literature’s applicability to the law enforcement context as an illustrative example

throughout, but we believe the implications of the literature reach beyond this context to

collaborative governance efforts across sectors and, indeed, countries.

Life cycle of collaborative governance

In 1998, the Crime and Disorder Act in Britain called explicitly on law enforcement organizations

to cooperate in reducing crime rates in their cities. In response, law enforcement organizations in

Coventry, a city northwest of London with relatively high crime rates, decided to form the Crime

and Disorder Reduction Partnership. Progress in the early years was slow: It was not clear what

problems really required partnership and what outcomes were expected. In addition, the

organizations were not used to work together, making coordinating collective actions complex.

In response, a new Chief Executive was appointed in 2001. She started by setting out a clearer

mission and objectives for the partnership. She also introduced a dedicated partnership staff to

coordinate the partners’ efforts and distribute funding from the central government not tied to

specific initiatives to the different partners. In addition, she chaired regular progress meetings to

drive the collaboration to newly specified, specific outcomes. These improvements to the

collaborative process finally helped to improve community safety in Coventry (Kelman and

Waterhead 2006).


8

The above example of Coventry illustrates the move towards collaborative governance to solve

complex crime problems. ‘Collaborative governance’ refers to some joint effort or process by

multiple organizations from the public and private sector with a certain public goal (Bardach

1998; Provan and Milward 2001; Agranoff and McGuire 2003; Bryson, Crosby, and Stone 2006;

McGuire 2006; Ansell and Gash 2008; Emerson, Nabatchi, and Balogh 2011; Klijn and Koppenjan

2000). This new form of governance supplements rather than replaces old forms of governance.

More precisely, “networks complement markets and hierarchies as governing structures for

authoritatively allocating resources and exercising control and coordination” (Rhodes 1996, 652).

The ultimate objective of this new form of governance is the creation of public value where

original process- or output-focused forms of governance failed (Benington and Moore 2010;

Bryson, Crosby, and Stone 2015; Moore 1995).

The Coventry example also illustrates the importance of managing the entire collaboration life

cycle from end-to-end, including determining the scope of collaboration upfront, coordinating

the ongoing collaborative process and ensuring a feedback loop from results to the activities

performed by collaborations (Bryson, Crosby, and Stone 2015). In line with this, synthesizing

across a variety of authors, collaborative governance efforts’ lifecycle seem to consist of at least

three steps: Managing starting conditions for collaboration, managing the collaborative process

and managing outcomes (Thomson and Perry 2006; Bryson, Crosby, and Stone 2006; Ansell and

Gash 2008; Bryson, Crosby, and Stone 2015). In this section, we attempt to give a systematic

description of these steps and the main authors who have attempted to define them, thereby

synthesizing the literature in the field while simultaneously showing how authors’ contributions

complement each other.

As a heuristic device, Figure 1 provides a high-level overview of some of the recent and most

influential contributions by authors that have described the steps of the collaborative

governance life cycle. First, it describes the contributions per step of the collaborative

governance life cycle of five of the most cited authors that attempted to describe the full

collaboration process in the last decade. Second, it describes other major influential

contributions to the collaborative life cycle that either preceded these five authors’ contributions

or represent a major contribution to a specific step in the collaborative process life cycle.


9

Managing starting conditions for collaboration

When thinking of starting a new collaboration, one of the first questions to ask is whether the

particular problem calls for a collaborative effort (Agranoff 2006). Not all issues require new

governance models to be solved; single cases, such as single criminal incidents, can and should

still be solved using a traditional, mono-disciplinary approach. Whether the process of

establishing a collaboration is initiated depends largely on antecedent conditions, which include

factors in the institutional environment, the problem’s complexity and the urgency of the public

issue (Bryson, Crosby, and Stone 2006).

Once it is clear that a collaboration is the preferred approach to tackle the issue in question, the

collaboration’s conveners need to decide on various dimensions of the collaboration. This

includes deciding on the aims of the partnership (Bryson, Crosby, and Stone 2006). In accordance

with these aims, it needs to be decided who needs to be involved and in which way. This depends

on partners’ incentives and constraints to participate, which in turn is influenced by possible

FIGURE 1: OVERVIEW OF STATE-OF-THE-ART CONTRIBUTIONS TO THE COLLABORATIVE GOVERNANCE LIFE

CYCLE


10

power, resource and knowledge asymmetries and whether there is a history of collaboration and

trust (Ansell and Gash 2008). According to Gazley (2010), prior experience can indeed influence

the success of new partnerships, as can formal contracts, but these are not a full substitute for

actual shared objectives.

Managing the collaborative process

Once the collaboration has been initiated, collaborators need to manage their ongoing process

(Bryson, Crosby, and Stone 2015). The collaborative process consists of a cocktail of several, often

simultaneously transpiring, components that together make up the actual activities of a

collaboration (Emerson, Nabatchi, and Balogh 2011). Thomson and Perry (2006) claim that this –

what seems to be – black box of the collaborative process consists of at least five dimensions:

The governance dimension pertains to structures for reaching agreements and sharing

power. These look markedly different from traditional hierarchical organization

governance models that may seem linear and singular in purpose. To the contrary,

governance relationships in collaborations are made up of non-linear, complex policy

games: “series of interactions between actors concerning certain policy issues”

(Koppenjan and Klijn 2004, 10). Provan and Kenis (2008) claim there are essentially three

modes of governance: Shared governance where participants are in the lead together,

lead organization governance where one organization leads the others, and network

administrative organization governance where a separate entity is established to govern

all the participants (as is the case for the Coventry example). What model of governance

is most appropriate depends on levels of trust, number of participants, degree of goal

consensus and the nature of the task at hand. According to Bardach (1998, 216): “how a

collaboration should structure its steering process ought to be contingent, among other

things, on exactly what functions the structure has to perform.”

The administration dimension refers to administrative structures in place to move

collaborations from governance to actual implementation. Similar to policy games, these

structures tend to be much more complex than in single organizations: “Hierarchy,

standardization and routinization” make place for “the right combination of

administrative capacity (through coordination and elements of hierarchy) and social

capacity to build relationships” (Thomson and Perry 2006, 25).

The autonomy dimension consists of “the process of reconciling individual and collective

interests” (Thomson and Perry 2006, 26) to ensure that collaborations can act as an

autonomous entity. Mutual interest is seen by Huxham (2000, 351) as an “essential

precursor to taking joint action”, but it may be just as important to continuously manage

individual interests to ensure they do not trump the purported mutual interest that


11

existed at the start of the collaboration. Interests change, so collaborations’ aims need to

keep up.

The mutuality dimension goes beyond the above described need to reconcile individual

and collective interests. Beyond mutual objectives, there is a need to create strategic

interdependence or complementarities. If this dimension is fulfilled, the participation of

each partner becomes critical to success of the collaboration and the collaboration falls

or stands with the participation of each (Thomson and Perry 2006).

The trust and reciprocity dimension is about believing in and understanding each other’s’

good intentions. Mutual trust promotes “a positive attitude between the partners” and

allows “autonomy of action by individual partners” within the collaboration without the

fear of wrongful intent (Huxham 2000, 351).

All activity in this stage of collaborating is influenced by the extent of leadership and the capacity

and competencies of the participants (Bryson, Crosby, and Stone 2015). In collaborative settings,

“traditional heroic or position top-down leadership [...] are often suspect” (Bardach 1998). The

partners in a collaboration prefer “leadership of a kinder, gentler sort” (Bardach 1998). Ansell

and Gash (2008) refer to this type of leadership as facilitative leadership. In addition,

collaboration requires a certain type of participants with interpersonal capabilities (Crosby and

Bryson 2010). In fact, several researchers have found that “individuals and organizations are

more reliable and productive partners in collaborations when they have particular attitudes,

competencies, and capacities” (Bryson, Crosby, and Stone 2015).

Managing outcomes

The ultimate aim of many collaborations is to produce some kind of publicly valuable result,

beyond mere efforts (Moore 1995; Benington and Moore 2010; Bryson, Crosby, and Stone 2015).

The last step of the collaborative life cycle is, thus, to collectively investigate the outcomes that

have been generated by the collaborative process and account for these outcomes (Emerson,

Nabatchi, and Balogh 2011). Bryson, Crosby and Stone (2006, 52) note that “cross-sector

collaborations are more likely to be successful when they have an accountability system that

tracks inputs, process, and outcomes; [...] and use a results management system that is built on

strong relationships with key political and professional constituencies.” Measuring such

outcomes, accounting for these outcomes and adjusting the collaborative process based on this

is a complicated process generally referred to as performance management. The next section of

the article discusses the design of such a performance management system in more detail.


12

Life cycle of performance management in the public sector

In 2009, New Orleans had the highest murder rate in the United States at 52 per 100,000

residents. The new Mayor Landrieu made fighting this extremely high murder rate his top priority.

An innovation team (‘i-team’) was put to work to devise and implement plans to reduce the

murder rate. The team started by analyzing the core of the issue. Contrary to popular belief,

careful analysis showed that most of the murders were committed by a small number of people.

Based on this information, the team worked together with numerous public sector organizations

and community stakeholders to identify and prioritize a set of evidence-based initiatives targeted

at these individuals. To ensure the implementation of the thirty initiatives that were selected,

frequent stock-take meetings were held between all the partners. These meetings were used to

coordinate the partners’ efforts, build trust and reach agreements, and check progress against

key milestones. Progress was checked continuously, using a proprietary data analytics

infrastructure capable of analyzing data from a variety of partners including the police. The team

reported this progress and the results of the stock-take meeting directly to the mayor. With this

tight performance management infrastructure in place, murders in New Orleans declined by 19%

from 2012 to 2013 (De Jong and Winig 2015).

The New Orleans example illustrates the importance of performance management systems for

the collaborations to be effective. At a minimum, such performance management can be defined

as some ongoing cycle of measuring performance, analyzing this performance data to track

progress against targets, and using this performance information to adjust practices (Johnsen

2005; Moynihan 2005; Moynihan 2008; Behn 2014). In the example of the New Orleans

collaboration, performance was measured using existing data collection mechanisms of partner

organizations, analyzed by the i-team and then incorporated in the decision-making process

during stock-take meetings.

As the collaboration moves through its life cycle, performance management systems tend to

grow along. Although authors’ number and descriptions of steps differ, we identified broadly

three steps in the performance management life cycle: setting the objectives of the performance

management system, designing and operationalizing the performance measurement process,

and incorporating and using the performance data (Moynihan 2008; Van Dooren, Bouckaert, and

Halligan 2015; De Bruijn 2007; Poister 2008; Behn 2014). In this section, we attempt to give a

systematic description of this public performance management life cycle and the efforts of key

authors to define the various steps, thereby synthesizing the literature in the field while

simultaneously showing how authors’ contributions complement each other.


13

As a heuristic device, Figure 2 provides a high-level overview of some of the recent and most

influential contributions by authors that have described the steps of the performance

management life cycle. First, it describes the contributions per step of the performance

management life cycle of five of the most cited authors that attempted to describe the full life

cycle in the last decade. Second, it describes other major influential contributions to the

performance management life cycle that either preceded these five authors’ contributions or

represent a major contribution to a specific step in the performance management life cycle.

Setting the objectives of the performance management system

To be able to decide what to measure, collaborators must first of all understand the purpose

their performance management system is to serve. Performance management systems can serve

a variety of purposes in the public sector. In general, possible rationales for performance

management seem to fall in three broad categories: accountability, learning and motivation

FIGURE 2: OVERVIEW OF STATE-OF-THE-ART CONTRIBUTIONS TO THE PERFORMANCE MANAGEMENT LIFE

CYCLE IN THE PUBLIC SECTOR


14

(McLaughlin and Jordan 1999; De Bruijn 2002; Behn 2003; Propper and Wilson 2003; Hatry 2006).

For example, Behn (2003) claims there are at least eight specific reasons to measure performance

that fall into these three categories. Depending on the purpose, the system requires different

types of performance measures (see Table 1).

Whatever list one ends up using, it is important that public managers explicitly and carefully

consider the objective of their performance management system and adjust its characteristics

accordingly (Behn 2003). This process is not linear nor clearly defined. Often, there is no singular

purpose, but rather a myriad of symbolic and instrumental purposes for performance

management systems. The process to align on these purposes is messy, but navigating through

this creative messiness is essential to avoid over-simplification (Moynihan 2008). In fact, overly

simplistic objectives may lead to dysfunctional performance management regimes that steer

public officials to simplistic output measures or lead to system gaming (Van Dooren, Bouckaert,

and Halligan 2015; De Bruijn 2007).

TABLE 1: RATIONALES FOR PERFORMANCE MANAGEMENT, ADAPTED FROM BEHN (2003).


15

The New Orleans i-team is a case in point. The team intended to use its performance

management system to continuously improve their approach and motivate their partners for the

thirty initiatives. Recognizing the importance of defining the purpose of their performance

management regime per initiative, the New Orleans i-team went through the effort of defining

the individual criteria for success of each of the initiatives and designed individual performance

management regimes accordingly; rather than settling for a singular set of metrics across each.

Designing and operationalizing performance measurement process

The second step in the process of performance management starts with the actual design of

the system. This requires defining the program’s theory of change (Weiss 1998). According to

Rogers et al. (2000, 5), a theory of change is: “an explicit theory or model of how the program

causes the intended or observed outcomes.” In other words, it is “a set of hypotheses upon

which people build their program plans” (Weiss 1998, 55). This theory of change will help to

identify the combination of input, process and output intended to produce outcomes (Thomas

2010). The performance management system can be designed to measure any of these

elements. Possible measures, according to Poister (2008), are productivity measures (output

over input), service quality measures (output over process), effectiveness measures (outcomes

over output), and cost-effectiveness measures (outcomes over inputs).

As described above, to determine which actual combination and interaction of these indicators

is most important to include, one should reflect on the system’s intended purpose and envisioned

impact (Behn 2014). De Bruijn (2007) provides some guidance on the range of indicators to

include: In order to avoid overly simplistic systems that are easily gamed and focused on one

particular aspect of a program, one should consider developing these systems interactively, and

providing a great variety of indicators along the theory of change. Van Dooren, Bouckaert and

Halligan (2015) add to this the importance of minimum quality standards of the data and

methodology behind measuring the indicators, something that often gets overlooked. Finally,

Speklé and Verbeeten (2014) add that effective performance metrics typically have a high degree

of “contractibility,” in that they set clear, unambiguous targets and managers know and control

the transformation process to achieve them. As we will see below, this may introduce special

challenges for collaborations that are often called into life to solve complex, multi-faceted

problems.

Once the system is designed and indicators are confirmed, “systems and procedures are put in

place to collect and process the data that enable measurements to be made regularly” (Bourne

et al. 2000, 758) to operationalize the performance management system. Data collection may

involve gathering information from sources already in existence – as did the i-team in New


16

Orleans – or setting up new methodologies for data-gathering, such as client questionnaire

surveys, observer ratings and qualitative assessments (Thomas 2010).

The requirements for data processing vary greatly depending on the intended use and scope of

the performance management system. If data processing requirements are limited,

collaborations merely need to be willing to send each other basic data and have one person on

board who can draw patterns from this data. If there is a great amount of data available and data

processing requirements are elaborate, data processing may entail investments in new data-

sharing infrastructure and elaborate data mining systems (Brown and Brudney 2003). Yet, for

most of “this kind of analytic thinking, there exists no formula. It requires creative people who

are willing to muck around in the data, rummaging for patterns others don’t see” (Behn 2014,

149).

Incorporating the performance data

In the final phase, all gathered information is compared against the theory of change, and

possible pre-set targets, to create an account of the performance of the collaboration (Bourne

et al. 2000). This performance account is then used to provide feedback, determine rewards

and consequences, identify areas for improvement, and/or celebrate successes in line with the

intended use of the tool. Whatever the purpose, the information should somehow be

communicated (Fryer, Antony, and Ogden 2009), whether it be reported publicly (Thomas

2010) or discussed in a meeting internally (Behn 2014).

Many authors stress the need to have a constructive discussion or dialogue about the data at a

minimum (Behn 2014; Moynihan 2008; Poister 2008; De Bruijn 2007). Oftentimes, performance

data is susceptive to different (valid) interpretations by different stakeholders. For example, the

reduction in murder rates in New Orleans may be interpreted as success by some, but others may

stress that the murder rate remains above the national average or have an alternative,

exogenous explanation for the reduction. Such accounts cannot be reconciled without a

constructive discussion about different possible interpretations, causal chains and alternative

explanations (Moynihan 2008).

Finally, and perhaps most importantly, incorporating performance data into the organizational

process requires an organizational mind shift. Professionals used to having professional

discretion may view a new performance management regime as threatening. They may also

perceive the time, resources and effort required to make the system work as overly burdensome.

In this sense, creating a performance-driven organization requires organizational change with

“champions” to drive the process forward (Poister 2008). This can only be achieved through

committed leadership and strong stakeholder involvement across the organization hierarchy. In


17

this sense, performance management may be a leadership or organizational strategy as much as

it is a technical measurement process (Behn 2014).

Performance management challenges and ways to deal with these

challenges particular to collaborative efforts

As the above shows, the basics of the collaborative governance and performance management

lifecycles are well understood. Bryson, Crosby and Stone (2015, 647) call the research into

collaborative governance a “staple of public management research.” As for performance

management, according to De Waal (2013), there are more than 6,618 articles in 546 journals

and 12 million websites dedicated to the topic in general.

Yet, most of the performance management literature was written with single, mostly private

sector, organizations in mind. Similarly, most research on collaborative governance efforts did

not specifically study their performance management journeys. Even so, there is a need for more

research in this area: Collaborative work environments introduce new performance management

challenges not familiar to the single organizational context (Moynihan et al. 2011). For example,

defining a collective theory of change to combat a wicked crime problem together requires

coordination across partners and a deep understanding of the interests and capabilities each

partner brings to the table. Once all partners agree with a collective approach, they then need to

account for their performance, not only to their parent organization, but also to their

collaboration partners as well as the wider authorizing environment. These new channels of

accountability often cause stress and fear, inhibiting the collective endeavor (Behn 2002; O’Flynn

2007).


18

To start to fill this knowledge gap, we synthesized across a variety of key authors who describe

subsets of performance management challenges in collaborative governance settings to derive

three broad categories of “collaborative performance management challenges”: substantive

problem-solving challenges, collaborative process challenges and multi-relational accountability

challenges (see Figure 3). We placed these challenges along the different stages of the

collaborative and performance management lifecycles according to where they most typically

manifest. We focused on challenges that are directly relevant for the collaboration and its

members, not necessarily always for the broader environment in which the collaboration does

its work. We also comment on possible “ways out” to deal with the identified challenges as

presented by various authors.

Our synthesis forms a much-needed overview of the collaborative performance management

challenges identified by the literature on the topic, and serves as a starting point for academics

and practitioners alike to anticipate and deal with particular challenges at different stages of the

collaborative process as well as identifying key knowledge gaps.

FIGURE 3: AN OVERVIEW OF THE CHALLENGES OF COLLABORATIVE PERFORMANCE MANAGEMENT


19

Substantive problem-solving challenges

Collaborations face a host of technical problem-solving challenges related to defining the

problem at hand, developing a collective approach and designing performance measures with

which to measure the success of this approach. This challenge occurs from the start of the

collaboration, when those taking the initiative for the collaboration have to decide what problem

to focus on, until deep into the collaborative process.

Challenges The first substantive problem-solving challenge is identifying and diagnosing the

problem at hand. For example, in the case of crime problems, this process starts by the selection

of a certain undesirable behavior exhibited by a set of certain individuals and/or in a set of

locations (Sherman 1995; Goldstein 1990; Sparrow 2008). For collaborations, this first step can

be extremely challenging given that they are often established to deal with intractable, multi-

dimensional problems. The crime problems law enforcement collaborations work on, for

instance, are extremely hard to define (Head and Alford 2015), as, almost per definition, their

scope and dimensions are often unknown.

When the problem has been defined, at least tentatively, a sound theory of change has to be

developed to solve the problem (Poister 2008). The collaborative governance setting makes this

a particularly complex task. To explicate a sound theory of change a lot of information is needed

about inputs, activities, outputs, and (desired) outcomes (Poister 2008). As mentioned,

collaborations are usually set up to deal with complicated or even wicked problems. Developing

the causal links in a theory of change to tackle a wicked problem is notoriously difficult given that

one of the defining features of such problems is that their causes are poorly understood (Rittel

and Webber 1973; Sparrow 2008). In this context, the need to develop a theory of change may

actually lead to the development of an approach that focuses on only part of the problem and

thus results in myopia (De Bruijn 2007). Further complicating the process, collaborative partners

have different resources in their toolbox and come from diverse organizational backgrounds. In

this context, it is often very complex to come to a collective approach on the basis of the full

spectrum of means at the disposal of all collaboration partners (Huxham and Vangen 2013).

Collaborations also face new challenges related to measuring their performance against their

theory of change. The two key questions of performance measurement are: What to measure

and how to measure it? In terms of what to measure, collaborations in the public sector are

expected to monitor a wider range of performance standards than traditional organizational

units (Rogers and Weber 2010). Collaborative efforts, as we pointed out above, are usually set

up to deal with multi-faceted problems or ventures to generate a set of desired (publicly valuable)

outcomes where unilateral approaches have performed poorly or failed altogether. (Moore

1995; Benington and Moore 2010; Bryson, Crosby, and Stone 2015). These are not typically the


20

issues where there is a clear link between outputs and outcomes, and where outcomes are

singular and clearly defined. Rather, desired outcomes and the outputs needed to generate them

can be complex, interdependent and contended (Benington and Moore 2010). As such,

collaborative governance calls for an increasing focus on outcome indicators that go beyond

simple input and output measures (Heinrich 2002). In addition, to judge whether the

collaboration is perceived to generating its desired outcomes, collaborations must attempt to

measure the extent to which their efforts are (politically and/or publicly) supported and seen as

(democratically) legitimate (Page et al. 2015; O’Toole, Meier, and Nicholson-Crotty 2005).

Measuring such public value outcomes requires the design, implementation and use of new

performance indicators. Typically, a performance indicator should be relevant, measurable and

attributable (De Bruijn 2007). Developing performance indicators that are measurable and

attributable is complicated by the intricate nature of the problems, the collaborative character

of the work and the relationship between both of them. First of all, intended effects usually only

manifest themselves in the long term, and deciding whether they are valuable can only be done

after short-term unintended consequences of the collaborative effort have been taken into

account. For effects that do manifest themselves in the short term, it is difficult to pinpoint who

is responsible for the effect because of the involvement of multiple agencies – a version of the

‘many hands’ problem. In most cases it is impossible to establish whether there is a direct relation

at all between the collaborative effort’s work and the reduction of societal loss or generation of

public value (Sparrow 2008). The only way to truly determine whether a collaboration has

achieved a certain societal outcome is using randomized control trials or statistical field

experiments (Sparrow 2008), which are most often prohibitively expensive, ethically

questionable or impossible to conduct due to a lack of a representative sample size (Behn 2014).

At the same time, indicators that are easily measurable and attributable are often not, or only

partly, relevant in light of the societal effect to be realized.

Ways out The literature refers to a variety of ways to deal with the above challenges. Some

authors have suggested ways of dissecting wicked problems. For instance, Sparrow (2008)

proposes ways of selecting and analyzing problems to work on. He proposes to use an “unfolding

chronology” to make clear how harms come to the fore at a certain moment in time. This

chronology helps practitioners determine a moment within the unfolding chronology where they

want to intervene (ideally at some ‘critical point of control’ or ‘clean point of access’), an object

on which to focus and a method for intervention.

Many models have also been developed to build a theory of change in a structured way. One of

these models is the logic model (McLaughlin and Jordan 1999; Poister 2008). A logic model is a

diagram describing the causal link between inputs, activities, and outputs with which an


21

organization or collaboration wants to achieve its desired – initial, intermediate, and ultimate –

outcomes (see Figure 4 for an example). Inputs refer to the resources that the collaboration has

at its disposal, activities to the processes it undertakes, outputs to immediate results, and

outcomes to longer-term impacts of the collaborations’ work. Outcomes are the ultimate

objectives of collaborations, but, as discussed above, these are often hard to define and measure.

As such, collaborations oftentimes opt for a more programmatic approach to monitoring and

evaluation, measuring only outputs and immediate outcomes. Heinrich (2002) shows that, if the

theory of change is carefully defined, such output and immediate outcome measures are often

correlated with actual long-term outcome improvements, suggesting such a programmatic

approach can serve as a proxy for outcome-oriented performance measurement. In addition,

process measures may be used to track the sustainability of collaborations (Head 2008).

In order for the above-described solutions based on the theory of change to be effective, though,

they must use a range of different types of indicators and carefully distinguish between

processes, outputs and outcomes (Koontz and Thomas 2012; Thomas 2010). In addition, Rogers

(2008) shows how program theory should align with the complicated (multiple components) and

complex (emergent and non-linear models of causality) nature of programs in order to serve as

a basis for evaluation effectively.

In addition to these methods to build an effective theory of change and generate proxies for

performance measurement, there are a number of pre-structured performance measurement

frameworks. Kaplan and Norton (1996) introduced the Balanced Scorecard in the private sector,

prescribing organizations to measure indicators beyond financial success. In a similar spirit, a

number of performance measurement frameworks specifically for the public sector were

FIGURE 4: AN EXEMPLARY LOGIC MODEL FOR THE CASE OF ILLEGAL HOTEL PROSTITUTION


22

developed (Jobin 2008; Moore 2014; Moore and Braga 2003; Page et al. 2015). For example,

Moore’s (2003; 2013) public value scorecard asks public managers to measure their performance

in terms of public value creation, their operational efficiency and the support from their

authorizing environment. Emerson and Nabatchi (2015b) even created a “performance matrix”

specific to the context of collaborative governance, suggesting measures at the intersection of

three performance levels – action/outputs, outcomes, and long-term adaptation – and three

units of analysis – the participant organization, the collaborative governance regime, and the

ultimate target goals.

While these scorecards can be useful, Skelcher and Sullivan (2008) caution against solely using

metric-driven approaches that start from what metrics are available in favor of theory-driven

approaches that rely on a priori models of the causal relationships between different elements

of a theory of change. Only the latter can accurately reflect the multi-faceted performance of

collaborative performance according to them. In addition, Poocharoen and Hoi-Lam Wong (2016)

highlight how performance measurement systems should adapt their performance metrics over

time to capture more intangible aspects of performance and allow for increased learning as

collaborative relationships deepen.

Knowledge gaps The above pre-structured frameworks, and others like them, help to structure

collaborators’ problem-solving process and make explicit what is to be measured. Yet, even when

these are implemented, substantive problem-solving challenges seem to linger, as the inherent

causal intricacies of the wicked problems that these collaborations are dealing with remain

(Roberts 2000; Weber and Khademian 2008; Head and Alford 2015). In addition, most of them

have been developed independently from the needs of collaborative governance efforts and do

not explicitly zoom in on the particularities of this context. For example, even if the causal

intricacies referred to above could be dissected, the job of defining a common approach along

with partners whose full range of means are unknown to each other lingers. The call for more

thorough research and practical advice to deal with such persistent challenges remains, in

particular to move from the suggested abstract theoretical frameworks to the day-to-day realities

of collaborating.

Collaborative process challenges

Collaborative process challenges arise during the actual day-to-day collaborative interactions due

to the difficulties in working together with organizations different from one’s own. From the

outset, these collaborative process challenges may not look like performance management

challenges, but overcoming these challenges is key to accepting a collaborative performance

management regime and channeling performance in the right direction. After all, it is only by

reaching agreement about and committing to a mutually beneficial approach that collaborations


23

can ever hope to perform, although collaborative partners might still have different reasons to

take part in the collaboration (Ansell and Gash 2008; Emerson, Nabatchi, and Balogh 2011).

Challenges For collaborators to accept any form of dedicated collaborative performance

management, they must trust each other. Initially, before a collaboration can become effective,

collaborators may be apprehensive about other collaborators’ good intentions or capacity. Both

are sources of considerable risk to the success of the strategic effort that the collaboration

pursues (Das and Teng 2001). To build trust, the collaborators must generate reciprocal

commitment to each other’s willingness to collaborate (Ostrom 1998; Bardach 1998; Ansell and

Gash 2008). Given the sometimes arbitrary nature of how collaborations are established and

possible prejudices about other collaborators’ way of working, this process of building trust is

considered key in establishing effective partnerships (Bryson, Crosby, and Stone 2006; Thomson

and Perry 2006).

Once basic trust has been established, collaborators should attempt to generate common

commitment to the collaboration, which can be quite difficult in the midst of competing

commitments with, for instance, the parent organizations. Stakeholders will only get committed

if there is enough respect for their perspectives and interests (Ansell and Gash 2008). In addition,

there must be a meaningful prospect of mutual gain for all stakeholders (Bryson, Crosby, and

Stone 2006). It is only in such a context of interdependency that collaborators can be expected

to “abide by the results of deliberation, even if they should go in the direction that a stakeholder

does not fully support” (Ansell and Gash 2008).

To design an effective performance management system, collaborators must also attempt to

reach a shared understanding on their goals and approach (Ansell and Gash 2008). There may

well be a misalignment over goals or targets to be achieved as part of the collaborative effort

(Moynihan et al. 2011). As Page (2004, 591) notes: “Reasonable people may disagree about which

results to measure.” Moynihan et al. (2011) stress that this is especially true for the complex

governance contexts in which collaborations find themselves: “In complex governance settings,

the greater heterogeneity of influential actors is likely to result in more marked battles about the

definition of performance”. Reaching a shared understanding will be more complicated the

denser the collaboration’s network is (Provan and Kenis 2008).

Ways out Given all the scholarly work on team processes and human interactions, many possible

solutions for collaborative process challenges can be found across several fields of study –

teaming research, organizational science, human resource management, and leadership studies,

to name a few. It would be impossible to mention all of these potential solutions here, so we will

only mention a few key ones: Firstly, to build trust one must “assess the trustworthiness of


24

[other] individuals and the institutions they represent; persuade others of one’s own

trustworthiness; learn how to trust; and take collective actions to develop trust” (Bardach 1998).

Procedural tools such as collaboration capability-building and physical collocation of the

collaborators can be used to further this process (Bardach 1998).

After the establishment of a minimum level of trust, the commitment of the collaboration

partners can be generated by using clear, fair and transparent procedures. This creates an

environment in which all partners know whether their interests are taken into account and can

hold the collaboration accountable (Ansell and Gash 2008). In addition, to reach agreement,

facilitative leadership can be of instrumental importance: For example, the facilitation of external

managers can generate a culture of joint problem-solving that can strongly help in reaching

agreement (Agranoff 2005).

Knowledge gaps While there is a long list of potential solutions to deal with collaborative process

challenges that reaches far beyond the solutions listed above, their impact remains to be tested

in the context of collaborative governance efforts. The intricate nature of the issues collaborative

governance endeavors typically deal with make designing a functional collaborative process

especially complicated. In particular, one must be careful not to over-structure the collaborative

process, thereby killing the very constructive conflict and continuous adaptation that makes

collaboration a successful mechanism to tackle multi-faceted problems (Stoker 2006). In fact,

Ansell and Gash (2008) view the collaborative process as a non-linear, iterative process that is

essentially impossible to pre-structure or capture on paper. Designing interventions for these

particular challenges will thus require a deeper understanding of the causes of collaborative

process challenges and a more detailed exploration of how such interventions sustain non-linear

processes and constructive group conflict in practice.

Multi-relational accountability challenges

The collaborative performance management life cycle ends in a feedback loop: The collaborative

process leads to some results, for which the collaboration needs to account itself (Bryson, Crosby,

and Stone 2015). When one thinks of collaborative performance management challenges,

accountability challenges may come to mind first, as the very nature of a collaboration opens

new channels of accountability and brings tensions with the old channels that exist within

organizations (Gains and Stoker 2009; Ryan and Walsh 2004; Page et al. 2015).

Challenges The first new channel of accountability is towards collaboration partners. This new

channel may raise a variety of tensions. Who is accountable to whom and for what is often

unclear (Bardach and Lesser 1996; Page 2004). Collaboration “creates complex lines of

accountability making attribution of blame or credit for collaborative actions difficult” (Page et


25

al. 2015, 3) Some collaborators may resist being held accountable by other organizations at all,

fearing that “they will not perform well — either because they doubt their own capacity, or

because circumstances beyond their control may influence the results they are asked to achieve”

(Page 2004, 591). In this blurred network of accountability, collaborators may attempt to game

the collaborative performance management system (De Bruijn 2007). For example, an individual

collaborator’s performance may be judged based on a mix of that collaborator’s own inputs as

well as the outputs and outcomes of the entire collaboration. In this system, individual

collaborators may steer their effort such that they contribute the minimum required input and

take credit for the entire collaboration’s achievement. Such shirking behavior is often observed

in collaborative governance efforts (Ring and Van De Ven 1994).

In addition to intra-collaborative accountability, collaborations may face new channels of

accountability towards their authorizing environment. The new model of collaborative

governance assumes that professionals take increasing discretion to coordinate with partners

and generate (publicly valuable) outcomes beyond their traditional role descriptions (Moore

1995; Bardach 1998). Given the increased discretion required to interpret policies and move

beyond traditional professional roles, the results of collaborative governance efforts must be

defended in the democratic arena (O’Flynn 2007). As a result, collaborators “find themselves

more frequently being held to account publicly yet possibly for policies they do not own” (Gains

and Stoker 2009, 448). New channels of accountability may include shorter reporting lines to top

management or politicians as well as direct communication with citizens (Klijn and Koppenjan

2000; Gains and Stoker 2009). For instance, in the case of the New Orleans i-team, the team

reported directly to the mayor and communicated their results to the public at large.

There is a tension between the very concept of increased discretion and accountability towards

other stakeholders and traditional channels of accountability (Moynihan et al. 2011).

Collaborators may not be able to account for time spent on collaborative work to their parent

organizations or they may find it hard to explain why they prioritized collaborative work over

routine tasks to their superiors. In fact, “existing accountability mechanisms are designed for

vertical accountability relationships [...]. These are inadequate for horizontal or ‘networked’

accountability across government agencies” (Ryan and Walsh 2004, 621). In networked

governance, responsibility is often shared and performance cannot be attributed to one single

person. Traditional accountability systems do not have the capacity to deal with such complexity

(Kettl 2000). To resolve this tension, a “complete mental reorientation” (Behn 2002, 9) of public

managers and their authorizing environment is required (Page 2004). Yet, every organization in

a collaboration will continue to have its own mandates and reporting requirements. As a result,

the accountability relations of the collaboration itself must take shape in the wider context of

existing accountability structures (Page et al. 2015).


26

Ways out The literature proposes a number of ways to deal with multi-relational accountability

challenges. First of all, many authors suggest that it must be made very explicit for what (to the

extent that this is possible to capture) and to whom a collaboration and collaborators are

accountable. Such clarity is essential in shaping healthy accountability relations (Bardach and

Lesser 1996). When this is clear, the next step of the process is to actually review and discuss the

collaborations’ performance regularly and consistently. According to Behn (2014), an

organization’s leadership should be committed to monitor and steer the performance of

subordinates continuously to ensure an actual impact on their approach. Of course, this raises

further questions about what the ‘leadership’ of a collaboration actually consists of. As we saw

above, this could be the collaborators themselves, their parent organizations or even the public

at large.

While leaders must continuously monitor and review a collaboration’s performance, leaders

must also give an unusual degree of discretion to collaborations and collaborators (Bardach

1998), thereby creating an atmosphere of responsibility, participation and joint learning.

Agranoff (2005, 34) agrees: “Leaders must go beyond lip service to facilitate an internal

atmosphere that generates participation, sharing of information, joint learning, and participative

decision making”. This simultaneously entails not steering on precise inputs, processes and

outputs.

Knowledge gaps In practice, these seemingly conflicting messages of clearly delineating

accountability and closely monitoring performance versus providing greater space and flexibility

to subordinates to collaborate may confuse public managers. For example, the i-team in New

Orleans needed to get ample space to select and test trial solutions. Yet, the direct reporting line

to the mayor and pressure to solve this priority issue incentivized the team to show progress.

How to deal with this underlying accountability paradox remains a question for public managers

seeking to stimulate collaborative governance today, a question that was already tough to crack

within single public sector organizations (Van Thiel and Leeuw 2002).

Conclusion: towards a greater understanding of collaborative

performance management challenges and ways to deal with them

This article aimed to provide an overview of the literature on collaborative performance

management. We have shown that there is a significant body of knowledge on performance

management in collaborative settings that can help inform practice. The literature provides

ample insight in the functioning of collaborative efforts and performance management in the

public sector in general, and highlights particular performance management challenges along the


27

collaborative governance life cycle and ways to deal with them. Nevertheless, in our synthesis,

we also showed that there remain some key knowledge gaps regarding the nature and causes of

these challenges and the effectiveness of interventions. This section summarizes our findings and

highlights avenues for further research to deal with these knowledge gaps.

Synthesis of findings and knowledge gaps

Although studies on these individual topics will never be completed, our understanding of the

performance management and collaborative governance life cycles by themselves could be

described as relatively mature. To the contrary, understanding the performance management

challenges that collaborations face in particular is a relatively nascent, but vibrant, knowledge

area. To contribute to the field’s understanding of this particular area, we synthesized across

different authors’ work in the field and identified three broad categories of collaborative

performance management challenges: substantive problem-solving challenges, collaborative

process challenges, and multi-relational accountability challenges. We described these

performance management challenges and ways that have been suggested to deal with them. We

also placed them along the collaborative governance and performance management life cycles

to guide researchers and practitioners alike in their journey to understand what matters at what

stage of the life cycle to collaborate more effectively.

Through our synthesis, we were able to generate a rich picture of the current landscape of

collaborative performance management challenges, which also helped us to identify current

knowledge gaps. In this way, we identified three categories of knowledge gaps across all the

challenges:

1. Lack of empirical understanding of manifestations and causes of collaborative

performance management challenges The literature shows a relatively comprehensive

picture of the categories of challenges we describe. Yet, many of the sub-challenges have

either been developed conceptually only and are yet to be studied in practice or have

been distilled from single case studies with limited external validity. As a result, the

various possible manifestations of these challenges in different contexts as well as their

underlying causes are still imperfectly understood. For example, while it is clear that

there is a tension between old vertical channels of accountability and new horizontal

channels of accountability, we have an incomplete view on how this tension manifests

itself differently in different sectors.

2. Incomplete and unsystematic understanding of ways to deal with collaborative

performance management challenges In addition to the knowledge gaps regarding the

nature and causes of collaborative performance management challenges, the landscape

of ways to deal with them is also quite patchy. While we showed there are certainly tried


28

and accepted ways to deal with some aspects of the challenges described, there is no

systematic solution to deal with the challenges holistically and prioritize amongst the

different possible strategies based on contextual factors. For example, it is unclear

whether a collaboration should already focus on developing their accountability

mechanism when they have not yet clearly defined the problem they are working on. In

another example, it remains unclear how the tension between the need for increased

discretion and closely monitoring results can be resolved.

3. Lack of empirical understanding of ways to deal with performance management

challenges and their applicability to the context of collaborative governance In our

synthesis, we showed that many of the ways to deal with performance management

challenges that were developed outside of the collaborative governance settings can be

considered as potential interventions in this particular context. Simultaneously, this

creates questions about their applicability. Many pre-structured problem-solving

frameworks and collaborative process methods were developed in the context of single

organizations or private sector teams. Momentarily, our empirical understanding of their

impact in the collaborative governance context is limited. Where the impact of such

interventions on collaborative performance has been explored, it often remains at the

level of single case studies with insufficient scale to confirm their generalizable impact

beyond the case.

Further research

The knowledge gaps described imply that future research should focus on building on the broad

body of existing studies together with practitioners to further root existing findings and confirm

and test them in practice. Continued conceptual research and literature synthesis is valuable: For

example, recent work by Bryson, Crosby and Stone (2015) synthesized our knowledge on the

collaborative process and Head and Alford (2015) helped us identify strategies to deal with

wicked problems in the public sector. However, there is a marked need to test these conceptual

models against actual practice.

Perry (2012) recommends increased usage of experimental research and meta-analyses in public

administration research to root conclusions in empiricism. In line with this recommendation,

more and more authors have indeed opted for a more empirical approach to study collaborative

governance efforts. Bryson, Crosby and Stone (2015), for instance, identified 191 such empirical

studies on varying aspects of the collaborative governance life cycle. Yet, there is a need to build

on these research methodologies in collaborative governance to examine questions pertaining

specifically to performance management.


29

In this regard, we propose several avenues for further research. First of all, possible underlying

causes for the challenges of collaborative performance management must be explored in more

detail in practice. To do this, action research, in which researchers observe the struggles of

collaborations at first hand, could help to sharpen our understanding of the intricacies of

“collaborative inertia.” In addition, comparative case studies could explore the different

manifestations of the challenges in different sectors or countries.

Insights into the root causes of these challenges and contextual differences can then be used to

design effective and comprehensive performance management strategies for collaborations. The

impact of these strategies should be investigated in more detail in practice as well. Initially,

empirical research should focus on compiling case studies of effective performance management

strategies that have been used across collaborations in different contexts. Following this, action

research could be used to pilot and perfect such possible interventions. Eventually, if knowledge

on possible effective collaborative performance management strategies has matured, such

interventions could be tested at larger scale using field experiments.

If these research avenues are pursued effectively, today’s case studies, including the cases on the

collaborative fight against human trafficking in the Netherlands, the Crime and Disorder

Reduction Partnership in Coventry and the i-team to reduce New Orleans’ murder rate only

comprise the start of exploration in this field.


30

References

Agranoff, Robert. 2005. “Managing Collaborative Performance: Changing the Boundaries of the

State?” Public Performance & Management Review 29 (1): 18–45.

———. 2006. “Inside Collaborative Networks: Ten Lessons for Public Managers.” Public

Administration Review 66 (Special Issue): 56–65. doi:10.1111/j.1540-6210.2006.00666.x.

———. 2007. Managing within Networks: Adding Value to Public Organisations. Washington

DC: Georgetown University Press.

Agranoff, Robert, and Michael McGuire. 2001. “Big Questions in Public Network Management

Research.” Journal of Public Administration Research and Theory 11 (3): 295–326.

doi:10.2307/3525819.

———. 2003. Collaborative Public Management: New Strategies for Local Governments.

Washington, DC: Georgetown University Press.

Alford, John, and Owen Hughes. 2008. “Public Value Pragmatism as the Next Phase of Public

Management.” The American Review of Public Administration 38 (2): 130–48.

doi:10.1177/0275074008314203.

Ansell, Chris, and Alison Gash. 2008. “Collaborative Governance in Theory and Practice.” Journal

of Public Administration Research and Theory 18 (4): 543–71. doi:10.1093/jopart/mum032.

Bardach, Eugene. 1998. Getting Agencies to Work Together: The Practice and Theory of

Managerial Craftsmanship. Washington, DC: Brookings Institution Press.

Bardach, Eugene, and Cara Lesser. 1996. “Accountability in Human Services Collaboratives—For

What? And To Whom?” Journal of Public Administration Research and Theory 6 (2): 197–

224. doi:10.1093/oxfordjournals.jpart.a024307.

Behn, Robert D. 2002. “The Psychological Barriers to Performance Management: Or Why Isn’t

Everyone Jumping on the Performance-Management Bandwagon?” Public Performance &

Management Review 26 (1): 5–25.

———. 2003. “Why Measure Performance? Different Purposes Require Different Measures.”

Public Administration Review 63 (5): 586–606. doi:10.1111/1540-6210.00322.

———. 2014. The PerformanceStat Potential: A Leadership Strategy for Producing Results.

Washington, DC: Brookings Institution Press.

Benington, John, and Mark H. Moore. 2010. Public Value: Theory and Practice. New York:

Palgrave Macmillan.

Bevan, Gwyn, and Christopher Hood. 2006. “What’s Measured Is What Matters: Targets and

Gaming in the English Public Health Care System.” Public Administration 84 (3): 517–38.

doi:10.1111/j.1467-9299.2006.00600.x.

Bingham, Lisa B., Tina Nabatchi, and Rosemary O’Leary. 2005. “The New Governance: Practices

and Processes for Stakeholder and Citizen Participation in the Work of Government.”

Public Administration Review 65 (5): 547–58. doi:10.1111/j.1540-6210.2005.00482.x.


31

Bourne, Mike, John Mills, Mark Wilcox, Andy Neely, and Ken Platts. 2000. “Designing,

Implementing and Updating Performance Measurement Systems.” International Journal of

Operations & Production Management 20 (7): 754–71. doi:10.1108/01443570010330739.

Brown, Mary M., and Jeffrey L. Brudney. 2003. “Learning Organizations in the Public Sector? A

Study of Police Agencies Employing Information and Technology to Advance Knowledge.”

Public Administration Review 63 (1): 30–43. doi:10.1111/1540-6210.00262.

Bruijn, Hans De. 2002. “Performance Measurement in the Public Sector: Strategies to Cope with

the Risks of Performance Measurement.” International Journal of Public Sector

Management 15 (7): 578–94. doi:doi:10.1108/09513550210448607.

———. 2007. Managing Performance in the Public Sector. Second ed. New York: Routledge.

Bryson, John M., Barbara C. Crosby, and Laura Bloomberg. 2014. “Public Value Governance:

Moving Beyond Traditional Public Administration and the New Public Management.”

Public Administration Review 74 (4): 445–56. doi:10.1111/puar.12238.

Bryson, John M., Barbara C. Crosby, and Melissa M. Stone. 2006. “The Design and

Implementation of Cross-Sector Collaboration : Propositions from the Literature.” Public

Administration Reviewstration Review 66 (Special Issue): 17–18. doi:10.1111/j.1540-

6210.2006.00665.x.

———. 2015. “Designing and Implementing Cross-Sector Collaborations: Needed and

Challenging.” Public Administration Review 75: 647–63.

doi:10.1111/puar.12432.Designing.

Clarke, Michael, and John Stewart. 1997. Handling the Wicked Issues: A Challenge for

Government. Birmingham: University of Birmingham, Institute of Local Government

Studies.

Crosby, Barbara C., and John M. Bryson. 2010. “Integrative Leadership and the Creation and

Maintenance of Cross-Sector Collaborations.” Leadership Quarterly 21 (2): 211–30.

doi:10.1016/j.leaqua.2010.01.003.

Das, T. K., and Bing-Sheng Teng. 2001. “Trust, Control, and Risk in Strategic Alliances: An

Integrated Framework.” Organization Studies 22 (2): 251–83.

doi:10.1177/07399863870092005.

Dijk, Jan J. M. van. 1995. “In Search of Synergy: Coalition-Building against Crime in the

Netherlands.” Security Journal 1 (6): 7–11.

Dooren, Wouter van, Geert Bouckaert, and J. Halligan. 2015. Performance Management in the

Public Sector. London: Routledge.

Emerson, Kirk, and Tina Nabatchi. 2015a. Collaborative Governance Regimes. Washington DC:

Georgetown University Press.

———. 2015b. “Evaluating the Productivity of Collaborative Governance Regimes: A

Performance Matrix.” Public Performance & Management Review 38 (4): 717–47.

doi:10.1080/15309576.2015.1031016.


32

Emerson, Kirk, Tina Nabatchi, and Stephen Balogh. 2011. “An Integrative Framework for

Collaborative Governance.” Journal of Public Administration Research and Theory 22 (1):

1–29. doi:10.1093/jopart/mur011.

Fryer, Karen, Jiju Antony, and Susan Ogden. 2009. “Performance Management in the Public

Sector.” International Journal of Public Sector Management 22 (6): 478–98.

doi:10.1108/09513550910982850.

Gains, Francesca, and Gerry Stoker. 2009. “Delivering ‘Public Value’: Implications for

Accountability and Legitimacy.” Parliamentary Affairs 62 (3): 438–55.

doi:10.1093/pa/gsp007.

Garland, David. 1996. “The Limits of the Sovereign State: Strategies of Crime Control in

Contemporary Society.” The British Journal of Criminology 36 (4): 445–71.

doi:10.1093/oxfordjournals.bjc.a014105.

Gazley, Beth. 2010. “Linking Collaborative Capacity to Performance Measurement in

Government--Nonprofit Partnerships.” Nonprofit and Voluntary Sector Quarterly 39 (4):

653–73. doi:10.1177/0899764009360823.

Goldsmith, Stephen, and William D. Eggers. 2004. Governing by Network: The New Shape of the

Public Sector. Washington, DC: Brookings Institution Press.

Goldstein, Herman. 1990. Problem-Oriented Policing. New York: McGraw-Hill, Inc.

Hatry, Harry P. 2006. Performance Measurement: Getting Results. Washington, DC: The Urban

Institute Press.

Head, Brian W. 2008. “Assessing Network-Based Collaborations.” Public Management Review

10 (6): 733–49. doi:10.1080/14719030802423087.

Head, Brian W., and John Alford. 2015. “Wicked Problems: Implications for Public Policy and

Management.” Administration & Society 47 (6): 711–39. doi:10.1177/0095399713481601.

Heinrich, Carolyn J. 2002. “Outcome-Based Performance Management in the Public Sector :

Implications for Government Accountability and Efectiveness.” Public Administration

Review 62 (6): 712–25. doi:10.1111/1540-6210.00253.

Huxham, Chris. 2000. “The Challenge of Collaborative Governance.” Public Management: An

International Journal of Research and Theory 2 (3): 337–58.

doi:10.1080/14719030000000021.

Huxham, Chris, and Siv Vangen. 2013. Managing to Collaborate: The Theory and Practice of

Collaborative Advantage. New York: Routledge.

Jobin, Denis. 2008. “A Transaction Cost-Based Approach to Partnership Performance

Evaluation.” Evaluation 14 (4): 437–65. doi:10.1177/1356389008095487.

Johnsen, Åge. 2005. “What Does 25 Years of Experience Tell Us about the State of Performance

Measurement in Public Policy and Management?” Public Money and Management 25 (1):

9–17. doi:10.1111/j.1467-9302.2005.00445.x.

Jong, Jorrit de, and Laura Winig. 2015. “Innovating by the Book: The Introduction of Innovation


33

Delivery Teams in Memphis and New Orleans.” Cambridge, Massachusetts: Harvard

Kennedy School of Government.

Kaplan, Robert S., and David P. Norton. 1996. The Balanced Scorecard: Translating Strategy into

Action. Boston, Massachusetts: Harvard Business Press.

Kelman, Steven, and Richard W. Waterhead. 2006. “Going It Together: Coventry’s Community

Safety Partnership.” Cambridge, Massachusetts: Harvard Kennedy School of Government.

Kettl, Donald F. 2000. “The Transformation of Governance: Globalization, Devoution, and the

Role of Government.” Public Administration Review 60 (6): 488–97. doi:10.1111/0033-

3352.00112.

Klijn, Erik-Hans, and Joop F. M. Koppenjan. 2000. “Public Management and Policy Networks.”

Public Management: An International Journal of Research and Theory 2 (2): 135–58.

doi:10.1080/14719030000000007.

Klijn, Erik-Hans, Joop F. M. Koppenjan, and Katrien Termeer. 1995. “Managing Networks in the

Public Sector: A Theoretical Study of Management Strategies in Policy Networks.” Public

Administration 73 (3): 437–54. doi:10.1111/j.1467-9299.1995.tb00837.x.

Koontz, Tomas M., and Craig W. Thomas. 2012. “Measuring the Performance of Public-Private

Partnerships.” Public Performance & Management Review 35 (4): 769–86.

doi:10.2753/PMR1530-9576350410.

Koppenjan, Joop F. M., and Erik-Hans Klijn. 2004. Managing Uncertainties in Networks: A

Network Approach to Problem Solving and Decision Making. London: Routledge.

Levi, Michael, and Mike Maguire. 2004. “Reducing and Preventing Organised Crime: An

Evidence-Based Critique.” Crime, Law and Social Change 41 (5): 397–469.

doi:10.1023/B:CRIS.0000039600.88691.af.

McGuire, Michael. 2006. “Collaborative Public Management: Assessing What We Know and

How We Know It.” Public Administration Review 66 (Special issue): 33–43.

doi:10.2307/4096568.

McLaughlin, John A., and Gretchen B. Jordan. 1999. “Logic Models: A Tool for Telling Your

Program’s Performance Story.” Evaluation and Program Planning 22 (1): 65–72.

Modell, Sven. 2004. “Performance Measurement Myths in the Public Sector: A Research Note.”

Financial Accountability & Management 20 (1): 39–55. doi:10.1111/j.1468-

0408.2004.00185.x.

Moore, Mark H. 1995. Creating Public Value: Strategic Management in Government.

Cambridge, Massachusetts: Harvard University Press. doi:10.2307/2952401.

———. 2003. “The Public Value Scorecard: A Rejoinder and an Alternative to ‘Strategic

Performance Measurement and Management in Non-Profit Organizations’ by Robert

Kaplan.” Working Paper No. 18. Cambridge, MS: Hauser Center for Nonprofit

Organizations. doi:10.2139/ssrn.354082.

———. 2014. “Public Value Accounting: Establishing the Philosophical Basis.” Public


34

Administration Review 74 (4): 465–77. doi:10.1111/puar.12198.Public.

Moore, Mark H., and Anthony A. Braga. 2003. “Measuring and Improving Police Performance:

The Lessons of Compstat and Its Progeny.” Policing: An International Journal of Police

Strategies & Management 26: 439–53. doi:10.1108/13639510310489485.

Moynihan, Donald P. 2005. “Goal Based Learning and the Future of Performance

Management.” Public Administration Review 65 (2): 203–16. doi:10.1111/j.1540-

6210.2005.00445.x.

———. 2008. The Dynamics of Performance Management: Constructing Information and

Reform. Washington, DC: Georgetown University Press.

Moynihan, Donald P., Sergio Fernandez, Soonhee Kim, Kelly M. Leroux, Suzanne J. Piotrowski,

Bradley E. Wright, and Kaifeng Yang. 2011. “Performance Regimes amidst Governance

Complexity.” Journal of Public Administration Research and Theory 21 (Suppl. 1): 141–55.

doi:10.1093/jopart/muq059.

O’Flynn, Janine. 2007. “From New Public Management to Public Value: Paradigmatic Change

and Managerial Implications.” Australian Journal of Public Administration 66 (3): 353–66.

doi:10.1111/j.1467-8500.2007.00545.x.

O’Toole Jr., Laurence J., Kenneth J. Meier, and Sean Nicholson-Crotty. 2005. “Managing

Upward, Downward and Outward.” Public Management Review 7 (1): 45–68.

doi:10.1080/1471903042000339419.

Ostrom, Elinor. 1998. “Theory of Collective Action A Behavioral Approach to the Rational Choice

Presidential Address, American Political Science Association, 1997.” American Political

Science Review 92 (1): 1–22.

Page, Stephen B. 2004. “Measuring Accountability for Results in Interagency Collaborates.”

Public Administration Review 64 (5): 591–606. doi:10.1111/j.1540-6210.2004.00406.x.

Page, Stephen B., Melissa M. Stone, John M. Bryson, and Barbara C. Crosby. 2015. “Public Value

Creation by Cross-Sector Collaborations: A Framework and Challenges of Assessment.”

Public Administration 93 (3): 715–32. doi:10.1111/padm.12161.

Perry, James L. 2012. “How Can We Improve Our Science to Generate More Usable Knowledge

for Public Professionals?” Public Administration Review 72 (4): 479–82. doi:10.1111/j.1540-

6210.2012.02607.x.

Poister, Theodore H. 2008. Measuring Performance in Public and Nonprofit Organizations. San

Francisco: Jossey-Bass.

Poocharoen, Ora-orn, and Norma Hoi-Lam Wong. 2016. “Performance Management of

Collaborative Projects: The Stronger the Collaboration, the Less Is Measured.” Public

Performance & Management Review 39 (3): 607–29.

Propper, Carol, and Deborah Wilson. 2003. “The Use and Usefulness of Performance Measures

in the Public Sector.” Oxford Review of Economic Policy 19 (2): 250–67.

doi:10.1093/oxrep/19.2.250.


35

Provan, Keith G., and Patrick Kenis. 2008. “Modes of Network Governance: Structure,

Management, and Effectiveness.” Journal of Public Administration Research and Theory 18

(2): 229–52. doi:10.1093/jopart/mum015.

Provan, Keith G., and H. Brinton Milward. 2001. “Do Networks Really Work? - A Framework for

Evaluating Public-Sector Organizational Networks.” Public Administration Review 61 (4):

414–23. doi:10.5465/APBPP.1999.27628083.

Rhodes, Roderick A. W. 1996. “The New Governance: Governing without Government.” Political

Studies 44: 652–67. doi:10.1111/j.1467-9248.1996.tb01747.x.

Ring, Peter S., and Andrew H. van de Ven. 1994. “Developmental Processes of Cooperative

Interorganizational Relationships.” Academy of Management Review 19 (1): 90–118.

doi:10.5465/AMR.1994.9410122009.

Rittel, Horst W. J., and Melvin M. Webber. 1973. “Dilemmas in a General Theory of Planning.”

Policy Sciences 4 (1973): 155–69.

Roberts, Nancy. 2000. “Wicked Problems and Network Approaches to Resolution.”

International Public Management Review 1 (1): 1–19.

Rogers, Ellen, and Edward P. Weber. 2010. “Thinking Harder About Outcomes for Collaborative

Governance Arrangements.” The American Review of Public Administration 40 (5): 546–67.

doi:10.1177/0275074009359024.

Rogers, Patricia J. 2008. “Using Programme Theory to Evaluate Complicated and Complex

Aspects of Interventions.” Evaluation 14 (1): 29–48. doi:10.1177/1356389007084674.

Rogers, Patricia J., Anthony Petrosino, Tracy A. Huebner, and Timothy A. Hacsi. 2000. “Program

Theory Evaluation: Practice, Promise, and Problems.” New Directions for Evaluation 2000

(87): 5–13. doi:10.1002/ev.1177.

Rosenbaum, Dennis P. 2002. “Evaluating Multi-Agency Anti-Crime Partnerships: Theory, Design,

and Measurement Issues.” Crime Prevention Studies 14: 171–225.

Ryan, Christine, and Peter Walsh. 2004. “Collaboration of Public Sector Agencies: Reporting and

Accountability Challenges.” International Journal of Public Sector Management 17 (7):

621–31. doi:10.1108/09513550410562284.

Sherman, Lawrence W. 1995. “Hot Spots of Crime and Criminal Careers of Places.” Crime and

Place 4: 35–52.

Skelcher, Chris, and Helen Sullivan. 2008. “Theory-Driven Approaches to Analysing Collaborative

Performance.” Public Management Review 10 (6): 751–71.

doi:10.1080/14719030802423103.

Sparrow, Malcolm K. 2008. The Character of Harms: Operational Challenges in Control.

Cambridge: Cambridge University Press.

Speklé, Roland F., and Frank H.M. Verbeeten. 2014. “The Use of Performance Measurement

Systems in the Public Sector: Effects on Performance.” Management Accounting Research

25 (2). Elsevier Ltd: 131–46. doi:10.1016/j.mar.2013.07.004.


36

Stoker, Gerry. 2006. “Public Value Management: A New Narrative for Networked Governance?”

American Review of Public Administration 36 (1): 41–57.

Thiel, Sandra Van, and Frans L. Leeuw. 2002. “The Performance Paradox in the Public Sector.”

Public Performance & Management Review 25 (3): 267–81. doi:10.2307/3381236.

Thomas, John C. 2010. “Outcome Assessment and Program Evaluation.” In The Jossey-Bass

Handbook of Nonprofit Leadership and Management, edited by David O. Renz, Third ed.,

401–30. San Francisco: Jossey-Bass.

Thomson, Ann M., and James L. Perry. 2006. “Collaboration Processes: Inside the Black Box.”

Public Administration Review 66 (December): 20–32. doi:10.1111/j.1540-

6210.2006.00663.x.

Waal, Andre de. 2013. Strategic Performance Management: A Managerial and Behavioral

Approach. Second Ed. New York: Palgrave Macmillan.

Weber, Edward P., and Anne M. Khademian. 2008. “Wicked Problems, Knowledge Challenges,

and Collaborative Capacity Builders in Network Settings.” Public Administration Review 68

(2): 334–49. doi:10.1111/j.1540-6210.2007.00866.x.

Weiss, Carol H. 1998. Evaluation. Sec. ed. Upper Saddle River, NJ: Prentice-Hall.

Maurits Waardenburg Martijn Groenleer · 2 Managing the Performance of Public Sector...

Documents

Transcript of Maurits Waardenburg Martijn Groenleer · 2 Managing the Performance of Public Sector...