INNOVATIONS IN GOVERNANCE MEASUREMENT Public Accountability Mechanisms (PAM) Initiative Francesca...
-
Upload
jaylen-crosthwaite -
Category
Documents
-
view
213 -
download
1
Transcript of INNOVATIONS IN GOVERNANCE MEASUREMENT Public Accountability Mechanisms (PAM) Initiative Francesca...
INNOVATIONS IN GOVERNANCE MEASUREMENT
Public Accountability Mechanisms (PAM) Initiative
Francesca Recanatini & Stephanie E. TrapnellApril 26, 2013
1
Outline
IntroductionsObjectives and potential outputs ISPMS, AGI, and behavioral dimensions of measurement
Public Accountability Mechanisms De jure and De facto
“Big ideas” for today
2
Objectives of the Workshop
To discuss innovations in the measurement of “de facto” aspects of governance focusing on: the gaps in existing data coverage, the strengths and weaknesses of recent
data collection methodologies, and the sustainability of these efforts.
If possible, to identify a “core” set of governance areas and related indicators (both existing and potential)
3
Outputs4
Menu of governance indicators and efforts
Draft note on methodologies Annotated bibliography of sources
Technical expert group (TEG)
A proposal and a new way forward
ISPMS, AGI, and behavioral impacts5
The challenge
To measure performance of institutions, and in particular, public sector institutions
To identify areas for institutional reform To evaluate the impact of change in
policies at the country and local level
6
Complementary sets of indicators
Measurement of the performance of Public Sector Management (PSM) systems in the middle of the public sector results chain.
Measurement of the way in which the government is held to account through political and non-executive institutions and directly by the public.
7
Indicators of the Strength of Public Management Systems (ISPMS)
Actionable Governance Indicators (AGIs)
Public sector results chain within a broader governance environment
8B
roader
Govern
ance
Envir
onm
ent B
roader G
overn
ance
Enviro
nm
ent
Moving forward: Utility criteria + feasibility
Criterion Definition
1. Action-worthy We know (or strongly believe) that they contribute to results
2. Actionable They are amenable to government action and project interventions
3. Behavioral Focus on function, not form
4. Replicable Generated transparently and can be reproduced by others
…and 2 “feasibility” criteria.
Criterion Definition
1. Country coverage
Available for a minimum of 20 countries
2. Time series Collected repeatedly
Selected Examples
1. “Does the executive’s budget or any supporting budget documentation present expenditures for the budget year that are classified by administrative units?” (Open Budget Survey)
2. Does a legislative committee hold public hearings on the individual budgets of central government administrative units in which testimony from the public is heard? (Open Budget Survey)
3. Tax payers access to information on tax liabilities and administrative procedures (PEFA)
10
De jure vs De facto
An illustration: Public Accountability Mechanisms
11
Public Accountability Mechanisms (PAM)de jure indicators
Financial disclosure (interests, assets,
income)217 indicators
2008, 2012
Freedom of information
37 indicators2010
Conflict of interest restrictions
124 indicators2011
Immunity protections56 indicators
2012
12
Outputs of de jure data13
Library of laws Analytical publications on the design of
public accountability mechanisms Data, including qualitative and quantitative
datasets, country profiles, and descriptive statistics
Country reports on enabling governance environment
….But how do we know what happens in practice?
Freedom of information systems in practice:
public sector management functions
1.1 Facilities 1.2 Data/Records 1.3 Human
resources 1.4 Financial 1.5 Policy
2.1 Demystification*
2.2 Responsiveness
2.3 Appeals 2.4 Proactive
Disclosure
14
1.0 Administrative Functions
2.0 Disclosure Functions
* clarity about government processes, rules, and decisions
Freedom of information systems:
Immediate impacts/Intermediate outcomes
15
3.1 Public engagement: Extent to which the public understands, believes in, and engages with the freedom of information process.
3.2 Government commitment: Extent to which the government supports the freedom of information regime, including efforts to establish participatory decision-making
3.3 Administrative culture: Extent to which the bureaucratic culture has shifted from principle of secrecy to openness.
3.4 Operational efficiency: Extent of improvement in operations and decision-making within the organization.
How do you measure impacts in a governance “ecosystem”
16
Participation of civil society organizations and citizens Accountability and enforcement by parliament, judiciary,
police, supreme audit institution and the ombudsman Political will that enables meaningful change, including
protection from retribution, lack of clientilism, and an environment that maintains appropriate incentives for civil servants
Foundational support mechanisms found in public sector systems, such as freedom of information frameworks, conflict of interest rules, financial disclosure practices, and human resources and policy management.
Methodologies matter17
Surveys: validation issues, response problems Expert assessment: time-consuming, with
degrees of subjectivity Household surveys: costly, time-consuming,
logistical difficulties Economic/social indicators: collected by
whom? National statistics agencies, multi-laterals, NGOs, donors….
Outcome mapping/process tracing: lengthy, involved, requires local capacity
So where do we start?18
Methodology
IndicatorsConcepts
“Big ideas” for today19
Actionability and action-worthiness Change in behavior and not in function Sustainable measurement practices Fuzzy concepts and methodologies Practical applications of data Challenges in de facto measurement
Some emerging trade-offs (or complementarities?)
Global focus versus sub-national focus? Existing data or new data? Focused on public sector or broad
governance issues? Perception-based versus objective data?
Or both? Using a single tool or multiple tools? Detailed focus on a specific case/country
20
Some emerging issues
Areas of focus? Tax administration Human resources Rule of law
Looking forward Clarify the focus of the effort and
link to a clear research question Integrate the issue of sustainability Benchmarking Understanding strengths of
different types of respondents Clear definition of the context Dissemination and wide use of the
data (and need for a strategy) New set of tools for certain
phenomena (high level corruption)
21
PAM website: www.agidata.org/pamAGI data portal: www.agidata.orgMetrics & Accountability: http://go.worldbank.org/H1K725TJV0
Thank you!22
23
Inputs Outputs
Rules (Institutiona
l Arrangeme
nts)
Organizational
Capacities
Governance System
Performance
Outcomes
The Missing Middle
Resources invested in projects to deliver its outputs. Examples: Funding, contracts, materials
Goods and services produced by the project.Examples: Surveys and Trainings conducted, Laws revised, Agencies established
Desired state of well-being -- a set of conditions, experiences or behaviors – that is the goal for change or improvement.Examples: Maternal/infant mortality rates, Standardized test scores of K-8 students
Actionable Governance Indicators (AGIs) focus
on specific and narrowly-defined
aspects of governance.
24