11
Actionable Governance Indicators: Actionable Governance Indicators: Progress and Prospects Progress and Prospects
DEC Seminar on DEC Seminar on
Measuring Governance Measuring Governance
May 1-2, 2008May 1-2, 2008
22
Purposes of broadly-defined indicatorsPurposes of broadly-defined indicators
Research on development impact of civil Research on development impact of civil liberties, corruption, rule of law, etc.liberties, corruption, rule of law, etc.
Research on determinants/correlates of Research on determinants/correlates of “good governance” “good governance”
Tool for reformers in governmentTool for reformers in government Country-level diagnosis: more sensitive to Country-level diagnosis: more sensitive to
random measurement error random measurement error
33
Purposes of narrowly-defined indicatorsPurposes of narrowly-defined indicators
Monitoring progress of public sector reform Monitoring progress of public sector reform programs: are reforms being implemented? (e.g. programs: are reforms being implemented? (e.g. pay de-compression)pay de-compression)
Measuring results: are intermediate-level Measuring results: are intermediate-level outcomes improving? (e.g. number of qualified outcomes improving? (e.g. number of qualified applicants for high-skill positions increasing)applicants for high-skill positions increasing)
Research on impacts of specific reforms on Research on impacts of specific reforms on performance of governance sub-systems (budget performance of governance sub-systems (budget management, civil service, tax administration)management, civil service, tax administration)
Input into “actionable” parts of CPIA criteria Input into “actionable” parts of CPIA criteria
44
Actionable Governance Indicators Actionable Governance Indicators are the are the “Missing Middle”“Missing Middle”
Inputs and outputsInputs and outputs Bank operations consistently trackBank operations consistently track
project inputsproject inputs and and outputsoutputs A growing number of Bank operations A growing number of Bank operations
and client countries have begun to and client countries have begun to tracktrack program and policy outputs program and policy outputs
OutcomesOutcomes Broad governance Broad governance outcomes outcomes are are
measured bymeasured by Freedom House, Freedom House, ICRG, etc. and by TI, WGI ICRG, etc. and by TI, WGI aggregate indexesaggregate indexes
The Missing Middle:The Missing Middle: Direct impacts of institutional Direct impacts of institutional
reform efforts on how reform efforts on how particular governance sub-particular governance sub-sub-systems function: sub-systems function: institutional arrangements, institutional arrangements, procedures, and practicesprocedures, and practices
Examples include PEFA, Examples include PEFA, Doing Business, Global Doing Business, Global Integrity, Open Budget Index, Integrity, Open Budget Index, Statistical Capacity IndexStatistical Capacity Index,, OECD procurement indexOECD procurement index
55
"Effective implementation, including further development of actionable and disaggregated indicators, will now be critical to achieving the GAC strategy's desired results."
Development Committee Communiqué, Washington DC, 15 April, 2007
66
ActionabilityActionability
implies greater clarity regarding the steps implies greater clarity regarding the steps governments can take to improve their score on governments can take to improve their score on an indicatoran indicator
is reduced to the degree extraneous factors can is reduced to the degree extraneous factors can affect indicator scores affect indicator scores
in general requires that indicators have in general requires that indicators have narrownarrow and and explicitexplicit definitions definitions
does not preclude subjective or perceptions-based does not preclude subjective or perceptions-based indicators (if they have explicit and narrow indicators (if they have explicit and narrow definitions) definitions)
77
Specific indicators enhance: Specific indicators enhance: • Actionability: clear link between actions Actionability: clear link between actions
and indicator values and indicator values • Replicability: independent informed Replicability: independent informed
observers should arrive at same scoreobservers should arrive at same score• Contestability: debates over narrower Contestability: debates over narrower
concepts are more tractable and concepts are more tractable and meaningful (DB)meaningful (DB)
88
Some specific indicators may also be Some specific indicators may also be • Manipulable (design reforms narrowly Manipulable (design reforms narrowly
for sole purpose of improving score) for sole purpose of improving score) • Irrelevant (rule of law matters for Irrelevant (rule of law matters for
development, but some “best development, but some “best practice” advice on how to improve practice” advice on how to improve rule of law may be wrong)rule of law may be wrong)
99
examples of narrow indicatorsexamples of narrow indicators
PEFAPEFA Open Budget Index Open Budget Index OECD Procurement AssessmentOECD Procurement Assessment Doing BusinessDoing Business Global Integrity Global Integrity Statistical Capacity Index Statistical Capacity Index Human Resource Management Human Resource Management Enterprise surveysEnterprise surveys Household surveys Household surveys
1010
PEFAPEFA
• 28 high-level indicators measuring performance 28 high-level indicators measuring performance of PFM systems (3 more on donor practices)of PFM systems (3 more on donor practices)
• a performance report based on the indicatorsa performance report based on the indicators
• applied in approximately 80 countries and by 19 applied in approximately 80 countries and by 19 donors (mostly WB and EC)donors (mostly WB and EC)
• common pool of information to promote country common pool of information to promote country leadership, coordinated and aligned donor leadership, coordinated and aligned donor support (“the strengthened approach”)support (“the strengthened approach”)
• about 30 assessments available at pefa.orgabout 30 assessments available at pefa.org
1111
Open Budget IndexOpen Budget Index
Using questionnaire with 122 items, Using questionnaire with 122 items, the OBI assesses: the OBI assesses:
public availability of key budget public availability of key budget documents documents
quality of information they providequality of information they provide timeliness of their dissemination timeliness of their dissemination
59 countries assessed in 2006 study; 59 countries assessed in 2006 study; goal of 80 or more in 2008goal of 80 or more in 2008
1212
OECD Assessment Methodology OECD Assessment Methodology for Public Procurement Systems for Public Procurement Systems
22 pilot countries22 pilot countriesBaseline indicatorsBaseline indicators 12 indicators12 indicators 54 sub-indicators54 sub-indicators Based on review of regulatory framework Based on review of regulatory framework
and practicesand practicesCompliance and performance indicatorsCompliance and performance indicators• Based on sample of procurement Based on sample of procurement
transactions and other performance transactions and other performance informationinformation
Assessment reportAssessment report
1313
Doing BusinessDoing Business
Contract enforceabilityContract enforceability Registering propertyRegistering property Paying taxesPaying taxes
Number of proceduresNumber of procedures TimeTime CostCost
1414
statistical capacity indicator (DECDG)statistical capacity indicator (DECDG)
Countries scored against specific criteria, Countries scored against specific criteria, using information available from the World using information available from the World Bank, IMF, UN, UNESCO, and WHO. Bank, IMF, UN, UNESCO, and WHO.
Overall score based on 3 dimensions:Overall score based on 3 dimensions: statistical practicestatistical practice: ability to adhere to : ability to adhere to
internationally recommended standards and internationally recommended standards and methodsmethods
data collection:data collection: frequency of frequency of censuses/surveys and completeness of vital censuses/surveys and completeness of vital registrationregistration
indicator availability: indicator availability: availability and availability and frequency of key socioeconomic indicators frequency of key socioeconomic indicators
1515
Enterprise Surveys Enterprise Surveys
107 countries, 75000+ firms107 countries, 75000+ firms ““unofficial payments” as share of firm unofficial payments” as share of firm
salessales time required to resolve dispute in courtstime required to resolve dispute in courts costs of providing security against crimecosts of providing security against crime time spent meeting with tax officialstime spent meeting with tax officials percentage of firm sales reported for tax percentage of firm sales reported for tax
purposespurposes major constraints to doing business major constraints to doing business
1616
Household surveysHousehold surveys
Gallup World Poll Gallup World Poll TI/Gallup International “Global TI/Gallup International “Global
Corruption Barometer”Corruption Barometer” Regional “barometer” surveys Regional “barometer” surveys World Values SurveyWorld Values Survey ““is corruption widespread” is corruption widespread” paid bribes to police, tax officials, paid bribes to police, tax officials,
etc. etc.
1717
in-country diagnostic toolsin-country diagnostic tools PER, CFAA, CPAR, fiscal ROSC PER, CFAA, CPAR, fiscal ROSC WBI GAC diagnostic surveyWBI GAC diagnostic survey PETS (Uganda & others) PETS (Uganda & others) Citizen scorecards (Bangalore)Citizen scorecards (Bangalore) Road composition (Indonesia)Road composition (Indonesia) Municipal audits (Brazil)Municipal audits (Brazil) Pharmaceuticals (Argentina)Pharmaceuticals (Argentina) HRM (Albania, Macedonia, Romania)HRM (Albania, Macedonia, Romania)
1818
Global IntegrityGlobal Integrity Assesses existence of laws and institutions Assesses existence of laws and institutions
designed to curb corruption, and their designed to curb corruption, and their implementationimplementation
Covers most UNCaC provisionsCovers most UNCaC provisions Assessment by lead researcher in country; Assessment by lead researcher in country;
two peer reviewers per countrytwo peer reviewers per country Six broad categories, 23 sub-categories, Six broad categories, 23 sub-categories,
81 questions and numerous sub-questions81 questions and numerous sub-questions Assessments in 2007 for 55 countries; 43 Assessments in 2007 for 55 countries; 43
countries in 2006countries in 2006
1919
Egypt 2007Egypt 2007
strengthsstrengths•procurementprocurement•political financingpolitical financing
weaknessesweaknesses•public access to public access to informationinformation•election integrityelection integrity•accountability of accountability of public officials public officials •external auditexternal audit
2020
Hierarchy of Global Integrity IndexHierarchy of Global Integrity Index
Overall indexOverall index
Government Accountability (1 of 6 major categories)Government Accountability (1 of 6 major categories)
Executive Accountability (1 of 23 total sub-categories)Executive Accountability (1 of 23 total sub-categories)
Regulations governing conflict of interest by executive branch Regulations governing conflict of interest by executive branch (1 of 81 questions)(1 of 81 questions)
Are heads of state/government required Are heads of state/government required to file regular asset disclosure forms? Can public access them?to file regular asset disclosure forms? Can public access them?
2121
not all categories are highly inter-correlatednot all categories are highly inter-correlated
Procurement rules
100806040200
Pu
blic
acc
ess
to
info
rma
tion
100
80
60
40
20
0 Rsq = 0.0806
2222
implementation not as easy as passing lawsimplementation not as easy as passing laws
"in law" sub-index
1009080706050403020100
"in
pra
ctic
e"
ind
ex
100
90
80
70
60
50
40
30
20
10
0
2323
enforcement: a function of income per capitaenforcement: a function of income per capita
GDP per capita
45000
40000
35000
30000
25000
20000
15000
10000
5000
0
law
s-p
ract
ice
s g
ap
60
50
40
30
20
10
0 Rsq = 0.1783
2424
How can ratings on broad How can ratings on broad indicators be improved? indicators be improved?
little understanding of what actions most little understanding of what actions most effective in reduce incidence of corruptioneffective in reduce incidence of corruption
actionable indicators may improve this actionable indicators may improve this understandingunderstanding
criteria in broad expert ratings often not criteria in broad expert ratings often not detailed or even availabledetailed or even available
unclear how different aspects (e.g. grand unclear how different aspects (e.g. grand v. petty corruption) are weightedv. petty corruption) are weighted
unclear what information sources are used unclear what information sources are used by expertsby experts
ratings may lag actions or events ratings may lag actions or events
2525
ICRG corruption ratings for IrelandICRG corruption ratings for Ireland
date
OCT 2007
JUL 2006
APR 2005
JAN 2004
OCT 2002
JUL 2001
APR 2000
JAN 1999
OCT 1997
JUL 1996
APR 1995
JAN 1994
OCT 1992
JUL 1991
APR 1990
JAN 1989
OCT 1987
JUL 1986
APR 1985
JAN 1984
corr
up
tion
ra
ting
5.5
5.0
4.5
4.0
3.5
3.0
2.5
2.0
1.5
2626
timelinetimeline
Dec. 1996: story breaks Dec. 1996: story breaks 5 to 4 5 to 4
Oct. 1997: tribunal convenes 4 to 3Oct. 1997: tribunal convenes 4 to 3
Jan. 1999: hears 1Jan. 1999: hears 1stst witnesses 3 to 2 witnesses 3 to 2
Feb. 1992: Haughey resigns as PMFeb. 1992: Haughey resigns as PMNov. 1992: retires from politicsNov. 1992: retires from politics
2727
some hints from the datasome hints from the data
Examine correlations of 23 anti-corruption Examine correlations of 23 anti-corruption policy areas from Global Integrity with policy areas from Global Integrity with outcomes (TI, WEF) for up to 66 countriesoutcomes (TI, WEF) for up to 66 countries
High: conflict of interest & asset disclosure High: conflict of interest & asset disclosure rules for executive branch, public access rules for executive branch, public access to informationto information
Low: conflict of interest & asset disclosure Low: conflict of interest & asset disclosure rules for judiciary, A-C laws & agencies, rules for judiciary, A-C laws & agencies, ombudsmanombudsman
2828
TI weakly related to GI “in law”TI weakly related to GI “in law”
LAWS
1009080706050
CP
I9
8
7
6
5
4
3
2
1 Rsq = 0.0749
ZWEZAR
ZAF
YEM VUTVNM
USA
UKR UGATZA
TMPTJK
THA
SRB
SLE
SEN
SDN
RUS
ROM
PHL
PER
PAKNPLNIC
NGA
MWIMOZ
MEXMDA
LVA
LKA
LBR
LBN
KGZKENKAZ
JPN
JORITA
ISR
IND
IDNGTM
GHA
GEO
FRA
ETH
ESP
EGY
ECU
DZA
CRICOL
CMR
CHN
CAN
BRABIH
BGR
BGD
BENBDI AZE
ARM ARG
2929
TI strongly related to GI “in practice”TI strongly related to GI “in practice”
IMPLEM
9080706050403020
CP
I9
8
7
6
5
4
3
2
1 Rsq = 0.4167
ZWEZAR
ZAF
YEM VUTVNM
USA
UKRUGATZA
TMPTJK
THA
SRB
SLE
SEN
SDN
RUS
ROM
PHL
PER
PAKNPLNIC
NGA
MWIMOZ
MEXMDA
LVA
LKA
LBR
LBN
KGZ KENKAZ
JPN
JORITA
ISR
IND
IDNGTM
GHA
GEO
FRA
ETH
ESP
EGY
ECU
DZA
CRICOL
CMR
CHN
CAN
BRABIH
BGR
BGD
BENBDI AZE
ARM ARG
3030
Global Integrity: Global Integrity: not all categories are highly inter-correlatednot all categories are highly inter-correlated
Procurement rules
100806040200
Pu
blic
acc
ess
to
info
rma
tion
100
80
60
40
20
0 Rsq = 0.0806
Top Related