t? n e m Agence Française de Développement 05 Evaluation ...

Evalua

tion

and

its Disco

nten

ts: D

o We Learn from

Exp

erience in Develop

men

t?

François BOURGUIGNONMichael A. CLEMENSJames MIRRLEESJean-David NAUDETLeonce NDIKUMANAJodi NELSONCatherine PARADEISERuerd RUBENMiguel SZÉKELY

Evaluation and its Discontents: Do We Learnfrom Experience in Development?The Agence Française de Développement and EUDN (European Development ResearchNetwork) have been co-organising an annual conference on development since 2003. Over theyears, this conference has become a landmark event in Europe for the development community.The March 26 2012 conference gathered over 1,000 participants from more than thirty countries.It challenged a central issue: can we learn from experience in the field of development? If so, howcan evaluation contribute and how is it that we seem unable to translate these experiencesinto practice?

Sir James A. Mirrlees (University of Cambridge, Chinese University of Hong Kong), Paul Gertler(University of California, Berkeley), Jean David Naudet (Agence Française de Développement),Jodi Nelson (Bill & Melinda Gates Foundation), Catherine Paradeise (University of Marne-la-Vallée),Ruerd Ruben (Ministry of Foreign Affairs, the Netherlands), Leonce Ndikumana (University ofMassachusetts Amherst), Miguel Szekely (former Under Secretary for Planning and Evaluationat the Ministry of Social Development, Mexico) and Michael Clemens (Center for GlobalDevelopment, Washington) commented the extent to which evaluation methods and approachescan usefully support the learning process in development. They presented facts, questions andreflections to identify how evaluation could, in the long run, contribute to experience-based andbetter development strategies. Francois Bourguignon (Paris School of Economics), Pierre Jacquet(Agence Française de Développement), Mamadou Diouf (Columbia University) and Jean PhilippePlatteau (University of Namur) also participated actively in the debates.

Con

férences &

Sém

inaires

/ Decem

ber 20

12 /

Evaluation and itsDiscontents: Do We Learn fromExperience inDevelopment?

05

Proceedings of the 9th AFD-EUDNConference, 2012

Table of Contents

Acknowledgements 5

Introduction 7

Part 1: Evaluating Development Policies 9

•Chapter 1 : Evaluating Development Policies, by James Mirrlees 11

Part 2: Impact Evaluations: a Tool for Accountability? 23

•Chapter 2 : Impact Evaluations: a Tool for Accountability? Lessons from Experienceat AFD, by Jean-David Naudet, Jocelyne Delarue and Tanguy Bernard 25

Part 3: Is Indicator-Based Management a Guarantee ofEfficiency? 45

•Chapter 3 : History Revisited : Measurement for Management in Development, by Jodi Nelson 47

•Chapter 4 : How much is Enough? Does Indicator-Based Management GuaranteeEffectiveness? by Catherine Paradeise 63

Part 4: Applying Evaluation to Development and Development Aid 93

•Chapter 5 : Dimensioning Development Aid: some Lessons from Evaluation, by Ruerd Ruben 95

•Chapter 6 : Applying Evaluation to Development and Aid: Can Evaluation Bridge theMicro-macro Gaps in Aid Effectiveness? by Leonce Ndikumana 123

•Chapter 7 : Applying Evaluation to Development Policy, by Miguel Székely 151

•Chapter 8 : The Collision of Development Goals and Impact Evaluation, by Michael A. Clemens 169

Authors’ Biographies 198

3December 2012 / Evaluation and its Discontents: Do We Learn from Experience in Development? /©AFD [ ]

December 2012 / Evaluation and its Discontents: Do We Learn from Experience in Development? /©AFD [ ]123

6. Applying Evaluation toDevelopment and Aid:

Can Evaluation Bridge the Micro-macro Gaps in Aid Effectiveness?

Leonce Ndikumana, Department of Economics and Political EconomyResearch Institute (PERI), University of Massachusetts[ 45]

Abstract

Donors and governments in aid recipient countries are under pressure to demonstrate theeffectiveness of aid, especially given the growing stress on fiscal balances in the context ofthe global financial and economic crisis. The evidence on aid effectiveness remains mixed atbest: while individual targeted aid interventions appear to produce positive results, theimpact of aid at the macroeconomic level remains limited. Furthermore, the reporting onconcrete outcomes of aid interventions remains inadequate, thus perpetuating doubts as toaid effectiveness. This paper discusses these micro-macro gaps in aid effectiveness and thereporting problem. It proposes some ways in which well-designed and carefully implementedevaluations can help bridge these gaps, and how better reporting and transparency on aidresults can advance the aid effectiveness agenda.

[45] The author is grateful for excellent assistance from Theresa Owusu-Danso, doctorate candidate at the University ofMassachusetts

There is growing pressure on donors andrecipient governments to demonstrate theeffectiveness of aid. In donor countries,taxpayers demand tangible proof of the useof the tax money channelled throughnational aid agencies and multi lateralinstitutions. This pressure has beenexacerbated by the adverse impact of theglobal financial and economic crisis ondonors’ fiscal balances. At the same time,populations in recipient countries are moreand more openly demanding tangibledevelopment outcomes, more transparencyin the management of aid and better accessto reports containing systematic andobjective assessments of aid effectiveness. Indeveloping countries, along with increasingdemocratisation and a free press,governments are facing a growing pool ofeducated but disfranchised youthdemanding genuine improvements in livingstandards.

These growing pressures and demands fortransparency and aid effectiveness arefurther fuelled by criticisms ranging fromanalysts arguing that aid has no robustimpact on development, to activists who areopenly opposed to aid on various grounds.Furthermore, the complexity of thedevelopment process makes it difficult totrack the impact of aid, which is influencedby many factors emanating from the donorside, the recipient’s context and exogenousfactors. In addition, the aid industry is acongested market where multiple actorspursue similar goals on the same terrain,

making it even more difficult to sort out theincremental impact of aid interventions.

Nonetheless, evidence shows thatdevelopment aid has produced substantialpositive results at the micro level, whether atproject or programme level. Well managedprogrammes have yielded improvements inschool enrolment, access to healthcare,reforms of tax systems and other valuableoutcomes. However, at the aggregate levelthe record remains very mixed, fuelling thedebate about overall weak aid effectiveness.Bridging the micro-macro gap remains acritical challenge for the development aidcommunity and national policy makers.

These practical challenges, criticisms, gapsbetween micro and macro outcomes anddomestic political pressures on donors andrecipient governments for moretransparency on aid call for more effectivemechanisms to analyse, monitor, evaluateand disseminate the concrete impacts of aidon development; in other words, there is acall for better aid effectiveness evaluation.While there has been substantial progress inevaluation methods and practice, importantgaps remain and there is room forimprovement. Moreover, the disseminationof impact evaluation results remainsinadequate, which further perpetuatesdoubts about aid effectiveness. This paperargues that well designed and implementedevaluations, together with betterdissemination of evaluation results, can helpshed light on these gaps in aid effectiveness.The paper thus emphasises two problems:

[ ] ©AFD / Evaluation and its Discontents: Do We Learn from Experience in Development? / December 2012124

6.1. Introduction

the dichotomy between micro-level andmacro-level aid effectiveness; and the lack oftransparency and inadequate reporting onthe concrete impacts of aid. These problemsare at the heart of the concerns regarding aideffectiveness in the aid community as well asin recipient countries.

Following this introduction, the paperprovides a brief review of the mixed recordof aid effectiveness in Section 2. Section 3highlights the problems at the origin of themicro-macro dichotomy in aid effectiveness.Section 4 discusses the role that evaluationcan play in bridging these gaps, and Section 5concludes.


6.2.1. A backdrop of rising aid volumes

The debate on aid effectiveness andevaluation is taking place in the context of anupswing of aid flows to developing countries.Following a steady decline in the 1990s, totalaid by the Development AssistanceCommunity (DAC) member countries has

increased substantially since the turn of thecurrent century. Between 1990 and 2009,total aid to all developing countries by DACdonors rose from USD119.9 billion to 165.3billion, a 37.8% increase. Based on the troughlevels of 2000 (USD84.7 bil l ion), thisrepresents a doubling of the volumes of aidover a decade (Table 4).


6.2. Aid effectiveness: a less than stellar record

Year Total AfricaLatin

AmericaAsia Europe Oceania Unspecified

1960 30.8 9.5 1.6 15.7 2.8 0.2 1.0

1970 44.9 9.3 5.7 18.5 1.0 1.5 1.9

1980 111.4 27.1 5.8 35.3 3.1 2.7 12.8

1990 119.9 41.2 8.6 29.5 2.3 2.3 9.6

2000 84.8 19.2 6.0 19.7 4.6 1.0 11.1

2009 165.3 47.7 9.1 38.6 5.8 1.6 24.9

Change 1990-2009 (%)

37.8 15.8 5.9 30.7 147.6 -26.9 159.1

Change 2000-2009 (%)

95.0 148.7 50.8 95.7 25.0 62.0 122.9

Real annual flows of official aid (constant 2009 US dollars, billions)4Table

Source: DAC database (online). Nominal values are deflated into real values using the US CPI index.

In per capita terms, the upward trend is mostnotable in sub-Saharan Africa, Asia and LatinAmerica (Figure 8). The substantial increasein aid to these regions since 2000 has beencredited for contributing to high growth inthe pre-crisis period.[ 46] Aid per capita in sub-

Saharan Africa more than doubled between2000 and 2009, rising from USD24 toUSD54. However, the 2009 levels are sti l lbelow the peak of USD57 per capita reachedin 1990.


[46] Various reports by the multilateral development institutions have listed increasing volumes of aid as one of the keydrivers of the high growth in Africa during the years leading to 2008–09. These include the African EconomicOutlook (by the African Development Bank, OECD, UNECA and UNDP), UNECA’s Economic Report on Africa ,UNDESA’s World Economic and Social Prospects , and the IMF’s World Economic Outlook.

Real aid per capita by region, 1960-2009 (constant 2009 US dollars)

8Figure

0

20

40

60

80

100

120

140

2008200420001996199219881984198019761972196819641960

EAP ECA LAC MENA SSA SA

Source: DAC database (online). Nominal values are deflated into real values using the US CPI index. SSA = sub-Saharan Africa; MENA =Middle-East and North Africa; SA = South Asia; LAC = Latin America and the Caribbean; EAP = East Asia and Pacific; ECA = Europe andCentral Asia.

Despite the considerable increase in thevolumes of aid over the recent years, thequantity of aid remains inadequate relative tothe financing needs of developing countriesas well as relative to donors’ targets. TheHigh-Level Plenary Meeting on the

Millennium Development Goals (MDGs)held in New York in 2010 under the theme of“keeping the promise” soberly lamented thefact that donors had not kept their promiseof increasing aid delivery (UN MDG TaskForce, 2011). The report of the MDG Task

Force found that, while official aid hadreached a record high of USD129 billion in2010, this represented only 0.32% of thegross national income (GNI) of DACmembers. Only five countries have met theUN target of 0.7% of GNI in official aid.[ 47]

The report noted a large gap of USD153bil l ion in 2010 in actual aid delivery. Aiddelivery to Africa in 2010 was USD15 billion(in 2004 dollars) below the pledges made in2005 at Gleneagles (UN MDG Task Force,2011: 15).

At the same time, developing countries arefacing large financing gaps in economic

infrastructure and social sectors. It isestimated that Africa faces an annual gap ofUSD48 bil l ion in infrastructure financingalone. In 2008, the MDG Africa SteeringGroup Report (2008) concluded that forAfrican countries to reach the MDGs, publicexternal financing would need to be scaledup by about USD72 bil l ion per year until2010. Actual disbursements fall far belowthese targets.

But most importantly, despite the fact thatthe volumes of aid to developing countries ingeneral have increased over the past years,the record of the impact of aid on


Growth gains from aid in SSA: GDP growth/aid growth

9Figure

0

10

20

30

40

50

60

2008200420001996199219881984198019761972196819641960

-8

-6

-4

-2

0

2

4

6

GDP growth per inhabitant (% per year) Aid "dividend"Volume of aid per inhabitant (USD)

Gro

wth

rat

e: G

DP

per

inha

bita

nt a

nd a

idV

olu

me of aid

per inhabitant

Source: DAC database (online). Nominal values are deflated into real values using the US CPI index. Aid dividend is proxied by the ratio ofreal GDP growth to the growth rate of real aid per capita.

[47] The five countries are: Norway (1.10% of aid/GNI), Luxemburg (1.05%), Sweden (0.97%), Denmark (0.91%), andNetherlands (0.81%). (Source: OECD-DAC online database).

development remains rather wanting.Growth in sub-Saharan Africa remains belowthe levels needed to reach nationaldevelopment targets; growth is volatile andhas generated inadequate job creation. Thecredit given to aid for stimulating the recentresurgence of growth in Africa is oftenexaggerated. Over the long run, the gainsfrom growth are limited. As can be seen onFigure 9, growth elasticity of aid has been lowand flat. The recent upswing in aid has onlyyielded a short-lived spike in “aid dividend”,reverting to a stagnant mean.

The gains from aid in terms of socialdevelopment are also less than satisfactory.While aid has supported important projectsin education and health, the overall impact

remains inadequate. The deficiencies aremost notable in the case of Africa, the regionpointed out as having received relativelyhigher volumes of aid. Infant mortality hasdeclined much more slowly in Africa than inthe other developing regions (Figure 10) andmany African countries are not likely to reachthe MDG target for this developmentobjective.

Although the debate on aid effectiveness hasheated up in recent years, efforts to assessthe effectiveness of aid date from as far backas the 1960s (Roodman, 2007a) .Doucouliagos and Paldam (2005) provide acomprehensive l iterature review, pointingout the “sad results” of four decades ofresearch on the theme. Over the years, the


Infant mortality by region (per 1,000 live births)

10Figure

0

20

40

60

80

100

120

140

160

180

20082005200219991996199319901987198419811978197519721969196619631960

Afrique subsaharienne Asie du Sud MENA Amérique latine Asie de l'Est et Pacifique

Note: MENA = Middle-East and North Africa.6.2.2. Disputed evidence at the macro level but more encouraging results at the micro level

work on aid effectiveness has been reviewedseveral times, with conclusions ranging fromextreme optimism to quasi-mil itantpessimism.[ 48] But the literature shows a cleardemarcation between the findings at themacro level, where the results are mixed, andresults at the micro level, where the evidenceis much more encouraging regarding thegains from aid. The sections below discussthe evidence at the two levels in turn.

Mixed evidence at the macro level

The ultimate objective of aid is to contributeto improved economic performance and thewellbeing of the population through theprovision of financial resources and technicalassistance. Naturally, governments and thepublic from donor and recipient countriesexpect to see improvements in indicators ofeconomic performance and wellbeing asreturns to aid. This explains why theattention in the aid effectiveness analysis hasbeen focused on national level indicators,mainly economic growth, health outcomesand human capital development. Implicitly,the analyst assumes, or rather hopes, thatthe impacts of aid at the micro level, wherethe action takes place, somehow translateinto macro level impacts at the national level.But the process of aggregation of micro leveloutcomes into the macro level impactsremains a black box.

The literature on the macro level impact ofaid fal ls into three camps, with a l imitednumber of agnostics along the spectrum ofaid effectiveness beliefs: aid works; aid does

not work; aid works under certain conditions(but it works after all). The first camp claimsthat aid works, and the only concern is thatthere is not enough aid and that it may notreach the intended recipients. The mostvocal advocates in this camp include JefferySachs, who has argued forcefully for a “big-push”-led growth financed by scaling up aidto developing countries (Sachs, 2005). [ 49]

Sachs and his colleagues argue that externaldevelopment assistance can help break thepoverty trap and that scaling up aid is theexit strategy from the poverty trap. Thefollowing excerpt says it all in reference toAfrica:

If Africa is caught below the thresholdlevel of infrastructure, and therefore isstuck in chronic low or negative growth,the main policy implication is to raisecapital above this threshold . . . Wepropose to increase the capital stock inone step, as it were, through a large, well-targeted infusion of foreign assistance. Inother words, we are arguing not forendless flows of increased aid, and notfor aid as simple charity, but rather forincreased aid as an exit strategy from thepoverty trap. For those who fear that aidincreases dependency, our response isthat aid that is ambitious enough wouldactually end Africa’s dependency.Moreover, we see no other l ikelysuccessful strategy for ending Africa’spoverty trap. (Sachs et al. 2004: 144)


[48] Key reviews of the literature on aid effectiveness include: Mosley (1980); Hansen and Tarp (2000); Clemens et al.(2004); McGillvray et al. (2005); Roodman (2007).

[49] See Easterly (2006a) for a critical review of Sach’s argument for a big-push approach to development assistance.

This camp includes analysts who argue thataid has been effective in stimulating growth(see Hansen and Tarp 2000, 2001). Besidesthe quantitative analysis supporting the aideffectiveness view, there is a large literaturefrom the activist world (NGOs and civi lsociety organisations) calling for a scaling-upof aid.

The second camp argues that aid works, butthat it works under certain specificcondition. One of the most widely citedstudies in this group is by Craig Burnside andDavid Dollar (2000), which claimed that aidworks but only in a good policy environment.This study generated considerable debateand controversies. Some analysts questionedthe robustness of the results and the meritof the methodology (see Roodman, 2007bfor a review). Further investigations refutedthe results as too fragile, not robust tosample selection and subject to particularspecification of the empirical model(Dalgaard and Hansen, 2001; Easterly et al.,2003). For instance, the interaction of aidand policy and the particular coding of thegood policy indicator are found to be the keydrivers of Burnside-Dollar results. Outside ofacademia, the concerns with the Burnsideand Dollar proposition were about its policyimplications. The proposition implied thataid should go to countries withdemonstrated evidence of good policy; thatis , aid should be conditioned to goodpolicies. This reopened a can of worms in thedebate on aid conditionality. Mostfundamentally, it meant that, given that low-income countries and especial ly thosecoming out of conflicts also have weak

institutions and policy frameworks, anapplication of the Burnside-Dollarproposition would leave these countries asaid orphans and trapped into a low-aid–poverty vicious circle.

A number of other analysts have supportedthe view that aid is effective under certainconditions. Noteworthy studies includeCollier and Dollar (2004), who argue that aideffectiveness requires good governmentinstitutions. Similarly, Svensson (1999) arguesthat aid is effective only in democracies.Collier and Dehn (2001) posit that aid can beeffective in countries experiencing shocks,but point out that aid effectiveness requiresgood policies. Patrick Guillaumont and hiscolleagues argue that aid helps absorbeconomic and natural shocks, and stronglyadvocate for allocating official developmentaid on the basis of economic vulnerability.[ 50]

Within this strand of conditional aideffectiveness l iterature, Dalgaard et al .(2004) controversial ly suggested that aidworks outside of the tropics but not withinthe tropics! The study is empirically fragile,the results being driven by a few countrieswith specific features, namely Botswana,Egypt, Jordan and Syria (Roodman, 2007b).This kind of conclusion feeds the usualdeterministic view of development thattends to attribute underdevelopment tofixed factors such as geography. But this viewis tenuous; it fails to explain, for example,why geography would prevent Burundi fromdeveloping while Switzerland developed,although both countries are landlocked andsmall.


[50] Guillaumont (2007, 2009, 2010); Guillaumont and Chauvret 2001; Guillaumont and Guillaumont-Jeanneney (2009);Guillaumont and Simonet (2011).

There is a smaller strand of the literature thatargues that aid simply does not work,conditionally or absolutely. Rajan andSubramanian (2005) challenge therobustness of the results in studies thatconclude that aid works even if it isconditional on good policies. They concludethat aid does not have any statistical lyconsistent effect of growth and that, even incases where it may exist, the effect is toosmall to be statistically observable.

William Easterly argues that aid works onlyif it is well targeted and al igned withindividual country’s cultural , social andeconomic conditions (Easterly, 2006b). He iscritical of large-scale or grand-scheme typesof aid interventions, or what he calls a“transformational” approach to aid. Theproblem is not the money; it is whether thefunds are used to meet the specific needs ofthe intended recipients. Easterly suggeststhat well-managed aid produces positiveresults at the micro level in areas such aseducation and health. He thus favours the“marginal” approach with small-scaletargeted interventions (Easterly, 2009).

Overall, the review of the literature suggeststhat the evidence on the macro-leveleffectiveness of aid remains mixed with noapparent movement towards any consensus.Now we turn to the micro level impact ofaid, where the results are much morepromising.

More encouraging evidence on aid effectiveness at the micro level

The key challenge to efforts to documentand quantify the effectiveness of aid at themacro level is that macroeconomicoutcomes are the result of a multiplicity of

factors, many of which are unrelated to aid,and some of which can affect theeffectiveness of aid either positively ornegatively. Economies are complex systemswhere virtually everything depends on andinfluences everything. Disentangling theimpact of a single factor such as aid onmacroeconomic outcomes l ike growth,human capital , health, etc. , is a dauntingexercise both conceptually and empirically.Moreover, project level interventions mayproduce macro level results, but these wouldbe observed only several years down theroad (Radelet and Banana, 2004).

Moreover, and most fundamentally, aid isonly an instrument used to achieve ultimatemacro level goals. For the instrument to havean impact on the ultimate goal, a long chainof causalities needs to hold systematically. Achain is as strong as its weakest link; if onenode in the chain of causalities fails, then thefinal result is compromised. For example, theultimate goal of aid interventions ineducation is to increase human capital, whichin turn would increase growth and generateimprovements in the overall wellbeing. Sodonors finance school construction with thehope that the recipient country wil l reapfuture benefits in terms of improved humancapital and higher growth. However inpractice, for the final result to materialise, notonly does aid need to be spent and useddiligently, but also agents’ behaviour needsto respond appropriately and significantlyalong the way. So, effectiveness of aidoperates at multiple levels and it is theaggregation of the intermediate levels ofeffectiveness that determines effectivenessat the macro level. Using the example of aidto education through construction ofschools, Roodman (2007a : 2) summarises


some of the questions that need to beaddressed, which points to many ways inwhich aid effectiveness may becompromised: “Was a school built? Didchildren come? Did they learn? When theygrew up, did they have fewer children oftheir own? Did they find more rewardingand productive work? Did economic outputgo up? Did poverty or inequality fall?”

For aid to education to have macro leveleffects, there are too many “ifs” that need tobe satisfied. If more schools are built, schoolattendance will increase, literacy will increase,households wil l make more efficientdecisions regarding matters relevant for theirwellbeing , workers (educated) will be moreproductive, more output and income will beproduced and the l iving standards wil lincrease. Trying to demonstrate empiricallyeach of these causal statements is amonumental task.

One possible solution to the challenge ofdemonstrating aid effectiveness is to be lessambitious in the quantitative assessment ofaid effectiveness and look not at the macrooutcomes but at the micro level outcomes;that is , look at narrower goals. Such an

investigation typically reveals what Clemenset al. (2004) refer to as a “micro-macroparadox”: despite the disappointing resultsat the macro level , there is evidence ofsuccessful targeted aid interventions at themicro level. At the sectoral level, aid has alsobeen found to be effective, especially in theareas of education and health. Michaelowaand Weber (2006) find that aid contributesto increasing primary school enrolment.Dreher et al. (2007) find similar results. In thearea of health, Mishra and Newhouse (2007)find that aid helps reduce infant mortality.

The reality, therefore, is that the aidlandscape includes a mixture of successesand failures. The problem is the technologyused to aggregate the impact of aid. This is atthe root of the fact that the successes havenot been able to outshine the fai lures toproduce robust overall positive impacts ofaid. There are many reasons for this. Keyamong these is that the aid enterprise hasmany structural deficiencies that undermineits effectiveness. These inefficiencies preventthe aggregation of positive micro level resultsinto visible positive outcomes at the macrolevel.


This section discusses succinctly the keystructural problems of aid effectiveness thatmay be at the origin of the micro-macroparadox. The focus is on problems that maybe addressed through effectiveconceptualisation and implementation ofevaluation.

6.3.1. A quantity and quality problem

To the extent that aid effectiveness meansdevelopment effectiveness, then both thequantity and the quality of aid matter.Regarding the quantity of aid, if aid is toproduce positive and visible results at theaggregate level , it must reach a minimumthreshold. It has been pointed out in severalstudies and reports that the current levels ofaid remain inadequate. They fall short of theinvestment gaps faced by developingcountries and are below the OECD targetsof 0.7% of donor countries’ gross nationalincome.

Various studies have documented large andeven growing financing gaps faced bydeveloping countries. In the case of Africa,for instance, it is estimated that to reach thehigh growth required to substantially reducepoverty, the continent would need to investabout USD93 bil l ion per annum ininfrastructure, including USD41 bil l ion forthe power sector (Africa InfrastructureCountry Diagnostics, 2009). Currently onlyUSD45 billion are covered, leaving a gap of

USD48 bil l ion, of which USD23 bil l ion isrequired for the energy sector alone.

For aid to generate meaningful impacts atthe macro level , the current levels wouldhave to be substantial ly increased in apredictable manner to fi l l the investmentfinancing gaps. Higher and more predictablefunding would help achieve higher and lessvolatile growth, and ultimately foster socialdevelopment. Improvement in aideffectiveness at the macro level is conditionalon increasing the volumes of aid delivery.

The quality of aid is also essential for aideffectiveness. When aid effectiveness isdefined in terms of developmenteffectiveness rather than financial soundnessor operational/process conformity, thequality of aid raises a number of issues. Thetwo most prominent issues are al locativeeffectiveness and predictabil ity. Thechallenge of al locative effectivenessemanates primarily from the fact that (1)resources are scarce and therefore donorshave to make difficult choices on where toinvest these resources; (2) there isinadequate information on effective returnsto investment in various activities; and (3)there is imperfect evidence on the keydrivers of growth at the country level. As aresult, aid effectiveness is constrained by thefact that some resources are al located tosectors with limited returns to investmentand with l ittle impact on growth anddevelopment.


6.3. Problems leading to themicro-macro paradox

Allocative efficiency problems areexacerbated by the lack of consistency in thedecision-making of donors. Over time,donors change their aid targets andpreferences, but it is not always clearwhether these shifts are inspired by carefulanalysis of the expected relative gains frominvestment in various sectors. So, forexample, donors have exhibited a strong biastoward social infrastructure and services,especially since 2000 (Figure 11). In contrast,aid to productive sectors declined since thebeginning of the 1990s. As donors focusedon poverty reduction as the ultimate goal ofaid, the attention shifted to activities and

sectors that were deemed closer to thisobjective, hence the emphasis on socialsectors. This shift, however, is problematic. Ithas been well documented that sustainedpoverty reduction requires higher andsustained growth and job creation, which inturn requires adequate investments inproductive sectors. Ironically, focussing onthe poor by increasing spending on socialservices has not contributed much toreducing poverty. It is by supporting wealthand job creation through strong , sustainedand broad-based growth that sustainedpoverty reduction can be achieved.


Sectoral allocation of ODA (total all donors, constant 2009 dollars, billions)

11Figure

0

5

10

15

20

25

20072004200119981995199219891986198319800

20

40

60

80

100

120

140

Infrastructu

re and so

cial services

Infrastructure and social services Multisectoral/cross-cutting Productive sectors Infrastructure and economic services

Source: DAC database (online). Nominal values are deflated into real values using the US CPI index (obtained from the IMF’s InternationalFinancial Statistics).

The problem of targeting of aid is amplifiedby “herd behaviour” among donors, and thetendency to “follow the winner” in a contextof high pressure to show results. Individualdonors seek to minimise risks by avoidinguntapped terrains and focussing on sectorsand activities that have gained consensusamong the donor community. Moreover,multilateral development institutions, whichare key players in the aid landscape, do nothave a genuine capacity to set their owntargets. They are all accountable to the samegovernments of member states. As a result,the preferences of dominant donorcountries permeate the strategic decisions ofmultilateral development institutions, so thatthe preferences of the latter mimic those ofthe former.

In addition, aid effectiveness is hampered bypoor cost-effectiveness, notably due to longand cumbersome aid delivery processes. Aidfor seeds and fertilisers is of little help whenit is delivered after the end of the plantingseason. The high cost of aid delivery is alsodue to ineffective donor coordination in alandscape marked by a proliferation ofdonors and projects. This increases theburden on recipient governments called toexecute, monitor, and evaluate multipleprojects and dialogue with multiple donors.It is not surprising , for example, that somerecipient governments occasionally call for amoratorium on donor missions duringcertain periods so they can get down torunning their business.[ 51]

6.3.2. Weak additionality of aid

The limited record of aid effectiveness atmacro level can also be attributed to weakadditionality of official development aid.One of the reasons for the weakadditionality of aid is that additionality is notintegrated in the planning of aidprogrammes. Additionality of aid can beevaluated at three levels: f inancialadditionality; positive spillover effects intothe local economy from aid-funded projectsand programmes; and technology andknowledge transfer. Financial additionality ofaid stems from the role that aid plays inattracting additional public and privateresources. From the public side, aid cancrowd in domestic public resources byincreasing the capacity to mobilise tax andnon-tax revenue. A donor intervention canalso crowd in external public resources byenticing other donors to co-fundprogrammes and projects.

From the private side, aid can play a catalyticrole in attracting private financiers or byfacilitating public-private partnership fundingarrangements. In practice, however, insteadof crowding in domestic public resources, aidoften tends to have a disincentive effect ontax mobilisation (Ostrom et al. 2001: Xviii).Because aid is fungible with othergovernment resources, especially in the caseof budget support, high volumes of aidalleviate pressure on the government tomobilise taxes that are politically undesirable.


[51] In Tanzania for example, the government declares a mission-free month during the budget preparation process.

The ability of aid to catalyse additional publicand private resources is also limited becausethis is not explicitly built into aidprogramming. When institutions make it anobjective for their funds to play a catalyticrole, aid indeed can crowd in substantialamounts of private finance.[ 52] Unfortunately,this practice is not part of the normalbusiness of public sector aid programmingamong bilateral donors or multi lateraldevelopment institutions.

Aid projects tend to also have suboptimalspillover effects in the local economy, whichlimits overall aid effectiveness. Aidprogrammes often remain virtual islands inthe economic landscape, thus minimisingtheir impact at the macro level.

In addition to financing , developmentassistance can also provide an avenue for thetransfer of technological know-how fromdonors to aid recipients. This in turn wouldeventually increase productivity in recipientcountries, leading to overall higher economicperformance. The record of aid effectivenessin this regard is weak. The gains throughtechnology transfer are particularly low inthe case of tied aid. Despite calls to moveaway from tied aid, it sti l l represents asubstantial fraction of total aid for manydonors, whether de jure or de facto . Thisfurther undermines aid effectiveness.

6.3.3. Failure to influence policy andinstitutions

There is broad consensus that institutionsand good policies are important ingredients

for sustained high long-run growth. Yet, theaid community has not made up its mindwhether aid should be used to induceimprovements in institutions and policies.For a long time, donors have had itbackwards: they have conditioned aid togood institutions and policies. Given that themajority of low-income countries have weakinstitutions and policies, they then end upreceiving less aid. As a result, poor countriesare trapped in a stable equilibrium of badinstitutions and low growth. Indeed, Birdsall(2007) argues that what is holding Africaneconomies in a low-growth high-povertytrap is an “institutional trap”. While growth isbelieved to be a function of institutions,donors have primarily focused on the directl ink between aid and growth, and less oninstitutions. One of the causes of l imitedperformance of aid in stimulating growth isthat little aid has been invested in institutionbuilding and that aid has not beensufficiently leveraged to improve theinstitutional framework in low-incomecountries.

There are reasons for this limited emphasison using aid to develop institutions. Some ofthese constraints are political , wherebydonors put their national strategic interestsahead of recipient countries’ economicdevelopment goals (Kil l ick, 1998; Kanbur,2000; Mold, 2009). Thus, bad governance inrecipient countries goes unchallenged, andeven worse is rewarded by additional aidinflows in the name of national strategicinterests. On the recipient side, there isresistance against interventions with an


[52] IFC’s evaluation reports show strong additionality of private sector lending , with the most common form of addi-tionality arising from funds mobilisation. See for example, Independent Evaluation Note No.1, 2008. “IFC’sExperience and Additionality in Middle Income Countries Results and Challenges”; also see IFC (2010) “IFC’sAdditionality: A Primer”, September 30, 2010.

institutional emphasis, especially inundemocratic regimes, on the pretence ofnational sovereignty. Moreover, there isl imited knowledge on how exactly toinfluence the development of good policiesand institutions. Donors know goodinstitutions when they see them, but theyknow less how to engineer them in aparticular country. Furthermore, institutionsdevelop very slowly and in a complex fashion.This is not particularly encouraging for typicalaid agencies that are bound by short-term“key performance indicators” tied to short-run results. The lack of patience thereforeexplains the inadequate invest ment ininstitution building and in developing thecapacity to implement good policies.

6.3.4. Poor alignment of incentivesand interests

Aid effectiveness is also compromised by thelack of convergence between the interests of

donors and those of recipients. Withindonor governments and developmentfinancing institutions (DFIs), there is oftenalso a lack of consistency between theincentives and interests of the institutionsand those of operations officers. Like anyinvestment venture, development aid carriesrisks. Yet, it is by taking risks that aid cangenerate the highest rewards in terms ofdevelopment outcomes. Thus, the donormust strike a balance between financial riskand development outcomes (Figure 12). Thetendency of programme and project officersis to err on the safe side, minimising financialrisks to demonstrate that money has beenused well, thus staying on good terms withinternal audit. For development financialinstitutions, this risk aversion is furthermotivated by the need to preserve thefinancial bottom line and good credit ratings.As i l lustrated in Figure 12, donors facetensions between maximising development


Development-risk trade-off

12Figure

Benevolentdevelopmentalist

Mercantile banker

Optimal path

Financial visibility

Dev

elop

men

t ou

tcom

es

outcomes, or behaving as “benevolentdevelopmentalists”, and maximising financialviability, or acting as “mercantilist bankers”.In such a context, risk aversion tends to keepaid below the optimal path with regard todevelopment outcomes.

Another way in which incentiveincompatibility undermines aid effectivenessis through the pressure to spend the aidbudget in the context of the “spend it or loseit” culture of aid budgetisation. This practiceinduces inefficient allocation of aid resourcesto activities with little development gains.This causes problems with aid absorption, aslarge volumes of aid are appropriated toprojects and programmes but remainunutilised for long periods. The pressure to“move the money” creates allocative as wellas management inefficiencies, especial lysince programme officers are evaluated noton the basis of effectiveness but on the basisof approvals. Ultimately, these micro levelinefficiencies contribute to the overall weakeffectiveness of aid at the macro level.

6.3.5. Lack of learning

The challenges described above have beenpointed out repeatedly for a long time; yetthey continue to permeate the developmentaid practice. A key reason for this is theimperfection of the learning process in thedevelopment aid industry. Evaluation is oftennot integrated into aid and developmentpolicy, and there are inadequate investmentsin developing evaluation mechanisms. Thisprevents the development of what Ostromet al. (2001) cal l the “error-correctioncapabilities” of systems and institutions thatprevent mistakes from generating self-perpetuating inefficiencies. Thus

improvements in aid effectiveness areconditional to the development of effectivelearning. The question this paper raises iswhether evaluation can help fill this learninggap and, as a result, contribute to bridgingthe gaps between macro level and microlevel aid effectiveness. This question is theobject of the next section.

6.3.6. Inadequate reporting and dissemination of the concreteimpacts of aid

Even in situations where aid has beeneffective and produced tangible results,especial ly at the micro and sectoral level ,often the results remain unknown to therecipients as well as to the public in thedonor countries. It is generally observed thatNon-Governmental Organisations do abetter job in publicising their interventionsand drumming up the results of theirprojects. This somehow explains the relativesuccess of actions by specialised agencies,and it is consistent with the evidence on aideffectiveness at the micro level discussedabove.

The lack of transparency and inadequatedissemination of the results of aid ariseprimarily from the tradition that aidmanagement is the government’s domain. Indeveloping countries where institutions ofpublic accountabil ity remain under -developed, government operations are notopen to the public, and participatorybudgeting is not part of the policy andpolitical culture. Thus, while the population isarguably the ultimate beneficiary of aidinterventions, it is not systematicallyinformed of the nature of interventions andtheir concrete results. It is expected thatincreased democratic consolidation and the


development of a free press will lead to morepressure on governments to open up the aidmanagement process, which wil l result inbetter access to information on aideffectiveness for the general public. This iskey to building political support fordevelopment aid.

Moreover, the technical process of reportingon aid effectiveness remains inadequate andit is not systematically integrated into the

programming and delivery of aid. Even indonor countries where institutions of publicaccountabil ity are developed, the generalpublic has inadequate access to reports onthe results of aid interventions. Reporting tothe general public is often defensive, reactingto criticisms from the media and theresearch community rather than being seenas an inherent obligation of government aidagencies. This tradition undermines theoverall aid effectiveness agenda.


6.4.1. Substantial progress has beenmade in evaluation practice butchallenges still remain

Evaluation is a key component of national aidpolicy, helping to set goals and accessperformance. For the USAID (2011: 1) ,“evaluation is the means through which itcan obtain systematic, meaningful feedbackabout the success and shortcomings of itsinterventions. Evaluation provides theinformation and analysis that preventsmistakes from being repeated, and thatincreases the chance that future investmentswil l yield even more benefits than pastinvestments.” Similarly, the JapaneseInternational Cooperation Agency (JICA)considers that “the primary objective ofevaluation is to improve the effectivenessand efficiency of projects by using evaluationresults for better planning andimplementation” (JICA, 2004).

There has been progress in evaluationmethods and practice, especially with theintroduction of experimental methods in thedesign and implementation of evaluation.The use of randomised experiments andrandomised control trials (RCTs) holdspromise; they are scientific, objective, andminimise sampling bias, thus enhancing thereliabil ity of evaluation results. Thesemethods also have the advantage of being

replicable in various settings (Duflo andBarnejee, 2009; Duflo and Kramer, 2005).The use of control groups enables theanalyst to get closer to establishing a causalrelationship between a particularintervention and the targeted outcomes.

But there still are many issues, even with theRCT methodology. In particular, thelimitations of RCTs stem from the fact thatthe method works well in situations wherean intervention is truly discrete andhomogeneous across space and time(Bamberger and White, 2007). This obviouslyhappens in scientific laboratories, but rarelyin real social settings. Moreover, RCTs do notcompletely overcome the perennialproblems of attribution of outcomes in acomplex system like an economy, wheremany factors are likely to influence directlyand indirectly a particular outcome (Vaessen,2010). Furthermore, randomisation may faceethical problems as the exercise involvesgroups that are not benefiting from anintervention that they would otherwise havewished to benefit from. In such a context, itis difficult to explain why some groups wouldonly serve as experimentation objects whileothers are beneficiaries of the aidintervention under evaluation.

Regardless of the particular methodologyused, evaluation faces structural problems.


6.4. Evaluation can play a majorrole in bridging the micro-macro

gaps

Key problems are briefly described here. Oneof the frequent challenges in evaluation isthe lack of a clear definition of instrumentsand targets as well as limited understandingof the exact channels through which theinstruments are expected to generateoutcomes. This manifests itself in weaklogical frameworks of programmes andprojects. So, for example, the logical frameof a road project often lists among expectedimpacts an increase in GDP growth.Certainly, there are many intermediatetargets between road construction and GDPgrowth, and unless the entire chain ofcausation is clearly defined, it is difficult forevaluation to be effective. Moreover,evaluation faces the classic problem ofdiscrepancy between instruments and goalswhen too many goals are set with too fewinstruments. Thus, when the evaluation failsto find the expected outcome of aid, it isdifficult to know whether the failure is dueto a bad choice of instruments orbottlenecks in the intermediate causalrelationships. This also means thatinnovations such as the so-called “results-based-management” frameworks cannot beeffective without a sound definition oftargets, instruments and transmissionmechanisms.

Moreover, without a good baseline andcontrol groups, reasonable relative progressmay be misjudged as failure. For example, ina post-conflict country, large improvementsin institutional and economic performanceare difficult to achieve in the short run. Toil lustrate the point, in these countries,achieving the MDGs to reduce poverty byhalf by 2015 may be impossible. But theremay be substantial improvements relative tothe no-project scenario. In this case, without

reasonable evaluation criteria, interventionsin such settings are inherently set to fail – the“set-to-fail” syndrome. Here therecommendation would be to look at notonly the achievement of final targets but alsothe extent of efforts and relativeimprovements. To use a sports analogy,evaluation should seek to crown not onlymost valuable players but also mostimproved players.

While all donors consider evaluation as animportant tool for aid planning andmanagement, they are nevertheless aware ofthe possible negative repercussions thatstem from negative evaluation results.Unsatisfactory evaluations may jeopardisenew aid budget allocations (by Congress orParl iament) and even damage therelationships between donor and recipientgovernments. On the recipient side, there isa risk that negative evaluations mayjeopardise new aid. These risks may causeboth an underinvestment in evaluation anddelayed evaluations due to the “fear-of-the-unknown” effect.

6.4.2. How can evaluation contributeto bridging the micro-macrogap?

Well designed and carefully executedevaluations can help establish better linkagesbetween micro level aid outcomes and themacro level impacts. In other words, to theextent that evaluations are well implementedand systematically integrated along the entireoperational cycle, they can help optimise theaggregation of project or programme-leveloutcomes into national level impacts. Thisrequires a number of innovations in the wayevaluations are designed, implemented and


utilised in aid policy. Below are key avenuesof possible improvements in that regard.

Evaluation as development diagnostics

Evaluations can help bridge the macro-microgap to the extent that they are conceived as“development diagnostics” aimed atuncovering the key drivers of intendeddevelopment outcomes, as well as thechannels of transmission from theintervention to the ultimate outcomes. Thisrequires deep knowledge of the sectorsinvolved, the specificities of the country andregion, including economic and non-economic features that influence thebehaviour of agents and overall economicperformance. Evidently such knowledgedoes not necessarily have to be generatedwithin a particular evaluation exercise. Whatis needed is close synergies between appliedresearch and evaluation to make evaluationgenuinely knowledge-intensive.

Comprehensive ex-ante evaluation as deci-sion-making tool

Many multilateral DFIs have endorsed thepractice of ex-ante evaluation ofdevelopment outcomes as a tool for guidingdecisions in private sector financingoperations. The objective is to identify andattempt to quantify the expectedadditionality and development outcomes ofprivate sector operations. However, despitethe increase in private sector portfolios ofDFIs, the dominant lending window remainsthe public sector. The latter is sti l l notcovered by ex-ante evaluation ofadditionality and development outcomes inmost DFIs. Two innovations are needed toharness the value added of ex-anteevaluations. First, these evaluations need to

be extended to the entire portfolio ofmultilateral financing institutions includingpublic sector operations. Second, ex-anteevaluations need to be more comprehensiveand address al l aspects of developmentoutcomes, including policy and institutionalimpacts. At the moment, the analysis oninstitutional impacts and potentialimplications for policy is still limited. Yet, thisis an area of great potential value addedtowards improving aid effectiveness andbridging the micro-macro gaps in aideffectiveness.

Evaluation to enhance policy and institu-tional impact of aid

As discussed earlier, a major weak link in theaid effectiveness chain is the l imitedcontribution of aid to improvements inpolicy and institutions. Part of the reason isthat it is typically not an explicit goal of aidinterventions to improve policy andinstitutions. This is either becauseinstitutions and policy are seen as toosensitive or simply because it is believed thataid cannot meaningfully influence policy andinstitutions. Yet for aid to be more effectiveat the macro level , or for micro levelinterventions to translate into nationaldevelopment outcomes, it is indispensableto incorporate improvements of policy andinstitutions as part of the goals of aid. It istherefore important for evaluations to payparticular attention to the impact of aid onpolicy and institutions. In this way,evaluations can help the aid process byidentifying the factors that make aideffective in improving policy and institutions,and by uncovering the mechanisms andchannels that generate such positive impacts.This requires a rethinking of the design andimplementation of evaluation frameworks to


bolster the policy and institutionaldimensions.

Better integration of evaluation outcomesinto operations

The evaluation functions are understandablytypically separated from the lendingfunctions of most institutions andgovernments. This preserves theindependence of the evaluator to ensure thecredibil ity and reliabil ity of evaluationfindings. However, independence carriessome costs. It prevents optimal use of thefeedback from evaluation in policyformulation and in the design andimplementation of operations. Moreover,the feedback from policy design andoperations into evaluations is imperfect.Hence the learning is suboptimal, with therisk that errors and mistakes be repeatedover time. An evaluation is good only if itinforms policy. One way out is to require thatprogramme officers systematicallydemonstrate that past evaluation has beenintegrated in the design and implementationof new interventions. Explicit requirement tobuild upon lessons from past evaluationswould promote the institutionalisation ofintegration of evaluation into operations.

Transparency, participation and public dis-closure of information

For evaluations to serve as an effective toolof aid effectiveness, it is essential to developa culture of transparency, participation andpublic disclosure of information in donoragencies and the donor community ingeneral. While most DFIs have establishedpolicies on disclosure and access toinformation, [ 53] these policies are rarelyknown by the target public and they arepoorly implemented partly due to lack ofadequate resources. Disclosure ofinformation on aid is even less frequent inmany government agencies. The increase inthe number of donors is accompanied bygrowing disparities in the practice ofinformation disclosure on aid, despite callsfor donor coordination and harmonisation.Yet, public disclosure of information isimportant to enable the recipientpopulations as well as the public in donorcountries to keep up with the use of aidresources and their concrete impact ondevelopment. Thus, transparency and publicdisclosure of information are key to the aideffectiveness agenda.


[53] The World Bank Policy on Access to Information dates from 2010; The African Development Bank is at an advancedstage of updating its Policy on Disclosure of Information.

The debate on aid effectiveness has beendisproportionately focused on macro leveloutcomes that look at the impact of aid onnational development outcomes such asgrowth, improvements in the quality of lifebrought about by better education, and thehealth status of the general population.However, individual aid interventions do notactually affect these outcomes directly.While aid effectiveness at the aggregate levelremains unsatisfactory, the aid landscapecontains individual success stories at themicro level. The dilemma is how to bridgethis micro-macro gap.

The analysis in this paper suggests thatincreasing aid effectiveness wil l requireimprovements at three levels: (1) improvedeffectiveness of aid at the micro-level; i.e. atproject and sector level; (2) moretransparency, better reporting and publicdisclosure of information on developmentoutcomes; and (3) better aggregation ofmicro level outcomes into macro levelimpacts. The paper argues that such anaggregation technology must be knowledge-and institution-intensive. Institutions arecritical for not only the quality of outcomesof individual policy interventions (i.e. microlevel effectiveness), but also for facilitatingpositive spil lover effects of individualinterventions into the rest of the economy.Institutions also facil itate learning fromexperience, which is essential toimprovement in policy design, implemen -

tation mechanisms and the overall impact ofaid at the macro level.

For evaluations to contribute to bridging themacro-micro gaps in aid effectiveness andhelp dissipate the clouds surrounding theimpact of aid, some innovations in the designand implementation of evaluations, andreporting on aid results are essential. First,this requires a substantive increase in theknowledge intensity of evaluations. Secondly,it is important to achieve higher systematicutil isation of evaluation-generatedknowledge in policy and programming thanobserved in current practice. Third, it isimportant to improve the reportingmechanisms and systematically embed aidreporting and dissemination of results intothe aid planning and delivery processes bothat the project and programme levels.

To achieve these innovations, donors andgovernments in recipient countries need toput their money where their mouth is: ifthey believe in evaluation, then they mustadequately resource it . This requiresinvesting more in evaluation through higherbudgetary allocations. It is also necessary toinvest more in building capacity and skills inevaluation at the donor and recipientcountry level. Moreover, it is imperative todevelop a culture of transparency, opennessand public disclosure of information on aidmanagement in donor and recipientcountries. This will improve accountabilityand ultimately enhance aid effectiveness.


6.5. Conclusion

AFRICA INFRASTRUCTURE COUNTRY DIAGNOSTIC (2009) , Africa’s Infrastructure: A Time forTransformation, World Bank, Washington D.C, available at www.infrastructureafrica.org

BAMBERGER, M. and H. WHITE (2007), “Using strong evaluation designs in developing countries,Experience and challenges”, Journal of Multidisciplinary Evaluation, 4(8): 58–72.

BIRDSALL, L, N. (2007), “Do no harm, aid, weak institutions and the missing middle in Africa”,Development Policy Review, 25(5): 575–598.

BURNSIDE, C. and D. DOLLAR (2000), “Aid, policies, and growth”, American Economic Review,90(4): 847–68.

CLEMENS, M., S. RADELET, and R. BHAVNANI (2004), “Counting chickens when they hatch, theshort-term effect of aid on growth”, Working Paper 44, Center for Global Development,Washington DC. (July).

COLLIER, P. and J. DHEN (2001), “Aid, shocks, and growth”, Working Paper 2688, World Bank,Washington DC (October).

COLLIER, P., and D. DOLLAR (2004), “Development effectiveness: What have we learnt?” TheEconomic Journal, 114 (496), F244–71.

COLLIER, P. and A. HOEFFLER (2004), “Aid, policy and growth in post-conflict societies”, EuropeanEconomic Review, 48(5): 1125–45.

DALGAARD, C.-J. and H. HANSEN (2001) , “On aid, growth and good policies”, Journal ofDevelopment Studies, 37(6): 17–41.

DALGAARD, C.-J., H. HANSEN and F. TARP (2004), “On the empirics of foreign aid and growth”,The Economic Journal, 114(496): F191–216.

DOUCOULIAGOS, H. and M. PALDAM (2005), “The aid effectiveness literature, the sad results of40 years of research”, Working Paper 2005–15, Department of Economics, University ofAarhus, Aarhus, Denmark.

DREHER, A., P. NUNNENKAMP and R. THIELE (2007), “Does aid for education educate children?Evidence from panel data”, Mimeo, The Kiel Institute for the World Economy (January).

DUFLO, E. and A. BANERJEE (2009), “The Experimental approach to development economics”,Annual Review of Economics, 1: 151-178.


References

DUFLO, E. and M. KREMER (2005) , “Use of randomization in the evaluation of developmenteffectiveness”, in George Pitman, Osvaldo Feinstein and Gregory Ingram (eds.), EvaluatingDevelopment Effectiveness , Transaction Publishers, New Brunswick, NJ 2005, pp. 205–232.

EASTERLY, W. (2006a), “The Big Push Déjà Vu, A Review of Jeffrey Sachs’s The End of Poverty,Economic Possibilities for Our Time”, Journal of Economic Literature, 44 (March): 96–105.

EASTERLY, W. (2006b), The White Man’s Burden, Why the West’s Efforts to Aid the Rest HaveDone So Much Ill and So Little Good , Penguin Press, New York.

EASTERLY, W. (2009) , “Can the West Save Africa?”, Journal of Economic Literature, 47(2):373–447.

EASTERLY, W., R. LEVINE and D. ROODMAN (2004) , “Aid, policies, and growth: Comment,”American Economic Review, 94(3): 774–80.

GUILLAUMONT, P. (2007) , « La vulnérabil ité économique, défi persistant a la croissanceafricaine », Revue Africaine de Développement, 19(1): 123–162.

GUILLAUMONT, P. (2009), “An economic vulnerability index, its design and use for internationaldevelopment policy”, Oxford Development Studies, 37(3): 193–228.

GUILLAUMONT, P. (2010), “Assessing the economic vulnerability of small island developing statesand the least developed countries”, Journal of Development Studies, 46(5): 828–854.

GUILLAUMONT, P. and L. CHAUVET (2001), “Aid and performance, a reassessment”, Journal ofDevelopment Studies, 37(6): 66–92.

GUILLAUMONT, P. and S. GUILLAUMONT-JEANNENEY (2009) , “Accounting for vulnerability ofAfrican countries in performance based aid allocation”, African Development Bank WorkingPaper No.103 (October).

GUILLAUMONT, P. and C. SIMONET (2011), “To what extent are African countries vulnerable toclimate change? Lessons of a new indicator of physical vulnerability to climate change”, CERDI/ Université d’Auvergne.

HANSEN, H. and F. TARP (2000) , “Aid effectiveness disputed”, Journal of InternationalDevelopment, 12(3): 375–98.

HANSEN, H. and F. TARP (2001) , “Aid and growth regressions”, Journal of DevelopmentEconomics, 64(2): 547–70.

INDEPENDENT EVALUATION GROUP (2008), “IFC’s Experience and Additionality in Middle IncomeCountries Results and Challenges,” Note No.1.

IFC (2010), “IFC’s Additionality: A Primer”, September 30, 2010.


JICA (2004), JICA Guideline for Project Evaluation – Practical Methods for Project Evaluation.(September).

KANBUR, R. (2000) , “Aid conditionality and debt in Africa”, In F. Tarp (ed.) Foreign Aid andDevelopment – Lessons Learnt and Directions for the Future , Routledge, London, pp.409–422.

KILLICK, T. (1998), Aid and the Political Economy of Policy Change , Routledge/ODI, London.

MCGILLIVRAY, M., M. S. FEENEY, N. HERMES and R. LENSINK (2005), “It works; it doesn’t; it can, butthat depends..., 50 years of controversy over the macroeconomic impact of development aid”,Working Paper 2005/24, World Institute for Development Economics Research, Helsinki(August).

MDGS AFRICA STEERING GROUP (2008), “Achieving MDGs in Africa”, New York, United Nations.

MICHAELOWA, K. and A. WEBER (2006), “Aid effectiveness reconsidered, panel data evidence forthe education sector”, Discussion Paper 264, Hamburg Institute of International Economics.

MISHRA, P. and D. NEWHOUSE (2007), “Health aid and infant mortality,” Working Paper 07/100,International Monetary Fund, Washington, DC. (April),

MOLD, A. (2009), Policy Ownership and Aid Conditionality in Light of the Financial Crisis –A Critical Review, OECD, Paris.

MOSLEY, P. (1980) , “Aid, savings and growth revisited,” Oxford Bulletin of Economics andStatistics, 42(2), 79–96.

OSTROM, E. , C. GIBSON, S. SHIVAKUMAR and K. ANDERSSON (2001) , Aid, Incentives, andSustainability, An Institutional Analysis of Development Cooperation . Sida Studies inEvaluation 02/01.

CLEMENS, M., S. RADELET, and R. BHAVNANI (2004), “Counting Chickens When They Hatch: TheShort-Term Effect of Aid on Growth”, CGD Working Paper No. 44, the Center for GlobalDevelopment,

WASHINGTON DC RAJAN, R. and A. SUBRAMANIAN (2005), “What Undermines Aid’s Impact onGrowth?”, IMF Working Paper 05/126, International Monetary Fund, Washington.

ROODMAN, D. (2007a) , “Macro Aid Effectiveness Research: A Guide for the Perplexed”,Working Paper 134, Center for Global Development.

ROODMAN, D. (2007b) , “The anarchy of numbers: aid, development, and cross-countryempirics”, World Bank Economic Review, 21(2): 255–77.

SACHS, J. (2005), The End of Poverty, Economic Possibilities for our Time , Penguin Press, NewYork.


SACHS, J., J. MCARTHUR, G. SCHMIDT-TRAUB, K. KRUK, C. BAHADUR, M. FAYE and G. MCCORD.(2004), “Ending Africa’s poverty trap”, Brookings Papers on Economic Activity , 1.

SVENSSON, J. (1999), “Aid, growth and democracy”, Economics and Politics,11(3): 275–97.

UNITTED NATIONS MDG TASK FORCE (2011), Millennium Development Goal 8 - The GlobalPartnership for Development, Time to Deliver , United Nations, New York.

USAID (2011), Evaluation – Learning from Experience. USAID Evaluation Policy. (January)

VAESSEN, J. (2010) , “Challenges in impact evaluation of development interventions,opportunities & limitations”, Discussion Paper 2010/01, Institute of Development Policy andManagement, University of Antwerp.


t? n e m Agence Française de Développement 05 Evaluation ...

Documents

Transcript of t? n e m Agence Française de Développement 05 Evaluation ...