1 | P a g e

1. BASIC INFORMATION

a. Basic project data

Project title: Growth and Public Sector Efficiency (GPSE) Program Project code: P-MZ-K00-009 Instrument number(s): 2100150025096 Project type: Program-Based Operation (PBO)

Sector: Multi-sector & Finance (Budget Support)

Country: Mozambique Environmental categorization (1-3) : 3 Processing Milestones Key Events Disbursement and Closing date

Date approved: 20/09/2011 Cancelled amount: 0 Original disbursement deadline: 31/12/2013 Date signed: 31/10/2011 Supplementary financing: 0 Original closing date: 31/12/2013 Date of entry into force : 15/12/2011

Restructuring: N/A Revised disbursement deadline: N/A

Date effective for 1st disbursement: 15/12/2011

Extensions (specify dates): None Revised closing date: N/A

Date of actual 1st : 22.12.2011

b. Financing sources

Financing source/ instrument

(MUA)

Approved amount

(MUA) :

Disbursed

amount

(MUA) :

Percentage disbursed

(%):

Loan: 60 60 100% Grant: N/A N/A N/A Government: N/A N/A N/A Other (ex. Co-financiers): 927 920.5 99.3%

TOTAL : 987 980.5 99.3%

Co-financiers and other external partners:

Execution and implementation agencies: Ministry of Planning and Development (MPD)

c. Responsible Bank staff

Position At approval At completion

Regional Director Ebrima FAAL Sector Director Isaac Ndoumbe LOBE Isaac Ndoumbe LOBE Sector Manager Jacob MUKETE Jacob MUKETE Task Manager Carlos MOLLINEDO Emilio DAVA Alternate Task Manager Antón Leis GARCIA PCR Team Leader Emílio DAVA PCR Team Members Antón LEIS GARCIA, Abel MENETE and

Jorgina MUANANOUA

d. Report data

PCR Date : 14/4/2014 PCR Mission Date: From: 17/10/2013 To: 25/10/2013 PCR-EN Date: N/A

Evaluator 1: Boubacar LY Evaluator 2: Ringo Star

PCR EVALUATION NOTE FOR PUBLIC SECTOR OPERATIONS

2 | P a g e

2. PROJECT DESCRIPTION Summary from Appraisal Report including addendum/corrigendum or loan agreement, and taking into account any modification that occurred during the implementation phase.

a. Rationale and expected impacts: Provide a brief and precise description on the project/programme rationale (concerns/questions raised), expected impacts and the intended beneficiaries (directly or indirectly impacted by the project/programme). Highlight any change that occurred during the execution phase.

Rationale: The Government of Mozambique (GoM) has undertaken numerous concerted reforms to bolster the economy and address its fragility. The economy grew 7.5% per year between 1993-2009 driven by large inflows of Foreign Direct Investment (FDI) mostly linked to natural resources exploitation, Official Development Assistance (ODA) and agriculture. The country still faces severe constraints, in particular, the lack of economic diversification, weak institutions, high costs of financing and weak infrastructure development that hinders private sector competitiveness, especially the Small and Medium Enterprises (SMEs). The Growth and Public Sector Efficiency Program (GPSE), a bedrock to sustain and deepen reforms supported by the Bank‘s previous two Development Budget Support Loans (PRSL I and II), is closely aligned to Mozambique‘s development objectives as set forth in the country‘s Poverty Reduction Action Plan (PRAP) 2011-2014. At the time of project approval, the GoM was expected to face growing inflationary and fiscal pressures due to external factors such as international food and fuel prices and increased spending needs to address social concerns. These inflationary and fiscal pressures could have adverse immediate effects on the country‘s hard-won macroeconomic stability. In response, the GoM took measures to maintain a prudent macroeconomic policy stance by reducing the overall spending envelope through austerity measures, including the gradual removal of the fuel subsidy, while at the same time strengthening the targeting of social protection programs. Since the consolidation of these efforts was to be progressive, the GPSE program financing was instrumental to assist the GoM overcome these short-term fiscal needs. Expected impacts: The GPSE loan is expected to contribute to providing the GoM with the requisite fiscal space to undertake the above measures while continuing structural reforms and stepping-up programmes of infrastructure investments programs needed to achieve sustained and inclusive economic growth and contribute to poverty reduction. Intended beneficiaries: In light of the broad set of policies encompassed in the GPSE, the spectrum of beneficiaries along the value chain were expected to be the Mozambican population as whole, in particular small-scale private sector operators and their employees and youths. Specific beneficiary groups include: (i) SME owners and their employees due to smoother bureaucratic requirements for business and trade, greater access to credit and labor opportunities and a better-trained workforce; (ii) rural businesses and individuals due to expanded rural micro-lending, PPP-led transport and service infrastructure and local distribution of revenues from resource extractive industries; (iii) foreign and domestic investors due to a more reliable and transparent legal and regulatory framework; and (iv) taxpayers through strengthened PFM and greater revenue from mega-project concessions. Project Cost and Financing: The project total cost at appraisal amounted to UA 799 million. The project cost to be supported by the AfDB group (ADF loan) was estimated at UA 60 million, against UA 739 million parallel financing, from a DfID grant (UA 148 million), a World Bank loan (UA 127 million), and a mix of loans and grants from other G19 donors (UA 464 million). The borrower did not contribute. At completion, the project total cost was estimated at UA 987 million. Although financing from the

3 | P a g e

AfDB and the borrower remained unchanged, those of the DfID (UA 174.9 million), the World Bank (UA 205.2 million), and the other G19 donors (UA 553.6 million) increased significantly.

b. Objectives/Expected Outcomes: Provide a clear and concise description of the project objectives, expected outcomes, and intended beneficiaries. In so doing, highlight any revision/amendment.

The overarching objective of the GSPE is to enhance a more conducive environment for job creation and skills acquisition, with a view to promote more inclusive growth and accelerated poverty reduction. The expected outcomes of the GSPE reform program are to: 1) Enhance public sector efficiency and transparency 2) Strengthen business creation, skill and technology acquisition Program beneficiaries targeted on a needs basis by the GPSE are: 1) SME owners and their employees 2) Rural businesses and individuals 3) Foreign and domestic investors 4) Taxpayers

c. Outputs and intended beneficiaries: Provide a clear and concise description the expected outputs and intended beneficiaries. In so doing, highlight any revision/amendment.

In congruence with the Appraisal Report, the GPSE program seeks to deliver the following set of 12 outputs pertinent to its two primary components which are: Component 1: Enhance Public Sector Efficiency and Transparency

The 6 outputs connecting to Component 1 are:

1. Increased total value of budgetary spending using direct execution 2. Increased number of organs with operational control units at central and provincial levels 3. Budget covered by external audits in accordance with norms 4. EITI report to the international EITI secretariat 5. Cabinet approval of PPP Law (Regulatory instrument) 6. Cabinet approval of the revised draft procurement decree, dissemination plan implemented

Component 2: Improve the Environment for Business and Employment Creation

The 6 outputs relating to Component 2 are:

1. Revised decree relative to the one-stop shops for creating enterprises 2. Operationalized an electronic single window customs management 3. Implemented a first one-stop border Post on the Mozambique – South Africa border 4. Approved the revised microfinance regulations 5. Approved the decree creating the National Professional Education Authority (ANEP) by Cabinet 6. Established an Observatory and Labor Market Information System

4 | P a g e

The delivery of these outputs arguably would comprehensively and reasonably contribute towards the attainment of the two key outcomes namely; Enhanced public sector efficiency and transparency and strengthened business creation, skill and technology acquisition. This will translate into benefits for SME owners and their employees, rural businesses and individuals, foreign and domestic investors and taxpayers.

d. Principal activities/Components: Provide a clear and concise description of the principal activities/components. In so doing, highlight any revision/amendment.

The program has 2 key components with associated sub-components which are: Component I: Enhance Public Sector Efficiency and Transparency which entails:

Making partnerships between the public and private sectors more inclusive and transparent Enhancing the public finance management framework

Component II: Improve the Environment for Business and Employment Creation involving: Simplifying business procedures and facilitate trade to promote job creation Improving SME access to financial services and skilled human capital

3. PROJECT PERFORMANCE ASSESSMENT

5 | P a g e

RELEVANCE a. Relevance of the project development objective: Evaluation of the relevance ex-ante and ex-post (including during the implementation phase). The relevance of the project objective (during the evaluation ex-ante and the post-evaluation) in terms of alignment with country’s development priorities and strategies, the beneficiary needs (including any changes that may have occurred during the implementation), applicable Bank sector strategies, the Bank country/ regional strategy, and general strategic priorities of the Bank. This criterion equally assesses the extent to which the project’s development objective was clearly stated and focused on outcomes and the realism of the intended outcomes in the project setting.

The review accedes that the relevance of the project development objective is highly satisfactory

(4).

The PCR states the development objective clearly in line with the Appraisal Report, its link with the Bank CSP 2011-2015, country strategy and development priorities leading to the realisation of the envisaged outcomes. The objective of the GPSE was to create conditions to foster more inclusive economic growth and poverty reduction by enhancing public sector efficiency, skilled human capital development and job creation. The GPSE thus, sought to achieve sustained, inclusive economic growth based on increasing investments and contribute to poverty reduction. The outcomes were enhanced public sector efficiency and transparency and strengthened business creation, skills and technology acquisition. In the PCR, it is noted that the GPSE is consistent with the Bank’s CSP 2011-2015, and governance in support of inclusive growth pillar” which focusses on “promoting SMEs competitiveness, job creation, income generation, trade facilitation to better position local products in the region and the country’s PRSP for 2010-2014. The relevance of the development objective is further demonstrated by the following specifically that:

The GPSE is strongly aligned with the Bank Governance Strategic Directions and Action Plan (GAP 2008-2012), the Medium-Term Strategy 2008-12 and the RISP 20111-2015. At the time of this review, the GPSE commands delicate relevance to the TYS 2013-2022. It further builds on the gains of the FSTAP 2005-2010 and the Financial Sector Development Strategy 2012-2022.

The project linked crisply with the overarching goal and the complimentary outcomes of the country’s Vision, Agenda 2025, PARP 2011-2014 and the PRSP for 2010-2014 which targets employment creation; improving employability; and facilitating linkages between labor supply and demand.

The program had causal links with its targeted beneficiaries in its formulation and account of the development objective. The review acknowledges the clear identification of constraints facing SME owners and their staff are constrained by bureaucratic requirements for business and trade, greater access to credit and labor opportunities and a better-trained workforce, rural businesses and individuals due to expanded rural micro-lending, PPP-led transport and service infrastructure and local distribution of revenues from resource extractive industries. It was critical that the PCR explored and stated other beneficiaries for instance foreign and domestic investors due to a more reliable and transparent legal and regulatory framework; and taxpayers through strengthened PFM and greater revenue from mega-project concessions.

The alignment with countries’ national development strategies, institutions, and procedures is

6 | P a g e

well emphasized as in the Paris Declaration (alignment, ownership, and harmonisation). b. Relevance of project design (from approval to completion): The evaluator should provide an assessment of the relevance of the project design regardless of the one provided in the PCR. The evaluator will also comment on the PCR conclusion for this section, and will provide an evaluation of the relevance of the project design. The latter assesses the soundness and the timing of eventual adjustments, or technical solutions to ensure the achievement of the intended results (outcomes and outputs), the adequacy of the risk assessment, environmental and social protection measures, as well as the implementation arrangements. For Programme Based Operations (PBO), an assessment will be made on the relevance of the prior actions, the policy dialogue and the extent to which the operation could have been more pro-poor in its design.

The review concurs that the design of the project was consistently conducive to achieving the

intended results and therefore satisfactory (3).

The review noted that the project was designed to enhancing public sector efficiency and transparency and was aligned with the two support pillars of the country’s medium-term strategy, namely, good governance and macroeconomic and fiscal management. Its design was consistently conducive to achieving the intended result. No adjustment was made to the scope of the program. However, the implementation faced some challenges that required corrective actions in order to ensure the achievement of the intended outputs and outcomes. Designed in a context of declining real government spending in priority sectors from 2009 to 2011, the GPSE program aimed at providing financial resources to the government in to order to enhance the implementation of policy reforms, improve labor skills and the conduciveness of the overall business environment for business development purposes. At the time of the design, education, infrastructure; stagnation in poverty reduction (headcount poverty index was 54.1% in 2005 and 54,7% in 2009); increase in the primary fiscal deficit (from -4 % of the GDP in 2009 to -5%, in 2011); deterioration of the business enabling environment, a steep HDI and lower international competiveness were among insurmountable challenges facing the GoM. This created significant demand and a pervasive need for continued and increased support to the expansion of government spending and funding for policy reforms. The design profited from the practices of institutional support projects and the lessons from the evaluation of past operations, including the PRSL II, 2008-2010 and lessons drawn from the Country Portfolio Improvement Plan (CPIP) 2011. The review noted that the design put in place Monitoring and Implementation arrangements (p 16 Appraisal Report). The Directorate of Investment and Cooperation within the MPD was dedicated with the facilitation of the data required for monitoring the program. In its design, the Bank was tasked with monitoring progress of the operation in coordination with Mozambique‘s main DPs and maintain dialogue with counterparts in MPD; the ministries of Finance, Commerce, Mining Resources, Labor and Education; the Bank of Mozambique; tax and customs authorities; and other offices critical to the success of the program. The review further noted that in its design periodic, monitoring by the Bank would take place during semi-annual field missions, while the country economist was tasked to undertake more frequent in-country monitoring. It was noted that the key tenets governing evaluation were considered in the MoU signed by the GoM and program donors in 2009 while the GoM and the G19 shared jointly the responsibility for monitoring the implementation of mutually agreed indicators through bi-annual joint reviews.

7 | P a g e

Implementation arrangements: The programme duration is three FYs between 2011 and 2013. Disbursements will consist of three equal tranches of UA 20 million each. The first disbursement will take place upon the entry into force of the Loan Agreement and fulfillment of the conditions precedent to the first disbursement; while the other two disbursements will take place after the satisfactory reviews of program execution, and subject to GoM satisfying the conditions stipulated in the Loan Agreement. In addition, despite the progress made by the GoM’s in developing a sound procurement system, further improvements were needed; and the GPSE was to support these efforts. However, details on the nature and scope of the support are not presented. Furthermore, the country’s Administrative tribunal was in charge of auditing the funds flow in the program designated account no later than six months after year end. The primary counterpart of the Bank within the MPD is clearly identified and a monitoring and evaluation plan although not very well detailed is presented.

Soundness and timing of eventual adjustments and technical solutions: Two challenges to project implementation have been identified in the Implementation Progress Report (IPR) and corrective actions for each one of them have been proposed. These are: (i) addressing the shortcoming reflected in poor business enabling environment indicators; and (ii) the decision not to carry out a PEFA assessment in 2013 as expected when the program was prepared. The actions proposed for the first challenge lie in the several key reform introduced by the GoM, with impact to be felt in 2013 indicators. To the second challenge, a detail qualitative evaluation of relevant PFM systems was proposed and was to be carried out during the PCR exercise. Inclusiveness: The design provided gender data on women beneficiation. Limitations abound. The first limitation was noted in the extent of alignment of the PBO with the country’s PAF framework and rigidity under a 3-year program with pre-defined disbursement conditions and policy actions for 3 years did not offer much room for an improved and smooth implementation of the program. This created a design loophole over and above lack of consideration of prior actions. A further limitation was noted in the disconnect between the RLF’s clear outline of the risks and mitigation measures which in the opinion of the review should be wielded together with the PCR’s treatment of the design aspects. The ramifications of the limitations at output and outcomes level is significant. In tracking the theory of change, it is noted that intended outcomes were unsatisfactory.

8 | P a g e

EFFECTIVENESS

c. Effectiveness in delivering outputs: Evaluation of the extent to which the project achieved its stated results (obtained from the logical framework) based on the last Implementation Progress and Results Report (IPR) and by considering accurate reporting of direct or indirect evidence on intended and unanticipated outputs. In the absence of sufficient data (as direct evidence), indirect evidence (such as project outcomes and other pertinent processes/elements of the causal chain) should be used particularly in the evaluation of the extent to which the project is expected to achieve its stated results/ objectives. The absence of sufficient data to assess the effectiveness should be indicated (and clearly detailed in the PCR quality evaluation section). The PCR score should equally be indicated in this section.

The review validates that the effectiveness of the project in delivering intended outputs is

satisfactory (3). In its analysis, the review finds that the delivery of project outputs was reasonably strong and had a higher likelihood of achievement based on an 83% output execution rate as confirmed by the PCR and the supervision mission report of June 3rd -5th 2013. The review identified complementarity of outputs but questioned the robustness, quality, thought-provoking nature and functionality of outputs in relation to what obtained on the ground. The review arrived at this satisfactory determination on the following basis:

The project has a reasonably high output execution rate of 83%. The majority of the outputs (8) were fully met and delivered on time. Due to the nature of the legislative process and relatively higher turnarounds involved in

reviewing, amending and ascending legislation, 2 outputs were on track and one of the 2 awaiting funding. One output out of the 12 is unmet and unaccounted for.

At this execution rate, the review reasonably foresees the project being able to achieve its expected results, i.e. the intended set of outputs and outcomes.

The review’s analysed and established that the number of outputs achieved were actually 10 out of 12 and not 11 out of 12 as in the PCR (p 9). The review noted that on the basis of reasonable foreseeability, the probability of achieving the planned microfinance regulatory framework was high on account of 75% execution rate and therefore perceived the possibility of its achievement as highly likely. It however cautions it risk to guarantee and report legislative provisions as achieved when still at drafting stage of the legal process especially when it is subordinate legislation as the case. A further related challenge is the confirmation of legislative capacity of the system. The review criticised the lack of attribution for failure of approval and decree establishing an Observatory and Labor Market Information System (0%) and delays in the approval of revised microfinance regulations by the Bank of Mozambique (BoM), supposedly expected to move with speed and decisiveness as a monetary authority. The review found it more pragmatic to have reported at a far more advanced phase such as the approval stage of the prudential instruments and further observes that the PCR does not systematically link the outputs to each component towards which it contributes as outlined below.

Outputs for Component 1: Enhance Public Sector Efficiency and Transparency The outputs were delivered as follows and is confirmed by independence evidence cited in this report:

a) 117% scored on the total value of budgetary spending using direct execution

9 | P a g e

b) 100% of organs with operational control units at central and provincial levels, c) 100% of the budget covered by external audits in accordance with norms, d) 100% EITI report to the international EITI secretariat, https://eiti.org/news/mozambique-

declared-eiti-compliant e) 100% Cabinet approval of PPP Law, https://www.google.com/search?q=FSDS+2013-

2022+strategy+Mozambique&ie=utf-8&oe=utf-8&client=firefox-

b#q=Mozambique+Cabinet+approval+of+PPP+Law+2011 f) 100% Cabinet approval of the revised draft procurement decree, dissemination plan

implemented All of the outputs for component 1 are on track to reach final targets and were investigated by the review though unconnected multiple lines of evidence. Outputs for Component 2: Improve the Environment for Business and Employment Creation The outputs were delivered as follows:

g) 100%: The decree relative to the one-stop shops for creating enterprises revised in 2012 as planned and scheduled

h) 100%: Operationalized an electronic single window customs management, https://www.mcnet.co.mz/procedures.aspx?lang=en-US&chapter=1&subchapter=1

i) 100%: Implemented a first one-stop border Post on the Mozambique – South Africa border

j) 75%; Approval of revised microfinance regulations effected k) 80%; Approval of the decree creating the National Professional Education Authority

(ANEP) by Cabinet effected l) 0%: Establish an Observatory and Labor Market Information System not effected

Thus the review confirmed the probable delivery of outputs for Component 2 satisfactory. The review had concerns regarding the formulation of specific outputs namely outputs 1 -3 which it reformulated for clarity as: outpu1- increase the percentage of total value of budgetary spending; output 2 – increase the percentage of organs with operational control units at central and provincial levels; output 3 – increase the percentage of budget covered by external audits in accordance with norms.

d. Effectiveness in delivering outcomes: Evaluation of the extent to which the project achieved its intended set of outcomes (including for Program Based Operations (PBOs) where complementary measures are necessary for their implementation, namely public awareness, policy dialogue and institutional arrangements for instance). The evaluator should make an assessment based on the results of the last project Implementation Progress and Results (IPR). The evaluator shall indicate the degree to which project outcomes (intended and unanticipated) as well as reasons for any eventual gap were discussed in the PCR.

The review disputes that the effectiveness in the delivery of outcomes is satisfactory (3) and rates it

unsatisfactory (2) on re-examination.

In its investigation, the review found that overall, 4 out of 8 outcome indicators are 100% on track while 3 were not met and one was not evaluable. The outcomes performance is relatively weaker based on the yardsticks provided in the Staff Guidelines, Project Completion Reporting and Rating 2012. A huge continuum in the delivery of outcomes was identified by the review. The review analysed and established that while outcome 1 category indicators (Enhanced public sector efficiency and transparency) were 100% on course, indicators pertaining to outcome 2 (Strengthened business creation, skill and technology acquisition) diverged significantly from the end targets.

10 | P a g e

Surprisingly, the outcome indicators in question here were found to be fundamentally realistic and practical as Outcome 1 if not less challenging conventionally. The PCR is silent on the factors accounting for this significant variation on outcomes delivery. It would have been intellectually motivating to invoke attribution analysis and explore the driving factors in pursuit of this variance, provide strategic and qualitative insight into development challenges. For instance exploring the ramifications of legislative reforms on the DBR indicator. The PCR’s basis to claim that both outcomes are on track is unfounded in the absence of a strong evidence based explanation for the reason that as provided by the PCR only one (EITI country compliance-Page 10 of the EITI 2016 progress report) out of the 5 indicators was met on Outcome 2 indicators. Training of women was unsatisfactory with 42.7% performance rate falling short of 57.3% delivery on the target, financial sector deepening data unavailable and a marginal reduction in DBR turnarounds. Thus the PCR needed to investigate more on Outcome 2 challenges. The review concurred with the Project Assessment Result (PRA) which concluded that the achievement of the project’s ultimate impact “achieving sustained, inclusive economic growth based on increasing investments and contribute to poverty reduction” could be compromised. However, based on evidence from the African Economic Outlook, the review confirms that reach and depth of the financial sector in Mozambique is still limited. The program has 2 clear outcomes specifically: Outcome 1. Enhanced public sector efficiency and transparency The program development objective is to create the conditions to foster more inclusive economic growth and poverty reduction by enhancing public sector efficiency, skilled human capital development and job creation. Progress towards achieving this objective is satisfactory with a 100% achievement on all three outcome indicators (Deviation between budgeted and actual expenditure, PEFA indicator PI-19 on competition and procurement and PEFA indicator PI-26 on scope, nature and follow-up on external audits). The EITI was tracked and found that GoM was verified to be on the active list, https://eiti.org/sites/default/files/migrated_files/progressreport.pdf Evidence from the World Bank report, entitled “Mozambique Economic Update: Facing hard choices”, remarks that “an ongoing downturn, brought about by low commodity prices, drought and conflict, was compounded by the fallout from the discovery of hidden debts in April 2016”. In addition as per Supervision report findings, 17 out of 35 PAF indicators were realised, much lower than anticipated. The review finds from the African Economic Outlook, p 10, 2012, OECD, UNECA, UNDP that Mozambique faces challenges with its governance and anti-corruption framework. Government accountability (executive, legislative, judicial) and the civil service are all rated as weak, Outcome 2. Strengthened business creation, skill and technology acquisition. The program is expected not to achieve at least one of its outcomes. Only 1 (EITI country compliance) out of the 8 outcome indicators was on track at the time of the review note. Two outcome indicators were lagging (Number of people (women) trained in professional skills at 42%, trading across borders: time to export and import (80%) and the cumulative number of rural recipients (women) of microfinance credit lacked evidence to assess. The outcomes are on track and likely to be achieved with corrective actions. The review rates outcome 2 as unsatisfactory

The PCR shows that there are multiple output indicators contributing to the outcome 2 which further contributes to impact in the chain. Gross investment as a percentage of GDP, in particular from the private sector, increased considerably since 2011 to 39% and 24.3%, respectively, well in excess of the

11 | P a g e

targets set out in the PAR (23% and 10%). The country has become EITI compliant ahead of schedule and significant progress has been made in the introduction and implementation of PFM reforms. On the business environment front, international indicators (Doing Business, Global Competitiveness Report) do not yet reflect the impact of recent reforms. On technical and vocational education, the number of people trained was only 42, 7% of the final target by end of 2012. The annual assessment of the government performance, carried out by a group of donors providing direct budget support in Mozambique in 2010/2011/ 2012 (EU Report on Assessment of Public Finance Management in Mozambique) shows that overall, 15 of the 28 PEFA indicators relating directly to PFM performance have shown no change and unfortunately, that they are not, as yet, fully reflected in gains in PEFA scores, although they are a necessary condition for the future gains, which might elevate the quality of the Mozambican PFM system to one fully consistent with international good practice. Evidence from the Independent Evaluation of Budget Support in Mozambique Final Report Volume II Annex Table 7, 2014 confirms that 25%, 46% and 33% progress was recorded as achieved in the public sector reform, good governance and PAP indicators respectively. A possible explanation could be that there is a higher likelihood of lack of political will and demand present generic challenges in public sector projects, https://www.oecd.org/dac/peer-

reviews/World%20bank%202004%2010_Steps_to_a_Results_Based_ME_System.pdf.

e. Project development outcome: The ratings derived for outcomes and output are combined to assess the progress the project has made towards realizing its development objectives, based on the rating methodology recommended in the Staff Guidance Note on project completion reporting and rating (see IPR Guidance Note for further instruction on development objective rating).

The review rejects the claim of the PCR that both outcomes and outputs are satisfactory. The review arrives at this decision by applying the formulae for computation of outputs (3) and outcomes (2) consistent with the Staff Guidance on Project Completion Reporting and Rating, August 2012. This results in a rating of 2, being attributed to the score of 2 arising from outcome 2 and the subsequent series of arguments highlighted in the previous section. Reasonably, the development objective could have been probably stronger with outcome 2’s share of the DO being better executed. This is confirmed by multiple independent lines of evidence that illustrate practical challenges with public sector efficiency and accountability as cited in http://allafrica.com/stories/201703160638.html,

and the Africa Research Institute Report (London), http://allafrica.com/stories/201703160638.html

and a possible lack of political will and demand present generic challenges in public sector,

https://www.oecd.org/dac/peer-

reviews/World%20bank%202004%2010_Steps_to_a_Results_Based_ME_System.pdf.

f. Beneficiaries: Using evidence, the evaluator should provide an assessment of the relevance of the total number of beneficiaries by categories and disaggregated by sex.

The review found that the PCR reasonably treated inclusiveness. 42.7% of women, short of 3.3% of the target were trained in professional skills. The review cannot confirm the translation of the knowledge gained into benefits on the ground on account of lack of evidence or at least mentioning of for instance MOVs relating to training having been delivered such as workshop training reports, manuals, feedback on satisfaction or any other accompanying evidence. The review also noted the exclusion of the male

12 | P a g e

counterparts in the design of the project as a deficiency. An inconsistency in the percentages actually trained has been picked up by the review as the PCR. The varying program beneficiaries along the value chain include different categories of the population, due to the broadness of the reform agenda. The cumulative number of rural recipients (women) of microfinance credit the end target being 300 000 and those earmarked for capacity building being (women) trained in professional skills (60% in 2013). The program therefore is relevant and gender responsive, hence covered cross-cutting issues. The Appraisal Report set out to address HIV and AIDS issues. The PCR excluded this aspect which is important in the country’s HDI.

g. Unanticipated additional outcomes (positive or negative, not taken into consideration in the project

logical framework): This includes gender, climate change, as well as social and socio-economic- related issues. Provide an assessment of the extent to which intended or unanticipated additional and important outcomes have been taken into consideration by the PCR. The assessment should also look at the manner the PCR accounted for these outcomes.

The review noted two unanticipated additional outcomes: One positive and one negative. Both are described below. The PCR only took into consideration the first one with regards to the acceleration of the implementation of the Governance Action Plan’s. The second, with regards to the decrease in aid inflow from the G19 was captured in the Project Result Assessment (PRA). Acceleration of the Governance Action Plan’s Implementation: Policy dialogue in the context of the

G-19 development partner group has been instrumental in accelerating the pace of the implementation of the GoM’s Governance Action Plan presented in March 2010 and in increasing funding to the governance sector. While no specific target was included in the Program, this can be considered a positive additional outcome.

Reduction of total aid inflow: The 2010 stand-off between the GoM and the G19 donors unexpectedly hampered the project. The stand-off was result of concerns over fiduciary in the health sector, which has disrupted the predictability of the State Budget and the actual flow of funds that is against the transparency and corruption improvements that the project foresees. As a result, aid disbursement flows from the G19 group decreased by 16% between 2010 and 2011 (from USD 1.35 billion in 2010 to USD 1.14 billion in 2011). This decrease in aid inflow, was expected to negatively impact state’s capacity.

The standoff with donors on fiduciary concerns and the consequential pull of some of the donors presented a negative factor, although eventually resolved. Negative impacts were recorded on the amount received by the government due to exchange rate losses. Furthermore, the Mozambican Central Bank has not received a document confirmation of the SWIFT transfer from Deutsche Bank as stated in the above mentioned MoU. As a way to overcome this impasse, the above mentioned MoU is under review. The PCR proffered no treatment on these issues.

13 | P a g e

EFFICIENCY

h. Timeliness: The timeliness of project implementation is based on a comparison between the planned and actual period of implementation from the date of effectiveness for first disbursement. For Programme Based Operations (PBOs), the timely release of the tranche(s) are assessed through this same criterion.

The review affirms that the timeliness of the GPSE is highly satisfactory (4). The ratio of planned (3 years) to actual implementation (3 years) time for the date of the effective first disbursement is 1. Accordingly, this is highly satisfactory as derived for the Staff Guidance Note, August 2012. Although two tranches were released as planned, the second tranche was released with a minor delay of one month. In addition, all policy actions supported by the GPSE was implemented on a timely manner.

This is a Program Based Operation (PBO) and the timeliness of implementation is based on the timely release of the tranche(s).

As noted above, information extracted from implementation arrangement part of the PAR stipulated that: The programme duration is three FYs between 2011 and 2013. Disbursements will consist of three equal tranches of UA 20 million each. The first disbursement was expected to take place upon the entry into force of the Loan Agreement and fulfillment of the conditions precedent to the first disbursement; while the other two disbursements would take place after the satisfactory reviews of program execution, and subject to GoM satisfying the conditions stipulated in Loan Agreement.

The PCR notes that in 2011 and 2013 the Bank disbursed the tranches exactly in the months it had planned to; while in 2012 the disbursement only experienced a month delay. In addition the completion reporting highlights that nearly all policy actions supported by the programme were implemented in a timely manner. The PRA echoed the information presented in the PCR.

Similarly, the July 2013 Implementation Progress Report (IPR) noted that the first tranche was disbursed immediately following loan effectiveness in December 201; and the two subsequent tranches were disbursed in July 2012 and July 2013, respectively.

i. Resource use efficiency: Provide and assessment of physical implementation (based on outputs delivered) against resources used (based on cumulative commitments) at completion for all contributors to the project (the Bank, Government, and others). This criterion would normally not apply to PBOs, as there is often no direct link between the outputs and the amount of contribution (in which case the rater would indicate N/A).

This not applicable as it is a PBO.

j. Cost-benefit analysis: Provide an assessment of the timeliness of the development outputs, and the extent to which costs of the costs have been effective and have been provided in the most efficient manner. The PCR rating should be discussed. The evaluator should verify whether the benefits of the project (achieved or expected) exceed its actual costs. To achieve this, evidences will mainly be based on a comparison between Economic Rates of Return (ERR) calculated at appraisal, the mid-term review and completion. When commenting PCR ratings, the degree of utilization of valid sources for evidence justifying the rating assigned should be taken into consideration. The evaluator should ensure of the validity of assumptions and that the same model was used for the calculation of others ERRs. For PBOs for which this calculation model

14 | P a g e

does not apply, an assessment could be done with regards to the contribution of policy reforms to economic growth. In the absence of sufficient evidence, an appropriate rating should be assigned.

This not applicable. As a result the review did not score this based on the very nature of the program.

k. Implementation progress: The assessment of the Implementation Progress (IP) on the PCR is derived from the updated IPR and takes into account the all applicable IP criteria assessed under the three categories : i) Compliance with covenants (project covenants, environmental and social safeguards and audit compliance), ii) project systems and procedures (procurement, financial management and monitoring and evaluation), and iii) project execution and financing (disbursement, budget commitments, counterpart funding and co-financing).

The review consents that the implementation progress (IP) is satisfactory (3).

The review notes that the average rating for the applicable IP criteria is ratings is 3, and implementation process has for the most led to anticipated results.

Compliance with covenant: All legal covenants have been met. This a category III Program so no ESMP and or RAP were required. And he Tribunal Administrativo regularly audited the General State accounts supported by thus budget support operation. Regular audits of the flow of funds have been submitted annually to the G19 donor group.

Project systems and procedures: As this was a PBO (budget support operation), procurement and

financial management were implemented in accordance with country systems. In addition, regular M&E was undertaken in the context of the Performance Assessment Framework (PAF) of the G19 donor group based on common indicators. However, M&E experienced some constraints due to the difficulties to obtain some of the data. Certain indicators were no longer collected as part of the PAF, and PBO design limited the flexibility to accommodate those changes.

Tranching: Project execution and financing: Two tranches were disbursed within planned

schedule as conditions precedent were implemented on time. This is a Budget Support operation, as a result, budget commitment and counterpart funding disbursements are not required. There was not data available on disbursement of parallel budget support operations from other G19 donors. July 2013 IPR noted that this information would be available for the preparation of the PCR. Unfortunately there is no mention on other stakeholder’s disbursement in the PCR. However, consistent with the PCR, they were challenges with data collection.

15 | P a g e

SUSTAINABILITY

l. Financial sustainability: Provide an assessment of the extent to which funding mechanisms and modalities (eg. Tariffs, user fees, maintenance fees, budgetary allocations, other stakeholder contributions, aid flows, etc.) have been put in place to ensure the continued flow of benefits after completion, with particular emphasis on financial sustainability. For PBOs, the assessment should focus on financial sustainability of reforms, as well as the Bank’s policy dialogue to promote financial sustainability of the reforms.

The review diverges from the findings of the PCR that financial sustainability is satisfactory (3) but instead unsatisfactory (2). The project’s gruesome challenge lies in the lack of reasonable foreseeability of guaranteed post project political commitment especially on transparency and accountability in the management of revenues and sustained budgetary allocations. The basis for this argument is that despite the Decree approved by Cabinet in 2013, ANEP was not operational due to lack of funds, an even earlier signal of sustainability challenges in spite of the vocal points 10 and 11 in the Letter of Development Policy for the GPSE, 17 June 2011, Annex 1. Further the limitation with Decrees is that its only subordinate legislation and commitment and longevity cannot be guaranteed. Candidly, the review is concerned with the potential grip of power by the executive and the implications on financial sustainability mechanisms. However, the review notes that the mechanisms deemed necessary to sustain the benefits and to ensure continued flow of benefits were put in place, that is effective and better budget planning and execution; transparency in the management of revenues from the exploration of natural resources which mitigated potential governance and social instability risks (e.g. 2013 PAF indicators for Governance), increased domestic resource mobilization, tax policy and administration reforms to increase fiscal and non –fiscal revenues and options to exploit natural resources (coal, gas and oil) reduce fiscal dependence from external sources such as ODA or budget support. There is no evidence to demonstrate functionality and operationalisation of the aforesaid measures. The review believes that the commitment of the G19 donors, the Bank’ s engagement in policy dialogue in fiscal resources and natural resources management, as well as efforts by the GoM to increase fiscal and non-fiscal revenues collection, are deemed sufficient to ensure the flow of benefits associated with the project after completion. However, the flow of capital is closely tied to a governance environment that has recently been criticized by a large part of the donor community.

G19 donors’ support: The long term commitment of the G19 donors to support the GoM’s reforms aimed at strengthening public expenditure, budget planning and execution, as well as transparent management of natural revenues is instrumental for the financial sustainability of the programme. However, in 2014 some of the countries have threatened to suspend their financing of Mozambique’s national budget over allegation of corruption and bad governance.

Bank’s policy dialogue: The Bank is engaging in policy dialogue with other donors and the GoM

16 | P a g e

to explore domestic resource mobilization through Economic Sector Work (ESW), and through a Policy Note on the fiscal framework for natural resource exploitation. This is timely as the recent discovery of natural resources (coal, gas and oil) suggests a future scenario of reduced fiscal dependence form Official Developmental Assistance (ODA) or budget support, provided these resources are managed in transparent manner.

Government of Mozambique: The GoM on its own has put in place tax policy and administration

reforms aimed at increasing fiscal and non-fiscal revenue. As a result, domestic revenue collection has increased steadily (from 14.1% to 23.5% of GDP between 2005 and 2012); while in parallel, a clear reduction of GBS (from 15% to 8% of national budget between 2010 and 2012) over time is noticeable. At completion, domestic revenues represented 72% of the national budget versus 62% in 2010. In addition, macroeconomic forecast for the country at programme’s completion were positive.

However, there is no basis for sustained growth as the country remains in vicious cycles of

fragility.

m. Institutional sustainability and strengthening of capacities: Provide an assessment of the extent to which the project has contributed to the strengthening of institutional capacities – including for instance through the use of country systems – that will continue to facilitate the continued flow of benefits associated with the project. An appreciation should be made with regards to whether or not improved governance practices or improved skills, procedures, incentives, structures, or institutional mechanisms came into effect as a result of the operation. For PBOs, this should include an assessment on the contributions made to building the capacity to lead and manage the policy reform process; the extent to which the political economy of decision making was conducive to reform; the Government’s commitment to reform; and how the design reinforced national ownership.

The review confirms that the institutional sustainability of the project is satisfactory (3).

The review found that in addition of the Government’s commitment to reforms and governance agenda, the design of the GPSE and the programme itself contributed to building lasting capacities to plan and implement policy reforms in Mozambique. At completion, the country systems and capacities are deemed sufficient to ensure the continued flow of benefits associated with the GPSE.

Contribution to building capacity to plan and implement policy reforms: A 2014 Independent Evaluation of the impact of budget support in Mozambique commissioned by the European Union (EU) found that GBS has provided the GoM with the means to implement its national and sectoral development strategies, with significant progress in macroeconomic management, governance, and education. In addition, the GoM’s solid performance in the achievement of key milestones under the GPSE and in maintaining a prudent fiscal and macroeconomic policy is mainly due to a parallel strengthening of capacity in policy-making and public management. Indeed, although capacity building was not the primary objective of the GPSE, some of the policy measures supported by the programme were geared towards professional training.

Government of Mozambique’s commitment to reforms and governance: Commitment to reform has remained strong during the duration of the Programme, despite a few stand-off episodes over financial governance issues. This caused some donors to delay and withhold their budget support disbursements. These issues include fiduciary concerns in the Ministry of Health in 2010 following some reports of corruption, and transparency and fiduciary concerns raised amongst donors over the issuing in 2013 by a State-own-enterprise (SOE), namely, the Tuna National Company – EMATUM, of the first-ever US dollar denominated Government-guaranteed bond on international markets, worth about $850 million. These concerns seem to be dispelled following

17 | P a g e

an IMF’s recent Financial Transparency Assessment. Furthermore, dialogue between the GoM and the donor community is ongoing for reforms to improve transparency and investment planning systems.

Impact of the GPSE design on national ownership: Conditions precedent to disbursement required the creation of certain human and institutional capacities in the country. For example, the disbursement of the third tranche was conditional on the creation of four one-stop-business-shops at the district level in the year 2012 alone.

An EU commissioned assessment in 2012-2013 on the impact of budget support in Mozambique confirmed that GoM has been equipped with the capacity and means to implement its national and sectoral development strategies. This package included, specially, financial resources to expand spending/investment in health and education and met the disbursement conditions of the project to a great extent which prescribed and required the creation of human and institutional capacities in the country. The disbursement of the third tranche was necessary to create four one-stop-business-shops at the district level in the year 2012 alone. The Bank ensured alignment of the program with Governance Strategic Directions and Action Plan (GAP 2008-2012) and fulfilled the six technical prerequisite conditions for budget support operation were fulfilled (Readiness Assessment). Laws governing the modalities of public-private partnerships (PPPs) and concessions were enacted creating an enabling environment for sustainability purposes. This strengthened capacity for business licensing and registration. However, the review highlights that there re capacity constraints in the public sector as confirmed by independent reports such as the IMF report, https://www.imf.org/external/pubs/ft/scr/2015/cr1512.pdf n. Ownership and sustainability of partnerships: Provide an assessment of whether the project has effectively involved relevant stakeholders, promoted a sense of ownership amongst the beneficiaries (both men and women) and put in place effective partnerships with relevant stakeholders (eg. local authorities, civil society organizations, private sector, donors) as required for the continued maintenance of the project outputs. For PBOs, the assessment should measure the extent to which the Government’s capacity to conduct consultations during policy dialogue and the extent to which the Bank supported the Government in deepening the consultation processes.

The review consents that the ownership and sustainability of partnerships is satisfactory (3). It however particularly cautioned on fiduciary concerns as a threat to sustainability and the stand of between donors and government as a grave concern in the light of the precious call of the Paris Declaration on harmonisation and ownership. The program was designed and implemented in alignment with donors providing (direct) budget support (the G-19 group) characterised by a platform for political dialogue between the G-19. This mechanism enhanced discussion and resolution of various issues between donors and the government. The annual PAF assessment enabled donors and the government to exchange notes and experiences generating synergy. The Memorandum of Understanding (MoU, March 2009) between the GoM and G19, including the Bank, established significant donor harmonization based on solid principles among them (i) predictability and alignment with domestic systems; (ii) joint monitoring and all policy actions or

18 | P a g e

expected outcomes are based on the common PAF; (iii) no separate reporting to the donors; and (iv) mutual accountability The Bank’s proven track record of collaboration in Mozambique through support to public sector reforms and PFM issues enhanced continuation prospects of the benefits. The Bank undertook joint missions for institutional financial sector reforms with the World Bank, GTZ and KfW and strengthened jointly supported transparency and accountability reforms as part of the Extractive Industries Transparency Initiative (EITI). The coordination of intervention was also organized through thematic working groups. The program applied good practice principles on conditionality. The Bank worked closely with other DPs providing budget support and the IMF within the PAF framework, to ensure better coordination and increase synergy. Challenges to sustainability include the stand-off moments over financial governance issues in the health and education sectors caused some donors to delay and withhold their contributions to the state and sectoral budget as exemplified in the case of the 2010 fiduciary concerns in the ministry of health and 2013, when the a government-guaranteed firm (the Tuna National Company – EMATUM) issued a first-ever US dollar denominated bond on international markets, worth about $500 m, raised transparency and fiduciary concerns among donors. The Financial Transparency Assessment carried out by the IMF in 2013 reported a positive and strong fiscal transparency stance in government spending and revenues collection mechanisms. This has resulted in a positive platform on which donors can decide on their continuation of budget support in Mozambique.

o. Environmental and social sustainability: Provide an assessment of the objectivity of the PCR rating on the project’s implementation of environmental and social mitigation/enhancement measures with regard to the Environmental and Social Management Plan (ESMP), the capacity of country institutions and systems, as well as the availability of funding to ensure the environmental and social sustainability of the operation. This criterion would normally only apply to Environmental Category I and II projects.

Not Applicable. This is a category 3 project with no impact on the environment, hence it is not evaluable.

19 | P a g e

4. PERFORMANCE OF STAKEHOLDERS

a. Bank performance: (Preparation/approval, ensure of Quality at Entry (QAE) : quality of the supervision, completion) : Provide observations on the objectivity of the PCR ratings and feedback provided by the Borrower, and if necessary, re-assess the Bank’s performance throughout the project cycle (design, implementation, completion) by focusing on evidence from the PCR in relation to 7 criteria defined in the PCR Guidance Note.

The review goes along with the findings of the PCR that the performance of the Bank is

satisfactory (3).

The Government has expressed its satisfaction with the Bank’s performance under the Program. In March 2014 the Government requested a successor program to the GPSE covering the 2014-2016 period. The Bank conducted supervision missions accordingly and disbursed the three full tranches (100%). However, some aspects in the PCR’s assessment of the Bank performance are not discussed. These are the timeliness of responses to requests and lessons learned from previous operations. Although the evaluation was able to collect information on the latter criteria, no information was available to substantiate the former. Quality at entry:

Proactive identification and resolution of problems at different stages of the project cycle: At appraisal a thorough assessment of Program perquisites conditions was conducted by the Bank, and concluded that the country fulfilled both general and technical prerequisite conditions for GBS. The problem identified at the design stage was the misalignment between the M&E framework used for the GPSE and the PAF assessment matrix.

Use of lessons learned from previous operations during the design and implementation: The design of the GPSE was guided by various analytical works, including the key findings and recommendations of the 2010 PEFA, the assessment of governance challenges, PFM Vision 2010-20, and the PCR of the Bank‘s PRSL II, 2008-2010. The operation also benefited from other analytical works by the World Bank, the IMF, the USAID, and the findings of a joint reviews 2008-2011 between the GoM and G19. The GPSE also benefited from lessons learned from the Financial Sector Technical Assistance Program (FSTA) co-financed by the Bank, the World Bank, the IMF and other donors. The design of the Program also benefited from lessons past PBOs in the country.

Promotion of stakeholder participation to strengthen ownership: As discussed above, ownership

is strong and accompanied by extensive consultations, which preceded the adoption of the PARP 2011-2014 supported by the present GPSE. Dialogue during the Bank‘s 2011-2014 CSP preparation with the GoM, country-level stakeholders and other DPs identified policy priorities

20 | P a g e

to address the current challenges. Quality of implementation:

Enforcement of safeguards and fiduciary requirements: Disbursements were made according to the schedule agreed upon with the Government following the 14 timely fulfillment of the conditions of disbursement for each of the three tranches. During supervision mission fiduciary requirements and standards were verified through annual audits of the flow of GBS funds and analyzed by the Bank’s fiduciary services.

Quality of Bank supervision and the timeliness of response to requests: The Bank has regularly (once every six months since 2011) carried out supervision missions to assess progress in the implementation of the GPSE. The missions comprised adequate skills drawn from staff in OSGE and other departments as needed, such as FFCO, ORPF and ECON. No information was available for the review to assess the timeliness of response to requests.

Design and implementation of the M&E system: Program monitoring has taken place in the

context of the joint PAF framework and led by the Mozambique Field Office, thus reducing transaction costs for the Government. However, the PCR notes that the design of the GPSE as a 3-year PBO with predefined performance indicators and disbursement conditions resulted in some difficulties to respond to shifting priorities at the government and PAF levels. The potential consequences of this concern were that it results in poor monitoring and delayed disbursement. Although the disbursement of the three tranches is satisfactory, information collected in the IPR suggest that the monitoring of the Program has been poor. The review has no evidence that a particular solution was proposed for this particular issue, though the lesson emphasized in the PCR.

Completion:

Participation to policy dialogue and/or leadership role: Throughout and beside the project cycle, the Bank actively participated to policy dialogue and the played a leadership role in the context of the Cooperation group. The Bank is indeed a member of the G19 group, as well as its various working groups and sub-groups. Since 2012, the Bank is the chair of the Infrastructure and the Budget Analysis groups. Moreover, the Bank, through the Mozambique Field Office, is an active member of the Economist, Private Sector, and Financial Sector working groups. Policy dialogue at the level of the Heads of Cooperation group and technical discussions through the working groups has been effective during the duration of the Program. The Bank has also actively participated in PAF Annual Reviews and Planning meetings on a semi-annual basis.

(a) Disbursed timeously as planned enabled smooth flow of program implementation and met the

conditionalities for the first and second disbursement (BTOR p 6). The third disbursement was not accounted for creating incomplete analysis.

(b) Engaged in policy dialogue as a member of the G19 group (c) Chaired the infrastructure budget analysis since 2012 and ensured program direction. (d) Carried out regular (once, every six months since 2011) supervision missions to assess the

implementation of the current project. (e) Constituted a good mix of skills during the missions (task managers and co-workers from other

departments as needed, such as FFCO, ORPF and ECON)

21 | P a g e

(f) Fiduciary requirements and standards were verified through annual audits carried out to the flow of ODA funds and analyzed by the Bank’s fiduciary services.

(g) Applied good practice policy conditions (h) Harmonised the program with other cooperating donors The Bank, in the supervision report of June, 2013, noted the need to adjust the PBO disbursement schedules to the PAF’s calendar for dissemination of annual results. It also allowed a certain degree of flexibility in the design of indicators, targets and disbursement conditions that may change as the basis for their design and assessment (e.g. PAF indicators).

b. Borrower performance: Provide observations on the objectivity of the PCR ratings, and if necessary, re-assess the Borrower’s performance throughout the project cycle (design, implementation, completion) by focusing on evidence from the PCR in relation to questions defined in the PCR Guidance Note.

The review concurs that the performance of the borrower is satisfactory (3). The review assessed the project in the context of the PAF Annual Review and concluded that the Government’s performance was consistently assessed quantitatively and qualitatively satisfactory in 2011, 2012 and 2013. However, the assessment provided in the PCR did not provide enough data to properly assess this criteria. Several of the sub-criteria, namely, the quality of the preparation and implementation, the responsiveness to supervision and recommendations, were not assessed, and the review was not able to collect sufficient data to assess them. Counterpart funding was not a requirement for this Program.

Compliance with covenant, agreements and safeguards: Information collected throughout the

review confirm that the GoM was compliant with all legal covenants. No ESMP was

required as this was a category III operation.

Measures taken to establish basis for project sustainability: As stated above, in addition of the

GoM’s measures to improve governance and transparency and its commitment for

reforms, in March 2014 it requested a successor program to the GPSE covering the 2014-

2016 period.

Timeliness of preparing request: Disbursements were made on time as a result of timely

fulfilment of conditions. As stipulated in the July 2013 IPR, the Program implementation

proceeded satisfactorily because all loan conditions had been satisfied on time. The review’s analysis also noted that the level of outputs indicates the commitment of the borrower. The GoM improved the credibility of the government’s mechanisms for budget preparation, approval and execution. The GBS was well aligned with the government’s Budget cycle. Commitments from G19 as well as from government are generally honored. Annual reviews, annual audits were performed every year and annual audit reports delivered to the nominated G19’s Secretary on time. The Mozambican Central Bank (Banco de Moçambique) adhered to internal procedures and the 2010 and 2012 auditor’s opinions were qualified. Limitations were as follows: fiscal years, 2010, 2011 and 2012 the audit reports were not delivered within the deadline. The new policy did not comply with the Memorandum of Understanding (MoU)

22 | P a g e

signed between Government of Mozambique and Programmatic Aid Partners - on 18 March 2009. Further the PCR did not substantiate the basis of its rating adequately. c. Performance of other stakeholders: Provide observations on the objectivity of the PCR ratings, and if necessary, re-assess the other shareholders’ performance throughout the project cycle (design, implementation, completion) by focusing on evidence from the PCR in relation to relevant questions specific to each stakeholder (co-financiers, NGO, contractors and service providers).

The review agrees that the performance of other stakeholders is unsatisfactory (2). The assessment provided in the PCR only examines the functioning of the collaborative agreements, whose performance is below expectation. Other factors such as the timeliness of disbursement by co-financiers, the quality of the policy dialogue with co-financiers, the quality of work by other service providers, as well as the responsiveness to demand are not examined. Of the four missing sub-criteria, the review was able to collect sufficient information to assess the timeliness of disbursement by co-financiers and the quality of the policy dialogue with co-financiers only. However, the assessment of the former sub-criteria presents a timid picture.

Timeliness of disbursements by co-financiers: There is no details data available on the timeliness of disbursement by co-financiers. At completion, the others co-financiers had disbursed 99.3% of their commitments. However, some episodes of stand –off between the GoM and the G19 resulted in some caused some donors to delay their budget support disbursements. As mentioned in the PCR, the GoM expressed dissatisfaction with the increasing number of partners using the variable tranche option for disbursements. The practice is believed to be associated with an increased use of conditionalities attached to budget support.

Functioning of collaborative agreements: The PCR notes that the performance of G-19 co-financiers, including the Bank, is jointly assessed through the G-19 Programmatic Aid Partners (PAPs) PAF, which relies on a set of qualitative measures and quantitative indicators. The same PCR added that although some improvement had been made on in certain indicators, performance was generally below expectations, and even deteriorated in some cases.

Quality of Policy dialogue with co-financiers (for PBO only): The PCR does integrate this sub-

criteria in its assessment of the performance of other stakeholders. As already discussed above, the dialogue with other co-financiers is strong. On the one hand, the Bank actively participated to policy dialogue and the played a leadership role in the context of the Cooperation group; and on the other hand, despite a few stand-off episodes over financial governance and transparency, the relationship between the GoM and donors has improved and cooperation was on-going. All these contributed to reinforcing the quality of the dialogue between the GoM and the G19 (now G14).

23 | P a g e

5. SUMMARY OF OVERALL PROJECT PERFORMANCE

a. Overall assessment: Provide a summary of the project/programme’s overall performance based on the PCR 4 key components (Relevance, Effectiveness, Efficiency and Sustainability). Any difference with the PCR and the reasons that have resulted in them should be mentioned. For cases with insufficient evidence (from the PCR and other documents) available, the evaluator should assign a partly satisfactory rating (to be revised) until a PPAR is complete.

Candidly, the review finds that the program was unsatisfactory. Multiple independent bodies of evidence consulted reveals that there is convergence of facts in the direction that there are challenges with public sector efficiency and transparency as cited in the review's account. Restated, these multiple independent lines of evidence included though not limited to the Africa Economic Outlook Report, World Bank reports (Mozambique Economic Update: Facing hard choices 2016), EITI indicators, PEFA indicators and other sites cross-checked by the review (http://allafrica.com/stories/201703160638.html, and the Africa Research Institute Report (London), http://allafrica.com/stories/201703160638.html) and the African Economic Outlook 2012, OECD, UNDP, UNECA, http://www.afdb.org/fileadmin/uploads/afdb/Documents/Publications/Mozambique Full PDF Country Note.pdf, https://www.oecd.org/daf/inv/investment-policy/IPR-Mozambique-Oct2013-Summary.pdf.

First it is not clear from the PCR’s conclusion to what extent the GPSE practically contributed to change or answered the purpose question in the Appraisal Report. In the short to medium term, the review is not convinced that the PCR clearly demonstrated that there was a more efficient and transparent use of public fiscal resources and a more conducive environment for job creation and skill acquisition, all with a view to promote more inclusive growth and accelerated poverty reduction nor connected indisputably the outcomes with a sustained, inclusive economic growth based on increasing investments and contribute to poverty reduction as a direct result of the GPSE. The PCR did not engage in adequate literature review and content analysis from various MOVs developed in the Appraisal Report to bring about a clear answer to Mozambique’s reeling fiscal and public sector efficiency dilemma. It did not. The fiscal deficit increased at an increasing rate from 6.4% in 2012 to 6.8% 2013. Neither did it tackle the intractable problem it was designed to address for instance what was the poverty incidence by 2013 against the 50% target set nor the gross investment % of the GDP set a target of 23% in 2013 (based on 20% in 2010) and what was the share of the change attributable to GPSE in answering this fundamental development question. Possibly the sharp drop form the project base year 2011 (7.2% of GDP) could be attributed to the wakeup call from the reform

24 | P a g e

announcement effect but this was only a short run gain as shown above from the outlook of the 2012 and 2013 or could have been a result of a combination of multiple macro variables as the GDP marginally increased in the same period of the GPSE’s operation. The review commented the relatively higher output execution (83%), although the PCR overstated that (88%).The review is comfortable with the prevalence of the outputs but rather the disconnectedness of the outputs with the desired changes (TOC probe) and of course the contribution to the ultimate impact. Thus the review finds accountability for outcomes heavy, painful and unacceptable. The GPSE was constrained by the deficiency in the likelihood of the delivery of outcome 2 and could possibly have generated reasonably more catalytic effect and leverage in enhancing public sector efficiency and transparency, strengthening business creation, skill and technology acquisition. Ultimately the DO is rated unsatisfactory. Conventionally, the review noted that this does equate to the programme not being far from instrumental in contributing to some limited extent or in part to the wider goal to achieving sustained, inclusive economic growth based on increasing investments and contribute to poverty reduction. The review raises a contentious and thought provoking issue relating to what solid checks and balances were in place to address the country’s prevalence of governance challenges as the executive exercise strong influence over the legislative and the judiciary and the ramifications of this to the GPSE are magnificent. This is particularly concerning given that the Global Competitiveness Report (2010) revealed that corruption is the second most severe problem affecting the business environment, followed by an inefficient government bureaucracy (12,2%) (World Economic Forum, 2010- 2011). Poverty, inequality and vulnerability remain stubbornly high and a problematic issue. The delivery of outputs and outcomes is summarised below: 1. Relevance ( Satisfactory)

Highly relevant, fully aligned and demand driven Development objective aligned to the CSP, country’s development strategy, sector strategies and

the needs of the beneficiaries, hence fully aligned. Design based on needs and lessons from the preceding two PRSL and therefore highly conducive

to achieving program objectives. Strong synergies and harmonization Clear and explicit focus between the objectives of the program and national priorities

2. Effectiveness in delivering outputs ( Satisfactory) The program achieved its stated results satisfactorily in that: 2.1. All 6 outputs constituting Component 1 scored at least 100% score

a) 117% scored against of total value of budgetary spending using direct execution b) 100% of organs with operational control units at central and provincial levels c) 100% of the budget covered by external audits in accordance with norms d) 100% EITI report to the international EITI secretariat e) 100% Cabinet approval of PPP Law f) 100% Cabinet approval of the revised draft procurement decree, dissemination plan

implemented

2.2. 5 out of 6 outputs comprising component 2 ranged between 75% -100%, (3 scored

100%) g) 100% The decree relative to the one-stop shops for creating enterprises revised in 2012

25 | P a g e

h) 100% Operationalized an electronic single window customs management i) 100% Implemented a first one-stop border Post on the Mozambique – South Africa border j) 75% Approval of revised microfinance regulations effected

m) 80% Approval of the decree creating the National Professional Education Authority effected

n) 0%: Establish an Observatory and Labor Market Information System not effected

2.3 Effectiveness in delivering outcomes (Unsatisfactory) Outcome 1. Enhanced public sector efficiency and transparency: The program development objective was to create the conditions to foster more inclusive economic growth and poverty reduction by enhancing public sector efficiency, skilled human capital development and job creation. Based on the PCR, progress towards achieving had a higher likelihood with a 100% achievement on all three outcome indicators (Deviation between budgeted and actual expenditure, PEFA indicator PI-19 on competition and procurement and PEFA indicator PI-26 on scope, nature and follow-up on external audits). Outcome 2. Strengthened business creation, skill and technology acquisition. The program is expected not to achieve at least one of its outcomes. The review had concern with the fact that based on the PCR, only 1 (EITI country compliance) out of the 8 outcome indicators was on track at the time of the review note. Two outcome indicators were found lagging (Number of people (women) trained in professional skills at 42% (a contradiction and inconsistency was also noted with the June 3rd 2013 Mission report), trading across borders: time to export and import (80%) and the cumulative number of rural recipients (women) of microfinance credit lacked evidence to assess. The outcomes are far out of track and not likely to be achieved without corrective actions. The review judged outcome 2 as unsatisfactory and therefore based on the Project completion reporting and rating criteria 2012, the overall outcome was found unsatisfactory on account of outcome 2 indicators as argued above. 3. Efficiency: The review confirmed a highly satisfactory rating. The rating in the PCR is based on

aspect of the efficiency dimension, specifically that the program scored 4 on timeliness and 3 on implementation progress. The other 2 subcategories of resource use efficiency and environmental safeguards were not applicable. That tends to trump up the efficiency score by naturally reducing this 2 subcategories, a limitation noted.

4. Sustainability: Overall, the sustainability is satisfactory with sufficient mechanisms to perpetuate program benefits. The PCR-EN noted a red flag on ownership and institutional sustainability based on the fiduciary concern of donors though it was eventually cleared. The scope of sustainability measures were moderate rather than robust and critical. Lasting and effective cost recovery measures on the financial component are not clear.

b. Design, implementation and utilization of the M&E (appreciation of the evaluator): Provide an assessment of planned and actual cost of the design, implementation and utilization of the M&E system. Design : To which extent the project M&E system was explicit, adequate and realistic to generate and analyse relevant data ; Implementation : To which extent relevant data was collected – Elements of M&E implementation and effectiveness in the PCR ; Utilization : degree of utilization of data generated for decision-making and resource allocation – elements of M&E utilization in the PCR.

The review confirmed that the baselines provided in the Appraisal Report were tracked and reported in the PCR. The review subscribed that the smart indicators in the RLF in the Appraisal Report are realistic and reasonably challenging based on the complexities of measuring governance and legislative phenomena.

26 | P a g e

An area of deficiency established by the review was constant lack of synergy with 2 key program components which only appear in the Appraisal Report and is not tracked by the PCR. Apparently the 2 components are silent throughout. The PCR did not treat risk and mitigation components as outlined in the Appraisal Report. This is a useful enabler in sustainability enhancement to ensure the unrelenting flow of program benefits. The review, while upholding the project log frame and the monitoring indicators, reasonably beliefs that they were discussed and agreed upon with the executing agency during appraisal and was captured as part of the covenants and a basis for the PIU. The absence of mid-term indicators could be attributed to the duration of the project. The PCR did not explore this. The PCR partially showed the utilisation of the data collected from the M and E pipeline in its account, in particular the Mission report of June 2013 established the current status from mission findings. Its limitation was that it created a ‘vacuum” by not addressing the next action steps either explicitly or implicitly, based on the mission findings.

6. EVALUATION OF KEY LESSONS LEARNED AND RECOMMENDATIONS

a. Lessons learned: Provide a brief description of any agreement/disagreement with all or part of the lessons learned from the PCR after analysis of the project performance with regards to each of the key components of the evaluation (Relevance, Effectiveness, Efficiency, and Sustainability). List the PCR main new and/or reformulated pertinent (and generic) lessons learned for each of these components here. It is recommended that no more than five lessons learned are discussed. Key questions and targeted audience must also be specified for each lesson learned.

The review notes that the two lessons learnt and three recommendations formulated by the PCR were largely design driven. However, they are not strongly linked with the problematic and transformation challenges faced in Mozambique and tended to be loose in this respect as the GPSE sought to address the chronic governance, legislative enforcement commitment and poverty reduction issues. The key lessons are summarised below:

1. Relevance: Reliance on PAF (Performance Assessment Framework) indicators strengthens

donor alignment and coordination (Bank group, Government and development Partners) Effectiveness: PBO’s disbursement conditions and indicators need to be aligned with and adjusted to changes in PAF indicators (Bank group); Align disbursements with the period l in which the government PAF assessment results are made public to avoid disrupting the predictability of flow of funds (Bank group) 2. Efficiency: A reasonable and focused number of outputs is necessary depending on the program context (Bank group) 3. Sustainability: Use of conditionalities and variable tranche versus (expected) increased domestic resources (Bank group, Donors). 4. Harmonization: In line with the Paris Declaration (2005), the review added one lesson arising from the GPSE namely that good harmonisation of programs and its implications on effectiveness, synergy and sustainability. 5. Political will and legislative enforcement commitment: It is key to enforce legislative provisions to enhance true accountability and transparency

b. Recommendations:

27 | P a g e

Provide a brief description of any agreement/ disagreement with all or part of the recommendations from the PCR. List the PCR main new and/or reformulated recommendations (requiring more actions by the Borrower and/or the Bank) here.

The lessons were built into recommendations. However they tend to closely mirror each other and make a little difference. The specific recommendations are as follows: 1. Build and maintain improved capacity to manage for development results 2. Align (and adjust) disbursement conditions and indicators with (to) changes in PAF indicators. 3. Account for outcomes in development reporting as part of evaluation culture

4. Political will, enforcement commitment of higher order legislative provisions to hold the public sector accountable.

7. COMMENTS ON PCR QUALITY AND TIMELINESS The overall PCR rating is based on all or part of the criteria presented in the annexe and other: The quality of the PCR is rated as highly satisfactory (4), satisfactory (3), unsatisfactory (2), and highly unsatisfactory (1). Le timeliness of the PCR is rated as on time (4) or late (1). The participation of the Borrower, co-financier, and the bank’s external office(s) are rated as follows: Very Good (4), Good (3), Fair (2), Poor (1).

The PCR Quality is satisfactory and was submitted timely according to the SNQO PCR Inbox. The PCR was prepared 8 months after the mission (17-25 October 2013) and submitted on the 17th of December 2013. The original closing date of the project was 31 December 2013. Accordingly, the PCR was submitted right on time. The timeliness of the submission of the report will be confirmed with the actual date of submission in the PCR box as requested. The PCR met the reporting requirements in the Staff Guidance on Project Completion Rating and Reporting (August 2012). The PCR attempted to intensively and logically account for the ultimate impact on poverty reduction through an elaborate chain of output indicators that justify outcome level indicators as evidence The Borrower and Bank performance is adequately tracked and reported both being satisfactory.

28 | P a g e

8. SUMMARY OF THE EVALUATION This is a summary of both the PCR and IDEV ratings with justification for deviations/comments. Appropriate section of the PCR Evaluation should be indicated in the last column in order to avoid detailed comments. The evaluator must provide a reasonable explanation for each criterion the PCR rating is not validated by IDEV. Consequently, the overall rating of the project could be “equally satisfactory”.

Criteria PCR PCREN Reason for disagreement/ Comments

RELEVANCE 4 3 Overall the relevance of the project ex-ante and ex-post is relevant.

Relevance of project development objective 4 4 The explicit link of DO with pertinent priorities of the country, policies and the Bank’s vision accompanied by a deficiency in exhaustive beneficiary identification and strong linkages with the country strategy were established as the limitations. The project is relevant to the country’s development priorities. Strong alignment of country strategies with national development priorities to reflect a sound analysis of the country context in the design. Post project relevance was found strong as the need to escalate transparency, accountability and efficiency is imminent to date.

Relevance of project design 3 3 The diagnostics, design and the intended changes of the intervention was reasonably congruent with project purpose. Its inadequacy was the lack of definition of MOVs for component 3 indicators. According to the Doing Business IMF PRSP Progress Report 2014 (p 10), Mozambique was ranked 142 in 2013, the same position it had in 2011 implying no change. The country still far from the best-ranked country in the Southern African Development Community (SADC), in terms of improving the business environment by 2015, and the current ranking puts the achievement of the target at risk with weak enforcement of legislative provisions.

EFFECTIVENESS 3 2 Despite posting a satisfactory output performance, the relatively weaker and huge concern with outcome performance on component 2 hamstrung the likelihood of staying on course to achieve the intended wider objective. Only one outcome component out of 5 was delivered yet the challenge and quality of the outcome are not proportionately more demanding that the single

29 | P a g e

outcome achieved. The review found no intensive and convincing account of the variance compared to higher execution against Outcome component 1.

Development objective (DO) 3 2 The relatively weak outcomes (unsatisfactory) had implications on the DO in overall terms. Consistent with the Project Completion Reporting and Rating 2012 Guidelines, the DO result was unsatisfactory. The probability of outcomes being achieved was found concernedly weak. Multiple evidence converged on this finding, http://allafrica.com/stories/201703160638.html,

https://www.afdb.org/fileadmin/uploads/afdb/Documents/Publications/Mozambique Full PDF Country Note.pdf. https://www.oecd.org/daf/inv/investment-policy/IPR-

Mozambique-Oct2013-Summary.pdf

EFFICIENCY 3 3

Timeliness 4 4 Albeit two tranches being released as planned, the second tranche was released with a minor delay of one month. This effects of the delay cannot be underestimated in a chain of events in a project execution environment unless the PCR can provide evidence to the extent that this had no effect. The delay is a shortcoming.

Resource use efficiency NA NA

Cost-benefit analysis N/A N/A

Implementation progress (IP) 3 3 Two tranches were disbursed within planned schedule as conditions precedent were implemented. One was

delayed and therefore had a shortcoming. The application of the formulae by sub-components is adhered to in the PCR.

SUSTAINABILITY 3 3

Financial sustainability 3 2 The review did not find solid and nascent evidence on financial sustainability mechanisms. For instance, there is no mention of cost recovery and resource mobilisation initiatives, strategic and financial planning, income diversification, sound administration and finance and own income generation options serve for natural resources options highlighted in the PCR. Despite the Decree approving ANEP by Cabinet in 2013, this did not become operational due to lack of funds. As a Decree is merely subordinate legislation, the prospects of financial sustainability based on this is risky.

Institutional sustainability and strengthening of capacities

3 3 Albeit the claim that the operation significantly contributed to strengthening institutional capacities, the review is not convinced that the measures were robust unless proven. In its opinion the review sees challenges in the ability to secure necessary inputs and support; to provide, efficiently and effectively, a continuing stream of activities and outputs as attested by ANEP’s lack of operations.

Ownership and sustainability of partnerships

3 3 Red flags were noted on donor concerns from some of the donors. There is no evidence that this was resolved. Furthermore, the review finds no evidence that all relevant stakeholders were involved.

Environmental and social sustainability N/A N/A This is a category 3 project.

OVERALL PROJECT COMPLETION 3

30 | P a g e

RATING Bank performance: 3 3 The review is satisfied that the Bank fulfilled all the

majority of its requirement (IPR, PCR).

Borrower performance: 3 3 For fiscal years, 2010, 2011 and 2012 the audit reports were not delivered swiftly within deadlines (BTOR p 6). The new policy did not comply with the Memorandum of Understanding (MoU) signed between Government of Mozambique and Programmatic Aid Partners - on 18 March 2009. Reasonably, the achievement rate of outputs is a direct reflection of the performance of the borrower.

Performance of other shareholders: 2 2 Other DPs (Netherlands, Belgium and Spain) have stopped for various reasons raising concerns for the program resource portfolio and the consequential implications are not good on the project.

Overall PCR quality: 3 3 The review found deficiencies in comprehensively wielding together the GPSE story despite the RBLF parameters in the Appraisal Report to fullness, and attempts to account for outcomes via outputs and input indicators though not in-depth. It provides useful lessons. It could be stronger by exploring low execution of outcomes and draw lessons and recommendations to inform project cycle related decisions. It does not provide evidence on SME performance and gender disaggregated data. The PCR management and review section does not show that any of the four managers reviewed the PCR.

9. PRIORITY FOR FUTURE EVALUATIVE WORK: PROJECT FOR PERFORMANCE

EVALUTION REPORT, IMPACT EVALUTION, COUNTRY/SECTOR REVIEWS OR

THEMATIC EVALUATION STUDIES:

- Project is part of a series and suitable for cluster evaluation

- Project is a success story

- High priority for impact evaluation

- Performance evaluation is required to sector/country review

- High priority for thematic or special evaluation studies (Country)

- PPER is required because of incomplete validation rating

Major areas of focus for future evaluation work:

a) Performance evaluation is required for sector/ country review

b) Cluster evaluation (institutional support)

c) Sector evaluation (budgetary support or public finance management reforms)

Follow up action by IDEV: Identify same cluster or sector operations; organize appropriate work or consultation mission to

facilitate a), b) and/or c).

Division Manager clearance Director signing off

31 | P a g e

Appendice 1

PROJECT COMPLETION REPORT EVALUATION NOTE

Validation of PCR performance ratings PCR rating scale:

Score Description 4 Very Good – Fully achieved with no shortcomings 3 Good – Mostly achieved despite a few shortcomings 2 Fair – Partially achieved. Shortcomings and achievements are roughly balanced 1 Poor – very limited achievement with extensive shortcomings

UTS Unable to score/rate NA Non Applicable

Criteria Sub-criteria PCR work score

IDEV review

Reasons for deviation/comments

RELEVANCE Relevance of the project development objective (DO) during implementation

4 4 The review confirmed that the relevance of the GPSE programme’s development objective is highly satisfactory. Not only was the GPSE’s development objective dovetails with but is also clearly stated in the project appraisal report (PAR); but it hinges upon two clear and measurable outcomes, which if achieved together, are likely to deliver the desired impact. In addition, the operation’s development objective is fully aligned with the country’s development priorities, beneficiaries’ needs, and the Bank’s general and country-specific applicable strategies at appraisal and throughout implementation.

Relevance of project design (from approval to completion)

3 3 Design took into account lessons in the past and leverages on the Bank’s experiences in Mozambique and the harmonisation of donors. The review noted that the design was consistently conducive to achieve result. No adjustment was made to the scope of the programme, however, the implementation faced some challenges that required corrective actions in order to ensure the achievement of the intended outcomes and outputs. The theory of change does not come strong in the GPSE design case.

OVERALL RELEVANCE SCORE 4 4

EFFECTIVENESS*

Effectiveness in delivering outcomes

Outcome1: Enhance public sector efficiency and transparency

3

The PCR treats the outcome assessment on the basis of outcome 1 only and excludes Outcome 2 in its analysis. The omission impairs the intention to be systematic and arrive at determining the holistic results of the programme in view of accountability on the ground. All

32 | P a g e

Criteria Sub-criteria PCR work score

IDEV review

Reasons for deviation/comments

indicators for outcome1 have been achieved satisfactorily. Budget execution has consistently exceeded 95% since 2010. Though, no PEFA assessment has been carried out in 2013, a qualitative assessment of the progress made since the 2010 PEFA report (which covered the 2009 budget cycle) suggests that both PI- 19 on competition and procurement and PI-26 on scope, nature and follow-up on external audits indicators have been met. Mozambique declared compliant on 26 October 2012 by the EITI Board, ahead of schedule. Capacity constraints are prevalent in the public sector.

Outcome2 Strengthen business creation, skill and technology acquisition

1

Based on the data used, the prospects of achieving the expected results is constrained by the fact that only one outcome based indicator out of 8 is on track. The review thus found no reasonable basis for a higher probability of achieving this outcome nor this being driven to such a probability by related complimentary outputs. Recent reforms have not had an impact on this indicator, although improvements are expected in 2014. Over 114,000 individuals had benefited from professional training from both public and private training centers. However, the proportion of female trainees (36%) fell well short of the target (60%). The indicator for the reduction of time to import and export was not fully achieved, but is in progress and the effects of recent reforms are expected to be further reflected in 2014

Effectiveness in delivering output

Output1: Deviation between budgeted and actual expenditure (in %)

3 Budget execution has consistently exceeded 95% since 2010. This was met without reported shortcomings and was reported on track in the 3rd June Supervision mission Report. However, end target for Output 1 is overstated as it is below target.

Output2: PEFA indicator PI-19 on competition and procurement

3 Prima facie, the review noted that the PCR reported that the adoption of Decree 15/2010 has introduced various improvements to the existing procurement framework (Decree 54/2005) and capacity building. The last supervision indicated this was positive. According to the Internal Audit Department (IGF), the introduction of internal control units at central and provincial levels increased from 75% in 2009 to 85% in 2010, and 100% in 2011 (accounting for a total of 82 units, 34 at the central level and 48 at the provincial level).There is no evidence on functionality and outcomes of the post introduction era. The PEFA Assessment report of the Public Financial Sector in Mozambique (p 14) states that there are

33 | P a g e

Criteria Sub-criteria PCR work score

IDEV review

Reasons for deviation/comments

also areas where deficiencies have remained since the 2010 assessment, in particular, procurement practices (PI-19) continue to lag behind international good practice, despite the improvements achieved, https://ec.europa.eu/europeaid/sites/devco/files/evaluation-budget-support-mozambique-1330-annex1-2014_en_0.pdf

Output 3: PEFA indicator PI-26 on scope, nature and follow-up on external audits

3 The PCR reported that the 2012 PAF for the Tribunal Administrativo (TA) was met and that the TA has also improved transparency. According to the Internal Audit Department (IGF), the introduction of internal control units at central and provincial levels increased from 75% in 2009 to 85% in 2010, and 100% in 2011 (accounting for a total of 82 units, 34 at the central level and 48 at the provincial level). The review noted that the supervision report noted a positive trend claim. The challenge of the review was to confirm the purported improvements in transparency or related indication to that effect based on the specific evidence adduced which actually is missing. The PCR should have made further investigations into the PEFA report, INTOSAI performance benchmarks to provide relevant evidence. The PEFA Assessment report of the Public Financial Sector in Mozambique (p 19) shows that there were no changes in the external audit and scrutiny function (P1-26,27,28) https://pefa.org/sites/default/files/MZ-Dec15-PFMPR-Public-Meth11-ENG%20with%20PEFA%20Check.pdf

Outcome level 2

Output 4: 4 EITI country compliance

4 The review conducted a follow through analysis of this indicator and confirmed the country status of Mozambique as declared compliant on 26 October 2012 by the EITI Board and is still active on the list https://eiti.org/news/mozambique-declared-eiti-compliant (Page 11 of the EITI 2016 progress report). Supervision Report. The review raised the question on the practicality of the functionality of this on the ground and the requisite political commitment.

Output 5: Time to start a business (DBR)

1 The supervision report reported no progress towards indicator. The time to start a business was 13 days in 2010 and remained static in 2013. There is no reasonable ground for the review to agree with the PCR that it would improve a few months before project exit. According to the Doing Business IMF PRSP Progress Report 2014

34 | P a g e

Criteria Sub-criteria PCR work score

IDEV review

Reasons for deviation/comments

(p 10), Mozambique was ranked 142 in 2013, the same position it had in 2011. https://www.imf.org/external/pubs/ft/scr/2014/cr14147.pd

Output 6 Number of people (women) trained in professional skills

1 The target was 60,000 women trained by 2013.33.6% women were trained by 2012.The review found no evidence from the PCR of the translation of knowledge from training or the number of women owned enterprises and downstream effects, e.g. jobs created and the quality of the job created. The PCR (p6) unlike the supervision report reports 42.7% as the women trained and conflicted with the supervision report 33.6% (p12) for the same year 2012. The deficiency was the exclusion of male counterparts in line with the Bank’s inclusion requirements.

Indicator 7 Trading across borders: time to export and import (DBR)

1 The days actually increased to 23 in 2013 rather than decrease to 20 being the target. This conflicts with the PCR’s statement that: “Operationalization of single windows, etc.) have considerably reduced the time and requirements to import goods into Mozambique”, https://www.imf.org/external/pubs/ft/scr/2014/cr14147.pdf

Cum. number of rural recipients (women) of microfinance credit

UTS Data on this could not be provided at the time of PCR preparation.

Output 1: Percentage of the total value of budgetary spending using direct execution

3 Target was fully met with shortcomings (delays) and exceeded and intensively explored in the PCR. The target was set as 42% and actual result was 117% in 2012.

Percentage of organs with operational control units at central and provincial levels (100% in 2012)

4 The review found the target met fully without shortcomings. The Internal Audit Department (IGF), the introduction of internal control units at central and provincial levels increased from 75% in 2009 to 85% in 2010, and 100% in 2011 (accounting for a total of 82 units, 34 at the central level and 48 at the provincial level).

Percentage of the budget covered by external audits in accordance with norms

4 The target was met without shortcomings. External Audit Institution (Tribunal Administrativo), the proportion of the State Budget covered by its audit verification activities in accordance with INTOSAI increased from 37% in 2010 to 40.6% in 2011 and 40% in 2012. The total number of audits carried out in 2011 reached 600. In parallel, the capacity of the Tribunal Administrativo has been considerably reinforced with donor support.

Output 4: Submit the first EITI report to the international EITI

3 The PCR found no evidence to confirm submission and the aftermath of the submission. The PCR reports that three EITI Reports have

35 | P a g e

Criteria Sub-criteria PCR work score

IDEV review

Reasons for deviation/comments

secretariat been prepared (Jan. 2011, March 2012, Dec. 2012) but does not say whether that was submitted for review, consideration and approval.

Output 5: Cabinet approval of PPP Law

3 The review investigated that the New PPP law 15/2011 was enacted in Aug. 2011; regulation adopted in July 2012. The new law provided different options for PPPs (BOT, BOOT, DBOT etc). However, the review would not determine the translation of the legislative package worked to the benefit of Mozambique.

Output 6: Cabinet approval of the revised draft procurement decree, dissemination plan implemented

3 Met without shortcomings Decree adopted (2010) and significant participation of SMEs in specific types of public procurement-70% of SMEs benefited.

Output 7: Revise the decree relative to the one-stop shops for creating enterprises

3 Met without shortcomings Decree 5/2012, of March 7, on simplified business licensing facilitated speedier licensing. This has been reinforced by the approval of new commercial licensing regulations approved on May 21, 2013

Output 8: Operationalize an electronic single window customs management

3 Met without shortcomings Electronic single window customs is operative for Maputo, Beira and Nacala ports (since April 2012) as well as air freight and land transportation. The central database was made operational in 2011

Output 9: Implement a first one-stop border Post on the Mozambique – South Africa border

4 Met without shortcomings One-stop border Post was confirmed operational in 2012.

Approval of revised microfinance regulations by the Bank of Mozambique

3 Met with delays Regulations approved by December 2012. The review found evidence on financial deepening and inclusionary impact from bank rolling micro-finance operations with the number of having increased from 202 to 285 based on the Mozambique’s Experience With Implementation Of Financial Sector Reforms Report June 2015. Key concerns was absence of data on registered vs active MFIs, quality of loans and assets, compliance with the legislation or at least a snapshot of inclusion and financial sector deepening in Mozambique, https://www.imf.org/external/pubs/ft/scr/2015/cr1512.pdf

Approval of the decree creating the National Professional Education Authority (ANEP) by Cabinet

1 Despite the Decree approved by Cabinet in 2013, this is not operational due to lack of funds. This confirmed the skeptical view of the review about the practical aspects of sustainability raised earlier.

36 | P a g e

Criteria Sub-criteria PCR work score

IDEV review

Reasons for deviation/comments

Output 12: Establish an Observatory and Labor Market Information System

1 Observatory creation had not happened by end-2013. The PCR did not reasonably care for this output as shown by lack of providing explanatory points on non-delivery.

Development objective (DO)

Development objective rating

3 2 This is a consequential outcome of weak

realisation from outcome 2 of the project.

Beneficiaries

Beneficiary 1: SME owners and their employees

3 Satisfactory

Beneficiary 2 Rural businesses and individuals (including women)

3 Satisfactory

Beneficiary 3 Foreign and domestic investors

3 Satisfactory

Beneficiary 4 Taxpayer

3 Satisfactory

Unanticipated outcomes (positive or negative not considered in the project logical

framework) and their level of impact on the project (high, moderate, low)

Institutional development

3 (+)

The PCR did not identify institutional development implications from available information. There are though apparent for instance strengthening of institutions.

Gender

3 (+)

Women SME owners benefited. The percentage cannot be established as the PCR and the Supervision report provided conflicting figures of 42% and 36% respectively. Mozambique has a GII value of 0.591, ranking it 135 out of 155 countries in the 2014 index. This is a significant challenge.

Environment & climate change

N/A

Poverty reduction 3

(+)

The SME are meant to be engines of growth and therefore drive growth, incomes and employment impact.

Private sector development 3

Regional integration 3

(+)

The number of days to do business increased impacting trade, the one stop shop concept on the other hand fast tracks trade with neighbouring SA.

Other (specify)

EFFECTIVENESS OVERALL SCORE 2

EFFICIENCY Timeliness (based on the initial closing date)

4 4

Resource used efficiency NA NA

Cost-benefit analysis NA NA

37 | P a g e

Criteria Sub-criteria PCR work score

IDEV review

Reasons for deviation/comments

Implementation progress (from the IPR)

3 3

Other (specify)

OVERALL EFFICIENCY SCORE 3

SUSTAINABILITY

Financial sustainability

3 2 The review had challenges with sustainability mechanisms in place. For instance the National Professional Education Authority (ANEP) could not go operational due to lack of funds

Institutional sustainability and strengthening of capacities

3 3 This was partially met. The measures are not robust though. The review finds that in addition of the Government’s commitment to reforms and governance agenda, the design of the GPSE and the programme itself contributed to building lasting capacities to plan and implement policy reforms in Mozambique. At completion, the country systems and capacities are deemed sufficient to ensure the continued flow of benefits associated with the GPSE.

Ownership and sustainability of partnerships

3 3 While the review confirmed that the ownership and sustainability of partnerships in the context of the GSPE is satisfactory (3), the programme did not involve all relevant stakeholders. No evidence points in this direction.

Environmental and social sustainability

NA NA

*The rating of the effectiveness component is obtained from the development objective (DO) rating in the latest IPR of the project (see Guidance Note on the IPR). The ratings for outputs and outcomes are determined based on the project’s progress towards realizing its targets, and the overall development objective of the project (DO) is obtained by combining the ratings obtained for outputs and outcomes following the method defined in the IPR Guidance Note. The following method is applied: Highly satisfactory (4), Satisfactory (3), Unsatisfactory (2) and Highly unsatisfactory (1).

Criteria Sub-criteria PCR Work score

IDEV review

Reasons for deviation/comments

BANK PERFORMANCE

Proactive identification and resolution of problems at different stage of the project cycle

3 Issues requiring prompt attention were flagged and actioned.

Use of previous lessons learned from previous operations during design and implementation

3 Based on the PCR and Appraisal lessons were fully taken into consideration in design.

Promotion of stakeholder participation to strengthen ownership

3 Harmonisation and ownership among DPs was imminent.

Enforcement of safeguard and fiduciary requirements

3 This is adequately treated as the PCR pointed the benefits of the cooperation.

38 | P a g e

Disbursement conditions were reasonably met except for lack of account of the third disbursement.

Design and implementation of Monitoring & Evaluation system

3 The M and E system is described with clear baselines, MOVs and tracking of output performance (Supervision Mission Report p 10 and 11). However the current status on Components B, C and partly A remained unaccounted for creating a loophole in the adequacy of data.

Quality of Bank supervision (mix of skills in supervisory teams, etc)

3 The mix of skills was adequate as noted in the Supervision mission composition.

Timeliness of responses to requests

NA

OVERALL BANK PERFORMANCE SCORE 3 3 The review confirmed that the Bank performance is rated satisfactory (3). The Government has expressed its satisfaction with the Bank’s performance under the Program. In March 2014 the Government requested a successor program to the GPSE covering the 2014-2016 period. However, some aspects in the PCR’s assessment of the Bank performance are not discussed. These are the timeliness of responses to requests and lessons learned from previous operations. Although the review was able to collect information on the latter criteria, no information was available to substantiate the former.

BORROWER PERFORMANCE

Quality of preparation and implementation

UTS

Compliance with covenants, agreements and safeguards

3 A qualitative assessment of the application of Decree 54/2010 has resulted in improvements in procurement,

Provision of timely counterpart funding

N/A Not require for PBO.

Responsiveness to supervision recommendations

UTS No information available to form a basis for rating.

Measures taken to establish basis for project sustainability

3 The measures were put in place but not robust to guarantee sustainability. ANEP was constrained by sustainability as outlined earlier.

Timeliness of preparing requests 3

OVERALL BORROWER PERFORMANCE SCORE 3 3 The review confirmed that the Borrower’s performance is rated satisfactory (3). The review arrives at this conclusion by taking into account the assessment provided in the context of the PAF Annual Review. The latter review concluded that the Government’s performance was consistently assessed quantitatively and qualitatively satisfactory in 2011, 2012 and 2013. However, the assessment provided in the PCR did not provide enough data to properly assess this criteria. Several of the sub-criteria, namely, the quality of the preparation and implementation, the

39 | P a g e

responsiveness to supervision and recommendations, were not assessed, and the review was not able to collect sufficient data to assess them. Counterpart funding was not a requirement for this Program.

PERFORMANCE OF OTHER STAKEHOLDERS

Timeliness of disbursements by co-financiers

2 The review found shortcomings.

Functioning of collaborative agreements

2 The standoff between other stakeholders and the GoM as noted in the PCR was a huge concern and limitation.

Quality of policy dialogue with co-financiers (for PBOs only)

3

Quality of work by service providers

NA

Responsiveness to client demands NA

OVERALL PERFORMANCE OF OTHER

STAKEHOLDERS

2 2 The review agreed that the performance of other stakeholders is rated unsatisfactory (2). The assessment provided in the PCR only examines the functioning of the collaborative agreements, whose performance is below expectation. Other factors such as the timeliness of disbursement by co-financiers, the quality of the policy dialogue with co-financiers, the quality of work by other service providers, as well as the responsiveness to demand are not examined. Of the four missing sub-criteria, the review was able to collect sufficient information to assess the timeliness of disbursement by co-financiers and the quality of the policy dialogue with co-financiers only. However, the assessment of the former sub-criteria presents a timid picture.

The overall rating is given: Very Good, Good, Fair and Poor.

(i) Very Good (HS) : 4 (ii) Good (H) : 3 (iii) Fair (US) : 2 (iv) Poor (HUS): 1

DESIGN, IMPLEMENTATION AND UTILIZATION OF MONITORING AND

EVALUATION (M&E)

Criteria Sub-criteria IDEV Score

Comments

M&E DESIGN

M&E system is in place, clear, appropriate and realistic

3

The review found out that there is an M and E in place and partially described. Its inadequacy was the lack of definition of MOVs for component 3 indicators. This comprised the RLF in the Appraisal report, evidence in form of Quarterly reports, BTORs and supervision mission reports. Various means of verification cited included BoM

40 | P a g e

Criteria Sub-criteria IDEV Score

Comments

statistical reports, PEFA Final reports. However, sector specific indicators for instance on microfinance are not highlighted amongst others.

Monitoring indicators and monitoring plan were duly approved

3

The review found out that the number of indicators in the PCR were consistent with those in the RLF/Appraisal report. This possibly explains a higher output execution ratio over and above commitment.

Existence of disaggregated gender indicator

2

Satisfactory though there is no mention or an attempt to provide disaggregated data for the male counterpart nor on gender disaggregated data on SME owners.

Baseline data were available or collected during the design

3

Baselines provided in the Appraisal Report are followed through the PCR. Evidence was provided by the Supervision Mission Report. Other forms of MOV were not exhausted in data collection and reporting resulting in methodological limitations. The list of interviewers met during the PCR mission was found annexed. It would have value added to hear what their responses were first hand.

Other, specify

OVERALL M&E DESIGN SCORE 3

M&E IMPLEMENTA-TION

The M&E function is adequately equipped and staffed

3

The review established that the 2 missions comprised diverse skills namely economists, governance and private sector specialists, finance and procurement specialists.

OVERALL M&E IMPLEMENTATION SCORE 3

M&E UTILIZATION

The borrower used the tracking information for decision

3

The review was disturbed by the quality and value for money from the Supervision Mission Report of November 5th -9th 2012. A concern was recorded on conflicting data on women training output. Further reference in the PCR to the MOVs in the Appraisal Report is mentioned to a very limited extent, eg PEFA Final report and Directives from the Presidium. The supervision missions identified and flagged critical items requiring special attention (June Mission Report p 8) and demonstrated outputs tracking. The Supervision mission June 3rd-6th was relatively more robust in tracking indicators and progress. In its analysis, the review observed that a matching of timelines for most indicators ad targets was consistent (Appraisal Report, PCR, BTOR 2013).

OVERALL M&E UTILIZATION SCORE 3 The M and E system was partially utilised.

OVERALL M&E PERFORMANCE SCORE 3

41 | P a g e

PCR QUALITY EVALUATION

Criteria PCR-EVN

(1-4) Comments

QUALITY OF PCR

1. Extent of quality and completeness of the PCR evidence and analysis to substantiate the ratings of the various sections

3 The quality of evidence to substantiate the ratings in the PCR was found challenging. The components are tracked and presented systematically despite the indicators being well captured comprehensively. The outcomes in the analysis table p 5 are not stated whilst the indicators are. First the supervision mission report of November the 5th of 2012 does not proffer utility to project decision making on the ground. Secondly, while the second supervision report provided value-add from the mission, there is little reference to the vast literature and documents (PEFA Final Report, BoM Statistical Reports, EITI reports, BOM directives) and other MOV in the Appraisal Report as evidential basis to strengthen the determinations of the PCR. Thirdly, there were consistent inconsistencies in gender reporting on the exact beneficiation of women. Despite these limitations, the PCR attempted to intensively and logically account for the ultimate impact on poverty reduction through an elaborate chain of output indicators that justify outcome level indicators.

2. Extent of objectivity of PCR assessment score 3 The assessment was overall objective. However some challenges for instance, the review questioned whether the PCR’s account of outputs delivered being reported erroneously as 88% in the IPR was an error or a bias as the PCR recalibrated off the calculable outputs delivered against a base of 8 instead of 12. End target for Output 1 is overstated as it is below target. A shortcoming was noted in relation to timeliness of disbursements. Strictly, the shortcoming is a limitation and a delay was incurred with consequential implications on the project. The assessment of the DO was not objectively treated and demonstrated bias based on Staff Guide 2012 for Project Completion Reporting and Rating. Some output ratings were overrated and rebutted by independent evidence for instance P1-19 and P1-26 scores.

3. Extent of internal consistency of PCR assessment ratings; inaccuracies; inconsistencies; (in various sections; between text and ratings; consistency of overall rating with individual component ratings)

3 Overall, reading together text with evidence, the review found that there are incidences of inaccuracies and inconsistencies. The first one was identified in the gender aspects of the training output for women. The PCR states that: “The number of people trained was only 42, 7% by end of 2012” On the other hand the supervision report of 6th of June 2013 states that: “33.6% women were trained in 2012”. Other inconsistencies related to outcome 1 indicator in the IPR and mixed lessons and

42 | P a g e

recommendations. PCR Project data converged while loan data, specifically, effective first date of disbursement diverged from the EVRD.

4. Extent of identification and assessment of key factors (internal and exogenous) and unintended effects (positive or negative) affecting design and implementation

2 Whilst the PCR partially mentioned that the standoff between Government and donors posed a medium negative impact, it did not explore the institutional developmental issues, private sector implications, value chain issues arising from SMEs, the value added by the SMEs in terms of jobs created and the quality of those jobs and the income effect in relation income poverty or other multidimensional forms of poverty hinged on incomes. The appraisal report details risks and mitigation measures which the PCR did not treat adequately but rather sharply disconnected with the Appraisal Report and posed sustainability challenges.

5. Adequacy of treatment of safeguards, fiduciary issues, and alignment and harmonization

3 Audit reports and compliance with INTOSAI proportionately improved from 5 to 600 2011.

6. Extent of soundness of data generating and analysis process (including rates of returns) in support of PCR assessment

1 Based on data, the review established this was not strong enough as the data from the evidence availed was limited as argued earlier.

7. Overall adequacy of the accessible evidence (from PCR including annexure and other data provided)

3 The literature reviewed- PCR, IPR, Appraisal Report, Country Strategy and Bank CSP, Annexure 2. Supervision and BTOR reports demonstrated the extent to which outputs were tracked although the first supervision report had huge limitations in its quality. Other limitations were sector specific evidence not being reflected at a snapshot level for instance the level of inclusion of MFIs, SACCOs, credit risk and compliance with the MFI regulatory and non-prudential supervision framework.

8. Extent to which lessons learned (and recommendations) are clear and based on the PCR assessment (evidence & analysis)

1 The review noted that the two lessons and three recommendations are properly formulated. This is fewer than the requirements provide in the PCR Guidelines and more pertinent development effectiveness enhancing lessons and recommendations could have been proffered in view of the Bank Group’s vision. Lessons and the recommendations were mixed e.g. page 15 of the PCR presents which are in actual fact are recommendations. In addition, the formulation of lessons and recommendations did not address deficiency in outcomes and propose measures to strengthen outcomes and output execution to drive the desired changes in the public sector and contribute to the achievement of nationally owned objectives. This is missing in the PCR’s story of public sector reforms in Mozambique. No legislative related lessons and recommendations as well as sustainability/risk mitigation were highlighted despite such lessons and recommendations being a game changer in the ownership and commitment of public sector efficiency and transparency initiatives. The lessons and recommendations could also have derived significant value from observing the 4 primary questions prescribed by the Staff Guidance Note on Project Completion and Reporting (Aug 2012, p 18). In this respect, the lessons and recommendations eluded confronting strategic aspects of continued financing, maintenance and institutional, technical and political risks to the project unless there is evidence in rebut to this finding of the

43 | P a g e

review.

9. Extent of overall clarity and completeness of the PCR

2 The review found a pertinent challenge in identifying a crisp, solid conclusion and position from the PCR regarding the contribution and translation onto the ground of functionality of the GPSE in answering the purpose questioned and overarching goal as outlined in the Appraisal Report and addressing the practical problematic public sector efficiency and transparency issues. Fundamentally, in its humble opinion, the review has a pragmatic concern with the effectiveness, ownership and commitment and sustainability of the GPSE and how this is treated in the PCR.

Other (specify)

PCR QUALITY SCORE 3 The PCR quality is satisfactory.

PCR compliance with guidelines (PCR/OM; IDEV)

1. PCR Timeliness (On time = 4; Late= 1) 4 The PCR was submitted on time on the 31st of December 2013, being the original closing date of the project. This Confirmed with the SNQO PCR Inbox.

2. Extent of participation of borrower, Co-financiers & field offices in PCR preparation

NA

3. Other aspect(s) (specify)

PCR COMPLIANCE SCORE 3 The PCR adhered to the reporting format and guidelines of the Staff Guidance Note on Project Completion and Reporting (Aug 2012).

*** rated as Very Good (4), or Good (3), or Fair (2), or Poor (1)

Acronyms

ANEP-National Professional Education Authority GoM-Government of Mozambique ODA-Official Development Assistance PAF-Performance Assessment Framework PBO-Policy Based Operation

References

1) PCR, 17-12-2013 2) IPR,17-07-2013 3) Appraisal Report 4) Letter of Development Policy for the GPSE, 17 June 2011 5) Supervision Missions (June 2013,Nov 2012) 6) Country Strategy Paper 2011-2015 7) Country Development Plans 8) Staff Guide on Project Completion Reporting and Rating 2012

44 | P a g e

Lines of Evidence

1) https://ec.europa.eu/europeaid/sites/devco/files/evaluation-budget-support-mozambique-

1330-annex1-2014_en_0.pdf 2) https://pefa.org/sites/default/files/MZ-Dec15-PFMPR-Public-Meth11-

ENG%20with%20PEFA%20Check.pdf 3) https://eiti.org/news/mozambique-declared-eiti-compliant 4) https://www.google.com/search?q=FSDS+2013-2022+strategy+Mozambique&ie=utf-

8&oe=utf-8&client=firefox-b#q=Mozambique+Cabinet+approval+of+PPP+Law+2011

5) https://www.mcnet.co.mz/procedures.aspx?lang=en-US&chapter=1&subchapter=1

6) Page 10 of the EITI 2016 progress report 7) https://eiti.org/sites/default/files/migrated_files/progressreport.pdf 8) https://www.afdb.org/fileadmin/uploads/afdb/Documents/Publications/Mozambique Full

PDF Country Note.pdf. 9) https://ec.europa.eu/europeaid/sites/devco/files/evaluation-budget-support-mozambique-

1330-annex1-2014_en_0.pdf. 10) https://www.oecd.org/dac/peer-

reviews/World%20bank%202004%2010_Steps_to_a_Results_Based_ME_System.pdf. 11) http://allafrica.com/stories/201703160638.html, 12) http://allafrica.com/stories/201703160638.html 13) https://www.oecd.org/dac/peer-

reviews/World%20bank%202004%2010_Steps_to_a_Results_Based_ME_System.pdf