Evaluation of sector programmes and budget support operations in the context of EU development cooperation

1st M&E Network Forum07 to 08 November 2011

Manila

Outline

Background Evaluation in the context of EU Development

Cooperation - General Principles

EU Health Sector Programme and Support in the Philippines Overview

Evaluation Methodology

Outlook Evaluation of Budget Support

Evaluation – General Principles

Definition: An evaluation is a judgement of interventions according to their results, impacts and needs they aim to satisfy - European Commission, 2000

Role: Lesson learning

Improving quality and the design of future interventions

Accountability

Ex-ante, mid-term, ex-post Independent, external, balanced Identifying performance-related questions,

information needs and indicators

Evaluation – General Principles

INPUT/PROCESS(supply driven)

INPUT/PROCESS(supply driven)

OUTPUT(supply driven)

OUTPUT(supply driven)

OUTCOME/RESULT

OUTCOME/RESULT

IMPACT(demand driven)

IMPACT(demand driven) Infection ratesInfection rates

Vaccination ratesVaccination rates

Vaccines providedVaccines provided

Funds Budgeted/spent,

laws passed

Funds Budgeted/spent,

laws passed

EXAMPLEEXAMPLE

Wider effectsConsequences of outcomes

Wider effectsConsequences of outcomes

Results at the level of beneficiaries(supply meets

demand)

Results at the level of beneficiaries(supply meets

demand)

Concrete consequences of

resources or measures taken

Concrete consequences of

resources or measures taken

Money or measures taken

Money or measures taken

IDEAIDEA

Focus

Relevance

Evaluation – General Principles

Effectiveness, sustainabilityimpact

Efficiency

Inputs

NeedsChallenges

Objectives Outputs

Impacts

Outcomes

Coherence Effectiveness, sustainabilityimpact, EC value added

EU Health Sector Support

Health as focal sector of EU support Assistance to Health is core activity of Delegation

3 ongoing programmes

1.1.Health Sector Policy Support Programme IHealth Sector Policy Support Programme I (2006 -2011; € 33 M)(2006 -2011; € 33 M) In support of FOURmulaOne for HealthIn support of FOURmulaOne for Health

Support to central/national level (DoH, DBM) & Support to central/national level (DoH, DBM) & provincial/local level (16 convergence provinces)provincial/local level (16 convergence provinces)

Sector Budget Support, WB trust fund & EU TA for Sector Budget Support, WB trust fund & EU TA for DoH and provincesDoH and provinces

EU Health Sector Support

2. Mindanao Health Sector Policy Support Programme (2007-2012; € 12 M)

Support to 13 out of 21 roll-out provinces

WB trust fund & EU TA for LGUs

3. Health Sector Policy Support Programme HSPSP II (2011-2014; € 36 m)

In support of Aquino Health Agenda to achieve Universal Health Care

Sector Budget Support

2 more in the pipeline Addressing Maternal Neonatal Child Health and Nutrition

(MNCHN) Needs of ICCs/IPs in Mindanao (€ 7 m)

HSPSP III (€ 35 m)

EvaluationEU Health Sector Support

Objective to provide the DoH, the European Commission

and other stakeholders with sufficient information to make an informed judgment about the past performance of the programme, particularly its contribution to the implementation and performance of the Health Sector Reforms with particular attention to performance against the established targets and indicators, and to provide lessons learnt and practical recommendations for ongoing and future interventions.

Ex-post Ex-ante

EvaluationEU Health Sector Support

Focus on HSPSP I

But also take into account MHSPSP and HSPSP II

External evaluation team

Evaluation criteria based on OECD DAC criteria

Adherence to aid effectiveness principles Partnership approach under government leadership

As much as possible jointly with other development partners within the Joint Assessment and Planning Initiative (JAPI)

Use of country systems

EvaluationEU Health Sector Support

Key issues to be looked at:

Achievements in Health Sector Reforms Sector perspective at national as well as LGU

level

Contribution of EU support Technical component

Financial component

Role and value added of technical assistance

Economic Growth

Social Development

Poverty Reduction

OutputClusters

Results(Outcomes)

Specific Impacts

IntermediateImpacts

Global Impacts

Service Delivery Reforms

Regulatory Reforms

Governance

Health Financing

Reform

MDG 1

Improved Health Insurance coverage

and financial protection

Improved financial management and

procurement

Improved planning, monitoring and

reporting

Access to and utilisation of

essential and basic

health packages

Better and sustained financing for health

Improve the performance of the

health system

Reduction in Mortality

Reduction in Morbidity & Disability

MDG 4

MDG 5

MDG 6

Quality and affordability of

health goods and services

MDG 8

Enhanced Service and Drugs

Availability & Quality

Evaluation of Budget Support

EC has developed a new methodology

Role The results of an evaluation are expected to

feed into (i) the implementation of future budget support operations; (ii) possible changes in policy, approach, and/or methods; and (iii) to support decisions on the circumstances when/where budget support is an appropriate and relevant instrument.

Basis for disbursement of budget support

Evaluation of Budget Support

Results are difficult to evaluate

Very heavy, time-consuming and costly exercise

2 main categories of effects: “flow-of-funds”

policy or institutional effects

Emphasis on access to services and use of quality services

Evaluation of Budget SupportInput

Transfer of funds

TC/Instit. support

Policy dialogue

Conditions/Indicators

Domestic resources

Other donor projects

Induced output

Fiscal discipline

PFMimproved

Policy design process

improved

Increased quantity and

quality of public services

Strategic allocations of

resources

Impact

Sustainable growth

Poverty reduction

Mortality reduction

Literacy

Social inclusion

Other objectives acc. to operations

covered

Outcome

Business climate

Private investment

Beneficiaries’ response, satisfaction and demand

Domestic accountability

Direct output

Budget level and structure

Aid predictability

AlignmentTC/instit.

support with Gov.

strategies

Effects of other

domestic resources

Other donor projects

STEP 3

STEP 1 STEP 2

Questions

Role and involvement of NEDA and other gov agencies, civil society, other development partners in evaluation?

Link with other development partners assistance?

How to strengthen policy dialogue? Moving towards a sector reform contract in

the future

Salamat Po!Thank you!