Bank Netherlands Partnership Program (BNPP)HD Learning WeekNovember 13, 2008

Arianna Legovini, Lead Impact Evaluation SpecialistEdit V. Velenyi, MIEP Coordinator

Africa Impact Evaluation Initiative (AIM)Malaria Impact Evaluation Program (MIEP)

“The impact of malaria extends far beyond the realm of public health and exacts a heavy toll on human and economic prosperity...”

Malaria kills an estimated 1 million people every year

Costs Africa US$12 billion annually in lost GDP 90 percent of malaria deaths worldwide are in

Africa Yet malaria is both preventable and treatable Universal coverage would bring the scourge to

a halt

Strong epidemiological experimental evidence: prevention and treatment work

Lacking behavioral and economic evidence What will induce people to have their children sleep

under a net? Or take the child for treatment within 24 hours?

What are the socio-economic returns to malaria prevention and treatment?▪ Labor participation and productivity, child cognitive

development

Which interventions are most cost-effective?▪ Private delivery mechanisms▪ School based malaria interventions

AIM - Africa Impact Evaluation Initiative 80+ Experimental or Quasi Experimental Evaluations

MIEP - Malaria Impact Evaluation Program Inaugural Workshop, Cape Town May 2007 Second Workshop Asmara Feb 2008 Countries: Eritrea, DRC, Kenya, India, Nigeria, Senegal, Zambia Areas of investigation

▪ Alternative delivery mechanisms and PPP▪ Complementary interventions (IRS, test kits)▪ Preventive treatment (IPTs)▪ Innovative financing for ACTs

Improve quality of operations Separate process from quality of intervention Test alternatives and inform design in real

time Increase project effectiveness Answer the “so what” questions

Build relationship with client Come with options not solutions Find out together what works best Assist client adopt better way of doing

business and taking decisions

Quality assurance Client-driven & operationally relevant Feedback loop into policy design

Lead researcher & Field Coordinator

MIEP ResearchTeam

MBP ProjectTeam

Client Government

Coordinating UnitIE Leads

Sector LeadProgram Coordinator

Team Support

Technical Advisory GroupResearchers & Specialists

Working Groups

BiometricsBiometrics

CognitiveCognitive

Socio-EconomicSocio-EconomicKnowledge, Attitudes and Practice

Knowledge, Attitudes and Practice

Cost-EffectivenessCost-Effectiveness

Alternative delivery mechanisms and PPP Complementary interventions (IRS, test kits) Preventive treatment (IPTs) Innovative financing for ACTs

MIEP Portfolio BNPP Recipients

• DRC• Nigeria

MIEP Under- 5 mortality

GNI/capita PopulationMBP

Commitment

SSA Portfolio

Number of Fever Cases Reported*

Number of Confirmed

Malaria Cases Reported

As % of population

Malaria mortality as %

of total mortality

Absent / Rarely Epidemic

Marginal / Epidemic Endemic (US$)

(Phase I - US$ million)

DRC 75,957,396 23,619,960 38.9% 16.9% 9% 6% 85% $140 60,643,890 30Eritrea 166,163 18,964 0.4% 13.6% 12% 72% 16% $230 4,692,115 2Kenya 30,192,924 11,341,750 31.0% 13.6% 53% 26% 21% $680 36,553,490 4Nigeria 155,556,109 57,506,430 39.7% 24.1% 0% 1% 99% $930 144,719,953 180Senegal 4,471,534 1,456,336 12.1% 27.6% 0% 3% 97% $820 12,072,479 5Zambia 10,209,192 3,655,203 31.3% 19.4% 5% 12% 83% $800 11,696,160 28TOTAL 276,553,318 97,598,643 NA NA NA NA NA NA 270,378,087 249

Population at Risk (%)Morbidity

9%

6%

85%(5.4m)

12%

72%

16%

53%

26%

21%

1%

99%(122.5m)

0%3%

97%

5%

12%

83%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

DRC Eritrea Kenya Nigeria Senegal Zambia

Population at Risk by Malaria Epidemiology (%)

Absent / Rarely Epidemic Marginal / Epidemic Endemic

Number one public health problem in Nigeria 99% of the population in endemic areas

(MARA/ARMA 2002) High Federal and States’ Governments

commitment to controlling the disease 2004 National Economic Empowerment and

Development Strategy▪ development of strategic plans for malaria

Presidential Initiative for Accelerated Achievement of the MDGs ▪ malaria control as a key component of health-related MDGs

2004 National Health Policy ▪ malaria control as a priority health program

Nigeria Total Health Expenditures 2003-2004 (NLSS 2004, CAS 2005)

$2.34 , 7%$3.42 , 10%

$2.45 , 7%

$1.22 , 4%$22.55 , 68%

$0.24 , 1%

$0.83 , 3%

Federal State Local

Donors Private Organizations Private Insurance

Private Out-of-Pockket

Explore innovative modes of delivery and complementary interventions to address limitations of the health system Weak / insufficient public sector delivery Human resource constraints Heterogeneity in the quality of private

sector delivery Insufficient coordination between the public

and private sectors

Consultations and Capacity Building – National and States Cape Town Inaugural Workshop: May 2007 Abuja Design Workshops

▪ MBP7: August 2007 ▪ MBP 4: September 2008

State Field Visits▪ Anambra: October 2008▪ Gombe: October 2008▪ Akwa Ibom: December 2008▪ Jigawa: December 2008

Nigeria Malaria IE Cycle and Timeline

STATE REGIONPROGRAM FUND ($mil)

AREA (Sq.Km) POPULATION

POPULATION DENSITY

NUMBER OF LGAs CWIQ SAMPLE

AKWA IBOM SOUTH 8.5 7,081 3,722,605 526 31 3,100ANAMBRA SOUTH 9.5 4,844 4,320,963 892 21 2,100BAUCHI NORTH 9.75 64,605 4,422,336 68 20 2,000GOMBE NORTH 6 5,530 2,301,044 416 11 1,100JIGAWA NORTH 9.75 23,154 4,443,452 192 27 2,700KANO NORTH 17.5 20,131 8,978,865 446 44 4,400RIVERS SOUTH 10.5 21,850 4,926,440 225 23 2,300

TOTAL NA 71.5 147,195 33,115,705 395 177 17,700SOUTH NA 28.5 33,775 12,970,008 548 75 7,500NORTH NA 43 113,420 20,145,697 281 102 10,200

ANAMBRA

AKWA IBOM

GOMBE

JIGAWA

1. FMOH / NMCP

Program Coordinator

2. STATE MOH

Program Managers

3. LGAMBP Focal Person

WardsWDC

WardsWDC

WardsWDC

Vulnerable Target Group: U5 Children & Pregnant Women

Public Facility (K) Private – PMV T1 Community – RMM T2

ACT Supply

STEWARDSHIPNational Policy Making

Resource AllocationSystem Harmonization

State-Level Policy Making

Implementation Decision

Supply

M&E

Program Implementation

Service Delivery & Reporting

IE

EvidenceBased Policy

Making

(LLIN – BCC)

Counterfactual: Public Sector (K) Private Sector Intervention Arm: PMV (T1)

What is the added value of using trained Patent Medicine Vendors (PMVs) to increase timely treatment of uncomplicated malaria with ACTs?

Community-Based Intervention Arm: RMM (T2) What is the added value of using trained Role Model

Mothers (RMMs) to increase▪ increase timely treatment of uncomplicated malaria with ACTs?▪ effective coverage of LLINs by the vulnerable target groups?

Cross-Over / Factorial Design (T1 + T2) What is the joint effect of using trained PMVs and

RMMs …?

Uniform Intervention Across States Supply Side: Training

Variations Supply Side: Incentives for PMVs and RMMs

▪ Monetary v. Non-Monetary Demand Side Approach: Communication

▪ BCC Component in Training for RMMs to Improve Use LLINs

▪ BCC (Community-Based Orientation)

Effect of Trained PMVs and RMMs on coverage and use of ACTs Target Population - U5 Children: 470,770 MB Program Funds: 6 million Design: Covers whole state, including all 11 LGAs and 114 Wards Intervention Arms: 1) PMV 2) RMM 3) PMV + RMM 4) Control Intensity: 114 Trained PMVs and 114 Trained RMMs HH Survey: 2,280 HH | Community Survey: 228 | Facility Survey | MIS

PMV in 57 Wards

PMV in 57 WardsRMM in 57 Communities Pure Control in 57 Wards

RMM in 57 Communities

Effect of Trained PMVs & RMMs on coverage and use of ACTs, and RRMs’ on LLIN use

Target Population: U5 (836,400) and Pregnant Women (209,100) MB Program Funds: 9.5 million Design for State (21 LGAs and 329 Wards) Intervention Arms: 1) PMV 2) RMM 3) PMV + RMM 4) Control Intensity: 172 Trained PMVs and 172 Trained RMMs (2 per Ward) HH Survey: 2,064 HH | Community Survey: 172 | Facility Survey | MIS

PMV in 43 Wards

PMV + RMM in 43 Wards Pure Control of 43 Wards

RMM in 43 Wards

Effect of incentives for PMVs & RMMs on coverage and use of ACT, and RRMs’ on LLIN use.

Effect of BCC on coverage and use of ACTs, and RMMs on LLIN use. Target Population: U5 (784,042) Pregnant Women (196,010) MB Program Funds: 8.5 million Design for State (31 LGAs and 329 Wards) Arms: 1) Incentives 2) BCC 3) Control 329 Trained PMVs and 329 Trained RMMs (Using 1 each for Focal

Village per Ward) HH Survey: 3,290 HH | Community Survey: 329 | Facility Survey | MIS

Incentives110 Focal Villages(1 FV per Ward)

Control (Training Only)109 Villages

(1FV per Ward)

BCC110 Focal Villages(1 FV per Ward)

Upgrade Routine MIS: Case Management Reporting Training and Rollout of Case Management Tool to Health Facilities

Management Tool (McKinsey) Costing Tool

Cost of Standard Delivery Cost of Innovative Arms (Incremental Costs)

▪ System Start Up (Intervention Specific M&E Tool)

▪ Intervention Cost (Commodity and Training)

▪ Management Cost (Operating the New System)

Third Party Audit Focal Person and State Manager Contracting of NGO/CSO

Routine Reporting Upgraded MIS Customized MIS Tools Random Audits

Household Survey Baseline CWIQ (November - December 2008)

▪ Developed Malaria Section for CWIQ Follow up Dedicated Survey (November – December 2009)

Community Surveys Component of HH Survey

Facility Surveys SFH Facility Census (November 2008) Facility Survey (November 2009)