1

Zimbabwe Results Based Financing

ProjectEvidence and Lessons

Brigadier General Dr. Gwinji, MOHCCDr. Goverwa -Sibanda, MOHCC

• Key messages, strategic context and overview of RBF program• Brigadier General Dr. Gwinji, MOHCC

• Preliminary results from impact evaluation and process evaluation• Dr. Sibanda, MOHCC

Outline of Presentation

• RBF is a health systems management tool intended to improve the efficiency of utilization of system inputs

• RBF in Zimbabwe piloted in 2 front runner districts in July 2011, then scaled up to 16 additional districts in March 2012

• Contextual background: Dramatic improvements in MCH indicators witnessed throughout the country (MICS 2014)

• Yet faster rates of improvement in RBF districts for key indicators• 13 percentage point improvement in the in-facility delivery rate• 12 percentage point improvement in post-natal care coverage• Significant improvement in the quality of ANC services

• Not all indicators show relative improvement under RBF• No differential gain in coverage of ANC services• Small gain in use of modern contraceptives

Main Takeaway Messages I

• Improvements in part due to• Team based incentives facilitating teamwork• Regular and structured supervision visits yielding feedback to improve

performance• Enhanced community participation

• Challenges to be addressed include• Facilities in remote areas or with small catchment populations• Capacity to fully operationalize the quality of care components• Demand side barriers related to religious and socio-economic factors

Main Takeaway Messages II

• Population: 13 million (2002 Census estimates) • Decline in public sector financing • Effect on management• supportive supervision

• Increase in household out-of-pocket expenditures due to various forms of user-fees

• Decline in outcomes & slow progress on some key health MDGs [MDGs 4 & 5]

Country & Project Context During Design Phase [2010-2011]

RBF aligned with and supports National Health Strategy and policy–equity in access to health services;◦ User fee removal (package of high impact services)◦ Rebuild the quality of care standards◦ Increase access to priority maternal, family planning and child

health services◦ Strengthen the referral system (promotes appropriate care

seeking at appropriate levels)◦ Decentralized service delivery and revitalized primary health care

Prioritized package of services directly linked to burden of disease for mothers, newborns and children under 5

RBF used to operationalize GoZ Results-Based Management Strategy and Results-Based Budgeting Pilot

RBF in the Zimbabwe Context

• Fee-for-services : for both quality and quantity – partial replacement of user fees

• Functions separated: purchaser, provider, regulator & external verifier

• Key role for community –Health Center Committees

1. Results-Based Contracting

• Strengthening planning and RBF management capacity: RBF national management team

• Purchasing, verification, strategic management

2. Management and Capacity Building

• Capture effect on health outcomes and various aspects of the health system

• Accountability through community tracer surveys (CBOs)

3. Monitoring and Documentation

Three Project Components

Rural Health Centers

District Hospital

Package of RBF Services

9. Tetanus TT2+10. ARVs to HIV+ preg. Women (PMTCT)11. Family planning short and long term

methods12. High risk perinatal referrals13. Vitamin A supplementation14. Children fully immunized15. Growth monitoring, children < 5yrs16. Cure discharged acute malnutrition

children < 5yrs (October 2012)

1. OPD new consultations2. First ANC visit during the first 16 weeks

of pregnancy (October 2012)3. Ante natal care 4 visits completed4. Post natal care 2 or more 5. Normal deliveries6. HIV VCT in ANC7. Syphilis RPR test8. IPT (x2 doses)

1. Normal deliveries in district hospital2. Deliveries with complications (caesareans excluded) and post partum

complications3. Caesareans performed4. Family planning: Tuba Ligations5. Counter referral note arrives at RHC (October 2012)

9

Data Quality and Verification

Verified data from health facility registers collected by Local Purchasing Units (LPUs) and entered in the RBF database is utilized for payments

Data flow integrated within national HMIS, no parallel system used

Quantity verification by the LPU undertaken every month

Quality verification by the DHE (for RHCs) and by PHE (for DH) undertaken every quarter

Client satisfaction performed by the CBOs every month and by the counter-verifier every quarter

04/12/2023

HMIS - DHIS 2

District Facility

Programme Database

HFOs

≈

Routine Reporting - T-SeriesEntry and Submission into DHIS2

Data Accessible in Programme

System:Same

Platform RBF Indicators Downloaded

Verified Data Entered Into System

Payment to Facil

ityUpdating

HMIS with Verified Data Verificati

on

Interaction between Programme Data and HMIS

04/12/2023

RBF Financing & Coverage

11

RBF Funding DFID and Government of Norway (US$35m) Ministry of Finance: US$5 million per year from 2014 US$ 28 million disbursed (including $5 million Government

counterpart funding) Population Coverage : 4,1 million Geographic coverage : 18 rural and 2 low-income

urban and peri-urban districts (Harare and Bulawayo)

July, 2011 to October, 2015

04/12/2023

Contract

Tracing clients and client satisfaction

Policy and Supervision

Policy and Supervision

Policy and Supervision

MoHCW

Provincial Health Executive

Health Facilities and HCC (415)

District Health Executive

National Steering Committee

District Steering Committee

CORDAID

Private Purchasing Agency (NPA)

Contract + Payment

Payment

Community Based OrganisationsClients

CORDAID

Local Purchasing Unit

Payment

Contract + Verification

Contract

External verification12

Governance & Institutional Arrangements

Impact and Process Evaluations: Answering Policy Relevant Questions

13

Baseline(2011)

Midline Impact Evaluation(Mar-Sept 2014)

Program Inception

Endline IE(TBD)

Process Monitoring and Evaluation (PME)(November 2013)

Routine Performance Review (Quarterly) – Operational Data

Technical Review(June 2012)

Mid-Term Review(January 2013)

Technical Adjustments: Prices and Services

Technical Modifications –clinical quality, streamlining verification, equity monitoring

2nd PME Round Planned for October 2014

14

The IE seeks to determine the causal impact of RBF on priority service utilization and related health indicators

◦ Treatment: Facilities and patients residing in districts that introduce the RBF program

◦ Comparison: Facilities and households in matched “business as usual” districts

◦ Districts matched on various characteristics including: Average catchment size of facility Proportion of staff positions filled Population rates over 2008 – 2010 of ANC coverage, in-facility

delivery rates, immunization coverage

Evaluation Design I

Participating Districts

16

Given the purposive selection of study districts, the evaluation must be quasi-experimental. Specifically,

◦ A difference-in-difference (“diff-n-diff”) estimator between matched districts in treatment and control (16 in each arm) estimates program impact

◦ To estimate actual impact: two years of program exposure (2012-2014) is contrasted with a two year period immediately (2008-2010) before the program

Evaluation Design II

17

Household information – Population representative surveys of health behavior,• including utilization, recall of procedures health outcomes,• including anthropometry, satisfaction with care, knowledge and mediating variables• socio-demographics

Baseline data for community and household utilized the 2011 DHS Follow up data at community, household structured to replicate and

supplement the earlier DHS Yields a sample of ~2800 recent pregnancies/births

Data Sources I

18

Facility survey – a comprehensive review of the structure, provision, and quality of care at clinic level Instruments

◦ Facility checklist◦ Health worker tool ◦ Exit interview tool (ANC, child illness) ◦ Direct observation (ANC, labor and delivery and child illness) ◦ Chart audit* of routine and complicated delivery• Only in the follow up

180 facilities in 2011 baseline (the NIHFA) and 231 in follow up

Technical support from USAID and UNICEF was critical in this undertaking

Data Sources II

19

• Data has just been collected and still being processed• However the preliminary results from both population

and facility data are now available…• Overall it is a story of strong gains in select health

indicators for the entire nation (consistent with the MICS results), with yet more rapid improvement in RBF districts

• RBF gains in both the quantity and quality of care, but not for all prioritized indicators

Preliminary Results I

20

• RBF led to gains in both the quantity and quality of care• A 13 percentage point increase in the in-facility delivery

rate• A 12 percentage point increase in post-natal care

coverage• More women receiving full package of ANC services

including urine tests, blood tests, tetanus shots• But not for all prioritized indicators• Little change in ANC coverage and contraceptive use–

baseline rates already high

Preliminary Results II

Dif-n-dif in Practice: Receipt of Urine Test during ANC Care

Start of RBF

2008 2009 2010 2011 2012 2013 20140.4

0.45

0.5

0.55

0.6

0.65

RBF pre-trend

RBF trend

comparison pre-trend

comparison trend

22

Outcome ImpactLevel of

significance

Any modern contraception 0.05 0.21Receipt of any antenatal care (ANC) 0.02 0.43Time of first ANC -0.19 0.22Number of ANC visits 0.40 0.15Blood pressure checked during ANC 0.03 0.55Urine sample collected during ANC 0.16 0.02Blood sample collected during ANC 0.08 0.14Receipt of tetanus vaccine during pregnancy 0.08 0.05Number of tetanus vaccine 0.33 0.05Any iron supplements received during pregnancy 0.00 0.99Receipt of any postpartum care (PPC) 0.12 0.05A PPC within 2 months of delivery 0.12 0.06PPC by a skilled provider 0.14 0.02Facility delivery 0.13 0.00Skilled delivery 0.14 0.00

Summary Regression Measures of Impact

Most impacts measured in proportional terms. Statistically significant results with in red.

23

Complementary Results from Facility Level Data

Patient exit interviews◦ ANC interviews (n=1105) indicate significant improvements in care

processes such as measured abdomen and urine sample◦ Child health interviews (n=1612) indicate significant improvement in

measurement and growth monitoring◦ Client satisfaction is also higher (although at marginal significance,

p=.128) Facility measures◦ Improvements in select facility conditions after RBF:

8 percentage point increase in availability of bio-med waste disposal◦ Increases in supervision and community involvement after RBF:

Increase in number of HCC meetings (2.7 more per year) Increase in external assessments of staff (2.4 more per year) 34 percentage point increase in presence of facility work plan

Moving Beyond the ‘Black Box’

RBF Investments ProgramOutcomes

25

Need to go beyond numbers ◦ Capture rich experiences and lessons from frontlines of PBF

implementation & context Community engagement and support (HCC) Geographic influences (supervision aspect, cross-catchment area

patient movement) Health facility management skills and dynamism Extent of mentorship and clinical supervision by district/province

Explain contextual factors that matter the most & account for variation in provider performance

Supporting impact evaluation and not substituting it –better understanding of the our intervention

Process Monitoring & EvaluationRationale

Providers at high performing health facilities (HPF) were reported to be highly motivated.

Regular and well structured supportive supervision improved the capacity to earn more subsidies e.g one (HPF) had an increase in subsidies of 53% to 94% between the first and second quarters.

Team work improved communication...we are now working as a team to ensure that the T5 (data

compilation form) is correct and accurate and we are able to send it to the district in time. We are motivated to work as we get paid for our effort. --( HF Staff Member)

Higher client satisfaction attracted more patients (patient choice)

Select Findings from Qualitative Study

When low performance is found, facilities attribute reasons to:Smaller catchment populations & at times

geographic remoteness Irregular supportive supervisionSocio-economic and religious factors in the

population

Challenges Affecting HF Performance

28

Revisit the remoteness criteria & facility catchment issue RBF performance payment calculation formula being

reviewed (higher remoteness bonus) Content & structure of quality supervision checklists

Written feedback by district managers mandatory Modified supervision checklist – more process measures

of clinical care introduced District Health Executive supervision contract revisited –

ensure incentive to regularly supervise remote facilities Integrate RBF into HR management

Policy Actions Considered & Next Steps

29

Thank You!

30

Extra slides

31

What is the impact of RBF on maternal and child health services utilization and outcomes?

What is the impact of RBF (counter verification) on health management information system (HMIS) and on supportive supervision?

What is the effect of RBF on community participation and social determinants of health?

What is the effect of RBF on health workers’ attitude, job satisfaction, retention and attrition, etc?

What is the effect of RBF on patient/client satisfaction and in health seeking behavior?

Policy Relevant Questions - Zimbabwe

32

Given the purposive selection of study districts, the evaluation must be quasi-experimental. Specifically,

◦ A difference-in-difference (“diff-n-diff”) estimator between matched districts in treatment and control (16 in each arm) estimates program impact In a diff-n-diff estimator, the observed trends in outcomes for the

comparison districts stand for what would have happened in the RBF districts if not for the RBF program

The validity of this approach can be assessed by comparing pre-RBF trends of same outcomes in RBF and comparison districts before RBF program

To estimate actual impact: two years of program exposure (2012-2014) will be contrasted with a two year period immediately (2008-2010) before the program

Evaluation Design II

Dif-n-dif in Practice: In-facility Delivery Rate

Start of RBF

2008 2009 2010 2011 2012 2013 20140.5

0.55

0.6

0.65

0.7

0.75

0.8

0.85

0.9

0.95

RBF pre-trend

RBF trend

comparison pre-trend

comparison trend

Dif-n-dif in practice: Rate of ANC coverage (any ANC)

After introduction of RBF

2007 2008 2009 2010 2011 2012 20130.88

0.9

0.92

0.94

0.96

0.98

1

RBF pre-trend

RBF post-trend

comparison pre-trend

comparison post-trend

35

Key areas PME assessed

• Factors affecting health provider performance

• Factors influencing changes on the demand (community) side

• Compliance of the RBF stakeholders to the project’s implementation guidelines.

Moving beyond the ‘Black Box’ II

PME Study sites

Binga, Chipinge, Kariba, Mazowe and Zvishavane.

Selected three facilities in each of five districts

Sequential mixed method deployed

Participants selected included mothers of reproductive age, male community members, influential leaders, health center committee members, district management team, as well as clinic health staff

Sampling & Approach

38

RBF:◦ Strengthens relationships between health care providers &

communities and improves access to services at community level (e.g. community ambulance)

◦ Fosters innovations and entrepreneurship among health workers◦ Improves district managers supportive supervision◦ Improves health facility infrastructure (service delivery environment)◦ Improves staff morale and promotes team work

Summary of Qualitative Evidence

Staff shortage “When one of us attends a workshop only one person is left to deal with registers and the workload is huge and most of the mistakes we make are due to fatigue. We are now losing patients to other institutions…”HF staff

“There are a number of clinics surrounding therefore there is competition for example X clinic is GVT owned and well equipped, has drugs and well-staffed so patients prefer going there than spending more time in long queues here where there is staff shortage”. -- HF Staff Member Shortage of drugs e.g. vaccines Inadequate infrastructure

Challenges (cont.)