2

NU Health Lessons Learned II Report

TABLE OF CONTENTS

Table of Contents ........................................................................................................................ 2

Abbreviation and Acronyms ......................................................................................................... 3

1 Executive Summary ............................................................................................................. 5

2 Introduction .......................................................................................................................... 6

3 Study Design ........................................................................................................................ 8

4 Experience working with District Health Teams .................................................................. 11

5 Experience working with PNFP Health Facility Staff ........................................................... 15

6 Quality of Care ................................................................................................................... 18

7 Health Outcomes................................................................................................................ 32

8 Value for Money ................................................................................................................. 42

9 Operation / Management Processes .................................................................................. 56

10 Risk, particularly Fiduciary Risks, and its Mitigation ........................................................ 61

11 Conclusions: Good and Bad Practices in RBF ................................................................ 64

References ................................................................................................................................ 68

Annexes .................................................................................................................................... 69

Annex 1: Indicators Assessed and the Associated Calculations for Payments - Year 2 ......... 69

Annex 2: Standard Unit of Output (SUO) formula used by UCMB .......................................... 71

3


ABBREVIATION AND ACRONYMS

ANC

BP

CI

Antenatal Care

Business Plan

Confidence Interval

CL Credit Line

CSP

DCV

DFID

DHIS2

DHT

DQA

DR

EMHS

Capacity Strengthening Plan

Direct Client Verification

Department for International Development

District Health Information System

District Health Team

Data Quality Assessment

Discrepancy Rate

Essential Medicines and Health Supplies

FGD Focus Group Discussion

HC

HF

Health Centre

Health Facility

HMIS Health Management Information System

HPI

IBF

Health Partners International

Input-Based Financing

IMNCI Integrated Management of Newborn and Childhood Illnesses

JMS

KII

LL

LLU

Joint Medical Stores

Key Informant Interviews

Lessons Learned

Lower Level Unit

MNCH

MoH

NAO

NU Health

Maternal, Newborn and Child Health

Ministry of Health

National Audit Office

Northern Uganda Health Programme

OPD Out-Patient Department

OR Odds Ratio

PCDP

PMTCT

Post-Conflict Development Programme

Prevention of Mother to Child Transmission (of HIV)

PNFP Private-not-for-Profit

PO

QoC

QQA

RBF

Purchase Order

Quality of Care

Quarterly Quality Assessment

Results-Based Financing (Health Facility)

RDT Rapid Diagnostic Test

SUO

ToC

UCMB

UGX

Standard Unit of Output

Theory of Change

Uganda Catholic Medical Bureau

Ugandan Shilling

UHC Universal Health Coverage

4


UKaid

VfM

United Kingdom Aid

Value for Money

5


1 EXECUTIVE SUMMARY

This report summarises lessons learned in the implementation of the Department for International

Development (DFID) funded study Northern Uganda Health Programme (NU Health). It covers a

set of topics identified as areas of substantive interest to DFID as well as the larger Results-Based

Financing (RBF) community. Drawing on a range of methodologies, these lessons learned build

on data collected specifically for this report, as well as analysis of existing programme-generated

data. Each section of this report is largely self-standing to enable the discerning reader to review

specific topics consistent with his or her own particular interest. The final section serves as a

conclusion on what has been learned and what might be done with that learning going forward.

Following the completion of its inception period, NU Health underwent a significant revision shifting

from a demonstration project to an operations research study with the aim of generating evidence

and insight into how RBF can contribute to improving health outcomes for the poor. The

programme was refocused to attend to some of the evidence gaps in the current debate on

achieving universal health coverage through RBF, better accountability and improved health

sector performance. NU Health is contributing to the policy discussion in Uganda and it aspires

to make contributions to the larger global debate. This report does not aim to provide the final

word on what has worked and at what price, rather it does aspire to share relatively unvarnished,

practical insights into what seems to work and lessons learned that others active in this area may

benefit from adapting or avoiding in their own efforts.

The report seeks to capture lessons learned along the programme’s value chain, from inputs

through processes, to outputs and outcomes. It is structured to address specific aspects of the

study and their role in the programme and the evidence it generates.

1. Executive Summary

2. Introduction

3. Programme design

4. Experiences working with District Health Teams

5. Experiences of Health Facility Staff and Management

6. Quality of Care

7. Health Outcomes

8. Value for Money (VfM)

9. Operational/Management Process

10. Risks, including fiduciary risk and mitigation in both RBF and Input-Based Financing (IBF)

11. Good and bad practices in RBF

Given the multiple stakeholder groups or constituencies involved in NU Health, many sections of

this report highlight points around perspective, timing and time horizon. Stakeholders at both the

District Health Team (DHT) and Private-Not-For-Profit (PNFP) facility levels appreciate that the

fundamental focus of NU Health as a study is to generate evidence. Still, at the DHT level, though

NU Health may have represented an interesting opportunity to improve health system

performance, it is seen as another priority to juggle by many DHTs who already feel under-

6


resourced and over-burdened. At the facility level, NU Health was generally appreciated as a

vehicle for improving quality of care, and coming as it did with the cessation of humanitarian

support to the health sector, it was seen as a lifeline by many health facilities. In reviewing the

lessons learned outlined in this report, the view point of these stakeholder groups is highlighted

where relevant.

As NU Health looks toward its final months, and the prospects for evidence uptake, it is worth

noting that accompanying the strengthening of the programme’s research study design, DFID also

advised the programme managers to curtail advocacy and communications activities. While the

proscription on advocacy in the absence of evidence certainly made sense, the proscription on

communications was, with hindsight, less helpful: NU Health was off the radar screen for many

key stakeholders and certainly at a policy level. More recently, as restrictions on communication

have eased, stakeholders are taking a real interest in the initial insights arising from the NU Health

experience, while sharing their concerns about how NU Health and similar efforts will feed into

Uganda’s roadmap for establishing more accountable and higher functioning health services

through Results-Based Financing (RBF).

2 INTRODUCTION

The Northern Uganda Health programme (NU Health 2011 - 2015) is part of the UKaid-supported

Post-Conflict Development Programme (PCDP) which aims to strengthen local and national

mechanisms for governance and accountability to improve access to health care, particularly for

the most vulnerable populations in the Acholi sub-region. NU Health focuses on generating

evidence on RBF and the extent to which this is an efficient and effective financing mechanism for

improving accountability and access to quality health care with faith-based PNFPs. Following its

recent contract amendment, the programme will run to the end of October 2015.

After the programme’s inception period, major changes were made to programme design and

implementation approach in consultation with DFID, and the programme was modified to better

assess the cost effectiveness of RBF, by introducing an input-based financing (IBF) comparison,

or control arm, in addition to the RBF intervention. Rather than running the programme in

“implementation mode” to maximise prospects for positive outcomes associated with RBF, NU

Health would adopt a “hands off” approach to avoid influencing how RBF facilities would use any

financing they received for attaining targeted results. Other programme variables aside from the

financing mechanism, such as the provision of a funded credit line and supportive supervision,

are kept constant for the RBF and IBF facilities to isolate the main effort of the funding modality

and strengthen the validity of results.

Prior to refining the study design, the programme had five outputs:

7


Output 1 Improve availability of essential medicines in participating PNFP health facilities

through the provision of a funded credit line for essential medicines and health

supplies (EMHS) for all participating PNFP facilities through the Joint Medical

Stores (JMS)

Output 2 Support improvements in service delivery and quality at PNFP facilities through

the implementation of a RBF model in the Acholi sub-region and for comparison

an input-based financing (IBF) model in Lango

Output 3 Strengthen the capacity of the District Health Teams (DHTs) to monitor and

provide supportive supervision to PNFP facilities

Output 4 Generate stakeholder engagement on the project, the evidence it generates, and

the concept of Results Based Financing at local & national levels

Output 5 Ensure the verification role within the RBF scheme is effectively fulfilled by the

DHTs with support from the NU Health team

With the refined study design, Output 4 was significantly curtailed. While stakeholders directly

involved with the study, i.e. DHTs and diocese and PNFP staff, would be engaged, DFID advised

that NU Health should not proceed with advocacy or communications with the broader stakeholder

group, particularly those at the national level, during the course of the study. Networking and

communications with these groups recommenced following approval of Contract Amendment 3 in

January 2015.

The Lessons Learned report originated from discussions between DFID and NU Health on key

themes for a series of Technical and Policy Briefs to be produced by the programme, exploring

specific areas of interest and programme findings in greater detail. Following further discussion, it

was agreed that two longer Lessons Learned documents would be developed and in March 2014

NU Health produced the first report which summarised lessons learned in the implementation of

the programme to date. That report covered a number of topics identified as areas of substantive

interest relevant to DFID as well as the larger RBF community. Drawing on a range of

methodologies, these lessons learned built on data collected specifically for the report, as well as

alternative analysis of existing programme-generated data. This second report draws additional

attention to implementation issues and their potential implications for any roll out or scale up of

RBF programming.

As working papers, there are limitations to these Lessons Learned documents. A principle

limitation is the reliance on data from the broader Health Management Information System (HMIS)

for some areas of comparison between NU Health-supported facilities and others in the ten

districts, as well as the use of qualitative methodologies on some issues which do not necessarily

provide a basis for wider inference. The findings, therefore, should be interpreted with some

caution and within their specific context. Specific limitations are noted in each section where

relevant. Evidence generated by an independent assessment undertaken by Liverpool School of

Tropical Medicine/LATH will augment these data and may provide a basis for more robust

inference.

8


The objective of this report is to build on the findings in the first report and further examine the role

of RBF or IBF in affecting changes through individual and collective behaviour at facility and district

levels, ultimately in the provision of health services.

This report is structured around the following nine key themes in sections 3 – 11 of this report:

3. Study design

4. Experiences working with DHTs

5. Experiences of Health Facility Staff and Management

6. Quality of Care

7. Health Outcomes

8. Value for Money (VfM)

9. Operational/Management Process

10. Risks, including fiduciary risk and mitigation in both RBF and IBF

11. Good and bad practices in RBF

3 STUDY DESIGN

In its assessment of the costs and benefits of RBF versus IBF, NU Health was designed to isolate

the main effect on financing modality, i.e. RBF versus IBF, on health output. Both this study design

and the inclusion of an evaluation are intended to generate robust evidence. NU Health is

designed to help address key gaps in the body of evidence on the relative value added of RBF.1

One of the assumptions outlined in the project design was that supported PNFP would reduce

their user fees as (i) this loss of funding from patients contribution would be compensated by DFID

funding and (ii) reduced fees would serve the purpose of increasing affordability and thus

accessibility. However, during the inception period this assumption revealed to be impractical as

PNFP were not prepared to remove their user fee levels. The reasons given by PNFP were that,

as NU Health support was limited in time, facilities would have to re-introduce their user fees after

NU Health’s exit. This would likely turn away patients/clients and risk the PNFP’s ability to provide

quality services with any degree of sustainability.

With the aim of isolating the main effect of the financing modality, there were several revisions to

NU Health’s design. First, there was the addition of a control or comparison group, a set of

facilities in Lira which by a range of indicators were similar to those in the Acholi intervention

districts. Secondly, the study adopted inclusion criteria related to facility readiness: despite some

modest investment to reach sample size, most facilities were included “as is” with minimal

intervention to affect facility systems or capabilities. Thirdly, the study emphasised providing

significant capacity development support to the DHTs to enable them to fulfil their

regulator/verifiers function in the study. Finally, there was a decision to be “hands off” in advising

the RBF facilities how they might use their results payments; such that each facility within the

1 For example; Eldridge, C & Palmer, N. Health Policy & Planning. 24 (3):160-166. (2009)

9


strictures of their own standard operating procedures could use the funds it earned as it deemed

fit.

The Theory of Change (ToC) behind the study design is summarised graphically as follows:

Following an assessment to identify PNFPs that met standards of functional service delivery, 21

facilities were originally enrolled in Acholi sub-region and these facilities were matched with ten

control PNFPs in neighbouring Lango sub-region. The Lango region was chosen as it is also post-

conflict and shares a similar socio-cultural and economic setting to Acholi. The range of levels of

health facilities within the RBF and IBF regions are represented in Table 1 below:

Table 1: Range of PNFPs in RBF and IBF regions

Level Acholi/RBF

region

Lango/IBF

region

Hospital /

Health Centre (HC) 4

3 2

HC3 3 6

HC2 15 2

After the disqualification of one facility, St Joseph Minakulu HC2 in July 2014, the 20 Acholi

facilities receive a variable payment each quarter based upon their performance over the

preceding three months. The ten control PNFPs in Lango each prepare a work plan to address

core areas of service improvement and receive input-based funding to support implementation of

these plans. The amount of funding allocated per level of facility roughly matches the amount

anticipated for the RBF PNFPs at the same level over the course of the programme.

A key feature of RBF design is the separation of the roles of Funder, Purchaser, Provider and

Regulator/Verifier. In NU Health these roles are defined as follows:

The Funding Agency is DFID, who transfers funds to NU Health.

The Purchaser is NU Health on behalf of DFID. NU Health oversees the transparency of

the RBF system, and controls the checks and balances. Total system oversight rests

physically at the NU Health Gulu office, with oversight from the NU Health team based in

Kampala.

The Providers are the PNFPs providing health services. They each hold a contract with

the purchaser which is contractually binding in terms of remuneration/incentive payments.

5 Outputs 1. Credit line 2. Financial support 3. DHT strengthening 4. Advocacy 5. Verification/ M&E

Outcome Increased access to quality health care services by the poor in Northern Uganda

Impact Increase in the economic, social and political opportunities that improve the lives of people affected by conflict in Northern Uganda

10


The Regulator/Verifier is the DHT in each district, which has the primary responsibility of

providing guidance and oversight to the PNFPs in implementation of RBF and general

troubleshooting, largely through feedback on HMIS reporting and planned supervisions.

Initially NU Health supported all the participating DHTs in this role both financially and with

the provision of secondees to support capacity, whilst gradually building their capacity to

fulfil their supervision/verification role independently. The programme design aims to see

DHT secondees becoming absorbed within, and financed by, the formal DHT team.

For the purposes of the study, a key aspect of NU Health is an intensive and rigorous verification

process to validate the service delivery data provided by PNFPs. This is a fundamental component

of the NU Health model as the value of RBF payments is calculated on the basis of verified

performance data. Data quality assessments (DQA) are undertaken by DHTs in collaboration with

NU Health staff and are complemented by a quarterly quality assessment (QQA) in each health

facility. The indicators selected to assess PNFP performance include the following:

Antenatal care with defined quality parameters – starting before 16 weeks, 4+ visits,

including provision of tetanus vaccination and malaria prevention, with appropriate

measures for the prevention of mother-to-child transmission (PMTCT) of HIV.

Delivery in the health facility – using a partograph, with emergency obstetric care provided

as needed, early breastfeeding,

appropriate postnatal care.

Child care – full vaccination,

appropriate diagnosis and

treatment of common illnesses.

Adult care – appropriate

diagnosis and treatment of

common illnesses.

Based on the results of the DQA and

QQA, RBF PNFPs receive payment

according to the formula in Figure 1.

Details of all indicators assessed and the

associated calculations for payments

are provided in Annex 1.

In the IBF control or comparison group, facilities received grants matched to facility-level specific

levels of RBF payment. In contrast to the RBF “hands off” approach, the use of grant funds are

highly regulated, with extensive scrutiny related to adherence to plan and procurement procedure.

P RBF Payment to a PNFP for the quarter

S Standard Subsidy for a particular indicator

z Quality Multiplier determined by the PNFP quality score

x Base Incentive per level of care

y Quality Incentive per level of care

n Number of patients seen by the PNFP for that particular indicator

P = S ( x + (y z)) nRBF INCENTIVE

PAYMENT

FORMULA

RBF Formula

13

Figure 1 RBF Formula

11


4 EXPERIENCE WORKING WITH DISTRICT HEALTH TEAMS

This section reviews the perspective of the DHTs along with those of the facilities and NU Health

Staff to summarise the experience of working with DHTs. As the most decentralised levels of

regulation within the Uganda Ministry of Health (MoH), the DHTs and Health Sub-District

authorities have played a critical function in the role as regulator/verifier in the RBF intervention

area, as well as assessor in the IBF control area. In addition, the DHTs have a set of tasks around

guidance and supervision of health facilities located within their area of responsibility, data

collection and analysis, the planning of health service delivery, and the coordination of

stakeholders at district level (across both public and private sectors). Determinants of DHT

performance relate to, among others, adequate staffing to cover these core functions, appropriate

skills levels among those staff, and the availability of key support measures, for example, funding

for activities and equipment such as vehicles, computers, power supply and so on.

At the time of programme inception, the most prominent obstacle to DHTs carrying out their

responsibilities were the capacity constraints. There was a combination of understaffed district

health offices, under-skilled DHT members, and a lack of transport to enable outreach to the

facilities. Both the lack of transportation and the shortage of qualified district health officials stems

from budget constraints, including staff attrition, unattractive employment packages and the

absence of facilities at the remote areas where some officials should be posted.

NU Health sought to address these issues by supporting three key areas:

Human resources strengthening;

Improved supportive supervision;

Improved data collection, analysis and utilization.

NU Health’s interventions in the aforementioned areas have resulted in improvements in DHT

overall effectiveness and efficiency, including the quality of health facility assessment and

supportive supervision.

Assessment Methodology

Focus Group Discussions (FGDs) and Key Informant Interviews (KIIs) were conducted with 48

District Health Team members according to a predetermined questionnaire. The interview

questions focused on the participants’ perceptions and opinions of:

the impact of support to the DHTs and health facilities on the delivery of services;

the main challenges and successes experienced during the life span of the NU Health

project;

the sustainability of the changes made; and,

recommendations for future RBF initiatives.

In addition, FGD were conducted with NU Health Staff which helped to inform the agenda for a

broader workshop convening key DHT with PNFP personnel, to discuss these issues in greater

12


depth. Many of the core issues raised are expanded with quantitative programme findings

throughout the remaining report sections.

Findings

Overall, the district health officials continued to report a positive view of the support provided by

the NU Health programme both at the district and facility levels. District health officials observed

many improvements at the PNFP health facilities particularly in the following areas:

quality of care provided;

patient attendance and satisfaction with services rendered;

staff management and motivation;

data management & reporting;

investment in equipment and capital development;

financial management;

drug management; and,

prescription habits.

The sections below present the district health officials’ experience in working with the NU Health

study in the three key areas of support provided.

Human Resources Strengthening

In most districts NU Health filled critical staffing gaps at the district health office by seconding staff

to support maternal, newborn and child health (MNCH) related activities. In cases such as Nwoya

district, DHT performance improved greatly with the support from much needed team members

and resulted in the district rising from 20th to 1st position in health sector performance nationwide

due to improved reporting. Despite the assistance that NU Health provided to the various districts

in this regard, many remain understaffed and it is likely that without support from the central

government or other external interventions, many of these positions will remain unfilled.

Supportive Supervision Capacity

NU Health supported each DHT to draw its Capacity Strengthening Plan (CSP) outlining the

support that each DHT would require to enhance its ability to play its expected supervisory role

and also to perform the additional role of independent verifier of the NU Health programme and to

eventually roll out the DQA/QQA process to other public or private facilities within its jurisdiction if

deemed relevant by the DHT. With Uganda Health policy which keeps on changing, most of the

CSP were aimed at mentoring health facility staff on implementing new health policies.

Participants reported that the DHTs’ capacities had been strengthened and that the support from

NU Health now prepared them for the management of RBF programmes. However, some felt that

the recruitment of additional vacant positions within the DHTs and further capacity building could

have resulted in a greater positive impact in supervision and consequently better health service

delivery and would certainly be necessary if RBF programming were expanded. One of the

recommendations made was that there be increased inter-district DHT dialogue and exchange

visits to make up for lack of capacity in some areas. District health officials also recommended

13


that PNFP staff be trained to do a “peer” QQA supportive supervision in other facilities to sustain

the positive impact of the programme on health service delivery. One of the obstacles in achieving

positive results from the DHTs was the attitude among some DHT members who are mainly

motivated by additional allowances given by the programme.

Logistics

Transport to and from the facilities during the quarterly assessments remained a challenge for the

DHTs. This was primarily a result of the competing claims on the limited availability of well-

maintained vehicles at the district health offices.

Conducting Supportive Supervision in Non-NU Health Supported Facilities

Many of the health officials interviewed reported that they had used the NU Health supervision

tools with facilities outside the NU Health study area. It was reported that although the MoH has a

supervision tool, the NU Health tools were preferable as they are more comprehensive. At the

same time, despite the shortcomings of the MoH tool, some officials felt that NU Health should

have harmonised their tools with the pre-existing ones for the purposes of uniformity and ease of

transition in a post-NU Health setting. District health officials also noted that seconded staff

performed their routine duties outside of NU Health study activities and that there were sometimes

conflicting priorities between fulfilling routine responsibilities related to their role within the DHT

and those associated with the NU Health study. Additionally the DHT personnel, including the

secondees, are being solicited by many implementing partners for various programmes and thus,

their availability is often limited.

Data collection, analysis and utilization

With the NU Health study requirements for verification in RBF facilities and assessment in the IBF

facilities, DHT participation in data collection, analysis and utilization increased tremendously.

Over time, a number of DHT members became conversant with data management and use,

particularly in the RBF areas.

As reported in the final section of this report, over time NU Health observed improvements in

reporting, data management and utilisation. Report accuracy increased over time, especially in

the RBF group, as did, to some extent, their completeness. However, the timeliness of reporting

remains a challenge with an average timeliness rate of reporting shown on District Health

Information System (DHIS2) below 70% in both groups (see section 11 for further details).

The demands of the study also served to highlight capacity constraints at both the systems and

individual levels. There were challenges in working with DHIS2 -- including familiarity with the

programme, lack of regular electrical supply, and unreliable internet connectivity. Although the

DHTs were mindful of the time bound nature of the study, many noted the critical gaps in staffing

as well as finance for transport which would adversely affect the DHTs’ ability to verify quality

indicators in the future. It was also noted that in some districts there were relatively low levels of

motivation of DHT members to supervise the facilities let alone review their data.

14


Feedback on process improvements suggested by DHTs

Participants stressed the importance of documenting NU Health’s activities, analysing the

changes made in the facilities and from this, making informed decisions and recommendations for

future RBF initiatives. District health officials also felt that it would have been beneficial for the

programme to have run over a longer period of time since the facilities are just starting to have

consistent performance in the delivery of quality health services.

There is also a general sense that a similar programme should be designed for government health

facilities since the PNFPs are mostly situated in urban and peri-urban areas. Supporting

government facilities in this way would have made a greater difference in rural and underserved

communities. In some areas where only one facility was supported in the whole district, it was

recommended that in future programmes, multiple facilities be supported to be able to better

assess programme impact on service delivery.

PNFP Perspectives

Principle observations from PNFP staff about the DHTs related both to their technical capabilities

and their ability to independently assess and verify facility performance and provide supportive

supervision. This first point, confirmed by NU Health staff, points to the limited capacity of many

DHTs and their absence of redundant capability on issues, such as MNCH, as well as broader

technical and policy concerns such as the role out and use of DHIS2.

The second point was slightly more sensitive. By design, RBF established a segregation of

responsibility that entails a tension between providers and regulators of services. PNFP staff

noted the difficulty at a human level of both assessing performance and defining payment, while

also providing supportive supervision, without being drawn into any gaming of the system. The

PNFP suggestion was to have these two functions separated in the future if possible, which would

have further implications for the, already limited, DHT staff time.

NU Health Staff Perspectives

NU Health field staff also flagged the technical and managerial capacity constraints described by

PNFP staff. In addition, they highlighted the absence of financial analytic and management skills

on the part of DHT staff.

Their more salient observation however was of a strain placed on the DHTs by donor and NGO

programme fragmentation contributing to competing demands on their time. They were clear that

the range of initiatives and their associated level of effort was beyond what the DHTs could

reasonably accommodate. The NU Health field team noted the risk of the largely unregulated

development assistance project market and the tendency for DHT staff to participate in activities

that yielded better allowances, rather than on the basis of policy priority or other system level

criteria.

Policy implications for future RBF initiatives

Overall, the district health officials are supportive and enthusiastic about government

implementing the RBF model. However, some of the district health officials interviewed

15


commented that the central MoH was not fully engaged with the programme. They felt that had

they been more engaged and therefore witnessed some of the changes that the programme has

influenced in the facilities, there would be greater support for government to push for RBF at a

policy level. District health officials have found the interventions and tools introduced by NU Health

useful, and expressed a willingness to use them beyond the life of the programme. However, it is

important to note that future RBF programmes would need to integrate and encourage increased

involvement of the MoH and other stakeholders to be able to push the RBF model as a policy, if

the final evidence gained from the programme evaluation (and other similar initiatives) supports

the intervention model. It was noted that district local government authorities and politicians should

have been better informed about the progress of the study activities to raise awareness and

support for the RBF model. There is a general appreciation that DHTs will need significant

capacity strengthening if they are to support a roll out of RBF to public as well as PNFP facilities.

5 EXPERIENCE WORKING WITH PNFP HEALTH FACILITY STAFF

This section reviews the perspective of the PNFP staff along with those of the DHTs and NU

Health Staff to summarise the experience of working with PNFP staff. A key early insight related

to the relative operational autonomy of the PNFP and the often modest level of oversight that the

dioceses or medical boards had on PNFP operations. Although general observations are valid on

functionality at a particular level of facility, e.g. HC2 or HC3, there was a high level of variation

within any particular level in how facilities functioned.

Many of the PNFPs supported by NU Health were new to the idea of business planning. Yet for

health facilities to participate in NU Health, they had to develop individual plans to outline their

spending priorities. The potential value of a business plan is clear to most, in so far as it aims to

support providers to assess where they are and plan realistic targets; and supports purchasers to

understand in which resources the facility may invest and which strategies they were likely to

apply. However, despite these apparent benefits, many facilities struggled with the exercise. While

many higher level facilities, such as hospitals, already had an established annual planning

process, most lower-level facilities did not, and required substantial assistance. The study’s

“hands off” approach, however, limited the level of support provided to the facilities for the

implementation of their business planning as no advice was provided on the allocation of funds

and there was no monitoring on the use of the funds in the RBF region. This meant that the

expected improvement in business planning from Year 1 to Year 2 was somewhat erratic.

The NU Health study tracked how well the supported health facilities under both models of

financing developed, monitored and revised their business plans to prioritise their spending and

improve service delivery. The NU Health team assessed how participatory the business planning

process was and what incentives exist for employees when the targets were met. As noted in the

first Lessons Learned Report, there were no significant differences in the business planning

process between the IBF and RBF health facilities. Most of the health facilities had participatory

business planning processes and were implementing the plans and making appropriate revisions

to reflect the facility’s evolving needs.

16



FGDs were conducted with health facility staff in three areas of interest listed below:

Business planning;

Staff motivation; and,

Perceptions on financing.

Findings

Overall, staff from both RBF and IBF health facilities reported an improvement in their work

environment and the quality of services that their facilities are now providing. Many reported an

increase in the number of patients, a reduction in drug stock outs and the upgrade or renovation

of health facility infrastructure. Although there were improvements observed in how healthcare

services are delivered at the health facilities, it was noted that some of these improvements would

not be sustainable without NU Health support once the programme ends. For example, most

health facilities reported that staff salaries or salary increments were being paid out of the funds

being provided by the NU Health programme to boost staff motivation. Concern also included how

subsidies for medicines will be maintained in the absence of credit line support.

Business planning and monitoring

It was found that health facility staff did not always participate in the development of the business

plans which were perceived to fall under their financial teams’ roles. This undermined the staff’s

ownership of the changes being made at the health facilities in the interest of meeting assessment

targets. The limited interaction between the financial and health management also affected efforts

to improve conditions at the facilities since the financial teams did not typically have the technical

knowledge about health facility operations. In the RBF health facilities where staff participated in

the business planning to some degree, staff felt that not enough incentives accrued to them

individually when the health facilities were awarded good scores by the NU Heath teams and

DHTs.

Motivation

Some of incentives provided to health workers and staff in return for achievement of health facility

targets included:

Allowances for or provision of accommodation: such as Minakulu HC3 providing a rent

allowance to the staff or Aber Hospital providing accommodation to the staff;

Allowances for or provision of meals to the staff during working days; and,

Bonuses or salary increments such as Alanyi HC3 providing bonuses once last year while

in Aduku HC2 bonuses are provided once a quarter.

However in some of the health facilities, staff remained demotivated. Some staff were displeased

with how facility leadership apportioned funding. It was noted that in some cases staff’s living

conditions were unimproved or salaries unchanged. Staff continually stressed the importance of

health facilities allocating some funds towards further professional development and other

employment benefits such as retirement packages or more bonuses.

17


Perceptions on Financing: Funding autonomy and innovation

Supported facilities invested the NU Health funds in a number of areas that they felt needed

improvement including:

Procurement of medical equipment;

Payment of staff salaries and salary increments;

Construction or upgrading of health facility infrastructure;

Accommodation allowances; and,

Recruitment of additional staff.

Health workers in the RBF health facilities were aware of the new funding model and in some

cases they shared in the success of their health facility. In the cases where this was practised,

better teamwork was fostered and health workers reported increased accountability amongst

themselves and to management. In some cases, the staff reported receiving bonuses depending

on how well the health facility had performed during its assessments. When and where health

facility (HF) staff were not receiving what Management had planned or promised, there were some

examples of disgruntled staff sabotaging the QQA process by reducing quality of services to

ensure that the following QQA results would be poor, which would eventually penalise the HF.

Despite their recognition of the NU Health Study’s purpose, many staff were concerned about the

sustainability of the changes brought about once the study comes to an end. There were concerns

about shortages of essential medical equipment that the facilities needed to be able to provide

better quality health services. Sustaining the rates of subsidised medication was also cited as one

of the challenges that the health facilities would struggle with when the NU Health study closes.

There was a concern that in the absence of other support, the provision of improved quality

healthcare may not be sustained beyond the end of NU Health study.

Feedback on process improvements suggested by PNFPs

Respondents generally reported that health facility management would have benefitted from more

guidance in the investment or budgeting of received programme funds. For example in the RBF

facilities where improved delivery of health services did not reflect in any additional benefits for

employees, health workers suggested that future RBF programmes set a standard salary or that

more stringent conditions be put in place that provide for staff welfare.

Many also recommended that the programme be extended to run over a longer period of time for

it to have a longer lasting impact on the provision of healthcare services in this region. Some

suggested that the DHTs’ capacities should have been strengthened further to enable them to

have the same knowledge on quality health service delivery as the NU Health team. Some health

facility staff also felt that health facilities in more remote areas should have been awarded more

support given the difficulties in providing health services especially with the high staff turn-over

rate experienced in such settings.

18


DHT Perspectives

One of the larger tensions with NU Health from the DHT perspective was the projects focus solely

on PNFPs. Understandably, the DHTs have an allegiance based on affiliation with the public

sector facilities. Despite their stated appreciation about the scope of the NU Health study, many

expressed an underlying resentment that the PNFPs get preferential treatment vis-à-vis public

facilities.

That said, many DHT staff noted their real appreciation for the opportunity to engage substantively

with the PNFPs. A number noted that they had applied their new skills and tools in results

verification and supportive supervision to facilities not participating in the study.

NU Health Staff Perspectives

NU Health staff noted the absence of financial analytic and management skills on the part of many

of the PNFPs. In the lower level facilities, there were also basic concerns about quality of care in

addition to data management/ utilisation capabilities.

NU Health staff had more salient observation however was around the “hands off” approach in the

RBF intervention facilities. As facilities came to focus on quality of care—which was good—and

to improve their results payments, staff were too frequently disappointed with the way many

PNFPs opted to use their results payments and often frustrated that they could do nothing about

it. By contrast, in the IBF facilities, where payments were necessary for business plan priorities

according to proscribed procurement process, there may have been some frustration with the

pace at which these business processes were adopted, but less with how funds were actually

used.

Policy implications for future RBF initiatives

While there are a range of implications for any future RBF initiatives working with PNFPs, the most

significant of these will be the parameters attached to the use of results payments. While the

“hands off” approach made sense for the purposes of the study, incite would have come with

tracking how facilities use their funds. There is a general view among participating stakeholders

that such a carte blanche approach would be neither desirable nor politically acceptable in an

operational context. Defining how to regulate the use of results payments, without inadvertently

creating perverse incentives or other unintended consequences will warrant significant attention.

6 QUALITY OF CARE

Background

In NU Health’s ToC, Quality of Care (QoC) is a critical factor on the path to better health. For the

purpose of the study, QoC is considered in terms of availability of essential equipment and

supplies, the availability of qualified and skilled human resources and how those personnel

performed in terms of prescription practice and adherence to clinical guidelines. The study also

considered client satisfaction including verification of service provision.

19


QoC is dependent on the alignment of a range of critical inputs including adequate number, skills

and motivation of staff, as well as the availability of essential commodities that are necessary to

provide basic, routine health services and emergency treatment. To this end, the Ministry of Health

of Uganda has set out standards for the number of staff and the required skills mix for a given

level of facility to offer the required minimum level of health care delivery, without which the level

of quality of service will not be satisfactory. The baseline survey conducted by NU Health,

however, confirmed that many health facilities lacked essential drugs and equipment, as well as

the requisite number of appropriately skilled personnel for the particular level of service delivery

according to minimum government standards.

One of the inputs of the study was to increase availability, improve stock management and rational

use of essential medicines and health supplies (EMHS) in the 31 supported PNFP facilities in

both RBF and IBF regions. A credit line system for the EMHS was instituted with the Joint Medical

Stores, where the 31 PNFP facilities were able to order and receive EMHS directly to the facility.

During the first year of the study, laboratory and health facility equipment was procured and

distributed to category two facilities in both the RBF and IBF regions. The rationale was to ensure

that the level two facilities2 were in a position to provide basic laboratory and medical procedures

as a requirement of the minimum health care package hence creating a level ground to perform

the RBF/IBF experiment. The equipment provided in the seed grant included delivery beds,

autoclaves, delivery sets, drip stands, and examination couches, amongst others.

Having the requisite number and type of staff according to MoH norms was an inclusion criteria

for the study. The NU-Health human resource intervention is two pronged with direct support to

the District Health Team (DHT) through recruitment of vital staff and indirectly to the facilities

through the creation of a viable financial environment for the facilities to recruit the required

number of staff with the right skills mix. Ongoing availability of EMHS, equipment and appropriate

staffing levels is monitored regularly throughout the programme.

The programme established a range of verification mechanisms to monitor the data provided

by health facility managers. These are used to determine the level of funding received in RBF

facilities which is then matched for the IBF facilities. Firstly, a monthly check of the claimed

performance in service provision (Data Quality Assessment or DQA) is undertaken by external

verifiers from the programme in collaboration with the DHT. Secondly, to complement the DQA, a

Quarterly Quality Assessment (QQA) is conducted at each health facility and a quality score is

given. This score acts as an additional factor in calculating the level of funding attained by the

facility. A third mechanism of verification was Direct Client Verification (DCV) of a sample of clients

from each health facility who were contacted by mobile phone to verify whether or not they actually

received the service they were recorded as receiving. The main objectives of the verification were

to ensure the validity of facility reported data; complement Quarterly Quality Assessment (QQA)

data on client satisfaction; and understand from a patient viewpoint the cost incurred for different

services. The QQAs, undertaken by NU Health staff with DHT members, exposed on-going

2 These were lower level facilities which were close to but did not meet standard with respect to basic medical

equipment and infrastructure at the time of project start up.

20


problems with adherence to appropriate clinical practices and MoH standard protocols. NU

Health put in place a range of measures to address these weaknesses in the health system,

monitored the effects of these actions and subsequently assessed the impact on clinical practice

regarding common illnesses and client satisfaction.


Different methodologies were used to monitor the various aspects of quality of care:

Concerning availability of EMHS, health facilities were assessed on a quarterly basis to

see whether or not a particular item of equipment or commodity was available on the day

of the survey. Equipment availability was checked once a year and included in an

assessment of the level of functionality in order to monitor maintenance practices.

The level of staffing of qualified and skilled people was assessed annually according to

the MOH staffing norms for each level of care. A facility passes the required staffing

number and skills mix if they have the recommended number and personnel per position

at a given level of care.

Prescription practices for common childhood illnesses (malaria, pneumonia and

diarrhoea) are regularly monitored during the DQA/QQA process. In addition, they were

compared between the RBF and IBF health facilities during the baseline year (2011/12),

after one year of implementation (2012/13) and again after two years of implementation

(2013/14) through the conducting of a clinical audit. This audit also assessed the

competence of the most senior clinician to manage sick children by observing them during

up to ten consultations per health facility and scoring them according to the Integrated

Management of Newborn and Childhood Illnesses (IMNCI) parameters of good care. A

detailed methodology of the entire clinical audit study is provided in the full clinical audit

report (available on request).

Client satisfaction was gauged directly by conducting exit interviews with clients present

at the health facilities during the QQAs. This feedback was incorporated into the overall

QQA score. The DCV method was based on telephone interviews with clients sampled

from a range of clinic registers and followed a structured questionnaire guideline. The

interviews were conducted by independent non-clinical staff that were not engaged in the

usual day-to-day running of the field activities or engaged in clinical care, in order to

preserve confidentiality and avoid bias. The sample of patients who had attended services

relevant to NU Health was developed.3 A lower sampling percentage was used for

hospitals than in lower level facilities, with 1% of patients who received care at Hospital

level being sampled, 2% of patients who received care at HC3 level and 3% of patients

who received care at HC2. An exception was made for St. Mary’s Lacor Hospital that has

a large number of patients and as a result only 0.5% of eligible patients were sampled.

3 Cordaid, Sina (2010) PBF in action – theory and instruments. Performance Based Financing Course Guide. Cordaid.

21


Table 2 illustrates the methodology used for each component:

Table 2 Methodology Summary

Area of

assessment

Methodology Sample size / scope Period

covered

Essential

equipment/

supplies

Facility

Survey

- Assessment of the availability and functionality of

selected equipment once annually:

i) delivery beds; ii) sterilisation equipment; iii) Rapid

Diagnostic Tests (RDTs); iv) ORT Corner; v)

microscope;

vi) oxygen cylinder; vii) oxygen delivery

equipment/oxygen concentrator

-Availability of 20 Essential Medicines and Health

Supplies

-31 health facilities (21 Acholi/RBF & 10 Lango/IBF)

2012/13/14

Skilled human

resources

Facility

Survey

- 31 health facilities (21 Acholi/RBF & 10 Lango/IBF)

- Staff availability according to MoH norms for each level

of care

2012/13/14

Prescription

habits

DQA/QQA

Clinical audit

IMNCI

31 health facilities (21 Acholi/RBF & 10 Lango/IBF).

Quarterly verification of data and quality scored against

comprehensive set of indicators (see NU Health’s DQA

and QQA tools)

Review of clinical practices at baseline, and after one

and two years of implementation using three cross

sectional studies for malaria (n=2,911), pneumonia

(n=1,520) and diarrhoea (n=2,608) case management

from Health Management Information System data in

the 31 PNFPs.

Observation of 247 child examinations in RBF PNFPs

and 105 examinations in IBF PNFPs.

2012/13/14

2011/12

2012/13

2013/14

2014

Client

Satisfaction

Exit

interviews

during QQA

Direct Client

Verification

Around 200 interviews held on average each quarter

from PNFPs assessed in that quarter with a total of 1520

interviews to date.

1,995 records were collected across all health facilities.

2012/13/14

Feb to Oct

2014

Findings

Highlights

Overall availability of EMHS items improved from 72% at baseline to 97% in the latest

assessment.

Average availability of seven items of essential equipment increased from 78% in 2013 to

82% in 2014.

22


Availability of appropriately skilled staff increased from 52% to 90% in the RBF region and

from 70% to 100% in the IBF region.

Average QQA scores increased during programme implementation from around 60% to 80%

in the RBF region and from under 50% to over 60% in the IBF region, with the highest

increases occurring at HC3 level.

A clinical audit showed that after two years of programme implementation and after adjusting

for confounding factors, compared to a child in the IBF region, a child in the RBF region was:

three times more likely to be treated correctly for malaria;

almost seven times more likely to be treated correctly for pneumonia; and,

over eight times more likely to be treated correctly for diarrhoea.

Clinical management of sick children, assessed using the IMNCI checklist, improved over

time in both RBF and IBF regions.

The direct client verification exercise to assess satisfaction revealed that 78% of those

contacted were satisfied with the service received.

Availability of EMHS and Equipment

The average availability of EMHS items improved to 96.5% in the most recent assessment with a

slightly higher availability recorded in the intervention RBF facilities (97%) compared to in the IBF

facilities (96%). This is higher than the overall availability results in the baseline and the first

Lessons Learned (LL) report which were 72% and 94% respectively.

All RBF facilities and 90% IBF facilities had over 90% of the basket of EMHS available on the day

of the survey; this is an improvement from the 81% and 70% in the last LL report respectively.

This is presumably still a direct reflection of the impact of monetary incentives which are provided

to facilities that score 100% in availability of the priority EMHS in the RBF health facilities during

the QQA. It is, however, worth noting that despite the provision of incentives to facilities that score

highly in the pharmaceutical indicators, some RBF facilities have failed to score 100% while some

facilities in the IBF region have scored 100%. The good performance in the IBF region can be

attributed to the regular supportive supervision and the credit line which is provided in both arms

of the study. The stock-out incidence of Insecticide treated mosquito nets and vitamin A has

continued to be registered in the IBF region with their stock out rate at 30% and 20% respectively.

Three commodities registered the highest incidence of stock outs in the RBF region, namely;

mosquito nets, suphadoxine-pyramethamine (SP) and measles vaccine, all at 10% on the day of

the survey. It is worth noting that the EMHS which were out of stock are those that are provided

fully by the government structures. There is sometimes a break in the supply chain of these

commodities hence rendering a stock-out in ill-prepared facilities. Availability has continued to be

better at higher level facilities compared to the lower level facilities; this is due to a better skill set

of pharmacists, dispensers and trained pharmacy staff who run this department. Hospitals also

have much bigger budgets than lower health facilities and prioritization is given to good

pharmaceutical practices.

In both the first (2013) and second (2014) annual follow-up rounds of assessment, availability of

essential equipment was highest in hospitals and decreased by level of care. Hospitals and HC4

had a 100% availability of the seven commodities assessed, followed by HC3 and lastly HC2. The

23


average availability of equipment was 78% in 2013 and increased to 82% in 2014 with a higher

availability recorded in the intervention RBF facilities on both occasions (81% and 86% in the first

and second rounds respectively) compared to the control IBF facilities (78% and 81%

respectively). This minimal difference in availability may reflect the impact of monetary incentives

in the RBF health facilities. Oxygen cylinders, oxygen delivery equipment and oxygen

concentrators were least available in both occasions for both the RBF and IBF regions. It is

presumed that this reflects the relative cost of these items which meant only hospitals and large

HC3s purchased them. Sterilizers, delivery beds, RDTs and ORS were provided as a seed grant

and quarterly through the credit line; this explains their high percentage availability in both rounds

of assessment.

Figure 2 Availability of equipment in the intervention and control facilities

Availability of qualified and skilled human resources

The findings showed that there were tremendous improvements in the recruitment and retention

of skilled staff for both the RBF and IBF health facilities. The Lango region demonstrated a better

staffing skills mix for 10/10 (100%) of the facilities compared to 18/20 (90%) in the Acholi RBF

region. Both regions registered an improvement from the first LL period from 52% and 70%

respectively for RBF and IBF regions. St Francis Akia, PAG and Aber hospital that were lagging

in the last LL period have since improved in their recruitment for the required personnel and skills

24


mix. In the RBF Acholi region the recommended staffing and skills mix was recorded in 90% of

the facilities with gaps only noted in the lower level facilities of Wi Anaka and St Luke HC2, which

both lack a midwife and yet conduct deliveries. It is worth noting that there is no association

between funding mechanism and the adequate numbers of the appropriate skilled staff. All

hospitals and HC3 across both the RBF and IBF regions had the required number of staff and skill

mix to offer the minimum standard of care.

Prescription practices in health service delivery

QQA Scores

The QQA is one of the tools used to monitor the provision of quality services: it includes an

assessment of the quality of ANC services, delivery care, under-five Out-Patient Department

(OPD) services, immunisation, general OPD services, hygiene and infection prevention,

laboratory standards, pharmaceutical management, inpatient care and data management. The

QQA scores obtained by PNFPs have gradually increased over time as shown in the graphs

below. The most significant improvement has been made at the HC3 level in both regions while

hospitals have made the least impressive progress. While this assessment looks at broader quality

issues rather than simply clinical practice, it includes areas such as use of the partograph,

appropriate treatment of malaria, pneumonia and diarrhoea cases, and rational antibiotic use.

Figure 3 QQA scores in Acholi/RBF region Figure 4 QQA scores in Lango/IBF region

Clinical Audit

The clinical audit found discernible improvement after two years of implementation for all clinical

practices assessed by the audit. These changes occurred mostly in the second year, and were

generally more pronounced in the RBF health facilities. The key findings regarding prescription

practices for common illnesses are described below, along with a summary of IMNCI-related

clinical practice.

Malaria

A malaria prescription was considered correct when each definitive diagnosis was supported by

either positive microscopy or Rapid Diagnostic Test (RDT) and the choice of medication matched

the illness classification. The first line of treatment for uncomplicated malaria is a combination of

artemether and lumefantrine (Ministry of Health Uganda, 2010). Until recently, the first line for

complicated malaria was oral or injectable quinine but artesunate is now recommended as a safer

and more efficacious option (Dondorp, et al., 2010).

25


At baseline and during year one and two of implementation, the distribution of correctly treated

malaria cases was different between RBF and IBF health facilities. In all periods, the proportion of

correctly treated cases was higher in the RBF setting and the difference between RBF and IBF

widened in the second year of implementation.

Figure 5 Proportion of correctly treated malaria cases in RBF and IBF regions

Multivariate analysis looked at the effect of age and sex of the child, and level of health facility:

only the latter was a confounding factor. After adjusting for this effect, the odds of a child being

correctly treated showed an increasing trend from baseline (Odds Ration [OR] 1.71; Confidence

Interval [CI] 1.21-2.40) through to year one (OR 1.80; CI 1.33-2.44) and year two (OR 3.15; CI

2.13-4.65) in the RBF as compared to the IBF region. These differences were statistically

significant.

Figure 6 Trends in correct treatment of malaria cases at different facility levels in RBF and

IBF regions

In summary, after adjusting for confounding factors, a child in the RBF region was three times

more likely to be treated correctly for malaria than a child in the IBF region after two years of

26


programme implementation. This suggests that the RBF mechanism is having a positive effect on

influencing clinical practice of health care providers.

Pneumonia

The national clinical guidelines recommend that each child with a diagnosis of pneumonia should

receive an antibiotic and Vitamin A supplementation (Ministry of Health Uganda, 2010). At

baseline and during year one and two of implementation, the distribution of correctly treated

pneumonia cases varied between RBF and IBF health facilities. In all periods, the proportion of

correctly treated pneumonia cases was higher in the RBF setting, and the difference between RBF

and IBF widened in the second year of implementation

Figure 7 Percentage of pneumonia cases treated correctly in RBF vs IBF regions

Multivariate analysis looked at the effect of age and sex of the child, and level of health facility: as

was the case for malaria, only the latter was a confounding factor. After adjusting for this effect,

the odds of a child being correctly treated were still higher in the RBF region when compared to

IBF at baseline (OR 22.14 CI 7.75-63.24), year one (OR 4.16 CI 2.80-6.17) and year two (OR 6.63

CI 3.34-13.17). These differences were statistically significant.

27


Figure 8 Trends in correct treatment of pneumonia cases at different facility levels in RBF

and IBF regions

In summary, after adjusting for the influence of the health facility level, a child in the RBF region

was almost seven times more likely to be treated correctly for pneumonia than a child in the IBF

region after two years of programme implementation. This suggests that the RBF mechanism is

having a positive effect on influencing clinical practice of health care providers.

Diarrhoea

The national clinical guidelines recommend that each child with diarrhoea should receive

rehydration therapy and Zinc supplementation (Ministry of Health Uganda, 2010). At baseline and

during year one and two of implementation, the distribution of correctly treated diarrhoea cases

varied between RBF and IBF health facilities. In all periods, the proportion of correctly treated

diarrhoea cases was higher in the RBF setting and the difference between RBF and IBF widened

in the second year of implementation

Figure 9 Percentage of diarrhoea cases treated correctly in RBF vs IBF regions

28


Multivariate analysis looked at the effect of age and sex of the child, and level of health facility:

once again, only the latter was a confounding factor. After adjusting for this effect, the odds of a

child being correctly treated were higher in the RBF compared to the IBF region at baseline (OR

2.41; CI 1.71-3.39), in the first year of implementation (OR 5.34 CI 3.84-7.42) and in the second

year (OR 8.34 CI 4.95-14.08). These differences were statistically significant.

Figure 10 Trends in correct treatment of diarrhoea cases at different facility levels in RBF

and IBF regions

In summary, after adjusting for the influence of the health facility level, a child in the RBF region

was over eight times more likely to be treated correctly for diarrhoea than a child in the IBF region

after two years of programme implementation. This suggests that the RBF mechanism is having

a positive effect on influencing clinical practice of health care providers.

IMNCI-related Clinical Practice

Clinician competence in both RBF and IBF health facilities was observed and assessed against

IMNCI parameters of good clinical practice (assessment tool available in clinical audit report)

during consultations in OPD. The first round of observations for each health facility was conducted

in early 2014 and subsequently repeated after a minimum period of three months. The scores are

given for a total of 227 cases observed in the RBF and 74 in the IBF health facilities.

Table 3 below summarizes the number of cases seen in each health facility by level of care and

region during the second year of implementation and shows the summary of scores for the three

rounds of assessment in the RBF region and two rounds in the IBF region.

29


Table 3 Summary of overall scores in clinical management of sick children in OPD using

IMNCI checklist in the RBF & IBF health facilities

RBF % (range) IBF % (range)

Type of

Health

Facility Round 1 Round 2 Round 3 Round 14 Round 2

HC2

54 (22-83

n=53)

57 (31-92

n=42) 64 (54-85 n=15) 38 (n=10) 71 (61-82 n=20)

HC3

53 (37-80

n=18)

80 (65-89

n=19) 74 (65-80 n=20) 48 (n=5) 64 (47-89 n=22)

Hospitals

36 (13-48

n=20)

69 (53-68

n=20) 68 (53-84 n=20) 79 (75-83 n=2) 66 (62-71 n=15)

In general, there was an improvement among all health facilities in the RBF group from the

assessment in the first round to the second round, which was more or less maintained in the third

round. In the IBF health facilities, improvement between the first and second rounds was observed

only at HC2 and HC3 level of care but not in hospitals. The apparent lack of improvement on

average at hospital level in the IBF hospitals is attributed to the lower performance by Aber

Hospital in the second round where high staff turnover seems to have influenced the lack of

continuity in adhering to the recommended clinical practices.

The following graphs compare the trends in changes and shows that the improvements have been

equally impressive (and more so in the case of HC2 level) in IBF facilities compared to RBF

facilities, apart from hospital level as noted above. The effect of observation itself, rather than the

incentive mechanism in RBF facilities, is therefore likely to be the main reason for improvements

although there could be additional external factors influencing change that are unrelated to the

programme effect. In any case, the third round of assessment in RBF facilities shows that these

improvements may not be maintained as there was a levelling off in average scores at HC3 and

hospital level. This could be due to some level of complacency after a period of time or the limited

capacity of the clinical staff to improve without more training.

4 Only one facility each was included at HC2 and HC3 levels.

30


Figure 11 Changes in average IMNCI assessment scores across RBF and IBF health

facilities at HC2, HC3 and hospital level

Client Satisfaction

In the DCV exercise, 40% of 1,995 patients sampled could be contacted by telephone. The two

most common reasons for unsuccessful calls were that the number was not available on the

network/did not connect (71.7%) or that the phone was switched off (13.5%). When a number

could not be reached, it was dialled again twice over the next 24h. The vast majority of clients

(78%) who were contacted, reported that they had received the service in question and that they

were satisfied with the service. Reasons given for satisfaction included provision of

comprehensive services and positive staff attitudes. Reasons for dissatisfaction with services

included long waiting times, overcrowding of facilities, lack of medicines and poor facility

upkeep/cleanliness. Information on costs incurred highlighted a wide range of fees; however, there

was insufficient detail gathered to draw conclusions. More in-depth analysis would be required to

achieve this, such as comparing receipts given to patients with health facility accounting records.

A broader look at the client satisfaction is outlined in the recent NU Health Client Verification

report, December 2014).

Policy Implications for Future RBF Initiatives

The widespread improved availability of essential equipment, drugs and other supplies is

associated with positive changes in prescription practices and adherence to national protocols,

particularly in the RBF region. It is also associated with high levels of client satisfaction in general.

As the availability of commodities was similar and consistent in both RBF and IBF regions, the

significant improvement in malaria, pneumonia and diarrhoea treatment practices suggests that

RBF may specifically contribute to some aspects of improved clinical behaviour. In the first LL

period, compliance to recommended malaria treatment was not significantly better in the RBF

region compared to the IBF region and this was attributed to malaria requiring more complex

31


management. However, over time this improved substantially in the RBF region, suggesting that

one year is too short a time period for more extensive change in clinical practice to take place.

The improvement noted in the use of recommended treatment protocols for malaria, diarrhoea

and pneumonia in the RBF relative to IBF region is not surprising. This observation is also

supported by the generally positive trend in compliance to good clinical practice among clinicians

seeing children aged less than five years in the Out-Patient Department during the IMNCI

assessment. The scale of improvement is quite large, illustrating how quickly incentives in the

RBF context can turn around quality of care for a range of clinical practices. Secondly, the rapid

improvement for lower level facilities relative to hospitals is attractive for strategies such as

universal health coverage that aim to increase access to quality health care with a relatively low

additional level of inputs.

As noted in the clinical audit report after year one, there is clearly scope to stimulate change in

ways other than using RBF mechanisms since improvements were also observed in the IBF

region. Important inputs in this case include regular supportive supervision, and a regular supply

of essential drugs and other commodities to all health facilities.

Concerning client verification, a high level of concurrence between health facility records and client

feedback was observed. It was also established that where records are available, client verification

can be conducted using telephone calls. Furthermore, the methodology used with telephone-

based validation was documented, providing a framework that might be adopted for use in similar

settings. However, given the current pattern of phone ownership and network coverage, this

approach will not likely provide a statistically representative indication of the experience of the

overall patient/client population.

Identifying specific reasons for satisfaction or dissatisfaction with services requires detailed

probing as clients generally report high levels of satisfaction which may simply be reluctance to

criticise people of power. Understanding the specific needs and desires of clients could be helpful

in orienting health care providers to be more responsive and considerate towards their clientele.

The DCV method can, therefore, complement other measures of eliciting client satisfaction by

providing more detailed information so long as its current biases are taken into account.

Institutionalising the use of telephones as a means of validation of service provision and

performance may have a place in low resource rural settings. The MoH has already revised

registers to include a phone contact. Further investigation, however, is in order to assess whether

the relatively low response rates have biases in phone based DCV findings. If Uganda adopts

plans to take RBF to scale, the cost of verification will need to be considered very seriously. A

comparison of phone versus the gold standard of physical verification in the client’s home might

provide an indication, even on a flawed sample basis, of whether phone-based DCV is good

enough to fulfil its purpose.

32


7 HEALTH OUTCOMES

In this section NU Health examines three aspects of health outcomes, namely: disease

presentation patterns for clients attending health facilities in the RBF and IBF regions (case load);

the client profile by age and sex (case mix); and perinatal outcomes from a clinical audit.

Background

Uganda still faces a high burden of preventable communicable diseases in the vulnerable

populations of women and children aged less than five years as well as a growing problem with

non-communicable disease. Amongst many other factors, quality of care is an important supply

side barrier that influences the use of services by the general population (Ministry of Health

Uganda & Macro International, 2007). For example, there is a known association between

improved monitoring during labour and better perinatal outcomes (Tayade & Jadhao, 2012); yet

use of the partograph, a key tool in monitoring the progress of labour, is still low in Northern

Uganda.

The NU Health programme targets the greatest weaknesses in the health system and the most

commonly occurring illnesses by providing inputs to essential medicines and commodities and

supportive supervision to promote adherence to the best clinical practices. This is enhanced by

additional financing coupled with facility autonomy that is bound in an agreement to ensure the

maintenance of basic health system inputs for participating providers.

Assessment methodology

The electronic DHIS2 that captures data at the district health office from all facility based providers

was used as the source to examine disease ranking patterns and changes in numbers of clients

attending services at NU Health supported health facilities from baseline (2011/12) through the

first year (2012/13) and second year (2013/14) of implementation.

The practice of labour monitoring and associated perinatal outcomes was assessed in a clinical

audit which examined records from the previous three years in all NU Health-supported PNFPs.

Cases constituted a sample of mothers in each year with perinatal death (neonatal death, fresh

still birth, macerated still birth) as an outcome. The cases were matched with controls with live

births of similar maternal age and parity. The study looked at correlations with the level of partial

or complete monitoring using partographs.

Findings

Disease Presentation Patterns

Clients attending the health facilities supported by NU Health in both the Acholi (RBF) and Lango

(IBF) regions were ranked by age and sex for the top ten most prevalent conditions in the baseline

year 2011/12, the first year of implementation 2012/2013 and the second year of implementation

2013/2014. The data presented have been extracted from DHIS2 and are been based on OPD

figures reported by the facilities.

33


Children

Among children aged less than five years, the top ten conditions were similar for both boys and

girls and in the two regions (tables below), especially after injuries dropped from the girls’ ranking

after the baseline. Malaria, pneumonia and diarrhoea remain among the top five conditions in both

regions over the last three years. In the Acholi region, the ranking for the top five conditions was

consistent across the three years for both boys and girls with pneumonia and cough being more

prominent and malaria less prominent in the last year. This is consistent with the findings of the

clinical audit which shows improving clinical management of malaria in RBF facilities. The ranking

for the top five conditions was similar in the Lango region except that anaemia featured more

prominently, being ranked as either the 2nd or 3rd for all three years for both boys and girls. Malaria

remained the most prevalent disease in Lango for boys in the latter two years and for girls for all

three years.

Table 4 Top 10 ranking for U5 boys over three years

Acholi Lango

Disease 2011/12 2012/13 2013/2014 2011/12 2012/13 2013/2014

Cough or cold 1 1 2 4 3 3

Malaria 2 2 4 2 1 1

Pneumonia 3 3 1 5 5 5

Acute Diarrhoea 4 4 3 1 4 4

Skin diseases 5 5 5 6 7 6

Ear, Nose,

Throat (ENT)

conditions

6 10 7 10 6 10

Intestinal Worms 7 6 8 9 8 8

Urinary Tract

Infections

8 9 10 8 11

Other eye

conditions

9 8 6 10

Anaemia 10 3 2 2

Gastrointestinal

disorders (non-

infective)

7 9 7 9 7

Table 5 Top 10 ranking for U5 girls over three years

Acholi Lango

Disease 2011/12 2012/13 2013/2014 2011/12 2012/13 2013/2014


Malaria 2 2 4 1 1 1

Pneumonia 3 3 2 5 5 5

Acute Diarrhoea 4 4 3 4 4 4


Ear, Nose,

Throat (ENT)

conditions

6 10 7 8 6 9

34



Urinary Tract

Infections

9 10 10 10

Other eye

conditions

9 8 6 10

Anaemia 10 2 2 3

Gastrointestinal

disorders (non-

infective)

7 9 7 8 7

Injuries 8

Adults

The most prevalent health conditions among adults in the two regions (tables below) are malaria

and cough although anaemia and diabetes are prevalent in Lango and to a lesser extent in Acholi,

while pneumonia and cardiovascular diseases are more prevalent in Acholi and less so in Lango.

Reproductive health problems (Pelvic Inflammatory Disease) represent an increasing non-

communicable condition reported among adult female patients in both regions over the last three

years. This pattern of conditions remains consistent with the emerging national profile of a double

burden of both communicable and non-communicable illnesses.

Table 6 Top 10 ranking for male adults over three years

Acholi Lango

Disease 2011/12 2012/13 2013/2014 2011/12 2012/13 2013/2014


Malaria 2 2 2 1 1 2

Injuries 3 3 3 10 8 5

Pneumonia 4 8 8


Urinary Tract

Infections

6 6 5 6 6 6

Anaemia 2 3 1

Gastrointestinal

disorders (non-

infective)

4 4 7 7 4


Cardiovascular

diseases

8 10 10

Hypertension 9 9 9 4 4 7

Diabetes Mellitus 5 5 9

Acute Diarrhoea 10

35


Table 7 Top 10 ranking for female adults over three years

Acholi Lango

Disease 2011/12 2012/13 2013/2014 2011/12 2012/13 2013/2014


Malaria 2 2 4 1 1 1

Injuries 4 9 9 10

Pneumonia 6 10 10


Skin diseases 7 7 7 9 9

Urinary Tract

Infections 5 5 5 6 6 4

Anaemia 2 2 8

Gastrointestinal

disorders (non-

infective)

3 2 3 5 2

Cardiovascular

diseases 9

Hypertension 10 8 6 5 4 5

Diabetes

Mellitus 7

Pelvic

Inflammatory

Disease (PID)

8 4 3 8 8 6

Malaria in

Pregnancy 7 7

Acute Diarrhoea

Rankings have been relatively consistent over the three years, particularly for children. Some

exceptions have occurred in adults in relation to non-communicable diseases as noted above. A

hypothesis around ranking was that initially more vulnerable children with illnesses constituting

the larger burden of disease would attend health care services at facilities supported by NU Health

and increasingly occupy the top rankings. Although in the longer term, one might see a reduction

in illnesses that are being better managed and where more effective prevention measures are

being provided, NU Health is not conducting enough targeted preventive level efforts to claim any

attribution in the short to medium term.

The information in the DHIS2 used to rank health-related conditions is reported data and, although

discrepancy rates between reported and verified data have improved significantly over time, they

are still subject to variation, especially in the IBF region where hospital/HC4 level data currently

shows a discrepancy rate of -19%. It is possible that the changes in rankings observed reflect

seasonal variations in the prevalence of illness and that the time period studied may also be too

short to draw substantive conclusions. It is furthermore unclear whether these changes observed

are in any way related to the funding mechanism or to changes in the quality of care provided or

other contributory factors.

36


The data used reflect service statistics from all NU Health supported facilities in the two regions. These facilities provided approximately 17% of overall services delivered in Acholi and just under 10% in the Lango sub-region after one year of implementation. In the second year of implementation, these proportions fell in both regions to 12% and just under 6% as shown below in Table 8.

Table 8 Contribution of OPD Consultations by NU Health Supported Health Facilities in

Acholi and Lango

OPD contribution of 20* PNFP to each Acholi District

(FY13-14) from DHIS2

0-4 Years 5 and Over Total

Agago 8.77% 8.62% 8.66%

Amuru 30.62% 7.82% 15.16%

Gulu 27.83% 12.98% 16.68%

Kitgum 17.51% 13.08% 14.22%

Nwoya 8.36% 3.22% 4.45%

Pader 2.84% 1.41% 1.78%

Average 19.54% 9.74% 12.33%

OPD contribution of 10 PNFP to each Lango District (FY13-14)

0-4 Years 5 and Over Total

Alebtong 4.49% 3.73% 3.94%

Apac 1.11% 0.56% 0.66%

Lira 11.32% 8.20% 9.03%

Oyam 6.13% 6.70% 6.58%

Average 7.02% 5.21% 5.63%

*It should be noted that the FY13-14 figures in Acholi are based on 20 and not 21 PNFP due to

the suspension of St Joseph Minakulu HC2.

When comparing the two years of data, we see that OPD attendance has increased slightly in the

Acholi districts (8%) but much more significantly (51%) in the Lango districts while OPD

attendance in PNFP supported by NU Health has decreased in both regions. Clearly this is a

matter of concern as one of the objectives of NU Health is to increase access to services and thus,

we would expect the OPD numbers to increase. However, these figures are based on DHIS2 data

and our assumption is that the DHIS2 OPD data of FY12-13 were seriously inflated while the

FY13-14 ones are more realistic due to the NU Health intervention and for rewarding/stimulating

more reliable reporting of data.

37


Table 9 Service Utilisation Change

ANALYSIS OPD ATTENDANCE FY12-13 versus FY13-14

AT DISTRICT LEVEL FY12-13 FY13-14 DIFFERENCE ∆%

6 districts in Acholi 1,893,473 2,051,957 158,484 8%

4 districts in Lango 1,101,675 1,667,525 565,850 51%

AT PNFP LEVEL FY12-13 FY13-14 DIFFERENCE ∆%

20 Acholi PNFP 309,800 252,991 -56,809 -18%

10 Lango PNFP 107,394 93,827 -13,567 -13%

In support of this, the figures below shows the verified OPD figure trend which has increased

substantially in Acholi PNFP but less so in Lango facilities:

Figure 12 Verified OPD numbers

Client Profile

The patients attending health facilities participating in the NU Health programme in both regions

were compared by age and gender for the top fifteen most prevalent health conditions in the

baseline year 2011/12 and the two years of implementation 2012/2013 and 2013/2014. Error!

eference source not found.In both regions, adult males formed the lowest proportion of

attendance (figure below), except in Lango region during the last year where they represent the

second largest group. The relative contribution of each of the gender and age categories remained

similar from baseline to end of year two in the Acholi region, apart from a slight increase in the

adult female category over the three years. In the Lango region there was a sharp drop in the

proportion of younger (<5 years) males reported to have attended health care services from

baseline (49%) to end of year one (21%); this reducing trend continued in year two (18%) but was

less pronounced. There was also a sharp increase in the proportion of adult females (>5 years)

reported to have attended health care services from baseline (26%) to the end of year one and

year two (to 39%). There is no obvious association between these changes and the funding

mechanism.

38


Table 10 Client Profile by region, age and gender based on the top fifteen prevalent health

conditions

RBF IBF

2011-12 2012-13 2013-14 2011-12 2012-13 2013-14

Fem 24% 29% 31% 26% 39% 39%

Male 17% 17% 16% 12% 19% 26%

Girl <5 29% 26% 24% 13% 21% 17%

Boy <5 30% 28% 29% 49% 21% 18%

U5 Male29%

U5 Fem24%

Adult Male16%

Adult Fem31%

Client profile of 20 Acholi PNFPFY 13-14

U5 Male18%

U5 Fem17%

Adult Male26%

Adult Fem39%

Client profile of 10 Lango PNFPFY 13-14

39


Perinatal Outcomes

Neonatal mortality, defined as death of an infant during the first 28 days of life, is a major aspect

of infant and child mortality. In 2013 it accounted for 44% of deaths in children under the age of

five years globally5 and 33% in Uganda6. The bulk of neonatal deaths occur during labour or in

the first week of life, with half of these occurring within the first 24 hours after birth. About 70% of

these deaths could be prevented if emergency and obstetric care interventions were made

available and used by 90% of mothers and babies (Lawn and Kerber, 2006). Instituting and

maintaining the correct use of partographs has been associated with up to 50% improvement in

both maternal and perinatal outcomes (WHO, 2012).

The clinical audit undertaken in NU Health assessed the use of the partograph during labour and

its association with perinatal outcomes. The figure below illustrates that the overall level of

complete labour monitoring improved over time and this was more pronounced for mothers (both

cases and controls) in the RBF region. In the IBF region, improvements were higher for live births

(controls) than for perinatal deaths (cases) but the absolute levels of monitoring were lower for

both groups compared to the RBF region.

Figure 13 Percentage of labour cases (perinatal deaths) and controls (live births) that

received complete monitoring with a partograph

Figure 14 below further summarises the comparison in performance between RBF and IBF regions

for any type of labour monitoring among mothers with perinatal deaths (cases) and those with live

births (controls) from baseline to year 2 of implementation. In both the RBF and IBF regions, there

was steady improvement in any type of labour monitoring for cases and controls. Overall, the

improvements in absolute levels of labour monitoring were consistently better in the RBF region

although the relative improvements were also impressive in the IBF region.

5 UNICEF 2014, http://data.unicef.org/child-mortality/neonatal 6 Countdown to 2015 - Uganda 2014 report

http://data.unicef.org/child-mortality/neonatal

40


Figure 14 Percentage of labour cases (perinatal deaths) and controls (live births) that received any

monitoring (partial or complete) with a partograph

The difference in means calculation (summarised in the table below) suggests that after the first

and second years of implementation, there were increases in the proportion of all mothers (cases

and controls combined) that had been completely monitored during labour. The increases in labour

monitoring were more pronounced in the RBF region.

Table 10 Difference in means calculation for complete monitoring of labour progress combined for

mothers with live births (controls) and perinatal deaths (cases) in the RBF and IBF health facilities

RBF IBF

Mean % with

Confidence

Interval

% Difference in

means between

years

Mean % with

Confidence

Interval

% Difference in

means between

years

Baseline

(2011/2012)

9.3 (7.1-11.7) 0.0 0.0 0.0

Year 1

(2012/2013)

27.2 (22.7-31.6) 27.2 2.5 (1.0-4.0) 2.5

Year 2

(2013/2014)

54.4 (49.6-59.2) 28.0 19.9 (15.3-24.5) 17.4

Total after two

years

55.2 19.9

As summarised in Table 11 the difference in mean increase of the proportion of all mothers that

received any monitoring during labour was slightly more in the IBF region after two years.

41


Table 11 Difference in means calculation for any monitoring of labour progress for mothers with

live births (controls) and perinatal deaths (cases)

RBF IBF

Mean % with

Confidence

Interval

% Difference in

means between

years

Mean % with

Confidence

Interval

% Difference in

means between

years

Baseline

(2011/2012)

19.2 (16.2-22.3) 0.0 0.0 0.0

Year 1

(2012/2013)

36.2 (31.4-40.9) 17.0 26.9 (22.7-31.0) 26.9

Year 2

(2013/2014)

64.0 (59.4-68.7) 27.8 50.2 (44.4-56.0) 23.3

Total after two

years

44.8 50.2

In summary, at baseline in the RBF region, mothers with perinatal deaths were more likely to have

received complete or any (including partial or complete) labour monitoring compared to those with

live births. In the IBF region there was no evidence that either mothers with perinatal deaths or

live births had been partially or completely monitored during labour as no partograph records were

found. During the first and second years of implementation, although there were substantial

increases in the overall level of monitoring during labour, there was no significant difference in the

level of any or complete labour monitoring between mothers with live births and those with

perinatal deaths in the RBF region. Similarly there was no significant difference between the level

of any or complete labour monitoring for live births and perinatal deaths in the IBF region for the

first year, but in the second year, mothers with perinatal deaths were only half as likely to have

received complete or any monitoring during labour as compared to those with live births.

Policy Implications for Future RBF Initiatives

There is a considerable and growing burden of non-communicable diseases, as well as the

existing communicable ones and RBF programmes targeting adult populations will increasingly

need to take this into account. The general trend emerging in NU Health-supported PNFPs shows

an increase in the volume of services which supports the hypothesis that improvements in quality

are associated with increased use. Better perinatal outcomes are associated with improved

monitoring during labour and the clinical audit demonstrated that complete labour monitoring

improved substantially after two years of implementation. The lack of variation between mothers

with live births and perinatal deaths is suggestive of systematic institutionalisation of the practice

in the RBF region. The relatively lower monitoring of mothers with perinatal deaths compared to

those with live births in the IBF region could potentially be explained in part by the lack of incentive

for faster change. The quality of labour monitoring in the RBF health facilities was monetarily

incentivised whereas this was not so in the IBF region. Lack of consistent use of the partograph

as a monitoring tool to assess progress during delivery should be of serious concern to health

managers and facility staff, as it prevents mothers receiving quality obstetric care and endangers

the lives of both mothers and babies.

42


8 VALUE FOR MONEY

The UK’s National Audit Office (NAO) defines good value for money as “the optimal use of

resources to achieve the intended outcomes”, where ‘optimal’ is taken to be “the most desirable

possible given expressed or implied restrictions or constraints.” (NAO)

To achieve the optimal use of resources, value for money approaches tend to be guided by

aspects of economy -spending less, efficiency - spending well, and effectiveness - achieving

greatest possible outcomes from output, or, spending wisely. These concepts are now the

generally accepted starting point for Value for Money (VfM) analysis, along with the cross-cutting

considerations of equity and sustainability.

Economy Getting the best value inputs

Efficiency Maximising outputs for a given level of inputs

Effectiveness Ensuring that the outputs deliver the desired outcome

Equity Ensuring that the benefits are distributed fairly (ICAI 2011)

Cost

effectiveness

A measure of impact that the programme interventions achieve

relative to inputs

Sustainability A programme approach which aims to achieve high intervention

coverage as a means to lasting impact

£ Inputs Outputs Outcomes Impact

Economy Efficiency

Effectiveness

Cost Effectiveness

Equity

43


VFM can be seen as a simple, but continuous, iterative process of identifying and implementing

further efficiencies and cost savings, applied consistently to an intervention to achieve lasting

impact, while maintaining programme effectiveness.

Value for Money (VfM) is also important for public accountability, and particularly in the Ugandan

context for maximising the prospects towards achieving Universal Health Coverage (UHC). In

addition to increasing absolute levels of investment and expenditure, countries like Uganda need

to ensure that the existing investment/expenditure is well utilised and leads to the desired

outcomes in terms of approaching UHC in addressing people’s health needs in an affordable and

sustainable way.

The aim of the NU Health study is to generate greater evidence about costs and benefits of RBF

as a means of achieving better VfM and accountability in health. The theory is that aligning

incentives will stimulate improved performance within the health system, while accountability is

ensured by making the ‘incentive to performance’ link visible to beneficiaries. Therefore, to

demonstrate what is being achieved at what cost, NU Health monitors financial inputs provided to

both RBF and IBF facilities, and the associated health outputs and outcomes.


This section aims to take a broad look at the VfM ‘Three ‘e’s’ principles to draw out:

What limitations are built into the programme design which would restrict full VfM

analysis?

How can each of the ‘three ‘e’s’ be applied in a NU Health context?

What programmatic lessons have emerged?

What are the implications for policy development stemming from the programmatic

example?

Throughout the review the NAO’s analytical framework on VfM depicted below, will underline the

review.

44


Figure 15 - NAO Analytical Framework for assessing Value for Money

Limitations

VfM – A controlled study vs a national programme? The NU Health programme provides a

degree of verification, data collection and analysis which is significantly more robust and resource

intensive than would be anticipated in a more routine ‘implementation only’ model, or indeed a

national system. This not only increases the financial inputs but also, adds a confounding factor

to the overall impact assessment. It is unclear the degree to which the frequent, intensive and

interactive verification visits in themselves incentivise improved quality and performance. The

assumption is that by comparing IBF with RBF the impact of the financial incentive can be

measured. However, at a national level the budgetary implications of the verification process may

limit the replicability of the study at national programme level.

Additional factors which are built in as controls for the NU Health programme must be examined

when considering a possible national level intervention. A few examples include:

Impact of the timeliness of payments on the facilities’ investment decisions and service

provision – what is the minimum level of financial responsiveness required to incentive

improved performance?

Impact of predictable and appropriate drug credit line – what is the minimum standard for

drug supply management for a facility to function effectively, and how long after a

predictable drug supply has commenced, do health facilities begin to make longer term

investment decisions?

Impact of DHT level engagement – besides the supervision question, to what extent does

the coordination function of the DHT add to the facilities ability and willingness to improve

45


performance, and to what extent does the DHT’s official role allow for time to support and

coordinate facility performance?

Control or oversight of inputs: It is worth noting that in the Acholi (RBF) group, NU Health did

not have either control over, or feedback about, the choice of inputs or investment options made

by the facilities, besides the credit line which was directly controlled by the programme. In the

Lango (IBF) group the programme had both veto control if expenditures did not align to previously

agreed criteria, and full access to the investment data. This limits the degree to which the

‘economy’ analysis can quantitatively reflect the difference in investment choices between the two

financing models.

Complexity of outcomes: As with any systems-wide health intervention, the complete picture of

outcomes and impact is highly complex. In order to fully map the cost-effectiveness of the

intervention, some core proxy indicators have been identified which can provide a highlight of the

impact the intervention is having. If a full mapping were to be attempted, the potential health impact

of every treatment made (across a range of medical departments), would be required. Even this

would only capture a partial impact assessment, as the systems approach also aims to embed

long term quality improvements, which means the outcome at the systems and infrastructure

levels would also need to be mapped (from the impact of health worker knowledge and practice,

to improvements in lab practices, increases to service provision through capital investment in

drugs & equipment). In summary, the long term positive impact of the intervention could be greater

than the immediate impact mapped at facility level.

Alignment between the quality controls and health outcomes: On a practical level the

verification process maps the ability of any facility to hit standard quality indicator targets for

service provision and to increase footfall. This relates directly to an increased resource envelop in

the RBF facilities. However, unless the indicators mapped at the verification stage are those which

will most effectively lead to the longer term health outcomes measured for VfM purposes, there

remains a risk that the benchmark for success shifts between implementation and analysis.

Timeframe: A basic but important note to stress throughout the subsequent analysis is that due

to the short programme timeframe, in most cases the programme has only two time related data

points, which severely limits the ability to determined trends effectively.

Data availability: HMIS data was extracted from the DHIS2 website for each facility in order to

obtain the number of cases for each of the six indicators included in the SUO formula which are

numbers of OPD patients, ANC visits, deliveries, family planning contacts, immunisation contact

and IPD patients. The NU Health team also requested the Annual Reports from each PNFP and

extracted the annual income and expenditure data that were analysed in the VfM section. The

quality of financial data from the RBF HF could not be wholly assessed as financial monitoring is

not directly carried out by NU Health and lower level units (LLUs) do not have a legal obligation to

be scrutinised by an external audit. Expressing the total health output of different levels of care in

a comparable way brings additional limitations.

46


1. Economy

An economy analysis aims to analyse the degree to which inputs both demonstrate, and contribute

to, value for money. While the programme has access to the input decisions of the IBF facilities

through the financial accounts, the variety of inputs is so diverse that no core trends can be

identified. This stems in part from the freedom for each facility to determine whether it uses UKaid

funding to pay salaries, buy drugs and equipment, capital investment or for other allowable

expenses as defined by the contract with NU Health and the annual Business plan. For RBF

facilities the challenge is even greater as the ‘hands-off’ approach prevents the programme

knowing in what inputs the facility chose to invest in order to deliver improved health services.

There are, however, key ‘economy’ areas that the programme can directly analyse and use to

provide insights into potential policy recommendations. These include:

Credit line

DHT resources

Overall investment

1.2 Credit Line inputs

The chart below show the average Credit Line (CL) support from NU Health to PNFP per level of

care in the two regions over the two years of implementation (in GBP). The red lines show the

average CL ceilings per level of care of Acholi PNFPs and the green lines the CL of Lango’s.

While levels of support are similar in the LLU, the average CL support to RBF Hospital exceeds

the one for IBF hospitals due to the inclusion of PAG HC4 which required lower CL support than

a hospital. Variation overtime is also due to the exchange rate between the GBP and the UGX as

CL ceilings were labelled in UGX while this table reflect the actual value of the ceilings in GBP.

£0

£5,000

£10,000

£15,000

£20,000

£25,000

Y1Q1

Y1Q2

Y1Q3

Y1Q4

Y2Q1

Y2Q2

Y2Q3

Y2Q4

Average CL support to Hospital/HC4 PNFP

Per RBF Hospital*

Per IBF Hospital/HC4

£0

£200

£400

£600

£800

£1,000

£1,200

£1,400

£1,600

£1,800

£2,000

Y1 Q1Y1 Q2Y1 Q3Y1 Q4Y2 Q1Y2 Q2Y2 Q3Y2 Q4

Average CL support to LLU PNFP

Per RBFHC3Per RBFHC2Per IBFHC3Per IBFHC2

Figure 16 Credit line Support to PNPF per level of Care

47


From a VfM perspective, the economy argument for the CL comes not from the cost per facility

model, but would rather sit at the procurement stage, as the cost per drug was fixed and in no way

affected by which facility it was going to. From a programme side, during the inception period NU

Health identified JMS as the most cost and context appropriate provider for its own credit line set-

up. Up to the end of December 2011, DANIDA had directed its contribution to PNFPs through

JMS. What the analysis does confirm is that both facility groups maintain a predictable drug supply

which responded to increases in service usage and contributed to the facilities ability to provide

quality care.

From a policy perspective, given the goal of increased access to services, there is an assumption

that not only would facilities require a consistent and predictable supply of drugs to function, but

that the cost of this would increase as service uptake improves. This requirement would likely

remain constant for both an IBF and an RBF model.

1.3 DHT inputs

Figure 17 below show the average financial support from NU Health to DHT for carrying out the

verification process in the two regions over the two years of implementation (in GBP). These

figures include the fees and allowances paid to the DHT personnel and fees-for-service provided

to the DHT after they have submitted the QQA report. These are maximum estimates based on

the agreed fees when reports are submitted in a timely manner.

Figure 17 Average financial support to DHT per level of care

The red lines show the average payment to Acholi DHT per level of care and the green lines the

same for Lango DHT. Variation between Year 1 and Year 2 trends is due to the correction of the

fee-for-service agreement with DHT in order to better reflect the costs really occurred by the DHT

to carry out the DQA/QQA.

£0

£50

£100

£150

£200

£250

£300

£350

£400

Y1 Q1Y1 Q2Y1 Q3Y1 Q4Y2 Q1Y2 Q2Y2 Q3Y2 Q4

Average financial support to DHTper level of care and per quarter

Per Hospital*

Per HC3

Per HC2

Per Hospital/HC4

Per HC3

Per HC2

48


Again from a VfM perspective, this does not demonstrate a significant differential between RBF

and IBF as the financials were controlled by the model. However, linking this with the earlier DHT

feedback around conflicting priorities, large roles and limited district resources, it does help to

raise sustainability questions as to the extent to which resourcing similar activity would be

essential for any future intervention. It is not clear whether the DHT involvement could have been

maintained without this external financial support to operations.

1.4 RBF payments and IBF grants

Figure 18 below shows the average financial support from NU Health to PNFP per level of care in

the two regions over the two years of implementation (in GBP).

Figure 18 Average Financial Support to PNFP per level of care

The red lines show the average RBF payments per level of care and the green lines the average

IBF grants. Efforts were made to achieve facility patient load weighted parity. The average RBF

payments exceed the average IBF grants for all three levels of care during the two years of

implementation (except at Year 1 Quarter 2 at HC2 level). This is because IBF grants were

calculated prospectively at the beginning of each implementation year, based on the average

estimated performance results of the experiment group of Acholi PNFPs and according to the

available IBF budget.

The sudden increment of the green lines between Y1 Q4 and Y2 Q1 is due to the revised IBF

grant after the first year of implementation in order to better match the level of funding with the

RBF payments made to Acholi PNFP.

We see the gap in financing increasing as the RBF facilities begin to provide consistently higher

quality scores during verification than the anticipated average. In essence the analysis at the

economy level clarifies a skewing at effectiveness level. If the financial input rises proportionate

£0

£10,000

£20,000

£30,000

£40,000

£50,000

£60,000

£70,000

£80,000

£90,000

Y1 Q1 Y1 Q2 Y1 Q3 Y1 Q4 Y2 Q1 Y2 Q2 Y2 Q3 Y2 Q4

Average financial support to PNFPper level of care and per quarter

Per RBF Hospital*

Per RBF HC3

Per RBF HC2

Per IBF Hospital/HC4

Per IBF HC3

Per IBF HC2

49


to quality, with the assumption that this relationship will be steady, there is no way for a facility to

demonstrate improved cost effectiveness – as the additional quality is undermined by the

additional input.

If the pilot were to be expanded elsewhere, there could be merit in determining the quality impact

of two different RBF scenarios compared to the IBF. In one, the financing would work as in NU

Health, increasing in line with quality, in the other, a deduction model could apply. This would help

to highlight the degree to which the financial value has the greatest impact on quality, compared

to ‘grant vs potential’ concept in which staff are incentivised by an improving quality score.

1.5 NU Health inputs as proportion of facility revenue

The final economy analysis looks at the degree to which the inputs from NU Health contribute to the facilities’ overall purchasing power. Table 12 NU Health Support as % of Total Income by facility below summarises the average

annual income per level of care against the average NU Health financial support, thus providing

an indication of the NU Health contribution to their annual activities. This gives us an indication

that while we look for trends in activity across the two study groups, the weight of the contribution

and therefore likely attribution, varies considerably.

The NU Health contribution to the HC3 budget is significant (70%) and modest for Hospital/HC4s

(around 20%). At HC2 there is a major difference between the NU health contribution in the two

groups, being significant in the RBF group (70%) and less so in the IBF ones (40%), the latter

charging higher user fees ensuring a better sustainability ratio as analysed later.

Table 12 NU Health Support as % of Total Income by facility

Av. annual

income

FY13-14 (UGX)

Av. RBF/IBF

grant

FY13-14 (UGX)

Av. % NU

Health

contribution to

income

Per RBF Hospital* 4,432,299,991 797,614,480 18.04%

Per RBF HC3 728,110,549 485,149,883 69.46%

Per RBF HC2 180,949,534 108,030,679 69.11%

Per IBF Hospital/HC4 3,028,387,342 590,316,007 22.16%

Per IBF HC3 398,226,247 279,712,421 71.54%

Per IBF HC2 137,634,921 55,525,525 42.17%

Overall, the key economy lessons learned relate to the necessary inputs required for a similar

intervention to succeed. These include:

An adequate, consistent and predictable drug supply;

A financial governance procedure to ensure costs are captured at facility level (whether or

not investment decisions are controlled);

Clarity around the roles, budget and priorities of the DHT;

Clarity around the level of freedom with which the facility may invest;

50


Clarity around any ‘quality cap’ for highly successful facilities.

2. Efficiency

This section analyses how well these inputs are turned into outputs. Key areas for review in this

section include:

Relative cost to run the programme

Comparative investment across IBF / RBF groups

Identification of core cost drivers and recommendations

2.1 NU Health Milestone Payments

The table below summarises the overall support to facilities (including facility grants and credit line

costs) and compares them to the wider cost of running the study. As an implementation based

research piece, these include amounts paid each quarter by DFID to Health Partners International

(HPI)/Montrose for delivering pre-agreed milestones that covered a range of costs including DHT

mentoring and organisational support through business planning, development of capacity

strengthening plans, provision of secondees to support DHTs; data collection & management

through extensive DQA / QQA activities; actual management costs and overheads. The bottom

row shows the value of the milestones relative to direct support to the facilities.

Figure 19 Milestone value proportion of total budget

The initial ratios are higher due to the staggered approach implemented by NU Health at the

beginning of the programme. Considerable improvements in efficiency have been demonstrated

both by facilities achieving higher performance and operational savings allowing the study to

streamline as the model becomes more established.

Economy analysis of inputs Y1 Q1 Y1 Q2 Y1 Q3 Y1 Q4 Y2 Q1 Y2 Q2 Y2 Q3 Y2 Q4TOTAL

2 YEARS

Total inputs to PNFP (Grants and CL) £388,210 £456,127 £483,960 £562,773 £734,001 £857,576 £882,559 £956,514 £5,321,721

NUH costs (MS value) £357,015 £329,060 £292,138 £465,188 £354,920 £511,120 £265,013 £246,875 £2,821,329

Grand Total £745,225 £785,187 £776,098 £1,027,961 £1,088,921 £1,368,696 £1,147,572 £1,203,389 £8,143,050

Ratio MS / Grand total 48% 42% 38% 45% 33% 37% 23% 21% 35%

0%

10%

20%

30%

40%

50%

60%

Y1 Q1 Y1 Q2 Y1 Q3 Y1 Q4 Y2 Q1 Y2 Q2 Y2 Q3 Y2 Q4

51


IBF vs RBF efficiency measures

While the full array of health outcomes cannot be captured for a programme as broad as NU

Health, the programme has been able to identify proxies by using the Standard Unit of Output

model identified by the UCMB. This monitors a wealth of indicators including OPD patient visits,

ANC visits, deliveries, family planning contacts, immunisation contact and IPD patient visits,

weighting each and coming up with an aggregate scoring (more of which is shown in Annex 2).

Analysis for FY13/14 demonstrates that by facility type, the RBF group provided a greater

proportion of the overall SUO score than the IBF group for higher level facilities with approximately

equal impact at the HC2 level. While it is impossible to draw full conclusions at this stage, we may

be seeing a situation in which the routine support, a predictable grant, drug credit line and DHT

inputs provide as much of a boost in lower facilities, as performance based incentives. It does,

however, also point to an inherent flaw in the model which includes many of the services which

HC2s do not perform.

Limitations & Recommendations

One core limitation for a balanced VfM assessment is that the incentivisation of performance in

RBF facilities masks the ability of the facility to demonstrate savings. If the model directly links

performance with increased input, and provides a specific payment per service, the efficiency of

the system will be inherently linked to the value placed on each service, and will not freely

demonstrate the efficiency or inefficiency of the facility.

For instance, if a facility is rewarded $x per ANC visit but only uses $(0.7x) to achieve it, the

assumption from an efficiency side is that the facility has still used $x for a single ANC visit. The

52


saving is only demonstrated if the remaining $(0.3x) goes directly towards another ANC visit.

However, if the saving instead goes to something less tangible, such as a staff incentive,

construction etc, the change in output may not be seen for some time after the investment.

Therefore, without understanding the way in which the facility invests, and the impact of a RBF

model over a longer time period, it is not possible to accurately determine comparative efficiencies

across IBF and RBF. Investments directly into ‘score-able’ areas will naturally be regarded as

more ‘efficient’ than longer term investments in the system. This is particularly important when

looking at the external learning from the programme, as with all health investments, the apparent

efficiency of each model could vary considerably over time. This is particularly compounded when

the NU Health grant forms a varying proportion of the overall facility revenue.

From a policy perspective, another key VfM learning is that operational savings are feasible as

both the RBF and IBF models become more established. However, in planning discussions the

operational budget would need to remain a key component. Further savings could be sourced by

reducing the robustness (frequency / on site time etc) of the verification exercises, however,

ultimately a balance would need to be struck to ensure appropriate quality and cost.

3. Effectiveness & Cost Effectiveness

This section aims to analyse whether the ‘right outputs’ have been invested in to achieve the

intended outcomes. Following the discussion above around the feasibility of mapping efficiency

without a full picture of inputs and outputs, a similar issue arises at the effectiveness level. The

effect is more pronounced as for efficiency there are at least consistent data elements coming

from the DQA/QQA, even if the input picture is less clear of RBF. At the output-to-outcome

transition, limitations stem from the lack of a full mapping of potential outcomes with associated

timeframe.

As clinical outcome data are not available, analysis consists of the cost per quality-adjusted SUO

to give an impression of relative cost effectiveness in RBF versus IBF facilities, across the different

levels of the system. As discussed above, with only two data points this presents only a tentative

finding, however, if the trend were to continue, one would expect to see improving effectiveness

and perhaps cost effectiveness.

Figure 20 provides a snapshot of the relative quality adjusted cost ratio per SUO for the two groups

by facility type. This figure shows a dramatic shift in the relationship between RBF and IBF cost

effectiveness, when the differential improvement in quality, as measured by the QQA is taken into

account. While the IBF appeared far more cost effective in year one for lower level facilities and

reasonably more cost effective for hospitals, RBF appeared to be the more cost effective model

for HC3s. A year further along, we can see a dramatic improvement in relative cost effectiveness

for RBF HC2s and a moderate adjustment for hospitals. At the end of the first full year of

implementation, the grant formula was shifted to help address weaker performing areas.

“The increases proposed would still represent a subsidy towards the total cost of service delivery,

but would allow facilities greater flexibility for innovation. The indicators proposed for increase

included: i) 1st ANC visit, ii) LLIN distribution iii) Caesarean Sections iv) Measles vaccination, and,

53


v) Vitamin A supplementation. Meeting participants suggested that the increments be spread out

across a broader selection of indicators, and nominated areas where they felt greater

incentivisation was needed most. The NU Health team agreed to look into the suggestions raised

when revising subsidy levels for implementation Year 2.”7

It is possible that by amending the performance contract formula, the incentivisation overtook the

facilities’ capacity to invest which would explain why HC3 facilities appeared to move from a RBF

to IBF as the most cost effective model. A third data point would provide an interesting comparator

to examine the extent to which facilities ‘settled’ into the new grant formula and were able to once

again reinvest more effectively in the RBF group.

Figure 20 Quality adjusted RBF v IBF Cost Ratio per SUO by Facility Type

4. Equity & User Fees

This section describes the shifts seen in the socioeconomic profile of beneficiaries through a

description on user fee policy. Despite the recognition that user fees can prove a barrier to access

for the poor, they often end up providing a significant level of necessary health facility revenue.

Although this has a regressive implication with the poor paying proportionally more than the better-

off, the revenue associated with user fees provides a level of health service sustainability which is

analysed below.

Table 13 summarises user fees per SUO per facility type in FY13-14 and FY12-13.

7 NU Health Quarterly Progress Report Jul-Sept 2013.

-

0.50

1.00

1.50

2.00

2.50

3.00

FY12-13 FY13-14

HospitalRatio

HC3 Ratio

HC2 Ratio

54


Table 13 User Fees by Facility Type

Av SUOAv User fees

collected

Av user fees

per SUOAv SUO

Av User fees

collected

Av user fees

per SUO

Per Hospital* 214,465 550,820,851 2,568 Per Hosp./ HC4 164,484 569,122,605 3,460

Per HC3 39,451 42,225,283 1,070 Per HC3 17,479 65,464,242 3,745

Per HC2 7,272 8,594,679 1,182 Per HC2 8,866 60,797,800 6,858

Acholi PNFP - FY13-14 Lango PNFP - FY13-14

Av SUOAv User fees

collected

Av user fees

per SUOAv SUO

Av User fees

collected

Av user fees

per SUO

Per Hospital* 225,455 449,282,513 1,993 Per Hosp./ HC4 144,051 673,524,425 4,676

Per HC3 56,412 39,743,967 705 Per HC3 16,875 61,248,262 3,630

Per HC2 4,201 12,379,791 2,947 Per HC2 9,043 37,620,775 4,160


The Acholi Hospitals and HC3 collected more user fees this year, despite lowering their output

while HC2 have decreased their collection of user fees while increasing their output. This

translates into a higher average user fee per output at Hospital and HC3 level compared to the

previous year while being divided by three at HC2 level. This could be explained by the increased

incentive provided by NU Health to that level of care which allowed them to reduce their use fees

to attract more patients and thus boost their output.

In Lango the opposite has been observed, with HC2 almost doubling their collected amount of

fees while slightly reducing their output, but Hospital/HC4 collecting fewer fees while increasing

their output. HC3 figures remained stable over the two FY. This translates into an increment of

average fees at HC2 level and a decrease at Hospital level.

When comparing the two regions, Lango PNFP charge their patients more than the Acholi ones,

with 5 times more at HC2 level and 3 times more at HC3 level, which reflects the same difference

as that noticed last year.

While this analysis is useful, the implications for broader policy or operational learning would be

strengthened by additional review of the following areas:

Key drivers for access (do people seek the facility nearest them first and then progress

through the facility levels or self-refer to the facility level providing the care they need?

Duplicability of user fees – if an individual is referred, must they always pay a second round

of user fees?

Drivers behind user fees – to what degree are user fees set by the owner or diocese vs

the in-charge? How does this link to planned income generation and budget mapping?

What is the socio-economic spread of patients for each level of care, and from this would

an equitable user fee policy look at having a flattened spread of costs to prevent anyone

from being unable to access the suitable level of care based on cost comparative to a lower

level facility? Or would it rather look to have a lower user fee rate at the lower level of

facility on the grounds that not only would this be more likely to be accessed by lower

55


socio-economic groups, but that an individual household would likely require more lower

level services than higher ones in a given time period?

While many of these questions are market driven, there is a broader equity question behind the

user fee analysis.

5. Sustainability & User Fees

The sustainability ratio of PNFP shows the portion of annual income that is covered by the collection of user fees. Table 14 summarises user fees as a proportion of the facilities’ income across facility types in

FY13-14 and FY 12-13.

Table 14 User Fees and Facility Income

Av IncomeAv User fees

collected

Av % user

fees vs

income


collected

Av % user

fees vs

income

Per Hospital* 4,432,299,991 550,820,851 12.43% Per Hosp./ HC4 3,028,387,342 569,122,605 18.79%

Per HC3 728,110,549 42,225,283 5.80% Per HC3 398,226,247 65,464,242 16.44%

Per HC2 180,949,534 8,594,679 4.75% Per HC2 137,634,921 60,797,800 44.17%



collected

Av % user fees

vs incomeAv Income

Av User fees

collected

Av % user fees

vs income

Per Hospital* 3,738,416,177 449,282,513 12.02% Per Hosp./ HC4 2,001,454,620 673,524,425 33.65%

Per HC3 390,014,987 39,743,967 10.19% Per HC3 175,978,784 61,248,262 34.80%

Per HC2 82,839,952 12,379,791 14.94% Per HC2 66,947,232 37,620,775 56.19%


On average, Lango/IBF HF are collecting more user fees per annum compared to those in Acholi.

When comparing these amounts with the average income per level of care, it shows higher

sustainability ratios in the Lango/IBF region compared to the Acholi one (all three percentages are

higher in Lango), and more so at LLU level.

The average incomes (which include the NU Health contribution) in both regions and at all three

levels of care increased from last year. When comparing these figures, it appears that on average

Acholi facilities collected more user fees this year than last year, though these constitute a smaller

proportion of overall budget. In Lango, by contrast, Hospitals/HC4 collected fewer fees and HC2

more fees, but that these represented a significant decline in their contribution to the overall

budget.

56


9 OPERATION / MANAGEMENT PROCESSES

This section describes the key operations and management processes that were established during the programme and the core lessons learned. Figure 21 illustrates the seven stages of the RBF programme process.

Figure 21 RBF Programme Process

1. Facility Selection

As the PNFPs in Northern Uganda were at different levels of functionality and preparedness to

engage in RBF, they were assessed by NU Health during the inception phase, using the five

following criteria:

1. Staffing norms;

2. Functioning equipment as required per level of care;

3. Functioning infrastructure;

4. Registered with UCMB, UPMB or have other legal status8;

5. Provide most of the preventive and curative maternal and/or child health services as

specified in the Health Sector Strategic and Investment Plan as appropriate for the

level of care.

Based on the above, the PNFP were classified by NU Health in three categories:

Category 1 when all five criteria were in place

Category 2 when two of the criteria were missing

Category 3 when more than 2 criteria were missing

In Acholi sub-region, 9 PNFP were in Category 1 and 12 in Category 2, while in Lango sub-region

there were 5 PNPF in Category 1 and 5 in Category 2. Facilities falling under Category 3 were

excluded from the RBF and IBF scheme as they were deemed too weak for consideration.

Following this selection, RBF implementation was phased with Category 1 HF joining immediately

into the RBF scheme without delay and without a seed grant; and Category 2 HF joining during

the second quarter after receiving a seed grant (explained below). This was to enable the PNFPs

in less-prepared classifications to enhance their level of service and capacity to set thresholds

which allowed them to participate on an equal footing with better-prepared health facilities.

2. Development of business plans by the PNFPs

8 Licensed with the Medical and Dental Practitioners Council and with the Medical Bureau or under the NGO status,

and recommended by the DHT.

Facility Selection

Capacity Building

Business Plan

Seed grantsMOU

Cycles &

Batches

Scale up

app-roach

57


Business plans formed the basis on which health facilities were able to plan improvements in

service delivery/quality which would be funded by the RBF or IBF funds. The business plans state

targets for improvements in quality and the resulting expected volume of services, as well as the

investments needed to attain these. Thus, unlike action plans, the business plan reflects realistic

goals which have financial implications for the health facility and the programme.

The business plan is an essential element of the RBF process as it outlines how the facility plans

to respond to the financial incentives provided, how it plans to achieve improvements in both levels

of service provision and quality of services, and hence meet its targets and receive funding.

A lesson learned from the business plan (BP) process is that most HF, especially LLU, had no

idea of the concept of a BP and were unable to prepare one prior to the programme. Therefore,

NU Health organised several training sessions aimed at explaining to all the PNFP and DHT the

BP concept, use, modalities, etc. as well the overall RBF concept.

Whilst NU Health was facilitating and supporting the BP process, the PNFP had autonomy over

what it did or did not include in the business plan and complete freedom for innovation to ensure

they met their objectives. As a result the PNFP subsequently bears the risk for non-performance

against targets.

The facilities were then able to prepare their business plans detailing scope and volume of

services which were shared with the DHT.

The business plan included the following information:

For the initial entry into the RBF scheme, facilities had to outline their plans for the Seed

Grant, which is an initial investment to prepare the health facility adequately for the

introduction of RBF. This grant was based on a case by case negotiation with NU Health,

and was available for activities or inputs related to improving quality of services and/or

implementation/management of an RBF scheme. This did not cover additional staffing.

Thereafter, NU Health negotiated with each facility to agree a set of services and quality

and quantity indicators (in line with the list of services and indicators included above)

against which their performance will be monitored. The following outlines the structure of

the business plan per service:

Selected services denote those services which each facility will include in line with

their core values and key competencies

Operations targets aimed at compliance with national minimum norms covering for

example human resources, infrastructure, essential medicines, medical equipment,

office equipment

Facilities were asked to outline for each service selected

Monthly targets

Issues arising with achieving targets

Strategies/steps to solve these issues to allow for achievement of targets

58


Finally, facilities were asked to plan revenues and expenses against current revenues and

expenses.

It is also worth noting here that while the initial plan had been to use the BP as a monitoring tool

for the PNPF usage of RBF funding, the revised hands-off approach applied by DFID on the

programme, rendered the BP somewhat irrelevant as a monitoring tool. On the other hand, in the

Lango sub-region, the annual BP has been a useful tool as it formed the basis for the quarterly

business plan and budget reviews which were approved and closely monitored by NU Health.

3. Capacity Building

Based on the above, the first training/capacity building focus was on Business Planning for both

the PNFP and the DHT.

This was followed by Financial Management training in the IBF/Lango region as IBF grants were

to be managed and monitored by NU Health. The existing financial guidelines from the Diocese

of Lira were used to roll out to all the Lango PNFP. Additional procurement guidelines were added

by NU Health to complement these existing procedures.

With NU Health providing medical supplies to all PNFPs, training on drug supply management

was also organised by NU Health twice over the lifespan of the programme. Following a baseline

survey assessing the gaps and weaknesses in drug management, training sessions tailored to

tackle the most pressing gaps were organised by NU Health.

Finally, as a new tool had been introduced by NU Health, the Quality of Care checklist, the tool

had been explained to PNFP and DHT in each District with theoretical and practical training

sessions.

4. Seed grants provided to Category 2 HF

As mentioned above, the PNFPs were in variable states of readiness to undertake the RBF

initiative. There were gaps in infrastructure, equipment, supplies, and human resources which

required attention before PNFPs could provide the services identified in the RBF programme as

triggers for payment. Seed grants were therefore provided by NU Health prior to engaging in the

RBF programme in order to enable them to participate effectively.

The initial assessment indicated that the capital needed to support equipment needs was relatively

small. Seed grant for equipment was used to procure equipment items such as instrument drums,

delivery sets, delivery beds, adult weighing scales, examination couches, drip stands, centrifuges,

autoclaves and dressing kits. These seed grant equipment items were sourced directly from JMS.

The human resource gaps were more substantial. A critical assumption was that with an enhanced

salary package (from the overall increased funding through the RBF/IBF mechanism), the facilities

would be able to attract the required staff mix during the first six to 12 months of implementation.

In reality PNFPs were not willing to change their human resource policies because the NU Health

support was limited in time. PNFP therefore consistently struggled to retain staff and often lost

59


staff to GoU recruitment campaigns to staff the public facilities. This was broadly seen to be a

reflection of the fact that in private facilities, staff have to do more work for less pay, while

performance is much laxer in public facilities and so it is seen to be a more attractive role for many.

5. MoU signed with each HF and DHT

Once the business plans had been endorsed by NU Health, the RBF modalities were sealed into

legal agreements with each PNPF. As PNFPs belonging to a Diocese had no legal status in their

own right as entities capable of entering into a legally binding contract (while the Diocese had),

legal agreements were therefore signed between NU Health, the Dioceses (as legal owners) and

the PNFP; the Dioceses taking legal responsibility for the participation of the PNFPs in the RBF

scheme.

It is worth noting here that the Medical Bureaus were not involved in the RBF scheme and thus

not in the legal agreements signed with PNFP, despite their technical role in the health sector

supporting the Dioceses. It is also worth noting that the Diocesan Health Coordinator Office was

not supported by NU Health –technically or financially - despite their legal responsibility as

representative of the legal owner of the HF.

6. Planning the verification in cycles and batches

The following processes are being followed in the RBF verification process:

On a routine basis, the PNFP records data on levels of service provision;

On a quarterly basis, data is being verified by the DHT with NU Health’s support;

DHT and NU Health also assess the quality of the health care services being provided by

the PNFP, using the Quality of care checklist;

NU Health then calculates performance-based payments due to PNFP;

PNPF prepares an invoice;

Invoices are crosschecked by NU Health and passed to HPI for payment;

HPI to match invoice with purchase order (PO) to check the amount is within agreed

maximum quarterly incentive payment per PNFP, however overall responsibility for

correctness of incentive invoice rests with NU Health;

The raising of a PO per PNFP allow HPI to monitor independently the actual funds

disbursement per PNFP versus target;

Funds are transferred by HPI directly to PNFPs;

PNPF raise a receipt.

The fund flows process for both RBF and the credit line are summarised in the following diagram.

60


Due to the above process based on quarterly verification, the supported PNPF HF in Acholi were

shared into three equivalent batches with each batch composed of one hospital, one HC3 and 5

HC2. In Lango, due to a different level of care composition, two batches had one Hospital or one

HC4, one HC3 and one HC2 while the third batch had four HC3. The aim was to spread as much

as possible the workload evenly across each month of verification.

As illustrated in Table 15, due to the above, one RBF quarter, broken into three cycles would then

spill over a five-month period as follows: Cycle 1 starting 1st September while the last cycle of that

quarter, Cycle 3, ending on 31st January, with the verification process (highlighted in red below)

being organised at the end of the three-month cycle.

Table 15 NU Health Planning Cycle

NU Health - CYCLE PLANNING

Quarter 1 Quarter 2 Quarter 3

Batch Sept Oct Nov Dec Jan Feb Mar Apr May

A Cycle 1 DQA/QQA DQA/QQA

Figure 24 NU Health Fund Flow Diagram

61


B Cycle 2 DQA/QQA DQA/QQA

C Cycle 3 DQA/QQA DQA/QQA

With the extension of the contract, NU Health support will continue to the facilities through the end

of June 2015. The final round of DQA/QQA will be completed in two rather than three months.

10 RISK, PARTICULARLY FIDUCIARY RISKS, AND ITS MITIGATION

This section examines the potential risks related to the different components of the programme:

the implementation of the RBF scheme, the IBF component, the Credit Line, and support to the

DHT. Given the “hands off” policy, where no advice was provided on the allocation of funds and

there was no monitoring on the use of the funds in the RBF region, particular emphasis is placed

on fiduciary risks. This section also examines the potential consequences of these risks and

mitigation measures taken by NU Health.

1. Fiduciary risk

Fiduciary risk is the risk that funds are not used for the intended purposes; do not achieve value

for money; and/or are not properly accounted for. Fiduciary risk arises due to a variety of factors,

including lack of capacity, competency or knowledge; bureaucratic inefficiency; and/or active

malfeasance and corruption.

By design, the RBF’s hands-off approach on fund management use entails risk. By design, DFID

and NU Health accepted this risk to ensure the methodological integrity of the study.

Nevertheless, the programme has criteria and standards related to how facilities adhere to their

own standard operating procedures, and there was one case brought to NU Health and DFID over

the potential mismanagement of RBF funding in one HC2. Additional financial monitoring carried

out subsequently by NU Health and the Diocesan Health Office indicated that there was poor

adherence to financial guidelines in that HF. Per its agreement, the facility was provided with an

opportunity to make corrections and was subsequently suspended from the programme when it

failed to make these corrections.

In the IBF group, by contrast, the fiduciary risk was mitigated by a range of actions. Basic

expenditure plans were based on NU Health approved business plans. Procurement and

payments, including accounting for project funds were closely monitored on a routine basis. IBF

facilities were audited by an external firm selected by NU Health. NU Health also received

management responses from each facility on issues from these audits.

2. Corruption

The likelihood and magnitude of corruption occurring, as opposed to the other factors (lack of

capacity, inefficiency, etc.) is a key aspect of fiduciary risk. Defined by Transparency International

62


as ‘the abuse of entrusted power for private gain,’ corruption is a problem in Uganda, as reflected

in the UK government’s suspension of direct funding in 2012. Building on the RBF segregation of

authority model, risk is mitigated by ensuring a high level of transparency among the DHT, the

PNFP and NU Health in the verification process with three parties jointly verifying the PNFP

performance which triggered RBF payment.

3. Displacement of existing funding

NU Health’s inception came at the time when humanitarian financing for health was tailing off,

according to many participating facilities. With NU Health funding flowing to PNFPs, there was a

risk that GoU and Districts might use NU Health support as an opportunity to displace some of the

Primary Health Care grants allocated to PNFP by the GoU. While tracking these fund flows was

beyond the scope of NU Health, no report of fund interruption was received, although there were

reports of delays.

4. Data reliability and gaming

Reported service delivery figures were generally higher than the verified figures because of poor

adherence to data collection and reporting procedures. As performance payment came on line,

there was also some tendency to inflate the performance. For the purpose of the study, primary

data were employed rather than those reported in the HMIS. Data triangulation was carried out

by the DHT and NU Health in order to obtain the verified numbers. Primary data was then

compared in order to assess the discrepancy rate. In addition, data quality assessment was

incorporated within the QQA tool.

5. DHT capacity

As the regulator/verifier of the RBF intervention arm as well as the IBF comparison arm, the DHT

plays a crucial role in the study. In Northern Uganda, however, the range of demands by externally

supported programmes and projects outstrips the capacity of the DHTs to deliver. Often the DHTs

were stretched thin to perform the results verification with consistent teams having the right skill

set.

Mitigation measures taken by NU Health included training all DHT personnel on the DQA and

QQA tools so that all of them were able to verify the data quantity and to a certain extent the

quality of health care services being provided. A second measure was to also train the District

Sub-District personnel (public health workers from HC4) on the tools so that they also could be

included when needed.

6. Management of medical supplies

The risks related to the Credit Line of EMHS can be divided into two main types:

1) Risks at the drug supplier level—JMS:

Changing prices of EMHS: there is a risk of EMHS prices rising beyond the agreed upon

price maximum in the MOU. This may lead to budget increase for EMHS.

63


There is a risk of JMS not being able to stock enough EMHS in terms of quantity, quality

and variety. This may eventually lead to very low order fill rates for the facilities and poor

quality drugs may be delivered to facilities

There is a potential risk in the delay in distribution of EMHS due to inadequate quantity of

EMHS available to transport at JMS, transport break down due to mechanical failure of the

trucks.

There is a risk of loss due to theft of some or all the EMHS being transported by the third

party transporters. Though it is a risk of JMS it eventually affects the PNFPs too.

The probability of the JMS risks occurring is minimal as JMS has a sound supply chain

management system in place that minimizes the occurrence of these risks. The impact of

these risks would be organisational and financial as other sources of supply would have to be

sourced which might have a financial implication.

The mitigation actions put in place by NU Health were to:

Signing a detailed MoU with JMS that clearly stipulates the roles and responsibilities

between NU Health and JMS.

There are procedures in place at NU Health to check the actions of JMS during execution

of the EMHS process with a routine back and forth communication between JMS, NU

Health and the PNFP about the purchase orders, the unavailability of some items at JMS,

suggested replacements, progress in the delivery rate, etc.

2) Risks at PNFP level:

Risk of theft of EMHS by the staff

responsible.

There is potential risk of a facility not

practicing good pharmaceutical

management practices hence leading to

loss due to expiries of medicines or

damage of EMHS due to poor storage

practices.

There is a risk of a facility not honouring

the terms of its NU Health MOU leading

to its suspension from the study.

The probability of loss occurring at facility level is

lower in higher level facilities due to their more

robust supply management procedures in place.

That probability is higher (medium) but

consequences lower in LLU that lack trained

staff, have weak supervision and poor

infrastructure. The impact of these risks might be

serious for beneficiary health services.

Despite the “hands off” policy, DHT and NU

Health keep their “ear to the ground.” Over the

course of the last three years, the programme

has received two notifications of potential

financial impropriety. With the first of these,

NU Health conducted an assessment, and

reported back to the Facility Management

Committee on matters that were inconsistent

with their NU Health agreement. Per

agreement, the facility was provided an agreed

period to implement the agreed changes in

management and control procedures. The

agreed changes were not made, and the

facility was excluded from the programme.

In the second instance, the programme

received an anonymous report of procurement

impropriety. In this case, there were apparent

weaknesses in adherence to procurement

procedure, but with no apparent malfeasance.

After further review by DFID, NU Health was

instructed not to certify the payment requested.

Figure 22 Risks & Consequences

64


The mitigation actions put in place by NU Health were to:

Train facility staff in good pharmaceutical management;

Conduct routine supportive supervision and EMHS audits at facilities;

Incentivise good pharmaceutical practises by adding two pharmaceutical management

indicators added to the QQA tool; and,

Use “the stick” in case of mismanagement of EMHS with the possibility of suspension of

credit line support.

11 CONCLUSIONS: GOOD AND BAD PRACTICES IN RBF

Background

This final section of the Lesson Learned paper will serve as its conclusion. Building on and

synthesizing points made in other sections of this report, evidence, insight and impression are

brought together to summarise NU Health’s observation on what has happened in health facilities

in the provision of services and the implications of those observations on policy and programming.

Management Process

With NU Health’s “Hands Off” policy the responsibility of the programme focused on ensuring

requisite inputs were in place, the appropriate reporting was undertaken, and, in the case of the

IBF control facilities, that procurement was carried out in accordance with facility plans and facility

procedures. While facilities were provided support for business planning at the beginning of the

programme, there was neither technical assistance nor supportive supervision to assist them in

using their plans in routine operations.

NU Health had a mandate not to guide the development or use of facility plans. Informally,

however, it received reports on heterogeneity on both how plans were developed and how they

were used. Some facilities actively overcame the management-clinical divide and achieved

success in increasing ownership of staff involved in clinical care as well as management and

administration. In these facilities, it seemed that there was a higher level of transparency in

communication with both organisational performance and incentive payments. Though it was

beyond the scope of this study, it would be interesting to better characterise the determinants of

more participatory and transparent planning and management.

Clinical Care

In interpreting what happened in terms of clinical care in facilities enrolled in the NU Health Study

it is important to note that although the study was designed around isolating the effect of funding

modality on provider behaviour, there are many other considerations that affect provider behaviour

outside the scope of the study. It is also interesting that anecdotally, at least, the health sectors

in Acholi and Lira are said to have different cultures born out of their differential experiences during

the conflict.

As noted in the Quality of Care and Health Outcomes sections of this report, data generated by

the programme provide possible preliminary insight into how financing modality affected clinical

65


care. Overall, facilities in the RBF arm of the study provided higher quality of care as measured

by adherence to treatment guidelines and maintenance of tracer medicine stocks.

Reporting

Discrepancy Rates in data

The discrepancy rates in the NU Health study refer to the discrepancy between service data reported in the HMIS and those verified by NU Health. As summarised in

Figure 26, while a general tendency toward under-reporting is noted, there is a trend toward lower

discrepancy rates over the life of the study overall and, not surprisingly, in the RBF facilities in

particular.

Figure 26 Discrepancy Rates

Discrepancy rate (DR) = (number of verified cases – number of reported cases by the PNFP) / number of reported

cases. A DR within a +-5% margin is considered acceptable and therefore incentivised in the RBF scheme in the quality

assessment tool.

Despite the absence of baseline data on reporting discrepancy, the pattern observed over the life

of the project is noticeably different in RBF versus IBF facilities. It no surprise that the discrepancy

rates are better in the RBF facilities, by virtue of their role in defining performance payment. No

doubt the intensity of QQA also reinforced the importance of good data management. Even with

the relatively more reliable HMIS of the RBF facilities, these data do underscore the need to

strengthen HMIS to strengthen service delivery and accountability, by enabling better targeting of

scarce resources to under-served or under-performing areas and facilities.

Completeness and Timeliness of data entry

In addition to managing accurate data, HMIS function is improved with complete and timely

information. While this exercise had some implications for the programme, it serves more as a

diagnostic on the state of HMIS in Uganda, or at least Northern Uganda in general.

66


Table 17: Average Completeness and Timeliness of Reporting rate of supported PNFP per

Level of Care during FY 13-14

Level of care Sub-

region

Actual

Reports

received

per year

Expected

Reports

per year

Completeness

Ratio

Reports

On

Time

per

year

Timeliness

Ratio

AVG PER

HOSPITAL*/HC4

ACHOLI 132.5 183 72% 89.5 49%

LANGO 152 182.5 83% 124 68%

AVG PER HC3 ACHOLI 153 235 65% 126 54%

LANGO 140 182 77% 116 64%

AVG PER HC2 ACHOLI 90 123 73% 71 58%

LANGO 119 146 82% 94 64%

AVG PER PNFP

(all level)

ACHOLI 104 147 71% 82 56%

LANGO 138 175 79% 113 65%

* excluding Lacor Hospital

In both intervention and comparison areas, performance was relatively low. It is interesting to

note that the Lango intervention area generally had more complete and timely submission.

Nevertheless, the larger point relates to how well HMIS can support effective and efficient delivery

of sectoral oversight and health services without good enough or timely enough information as

this is an essential factor in deciding on appropriate allocation of resources to those areas most

in need.

Financial Management

Financial management, including the mitigation of fiduciary risk, is vital to any project or

programme. When it is the substantive focus of the intervention, in this case the assessment of a

service financing mechanism, it becomes the central focus of interest.

When NU Health was realigned to provide a stronger basis of evidence, the project was advised

to adopt a “hands off” approach with relation to how RBF funds might be used. IBF facilities, which

received grants matching the value of similar situated RBF facilities, had facility specific plans and

specific procurement and documentation requirements guiding the use of NU Health funds. The

RBF facilities received payment on the basis of their verified results without investment

qualification from NU Health. It was stipulated that performance payments be used in a manner

consistent with any non-NU Health existing governance arrangements.

To this end, there were a range of ways facilities opted to utilise their funds. Although NU Health

did not track it, we have some impressions about investment tendencies. In general, in the larger

facilities, finance from NU Health was pooled with other facility operating funds. Funds also seem

to have flowed towards investment rather than consumption in many of these larger institutions.

67


In general lower level facilities are using NU Health finance to fund operating expenses and

allowances rather than choosing broader investment options. It was also noticed that IBF facilities

in general invested more in infrastructure and equipment than the RBF facilities.

That said, there were some RBF facilities who opted to save some of their results payments. A

number of HC2s such as New Life, St Monica and Wi Anaka adopted saving strategies in part for

concern about what happens after the end of NU Health.

Conclusion

Over the last three years, NU Health has developed into a quasi-experimental study focused on

isolating, to the extent possible, the different practices, management methods and cost-

effectiveness of RBF with non-state PNFPs vis-à-vis a conventional IBF model. Some significant

lessons can be drawn from this experiment as can be seen through the two Lessons Learned

documents prepared.

It is expected that with its formal evaluation, NU Health will contribute important evidence to inform

policy and programming in Uganda and beyond. In terms of effectiveness, preliminary data

suggest that RBF can lead to better quality of care along with improvements in a number of the

critical pathways that result in accessible quality care regardless of gender or socio-economic

status. The programme is also working with cost data to develop preliminary models of the extent

to which a similar intervention design, with the research elements and some data management

functions stripped out, could serve as an appropriate financing model for public facilities in

Uganda.

68


REFERENCES

CORDAID & SINA (2010) PBF in Action Theory and Instruments Course Guide

COUNTDOWN TO 2015 (2014) Fulfilling the Health Agenda for Women and Children: The 2014

Report

DONDORP AL, FANELLO CI, HENDRIKSEN ICE, GOMES E, SENI A, CHHAGANLAL KD, et al.

(2010) “Artesunate versus quinine in the treatment of severe falciparum malaria in African children

(AQUAMAT): an open-label, randomised trial”. The Lancet 376(9753): 1647-1657

ELDRIDGE, C & PALMER, N (2009) Performance-based payment: some reflections on the

discourse, evidence and unanswered questions. Health Policy and Planning 24(3): 160-166

ICAI (2011) ICAI’s report on Approach to effectiveness and Value for Money

LAWN JE & KERBER K (Eds) (2006) Opportunities for Africa’s Newborn. Practical data, policy

and support for newborn care in Africa, Cape Town, PMNCH, Save the Children, UNFPA,

UNICEF, USAID, WHO

MINISTRY OF HEALTH UGANDA & MACRO INTERNATIONAL INC. (2007) Uganda Service

Provision Assessment Survey 2008

MINISTRY OF HEALTH UGANDA (2010) Uganda Clinical Guidelines: National Guidelines on

Management of Common Conditions

NAO (Date unknown) Analytical Framework for assessing Value for Money -

http://www.bond.org.uk/data/files/National_Audit_Office__Analytical_framework_for_assessing_

Value_for_Money.pdf

TAYADE S & JADHAO P (2012) The Impact of Use of Modified WHO Partograph on Maternal and

Perinatal Outcome International Journal of Biomedical and Advance Research 256

UNICEF (2014) http://data.unicef.org/child-mortality/neonatal

http://www.bond.org.uk/data/files/National_Audit_Office__Analytical_framework_for_assessing_Value_for_Money.pdf

http://www.bond.org.uk/data/files/National_Audit_Office__Analytical_framework_for_assessing_Value_for_Money.pdf

http://data.unicef.org/child-mortality/neonatal

ANNEXES

ANNEX 1: INDICATORS ASSESSED AND THE ASSOCIATED CALCULATIONS FOR PAYMENTS - YEAR 2

70


71

NU Health Lesson Learned II Report

ANNEX 2: STANDARD UNIT OF OUTPUT (SUO) FORMULA USED BY UCMB

As the range of health care services provided by the PNFP is very broad and varies substantially

across the different levels of health care, from HC2 to hospital, it was suggested by NU Health

that the total health output of facilities be limited to a certain number of indicators, for which cases

will be collected from DHIS2 for each PNFP.

To make the data meaningful, it is presented under the form of Standard Unit of Outputs (SUO)

which is a method used by the UCMB (Uganda Catholic Medical Bureau)9 , by hospitals in their

annual reports and by Cordaid to reflect the health output of facilitates supported by its

Performance Based Financing scheme in Jinja and Kamuli Districts.

As described in the UCMB documents, this method is a composite index calculated with a

combination of the six following parameters which are then converted and expressed into their

outpatient equivalents to determine the health output of a HF:

In-patients episodes (IP);

Out-patient contacts (OP);

Deliveries carried out (DEL);

Immunisation doses administered (IMM);

Antenatal contact (ANC); and

Family Planning contacts (FP).

The composite indicator of activity, called Standard Unit of Output (SUO), is calculated by

attributing to each output a relative weight according to the level of care. The weighting attributed

to each of the 6 parameters, based on relevant literature described in the UCMB report, is

equivalent in terms of costs for managing one outpatient, when you manage for example one

inpatient from admission to discharge. Because the costs for managing patients vary with the

level of care, two different weighting scales have been designed by UCMB to reflect this

difference, as shown in the table below:

Equivalent weighting of each

parameter of the SUO formula

Hospital

and HC4

LLU (HC2

and HC3)

IP *15 *5

DEL *5 *2

OP *1 *1

9 PUBLIC PRIVATE PARTNERSHIP IN HEALTH. WHAT IS ITS EFFECT ON THE PERFORMANCE OF THE HEALTH SECTOR? Daniele Giusti, MD DTM&H MPH, Peter Lochoro, MB ChB MHSM, Andrea Mandelli, BA Economics http://www.ucmb.co.ug/data%20on%20ucmb/Reports/ARTICLES/Article%20Effect%20of%20PPP%20on%20Health%20System.pdf

http://www.ucmb.co.ug/data%20on%20ucmb/Reports/ARTICLES/Article%20Effect%20of%20PPP%20on%20Health%20System.pdf

http://www.ucmb.co.ug/data%20on%20ucmb/Reports/ARTICLES/Article%20Effect%20of%20PPP%20on%20Health%20System.pdf

72

NU Health Lesson Learned II Report

ANC *0.5 *0.3

IMM *0.2 *0.2

FP *0.2 *0.2

In other words, SUO provides a general idea of the volume of the main services produced by a

health unit by using the following two formulas:

- SUO for a hospital and HC4 = [15*IP+5*DEL+1*OP+0.5*ANC+0.2*IMM+0.2*FP]

- SUO for a LLU = [5*IP+2*DEL+1*OP+0.3*ANC+0.2*IMM+0.2*FP]

Using this formula and based on data extracted from DHIS2 on these 6 indicators for each PNFP

for the FY 2013-14, NU Health has calculated the Total Health Output for each HF, and been able

to compare the two groups of PNFP. The two PNFP groups of Lango and Acholi sub-regions are

sufficiently homogenous to ensure that the SUO method for comparison of Total Health Output is

adequate and useful.

Limitations of this method

One of the limitations of this method is the quality of the data presented by DHIS2. As per NU

Health experience, the reported figures are not totally accurate due to miscalculation or

misclassification by PNFP in the way their data are submitted to the DHT. A second limitation is

due to the SUO formula which, as any indicator, is limited to certain factors while excluding others.

Finally, it also does not differentiate weighting between HC3 and HC2 and between hospitals and

HC4, while in reality the higher the level of care, the higher the cost of delivering the same health

care services.

It should also be noted here that Lacor Hospital is excluded from this study as there is no

comparable institution in the Lango region offering a similarly large range of services. See page

18 of NU Health’s inception report for more detailed background on this statement.

Lessons Learned Report II March 2015 - Health Partners...

Documents

Transcript of Lessons Learned Report II March 2015 - Health Partners...