Practical Issues in Implementing Performance-Based Contracting

Practical Issues in Implementing Performance-Based Contracting

Health System Innovations Workshop

Abuja, Jan. 25-29, 2010

A Few Thoughts to Start• Don’t Panic: Not everything has to be perfect,

be creative! Make new & interesting mistakes –don’t repeat old ones

• Be Systematic: Write things down in a contract and contracting manual

• The Limits of Planning: Endless planning and analysis can get in the way of action & learning

• Humility: a) don’t be too sure of thingsb) knowledge must be larger than our experiencec) give people at local levels sufficient autonomyd) keep learning, evaluating, adapting

• :

Performance-Based Contracting for Health Services in Developing Countries-A Toolkit

Outline of the Toolkit1. Summary of the Toolkit – pages 1-8

2. What is performance-based contracting? Definitions and Concepts – pages 9-17

3. How to Contract? Going through the 7 steps of the contracting cycle – pages 19-66

4. Checklist for Contracting – quick summary of tasks in contracting – pages 67-68

5. Whether to contract? Review of the global experience with contracting – pages 69-96

Outline of the Toolkit• Appendix A: Example of a Contracting Manual

including an example of a contract

• Appendix B: World Bank procurement approach to contracting for health services

• Appendix C: Description of Evaluated Contracting Experiences

• Appendix D: TORs for 3rd party evaluation

• Appendix E: TORs for contracts involving: (i) PHC delivery; and (ii) HIV prevention for CSWs

• WWW.rbfhealth.org - tools and guidelines

http://www.rbfhealth.org/

2. Define the

Services

3. Design the

Monitoring and

Evaluation

5. Arrange for

Contract

Management

7. Carry out

Bidding Process and

Manage the

Contracts

1. Dialogue with

Stakeholders

4. Decide how to

Select Contractors

and Establish Price

6. Draft Contract &

Bidding Documents

The Contracting Cycle: A Systematic Approach (page 20)

The 6 most important mistakes1. Not clearly defining the objectives of the contract

and the indicators of success

2. Allowing contractors & purchasers to forget the stated objectives & targets

3. Limiting the managerial autonomy of contractors

4. Not defining the size & location of each “lot”

5. Not having a contracting plan: not indicating how contracts will be managed & M&E will be done

6. Setting prices irrationally and not choosing the best contractors

Defining Objectives – More than SMART

• Big advantage of contracting is results focus so concentrate on outputs not inputs.

• The purchaser should objectively define:– Quantity of services (e.g. % DTP3 coverage, skilled

birth attendance)

– Technical Quality (national technical guidelines)

– Equity (ensuring the poor receive services)


• Need to address the most important challenges

– In Ghana 70% of HIV transmitted through FSWs, guess what % of grants went to FSWs?

• Focus on a Few!! Indicator inflation if > 10 loss of focus, less data collection

• Bias towards outputs & outcomes not inputs & processes e.g. DPT3 coverage better than vaccine availability or “micro-planning” – the exception is in measuring “quality”


• They need to be defined in a measurable way –define numerator & denominator precisely.

• Avoid undefined terms like “functional” health facility

• Set targets broadly: 20% 58% DPT3 coverage is good even if the target was 60%

• Within the “span of control” of the contractor, e.g., measuring availability of staff if contractor not allowed hire, transfer, decide payments to health workers

Some Examples of Indicators –What’s Wrong with Them?

1. % of county hospitals with functioning x-ray machines

2. Number of health workers receiving appropriate training

3. % of patients using a PHC Center who are satisfied with the services

4. Low rate of health workers leaving their positions


• Refer to Tasks 4 and 5 (pages 26-31)

• Look at table 3.2 (page 28)

• Congratulations!! You’ve avoided the first important mistake

Ensuring Focus on Outputs

• Regular (quarterly) discussion of progress on indicators between purchaser and contractor

• Carry out independent M&E

• Credible threat of sanctions:

– Embarrassment works!! Special meetings

– Letters to NGO board

– Replacement of key managers

– Termination of contract

• RBC!!!! (Read the bloody contract)

Ensuring Focus on Outputs –Performance Bonuses

• Sends signals to contractors about important indicators

• Challenges of performance bonuses:

– Finding indicators that are important & can be measured reasonably frequently

– Amount of bonus – enough to signal but not too expensive – about 10%

– What contractor can do with the bonus

– Design of bonus: (i) improvement from baseline; (ii) absolute target

Design of Bonus System

Lot A Lot B Lot C

Score year 0 40 65 50

Score Year 1 50 70 70

Score Year 2 60 75 75

Change from previous best (+10 points)

Yes,Yes

No,No

Yes,No

Absolute Target (=70 points) No,No

Yes,Yes

Yes,Yes

Change and/or above 70 points Yes,Yes

Yes,Yes

YesYes

Ensuring Focus on Outputs-Results-Based Financing

• “Transfer of money or material goods conditional on taking a measurable health related action or achieving a predetermined performance target.” - Eichler and Levine

• Contractors are provided payments based on the amount of services they actually deliver

Ensuring Managerial Autonomy

• Decentralize management to people who are closest to the ground reality -

• Purchaser can hold contractors accountable for results when managers have responsibility & autonomy - avoids the “blame game”

• Encourages innovation – RBF, sub-centers

• Take advantage of private sector’s flexibility –that’s why they’re called Non-GovernmentalOrganizations

Impediments to Managerial Autonomy

• Telling contractors “how” they should deliver services (define objectives, “what”)

• Line item budgets with reimbursement of actual expenditures– Limit flexibility to move money where it’s

needed

– Encourages micro-management

– Increases transaction costs & arguments

– Encourages focus on inputs

Lump-sum allows proper FM!!!

Example of a Line-Item Budget

Item Amount

1. Staff salaries $150,000

2. Drugs $40,000

3. Medical Equipment $20,000

4. Medical supplies $30,000

5. Maintenance & Repair $10,000

TOTAL $250,000

Impediments to Managerial Autonomy

• Following Government procedures for staff hiring, firing, transfer, & pay.

• Unclear authority of purchaser’s officials– Will try to force certain approaches

– Will claim power over more decisions including staff recruitment

• Government procures important inputs (allow contractors to do procurement, use private auditors)

Drug Availability Index (max = 100) according to who is responsible for drug procurement -

Afghanistan

30

40

50

60

70

80

90

100

Decentralized to NGO Centralized (CI) Centralized 3rd party

2004

2007

A Tale of 2 Countries: Afghanistan & DRC

• Both severely affected by conflict

• Poor countries with limited infrastructure

• Lots of donor money flowing in

• Lots of NGOs

• Both started contracting with NGOs to deliver health services around 2004

• Used different approaches to contracting

Government Controls on Contracted NGOs

Type of Control Afghanistan DRC

Plans and Fiduciary Controls

Annual work plans approved by Government NO YES

Annual procurement plans approved by Government NO YES

Centralized procurement of goods by Government NO YES

Reimbursement for individual expenditures NO YES

Ex ante approval of payments to health workers NO YES

Innovations have to be pre-approved NO YES

Monitoring & Evaluation

Regular quarterly review of reported results YES NO

High quality annual health facility surveys YES NO

Frequent field supervision YES NO

Change in the utilization rate for curative care (per 100 population per year) during the first

year of the contracts

0

10

20

30

40

50

60

70

1

2007 2008DRC

2004 2005Afghanistan

Increasing Managerial Autonomy

• See tasks 26 – 29 pages 55-59

Size of “Lots” – Economies of Scale

• Economies of scale in price per beneficiary– Fixed management and admin. costs.

• Likely more competition lower prices

• Large packages facilitates contract management

• Easier & cheaper to monitor and evaluate contractor performance with fewer lots

• Likely to reduce opportunities for corruption– Unscrupulous officials will intimidate small

contractors

– Easier to avoid “ghost” NGOs

Scale of Contracts – Provinces vs. Clusters in Afghanistan

• Provinces: 300,000 – 900,000 population

• “Clusters” of districts – 100,000 – 180,000

• Issues related to NGO capacity, equity

• Bid price of provinces = $4.05 per capita per year vs. $7.80 for clusters, high admin. costs

• MOH decided cluster approach was too expensive

Size of “Lots” – Arguments for Smaller Scale

• Concerns regarding contractor “capacity”– hard to predict & compared to what alternative

• Increased diversity – no oligopoly

• No disruption to existing providers– But could suffer from “Swiss cheese”

Distribution of NGO HCs in Afghanistan

un-served

Size of “Lots” - Recommendations

• Lots should cover at least 0.5 million population

• 7-20 contracts

• See task 8, pages 34-36

Use a Contracting Plan

• People tend to focus on the contract itself and the recruitment procedures

• Fail to systematically address:

1. How monitoring and evaluation will be carried out (otherwise indicators in contract are meaningless)

2. How contracts will be managed (client’s activities not described in the contact)

Monitoring and Evaluation

• Ensure that contracts remain output and outcome focused

• Learn lessons and improve performance

• Definitions: – “Monitoring” tracking the performance of

individual contractors

– “Evaluation” tracking the overall progress in service delivery of all contractors (comparison to other service providers)

The Process of M&E

• Collect baseline data early on & provide to all stakeholders (helps contractors identify important issues)

• Recruit 3rd party to help with M&E design & data collection while maintaining government oversight & involvement

• Clear responsibility for analysis of M&E data

• Ensure there’s data on effectiveness (coverage), equity, quality of care, cost

Recommendations

• M&E key to achieving good results –worth the investment needed.

• Use different sources of data – none are perfect

• Make M&E somebody’s job

• See tasks 10-15, pages 37-44

Contract Management

• Often done poorly

• Ministries of Health often have limited experience and understanding of contracting

• Often not clear who in the MOH is responsible for contract management

Major Issues and Challenges in Contract Management

• Paying contractors on time

• Avoiding corruption

• Proper supervision, monitoring & evaluation

• Solving problems – many related to relationships

• Maintaining government ownership, oversight, & involvement while avoiding micro-management

Some Possible General Solutions to Contract Management

• Recruiting sufficient number of talented people to manage contracts

• Allow sufficient budget

• Computerized contract management systems

• Provide incentives to contract managers based on results achieved by contractors, timely payment, lack of audit objections, etc.

Determining the Price of a Contract

There are basically 3 options, each with benefits and issues:

1. Competition at least partly on the basis of price

2. Negotiation of price with selected bidder

3. Fixed price where client sets price in advance

Competition, at least partly on Price

Advantages

• lowest price

• transparent

• encourages innovation

• reflects local realities

• benefits local NGOs

Disadvantages

• may take longer

• may end up with prices that are inconsistent

• may end up with prices that are too low or too high

Negotiations with Contractor

Advantages

• fairly quick

• reflects local realities

• if purchaser well motivated & savvy can be low cost

Disadvantages

• not transparent!!!

• may end up with prices that are inconsistent

• may end up with prices that are too low or too high

Fixed Price

Advantages

• fairly quick

• transparent

• uniform, hence “fair” (?)

Disadvantages

• rigid, “one size fits all”

• discourages innovation or cost savings

• hard to estimate real costs (be humble!)

Example of Competition (LCS or QCBS)

• Bidders submit technical and financial proposals

• technical proposals scored by evaluation committee

• financial proposals opened publicly

• In LCS, lowest price among technically responsive bidders is selected.

• In QCBS technical “score” combined with financial score using a weight (e.g. 80/20)

Example of LCS: Minimum Acceptable Technical Score = 60

Name

of

NGO

Technical

Score (St)

Financial

Proposal

($)

Financial

Score Sf =100

x Fm/F

Total

Score (S) =

St x T+ Sf

x P

A* 60 $2.0

million

100 68.0

B 70 $2.5

million

80 72.0

C 75 $4.0

million

50 70.0

Example of Competition (QCBS)

Name

of

NGO

Technical

Score (St)

Financial

Proposal

($)

Financial

Score Sf =100

x Fm/F

Total

Score (S) =

St x T+ Sf

x P

A 60 $2.0

million

100 68.0

B* 70 $2.5

million

80 72.0*

C 75 $4.0

million

50 70.0

* 2.0m/2.5m x 100=80 and (70 x 0.8) + (80 x 0.2) = 72.0

Contractor Selection – Competition vs. Sole Source

• Bidders want transparent process with a “level playing field” competition

• Competitive process will generally lead to “best” managers, most innovative ideas, “best” organizations, best prices

• Sole source selection is quick but not fair, not transparent, creates resentment, leads to “fat & happy” contractors, limits innovation & creative thinking

Situations Where Sole Source Might Make Sense

• Where a mission clinic has been providing services for 50 years

• Limited competition possible due to ongoing armed conflict

• Contractor bringing significant funds into a partnership

Selection Criteria

• Need to be clearly defined in advance

• Should not be excessively detailed

• Should not be excessively difficult

• Should look at:

– Experience of organization (track record)

– Key staff (quality of managers)

– Work-plan/strategy (lowest % of score)

Evaluation Process

• Independent evaluation committee

• Committee should include members external to client/purchaser:

– technical agency (e.g. WHO, UNICEF)

– representative of NGO community (obviously need to avoid conflicts of interest)

Maximizing Participation

• Prior consultations with many NGOs

• Advertise widely

• Hold information sessions

• Don’t make process too daunting

• Establish reasonable selection criteria

• Allow NGOs to form consortia (< 3 or 4)

• Avoid bid & performance bonds, guarantees

• “Grow” local NGOs by having a few small packages

Recommendation

• Use competition based at least partly on price whenever it’s feasible

• Modify approach to minimize disadvantages, e.g. fixed price is adjusted to reflect local variation

• See tasks 16-20 pages 44-51

Practical Issues in Implementing Performance-Based Contracting

Documents

Transcript of Practical Issues in Implementing Performance-Based Contracting