PERFORMANCE AUDIT FOR TRIGGERING IDA BUY-DOWN

Dr.Mizan Siddiqi, Principal InvestigatorPUBLIC HEALTH SERVICES AND SOLUTIONS (PHSS)

NOW MORE THEN EVER: STOP POLIO FOREVER

www.thephss.org

BACKGROUND• PARTNERSHIP FOR POLIO ERADICATION PROJECT

(PPEP) was funded by WB since 2003 through “IDA Buy-down mechanism”

• The buy-down will be triggered based on the results of performance audit by an independent auditor

• PHSS has been awarded to conduct the audit

NOW MORE THEN EVER: STOP POLIO FOREVER

STAKEHOLDES MEETING: PURPOSE• Present objective, methodology, timeline and

details of the plan• Agree on the plan, objective, methodology and

timeline • Define Roles and Responsibilities of partners in

the exercise • Provide feedback on the instruments, analysis

and outcome of the exercise

NOW MORE THEN EVER: STOP POLIO FOREVER

APPROACH

• Participatory• Consultative

• Objective: Evidence BasedBoth qualitative and qualitative information will be used to analyze performance and conclude findings

NOW MORE THEN EVER: STOP POLIO FOREVER

OBJECTIVE• To determine if the IDA funds achieved

intended results against agreed indicators.

• The achievement will be measured by two indicators. These are:

NOW MORE THAN EVER: STOP POLIO FOREVER

INDICATORS• INDICATOR A: Timely arrival of OPV vaccine

at national level (3 weeks before SIA)

• INDICATOR B: OPV Immunization coverage of SIA (at least 80 percent in endemic states)

NOW MORE THAN EVER: STOP POLIO FOREVER

METHODOLGY: INDICATOR A Timely arrival of OPV vaccine at national level (3 weeks before SIA)

• Review of documents• Stake-holders’ Interview

• Analysis of Vaccine Arrival Report (VAR)

NOW MORE THAN EVER: STOP POLIO FOREVER

LIST OF STAKEHOLDERS

S/N Organization Position

1 NPHCDA ED2 NPHCDA Director, DC and Imm.3 ICC Member4 Unicef EPI Team Leader5 Unicef EPI Logistics6 Unicef Country Represntative7 Unicef Contract Specialist8 Unicef Sharif S Alam9 Rotary International Team Leader, Polio Plus10 WHO Team Leader, EPI11 WHO Country Represntative12 DfID Health Specialist13 BMGF Health Specialist

14 BMGF Health Specialist

15 World Bank16 World Bank Health Specialist17 World Bank Health Specialist18 JAICA Health Specialist19 USAID Health Specialist20 CDC Polio Team Leader

EXPECTED OUTPUT (INDICATOR A): VACCINE ARRIVAL TIMEThe analysis of Vaccine Arrival Time will be influenced by external and internal factors:

NOW MORE THAN EVER: STOP POLIO FOREVER

Factor External/Internal

Delay in forecasting InternalDelay in fund transfer InternalAvailability of vaccine in global market ExternalChange in epidemiology of WPV ExternalRegistration of new type of vaccine InternalDelay in transportation of vaccine (availability of flight) ExternalDelay in clearing of vaccine on arrival InternalStorage capacity in the country Internal

The assessment will look into these factors of vaccine procurement to determine if the IDA funds achieved intended results against the Indicator A (Vaccine Arrival Time). External Factors will not be considered as a failure by GON for delay in vaccine arrival.

INDICATOR BOPV Coverage of SIA

(at least 80 percent in selected endemic states)

NOW MORE THAN EVER: STOP POLIO FOREVER

INDICATOR B: OBJECTIVE• To determine OPV coverage among 0-59

months old children in four high risk endemic states

-Katsina, Zamfara, Kano and Borno (19th ERC)

NOW MORE THAN EVER: STOP POLIO FOREVER

INDICATOR B: METHODOLOGY• Four individual cluster surveys one for each state

will be conducted during the week of October 29, 2010 (October 29-31) after the IPD is completed (Oct 23-28, 2010)

• WHO 30-cluster coverage survey methodology will be used to select sample at the household level

(WHO Immunization Coverage Cluster Survey- Reference Manual 2005)

NOW MORE THAN EVER: STOP POLIO FOREVER

INDICATOR B: METHODOLOGYSAMPLE SIZE

State Number of survey unit

Number of Clusters

Number of 0 to 59 Months Old Children (OPV coverage in October 2010 IPD)

Katsina 1 30 210

Zamfara 1 30 210

Kano 1 30 210

Borno 1 30 210

Total 4 120 840

NOW MORE THAN EVER: STOP POLIO FOREVER

INDICATOR B: METHODOLOGYSELECTION OF CLUSTERS

• 30 clusters will be selected using stratified cluster sampling method from three strata of wards i.e. VHR, HR and LR wards.

• The number of clusters in each stratum (sample size of each stratum) will be allocated proportionally to size of the strata .

NOW MORE THAN EVER: STOP POLIO FOREVER

INDICATOR B: METHODOLOGY (contd.)SELECTION OF HOUSEHOLD

• Households will be selected strictly randomly.

• WHO-30 cluster methodology will be used to select starting and subsequent household.

NOW MORE THAN EVER: STOP POLIO FOREVER

INDICATOR B: METHODOLOGY (CONTD.)

SELECTION OF RESPONDENT AND CHILD

• One child under five years of age per household will be selected

• The vaccination status of the child will be checked from the marking on the little finger of the child’s left

• For all eligible children (one per household), the parent/caregiver will be interviewed of the selected child questions.

NOW MORE THAN EVER: STOP POLIO FOREVER

INDICATOR B: METHODOLOGY (CONTD.)SURVEY INSTRUMENT

The questionnaire will be developed based on the recent LQAS and Outside monitoring survey. In addition to OPV status, few additional questions will be asked to find reasons for non-vaccination.

NOW MORE THAN EVER: STOP POLIO FOREVER

INDICATOR B: METHODOLOGY (CONTD.)FIELD WORK

Training • A one day training will be given to interviewers on sample

selection and data collection instrument.• An operational manual will be developed to guide training

and filed work.

Data CollectionSix teams (one interviewer and one recorder) will collect data in each state. They will be supervised by State Coordinator and Senatorial District Monitors.

NOW MORE THAN EVER: STOP POLIO FOREVER

SMS ReceiverModem

Public Health Services and Solutions (PHSS)

Sender SMS Sent to Web

Data analysis

Dashboard

PHSS Real Time Monitoring System using web/sms will be deployed

www.thephss.org

MONITORING

NOW MORE THAN EVER: STOP POLIO FOREVER

FILED WORK (Contd.)Quality Control

• The state monitor will conduct supervised interviews in his/her designated clusters with the teams on a daily basis

• State coordinator will conduct random spot checks for all teams in the state.

NOW MORE THAN EVER: STOP POLIO FOREVER

Data Management and Analysis• All data will be entered in a computerized data

base and analyzed using Epi-Info software

• Various tables and charts will be produced using MS Excel and MSWord

• The analysis will be limited to key indicators and expected output

NOW MORE THAN EVER: STOP POLIO FOREVER

INDICATOR B:Expected Output

OPV Coverage in 4 Endemic States(Katsina, Zamfara, Kano and Borno)

NOW MORE THAN EVER: STOP POLIO FOREVER

Report Outline

The report will have 4 chapters:

• Chapter 1 Background, objectives, design and implementation of the audit

• Chapter 2 Analysis of the findings of Indicator A including comments on results against target

• Chapter 3 Analysis of the findings of Indicator B received from cluster survey in four states including comments on results against target

• Chapter 4 Discussions and Conclusion

TEAM

Principal InvestigatorDr.Mizan Siddiqi

Co-Principal Investigator Garry Presthus

Other Staffs:-M&E Specialist-Immunization Specialist-Biostatistician-Data Management Specialist -State Coordinators, State Monitors, Interviewers.

WORKPLANItem No. Activity WK

1WK2

WK 3

WK4

1. Presentation of work plan, methodology, reporting format and timeline to WB and FGoN and make adjustments as necessary, recruitment of staff

X

Indicator A (Vaccine Arrival)

2. Review of documents X

3. Key Informant Interview X X

4. Analysis of information X X

Draft Report on Indicator A X

Indicator B (Cluster Survey)

5. Collection of population data by ward and risk category from states for sampling X

6. Design, develop and host web site and survey monitoring software X

7. Training of field staff X

8. Data Collection in 4 states X

9. Design of data entry screen X

10. Data entry X

11. Data Cleaning and analysis X

12. Draft Report on Indicator B X

13. Presentation of overall draft report X

14. Final Report X

Assistance from StakeholdersNPHCDA1. Leadership, Coordination and Facilitation of the study2. Support at the State and LGA level

WHO 3. Review of methodology, sampling frame and data collection instruments4. Out side and inside monitoring data of OPV coverage (2007-2010 by SIA) four states5. LQAS data of OPV coverage for four selected states (2009-2010 by SIA)6. Support for survey at the State and LGA level7. ERC Recommendations and SIA schedule (2007-2010) 8. Assign Technical Staff to support the team

Unicef9. VAR report (2007-2010) and information surrounding global vaccine market, availability and reasons

for delay in arrival due to external factors10. Stakeholder Interview11. Documents related to vaccine procurement12. Assign Technical Staff to support team

World Bank13. Project Documents14. Coordination with stakeholders

All partners1. Stakeholder Interview2. Review of draft report

YOUR COOPERATION IS MUCH APPRECIATEDThank You All

NOW MORE THAN EVER: STOP POLIO FORVER

Please visit www.thephss.org/ppep Register

Join the forum and post comments

If we can not meet you because of your busy schedule/travel etc. please fill up the online questionnaire and submit.

All materials related to the exercise are posted for your information and support.

For any question please send an e-mail to: mizansiddiqi@thephss.org or call 07067275270