Performance Audit The GAVI Alliance Data Quality Audit experience
description
Transcript of Performance Audit The GAVI Alliance Data Quality Audit experience
Performance Audit
The GAVI Alliance Data Quality Audit experience
May 2008 ICGFM conference
Dr. Mercy AhunGAVI Alliance Secretariat, Geneva
Performance Audit• DQA history and rationale
• DQA overview
• Requirements
• Report Content
• GAVI DQA Experience
• Strengths of DQA
• Limitations of DQA
• Future Opportunities
History and rationale for DQA
• History: DQA emerged as an assessment tool to accompany the investment component of GAVI’s Immunization System Support (ISS) window
• Rationale: DQA aims to provide GAVI with assurance that the level of reward payable through ISS is based on a data validation process
Overview of Data Quality Audit
• Goal: assure that management of immunisation services are based on sound and accurate data
• Operational objectives:
1. assess the quality, accuracy, timeliness and completeness of immunisation reporting systems
2. audit the reported DTP3 coverage in infants in a specific calendar year and then estimate the National verification factor to use in reward allocations
Requirements• Countries receiving Immunisation Services
Support are required to conduct DQA during 2nd year of support
• External audit team engaged to review records & reports
• 4 districts randomly selected & data for previous year reviewed
• 6 Health facilities visited per district
• Number of vaccinations checked at facility, district & national levels
DQA report content• Performance indicators (DTP drop out rate,
wastage, coverage, Verification factor)
• Verification factor– Estimation based on all DTP3 records and reports
audited at facility & district level– Based on ratio of recounted DTP3 and reported
DTP3 (consistency )– Successful DQAs – VF of at least 0.80
• Quality of the system index: Composite index based on the recording, reporting, storage, M&E and system design for data
DQA reporting process
• In-country presentation
• Draft report with electronic copy
• Opportunities for feedback
• Confidentiality
• Monitoring of plan of action to address weaknesses in GAVI Annual Progress Report
GAVI DQA experience
Of the 52* GAVI-eligible countries that applied for ISS funding 2000-2007:
Number of Countries
Presented for review 43
Passed
Under review
* 9 countries did not require a DQA as ISS expenditure was below the minimum needed for review
39
4
Innovative: improving MIS
40
50
60
70
80
90
100
110
2002 2003 2004 2005 2006 2007
Average DQA 'verification
factor' (80=passing
score)
With introduction of Data Quality Audit (DQA), health data are becoming more reliable in all countries
Passed
All countries
Failed
78 11
5
415
1115
8
4
3
00 1
1
2
DQA strengths• Innovative Management, diagnostic and
capacity building tool – DQS – self assessment tool developed to
help countries diagnose and address data weaknesses
• Exposed need to improve skills in management of health data
• Induced improvements in repeat DQA countries• Enhances donor confidence and contributes to
transparency and accountability
Limitations of DQA
• Expensive & labour intensive
• Imprecision of verification factor– Wide confidence intervals
• Unable to capture verification of vaccination at point of delivery
Future opportunities for DQA
• Major evaluation underway to examine methods, sources scope
• Will review best practice and decision model
• Opportunities for expanding DQA to include other health system metrics
• Capacity building to enable local DQA or peer review process