INDEPTH Network Effectiveness and Safety Studies Platform (INESS)
Introduction to Systems Effectiveness Modules
Don de Savigny
INDEPTH Scientific Advisory Committee
Swiss Tropical Institute, Basel
Pune, 28 October, 2009
INESS systems effectiveness objective
To assess the effectiveness, and determinants of effectiveness, of new malaria treatments in real world health systems.
Challenge
For INDEPTH DSS Sites…
To move beyond population health observatories to include a health system observatory function
To link population health and health behaviours to health services and to health system behaviours
= 37%
98%
Driving with the brakes on:How interventions lose traction in health systems
Example of ACT anti-malarial treatment in Rufiji DSS in 2006
Efficacy
X Access
X Diagnostics
X Provider compliance
X Patient adherence
Effectiveness
X 95%
X 95%
X 70%
X 60%Health system factors
Averages mask inequities
X 40%
X 90%
X 75%
X 60%
= 16%Poorest quintile
Data source: IMPACT Tanzania. Effectiveness data are actual. Poorest quintile estimates are hypothetical
What does this mean?
Presently more traction can be gained by removing health system bottlenecks than by improving the efficacy of new drugs.
INESS Technical approach for systems effectiveness Seven linked study modules provide the ingredients
for the effectiveness estimation:
Module Task team facilitator
1 Access STI
2 Diagnostic targeting CDC
3 Provider compliance CDC
4 Patient Adherence STI
5 Community acceptability CDC
6 Contexts and other effects STI
7 Costs and cost effectiveness SPH
Level
HH
HF
HF
HH
Community & HF
District & HH
District, HF & HH
INESS: Understanding barriers to effectiveness
Therapeutic efficacy
Targeting Actual PracticeComplianceAccess Adherence Effectiveness
Costs
HH HHHF HF
Module 1. Access
Main purpose: determine proportion of cases needing to seek care
that actually gain physical access to a point of provision
Quick overview:
Household surveys of fevers in prior two weeks Determines who was able to access authorized provider within 24h Determines reasons for choices and failed access Analyzes across time, space, socio-economic quintiles and provider
characteristics
Module 1. AccessMore details: DSS Total Population Monitoring via three core questions for every DSS household
Any fever in prior two weeks If yes, who (name, permanent ID) Did he/she take an antimalarial
Provides annual pattern of fever burden
DSS Household Access Sample Survey for in-depth assessment of care seeking and access on sample of those with fever (on PDAs) Sample size ~ 21,000 per year 2 hh per routine DSS enumerator per week requiring full interview Modified Malaria Indicator Survey instrument to identify:
ACT provider Delay and sequence of care seeking Whether any diagnostic test done for the ACT Whether and what treatment(s) obtained Whether full ACT course continuing or completed Costs of episode
RDT conducted and referral if needed Sample size ~ 1,690 per year
Powered to provide estimate +/-5% of proportion of RDT +ve febrile individuals having access to a source of ACT within 24 and 48h in both rainy and dry seasons by equity quintile.
Module 2 & 3. Diagnostic targeting & Provider compliance
Main purpose: determine the proportion of cases having access that are
correctly diagnosed / classified determine the proportion of correctly diagnosed cases
that are provided with the correct treatment
Quick overview:
Health facility / provider surveys Sampled at peak and low seasons Exit interviews with gold standard diagnostic Determines the drug and instructions provided or prescribed Assesses stock-outs and quality of drugs on hand Identifies cohorts for adherence follow-up survey at home
Module 2 & 3. Diagnostic targeting & Provider compliance
More details: Stratified sampling of ACT providers Sample size: 1,750 patients per year over two seasons All patients presenting as initial illness to sampled ACT provider on
day of survey Gold standard diagnostic Patient exit interview Pharmacy and supply inventory Health worker interview
Module 4. Patient adherence
Main purpose: Estimate proportion of patients who receive treatment who
use it as intended; and the proportion who are satisfied with the treatment
Quick overview:
Household survey Sample of exit subjects from Module 3 followed at home on day after last
scheduled dose (plus filter paper blood sample) Standard interviews for adherence and acceptability Further follow-up and filter paper blood at day 28 (and 42 depending on
ACT) Gold standard diagnostic available
Sample size: AdherenceThree levels of adherence: High (complete):65% of users with treatment failure of 5% Medium: 25% of users with treatment failure of 30% Low: 10% of users with treatment failure of 50%
Calculations Sample size required to detect a difference in treatment failure rate between the
two smallest groups, medium and low. With following parameters Confidence level: 95% Power 80% Ratio unexposed (medium adherence)/ Exposed (low adherence)
= 25% / 10% = 2.1 Prevalence of disease (treatment failure rate) in Exposed group: 50% Rate ratio = 50% / 30% = 1.67 We need 175 in medium adherence group and 70 in low adherence group. As the low group is expected to be 10% of all, we will need a total of 700 patients
to be followed through to the last day. This would have to be corrected upwards to account for the losses. Perhaps to 1000 patients per treatment for each drug.
Module 4. Patient adherence
More details: Sample size: 400 per season Visited one day after expected end of treatment course Asked about:
Doses taken on each day, individually Time specificity limited to morning, noon, afternoon, night How drugs were taken (with food, drink, etc) Vomiting and specifics
Pills remaining and packaging examined Filter paper blood sample taken 28 day interview and filter paper blood sampling visit scheduled
Module 5. Community acceptance
Main purpose: Examine the social, cultural and behavioural factors
that facilitate or impede uptake and adherence to new ACTs when introduced
Quick overview:
Community survey of three different populations Persons having a recent malaria fever episode (45-50 interviews) Adult men & women living in DSS area (15 FGDs per year) ACT providers (15-20 interviews)
Two communities <5km and two communities >5km from ACT
Module 6. Contexts and additional effects
Main purpose: Estimate the contribution to reduced morbidity &
mortality.
HMIS document reviews for trends and patterns in: Proportion of fevers recorded as malaria (OPD, IPD) Severe anemia Incidence of severe malaria Proportion requiring transfusion
DSS database and VA review for trends in: All cause and malaria specific mortality Health seeking prior to malaria death from verbal autopsy ITNs and IRS coverage
District plan and budget reviews for trends in: Health system changes Malaria expenditures as a share of all expenditure
Other contextual data (rainfall, EIR, molecular markers for resistance) Repeat therapeutic efficacy (100 patients)
Module 7. Overall effectiveness and costs
Main purpose: Determine the effectiveness, and the
determinants of effectiveness
Putting it all together Determine overall population effectiveness by equity quintile Determine the efficacy losses, and where the greatest losses occur Determine the costs of change, comparative financial costs, and
expected cost-effectiveness
Systems effectiveness: 20 IndicatorsElement of effectiveness Indicator
Access •Proportion of people with fever who have sought contact with a provider who should have the drug •Proportion of people with fever who seek care from other providers
Availability •Proportion of providers with the product in stock•Proportion of time product is in stock
Targeting accuracy •Proportion of malaria positive patients correctly diagnosed/classified by health providers
Compliance (health system and worker)
•Proportion of prescriptions which are correct (in accordance with manufacturer’s or MOH guidelines)
Adherence (individual) •Proportion of people who receive product and take as prescribed
Acceptability •Proportion of people who are satisfied with the tested antimalarial (qualitative assessment) •Proportion of people actually opting for the tested antimalarial
Other measures of effectiveness (including sensitivity of drug overtime)
•Parasitological cure rate (clearance of the initial parasite infection by day 7, persisting at D28, with PCR correction and/or in vitro and molecular markers as proxies for these measures optional for sites with capacity to measure them) •Parasite and anaemia prevalence•Blood drug level •Proportion of cases recorded as malaria in health facilities (outpatient + admissions) •Incidence of severe malaria and malaria-related anemia •Proportion of malaria cases requiring blood transfusions •Mortality rate (all causes, malaria-specific)
Related malariologic parameters that could influence findings
•Entomological Inoculation Rate (EIR)•Coverage of other malaria-control interventions
Costs / cost-effectiveness (based on a standardized cost tracking system)
•Incremental financial costs of drugs policy (costs of drugs + costs of other activities required to change policy)•Costs per clinical outcome
Systems Effectiveness Task Teams
Will assist with: Development of field protocols Piloting protocols in initial sites / countries Developing training and capacity strengthening approaches General oversight on module performance Data management & analysis
Thank you
District expenditure shares – all strategies
District absolute annual per capita expenditure – communicable diseases
Rufiji District 2007
Estimating District ACT requirements
Top Related