Post on 13-Jan-2016
Extreme Makeover, Data EditionInside the Box
2007 City MatCH Conference – Skills Building Session
Juan M. Acuña M.D., M.Sc.
MCH Medical Epidemiologist
MCHEPI Program Team Leader, CDC
Objectives of the presentation
1. Review validity and the EBPH process
2. Review EPI principles guiding evidence application as a MCH decision driver
3. Review on current examples based on programmatic information in MCH
Relevant questions based on the previous presentation
• Is there a need for EBPH?
• What do we call evidence?
• How do we assess evidence?
• How do we provide recommendations based on the best evidence?
• How do we implement an action plan?
• How do we evaluate impact?
Changes or new programs are based on recommendations
A. Good evidence to support decisions
B. Fair evidence to support decisions
C. Poor evidence that does not provide direction to do or don’t do
D. Fair evidence to support don’t do
E. Good evidence to support don’t do
Let’s change the paradigm…
• MCH covers maternal, child, health
• Other close fields (injury, infections, death, behaviors, etc”
• MCH belongs to public health
• We are temporary MCH practitioners
• Let’s talk about public health
Basic concepts about EBPH
WORKING DEFINITION (PH):
Promoting healthy people in
healthy communities
Public Health• Prevents epidemics and spread of disease
• Protects against environmental hazards
• Prevents injuries
• Promotes and encourages healthy behaviors
• Responds to disasters and assists communities in recovery
• Assures the quality and access to services
APHA
Services
Monitor health status YesDiagnose, investigate public hazards YesInform, educate, empower NoMobilize community partners NoDevelop policies and plans NoEnforce laws and regulations NoLink people to health services NoAssure expert workforce NoEvaluate effectiveness, access, quality YesResearch new insights and solutions yes
Services data-based
Public Health data-action loop:
Case recollection
Population information
Risk factor data (PRAMS)
ANALYSIS
Programs and policies
RATES AND PROPORTIONS
1. absolute risk
2. population “mapping”
3. tendencies
Complex Analyses
• Cause
• risk factors
• costs
• morbidity
Program evaluation
INFORMATION
Inside the box
outside t
he box
Sources of evidence in PH
• “soft” information: review processes, personal information, “gut” feelings
• “adequate” information: routinely collected information, case review programs, passive systems
• “strong” information: active surveillance, some clinical studies
• “very strong”: randomized clinical trials
2. EPI components in public health problems
Examples
• Referral distance is associated to mortality
• Implications of fetal intervention (clinical and public health) to pediatrics and PH
Perinatal Mortality Rate by Distance from Shreveport, Region 7, 1995-9
57.4
18.324.3
61.9
16.5
30.3
9
34.444
010203040506070
0 20 40 60 80 100
Distance from Shreveport
Mo
rtal
ity
Rat
e
p:0.38
r:0.48
Example 1
So the main issue here is Validity
Valid truth
• Unbiased (bias needs to be avoided)• True and tolerable level of chance (chance
needs to be measured)• Took into account the presence of
characteristics that change the relationship of interest (controlled for confounding)
LBW - SGA LAPRAMS data 1998-1999
Population at risk
LA 1998-1999:
130,294 pregnancies
Smoking OR: 3.5
Wt-Gain OR: 3
Counseling OR: 1.7
Prevalence:
LBW: 7% (9,120)
VLBW: 2% (2,605)
SGA: 15% (19,544)
AFp:
LBW: 9% (820)(+?)
VLBW: 2% (52) (+?)
SGA: 2% (390)(+?)
Why the concern?
• Knowledge is rapidly expanding
• The use of “EB decision-making” is common
• Large amount of published (scientific) literature
• Larger amounts of (unused) stored data
• Lack of guidelines for the EBPH process
• Large degree of uncertainty about change
Critical appraisal of the evidence
• The most difficult step (for PH officials)
• Needs technical evaluation (epi)
• Needs “special” skills
• Less than 10% of active personnel has the skills required. Less than 1% has a method.
• Time-consuming step
• Source of “biased” decisions
Critical appraisal of the evidence• Needs assessment,
surveillance (diagnosis):– Prevalence
– Sensitivity
– Specificity
– PPV / NPV
– LRP / LRN
– Pretest odds / postT odds
– post-test probability
– Receiver-operator curve
• Prognosis:– time variables
– survival curves
– time series
– prognosis estimates
– precision
– relationship with screening and diagnosis
Critical appraisal of the evidence
• Intervention:– Blindness (evaluation)
– Randomization (evaluation)
– Control event rate
– Relative Risk reduction
– Absolute Risk reduction
– Number needed to treat
• Risk:– Odds ratio
– Risk ratio
– Incidence
– Prevalence
– Exposure
– inference
– probability distribution
Critical appraisal of the evidence
• Other more “general” terms:– Bias– confounders– probability distribution and chance– p value– confidence intervals– logistic regression, multivariate analysis– univariate analysis
Surveillance Systems
Epidemiological Studies
Prevention Programs
Risk factors
Protective factors
Public concerns
Prevention strategies
Public policy
Education
Prevalence rates
Registry of cases for study or referral
Monitor prevention
Example # 3:
Birth CertificatesPredictive Value Positive 76%Sensitivity 28%
Hospital Discharge DataPredictive Value Positive 85-95%Sensitivity 70-90%
Example # 3:
Evaluation of Data Studies
Conflict in PH
To do things right
To do the right things (right)
DRIVING FORCE: best evidence for the best practice
PROBLEMS: How is this done? How to do it always? How to do it always the same?
Best Evidence
• Makes sense (relevant)
• Unbiased
• Available
• Statistically significant (chance)
• Significant to Public Health
• Leads to correct decisions
Best EvidenceUnbiased: well designed (bias cannot be measured)
random error
average: 10 (very precise) average: 4
Best EvidenceUnbiased: well designed (bias cannot be measured)
Systematic error random error
average: 4 average: 4
Best Evidence
Available:
• Published (strength of evidence)
• Surveillance systems
• Routinely collected information
• Peer information
• Smart opinion
• Other
Evidence
I - Evidence from RCT
II-1 - Well designed non-randomized trials
II-2 - Cohort, Case Control analysis
II-3 - Comparisons of places, time, interventions, better more than 1 center
III - Opinion of authorities, descriptive studies, expert peer groups or committees
Best Evidence
Statistically significant:
Chance (randomness):
•p values
•Confidence intervals
95% p
no association
no difference
Measurable!
Evidence
Statistical significance
Meaningful to Public Health
BOTH
good best fair
We have been taught to accept statistical significance. If large samples (as in many cases), we are bound to have it, even if it is not meaningful.
Best EvidenceLeads to correct decisions: MSAFP vs. Folic Acid for NTDs
• UK and Northern Ireland: one of the highest prevalences in the world
• Prenatal screening for more than 35 years
• Prenatal termination of the affected pregnancies
0
5
10
15
20
25
UK
Ireland
MSAFP
Folic Acid Fortification
Fortificationlevel
Diet + %population> 0.4mg
Casesprevented
% casesprevented
Cases tooccur
0 0 0.6 0 0 2000140 100 2.2 405 20 1595350 250 19.8 919 46 1081420 300 30.6 1082 54 918360 400 48.3 1324 66 676700 500 61.3 1499 75 501
4. Change PH practices
Public Health is about:
• Research
• Advocacy
• Community Services
• Education
• Wisely invest as little money as possible to make the biggest and better change possible
Changes are based on recommendations
A. Good evidence to support decisions
B. Fair evidence to support decisions
C. Poor evidence that does not provide direction to do or don’t do
D. Fair evidence to support don’t do
E. Good evidence to support don’t do
Evidence
I - Evidence from RCT
II-1 - Well designed non-randomized trials
II-2 - Cohort, Case Control analysis
II-3 - Comparisons of places, time, interventions, better more than 1 center
III - Opinion of authorities, descriptive studies, expert peer groups or committees
Framework for evaluation• Do I want (have) to evaluate the study?
• Outline the study
• Is the study believable?
• What is the Public Health relevant finding?– Are the variables of interest included?
– Are findings explainable by chance, bias or confounders?
– Is the finding believable within our knowledge?
• Will the study help me with my population?– Is my population similar?
– Is my problem similar?
– Will the findings benefit my programs and policies?
Inside the Box Workshop Summary
We have:
1. Reviewed concept and situation of “evidence based” practice of public health
2. Reviewed relevant basic EPI concepts
3. Reviewed EPI aspects of public health
4. Provided element to build a framework to evaluate evidence
5. Evaluated the role of evidence in PH decisions