Understanding Web Searching Secondary Readings and So On… Will Meurer for WIRED October 7, 2004.
Systematic Synthesis of the Literature: Introduction to Meta-analysis Linda N. Meurer, MD, MPH...
-
Upload
chad-parker -
Category
Documents
-
view
241 -
download
0
Transcript of Systematic Synthesis of the Literature: Introduction to Meta-analysis Linda N. Meurer, MD, MPH...
Systematic Synthesis of the Literature: Introduction to Meta-analysis
Linda N. Meurer, MD, MPHDepartment of Family and Community Medicine
. . It is necessary, while formulating the problems of which in our advance we are to find solutions, to call into council the views of those of our predecessors who have declared an opinion on the subject, in order that we may profit by whatever is sound in their suggestions and avoid their errors.
Aristotle, De Anima
Clinical Overview Purpose:
Provide general information on a topic Good to review diagnosis and management Disseminate experience and opinions of an expert
Methods: Usually don’t include a methods secsion References chosen to illustrate points Conclusions may or may not be “evidence-based”
Critical/ Systematic Review Purpose
More focused topic; answers specific question(s) Represent a summary of systematically gathered and analyzed
primary research May lead to new conclusions/ knowledge Saves the busy clinician the work of interpreting multiple studies
on the same subject
Methods Should always include methods section with at least:
Study inclusion criteria, search strategy, analysis method References chosen through clear criteria to minimize author bias
Key characteristics of a systematic review
Clearly stated title and objectives Comprehensive strategy to search for relevant
studies (unpublished and published) Explicit and justified criteria for the inclusion or
exclusion of any study Clear presentation of characteristics of each study
included and an analysis of methodological quality Comprehensive list of all studies excluded and
justification for exclusion
Key characteristics of a systematic review (cont.)
Clear analysis of the results of the eligible studies statistical synthesis of data (meta-analysis) if
appropriate and possible; or qualitative synthesis
Structured report of the review clearly stating the aims, describing the methods and materials and reporting the results
Meta-analysis – Systematic Review with statistical synthesis
Purpose Usually answers one specific question Can generate summary estimates of effect from multiple
studies Considered primary research with included studies treated as
data
Methods Identical to other types of Systematic Reviews Explicit, systematic collection of studies Uses statistical procedures to combine data or results from
different but similar studies
Meta-analysis - advantages Increase statistical power
Resolve uncertainty when reports disagree
Improve precision of estimates of effect size
Answer questions not posed at the beginning of original studies through examination of study differences, sensitivity analyses
0.25 0.5 1.0 2.0 4.0
XXX
XX
XX
X
XX
Relative Risk
Example: Forrest Plot
Meta-analysis results often displayed graphically
Each X = results of a single study
Horizontal lines = 95% CI -X- represents weighted
summary estimate after combining all studies. Note better precision Most studies not significant
by themselves contribute to highly significant summary
Threats to validity
When considering whether the results of any study reflect ‘truth’, there are generally 4 threats:
Selection bias Study sample doesn’t represent the population of interest
Information bias Measurement errors, misclassification etc.
Confounding Association between variables due to or affected by their shared
association with another variable Chance
The probability that data reveals an association that is not real
Meta-analysis - limitations
Threat #1: Selection bias In the case of meta-analysis, reflects bias in the
selection or availability of studies included: Retrieval bias: Investigator conducting review
selects studies that support hypothesis (or are otherwise biased)
Reporting bias: Investigators of original studies only report data that supports view (e.g. drug sponsored?)
Publication bias: Only studies with statistically significant results make it to the journals
Minimizing selection bias Retrieval bias:
Systematic protocol a priori (before study starts) that includes Clear selection criteria Explicit exhaustive search for relevant articles Multiple reviewers
Reporting bias: Examine the source of support for work Conduct sub-analyses to see if source influences results
Publication bias: Seek unpublished sources of data Demonstrate through use of a funnel plot
Funnel Plot
A scatterplot of individual study results
(effect size) on the x-axis;
A measure of study size on the y-axis
As sample size goes up variance decreases a funnel shape forms
-4 -2 0 2 4 6 8
If a publication bias exists:
You might see a skewed plot
Hole in the funnel plot around the null suggests a bias
Results in an overestimate of pooled effect
-4 -2 0 2 4 6 8
Meta-analysis: potential threats to validity
Threat #2: Information biasQuality of a meta-analysis is dependent on
quality of original articles, including: Selection Measurement Confounding
The author should conduct a very careful validity assessment of each article included in the study
Meta-analysis: potential threats to validity
Threat #3: Confounding As with information bias, confounding in individual
studies will be transmitted into the meta-analysis.
Differences in populations studied, settings, specific intervention details (dose, duration), measurements used, etc. may result in differences in study results
This can increase generalizability if studies agree If studies do not agree, may need to explore confounders that
might account for disagreement (heterogeneity)
Meta-analysis: potential threats to validity
Threat #4: Chance By combining the results of smaller studies, the
increased power achieved produces a more precise estimate with greater statistical significance
Assuming the included studies are valid, inter-study variability will still occur
Statistical testing for homogeneity can determine whether this variability is greater than one would expect due to chance alone
Tests for homogeneity
Test the probability that observed differences among the results of individual studies were due to chance alone.
Reported as a Cochrane Chi Square (Q-statistic): statistical significance shows results are not
homogenous Due to outliers? What do you do?
Heterogeneity A clue that differences in the studies exist that may lead to
new discoveries: Design Population: risk factors, setting Intervention: dose, duration, preparation Measurements
Finding heterogeneity should prompt an author to explore these factors more fully
Finding heterogeneity also influences the choice of statistical model used to combine the data
You’ve found heterogeneity.So what does one do? Try to explain
Eliminate obvious outliers and retestSubgroup analysesRegression analysis on study characteristics
Incorporate between-study differencesUse a random effects model
Method/model2 statistical models used Fixed effect model
estimates treatment effect as if all studies are estimating one single true value
ignores between study variability;
Used when study results are homogeneous
Random effects model estimates treatment effect
as if each study is estimating a distinct value from a distribution of possible results
accounts for between- study variability
Should be used when heterogeneity exists
Summary
You have been introduce to the basic concepts and terminology you need to critically use a meta-analysis, including: Purpose Advantages Potential threats to validity Analysis methods
Please return to the ANGEL course page (should still be open in another window) and click proceed to move on.