Systematic Review Meta-analysis - med. · PDF file16/02/53 1 Systematic Review & Meta-analysis...
Transcript of Systematic Review Meta-analysis - med. · PDF file16/02/53 1 Systematic Review & Meta-analysis...
16/02/53
1
Systematic Review &
Meta analysisMeta-analysisAmmarin Thakkinstian, Ph.D.
Section for Clinical Epidemiology and BiostatisticsFaculty of Medicine, Ramathibodi Hospital
Tel: 02-201-1269, 02-201-1762 Fax: 02-2011284
e-mail: [email protected]
Course outline
• Lecture (20%)– Review methods– Meta-analysis
• Class discussion & tutorial (30%)– Review topics
• Assignments (50%)• Assignments (50%)
16/02/53
2
Topics Score Due DateAssignment I: - Initiating review topic- Literature review
o Background & rationale , o Formulate a review question & objectives
15% Feb 26, 2009
Assignment II: Locate and select studies- Define inclusion & exclusion criteria 12.5% Mar 6 , 2009- Search terms and strategies - Identifying studies- Selecting studiesAssignment III: - Design
o Data extraction formso Quality or risk of bias assessment
- Perform data extraction & assess quality of studies
12.5% Mar 13 , 2009
Assignment IV: Assignment IV: - Data analysis plan - Dummy tables & figures
15% Mar 20, 2009
Assignment V: Data analysis 20% April 17, 2009Assignment VI: Writing up paper (as a manuscript)- Presentation (verbal)- Submit paper
5%25%
May 7, 2009May 8, 2009
Systematic review
• Review methodology• Review methodology• Pooling methods: meta-analysis
– Dichotomous outcome– Continuous outcome
16/02/53
3
Systematic review
A review that has been prepared• A review that has been prepared using a systematic approach to minimise biases and random error
Rationale
• Tool for health care worker• Tool for health care worker, researchers, consumers, and also policy maker who want to keep up with the evidence that is accumulating in their field
16/02/53
4
Rationale• More objective appraisal of the
evidence than traditional narrativeevidence than traditional narrative reviews- Narrative review: subjective, selection
bias, limitation of single or few studies, unhelpful descriptions, e.g., no clear evidence, a weak relationship, a strongevidence, a weak relationship, a strong relationship.
- Systematic review: more transparent appraisal, allow reader to replicate, quantitative conclusion.
• Meta-analysis:
Rationale
– Estimates treatment effects– Leading to reduces probability of false
negative results (increase power of test)– Potentially to a more timely introduction
of effective treatments.of effective treatments.
16/02/53
5
Rationale• Exploratory analyses:
– subgroups of patients who are likely to respond particularly well to a treatment (or the reverse)
– may generate promising new research questions to be addressed in future studies.
• Systematic review may demonstrate th l k f d t id dthe lack of adequate evidence and thus identify area where further studies are needed
Terminology
• Systematic reviewSystematic review• Overview• Meta-analysis• Research synthesis• Summarizing• Summarizing• Pooling
16/02/53
6
Review Methodology• Good Rationale• Clearly state research question• Clearly state research question• Objective• Identify relevant studies/locate studies• Explicitly describe inclusion & exclusion
criteria of studies • Data extraction: study factor outcome• Data extraction: study factor, outcome• Data analysis• Results• Discussion
Review proposal• Introduction & background• Research question/objective
R i th d• Review methods– Locating studies– Selecting studies
• Inclusion/exclusion criteria – Data extraction – Quality/risk assessment – Data analysis
• Results• Discussion
16/02/53
7
Rationale • Why do we need to perform the review• How were results of previous individual p
and review study (if any) – Positive results – Negative results
• Methodologic issues – Sample size/Power of test p– Narrative reviews?
• Selective bias • Pooling effect sizes?
– Quality of the studies
16/02/53
8
Good research question • Evidence-base Medicine (EBM)
– Patient– Intervention– Comparator– Outcome– PICO
16/02/53
9
Research question• Is there association between VDR and
BMD/osteoporosis in women?BMD/osteoporosis in women? – To compare rates of osteoporosis in women
with BB with bb genotypes • Is Rosiglitazone higher risk of developing
myocardial infarction compare with Metformin indiabetic patients?diabetic patients? – To compare incidence of MI between diabetic
patients received Rosiglitazone and Metformin
Locate studies1. Defines source of database
– MEDLINE– MEDLINE- 1949to present• Over 16 million references • Since 2005, between 2,000-4,000 completed references are
added each day Tuesday through Saturday• Cover 5200 worldwide journals in 40 languages- Uses medical subject heading (MeSH) for index
Includes biomedicine and health science journals- Includes biomedicine and health science journals- English abstracts for 79% on references- 90% are English language articles- 47% of journals covered are published in the US
- PubMed available free of charge
From http://www.nlm.nih.gov/pubs/factsheets/medline.html
16/02/53
10
Defines source of databasesEMBASE- Over 12 million records from 1974-present- Over 12 million records from 1974-present - More than 600,000 records added annually- Covers over 4,800 active peer-reviewed journals
published in 70 countries/ 30 languages - uses EMTREE for indexing
includes English abstracts for 80% of references- includes English abstracts for 80% of references- daily update, within two weeks of receipt of the
original journal- Produced by Elsevier, no free version available
Defines source of database
• The Cochrane Controlled Trials Register (CCTR)(CCTR)
• ClinicalTrials.gov • HUGE NET Review• Reference lists
H d hi f k j l• Hand searching of key journals• Personal communication with expert in the
field
16/02/53
11
Locate studies
2 Define the software & version used2. Define the software & version used for searching
- PubMed- OVID version 1.0.2- Silver Platter
National Center for Biotechnology Information
16/02/53
14
3. Defines searching terms• Specify period of searching• Specify period of searching• Plan for update searching • Combinations of search terms
• Intervention: treatment/study factor• Outcome of interest • Comparator* • Patient
16/02/53
15
Example• VDR& BMD/Osteoporosis(J Bone Miner Res.
2004;19(3):419-28.)
1. vitamin D receptor or VDR (MeSH)2. genotype(s) or allele(s) or polymorphism(s) (MeSH)3. bone mineral density or BMD or bone density (MeSH)4. low bone mineral density or low density (textword)5. osteoporosis (MeSH)6. fracture (MeSH)6. fracture (MeSH)7. 1 and 2 and 38. 1 and 2 and 49. 1 and 2 and 510. 1 and 2 and 6
Selecting studies • Clearly define inclusion & exclusion criteria• Inclusion criteriaInclusion criteria
– Type of subjects• Treatment or exposure or gene• Comparator (if needed) • Outcome– Study design
• randomized controlled trial, • observational studies (cohort, case-control,
cross-sectional study)– Full paper
•
16/02/53
16
– Languages• English, French, others
– Restrictions due to sample size or other criteria
– Multiple publications of the same studies, choose the recent one or the
h id d l t fone has provided more completeness of data
Exclusion
– Incompleteness of information• Contact authors at least two times for
incomplete data– Not the outcome of interest – Review studies
Narrati e re ie• Narrative review • Systematic review
– Not comparative studies • No control group
16/02/53
17
Selecting studies
• Merge studies identified from databases i f ( E d t )using reference manager (e.g. Endnote)
– Remove duplicates • Two reviewers independently select
studies – Screen title/abstract to remove non-relevantScreen title/abstract to remove non relevant
studies base on eligibility criteria – Access full papers
– Examine other sources of studies C t t th if d d– Contact author if needed
– Final decision• Perform searching every 3 months
while doing a review
16/02/53
18
Thakkinstian et al. Meta-analysis of molecular association studies: vitamin D receptor gene polymorphisms and BMD as a case study J Bone Miner ResBMD as a case study. J Bone Miner Res 2004;19(3):419-28.
Any observational study (cohort, case-control, and cross-sectional study), regardless of sample size, which determined the difference in mean BMD di t th VDR t t t dBMD according to the VDR genotype, or tested the association between osteoporosis or fractureand the VDR genotypes and which fulfilled the following criteria:
16/02/53
19
VDR & BMD/Osteoporosis- The BMD measurements were performed at
lumbar spine or hip by Dual Energy X-raylumbar spine or hip by Dual Energy X-ray Absorptiometry (DEXA) or Dual Photon Absorptiometry (DPA) method.
- sufficient details: mean and SD of BMD, and number of subjects for each VDR genotype, and frequencies of genotype among case and control group were reported for dichotomous outcomes
VDR & BMD/Osteoporosis
• Participants were adult womenParticipants were adult women who were either pre- or post menopausal.
• Study factor & outcome– Bsm/Apa/Taq/Fok– Bsm/Apa/Taq/Fok– BMD/Osteoporosis
16/02/53
20
Data extraction (DE)• At least two reviewers• Design DEF, pilot, & revise DEF
Th ti l– The article• Study ID, • Author, • Year & source of publication
– The study characteristics• Type of studies subjects
th i it– ethnicity, – Adults vs children– Postmenopause, premenopause
• study design (RCT, CS, CC, CrS)• Methods used/criteria for measuring outcomes
DE (cont.)– Patients
• Demographic and clinical features of study's participants that might affect outcomes
–mean age, gender, BMI, smoking, underlying diseases
T bl f t d f t /i t ti– Table of study factors/interventions versus outcomes
16/02/53
21
DE (cont.)• Dichotomous outcome
– Frequencies between study factor/intervention vs outcomefactor/intervention vs outcome
GroupDisease
Yes No N I
Rx (Exp+) a b n1 a/n1Rx (Exp+) a b n1 a/n1
Placebo (Exp-) c d n2 c/n2
- OR (95% CI), RR (95% CI), HR (95% CI)
DE (cont.)
• Continuous outcome n mean (95% CI) G SD– n, mean (95% CI) Group n mean SD
A n1 mean1 SD1
B n2 mean2 SD2
16/02/53
22
Risk of bias in individual studies
• Quality Assessment (QA) • Consider internal & external validity
Risk of bias (cont.)• RCT
TC Chalmers From Egger M Smith GD– TC Chalmers From Egger M, Smith GD, Altman DG. Systematic reviews in health care: Meta-analysis in context. London: BMJ Books, 2001.
• Jadad’s quality assessment scale– The Cochrane Collaboration’s tool for
assessing risk of bias 2009 – Preferred reports of items for systematic
review and meta-analysis-PRISMA guideline
16/02/53
23
Domain Description Review authors’ judgement
Sequence generation. Describe the method used to generate the allocation sequence in sufficient detail to allow an
f h h i
Was the allocation sequence adequately generated?
assessment of whether it should produce comparable groups.
Allocation concealment. Describe the method used to conceal the allocation sequence in sufficient detail to determine whether
Was allocation adequately concealed?
detail to determine whether intervention allocations could have been foreseen in advance of, or during, enrolment.
Blinding of participants, personnel and outcome assessors Assessments should be made for each main outcome (or class of outcomes).
Describe all measures used, if any, to blind study participants and personnel from knowledge of which intervention a participant received. Provide any information relating to
Was knowledge of the allocated intervention adequately prevented during the study?
whether the intended blinding was effective.
Incomplete outcome data Assessments should be made for each main outcome (or class of outcomes).
Describe the completeness of outcome data for each main outcome, including attrition and exclusions from the analysis. State whether attrition and exclusions were reported, the
Were incomplete outcome data adequately addressed?
numbers in each intervention group (compared with total randomized participants), reasons for attrition/exclusions where reported, and any re-inclusions in analyses performed by the review authors.
16/02/53
24
Selective outcome reporting. State how the possibility of selective outcome reporting was examined by the review authors, and what was found.
Are reports of the study free of suggestion of selective outcome reporting?
Other sources of bias. State any important concerns Was the study f fabout bias not addressed in
the other domains in the tool.
If particular questions/entries were pre-specified in the review’s protocol, responses should be provided for each question/entry.
Trial methodology
apparently free of other problems that could put it at a high risk of bias?
Premature trial termination
Post-randomization exclusion
Unbalance baselineStatistical analysis
Unbalance baseline characteristics
Adequately describe methods of data analysis
-use per-protocol analysis, modified ITT
• Observational studies– Thakkinstian A D'Este C Eisman JThakkinstian A, D Este C, Eisman J,
Nguyen T, Attia J. Meta-analysis of molecular association studies: vitamin D receptor gene polymorphisms and BMD as a case study. J Bone Miner Res 2004;19:419-28.;
16/02/53
25
Criteria ScoreA. Representativeness of cases - Consecutive/randomly selected from
cases population with clearly defined random frame
2
1- Consecutive/randomly selected from cases population without clearly defined random frame or with extensively inclusion criteria
- Not describe method of selection
1
0
B. Representativeness of controls- Consecutive/randomly drawn from area
( d/ i ) i h h2
(ward/community) as cases with the same criteria
- Consecutive/randomly drawn from different area as cases
- Not describe
1
0
C. Ascertainment of osteoporosis/fracture- Clearly described objective criteria of
diagnosis of osteoporosis/osteoporotic fracture with proving diagnosis e g measure
2
Criteria Score
fracture with proving diagnosis, e.g. measure BMD using DEXA, X-ray for fracture
- Diagnosis of osteoporosis/osteoporotic fracture by patients history
- Not describe
1
0
D. Ascertainment of control- Controls were proved that they were not
osteoporosis/osteoporotic fracture, i.e. 2
p p ,measured BMD
- Only mentioned that controls were subjects who were not osteoporosis/osteoporotic fracture without proving
- Not describe
1
0
16/02/53
26
E. Ascertainment of VDR examination- Genotyping done under “blind” condition- Unblind or not mention
10
F. Data analysisHWE
Criteria Score
HWE:- Checking goodness of fit in control group
with appropriate statistics - Checking goodness of fit of HWE in case &
control group, or using inappropriate statistics
- Not mentioned
2
1
0
G Association assessment:G. Association assessment: - Appropriate statistic used with adjusting for
confounders e.g. logistic regression, or matched case-control design
- Appropriate statistic used without adjusting - Inappropriate statistic
2
10
H. Response rate- Response rates for both group are the 2
Criteria Score
Response rates for both group are the same or different between groups ≤ 5%
- Response rates are different between 5% - 10%
- Response rates are different 10% or more, or not mention about response rates
2
1
0
Total 15
16/02/53
30
Statistical analysis
• Unit of analysis is study NOT subject• Describe how data will be pooled
– OR, RR, RD, HR– Mean
• Heterogeneity Test– Test
– Degree (I2)
• Explore source(s) of heterogeneity– Meta-regression– Sub-group analysis
• Publication bias – Graph
Test– Test • Software • P value
16/02/53
31
Dummy tables
• Table 1. study’s characteristics • Table 2. cross-tabulate table
– Treatment vs primary outcome– Treatment effects & 95% CI– Pooled treatment effects
• Table 3 cross tabulate table of subgroup• Table 3. cross-tabulate table of subgroup• Table 4. cross-tabulate table of 2nd
outcome
Dummy figures
• Figure 1. Flow chart of selection of studies• Figure 2. Forest plot of 1st outcome• Figure 3. Forest plot of 2nd outcome