A precise water_abundance_measurement_for_the_hot_jupiter_wasp_43b
Meta-Analysis Concepts and...
Transcript of Meta-Analysis Concepts and...
![Page 1: Meta-Analysis Concepts and Applicationscms.education.gov.il/.../37798/MetaAnalysisConceptsandApplications.pdf · When effect is consistent • Provide precise estimate of the effect](https://reader031.fdocuments.us/reader031/viewer/2022031318/5c4e57c893f3c31436491f0b/html5/thumbnails/1.jpg)
Meta-AnalysisConcepts and Applications
Michael BorensteinHannah Rothstein
![Page 2: Meta-Analysis Concepts and Applicationscms.education.gov.il/.../37798/MetaAnalysisConceptsandApplications.pdf · When effect is consistent • Provide precise estimate of the effect](https://reader031.fdocuments.us/reader031/viewer/2022031318/5c4e57c893f3c31436491f0b/html5/thumbnails/2.jpg)
Table of ContentsIntroduction Slides 3- 15Goals Slides 16-17Effect Sizes Slides 18-67Fixed Effect Computations Slides 68-110Fixed Effects vs. Random
Effects Slides 111-133
![Page 3: Meta-Analysis Concepts and Applicationscms.education.gov.il/.../37798/MetaAnalysisConceptsandApplications.pdf · When effect is consistent • Provide precise estimate of the effect](https://reader031.fdocuments.us/reader031/viewer/2022031318/5c4e57c893f3c31436491f0b/html5/thumbnails/3.jpg)
Acknowledgments
![Page 4: Meta-Analysis Concepts and Applicationscms.education.gov.il/.../37798/MetaAnalysisConceptsandApplications.pdf · When effect is consistent • Provide precise estimate of the effect](https://reader031.fdocuments.us/reader031/viewer/2022031318/5c4e57c893f3c31436491f0b/html5/thumbnails/4.jpg)
Development funded by NIH
• National Institute of Mental Health• National Institute on Aging• National Institute of Drug Abuse
![Page 5: Meta-Analysis Concepts and Applicationscms.education.gov.il/.../37798/MetaAnalysisConceptsandApplications.pdf · When effect is consistent • Provide precise estimate of the effect](https://reader031.fdocuments.us/reader031/viewer/2022031318/5c4e57c893f3c31436491f0b/html5/thumbnails/5.jpg)
What is the goal of a meta-analysis?
![Page 6: Meta-Analysis Concepts and Applicationscms.education.gov.il/.../37798/MetaAnalysisConceptsandApplications.pdf · When effect is consistent • Provide precise estimate of the effect](https://reader031.fdocuments.us/reader031/viewer/2022031318/5c4e57c893f3c31436491f0b/html5/thumbnails/6.jpg)
When effect is consistent
• Provide precise estimate of the effect• Report if it is robust across range of
populations
![Page 7: Meta-Analysis Concepts and Applicationscms.education.gov.il/.../37798/MetaAnalysisConceptsandApplications.pdf · When effect is consistent • Provide precise estimate of the effect](https://reader031.fdocuments.us/reader031/viewer/2022031318/5c4e57c893f3c31436491f0b/html5/thumbnails/7.jpg)
When the effect varies
• May be used to qualify the mean effect• May make the mean effect irrelevant• May be of more interest than the
combined effect• What factors may explain the variation
![Page 8: Meta-Analysis Concepts and Applicationscms.education.gov.il/.../37798/MetaAnalysisConceptsandApplications.pdf · When effect is consistent • Provide precise estimate of the effect](https://reader031.fdocuments.us/reader031/viewer/2022031318/5c4e57c893f3c31436491f0b/html5/thumbnails/8.jpg)
Why perform a meta-analysis?
![Page 9: Meta-Analysis Concepts and Applicationscms.education.gov.il/.../37798/MetaAnalysisConceptsandApplications.pdf · When effect is consistent • Provide precise estimate of the effect](https://reader031.fdocuments.us/reader031/viewer/2022031318/5c4e57c893f3c31436491f0b/html5/thumbnails/9.jpg)
![Page 10: Meta-Analysis Concepts and Applicationscms.education.gov.il/.../37798/MetaAnalysisConceptsandApplications.pdf · When effect is consistent • Provide precise estimate of the effect](https://reader031.fdocuments.us/reader031/viewer/2022031318/5c4e57c893f3c31436491f0b/html5/thumbnails/10.jpg)
Streptokinase
• Meta-analysis in 1977 could have been definitive
• Additional 40,000 patients randomized• Additional ???? Patients not treated• Even in 1992, narrative review was not
definitive• Without meta-analysis, studies could have
continued indefinitely
![Page 11: Meta-Analysis Concepts and Applicationscms.education.gov.il/.../37798/MetaAnalysisConceptsandApplications.pdf · When effect is consistent • Provide precise estimate of the effect](https://reader031.fdocuments.us/reader031/viewer/2022031318/5c4e57c893f3c31436491f0b/html5/thumbnails/11.jpg)
Forest plot
• Transparent• A mechanism for understanding the
statistics• A mechanism for communicating the
statistics
![Page 12: Meta-Analysis Concepts and Applicationscms.education.gov.il/.../37798/MetaAnalysisConceptsandApplications.pdf · When effect is consistent • Provide precise estimate of the effect](https://reader031.fdocuments.us/reader031/viewer/2022031318/5c4e57c893f3c31436491f0b/html5/thumbnails/12.jpg)
Goals of meta-analysis
![Page 13: Meta-Analysis Concepts and Applicationscms.education.gov.il/.../37798/MetaAnalysisConceptsandApplications.pdf · When effect is consistent • Provide precise estimate of the effect](https://reader031.fdocuments.us/reader031/viewer/2022031318/5c4e57c893f3c31436491f0b/html5/thumbnails/13.jpg)
Assigning weights
• Compute combined effect• Assess heterogeneity• Use to qualify combined effect• Focus on heterogeneity
![Page 14: Meta-Analysis Concepts and Applicationscms.education.gov.il/.../37798/MetaAnalysisConceptsandApplications.pdf · When effect is consistent • Provide precise estimate of the effect](https://reader031.fdocuments.us/reader031/viewer/2022031318/5c4e57c893f3c31436491f0b/html5/thumbnails/14.jpg)
Steps
• Show sample• Show effects• Show weights• Show combined effect• How compute effects• How compute weights• How compute combined effect
![Page 15: Meta-Analysis Concepts and Applicationscms.education.gov.il/.../37798/MetaAnalysisConceptsandApplications.pdf · When effect is consistent • Provide precise estimate of the effect](https://reader031.fdocuments.us/reader031/viewer/2022031318/5c4e57c893f3c31436491f0b/html5/thumbnails/15.jpg)
Steps
• Show heterogeneity• How compute heterogeneity• Statistical implications• Practical implications
![Page 16: Meta-Analysis Concepts and Applicationscms.education.gov.il/.../37798/MetaAnalysisConceptsandApplications.pdf · When effect is consistent • Provide precise estimate of the effect](https://reader031.fdocuments.us/reader031/viewer/2022031318/5c4e57c893f3c31436491f0b/html5/thumbnails/16.jpg)
Goals of meta-analysis
![Page 17: Meta-Analysis Concepts and Applicationscms.education.gov.il/.../37798/MetaAnalysisConceptsandApplications.pdf · When effect is consistent • Provide precise estimate of the effect](https://reader031.fdocuments.us/reader031/viewer/2022031318/5c4e57c893f3c31436491f0b/html5/thumbnails/17.jpg)
Assigning weights
• Compute combined effect• Assess heterogeneity• Use to qualify combined effect• Focus on heterogeneity
![Page 18: Meta-Analysis Concepts and Applicationscms.education.gov.il/.../37798/MetaAnalysisConceptsandApplications.pdf · When effect is consistent • Provide precise estimate of the effect](https://reader031.fdocuments.us/reader031/viewer/2022031318/5c4e57c893f3c31436491f0b/html5/thumbnails/18.jpg)
Computing effect sizeand variance
![Page 19: Meta-Analysis Concepts and Applicationscms.education.gov.il/.../37798/MetaAnalysisConceptsandApplications.pdf · When effect is consistent • Provide precise estimate of the effect](https://reader031.fdocuments.us/reader031/viewer/2022031318/5c4e57c893f3c31436491f0b/html5/thumbnails/19.jpg)
Effect size
![Page 20: Meta-Analysis Concepts and Applicationscms.education.gov.il/.../37798/MetaAnalysisConceptsandApplications.pdf · When effect is consistent • Provide precise estimate of the effect](https://reader031.fdocuments.us/reader031/viewer/2022031318/5c4e57c893f3c31436491f0b/html5/thumbnails/20.jpg)
Effect size AND precision
![Page 21: Meta-Analysis Concepts and Applicationscms.education.gov.il/.../37798/MetaAnalysisConceptsandApplications.pdf · When effect is consistent • Provide precise estimate of the effect](https://reader031.fdocuments.us/reader031/viewer/2022031318/5c4e57c893f3c31436491f0b/html5/thumbnails/21.jpg)
Reporting precision
• Standard error• Confidence interval• Variance
![Page 22: Meta-Analysis Concepts and Applicationscms.education.gov.il/.../37798/MetaAnalysisConceptsandApplications.pdf · When effect is consistent • Provide precise estimate of the effect](https://reader031.fdocuments.us/reader031/viewer/2022031318/5c4e57c893f3c31436491f0b/html5/thumbnails/22.jpg)
Precision
• In primary study, qualifies the effect size• In meta-analysis, is used to assign weight
to effect size
![Page 23: Meta-Analysis Concepts and Applicationscms.education.gov.il/.../37798/MetaAnalysisConceptsandApplications.pdf · When effect is consistent • Provide precise estimate of the effect](https://reader031.fdocuments.us/reader031/viewer/2022031318/5c4e57c893f3c31436491f0b/html5/thumbnails/23.jpg)
Continuous data
• Start with means and SD• Raw difference• Standardized mean difference (d)• Bias-corrected standard difference (G)
![Page 24: Meta-Analysis Concepts and Applicationscms.education.gov.il/.../37798/MetaAnalysisConceptsandApplications.pdf · When effect is consistent • Provide precise estimate of the effect](https://reader031.fdocuments.us/reader031/viewer/2022031318/5c4e57c893f3c31436491f0b/html5/thumbnails/24.jpg)
Means and SD’s
5020100Control
5020110Treated
NSDMean
![Page 25: Meta-Analysis Concepts and Applicationscms.education.gov.il/.../37798/MetaAnalysisConceptsandApplications.pdf · When effect is consistent • Provide precise estimate of the effect](https://reader031.fdocuments.us/reader031/viewer/2022031318/5c4e57c893f3c31436491f0b/html5/thumbnails/25.jpg)
Raw mean difference
• Natural scale• Well known scale• All studies on same scale
![Page 26: Meta-Analysis Concepts and Applicationscms.education.gov.il/.../37798/MetaAnalysisConceptsandApplications.pdf · When effect is consistent • Provide precise estimate of the effect](https://reader031.fdocuments.us/reader031/viewer/2022031318/5c4e57c893f3c31436491f0b/html5/thumbnails/26.jpg)
Raw mean difference
1 2MeanDifference Mean Mean= −
2 21 1 2 2
1 2
( 1) * ( 1) *2Pooled
N SD N SDSDN N
− + −=
+ −
1 2
1 1 *MeanDifference PooledSE SDN N
= +
![Page 27: Meta-Analysis Concepts and Applicationscms.education.gov.il/.../37798/MetaAnalysisConceptsandApplications.pdf · When effect is consistent • Provide precise estimate of the effect](https://reader031.fdocuments.us/reader031/viewer/2022031318/5c4e57c893f3c31436491f0b/html5/thumbnails/27.jpg)
Raw mean difference
110 100 10MeanDifference = − =
2 2(50 1) *10 (50 1) *10 2050 50 2PooledSD − + −
= =+ −
1 1 * 20 4.050 50MeanDifferenceSE = + =
![Page 28: Meta-Analysis Concepts and Applicationscms.education.gov.il/.../37798/MetaAnalysisConceptsandApplications.pdf · When effect is consistent • Provide precise estimate of the effect](https://reader031.fdocuments.us/reader031/viewer/2022031318/5c4e57c893f3c31436491f0b/html5/thumbnails/28.jpg)
Raw mean difference
• Effect size – difference in means• Precision – SD within group, N
![Page 29: Meta-Analysis Concepts and Applicationscms.education.gov.il/.../37798/MetaAnalysisConceptsandApplications.pdf · When effect is consistent • Provide precise estimate of the effect](https://reader031.fdocuments.us/reader031/viewer/2022031318/5c4e57c893f3c31436491f0b/html5/thumbnails/29.jpg)
Raw mean difference
![Page 30: Meta-Analysis Concepts and Applicationscms.education.gov.il/.../37798/MetaAnalysisConceptsandApplications.pdf · When effect is consistent • Provide precise estimate of the effect](https://reader031.fdocuments.us/reader031/viewer/2022031318/5c4e57c893f3c31436491f0b/html5/thumbnails/30.jpg)
Raw mean difference
![Page 31: Meta-Analysis Concepts and Applicationscms.education.gov.il/.../37798/MetaAnalysisConceptsandApplications.pdf · When effect is consistent • Provide precise estimate of the effect](https://reader031.fdocuments.us/reader031/viewer/2022031318/5c4e57c893f3c31436491f0b/html5/thumbnails/31.jpg)
Standardized mean difference
• Proprietary scales • Multiple scales
![Page 32: Meta-Analysis Concepts and Applicationscms.education.gov.il/.../37798/MetaAnalysisConceptsandApplications.pdf · When effect is consistent • Provide precise estimate of the effect](https://reader031.fdocuments.us/reader031/viewer/2022031318/5c4e57c893f3c31436491f0b/html5/thumbnails/32.jpg)
![Page 33: Meta-Analysis Concepts and Applicationscms.education.gov.il/.../37798/MetaAnalysisConceptsandApplications.pdf · When effect is consistent • Provide precise estimate of the effect](https://reader031.fdocuments.us/reader031/viewer/2022031318/5c4e57c893f3c31436491f0b/html5/thumbnails/33.jpg)
Standardized mean difference (d)
Within
MeanDifferencedSD
=
21 2
1 2
1/ 1/2 * ( )dN N dSE
N N+ +
=+
![Page 34: Meta-Analysis Concepts and Applicationscms.education.gov.il/.../37798/MetaAnalysisConceptsandApplications.pdf · When effect is consistent • Provide precise estimate of the effect](https://reader031.fdocuments.us/reader031/viewer/2022031318/5c4e57c893f3c31436491f0b/html5/thumbnails/34.jpg)
Standardized mean difference (d)
10 0.5020
d = =
21/ 50 1/ 50 0.5 .2032 * (50 50)dSE + +
= =+
![Page 35: Meta-Analysis Concepts and Applicationscms.education.gov.il/.../37798/MetaAnalysisConceptsandApplications.pdf · When effect is consistent • Provide precise estimate of the effect](https://reader031.fdocuments.us/reader031/viewer/2022031318/5c4e57c893f3c31436491f0b/html5/thumbnails/35.jpg)
Standardized mean difference
• Effect size – Difference in means relative to SD within groups
• Precision – Sample size
![Page 36: Meta-Analysis Concepts and Applicationscms.education.gov.il/.../37798/MetaAnalysisConceptsandApplications.pdf · When effect is consistent • Provide precise estimate of the effect](https://reader031.fdocuments.us/reader031/viewer/2022031318/5c4e57c893f3c31436491f0b/html5/thumbnails/36.jpg)
Standardized mean difference (d)
![Page 37: Meta-Analysis Concepts and Applicationscms.education.gov.il/.../37798/MetaAnalysisConceptsandApplications.pdf · When effect is consistent • Provide precise estimate of the effect](https://reader031.fdocuments.us/reader031/viewer/2022031318/5c4e57c893f3c31436491f0b/html5/thumbnails/37.jpg)
Standardized mean difference (d)
![Page 38: Meta-Analysis Concepts and Applicationscms.education.gov.il/.../37798/MetaAnalysisConceptsandApplications.pdf · When effect is consistent • Provide precise estimate of the effect](https://reader031.fdocuments.us/reader031/viewer/2022031318/5c4e57c893f3c31436491f0b/html5/thumbnails/38.jpg)
Bias-corrected d (Hedges g)
314 * 1
Jdf
= −−
*G d J=
*SE SEG D J=
![Page 39: Meta-Analysis Concepts and Applicationscms.education.gov.il/.../37798/MetaAnalysisConceptsandApplications.pdf · When effect is consistent • Provide precise estimate of the effect](https://reader031.fdocuments.us/reader031/viewer/2022031318/5c4e57c893f3c31436491f0b/html5/thumbnails/39.jpg)
Bias-corrected d (Hedges g)
31 .9924 * 98 1
J = − =−
0.500 * 0.992 0.496G = =
0.203 * 0.992 0.202SEG = =
![Page 40: Meta-Analysis Concepts and Applicationscms.education.gov.il/.../37798/MetaAnalysisConceptsandApplications.pdf · When effect is consistent • Provide precise estimate of the effect](https://reader031.fdocuments.us/reader031/viewer/2022031318/5c4e57c893f3c31436491f0b/html5/thumbnails/40.jpg)
Bias-corrected d (Hedges g)
![Page 41: Meta-Analysis Concepts and Applicationscms.education.gov.il/.../37798/MetaAnalysisConceptsandApplications.pdf · When effect is consistent • Provide precise estimate of the effect](https://reader031.fdocuments.us/reader031/viewer/2022031318/5c4e57c893f3c31436491f0b/html5/thumbnails/41.jpg)
Bias-corrected d (Hedges g)
![Page 42: Meta-Analysis Concepts and Applicationscms.education.gov.il/.../37798/MetaAnalysisConceptsandApplications.pdf · When effect is consistent • Provide precise estimate of the effect](https://reader031.fdocuments.us/reader031/viewer/2022031318/5c4e57c893f3c31436491f0b/html5/thumbnails/42.jpg)
Multiple indices
![Page 43: Meta-Analysis Concepts and Applicationscms.education.gov.il/.../37798/MetaAnalysisConceptsandApplications.pdf · When effect is consistent • Provide precise estimate of the effect](https://reader031.fdocuments.us/reader031/viewer/2022031318/5c4e57c893f3c31436491f0b/html5/thumbnails/43.jpg)
Other data types
• Correlation• Survival• Events by person/years• One-armed studies• Generic indices
![Page 44: Meta-Analysis Concepts and Applicationscms.education.gov.il/.../37798/MetaAnalysisConceptsandApplications.pdf · When effect is consistent • Provide precise estimate of the effect](https://reader031.fdocuments.us/reader031/viewer/2022031318/5c4e57c893f3c31436491f0b/html5/thumbnails/44.jpg)
Study design and precision
• Independent groups vs. matched designs• Effect size is the same• Precision is different• Can combine in analysis
![Page 45: Meta-Analysis Concepts and Applicationscms.education.gov.il/.../37798/MetaAnalysisConceptsandApplications.pdf · When effect is consistent • Provide precise estimate of the effect](https://reader031.fdocuments.us/reader031/viewer/2022031318/5c4e57c893f3c31436491f0b/html5/thumbnails/45.jpg)
Data format
• Back-compute effect size and variance• Test statistics or p-values• Confidence limits
![Page 46: Meta-Analysis Concepts and Applicationscms.education.gov.il/.../37798/MetaAnalysisConceptsandApplications.pdf · When effect is consistent • Provide precise estimate of the effect](https://reader031.fdocuments.us/reader031/viewer/2022031318/5c4e57c893f3c31436491f0b/html5/thumbnails/46.jpg)
Compute d from p-value
![Page 47: Meta-Analysis Concepts and Applicationscms.education.gov.il/.../37798/MetaAnalysisConceptsandApplications.pdf · When effect is consistent • Provide precise estimate of the effect](https://reader031.fdocuments.us/reader031/viewer/2022031318/5c4e57c893f3c31436491f0b/html5/thumbnails/47.jpg)
Compute d from p-value
![Page 48: Meta-Analysis Concepts and Applicationscms.education.gov.il/.../37798/MetaAnalysisConceptsandApplications.pdf · When effect is consistent • Provide precise estimate of the effect](https://reader031.fdocuments.us/reader031/viewer/2022031318/5c4e57c893f3c31436491f0b/html5/thumbnails/48.jpg)
Compute SEOdds ratio from CI
![Page 49: Meta-Analysis Concepts and Applicationscms.education.gov.il/.../37798/MetaAnalysisConceptsandApplications.pdf · When effect is consistent • Provide precise estimate of the effect](https://reader031.fdocuments.us/reader031/viewer/2022031318/5c4e57c893f3c31436491f0b/html5/thumbnails/49.jpg)
Compute SEOdds ratio from CI
![Page 50: Meta-Analysis Concepts and Applicationscms.education.gov.il/.../37798/MetaAnalysisConceptsandApplications.pdf · When effect is consistent • Provide precise estimate of the effect](https://reader031.fdocuments.us/reader031/viewer/2022031318/5c4e57c893f3c31436491f0b/html5/thumbnails/50.jpg)
Multiple data formats
![Page 51: Meta-Analysis Concepts and Applicationscms.education.gov.il/.../37798/MetaAnalysisConceptsandApplications.pdf · When effect is consistent • Provide precise estimate of the effect](https://reader031.fdocuments.us/reader031/viewer/2022031318/5c4e57c893f3c31436491f0b/html5/thumbnails/51.jpg)
Caveat
• These slides are meant as a general introduction.
• They do not deal with special cases such as empty cells.
• They do not address variations in computational formulas.
![Page 52: Meta-Analysis Concepts and Applicationscms.education.gov.il/.../37798/MetaAnalysisConceptsandApplications.pdf · When effect is consistent • Provide precise estimate of the effect](https://reader031.fdocuments.us/reader031/viewer/2022031318/5c4e57c893f3c31436491f0b/html5/thumbnails/52.jpg)
Binary data
• Start with 2x2 table• Odds ratio• Risk ratio• Risk difference
![Page 53: Meta-Analysis Concepts and Applicationscms.education.gov.il/.../37798/MetaAnalysisConceptsandApplications.pdf · When effect is consistent • Provide precise estimate of the effect](https://reader031.fdocuments.us/reader031/viewer/2022031318/5c4e57c893f3c31436491f0b/html5/thumbnails/53.jpg)
2 x 2 Table
N2DCControl
N1BATreated
Non-EventsEvents
![Page 54: Meta-Analysis Concepts and Applicationscms.education.gov.il/.../37798/MetaAnalysisConceptsandApplications.pdf · When effect is consistent • Provide precise estimate of the effect](https://reader031.fdocuments.us/reader031/viewer/2022031318/5c4e57c893f3c31436491f0b/html5/thumbnails/54.jpg)
2 x 2 Table
1008812Control
100928Treated
Non-EventsEvents
![Page 55: Meta-Analysis Concepts and Applicationscms.education.gov.il/.../37798/MetaAnalysisConceptsandApplications.pdf · When effect is consistent • Provide precise estimate of the effect](https://reader031.fdocuments.us/reader031/viewer/2022031318/5c4e57c893f3c31436491f0b/html5/thumbnails/55.jpg)
Log odds ratio
( ) ADLog OddsRatio LogBC
⎛ ⎞= ⎜ ⎟⎝ ⎠
( )1 1 1 1
Log OddsRatioSEA B C D
= + + +
![Page 56: Meta-Analysis Concepts and Applicationscms.education.gov.il/.../37798/MetaAnalysisConceptsandApplications.pdf · When effect is consistent • Provide precise estimate of the effect](https://reader031.fdocuments.us/reader031/viewer/2022031318/5c4e57c893f3c31436491f0b/html5/thumbnails/56.jpg)
Log odds ratio
8 * 88( ) 0.45092 *12
Log OddsRatio Log ⎛ ⎞= = −⎜ ⎟⎝ ⎠
( )1 1 1 1 0.4808 92 12 88Log OddsRatioSE = + + + =
![Page 57: Meta-Analysis Concepts and Applicationscms.education.gov.il/.../37798/MetaAnalysisConceptsandApplications.pdf · When effect is consistent • Provide precise estimate of the effect](https://reader031.fdocuments.us/reader031/viewer/2022031318/5c4e57c893f3c31436491f0b/html5/thumbnails/57.jpg)
Log odds ratio
![Page 58: Meta-Analysis Concepts and Applicationscms.education.gov.il/.../37798/MetaAnalysisConceptsandApplications.pdf · When effect is consistent • Provide precise estimate of the effect](https://reader031.fdocuments.us/reader031/viewer/2022031318/5c4e57c893f3c31436491f0b/html5/thumbnails/58.jpg)
Log odds ratio
![Page 59: Meta-Analysis Concepts and Applicationscms.education.gov.il/.../37798/MetaAnalysisConceptsandApplications.pdf · When effect is consistent • Provide precise estimate of the effect](https://reader031.fdocuments.us/reader031/viewer/2022031318/5c4e57c893f3c31436491f0b/html5/thumbnails/59.jpg)
Log risk ratio
1
2
/( )/
A NLog RiskRatio LogC N⎛ ⎞
= ⎜ ⎟⎝ ⎠
( )/ /
Log RiskRatioB A D CSEB A D C
= ++ +
![Page 60: Meta-Analysis Concepts and Applicationscms.education.gov.il/.../37798/MetaAnalysisConceptsandApplications.pdf · When effect is consistent • Provide precise estimate of the effect](https://reader031.fdocuments.us/reader031/viewer/2022031318/5c4e57c893f3c31436491f0b/html5/thumbnails/60.jpg)
Log risk ratio
8 /100( ) 0.40512 /100
Log RiskRatio Log ⎛ ⎞= = −⎜ ⎟⎝ ⎠
( )92 / 8 88 /12 0.43492 8 88 12Log RiskRatioSE = + =
+ +
![Page 61: Meta-Analysis Concepts and Applicationscms.education.gov.il/.../37798/MetaAnalysisConceptsandApplications.pdf · When effect is consistent • Provide precise estimate of the effect](https://reader031.fdocuments.us/reader031/viewer/2022031318/5c4e57c893f3c31436491f0b/html5/thumbnails/61.jpg)
Log risk ratio
![Page 62: Meta-Analysis Concepts and Applicationscms.education.gov.il/.../37798/MetaAnalysisConceptsandApplications.pdf · When effect is consistent • Provide precise estimate of the effect](https://reader031.fdocuments.us/reader031/viewer/2022031318/5c4e57c893f3c31436491f0b/html5/thumbnails/62.jpg)
Log risk ratio
![Page 63: Meta-Analysis Concepts and Applicationscms.education.gov.il/.../37798/MetaAnalysisConceptsandApplications.pdf · When effect is consistent • Provide precise estimate of the effect](https://reader031.fdocuments.us/reader031/viewer/2022031318/5c4e57c893f3c31436491f0b/html5/thumbnails/63.jpg)
Risk Difference
A CRDA B C D
= −+ +
1 1 2 2* *RD
P Q P QSEA B C D
⎛ ⎞= +⎜ ⎟+ +⎝ ⎠
//
i i i
i i i
p Events Nq NonEvents N
=
=
![Page 64: Meta-Analysis Concepts and Applicationscms.education.gov.il/.../37798/MetaAnalysisConceptsandApplications.pdf · When effect is consistent • Provide precise estimate of the effect](https://reader031.fdocuments.us/reader031/viewer/2022031318/5c4e57c893f3c31436491f0b/html5/thumbnails/64.jpg)
Risk Difference
8 12 0.0408 92 12 88
RD = − = −+ +
.08 * .92 .12 * .88 0.4348 92 12 88RDSE ⎛ ⎞= + =⎜ ⎟+ +⎝ ⎠
![Page 65: Meta-Analysis Concepts and Applicationscms.education.gov.il/.../37798/MetaAnalysisConceptsandApplications.pdf · When effect is consistent • Provide precise estimate of the effect](https://reader031.fdocuments.us/reader031/viewer/2022031318/5c4e57c893f3c31436491f0b/html5/thumbnails/65.jpg)
Risk Difference
![Page 66: Meta-Analysis Concepts and Applicationscms.education.gov.il/.../37798/MetaAnalysisConceptsandApplications.pdf · When effect is consistent • Provide precise estimate of the effect](https://reader031.fdocuments.us/reader031/viewer/2022031318/5c4e57c893f3c31436491f0b/html5/thumbnails/66.jpg)
Risk Difference
![Page 67: Meta-Analysis Concepts and Applicationscms.education.gov.il/.../37798/MetaAnalysisConceptsandApplications.pdf · When effect is consistent • Provide precise estimate of the effect](https://reader031.fdocuments.us/reader031/viewer/2022031318/5c4e57c893f3c31436491f0b/html5/thumbnails/67.jpg)
Multiple indices
![Page 68: Meta-Analysis Concepts and Applicationscms.education.gov.il/.../37798/MetaAnalysisConceptsandApplications.pdf · When effect is consistent • Provide precise estimate of the effect](https://reader031.fdocuments.us/reader031/viewer/2022031318/5c4e57c893f3c31436491f0b/html5/thumbnails/68.jpg)
Fixed effect computations
![Page 69: Meta-Analysis Concepts and Applicationscms.education.gov.il/.../37798/MetaAnalysisConceptsandApplications.pdf · When effect is consistent • Provide precise estimate of the effect](https://reader031.fdocuments.us/reader031/viewer/2022031318/5c4e57c893f3c31436491f0b/html5/thumbnails/69.jpg)
Assigning weights
• To get most precise effect• To give more weight to the more precise
studies
![Page 70: Meta-Analysis Concepts and Applicationscms.education.gov.il/.../37798/MetaAnalysisConceptsandApplications.pdf · When effect is consistent • Provide precise estimate of the effect](https://reader031.fdocuments.us/reader031/viewer/2022031318/5c4e57c893f3c31436491f0b/html5/thumbnails/70.jpg)
Assign weight to each study
• Weight by 1/variance, or the “Inverse variance”
1i
i
wv
=wi=Study weight
vi=Study variance
![Page 71: Meta-Analysis Concepts and Applicationscms.education.gov.il/.../37798/MetaAnalysisConceptsandApplications.pdf · When effect is consistent • Provide precise estimate of the effect](https://reader031.fdocuments.us/reader031/viewer/2022031318/5c4e57c893f3c31436491f0b/html5/thumbnails/71.jpg)
Combined mean
1
1
ˆ
k
i ii
k
ii
w y
wθ =
=
=∑
∑
wi=Study weight
yi=Study mean
![Page 72: Meta-Analysis Concepts and Applicationscms.education.gov.il/.../37798/MetaAnalysisConceptsandApplications.pdf · When effect is consistent • Provide precise estimate of the effect](https://reader031.fdocuments.us/reader031/viewer/2022031318/5c4e57c893f3c31436491f0b/html5/thumbnails/72.jpg)
Variance of combined mean
1
1ˆ( ) k
ii
Varw
θ
=
=
∑wi=Study weight
Var(θ)=Variance of combined mean
![Page 73: Meta-Analysis Concepts and Applicationscms.education.gov.il/.../37798/MetaAnalysisConceptsandApplications.pdf · When effect is consistent • Provide precise estimate of the effect](https://reader031.fdocuments.us/reader031/viewer/2022031318/5c4e57c893f3c31436491f0b/html5/thumbnails/73.jpg)
Test of the null
0ˆ( )
ˆ( )Z
Var
θ θ
θ
−=
![Page 74: Meta-Analysis Concepts and Applicationscms.education.gov.il/.../37798/MetaAnalysisConceptsandApplications.pdf · When effect is consistent • Provide precise estimate of the effect](https://reader031.fdocuments.us/reader031/viewer/2022031318/5c4e57c893f3c31436491f0b/html5/thumbnails/74.jpg)
Example using Excel
![Page 75: Meta-Analysis Concepts and Applicationscms.education.gov.il/.../37798/MetaAnalysisConceptsandApplications.pdf · When effect is consistent • Provide precise estimate of the effect](https://reader031.fdocuments.us/reader031/viewer/2022031318/5c4e57c893f3c31436491f0b/html5/thumbnails/75.jpg)
Enter the summary data
![Page 76: Meta-Analysis Concepts and Applicationscms.education.gov.il/.../37798/MetaAnalysisConceptsandApplications.pdf · When effect is consistent • Provide precise estimate of the effect](https://reader031.fdocuments.us/reader031/viewer/2022031318/5c4e57c893f3c31436491f0b/html5/thumbnails/76.jpg)
Compute effect size and variancefor each study
![Page 77: Meta-Analysis Concepts and Applicationscms.education.gov.il/.../37798/MetaAnalysisConceptsandApplications.pdf · When effect is consistent • Provide precise estimate of the effect](https://reader031.fdocuments.us/reader031/viewer/2022031318/5c4e57c893f3c31436491f0b/html5/thumbnails/77.jpg)
Assign weight to each study
![Page 78: Meta-Analysis Concepts and Applicationscms.education.gov.il/.../37798/MetaAnalysisConceptsandApplications.pdf · When effect is consistent • Provide precise estimate of the effect](https://reader031.fdocuments.us/reader031/viewer/2022031318/5c4e57c893f3c31436491f0b/html5/thumbnails/78.jpg)
Compute combined effect
![Page 79: Meta-Analysis Concepts and Applicationscms.education.gov.il/.../37798/MetaAnalysisConceptsandApplications.pdf · When effect is consistent • Provide precise estimate of the effect](https://reader031.fdocuments.us/reader031/viewer/2022031318/5c4e57c893f3c31436491f0b/html5/thumbnails/79.jpg)
Variance of combined effect
![Page 80: Meta-Analysis Concepts and Applicationscms.education.gov.il/.../37798/MetaAnalysisConceptsandApplications.pdf · When effect is consistent • Provide precise estimate of the effect](https://reader031.fdocuments.us/reader031/viewer/2022031318/5c4e57c893f3c31436491f0b/html5/thumbnails/80.jpg)
Same example in CMA
![Page 81: Meta-Analysis Concepts and Applicationscms.education.gov.il/.../37798/MetaAnalysisConceptsandApplications.pdf · When effect is consistent • Provide precise estimate of the effect](https://reader031.fdocuments.us/reader031/viewer/2022031318/5c4e57c893f3c31436491f0b/html5/thumbnails/81.jpg)
Enter summary data
![Page 82: Meta-Analysis Concepts and Applicationscms.education.gov.il/.../37798/MetaAnalysisConceptsandApplications.pdf · When effect is consistent • Provide precise estimate of the effect](https://reader031.fdocuments.us/reader031/viewer/2022031318/5c4e57c893f3c31436491f0b/html5/thumbnails/82.jpg)
Compute effect size
![Page 83: Meta-Analysis Concepts and Applicationscms.education.gov.il/.../37798/MetaAnalysisConceptsandApplications.pdf · When effect is consistent • Provide precise estimate of the effect](https://reader031.fdocuments.us/reader031/viewer/2022031318/5c4e57c893f3c31436491f0b/html5/thumbnails/83.jpg)
Display formula
![Page 84: Meta-Analysis Concepts and Applicationscms.education.gov.il/.../37798/MetaAnalysisConceptsandApplications.pdf · When effect is consistent • Provide precise estimate of the effect](https://reader031.fdocuments.us/reader031/viewer/2022031318/5c4e57c893f3c31436491f0b/html5/thumbnails/84.jpg)
Combined effect and variance
![Page 85: Meta-Analysis Concepts and Applicationscms.education.gov.il/.../37798/MetaAnalysisConceptsandApplications.pdf · When effect is consistent • Provide precise estimate of the effect](https://reader031.fdocuments.us/reader031/viewer/2022031318/5c4e57c893f3c31436491f0b/html5/thumbnails/85.jpg)
Weights
![Page 86: Meta-Analysis Concepts and Applicationscms.education.gov.il/.../37798/MetaAnalysisConceptsandApplications.pdf · When effect is consistent • Provide precise estimate of the effect](https://reader031.fdocuments.us/reader031/viewer/2022031318/5c4e57c893f3c31436491f0b/html5/thumbnails/86.jpg)
Combined effect and variance
![Page 87: Meta-Analysis Concepts and Applicationscms.education.gov.il/.../37798/MetaAnalysisConceptsandApplications.pdf · When effect is consistent • Provide precise estimate of the effect](https://reader031.fdocuments.us/reader031/viewer/2022031318/5c4e57c893f3c31436491f0b/html5/thumbnails/87.jpg)
Combined effect and variance
![Page 88: Meta-Analysis Concepts and Applicationscms.education.gov.il/.../37798/MetaAnalysisConceptsandApplications.pdf · When effect is consistent • Provide precise estimate of the effect](https://reader031.fdocuments.us/reader031/viewer/2022031318/5c4e57c893f3c31436491f0b/html5/thumbnails/88.jpg)
More information leads to greater precision
![Page 89: Meta-Analysis Concepts and Applicationscms.education.gov.il/.../37798/MetaAnalysisConceptsandApplications.pdf · When effect is consistent • Provide precise estimate of the effect](https://reader031.fdocuments.us/reader031/viewer/2022031318/5c4e57c893f3c31436491f0b/html5/thumbnails/89.jpg)
Increase the N within studies
![Page 90: Meta-Analysis Concepts and Applicationscms.education.gov.il/.../37798/MetaAnalysisConceptsandApplications.pdf · When effect is consistent • Provide precise estimate of the effect](https://reader031.fdocuments.us/reader031/viewer/2022031318/5c4e57c893f3c31436491f0b/html5/thumbnails/90.jpg)
N=50 per group
![Page 91: Meta-Analysis Concepts and Applicationscms.education.gov.il/.../37798/MetaAnalysisConceptsandApplications.pdf · When effect is consistent • Provide precise estimate of the effect](https://reader031.fdocuments.us/reader031/viewer/2022031318/5c4e57c893f3c31436491f0b/html5/thumbnails/91.jpg)
N=100 per group
![Page 92: Meta-Analysis Concepts and Applicationscms.education.gov.il/.../37798/MetaAnalysisConceptsandApplications.pdf · When effect is consistent • Provide precise estimate of the effect](https://reader031.fdocuments.us/reader031/viewer/2022031318/5c4e57c893f3c31436491f0b/html5/thumbnails/92.jpg)
Increase the number of studies
![Page 93: Meta-Analysis Concepts and Applicationscms.education.gov.il/.../37798/MetaAnalysisConceptsandApplications.pdf · When effect is consistent • Provide precise estimate of the effect](https://reader031.fdocuments.us/reader031/viewer/2022031318/5c4e57c893f3c31436491f0b/html5/thumbnails/93.jpg)
Number studies = 3
![Page 94: Meta-Analysis Concepts and Applicationscms.education.gov.il/.../37798/MetaAnalysisConceptsandApplications.pdf · When effect is consistent • Provide precise estimate of the effect](https://reader031.fdocuments.us/reader031/viewer/2022031318/5c4e57c893f3c31436491f0b/html5/thumbnails/94.jpg)
Number studies = 6
![Page 95: Meta-Analysis Concepts and Applicationscms.education.gov.il/.../37798/MetaAnalysisConceptsandApplications.pdf · When effect is consistent • Provide precise estimate of the effect](https://reader031.fdocuments.us/reader031/viewer/2022031318/5c4e57c893f3c31436491f0b/html5/thumbnails/95.jpg)
More precise studies are given more weight
![Page 96: Meta-Analysis Concepts and Applicationscms.education.gov.il/.../37798/MetaAnalysisConceptsandApplications.pdf · When effect is consistent • Provide precise estimate of the effect](https://reader031.fdocuments.us/reader031/viewer/2022031318/5c4e57c893f3c31436491f0b/html5/thumbnails/96.jpg)
Same N for each study
![Page 97: Meta-Analysis Concepts and Applicationscms.education.gov.il/.../37798/MetaAnalysisConceptsandApplications.pdf · When effect is consistent • Provide precise estimate of the effect](https://reader031.fdocuments.us/reader031/viewer/2022031318/5c4e57c893f3c31436491f0b/html5/thumbnails/97.jpg)
Same N for each study
![Page 98: Meta-Analysis Concepts and Applicationscms.education.gov.il/.../37798/MetaAnalysisConceptsandApplications.pdf · When effect is consistent • Provide precise estimate of the effect](https://reader031.fdocuments.us/reader031/viewer/2022031318/5c4e57c893f3c31436491f0b/html5/thumbnails/98.jpg)
Same N for each study
![Page 99: Meta-Analysis Concepts and Applicationscms.education.gov.il/.../37798/MetaAnalysisConceptsandApplications.pdf · When effect is consistent • Provide precise estimate of the effect](https://reader031.fdocuments.us/reader031/viewer/2022031318/5c4e57c893f3c31436491f0b/html5/thumbnails/99.jpg)
N varies by study
![Page 100: Meta-Analysis Concepts and Applicationscms.education.gov.il/.../37798/MetaAnalysisConceptsandApplications.pdf · When effect is consistent • Provide precise estimate of the effect](https://reader031.fdocuments.us/reader031/viewer/2022031318/5c4e57c893f3c31436491f0b/html5/thumbnails/100.jpg)
N varies by study
![Page 101: Meta-Analysis Concepts and Applicationscms.education.gov.il/.../37798/MetaAnalysisConceptsandApplications.pdf · When effect is consistent • Provide precise estimate of the effect](https://reader031.fdocuments.us/reader031/viewer/2022031318/5c4e57c893f3c31436491f0b/html5/thumbnails/101.jpg)
N varies by study
![Page 102: Meta-Analysis Concepts and Applicationscms.education.gov.il/.../37798/MetaAnalysisConceptsandApplications.pdf · When effect is consistent • Provide precise estimate of the effect](https://reader031.fdocuments.us/reader031/viewer/2022031318/5c4e57c893f3c31436491f0b/html5/thumbnails/102.jpg)
Effect size pulled by larger study
d moved from .48 to .66
![Page 103: Meta-Analysis Concepts and Applicationscms.education.gov.il/.../37798/MetaAnalysisConceptsandApplications.pdf · When effect is consistent • Provide precise estimate of the effect](https://reader031.fdocuments.us/reader031/viewer/2022031318/5c4e57c893f3c31436491f0b/html5/thumbnails/103.jpg)
Relative weights in forest plot
![Page 104: Meta-Analysis Concepts and Applicationscms.education.gov.il/.../37798/MetaAnalysisConceptsandApplications.pdf · When effect is consistent • Provide precise estimate of the effect](https://reader031.fdocuments.us/reader031/viewer/2022031318/5c4e57c893f3c31436491f0b/html5/thumbnails/104.jpg)
Study name Std diff in means and 95% CIStd diff Standard
in means error
A 0.400 0.202B 0.250 0.201C 0.800 0.208
0.476 0.117
-2.00 -1.00 0.00 1.00 2.00
Favours A Favours B
Example 01
Meta Analysis
![Page 105: Meta-Analysis Concepts and Applicationscms.education.gov.il/.../37798/MetaAnalysisConceptsandApplications.pdf · When effect is consistent • Provide precise estimate of the effect](https://reader031.fdocuments.us/reader031/viewer/2022031318/5c4e57c893f3c31436491f0b/html5/thumbnails/105.jpg)
Study name Std diff in means and 95% CIStd diff Standard
in means error
A 0.400 0.202B 0.250 0.201C 0.800 0.093
0.658 0.078
-2.00 -1.00 0.00 1.00 2.00
Favours A Favours B
Example 02
Meta Analysis
![Page 106: Meta-Analysis Concepts and Applicationscms.education.gov.il/.../37798/MetaAnalysisConceptsandApplications.pdf · When effect is consistent • Provide precise estimate of the effect](https://reader031.fdocuments.us/reader031/viewer/2022031318/5c4e57c893f3c31436491f0b/html5/thumbnails/106.jpg)
References
• Hedges and Olkin• Lipsey and Wilson
![Page 107: Meta-Analysis Concepts and Applicationscms.education.gov.il/.../37798/MetaAnalysisConceptsandApplications.pdf · When effect is consistent • Provide precise estimate of the effect](https://reader031.fdocuments.us/reader031/viewer/2022031318/5c4e57c893f3c31436491f0b/html5/thumbnails/107.jpg)
Files available by e-mail
• Standardized difference. xls• Standardized difference. cma
![Page 108: Meta-Analysis Concepts and Applicationscms.education.gov.il/.../37798/MetaAnalysisConceptsandApplications.pdf · When effect is consistent • Provide precise estimate of the effect](https://reader031.fdocuments.us/reader031/viewer/2022031318/5c4e57c893f3c31436491f0b/html5/thumbnails/108.jpg)
![Page 109: Meta-Analysis Concepts and Applicationscms.education.gov.il/.../37798/MetaAnalysisConceptsandApplications.pdf · When effect is consistent • Provide precise estimate of the effect](https://reader031.fdocuments.us/reader031/viewer/2022031318/5c4e57c893f3c31436491f0b/html5/thumbnails/109.jpg)
![Page 110: Meta-Analysis Concepts and Applicationscms.education.gov.il/.../37798/MetaAnalysisConceptsandApplications.pdf · When effect is consistent • Provide precise estimate of the effect](https://reader031.fdocuments.us/reader031/viewer/2022031318/5c4e57c893f3c31436491f0b/html5/thumbnails/110.jpg)
![Page 111: Meta-Analysis Concepts and Applicationscms.education.gov.il/.../37798/MetaAnalysisConceptsandApplications.pdf · When effect is consistent • Provide precise estimate of the effect](https://reader031.fdocuments.us/reader031/viewer/2022031318/5c4e57c893f3c31436491f0b/html5/thumbnails/111.jpg)
Fixed effect vs.Random effects
![Page 112: Meta-Analysis Concepts and Applicationscms.education.gov.il/.../37798/MetaAnalysisConceptsandApplications.pdf · When effect is consistent • Provide precise estimate of the effect](https://reader031.fdocuments.us/reader031/viewer/2022031318/5c4e57c893f3c31436491f0b/html5/thumbnails/112.jpg)
Fixed vs. Random
• Concept• Definition• How weights affect
– Combined value– Confidence interval width
• Which should we use
![Page 113: Meta-Analysis Concepts and Applicationscms.education.gov.il/.../37798/MetaAnalysisConceptsandApplications.pdf · When effect is consistent • Provide precise estimate of the effect](https://reader031.fdocuments.us/reader031/viewer/2022031318/5c4e57c893f3c31436491f0b/html5/thumbnails/113.jpg)
Concept
• Fixed effect model– Common population– Effect size varies only because of random
error• Random effects model
– Multiple populations– Effect size will vary because of random error– Effect size will vary because of true variation
![Page 114: Meta-Analysis Concepts and Applicationscms.education.gov.il/.../37798/MetaAnalysisConceptsandApplications.pdf · When effect is consistent • Provide precise estimate of the effect](https://reader031.fdocuments.us/reader031/viewer/2022031318/5c4e57c893f3c31436491f0b/html5/thumbnails/114.jpg)
Definition of combined effect
• Fixed effect model– There is one true effect.– Combined effect is estimate of this value.
• Random effects model– There are a series of effects.– Combined effect is average of a series of
values.
![Page 115: Meta-Analysis Concepts and Applicationscms.education.gov.il/.../37798/MetaAnalysisConceptsandApplications.pdf · When effect is consistent • Provide precise estimate of the effect](https://reader031.fdocuments.us/reader031/viewer/2022031318/5c4e57c893f3c31436491f0b/html5/thumbnails/115.jpg)
Fixed vs. Random
i i iT µ ξ ε= + +
i iT µ ε= +
![Page 116: Meta-Analysis Concepts and Applicationscms.education.gov.il/.../37798/MetaAnalysisConceptsandApplications.pdf · When effect is consistent • Provide precise estimate of the effect](https://reader031.fdocuments.us/reader031/viewer/2022031318/5c4e57c893f3c31436491f0b/html5/thumbnails/116.jpg)
Factors affecting Tau-squared
![Page 117: Meta-Analysis Concepts and Applicationscms.education.gov.il/.../37798/MetaAnalysisConceptsandApplications.pdf · When effect is consistent • Provide precise estimate of the effect](https://reader031.fdocuments.us/reader031/viewer/2022031318/5c4e57c893f3c31436491f0b/html5/thumbnails/117.jpg)
When Tau2 is zero
• Random effects model reduces to the fixed effect model.
![Page 118: Meta-Analysis Concepts and Applicationscms.education.gov.il/.../37798/MetaAnalysisConceptsandApplications.pdf · When effect is consistent • Provide precise estimate of the effect](https://reader031.fdocuments.us/reader031/viewer/2022031318/5c4e57c893f3c31436491f0b/html5/thumbnails/118.jpg)
Weights
• Fixed effect– One true effect– All variation is random error– Largely ignore the smaller studies
• Random effects– Range of effects– Each study provides information about a different
population– Cannot ignore small studies, nor give too much
weight to large studies
![Page 119: Meta-Analysis Concepts and Applicationscms.education.gov.il/.../37798/MetaAnalysisConceptsandApplications.pdf · When effect is consistent • Provide precise estimate of the effect](https://reader031.fdocuments.us/reader031/viewer/2022031318/5c4e57c893f3c31436491f0b/html5/thumbnails/119.jpg)
Fixed effect model
Within-Study Error
Total Variance
+ =
![Page 120: Meta-Analysis Concepts and Applicationscms.education.gov.il/.../37798/MetaAnalysisConceptsandApplications.pdf · When effect is consistent • Provide precise estimate of the effect](https://reader031.fdocuments.us/reader031/viewer/2022031318/5c4e57c893f3c31436491f0b/html5/thumbnails/120.jpg)
Random effects model
Within-Study Error
Total Variance
+ =Between-study
variance
![Page 121: Meta-Analysis Concepts and Applicationscms.education.gov.il/.../37798/MetaAnalysisConceptsandApplications.pdf · When effect is consistent • Provide precise estimate of the effect](https://reader031.fdocuments.us/reader031/viewer/2022031318/5c4e57c893f3c31436491f0b/html5/thumbnails/121.jpg)
Extreme effect in large study
![Page 122: Meta-Analysis Concepts and Applicationscms.education.gov.il/.../37798/MetaAnalysisConceptsandApplications.pdf · When effect is consistent • Provide precise estimate of the effect](https://reader031.fdocuments.us/reader031/viewer/2022031318/5c4e57c893f3c31436491f0b/html5/thumbnails/122.jpg)
Extreme effect in small study
![Page 123: Meta-Analysis Concepts and Applicationscms.education.gov.il/.../37798/MetaAnalysisConceptsandApplications.pdf · When effect is consistent • Provide precise estimate of the effect](https://reader031.fdocuments.us/reader031/viewer/2022031318/5c4e57c893f3c31436491f0b/html5/thumbnails/123.jpg)
Study name Statistics for each study Odds ratio and 95% CI
Odds Lower Upper ratio limit limit
Morton 0.436 0.038 5.022Rasmussen 0.348 0.154 0.783Smith 0.278 0.057 1.357Abraham 0.957 0.058 15.773Feldstedt 1.250 0.479 3.261Shechter-89 0.090 0.011 0.736Ceremuzynski 0.278 0.027 2.883Berschat 0.304 0.012 7.880Singh 0.499 0.174 1.426Pereira 0.110 0.012 0.967Golf 0.427 0.127 1.436Thogersen 0.452 0.133 1.543LIMIT-2 0.741 0.556 0.988Shechter-95 0.208 0.067 0.640ISIS-4 1.059 0.996 1.127MAGIC 1.003 0.873 1.152
0.712 0.564 0.9000.01 0.1 1 10 100
Favours A Favours B
Magnesium Fixed effect
Meta Analysis
![Page 124: Meta-Analysis Concepts and Applicationscms.education.gov.il/.../37798/MetaAnalysisConceptsandApplications.pdf · When effect is consistent • Provide precise estimate of the effect](https://reader031.fdocuments.us/reader031/viewer/2022031318/5c4e57c893f3c31436491f0b/html5/thumbnails/124.jpg)
Study name Statistics for each study Odds ratio and 95% CI
Odds Lower Upper ratio limit limit
Morton 0.436 0.038 5.022Rasmussen 0.348 0.154 0.783Smith 0.278 0.057 1.357Abraham 0.957 0.058 15.773Feldstedt 1.250 0.479 3.261Shechter-89 0.090 0.011 0.736Ceremuzynski 0.278 0.027 2.883Berschat 0.304 0.012 7.880Singh 0.499 0.174 1.426Pereira 0.110 0.012 0.967Golf 0.427 0.127 1.436Thogersen 0.452 0.133 1.543LIMIT-2 0.741 0.556 0.988Shechter-95 0.208 0.067 0.640ISIS-4 1.059 0.996 1.127MAGIC 1.003 0.873 1.152
1.016 0.961 1.0730.01 0.1 1 10 100
Favours A Favours B
Magnesium Fixed effect
Meta Analysis
![Page 125: Meta-Analysis Concepts and Applicationscms.education.gov.il/.../37798/MetaAnalysisConceptsandApplications.pdf · When effect is consistent • Provide precise estimate of the effect](https://reader031.fdocuments.us/reader031/viewer/2022031318/5c4e57c893f3c31436491f0b/html5/thumbnails/125.jpg)
Key idea
• Relative weights assigned under random effects will be more balanced than those assigned under fixed effects.
• As we move from fixed effect to random effects, extreme studies will lose influence if they are large, and will gain influence if they are small.
![Page 126: Meta-Analysis Concepts and Applicationscms.education.gov.il/.../37798/MetaAnalysisConceptsandApplications.pdf · When effect is consistent • Provide precise estimate of the effect](https://reader031.fdocuments.us/reader031/viewer/2022031318/5c4e57c893f3c31436491f0b/html5/thumbnails/126.jpg)
Confidence interval width
• Both models include within-study variance.• Random effects model includes also
between-study variance.• Therefore, the confidence interval for the
random effects model will always be as wide or wider than for the fixed effect model.
![Page 127: Meta-Analysis Concepts and Applicationscms.education.gov.il/.../37798/MetaAnalysisConceptsandApplications.pdf · When effect is consistent • Provide precise estimate of the effect](https://reader031.fdocuments.us/reader031/viewer/2022031318/5c4e57c893f3c31436491f0b/html5/thumbnails/127.jpg)
Study name Std diff in means and 95% CI
Std diff Standard in means error
A 0.400 0.001B 0.400 0.001C 0.400 0.001D 0.400 0.001E 0.400 0.001
0.400 0.000
-1.00 -0.50 0.00 0.50 1.00
Favours A Favours B
Fixed effect model with huge N
Meta Analysis
![Page 128: Meta-Analysis Concepts and Applicationscms.education.gov.il/.../37798/MetaAnalysisConceptsandApplications.pdf · When effect is consistent • Provide precise estimate of the effect](https://reader031.fdocuments.us/reader031/viewer/2022031318/5c4e57c893f3c31436491f0b/html5/thumbnails/128.jpg)
Study name Std diff in means and 95% CI
Std diff Standard in means error
A 0.400 0.001B 0.450 0.001C 0.350 0.001D 0.450 0.001E 0.350 0.001
0.400 0.022
-1.00 -0.50 0.00 0.50 1.00
Favours A Favours B
Random effects model with huge N
Meta Analysis
![Page 129: Meta-Analysis Concepts and Applicationscms.education.gov.il/.../37798/MetaAnalysisConceptsandApplications.pdf · When effect is consistent • Provide precise estimate of the effect](https://reader031.fdocuments.us/reader031/viewer/2022031318/5c4e57c893f3c31436491f0b/html5/thumbnails/129.jpg)
Which model should we use?
![Page 130: Meta-Analysis Concepts and Applicationscms.education.gov.il/.../37798/MetaAnalysisConceptsandApplications.pdf · When effect is consistent • Provide precise estimate of the effect](https://reader031.fdocuments.us/reader031/viewer/2022031318/5c4e57c893f3c31436491f0b/html5/thumbnails/130.jpg)
Fixed effect
• If there is reason to believe that all the studies are functionally identical
• Our goal is to compute the common effect size, which would then be generalized to other examples of this same population.
• Example, of drug company has run five studies to assess the effect of a drug.
![Page 131: Meta-Analysis Concepts and Applicationscms.education.gov.il/.../37798/MetaAnalysisConceptsandApplications.pdf · When effect is consistent • Provide precise estimate of the effect](https://reader031.fdocuments.us/reader031/viewer/2022031318/5c4e57c893f3c31436491f0b/html5/thumbnails/131.jpg)
Random effects
• When not likely that all the studies were functionally equivalent.
• When the goal of this analysis is to generalize to a range of populations.
![Page 132: Meta-Analysis Concepts and Applicationscms.education.gov.il/.../37798/MetaAnalysisConceptsandApplications.pdf · When effect is consistent • Provide precise estimate of the effect](https://reader031.fdocuments.us/reader031/viewer/2022031318/5c4e57c893f3c31436491f0b/html5/thumbnails/132.jpg)
Choice of model should not be based on significance test
• Practical issue– Type-II error
• Fundamental issue– The difference between fixed and random
effects is really conceptual
![Page 133: Meta-Analysis Concepts and Applicationscms.education.gov.il/.../37798/MetaAnalysisConceptsandApplications.pdf · When effect is consistent • Provide precise estimate of the effect](https://reader031.fdocuments.us/reader031/viewer/2022031318/5c4e57c893f3c31436491f0b/html5/thumbnails/133.jpg)
Common (incorrect) wisdom about significance tests
• “Significance test for the effect size will always be more significant using the fixed effect model” rather than the random effects model.
• Is not true• In any event, should never be a factor in
selecting a computational model.
![Page 134: Meta-Analysis Concepts and Applicationscms.education.gov.il/.../37798/MetaAnalysisConceptsandApplications.pdf · When effect is consistent • Provide precise estimate of the effect](https://reader031.fdocuments.us/reader031/viewer/2022031318/5c4e57c893f3c31436491f0b/html5/thumbnails/134.jpg)
Criticisms of meta-analysis
Meta-Analysis – Concepts and Applications
SCT Orlando May 21, 2006Michael Borenstein and Julian Higgins
![Page 135: Meta-Analysis Concepts and Applicationscms.education.gov.il/.../37798/MetaAnalysisConceptsandApplications.pdf · When effect is consistent • Provide precise estimate of the effect](https://reader031.fdocuments.us/reader031/viewer/2022031318/5c4e57c893f3c31436491f0b/html5/thumbnails/135.jpg)
Meta-AnalysisConcepts and Applications
Michael Borenstein and Julian HigginsWorkshops Chairman Domenic RedaSCT Orlando May 21, 2006Additional materials available at www.Meta-
Analysis.comQuestions to [email protected]