Sensitivity Analysis and Meta-analysisEPI 811 Individual PresentationChapter 10 of Szklo and Nieto’s Epidemiology: Beyond the BasicsAnton Frattaroli

Sensitivity Analysis• Generally, an assessment of how systematic or random errors

affect an effect estimates’ representativeness of the actual effect (the validity of the effect estimate).

• Misclassification error is a primary inhibitor of validity and can be difficult to correct for.

• Executed by adjusting model parameters over a reasonable range, and observing the results.

Sensitivity Analysis Applied to Misclassification: Example• You: “The relative risk (RR) of coronary heart disease (CHD) for

second-hand (passive) smoke exposure in non-smokers is between 1.15 and 1.3.”

• They: “Maybe some of your ‘non-smokers’ are actually smokers, and your RR is too high.”

• You: “I can do a sensitivity analysis. Assume 5% of my exposed ‘non-smoker’ cases are just misclassified smokers. In that case, the effect of CHD on active smokers would have to exhibit a RR of 7.0 in order to entirely account for the difference in RR. But, since the RR of CHD in smokers is 2.0, you’re wrong.”

• They: “Try 10%.”

STATA: episensi

Sensitivity Analysis Applied to Vaccine Effectiveness: Example

Meta-analysis• A “quantitative approach for systematically assessing the

results of previous research in order to arrive at conclusions about the body of research (Petitti)”.

• Unit of analysis is the study, rather than a group or individual. Study selection is similar to the selection of subjects in a study.

• In a meta-analysis study of the relationship of major depression to socioeconomic class, only 51 studies were chosen out of a 743 found.

Meta-analysis in Action!

STATA: metan

You can do it too!

Meta-analysis Styles• Some prefer the Mantel-Haenszel method of weighting study estimate

results by the power of the study (the “fixed-effects model”).• Others like a “random-effects model”, which takes into account both

within-study variance and cross-study variance.• The random-effects model is more conservative. The real difference is

in the generalizability: fixed-effects is limited to the included studies, while random-effects can be applied to a hypothetical “population of studies.”

• Neither model is advisable when the direction of the studies is not consistent.

• Still some argument about the effectiveness of meta-analysis, given the differences in participant selection and data collection methods across studies.

• EPI 810 tip: Prospective meta-analysis can be used as an agreement across research groups to avoid some of these pitfalls.

Questions?• Where do I get that totally sweet Stata module?• episens: http://ideas.repec.org/c/boc/bocode/s456792.html• metan: http://ideas.repec.org/c/boc/bocode/s456798.html