Lecture 17: Prevention of bias in RCTs
description
Transcript of Lecture 17: Prevention of bias in RCTs
![Page 1: Lecture 17: Prevention of bias in RCTs](https://reader036.fdocuments.us/reader036/viewer/2022062304/5681404c550346895dabbe34/html5/thumbnails/1.jpg)
Lecture 17 (Oct 28,2004) 1
Lecture 17: Prevention of bias in RCTs
• Statistical/analytic issues in RCTs– Measures of effect– Precision/hypothesis testing– Compliance/intention to treat– RCTs of effectiveness of screening
• Effects of study design (Schultz paper)• Strengths and weaknesses of RCTs
![Page 2: Lecture 17: Prevention of bias in RCTs](https://reader036.fdocuments.us/reader036/viewer/2022062304/5681404c550346895dabbe34/html5/thumbnails/2.jpg)
Lecture 17 (Oct 28,2004) 2
Analysis of RCTs
• Planning stage:– Pre-specified hypotheses– Primary and secondary outcomes– Measure of effect– Sample size calculation
![Page 3: Lecture 17: Prevention of bias in RCTs](https://reader036.fdocuments.us/reader036/viewer/2022062304/5681404c550346895dabbe34/html5/thumbnails/3.jpg)
Lecture 17 (Oct 28,2004) 3
Analysis of RCTs
• Analysis stage:– Check on success of randomization– Analyze adherence to interventions – Intention to treat - why?– Should the analyses be blinded?
![Page 4: Lecture 17: Prevention of bias in RCTs](https://reader036.fdocuments.us/reader036/viewer/2022062304/5681404c550346895dabbe34/html5/thumbnails/4.jpg)
Lecture 17 (Oct 28,2004) 4
MRFIT study(Multiple Risk Factor Intervention Trial)• Prevention of coronary heart disease (CHD)
– Followed Framingham and other observational studies• Multi-site RCT • High-risk men age 35-57 (Framingham algorithm)
– N = 12,866• Comparison groups:
– Special intervention (SI): • Reduction of serum cholesterol via smoking cessation,
hypertension treatment, dietary modification– Usual care (UC):
• Notification of physician of results of risk status
![Page 5: Lecture 17: Prevention of bias in RCTs](https://reader036.fdocuments.us/reader036/viewer/2022062304/5681404c550346895dabbe34/html5/thumbnails/5.jpg)
Lecture 17 (Oct 28,2004) 5
MRFIT study (cont)
• Primary outcome: Death from CHD– Method of analysis?
• Secondary outcomes: – Death from any cardiovascular disease
– Death from any cause
– Overall CHD incidence (fatal and non-fatal cases)
• Intermediate outcomes:– Risk factor levels
![Page 6: Lecture 17: Prevention of bias in RCTs](https://reader036.fdocuments.us/reader036/viewer/2022062304/5681404c550346895dabbe34/html5/thumbnails/6.jpg)
Lecture 17 (Oct 28,2004) 6
MRFIT study(Multiple Risk Factor Intervention Trial)
• Sample size estimation:– Expected 6-year CHD death rate = 29.0/1,000– Hypothesized rate in SI group = 21.3/1,000 (26.6% reduction)– P (type 1 error) = 0.05 (one-sided test)– Power = 0.88
• Basis for projection:– 10% reduction of serum cholesterol if >220 mg/dL (vs no change in UC)– Reduction in smoking rate:
• 25% for smokers of 40+ cigs/day (vs 5% UC)• 40% for smokers of 20-39 cigs/day (vs 10% UC)• 55% for smokers of <20 cigs/day (vs 15% UC)
• Sub-group hypotheses:– Formulated during trial, blind to interim mortality data– Example: SI would be especially effective in men with normal resting
ECGs
![Page 7: Lecture 17: Prevention of bias in RCTs](https://reader036.fdocuments.us/reader036/viewer/2022062304/5681404c550346895dabbe34/html5/thumbnails/7.jpg)
Lecture 17 (Oct 28,2004) 7
![Page 8: Lecture 17: Prevention of bias in RCTs](https://reader036.fdocuments.us/reader036/viewer/2022062304/5681404c550346895dabbe34/html5/thumbnails/8.jpg)
Lecture 17 (Oct 28,2004) 8
![Page 9: Lecture 17: Prevention of bias in RCTs](https://reader036.fdocuments.us/reader036/viewer/2022062304/5681404c550346895dabbe34/html5/thumbnails/9.jpg)
Lecture 17 (Oct 28,2004) 9
MRFIT: explanations?
• Intervention not effective• Intervention is effective, but less than expected
because:– Lower than expected mortality in UC group– Risk reduction in UC group
• Positive effect in some sub-groups offset by negative effect in others– In subgroup with hypertension and ECG abnormalities,
higher death rate in SI vs UC– Possibly unfavorable response to antihypertensive drug
therapy?
![Page 10: Lecture 17: Prevention of bias in RCTs](https://reader036.fdocuments.us/reader036/viewer/2022062304/5681404c550346895dabbe34/html5/thumbnails/10.jpg)
Lecture 17 (Oct 28,2004) 10
MRFIT - lessons
• Consider “contamination” and “compensatory” effects in study design
• Clear specification a priori of planned sub-group analyses (with sample size calculations)
• (Reference: JAMA 1982, 248: 1465-1477)
![Page 11: Lecture 17: Prevention of bias in RCTs](https://reader036.fdocuments.us/reader036/viewer/2022062304/5681404c550346895dabbe34/html5/thumbnails/11.jpg)
Lecture 17 (Oct 28,2004) 11
Measures of effect
• Types of data to be analyzed:– incidence rate of an adverse event (death, etc)
• It = incidence rate in treatment group
• Ic = incidence rate in control group
• Example (mammography and mortality):• It = 2/10,000/year
• Ic = 4/10,000/year
![Page 12: Lecture 17: Prevention of bias in RCTs](https://reader036.fdocuments.us/reader036/viewer/2022062304/5681404c550346895dabbe34/html5/thumbnails/12.jpg)
Lecture 17 (Oct 28,2004) 12
Risk difference and ratio
Risk difference = Ic - It/units
– usually easier to express as risk reduction
– 4 - 2/10,000/year = 1/10,000/year
Risk ratio (relative risk) = Ic = 4/2 = 2.0
It
Alternatively: = It = 2/4 = 0.50
Ic
![Page 13: Lecture 17: Prevention of bias in RCTs](https://reader036.fdocuments.us/reader036/viewer/2022062304/5681404c550346895dabbe34/html5/thumbnails/13.jpg)
Lecture 17 (Oct 28,2004) 13
Relative risk reduction
• Analogous to attributable risk percent
• Sometimes called percent effectiveness
= risk difference = Ic - It
risk in control group Ic
= 2/4 = 50%
• Can be computed from the risk ratio: 1 - 1
RR
= 1 -1/2
![Page 14: Lecture 17: Prevention of bias in RCTs](https://reader036.fdocuments.us/reader036/viewer/2022062304/5681404c550346895dabbe34/html5/thumbnails/14.jpg)
Lecture 17 (Oct 28,2004) 14
Example from GUSTO trial
• tissue plasminogen activator (TPA) vs streptokinase (SK) as thrombolytic strategy in treatment of AMI.
30-day mortality in TPA group = 6.3%
• 30-day mortality in SK group = 7.3%
![Page 15: Lecture 17: Prevention of bias in RCTs](https://reader036.fdocuments.us/reader036/viewer/2022062304/5681404c550346895dabbe34/html5/thumbnails/15.jpg)
Lecture 17 (Oct 28,2004) 15
Measures of effect
RATE/RISK RATIO
SK rate = 7.3 = 1.16
TPA rate 6.3
RELATIVE RISK REDUCTION
SK rate – TPA rate = 7.3 – 6.3 = 14%
SK rate 7.3
[also calculated as 1 – (1/rate ratio)]
![Page 16: Lecture 17: Prevention of bias in RCTs](https://reader036.fdocuments.us/reader036/viewer/2022062304/5681404c550346895dabbe34/html5/thumbnails/16.jpg)
Lecture 17 (Oct 28,2004) 16
Measures of effect (cont)
ABSOLUTE RISK REDUCTION (rate/risk difference; attributable risk)
SK rate – TPA rate = 7.3% – 6.3% = 1.0%
NUMBER NEEDED TO TREAT (NNT)(Reciprocal of risk difference)
1 = 1 = 100
SK rate – TPA rate .01
![Page 17: Lecture 17: Prevention of bias in RCTs](https://reader036.fdocuments.us/reader036/viewer/2022062304/5681404c550346895dabbe34/html5/thumbnails/17.jpg)
Lecture 17 (Oct 28,2004) 17
SELECTION OF EFFECT MEASURES
Ratio measures assess strength of effect - how effective is the treatment?
Difference measures take into account frequency of the outcome – can assess whether it is worthwhile (allocation of time and $$)
Both ratio and difference measures are needed
All these measures are estimates and are subject to sampling error – need confidence intervals to determine their precision
All the measures are limited by the study(ies) that generated them – they may vary by patient characteristics, adherence to treatment, duration of follow-up, etc)
Measures consider only beneficial and not adverse effects of treatment.
![Page 18: Lecture 17: Prevention of bias in RCTs](https://reader036.fdocuments.us/reader036/viewer/2022062304/5681404c550346895dabbe34/html5/thumbnails/18.jpg)
Lecture 17 (Oct 28,2004) 18
Aspirin in prevention of MI among male smokers
(data from Physicians’ Health Study)
5-year incidence of MI:
aspirin group = 1.2%
placebo group = 2.2%
Risk ratio = 1.8
Relative risk reduction = 45%
Absolute risk reduction = 1.0% in 5 years
NNT = 100 for 5 years (to prevent 1 MI)
![Page 19: Lecture 17: Prevention of bias in RCTs](https://reader036.fdocuments.us/reader036/viewer/2022062304/5681404c550346895dabbe34/html5/thumbnails/19.jpg)
Lecture 17 (Oct 28,2004) 19
Antihypertensive treatment in 75-year old women with BP of 170/80
(data from SHEP study)
• 5-year incidence of stroke:
treatment group = 5.2%
placebo group = 8.2%– Risk ratio = 1.6– Relative risk reduction = 37%– Absolute risk reduction = 3.0% in 5 years– NNT = 33 / 5 years (to prevent 1 stroke)
![Page 20: Lecture 17: Prevention of bias in RCTs](https://reader036.fdocuments.us/reader036/viewer/2022062304/5681404c550346895dabbe34/html5/thumbnails/20.jpg)
Lecture 17 (Oct 28,2004) 20
Measures of effect in RCTs: continuous outcomes
• Example: RCT of antidepressant vs placebo:
• Measures on depression scale at baseline and at follow-up
• Possible measures:– Difference in mean scores at follow-up – Difference in change scores from baseline to
follow-up
![Page 21: Lecture 17: Prevention of bias in RCTs](https://reader036.fdocuments.us/reader036/viewer/2022062304/5681404c550346895dabbe34/html5/thumbnails/21.jpg)
Lecture 17 (Oct 28,2004) 21
Adherence to interventions
• Possible outcomes:– Low adherence in one or both study groups
• E.g. St John’s wort vs sertaline
– Cross-over • E.g., RCTs of medical vs surgical treatment of CHD
• How should results be analyzed?– By intervention to which randomized (“intention-to-
treat”)
– By intervention actually received?
![Page 22: Lecture 17: Prevention of bias in RCTs](https://reader036.fdocuments.us/reader036/viewer/2022062304/5681404c550346895dabbe34/html5/thumbnails/22.jpg)
Lecture 17 (Oct 28,2004) 22
RCTs of screeningExample: evaluation of the effectiveness of
breast cancer screening (HIP study)
• 1st RCT of breast cancer screening – Study population: Members of HMO
– Intervention: Invitation to receive annual mammography and clinical exam (3 years)
• Possible outcomes:– survival rate (1 year, 5 year)
– case-fatality rate
– mortality rate
• Which would you use?
![Page 23: Lecture 17: Prevention of bias in RCTs](https://reader036.fdocuments.us/reader036/viewer/2022062304/5681404c550346895dabbe34/html5/thumbnails/23.jpg)
Lecture 17 (Oct 28,2004) 23
![Page 24: Lecture 17: Prevention of bias in RCTs](https://reader036.fdocuments.us/reader036/viewer/2022062304/5681404c550346895dabbe34/html5/thumbnails/24.jpg)
Lecture 17 (Oct 28,2004) 24
Bias in RCTs of screening
• Definition of time zero?– Date of first symptoms?
– Date of detection?
– Date of diagnosis?
• Bias if difference in “time zero”between study groups:
– screening/early detection intervention shifts time zero
– intervention appears to lengthen time to outcome without real change in prognosis
– “lead time” bias
– “length” bias
![Page 25: Lecture 17: Prevention of bias in RCTs](https://reader036.fdocuments.us/reader036/viewer/2022062304/5681404c550346895dabbe34/html5/thumbnails/25.jpg)
Lecture 17 (Oct 28,2004) 25
Other types of bias in RCTs
• Hawthorne effect:– Non-specific effect of being in a study– Prevention?
• Contamination bias:– Control group receives some component(s) of
intervention– Prevention?
• Confounding variables – Variables associated with intervention group and
outcome, not in causal chain– Prevention?
![Page 26: Lecture 17: Prevention of bias in RCTs](https://reader036.fdocuments.us/reader036/viewer/2022062304/5681404c550346895dabbe34/html5/thumbnails/26.jpg)
Lecture 17 (Oct 28,2004) 26
Internal vs external validity
• Internal validity – Lack of bias in study
• External validity– Generalizability– Representativeness of study sample