1 Chapter 11 Issues in Analysis of Randomized Clinical Trials.
-
Upload
ally-berryman -
Category
Documents
-
view
213 -
download
0
Transcript of 1 Chapter 11 Issues in Analysis of Randomized Clinical Trials.
1
Chapter 11Chapter 11
Issues in Analysis of Issues in Analysis of Randomized Clinical TrialsRandomized Clinical Trials
2
Issues in Analysis of Issues in Analysis of Randomized Clinical TrialsRandomized Clinical Trials
• Reference:
May, DeMets et al (1981)
Circulation 64:669-673
Peto et al (1976)
British Journal of Cancer
3
Sources of BiasSources of Bias1. Patient selection
2. Treatment assignment
3. Patient Evaluation
4. Data Analysis
Methods to Minimize Bias1. Randomized Controls
2. Double blind (masked)
3. Analyze what is randomized
4
What Data Should Be Analyzed?What Data Should Be Analyzed?• Basic Intention-to-Treat Principle
– Analyze what is randomized!– All subjects randomized, all events during
follow-up• Randomized control trial is the “gold”
standard”
• Definitions
Exclusions– Screened but not randomized– Affects generalizability but validity OK
Withdrawals from Analysis– Randomized, but not included in data analysis– Possible to introduce bias!
5
Patient CloseoutPatient Closeout
• ICH E9 Glossary– “Intention-to-treat principle - …It has the
consequence that subjects allocated to a treatment group should be followed up, assessed, and analyzed as members of that group irrespective of their compliance with the planned course of treatment.”
6
Intention To Treat (ITT) Intention To Treat (ITT) PrinciplePrinciple
• Analyze all subjects randomized & all events
• Beware of “look alikes”– Modified ITT: Analyze subjects who get
some intervention– Per Protocol: Analyze subjects who comply
according to the protocol
7
Patient Withdrawn in Analysis (1)Patient Withdrawn in Analysis (1)
• Common Practice - 1980s– Over 3 years, 37/109 trials in New England Journal of Medicine
published papers with some patient data not included
• Typical Reasons Given
a. Patient ineligible (in retrospect)
b. Noncompliance
c. Competing events
d. Missing data
8
Patient Withdrawn in Analysis (2)Patient Withdrawn in Analysis (2)
A. Patient INELIGIBLE
– After randomization, discover some patients did not in fact meet entry criteria
– Concern ineligible patients may dilute treatment effect
– Temptation to withdraw ineligibles
– Withdrawl of ineligible patients, post hoc, may introduce bias
9
Betablocker Heart Attack TrialBetablocker Heart Attack Trial(JAMA, 1982)(JAMA, 1982)
• 3837 post MI patients randomized• 341 patients found by Central Review to be ineligible• Results
% Mortality
Propranolol Placebo
Eligible 7.3 9.6
Ineligible 6.7 11.3 Best
Total 7.2 9.8
In the ineligible patients, treatment works best
10
Acceptable PoliciesAcceptable PoliciesFor Ineligible SubjectsFor Ineligible Subjects
1. Delay randomization, confirm eligibility and allow no withdrawals (e.g. AMIS) (Chronic Studies)
2. Accept ineligibles, allow no withdrawals
(e.g. BHAT, MILIS) (Acute Studies)
3. Allow withdrawals if:
a. Procedures defined in advance
b. Decision made early (before event)
c. Decision independent and blinded
d. Use baseline covariates only (two subgroups)
e. Analysis done with and without
11
B. WITHDRAWL FOR NON-COMPLIANCE
References: Sackett & Gent (1979) NEJM, p. 1410Coronary Drug Project (1980) NEJM, p. 1038
• Two Types of Trials
1. Management
- "Intent to Treat" Principle
- Compare all subjects, regardless of compliance
2. Explanatory
- Estimate optimum effect, understand mechanism
- Analyze subjects who fully comply
WITHDRAWALS FOR NON-COMPLIANCE MAY LEAD TO BIAS!
12
Cancer Trial (5-FU & Radiation)Cancer Trial (5-FU & Radiation)Gastric CarcinomaGastric Carcinoma
• Reference: Moertel et al. (Journal of Clinical Oncology, 1984)
• 62 patients randomized– No surgical adjuvant therapy
vs.
– 5-FU and radiation
• 5 year survival results
Randomized Percent (%)
Treatment 23% P < 0.05
No Treatment 4%
13
Cancer Trial (5-FU & Radiation)Cancer Trial (5-FU & Radiation)Gastric CarcinomaGastric Carcinoma
• According to treatment received 5 year survival
Received % Survival
Treatment 20%
Refused Treatment 30% NS
Control 4%
14
Example: Coronary Drug ProjectExample: Coronary Drug Project5-Year Mortality5-Year Mortality
Clofibrate Placebo
N % Deaths N % Deaths
Total (as reported) 1103 20.0 2782 20.9
By Compliance 1065 18.2 2695 19.4
< 80% 357 24.6 882 28.2
> 80% 708 15.0 1813 15.1
• Adjusting for 40 covariates had little impact
• Compliance is an outcome
Compliers do better, regardless of treatment
15
Example: Coronary Drug ProjectExample: Coronary Drug Project2-Year Mortality2-Year Mortality
Compliance Assessed Estrogen Placebo
N % Deaths N % Deaths
Total 903 6.2 2361 5.7
< 80% 488 6.1 436 9.9
> 80% 415 6.3 1925 4.8
Comments• Higher % of estrogens patients did not comply• Beneficial to be randomized to estrogen & not take it • (6.1% vs. 9.9%)• Best to be randomized to placebo & comply (4.8%)
16
Example: Wilcox et al (1980) Trial, Example: Wilcox et al (1980) Trial, BMJBMJ6-Week Mortality6-Week Mortality
Propranolol Atenolol Placebo
N % Deaths N % Deaths N % Deaths
Total 132 7.6 127 8.7 129 11.6
Compliers 88 3.4 76 2.6 89 11.2
Non-compliers 44 15.9 51 17.6 40 12.5
Comments• Compliers did better than placebo• Treatment non-compliers did worse than placebo• Placebo non-compliers only slightly worse than compliers• Analysis by compliers overestimates benefit
17
Aspirin Myocardial Infarction Aspirin Myocardial Infarction Study (AMIS)Study (AMIS)
% Mortality
Compliance Aspirin Placebo
Good 6.1 5.1
Poor 21.9 22.0
Total 10.9 9.7
18
Summary of ComplianceSummary of Compliance
• No consistent pattern
Example Non-compliance Did Worse
AMIS Both Treatment & Control
CDP Estrogen Control OnlyBeta-blocker, Wilcox Two Treatments, Not
Control• Compliance an outcome, not always independent
of treatment• Withdrawal of non-compliers can lead to bias• Non-compliers dilute treatment• Try hard not to randomize non-compliers
19
II. Competing EventsII. Competing Events
• Subject may be censored from primary event by some other event (e.g. cancer vs. heart disease)
• Must assume independence
• If cause specific mortality used, should also look at total death
• If non-fatal event is primary, should also look at total death and non-fatal event
• Problem for some response measures
20
III. Problem of DefinitionsIII. Problem of Definitions
Classification Anturane Placebo P-value
ART 30/812 48/817 0.03
Another Committee 28/812 39/817 0.17
• Cause specific definitions hard to apply
• Example: Anturane Reinfarction Trail (ART)(NEJM, 1980)
Sudden Death
21
Anturane Reinfarction TrialAnturane Reinfarction TrialSudden DeathSudden Death
Category Source Placebo Anturane P-value
All patients & all NEJM 48/817 30/812 0.03
sudden deaths AC 39/817 28/812 0.17
"Eligible" patients & NEJM 46/785 28/775 0.03
all sudden deaths AC 37/782 25/773 0.12
• Problem of cause specific definitions• AC = Another review committee
22
IV. "Wrong", Inconsistent, IV. "Wrong", Inconsistent, Outlying DataOutlying Data
• "Wrong" or "outlying" data may in fact be real
• Decisions must be made blind of group assignment
• All modifications or withdrawals must be documented
23
V. Missing Outcome DataV. Missing Outcome Data
• Design with zero– missingness may be associated with treatment
• for analysis, data are not missing at random• even if same number missing, missing may be for
different reason in each treatment group
• Implement with minimum possible
• Analyze exploring different approaches– if all, or most, agree, then more persuasive
24
““Best” and “Worst” Best” and “Worst” Case AnalysesCase Analyses
Treatment Control
Total Events 170 220
Lost to Follow-up 30 10
"Best" Case 170 230
"Worst" Case 200 220
25
VI. Poor Quality DataVI. Poor Quality Data
26
Poor Quality Data (1)Poor Quality Data (1)
1. Lost to Follow-up (enforced withdrawals) NO DATA:
PROBLEMS:
– Not necessarily independent of treatment
– Raises questions about study conduct
27
Poor Quality Data (2)Poor Quality Data (2)
SOLUTIONS:
1. Keep to a minimum• Easiest if vital status is the outcome• Hardest if the response variables are
time-related measures requiring a hospital or clinic visit
2. Censor at the time lost– Can be done in survival analysis– Assumes independence of treatment
28
Poor Quality Data (3)Poor Quality Data (3)
SOLUTIONS:
3. Estimate missing data using previous data or averages
4. “Best” case and “worst” case analyses
29
VII. Poor Clinic Performance in VII. Poor Clinic Performance in a Multicenter Studya Multicenter Study
• If randomization was stratified by clinic, then withdrawal of a clinic is theoretically valid
• Withdrawal must be done independent of the outcome at that clinic
30
Mortality in Aspirin Myocardial Mortality in Aspirin Myocardial Infarction Study (AMIS)Infarction Study (AMIS)
Aspirin Placebo P-value
All 30 Centers 246/2267 219/2257 0.997 “Selected” Centers 39 66 < 0.01
• In “selected” centers, aspirin showed superiority
31
Mortality in Beta-Blocker Mortality in Beta-Blocker Heart Attack Trial (BHAT)Heart Attack Trial (BHAT)
Propranolol Placebo P-value
All 32 Centers 138/1916 188/1921 < 0.01
Cox adjusted Z = 3.05
6 “Selected” Centers 43 26 < 0.05
• In “selected” centers, propranolol worse
32
VIII. Special Counting RulesVIII. Special Counting Rules• Events beyond a specified number of days after
treatment stopped not counted "non-analyzable"
• Examples
1. "7 Day Rule" Anturane (1978) NEJM2. "28 Day Rule" Timolol (1981) NEJM
• If used, must– Specify in advance– Be a long period to insure termination not related to
outcome– Analyze results both ways
33
IX. Fishing orIX. Fishing orDichotomizing OutcomesDichotomizing Outcomes
• Common practice to define a response (S,F) from a non-dichotomous variable
• By changing our definition, we can alter results
• Thus, definitions stated in advance
• Definitions should be based on external data
34
Dichotomizing OutcomesDichotomizing Outcomes
Heart Rate
Trt A Trt B
Subject Pre Post Pre Post
1 72 72 0 72 702
2 74 73 1 71 68 3
...
25 73 73 0 79 79 0
Mean 74.0 73.2 0.8 74.4 74.0 0.4
Example
35
Three Possible Analyses (1)Three Possible Analyses (1)
Change Treatment A Treatment B P-Value
1.F = < 7 23 25 0.49
S = > 7 2 0
36
Three Possible Analyses (2)Three Possible Analyses (2)
Change Treatment A Treatment B P-Value
1.F = < 7 23 25 0.49
S = > 7 2 0
2.F = < 5 19 25 0.02
S = > 5 6 0
37
Three Possible Analyses (3)Three Possible Analyses (3)
Change Treatment A Treatment B P-Value
1.F = < 7 23 25 0.49
S = > 7 2 0
2.F = < 5 19 25 0.02
S = > 5 6 0
3.F = < 3 17 18 0.99
S = > 3 8 7
38
X. Time Dependent Covariate X. Time Dependent Covariate AdjustmentAdjustment
• Classic covariate adjustment uses baseline prognostic factors only– Adjust for Imbalance– Gain Efficiency
• Adjustment by time dependent variates not recommended in clinical trials (despite Cox time dependent regression model)
• Habit from epidemiology studies
39
Coronary Drug ProjectCoronary Drug Project5-Year Mortality5-Year Mortality
Baseline Cholesterol % DeathsCholesterol Change Clofibrate Placebo
< 250mg%* Fall 16.0 21.2
< 250 Rise 25.5 18.7
> 250 mg% Fall 18.1 20.2
> 250 ** Rise 15.5 21.3
• Little change in placebo group• Best to have
a. Low cholesterol getting lower *b. High cholesterol getting higher **
Example
40
Example: Cancer TrialsExample: Cancer Trials
• A common practice to compare survival on patients with a tumor response
• Problem is that patient must first survive to be a responder
length - bias sampling
41
Cancer Trials (1)Cancer Trials (1)
Advanced Breast Cancer: Surgery vs. Medicine
Santen et al. (1981) NEJM
(Letter to editor, Paul Meier, U of Chicago)
• A randomized clinical trial comparing surgical adrenalectomy vs. drug therapy in women with advanced breast cancer
• 17 pts withdrawn from surgery group
10 pts withdrawn from medical group
42
Cancer Trials (2)Cancer Trials (2)• Reasons
– Medical group (10 pts)
2 stopped taking their drugs
5 drug toxicity
– Surgical group (17 pts)
7 later refused surgery
8 rapid progression precluding surgery
• No follow-up data on these 27 pts presented
43
XI. Subgroup AnalysesXI. Subgroup Analyses
44
False Positive RatesFalse Positive RatesThe greater the number of subgroups analyzed separately, the larger the probability of making false positive conclusions.
No. of Subgroups False Positive Rate
1 .05
2 .08
3 .11
4 .13
5 .14
10 .19
45
Subgroup AnalysesSubgroup Analyses
• Focusing on a particular “significant” subgroup can be risky– Due to chance– Results not consistent
• Estimates not precise due to small sample size
46
MERIT Total MortalityMERIT Total Mortality
47
MERITMERIT
48
MERITMERIT(AHJ, 2001)
49
Praise IPraise IRef: NEJM, 1996
• Amlodipine vs. placebo• NYHA class II-III• Randomized double-blind• Mortality/hospitalization outcomes• Stratified by etiology (ischemic/non-ischemic)• 1153 patients
50
PRAISE IPRAISE I
51
PRAISE I - InteractionPRAISE I - Interaction
• Overall P = 0.07
• Etiology by Trt InteractionP = 0.004
• Ischemic P = NS
• Non-Ischemic P < 0.001
52
PRAISE I - IschemicPRAISE I - Ischemic
53
PRAISE I – Non- IschemicPRAISE I – Non- Ischemic
54
PRAISE IIPRAISE II
• Repeated non-ischemic strata
• Amlodipine vs. placebo
• Randomized double-blind
• 1653 patients
• Mortality outcome
• RR 1.0
55
Three ViewsThree Views
• Ignore subgroups and analyze only by treatment groups.
• Plan for subgroup analyses in advance. Do not “mine” data.
• Do subgroup analyses
However view all results with caution.
56
Analysis Issues SummaryAnalysis Issues Summary
• Important not to introduce bias into the analysis
• ITT principle is critical
• Important to have “complete” follow-up
• Off treatment is not off study