Bala Venkatesh Professor of Intensive Care Wesley...
Transcript of Bala Venkatesh Professor of Intensive Care Wesley...
![Page 1: Bala Venkatesh Professor of Intensive Care Wesley ...msic.org.my/sfnag402ndfbqzxn33084mn90a78aas0s9g/... · When is a trial stopped early •Treatments found to be convincingly different](https://reader036.fdocuments.us/reader036/viewer/2022080720/5f79dcdc5c839f5d9a6e778d/html5/thumbnails/1.jpg)
Bala Venkatesh
Professor of Intensive Care
Wesley & Princess Alexandra Hospitals
The George Institute for Global Health
University of Queensland & NSW
Risks of harm in stopping clinical trials early
![Page 2: Bala Venkatesh Professor of Intensive Care Wesley ...msic.org.my/sfnag402ndfbqzxn33084mn90a78aas0s9g/... · When is a trial stopped early •Treatments found to be convincingly different](https://reader036.fdocuments.us/reader036/viewer/2022080720/5f79dcdc5c839f5d9a6e778d/html5/thumbnails/2.jpg)
Scope of this talk
• The DMC or the DSMC – Origins
• Role of the DMC
• Trial stopping rules
• Risk of overestimation of benefit
• Examples of trials stopped early
![Page 3: Bala Venkatesh Professor of Intensive Care Wesley ...msic.org.my/sfnag402ndfbqzxn33084mn90a78aas0s9g/... · When is a trial stopped early •Treatments found to be convincingly different](https://reader036.fdocuments.us/reader036/viewer/2022080720/5f79dcdc5c839f5d9a6e778d/html5/thumbnails/3.jpg)
Controlled Clinical Trials 9:137-148 (1988)
![Page 4: Bala Venkatesh Professor of Intensive Care Wesley ...msic.org.my/sfnag402ndfbqzxn33084mn90a78aas0s9g/... · When is a trial stopped early •Treatments found to be convincingly different](https://reader036.fdocuments.us/reader036/viewer/2022080720/5f79dcdc5c839f5d9a6e778d/html5/thumbnails/4.jpg)
Monitoring of clinical trial progress
• early definitive evidence of benefit
• convincing evidence of harm,
• or sufficient evidence of no potential benefit to render continuation of the trial to be futile.
Recommendations and the discussions arising from the Greenberg report gave birth to the
DMC or DSMC
![Page 5: Bala Venkatesh Professor of Intensive Care Wesley ...msic.org.my/sfnag402ndfbqzxn33084mn90a78aas0s9g/... · When is a trial stopped early •Treatments found to be convincingly different](https://reader036.fdocuments.us/reader036/viewer/2022080720/5f79dcdc5c839f5d9a6e778d/html5/thumbnails/5.jpg)
![Page 6: Bala Venkatesh Professor of Intensive Care Wesley ...msic.org.my/sfnag402ndfbqzxn33084mn90a78aas0s9g/... · When is a trial stopped early •Treatments found to be convincingly different](https://reader036.fdocuments.us/reader036/viewer/2022080720/5f79dcdc5c839f5d9a6e778d/html5/thumbnails/6.jpg)
Role of the DMC
![Page 7: Bala Venkatesh Professor of Intensive Care Wesley ...msic.org.my/sfnag402ndfbqzxn33084mn90a78aas0s9g/... · When is a trial stopped early •Treatments found to be convincingly different](https://reader036.fdocuments.us/reader036/viewer/2022080720/5f79dcdc5c839f5d9a6e778d/html5/thumbnails/7.jpg)
DMC charter
![Page 8: Bala Venkatesh Professor of Intensive Care Wesley ...msic.org.my/sfnag402ndfbqzxn33084mn90a78aas0s9g/... · When is a trial stopped early •Treatments found to be convincingly different](https://reader036.fdocuments.us/reader036/viewer/2022080720/5f79dcdc5c839f5d9a6e778d/html5/thumbnails/8.jpg)
When is a trial stopped early • Treatments found to be convincingly different –
BHAT/CAST/Metoprolol • Trial stopped for futility –ARDSNet Hi vs Lo PEEP/ECMO • Side effects or toxicities are too severe - ILLUMINATE • Information external to the RCT making trial
unnecessary or unethical - CELECOXIB • Data quality is poor • Accrual is slow -CORTICUS • Scientific question is no longer important • Adherence to treatment low - CORTICUS • Resources to perform study are lost or diminished • Study integrity - undermined by fraud or misconduct
![Page 9: Bala Venkatesh Professor of Intensive Care Wesley ...msic.org.my/sfnag402ndfbqzxn33084mn90a78aas0s9g/... · When is a trial stopped early •Treatments found to be convincingly different](https://reader036.fdocuments.us/reader036/viewer/2022080720/5f79dcdc5c839f5d9a6e778d/html5/thumbnails/9.jpg)
Examples of stopping rules - benefit
![Page 10: Bala Venkatesh Professor of Intensive Care Wesley ...msic.org.my/sfnag402ndfbqzxn33084mn90a78aas0s9g/... · When is a trial stopped early •Treatments found to be convincingly different](https://reader036.fdocuments.us/reader036/viewer/2022080720/5f79dcdc5c839f5d9a6e778d/html5/thumbnails/10.jpg)
Stopping rules for futility
Stopping rules for harm
![Page 11: Bala Venkatesh Professor of Intensive Care Wesley ...msic.org.my/sfnag402ndfbqzxn33084mn90a78aas0s9g/... · When is a trial stopped early •Treatments found to be convincingly different](https://reader036.fdocuments.us/reader036/viewer/2022080720/5f79dcdc5c839f5d9a6e778d/html5/thumbnails/11.jpg)
Problems arise
• When there are liberal stopping rules
• When the totality of the evidence is not considered
• Biological plausibility of an effect not taken into consideration
![Page 12: Bala Venkatesh Professor of Intensive Care Wesley ...msic.org.my/sfnag402ndfbqzxn33084mn90a78aas0s9g/... · When is a trial stopped early •Treatments found to be convincingly different](https://reader036.fdocuments.us/reader036/viewer/2022080720/5f79dcdc5c839f5d9a6e778d/html5/thumbnails/12.jpg)
Epidemiology of trials stopped early
1) industry-funded drug trials in the areas of cardiology, hematology-oncology,
(HIV-AIDS).
2) Adequate descriptions of the methods used to inform the decision to stop the
trial were often lacking;
3) 76 of 143 prematurely terminated trials did not note one or more of the
following: a) the planned sample size, b) the interim
analysis after which the trial stopped, or c) whether a statistical
stopping rule informed the decision.
![Page 13: Bala Venkatesh Professor of Intensive Care Wesley ...msic.org.my/sfnag402ndfbqzxn33084mn90a78aas0s9g/... · When is a trial stopped early •Treatments found to be convincingly different](https://reader036.fdocuments.us/reader036/viewer/2022080720/5f79dcdc5c839f5d9a6e778d/html5/thumbnails/13.jpg)
Why is it problematic?
![Page 14: Bala Venkatesh Professor of Intensive Care Wesley ...msic.org.my/sfnag402ndfbqzxn33084mn90a78aas0s9g/... · When is a trial stopped early •Treatments found to be convincingly different](https://reader036.fdocuments.us/reader036/viewer/2022080720/5f79dcdc5c839f5d9a6e778d/html5/thumbnails/14.jpg)
cluster around the true effect; some trials will start and remain close to the true effect as data accumulate.
1) Effects of Multiple Testing and Overestimation of Treatment Effects
![Page 15: Bala Venkatesh Professor of Intensive Care Wesley ...msic.org.my/sfnag402ndfbqzxn33084mn90a78aas0s9g/... · When is a trial stopped early •Treatments found to be convincingly different](https://reader036.fdocuments.us/reader036/viewer/2022080720/5f79dcdc5c839f5d9a6e778d/html5/thumbnails/15.jpg)
1) 91 truncated RCTs with 424 matching non-truncated RCTs 2) Large differences in treatment effect size between truncated and nontruncated RCTs (ratio of relative risks 0.75) occurred with truncated RCTs having fewer than 500 events. 3) The pooled ratio of relative risks in truncated RCTs vs matching nontruncated RCTs was 0.71 (95% confidence interval, 0.65-0.77)
![Page 16: Bala Venkatesh Professor of Intensive Care Wesley ...msic.org.my/sfnag402ndfbqzxn33084mn90a78aas0s9g/... · When is a trial stopped early •Treatments found to be convincingly different](https://reader036.fdocuments.us/reader036/viewer/2022080720/5f79dcdc5c839f5d9a6e778d/html5/thumbnails/16.jpg)
What does the pooled ratio of RR in truncated vs matching non-truncated
RCTs of 0.71 mean?
• This implies that, for instance, if the RR from the non-truncated RCTs was 0.8 (a 20% RR reduction), the RR from the truncated RCTs would be on average approximately 0.57 (a 43% relative risk reduction, more than double the estimate of benefit).
• Non-truncated RCTs with no evidence of benefit—ie, with an RR of1.0— would on average be associated with a 29% relative risk reduction in truncated RCTs addressing the same question.
![Page 17: Bala Venkatesh Professor of Intensive Care Wesley ...msic.org.my/sfnag402ndfbqzxn33084mn90a78aas0s9g/... · When is a trial stopped early •Treatments found to be convincingly different](https://reader036.fdocuments.us/reader036/viewer/2022080720/5f79dcdc5c839f5d9a6e778d/html5/thumbnails/17.jpg)
2) Comprehensive Assessment of
Treatment Impact Limited
• Often, data are lacking on both long term effects and on patient-important outcomes that did not drive the decision to truncate the trial (
• e.g., overall survival
• Quality of life, or
• adverse events).
• Use of composite end points
![Page 18: Bala Venkatesh Professor of Intensive Care Wesley ...msic.org.my/sfnag402ndfbqzxn33084mn90a78aas0s9g/... · When is a trial stopped early •Treatments found to be convincingly different](https://reader036.fdocuments.us/reader036/viewer/2022080720/5f79dcdc5c839f5d9a6e778d/html5/thumbnails/18.jpg)
3) Lack of data on risk benefit ratios
• Fish WH, et al Effect of intramuscular vitamin E on mortality and intracranial hemorrhage in neonates of 1000 grams or less. Pediatrics. 1990;85:578-84.
• Brion LP, et al. Vitamin E supplementation for prevention of morbidity and mortality in preterm infants. Cochrane Database Syst Rev. 2003;4:CD003665.
![Page 19: Bala Venkatesh Professor of Intensive Care Wesley ...msic.org.my/sfnag402ndfbqzxn33084mn90a78aas0s9g/... · When is a trial stopped early •Treatments found to be convincingly different](https://reader036.fdocuments.us/reader036/viewer/2022080720/5f79dcdc5c839f5d9a6e778d/html5/thumbnails/19.jpg)
4) Misguided practice recommendations
• Publication in prominent journals
• Rapid dissemination in media
• Speedy incorporation into practice guidelines and quality assurance initiatives.
![Page 20: Bala Venkatesh Professor of Intensive Care Wesley ...msic.org.my/sfnag402ndfbqzxn33084mn90a78aas0s9g/... · When is a trial stopped early •Treatments found to be convincingly different](https://reader036.fdocuments.us/reader036/viewer/2022080720/5f79dcdc5c839f5d9a6e778d/html5/thumbnails/20.jpg)
Beta blockers in non-cardiac surgery
• A clinical trial of bisoprolol in patients with vascular
• disease having non-cardiac surgery with a planned sample size of 266 stopped early after enrolling 112 patients.
• Two of 59 patients in the bisoprolol group and 18 of 53 in the control group had experienced a composite endpoint event (cardiac death or myocardial infarction).
• The authors reported a 91% reduction in relative risk with a 95% confidence interval of 63% to 98%.
Rapidly incorporated into clinical practice
guidelines
![Page 21: Bala Venkatesh Professor of Intensive Care Wesley ...msic.org.my/sfnag402ndfbqzxn33084mn90a78aas0s9g/... · When is a trial stopped early •Treatments found to be convincingly different](https://reader036.fdocuments.us/reader036/viewer/2022080720/5f79dcdc5c839f5d9a6e778d/html5/thumbnails/21.jpg)
• In 2008 a systematic review and meta-analysis, including over 12 000 patients having non-cardiac surgery
• documented a 35% reduction in the odds of non-fatal myocardial infarction (95% CI 21% to 46%), but a twofold increase in non-fatal strokes (odds ratio 2.1, 27 to 3.68), and a possible increase in all cause mortality (1.20, 0.95 to 1.51)
![Page 22: Bala Venkatesh Professor of Intensive Care Wesley ...msic.org.my/sfnag402ndfbqzxn33084mn90a78aas0s9g/... · When is a trial stopped early •Treatments found to be convincingly different](https://reader036.fdocuments.us/reader036/viewer/2022080720/5f79dcdc5c839f5d9a6e778d/html5/thumbnails/22.jpg)
5) A freezing effect on the conduct of new trials
• Equipoise
• Ethical issues
• Disproportionate weight in SR-MA
![Page 23: Bala Venkatesh Professor of Intensive Care Wesley ...msic.org.my/sfnag402ndfbqzxn33084mn90a78aas0s9g/... · When is a trial stopped early •Treatments found to be convincingly different](https://reader036.fdocuments.us/reader036/viewer/2022080720/5f79dcdc5c839f5d9a6e778d/html5/thumbnails/23.jpg)
Examples of differences between interim analyses and final results
![Page 24: Bala Venkatesh Professor of Intensive Care Wesley ...msic.org.my/sfnag402ndfbqzxn33084mn90a78aas0s9g/... · When is a trial stopped early •Treatments found to be convincingly different](https://reader036.fdocuments.us/reader036/viewer/2022080720/5f79dcdc5c839f5d9a6e778d/html5/thumbnails/24.jpg)
Example 1: ADRENAL - Interim analyses
![Page 25: Bala Venkatesh Professor of Intensive Care Wesley ...msic.org.my/sfnag402ndfbqzxn33084mn90a78aas0s9g/... · When is a trial stopped early •Treatments found to be convincingly different](https://reader036.fdocuments.us/reader036/viewer/2022080720/5f79dcdc5c839f5d9a6e778d/html5/thumbnails/25.jpg)
DMC and close to 1st interim analysis
Sample size 950 – approximately CORTICUS X 2
and Annane X 3
Example 1: ADRENAL - Interim analyses
![Page 26: Bala Venkatesh Professor of Intensive Care Wesley ...msic.org.my/sfnag402ndfbqzxn33084mn90a78aas0s9g/... · When is a trial stopped early •Treatments found to be convincingly different](https://reader036.fdocuments.us/reader036/viewer/2022080720/5f79dcdc5c839f5d9a6e778d/html5/thumbnails/26.jpg)
As the study progressed….
Sample size 1840 – approximately 3.5 X
CORTICUS and Annane X 6
![Page 27: Bala Venkatesh Professor of Intensive Care Wesley ...msic.org.my/sfnag402ndfbqzxn33084mn90a78aas0s9g/... · When is a trial stopped early •Treatments found to be convincingly different](https://reader036.fdocuments.us/reader036/viewer/2022080720/5f79dcdc5c839f5d9a6e778d/html5/thumbnails/27.jpg)
DMC and 2nd interim analysis
Sample size 2900 – approximately CORTICUS X 6
and Annane X 10
![Page 28: Bala Venkatesh Professor of Intensive Care Wesley ...msic.org.my/sfnag402ndfbqzxn33084mn90a78aas0s9g/... · When is a trial stopped early •Treatments found to be convincingly different](https://reader036.fdocuments.us/reader036/viewer/2022080720/5f79dcdc5c839f5d9a6e778d/html5/thumbnails/28.jpg)
Study completion
![Page 29: Bala Venkatesh Professor of Intensive Care Wesley ...msic.org.my/sfnag402ndfbqzxn33084mn90a78aas0s9g/... · When is a trial stopped early •Treatments found to be convincingly different](https://reader036.fdocuments.us/reader036/viewer/2022080720/5f79dcdc5c839f5d9a6e778d/html5/thumbnails/29.jpg)
![Page 30: Bala Venkatesh Professor of Intensive Care Wesley ...msic.org.my/sfnag402ndfbqzxn33084mn90a78aas0s9g/... · When is a trial stopped early •Treatments found to be convincingly different](https://reader036.fdocuments.us/reader036/viewer/2022080720/5f79dcdc5c839f5d9a6e778d/html5/thumbnails/30.jpg)
Example 2: AML trial – 4 vs 5 courses of chemo
![Page 31: Bala Venkatesh Professor of Intensive Care Wesley ...msic.org.my/sfnag402ndfbqzxn33084mn90a78aas0s9g/... · When is a trial stopped early •Treatments found to be convincingly different](https://reader036.fdocuments.us/reader036/viewer/2022080720/5f79dcdc5c839f5d9a6e778d/html5/thumbnails/31.jpg)
Example 2: AML trial – 4 vs 5 courses of chemo
![Page 32: Bala Venkatesh Professor of Intensive Care Wesley ...msic.org.my/sfnag402ndfbqzxn33084mn90a78aas0s9g/... · When is a trial stopped early •Treatments found to be convincingly different](https://reader036.fdocuments.us/reader036/viewer/2022080720/5f79dcdc5c839f5d9a6e778d/html5/thumbnails/32.jpg)
Example 3: Intensive insulin therapy
in critically ill patients
![Page 33: Bala Venkatesh Professor of Intensive Care Wesley ...msic.org.my/sfnag402ndfbqzxn33084mn90a78aas0s9g/... · When is a trial stopped early •Treatments found to be convincingly different](https://reader036.fdocuments.us/reader036/viewer/2022080720/5f79dcdc5c839f5d9a6e778d/html5/thumbnails/33.jpg)
Example 4: Activated protein C
The trial was designed to enroll 2280 patients; two planned interim analyses by
an independent data and safety monitoring board occurred after 760 and 1520
patients had been enrolled. Statistical guidelines to suspend enrollment if
drotrecogin alfa acti- vated was found to be significantly more efficacious than
placebo were determined a priori and used the O’Brien–Fleming spending
function according to the method of Lan and Demets.
At the time of the second interim analysis of data from 1520 patients,
enrollment was suspended because the differences in the mortality rate
between the two groups exceeded the a priori guideline for stopping
![Page 34: Bala Venkatesh Professor of Intensive Care Wesley ...msic.org.my/sfnag402ndfbqzxn33084mn90a78aas0s9g/... · When is a trial stopped early •Treatments found to be convincingly different](https://reader036.fdocuments.us/reader036/viewer/2022080720/5f79dcdc5c839f5d9a6e778d/html5/thumbnails/34.jpg)
![Page 35: Bala Venkatesh Professor of Intensive Care Wesley ...msic.org.my/sfnag402ndfbqzxn33084mn90a78aas0s9g/... · When is a trial stopped early •Treatments found to be convincingly different](https://reader036.fdocuments.us/reader036/viewer/2022080720/5f79dcdc5c839f5d9a6e778d/html5/thumbnails/35.jpg)
Example 5: ECMO trial by Coombes at al stopped for futility
![Page 36: Bala Venkatesh Professor of Intensive Care Wesley ...msic.org.my/sfnag402ndfbqzxn33084mn90a78aas0s9g/... · When is a trial stopped early •Treatments found to be convincingly different](https://reader036.fdocuments.us/reader036/viewer/2022080720/5f79dcdc5c839f5d9a6e778d/html5/thumbnails/36.jpg)
Take home message – Stopping clinical trials early for benefit
• Risk of type 1 error
• Not just look at primary outcome
• Secondary outcomes and safety issues
• Totality of available evidence
• Statistical boundary is just one guideline