Controversy? What controversy? An attempt to structure the debate on adaptive designs Marc...
-
Upload
aldous-bradley -
Category
Documents
-
view
214 -
download
0
Transcript of Controversy? What controversy? An attempt to structure the debate on adaptive designs Marc...
Controversy? What controversy?An attempt to structure the debate on adaptive designs
Marc Vandemeulebroecke
MCP, Vienna, 11.7.2007
Vandemeulebroecke Controversy? What controversy?1
Vandemeulebroecke Controversy? What controversy?2
A. Wald (1945): „The National Defense Research Committee considered these developments sufficiently useful for the war effort to make it desirable to keep the results out of the reach of the enemy.“
Examples (1): Sequential analysis
F. J. Anscombe (1963): „‘Sequential analysis‘ is a hoax.“
P. Armitage (1989): „If [...] ‘sequential analysis is a hoax‘, then its proponents are amongst the most persuasive swindlers, and their clients amongst the most gullible consumers, in the history of scientific endeavour.“
Vandemeulebroecke Controversy? What controversy?3
Example (2): Group sequential designs
such that for all,1, KkIk 21)( kIP k
Repeated confidence intervals (C. Jennison, B. Turnbull 1989)
P. Armitage: „ingenious derivation“
S. J. Pocock: „a minor change of presentation style“
Example (3): Adaptive/flexible designs
Vandemeulebroecke Controversy? What controversy?4
C.-F. Burman, C. Sonesson (2006): Are flexible designs sound?
M. Proschan: „Such methods, if extremely abused, produce illogical conclusions, but that is no more a condemnation of adaptive methods than Jack the Ripper is a condemnation of cutlery.“
P. Bauer: „Is wine sound? It is the way to drink it that matters.“
„The weighted test violates inference principles [...] and may lead to unreasonable results.“
03.010000 100 XNTest H0: with ; interim:
005.0,28.25.2,1 1011012 XZXnChange to observe
Example (4): Terminology
Vandemeulebroecke Controversy? What controversy?5
A. J. Phillips, O. N. Keene (2006): „The terms ‘flexible design’ and ‘adaptive design’ are used interchangeably.”
C. Jennison, B. Turnbull (2006): „The adjectives ‘flexible’ and ‘adaptive’ are not synonymous. Indeed adaptive designs can range from being totally inflexible to very flexible.”
Vandemeulebroecke Controversy? What controversy?6
Issues
Imagine a study protocol:• Double-blind multicenter trial in Crohn‘s Disease• Active vs placebo 1:1• Endpoint: CDAI after 4 weeks• IA at half of the sample size, switch to 2:1 if encouraging
Trial conduct:• IA disappointing, but new budget available Continue 1:1, ad hoc increase of sample size• Final analysis correctly adjusted and significant• In stage 2, patients are more severe, and placebo is doing much worse
Issue 1: Feasibility
Vandemeulebroecke Controversy? What controversy?7
Practical, logistical or operational issues regarding the conduct of the study
• Speed of recruitment vs. speed of response
• Availability of specialized software
• „Supply chain management“: Data capture/cleaning/ processing; real-time randomization and drug supply
• Training/SOPs/“charter“
• Investigate behavior of procedure by simulation, play through scenarios with decision makers / IDMC
Vandemeulebroecke Controversy? What controversy?8
Statistical/mathematical property.
• Correct type I error
• Point estimates available and sensible (e.g., mean/median unbiased)
• Confidence intervals available, sensible and correct
Issue 2: Validity
Vandemeulebroecke Controversy? What controversy?9
Hard to quantify. Concern: „operational bias“
Consistency/homogeneity across trial stages (A. Koch: identification problem) MCP: T. Friede, A. Faldum...
Jeopardizing integrity means jeopardizing credibility.
Measures to maintain trial integrity:
• avoid unblinding if possible • control dissemination of results• control scope and amount of adaptations
Issue 3: Integrity
Vandemeulebroecke Controversy? What controversy?10
V. Dragalin (2006, PhRMA):
„To maintain study validity means providing correct statistical inference [...], ensuring consistency between different stages of the study, and minimizing operational bias.
To maintain study integrity means providing convincing results to a broader scientific community; preplanning, as much as possible, based on intended adaptations; and maintaining the blind of interim analysis results.“
Issues 2/3: Validity/Integrity
H. Schäfer (2006): internal validity - external validity
Vandemeulebroecke Controversy? What controversy?11
Only interpretable results are communicable.
Intregrity and interpretability have the main impact of the persuasiveness of the study results.
• Endpoints meaningful
• Point estimates and confidence intervals available and sensible
Jeopardizing integrity (e.g., across-stage consistency or amount of adaptations) will also reduce interpretability.
Issue 4: Interpretability
Vandemeulebroecke Controversy? What controversy?12
Issue 5: Efficiency
Often, adaptation comes at the cost of efficiency.
• ...in terms of power / expected information
• Reason: non-sufficient statistics
Any adaptive design can be outperformed by a nonadaptive design - and vice versa!
„Statistical efficiency is not always the priority“ (Koch 2006)
„Response-dependent choice of group sizes can itself be a source of improved efficiency“ (Schmitz 1993, Jennison/Turnbull 2006)
Vandemeulebroecke Controversy? What controversy?13
Two main questions:
• Should there be an a priori selection of study features that may be adapted? If so, now many and which?
• Should the design changes follow prespecified rules or may they be done ad hoc? Should they depend on information internal or external to the trial?
Issue 6: Flexibility
stopping
+ stage sizes
+ other
Ada
ptab
le
feat
ures
Prespeficied rule Not dep. on trial-internal data Completely freeNature of change
Group sequential
Spending functions
p-value combinations
„Schmitz designs“
Vandemeulebroecke Controversy? What controversy?14
Issues• Feasibility: „Only“ operational issues
• Validity: „Only“ mathematical issues
• Integrity: Requirements?
„Operational bias“? Homogeneity across stages?
• Interpret‘y: Requirements?
• Efficiency: Weigh against flexibility (sponsor‘s choice)
• Flexibility:
Summary/conclusions
A priori selection of adaptable features? How many/which?
Adaptations ad hoc or following prespecified rules? Depending on internal or external information ?
Biom J 48 (2006, special issue), in particular: O‘Neil, Hung et al., Koch, and discussants: Section on „Regulatory perspectives“, pp. 559-622
Burmann, Sonesson (2006): Are flexible designs sound? Biometrics 62, pp. 664-683 (with discussion)
EMEA (2006): Reflection paper on methodological issues in confirmatory clinical trials with flexible design and analysis plan. CHMP/EWP/2459/02
Fleming, DeMets (1993): Monitoring of clinical trials: Issues and Recommendations. Contr Clin Trials 14, pp. 183-197
Gallo et al. (PhRMA, 2006): Adaptive designs in clinical drug development - an executive summary of the PhRMA working group. J Biopharm Stat 16, pp. 275-283 (with discussion)
Jennison, Turnbull (2006): Adaptive and nonadaptive group sequential tests. Biometrika 93, pp. 1-21
Phillips, Keene (PSI, 2006): Adaptive designs for pivotal trials: discussion points from the PSI adaptive design expert group. Pharm Stat 5, pp. 61-66
PhRMA (2006): Full white paper. DIJ 40, pp. 421-484
Posch, Bauer, Brannath (2003): Issues in designing flexible trials. Stat in Med 22, pp. 953-969
Stat in Med 25 (2006, special issue), pp. 3229-3408
Some references
Vandemeulebroecke Controversy? What controversy?16
...and many thanks to Norbert Benda!
Appendix: Terminology
Vandemeulebroecke Controversy? What controversy?A1
PhRMA: Definition of “adaptive design”
P. Gallo et al (2006): “a clinical study design that uses accumu-lating data to decide how to modify aspects of the study as it continues, without undermining the validity and integrity of the trial. [...] In such trials, changes are made ‘by design’, and not on an ad hoc basis.”
V. Dragalin (2006): “a multistage study design that uses accumulating data to decide how to modify aspects of the study without undermining the validity and integrity of the trial. [...] preplanning, as much as possible, based on intended adaptations.”
Appendix: Terminology
Vandemeulebroecke Controversy? What controversy?A2
Other definitions of “adaptive design”
EMEA (2006): “A study is called ‘adaptive’ if statistical methodology allows the modification of a design element [...] at an interim analysis with full control of the type I error.”
Novartis slide deck: “Adaptive designs allow for initial uncertainties in trial design to be confirmed/adapted during the trial. The integrity of the trial is maintained and the evidence for the same hypothesis before and after the adaptation is combined.”
Appendix: Terminology
Vandemeulebroecke Controversy? What controversy?A3
Other terms
• flexible• adaptive combination• flexible group sequential• flexible adaptive• adaptive group sequential• internal pilot• self designing• seamless adaptive• design automates• ....
(List stolen from J. Röhmel‘s talk at BfArM Discussion on Adaptive Designs, 2006)
Vandemeulebroecke Controversy? What controversy?A4
Jennison, Turnbull (2006): „The strength of adaptive redesign lies in coping with the unexpected, in particular responding to external information that could not have been anticipated at the start of a study.“
Appendix: Flexibility
P. Gallo et al. (2006): „Should or should it not be viewed that adaptive designs might remedy inadequate planning?“
M. Frisén (2006): „One should use planned adaptive designs when one expects that it will be necessary to adapt the design. If one unexpectedly has strong reasons to change the plans, one should be very careful.“