Adaptive Clinical Trials: A Necessary Step toward

Personalized Medicine

Adaptive Clinical Trials: A Necessary Step toward

Personalized Medicine Donald A. Berry

dberry@mdanderson.orgDonald A. Berry

dberry@mdanderson.org

22

Financial DisclosureFinancial Disclosure

Part owner Berry Consultants, LLCDesigns adaptive trials for Pharmaceutical companiesMedical device companiesNIH-sponsored cooperative groups

Part owner Berry Consultants, LLCDesigns adaptive trials for Pharmaceutical companiesMedical device companiesNIH-sponsored cooperative groups

33

Janet Woodcock, Director CDER FDAJanet Woodcock,

Director CDER FDA

“Improved utilization of adaptive

and Bayesian methods” could help

resolve low success rate of and

expense of phase 3 clinical trials

“Improved utilization of adaptive

and Bayesian methods” could help

resolve low success rate of and

expense of phase 3 clinical trials

44

FDA’s Critical Path Opportunities Report (2006)

FDA’s Critical Path Opportunities Report (2006)

“uncovered a consensus that the two most important areas for improving medical product development are biomarker development and streamlining clinical trials.”

“uncovered a consensus that the two most important areas for improving medical product development are biomarker development and streamlining clinical trials.”

http://www.fda.gov/ScienceResearch/SpecialTopics/CriticalPathInitiative/default.htm

55

For example, in 2010, the Biomarkers Consortium–a public-private partnership that includes the NIH, the FDA, patient groups, and pharmaceutical and biotech–initiated a groundbreaking trial in breast cancer to predict drug responsiveness based on the presence or absence of genetic and biological markers, … I-SPY 2 (ClinicalTrials.gov NCT01042379).

66

"Our current approach [to trials] is horribly inefficient, and we need to do something better," says Roger Lewis, an emergency medicine physician at Harbor-University of California, Los Angeles, Medical Center. Lewis helps advise a company called Berry Consultants …

77

Lewis and Berry, along with emergency medicine physician William Barsan at the University of Michigan, will be studying whether "adaptive" trial designs that incorporate new information in midcourse can answer medical questions. They also want to learn what concerns researchers might have about this approach.

88

Current use of Bayesian adaptive designs

Current use of Bayesian adaptive designs

MDACC (> 300 trials) Device companies (> 25 PMAs)* Drug companies (Most of

top 40; many biotechs)**

MDACC (> 300 trials) Device companies (> 25 PMAs)* Drug companies (Most of

top 40; many biotechs)** *http://www.fda.gov/MedicalDevicesDeviceRegulationandGuidance/GuidanceDocuments/ucm071072.htm**http://www.fda.gov/downloads/DrugsGuidanceCompliance RegulatoryInformation/Guidances/UCM201790.pdf

99

Example: Troxacitabine in AML* (endpoint: CR by day 50)

Example: Troxacitabine in AML* (endpoint: CR by day 50)

RANDOMIZE

RANDOMIZE

IdarubicinAra-C

IdarubicinAra-C

Trox Idarubicin

Trox Idarubicin

TroxAra-C Trox

Ara-C

n = 25

n = 25

n = 25

Standard designStandard design

Adaptiverandomizationto learn, while

effectivelytreating

patients in trial

Adaptiverandomizationto learn, while

effectivelytreating

patients in trial

Our designOur design

* Giles JCO 2003

1010

Adaptive RandomizationAdaptive Randomization

Assign with higher probability to better performing therapies

TI dropped after 24th patient Trial stopped after 34 patients

Assign with higher probability to better performing therapies

TI dropped after 24th patient Trial stopped after 34 patients

1111

Summary of AML trial results

Summary of AML trial results

CR by 50 days:IA 10/18 = 56%TA 3/11 = 27%TI 0/5 = 0%

CR by 50 days:IA 10/18 = 56%TA 3/11 = 27%TI 0/5 = 0%

1212

Some areas of application of Bayesian adaptive drug trialsSome areas of application of Bayesian adaptive drug trials Migraine Oncology Rh Arthritis Lupus Diabetes Obesity Stroke Gastroparesis

Migraine Oncology Rh Arthritis Lupus Diabetes Obesity Stroke Gastroparesis

Spinal Cord Injury HIV Hepatitis C Pre-term labor Constipation Overactive bladder Libido Alzheimer’s

Spinal Cord Injury HIV Hepatitis C Pre-term labor Constipation Overactive bladder Libido Alzheimer’s

1313

Simulations Usually RequiredSimulations Usually Required To find operating characteristics: Type I error rate Power Sample size distribution

Prospective design essential Longitudinal modeling Many scenarios Accrual rate matters

To find operating characteristics: Type I error rate Power Sample size distribution

Prospective design essential Longitudinal modeling Many scenarios Accrual rate matters

1414

"Our current approach [to trials] is horribly inefficient, and we need to do something better," says Roger Lewis, an emergency medicine physician at Harbor-University of California, Los Angeles, Medical Center. Lewis helps advise a company called Berry Consultants …

1515

Lewis and Berry, along with emergency medicine physician William Barsan at the University of Michigan, will be studying whether "adaptive" trial designs that incorporate new information in midcourse can answer medical questions. They also want to learn what concerns researchers might have about this approach.

1616

Bayesian adaptive trialsBayesian adaptive trialsStopping early (or late)EfficacyFutility

Dose finding (& dose dropping)Seamless phasesPopulation findingAdaptive randomizationRamping up accrual

Stopping early (or late)EfficacyFutility

Dose finding (& dose dropping)Seamless phasesPopulation findingAdaptive randomizationRamping up accrual

1717

Why?Why?

Smaller trials (usually!)More accurate conclusionsCan focus on better

treatment of patients in trials

Smaller trials (usually!)More accurate conclusionsCan focus on better

treatment of patients in trials

1818

Three Recent Examples of Smaller Sample Size UsingThree Recent Examples of Smaller Sample Size Using

Bayesian predictive probabilities

Longitudinal modeling

Bayesian predictive probabilities

Longitudinal modeling

1919

A Bayesian statistical design was used with arange in sample size from 600 to 1800 patients.

2020

From “Methods”From “Methods”

“These interim analyses were not the standard type in which the trial results are announced when a boundary is crossed. Rather, the decision to discontinue enrollment was based on a prediction that future follow-up was likely to give a meaningful answer.”

“These interim analyses were not the standard type in which the trial results are announced when a boundary is crossed. Rather, the decision to discontinue enrollment was based on a prediction that future follow-up was likely to give a meaningful answer.”

2121

2222

Time to Chronic Failure by Randomization Group (Updated)

Time to Chronic Failure by Randomization Group (Updated)

64%

16%

TCool 103

69 69 66 63 62 61 54 52 37 15 3 2

AAD 56 39 29 19 16 13 11 10 7 2 0 0 0• Circles in the graph represent censored patients

THERMOCOOL

Free

dom

from

Chr

onic

Fai

lure

Days into Effectiveness Follow-up

64%

16% AAD

0 30 60 90 120 150 180 210 240 270 300 330 3600.0

0.2

0.4

0.6

0.8

1.0

Adaptive Design-Key Features

Response-Adaptive RandomizationUses interim data and Bayesian techniques to modify the treatment allocation

probabilities in favor of more informative treatment arms.• Maximize information about dose-response relationship• increases the probability of success in Phase 3 by bringing the correct dose forward

into Phase 3

Predefined Decision RulesFrequent interim analyses allows determination of futility or success• Allows Ph2a and Ph2b to be combined in a single trial• Succeed efficiently and fail efficiently

Trial simulationsDefine operating characteristics (false + rate, statistical power, and sample size

distributions)

Uses accumulating data to decide on how to modify aspects of the study without undermining the validity and integrity of the trial.

Robert Lenz, San Diego, Oct 5, 2009

Response-Adaptive Randomization

New Patient

Randomizationto placebo or optimal

dose

Find optimal

dose1

2

Sufficientinformation

obtained

Futility

Decision

Continue until max. sample size reached

6

9

8

7

10

Ongoing Pts Data

3 Estimation of Dose-Response

5

Longitudinal model predicts final outcome

4

Robert Lenz, San Diego, Oct 5, 2009

Response-Adaptive RandomizationA

DA

S-C

og S

core

Doses (mg)0 5 10 15 20 30 35

Area of the D-R relationship we want greatest information about

Robert Lenz, San Diego, Oct 5, 2009

Goal is to Find the ED90 and MEDA

DA

S-C

og S

core

Doses (mg)0 5 10 15 20 30 35

MED ED90

ED90: smallest dose that produces 90% of the maximal efficacy over the doses studied

MED: minimally effective dose; operationally defined as the smallest dose that produces at least 1.75 point improvement over placebo in ADAS-Cog

Robert Lenz, San Diego, Oct 5, 2009

Adaptive Randomization

• The probability of allocating the subsequent subject to a particular dose is proportional to the reduction in the variance of ED90 achieved when a subject is allocated to that dose.

• The allocation ratio will be altered in such as way as to maximize the information (minimize the variance) about the MED and ED90.

• Randomization probabilities are updated every 2 weeks.

Robert Lenz, San Diego, Oct 5, 2009

Normal Dynamic Linear Model

• Normal Dynamic Linear Model is used to describe the dose-response relationship.

• This is a very versatile and flexible model that allows for non-monotonic response functions.

• Allows borrowing strength from neighboring doses (don’t treat each dose as entirely independent), thereby narrowing the probability distribution around the dose.

• Thus, increasing the number of doses increases information about the entire dose-response curve, not just a single dose.

Robert Lenz, San Diego, Oct 5, 2009

Simulations: Sensitivity Analyses

• The primary simulation is based on the best guesses of accrual rate and correlation between early (week 4 and 8) and final time point (week 12) for the ADAS-Cog scores.

• Maximum sample size is 400

• When considering all the simulated D-R curves, very favorable operating characteristics

• False positive rate less than 5%• False negative rate < 20%

• Average sample size across D-R relationships is 319

Robert Lenz, San Diego, Oct 5, 2009

Model estimates of ACAScog(every two weeks)

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

46

47

48

49

50

May 14, 2009: Stop for futility

51

N = 322Completers = 238

Same as previous, but dropping incomplete patients52

53

Final

Sample sizes

54

55

56

57

58

59

60

61

62

63

64

65

66

67

68

69

70

71

72

73

74

May 14, 2009: Stop for futility75

76

Final

7777

Adaptive Phase II/III TrialAdaptive Phase II/III Trial

7878

IoM Report on Cancer Cooperative Groups 2010

7979

Example from Critical Path Initiative

Example from Critical Path Initiative

Type II diabetesSeamless Phase II/III: Dose

finding plus confirmationSample size 200 - 1566Active comparator & placeboPrimary endpoint:

Clinical Utility Index (12 months)

Type II diabetesSeamless Phase II/III: Dose

finding plus confirmationSample size 200 - 1566Active comparator & placeboPrimary endpoint:

Clinical Utility Index (12 months)

8080

Some DetailsSome DetailsPhase II: 7 doses experimental drug

plus placebo plus active controlPhase III1 or 2 doses experimental drugSample size via predictive power

considering available Phase II dataAdaptive transition: Bayesian

predictive probsBoth phases driven by CUI

Phase II: 7 doses experimental drug plus placebo plus active control

Phase III1 or 2 doses experimental drugSample size via predictive power

considering available Phase II dataAdaptive transition: Bayesian

predictive probsBoth phases driven by CUI

8181

Clinical Utility IndexClinical Utility Index

Dose-response modeling Longitudinal modelingDose-response modeling Longitudinal modeling

Exp – AC12 mos

Exp – AC12 mos

Exp – P6 mos

Exp – P6 mos

0 0 0 0

8282

Adaptive Biomarker TrialAdaptive Biomarker Trial

8383

EXAMPLE

TRIAL

8484

Savings possible in sample size when

using biomarkers …

Savings possible in sample size when

using biomarkers …

8585

Berry et al. SABCS 2009

8686

I‐SPY2: The Cartoon(Press conference* slides)

*<http://ispy2.org>

Standard Phase 2 Cancer Drug Trials

Experimental armExperimental armPopulationof patients

Outcome:Tumor 

shrinkage?

Populationof patients

Outcome:Longer timedisease free

RANDOMIZE

Standard Phase 2 Cancer Drug Trials

Experimental drugExperimental drug

Outcome:Longer timedisease free

RANDOMIZE

Outcome:Tumor 

shrinkage?

Populationof patients

Populationof patients

Consequence:34% success rateof Phase 3 Trials 

I‐SPY2 TRIAL

Outcome:Complete response at surgery

Populationof patients

ADAPTIVELY

RANDOMIZE

I‐SPY2 TRIAL

Outcome:Complete response at surgery

Arm 2 graduates to small focused Phase 3 trial

Populationof patients

ADAPTIVELY

RANDOMIZE

I‐SPY2 TRIAL

Outcome:Complete response at surgery

Populationof patients

Arm 3 dropsfor futility

ADAPTIVELY

RANDOMIZE

I‐SPY2 TRIAL

Outcome:Complete response at surgery

Arm 5 graduates to small focused Phase 3 trial

Populationof patients

RANDOMIZE

ADAPTIVELY

I‐SPY2 TRIAL

Outcome:Complete response at surgery

Arm 6 isadded tothe mix

Populationof patients

RANDOMIZE

ADAPTIVELY

Goal: Greater than 85% success rate in 

Phase 3, with focus onpatients who benefit

9595

9696

I-SPY2 Adaptive Design ProcessI-SPY2 Adaptive Design Process PI: Laura Esserman, UCSF Sponsored by FNIH: NCI, FDA,

industry, academia, Coordinated with FDA (CDER,

CBER & CDRH) from inception Current status: 20 centers, ~50

patients, experimental drugs so far: neratinib, ABT888, AMG386

PI: Laura Esserman, UCSF Sponsored by FNIH: NCI, FDA,

industry, academia, Coordinated with FDA (CDER,

CBER & CDRH) from inception Current status: 20 centers, ~50

patients, experimental drugs so far: neratinib, ABT888, AMG386

9797

I-SPY 2 EffectsI-SPY 2 Effects Match drugs (& combos) with

biomarker signatures Graduate drug/biomarker pairs to

smaller (n < 300), more focused, more successful Phase 3

Descendents of I-SPY 2 in melanoma, colorectal cancer, Alzheimer’s, acute heart failure, …

Match drugs (& combos) with biomarker signatures

Graduate drug/biomarker pairs to smaller (n < 300), more focused, more successful Phase 3

Descendents of I-SPY 2 in melanoma, colorectal cancer, Alzheimer’s, acute heart failure, …

9898

Software?Software?

SAS ADAPT?SAS ADAPT?

Copyright © 2010 Berry Consultants, LLC, SAS Institute Inc., and Tessella Inc. All rights reserved.

BeST: Solution for Smarter Clinical Trials

Solution Design

Trial Simulation Engine

Design Specs

SAS Analysis

Simulation Simulation Results

JMP/SASReview & 

Visualization

Design Selection/ Iteration

FinalDesign

Trial ExecutionEngine

Data Source

TrialData

Results

SAS Analysis

Simulation Actual trial Dashboard

DSMB

101101

Also, Selected Short Subject …Also, Selected

Short Subject …

102102

Designing a clinical trial is making a decision

Designing a clinical trial is making a decision

Motivated by dinner with Joel Tsevat and

John Perentisis …

Motivated by dinner with Joel Tsevat and

John Perentisis …

103103

Decision analysis as a medical research paradigm

Decision analysis as a medical research paradigm

Why do we carry out clinical trials of experimental products?(1) To see if they are safe and effective.(2) To see if the results are unusual

assuming product is ineffective.(3) To deliver good medicine to patients.

Does your answer matter?

104104

Standard Approach to Choosing Sample SizeStandard Approach to Choosing Sample Size

Consider time to event. Want 25% reduction in hazards Type I error 5%, two-sided Power 80% Accrual rate 2/month Median for control: 8 months Follow-up after accrual: 12 months

Answer: n = 650

Consider time to event. Want 25% reduction in hazards Type I error 5%, two-sided Power 80% Accrual rate 2/month Median for control: 8 months Follow-up after accrual: 12 months

Answer: n = 650

Where are the questions about the disease? the population? Is this

CABG or a rare pediatric cancer?

Where are the questions about the disease? the population? Is this

CABG or a rare pediatric cancer?

105105

We need a new paradigm!

We need a new paradigm!

106106

Choosing sample size using decision analysisChoosing sample size

using decision analysis Goal: Effective overall treatment of

patients, both those who come after the trial and those in the trial

Example formalizing this goal: Maximize expected number of successes over all patients

Goal: Effective overall treatment of patients, both those who come after the trial and those in the trial

Example formalizing this goal: Maximize expected number of successes over all patients

107107

Compare Joffe/Weeks JNCI Dec 18, 2002

Compare Joffe/Weeks JNCI Dec 18, 2002

“Many respondents viewed the main societal purpose of clinical trials as benefiting the participants rather than as creating generalizable knowledge to advance future therapy. This view, which was more prevalent among specialists such as pediatric oncologists that enrolled greater proportions of patients in trials, conflicts with established principles [from Belmont Report] of research ethics.”

“Many respondents viewed the main societal purpose of clinical trials as benefiting the participants rather than as creating generalizable knowledge to advance future therapy. This view, which was more prevalent among specialists such as pediatric oncologists that enrolled greater proportions of patients in trials, conflicts with established principles [from Belmont Report] of research ethics.”

We must rethink the “established principles

of research ethics”

We must rethink the “established principles

of research ethics”

108108

“More ethical” approach:Maximize overall benefit“More ethical” approach:Maximize overall benefit

What is “overall”?All patients who will be treated

with therapies assessed in trialCall it N, “patient horizon”Enough to know magnitude of N:

100? 1000? 1,000,000?

What is “overall”?All patients who will be treated

with therapies assessed in trialCall it N, “patient horizon”Enough to know magnitude of N:

100? 1000? 1,000,000?

109109

Goal: maximize expected number of successes in N Either one- or two-armed trial Suppose n = 1000 is right for one trial & N = 1,000,000 Then for other N’s use n =

Goal: maximize expected number of successes in N Either one- or two-armed trial Suppose n = 1000 is right for one trial & N = 1,000,000 Then for other N’s use n =

1000

1,000,000

110110

Prior distributions?Prior distributions?

111111

Optimal allocations in a two-armed trialOptimal allocations in a two-armed trial

112112

√N3√N

2 √N2

113113

ConclusionsConclusions Standard clinical research paradigm

doesn’t apply to rare populations Decision analysis addresses right

questions, hypothesis testing does not Bayesian adaptive approach balances

experimentation and treating trial participants effectively

Bayesian attitude uses all info “All info” includes longitudinal info and

borrowing adult info to children

Standard clinical research paradigm doesn’t apply to rare populations

Decision analysis addresses right questions, hypothesis testing does not

Bayesian adaptive approach balances experimentation and treating trial participants effectively

Bayesian attitude uses all info “All info” includes longitudinal info and

borrowing adult info to children