A Fireside Chat about Precision Medicine 3mm and a ... · Strati cation Free Adaptive Enrichment...

A Fireside Chat about Precision Medicine

and a [somewhat] more informed discussion of

Stratification Free Adaptive Enrichment

Noah Simon

July, 5 2017

1 / 53

An Dialogue about Precision Medicine

Let’s begin with a discussion about Precision Medicine.

2 / 53

The following ideas may have partly been borrowed from...

He’s the one in the center... giving everyone else a hard time atseminar!

3 / 53

What is Precision Medicine?

I will stick mostly to oncology...

Not because I know much there...

But I definitely know less about everything else!

4 / 53


The practice of medicine has always been about

I characterizing dysfunction

I treating based on specific characterizations

5 / 53


In the beginning this was based on simple observation alone:

you’ve been vomiting and missed your period −→ Pregnant

Now we have more sophisticated methods:

hCG in urine −→ Pregnant

In oncology, tumors are characterized using histology

6 / 53


My understanding is:

Medicine attempts to differentiate diseases...

to develop treatments that target specific disease characteristics

Precision medicine attempts to differentiate diseases...

more precisely?


That reads like a high schooler “not-plagiarizing” an essay...

7 / 53


My understanding is:

Medicine attempts to differentiate diseases...


[Biomolecular] Precision medicine attempts to differentiate diseasesusing biomolecular profiling

to develop treatments that target specific biomolecular diseasecharacteristics

8 / 53

What am I leaving out

Screening diagnostics

eg. cfDNA

Actionable prognostic biomarkers

eg. oncotypeDX

Often forgotten that the goal is to find actionable biomarkers

9 / 53

Back to “Predictive Biomarkers”

Two common scenarios:

Developing a targeted treatment + diagnostic

Developing a new diagnostic, for an existing, non-targetedtreatment

10 / 53

Targeted Treatments

30+ targeted cancer drugs1 with many different targets

The primary FDA-specified “biomolecular” indications were

I HER2/HR status

I KRAS/EGFR mutation

I BRAF mutation

Many with no “biomolecular indication”...

only approved in very specific cancer-types though!

(histology-based personalization!)

1from “Overview of FDA-approved Anti-Cancer Drugs Used for TargetedTherapy” WCRJ 2015; 2(3) e553

11 / 53

The Road to Failure in Precision Medicine

Where have I seen little success?

Characterizing the [in]effectiveness of non-targeted treatments

Why do poor treatments tend not to work?

???

Why do I tend to miss free throws?

Because I keep forgetting to wear my lucky shirt...?

Or maybe because I’m generally bad at basketball...

12 / 53

The Road to Failure in Precision Medicine

Where have I seen little success?

Characterizing the [in]effectiveness of non-targeted treatments

Why do poor treatments tend not to work?

Because they tend not to work...

Why do I tend to miss free throws?

Because I keep forgetting to wear my lucky shirt...?

Or maybe because I’m generally bad at basketball...

13 / 53

The Road to Success in Precision Medicine?

What is the best place for statisticians on that road?

Is it building fancier methods?

(in some avenues things work pretty well with simple methods)

Or domain expertise?

Or some other option?

14 / 53

Solve Easy Problems!

EE/CS does this well!

Very approximately solve useful + “easy” domain problems

Statistics seems to have more deep, but slow prodding phenotype.

sometimes the problems are messy...

15 / 53

Now to the real talk!

Stratification-free Adaptive Enrichment Designs

16 / 53

In Collaboration with...

17 / 53

Biomarkers

Prognostic

Gives information on outcome independent of treatment

Generally not informative for treatment decisions

(Main effect term)

Predictive

Gives information on relative effectiveness of treatments

Informative for treatment decisions

(Interaction term)

18 / 53

Developing a Drug and a Companion Biomarker

What is done now?

Have a targeted drug and potential clinical/genomic featuresof relevence.

Run a large clinical trial for general efficacy

Reserve a bit of α for sub-group exploration

Posthoc sub-group analysis

19 / 53

Developing a Drug and a Companion Biomarker

Adaptive Enrichment Trial

Have a targeted drug and potential clinical/genomic features ofrelevence.

Concurrently

I Develop a biomarker to target treatment

I Enrich enrollment population based on biomarker

20 / 53

An Adaptive Approach

Stratification

1. Break patients into arms by predefined biomarker

2. As the trial runs, discontinue arms which do not show benefit

3. At termination, cleverly test arms and determine beneficiarysubset

Strong Approach When

Biomarker is a simple score (1 dimensional)

Little adaptation is needed

21 / 53

Stratification-free Enrichment

Strata simplify analysis but Strata are rigid

I “Beneficiary group” may not align well with strata

I Difficult to fine-tune

I Number of strata is exponential in number of features

Instead we use models...

22 / 53

Example Biomarker Model

Something like

logit [pT (x)] = βT (0) +∑j

βT (j)xj

logit [pC (x)] = βC(0) +∑j

βC(j)xj

Make decisions based on

pT (x)− pC (x)

23 / 53

The Plan

Enroll patients in blocks (k = 1, . . . ,K )

Use information from past blocks to develop/updatebiomarker

Use current biomarker to restrict enrollment in current block

Randomize new patients to arms (fixed weights)

At the end of the trial combine all the information to test null

24 / 53

Frasian Statistics

1. Building biomarker/determining enrollment criteria Bayesian

2. Testing the null in a principled, model-free way Frequentist

25 / 53

Testing the Null

Let’s begin at the end:

Assume

I we have a method for building/updating a biomarker

I and we can leverage this to restrict enrollment

How do we test the null?

pT (x) ≤ pC (x) for all x

26 / 53

Testing the Null

Two designs which “obviously” work...

In reality only one controls type 1 error:

1. Permutation Test (within block)

2. Agglomerated z-test

27 / 53

Permutation Test

Use the statistic

T = (# Treatment successes)− (# Control successes)

For null distribution of T , permute labels within strata.

28 / 53

Agglomerated z-test

Use the statistic

T =∑

Tk

where

Tk =

√nkn

(p(T ,k) − p(C ,k)

2√

ppool ,k (1− ppool ,k) /nk

)(these are just weighted within block z-stats).

For null distribution of T , compare to N(0, 1)

29 / 53

Testing the Null

Two designs which “obviously” work...

Which one controls type 1 error?

1. Permutation Test (within block) 7

2. Agglomerated z-test 3

30 / 53

Failure of the Permutation Test

What goes wrong if we permute?

The prognosis of future patients contains information about theoutcome of previous blocks.

Thus, conditioning on the outcomes does not induce thepermutation distribution under the null.

31 / 53

Failure of the Permutation Test

Consider a simple 2 block design:

Check if there is significance after the first block

I Yes? → enroll extremely poor (or good) prognosis group (lowvariability).

I No? → enroll only patients in middle (high variability)

Can easily see type 1 error in excess of 0.2

32 / 53

Now for Part 1

How do we

I Construct/Update a Biomarker?

I Make Decisions about Enrollment?

I Decide treatment indication?

I Estimate treatment effect?

33 / 53

Bayesian Modeling!

As a frequentist, those problems are hard.

Uncertainty in biomarker estimates needs to take sampling intoaccount

A Bayesian model (with a prior we believe) makes much of thisdisappear

We need to do this well enough that scientific inefficienciesdominate statistical inefficiencies.

34 / 53

Decisions, Decisions, Decisions

How do we

Construct/Update a Biomarker?

35 / 53

Constructing/Updating Biomarkers

Simple Version:

Assume

logit [pT (x)] = βT (0) +∑j

βT (j)xj

logit [pC (x)] = βC(0) +∑j

βC(j)xj

with a prior distribution Π on the βs

Update by considering Π (pT (x), pC (x)|past data)

Π (pT (x), pC (x)|past data) can ignore decision rule

36 / 53


How do we

Construct/Update a Biomarker?Condition on previous data

Make Decisions about Enrollment?

37 / 53

Making Enrollment Decisions

Need 2 things

I A utility function U(·)

I and a decision rule D which maximizes ED [U]

38 / 53

Making Enrollment Decisions

What’s a good utility function?

Must combine power, and sensitivity:

One choice we like: expected future patient outcome

U (trial) =

∫X max E [pT (x)|trial] ,E [pC (x)|trial] dG (x) : reject∫X E [pC (x)|trial] dG (x) : reject

39 / 53

Maximizing in Practice

Choosing the optimal rule is hard...

Instead we restrict to a simpler class: rules based on

Π (pT (x) > pC (x)|past data)

Still requires a lot of computational power; who do we turn to?

Very easy and cheap! (using MIT tool starcluster)

costs ∼ 1$/hour for 120 cores (if you are careful)40 / 53


How do we


Make Decisions about Enrollment?Maximize a Utility Function

Decide treatment indication?

41 / 53

Deciding Treatment Indication

Who do we expect to benefit from treatment?

Treat those patients...

Ω = x st E [pT (x)|trial] > E [pC (x)|trial]

Can also take toxicity into account.

Could also use optimal rule from simpler class (eg. best linear rule)

42 / 53


How do we



Decide treatment indication?Subset with expected posterior benefit

Estimate treatment effect?

43 / 53

Estimating Treatment Effect

Given Ω; use expected posterior benefit∫Ω

(E [pT (x)|trial]− E [pC (x)|trial]) dG (x)

No selection bias (Assuming prior we trust)

Can substitute different (more conservative) prior here

44 / 53


How do we



Decide treatment indication?Subset with expected posterior benefit

Estimate treatment effect?Expected posterior benefit

45 / 53

Putting it all Together!

1. Choose a utility; and priors for pT , pC

2. Find an enrollment strategy to “optimize” this utility

3. Enroll patients in blocks according to:

I enrollment strategy using...

features, treatment labels and outcomes of previous blocks

4. At trial termination

I Test using blocked t-test

I Decide treatment indication based on posterior

I Estimate effect-size based on posterior

46 / 53

Bayesian Vs Frequentist?

Biomarker Construction Bayesian

Enrichment Bayesian

Early Termination Bayesian

Testing the Global Null Frequentist

Estimating the Effect-size Bayesian

47 / 53

Redesign of Bokemeyer et al.

Study of chemo vs chemo + targeted treatment (cetuximab)

Cetuximab targets EGFR

I In original trial only EGFR expression was considered

I The original trial did not find significance at 0.05 level.

I Later, tissue samples were re-assayed for KRAS mutation

I In KRAS wildtype-only samples, treatment effect was highlysignificant.

48 / 53


I 337 patients equally randomized

I EGFR expression was a 3-level factor.

I Simulated trials: EGFR/KRAS and ORR based on empiricalrates from study.

I For adaptive enrichment, used 2 and 3 block designs withseparate working models

PT (ORR = 1|EGFR,KRAS) = βT0 + βT1EGFR + βT2KRAS

+ βT3(KRAS ∗ EGFR)

49 / 53


Results averaged over 1000 simulated trials:

Non-Adapt 2 block 3 block

power 0.23 0.68 0.78true effect-size 0.06 0.21 0.21estimated effect-size 0.13 0.25 0.24

Effect-sizes were only calculated/estimated for successful trials.

Many more tables with discussion in the paper

50 / 53

Take-aways

We can conduct a clinical trial

which controls type 1 error under the global null of no benefitfor anyone

in which we concurrently build a biomarker to determine thoselikely to benefit

and apply the biomarker to enrich our patient population (andincrease our power)

In addition, this biomarker is naturally constructed using aBayesian framework.

51 / 53

The Papers

N. Simon, R. Simon, Adaptive Enrichment Designs for ClinicalTrials, Biostatistics 2013.

N. Simon, R. Simon, Using Bayesian Modeling in FrequentistAdaptive Enrichment Designs, Biostatistics 2017.

Cetuximab Study:

Bokemeyer and others, Fluorouracil, leucovorin, and oxaliplatinwith and without cetuximab in the first-line treatment of metasticcolorectal cancer, Journal of Clinical Oncology, 2009.

52 / 53

A Fireside Chat about Precision Medicine 3mm and a ... · Strati cation Free Adaptive Enrichment...

Documents

Transcript of A Fireside Chat about Precision Medicine 3mm and a ... · Strati cation Free Adaptive Enrichment...