Futility stopping

22
1 Carl-Fredrik Burman, 11 Nov 2008 RSS / MRC / NIHR HTA Futility Meeting Futility stopping Carl-Fredrik Burman, PhD Statistical Science Director AstraZeneca R&D

description

Futility stopping. Carl-Fredrik Burman, PhD Statistical Science Director AstraZeneca R&D. Stakeholder perspectives. The patient A pharmaceutical company The public (MRC, NIHR). The fundamental design requirement: Ethics. ”My old mother – principle” The trial is ethical if (and only if) - PowerPoint PPT Presentation

Transcript of Futility stopping

Page 1: Futility stopping

1 Carl-Fredrik Burman, 11 Nov 2008RSS / MRC / NIHR HTA Futility Meeting

Futility stopping

Carl-Fredrik Burman, PhD

Statistical Science Director

AstraZeneca R&D

Page 2: Futility stopping

Carl-Fredrik Burman, 11 Nov 2008RSS / MRC / NIHR HTA Futility Meeting

2

Stakeholder perspectives

The patient

A pharmaceutical company

The public (MRC, NIHR)

Page 3: Futility stopping

Carl-Fredrik Burman, 11 Nov 2008RSS / MRC / NIHR HTA Futility Meeting

3

The fundamental design requirement:Ethics

”My old mother – principle”The trial is ethical if (and only if)I would recommend my mother

to take part in the trial,given that she would be eligible.

Page 4: Futility stopping

Carl-Fredrik Burman, 11 Nov 2008RSS / MRC / NIHR HTA Futility Meeting

4

Interim stopping

Stop the trial as soon as I would not include my mother, e.g. if

One (publicly available) treatment is clearly better

A “new” treatment fails to show sufficient effect, when it has known safety disadvantages

No ethical obligation to stop If two treatments with similar safety have no clear difference in

effect

Page 5: Futility stopping

Carl-Fredrik Burman, 11 Nov 2008RSS / MRC / NIHR HTA Futility Meeting

5

(Genuine) informed consent

The patient should get Full information regarding the trial treatments (and procedures),

including previous data, potential risks, etc.

Help to understand the information and

Apply it to his/her specific situation (health status, preferences)

When would a fully informed, fully competent patient give consent?

If and only if it is better (not worse) for him/her to take part in the trial, as compared to receiving standard therapy.

Cf. “my old mother” principle

Page 6: Futility stopping

Carl-Fredrik Burman, 11 Nov 2008RSS / MRC / NIHR HTA Futility Meeting

6

Easy-going clinical equipose is not enough Clinical equipose

If there is uncertainty about which treatment is better

(Alternatively, compelling evidence of one treatment being better)

(Alternatively, medical experts disagree)

It’s far too easy to say that we are uncertain

I expect my doctor to say what he believes is best

Page 7: Futility stopping

7 RSS / MRC / NIHR HTA Futility Meeting

Carl-Fredrik Burman, 11 Nov 2008

Our old Mother

Scientific equipose Not every expert agree on

CO2-induced global warming

Do you suggest a randomised N-of-1 trial?

Of course not — choose the treatment we believe is best

Earth

Page 8: Futility stopping

Carl-Fredrik Burman, 11 Nov 2008RSS / MRC / NIHR HTA Futility Meeting

8

What is ”best” for the patient?May depend on e.g. Effect (best guess + uncertainty)

Safety

Better care in the trial?

Economic compensation (but beware of exploitation)

Altruism

Likely effect will differ between individuals (covariates)

Preferences are different

Decision theory may help decide (at least in theory …)

Page 9: Futility stopping

Carl-Fredrik Burman, 11 Nov 2008RSS / MRC / NIHR HTA Futility Meeting

9

Decision analysis (DA)

Patient perspective

Utility function U(effect, safety, QoL, cost, …)

Model for effect, safety, etc., based on best information (data, expert knowledge, …). Often Bayesian prior.

Choose decision (volunteer to participate in trial, or not) to maximise expected utility

The DA approach can also be used by a trial sponsor

Page 10: Futility stopping

Carl-Fredrik Burman, 11 Nov 2008RSS / MRC / NIHR HTA Futility Meeting10

A pharmaceutical company perspective(simplified) A new drug will be licensed if and only if the (next) phase

III trial has a statistically significant effect (p<5%) If licensed, the company will make a profit of V (unit: £) The trial cost is k·N, where N is the sample size The assumed (believed) treatment effect is . Maximise

V · Power(N) – k · N

Of course, this model is wrong (as all models are).

Should e.g. have V=V(T)=V(T(N)), where T is time.

Page 11: Futility stopping

11 RSS / MRC / NIHR HTA Futility Meeting

Carl-Fredrik Burman, 11 Nov 2008

0

20

40

60

80

100

0 500 1000 1500 2000

Number of patients

mUSD

Gain

Net gain = Gain – Cost

Cost

Optimal sample size

Nopt = 1010

Page 12: Futility stopping

Carl-Fredrik Burman, 11 Nov 2008RSS / MRC / NIHR HTA Futility Meeting12

The interim decision(continue vs. stop for futility) Value V if significant

Conditional power CP if trial is continued

C additional trial cost if continued (compared to if stopped)

Continue iff V · CP > C,

that is, iff CP > C / V

Page 13: Futility stopping

Carl-Fredrik Burman, 11 Nov 2008RSS / MRC / NIHR HTA Futility Meeting13

DA vs. ”least clinically relevant” effect

DA approach: Maximise expected utility based on ”best guess” effect (or prior)

Traditional approach: 90% power at ”least clinically relevant” effect

What is the least clinically relevant effect? If no adverse effects, no cost And the outcome is death

One single saved life is clinically relevant … at least to the one saved

What is a relevant effect depends on safety, cost etc.

Page 14: Futility stopping

Carl-Fredrik Burman, 11 Nov 2008RSS / MRC / NIHR HTA Futility Meeting14

Conditional power at interim

Final estimate is N(, 1/N).

Stage i has sample size Ni and estimate i. Then = (N1·1+N2·2)/N

Statistical significance if > C / N (where C=1.96 say)

CP = P( > C / N ) = ( ·N2+ 1·N1/N2-C (N/N2) )

But which to use when calculating CP? Original alternative Alternative ?

Interim estimate 1 ?

Linear combination of 1 and Alternative ?

Bayesian posterior based on interim data ?

Page 15: Futility stopping

Carl-Fredrik Burman, 11 Nov 2008RSS / MRC / NIHR HTA Futility Meeting15

Stop, continue, or something else?

Run a new trial?

Sample size reestimation, based on interim estimates Flexible design methodology (Bauer & Köhne –94)

Predefined weights for the different stages (generally, weight not proportional to information)

May change the sample size for stage 2 after viewing interim results

Discussion on CP

Somewhat controversial

May be better than design with only futility stopping

Group-sequential designs should often be preferred

Page 16: Futility stopping

Carl-Fredrik Burman, 11 Nov 2008RSS / MRC / NIHR HTA Futility Meeting16

Publicly funded trial:Treatments with similar safety Assume

Whole patient population will receive one of these treatments

Efficacy is the only unknown

Same safety, cost, etc.

The closer the interim effect is to zero, the more value in continuing

Thus, no reason to stop for futility

Page 17: Futility stopping

Carl-Fredrik Burman, 11 Nov 2008RSS / MRC / NIHR HTA Futility Meeting17

Example 1: Value of information

Compare 2 treatments with probabilities pA, pB for death.

Assume total future population size is T (10,000 say)

If we knew that , we would choose treatment A T·lives would be spared as compared to using B

Similarily, choose B if <0

Net value T·Abs()

or T·Abs()/2 if compared to using random treatment

Page 18: Futility stopping

Carl-Fredrik Burman, 11 Nov 2008RSS / MRC / NIHR HTA Futility Meeting18

Maximal value of information

Before trial, p2p1 has approximately normal prior with mean=0, SD= (say 10%)

What would the value be if we could learn the exact value of ?

Take the Bayesian expectation of the value T·Abs()/2,

Eprior [T·Abs()/2] = T· / (2)

With T=10,000 and =10%, about 400 lives would be spared

Example cont’d

Page 19: Futility stopping

Carl-Fredrik Burman, 11 Nov 2008RSS / MRC / NIHR HTA Futility Meeting19

0

50

100

150

200

-0.2 -0.15 -0.1 -0.05 0 0.05 0.1 0.15 0.2Prior mean

Sa

ved

live

s

Page 20: Futility stopping

Carl-Fredrik Burman, 11 Nov 2008RSS / MRC / NIHR HTA Futility Meeting20

Publicly funded trial:Intervention vs. no treatment (placebo) Assume

Intervention is associated with some cost, safety risks

Not clear whether intervention has a positive effect

If effect, then the size of the effect will determine the size of the patient population which will get a positive net benefit

First objective: is there any effect?

Reasonable to stop for futility if interim estimate is low

Expected value by continuing study is then small

Page 21: Futility stopping

Carl-Fredrik Burman, 11 Nov 2008RSS / MRC / NIHR HTA Futility Meeting21

Information leakage

In regulatory setting, large discussion on who should see interim data

Does the DMC have to be independent from the sponsor

What are the risks of potential information leakage?

Problems may be over-emphasised?

The ethical aspect

Page 22: Futility stopping

Carl-Fredrik Burman, 11 Nov 2008RSS / MRC / NIHR HTA Futility Meeting22

Summary

Futility stopping may be an ethical requirement

Industry funded trials: Tradeoff cost and expected value

Publicly funded trials (examples) Don’t stop for futility if two active treatments differ only in effect

May stop for futility if “active” treatment unlikely to have sufficient effect (tradeoff cost and value)

(If basic science objective …)