1

Issues in Selection of Deltas in Non-Inferiority Trials :

Acute Bacterial Meningitis and Hospital-Acquired Pneumonia

John H. Powers, M.D.

Medical Officer

Division of Special Pathogen and Immunologic Drug Products

Center for Drug Evaluation and Research

U.S. Food and Drug Administration

2

Introduction Clinical perspective on delta

– definition of delta and components– impact of deltas in clinical setting

Delta 1 issues in acute bacterial meningitis and HAP– data from pre-antibiotic and antibiotic eras– confounders in determining efficacy of control regimens

Delta 2 issues with acute bacterial meningitis and HAP– consequences of less effective therapy– practical issues in selecting delta

3

Clinical Trials Purpose of clinical trials

– Distinguish effects of drug from other influences

» spontaneous change in course of disease

» placebo effect

» biased observations

– difficult for clinicians to make judgments on drug efficacy/safety outside of setting of clinical trial

» high spontaneous resolution rate in less serious diseases» confounding factors for lack of patient improvement in serious diseases» lack of direct comparison of safety of two drugs in similar patient

population

4

Non-Inferiority Trials Non-inferiority trials attempt to prove test drug is not

inferior to control drug by some margin» cannot statistically prove two drugs are identical in

efficacy» need some way to estimate the variability around the

difference between two treatments

Non-inferiority margin (delta) = maximum degree of inferiority of test drug compared to control drug that trial will attempt to exclude statistically

» specified prior to initiation of trial

5

Non-Inferiority Margins After completion of trial:

1) calculate difference in point estimates of efficacy

of test agent minus control agent2) calculate 95% confidence interval around difference in point

estimates

» gives some idea of variability around the estimate in the differences

3) compare lower bound of 95% CI to pre-specified non-inferiority margin

-20% +20%-8%-15%

6

Components of Delta Delta 1

– conservative estimate of advantage of active control over placebo

» data-based

Delta 2– largest clinically acceptable difference between active control

and experimental drug» judgement based on consequences to patients of treatment

failure

overall delta for clinical trial smaller of the two values– if delta 1 is large, overall delta set by delta 2

7

Components of Delta - Delta 1 Historically-based data

– Do we really know what we think we know?» lack of data from pre-antibiotic era» change in resistance patterns and epidemiology of

organisms» differing response rates in sub- populations» changes in practice of medicine» problems with defining patients with bacterial

infection vs. non-bacterial/non-infectious causes» different definitions of success and failure in current

trials compared to previous mortality-based trials

8

Components of Delta - Delta 2 Judgement based “acceptable loss” relative to current therapy

– ideal situation» smaller delta for more severe disease

less loss relative to current therapy given potential for greater overall morbidity/mortality

» larger delta for less severe disease greater loss relative to current therapy may not translate into as

great a consequence for patients

– BUT we don’t live in an ideal world» practicalities of performing clinical trials

9

Components of Delta - various diseases

Acute bacterial meningitis– 1 = magnitude of advantage over placebo well-known

AND large– 2 = decision on “acceptable loss”

Hospital-acquired pneumonia– 1 = magnitude of advantage over placebo not as clear– 2 = decision on “acceptable loss”

Acute exacerbations of chronic bronchitis– 1 = advantage over placebo unclear (and small?)– 2 = decision on acceptable loss not as critical

10

Components of DeltaMeningitis and HAP

Delta 1 - important questions

Q1: What is the magnitude of benefit of any antibiotic therapy over placebo?

Q2: Is the benefit of antimicrobial therapy in current trials measured in the same way as in the original trials showing benefit?

Q3: Is the magnitude of benefit of therapy over placebo large enough that it should not affect the selection of the overall delta for a trial?

11

Components of DeltaMeningitis and HAP

Delta 2

Q: What is an “acceptable loss” of efficacy compared to accepted therapy in a serious disease?

– Scientific considerations» consequences of treatment failure in various patient

subsets with meningitis or HAP

– Practical considerations» effect of changes in delta on sample size as efficacy

rate changes

12

Historical Data - Meningitis Acute bacterial meningitis highly lethal in pre-antibiotic

era– meningococcal disease most common and occurred in

previously healthy young people– overall mortality 70-90% without specific therapy– mortality decreased to 30% with introduction of

antimeningococcal serum Flexner S. J Exp Med 1913;17:553-76

– sulfanilamide treatment reduced mortality to 10% (9/11 patients survived in original series)

Schwenker F et al. JAMA 1937;108:1407-8

13

Historical Data - Meningitis Problems with historical data

– different endpoints in current trials» developmental, neurologic, audiologic sequelae as

well as mortality– different epidemiology

» pneumococcal meningitis most common now in U.S. – different populations

» proportionately more older adults with meningitis since introduction of HIB vaccine

Schuchat A et al. N Engl J Med 1997;337:970-6.

14

Historical Data - HAP

Clinical entity of HAP not described in pre-antibiotic era

– only 2 spontaneous cures out of 151 cases in military

recruits in S. aureus outbreaks in 1918

– few reports of gram-negative pneumonias

» How certain is diagnosis in these case reports?

No way to compare antibiotic therapy to placebo

15

Celis R. Chest 93;318-24.1988

– 30.5% (33/108) all-cause mortality with “appropriate” antibiotics

– 91.6% (11/12) all-cause mortality with“inappropriate” antibiotics

Alvarez-Lerma et al. Intensive Care Med 1996;22:387-94.

– 16.2% (36 /146)attributable mortality with “appropriate” antibiotics

» all-cause mortality 34.9% (51/146)

– 24.7% ( 46/284) attributable mortality with “inappropriate” antibiotics

» all-cause mortality 32.4% (92/284)

Historical Data - HAP

16

Historical Data - HAP Problems with historical data

– Difficulty in clinical diagnosis of HAP» patients in study who do not have disease

– Change in nosocomial organisms over time» changes in resistance patterns

– Different outcomes in various patient populations» mechanically ventilated pts. Vs. others

– Death attributable to pneumonia vs. all-cause mortality– Clinical endpoints other than mortality in current trials

17

Components of DeltaMeningitis

Delta 1 - important questions

Q1: What is the magnitude of benefit of any antibiotic therapy over placebo?

Appears as large as 60%-80% mortality benefit but magnitude of benefit on clinical parameters not as clear

Q2: Is the benefit of antimicrobial therapy in current trials measured in the same way as in the original trials showing benefit?

Yes and No

Q3: Is the magnitude of benefit of therapy over placebo large enough that it should not affect the selection of the overall delta for a trial?

Yes

18

Components of DeltaHAP

Delta 1 - important questions

Q1: What is the magnitude of benefit of any antibiotic therapy over placebo?

May be anywhere from 8.5%-60% depending on how and in whom it is measured. Unclear benefit on clinical parameters

Q2: Is the benefit of antimicrobial therapy in current trials measured in the same way as in the original trials showing benefit?

Yes and No

Q3: Is the magnitude of benefit of therapy over placebo large enough that it should not affect the selection of the overall delta for a trial?

Point for committee discussion

19

Components of DeltaMeningitis and HAP

Delta 2

Q: What is an “acceptable loss” of efficacy compared to accepted therapy in a serious disease?

– Scientific considerations» consequences of treatment failure in various patient

subsets with HAP

– Practical considerations» effect of changes in delta on sample size as efficacy

rate changes

20

Consequences of Failure Meningitis

– clear mortality benefit of antibiotic therapy– morbidity is developmental, neurological and audiological

sequelae» what is magnitude of benefit of antibiotics?

HAP– mortality

» magnitude of benefit varies depending on how and in whom it is measured

– morbidity» increased costs and hospital stay» effect on rate of clinical resolution?

21

Practical Issues Effect of success rate and delta selection on sample size

– Selection of a smaller delta in more severe diseases with relatively lower success rates would increase sample size

– Is larger sample size practical given:

1) epidemiology of the disease

2) limitations of inclusion and exclusion criteria

3) inability to continue on randomized therapy in studies of severe disease

22

Clinical Trial Implications:Sample size per arm to achieve 80% power

1570

393

17599

1507

377

16895

1319

330

14783

1005

252

11263

0

200

400

600

800

1000

1200

1400

1600

Sam

ple

Siz

e

50 60 70 80

Success Rate (%)

5%10%15%20%

23

Epidemiology of Meningitis*

OrganismIncidence per

100,000Case fatality rate

(%)

S. pneumoniae 1.1 21

N. meningitidis 0.6 3

Group Bstreptococcus

0.3 7

H. influenzae 0.2 6

Listeria 0.2 15

*Based on 248 cases in 1995 from Schuchat et al. N Engl J Med. 1997;337:970-6.

24

Epidemiology of Meningitis Case fatality rates and incidence vary by organism

– H. influenzae lower case fatality rates than S. disease caused by S. pneumoniae

– S. pneumoniae now more common overall

– mortality rates in future trials may be higher than those in past given shift in epidemiology

Number of cases in U.S. declining since introduction of HIB vaccine

– estimated 12,920 cases in 1986

– estimated 5,755 cases in 1995» Schuchat et al. N Engl J Med 1997;337:970-76.

25

Epidemiology of HAP Actual incidence of HAP unclear (not a reportable illness)

– NNIS data estimates 250,000 cases/year in U.S.» uses clinical definition of HAP

– estimated 1% of all patients entering hospital develop pneumonia

– 15-18% of all hospital acquired infections» 2nd most common after UTI » most common infection in ICU setting

ICARE report. Am J Infect Control 1999;27:279-84.

26

Epidemiology Estimated U.S. cases per year (1994)

– acute otitis media 26,000,000

– acute sinusitis 23,000,000

– tonsillitis/pharyngitis 21,000,00

– pneumonia (community) 4,000,000

– hospital-acquired pneumonia 250,000

– acute bacterial meningitis <10,000

– acute bacterial endocarditis 10,000

27

Recent Trials Practical Points

– success rates in HAP trials in 50% - 70% range

» much larger sample size with smaller delta

– recent approvals with 20% delta based on 1992 guidance in all recent HAP trials

» theoretically a new drug could be as much as 20% less effective than comparator

– almost half of patients do not complete trial

» must take into account when planning sample size

28

Clinical Trial Implications:Sample size per arm to achieve 80% power

1570

393

17599

1507

377

168

95

1319

330

147

83

1005

252

11263

0

200

400

600

800

1000

1200

1400

1600

Sa

mp

le S

ize

50 60 70 80

Success Rate (%)

5%

10%

15%

20%

29

Components of DeltaMeningitis and HAP

Delta 2

Q: What is an “acceptable loss” of efficacy compared to accepted therapy in a serious disease?

» serious nature of meningitis and HAP would seem to call for selection of smaller deltas

» smaller deltas would result in larger sample size of clinical trials - is this practical?

» balance with risk of accepting drugs which may be 20% less effective than currently approved therapy

could be success rate of 40% for new drug for HAP

30

The Balance

Risk to patients of accepting larger deltas, especially in more severe disease

versus

Realities of performing clinical trials