Some Principles of Evidence-based Medicine

William J. Powers, M.D.

Financial Disclosures for William J. Powers, M.D.

Salary and Research Support: National Institutes of HealthWashington University School of

Medicine Barnes-Jewish Hospital

No personal financial relationships with any pharmaceutical or medical device companies including consulting, lecture fees, honoraria, gifts, licensing revenues, equity interests or other compensation

Users' Guides to the Medical Literature: XXV. Evidence-Based Medicine: Principles for Applying

the Users' Guides to Patient Care.

Guyatt, et al: JAMA 2000 284:1290-1296Guyatt, et al: JAMA 2000 284:1290-1296

Hierarchy of Clinical Evidence

In the N of 1 RCT, Patients undertake pairs of treatment periods in which they receive a target treatment in 1 period and a placebo or alternative in the other. Patients and clinicians are blind to allocation, The order of the target and control are randomizedPatients make quantitative ratings of their symptoms during each period N of 1 RCTs can provide definitive evidence of treatment effectiveness in individual patients, and may lead to long-term differences in treatment administration.

N of 1 RCTs are unsuitable for short-term problems; therapies that cure therapies that act over long periods of time or prevent rare or unique events (such as stroke, myocardial infarction, or death)

N of 1 RCTN of 1 RCT

Guyatt, et al: JAMA 2000 284:1290-1296Guyatt, et al: JAMA 2000 284:1290-1296

When considering any source of evidence about treatment other than N of 1 RCTs, clinicians are generalizing from results in other people to their patients, inevitably weakening inferences about treatment impact and introducing complex issues of how trial results apply to individuals.

Inferences may nevertheless be strong if results come from a systematic review of methodologically strong RCTs with consistent results and are generally somewhat weaker if we are dealing with only a single RCT unless it is large and has enrolled a diverse patient population.

Users' Guides to the Medical Literature: XXV. Evidence-Based Medicine: Principles for Applying

the Users' Guides to Patient Care.

Guyatt, et al: JAMA 2000 284:1290-1296Guyatt, et al: JAMA 2000 284:1290-1296

Meta-analysis

• Identification of Trials• Definition of Outcomes• 2 x 2 outcome tables summed over

all trials• Calculation of treatment effects• Statistical significance

An

nu

alize

d S

troke R

ate

900-1300 mg/day ASA 150-300 mg/day DPM

p=NS

Secondary Prevention of Stroke with High Dose Aspirin and Dipyridamole

Meta-analysis of 1574 patients from Three Trials*

Secondary Prevention of Stroke with High Dose Aspirin and Dipyridamole

Meta-analysis of 1574 patients from Three Trials*

*Rev Neurol 1082; 138:367-385 Stroke 1983; 14:5-14 Stroke 1985; 16:406-415

European Stroke Prevention Study 2Low Dose Aspirin plus Extended Release Dipyridamole

(J Neurol Sci 1996; 143:1-13)

01

2

3

4

5

67

8

9

10 Placebo

ASA

DPM

ASA+DPM

An

nu

alized

Str

oke R

ate

* ***

* Better than placebo

** Better than other 3 groups

ASA 50 mg/dayDPM 400ER mg/day

Secondary Prevention of Stroke with Aspirin and Dipyridamole

Meta-analysis of 4873 patients from Four Trials with Any Dose of Aspirin

Diener HC: Neurology1998; 51(Suppl 3):S17-S19

Secondary Prevention of Vascular Events in High Risk Patients

Aspirin vs. Dipyridamole and Aspirin

0

5

10

15

20

25

Aspirin

ASA+DPM

Perc

en

t of

Vascu

lar

Even

ts

p=NS

Meta-analysis of 25 Trials

Users' Guides to the Medical Literature: II. How to Use an Article About Therapy or Prevention:

A. Are the Results of the Study Valid?

Guyatt, et al: JAMA 1993; 270:2598-2601

Why is the intent-to-treat principle so important?

• It mimics the real life decisionAt diagnosis, you have to decide what treatment to try

• It accounts for sicker patients going off protocolBenefits can be inflated by non-responders dropping out

• It accounts for side effectsTreatment A for cancer yields a 90% 5-year survival but

only 10% survive treatmentTreatment B yields a 75% survival and all survive treatment

Measuring the Effects of Therapy

Guyatt, et al: JAMA.1994; 271:59-63

Comparison of Aspirin with other Anti-platelet Agents for the Secondary Prevention of Stroke:

Absolute Risks & Risk ReductionA

nn

ualiz

ed

Str

oke

Rate

(%

)

25

20

15

10

5

0ESPS2 TASS

25

20

15

10

5

0ESPS2 TASS

Aspirin Agent Relative Risk Reduction

Absolute Risk Reduction

Comparison of Aspirin with other Anti-platelet Agents for the Secondary Prevention of

Stroke: Prognosis

0

10

2030

40

50

6070

80

90

100

ESPS2 TASS

AspirinAgent

% W

ith

ou

t S

troke

At

On

e Y

ear

The solid line represents the confidence interval around the first example in which there were 100 patients per group and the number of events in the active and control groups were two and four, respectively. The broken line represents the confidence interval around the second example in which there were 1000 patients per group and the number of events in the active and control groups were 20 and 40, respectively.

From:   Guyatt: JAMA 1994: 271:59-63

Confidence Intervals for Risk Reduction

Warfarin-Aspirin Recurrent Stroke Study(WARSS)

(Mohr et al: NEJM 2001; 345:1444-1451)

PRIMARY RESULTSAlthough this trial demonstrated no

significant difference between warfarin and aspirin, it did not establish

equivalence Hazard ratio 1.13 95% CI 0.92-1.38

This means that there is less than a 5% chance

that warfarin is more than 8% better than aspirin

HAESTLancet 2000; 355:1205-1210

HAESTLancet 2000; 355:1205-1210

Recurrent ischemic stroke within 14 days in 449 patients with acute (< 30 hrs) ischemic stroke who had atrial fibrillation within 2 years (89%

present on admission)

Dalteparin 19/224 (8.5%)Aspirin 17/225 (7.5%)

p=.73

OR(95% CI) = 1.13 (.57 to 2.24)

Number Needed to Treat

The number needed to treat is 1 divided by the absolute

risk reduction

Number Needed to Treat to Prevent 1 Stroke at 2 Years After Carotid Endarterectomy

Gorelick PB: Stroke. 1999; 30:1745-50.

Study

% Stenosi

sNNT

Symptomatic    NASCET 70–99 8ECST 70–99 8NASCET 50–69 20NASCET <50 67

Asymptomatic   

Veterans Affairs 48ACAS 83

Economic Analysis

• Cost-effectiveness analysis - health outcomes are not valued, but reported in physical units such as life years gained or cases successfully treated.

• Cost-utility analysis - outcomes of different types are weighted to produce a composite index, such as the quality-adjusted life year (QALY) or healthy years equivalent.

• Cost-benefit analyses - the health consequences are valued by asking health care consumers what they would be willing to pay for health services that achieve combinations of outcomes of particular types.

Drummond, et al:. JAMA 1997; 277:1552-1557Drummond, et al:. JAMA 1997; 277:1552-1557

Cost Utility of PET Selection for EC/IC Bypass Surgery for Symptomatic Carotid Occlusion

Derdeyn et al, J Nucl Med 2000; 41:800-807

If the Carotid Occlusion Surgery Study hypothesis is proven true, EC/IC bypass surgery will improve outcome and reduce cost:

49 QALY/10 yrs/ 100 patients cost savings of $11,000 per patient

The Important Issue of the Primary Endpoint

The Primary Endpoint should be an event that is clinically important to

patients such as death, stroke or myocardial infarction

The Important Issue of the Primary Endpoint

• The choice of the Primary Endpoint will affect the size and duration of the study• Sample size is determined by the number of

events in the control group and the projected treatment effect

• The choice of the Primary Endpoint will affect the outcome of the study • All components of the primary endpoint are

weighted equally• Serious Adverse Events NOT included in Primary

Endpoint are not included in the primary analysis

Comparison of Stenting with Minimally Invasive Bypass Surgery for Stenosis of the Left Anterior Descending

Coronary Artery NEJM 2002; 347:561-566

Stenting Group

(N=108)

Surgical Group (N=106) P

Cardiac Death 0 2 .99

Acute MI 3 5 .68

Revascularization 31 9 .003

Primary Endpoint (All)

34 16 .02

Secondary Prevention of Stroke with Aspirin and Dipyridamole ESPS2

0

5

10

15

20

2 y

ear

Str

oke R

ate

(%

)

Bleeding

Tissue Plasminogen Activator for Acute Ischemic Stroke (NINDS)

NEJM 1995; 333:1581-1587; Stroke 1997; 28:2109-2118

Outcome at 3 Months in 624 Patients

Tissue Plasminogen Activator for Acute Ischemic Stroke (NINDS)

NEJM 1995; 333:1581-1587; Stroke 1997; 28:2109-2118

Outcome at 3 Months in 624 Patients

0102030405060708090100

rt-PA Placebo rt-PA Placebo rt-PA Placebo

p < .001

GoodRankin 0-1

PoorRankin 2- 5

Dead

SymptomaticHemorrhage

Observational studies may provide compelling evidence if untreated outcomes are consistent and treatment effects are sufficiently large and consistent. For instance , observational studies have allowed extremely strong inferences about the efficacy of insulin in diabetic ketoacidosis or penicillin for pneumococcal meningitis, both of which are almost uniformly fatal if untreated.

However, when outcome is variable and, especially, when the poor outcome occurs in only a small proportion of patients, evidence derived only from non-randomized observational studies should be regarded with extreme caution.

Observational Studies

Divergent Data on Post-Menopausal Hormone TherapyNEJM 2003; 348:645-650

Users' Guides to the Medical Literature: XXV. Evidence-Based Medicine: Principles for Applying

the Users' Guides to Patient Care.

Users' Guides to the Medical Literature: XXV. Evidence-Based Medicine: Principles for Applying

the Users' Guides to Patient Care.

Guyatt, et al: JAMA 2000; 284:1290-1296

Physiologic studies and unsystematic clinical observations provide the weakest inferences about treatment effects.

Problems in the "Evidence" of

"Evidence-Based Medicine" Feinstein AR & Horwitz RI: Am J Med 1997;103:529-535

• The data do not include many types of treatments or patients seen in clinical practice.

• The results show comparative efficacy of treatment for an "average" randomized patient, not for pertinent subgroups formed by such cogent clinical features as severity of symptoms, illness, co-morbidity, and other clinical nuances.

The laudable goal of making clinical decisions based on evidence can be impaired by the restricted quality and scope of the "best available evidence." The authoritative aura, however, may lead to major abuses that produce inappropriate guidelines or doctrinaire dogmas for clinical practice.

0

5

10

15

Primary Prevention Secondary Prevention

Aspirin

OAC

% P

rim

ary

End

poin

tp

er

year

Extrapolating from Primary Prevention to Secondary Prevention

Oral Anticoagulation vs. Aspirin in Atrial Fibrillation

AFASAK-1 Lancet 1989; 1:175-178; European Atrial Fibrillation Trial Lancet 1993; 342: 1255-1262

Extrapolating from Primary Prevention to Secondary PreventionAspirin for Primary Stroke Prevention

Non-fatal Stroke per 10,000 subject-years

Aspirin Control Odds Ratio

British1 35.6 31.7 1.12

US2 20.1 16.8 1.20

1 BMJ 1988; 296:313-3162 NEJM 1989; 321:129-135

Extrapolating from Primary Prevention to Secondary Prevention

Antiplatelet Therapy for Secondary Stroke Prevention1

Non-fatal Stroke

Antiplatelet Control Odds Ratio

8.3% 10.8% .77*

* p< .0001

1 BMJ 2002; 324:71-86

The Problem of Subgroup AnalysisTOAST Primary Analysis

JAMA 1998;279:1265

• 1281 patients within 24 hrs of ischemic stroke

• Dose-adjusted danaparoid IV vs. placebo x 7d

• Primary endpoint: favorable functional outcome (GOS I or II and BI 12-20) at 3 months

Favorable outcome

Danaparoid 75.2%Placebo 73.7%

p = .49OR 1.09 (0.85 – 1.41)

TOASTTOAST

Stroke Subtype Analysis

5 subtypes analyzed for two different endpoints in addition to 4 analyses for total group = 14 analyses

p < .05/14 = p< .0036

Large artery atherosclerosisFavorable outcome p=.04Very favorable outcome p=.02

NO STATISTICALLY VALID SIGNIFICANT DIFFERENCE

Lumping vs. Splitting WARSS

(Mohr et al: NEJM 2001; 345:1444-1451)

Design:Prospective, randomized, double-blind, multi-center clinical trialEligibility:Non-cardioembolic stroke < 30 d, no CEA plannedTreatment:Warfarin (INR 1.4-2.8, mean 2.1) vs aspirin 325 mg/day

Primary endpoint:Recurrent ischemic stroke or death within two years

WARSS (Mohr et al: NEJM 2001; 345:1444-1451)

PRIMARY RESULTS2206 patients followed for 2 years

Ischemic Stroke/ Death Major Hemorrhage /100 pt-yrs

Warfarin 17.8% 2.22Aspirin 16.0% 1.49

p=.25

Hazard ratio 1.13 95% CI 0.92-1.38

Lumping vs. SplittingWASID

(Chimowitz MI, et al: Neurology 1995; 45: 1488-1493)

Design:Retrospective, non-randomized, unblinded, multi-center concurrent cohort study based on angiogram review Eligibility:Symptomatic 50-99% stenosis of major intracranial artery by arteriographyTreatment:MD choice: Warfarin (PT 1.2 –1.6 x control) vs aspirin (usually 325 mg/d)

Primary endpoint:Stroke, MI or sudden death

WASID(Chimowitz MI, et al: Neurology 1995; 45: 1488-1493)

RESULTS OAC ASA

Number of Patients 88 63Median follow-up (mo) 14.7

19.3 Primary Endpoint 14 26

p < 0.01

Hazard ratio 0.46 95% CI 0.23-0.86

In addition to the strength of the evidence, the following considerations will bear on the strength of the treatment recommendation:

-the magnitude and precision of the treatment effect -patients’ risk of the target event being prevented

-the nature of the benefit-the magnitude of the risk associated with

treatment -variability in patient preferences

-variability in regional resource availability -cost considerations

Translating Evidence into Recommendations(Guyatt, et al: Chest 2001; 119:3S-7S)

Issues Examples

Less serious outcomePreventing postphlebitic syndrome with thrombolytic therapy in DVT rather than death from PE.

Smaller treatment effect

Clopidogrel vs aspirin leads to a smaller stroke reduction in TIA (8.7% RRR) than anticoagulation vs placebo in AF (68% RRR).

Imprecise estimate of treatment effect

Aspirin vs placebo in AF has a wider CI than aspirin for stroke prevention in patients with TIA.

Lower risk of target event

Some surgical patients are at very low risk of postoperative DVT and PE, while others surgical patients have considerably higher rates of DVT and PE.

Higher risk of therapy Warfarin has a much higher risk of serious hemorrhage than aspirin.

Higher costs tPA has much higher cost than streptokinase in acute MI.

Varying values

Most young, healthy people will put a high value on prolonging their lives (and thus incur suffering to do so); the elderly and infirm are likely to vary in the value they place on prolonging their lives (and may vary in the suffering they are ready to experience to do so).

Translating Evidence into Recommendations Factors That May Weaken a Recommendation to Treat

(Guyatt, et al: Chest 2001; 119:3S-7S)

Further Reading on Evidence Based Medicine

Users' Guides to the Medical Literature: Essentials of Evidence-Based Clinical Practice edited by Gordon Guyatt and Drummond Rennie, includes CD-ROM, 442 pp, soft cover, $34.95, ISBN 1-57947-191-9, Chicago, Ill, AMA Press, 2002.

Interpreting the Medical Literatureby Stephen H. Gehlbach, 4th ed, 296 pp, paper, $34.95, ISBN 0-07-138762-5, New York, NY, McGraw-Hill, 2002.

Evidence-Based Medicine: How to Practice and Teach EBMby David L. Sackett, Sharon E. Straus, W. Scott Richardson, William Rosenberg, and R. Brian Haynes, 2nd ed, with CD-ROM, 261 pp, softcover, $43.95, ISBN 0-443-06240-4, Philadelphia, Pa, Churchill Livingstone, 2000.