1

Levels of Evidence

• Why?• What?• How?

2

Time-poor clinician suffering from Information Overload

3

Evidence-Based Medicine

• EBM is ...”the conscientious, explicit and judicious use of current best evidence in making decisions about the care of an individual patient. It means integrating individual clinical expertise with the best available external clinical evidence from systematic research”

(Sackett, D. BMJ 1996;312:71-72).

4

5

Steps in EBM

6

The Evidence Pyramid is a guideline to the hierarchy of study design

7

Type of question = type of study design

Type of question Suggested study

Therapy RCT > prospective cohort

Diagnosis Prospective, blind comparison to a gold standard

Etiology/Harm RCT > cohort > case control > case series

Prognosis Cohort study > case control > Case series

Prevention RCT > cohort study >case control > case series

Cost Economic analysis

8

NHMRC Levels of Evidence

Level Intervention Diagnostic accuracy Prognosis Aetiology Screening Intervention

I 4 A systematic review of level II studies

A systematic review of level II studies A systematic review of level II studies

A systematic review of level II studies

A systematic review of level II studies

II A randomised controlled trial A study of test accuracy with: an independent, blinded comparison with a valid reference standard,5 among consecutive persons with a defined clinical presentation6

A prospective cohort study

A prospective cohort study A randomised controlled trial

III-1 A pseudorandomised controlled trial (i.e. alternate allocation or some other method)

A study of test accuracy with: an independent, blinded comparison with a valid reference standard,5 among non-consecutive persons with a defined clinical presentation6

All or none All or none A pseudorandomised controlled trial (i.e. alternate allocation or some other method)

III-2 A comparative study with concurrent controls: ▪ Non-randomised, experimental trial9 ▪ Cohort study ▪ Case-control study ▪ Interrupted time series with a control group

A comparison with reference standard that does not meet the criteria required for Level II and III-1 evidence

Analysis of prognostic factors amongst persons in a single arm of a randomised controlled trial

A retrospective cohort study A comparative study with concurrent controls: ▪ Non-randomised, experimental trial ▪ Cohort study ▪ Case-control study

III-3 A comparative study without concurrent controls: ▪ Historical control study ▪ Two or more single arm study10 ▪ Interrupted time series without a parallel control group

Diagnostic case-control study6 A retrospective cohort study

A case-control study A comparative study without concurrent controls: ▪ Historical control study ▪ Two or more single arm study

IV Case series with either post-test or pre-test/post-test outcomes

Study of diagnostic yield (no reference standard)

Case series, or cohort study of persons at different stages of disease

A cross-sectional study or case series

Case series

9

NHMRC Assessment of study quality – Grades of Recommendations1. The evidence base, in terms of the number of studies, level

of evidence and quality of studies (risk of bias).2. The consistency of the study results.3. The potential clinical impact of the proposed

recommendation.4. The generalisability of the body of evidence to the target

population for the guideline.5. The applicability of the body of evidence to the Australian

healthcare context.

10

Checklist for appraising the quality of studies of interventions (Cochrane handbook)

1. Method of treatment assignment

a. Correct, blinded randomisation method described

OR randomised, double-blind method stated

AND group similarity documented

b. Blinding and randomisation stated but method not described

OR suspect technique (eg allocation by drawing from an envelope)

c. Randomisation claimed but not described and investigator not blinded

d. Randomisation not mentioned

2. Control of selection bias after treatment assignment

a. Intention to treat analysis AND full follow-up

b. Intention to treat analysis AND <15% loss to follow-up

c. Analysis by treatment received only OR no mention of withdrawals

d. Analysis by treatment received

AND no mention of withdrawals

OR more than 15% withdrawals/loss-to-follow-up/post-randomisation exclusions

3. Blinding

a. Blinding of outcome assessor

AND patient and care giver

b. Blinding of outcome assessor

OR patient and care giver

c. Blinding not done

4. Outcome assessment (if blinding was not possible)

a. All patients had standardised assessment

b. No standardised assessment OR not mentioned

Source: National Health and Medical Research Council (NHMRC). How to review the evidence: systematic identification and review of the scientific literature. Canberra: NHMRC, 1999: p.45

11

NHMRC Grades of RecommendationsGrade of recommendation Description

A

Body of evidence can be trusted to guide practice

B

Body of evidence can be trusted to guide practice in most situations

C

Body of evidence provides some support for recommendation(s) but care should be taken in its application

D

Body of evidence is weak and recommendation must be appliedwith caution

12

Clinical Guidelines for Stroke Management 2010. National Stroke Foundation

13

Brain Trauma, F., S. American Association of Neurological, et al. (2007). "Guidelines for the management of severe traumatic brain injury." Journal of Neurotrauma 24 Suppl 1.

Class of evidence Study design Quality Criteria

I Good quality randomized controlled trial (RCT)

Adequate random assignment methodAllocation concealmentGroups similar at baselineOutcome assessors blindedAdequate sample sizeIntention-to-treat analysisFollow-up rate 85%No differential loss to follow-upMaintenance of comparable groups

II Moderate quality RCT Violation of one or more of the criteria for a good quality RCT

II Good quality cohort Blind or independent assessment in a prospective study, or use cohort of reliable data in a retrospective studyNon-biased selectionFollow-up rate 85%Adequate sample sizeStatistical analysis of potential confounders

II Good quality case-control Accurate ascertainment of casesNonbiased selection of cases/controls with exclusion criteria applied equally to bothAdequate response rateAppropriate attention to potential confounding variables

III Poor quality RCT Major violations of the criteria for a good or moderate quality RCT

III Moderate or poor quality cohort Violation of one or more criteria for a good quality cohort

III Moderate or poor quality case-control Violation of one or more criteria for a good quality case-control

III Case series, databases or registries

14

Table 1. Applying Classification of Recommendations and Level of Evidence

Morgenstern, L. B., J. C. Hemphill, 3rd, et al. (2010). "Guidelines for the management of spontaneous intracerebral haemorrhage: a guideline for healthcare professionals from the American Heart Association/American Stroke Association." Stroke 41(9): 2108-29.

15

Is Evidence-Based Surgery an oxymoron? What if there is no level I evidence? Surgical RCTs have well-recognized disadvantages:

high costs, administrative complexity, prolonged time to completion, recruitment difficulty, blinding, randomization technique standardization, poor generalizability or external validity, patient compliance, underpowered studies, crossovers and drop outs, multiple surgical options, technological advancement, patient complexity, variability and preference and selection bias ....

16

• It is the well-defined research question that dictates the study design, not that every study should be a RCT because it’s the gold standard”

• “The proper use of evidence-based information is not the strict adherence to only RCTs, but more accurately, the informed and effective use of all types of evidence … Large, prospective cohort studies in a surgical setting are often thought to be on a par with RCTs and provide superior generalizability”

Fisher, C. G. and K. B. Wood (2007). "Introduction to and techniques of evidence-based medicine." Spine 32(19 Suppl): S66-72.

17

ISAT (Lancet, 2002; Lancet, 2005)• 9559 eligible patients, 2143 randomised

– 43 Participating centres; enrolled 1-44% of eligible patients• Equipoise did not exist in over 75%

– 3615 underwent surgery– 2737 underwent endovascular treatment– 1064 unknown treatment

• Differences between groups– Cross-overs, time to treatment,

• Small but significant difference in time between randomisation and first procedure

– Coiling 1.1 days Surgery 1.7 days• Cross-overs

– Coiling surgery 9 patients– Surgery coiling 38 patients

• Different experience of INR and surgeons• Unknown differences between centres

18

• Know which levels and grades of recommendations are being used and quote/reference them

• Stay up to date with developments or changes to levels and grades

• Use NHMRC levels and grades where possible• Look for other levels of evidence when RCTs or level 1 studies are not possible

• Make evidence-based decisions• Become a lifelong learner of EBM http://libguides.mq.edu.au/content.php?pid=167579&sid=1412023

Using the evidence....