Literature at your fingertips May 24, 2013 Peter Tugwell.

Literature at your fingertips

May 24, 2013

Peter Tugwell

• Participants will learn how to formulate questions that can be answer by available evidence

• Participants will learn an approach to critically appraise the literature

• Participants will learn the key features of systematic reviews and how to use them to inform clinical practice

2

Learning Objectives

• None applicable.

Disclosures

EBM Key Concepts

6 A’s1. ASK2. ACCESS -3. APPRAISE4. AGGREGATE5. APPLY6. AUDIT

‘PICOT’ FIGURE

GUIDELINES GRADE AMSTAR

©

EBM Key Concepts


THE 6 A’S -STEPS OF EVIDENCE BASED PRACTICE

1. ASK a focussed question2. ACCESS - search for epidemiological evidence to help

answer question3. APPRAISE the evidence for its validity, effect size,

precision)4. AGGREGATE the evidence using the

triangle,circle,box,x FRAMEWORK5. APPLY your decision integrating the aggregated

evidence into the trade-off of (i)benefits versus harms;(ii)patient values and preferences,(iii) cost –effectiveness and cost- equity,to make an evidence-based decision

6. AUDIT your practice (i.e. check your actual practice against evidence-based practice on a regular basis)].

EBM Key Concepts


‘PICOT’

EBP STEP 1: ASK - TURN YOUR QUESTION INTO 5 PARTS (PICOT)

1. P2. I3. C4. O5. T


1. Participants (patient(s) you want to treat)


1. Participants (patient(s) you want to treat)2. Intervention( or ‘Exposure’ if no intervention )


1. Participants (patient(s) you want to treat)2. Intervention( or ‘Exposure’ if no intervention )3. Comparison (there is always an alternative! -

another therapy, nothing …



another therapy, nothing …4. Outcome (MCID [Minimal Clinically Important Difference] in the

Humanly Important Outcome [Distress,Disability,Dysfunction,Death]



another therapy, nothing …4. Outcome (MCID [Minimal Clinically Important Difference] in the

Humanly Important Outcome [Distress,Disability,Dysfunction,Death]

5. Time frame (over which you expect a result)

2012 ACR OA Scenarios-Knee

Base Case 1 (knee OA)Patients:An adult with symptomatic knee OA without co-morbidities, presents to her/his primary care provider for treatment with mild to severe pain in and/or around her knee(s) .No response to acetominophen.Intervention: NSAID Comparator : PlaceboOutcome: Pain, Function, Adverse Effects

Scenarios-Knee (con’t)

– Peptic ulcer Scenario : …she/he has a history of symptomatic and/or complicated peptic ulcer disease (e.g., history of upper GI bleed but not in the past year).

Scenarios-Knee (con’t)

– Low Dose Aspirin: …she/he is taking low-dose (325 mg/day or less) aspirin for cardioprotection

EBM Key Concepts


‘PICOT’ FIGURE


©

P

I C

OT

• P

• I

• C

• O

• T

• Recruitment

• Allocation

• Maintenance

• Blind or

• Objective measurements &

processes

the evidence using RAMBO on the PICOT frame

Figure

©

Epidemiologic evidence

Clinical / population

health considerations

Policy issues

Patient / community preferences

Guidelines: making evidence-based decisions

expertise: ‘putting it all together’ the art of practice

EBP Step 4: AGGREGATE the relevant information & make an evidence-based decision:’ the X-factor

EBM Key Concepts


‘PICOT’ FIGURE

GUIDELINES GRADE

©

OUR EXPERIENCE WITH DEVELOPING OA GUIDELINES FOR ACR

Roy Altman, Maria Benkhalti, Elizabeth Ghogumu, Gordon Guyatt, Marc Hochberg, Jessie McGowan,

Annette O’Connor, Karine Toupin-April, Peter Tugwell , Erin Ueffing, Vivian Welch, George Wells

June 1st, 2009Rheumatology city-wide rounds, Ottawa

Limitations of Previous Methods for developing Guidelines

• Methods Lack of clarity in presentation of results – Clinicians do not understand what they mean!

Ie diffference between 1a 2b 3d – Lack of validated yet simple qua 3d lity – Both Absolute and Relative effects often not reported – Absolute NNT/NNH often not calculated– Only SMDs

• Absence of clinically relevant scenarios in some

2008 Request from ACR for new OA Guidelines

To develop guidelines for the management of osteoarthritis of the hip, knee and handMethods considered

– Rand Appropriateness Panel– GRADE (Grades of Recommendation, Assessment,

Development and Evaluation) method

Proposal to ACR for OA Guidelines

Pharmacologic and non-pharmacologic interventionsUse of ‘GRADE’ to create Summary of Findings tables and to make recommendations

Use of the Wells calculator to find – The relative risk for continuous outcomes – The number needed to treat (NNT)

Use of clinical scenarios to ensure clinical relevance

Clinically relevant treatment comparisons (>50)

ACR Proposal

Pharmacologic and non-pharmacologic interventions

Knee and Hip OA TreatmentsAcupunctureTENS AcetaminophenChondroitin SulfateCortico-steroid injectionGlucosamine SulfateOpioidsTramadol

Oral NSAIDsTopical capsaicinTopical NSAIDsHyaluronates injectionOther non-pharmacologic PT and OT

– [incl weight loss, exercise, knee brace,foot insole]

> 50 comparisons

ACR Scope of work

Treatments specific to Hand OA

Oral Trolamine salicylateOral methotrexateOral hydroxychloroquineOral sulfasalazine

ACR Scope of work

ACR Proposal

Use of the new ‘GRADE’ Method

‘GRADE’ (Grades of Recommendation, Assessment, Development and Evaluation) method

– to create Summary of Findings tables and to make recommendations

GRADE system uptake (Grades of Recommendation, Assessment, Development and Evaluation)

Up-To-Date British Medical JournalACP BMJ Clinical EvidenceSociety of Vascular SurgeryEuropean Respiratory SocietySociety of Critical Care Medicine American College of Chest Physicians EBM Guidelines FinlandNational Institute for Clinical Excellence (NICE)Agency for Health Care Research and Quality (AHRQ)Swedish National Board of Health and WelfareCanadian Agency for Drugs and Technology in Health Ontario MOH Medical Advisory Secretariat

World Health OrganizationAmerican Thoracic Society Cochrane CollaborationPolish Institute for EBM Society of Pediatric EndocrinologyAmerican Endocrine SocietySurviving sepsis campaignEuropean Soc of Thoracic Surgeons Allergic Rhinitis in Asthma GuidelinesEvidence-based Nursing Sudtirol, Alta Adiga, ItalyNorwegian Knowledge Centre for the Health ServicesUniversity of Pennsylvania Health System Center for EB Practice Journal of Infection in Developing Countries - InternationalAgencia sanitaria regionale, Bologna, ItaliaThe German Agency for Quality in Medicine

E.g. Topical NSAIDs vs. placebo

SUMMARY OF FINDINGS TABLE -SOF TABLE

Oral NSAIDs vs. Placebo

SUMMARY OF FINDINGS TABLE-SOF TABLE

Cox-2 vs. Oral NSAIDsPatient or population: patients with OA and RA Intervention: Celecoxib Comparison: non-selective NSAIDs

Outcomes Illustrative comparative risks (95% CI)

Absolute difference

Relative effect (95% CI)

No of Participants (studies)

Quality of the evidence (GRADE)

NNT

Control group rate

Intervention rate

non-selective NSAIDs

Celecoxib

Benefits

Pain VAS. Scale from: 0 to 10 cm.

See NSAID vs placebo

See Cox vs placebo

Mean difference -0.42 cm on 0-10

cm scale (-2.4 to 1.6)

(14) O MODERATE

Not statistically significant

Harms

POBs 10%

4% (3% to 5%)

-6%

0.39 (0.31 to 0.50)

73449 (8)

HIGH

17 (15 to 20)

PUBs 10%

4% (3% to 5%)

-6%

0.41 (0.33 to 0.52)

85,955 (13)

HIGH

17 (15 to 21)

Myocardial infarction** 0.4%

0.3% (0.2% to

0.5%)

-0.1%

RR 0.80 (0.53 to 1.20)

29568 (13)

low1,2

Not statistically significant

1 RR CI ranged from 0.53 to 1.20 2 Studies were industry-funded ** Chen LC, Ashcroft DM. Risk of myocardial infarction associated with selective COX-2 inhibitors: meta-analysis of randomised controlled trials.

Pharmacoepidemiology and drug safety 2007;16:762-72.

SUMMARY OF FINDINGS TABLE-SOF TABLE

GRADE quality assessment (Grades of Recommendation, Assessment, Development and Evaluation)

EBM Key Concepts


‘PICOT’ FIGURE


©

How do you critically appraise a systematic review?

Assessing the Methodological Quality of Systematic Reviews

AMSTAR(A MeaSurement Tool to Assess

Systematic Reviews)

Acknowledgements

► BEVERLEY SHEA► Jeremy Grimshaw ► Lex Bouter► Maarten Boers► David Moher► Andy Oxman► Gord Guyatt► Peter Tugwell► Cochrane Musculoskeletal and HIV groups► CADTH► Internal and external validation teams

Main Considerations

►Why is the quality of SRs important?►How best to measure quality?►The development of AMSTAR►Future development of AMSTAR

The Perspective

► Decision makers

– Policy makers– Guidelines developers– Health professionals– Patients– Pharma

► Teachers/academics

Central Concepts

►In the context of a SR, validity is the extent to which its design and conduct minimize bias

►Methodological quality of SRs– how well the SR was conducted (AMSTAR)

►Reporting quality of SRs– how well systematic reviewers have reported their methodology

and findings (PRISMA)

AMSTAR and PRISMA:Overview

AMSTAR PRISMA

• Assesses methodological quality of systematic reviews (SRs)

• Assesses the reporting quality of systematic reviews (SRs)

• 11-item checklist • 27-item checklist

• Designed to improve the quality and conduct of SR methods

• Designed to improve transparency and consistency of reporting SR methods

• E.g.: Provides criteria for assessing a literature search as adequate or inadequate

• E.g.: Requires details of the literature search terms and sources used, but does not distinguish between adequate and inadequate searches

AMSTAR (1)

1. Was an ‘a priori’ design provided, outlining the reviewmethods prior to starting the review?The research question and inclusion criteria should be established before the

conduct of the review.

□ Yes □ No □ Can’t answer □ N/A

2. Was there duplicate study selection and data extraction? There should be at least two independent data extractors and a consensus procedure for disagreements should be in place.


3. Was a comprehensive literature search performed using at least two electronic sources and reasonably supplemented by other sources?At least two electronic sources should be searched. The report must include years and databases used (e.g. Central, EMBASE, and MEDLINE). Key words and/or MESH terms must be stated and where feasible the search strategy should be provided. All searches should be supplemented by consulting current contents, reviews, textbooks, specialized registers, or experts in the particular field of study, and by reviewing the references in the studies found.


AMSTAR (2)

4. Were the study inclusion criteria comprehensive, including

publications of all types and languages?The authors should state that they searched for reports regardless of their

publication type. The authorsshould state whether or not they excluded any reports (from the systematic review),

based on theirpublication status, language etc.


5. Was a list of studies (included and excluded) provided? A list of included and excluded studies should be provided.


6. Were the characteristics of the included studies provided?In an aggregated form such as a table, data from the original studies should be provided on the participants, interventions and outcomes. The ranges of characteristics in all the studies analyzed e.g. age, race, sex, relevant socioeconomic data, disease status, duration, severity, or other diseases should be reported.


7. Was the scientific quality of the included studies assessed and documented, using an established scale?‘A priori’ methods of assessment should be provided (e.g., for effectiveness studies if the author(s) chose to include only randomized, double-blind, placebo controlled studies, or allocation concealment as inclusion criteria); for other types of studies alternative items will be relevant.


AMSTAR (3)8. Was the scientific quality of the included studies used

appropriately in formulating conclusions?The results of the methodological rigor and scientific quality should be considered in

the analysis and theconclusions of the review, and explicitly stated in formulating recommendations.


9. Were the methods used to combine the findings of studies appropriate? For the pooled results, a test should be done to ensure the studies were combinable, to assess their homogeneity (i.e. Chi-squared test for homogeneity, I²). If heterogeneity exists a random effects model should be used and/or the clinical appropriateness of combining should be taken into consideration (i.e. is it sensible to combine?).


10. Was the likelihood of publication bias assessed?An assessment of publication bias should include a combination of graphical aids (e.g., funnel plot, other available tests) and/or statistical tests (e.g., Egger regression test).


11. Was the conflict of interest stated?Potential sources of support should be clearly acknowledged in both the systematic review and the included studies.


Feasibility

► Informal feedback on feasibilityEasy to use, understandable and easy to

apply, good teaching tool, not too difficult to score - like the individual components, some degree of difficulty with answering the question on inclusion of grey literature

► AMSTAR takes approximately 5 minutes less than the Oxman and Guyatt AMSTAR 14.9 (range 5-29) minutesOxman and Guyatt 20.3 (range 9-36)

minutesSacks 34.43 (range 22-52) minutes

Reasons to use AMSTAR?

►Value in using a common instrument►Builds on previous methodologically sound work►Good content and construct validity►Good reliability►Excellent feasibility►Published in peer reviewed journals►Has a memorable name!

Impact of AMSTAR

► Citations: 87 to date (Google Scholar)► Downloads > 6000

– http://www.plosone.org/article/info:doi%2F10.1371%2Fjournal.pone.0001350

– http://www.biomedcentral.com/content/pdf/1471-2288-7-10.pdf

► Accepted by a Canadian Association Drugs and Technology (CADTH) as the instrument to use in all assessments of methodological quality in SRs

► Published materials in support of using AMSTAR ‘Standard criteria, such as….(AMSTAR), should be used to critically appraise…’ (Andrew D Oxman*Health Research Policy and Systems 2006, 4:20)

► Modified CONSORT (Biomed Central in press) has a comment that it measures quality of reporting not methods

http://www.plosone.org/article/info:doi/10.1371/journal.pone.0001350

THANK –YOU !

QUESTIONS ?

Please complete your

GREEN EVALUATION SHEETS

Literature at your fingertips May 24, 2013 Peter Tugwell.

Documents

Transcript of Literature at your fingertips May 24, 2013 Peter Tugwell.