Systematic Review Module 12: Presentation of Findings

Systematic Review Module 12:Systematic Review Module 12:Presentation of FindingsPresentation of Findings

Melissa McPheeters, PhD, MPH Melissa McPheeters, PhD, MPH

Jeff Seroogy, BSJeff Seroogy, BS

Vanderbilt University EPCVanderbilt University EPC

Joseph Lau, MDJoseph Lau, MD

Thomas Trikalinos, MD, PhDThomas Trikalinos, MD, PhDTufts EPCTufts EPC

Learning ObjectivesLearning Objectives

To have a basic understanding of the ways that results are commonly presented in a CER

To understand the benefit of synthesizing nonquantitative results

To review examples of graphical presentations of nonquantitative findings

2

CER Process OverviewCER Process Overview

Prepare topic:

· Refine key questions

· Develop analytic frameworks

Search for and select

studies:

· Identify eligibility criteria

· Search for relevant studies

· Select evidence for inclusion

Abstract data:

· Extract evidence from studies

· Construct evidence tables

Analyze and synthesize data:

· Assess quality of studies

· Assess applicability of studies

· Apply qualitative methods

· Apply quantitative methods (meta-analyses)

· Rate the strength of a body of evidence

Present findings

3

Organizing and Organizing and Reporting FindingsReporting Findings

Once the evidence tables are completed, there is an enormous (sometimes overwhelming) amount of data yet to synthesize

Various types of heterogeneity may preclude a quantitative synthesis

Nonetheless, qualitatively synthesizing heterogeneous studies can capture underlying similarities to support conclusions

Making sense of the data requires good tabular presentation and clear organization and writing in the text

4

Combining StudiesCombining Studies

Synthesizing results from different studies allows for broader conclusions to be drawn– PICOTS can be employed to determine what criterion to

focus on when combining results Population, intervention, comparator, outcome, timing, setting The nature of the review and the substance of the literature will dictate

the most appropriate criterion to use

Not all studies can be (or should be) combined – Various statistical models are useful for identifying the level

of heterogeneity between studies (see Methods Guide)

– Even with statistically significant variation, studies still may be combined unless systematic differences among studies are evident or when combining would yield meaningless, skewed, or biased results

5

Challenges to Combining Challenges to Combining StudiesStudies

Even studies meeting the same inclusion criteria can vary substantially:– Clinical heterogeneity: variation in the study population, interventions, and

outcomes– Methodological heterogeneity: variation in study design– Statistical heterogeneity: variation in observed treatment effect (for trials)

Many factors can contribute to variation in seemingly similar studies:– Evolving diagnostic criteria, evolving diseases, difference in baseline

characteristics, differences in care, etc.

6

Solution …Solution …

Qualitative synthesis through well-constructed text and summary tables

Content and order of report depend on the available evidence, but should adhere to a set of guiding principles

Flow and clarity of the document support development of clear conclusions and summary statements, even with heterogeneous studies

7

Writing the ReportWriting the Report

A standard order for the overall report is recommended

Subsection ordering may vary but should adhere to principles of clarity, be consistent with key questions, and may be guided by PICO(TS)

Summary tables should be presented within each section; where multiple sections present comparable data, comparable tables should be available

8

Overall Order of the Overall Order of the DocumentDocument

Sections prior to and after results are standard Results are then usually structured per the

key questions Order of results presented within key

questions can vary, but when there is a primary outcome of interest (generally efficacy), it is presented first

Modifying questions are generally presented after direct efficacy and effectiveness questions, with the order possibly guided by PICOTS

9

Document StructureDocument Structure

Chapter 1: Introduction– The nuts and bolts: defining the project, purpose and scope of the

CER, key questions, analytic framework, experts involved, etc.

Chapter 2: Methods– Literature review methods: literature search methods, inclusion and

exclusion criteria, developing (NOT presenting) evidence tables, approach to quality, data abstraction, and data synthesis process No “results” (i.e., tables) presented in this chapter, but rather a guide as

to how the information was collected and the tables created

Chapter 3: Results– Broken down by key questions, with subsections when applicable

(focus of this module)

Chapter 4: Discussion– Sections discussing strength of the literature and evidence, principal

findings (broken down by key questions), areas of future research, and any conclusions that can be drawn

10

Presenting ResultsPresenting Results

Chapter 3: Results: example from Overactive Bladder review

Subsections determined over course of the review, dictated by the substance of the literature, technical expert panel guidance, and PICOTS

Depending on findings: some key questions will have several subsections (KQ2/KQ3); some will have few (KQ4/KQ5)

11

Presenting Results: Levels of Presenting Results: Levels of OrganizationOrganization

Level 1: Key Level 1: Key questionsquestions

Level 2: Level 2: Intervention typeIntervention type

Levels 3+: Use Levels 3+: Use key questions and key questions and PICO(TS) to PICO(TS) to organize …organize …

12

PICOTSPICOTS

PICOTS can be used to establish ordering of results

For example, if multiple interventions are being examined, interventions (I) might be the meta-organizational level, with other PICOTS reflected within

Or , if the same intervention is being studied in different populations (P), the report could be structured by population (e.g., results seen in women followed by results seen in men) with interventions (I) ordered within populations (P)

13

Organizing by and within Organizing by and within PICO(TS)PICO(TS)

Population: primary populations of interest followed by subpopulations

Intervention: hierarchy of interventions may reflect most common to least

Comparator: where multiple comparisons are made, one might present the more “usual” decision first

Outcome: a hierarchy of outcomes may reflect higher to lower acuity, or target outcomes followed by collateral ones; harms generally presented last

Timing: generally short term before long term Setting: an intervention may be available in various

settings (e.g., inpatient or outpatient)

14

PrinciplesPrinciples

The order of sections and subsections should reflect the key questions and a conscious attention to PICOTS. Importantly, it is: – Not necessarily a ranking of importance

– Structured to reflect the strength of the data and outcomes

– Useful in determining the applicability of different treatments or the likelihood of different outcomes

15

Organization within Organization within Intervention Category (Level 3)Intervention Category (Level 3)

Within intervention Within intervention category :category :– Content of the Content of the

literature (types literature (types of studies, total of studies, total N, outcomes N, outcomes included)included)

– Outcomes Outcomes (efficacy, (efficacy, effectiveness effectiveness and other)and other)

– HarmsHarms

16

Summary TablesSummary Tables

Within the structure described thus far, represent a map of the outcomes

Provide a snapshot of global characteristics of the CER; tables will differ in their structure and content based on the included literature of the CER

Highlight gaps in the present research and practical goals for future studies

17


Whereas evidence tables represent an individual study, summary tables combine multiple studies to illustrate overarching trends in the data– Can be designed to include characteristics of all

included studies i.e., funding sources, assessment method (mailed

questionnaire, telephone questionnaire, etc), country of study, etc.

– Can be designed for subsets of included studies i.e., summary tables for RCTs, prevalence studies,

harms/side effects, outcomes for specific treatments, etc.

– .

18

Overall Summary TableOverall Summary Table

19


Another basic summary table Another basic summary table is the “Study Characteristics” is the “Study Characteristics” table table

This will vary some between This will vary some between projects, but each should projects, but each should have some form of this tablehave some form of this table

Fields can be taken from the Fields can be taken from the global descriptors in the global descriptors in the evidence tables or at the evidence tables or at the recommendation of content recommendation of content experts experts

20

Balk et al. Effects of Statins on Nonlipid Serum Balk et al. Effects of Statins on Nonlipid Serum Markers Associated with Cardiovascular Markers Associated with Cardiovascular

Disease. Disease. Ann Intern Med Ann Intern Med 20032003

21

Summary MatrixSummary MatrixPrimary prevention: cohort study of fish consumption Primary prevention: cohort study of fish consumption

and all cause mortality (and all cause mortality (EPC Report # 94, Effects of EPC Report # 94, Effects of omega-3 fatty acids and cardiovascular disease 2004)omega-3 fatty acids and cardiovascular disease 2004)

22


Summarizes information for each study answering one question

Simplified entry (one row) for each study Some dimensions for table columns

– PICO (may be listed in table title or headers)

– Methodological quality

– Applicability

– Study size (weight)

– Magnitude of effect

One study may belong to multiple summary tables (e.g., different outcomes)

23


Summary tables can Summary tables can be specialized for be specialized for different subgroups different subgroups of the literatureof the literature

Example: a table Example: a table presenting the presenting the findings from only findings from only the included RCT the included RCT studies of one studies of one treatment type treatment type

24

Summary Table ExampleSummary Table ExampleSecondary prevention RCTs: supplementsSecondary prevention RCTs: supplements

25

Summary Table ExampleSummary Table ExamplePrimary prevention: cohort study Primary prevention: cohort study (EPC Report # 94, (EPC Report # 94, Effects of omega-3 fatty acids and cardiovascular disease 2004)Effects of omega-3 fatty acids and cardiovascular disease 2004)

26


Summary tables Summary tables can be specialized can be specialized for different for different subgroups of the subgroups of the literatureliterature

Example: a table Example: a table presenting the presenting the findings from findings from prevalence prevalence literature literature

27


Summary tables can be Summary tables can be specialized for different specialized for different subgroups of the literaturesubgroups of the literature

Example: a table Example: a table presenting the proportion presenting the proportion of participants of participants experiencing harms and experiencing harms and side effects from side effects from treatment studiestreatment studies

28


Summary tables Summary tables can be specialized can be specialized for different for different subgroups of the subgroups of the literatureliterature

Example: a table Example: a table presenting the presenting the outcomes of a outcomes of a certain treatment certain treatment

29

Summary Table ExampleSummary Table ExampleSecondary prevention RCTs: diet/dietary adviceSecondary prevention RCTs: diet/dietary advice

30

Forest Plot without SummaryForest Plot without Summary

31

Favors WACAFavors WACA Favors PVIFavors PVI

Wide area circumferential ablation (WACA) vs. pulmonary vein isolation (PVI): Freedom from atrial Wide area circumferential ablation (WACA) vs. pulmonary vein isolation (PVI): Freedom from atrial fibrillation recurrence; Evidence Report “Comparative Effectiveness of Radiofrequency Catheter fibrillation recurrence; Evidence Report “Comparative Effectiveness of Radiofrequency Catheter Ablation for Atrial Fibrillation” by Tufts EPCAblation for Atrial Fibrillation” by Tufts EPC

Evidence MapEvidence Map

A succinct visualization of studies available to address specific question(s) along one or more variables of interest– Results of outcomes typically not included

Can help to identify comparisons where evidence is sufficient to be analyzed

To guide readers in knowing what comparisons and outcomes are available in the report

To quickly appreciate evidence gaps

32

Update on Acute Bacterial Rhinosinusitis. Evidence Update on Acute Bacterial Rhinosinusitis. Evidence Report 124Report 124

33

Primary Studies on Vitamin D Primary Studies on Vitamin D Intake or ConcentrationIntake or Concentration

34

Example of Evidence MapExample of Evidence Map

Fig

ure

2. C

urr

ent

clin

ical

evi

den

ce o

n p

artic

le b

ea

m th

era

pie

s. T

era

saw

a T

, et

al.

Ann

Inte

rn M

ed 2

00

9

35

Evidence Map:Evidence Map: VKORC1 VKORC1 Single Nucleotide Single Nucleotide Polymorphisms and Mean Warfarin Maintenance Polymorphisms and Mean Warfarin Maintenance

Dose or Anticoagulation OutcomesDose or Anticoagulation Outcomes

36

Wrapping UpWrapping Up

PRISMA checklist– Preferred Reporting Items for Systematic Reviews

and Meta-Analyses

– A list of items that should be included in any evidence review and flow diagram detailing the reporting of included studies i.e., title, study selection process, synthesis of results,

etc. (27 total items in the 2009 version)

– Very useful tool to reference during the writing of a review, and a necessary checklist to work through during the closing stages of a project

– Available at www.prisma-statement.org/

37

Systematic Review Module 12: Presentation of Findings

Documents

Transcript of Systematic Review Module 12: Presentation of Findings