Ilaria Casetta, MD Ilaria Casetta, MD Neurological Clinic University of Ferrara Copyright © The...

50
Ilaria Casetta, Ilaria Casetta, MD MD Neurological Clinic University of Ferrara Copyright © The Cochrane Collaboration

Transcript of Ilaria Casetta, MD Ilaria Casetta, MD Neurological Clinic University of Ferrara Copyright © The...

Page 1: Ilaria Casetta, MD Ilaria Casetta, MD Neurological Clinic University of Ferrara Copyright © The Cochrane Collaboration.

Ilaria Casetta, MDIlaria Casetta, MD Ilaria Casetta, MDIlaria Casetta, MD

Neurological Clinic

University of Ferrara

Copyright © The Cochrane Collaboration

Page 2: Ilaria Casetta, MD Ilaria Casetta, MD Neurological Clinic University of Ferrara Copyright © The Cochrane Collaboration.

BRIEF HISTORY

The Cochrane Collaboration is named in honour of Archie Cochrane, a British medical researcher who contributed greatly to the development of epidemiology as a science.

Professor Archibald Leman Cochrane, CBE FRCP FFCM, (1909 - 1988)Cardiff University Library, Cochrane Archive, University Hospital Llandough.

Page 3: Ilaria Casetta, MD Ilaria Casetta, MD Neurological Clinic University of Ferrara Copyright © The Cochrane Collaboration.

Healthcare decision-making needs to be informed by high quality, timely research evidence

Prof. Cochrane stressed the importance of using evidence from randomised controlled trials (RCT's) because these were likely to provide much more reliable information than other sources of evidence.

Background

Page 4: Ilaria Casetta, MD Ilaria Casetta, MD Neurological Clinic University of Ferrara Copyright © The Cochrane Collaboration.

"It is surely a great criticism of our profession that we have not organised a critical summary, by specialty or subspecialty, adapted periodically, of all relevant randomised controlled trials."

(1979)

"It is surely a great criticism of our profession that we have not organised a critical summary, by specialty or subspecialty, adapted periodically, of all relevant randomised controlled trials."

(1979)

Background

Page 5: Ilaria Casetta, MD Ilaria Casetta, MD Neurological Clinic University of Ferrara Copyright © The Cochrane Collaboration.

Most doctors do not have the time to critically read all original papers

Often the results of available trials are not consistent

Textbooks become quickly out of date

Why do we need critical summaries?

Page 6: Ilaria Casetta, MD Ilaria Casetta, MD Neurological Clinic University of Ferrara Copyright © The Cochrane Collaboration.

Established as an international organisation in 1993, registered as a charity in the U.K.

Aim- To help people make well-informed decisions about health care.

How - by preparing, disseminating and maintaining,systematic reviews of the effects of healthcare interventions across all areas of health care.

Background and aims

Page 7: Ilaria Casetta, MD Ilaria Casetta, MD Neurological Clinic University of Ferrara Copyright © The Cochrane Collaboration.

More than 28,000 dedicated people from over 100 countries.

They work together to help healthcare providers, policy-makers, patients, their advocates and carers , make well-informed decisions about health care

The members of The Cochrane Collaboration are organised into groups, known as 'entities',

Cochrane Collaboration

Page 8: Ilaria Casetta, MD Ilaria Casetta, MD Neurological Clinic University of Ferrara Copyright © The Cochrane Collaboration.

The main work of The Cochrane Collaboration is done by more than fifty Cochrane Review Groups

Methods Groups have been established to develop and update methodology

The work of Cochrane Groups is facilitated in various ways by the work of Cochrane Centres around the world.

The consumer network help to establish priorities and identify outcomes relevant to patient

Organization

Page 9: Ilaria Casetta, MD Ilaria Casetta, MD Neurological Clinic University of Ferrara Copyright © The Cochrane Collaboration.

Activity and outputs

Training courses Methodology Handbooks

Software RevMan

Cochrane Library

Cochrane Handbook for Systematic Reviews of Interventions Cochrane Handbook for Diagnostic Test Accuracy Reviews

Page 10: Ilaria Casetta, MD Ilaria Casetta, MD Neurological Clinic University of Ferrara Copyright © The Cochrane Collaboration.

www.thecochranelibrary.com

Page 11: Ilaria Casetta, MD Ilaria Casetta, MD Neurological Clinic University of Ferrara Copyright © The Cochrane Collaboration.

About The Cochrane Library The Cochrane Library is a collection of six

databases that contain different types of high-quality, independent evidence to inform healthcare decision-making, and a seventh database that provides information about groups in The Cochrane Collaboration.

Page 12: Ilaria Casetta, MD Ilaria Casetta, MD Neurological Clinic University of Ferrara Copyright © The Cochrane Collaboration.

The Cochrane Database of Systematic Reviews (Cochrane Reviews)

The Cochrane Database of Abstracts of Reviews of Effects –DARE (Other reviews)

The Cochrane Central Register of Controlled Trials- CENTRAL (Clinical trials)

The Cochrane Methodology Register (Methods Studies) Health Technology Assessment Database (Technology

Assessments) NHS Economic Evaluation Database (Economic Evaluations) About the Collaboration

The Cochrane Library is a collection of evidence-based

medicine databases:

Page 13: Ilaria Casetta, MD Ilaria Casetta, MD Neurological Clinic University of Ferrara Copyright © The Cochrane Collaboration.

Cochrane Central Register of Controlled Trials (CENTRAL)

Includes details of published articles taken from bibliographic databases (notably MEDLINE and EMBASE), and other published and unpublished sources. CENTRAL records include the title of the article, information on where it was published (bibliographic details) and, in many cases, a summary of the article. They do not contain the full text of the article.

Each Cochrane Review Group maintains and updates a collection of controlled trials relevant to its own area of interest (handsearch)

Page 14: Ilaria Casetta, MD Ilaria Casetta, MD Neurological Clinic University of Ferrara Copyright © The Cochrane Collaboration.

Cochrane Methodology Register (CMR)

is produced by the UK Cochrane Centre, on behalf of the Cochrane Methodology Review Group.

A collection of paper that report on methods of controlled trials, and of methodological studies.

Page 15: Ilaria Casetta, MD Ilaria Casetta, MD Neurological Clinic University of Ferrara Copyright © The Cochrane Collaboration.

Database of Abstracts of Reviews of Effects (DARE)

Contains abstracts of systematic reviews that have been quality-assessed. Each abstract includes a summary of the review together with a critical commentary about the overall quality.

DARE complements the CDSR by quality-assessing and summarizing reviews that have not yet been carried out by The Cochrane Collaboration..

DARE is produced by the Centre for Reviews and Dissemination (CRD) at the University of York, UK.

Page 16: Ilaria Casetta, MD Ilaria Casetta, MD Neurological Clinic University of Ferrara Copyright © The Cochrane Collaboration.

Health Technology Assessment Database brings together details of completed and

ongoing health technology assessments (studies of the medical, social, ethical, and economic implications of healthcare interventions) from around the world. The aim of the HTA Database is to improve the quality and cost-effectiveness of health care

Page 17: Ilaria Casetta, MD Ilaria Casetta, MD Neurological Clinic University of Ferrara Copyright © The Cochrane Collaboration.

NHS Economic Evaluation Database

The NHS Economic Evaluation Database (EED) assists decision-makers by systematically identifying economic evaluations from around the world, appraising their quality, and highlighting their relative strengths and weaknesses.

Page 18: Ilaria Casetta, MD Ilaria Casetta, MD Neurological Clinic University of Ferrara Copyright © The Cochrane Collaboration.

Cochrane Database of Systematic Reviews (CDSR) Is the leading resource for systematic

reviews in health care. The CDSR includes all Cochrane Reviews

(and protocols) prepared by Cochrane Review Groups in The Cochrane Collaboration.

Page 19: Ilaria Casetta, MD Ilaria Casetta, MD Neurological Clinic University of Ferrara Copyright © The Cochrane Collaboration.

Cochrane Database of Systematic Reviews Each Cochrane Review is a peer-reviewed

systematic review that has been prepared and supervised by a Cochrane Review Group (editorial team) in The Cochrane Collaboration according to the Cochrane Handbook for Systematic Reviews of Interventions or Cochrane Handbook for Diagnostic Test Accuracy Reviews

Page 20: Ilaria Casetta, MD Ilaria Casetta, MD Neurological Clinic University of Ferrara Copyright © The Cochrane Collaboration.

Cochrane Reviews The Cochrane Reviews are prepared by authors

who register titles with one of the 53 Cochrane Review Groups (focusing on a specific topic area)

Co-ordinating Editor(s), and Editorial team including a Managing Editor and Trials Search Co-ordinator.

The Cochrane Review Groups provide authors with methodological and editorial support to prepare Cochrane Reviews, and manage the editorial process, including peer review

Page 21: Ilaria Casetta, MD Ilaria Casetta, MD Neurological Clinic University of Ferrara Copyright © The Cochrane Collaboration.

Systematic review

“A review that is conducted according to clearly stated, scientific research methods, and is designed to minimize biases and errors inherent to traditional, narrative reviews.”

Margaliot, Zvi, Kevin C. Chung. Systematic Reviews: A Primer for Plastic Surgery Research. PRS Journal. 120/7 (2007)

Page 22: Ilaria Casetta, MD Ilaria Casetta, MD Neurological Clinic University of Ferrara Copyright © The Cochrane Collaboration.
Page 23: Ilaria Casetta, MD Ilaria Casetta, MD Neurological Clinic University of Ferrara Copyright © The Cochrane Collaboration.

Systematic reviews Systematic reviews

Systematic in identification of Systematic in identification of literatureliterature

Explicit in statement of Explicit in statement of objectives, materials, and objectives, materials, and methodsmethods

Reproducible in its Reproducible in its methodology and conclusionsmethodology and conclusions

Scientific approach to a review article

Criteria determined at outset Comprehensive search for

relevant articles Explicit methods of appraisal

and synthesis Meta-analysis may be used to

combine data

Page 24: Ilaria Casetta, MD Ilaria Casetta, MD Neurological Clinic University of Ferrara Copyright © The Cochrane Collaboration.

Systematic process involving several steps

Formulate the question Plan the review (a priori protocol) Comprehensive search Unbiased selection and abstraction process Critical appraisal of data Synthesis of data (may include meta-analysis) Interpretation of results

All steps are described explicitly in the review.

Page 25: Ilaria Casetta, MD Ilaria Casetta, MD Neurological Clinic University of Ferrara Copyright © The Cochrane Collaboration.

Well formulated question

Population (Patient, Problem)

Intervention Comparison Outcomes (primary /

secondary)

Time

Patients with MS Beta Interferon Placebo Relapse rate/ disease

progression

After 1 year/3 years

EXAMPLE

Page 26: Ilaria Casetta, MD Ilaria Casetta, MD Neurological Clinic University of Ferrara Copyright © The Cochrane Collaboration.

Systematic process involving several steps

Formulate the question Plan the review (a priori protocol) Comprehensive search Unbiased selection and abstraction process Critical appraisal of data Synthesis of data (may include meta-analysis) Interpretation of results

All steps are described explicitly in the review.

Page 27: Ilaria Casetta, MD Ilaria Casetta, MD Neurological Clinic University of Ferrara Copyright © The Cochrane Collaboration.

Plan the review : protocolThe Protocol is the a priori work-plan for the

eventual review. It lays out the plan in detail. Cochrane researchers use the protocol to describe the

proposed approach for a systematic review. It outlines: the question that the review authors are addressing (the

health problem and the intervention under investigation, how benefits and harms will be measured, and the type of appropriate study design, such as the inclusion criteria of the studies).

The protocol also outlines the process for identifying, assessing, and summarizing studies in the review.

By making this information available the protocol is a public record, published in the CDSR

Page 28: Ilaria Casetta, MD Ilaria Casetta, MD Neurological Clinic University of Ferrara Copyright © The Cochrane Collaboration.

Systematic process involving several steps

Formulate the question Plan the review (a priori protocol) Comprehensive search Unbiased selection and abstraction process Critical appraisal of data Synthesis of data (may include meta-analysis) Interpretation of results

All steps are described explicitly in the review.

Page 29: Ilaria Casetta, MD Ilaria Casetta, MD Neurological Clinic University of Ferrara Copyright © The Cochrane Collaboration.

Clearly stated title and objectives

Comprehensive strategy to search for relevant studies

(unpublished and published)

Explicit and justified criteria for the inclusion or exclusion

of any study

Clear presentation of characteristics of each study

included and an analysis of methodological quality

Comprehensive list of all studies excluded and

justification for exclusion

Page 30: Ilaria Casetta, MD Ilaria Casetta, MD Neurological Clinic University of Ferrara Copyright © The Cochrane Collaboration.

Comprehensively search databases to find all relevant studies PubMed EMBASE Cochrane Central Register of Controlled Trials (CENTRAL) CINAHL Web of Science PsycINFO LILACS Databases from LMIC Dissertation abstracts Clinical trials registers Handsearching of journals Reference lists Conference abstracts

Personal communication Anything else you can think of.

Page 31: Ilaria Casetta, MD Ilaria Casetta, MD Neurological Clinic University of Ferrara Copyright © The Cochrane Collaboration.

Database Bias – Different database can record different studies “

Publication Bias - selective publication of articles that show positive effect of

treatment and statistical significance.

English-language bias - occurs when reviewers exclude papers published in

languages other than English

Citation bias - occurs when studies with significant or positive results are referenced

in other publications, compared with studies with inconclusive or negative findings

Page 32: Ilaria Casetta, MD Ilaria Casetta, MD Neurological Clinic University of Ferrara Copyright © The Cochrane Collaboration.

Systematic process involving several steps

Formulate the question Plan the review (a priori protocol) Comprehensive search Unbiased selection and abstraction process Critical appraisal of data Synthesis of data (may include meta-analysis) Interpretation of results

All steps are described explicitly in the review.

Page 33: Ilaria Casetta, MD Ilaria Casetta, MD Neurological Clinic University of Ferrara Copyright © The Cochrane Collaboration.

Selection process- Refine SearchBy excluding Duplicates Irrelevant Papers Studies not fulfilling

inclusion criteria

Obtaining full-text of relevant studies

1814 records

7 included

Page 34: Ilaria Casetta, MD Ilaria Casetta, MD Neurological Clinic University of Ferrara Copyright © The Cochrane Collaboration.

Abstraction process

Using a pre-define data abstraction form, authors inependently abstract the data and study characteristics from each study determined to be eligible for inclusion.

Page 35: Ilaria Casetta, MD Ilaria Casetta, MD Neurological Clinic University of Ferrara Copyright © The Cochrane Collaboration.

Systematic process involving several steps

Formulate the question Plan the review (a priori protocol) Comprehensive search Unbiased selection and abstraction process Critical appraisal of data Synthesis of data (may include meta-analysis) Interpretation of results

All steps are described explicitly in the review.

Page 36: Ilaria Casetta, MD Ilaria Casetta, MD Neurological Clinic University of Ferrara Copyright © The Cochrane Collaboration.

Assessment1. Sequence generation

(randomization)

2. Allocation concealment

3. Blinding of participants, personnel and outcomes

4. Incomplete outcome data (attrition and exclusions)

5. Selective outcome reporting

6. Other (including topic-specific, design-specific)

Ad

eq

ua

te s

eq

ue

nce

ge

ne

ratio

n

Barry 1988 +

Baylis 1989 +

Cooper 1987 +

Dodd 1985 +

Goodwin 1986 +

Sanders 1983 +

Allo

ca

tio

n c

on

ce

alm

en

t

-

+

?

?

+

+

Blin

din

g (

Pa

tie

nt-

rep

ort

ed

ou

tco

me

s)

+

+

-

+

+

-

Blin

din

g (

Mo

rta

lity

)

+

+

?

+

+

?

Inco

mp

lete

ou

tco

me

da

ta a

dd

resse

d (

Sh

ort

-te

rm o

utc

om

es (

2-6

wks))

-

?

-

+

+

-

Inco

mp

lete

ou

tco

me

da

ta a

dd

resse

d (

Lo

ng

er-

term

ou

tco

me

s (

> 6

ks))

-

?

-

-

+

-

Fre

e o

f se

lective

re

po

rtin

g

-

+

+

?

+

-

Fre

e o

f o

the

r b

ias

-

?

?

?

+

-

Risk of bias

Page 37: Ilaria Casetta, MD Ilaria Casetta, MD Neurological Clinic University of Ferrara Copyright © The Cochrane Collaboration.

Critical appraisal and assessment of the risk of bias for each study

Page 38: Ilaria Casetta, MD Ilaria Casetta, MD Neurological Clinic University of Ferrara Copyright © The Cochrane Collaboration.

Systematic process involving several steps

Formulate the question Plan the review (a priori protocol) Comprehensive search Unbiased selection and abstraction process Critical appraisal of data Synthesis of data (may include meta-analysis) Interpretation of results

All steps are described explicitly in the review.

Page 39: Ilaria Casetta, MD Ilaria Casetta, MD Neurological Clinic University of Ferrara Copyright © The Cochrane Collaboration.

Synthesis of data

“Once the data have been extracted and their quality and validity assessed, the outcomes of individual studies within a systematic review may be pooled and presented as summary outcome or effect

When data are NOT too sparse, of too low quality or too heterogeneous

Page 40: Ilaria Casetta, MD Ilaria Casetta, MD Neurological Clinic University of Ferrara Copyright © The Cochrane Collaboration.

Synthesis of data when:

Studies evaluated the same outcomes

Homogeneity

Good Quality

Page 41: Ilaria Casetta, MD Ilaria Casetta, MD Neurological Clinic University of Ferrara Copyright © The Cochrane Collaboration.

Meta-analysis

“Meta-analysis is a statistical technique for combining the results of independent, but similar, studies to obtain an overall estimate of treatment effect.”

The validity of a meta-analysis depends on the quality of the studies included,

Page 42: Ilaria Casetta, MD Ilaria Casetta, MD Neurological Clinic University of Ferrara Copyright © The Cochrane Collaboration.

Meta-analysis- Forest plot

At the bottom there’s a horizontal line. This is the scale measuring the treatment effect

The vertical line in the middle is where the treatment and control have the same effect

Page 43: Ilaria Casetta, MD Ilaria Casetta, MD Neurological Clinic University of Ferrara Copyright © The Cochrane Collaboration.

Take care to read what the labels say – things tothe left do not always mean the treatment is better than the

control.

Page 44: Ilaria Casetta, MD Ilaria Casetta, MD Neurological Clinic University of Ferrara Copyright © The Cochrane Collaboration.

Each study is given a blob, placed where the data measure the effect.

The size of the blob is proportional to the % weight The horizontal line is called a confidence interval and is a measure of how we think the result of this study might vary by chance.

The wider the horizontal line is, the less confident we are of the observed effect.

Page 45: Ilaria Casetta, MD Ilaria Casetta, MD Neurological Clinic University of Ferrara Copyright © The Cochrane Collaboration.

If the confidence interval crosses the line of no effect, we have found no statistically significant difference in the effects of the two interventions

The pooled analysis is given a diamond shape where the widest bit in the middle is located at the calculated best guess (point estimate), and the horizontal width is the confidence interval

Page 46: Ilaria Casetta, MD Ilaria Casetta, MD Neurological Clinic University of Ferrara Copyright © The Cochrane Collaboration.

Systematic process involving several steps

Formulate the question Plan the review (a priori protocol) Comprehensive search Unbiased selection and abstraction process Critical appraisal of data Synthesis of data (may include meta-analysis) Interpretation of results

All steps are described explicitly in the review.

Page 47: Ilaria Casetta, MD Ilaria Casetta, MD Neurological Clinic University of Ferrara Copyright © The Cochrane Collaboration.

Interpretation of the results

What does it all mean?

Implications for patient care and future research

Page 48: Ilaria Casetta, MD Ilaria Casetta, MD Neurological Clinic University of Ferrara Copyright © The Cochrane Collaboration.

The Cochrane Library databases

The CDSR is published online monthly (from 2010) with quarterly DVDs.Access is free for many people in low-income and middle-income countries via a number of initiatives. This is in addition to general access options, including national licenses and subscriptions.

Abstracts are free

Page 49: Ilaria Casetta, MD Ilaria Casetta, MD Neurological Clinic University of Ferrara Copyright © The Cochrane Collaboration.

"Every day someone, somewhere searches The Cochrane Library every second, reads an abstract every two seconds and downloads a full-text article every three seconds."

The Cochrane Library usage data , 2009

Page 50: Ilaria Casetta, MD Ilaria Casetta, MD Neurological Clinic University of Ferrara Copyright © The Cochrane Collaboration.

Novel methodologies

To summarize evidence across reviews Overview of Reviews Network Meta-Analysis