Research article structure: W here can reporting guidelines help? Iveta Simera
Guidelines for Reporting Research
Transcript of Guidelines for Reporting Research
![Page 1: Guidelines for Reporting Research](https://reader036.fdocuments.us/reader036/viewer/2022081421/554b3217b4c905d3088b50b9/html5/thumbnails/1.jpg)
C SM
CC SSMM
Guidelines for reporting research
Doug AltmanCentre for Statistics in
Medicine Oxford
NCRI, 24 January 2006
![Page 2: Guidelines for Reporting Research](https://reader036.fdocuments.us/reader036/viewer/2022081421/554b3217b4c905d3088b50b9/html5/thumbnails/2.jpg)
C SM
CC SSMM
Biased reporting is scientific misconduct
“Failure to publish an adequate account of a well-designed clinical trial is a form of scientific misconduct which can lead those caring for patients to make inappropriate treatment decisions.”
[Chalmers, 1990]
![Page 3: Guidelines for Reporting Research](https://reader036.fdocuments.us/reader036/viewer/2022081421/554b3217b4c905d3088b50b9/html5/thumbnails/3.jpg)
C SM
CC SSMM
Clinical trials
Focus on randomised clinical trials (RCTs)– Principles of good reporting apply to all types of
research
![Page 4: Guidelines for Reporting Research](https://reader036.fdocuments.us/reader036/viewer/2022081421/554b3217b4c905d3088b50b9/html5/thumbnails/4.jpg)
C SM
CC SSMM
Outline
Importance of clinical trials Importance of good reporting Evidence of bad reporting Reporting guidelines - CONSORT Publication (dissemination) bias
– Bias from unpublished research– Bias in published research
![Page 5: Guidelines for Reporting Research](https://reader036.fdocuments.us/reader036/viewer/2022081421/554b3217b4c905d3088b50b9/html5/thumbnails/5.jpg)
C SM
CC SSMM
Chalmers & Altman. How can medical journals help prevent poor medical research? Some
opportunities presented by electronic publishing. Lancet 1999
Electronic publication of a protocol could be simply the first element in a sequence of “threaded” electronic publications, which continues with reports of the resulting research (published in sufficient detail to meet some of the criticisms of less detailed reports published in print journals), followed by deposition of the complete data set. Not only would this approach allow alternative explorations of the data, it would help to address some of the growing concerns about research misconduct.
![Page 6: Guidelines for Reporting Research](https://reader036.fdocuments.us/reader036/viewer/2022081421/554b3217b4c905d3088b50b9/html5/thumbnails/6.jpg)
C SM
CC SSMM
Data for clinical trials
Knowledge of existence (registration) Trial protocol
– Including detailed analysis plan Results of analyses Raw data Other documents
– e.g. Data collection forms
![Page 7: Guidelines for Reporting Research](https://reader036.fdocuments.us/reader036/viewer/2022081421/554b3217b4c905d3088b50b9/html5/thumbnails/7.jpg)
C SM
CC SSMM
Importance of good reporting
“The whole of medicine depends on the transparent reporting of
clinical trials”
Rennie D. CONSORT revised—improving the reporting of randomized trials. JAMA
2001;285:2006-7.
![Page 8: Guidelines for Reporting Research](https://reader036.fdocuments.us/reader036/viewer/2022081421/554b3217b4c905d3088b50b9/html5/thumbnails/8.jpg)
C SM
CC SSMM
Importance of good reporting
Assessment of quality and relevance of research is seriously impeded by inadequate reporting
Randomised controlled trials (RCTs) should be reported fully and accurately– Existence of trials and main findings, to ensure that we
see all the evidence, not a biased subset– Clear details of what was done, to enable evaluation of
the reliability of findings – Full reporting of all results (outcomes), to contribute to
the overall evidence of interventions (benefits and harms)
Much evidence indicates that all areas are deficient
![Page 9: Guidelines for Reporting Research](https://reader036.fdocuments.us/reader036/viewer/2022081421/554b3217b4c905d3088b50b9/html5/thumbnails/9.jpg)
C SM
CC SSMM
Poor reporting of RCTs is common 1
Not reporting an adequate method for generating random numbers – 68% of 206 trials in obstetrics & gynecology journals– 52% of 80 trials in general medical journals
Not reporting the mechanism used to allocate interventions – 89% of 196 trials in rheumatoid arthritis – 48% of 206 trials in obstetrics & gynecology journals– 44% of 80 trials in general medical journals– 93% of 73 trials in one dermatology journal – 96% of 2000 trials in schizophrenia– 99% of 122 trials of SSRIs
![Page 10: Guidelines for Reporting Research](https://reader036.fdocuments.us/reader036/viewer/2022081421/554b3217b4c905d3088b50b9/html5/thumbnails/10.jpg)
C SM
CC SSMM
Poor reporting of RCTs is common 2
Not stating whether blinding was used– 51% of 506 trials in cystic fibrosis– 33% of 196 trials in rheumatoid arthritis– 38% of 68 trials in dermatology
Poor reporting of adverse effects of interventions– 61% of 192 reports in 7 medical areas
![Page 11: Guidelines for Reporting Research](https://reader036.fdocuments.us/reader036/viewer/2022081421/554b3217b4c905d3088b50b9/html5/thumbnails/11.jpg)
C SM
CC SSMM
The CONSORT statement[Begg et al 1996; Moher et al 2001]
22 items which should be reported in a paper
Also a flow diagram showing patient progress through the trial (to be included in the trial report)
Evidence-based, whenever possible Accompanied by a long explanatory
document
www.consort-statement.org
![Page 12: Guidelines for Reporting Research](https://reader036.fdocuments.us/reader036/viewer/2022081421/554b3217b4c905d3088b50b9/html5/thumbnails/12.jpg)
C SM
CC SSMM
ExcludedExcluded
Not meeting inclusion criteriaNot meeting inclusion criteria
Refused to participateRefused to participate
Other reasonOther reason
Assessed Assessed for eligibilityfor eligibility
(n=…)
RandomizedRandomized
Allocated to interventionAllocated to intervention
Received allocated interventionReceived allocated intervention
Did not receive allocatedDid not receive allocated intervention (give reasons) intervention (give reasons)
Lost to follow upLost to follow up
Discontinued intervention Discontinued intervention (give reasons)(give reasons)
AnalysedAnalysed
Excluded from analysisExcluded from analysis
Allocated to interventionAllocated to intervention
Received allocated interventionReceived allocated intervention
Did not receive allocatedDid not receive allocated intervention (give reasons) intervention (give reasons)
Lost to follow upLost to follow up
Discontinued intervention Discontinued intervention (give reasons)(give reasons)
AnalysedAnalysed
Excluded from analysisExcluded from analysis
Anal
ysis
Anal
ysis
Follo
w up
Follo
w up
Allo
catio
nAl
loca
tion
Enro
llmen
tEn
rollm
ent
![Page 13: Guidelines for Reporting Research](https://reader036.fdocuments.us/reader036/viewer/2022081421/554b3217b4c905d3088b50b9/html5/thumbnails/13.jpg)
C SM
CC SSMM
CONSORT checklist (22 items)TITLE & ABSTRACTINTRODUCTION BackgroundMETHODS Participants Interventions Objectives Outcomes Sample size Randomization
Sequence allocation Allocation concealment Implementation
Blinding (Masking) Statistical methods
RESULTS Participant flow Recruitment Baseline data Numbers analyzed Outcomes and
Estimation Ancillary analyses Adverse eventsDISCUSSION Interpretation Generalisability Overall evidence
![Page 14: Guidelines for Reporting Research](https://reader036.fdocuments.us/reader036/viewer/2022081421/554b3217b4c905d3088b50b9/html5/thumbnails/14.jpg)
C SM
CC SSMM
CONSORT items
There are really >22 items
Item 6:– Clearly defined primary and secondary outcome
measures and, when applicable, any methods used to enhance the quality of measurements (e.g., multiple observations, training of assessors).
![Page 15: Guidelines for Reporting Research](https://reader036.fdocuments.us/reader036/viewer/2022081421/554b3217b4c905d3088b50b9/html5/thumbnails/15.jpg)
C SM
CC SSMM
Avoid unhelpful labels
CONSORT does not use term “double blind”
Item 11:– Whether or not participants, those administering
the interventions, and those assessing the outcomes were blinded to group assignment.
![Page 16: Guidelines for Reporting Research](https://reader036.fdocuments.us/reader036/viewer/2022081421/554b3217b4c905d3088b50b9/html5/thumbnails/16.jpg)
C SM
CC SSMM
Secondary goal of CONSORT
“To encourage and provide incentives for researchers to conduct high-quality, unbiased randomized trials”– Arguably even more important
![Page 17: Guidelines for Reporting Research](https://reader036.fdocuments.us/reader036/viewer/2022081421/554b3217b4c905d3088b50b9/html5/thumbnails/17.jpg)
C SM
CC SSMM
Extensions
Cluster RCTs (BMJ 2004) Harms (Ann Intern Med 2004) Noninferiority RCTs (JAMA 2006)
More planned
![Page 18: Guidelines for Reporting Research](https://reader036.fdocuments.us/reader036/viewer/2022081421/554b3217b4c905d3088b50b9/html5/thumbnails/18.jpg)
C SM
CC SSMM
Factors in success of CONSORT
Membership of Group– Methodologists– Trialists– Editors
Reporting rather than conduct Focus on main issues
– ‘One side of paper’ No competitors High profile publications Supported by Editorial groups, >150
journals, peer review granting agencies
![Page 19: Guidelines for Reporting Research](https://reader036.fdocuments.us/reader036/viewer/2022081421/554b3217b4c905d3088b50b9/html5/thumbnails/19.jpg)
C SM
CC SSMM
Process 1
Preparation– Assessment of existing literature on reporting– Identification of relevant empirical evidence
Meeting of experts– Clear scope– Clear aim – reporting not conduct– Develop checklist (& flow diagram)
Small group fine-tunes checklist and develops publications– Seek input from wider group (e.g. on web)
![Page 20: Guidelines for Reporting Research](https://reader036.fdocuments.us/reader036/viewer/2022081421/554b3217b4c905d3088b50b9/html5/thumbnails/20.jpg)
C SM
CC SSMM
Process 2
Publications– Statement– Explanatory document
Web site – Extra material– Invite comments– etc
Periodic revision – Take account of feedback / criticism– Take account of new evidence
NB no funding!!
![Page 21: Guidelines for Reporting Research](https://reader036.fdocuments.us/reader036/viewer/2022081421/554b3217b4c905d3088b50b9/html5/thumbnails/21.jpg)
C SM
CC SSMM
Some history of reporting guidelines developed by
consensus 1993-94 SORT RCT1994 Asilomar RCT1995-96 CONSORT RCT1996-99 QUOROM SR/M-A of RCTs1997-00 MOOSE M-A of obs studies1999-01 CONSORT II RCT2000-03 STARD Diagnostic2003-04 TREND NonRCT/Behavioural2001-05 REMARK Prognostic (cancer)2004-?? STROBE Observational 2005-?? QUOROM II SR/M-A of RCTs
![Page 22: Guidelines for Reporting Research](https://reader036.fdocuments.us/reader036/viewer/2022081421/554b3217b4c905d3088b50b9/html5/thumbnails/22.jpg)
C SM
CC SSMM
Impact of CONSORT
Review of ‘Instructions to Authors’ of 167 high impact medical journals in 2003 – 36/166 (22%) referred to CONSORT
8 published studies show that adoption of CONSORT by journals is associated with improved reporting of RCTs– Much room remains for improvement
![Page 23: Guidelines for Reporting Research](https://reader036.fdocuments.us/reader036/viewer/2022081421/554b3217b4c905d3088b50b9/html5/thumbnails/23.jpg)
C SM
CC SSMM
![Page 24: Guidelines for Reporting Research](https://reader036.fdocuments.us/reader036/viewer/2022081421/554b3217b4c905d3088b50b9/html5/thumbnails/24.jpg)
C SM
CC SSMM
Publication (dissemination) bias
Study not published Only selected findings published Stopping a trial early for benefit
![Page 25: Guidelines for Reporting Research](https://reader036.fdocuments.us/reader036/viewer/2022081421/554b3217b4c905d3088b50b9/html5/thumbnails/25.jpg)
C SM
CC SSMM
Outcomes
Publication
Outcomes
From trial to review
Trial 2Trial 1 Trial 3
Systematic review
Publication Publication
Outcomes
![Page 26: Guidelines for Reporting Research](https://reader036.fdocuments.us/reader036/viewer/2022081421/554b3217b4c905d3088b50b9/html5/thumbnails/26.jpg)
C SM
CC SSMM
Publication biases
Study publication bias selective publication of entire studies
Outcome reporting bias selective reporting of outcomes within
published studies
![Page 27: Guidelines for Reporting Research](https://reader036.fdocuments.us/reader036/viewer/2022081421/554b3217b4c905d3088b50b9/html5/thumbnails/27.jpg)
C SM
CC SSMM
Study publication bias
Studies reported at conferences are less likely to be fully published if not significant– Statistically significant results are about 20% more
likely to be published [Scherer et al, CDMR 2004]
Even when published nonsignificant studies take longer to reach publication that those with significant findings– Trials with null or negative findings took an average
just over a year longer to be published than those with positive results [Hopewell et al, CDMR 2001]
![Page 28: Guidelines for Reporting Research](https://reader036.fdocuments.us/reader036/viewer/2022081421/554b3217b4c905d3088b50b9/html5/thumbnails/28.jpg)
C SM
CC SSMM
Full publication bias (abstracts not leading to full
papers)
Cohort of 500 oncology trials with >200 participants [Krzyzanowska et al, JAMA 2003]
Preliminary results presented at ASCO 26% not published in full within 5 years
– 81% with P<0.05 were published– 68% with P>0.05 were published
So not just a problem of small trials!
![Page 29: Guidelines for Reporting Research](https://reader036.fdocuments.us/reader036/viewer/2022081421/554b3217b4c905d3088b50b9/html5/thumbnails/29.jpg)
C SM
CC SSMM
3 studies of selective reporting of trial results [Chan et al, 2004
& 2005]
• Study of published reports of RCTs– All Medline-indexed RCTs published in Dec 2000
• 519 RCTs, >10000 outcomes
Comparison of protocols and publications– 102 RCTs submitted to the Copenhagen and
Frederiksberg Research Ethics committee in 1994-95
– 48 RCTs funded by CIHR 1990-98
Questionnaire sent to all authors in each cohort
![Page 30: Guidelines for Reporting Research](https://reader036.fdocuments.us/reader036/viewer/2022081421/554b3217b4c905d3088b50b9/html5/thumbnails/30.jpg)
C SM
CC SSMM
Pooled OR for outcome reporting bias (fully versus incompletely reported
outcomes) by study design and cohort
![Page 31: Guidelines for Reporting Research](https://reader036.fdocuments.us/reader036/viewer/2022081421/554b3217b4c905d3088b50b9/html5/thumbnails/31.jpg)
C SM
CC SSMM
Impact of these biases
Study publication bias: odds ratio of 2.0– 81% of studies with P<0.05 were published– 68% of studies with P>0.05 were published
Selective reporting: odds ratio of 2.4– 71% of outcomes with P<0.05 are fully reported
– 50% of outcomes with P>0.05 are fully reported
Biases are cumulative– Hard to quantify overall impact
![Page 32: Guidelines for Reporting Research](https://reader036.fdocuments.us/reader036/viewer/2022081421/554b3217b4c905d3088b50b9/html5/thumbnails/32.jpg)
C SM
CC SSMM
Major discrepancies in the specification of primary outcomes
between protocols and publications
![Page 33: Guidelines for Reporting Research](https://reader036.fdocuments.us/reader036/viewer/2022081421/554b3217b4c905d3088b50b9/html5/thumbnails/33.jpg)
C SM
CC SSMM
Protocols
Clearly, trial protocols need to be in the public domain– Preferably before the trial finishes
![Page 34: Guidelines for Reporting Research](https://reader036.fdocuments.us/reader036/viewer/2022081421/554b3217b4c905d3088b50b9/html5/thumbnails/34.jpg)
C SM
CC SSMM
Biased reporting is scientific misconduct
“In return for the altruism and trust that make clinical research possible, the research enterprise has an obligation to conduct research ethically and to report it honestly.”
[“Clinical Trial Registration: A Statement from the International Committee of Medical Journal Editors”, September 2004]
![Page 35: Guidelines for Reporting Research](https://reader036.fdocuments.us/reader036/viewer/2022081421/554b3217b4c905d3088b50b9/html5/thumbnails/35.jpg)
C SM
CC SSMM
Data for clinical trials
Knowledge of existence (registration) Trial protocol
– Including detailed analysis plan Results of analyses Raw data Other documents
– e.g. Data collection forms
![Page 36: Guidelines for Reporting Research](https://reader036.fdocuments.us/reader036/viewer/2022081421/554b3217b4c905d3088b50b9/html5/thumbnails/36.jpg)
C SM
CC SSMM