Making research & reporting better using the ORION statement to design or evaluate

infection control intervention studies and outbreak reports

Using the short “ORION Abstract” for conference and journal abstract submission & review

Dr Sheldon Stone, Royal Free Campus, University College London Medical School, UK

Professor Barry Cookson, HPA and LSHTMOn behalf of ORION group

Session Goals: This interactive session will : 1.   Help you write a high quality conference/journal abstract 2.    Provide you with a framework to help referee a conference abstract

3.   Help you select the best papers/conference presentations to inform continuing professional development.. what article to read?

what conference oral/poster session to go to? 4. Give us feedback on how to improve the abstract tool (short ORION) and how to teach it

5. Encourage you to study the full ORION statement for papers and grant applications www.idrn.org/orion.php

Meet the Expert Outline

1) Brief summary of ORION ~10minutes

2) We provide:

• the ORION abstract checklist, • four Abstracts:

o An antibiotic stewardship intervention

o A Hand Hygiene trial

o Two Outbreak Reports

Outline (2)

• Working alone or with neighbours:

• 5 mins to read first abstract: Stone et al Age & Aging 1998 • 5 mins to assess with the ORION abstract check-list• 5 mins individual feed-back by show of hands

• Repeat above for second abstract: P1381 ECCMID

• Repeat above for third abstract: Lopansri et al 2010

• Repeat if time for fourth abstract: Dror Marchaim et al 2010

The ORION statement: Guidelines for transparent reporting of

Outbreak Reports & Intervention studies Of Nosocomial Infection

A CONSORT equivalent for Infection Control Studies

Funded by Health Technology Assessement Board

Stone et al Lancet Infect Dis 2007; J Antimicrob Chemother 2007www.idrn.org/orion.php

Co-authors & Collaborating Institutions

• Ben Cooper Stats/Modelling

• Chris Kibbler Microbiology

• Jenny Roberts Health Economics

• Graham Medley Modelling

• GeorgiaDuckworth Public

Health

• Rosalind Lai Library Sciences

• Shah Ebrahim Epidemiology,

EBM

• Erwin Brown Microbiology

• Phil Wiffen EBM

• Peter Davey Infectious Diseases

Pharmaco-economics

Royal Free&University College Medical SchoolHealth Protection Agency, Colindale London School Hygiene & Tropical Medicine Warwick UniversityFrenchay Hospital, Bristol UK Cochrane Centre, Oxford ; University of Dundee Medical School

Evidence Base for Infection Control InterventionsDavey et al Cochrane 2005;Cooper et al BMJ 2004

• Cochrane review of interventions to change antibiotic prescription & evaluate HCAI outcomes (2005) & HTA (2003) review isolation practices in MRSA show limited evidence of some effect but inadequate reporting & major flaws in design & statistical analysis

• Lack of details eg on interventions & timings• Failure to assess & adjust for confounders/biases • Aggregation of outcomes (misses trends)• Analysis fails to account for dependencies of infectious outcomes

• Quality of infection control research must improve to provide robust evidence for policy & practice

To summarise the problem…..Cooper B et al BMJ 2004, HTA 20003, Davey et al Cochrane

2005;Ramsay et al JAC 2003

• Studies conclude interventions cause MRSA or antibiotic use or Clostridium difficile

• Validity of conclusions threatened by confounders & biases, unaccounted for in studies, which provide plausible alternative explanations of outcome and by inappropriate statistics e.g. aggregation of data (misses time trends) & assumption that infection outcomes are independent (Chi-Sq; OR)

The sort of problems: regression to mean, statistical analysis

Onesko KM, Infection Control 1987

Nosocomial MRSA (infections?). ICU & 4 South ward combined.

00.51

1.52

2.53

3.5

Aug Oct Dec Feb Apr Jun Aug Oct Dec Feb Apr Jun

% o

f pat

ien

ts

1983 1984 1985

Non-medicated soap Low-iodine soap

Interrupted time series

AIM OF ORION StatementCONSORT equivalent for infection control studies

• Improve standards research & publication• Transparency of reporting• Readers relate studies to their situation.• Facilitate synthesis of evidence • Framework for reviewers & editors to assess papers • Criteria research grant assessment panels

• Designed especially for Interrupted Time Series (with or without controls groups) and outbreak reports.

Key issues addressed by ORION

Transparency: Why was the study done? (hypothesis) What sort of study? (design) Exactly what was done, to whom, when?

Analysis: Disaggregated data Account for dependencies Confounders

Inference: How do findings relate to hypothesis? What else influenced the findings? Do findings generalise ?

Components of ORIONStone et al Lancet ID 2007;JAC 2007;www.idrn.org/orion.php

• 22 item checklist Title Abstract Introduction Methods Results Discussion

• Summary table Population Clinical setting Precise nature & timing of all interventions

• Graphical summary results

• adapted CONSORT statement to the wide variety settings interventions, designs & statistical issues infection control studies & outbreak reports

• Consultation with professional

societies

• Independent academic review in two journals

ORION exemplar paper….. .Fowler S et al JAC 2007

CONSORT journal & conference abstract checklistHopewell et al PLOS Med 2008

• ORION is a CONSORT equivalent for Infection Control and Antimicrobial Stewardship interventions & Outbreak Reports

• CONSORT for ABSRACTS : for submission & review of conference & journal abstracts of RCTs (same flaws as full articles)

• Aim to improve reporting of RCTs in journal abstracts & conference proceedings, giving readers the detail & clarity to assess validity & applicability of RCT and help them decide which full articles to read or conference presentations to attend

• ORION followed suit

• Used for SHEA Conference 2011

SUBMISSION & REVIEWER’S ABSTRACT CHECKLIST FOR INTERVENTION STUDIES

AND OUTBREAK REPORTS

Title 1.Clear statement that this is an intervention study or outbreak report.

Background 2.Rationale for study with clear hypothesis for intervention studies or objective for outbreak reports

Methods- 3.Clear statement of intervention study design1 or case and outbreak definition for outbreak report.

4.Brief description of participants, setting and of intervention or

outbreak control measures (with start & stop dates)

5. Clearly defined outcomes & denominators at regular time intervals2,

not as totals for each phase

(can be put in results instead)

6. Statistical analysis accounts for any dependencies in the data (can

be in results instead) ( analysis may not be appropriate for OR)

ABSTRACT CHECKLIST (cont)

Methods (cont)

7. Which potential confounders or biases were considered, recorded or

adjusted for3 (can be in results instead) 8. Where relevant: details of culture, typing, environmental sampling,

& risk factors for acquisition, root cause analysis or organisational risk assessment.

Results- 9. For main outcomes: estimated effect size & its precision (usually 95% CI)

(A graphical summary is often appropriate eg for most time series).

Conclusions 10. For intervention studies: consider in relation to original hypothesis, accounting for potential confounders & biases.

For outbreak reports: consider clinical significance of observations & hypothesis to explain them.

Stone et al abstract• Title- not explicit that intervention

study β

• Background- rationale but no clear hypothesis β

• Methods-

design: not stated β

• Brief description

of intervention but not setting, participants or dates β

• Outcomes-

(I) no regular time intervals ie not disaggregated β

(ii) clear CDI outcomes but not AB

• Statistical analysis:

fail to account for dependencies β (segmental regression for AB & Poisson for CDI)

• Confounders/bias

- no mention β

• Culture, typing etc

- N/R

• Results-

no estimated size effect &

precision, no graphical summary β

• Conclusions-

no original hypothesis …. β

P1381 Feedback Intervention Trial (poster this afternoon)

• Title-

RCT α

• Background- rationale but no explicit hypothesis

+

• Methods-

-design: clear α

-setting, participants, intervention, dates α

• Outcomes-

- clear primary & secondaryα

-regular time points (monthly- fig) α

- clear denominators α

• Statistical analysis:

accounts for dependencies α

• Confounders/bias

- mentioned & accounted for α

• Culture, typing etc- N/R

• Results- estimated size effect &

precision, graphical summary α

• Conclusions- no explicit hypothesis but

accounted for confounders

(implementation) …. +

Outbreak abstract 1Lopansri et al HPA SHEA 2010

• Title: outbreak report α

• Background: rationale and objectives α

• Methods- -case definition α -no outbreak definition -hospital setting & outbreak control

measures but not ward briefly described but little detail on participants +

- start & stop dates for original index cases but not for whole study +

- outcomes & stat analysis : NRα

• Methods (cont)-

-Risk factors analysed but are there others that should be mentioned as not having been present +

More details on micro/strain? Information on C&S from samples from

flash sterilised equipment?

• Results- -OK but graph might have been helpfulα-

• Conclusions- - Clinical significance observation &

explanatory hypothesis α

Outbreak abstract 2Marchaim D et al SHEA 2010

• Title: outbreak in titleα

• Background: rationale and objectiveα • Methods- -clear case definitionα -participants settings & outbreak control

measures not described but LOS, age, mortality given

- start & stop dates of outbreak given but not of study +

- outcomes: chart would have been useful especially as shared wards presented as important

• Methods (cont)- - stats:NRα -Confounders:NR α -Cultures etc : other risk factors (eg

patients characteristics, environmental contamination) not mentioned

• Results- -chart would have been useful

-colistin agrregated

• Conclusions- - Significance not stated. No hypothesis

generated. Effect of control measures stated with no evidence

Reflections 1

• Has the session achieved its original aims? 1.   Helping you write high quality conference or journal

abstracts 2.  Providing you with a framework to help referee a conference abstract 3.   Helping inform your CPD with a tool to help select

-which full articles to read -which conference sessions to hear

• Was this appropriate for a “meet the expert” session?

• Tool and meet the expert session materials is on website www.idrn.org/orion.php

Reflections 2

• Did ORION for abstracts work for intervention studies?• Did ORION for abstracts work in outbreak reports?

• Does it need an explanatory document like CONSORT with examples of compliant & non compliant practice

• Does it need more slightly more submission space for journals (2500 characters works for conferences)

• What doesn’t the checklist cover?

• What would make it better?

Thank you

Comments & suggestions to

s.stone@medsch.ucl.ac.uk

Barry.cookson@hpa.org.uk