Systematic Reviews Practicalities and Realities
description
Transcript of Systematic Reviews Practicalities and Realities
![Page 1: Systematic Reviews Practicalities and Realities](https://reader035.fdocuments.us/reader035/viewer/2022062519/568156f6550346895dc49c8d/html5/thumbnails/1.jpg)
Systematic Reviews Practicalities and Realities
Alison Brettle, Research Fellow (Information)
Salford Centre for Nursing, Midwifery and Collaborative Research
University of Salford
![Page 2: Systematic Reviews Practicalities and Realities](https://reader035.fdocuments.us/reader035/viewer/2022062519/568156f6550346895dc49c8d/html5/thumbnails/2.jpg)
Session Overview
Practical overview of systematic reviews and what you need to conduct one
Discussion
![Page 3: Systematic Reviews Practicalities and Realities](https://reader035.fdocuments.us/reader035/viewer/2022062519/568156f6550346895dc49c8d/html5/thumbnails/3.jpg)
What is a systematic review?
An overview of primary research studies conducted according to explicit and reproducible methodology
A rigorous method of summarising research evidence
Shows what we know and don’t know about a topic area
Provides evidence of effectiveness (or not) by summarising and appraising relevant evidence
![Page 4: Systematic Reviews Practicalities and Realities](https://reader035.fdocuments.us/reader035/viewer/2022062519/568156f6550346895dc49c8d/html5/thumbnails/4.jpg)
Systematic reviews aim
To find all relevant research studies (published and unpublished)
To assess each study on basis of defined criteria
Synthesise the findings in an unbiased way
Present a balanced and impartial summary of the findings taking any flaws into consideration
![Page 5: Systematic Reviews Practicalities and Realities](https://reader035.fdocuments.us/reader035/viewer/2022062519/568156f6550346895dc49c8d/html5/thumbnails/5.jpg)
Systematic review models
Medical/Health care Cochrane Collaboration, NHS Centre for
Reviews and Dissemination Usually includes “high quality” research
evidence – RCTs Often includes meta-analysis (mathematical
synthesis of results of 2+ studies that addressed same hypothesis in same way)
Social care/Social Sciences SCIE, EPPI Centre, Campbell Collaboration Often include wider range of studies including
qualitative Often narrative synthesis of evidence
![Page 6: Systematic Reviews Practicalities and Realities](https://reader035.fdocuments.us/reader035/viewer/2022062519/568156f6550346895dc49c8d/html5/thumbnails/6.jpg)
Systematic review process
Define/focus the question Develop a protocol Search the literature (possibly 2 stages scoping
and actual searches) Refine the inclusion/exclusion criteria Assess the studies (data extraction tools, 2
independent reviewers) Combine the results of the studies to produce
conclusion– can be a qualitative or quantitative (meta-analysis)
Place findings in context – quality and heterogeniety of studies, applicability of findings
![Page 7: Systematic Reviews Practicalities and Realities](https://reader035.fdocuments.us/reader035/viewer/2022062519/568156f6550346895dc49c8d/html5/thumbnails/7.jpg)
Task
You are part of a systematic review team and this is your first meeting. Your task is to draft a protocol by the end of the meeting then present to the other groups
![Page 8: Systematic Reviews Practicalities and Realities](https://reader035.fdocuments.us/reader035/viewer/2022062519/568156f6550346895dc49c8d/html5/thumbnails/8.jpg)
Components of a protocol
Background/context leading to question Review question Inclusion/Exclusion criteria Methods for
Identifying evidence Selecting studies Extracting studies Assessing quality of studies Synthesising the results Disseminating the findings
![Page 9: Systematic Reviews Practicalities and Realities](https://reader035.fdocuments.us/reader035/viewer/2022062519/568156f6550346895dc49c8d/html5/thumbnails/9.jpg)
Lessons learned in systematic reviews
![Page 10: Systematic Reviews Practicalities and Realities](https://reader035.fdocuments.us/reader035/viewer/2022062519/568156f6550346895dc49c8d/html5/thumbnails/10.jpg)
Lindsey Dugdill, Alison Brettle, Claire Hulme, Serena McCluskey, Andrew Long
Workplace physical activity interventions: a systematic review of the research evidence
![Page 11: Systematic Reviews Practicalities and Realities](https://reader035.fdocuments.us/reader035/viewer/2022062519/568156f6550346895dc49c8d/html5/thumbnails/11.jpg)
Background
Policy Increased PA important for
prevention of chronic illness Trend on a worldwide and UK scale
that PA levels are declining
![Page 12: Systematic Reviews Practicalities and Realities](https://reader035.fdocuments.us/reader035/viewer/2022062519/568156f6550346895dc49c8d/html5/thumbnails/12.jpg)
Objectives To identify which types of workplace PA
intervention were effective in changing PA behaviour for different sectors of the workforce and different types of workplace
To ascertain which aspects of intervention design and delivery contributed to effectiveness and contributory motivators, barriers and facilitators
![Page 13: Systematic Reviews Practicalities and Realities](https://reader035.fdocuments.us/reader035/viewer/2022062519/568156f6550346895dc49c8d/html5/thumbnails/13.jpg)
Lesson 1. Focus the question
More focussed the question, easier to undertake literature search and decide on relevant studies
Need to maintain a balance between a clearly defined topic and a meaningful and relevant question
Can help scope the literature to help develop the question
Refine question as part of review process
![Page 14: Systematic Reviews Practicalities and Realities](https://reader035.fdocuments.us/reader035/viewer/2022062519/568156f6550346895dc49c8d/html5/thumbnails/14.jpg)
Focussing the question
Options: empirically led -> literature scoping ->
narrowing down topic on the basis of ‘what is there’ in the evidence base
theoretical and conceptual -> concept mapping, explore the meaning given to the concepts, embracing user and carer notions of desired outcomes (based on previous research) themselves central to a ‘social care’ perspective
Funder/policy driven
![Page 15: Systematic Reviews Practicalities and Realities](https://reader035.fdocuments.us/reader035/viewer/2022062519/568156f6550346895dc49c8d/html5/thumbnails/15.jpg)
Lesson 2. Scope the literature
Provides an indication of literature available and what needs to be examined
Helps focus the question Helps clarify initial inclusion and
exclusion criteria
![Page 16: Systematic Reviews Practicalities and Realities](https://reader035.fdocuments.us/reader035/viewer/2022062519/568156f6550346895dc49c8d/html5/thumbnails/16.jpg)
Lesson 3. Write a protocol
Plan what you are going to do in the review
Set out the background and objectives
Outline the resources you will use Establish inclusion/exclusion criteria How will data be extracted (what
will be extracted) How will you synthesise literature
![Page 17: Systematic Reviews Practicalities and Realities](https://reader035.fdocuments.us/reader035/viewer/2022062519/568156f6550346895dc49c8d/html5/thumbnails/17.jpg)
Literature search
12 electronic databases 1996-2007Selected websites of known organisationsCitation tracking
Could also use hand-searching of journals “grey” literature
![Page 18: Systematic Reviews Practicalities and Realities](https://reader035.fdocuments.us/reader035/viewer/2022062519/568156f6550346895dc49c8d/html5/thumbnails/18.jpg)
Lesson 4. Use multiple sources of information
Databases only part of the picture Handsearching identifies further
references Citation tracking adds more references SMI review – 96 studies (73 databases,
23 handsearching/citation tracking) Counselling – 47 studies (aprox 10
handsearching, 4 citation tracking)
![Page 19: Systematic Reviews Practicalities and Realities](https://reader035.fdocuments.us/reader035/viewer/2022062519/568156f6550346895dc49c8d/html5/thumbnails/19.jpg)
Lesson 5. Refine your search plan as you carry out the literature searches
Theory Develop your search plan Everything that follows flows
smoothly and easilyPractice Modify your approach as the search
progresses Use spiral approach to refine Ongoing discussions within team
![Page 20: Systematic Reviews Practicalities and Realities](https://reader035.fdocuments.us/reader035/viewer/2022062519/568156f6550346895dc49c8d/html5/thumbnails/20.jpg)
Search spiral
Scoping (initial database searches) Refining (indepth searches of range
of sources) Confirming (hand searches, citation
tracking)
![Page 21: Systematic Reviews Practicalities and Realities](https://reader035.fdocuments.us/reader035/viewer/2022062519/568156f6550346895dc49c8d/html5/thumbnails/21.jpg)
Inclusion
Intervention aimed to increase PA Aimed at employed adults Initiated or endorsed by employer Outcome measures included a
measure of PA Evidence of behaviour change English Published post 1996 Took place in UK, Australia, NZ, Europe
![Page 22: Systematic Reviews Practicalities and Realities](https://reader035.fdocuments.us/reader035/viewer/2022062519/568156f6550346895dc49c8d/html5/thumbnails/22.jpg)
Exclusion
Self employed or unemployed adults
Adults needing specialist advice regarding PA
No evidence of behaviour change or PA measure
Insufficient detail reported in study Cost effectiveness Located in US or Asia
![Page 23: Systematic Reviews Practicalities and Realities](https://reader035.fdocuments.us/reader035/viewer/2022062519/568156f6550346895dc49c8d/html5/thumbnails/23.jpg)
Lesson 6. How will you refine the inclusion/exclusion criteria?
Tighter the criteria = less papers to review
BUT will your review draw any meaningful
conclusions? Focussing the question v quality of
studies
![Page 24: Systematic Reviews Practicalities and Realities](https://reader035.fdocuments.us/reader035/viewer/2022062519/568156f6550346895dc49c8d/html5/thumbnails/24.jpg)
Overview of search and appraisal process
![Page 25: Systematic Reviews Practicalities and Realities](https://reader035.fdocuments.us/reader035/viewer/2022062519/568156f6550346895dc49c8d/html5/thumbnails/25.jpg)
38 papers representing 33 studies Each paper appraised by 1 reviewer, using a
standardised protocol, 20% sample reviewed by 2 reviewers and each checked as put on database
Quality of papers ++ high, + good, - poor evidence (using protocols/guides provided by NICE)
Categorised by intervention and quality rating Summary table Conclusions on the whole derived from those
classified as high or good
Appraisal
![Page 26: Systematic Reviews Practicalities and Realities](https://reader035.fdocuments.us/reader035/viewer/2022062519/568156f6550346895dc49c8d/html5/thumbnails/26.jpg)
Lesson 7: Know what you want to find out
Select your critical appraisal/data extraction tool carefully
Make sure it captures the elements that you wish to write about in your final report
Think what data need to extract Narrative review v meta analysis Quality of studies – what evidence are you
going to include? How are you going to assess the quality
Do you need to refine the inclusion/exclusion criteria as you go along?
![Page 27: Systematic Reviews Practicalities and Realities](https://reader035.fdocuments.us/reader035/viewer/2022062519/568156f6550346895dc49c8d/html5/thumbnails/27.jpg)
Findings: published systematic reviews
Inconclusive review level evidence that workplace PA interventions were effective in increasing PA
![Page 28: Systematic Reviews Practicalities and Realities](https://reader035.fdocuments.us/reader035/viewer/2022062519/568156f6550346895dc49c8d/html5/thumbnails/28.jpg)
Findings: Stairwalking
7 studies (1 high, 2 good, 4 poor) 4 demonstrated that posters and
signs can increase stair use in the short term
Little evidence of effectiveness
![Page 29: Systematic Reviews Practicalities and Realities](https://reader035.fdocuments.us/reader035/viewer/2022062519/568156f6550346895dc49c8d/html5/thumbnails/29.jpg)
Findings: Walking interventions
4 studies (2 good, 2 poor) 3 used pedometers (objective
measure) All 4 also used self reported counts Evidence that workplace walking
interventions can increase daily step counts
![Page 30: Systematic Reviews Practicalities and Realities](https://reader035.fdocuments.us/reader035/viewer/2022062519/568156f6550346895dc49c8d/html5/thumbnails/30.jpg)
Findings: Active travel
3 studies (1 good, 2 poor) Evidence from 1 study that a
walking and cycling to work campaign using written materials can increase walking to work in economically advantaged women
![Page 31: Systematic Reviews Practicalities and Realities](https://reader035.fdocuments.us/reader035/viewer/2022062519/568156f6550346895dc49c8d/html5/thumbnails/31.jpg)
Findings: multi component interventions
16 studies (2 high, 1 good) Wide range of interventions Difficult to attribute which bit of the
intervention was causing the effect Limited evidence that counselling
has effect on workplace PA
![Page 32: Systematic Reviews Practicalities and Realities](https://reader035.fdocuments.us/reader035/viewer/2022062519/568156f6550346895dc49c8d/html5/thumbnails/32.jpg)
Limitations
Quality of evidence Exclusion of studies from US and
Asia Most of studies took place in large
public sector workplaces Reliance on self report measures
![Page 33: Systematic Reviews Practicalities and Realities](https://reader035.fdocuments.us/reader035/viewer/2022062519/568156f6550346895dc49c8d/html5/thumbnails/33.jpg)
Lesson 9. Set up a system to keep track of the review process
Use a package such as reference manager or endnote to keep track of references
Large reviews need method of keeping track of data extraction, exel, access or specialised software
Update at every stage of the process Compiles references at the end Need a hard copy system too for the actual
papers and a system for keeping track of results If you do not know where you are up to – it is
unlikely that your review will be systematic
![Page 34: Systematic Reviews Practicalities and Realities](https://reader035.fdocuments.us/reader035/viewer/2022062519/568156f6550346895dc49c8d/html5/thumbnails/34.jpg)
System to manage process – the physical
Putting material into endnote Direct export from many sources From a file Direct input
How are you going to manage the hard copies
What about your notes? What systems do you use?
![Page 35: Systematic Reviews Practicalities and Realities](https://reader035.fdocuments.us/reader035/viewer/2022062519/568156f6550346895dc49c8d/html5/thumbnails/35.jpg)
Reducing and organising
Using groups and adding notes to Endnote
Using a screening tool (once you’ve ditched the really irrelevant ones)
![Page 36: Systematic Reviews Practicalities and Realities](https://reader035.fdocuments.us/reader035/viewer/2022062519/568156f6550346895dc49c8d/html5/thumbnails/36.jpg)
Data extraction/Critical Appraisal
Tools Many widely available Can adapt to suit your own study Could design own Assessing quality – systems available – be
explicit if you are going to do this Storing the data
Template in word/Table in word Excel spreadsheet Other?
![Page 37: Systematic Reviews Practicalities and Realities](https://reader035.fdocuments.us/reader035/viewer/2022062519/568156f6550346895dc49c8d/html5/thumbnails/37.jpg)
Useful resources – systematic reviews
Cochrane Collaboration http://www.cochrane.org/ http://www.cochrane.org/docs/irmg.htm
Centre for Reviews and Dissemination http://www.york.ac.uk/inst/crd/
handbook for conducting systematic reviews, http://www.york.ac.uk/inst/crd/methods.htm Searching for systematic reviews http://
www.york.ac.uk/inst/crd/revs.htm EPPI-Centre – Stages of a review
http://eppi.ioe.ac.uk/cms/Default.aspx?tabid=89 SCIE - The conduct of systematic research reviews for SCIE
knowledge reviews http://www.scie.org.uk/publications/details.asp?
pubID=111