GRADE, Summary of Findings and ConQual Workshop · • Moving towards GRADE • Summary of Findings...
Transcript of GRADE, Summary of Findings and ConQual Workshop · • Moving towards GRADE • Summary of Findings...
![Page 1: GRADE, Summary of Findings and ConQual Workshop · • Moving towards GRADE • Summary of Findings tables ... 4. JBI adopts an approach based on GRADE but sensitive to the ... Measurement](https://reader031.fdocuments.us/reader031/viewer/2022030708/5af653e87f8b9a190c8f8d76/html5/thumbnails/1.jpg)
GRADE, Summary of Findings and ConQual Workshop
![Page 2: GRADE, Summary of Findings and ConQual Workshop · • Moving towards GRADE • Summary of Findings tables ... 4. JBI adopts an approach based on GRADE but sensitive to the ... Measurement](https://reader031.fdocuments.us/reader031/viewer/2022030708/5af653e87f8b9a190c8f8d76/html5/thumbnails/2.jpg)
To discuss…
• Introduction
• New JBI Levels of Evidence and Grades of Recommendation
• Moving towards GRADE
• Summary of Findings tables
• Qualitative Levels
• Conclusion
University of Adelaide 2
![Page 3: GRADE, Summary of Findings and ConQual Workshop · • Moving towards GRADE • Summary of Findings tables ... 4. JBI adopts an approach based on GRADE but sensitive to the ... Measurement](https://reader031.fdocuments.us/reader031/viewer/2022030708/5af653e87f8b9a190c8f8d76/html5/thumbnails/3.jpg)
JBI Methodology Groups
• Formed to provide guidance on emerging methods of evidence synthesis
• Mixed methods, GRADE, qualitative, umbrella, economic, effects, prevalence, correlation
• Group formed to review JBI Levels of Evidence
University of Adelaide 3
![Page 4: GRADE, Summary of Findings and ConQual Workshop · • Moving towards GRADE • Summary of Findings tables ... 4. JBI adopts an approach based on GRADE but sensitive to the ... Measurement](https://reader031.fdocuments.us/reader031/viewer/2022030708/5af653e87f8b9a190c8f8d76/html5/thumbnails/4.jpg)
Levels of Evidence
• Designate study type
• Better study designs, with greater methodological quality, are ranked higher
• Assist in applying research into practice
• Recommendations assigned a grade
Grades of Recommendation
![Page 5: GRADE, Summary of Findings and ConQual Workshop · • Moving towards GRADE • Summary of Findings tables ... 4. JBI adopts an approach based on GRADE but sensitive to the ... Measurement](https://reader031.fdocuments.us/reader031/viewer/2022030708/5af653e87f8b9a190c8f8d76/html5/thumbnails/5.jpg)
History
University of Adelaide 5
![Page 6: GRADE, Summary of Findings and ConQual Workshop · • Moving towards GRADE • Summary of Findings tables ... 4. JBI adopts an approach based on GRADE but sensitive to the ... Measurement](https://reader031.fdocuments.us/reader031/viewer/2022030708/5af653e87f8b9a190c8f8d76/html5/thumbnails/6.jpg)
University of Adelaide 6
![Page 7: GRADE, Summary of Findings and ConQual Workshop · • Moving towards GRADE • Summary of Findings tables ... 4. JBI adopts an approach based on GRADE but sensitive to the ... Measurement](https://reader031.fdocuments.us/reader031/viewer/2022030708/5af653e87f8b9a190c8f8d76/html5/thumbnails/7.jpg)
University of Adelaide 7
![Page 8: GRADE, Summary of Findings and ConQual Workshop · • Moving towards GRADE • Summary of Findings tables ... 4. JBI adopts an approach based on GRADE but sensitive to the ... Measurement](https://reader031.fdocuments.us/reader031/viewer/2022030708/5af653e87f8b9a190c8f8d76/html5/thumbnails/8.jpg)
Old levels of evidence
University of Adelaide 8
![Page 9: GRADE, Summary of Findings and ConQual Workshop · • Moving towards GRADE • Summary of Findings tables ... 4. JBI adopts an approach based on GRADE but sensitive to the ... Measurement](https://reader031.fdocuments.us/reader031/viewer/2022030708/5af653e87f8b9a190c8f8d76/html5/thumbnails/9.jpg)
Old Grades of recommendation
University of Adelaide 9
![Page 10: GRADE, Summary of Findings and ConQual Workshop · • Moving towards GRADE • Summary of Findings tables ... 4. JBI adopts an approach based on GRADE but sensitive to the ... Measurement](https://reader031.fdocuments.us/reader031/viewer/2022030708/5af653e87f8b9a190c8f8d76/html5/thumbnails/10.jpg)
How are they used?
University of Adelaide 10
![Page 11: GRADE, Summary of Findings and ConQual Workshop · • Moving towards GRADE • Summary of Findings tables ... 4. JBI adopts an approach based on GRADE but sensitive to the ... Measurement](https://reader031.fdocuments.us/reader031/viewer/2022030708/5af653e87f8b9a190c8f8d76/html5/thumbnails/11.jpg)
Justification for Change • Clarity (not specific)
• Quasi-experimental studies
• Not all types of evidence
• No supporting document
• Experimental studies
• Adopt GRADE
• Difficulties creating recommendations
• Don’t say ‘systematic review’
University of Adelaide 11
![Page 12: GRADE, Summary of Findings and ConQual Workshop · • Moving towards GRADE • Summary of Findings tables ... 4. JBI adopts an approach based on GRADE but sensitive to the ... Measurement](https://reader031.fdocuments.us/reader031/viewer/2022030708/5af653e87f8b9a190c8f8d76/html5/thumbnails/12.jpg)
Working Party Recommendations 1. JBI adopts the GRADE approach to summary of findings tables
for all reviews addressing questions of effect. 2. JBI adopts a modified GRADE approach, based on FAME, for
forming recommendations for practice. 3. New levels of evidence under the following headings:
Intervention/Therapy/ Harms, Diagnostic Accuracy, Prognosis, Economic Analysis, Qualitative research.
4. JBI adopts an approach based on GRADE but sensitive to the nature of qualitative research for qualitative systematic reviews.
5. JBI contact the GRADE working group to discuss GRADE for the use of JBI, and to offer a partnership for developing GRADE for qualitative research.
University of Adelaide 12
![Page 13: GRADE, Summary of Findings and ConQual Workshop · • Moving towards GRADE • Summary of Findings tables ... 4. JBI adopts an approach based on GRADE but sensitive to the ... Measurement](https://reader031.fdocuments.us/reader031/viewer/2022030708/5af653e87f8b9a190c8f8d76/html5/thumbnails/13.jpg)
Why GRADE?
• Grading of Recommendations Assessment, Development and Evaluation (GRADE)
• International working group
• Endorsed by many EBHC organisations (WHO, Cochrane, SIGN, etc)
University of Adelaide 13
Kerwin et al. 2012
![Page 14: GRADE, Summary of Findings and ConQual Workshop · • Moving towards GRADE • Summary of Findings tables ... 4. JBI adopts an approach based on GRADE but sensitive to the ... Measurement](https://reader031.fdocuments.us/reader031/viewer/2022030708/5af653e87f8b9a190c8f8d76/html5/thumbnails/14.jpg)
Why GRADE?
University of Adelaide 14
![Page 15: GRADE, Summary of Findings and ConQual Workshop · • Moving towards GRADE • Summary of Findings tables ... 4. JBI adopts an approach based on GRADE but sensitive to the ... Measurement](https://reader031.fdocuments.us/reader031/viewer/2022030708/5af653e87f8b9a190c8f8d76/html5/thumbnails/15.jpg)
Forming recommendations with GRADE
University of Adelaide 15
Balance between benefits, harms
and burdens
Resource use Patients
values and preferences
Quality of Evidence
How do we determine quality of the evidence?
![Page 16: GRADE, Summary of Findings and ConQual Workshop · • Moving towards GRADE • Summary of Findings tables ... 4. JBI adopts an approach based on GRADE but sensitive to the ... Measurement](https://reader031.fdocuments.us/reader031/viewer/2022030708/5af653e87f8b9a190c8f8d76/html5/thumbnails/16.jpg)
Example meta-analysis discussion
From the examples provided, what information would increase or decrease your confidence in these results?
University of Adelaide 16
![Page 17: GRADE, Summary of Findings and ConQual Workshop · • Moving towards GRADE • Summary of Findings tables ... 4. JBI adopts an approach based on GRADE but sensitive to the ... Measurement](https://reader031.fdocuments.us/reader031/viewer/2022030708/5af653e87f8b9a190c8f8d76/html5/thumbnails/17.jpg)
Discussion results
• Decrease
• Increase
University of Adelaide 17
![Page 18: GRADE, Summary of Findings and ConQual Workshop · • Moving towards GRADE • Summary of Findings tables ... 4. JBI adopts an approach based on GRADE but sensitive to the ... Measurement](https://reader031.fdocuments.us/reader031/viewer/2022030708/5af653e87f8b9a190c8f8d76/html5/thumbnails/18.jpg)
GRADE
• Decrease
– Methodological quality (risk of bias)
– Indirectness (i.e applicability, generalisability, transferability etc)
– Inconsistency (heterogeneity)
– Imprecision (uncertainty)
– Publication bias
• Increase
– Large, consistent, precise effect
– All plausible biases underestimate the effect
– Dose response effect
University of Adelaide 18
![Page 19: GRADE, Summary of Findings and ConQual Workshop · • Moving towards GRADE • Summary of Findings tables ... 4. JBI adopts an approach based on GRADE but sensitive to the ... Measurement](https://reader031.fdocuments.us/reader031/viewer/2022030708/5af653e87f8b9a190c8f8d76/html5/thumbnails/19.jpg)
Quality of evidence: beyond risk of bias Definition: The extent to which our confidence in an estimate of the treatment effect is adequate to support a particular recommendation
Methodological limitations
Inconsistency of results
Indirectness of evidence
Imprecision of results
Publication bias
Risk of bias: Allocation
concealment Blinding Intention-to-treat Follow-up Stopped early
Sources of indirectness:
Indirect
comparisons Patients Interventions Comparators Outcomes
19
![Page 20: GRADE, Summary of Findings and ConQual Workshop · • Moving towards GRADE • Summary of Findings tables ... 4. JBI adopts an approach based on GRADE but sensitive to the ... Measurement](https://reader031.fdocuments.us/reader031/viewer/2022030708/5af653e87f8b9a190c8f8d76/html5/thumbnails/20.jpg)
GRADEing the evidence • Pre-ranking
– RCTs start as high, Observational studies as low
• Quality of evidence ranges from
HIGH
MODERATE
LOW
VERY LOW
• RCTs start with high quality rating • Can be downgraded 1 or 2 points for each area of concern • Maximum downgrade of 3 points overall
Confidence
![Page 21: GRADE, Summary of Findings and ConQual Workshop · • Moving towards GRADE • Summary of Findings tables ... 4. JBI adopts an approach based on GRADE but sensitive to the ... Measurement](https://reader031.fdocuments.us/reader031/viewer/2022030708/5af653e87f8b9a190c8f8d76/html5/thumbnails/21.jpg)
GRADE Quality of Evidence
In the context of making recommendations:
• The quality of evidence reflects the extent of our confidence that the estimates of an effect are adequate to support a particular decision or recommendation
![Page 22: GRADE, Summary of Findings and ConQual Workshop · • Moving towards GRADE • Summary of Findings tables ... 4. JBI adopts an approach based on GRADE but sensitive to the ... Measurement](https://reader031.fdocuments.us/reader031/viewer/2022030708/5af653e87f8b9a190c8f8d76/html5/thumbnails/22.jpg)
Likelihood of and confidence in an outcome
![Page 23: GRADE, Summary of Findings and ConQual Workshop · • Moving towards GRADE • Summary of Findings tables ... 4. JBI adopts an approach based on GRADE but sensitive to the ... Measurement](https://reader031.fdocuments.us/reader031/viewer/2022030708/5af653e87f8b9a190c8f8d76/html5/thumbnails/23.jpg)
Interpretation of grades of evidence
• /A/High: Further research is very unlikely to change confidence in the estimate of effect.
• /B/Moderate: Further research is likely to have an important impact on confidence in the estimate of effect and may change the estimate.
• /C/Low: Further research is very likely to have an important impact on confidence in the estimate of effect and is likely to change the estimate.
• /D/Very low: We have very little confidence in the effect estimate: Any estimate of effect is very uncertain.
![Page 24: GRADE, Summary of Findings and ConQual Workshop · • Moving towards GRADE • Summary of Findings tables ... 4. JBI adopts an approach based on GRADE but sensitive to the ... Measurement](https://reader031.fdocuments.us/reader031/viewer/2022030708/5af653e87f8b9a190c8f8d76/html5/thumbnails/24.jpg)
Summary of Findings tables
• Standard table format • one for each comparison
• Focus on outcomes
• Includes: • context
• results
• GRADE
• reasons behind decisions
![Page 25: GRADE, Summary of Findings and ConQual Workshop · • Moving towards GRADE • Summary of Findings tables ... 4. JBI adopts an approach based on GRADE but sensitive to the ... Measurement](https://reader031.fdocuments.us/reader031/viewer/2022030708/5af653e87f8b9a190c8f8d76/html5/thumbnails/25.jpg)
Summary of findings table
• Improve understanding
• Improve accessibility
• Created with GRADEpro http://ims.cochrane.org/revman/gradepro
![Page 26: GRADE, Summary of Findings and ConQual Workshop · • Moving towards GRADE • Summary of Findings tables ... 4. JBI adopts an approach based on GRADE but sensitive to the ... Measurement](https://reader031.fdocuments.us/reader031/viewer/2022030708/5af653e87f8b9a190c8f8d76/html5/thumbnails/26.jpg)
Presented in a summary of findings table
University of Adelaide 26
![Page 27: GRADE, Summary of Findings and ConQual Workshop · • Moving towards GRADE • Summary of Findings tables ... 4. JBI adopts an approach based on GRADE but sensitive to the ... Measurement](https://reader031.fdocuments.us/reader031/viewer/2022030708/5af653e87f8b9a190c8f8d76/html5/thumbnails/27.jpg)
University of Adelaide 27
![Page 28: GRADE, Summary of Findings and ConQual Workshop · • Moving towards GRADE • Summary of Findings tables ... 4. JBI adopts an approach based on GRADE but sensitive to the ... Measurement](https://reader031.fdocuments.us/reader031/viewer/2022030708/5af653e87f8b9a190c8f8d76/html5/thumbnails/28.jpg)
University of Adelaide 28
![Page 29: GRADE, Summary of Findings and ConQual Workshop · • Moving towards GRADE • Summary of Findings tables ... 4. JBI adopts an approach based on GRADE but sensitive to the ... Measurement](https://reader031.fdocuments.us/reader031/viewer/2022030708/5af653e87f8b9a190c8f8d76/html5/thumbnails/29.jpg)
Forming recommendations with GRADE
• Two recommendations
– Strong and Weak
– For or against
University of Adelaide 29
Balance between benefits, harms
and burdens
Resource use Patients
values and preferences
Quality of Evidence
![Page 30: GRADE, Summary of Findings and ConQual Workshop · • Moving towards GRADE • Summary of Findings tables ... 4. JBI adopts an approach based on GRADE but sensitive to the ... Measurement](https://reader031.fdocuments.us/reader031/viewer/2022030708/5af653e87f8b9a190c8f8d76/html5/thumbnails/30.jpg)
Goldet et al. 2013
University of Adelaide 30
![Page 31: GRADE, Summary of Findings and ConQual Workshop · • Moving towards GRADE • Summary of Findings tables ... 4. JBI adopts an approach based on GRADE but sensitive to the ... Measurement](https://reader031.fdocuments.us/reader031/viewer/2022030708/5af653e87f8b9a190c8f8d76/html5/thumbnails/31.jpg)
Systematic review
Guideline development
P I C O
Outcome
Outcome
Outcome
Outcome
Critical
Important
Critical
Low Summary of findings & estimate of effect for each outcome
Rate overall quality of evidence across outcomes based on
lowest quality of critical outcomes
RCT start high, obs. data start low
1. Risk of bias 2. Inconsistency 3. Indirectness 4. Imprecision 5. Publication
bias
Gra
de
d
ow
n
Gra
de
u
p
1. Large effect 2. Dose
response 3. Confounders
Very low Low Moderate High
Formulate recommendations: • For or against (direction) • Strong or weak (strength)
By considering: Quality of evidence Balance benefits/harms Values and preferences
Revise if necessary by considering:
Resource use (cost)
• “We recommend using…” • “We suggest using…” • “We recommend against using…” • “We suggest against using…”
![Page 32: GRADE, Summary of Findings and ConQual Workshop · • Moving towards GRADE • Summary of Findings tables ... 4. JBI adopts an approach based on GRADE but sensitive to the ... Measurement](https://reader031.fdocuments.us/reader031/viewer/2022030708/5af653e87f8b9a190c8f8d76/html5/thumbnails/32.jpg)
Why still have levels?
• Other JBI resources (not just systematic reviews)
• Assist in pre-ranking
• Address evidence of other types
University of Adelaide 32
![Page 33: GRADE, Summary of Findings and ConQual Workshop · • Moving towards GRADE • Summary of Findings tables ... 4. JBI adopts an approach based on GRADE but sensitive to the ... Measurement](https://reader031.fdocuments.us/reader031/viewer/2022030708/5af653e87f8b9a190c8f8d76/html5/thumbnails/33.jpg)
JBI Levels of Evidence
OLD LEVELS OF EVIDENCE
• Feasibility
• Appropriateness
• Meaningfulness
• Effectiveness
NEW PROPOSED LEVELS OF EVIDENCE
• Therapy, harm
• Diagnostic/screening studies
• Prognosis
• Economic evaluations
• Meaningfulness
University of Adelaide 33
![Page 34: GRADE, Summary of Findings and ConQual Workshop · • Moving towards GRADE • Summary of Findings tables ... 4. JBI adopts an approach based on GRADE but sensitive to the ... Measurement](https://reader031.fdocuments.us/reader031/viewer/2022030708/5af653e87f8b9a190c8f8d76/html5/thumbnails/34.jpg)
University of Adelaide 34
![Page 35: GRADE, Summary of Findings and ConQual Workshop · • Moving towards GRADE • Summary of Findings tables ... 4. JBI adopts an approach based on GRADE but sensitive to the ... Measurement](https://reader031.fdocuments.us/reader031/viewer/2022030708/5af653e87f8b9a190c8f8d76/html5/thumbnails/35.jpg)
Levels of evidence: Diagnostic/screening studies
Level 1 – Studies of Test Accuracy among consecutive patients • Level 1.a – Systematic reviews of studies of test accuracy among consecutive patients • Level 1.b – Studies of test accuracy among consecutive patients Level 2 – Studies of Test Accuracy among non-consecutive patients • Level 2.a – Systematic reviews of studies of test accuracy among non-consecutive patients • Level 2.b – Studies of test accuracy among non-consecutive patients Level 3 – Diagnostic Case control studies • Level 3.a – Systematic reviews of diagnostic case control studies • Level 3.b – Diagnostic case-control study Level 4 – Diagnostic yield studies • Level 4.a – Systematic reviews of diagnostic yield studies • Level 4.b – Individual diagnostic yield study Level 5 – Expert Opinion and Bench Research • Level 5.a – Systematic reviews of expert opinion • Level 5.b – Expert consensus • Level 5.c – Bench research/ single expert opinion
University of Adelaide 35
![Page 36: GRADE, Summary of Findings and ConQual Workshop · • Moving towards GRADE • Summary of Findings tables ... 4. JBI adopts an approach based on GRADE but sensitive to the ... Measurement](https://reader031.fdocuments.us/reader031/viewer/2022030708/5af653e87f8b9a190c8f8d76/html5/thumbnails/36.jpg)
Levels of evidence: Prognosis
Level 1 – Inception Cohort Studies • Level 1.a – Systematic reviews of inception cohort studies • Level 1.b – Inception cohort studies Level 2 – Studies of All or none • Level 2.a – Systematic reviews of studies of all or none studies • Level 2.b – All or none studies Level 3 – Cohort studies • Level 3.a – Systematic reviews of cohort studies (or control arm of RCT) • Level 3.b – Cohort studies (or control arm of RCT) Level 4 – Case series/Case Controlled/ Historically Controlled studies • Level 4.a – Systematic reviews of Case series/Case Controlled/ Historically Controlled studies • Level 4.b – Individual Case series/Case Controlled/ Historically Controlled studies Level 5 – Expert Opinion and Bench Research • Level 5.a – Systematic reviews of expert opinion • Level 5.b – Expert consensus • Level 5.c – Bench research/ single expert opinion
University of Adelaide 36
![Page 37: GRADE, Summary of Findings and ConQual Workshop · • Moving towards GRADE • Summary of Findings tables ... 4. JBI adopts an approach based on GRADE but sensitive to the ... Measurement](https://reader031.fdocuments.us/reader031/viewer/2022030708/5af653e87f8b9a190c8f8d76/html5/thumbnails/37.jpg)
Levels of evidence: Economic evaluations
• Level 1: Systematic review of economic evaluations
• Level 2: Single economic evaluation
• Level 3: Systematic review of expert opinion
• Level 4: Expert opinion
University of Adelaide 37
![Page 38: GRADE, Summary of Findings and ConQual Workshop · • Moving towards GRADE • Summary of Findings tables ... 4. JBI adopts an approach based on GRADE but sensitive to the ... Measurement](https://reader031.fdocuments.us/reader031/viewer/2022030708/5af653e87f8b9a190c8f8d76/html5/thumbnails/38.jpg)
Levels of evidence: Meaningfulness
1. Qualitative or mixed-methods systematic review
2. Qualitative or mixed-methods synthesis
3. Single qualitative study
4. Systematic review of expert opinion
5. Expert opinion
University of Adelaide 38
![Page 39: GRADE, Summary of Findings and ConQual Workshop · • Moving towards GRADE • Summary of Findings tables ... 4. JBI adopts an approach based on GRADE but sensitive to the ... Measurement](https://reader031.fdocuments.us/reader031/viewer/2022030708/5af653e87f8b9a190c8f8d76/html5/thumbnails/39.jpg)
Summary of findings table: Meaningfulness
• Incorporating a ‘GRADE’ like rating of quality for studies related to meaningfulness – can it be done?
• Need to consider what increases or decrease confidence in the results
University of Adelaide 39
![Page 40: GRADE, Summary of Findings and ConQual Workshop · • Moving towards GRADE • Summary of Findings tables ... 4. JBI adopts an approach based on GRADE but sensitive to the ... Measurement](https://reader031.fdocuments.us/reader031/viewer/2022030708/5af653e87f8b9a190c8f8d76/html5/thumbnails/40.jpg)
Discussion Activity
EXAMPLE META-SYNTHESIS
From the examples provided, what information would increase or decrease your confidence in these results?
University of Adelaide 40
Synthesised finding 1
Munn & Jordan. The patient experience of high technology medical imaging: a systematic review of the qualitative evidence, JBI Library of
Systematic Reviews, 2011; 9(19): 631-678
People undergoing imaging often expect a health issue to be found during their
scan, which can then lead to anxiety and worry.
Synthesised finding 2
Adjust themselves: Caregivers need to adjust themselves to the caregiving role when living with and
taking care of persons with schizophrenia. The ultimate goal is to integrate the caregiving role into their
lives.
Tungpunkom, Napa, Chaniang & Srikhachin. Caregiving experiences of families living with persons with schizophrenia: a systematic review,
2013; 11(8): 415-564
![Page 41: GRADE, Summary of Findings and ConQual Workshop · • Moving towards GRADE • Summary of Findings tables ... 4. JBI adopts an approach based on GRADE but sensitive to the ... Measurement](https://reader031.fdocuments.us/reader031/viewer/2022030708/5af653e87f8b9a190c8f8d76/html5/thumbnails/41.jpg)
Discussion Activity
EXAMPLE META-SYNTHESIS
What information will increase or decrease your confidence in the
results?
University of Adelaide 41
![Page 42: GRADE, Summary of Findings and ConQual Workshop · • Moving towards GRADE • Summary of Findings tables ... 4. JBI adopts an approach based on GRADE but sensitive to the ... Measurement](https://reader031.fdocuments.us/reader031/viewer/2022030708/5af653e87f8b9a190c8f8d76/html5/thumbnails/42.jpg)
Discussion Activity (some answers)
• Type of data
• Dependability
• Confidence
EXAMPLE META-SYNTHESIS
What information will increase or decrease your confidence in the
results?
University of Adelaide 42
![Page 43: GRADE, Summary of Findings and ConQual Workshop · • Moving towards GRADE • Summary of Findings tables ... 4. JBI adopts an approach based on GRADE but sensitive to the ... Measurement](https://reader031.fdocuments.us/reader031/viewer/2022030708/5af653e87f8b9a190c8f8d76/html5/thumbnails/43.jpg)
Meaningfulness Summary of Findings Table
University of Adelaide 43
Systematic review title:
Population:
Phenomena of interest:
Context:
Synthesised
Finding
Type of data Dependability Confidence Score Comments
![Page 44: GRADE, Summary of Findings and ConQual Workshop · • Moving towards GRADE • Summary of Findings tables ... 4. JBI adopts an approach based on GRADE but sensitive to the ... Measurement](https://reader031.fdocuments.us/reader031/viewer/2022030708/5af653e87f8b9a190c8f8d76/html5/thumbnails/44.jpg)
Type of data
• Ranking scale consists of 4 levels
– High
– Moderate
– Low
– Very Low
• Begin by pre-ranking papers based on type of data
– High for qualitative studies
– Low for expert opinion
University of Adelaide 44
![Page 45: GRADE, Summary of Findings and ConQual Workshop · • Moving towards GRADE • Summary of Findings tables ... 4. JBI adopts an approach based on GRADE but sensitive to the ... Measurement](https://reader031.fdocuments.us/reader031/viewer/2022030708/5af653e87f8b9a190c8f8d76/html5/thumbnails/45.jpg)
Score of dependability: qualitative research
Measurement Measured by asking the following questions:
1. Is there congruity between the research methodology and the research question or objectives?
2. Is there congruity between the research methodology and the methods used to collect data?
3. Is there congruity between the research methodology and the representation and analysis of data?
4. Is there a statement locating the researcher culturally or theoretically?
5. Is the influence of the researcher on the research, and vice-versa, addressed?
Ranking system: 4-5 ‘yes’ responses, the finding remains unchanged
2-3 ‘yes’ responses: move down 1 level
0-1 ‘yes’ responses: move down 2 levels
University of Adelaide 45
![Page 46: GRADE, Summary of Findings and ConQual Workshop · • Moving towards GRADE • Summary of Findings tables ... 4. JBI adopts an approach based on GRADE but sensitive to the ... Measurement](https://reader031.fdocuments.us/reader031/viewer/2022030708/5af653e87f8b9a190c8f8d76/html5/thumbnails/46.jpg)
Score of dependability: expert opinion
Measured by asking questions related to the appropriateness of the conduct of the research with research aims and purpose:
1. Does the source of the opinion have standing in the field of expertise?
2. Is the opinion’s basis in logic/experience clearly argued?
3. Is the argument developed analytical? 4. Is there reference to the extant
literature/evidence and any incongruence with it logically defended?
5. Is the opinion supported by peers?
University of Adelaide 46
Ranking system:
4-5 ‘yes’ responses, the paper remains unchanged
(0)
2-3 ‘yes’ responses: move down 1 level (-1)
0-1 ‘yes’ responses: move down 2
levels (-2)
![Page 47: GRADE, Summary of Findings and ConQual Workshop · • Moving towards GRADE • Summary of Findings tables ... 4. JBI adopts an approach based on GRADE but sensitive to the ... Measurement](https://reader031.fdocuments.us/reader031/viewer/2022030708/5af653e87f8b9a190c8f8d76/html5/thumbnails/47.jpg)
Score of confidence
Measurement Assign a level of credibility to the findings:
Unequivocal (findings accompanied by an illustration that is beyond reasonable doubt and; therefore not open to challenge) Equivocal (findings accompanied by an illustration lacking clear association with it and therefore open to challenge) Unsupported (findings are not supported by the data, or with no illustration)
University of Adelaide 47
Ranking The synthesised findings contains only unequivocal findings Remains unchanged Mix of unequivocal/equivocal findings downgraded one (-1) All equivocal finding down grade 2 (-2)
Mix of plausible/unsupported findings downgraded three (-3)
Not-supported findings downgraded four (-4)
![Page 48: GRADE, Summary of Findings and ConQual Workshop · • Moving towards GRADE • Summary of Findings tables ... 4. JBI adopts an approach based on GRADE but sensitive to the ... Measurement](https://reader031.fdocuments.us/reader031/viewer/2022030708/5af653e87f8b9a190c8f8d76/html5/thumbnails/48.jpg)
Meaningfulness Summary of Findings Table
University of Adelaide 48
Systematic review title: The patient experience of high technology medical imaging: a systematic review of the
qualitative evidence
Population: Persons who had undergone high technology medical imaging
Phenomena of interest: The meaningfulness of a patients experience of undergoing diagnostic imaging using
high technology
Context: Male and Female Adult Patients presenting to a medical imaging department
Synthesised
Finding
Type of
research
Dependability Confidence Score Comments
People undergoing imaging often
expect a health issue to be found during their scan, which can then
lead to anxiety and worry
Qualitative
(HIGH)
(MODERATE)
(LOW)
LOW
![Page 49: GRADE, Summary of Findings and ConQual Workshop · • Moving towards GRADE • Summary of Findings tables ... 4. JBI adopts an approach based on GRADE but sensitive to the ... Measurement](https://reader031.fdocuments.us/reader031/viewer/2022030708/5af653e87f8b9a190c8f8d76/html5/thumbnails/49.jpg)
A new spin on FAME
University of Adelaide 49
The FAME (Feasibility, Appropriateness, Meaningfulness and Effectiveness) scale informs the recommendation. F – Feasibility; specifically: What is the cost effectiveness of the practice? Is the resource/practice available? Is their sufficient experience/levels of competency available? A – Appropriateness; specifically: Is it culturally acceptable? Is it transferable to the majority of the population? Is it easily adaptable to a variety of circumstances? M – Meaningfulness; specifically: Is it associated with positive experiences? Is it not associated with negative experiences? E – Effectiveness; specifically: Was there a beneficial effect? Is it safe? (i.e is there a lack of harm associated with the practice?
![Page 50: GRADE, Summary of Findings and ConQual Workshop · • Moving towards GRADE • Summary of Findings tables ... 4. JBI adopts an approach based on GRADE but sensitive to the ... Measurement](https://reader031.fdocuments.us/reader031/viewer/2022030708/5af653e87f8b9a190c8f8d76/html5/thumbnails/50.jpg)
JBI Grades of Recommendation
University of Adelaide 50
JBI Grades of Recommendation
Grade A A ‘strong’ recommendation for a certain health management strategy where: • it is clear that desirable effects outweigh undesirable effects of the
strategy; • where there is evidence of adequate quality supporting its use; • there is a benefit or no impact on resource use, and • values, preferences and the patient experience have been taken into
account
Grade B A ‘weak’ recommendation for a certain health management strategy where: • desirable effects appear to outweigh undesirable effects of the
strategy, although this is not as clear; • where there is evidence supporting its use, although this may not be
of high quality; • there is a benefit, no impact or minimal impact on resource use, and • values, preferences and the patient experience may or may not have
been taken into account.
![Page 51: GRADE, Summary of Findings and ConQual Workshop · • Moving towards GRADE • Summary of Findings tables ... 4. JBI adopts an approach based on GRADE but sensitive to the ... Measurement](https://reader031.fdocuments.us/reader031/viewer/2022030708/5af653e87f8b9a190c8f8d76/html5/thumbnails/51.jpg)
Recommendations
People undergoing scanning are aware that is has the power to detect significant health issues, which can lead to anxious waits
and uncertainty after imaging. Once imaging is completed, patients should be given information regarding when they will
receive their results, and what will happen now the images have been taken
(Grade B)
Munn & Jordan. The patient experience of high technology medical imaging: a systematic review of the qualitative evidence, JBI Library of
Systematic Reviews, 2011; 9(19): 631-678
![Page 52: GRADE, Summary of Findings and ConQual Workshop · • Moving towards GRADE • Summary of Findings tables ... 4. JBI adopts an approach based on GRADE but sensitive to the ... Measurement](https://reader031.fdocuments.us/reader031/viewer/2022030708/5af653e87f8b9a190c8f8d76/html5/thumbnails/52.jpg)
University of Adelaide 52
![Page 53: GRADE, Summary of Findings and ConQual Workshop · • Moving towards GRADE • Summary of Findings tables ... 4. JBI adopts an approach based on GRADE but sensitive to the ... Measurement](https://reader031.fdocuments.us/reader031/viewer/2022030708/5af653e87f8b9a190c8f8d76/html5/thumbnails/53.jpg)
Conclusions
• Improved clarity in levels of evidence
• In line with international organisations
• Changes made to fit with JBI’s broader view of what constitutes evidence
• Guidance provided
University of Adelaide 53