HOW GRADE COULD HELP TO IMPLEMENT THE EVIDENCE Holger Schünemann, MD, PhD Professor and Chair,...
-
Upload
ophelia-lindsey -
Category
Documents
-
view
212 -
download
0
Transcript of HOW GRADE COULD HELP TO IMPLEMENT THE EVIDENCE Holger Schünemann, MD, PhD Professor and Chair,...
HOW GRADE COULD HELP TO IMPLEMENT THE EVIDENCE
Holger Schünemann, MD, PhD Professor and Chair, Dept. of Clinical Epidemiology & BiostatisticsProfessor of MedicineMichael Gent Chair in Healthcare ResearchMcMaster University, Hamilton, Canada
ContentPulmonary rehabilitation compared to usual community care for COPD with recent exacerbation
Bibliography: Puhan M, et al. Pulmonary rehabilitation following exacerbations of chronic obstructive pulmonary disease. Cochrane Database of Systematic Reviews 2010, Issue 11.
Outcomes No of Participants(studies)Follow up
Quality of the evidence(GRADE)
Relative effect(95% CI)
Anticipated absolute effects
Risk with Usual community care
Risk difference with Pulmonary rehabilitation (95% CI)
Hospital admission
250(6 studies)3-18 months
⊕⊕⊕⊕HIGH
OR 0.22 (0.08 to 0.58)
405 per 1000 275 fewer per 1000(from 122 fewer to 353 fewer)
Mortality 110(3 studies)3-48 months
⊕⊕⊕⊝MODERATE2
due to imprecision
OR 0.28 (0.1 to 0.84)
Low1
100 per 1000 70 fewer per 1000(from 15 fewer to 89 fewer)
High1
500 per 1000 281 fewer per 1000(from 43 fewer to 409 fewer)
Quality of life (CRQ) dyspneaChronic Respiratory Questionnaire3. Scale from: 1 to 7.
258(5 studies)12 and 76 weeks
⊕⊕⊕⊝MODERATE4
due to imprecision
The mean quality of life (sgrq) total in the control groups was3.1
The mean quality of life (crq) dyspnea in the intervention groups was0.97 higher(0.35 to 1.58 higher)
Quality of life (SGRQ) totalSt George's Respiratory Questionnaire5. Scale from: 0 to 100.
127(3 studies)12 and 26 weeks
⊕⊕⊕⊝MODERATE4
due to imprecision
The mean quality of life (sgrq) total in the control groups was50
The mean quality of life (sgrq) total in the intervention groups was9.88 lower(5.37 to 14.4 lower)
Ambulation (as measured by 6 min walking distance)distance in meters6
299(6 studies)1 - 208 weeks7
⊕⊕⊕⊝MODERATE4,8
due to imprecision
The mean ambulation (as measured by 6 min walking distance) in the intervention groups was77.7 higher(12.21 to 143.2 higher)
Resource use - not reported
- - - See footnote See footnote
Systematic Reviews & GRADE
Evidence & judgments
Recommendation, health policy & implementation
ContentPulmonary rehabilitation compared to usual community care for COPD with recent exacerbation
Bibliography: Puhan M, et al. Pulmonary rehabilitation following exacerbations of chronic obstructive pulmonary disease. Cochrane Database of Systematic Reviews 2010, Issue 11.
Outcomes No of Participants(studies)Follow up
Quality of the evidence(GRADE)
Relative effect(95% CI)
Anticipated absolute effects
Risk with Usual community care
Risk difference with Pulmonary rehabilitation (95% CI)
Hospital admission
250(6 studies)3-18 months
⊕⊕⊕⊕HIGH
OR 0.22 (0.08 to 0.58)
405 per 1000 275 fewer per 1000(from 122 fewer to 353 fewer)
Mortality 110(3 studies)3-48 months
⊕⊕⊕⊝MODERATE2
due to imprecision
OR 0.28 (0.1 to 0.84)
Low1
100 per 1000 70 fewer per 1000(from 15 fewer to 89 fewer)
High1
500 per 1000 281 fewer per 1000(from 43 fewer to 409 fewer)
Quality of life (CRQ) dyspneaChronic Respiratory Questionnaire3. Scale from: 1 to 7.
258(5 studies)12 and 76 weeks
⊕⊕⊕⊝MODERATE4
due to imprecision
The mean quality of life (sgrq) total in the control groups was3.1
The mean quality of life (crq) dyspnea in the intervention groups was0.97 higher(0.35 to 1.58 higher)
Quality of life (SGRQ) totalSt George's Respiratory Questionnaire5. Scale from: 0 to 100.
127(3 studies)12 and 26 weeks
⊕⊕⊕⊝MODERATE4
due to imprecision
The mean quality of life (sgrq) total in the control groups was50
The mean quality of life (sgrq) total in the intervention groups was9.88 lower(5.37 to 14.4 lower)
Ambulation (as measured by 6 min walking distance)distance in meters6
299(6 studies)1 - 208 weeks7
⊕⊕⊕⊝MODERATE4,8
due to imprecision
The mean ambulation (as measured by 6 min walking distance) in the intervention groups was77.7 higher(12.21 to 143.2 higher)
Resource use - not reported
- - - See footnote See footnote
Systematic Reviews & GRADE
Evidence & judgments
Recommendation, health policy & implementation
Examples and summary from leading/co-leading 15 guideline projects
• 11 World Health Organization
• World Allergy Organization, Allergic Rhinitis in Asthma, American Thoracic Society (2)
Guideline development Process
Systematic review
Guideline development
PICO
OutcomeOutcomeOutcomeOutcome
Formulate
question
Rate
importa
nce
Critical
Important
Critical
Not important
Create
evidence
profile with
GRADEpro
Summary of findings & estimate of effect for each outcome
Grade overall quality of
evidence across outcomes based
on lowest quality of critical outcomes
Panel
Randomization increases initial
quality
1. Risk of bias2. Inconsisten
cy3. Indirectnes
s4. Imprecision5. Publication
bias
Gra
de d
own
Gra
de u
p 1. Large effect
2. Dose response
3. Confounders
Rate quality
of evidence
for each
outcomeSelect
outcomes
Very low
LowModerate
High
Formulate recommendations:• For or against (direction)• Strong or conditional/weak
(strength)
By considering: Quality of evidence Balance benefits/harms Values and preferences
Revise if necessary by considering: Resource use (cost)
• “We recommend using…”• “We suggest using…”• “We recommend against using…”• “We suggest against using…”
Outcomes
across
studies
Outcome generation and selection• Multidisciplinary panels
– Researchers, epidemiologists, public health officers, methodologists, patient representatives…
• Delphi process – 3 rounds
1. List of possible outcomes from literature– Panel members review and add
2. List of all outcomes grouped by theme– Panel members rate importance
3. Final agreement and results
Mortality from cervical cancer8.2
Cervical cancer Incidence8.3
Detected CIN 2,37. 9
Major Infections (requiring hospital admission and antibiotics, e.g. PID)6.0
Maternal bleeding5.8
Premature delivery5.7
Fertility5.4
Identification of STIs (benefit)5.0
Minor infections (requiring outpatient treatment only)3.8
Outcomes for screening on a scale of 1 (not important) to 9 (critical)
Challenges and advantages of this approach
• Often starting with many outcomes
• Experts initially focused on what they know from research studies
• Requires detailed explanations
• ↓ Participation of panel members
• Perspective taken
• Complete• Everyone involved
– Independent ratings• Numerical estimates• Well documented and
kept record• Transparent• Reduces work
WHO influenza guidelines
• New guideline on pharmacological management of influenza– Previously few randomized trials
• Low quality evidence for many outcomes (imprecision)• Industry sponsored – publication bias• Not all outcomes
• Review of observational studies– To inform guidelines
Methods
• Standard systematic review– MEDLINE, EMBASE, CENTRAL, CINAHL, SIGLE, the Chinese
Biomedical Literature Database, Panteleimon and LILACS for relevant studies up to November 2010
– contacted pharmaceutical companies and international agencies– RevMan 5.1
• 10 PICO → recommendations approach– Outcomes determined through Delphi process previously
• QoE according to GRADE approach– GRADEpro (www.gradeworkinggroup.org)– Risk of bias using modified Ottawa Newcastle scale
ResultsQuality assessment Summary of Findings
Participants(studies)
Risk of bias Inconsistency Indirectness Imprecision Publication bias
Overall quality of evidence
Study event rates (%) Relative effect(95% CI)
Anticipated absolute effects
With no antiviral treatment
With oseltamivir Risk with no antiviral treatment Absolute effect with Oseltamivir (95% CI)
Mortality
681(3 studies)
no serious risk of bias
no serious inconsistency
no serious indirectness
no serious imprecision
undetected1 ⊕⊕⊝⊝LOW1
59/242 (24.4%)
31/439 (7.1%)
adj OR 0.23 (0.13 to 0.43) 240 deaths per 1000 172 fewer deaths per 1000
(from 120 to 201 fewer)
1557(9 studies)
serious2 no serious inconsistency
no serious indirectness
no serious imprecision
undetected1 ⊕⊝⊝⊝VERY LOW1,2
due to risk of bias
61/320 (19.1%)
228/1237 (18.4%)
OR 0.51 (0.23 to 1.14)3 240 deaths per 1000 101 fewer deaths per 1000
(from 172 fewer to 25 more)
Hospitalisation
150710(5 studies)
no serious risk of bias
no serious inconsistency
no serious indirectness
no serious imprecision
undetected4 ⊕⊕⊝⊝LOW4
1238/100585 (1.2%)
431/50125 (0.86%)
adj OR 0.75 (0.66 to 0.89)
12 hospitalisations per 1000 3 fewer hospitalisations per 1000(from 1 to 4 fewer)
242762(6 studies)
serious2 no serious inconsistency
no serious indirectness
no serious imprecision
undetected4 ⊕⊝⊝⊝VERY LOW2,4
due to risk of bias
1738/146410 (1.2%)
1086/96352 (1.1%)
OR 0.75 (0.66 to 0.86)
12 hospitalisations per 1000 3 fewer hospitalisations per 1000(from 2 to 4 fewer)
ICU admissions/mechanical ventilation/respiratory failure
1032(6 studies5)
Serious5 serious6 no serious indirectness
no serious imprecision
undetected1 ⊕⊝⊝⊝VERY LOW1,6
due to risk of bias, inconsistency
- 200/1032 (19.4%)
- -
Complications - Pneumonia
150466(3 studies)
no serious risk of bias
serious6 no serious indirectness
no serious imprecision
undetected4 ⊕⊝⊝⊝VERY LOW4,6
due to inconsistency
2111/100449 (2.1%)
647/50017 (1.3%)
adj OR 0.83 (0.59 to 1.16)
21 pneumonias per 1000 4 fewer pneumonias per 1000(from 9 fewer to 3 more)
265276(6 studies)
serious2 serious6 no serious indirectness
no serious imprecision
undetected4 ⊕⊝⊝⊝VERY LOW2,4,6
due to risk of bias, inconsistency
3244/166256 (2%)
1273/99020 (1.3%)
OR 0.64 (0.46 to 0.88)
20 pneumonias per 1000 7 fewer pneumonias per 1000(from 2 to 10 fewer)
Question: Should oseltamivir vs. no antiviral treatment be used for influenza (follow-up: 30 days)?
ContentPulmonary rehabilitation compared to usual community care for COPD with recent exacerbation
Bibliography: Puhan M, et al. Pulmonary rehabilitation following exacerbations of chronic obstructive pulmonary disease. Cochrane Database of Systematic Reviews 2010, Issue 11.
Outcomes No of Participants(studies)Follow up
Quality of the evidence(GRADE)
Relative effect(95% CI)
Anticipated absolute effects
Risk with Usual community care
Risk difference with Pulmonary rehabilitation (95% CI)
Hospital admission
250(6 studies)3-18 months
⊕⊕⊕⊕HIGH
OR 0.22 (0.08 to 0.58)
405 per 1000 275 fewer per 1000(from 122 fewer to 353 fewer)
Mortality 110(3 studies)3-48 months
⊕⊕⊕⊝MODERATE2
due to imprecision
OR 0.28 (0.1 to 0.84)
Low1
100 per 1000 70 fewer per 1000(from 15 fewer to 89 fewer)
High1
500 per 1000 281 fewer per 1000(from 43 fewer to 409 fewer)
Quality of life (CRQ) dyspneaChronic Respiratory Questionnaire3. Scale from: 1 to 7.
258(5 studies)12 and 76 weeks
⊕⊕⊕⊝MODERATE4
due to imprecision
The mean quality of life (sgrq) total in the control groups was3.1
The mean quality of life (crq) dyspnea in the intervention groups was0.97 higher(0.35 to 1.58 higher)
Quality of life (SGRQ) totalSt George's Respiratory Questionnaire5. Scale from: 0 to 100.
127(3 studies)12 and 26 weeks
⊕⊕⊕⊝MODERATE4
due to imprecision
The mean quality of life (sgrq) total in the control groups was50
The mean quality of life (sgrq) total in the intervention groups was9.88 lower(5.37 to 14.4 lower)
Ambulation (as measured by 6 min walking distance)distance in meters6
299(6 studies)1 - 208 weeks7
⊕⊕⊕⊝MODERATE4,8
due to imprecision
The mean ambulation (as measured by 6 min walking distance) in the intervention groups was77.7 higher(12.21 to 143.2 higher)
Resource use - not reported
- - - See footnote See footnote
Systematic Reviews & GRADE
Evidence & judgments
Recommendation, health policy & implementation
• Systematic and transparent approach
• Transparently lay out rationale for recommendations
• Manage COI
Judgments/explanations
1 Although we did not downgrade, publication bias cannot be excluded and is of concern.2 Studies not adjusted for potential confounding factors.3 Significant differences in effect for pandemic versus seasonal influenza (see subgroup analyses table).4 Publication bias a concern since large studies had for-profit funding and weighted heavily in analyses.5 No independent comparison group.6 High heterogeneity among studies.7 Measured in select patients in trials.
Getting from evidence to recommendations - GRADE
Recommendations are based on judgments:– Quality of evidence (confidence in estimates of
effect)– Balance between benefits and downsides– Values and preferences– Resource use
But judgments need to be based on the best available evidence and transparent
Balancing desirable and undesirable consequences
↑ Allergic reactions
↑ Local skin reactions
↑ Nausea↑ Resources
↑ QoL ↓ Death
↓ Morbidity
↑ herd immunity
Conditional
Strong For Against
↑ Allergic
reactions
↑ Local skin
reactions
↑ Nausea
↑ Resources
↑ QoL↓ Death
↓
Morbidity↑ herd
immunityConditional
Strong For Against
Balancing desirable and undesirable consequences
↑ Allergic reactions
↑ Local skin reactions
↑ Nausea
↑ Resources↑ QoL↓ Death
↓ Morbidity
↑ herd immunity
Conditional
Strong For Against
Balancing desirable and undesirable consequences
Balancing desirable and undesirable consequences
↑ Allergic
reactions
↑ Local skin
reactions
↑ Nausea
↑ Resources
↑ QoL
↓ Death
↓
Morbidity
↑ herd
immunity
Conditional
Strong For Against
Balancing desirable and undesirable consequences
↑ Allergic reactions ↑ Local skin
reactions
↑ Nausea
↑ Resources
↑ QoL
↓ Death
↓ Morbidity
↑ herd immunity
Conditional
Strong For Against
ContentPulmonary rehabilitation compared to usual community care for COPD with recent exacerbation
Bibliography: Puhan M, et al. Pulmonary rehabilitation following exacerbations of chronic obstructive pulmonary disease. Cochrane Database of Systematic Reviews 2010, Issue 11.
Outcomes No of Participants(studies)Follow up
Quality of the evidence(GRADE)
Relative effect(95% CI)
Anticipated absolute effects
Risk with Usual community care
Risk difference with Pulmonary rehabilitation (95% CI)
Hospital admission
250(6 studies)3-18 months
⊕⊕⊕⊕HIGH
OR 0.22 (0.08 to 0.58)
405 per 1000 275 fewer per 1000(from 122 fewer to 353 fewer)
Mortality 110(3 studies)3-48 months
⊕⊕⊕⊝MODERATE2
due to imprecision
OR 0.28 (0.1 to 0.84)
Low1
100 per 1000 70 fewer per 1000(from 15 fewer to 89 fewer)
High1
500 per 1000 281 fewer per 1000(from 43 fewer to 409 fewer)
Quality of life (CRQ) dyspneaChronic Respiratory Questionnaire3. Scale from: 1 to 7.
258(5 studies)12 and 76 weeks
⊕⊕⊕⊝MODERATE4
due to imprecision
The mean quality of life (sgrq) total in the control groups was3.1
The mean quality of life (crq) dyspnea in the intervention groups was0.97 higher(0.35 to 1.58 higher)
Quality of life (SGRQ) totalSt George's Respiratory Questionnaire5. Scale from: 0 to 100.
127(3 studies)12 and 26 weeks
⊕⊕⊕⊝MODERATE4
due to imprecision
The mean quality of life (sgrq) total in the control groups was50
The mean quality of life (sgrq) total in the intervention groups was9.88 lower(5.37 to 14.4 lower)
Ambulation (as measured by 6 min walking distance)distance in meters6
299(6 studies)1 - 208 weeks7
⊕⊕⊕⊝MODERATE4,8
due to imprecision
The mean ambulation (as measured by 6 min walking distance) in the intervention groups was77.7 higher(12.21 to 143.2 higher)
Resource use - not reported
- - - See footnote See footnote
Systematic Reviews & GRADE
Evidence & judgments
Recommendations, health policy & implementation
Question/Recommendation: Should contacts of new or recurrent TB cases be investigated? Population: In people (at normal risk) who had contact with new or recurrent cases of TB (does)
Intervention: contact investigation Comparison: no investigation Setting (if relevant): high income countries
Decision domain: Judgment Summary of reason for judgment Explanation Subdomains influencing judgment
Quality of evidence (QoE) Is there high or moderate quality evidence? The higher the quality of evidence, the more likely is a strong recommendation
Yes No
There is very low quality evidence from observational studies that are moderate risk of bias for the critical outcomes.
QoE for benefits: Very low QoE for harms: Harms not explicitly evaluated QoE for resource use: Resource use not explicitly evaluated Key reasons for down- or upgrading? Risk of bias was a reason for downgrading for most critical outcomes All critical outcomes measured? Harms and resources not explicitly evaluated
Balance of benefits versus harms and burdens Are you confident that the benefits outweigh the harms and burden or vice versa? The larger the difference between the benefits and harms and the certainty around that difference, the more likely is a strong recommendation. The smaller the net benefit or net harm and the lower the certainty for that net effect, the more likely is a conditional/weak recommendation.
Yes No
There is considerable benefit while little clinical harm or downsides are expected
The yield for all tuberculosis (bacteriologically-confirmed and clinically diagnosed) was 4.5% of contacts investigated.
Latent tuberculosis infection was found in 51.4% of contacts investigated.
The yield for all tuberculosis (bacteriologically-confirmed and clinically diagnosed) was 7.0 % of pediatric contacts investigated.
Latent tuberculosis infection was found in 40.4% % of pediatric contacts investigated.
Baseline risk for benefits and harm and burden?
Is the baseline risk similar across subgroups?
Should there be separate recommendations for subgroups?
Relative risk for benefits and harms: Are the relative benefits large?
Yes, the relative benefits are probably large. Are the relative harms large?
No, the relative harms are probably small. Recommendations for other groups are made separately, pediatric and adult index cases were considered together. Requirement for modeling:
Is there a lot of extrapolation and modeling required for these outcomes?
Yes, modeling is required.
Values and preferences Are you confident about the assumed or identified relative values and are they similar across the target population? The more certainty or similarity in values and preferences, the more likely a strong recommendation.
Yes No
Benefits much higher valued than expected minor harms.
A high value was placed on avoiding consequences of TB, dissemination of TB and mortality. A low value was placed on possible adverse events. There is likely little variability and panel is quite certain
Perspective taken: Patients or public Source of values: Guideline panels assessment Source of variability if any: Not a lot of variability Method for determining values satisfactory for this recommendation: Yes, given the expected small variability and difference between guideline panel and patients.
Resource implications Are the resources worth the expected net benefit from following the recommendation? The lower the cost of an intervention compared to the alternative, and other costs related to the decision – that is, the fewer resources consumed – the more likely is a strong recommendation in favour of that intervention.
Yes No
Resources required are worth the net benefit considering the benefit on mortality and new TB cases.
There are resources required to conduct contact investigation but these resources are worth the expected benefits and downstream treatment costs.
What are the cost per resource unit? Feasibility: Opportunity cost: Differences across settings:
Overall strength of recommendation
?Strong/conditional?
The guideline panel recommends that contacts of patients with TB who are at normal risk be investigated. (NOTE: this is a hypothetical recommendation developed for this article and not intended for clinical decision making.)
Depending on contact investigation strategy used, resource utilization and implications will vary. Opportunity cost may be high.Feasibility is dependent on existing and well functioning programs.Resources worth in smear positive index cases
Quality assessment No of patients Effect
Quality ImportanceNo. of studies Design Limitations Inconsistency Indirectness Imprecision Other Cryotherapy LEEP Relative
(95% CI)Absolute effect at 1 year
(95% CI)
Recurrence CIN2–3 (follow-up 12 months randomized trials; 3–85 months observational studies)1 randomized
trialsno serious limitations
no serious inconsistency
no serious indirectness seriousa,b none 12/161 (7.5%)
4/168 (2.4%)
OR 3.3 (1.04 to 10.46)
51 more per 1000(from 1 to 179 more) O CRITICAL
3 observational studies
no serious limitations
no serious inconsistency
no serious indirectness no serious imprecision
none
2227/14 387(15.5%)
319/7454(4.3%)
OR 2.66 (1.89 to 3.75)
—
OO CRITICAL
2.4%c 37 more per 1000(from 20 to 60 more)
Cervical cancer (follow-up 12 months randomized trials; 3–85 months to 26 years observational studies)1 randomized
trialsno serious limitations
no serious inconsistency
no serious indirectness very seriousa none 0/200(0%)
0/200(0%) — 0 fewer per 1000d OO CRITICAL
2 observational studies
no serious limitations
no serious inconsistency
no serious indirectness no serious imprecision
none 2/679(0.3%)
3/3350(0.1%) — 0 fewer per 1000e OO CRITICAL
Treatment unacceptable to women (follow-up 2 weeks; acceptability question)1 randomized
trialsno serious limitations
no serious inconsistency
no serious indirectness very seriousf none 15/170 (8.8%)
8/186 (4.3%)
OR 2.15 (0.89 to 5.22)
45 more per 1000(from 5 fewer to 147 more) OO CRITICAL
All severe adverse events (follow-up mean 12–16 months; stenosis and PID)2 randomized
trialsno serious limitations
no serious inconsistency
no serious indirectness very seriousf none3/300 (1%)
2/298f (0.67%) — 0.4 more per 1000
(from 8 fewer to 9 more) OO CRITICAL
All severe adverse events (follow-up 33 months; PID, plug syndrome, stenosis, blood transfusion) 5 randomized
trialsno serious limitations
no serious inconsistency
serioush serioush none 136 480OR 0.53
(0.1 to 2.88)
—
OO CRITICAL 4%i 18 fewer per 1000
(from 36 fewer to 67 more)
All severe adverse events (follow-up 12 months; PID, stenosis, major bleeding)9 observational
studies serious limitationsj
no serious inconsistency
seriousi seriousf none 1/2233(0%)
38/960(4%)a — 10 fewer per 1000
(from 20 fewer to 0) OOO CRITICAL
Should cryotherapy versus LEEP be used in women with histologically confirmed cervical intraepithelial neoplasia?
Recommendation
• In settings where LEEP is available and accessible, and women present with CIN lesions extending into the cervical canal, the expert panel suggests treatment with LEEP over cryotherapy (conditional recommendation, OO quality evidence)
• Remarks: The benefits of LEEP were greater than those of cryotherapy, and the harms were fewer in these women. However, since there are greater resource implications for LEEP than cryotherapy, and thus LEEP is not available in all settings, a conditional recommendation was made.
Implications of a conditional/weak recommendation
• Patients: The majority of people in this situation would want the recommended course of action, but many would not
• Clinicians: Be more prepared to help patients to make a decision that is consistent with their own values/decision aids and shared decision making
• Policy makers: There is a need for substantial debate and involvement of stakeholders
Implications of a strong recommendation
• Patients: Most people in this situation would want the recommended course of action and only a small proportion would not
• Clinicians: Most patients should receive the recommended course of action
• Policy makers: The recommendation can be adapted as a policy in most situations , can be used as quality indicator/performance measure
Only two of six performance measures seemed reasonable
WHO evaluation & feedback
• WHO staff & guideline review committee members (GRC) invited to feedback (Jan 2011) about using GRADE approach– what worked– what did not work
• Group discussion (NGT), 11 + 2
Summary of feedback
• Transparency of the GRADE process helps• Requirement for a good Chair
– methods knowhow to move the process• For observational studies we need better/
¿different? summaries (e.g. narrative versions) • Integration and elicitation of values and
preferences was frequently challenging• For global guidelines: issues around the
description of baseline risks and applicability across countries require work
Summary of feedback• Variability in baseline risk → weak or conditional
recommendations should follow • A description of the modifying factors and the layout
of the evidence could be a great benefit and will facilitate implementation
• Measures should be taken to streamline the timing of the development of guidelines
• Working with centers, training, and capacity building of these centers who collaborate with WHO a priority for implementing GRADE
• Impact evaluation (of current process for development of guidelines) should take place
Conclusions• (WHO) guidelines should be based on the best
available evidence to be evidence based• GRADE not avoid judgments but provides
framework• combines what is known in health research methodology
and provides an approach to improve communication
• GRADE process works – is it better?• Change in culture towards the use of evidence• Transparency in decision making and judgments is
key
Thanks
• Nancy Santesso, Andy Oxman, Suzanne Hill• WHO staff who participated in providing
feedback
Results - PRISMA
Studies awaiting assessment
(n = 6)•Studies awaiting translation (1)•Papers could not obtain in full (5)
Records identified through database searching (all study
designs)EMBASE, MEDLINE = 9873
SIGLE = 7CINAHL = 1062
LILACS = 19COCHRANE = 301
Chinese Biomedical Literature Database = 914
Panteleimon = 12(Total n = 12176)
Additional records identified through
other sourcesPharmaceutical
companies(n = 12)
Reference lists of relevant papers
(n=15)Records after duplicates removed(n = 7456)
Records screened(n = 7483)
Records excluded(n = 6563)
Full-text articles assessed for eligibility
(n = 920 )
Full-text articles excluded (n = 825)
Excluded for•Not influenza or influenza like illness•Fewer than 25 people•Randomised controlled trial, or not an observational study•Not antiviral agent•Antiviral agents analysed together•Prophylaxis•No outcomes reported
Studies includedN = 89
Question•51 + 5 studies •7 studies•6 studies•0 studies•8•0 studies•16•0 studies•1 study•2 studiesNote: one study may be relevant to multiple questions
Results
Study or Subgroup
Hanshaoworakul 2009Liem 2009 (1)McGeer 2009 (2)
Total (95% CI)
Heterogeneity: Tau² = 0.00; Chi² = 1.58, df = 2 (P = 0.45); I² = 0%Test for overall effect: Z = 4.63 (P < 0.00001)
log[Odds Ratio]
-2.040221-0.941609-1.309333
SE
0.587394160.751132390.4270348
Total
3155569
439
Total
13012
100
242
Weight
28.5%17.5%54.0%
100.0%
IV, Random, 95% CI
0.13 [0.04, 0.41]0.39 [0.09, 1.70]0.27 [0.12, 0.62]
0.23 [0.13, 0.43]
oseltamivir no treatment Odds Ratio
(1) Adjusted for neutropenia and hospital admission(2) Does not specify what was adjusted for
Odds RatioIV, Random, 95% CI
0.1 0.2 0.5 1 2 5 10Favours oseltamivir Favours no treatment
Study or Subgroup
Chemaly 2007Estenssoro 2010Hien 2009Huang 2009Li 2010Liem 2009McGeer 2009Siston 2010 (1)Xi 2009
Total (95% CI)
Total eventsHeterogeneity: Tau² = 0.70; Chi² = 16.76, df = 7 (P = 0.02); I² = 58%Test for overall effect: Z = 1.64 (P = 0.10)
Events
0150
520
188
2124
228
Total
253282517
1185568
476125
1237
Events
352108
3453
61
Total
884
572712
1007430
320
Weight
5.2%13.4%8.6%7.3%
14.4%18.9%17.3%14.9%
100.0%
M-H, Random, 95% CI
0.03 [0.00, 0.69]0.51 [0.12, 2.15]0.25 [0.03, 2.24]
7.47 [0.63, 88.02]Not estimable
0.24 [0.06, 0.92]0.26 [0.11, 0.60]0.64 [0.23, 1.74]2.14 [0.60, 7.64]
0.51 [0.23, 1.14]
Oseltamivir No treatment Odds Ratio
(1) Pregnant women
Odds RatioM-H, Random, 95% CI
0.001 0.1 1 10 1000Favours oseltamivir Favours no treatment
Should oseltamivir versus no treatment be used to treat influenza?Mortality (adjusted)
Mortality (unadjusted)
Recommendation
- The Guidelines Group recommends that TB
programs/clinicians use/do not use
fluoroquinolones in the treatment of all
patients with MDR (Strong (conditional)
recommendation/ low (very low, low, moderate,
high) grade of evidence)
Example: Oseltamivir for Avian Flu
Recommendation: In patients with confirmed or strongly suspected infection with avian influenza A (H5N1) virus, clinicians should administer oseltamivir treatment as soon as possible (strong recommendation, very low quality evidence).
Remarks: This recommendation places a high value on the prevention of death in an illness with a high case fatality. It places relatively low values on adverse reactions, the development of resistance and costs of treatment.
Schunemann et al. The Lancet ID, 2007
Other explanationsRemarks: Despite the lack of controlled treatment data for H5N1, this is a strong recommendation, in part, because there is a lack of known effective alternative pharmacological interventions at this time.
The panel voted on whether this recommendation should be strong or weak and there was one abstention and one dissenting vote.
Recommendation: In settings where LEEP/LLETZ is available and accessible, the expert panel suggests treatment with LEEP/LLETZ over cryotherapy
Population: Women with histologically confirmed CINIntervention: Cryotherapy versus LEEPFactor Decision Explanation High or moderate evidence(is there high- or moderate-quality evidence?) The higher the quality of evidence, the more likely is a strong recommendation.
Yes No ÅÅOO
There is moderate-quality evidence from both randomized and observational controlled studies for recurrence rates. However, there is low-quality evidence for other outcomes that were considered critical and important for decision-making (e.g. severe adverse events, cervical cancer). There is uncertainty for fertility and other obstetric outcomes, and HIV acquisition/transmission was not measured.
Certainty about the balance of benefits versus harms and burdens(is there certainty?) The larger the difference between the desirable and undesirable consequences and the certainty around that difference, the more likely is a strong recommendation. The smaller the net benefit and the lower the certainty for that benefit, the more likely is a conditional/ weak recommendation.
Yes No
Benefits of LEEP were greater, and harms were fewer or similar
Recurrence rates of CIN1, CIN2–3 and all CINs are probably greater with cryotherapy:o CIN2–3, odds ratio (OR) 3.3 (CI 1.04 to 10.46)o CIN1, OR 2.74 (CI 0.62 to 12.07)o All CIN, OR 2.14 (CI 1.05 to 4.33).
Cryotherapy may be less acceptable to patients than LEEP. There may be little difference in serious adverse events
between cryotherapy and LEEP, but there may be fewer minor adverse events (such as pain) with cryotherapy.
It is unclear whether there is a difference in fertility/obstetric outcomes.
High value was placed on CIN recurrence, serious adverse events and acceptability to the patient.
Low value was placed on minor adverse events.
Certainty in or similar values (is there certainty or similarity?) The more certainty or similarity in values and preferences, the more likely is a strong recommendation.
Yes No
Similar values across women
There is not a lot of variability The panel felt secure in assuming the populations value
Resource implications(are resources worth expected benefits?) The lower the cost of an intervention compared to the alternative that is considered and other costs related to the decision – that is, fewer resources consumed – the more likely is a strong recommendation.
Yes No
More resources required for LEEP
Need for more skilled providers to perform LEEP Need for more or expensive equipment/supplies for LEEP;
electricity supply for LEEP Need for local anaesthesia with LEEP
Overall strength of recommendationConditional
MethodsTypes of participants• We included studies in all populations with influenza or influenza like-illness. Types of intervention• Oseltamivir, zanamivir, amantadine or rimantadine in any dose or by any route.Type of outcome measures• We determined a priori to report on the following outcomes because they were
judged to be important or critical for decision making:• Mortality, Hospitalisation, ICU Admission, mechanical ventilation and respiratory
failure, Duration of hospitalization, Time to alleviation of symptoms, Time to return to normal activity, Complications
• Critical adverse events (e.g. major psychotic disorders, encephalitis, stroke and seizure),
• Important adverse events (e.g. pain in extremities, clonic twitching, body weakness, dermatological changes such as uticaria and rash)
• Viral shedding and Resistance
GRADE Uptake World Health Organization Allergic Rhinitis in Asthma Guidelines (ARIA) American Thoracic Society American College of Physicians European Respiratory Society European Society of Thoracic Surgeons British Medical Journal Infectious Disease Society of America American College of Chest Physicians UpToDate® National Institutes of Health and Clinical Excellence (NICE) Scottish Intercollegiate Guideline Network (SIGN) Cochrane Collaboration Infectious Disease Society of America Clinical Evidence Agency for Health Care Research and Quality (AHRQ) Partner of GIN Over 60 major organizations
GRADE Uptake World Health Organization Allergic Rhinitis in Asthma Guidelines (ARIA) American Thoracic Society American College of Physicians European Respiratory Society European Society of Thoracic Surgeons British Medical Journal Infectious Disease Society of America American College of Chest Physicians UpToDate® National Institutes of Health and Clinical Excellence (NICE) Scottish Intercollegiate Guideline Network (SIGN) Cochrane Collaboration Infectious Disease Society of America Clinical Evidence Agency for Health Care Research and Quality (AHRQ) Partner of GIN Over 60 major organizations
Recommendation 1a
• The panel recommends that people who had household contact with smear positive or M/XDR TB index cases be investigated for active TB (strong recommendation, very low quality evidence).
Recommendation 1b
• The panel recommends that people who had household contact with TB index cases who are children younger than 5 years of age be investigated for active TB (strong recommendation, very low quality evidence).