Gary W. K. Wong1,2, Marc Miravitlles2,3, Alison Chisholm2, Jerry Krishnan2,4

1. Department of Paediatrics and School of Public Health, Prince of Wales Hospital, The Chinese University of Hong Kong, Hong Kong, China;

2. The Respiratory Effectiveness Group, Cambridge, United Kingdom; 3. Pneumology Department, Hospital Universitari Vall d’Hebron, CIBER de

Enfermedades Respiratorias (CIBERES), Barcelona, Spain; 4. Population Health Sciences, University of Illinois, Hospital and Health

Sciences System, Chicago, Illinois

Respiratory Guidelines—Which Real World?

Clinical practice guidelines: purpose• Remaining abreast of the latest advances in

medical research and practice is a challenge for many practicing clinicians. Need too Digest o Translate and Apply the perpetual flow of new research data

• Clinical guidance aims to: help guide clinicians to understand, translate, and apply new evidence into everyday practice

Clinical practice guidelines: limitations• This purposes can be thwarted by:

o A lack of diversity and plurality of committee members

o By limitations in evidence grading methodologies

o Limitations in the evidence base and in randomised controlled trail (RCT) design, which largely underpin their recommendations.

Limitations: guideline committees

• Lack of consistency in the composition of respiratory guideline groups

• Clinical non-specialist stakeholder are often under representedo Experts panels are often dominated by

clinicians from university centers or tertiary hospitals who treat a subset of (more severe) patients

o No primary care physicians are listed as authors on GOLD’s Strategy document for the diagnosis, management, and prevention of COPD11. Vestbo J, et al. Am J Respir Crit Care Med 2013;187:347–365.

Limitations: guideline committees• Lessons can be learned from:

o Global Initiative for Asthma (GINA):2 Practitioners from primary care and specialists with knowledge implementation and dissemination have been recruited so GINA’s recommendations better reflect the practical applicability of frontline daily practice around the world.

o Spanish COPD guidelines (GesEPOC). Pharmacological treatment of stable COPDIncluded representatives from all scientific societies involved in the care of patients with COPD within the development group to ensure proportional representation of primary specialists.3

2. GINA website: http://www.ginasthma.org/Committees3. Miravitlles M, S et al. Arch Bronconeumol 2012;48:247–257.

Limitations: GRADE evidence appraisal (I)• The best accepted methodology for developing

guidelines is the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) system.

• GRADE has been adopted by more than 70 organizations around the world, including the American Thoracic Society (ATS), the World Health Organization (WHO), the Cochrane Collaboration, and the NICE (6).

• GRADE classifies the quality of evidence as very low, low, moderate, or high based on the methodological quality of the evidence and likelihood of outcome bias

4. Guyatt GH, et al on behalf of the GRADE Working Group. BMJ 2008;336:924–926.

Limitations: GRADE evidence appraisal (II)• GRADE’s quality assessment starts:

o RCTs as high-quality evidence ando Observational studies as of low-quality evidence.

• The following can reduce the quality of evidence from RCTs:o Poorly detailed design and execution, inconsistency,

indirectness, reporting bias, and imprecision. • Factors that can increase the quality of evidence from

observational studies include:o robust methods of handling potential sources of bias

(selection bias, recall bias, information bias, detection bias) and a priori study registration and planning

• GRADE then offers a quality classification for the associated recommendation (weak or strong).

4. Guyatt GH, et al on behalf of the GRADE Working Group. BMJ 2008;336:924–926.

Limitations: GRADE evidence appraisal (III)• GRADE offers a systematic approach to evidence

evaluationBUT• It is not practically possible to address all

clinical questions that may arise in everyday clinical practice or to have answers to all the potential outcomes of interest.

• The result is a limitation in the published evidence available to inform practicing clinicians.o Statement issued by the American College of

Physicians, the American College of Chest Physicians, and the European Respiratory Society on the management of COPD offered only 7 evidence-based recommendations.5

5. Qaseem A, et al. Ann Intern Med 2011;155:179–191.

• Studies have shown that efficacy RCTs exclude about 95% of asthma and 90% of COPD routine care populations due to strict inclusion criteria.6

6. Herland K, et al. Respir Med 2005;99:11–19.

Limitations: RCTs inclusions/exclusions

COPD

Asthma

Patient RCT eligibility drop-off with sequential application of standard inclusion criteria

Limitations: RCT design• In addition to strict inclusion/exclusion criteria,

RCTs typically involve:o Close patient–physician interactiono Rigorous intensity of follow-up (often at a level

infeasible in routine care)• RCT design is necessary to discern a direct cause-

and-effect between an intervention and an outcome but limit the generalizability of their findings.

• Observational studies, postmarketing, and pragmatic trials typically lack some of the internal validity of RCTs but offer greater external validity (i.e., generalizable).

Limitations: who do RCTs miss?• A recent study7 in >1,000 patients age ≥40 years

found that 45% of patients with a physician diagnosis of COPD do not meet the typical clinical trial criteria for defining COPD (known risk factor, such as smoking, and a reduced FVC/FEV1)

• The study found also found that: patients with a physician diagnosis of COPD who were younger, female, not white, and had diabetes or depression were more likely to be excluded from clinical trials on the basis of the reference standard.

• Efficacy trials in COPD often enroll a small, non-representative subset of patients with physician-diagnosed COPD.

7. Prieto-Centurion V, et al. Am J Respir Crit Care Med 2013;187:A5017.

Improving guidelines: evidence appraisal• Sir Michael Rawlins, the former chairman of the

UK’s National Institute for Health and Clinical Excellence:7

o Hierarchies of evidence are over-simplistic and offer a pseudoquantitative assessment of the available evidence.

o All forms of evidence should be considered, while taking into account the limitations and strengths of their respective designs

o Differently designed studies should be considered as complementary and should be used in combination by guideline developers to help inform their judgments and recommendations. 7. Rawlins M. De testimonio: on the evidence for decisions about

the use of therapeutic interventions. Lancet 2008;372:2152–2161.

Improving Guidelines: addressing gaps• Greater emphasis should be placed on

recognizing the gaps in their evidence review (i.e., on the indirectness of the evidence that efficacy trials present for most patient populations).

• GRADE provides the opportunity to downgrade the quality of evidence for “indirectness,” and guideline committees should carefully review this criterion in assigning the quality of evidence.

• The Pragmatic–Explanatory Continuum Indicator Summary (PRECIS) wheel provides a structured approach to assessing the extent to which studies may provide indirect evidence

8. Thorpe KE, et al. CMAJ 2009;180:E47–E57.

Improving Guidelines: PRECIS Wheel (I)

• 9 “spokes,” each representing a different element of the study design (e.g., study eligibility criteria, expertise of individuals applying the intervention).

• Each spoke, or axis, represents an explanatory– pragmatic (i.e., efficacy–effectiveness) continuum, and aspects of a trial are scored/positioned along each respective axis depending on the extent to which they reflect the characteristics of an explanatory (efficacy) RCT or a pragmatic effectiveness trial

8. Thorpe KE, et al. CMAJ 2009;180:E47–E57.

Improving Guidelines: PRECIS Wheel (I)

Two asthma trials mapped on the PRECIS wheel domains – A: Price et al9), B: an efficacy trial (Malmstrom et al10). The visually representation shows the clear gaps in the efficacy trial design

9. Price D, et al. Health. Technol Assess 2011;15:1–132.10. Malmstrom K, et al. Ann Intern Med 1999;130:487–495.

Integrating evidence• Large representative observational studies:

offer the possibility of analyzing patient subgroups of clinical interest (e.g.,pediatrics, elderly, patients with comorbidities, smokers, etc.) to explore whether RCT efficacy findings deviate in any of the groups of patients that are typically excluded from RCTs.

• Pragmatic effectiveness trials: conducted in patients with clinical disease diagnoses (i.e., those patients the physician believes have a condition and are treated accordingly) can further complement data from observational studies and efficacy RCTs.

Integrating evidence: example• Inform asthma management decisions in current

smokers.11

o Cigarette smoking associated with worsening symptoms, accelerated decline in lung function, and impaired response to corticosteroids (28–30).

o Smokers are typically excluded from asthma RCTs and guidelines provide no specific management advice

o Pragmatic trial & observational study data suggest possible utility in:– higher-dose inhaled corticosteroids,– targeting the small airways of the lungs– Targeting leukotrienes, – Possibly a combinations of these options

(27).

Conclusions: enhancing guideline utility• To achieve their goal of helping to guide the

meaningful implementation of new research into practice, for the benefit of all stakeholders, clinical guidelines would benefit from:o Wider stakeholder representation within

guideline committees (e.g., patients, primary and secondary care clinicians, policy makers, and health insurers) while still respecting the rules of evidence evaluation

o Keeping the ideas of “usability” and “readability” at the forefront of the development process

o Integration of effectiveness data as well as efficacy data into recommendations.

Conclusions: enhancing guideline utility• Guidelines must provide a wider view of the

management of diseases, one that reflects the diversity of decisions faced by clinicians.

• When conducting evidence reviews, guideline committees should consider not only the robustness of the available data but also its applicability.

• Evidence from effectiveness studies—pragmatic trials and observational studies—should be more consistently incorporated when assessing the risks and benefits of different care strategies. In some cases, (e.g., understanding the long-term sequelae of therapy), observational studies (rather than clinical trials) are needed.