Rapid Guideline Development · Rapid Guideline Development For Professional Medical Associations...

Rapid Guideline Development For Professional Medical Associations

Richard Rosenfeld, SUNY Downstate Stephanie Jones, AAO-HNSF


Learning Objectives Recognize characteristics of trustworthy guidelines

Understand principles of rapid & efficient guideline development Identify strategies for conflict prevention and resolution

Competing Interests

Richard Rosenfeld: Journal Editor AAO-HNSF Stephanie Jones: Director Research & Quality AAO-HNSF

EMPOWERING PHYSICIANS TO DELIVER THE BEST PATIENT CARE

The American Academy of Otolaryngology—Head and Neck Surgery (AAO-HNS) is the world's largest organization representing specialists who treat the ear, nose, throat, and related structures of the head and neck. The Academy represents more than 12,000 otolaryngologist—head and neck surgeons who diagnose and treat disorders of those areas. Headquarters in Alexandria, VA

Who are we?

AAO-HNSF Staffing (4 FTE) 0.5 FTE Stephanie Jones, BS, Director, Research & Quality Improvement 0.5 FTE Gene Cunningham, MS, Senior Manager, Quality Measurement/Nat’l Coalitions 1 FTE Leslie Caspersen, MBA, Senior Manager, Guideline Dissemination &

Implementation 1 FTE Maureen D’Antuono Corrigan, BA, Analyst 1 FTE Vacant, Analyst

AAO-HNSF Volunteers Richard M. Rosenfeld, MD, MPH, Methodologist & Sr. Advisor Quality & Guideline Seth R. Schwartz, MD, MPH, Methodologist & Chair, Guideline Task Force Sujana S. Chandrasekhar, MD, Methodologist

Title Date released Page views

Clinical Practice Guideline: Acute Otitis Externa 7/14/2006 78,288

Clinical Practice Guideline: Adult Sinusitis 8/22/2008 61,861

Clinical Practice Guideline: Benign Paroxysmal Positional Vertigo 4/17/2009 47,068

Clinical Practice Guideline: Cerumen Impaction 4/17/2009 39,974

Clinical Practice Guideline: Hoarseness (dysphonia) 4/23/2010 28,771

Clinical Practice Guideline: Tonsillectomy in children 3/15/2011 27,246

Clinical Practice Guideline: Polysomnography for Sleep Disordered Breathing Prior to Tonsillectomy in Children

12/1/2011 14,824

Clinical Practice Guideline: Sudden Hearing Loss 4/1/2012 21,305

Clinical Practice Guideline: Improving Voice Outcomes after Thyroid Surgery

6/1/2013 *

Clinical Practice Guideline: Tympanostomy Tubes in Children 7/1/2013 *

* Submitted to the NGC, but not yet posted TOTAL 319,337

AAO-HNSF Guideline Usage SummaryThe following table contains the cumulative number of page views for each AAO-HNSF guideline listed on the National Guideline Clearinghouse (NGC) website from the time the guideline was posted to the NGC through June 2013


Have a plan and follow it

Learn from Others: Conferences, Workshops, Webinars, Publications

AAO-HNSF G-I-N Scholars Training Program

Francis Bacon, Sr. English lawyer and philosopher, 1561-1626

Reading maketh a full man; Conference a ready man;

and writing an exact man.

Rosenfeld’s addendum: Publishing and post-

publication peer review maketh a wise man.

Clinical Practice Guideline Development Manual: Third Edition

Pragmatic, transparent approach to creating guidelines for performance assessment

Evidence-based, multidisciplinary process leading to publication in 12-18 months

Emphasizes a focused set of key action statements to promote quality improvement

Uses action statement profiles to summarize decisions in recommendations

Rosenfeld, Shiffman, and Robertson

Otolaryngol Head Neck Surg 2013; 148(Suppl):S1-55

AAO-HNS CPG Manual, 3rd ed. Otolaryngol Head Neck Surg 2013; 148(Suppl):S3

Standards for Developing Trustworthy Clinical Practice Guidelines

Standard 1. Transparency

1.1 The processes by which a CPG is developed and funded should be detailed explicitly and publicly accessible

http://www.iom.edu/Reports/2011/Clinical-Practice-Guidelines-We-Can-Trust/Standards.aspx

Clinical Practice Guideline Development Manual: Third Edition



Get the right people on the bus

Who’s on Your Guideline Bus?

Jim Collins. Good to Great New York: Harper Business 2001

Most people assume that great bus drivers (read: business leaders) immediately start the

journey by announcing to the people on the bus where they’re going – by setting a new direction

or by articulating a fresh corporate vision.

In fact, leaders of companies that go from good to great start not with “where” but with “who.”

They start by getting the right people on the bus, the wrong people off the bus, and the right

people in the right seats.

Keeping the Wrong People Off the Bus

Standard 2. Conflict of Interest

2.1 Guideline development group (GDG) members should declare all interests

2.2 All COIs should be disclosed and discussed within the GDG

2.4 Chair should not have a COI, only a minority of members should have COI, funders have no role in CPG development


Intellectual Conflict of Interest

Academic activities that create the potential for an attachment to a specific point of view that could unduly affect an individual’s judgment – Guyatt

Research, publications, or grant support related to the guideline Being a chair or member of a related guideline committee Membership in a related lobbying or advocacy organization Leadership in a group that may gain from a guideline development

group member’s opinion Acting as an expert witness or having membership in a related

advisory or governing board with other organizations or funders Family members with the target condition

a.k.a. Anti-Rapid CPG Development Poison Pill

Graham R, CPGs We Can Trust, IOM 2011 Norris SL, PloS ONE 2011; 6:e25153 and Guyatt G, Intern Med 2010; 152:738-41.

Getting the Right People on the Bus

Standard 3. Guideline Development Group (GDG) Composition

3.1 The GDG should be multidisciplinary and balanced, comprising a variety of methodological experts and clinicians, and populations expected to be affected by the guideline.


Multidisciplinary Guideline Panels

Increases the probability that all relevant scientific evidence will be located and critically evaluated

Increases the chances that the panel will address practical problems relating to application of the guidelines

Helps build support among the groups for whom the guideline is intended

May produce more reliable results by balancing biases of the various individuals on the panel

Why Bother to Diversify?

Shekelle et al. Clinical guidelines: developing guidelines. BMJ 1999; 318:593-6 Institute of Medicine. Clinical practice guidelines. Washington DC: Nat’l Academy Press, 1990


Guideline Development Group Composition

Sir William Osler Canadian Physician, 1849-1919

Common sense in medical matters

is rare, and is usually in inverse ratio to

the degree of education.

What do Consumers contribute to GDGs?

Consumer Involvement in Guidelines What are the Possibilities?

Common Sense Perspective Skepticism

Respect for harms Patient education Shared decisions


Standard 3. Guideline Development Group (GDG) Composition

3.2 Patient and public involvement should be facilitated by including (at least at the time of clinical question formulation and draft CPG review) a current or former patient, and a patient advocate or patient/consumer organization representative in the GDG


Begin with the End in Mind

Consumers do not have to be content experts! (the same applies to clinicians…of whom “experts” should be a minority)

Habit #2, Stephen Covey

Consumers

AAO-HNS Guideline Panels

Avoid surprises and sabotage by ensuring that all stakeholders are represented on the guideline development group

Identify participants who are qualified to represent their constituency and communicate with leadership

Ensure that participants truly understand their role and represent their constituency (beware of “experts”)

Schedule dates and times for calls and meetings in advance, and make availability a prerequisite of participation

Promote ownership through authorship, including staff

Principles for Rapid Guideline Development




Keep the scope manageable

Guidelines ARE NOT Review Articles! Guidelines contain key statements that are action-oriented

prescriptions of specific behavior from a clinician

Monitor

Test

Gather Interpret Perform Dispose

Action

Conclude Prescribe

Educate

Document

Procedure

Consult

Advocate

Prepare

Keeping Guideline Scope Manageable

Discuss purpose before scope: Why was the topic picked? What are the QI goals? What is the intended impact?

Define target condition (procedure), target patient, intended audience and practice settings, outcomes

Acknowledge that some issues important to some stakeholders will inevitably be left out of the guideline

Move from broad topics to answerable questions


Ranked Topic List for AAO-HNS Guideline on Voice Outcomes after Thyroid Surgery


Standard 4. Systematic Reviews

4.1 CPG developers should use systematic reviews that meet IOM standards.

4.2 When reviews are conducted specifically to inform particular guidelines, the GDG and systematic review team should interact regarding the scope, approach, and output of both processes.


Two Approaches to Evidence and Guidelines

Evidence as Protagonist Model Development is driven by the literature search,

which takes center stage with exhaustive evidence tables or textual discussions that rank and summarize citations.

Evidence as Supporting Cast Model Development is driven by a priori considerations of

quality improvement, using the literature search as one of many factors that are used to translate evidence into action.

1. Promote appropriate care 2. Reduce inappropriate or harmful care 3. Reduce variations in delivery of care 4. Improve access to care 5. Facilitate ethical care 6. Educate & empower clinicians & patients 7. Facilitate coordination & continuity of care 8. Improve knowledge base across disciplines

Quality Improvement Opportunities

Eden J, Wheatley B, McNeil B, Sox H (eds).Washington, DC: Nat’l Academies Press

a.k.a. Potential topics for guideline action statements





Make group time productive

Group Facilitation

Have an agenda, follow it, and appoint a time keeper; focus on issues requiring group interaction and exchange

Remind group members they are representing a specific point or view or discipline, not serving as an official spokesperson

Encourage all to contribute; use round-robin when necessary

Beware of unbridled passion (and potential intellectual conflicts) from experts & leaders

Defer discussions to a later time if an impasse is reached (lets passion cool and reason return)



Standard 6. Articulation of Recommendations

6.1 Recommendations should be articulated in a standardized form detailing precisely: what the recommended action is, and under what circumstances it should be performed.

6.2 Strong recommendations should be worded so that compliance with the recommendation(s) can be evaluated.


Clear Methodology and Format Facilitates

Group Consensus

Key Action Statement

Action Statement Profile

Supporting & Amplifying Text


Standard 5. Evidence Foundations

5.1 For each recommendation provide: Clear description of benefits & harms Quality, quantity, and consistency of the

available aggregate evidence Role of values, opinion, theory, and clinical

experience in deriving the recommendation Rating of confidence in the evidence Rating of the strength of recommendation Explanation of any differences of opinion


Rosenfeld RM, et al. CPG: Tympanostomy Tubes in Children. Otolaryngol HNS 2013; 149:S1-35

Action Statement Profiles and Guideline Development 1. Encourage an explicit and transparent

approach to guideline writing

2. Force guideline developers to discuss and document the decision making process

3. Create “organizational memory” to avoid re-discussing already agreed upon issues

4. Allow guideline users to rapidly understand how and why statements were developed

5. Facilitate identifying aspects of guideline best suited to performance assessment

Key action statement with recommendation strength and justification Supporting text for key action statement Action statement profile: Aggregate evidence quality: Confidence in evidence: Benefit: Risk, harm, cost: Benefit-harm assessment: Value judgments: Intentional vagueness: Role of patient preferences: Differences of opinion: Exclusions:

Executive Summary: Tympanostomy Tubes in Children. Otolaryngol HNS 2013; 149:8-16

Building Better Guidelines with BRIDGE-Wiz

Description of a software assistant for structured action statement creation to promote clarity, transparency and implementability

Shiffman…Rosenfeld et al, JAMIA 2012

J Am Med Inform Assoc 2002; 19:94-101.

1. Choose an action type 2. Choose a verb 3. Define the object for the verb 4. Add actions 5. Check executability 6. Define conditions for the

action 7. Check decidability

8. Describe benefits, risks, harms & costs 9. Judge the benefit-harms balance 10. Select aggregate evidence quality 11. Review proposed strength of

recommendation and level of obligation 12. Define the actor 13. Choose recommendation style 14. Edit the final statement

Aristotle Philosopher, Scientist, Physician 384-322 BC

It is the mark of an educated mind to rest

satisfied with the degree of precision which the nature of

the subject admits…

…and not to seek exactness where only an

approximation is possible.

Adapted from GRADE






Plan for efficient review

Internal Guideline Review Guideline Implementability Appraisal (GLIA) Yale Center for Medical Informatics

BMC Med Informatics Decis Making 2005; 5:23-31

Decidability Precisely under what circumstances to do something Executability Exactly what to do under the circumstances defined Effect on process of care

Degree to which the recommendation impacts workflow in a typical case setting

Presentation and formatting

Degree to which the recommendation is recognizable and succinct

Measurable outcomes

Degree to which the guideline identifies markers or endpoints to track the effects of implementation

Apparent validity Degree to which the recommendation reflects the intent of the developer and the strength of evidence

Novelty / innovation

Degree to which the recommendation proposes behaviors considered unconventional

Flexibility Degree to which a recommendation permits interpretation and allows for alternatives in execution


Standard 7. External Review

7.1 External reviewers should comprise a full spectrum of relevant stakeholders, including scientific and clinical experts, organizations, agencies, patients, and representatives of the public.

7.3 The GDG should consider all external reviewer comments and keep a written record of the rationale for modifying or not modifying a CPG in response to comments.


Written Record of Reviewer Comment Disposition

External Review of Draft Guideline

Identify reviewers through organizational outreach; must include all relevant stakeholders

Reviewers complete COI and confidentiality forms; informed in advance of strict date for comment submission

Ensure understanding of the process: comment disposition, differences from standard peer-review, feedback to reviewers

Staff collate and organize all comments into a master grid with reviewer names removed

Chair and assistant chairs triage comments with group review


Additional External Review

Final CPG

Public Comment

Board of Directors

Journal Peer-

Review

Oliver Wendell Holmes, Jr. US Supreme Court Justice, 1841-1935

Certitude is not the test of certainty.

We have been cocksure of

many things that were not so.


[email protected] [email protected]





Plan for efficient review

Rapid Guideline Development · Rapid Guideline Development For Professional Medical Associations...

Documents

Transcript of Rapid Guideline Development · Rapid Guideline Development For Professional Medical Associations...