CAN-IMPLEMENT© for Guideline Adaptation and

Implementation

Part 2: The Methodology(August 2011)

3 Phase Methodology

PHASE 1Identification and

Clarification of Issue/Problem

PHASE 2Solution Building

PHASE 3Implementation, Evaluation and Sustainability

PHASE 1: Identification & Clarification of Issue/Problem

PHASE 1, Step 1: Call-to-Action

1.1 Clarify the motivation, purpose and scope of the proposed initiative. Consider:

  What is the agency/ institutional mandate and infrastructure

supporting evidence-informed practice? Is this a response to a specific practice challenge? Is a guideline the most appropriate solution to the challenge? Who (person/group) will lead, implement and maintain these

recommendations? What is the intended practice jurisdiction (local, regional,

national??

 

PHASE 1, Step 1: Call-to-Action

A critical, strategic element requiring strong facilitation and

leadership skills; establishes legitimate guideline development

mandate and infrastructure; especially important for new groups

Plan an orientation session for participants Discuss: What is a “guideline”?

Tools PHASE 1: Evidence-informed practice Training Resources

PHASE 1, Step 2: Plan

2.1 Establish scope of guideline and articulate Health Question(s).

2.2 Determine feasibility of adaptation.

2.3 Form steering committee and working panel(s) and determine key stakeholders and necessary resources

2.4 Determine consensus process.

2.5 Write the Adaptation Work Plan.

PHASE 1, Step 2: Plan

Use the PIPOH instrument to develop health questions Determine required expertise and resources;

(using the Skills Assessment Checklist) Understand Facilitation and the role of the Coordinator Draft Adaptation Work Plan – an essential document

outlining: Scope of topic and health questions Terms of reference (steering committee and working panel(s) Funding and resource commitments Consensus process Conflicts of interest Projected timeline Meeting arrangements

Tools PHASE 1: PIPOH Checklist

► Patient population: average risk women► Intervention: screening► Professionals: family physicians/ GPs/nurses► Outcomes: screening interval/modality► Healthcare setting: family practice

Example Question: What is appropriate cervical cancer screening (CCS)for average risk women seen in family practice?

Tools PHASE 1: Search Sources

Tools PHASE 1: Skills Assessment Checklist

Tools PHASE 1: Sample Declaration of Conflict of Interest CONFLICT OF INTEREST DISCLOSURE DECLARATION

Tools PHASE 1: Sample Adaptation Work Plan

PHASE 1, Step 3: Search and Screen

3.1 Search existing guidelines, systematic reviews, and new or emerging areas of evidence; confirm if guideline is de novo, adaptation or mixed initiative.

3.2 Screen search results to develop short list for full appraisal; document selection.

PHASE 1, Step 3: Search and Screen

Designing and executing the search - engaging services of a health science librarian or information specialist

Managing citations: Developing a screening protocol and documenting selection decisions

Library Science Supplement and Toolkit resources

Designing the search:Choosing inclusion/exclusion criteria

Selecting only evidence-based guidelines (guideline must include a report on systematic literature searches and explicit links between individual recommendations and their supporting evidence

Selecting only national and/or international guidelines; selecting guidelines written in a particular language Fr/Eng?)

Specifying a range of dates for publication; selecting only those published since an important review was published

Selecting peer reviewed publications only; excluding guidelines written by a single author not on behalf of an organization – ideally has multidisciplinary input

Excluding guidelines published without references – panel must have access to the evidence

Potential papers found(n=599)

Potential papers found(n=599)

Rejected – not CPGs(n=537)

Rejected – not CPGs(n=537)

Remaining more Carefully reviewed

(n=62)

Remaining more Carefully reviewed

(n=62)

Rejected – not population screening

(n=10)

Rejected – not population screening

(n=10)

Duplicates searched for in remaining

(n=52)

Duplicates searched for in remaining

(n=52)

Duplicates(n=32)

Duplicates(n=32)

Updates searchedfor in remaining

(n=20)

Updates searchedfor in remaining

(n=20)

Updated(n=2)

Updated(n=2)

Cervical cancer population

screening CPGs(n=18)

Cervical cancer population

screening CPGs(n=18)

8 from Medline9 from Web

1 from CCOHTA

8 from Medline9 from Web

1 from CCOHTA

Evidence/ Literature

Search and Screen

PHASE 1, Step 4: Assess and Select

4.1 Assess shortlisted guidelines (recommendations and supporting evidence) in detail for: quality (e.g. AGREE), currency, content, coherence between evidence and recommendations, and applicability and acceptability to local context.  4.2 Decision and Selection: review all assessments and achieve consensus with respect to Selecting, Rejecting or Modifying specific

PHASE 1, Step 4: Assess and Select

Assessing Quality of guidelines Using the AGREE II Instrument Summarizing and displaying AGREE scores Assessing Quality of Systematic Reviews

Assessing guideline Currency

Assessing guideline Content Preparing the ‘Recommendations Matrix’ The evidence: type and level; classification systems Critical appraisal (interpretation and Consistency of evidence)

Assessing Acceptability and Applicability

Making Decision to Accept, Reject or Modify Achieving and documenting consensus

www.agreetrust.org

Tools PHASE 1: AGREEII Instrument

23 items

7 point Scale

Overall Assessment

User Guide & Manual

1. Scope & purpose (3)

2. Stakeholder involvement (3)

3. Rigour of development (8)

4. Clarity of presentation (3)

5. Applicability (4)

6. Editorial independence (2)

SIX Domains

CURRENCY Assessment

Is there any new evidence relevant to guideline?

Does new evidence invalidate any of the recommendations?

Are there any plans to update the guideline in the near future?

When was the guideline last updated?

CONSISTENCY Assessment

Quality of source guideline search strategy and study selection ( ADAPTE Tool 13) Was the search for evidence comprehensive? Is there any bias in the selection of articles?

Consistency between evidence and interpretations; between interpretations and recommendations ( ADAPTE Tool 14) Is the evidence valid, overall? Are the recommendations based on data and

interpretations?

Tools PHASE 1: Evaluation: Search and Selection of Evidence

Tools PHASE 1: Evaluation Sheet: Scientific Validity of GuidelinesConsistency between evidence, its interpretation and recommendations)

Tools PHASE 1: Recommendations Matrix (template)

APPLICABILITY Assessment

Review of each of the recommendations with respect to 2 main questions (ADAPTE Tool 15)

Can the recommendation be put into practice? Consider patient similarity, interventions, outcomes, patient preferences, availability of equipment, availability of expertise, any constraints?

Is the benefit from this recommendation worth implementing?

Tools PHASE 1: Evaluation Sheet: Acceptability/Applicability

Decision and Selection; you might …

Use process of elimination based on overall AGREE scores

Eliminate any outdated CPGs

Look at recommendations of top 3 CPGs and decide which you could adopt in entirety; which you cannot reach agreement upon; which individual recommendations you can agree to adopt …

Decision and Selection Options

1. ACCEPT a whole guideline and all of its recommendations After reviewing all of the assessments, the panel accepts the guideline as is.

2. REJECT a whole guideline and all of its recommendations After reviewing all of the assessments, the panel decides to reject the complete

guideline. The decision will be based on how the panel weighs the assessments (e.g., poor AGREE scores, guideline is out-of-date, the recommendations do not apply to the panel’s context).

3. ACCEPT the evidence summary of the guideline After reviewing all of the assessments, the panel decides to accept the description of

the evidence (or parts) but to reject the interpretation and the recommendations.

4. ACCEPT a single recommendation After reviewing the recommendations from the guideline or guidelines, the panel

decides which to accept and which to reject which may be from one or more guidelines.

5. MODIFY single recommendations After reviewing all of the recommendations from the guideline(s), the panel decides

which are acceptable but need to be modified.

PHASE 1, Step 5: Draft, Review, and Endorse Recommendations

5.1 Draft Customized Guideline

5.2 Conduct internal review and make revisions

5.3 Conduct external review and obtain endorsement

5.4 Prepare final documents, including any practitioner and patient information, records or application tools, and appropriate source acknowledgments

5.5 Establish a Renewal Plan

PHASE 1, Step 5: Draft, Review, and Endorse Recommendations

Customizing recommendations: Using a template for structure and content Authors, acknowledgements, permissions and copyright

issues from source developers Using brief, unambiguous, actionable language Including application tools, algorithms, patient information Including a short preface summarizing recommendations,

and methodology; appendices and possible web links to documents

Important aspect: Transparency of all decision making (e.g., consensus process is described, how decisions were arrived at and resolved; if recommendations were modified, how and why they were modified);

External Review

External review with those affected by uptake, e.g., experts (practitioners, patients) and users (policy makers, managers)

Consultation with endorsement bodies Inclusion of representative on panel throughout process?

Consultation with source guideline developers

Acknowledgement of source documents

Tools PHASE 1: Sample External Review Survey

Sustainability/Planning for renewal

Guideline maintenance specifying in the guideline when, how and by whom the

recommendations will be updated

PHASE 2: Solution Building

PHASE 2, Step 1: Align Knowledge to Local Context (Practice and System)

Conduct a gap analysis

To focus an implementation action plan, it is necessary to first understand the difference between current practices and the new recommendations.

The objective of the gap analysis is to clarify exactly what and how much will need to change in the prevailing practice and

system.

Gathering information

Information about the local population, organization, and providers can be obtained from many sources using a variety of formal and informal methods, including:

Guideline documentation from PHASE 1 Steps 1-5, e.g. previous environmental scans or other baseline measurements, and meeting notes from discussions on re: acceptability and feasibility of recommendations

Clinical/Chart Audit

Interviews, focus groups, town hall sessions, surveys and questionnaires

PHASE 2, Step 2: Assess the Innovation, Adopters and Practice Environment for “Barriers and Supports”

Innovation (examples):

Attributes of the evidence-informed recommendations

The rigour/credibility of the guideline development process

Perceived compatibility of recommendations with existing routines (e.g. complexity, convenience, relative advantages or costs

PHASE 2, Step 2: Assess the Innovation, Adopters and Practice Environment for “Barriers and Supports”

Adopters (examples):

Awareness Knowledge and Skills Attitudes, expectations, motivation Current behaviours or routines Concerns, preferences

NOTE: Adopters include allied health practitioners (the care “providers”), administrative and organizational staff and leaders – and patients and their families!

PHASE 2, Step 2: Assess the Innovation, Adopters and Practice Environment for “Barriers and Supports”

Practice Environment/Organizational Context (examples):

Organizational culture and “readiness” for evidence-based practice Infrastructure; levels of authority and decision-making processes Resources; economic constraints Leadership; availability of Champions Work Load; levels of job stress Quality Improvement mechanisms

(Tools) RNAO “Environmental Readiness Assessment”

PHASE 2, Step 3: Select and Tailor Implementation Interventions

Best practice Implementation strategies should address barriers related to the individual practitioner, social context and organizational and environmental context - and should be tailored to different groups of stakeholders (i.e. nursing staff, project leaders and administrators.”) Ploeg, 2007

Implementation Strategies/Tactics

Educational Outreach Reminders: manual or computerized Introduction of computers in practice; computerized decision support Multi-faceted interventions Interactive small group meetings Multi-professional collaboration Mass media campaigns Financial interventions (fund-holding/budgets) Use of opinion leaders/champions Audit and feedback Conferences and courses Educational materials Patient-mediated interventions  Dissemination of printed educational material Didactic educational sessions (lectures) Total quality improvement/continuous quality improvement

Usability Testing

Barrier management and knowledge transfer are iterative processes.

Consider the value of testing new procedures, decision making algorithms, training information and other support tools with both patients and providers before proceeding with a full scale implementation.

Tools PHASE 2: Implementation Action Planning (template)

PHASE 3: Implementation, Evaluation & Sustainability

Evaluation Planning:Monitoring Uptake and Assessing Outcomes

In PHASE 3, groups evaluate both the level of evidence uptake and the impact of the new recommendations for care on the intended:

patient population health care providers organization

Evaluation Planning:Monitoring Uptake and Assessing Outcomes

Key Concepts:

Evaluating use of recommendations: Defining the evaluation period - from date of “knowledge

activation” to “targeted threshold” of guideline uptake

Evaluating implementation process Monitoring, revisiting and augmenting or modifying implementation “interventions” to enhance uptake

Evaluating impact: Defining specific ‘indicators’ Establishing tracking measures (data collection/resources) Assessing Patient, Practice (provider) and System outcomes

Using the CAN-IMPLEMENT© ResourceNavigation

Field Notes Tips Checklists Thinking about Implementation Progress Checks - Direction - Outputs - Documents - Tools