Developing and Managing Quality Improvement Learning Collaboratives

September, 2010

Contributors: Brenda Joly PhD, MPH Maureen Booth, MRP George Shaler, MPH Ann Conway, PhD

Lessons from the MLC States

Overview

• Evaluation background – Components– Case study process, rationale, theory

• Findings and opportunities– Planning and start-up– Managing the Learning Collaborative

• Implications – Sponsors– Local health departments

MLC Evaluation

• Quality improvement goal: – To promote the application of QI methods

• Evaluation tools:– Annual survey– Mini-Collaborative survey– Quarterly reports– Case studies– Key informant interviews

Case Studies

• Who? – Mini-collaborative members and organizers– LHD quality improvement teams

• Why? – Multiple perspectives– “On-the-ground” understanding

• What?– Site visits, interviews, document review– Observations of meetings

Literature-Based Theory

Case Study Theory

Successful Mini-Collaboratives

Clear

Expectations

Cohesive Team

Strong Facilitators &

Content Experts

Sufficient Training and TA

Productive Meetings

Resources Provided to Agencies

Member Motivation and Engagement

Work Plan in Place and Used

Senior Leader Commitment

Available Evidence and Best Practices

Use of an Improvement

Model

Sufficient Time

Planning and Start-Up

Key Finding #1:

• Who selected targets? – State sponsors or statewide body– Local input

• What criteria were use in selection? – Relevance and alignment with priorities– Need demonstrated by data

The Relevance of a Target Area is Critical and Impacts Engagement

Key Finding #2:

• What works best? – Involving senior leadership is essential– Having diversity in faculty adds value– Having a sufficient size to promote exchange– Working on a consistent QI project– Having clear roles and responsibilities– Having prior experience working together

The Structure of a Mini-Collaborative Affects its Effectiveness

Key Finding #3:

• What did we learn? – Goals of mini-collaboratives varied and evolved

• Build QI skills• Improve quality within target area

– Goals should be aligned with timeframe and participant readiness

– Expectations for QI projects should be focused, realistic and communicated

Defining Expectations & Communicating Them in Advance Builds Confidence

Key Finding #4:

• Who planned the mini-collaboratives? – State sponsors– Broader group of state, faculty and others

• What should be planned?– Model, curriculum, workplan, use of faculty and

evidence, tools to assess participant knowledge and measures to monitor progress

Advanced Planning Influences the Effectiveness of Learning Collaboratives

Key Finding #5:

• What did we learn? – The planning and start-up phase is especially

time consuming for sponsors– Having outside faculty helps– Level of effort is often underestimated if

expectations are unclear or scope is unrealistic– Competing priorities are a reality

The Level of Effort Among Sponsors and LHDs is Often Underestimated

Managing the Learning Collaborative

Key Finding #1:

• What was most helpful? – Learning/using fewer tools– Training on when and how to use– Providing opportunity for immediate application– Providing feedback on use– Giving LHDs a chance to practice, practice,

practice

Opportunities for Timely and Frequent Application of QI Tools are Essential

Key Finding #2:

• What was most helpful? – Opportunities for ongoing information exchange– In-person meetings– Structured learning sessions– Communicating the value of QI to LHDs– Site visits to LHDs– Communicating clear expectations

Communication With and Among Members Enhances Learning Experience

Key Finding #3:

• Facilitating factors? – Commitment to data driven QI and accreditation– Leadership buy-in

• Impeding factors?– Concurrent QI skill development at state and local

level– Lack of focus and clear expectations, unrealistic

goals, turnover, and competing priorities

Several Factors May Strengthen or Impede a QI Learning Collaborative

Key Finding #4:

• Technical assistance: – There should be a good mix of content expertise

and QI expertise among sponsors and faculty

• Other resources:– Evidence needed to select QI intervention should

be made available– Financial resources may boost efforts of LHDs

It is Critical for Participants to Have Access to Adequate Resources

Implications for Sponsors

• Some considerations…– Set time aside for advance planning – Recruit credible and skilled faculty– Assess QI knowledge beforehand– Recruit the enough and the right mix of

participants– Embed the application of QI into the process– Communicate with the group frequently– Follow a structured approach

Implications for LHDs

• Some considerations…– Garner support of leadership and staff– Promote the value of QI– Align QI project with job responsibilities– Actively engage in information exchange– Use faculty and available technical assistance– Remain open to the process and outcomes– Document your efforts– Share your findings

Developing and Managing a Quality Developing and Managing a Quality Improvement Learning CollaborativeImprovement Learning Collaborative

September 2010September 2010

Contributors:Contributors: Cathy Montgomery, M.S., ASQ-CQIACathy Montgomery, M.S., ASQ-CQIA Sandra Ruzycki, M.P.H., Quad-RSandra Ruzycki, M.P.H., Quad-R Baker, Clay, DeSoto, Duval, Glades, Martin, Nassau, St.Baker, Clay, DeSoto, Duval, Glades, Martin, Nassau, St. Johns and St. Lucie County Health DepartmentsJohns and St. Lucie County Health Departments

Lessons from FloridaLessons from Florida

Planning and Start-UpPlanning and Start-Up

Plan, Plan, PlanPlan, Plan, Plan

• Use data to determine target Use data to determine target areaarea– Overweight and obesity Overweight and obesity

among children 6-19among children 6-19

• Create partnershipsCreate partnerships

• Outline timeframes and Outline timeframes and expectations expectations ΨΨ

• Establish criteria for Establish criteria for selecting participantsselecting participants ΨΨ = lesson learned = lesson learned

Use CriteriaUse Criteria

Childhood Obesity Mini-CollaborativeChildhood Obesity Mini-Collaborative

Establish (facilitator) GoalsEstablish (facilitator) Goals

• Use of quality Use of quality improvement tools and improvement tools and methodsmethods

• Support participants in Support participants in their projectstheir projects

• Communicate project Communicate project impacts and outcomesimpacts and outcomes– Local, state, national Local, state, national

venuesvenues

• Quality Improvement Control Quality Improvement Control Story Story ΨΨ– Founded on PDCAFounded on PDCA

• Integrates QI tools and Integrates QI tools and methods throughout the methods throughout the processprocess

Select an Improvement ModelSelect an Improvement Model

Training and ToolsTraining and Tools

• Assess participant’s Assess participant’s knowledge of QI tools knowledge of QI tools ΨΨ

• Provide relevant trainingProvide relevant training– Just-in-timeJust-in-time– Face-to-face Face-to-face ΨΨ

• Allow participants to Allow participants to practice using tools and practice using tools and processprocess

Results: TrainingResults: Training• Participation in QI training improved overall Participation in QI training improved overall

knowledge of QI tools – 84%knowledge of QI tools – 84%– Baseline = 84%Baseline = 84%

• Confident in their ability to use QI tools – 68%Confident in their ability to use QI tools – 68%– Baseline = 62%Baseline = 62%

• % of participants that rated level of use of QI tools % of participants that rated level of use of QI tools after training as: after training as:

moderate – 58%moderate – 58%– Baseline = 62%Baseline = 62%

high – 21%high – 21%– Baseline = 15%Baseline = 15%

*Surveys conducted March 2009 (baseline) and February 2010*Surveys conducted March 2009 (baseline) and February 2010

Results: ToolsResults: Tools

• Use QI tools in other initiatives – 83% Use QI tools in other initiatives – 83% (n = 15)(n = 15)

• Incorporate QI tools into ongoing work – 67% Incorporate QI tools into ongoing work – 67% (n=12)(n=12)

• Development of evaluation measures – 56% Development of evaluation measures – 56% (n = 10)(n = 10)

• Present QI tools to CHD staff – 50% Present QI tools to CHD staff – 50% (n=8) (n=8) ΨΨ

% of participants that indicated a high readiness to:% of participants that indicated a high readiness to:

Technical AssistanceTechnical Assistance

• Conduct coaching callsConduct coaching calls– Using quality improvement process and tools Using quality improvement process and tools – Creating action plans and storyboardsCreating action plans and storyboards– Developing methods and measures for evaluationDeveloping methods and measures for evaluation– Analyzing data resultsAnalyzing data results

• ResourcesResources– Subject matter expertiseSubject matter expertise– Books, articles, and research related to QI and projectsBooks, articles, and research related to QI and projects– Evaluation databaseEvaluation database

Evaluation DatabaseEvaluation Database

Managing the Learning Managing the Learning CollaborativeCollaborative

Collaborative TeamsCollaborative Teams

• Counties created teams Counties created teams consisting of internal consisting of internal staffstaff– 65% of participants 65% of participants

indicated multiple CHD indicated multiple CHD programs were programs were represented on teamrepresented on team

– 3 of 9 counties had QI 3 of 9 counties had QI program staff on the teamprogram staff on the team

Results: Collaborative TeamsResults: Collaborative Teams

• Team meetings were random and unproductive ΨΨ – 53% of participants stated team meetings were only

slightly productive

– 53% moderately or slightly agreed there was consensus on project goals and procedures

• % of participants that strongly or moderately agreed project team members:– Got along well – 94%Got along well – 94%– Treated each other fairly – 71%Treated each other fairly – 71%– Had necessary skills for this project – 65%Had necessary skills for this project – 65%

Results: CHD SupportResults: CHD Support

• 59% rated level of CHD 59% rated level of CHD support was very or support was very or somewhat supportivesomewhat supportive

• 71% indicated staff time 71% indicated staff time and resources were fully, and resources were fully, mostly or somewhat mostly or somewhat availableavailable

CommunicationCommunication

• Provide venues for participants to share ideasProvide venues for participants to share ideas– Monthly conference calls/web exMonthly conference calls/web ex– Face-to-face meetingsFace-to-face meetings– SharePoint siteSharePoint site

• Conduct one-on-one coaching callsConduct one-on-one coaching calls

• Communicate progress of collaborative projects Communicate progress of collaborative projects ΨΨ

Results: CommunicationResults: Communication• % of participants who rated the following resources % of participants who rated the following resources

as usefulas useful– One-on-one coaching calls:One-on-one coaching calls: 65%65%– Monthly conference calls: 65%Monthly conference calls: 65%– SharePoint site: 59%SharePoint site: 59%– Collaborating with other MCLC counties: 71%Collaborating with other MCLC counties: 71%– Collaborating with other agencies: 53% Collaborating with other agencies: 53% ΨΨ

• % of participants who agreed or strongly agreed HPI % of participants who agreed or strongly agreed HPI provided:provided:– Effective communication: 94%Effective communication: 94%– Effective project coordination: 95%Effective project coordination: 95%

• 76% rated their overall experience 76% rated their overall experience as excellent or goodas excellent or good

• 94% rated the level of success of 94% rated the level of success of their project as excellent or goodtheir project as excellent or good

• 88% indicated they were likely to 88% indicated they were likely to participate on another learning participate on another learning collaborativecollaborative

• 76% reported they were likely to 76% reported they were likely to implement a new QI project in implement a new QI project in their CHDtheir CHD

Overall ExperienceOverall Experience

Lessons Learned Lessons Learned ΨΨ

What else did we learn?What else did we learn?

• Planning and implementation is time consumingPlanning and implementation is time consuming– 41.2% of participants spend 1-5 hours per week on 41.2% of participants spend 1-5 hours per week on

project; others indicated they spend more time!project; others indicated they spend more time!

• Don’t assume what they knowDon’t assume what they know

• Competing priorities are a realityCompeting priorities are a reality