Better Quality Through Better Measurement · 3. I am familiar with this topic but would have to...
Transcript of Better Quality Through Better Measurement · 3. I am familiar with this topic but would have to...
Institute for Healthcare Improvement
Better Quality ThroughBetter MeasurementFaculty
Robert Lloyd, PhD, IHI
Mukesh Thakur, MD, Hamad General Hospital
Akhnuwkh Jones, MD, Hamad General Hospital
24 March 2018
10:30 AM – 11:30 AM
and
1:00 PM – 2:05 PM
2018 Mideast Forum on Quality and Safety in Healthcare
The presenters have nothing to declare
© 2016 Institute for Healthcare Improvement/R. Lloyd
Is the process standardized?
Organize a team
Diagnose the problem and
related process(es)
NO
YES
Standardize
theprocess(es)
Identify potential measures
Select process, outcome and
balancing measures
Develop Operational Definition(s)
Collect & plot the data on a run or control
chart
Are special causes
present?
NOYES Identify Change Concepts and Ideas
that can be placed into PDSAs
Identify an opportunity for improvement
Investigate & Eliminate
A
The Quality Improvement JourneySource: Carey, R. and Lloyd, R. Measuring Quality Improvement in Healthcare: A Guide to Statistical Process Control Applications. ASQ Press, Milwaukee, WI, 2001.
Steps in the QI Journey
Steps in the Quality Measurement Journey
Institute for Healthcare Improvement
© 2016 Institute for Healthcare Improvement/R. Lloyd
Did the ideas have the desired impact?
Run PDSA tests
Collect additional data
on the key measures and update the run
or control charts
NO
YES
Modify the improvement
strategy
Implement the ideas, sustain the gains and
consider spread
Continue to monitor the
new processes and report ongoing results
Identify other opportunities for
improvement
Select specific ideas for change
A
Disseminate results as
appropriate
The Quality Improvement JourneySource: Carey, R. and Lloyd, R. Measuring Quality Improvement in Healthcare: A Guide to Statistical Process Control Applications. ASQ Press, Milwaukee, WI, 2001.
Steps in the QI Journey
Steps in the Quality Measurement Journey
The Improvement Guide, API, 1996
A Model for Learning and Change
When you combine
the 3 questions with the…
…the Model for
Improvement.
PDSA cycle, you get…
Institute for Healthcare Improvement
5
Discussion Topics
1. What is your current level of knowledge about quality measurement?
2. What is your motivation for measuring?
3. Do you know the milestones in the Quality Measurement Journey (QMJ)?
4. Do you understand variation conceptually?
5. Do you understand variation statistically?
6. Do you link measurement to improvement?
Question #1What is your current level of knowledge about
quality measurement?
This self-assessment is designed to help quality facilitators and improvement team members gain a
better understanding of where they personally stand with respect to the milestones in the Quality
Measurement Journey (QMJ). What would your reaction be if you had to explain why is it
preferable to plot data over time rather than using aggregated statistics and tests of significance?
Can you construct a run chart or help a team decide which measure is more appropriate for their
project?
You may not be asked to do all of the things listed below today or even next week. But if you are
facilitating a QI team or expect to be able to demonstrate improvement, sooner or later these
questions will be posed. How will you deal with them?
The place to start is to be honest with yourself and see how much you know about concepts and
methods related to the QMJ. Once you have had this period of self-reflection, you will be ready to
develop a learning plan for yourself and those on your improvement team.
R. Lloyd, Quality Health Care: A Guide to Developing and Using Indicators.
Jones & Bartlett Publishers, 2004: 301-304.
Institute for Healthcare Improvement
ExerciseMeasurement Self-Assessment
Select the one response which best captures your opinion:
1. I'd definitely have to call in an outside expert to explain and apply this topic.
2. I’ve heard of this topic but I would not feel comfortable applying it to a team’s work.
3. I am familiar with this topic but would have to study it further before I felt comfortable
explaining it to a team.
4. I have knowledge about this topic and feel confident that I could help a team apply it to
their improvement efforts but I would not want to stand up and teach this to a large group.
5. I consider myself an expert in this area and could apply easily to a team’s work as well
teach this topic to large groups.
R. Lloyd, Quality Health Care: A Guide to Developing and Using Indicators.
Jones & Bartlett Publishers, 2004: 301-304.
Measurement Self-AssessmentR. Lloyd, Quality Health Care: A Guide to Developing and Using Indicators. 2nd Edition, Jones & Bartlett Publishers, 2017: 339-341.
Measurement Topic or SkillResponse Scale
1 2 3 4 5
Help people in my organization determine why they are measuring (improvement, judgment or research)
Move teams from concepts to specific quantifiable measures
Building clear and unambiguous operational definitions for our measures
Develop data collection plans (including stratification and sampling strategies)
Explain why plotting data over time (dynamic display) is preferable to using aggregated data and summary statistics (static display)
Explain the differences between random and non-random variation
Construct run charts (including locating the median)
Explain the reasoning behind the run chart rules
Interpret run charts by applying the run chart rules
Explain the statistical theory behind Shewhart control charts (e.g., sigma limits, zones, special cause rules)
Describe the basic 7 Shewhart charts and when to use each one
Help teams select the most appropriate Shewhart chart for their measures
Describe the rules for special cause variation on a Shewhart chart
Help teams link measurement to their improvement efforts
1. I'd definitely have to call in an outside expert to explain and apply this topic/method. 2. I'm not sure I could apply this appropriately to a project.3. I am familiar with this topic but would have to study it further before applying it to a project.4. I have knowledge about this topic, could apply it to a project but would not want to be asked to teach it to others.5. I consider myself an expert in this area, could apply it easily to a project and could teach this topic/method to others.
Institute for Healthcare Improvement
© 2016 Institute for Healthcare Improvement/R. Lloyd
9Quality
Improvement
Control Assurance
Quality Control is a process by
which procedures and methods are
established to review and
standardize the reliability and
quality of all factors involved in the
production of products or services.
Quality Assurance is any
systematic process of
checking or auditing
periodically to see if a
product or service being
developed is meeting
specified requirements,
targets or goals.
Quality Improvement is the combined
and unceasing efforts of everyone (e.g.,
healthcare professionals, patients and
their families, researchers, payers,
planners and educators) to make the
changes that will lead to better patient
outcomes (e.g., health), better system
performance (e.g., care) and better
professional development.
Question #2
What is your motivation for measuring?
R. Lloyd, Quality Health Care: A
Guide to Developing and Using
Indicators. 2nd Edition, Jones & Bartlett Publishers, 2017.
© 2016 Institute for Healthcare Improvement/R. Lloyd
QualityBetter
Old Way(Quality Assurance)
QualityBetter Worse
New Way(Quality Improvement)
Action taken on all occurrences
Reject
defectives
Quality Assurance vs Quality Improvement
Source: Robert Lloyd, Ph.D.
Requirement,
Specification or Target
No action
taken
here
Worse
Institute for Healthcare Improvement
11
AIM (Why are you measuring?)
Concept
Measure
Operational Definitions
Data Collection Plan
Data Collection
Analysis ACTION
Question #3
Do you know the Milestones in theQuality Measurement Journey
R. Lloyd, Quality Health Care: A Guide to Developing and Using Indicators. 2nd Edition, Jones & Bartlett Publishers, 2017.
© 2016 Institute for Healthcare Improvement/R. Lloyd
AIM – reduce inpatient harm by 37% by the end of the calendar year
Concept – reduce inpatient falls
Indicator – Inpatient falls rate (falls per 1000 patient days)
Operational Definitions - # falls/inpatient days
Data Collection Plan – monthly; no sampling; all IP units
Data Collection – unit submits data to QI Dept. for analysis
Analysis – control chartACTION
A CompletedQuality Measurement Journey
Institute for Healthcare Improvement
13Measurement is Central to the Team’s Ability to Improve
The purpose of measurement in QI work is for learning not judgment!
All measures have limitations, but the limitations do not negate their value for
learning.
You need a balanced set of measures reported daily, weekly or monthly to
determine if the process has improved, stayed the same or become worse.
These measures should be linked to the team’s Aim.
Measures should be used to guide improvement and test changes.
Measures should be integrated into the team’s daily routine.
Data should be plotted over time on annotate graphs.
Focus on the Vital Few!
14
AIM (How good? By when?)
Concept
Measure
Operational Definitions
Data Collection Plan
Data Collection
Analysis ACTION
R. Lloyd. Quality Health Care: A Guide to Developing and Using Indicators. 2nd Edition, Jones
and Bartlett Publishers, 2017.
Milestones in theQuality Measurement Journey
Institute for Healthcare Improvement
Moving from a Concept to Measure
“Hmmmm…how do I move from a concept
to an actual measure?
Every concept can have MANY measures. Which one is most appropriate?
16
Vision
End Result
Ideal State
Institute for Healthcare Improvement
17
Are these measures!
Reduce wait times
Improve patient satisfaction
Expand market share
Be more efficient
Increase health and well-being
Reduce waste
Improve our financial situation
Reduce inpatient discharge delays
Enhance Patient education
Deliver safe services
They are part of…
Every concept can have many measures!R. Lloyd. Quality Health Care: A Guide to Developing and Using Indicators. 2nd Edition, Jones and Bartlett, 2017.
Concept Potential Measures
Access • Number of days to the next 2nd appointment• Percent of add-ons who can be seen today• Number of walk-in appointments
• The number of minutes a caller is on hold before talking to a
staff person• Number of phone calls requesting an appointment this week
Wait Time • Wait time from check-in to discharge
• Wait time from check-in to seeing doctor
• Time spent with doctor
• Time it takes to have follow-up work done in the office (labs,
x-ray, ultra-sound, etc.)
Management of
Diabetes Patients
• Percent of diabetes patients with appropriate eye and foot
exams done during an office visit
• Percent of all diabetes patient in glucose control
• Percent of patients engaged in self-management goals
Institute for Healthcare Improvement
Three Types of Measures
Outcome Measures: Voice of the customer or patient. How is the system performing? What is the result?
Process Measures: Voice of the workings of the system. Are the parts/steps in the system performing as planned?
Balancing Measures: Looking at a system from different directions/dimensions. What happened to the system as we improved the outcome and process measures (e.g. unanticipated consequences, other factors influencing outcome)?
Potential Set of Measures for Improvement in a Family Practice Clinic
Topic
Outcome Measures
Process Measures Balancing Measures
Improve waiting time and patient satisfaction in the family practice clinic
Total Length of Stay (in minutes) for a scheduled appointment at the clinic
% of patients marking Strongly Agree to the question: “Would you recommend our clinic to family and friends?”
Time from check-in till seeing the doctor
Patient /staff comments on flow
% of patient receiving discharge materials
Wait time for ancillary services (lab, x-ray, ultra-sound) during a visit
Volume of patients
% of patients leaving without being seen by the doctor
Staff satisfaction
Financials
Institute for Healthcare Improvement
Balancing Measures:Looking at the System from Different Dimensions
Outcome (quality, time)
Transaction (volume, no. of patients)
Productivity (cycle time, efficiency, utilisation, flow, capacity, demand)
Financial (charges, staff hours, materials)
Appropriateness (validity, usefulness)
Patient satisfaction (surveys, customer complaints)
Staff satisfaction
Balancing measures help keep you from sub-optimizing the system!
22
“Without reflection, we go blindly on
our way, creating more unintended
consequences, and failing to achieve anything useful.”
~Margaret Wheatley
Balancing Measures help you capture
Unintended Consequences
Institute for Healthcare Improvement
© 2016 Institute for Healthcare Improvement/R. Lloyd
ExerciseOrganizing your Measures
1. A starting point for any QI project is to move from concepts to measures
that appropriately capture the concepts of interest.
2. Use the Organizing Your Measures Worksheet on the next page to start
this part of your journey.
3. List the concepts of interest in the far left column. Then identify potential
measures for these concepts in the second column. Remember that a
single concept might have more than one potential measure.
4. Finally, indicate whether each potential measure is an Outcome, Process
or Balancing measure.
© 2016 Institute for Healthcare Improvement/R. Lloyd
Concept Potential Measure(s) Outcome Process Balancing
Organizing Your Measures Worksheet
Topic for Improvement:
R. Lloyd. Quality Health Care: A Guide to Developing and Using Indicators. 2nd Edition, Jones and Bartlett, 2017.
Institute for Healthcare Improvement
© 2016 Institute for Healthcare Improvement/R. Lloyd
Concept Potential Measure(s) Outcome Process Balancing
Patient Harm Inpatient falls rate
Patient Harm Number of falls
Compliance Percent of inpatients assessed for falls
Staff Education
Percent of staff fully trained in falls assessment protocol
Assessment Time
The additional time it takes to conduct a proper falls assessment
ExampleOrganizing Your Measures Worksheet
Topic for Improvement: Inpatient Falls
R. Lloyd. Quality Health Care: A Guide to Developing and Using Indicators. 2nd Edition, Jones and Bartlett, 2017.
26
AIM (How good? By when?)
Concept
Measure
Operational Definitions
Data Collection Plan
Data Collection
Analysis ACTION
Milestones in theQuality Measurement Journey
R. Lloyd. Quality Health Care: A Guide to Developing and Using Indicators. 2nd Edition, Jones
and Bartlett Publishers, 2017.
Institute for Healthcare Improvement
An Operational Definition...
… is a description, in
quantifiable terms, of what to measure and the steps to follow to measure it consistently.
• It gives communicable meaning to a concept
• Is clear and unambiguous
• Specifies measurement methods and equipment
• Identifies criteria
27
R. Lloyd. Quality Health Care: A Guide to Developing and Using Indicators. 2nd Edition, Jones
and Bartlett Publishers, 2017.
28
What does it mean to “go wireless”?
Institute for Healthcare Improvement
How do you define the following healthcare concepts?
• World Class Performance
• A little pain and swelling
• Teenage pregnancy
• Cancer waiting times
• Health inequalities
• Asthma admissions
• Childhood obesity
• Patient education
• Health and wellbeing
• Adding life to years and years to life
• Children's palliative care
• Safe services
• Smoking cessation
• Urgent care
• Delayed discharges
• End of life care
• Falls (with/without injuries)
• Childhood immunizations
• Complete maternity service
• Patient engagement
• Moving services closer to home
• Successful breastfeeding
• Ambulatory care
• Access to health in deprived areas
• Diagnostics in the community
• Productive community services
• Vascular inequalities
• Breakthrough priorities
ExerciseOperational Definition
30
• Select an improvement project that is work related or a personal improvement project.
• Select one measure from this project and develop an operational definition that is:
• Clear and unambiguous
• Specifies measurement methods and equipment
• Identifies criteria if appropriate.
• Use the Operational Definition Worksheet to guide and record your work.
Institute for Healthcare Improvement
Team name: _______________________________________________________________________
Date: __________________ Contact person: ________________________________
WHAT PROCESS DID YOU SELECT?
WHAT SPECIFIC MEASURE DID YOU SELECT FOR THIS PROCESS?
OPERATIONAL DEFINITIONDefine the specific components of this measure. Specify the numerator and denominator if it is a percent
or a rate. If it is an average, identify the calculation for deriving the average. Include any special
equipment needed to capture the data. If it is a score (such as a patient satisfaction score) describe how
the score is derived. When a measure reflects concepts such as accuracy, complete, timely, or an error,
describe the criteria to be used to determine “accuracy.”
Operational Definition Worksheet
R. Lloyd. Quality Health Care: A Guide to Developing and Using Indicators. 2nd Edition, Jones
and Bartlett Publishers, 2017.
DATA COLLECTION PLANWho is responsible for actually collecting the data?How often will the data be collected? (e.g., hourly, daily, weekly or monthly?)What are the data sources (be specific)?What is to be included or excluded (e.g., only inpatients are to be included in this measure or only stat lab requests shouldbe tracked).How will these data be collected?Manually ______ From a log ______ From an automated systemWill sampling be required? If ‘yes’ what type of sample will you pull?
BASELINE MEASUREMENTWhat is the actual baseline number? ______________________________________________What time period was used to collect the baseline? ___________________________________
TARGET(S) OR GOAL(S) FOR THIS MEASUREDo you have target(s) or goal(s) for this measure?Yes ___ No ___
Specify the External target(s) or Goal(s) (specify the number, rate or volume, etc., as well as the source of the target/goal.)
Specify the Internal target(s) or Goal(s) (specify the number, rate or volume, etc., as well as the source of the target/goal.)
Operational Definition Worksheet(cont’d)
R. Lloyd. Quality Health Care: A Guide to Developing and Using Indicators. 2nd Edition, Jones
and Bartlett Publishers, 2017.
Institute for Healthcare Improvement
33
AIM (How good? By when?)
Concept
Measure
Operational Definitions
Data Collection Plan
Data Collection
Analysis ACTION
Milestones in theQuality Measurement Journey
R. Lloyd. Quality Health Care: A Guide to Developing and Using Indicators. 2nd Edition, Jones
and Bartlett Publishers, 2017.
Key Aspects of Data Collection
• Stratification
• Sampling Methods
• Frequency of Data Collection
• Duration of Data Collection
R. Lloyd. Quality Health Care: A Guide to Developing and Using Indicators. 2nd Edition, Jones
and Bartlett Publishers, 2017.
Institute for Healthcare Improvement
© 2015 Institute for Healthcare Improvement/R. Lloyd
35
Key Data Collection Strategies:Stratification
Stratification
• Separation & classification of data according to predetermined categories
• Designed to discover patterns in the data
• For example, are there differences by shift, time of day, day of week, severity of patients, age, gender or type of procedure?
• Consider stratification BEFORE you collect
the data
36
• Age
• Day of week
• Time of day or Shift
• Stat vs routine orders
• Severity of patients
Gender
Co-morbid conditions
Facility or service area
Units within a facility
Socio-economic status
Common Stratification Levels
R. Lloyd. Quality Health Care: A Guide to Developing and Using Indicators. 2nd Edition, Jones
and Bartlett Publishers, 2017.
Institute for Healthcare Improvement
© 2015 Institute for Healthcare Improvement/R. Lloyd
Probability Sampling Methods
• Simple random sampling
• Stratified random sampling
• Stratified proportional random sampling
• Systematic sampling
• Cluster sampling
Key Data Collection Strategies:Sampling Methods
Non-probability Sampling Methods
• Convenience sampling
• Quota sampling
• Judgment sampling
R. Lloyd. Quality Health Care: A Guide to Developing and Using Indicators. 2nd Edition, Jones and Bartlett Publishers, 2017.
© 2015 Institute for Healthcare Improvement/R. Lloyd
How often and for how long do you need to collect data?
Frequency – the period of time in which you collect data (i.e., how often will you dip into the
process to see the variation that exists?)
• Moment by moment (continuous monitoring)?
• Every hour?
• Every day? Once a week? Once a month?
Duration – how long you need to continue collecting data
• Do you collect data on an on-going basis and not end until the measure is always at the
specified target or goal?
• Do you conduct periodic audits?
• Do you just collect data at a single point in time to “check the pulse of the process”
Do you need to pull a sample or do you take every occurrence of the data (i.e., collect data for the total population)
R. Lloyd. Quality Health Care: A Guide to Developing and Using Indicators. 2nd Edition, Jones and Bartlett Publishers, 2017.
Institute for Healthcare Improvement
39
AIM (How good? By when?)
Concept
Measure
Operational Definitions
Data Collection Plan
Data Collection
Analysis ACTION
Milestones in theQuality Measurement Journey
R. Lloyd. Quality Health Care: A Guide to Developing and Using Indicators. 2nd Edition, Jones
and Bartlett Publishers, 2017.
40
You have performance data.Now what do you do with it?
Institute for Healthcare Improvement
“If I had to reduce my message for
management to just a few words, I’d say it all had to do with reducing
variation.”W. Edwards Deming
Question #4Do you understand variation conceptually?
42
The Problem!
Aggregated data presented in tabular formats or with summary statistics, will not
help you measure the impact of process improvement efforts.
Aggregated data can only lead to judgment, not to improvement.
Institute for Healthcare Improvement
“Managing a company by means of the monthly (or quarterly or yearly) report is like trying to drive a car by watching the yellow line
in the rear-view mirror.” Myron Tribus
If you are serious about your quality improvement efforts, you should be collecting and analyzed data as close to the production of work as possible.
• What would it take to collect data on individual patients waiting to see the doctor?
• To track the number of patients being assessed for pressure ulcers each day?
• The percent of “did not attend” appoints for each week?
• Most measures can be collected more frequently than monthly!
As quoted in Wheeler, Donald. Understanding Variation: The Key to Managing Chaos. SPC Press, Inc., 1993: 4.
The average of a set of numbers can be created by many different distributions
44
X (CL)
Measu
re
Time
= 76
Institute for Healthcare Improvement
If you don’t understand the variation that lives in your data, you will be tempted to ...
• Deny the data (It doesn’t fit my view of reality!)
• See trends where there are no trends
• Try to explain natural variation as special events
• Blame and give credit to people for things over which they have no control
• Distort the process that produced the data
• Kill the messenger!
45
© 2016 Institute for Healthcare Improvement/R. Lloyd
“A phenomenon will be said to be controlled when,
through the use of past experience, we can
predict, at least within limits, how the
phenomenon may be expected to vary in the
future”W. Shewhart. Economic Control of
Quality of Manufactured Product, 1931
Dr. Walter A Shewhart
Institute for Healthcare Improvement
© 2016 Institute for Healthcare Improvement/R. Lloyd
“What is the variation in one system over time?”Walter A. Shewhart - early 1920’s, Bell Laboratories
47
time
UCL
Every process displays variation:Controlled variation
• stable, consistent pattern of variation
• “chance”, constant causes
Special cause variation• “assignable”
• pattern changes over time
LCL
Static View
Dynamic View
Sta
tic V
iew
48
Common Cause Variation• Is inherent in the design of the
process
• Is due to regular, natural or ordinary
causes
• Affects all the outcomes of a
process
• Results in a “stable” process that is
predictable
• Also known as random or
unassignable variation
Special Cause Variation• Is due to irregular or unnatural causes
that are not inherent in the design of
the process
• Affect some, but not necessarily all
aspects of the process
• Results in an “unstable” process that
is not predictable
• Also known as non-random or
assignable variation
Types of Variation
Institute for Healthcare Improvement
© 2016 Institute for Healthcare Improvement/R. Lloyd
Common Cause Variation
• Points equally likely above or below center line
• There will be a high data point and a low, but this is expected
• No trends or shifts or other patterns
0
10
20
30
40
50
60
70
80
90
100
3/1/
2008
3/8/
2008
3/15
/200
8
3/22
/200
8
3/29
/200
8
4/5/
2008
4/12
/200
8
4/19
/200
8
4/26
/200
8
5/3/
2008
5/10
/200
8
5/17
/200
8
5/24
/200
8
5/31
/200
8
6/7/
2008
Courtesy of Richard Scoville, PhD, IHI Improvement Advisor
© 2016 Institute for Healthcare Improvement/R. Lloyd
Two Types of Special Causes
Unintentional
When the system is out of control and unstable due to unexpected forces
Intentional
When we’re trying to change the
systemCourtesy of Richard Scoville, PhD, IHI Improvement Advisor
Holding the Gain: Isolated Femur Fractures
0
200
400
600
800
1000
1200
1 4 7 10 13 16 19 22 25 28 31 34 37 40 43 46 49 52 55 58 61 64Sequential Patients
Min
ute
s E
D t
o O
R p
er
Patient
Institute for Healthcare Improvement
© 2016 Institute for Healthcare Improvement/R. Lloyd
Point …Variation exists!
Common Cause (random) Variation does not mean “Good Variation.” It only means that the process is stable and predictable. For example, if a patient’s systolic blood pressure averaged around
165 and was usually between 160 and 170 mmHg, this might be stable and predictable but completely unacceptable.
Similarly, Special Cause (non-random) Variation should not be viewed as “Bad Variation.” You could have a non-random variation
that represents a very good result (e.g., a low turnaround time), which you would want to emulate. Non-Random merely means that
the process is unstable and unpredictable.
© 2016 Institute for Healthcare Improvement/R. Lloyd
52
2 Questions …
1. Is the process stable?
If so, it is predictable.
2. Is the process capable?
The chart will tell you if the process is stable and predictable.
You have to decide if the output of the process is capable of meeting the target or goal under current operation condition!
Institute for Healthcare Improvement
© 2016 Institute for Healthcare Improvement/R. Lloyd
Random Variation
Normal Sinus Rhythm(a.k.a. Random Variation)
Ventricular Fibrillation(a.k.a. Non-Random Variation)
Non-Random VariationHolding the Gain: Isolated Femur Fractures
0
200
400
600
800
1000
1200
1 4 7 10 13 16 19 22 25 28 31 34 37 40 43 46 49 52 55 58 61 64Sequential Patients
Min
ute
s E
D t
o O
R p
er
Patie
nt
Appreciation is extended to Dr. Douglas Brosnan, JD, MD, Vice Chair, Department of Emergency Medicine, Sutter Roseville Inpatient
EHR Physician Champion for providing the example of normal sinus rhythm versus ventricular fibrillation.
Finally, find examples that work for your discipline!
© 2016 Institute for Healthcare Improvement/R. Lloyd
Leaders understand the different ways that variation is viewed.
They explain changes in terms of common causes and special causes.
They use graphical methods to learn from data and expect others to consider variation in their decisions and actions.
They understand the concept of stable and unstable processes and the potential losses due to tampering.
Capability of a process or system is understood before changes are attempted.
Attributes of a Leader WhoUnderstands Variation
Institute for Healthcare Improvement
© 2016 Institute for Healthcare Improvement/R. Lloyd
• Select several measures which your organization tracks regularly.
• Do you and the leaders of your organization evaluate these measures according the criteria for common and special causes of variation?
• If not, what criteria do you use to determine if your measures are improving or getting worse?
DialogueUnderstanding Variation
Antal patienter med vårdtid < 6dygn i % vid primär elektiv knäplastik
(operationsdag= dag1)
0
10
20
30
40
50
60
70
80
90
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18
Månad
An
tal p
atie
nte
r i %
© 2016 Institute for Healthcare Improvement/R. Lloyd
Question #5Do you understand variation statistically?
56
STATIC VIEW
Descriptive StatisticsMean, Median & Mode
Minimum/Maximum/RangeStandard Deviation
Bar graphs/Pie charts
DYNAMIC VIEWRun Chart
Control Chart
(plot data over time)
Statistical Process Control (SPC)
Rate
per 1
00
ED
Pa
tients
Unp lanned Retur ns to E d w/in 72 Hours
M41.78
17
A43.89
26
M39.86
13
J40.03
16
J38.01
24
A43.43
27
S39.21
19
O41 .90
14
N41.78
33
D43.00
20
J39.66
17
F40.03
22
M48.21
29
A43.89
17
M39.86
36
J36.21
19
J41.78
22
A43.89
24
S31.45
22
Mo nth
ED /1 00
Re tur ns
u ch a r tu ch a r tu ch a r tu ch a r t
1 2 3 4 5 6 7 8 910
11
12
13
14
15
16
17
18
19
0 .0
0 .2
0 .4
0 .6
0 .8
1 .0
1 .2
U C L = 0 .88
M ea n = 0 .5 4
LC L = 0.19
Institute for Healthcare Improvement
57
How do we analyze variation forquality improvement?
• We use Statistical process Control (SPC) methods and tools
• Run and Shewhart (Control) Charts are the best tools to determine:
─The variation that lives in the process
─ if our improvement strategies have had the desired effect.
Process Improvement: Isolated Femur Fractures
0
200
400
600
800
1000
1200
1 4 7 10 13 16 19 22 25 28 31 34 37 40 43 46 49 52 55 58 61 64Sequential Patients
Min
ute
s E
D t
o O
R p
er
Patient
Holding the Gain: Isolated Femur Fractures
0
200
400
600
800
1000
1200
1 4 7 10 13 16 19 22 25 28 31 34 37 40 43 46 49 52 55 58 61 64Sequential Patients
Min
ute
s E
D t
o O
R p
er
Patient
3. Determine if we are holding the gains
Current Process Performance: Isolated Femur Fractures
0
200
400
600
800
1000
1200
1 4 7 10 13 16 19 22 25 28 31 34 37 40 43 46 49 52 55 58 61 64Sequential Patients
Min
ute
s E
D t
o O
R p
er
Patient Three Uses of
SPC Charts
2. Determine if a change is an improvement
1. Make process performance visible
Institute for Healthcare Improvement
59
How do we analyze variation forquality improvement?
Measure
Time
Measure
Time
A Run Chart:• Is a time series plot of data• The centerline is the Median• 4 Run Chart rules are used to determine if there
are random or non-random patterns in the data
A Control Chart:• Is a time series plot of data• The centerline is the Mean• Added features include Upper and lower control
Limits (UCL & LCL)• 5 Control Chart rules are used to determine if the
data reflect common or special causes of variation
Run Chart
Control Chart
© 2017 Institute for Healthcare Improvement/R. Lloyd
Your next
move…
…to gain more knowledge about Run Charts and Shewhart (control) Charts)
Institute for Healthcare Improvement
© 2016 Institute for Healthcare Improvement/R. Lloyd
61
AIM (Target Condition)
Concept
Measures
Operational Definitions
Data Collection Plan
Data Collection
Analysis ACTION
R. Lloyd. Quality Health Care: A Guide to Developing and Using Indicators. 2nd Edition, Jones and Bartlett, 2017.
Milestones in theQuality Measurement Journey
Institute for Healthcare Improvement
Measurement Self-AssessmentR. Lloyd, Quality Health Care: A Guide to Developing and Using Indicators. 2nd Edition, Jones & Bartlett Publishers, 2017: 339-341.
Measurement Topic or SkillResponse Scale
1 2 3 4 5
Help people in my organization determine why they are measuring (improvement, judgment or research)
Move teams from concepts to specific quantifiable measures
Building clear and unambiguous operational definitions for our measures
Develop data collection plans (including stratification and sampling strategies)
Explain why plotting data over time (dynamic display) is preferable to using aggregated data and summary statistics (static display)
Explain the differences between random and non-random variation
Construct run charts (including locating the median)
Explain the reasoning behind the run chart rules
Interpret run charts by applying the run chart rules
Explain the statistical theory behind Shewhart control charts (e.g., sigma limits, zones, special cause rules)
Describe the basic 7 Shewhart charts and when to use each one
Help teams select the most appropriate Shewhart chart for their measures
Describe the rules for special cause variation on a Shewhart chart
Help teams link measurement to their improvement efforts
1. I'd definitely have to call in an outside expert to explain and apply this topic/method. 2. I'm not sure I could apply this appropriately to a project.3. I am familiar with this topic but would have to study it further before applying it to a project.4. I have knowledge about this topic, could apply it to a project but would not want to be asked to teach it to others.5. I consider myself an expert in this area, could apply it easily to a project and could teach this topic/method to others.
64Final tips for building an effective measurement system
Seek useful measures not perfection
Think about stratification
Use sampling (when appropriate)
Integrate measurement into daily routine
Collect qualitative and quantitative data
Plot data over time
Institute for Healthcare Improvement
But realize that theCharts Don’t Tell You…
• The reasons(s) for a Special Cause.
• Whether or not a Common Cause process should be
improved (is the performance of the process
acceptable?)
• How the process should actually be improved or
redesigned.
65
A Simple Improvement Plan
1. Which process do you want to improve or redesign?
2. Does the process contain common or special cause variation?
3. How do you plan on actually making improvements? What strategies
do you plan to follow to make things better?
4. What effect (if any) did your plan have on the process performance?
SPC methods and toolswill help you answer Questions 2 & 4.
YOU need to figure out the answers to Questions 1 & 3.
Institute for Healthcare Improvement
©Copyright 2013 Institute for Healthcare Improvement/R. Lloyd
Finally, remember that data is a necessary part of the Sequence of Improvement
Sustaining improvements and Spreading changes to other locations
Developing a change
Implementing a change
Testing a change
Theory and Prediction
Test under a variety of conditions
Make part of routine operations
R. Lloyd. Quality Health Care: A Guide to Developing and Using Indicators. 2nd Edition, Jones and
Bartlett, 2017: 343.
68
“Quality begins with intent, which
is fixed by management.”
W. E. Deming, Out of the Crisis, p.5
Institute for Healthcare Improvement
© 2017 Institute for Healthcare Improvement/R. Lloyd
69
Additional Resources
You can access the following free videos from the IHI website:
Dr. Lloyd has over 20 Whiteboard Videos that explain the concepts, tool and methods of QI in 4-8
minutes. http://www.ihi.org/education/IHIOpenSchool/resources/Pages/BobLloydWhiteboard.aspx
Also Dr. Lloyd’s On Demand Videos can also be accessed from the IHI Website:
Deming’s System of Profound Knowledge and the Model for Improvement
http://www.ihi.org/education/WebTraining/OnDemand/ImprovementModelIntro/Pages/default.aspx
Data Collection and Understanding Variation
http://www.ihi.org/education/WebTraining/OnDemand/DataCollection_Variation/Pages/default.asp
x
Using Run and Control Charts
http://www.ihi.org/education/WebTraining/OnDemand/Run_ControlCharts/Pages/default.aspx
70
Appendices• Appendix A: The Quality Improvement Tool Box
• Appendix B: Force Field Analysis
• Appendix C: Driver Diagrams
• Appendix D: References on Quality
• Appendix E: References on Measurement
• Appendix F: References on Spread
• Appendix G: Faculty bios and contact information
Institute for Healthcare Improvement
Thank you for joining us today!Good luck with your
Quality Journey!
Dr. BobDr. BobDr. BobDr. Bob
Dr. Dr. Dr. Dr. MukeshMukeshMukeshMukesh & Dr. & Dr. & Dr. & Dr. AkhnuwkhAkhnuwkhAkhnuwkhAkhnuwkh
Contact Information:
Bob Lloyd: [email protected]
Mukesh Thakur: [email protected]
Akhnuwkh Jones: [email protected]
71
Appendix A:The Quality Improvement Tool Box
72
Seven Basic
Tools
Seven Management
Tools
Creativity
ToolsMeasurement
Tools
Design
Tools
Statistical
Tools
Source: Oakes, D. “Organize Your Quality Tool Belt” Quality Progress,
American Society for Quality, July, 2002:25-29.
Institute for Healthcare Improvement
The Primary QI Tools…
• The Seven Basic Tools
• Flowchart
• Cause & effect diagram
• Pareto chart
• Check sheet
• Run & control charts
• Histograms
• Scatter diagrams
• The Seven Management Tools
• Affinity diagrams
• Interrelationship digraphs
• Matrix diagram
• Priorities matrix
• Activity network diagrams
• Tree diagrams
• Process decision program charts
73
What’s in your tool box?
CQI Tools by Function74
Creativity Tools• Brainstorming
• Mind mapping
• Six thinking hats
• Innovation/IDEO
Measurement Tools• Cost of quality analysis
• Benchmarking
• Dashboards/indicators
• Survey analysis
Design Tools• QFD
• House of quality
• FMEA
• Hoshin planning
Statistical Tools• SPC
• DOE
• Descriptive statistics
• Multivariate statistics
Institute for Healthcare Improvement
Methods and Tools for Improvement 75
Category Method or Tool Typical Use of Method or Tool
Viewing Systemsand Processes
1. Flow Diagram Develop a picture of a process. Communicate and standardize processes.
2. Linkage of Processes Develop a picture of a system composed of processes linked together.
GatheringInformation
3. Form for Collecting Data Plan and organize a data collection effort.
4. Surveys Obtain information from people.
5. Benchmarking Obtain information on performance and approaches from other organizations.
6. Creativity Methods Develop new ideas and fresh thinking.
OrganizingInformation
7. Affinity Diagram Organize and summarize qualitative information.
8. Force Field Analysis Summarize forces supporting and hindering change.
9. Cause and Effect Diagram Collect and organize current knowledge about potential causes of problems or variation.
10. Matrix Diagram Arrange information to understand relationships and make decisions.
11.Tree Diagram Visualize the structure of a problem, plan, or any other opportunity of interest.
12. Quality Function Deployment (QFD)
Communicate customer needs and requirements through the design and production processes.
UnderstandingVariation
13. Run Chart Study variation in data over time; understand the impact of changes on measures.
14. Control Chart Distinguish between special and common causes of variation.
15. Pareto Chart Focus on areas of improvement with greatest impact.
16. Frequency Plot Understand location, spread, shape, and patterns of data.
UnderstandingRelationships
17. Scatterplot Analyze the associations or relationship between two variables; test for possible cause-and-effect.
18. Two-Way Table Understand cause-and-effect for qualitative variables.
19. Planned Experimentation Design studies to evaluate cause-and-effect relationships and test changes.
Two Essential Tools
Flowcharting Cause & Effect Diagrams
76
KQC
People Equipment
Material Environment
Institute for Healthcare Improvement
Tools to Understand Variation in Data 77
Waiting Time for Clinic Visit
20
25
30
35
40
45
50
55
60
Avera
ge
Days
Waiting Time for Clinic Visit
20
25
30
35
40
45
50
55
60
Avera
ge D
ays
Distribution of Wait Times
0
10
20
30
40
50
60
5 15 25 35 45 55 65 75 85 95 105
Wait time (days) for Visit
nu
mb
er
of
vis
its
Clinic Wait Times > 30 days
0
2
4
6
8
10
12
14
16
C F G D A J H K B I L E
Clinic ID
# o
f w
ait
s >
30
da
ys
Relationship Between Long
Waits and Capacity
0
5
10
15
20
75 95Capacity Used
# w
ait
tim
es >
30 d
ays
Run Chart Shewhart Chart
Frequency Plot Pareto Chart Scatterplot
IH p. 8-34
Appendix B: Force Field Analysis 78
What is it?
Force Field Analysis is a QI tool designed to identify driving (positive) and restraining (negative) forces that support or work against the solution of an issue or problem.
When the driving and restraining forces are identified, steps can be taken to reinforce the driving forces and reduce the restraining forces
What does the Force Field do?
Allows comparisons of the “positives” and “negatives” of a situation
Enables easy comparisons
Forces people to think together about all the aspects of making the desired change a permanent one
Encourages people to agree about the relative priority of factors on each side of an issue
Supports the honest and open reflection on the underlying root causes of a problem and ways to break down barriers
Institute for Healthcare Improvement
How do I set up a Force Field Analysis? 79
1. Draw a letter “T” on a flipchart page
2. Write the name of the issue or project across the top of the page
3. Label the left column “Driving Forces” and the right column the
“Restraining Forces”
4. Use brainstorming or nominal group technique (NGT) to generate
the list of forces or factors that are driving the issue or project and
those that are restraining or the holding things back
5. Eliminate duplicate ideas and clarify any ideas that are vague or
not specific
6. If the team feels the need, they can use rank ordering to set
priorities for the driving and restraining forces
7. Generate a list of ideas about actions that can be taken to reduce
the restraining forces
Force Field Analysis Worksheet
Issue or Project: ______________________________________
Driving Forces (+) Restraining Forces (-)
Actions to reduce the Restraining Forces:
•
•
•
Institute for Healthcare Improvement
Driver Diagrams, a tool to help us understand the system and the
messiness of life.
Appendix C:Driver Diagrams
©Copyright 2013 IHI/R. Lloyd
82
A Driver Diagram is a good way to show your aim and the system you want to improve
Concept 1
Concept 2
Concept 3
OutcomePrimary Drivers
Secondary Drivers
Specific Change Ideas
Change Concepts
Ideas:
1
2
3
4
5
6
7
.
.
.
.
.
.
.
.N
Concept 4
Concept 5
Concept 6
Institute for Healthcare Improvement
83What Changes Can We Make?
Primary Drivers
System components which will contribute to moving the primary outcome
Secondary Drivers
Elements of the associated Primary Driver.
They can be used to create projects or a change package that will affect the Primary Driver.
To improve the inpatient
experience for adult female
inpatients on a mental health unit
in order to increase
satisfaction by 25% in 10 months
Ward Environment
Bed occupancy
Stop sleep outs
Multidisciplinary Ward Team
Process
Nursing input
Pharmacy input
Family support
Patient Choice
Ward round
Complaints
Ward ActivitiesOT programme
Add senior OT to project team
Review of delays at weekly bed meetings
AIM PRIMARY DRIVERS SECONDARY DRIVERS CHANGE IDEAS
Rewrite protocol
Offer pharmacy advice to every patient during
stay
Ensure daily 1:1 time with named nurse
Change concept of large MDT ward round
meetings
Train one staff member on each ward to use
support skills
To change OT programme content
Improving quality of care on an inpatient female
psychiatric ward
Source: East London Foundation Trust, London, England.
Institute for Healthcare Improvement
85
What Changes Can We Make? Understanding the System for Improving Dental Health
Process Changes
• System
knowledge
• Exemplars
• Change
Packages
Reduce burden of dental
disease
• % pts with new
cavitation
• % pts complaining of
pain
• % of pts with OR Tx
Active, informed families
Reliable delivery of
evidence based
preventive & restorative
care
Patient oral health literacy
Community support
• CHCs, private dentists,
pediatricians, PCPs
• Payers
Early, regular risk-based
evaluation & guidance
Use of conservative
procedures
• Fluoride exposure
• ART
Patient self management
• Improved diet
• Improved hygiene
Improved patient access:
‘Dental Home’
Qualified OR Tx
Team-based care
Coordination with PCPs:
referrals
Balancing demand and
capacitySource: Richard Scoville, Ph.D.
Appendix DGeneral References on Quality
The Improvement Guide: A Practical Approach to Enhancing Organizational Performance. G. Langley, K. Nolan, T. Nolan, C. Norman, L. Provost. Jossey-Bass Publishers., San Francisco, 1996.
Quality Improvement Through Planned Experimentation. 2nd edition. R. Moen, T. Nolan, L. Provost, McGraw-Hill, NY, 1998.
The Improvement Handbook. Associates in Process Improvement. Austin, TX, January, 2005.
A Primer on Leading the Improvement of Systems,” Don M. Berwick, BMJ, 312: pp 619-622, 1996.
“Accelerating the Pace of Improvement - An Interview with Thomas Nolan,” Journal of Quality Improvement, Volume 23, No. 4, The Joint Commission, April, 1997.
86
Institute for Healthcare Improvement
Appendix EReferences on Measurement
Brook, R. et. al. “Health System Reform and Quality.” Journal of the American Medical Association 276, no. 6 (1996): 476-480.
Carey, R. and Lloyd, R. Measuring Quality Improvement in healthcare: A Guide to Statistical Process Control Applications. ASQ Press, Milwaukee, WI, 2001.
Lloyd, R. Quality Health Care: A Guide to Developing and Using Indicators. Jones and Bartlett Publishers, Sudbury, MA, 2004.
Nelson, E. et al, “Report Cards or Instrument Panels: Who Needs What? Journal of Quality Improvement, Volume 21, Number 4, April, 1995.
Provost, L. and Murray, S. The Data Guide. Associates in Process Improvement, Austin, TX 1-512-708-0131.
Solberg. L. et. al. “The Three Faces of Performance Improvement: Improvement, Accountability and Research.” Journal of Quality Improvement 23, no.3 (1997): 135-147.
87
Appendix FReferences on Spread
Gladwell, M. The Tipping Point. Boston: Little, Brown and Company, 2000.
Kreitner, R. and Kinicki, A. Organizational Behavior (2nd ed.) Homewood, Il: Irwin, 1978.
Lomas J, Enkin M, Anderson G. Opinion Leaders vs Audit and Feedback to Implement Practice Guidelines. JAMA, Vol. 265(17); May 1, 1991, pg. 2202-2207.
Myers, D.G. Social Psychology (3rd ed.) New York: McGraw-Hill, 1990.
Prochaska J., Norcross J., Diclemente C. In Search of How People Change, American Psychologist, September, 1992.
Rogers E. Diffusion of Innovations. New York: The Free Press, 1995.
Wenger E. Communities of Practice. Cambridge, UK: Cambridge University Press, 1998.
88
Institute for Healthcare Improvement
Appendix G: Faculty BioP89
Robert Lloyd, PhD. Vice President, Institute for Healthcare Improvement
provides leadership in the areas of performance improvement strategies,
statistical process control methods, development of strategic dashboards
and capacity and capability building for quality improvement. He also
serves as faculty for the IHI Improvement Advisor (IA) Professional
Development programme and various IHI initiatives and demonstration
projects in the US, Canada, the UK, Sweden, Denmark, Norway, Africa,
the Middle East and New Zealand. Dr. Lloyd an internationally recognized
speaker on quality improvement concepts, methods and tools. He also
advises senior leadership teams on how to create the structures and
processes that will make quality thinking part of daily work. He is the
author of two leading books on measuring quality improvement in
healthcare settings and numerous articles and chapters on quality
measurement and improvement. He lives in Chicago, Illinois with his wife
Gwenn, daughter Devon and their ever entertaining dog Cricket.
@rlloyd66
© 2016 Institute for Healthcare Improvement/R. Lloyd
90
Dr. Lloyd’s books, Measuring Quality Improvement in Healthcare: A Guide
to Statistical Process Control Applications (ASQ Press, 2000),
https://asq.org/quality-press/display-item?item=H1091
Quality HealthCare: A Guide to Developing and Using Indicators, Jones &
Bartlett Learning, 1st Edition 2004 and 2nd Edition 2017.
http://www.jblearning.com/catalog/9781284023077/
1st Edition
2nd
Edition
Institute for Healthcare Improvement
Appendix G: Faculty BioP91
Dr. Mukesh Thakur MBBS, MRCP (UK), CCST (UK), FRCP (Edinburgh). He has
extensive clinical experience of over 17 years in various internal medicine
specialties, of which more than 12 years have been in the National Health
Service, UK. He worked as a senior consultant in the Acute Internal Medicine
Department at Hull and East Yorkshire Hospitals NHS Trust UK, one of the largest
healthcare facilities in England. He served as Director of Training Program in
General Internal Medicine and Lead for Simulation in Acute Internal Medicine at
Hull Institute of Learning and Simulation. In addition, he serves as Examiner for
The Royal College of Physicians UK and Core Faculty (East Yorkshire School of
Endoscopy). He has completed his training with the Institute for Healthcare
Improvement USA, as an Improvement Advisor and Lean for Healthcare from
University of Tennessee USA. Dr. Mukesh is leading many quality initiatives in
Hamad General Hospital, including improving the Flow in the process of
Admission and Discharge and use of Standard Communication in Healthcare
settings. He loves music, movies and spending time with family.
Mukesh Thakur <[email protected]>
Appendix G: Faculty BioP92
Dr. Akhnuwkh Jones, MD, was born and raised in Philadelphia, Pennsylvania,
first capitol of the United States of America. Graduated from Quba Institute, in
1997, in which he was able to memorize the Holy Qur’aan under tutelage of
Imam Anwar, and Anas Muhaimin. He then moved on to Penn State in which he
graduated with degree in biology in 2001. At the age of 13, his dream was to
become a physician, and that became true in 2006. He graduated from Temple
University School of Medicine in 2006, located in his home town of Philadelphia,
and completed a residency in internal medicine, at Lankenau Medical Center in
Philadelphia in 2009. In the same year he obtained his board certification in
medicine from the American Board of Internal Medicine.
Before Joining Hamad in July 2014, Dr. Jones served as hospitalist in Jennersville Regional
Hospital located in Pennsylvania. As a hospitalist he was recognized as one of the leading
physicians in the world in 2013. In 2014, he moved to Qatar to join the Medicine Department at
Hamad General Hospital. He completed an IHI internal fellowship for quality improvement at HMC,
with commendation and is currently completing IA (Improvement Advisor), program for IHI. His goal
is to be a leader in quality improvement in healthcare and a role model for young physicians.
Akhnuwkh Jones <[email protected]>