Post on 26-Mar-2015
Measuring Progress in Patient Safety
Peter Pronovost, MD, PhD, FCCMJohns Hopkins University
ExercisePlease answer each question with a score of 1 to 5.
1 is below average, 3 is average and 5 is above average
• How smart am I
• How hard do I work
• How kind am I
• How tall am I
• How good is the quality of care we provide
Improving Sepsis Care
(n= 19 ICUs)Mortality
13.1
21.9
41.8
0.0
10.0
20.0
30.0
40.0
50.0
Oct - Dec2003
Mar - May2004
July - Sept2004
%
69% Reduction (p < 0.001)
ICU LOS
6.2
7.6
10.0
0.0
2.0
4.0
6.0
8.0
10.0
12.0
14.0
Oct - Dec2003
Mar - May2004
July - Sept2004
Day
s
36% Reduction (NS)
Improving Sepsis Care
(n= 19 ICUs)Mortality
13.1
21.9
41.8
0.0
10.0
20.0
30.0
40.0
50.0
Oct - Dec2003
Mar - May2004
July - Sept2004
%
69% Reduction (p < 0.001)
ICU LOS
6.2
7.6
10.0
0.0
2.0
4.0
6.0
8.0
10.0
12.0
14.0
Oct - Dec2003
Mar - May2004
July - Sept2004
Day
s
36% Reduction (NS)
Central Mandate
Local Wisdom
Scientifically Sound Feasible
xx
Structure
ContextHave we created a culture of safety?
Process Outcome
Have we reduced the likelihood of harm?
How often do we do what we are supposed to?
How often do we harm?
Conceptual model for measuring safety
IT
Adapted from Donebedian
Keystone ICU Safety Dashboard
2004 2006
How often did we harm (BSI) 2.8/1000 0
How often do we do what we should
66% 95%
How often did we learn from mistakes
100s 100s
% Needs improvement in Safety climate
Teamwork climate
84%
82%
43%
42%
Pronovost JAMA 2007
Translating EvidenceInto Practice
* Envision the problemwithin the larger health
care system
* Engage collaborativemulti-disciplinaryteams centrally(stages 1,2,&3)
and locally(stage 4)
1. Summarize the Evidence
Convert interventions to behaviors
2. Identify local barriers toimplementation: understandthe process and context of
work
3. Measure Performance
4. Ensure all patientsreceive the interventions
Identify Interventions associatedwith improved outcomes
Select interventions with the largestbenefit and lowest barriers to use
Enlist all stakeholders to shareconcerns and identify potentialgains/losses associated withintervention implementation
Observe staff performing theinterventions
"Walk the process" to identifydefects in each step of intervention
implementation
Measure Baseline Performance
Develop and pilot test measures
Select Measures(process and/or outcome)
Engage
Explain why the interventions are
important
Execute
Design an intervention “toolkit” targeted to barriers employing standardization,
independent checks and reminders, and learning from mistakes
Educate
Share the evidence supporting the interventions
Evaluate
Regularly assess performance
measures
Pronovost BMJ 2008
Comprehensive Unit-based Safety Program (CUSP)
1. Educate staff on science of safety http://www.jhsph.edu/ctlt/training/patient_safety.html
2. Identify defects3. Assign executive to adopt unit4. Learn from one defect per quarter and implement
teamwork tools
Pronovost J, Pronovost J, Patient Safety,Patient Safety, 20052005
What can be measured as a valid rate?
• Rate requires– Numerator- event– Denominator- those at risk for event– Surveillance for events and those at risk
• Minimal and Known Error– Random error– Systematic error
Sources Variation in Safety measures
• True variation in Safety
• V data quality/definition/methods of collection
• V case mix
• V historical rates
• Chance
Measuring Preventable Harm
• Measure rate or counts directly– High sensitivity low specificity
• Estimate observed/expected (O/E)– Low sensitivity and specificity
• Link process and outcome– High specificity and moderate sensitivity
Process Measures
• Validity of the construct
• Validity of how we measure construct
It is Ok to have non-rate measures
Self reported measures are generally not valid as rates
A common mistake is interpreting a non-rate measure as a valid rate
Learning from Mistakes
• What happened?
• Why did it happen (system lenses)
• What could you do to reduce risk
• How to you know risk was reduced– Create policy/process/procedure– Ensure staff know policy– Evaluate if policy is used correctly
Pronovost 2005 JCJQI
•Identify Hazards•(
3. Mitigate Risks
2. Analyze & Prioritize Hazards
4. Evaluate Effectiveness of Risk Reduction
Patient Safety Learning Communities
Patient safety learning communities relate to each other in a gear like fashion: as the identified hazards require stronger levels of intervention to achieve mitigation, the next learning community is engaged in action, eventually feeding back to the group that provided the initial thrust. Each group (unit, hospital, industry) follows the same four- step process, but they engage unique matrices of stakeholders to mitigate hazards that are within their locus of control.
Pronovost Health affairs in press
GYN/OB JHOC Medicine Neurosciences Oncology Ophthalmology
FAC: Fetal Assessment Center/OB Ultrasound
GSS - Shared Specialty Suite
Asthma & Allergy - Allergy & Clinical Immunology
BRU GSS - Medical Oncology GSS - Wilmer 110
GSS - GYN/OB 420 JHOPC - Express Testing Asthma & Allergy -
Pulmonary EMU
IPOP Clinic - HIPOP Location
GSS - Wilmer Laser Center
GSS - GYN/REI JHOPC - OR Asthma & Allergy -
Rheumatology JHOPC Neurosciences
IPOP Clinic - IPOP Location
WECP & ER
HAL-2 JHOPC - PACU Blalock 4 - Endoscopy MEY 8 (12) Weinberg OPD - 1st Floor Wilmer OR
JHOPC GYN/OB WM - Shared Specialty
Suite Blalock 5 Echo Lab (2) MEY 9 (5) Weinberg OPD - 2nd Floor Wilmer PACU
MCE Cardiac CT NCCU7 WGA 5 (5) Wilmer White Marsh
NEL-2 Nursery CCP-5 (5) WGB 5 Wilmer: Other - E Balt
Divisions
NEL-2 Obstetric OR CCU-5 (7) WGC-5 (3) Wilmer: Other - Satellites
NEL-2 PACU CVC WGD 5
Nelson Harvey 2 CVIL- CardioVascular
Interventional Lab
OSL-2 Dialysis Unit
OSL-3 Nursery GSS - Internal Medicine
OSL-3 HAL-5 (5)
WGB-4 HAL-8 (7)
Hospitalist Unit (5)
JHOPC - Exec Health & Travel Clinic
JHOPC - Medicine Clinics
CAST• Each contributing factor rate
– importance of the problem and contributing factors in causing the accident
– importance of the problem and contributing factors in future accidents
• Each Intervention rate– How well the intervention solves the problem or mitigates
the contributing factors for the accident– Rates the team belief that the intervention will be
implemented and executed as intended
What is Culture*?:
“The way we do things around here”
*aka Climate
1 attitude = opinion…everyone’s attitude = culture
–Disagreements in the ICU are appropriately resolved (i.e., not who is right, but what is best for the patient)
–Our doctors and nurses work together as a well coordinated team
Teamwork climate: perceived quality of collaboration between the personnel in this unit
–Trainees in my discipline are adequately supervised
–This hospital deals constructively with problem personnel
Working conditions: perceived quality of the work environment and logistical support (staffing, training, etc.)
–I am less effective at work when fatigued
–When my workload becomes excessive, my performance is impaired
Stress recognition: acknowledgement of how performance is influenced by stressors (permitted to be human)
–Hospital management supports my daily efforts in the ICU
–Hospital management does not knowingly compromise the safety of patients
Perceptions of management: approval of managerial action
–I would feel safe being treated in this ICU
–Medical errors are handled appropriately in this ICUSafety climate: perceptions of a strong and proactive commitment to patient safety in this unit
–I like my job
–This hospital is a good place to workJob satisfaction: positivity about the work experience
Example itemsFactor: Definition
Executive Perceptions vs. Frontline Perceptions:
Executives overestimate:Teamwork Climate 4XSafety Climate 2.5X
Executive Confidence vs. Executive Accuracy:
-Often wrong but rarely in doubt…-Currently no incoming data-streams-Halo Effects-Frontline data fills the gap
* * * * * *
* Statistically Significant
64 Teamwork Climate 200667 Teamwork Climate 2007
71 Teamwork Climate 2008
62 Teamwork Climate 2005
60 Safety Climate 200665 Safety Climate 200770 Safety Climate 2008
59 Safety Climate 2005
CS
ICU
T1
CS
ICU
T2
0
10
20
30
40
50
60
70
80
90
100
% o
f res
pond
ents
with
in a
n IC
U th
at a
gree
#4. “I Would Feel Safe Being Treated Here As A Patient.”
CS
ICU
T1
CS
ICU
T2
0
10
20
30
40
50
60
70
80
90
100
C S IC U
T 2
% o
f res
pond
ents
with
in a
n IC
U th
at a
gree
#3. “Nurse Input Is Well Received In This ICU.”
CS
ICU
T2
CS
ICU
T1
0
10
20
30
40
50
60
70
80
90
100
% o
f res
pond
ents
with
in a
n IC
U th
at a
gree
#26. “In This ICU, It Is Difficult To Speak Up If I Perceive A Problem With Patient Care.”
CS
ICU
T1
CS
ICU
T2
0
10
20
30
40
50
60
70
80
90
100
% o
f res
pond
ents
with
in a
n IC
U th
at a
gree
#32. “Disagreements In This ICU Are Resolved Appropriately (i.e. not who is right, but what is best for the patient).”
Questions for Reflection
• How do you know you are safer?
• How will you become more efficient in your measurement efforts?
• How will you better tap into local wisdom?
Focus and Execute