Building and Measuring Safety Culture:

Getting Started & Planning Your Survey Process

August 21 & 23, 2012

2

Polling Question

1. Has your organization conducted a safety culture survey in the past 12 months?

a. Yes

b. No

c. Do not know

3

Polling Question

2. If yes, what survey instrument was used?a. Safety Attitude Questionnaire (SAQ)

b. Hospital Survey on Patient Safety (HSOPS)

c. Other survey or measurement tool

d. Do not know

4

Learning Objectives

1. Define safety culture

2. Describe why safety culture is important for improvement efforts

3. Identify best practices for measuring safety culture– Define the role and responsibilities of the survey

coordinator

– Identify characteristics of successful surveys and action items

5

A BRIEF OVERVIEW OF SAFETY CULTURE

Part I

6

The Armstrong Institute Model to Improve Care

http://www.hopkinsmedicine.org/armstrong_institute

7

What is Safety Culture?

Perceived priority of safety relative to other goals

Culture is the compass team members use to guide their behaviors, attitudes, & perceptions on the job

• What will I get praised for?• What will I get reprimanded for?• What is the “right” thing to do?

Culture provides the context for team success

Image source: Marysia Tomaszewska, August 8, 2012, used under a Creative Commons License

What Are the Core Aspects of Safety Culture…

Schein, 2007

9

Why Safety Culture Matters

1. Safety culture is related to outcomes

– Patient outcomes Patient care experience

Infection rates, sepsis

Postop. hemorrhage, respiratory failure, accidental puncture/laceration

Treatment errors

– Clinician outcomes Incident reporting, burnout, turnover

Huang et al., 2010; Mardon et al., 2010; MacDavitt et al., 2007; Singer et al., 2009; Sorra et al., 2012; Weaver, 2011.

10

Why Safety Culture Matters

2. Safety culture influences the effectiveness of other safety and quality interventions

– Can enhance or inhibit effects of other interventions

3. Safety culture can change through intervention

– Best evidence so far for culture interventions that use multiple components

Haynes et al., 2011; Morello et al., 2012; Van Nord et al., 2010; Weaver et al., in press

11

MEASURING SAFETY CULTUREPart II

12

CUSP & Safety Culture

• Safety Culture is typically measured “Pre-CUSP” (before interventions begin):– Provides a baseline to diagnose barriers and facilitators that

can impact improvement efforts– Then can be measured 12-18 months following start of

improvement efforts

• Use reliable and valid survey instrument– Hospital Survey on Patient Safety (HSOPS)

• CUSP is the intervention that you will use to help you improve culture results

13

What is the Hospital Survey on Patient Safety (HSOPS)?

Part of a suite of survey tools (SOPS) for hospitals, medical offices, nursing homes

• Sponsored by: Agency for Healthcare Research & Quality

• Developed by Westat, public release in 2004

The HSOPS App: An online survey tool Developed by the Armstrong Institute, in partnership with CeCity

Allows participants to complete the survey online and survey coordinators to access detailed reports of results

Participants are asked to choose 1 to 5 for each question:1 Strongly Disagree 2 Disagree 3 Neither Agree nor Disagree 4 Agree 5Strongly Agree

1 Never 2 Rarely 3 Sometimes 4 Most of the time 5 Always

14

HSOPS Questions & Composite Scores

10 Composite Scores (“Dimensions”)

No. of Questions

Example Question

1. Supervisor/manager expectations & actions promoting patient safety 4 B1. My supervisor/manager seriously considers staff

suggestions for improving patient safety.

2. Organizational learning-continuous improvement 3 A9. Mistakes have led to positive changes here

3. Teamwork within unit 4 A1. People support one another in this unit.

4. Communication openness 3 C4. Staff feel free to question the decisions or actions of those with more authority.

5. Feedback & communication about error 3 C1. We are given feedback about changes put into place based on event reports.

6. Nonpunitive response to error 3 A8. Staff feel like their mistakes are held against them. (negatively worded)

7. Staffing 4 A2. We have enough staff to handle the workload.

8. Hospital management support for patient safety 3 F8. The actions of hospital management show that

patient safety is a top priority.

9. Teamwork across hospital units 4 F4.There is good cooperation among hospital units that need to work together.

10. Hospital handoffs & transitions 4 F5.Important patient care information is often lost during shift changes. (negatively worded)

15

HSOPS Questions & Composite Scores (cont’d)

6 Background questions

4 Outcome Variables No. of Questions

Example Question

1. Overall perceptions of safety 4 A15. Patient safety is never sacrificed to get more work done.

2. Frequency of event reporting 3 D1. When a mistake is made, but is caught and corrected before affecting the patient, how often is this reported?

3. Patient safety grade (of hospital unit)

1 E1. Please give your work area/unit in this hospital an overall grade on patient safety.

4. Number of events reported in the last 12 months

1 G1. In the past 12 months, how many event reports have you filled out and submitted?

16

HSOPS Scoring

• Scoring guidelines created by AHRQ• Scores represent the % of positive responses

– % who gave a score of 4 or 5

17

OVERVIEW OF THE SURVEY ADMINISTRATION PROCESS

Part III

18

Effective surveys…

1. Are confidential and anonymous• No way to link responses with email address or other individual

information • Results reported at unit-level• Critical that the survey coordinator, project team, and leadership focus on

survey as a tool for learning and improvement

2. Have a clear reason and motivation to complete them• Must be clear to staff why you are asking them to complete this survey,

what will happen with results, and what will be done based on their input

3. Are publically supported by both formal and informal unit leaders

• If leaders and colleagues view the survey as important and worthy of their time, frontline staff will be more motivated to contribute their input

19

Two Options for HSOPS

Option 1 Option 2

Collect fresh HSOPS data from participating work areas

• Use online HSOPS App to survey work area team

members

Upload previously collected HSOPS data from participating

work areas

• Upload excel file to online HSOPS App

What are the key tasks for planning our survey?

Phase TaskPrep (~3 wks)

wk

1wk

2wk

3wk

4wk

5wk

6wk

7wk

8wk

9

Planning stage

• Logistical tasks

• Who will administer?

• Who will be surveyed?• Publicize! Motivate participation.

• Create a debriefing plan

Alert participants• Provide pre-notification (post, email,

meetings)

Upload unit data • Log-on and enter unit background info

Start!• Upload participant email addresses or

previously collected data

Track • Monitor response rates (target >60%)

Remind • 1st reminder

Remind again • Final reminder

End the survey • Close the survey

Collect data • Download results report

Execute • Debrief & plan improvements

21

Who are key personnel in the HSOPS process?

22

Survey Coordinators: Key Responsibilities

Survey coordinators…

• Help to coordinate the survey administration process,

• Learn how to use the online survey database (via training webinars and conference calls)

• Help survey participants with the survey process,

• Enter data about the work areas participating in the survey into the online survey database,

• Monitor the survey response rate using the online survey database,

• Work with hospital and work area leadership to distribute survey materials and information,

• Communicate with other HSOPS Coordinators in the project

23

Online HSOPS Application (“HSOPS App”)

24

Developing HSOPS Survey List

1. Determine whom to survey

Clinical staff only? Non-clinical support staff (e.g., environmental services)?

2. Compile unit email list

E-mail address Hospital area/unit Staffing category (e.g., physician, nurse, technician, administrative

support)

Make sure your list is up to date, check for:– Staff on administrative or extended sick leave, – Staff who appear in more than one staffing category or hospital area/unit, – Staff who have moved to another hospital area/unit, – Staff who no longer work at the hospital, and – Other changes that may affect the accuracy of your list email addresses

25

Pre-Notification Materials to Inform Unit Clinicians and Staff of HSOPS

1. Why the hospital is conducting the survey and how staff responses will be used,

2. Which hospital staff were selected to be surveyed (e.g., all staff, nursing staff, all clinical staff, a random sample of staff, etc.),

3. How much time is needed to complete the survey, Confidentiality or anonymity assurances,

4. Suggested reply timeframe and how to return completed surveys,

5. Incentives for which units will be eligible, if they respond (Optional), and

6. Contact information for the main hospital point-of-contact.

Sorra & Nieva, 2004

26

Why should we aim for a response rate of 60% or higher?

Response rate =Number of respondents who completed the online survey divided by the total number of unit members who were invited to complete the survey

Safety culture reflects the shared perceptions among unit members

• Response rate of 60% is minimum necessary for a relatively representative sample

• Important for validity and reliability of results

27

Thinking ahead: Creating a Debriefing Plan for your Survey Results

Debriefing is…

– A semi-structured conversation among frontline clinicians and staff that is usually led by a designated facilitator

Purpose…

1. Encourage open communication, transparency, and interactive discussion about the survey results across all levels

2. To engage clinicians and staff in generating and implementing their ideas about how to create an effective safety culture in their work area

28

Some points to cover in your debriefing plan

29

We are here to support & guide: HSOPS Calls

1. Getting Started & Planning the Process • Intro to measurement, basics of survey administration

2. The online HSOPS process• A more detailed, technical call for survey coordinators that will

walk through the details of how to use the online survey app

3. Interpreting HSOPS data and debriefing• Once data are collected and survey coordinators have download

their results, another call will be held to walk through how to read reports, debriefing staff ,and interpret results

30

What if we have other surveys going on?

It is common for hospitals to use several different surveys to elicit input from clinicians and staff

Efforts to measure safety culture can be aligned with other surveys

• E.g., JHM measures both safety culture and employee engagement annually

Some tips:

• Try to not have more than 1 survey going on at the same time if possible (Lack of time and survey fatigue can negatively impact results)

• Consider similar trends observed across surveys

• You can debrief results of similar surveys at the same time if similar trends or common themes are observed in results

31

Some Additional Tips for Effective Surveys

• Communicate to all staff on the unit the importance of their valuable input

• Check response rates periodically and send reminders at different times throughout the survey administration process: before the survey is administered and a few times during the survey administration process

• Send periodic reminders through various communication channels, such as emails, announcements, during staff meetings, etc.

• Create a debriefing plan for sharing results with clinicians and staff

• Don’t be afraid to reach out for assistance when in doubt

32

Action Items:

1. Identify a survey coordinator for participating work areas …Or coordinators if your team desires more than one coordinator

2. Send collated list of survey coordinators to your coordinating entity (HEN, Hospital association, or affinity group leader)

If collecting new data or uploading previously collected data

Coordinating entity to compile and send to JHU

3. Determine whom to survey and sample size …Or locate previously collected data file

4. If collecting fresh HSOPS data.: Compile unit email list

Create informational materials to publicize the survey (posters, flyers, newsletters, meeting agenda items)

Send pre-notification letter, post pre-notification information, or hold pre-notification meeting

5. Create draft debriefing plan

33

References

• Haynes AB, Weiser TG, Berry WR, Lipsitz SR, Breizat AH, Dellinger EP, Dziekan G, Herbosa T, Kibatala PL, Lapitan MC, Merry AF, Reznick RK, Taylor B, Vats A, Gawande AA; Safe Surgery Saves Lives Study Group. Changes in safety attitude and relationship to decreased postoperative morbidity and mortality following implementation of a checklist-based surgical safety intervention. BMJ Qual Saf. 2011 Jan;20(1):102-7.

• Huang DT, Clermont G, Kong L, Weissfeld LA, Sexton JB, Rowan KM, Angus DC. Intensive care unit safety culture and outcomes: a US multicenter study. Int J Qual Health Care. 2010 Jun;22(3):151-61.

• MacDavitt K, Chou SS, Stone PW. Organizational climate and health care outcomes. Jt Comm J Qual Patient Saf. 2007 Nov;33(11 Suppl):45-56.

• Mardon RE, Khanna K, Sorra J, Dyer N, Famolaro T. Exploring relationships between hospital patient safety culture and adverse events. J Patient Saf. 2010 Dec;6(4):226-32.

• Morello RT, Lowthian JA, Barker AL, McGinnes R, Dunt D, Brand C. Strategies for improving patient safety culture in hospitals: a systematic review. BMJ Qual Saf. 2012 Jul 31. [Epub ahead of print]

• Schein E. Organizational culture and leadership, 4th edition. San Francisco, CA: Jossey-Bass. 2010.

34

References

• Singer SJ, Falwell A, Gaba DM, Meterko M, Rosen A, Hartmann CW, Baker L. Identifying organizational cultures that promote patient safety. Health Care Manage Rev. 2009 Oct-Dec;34(4):300-11.

• Sorra J, Khanna K, Dyer N, Mardon R, Famolaro T. Exploring Relationships Between Patient Safety Culture and Patients' Assessments of Hospital Care. J Patient Saf. 2012 Jul 10. [Epub ahead of print].

• Sorra JS, Nieva VF. Hospital Survey on Patient Safety Culture. (Prepared by Westat, under Contract No. 290-96-0004). AHRQ Publication No. 04-0041. Rockville, MD: Agency for Healthcare Research and Quality. September 2004.

• van Noord I, de Bruijne MC, Twisk JW. The relationship between patient safety culture and the implementation of organizational patient safety defences at emergency departments.. Int J Qual Health Care. 2010 Jun;22(3):162-9.

• Weaver SJ. A configural approach to patient safety climate: The relationship between climate profile characteristics and patient safety. Doctoral dissertation. University of Central Florida. 2011.

• Weaver, S. J., Dy, S., Lubomski, L., & Wilson, R. Promoting a culture of safety. In R.M. Watcher, P.G. Shekelle, P. Pronovost (Eds.). Making healthcare safer: A critical analysis of the evidence of patient safety practices (AHRQ report # TBD). Rockville, MD. In press.