Design and Validation of a Lean Leader Behavior Self...
Transcript of Design and Validation of a Lean Leader Behavior Self...
Design and Validation of a Lean Leader Behavior Self-Assessment Tool
Laura Winner, DrPH, RN Senior Director, Lean Sigma Deployment
Erica Reinhardt, MS, RD Lean Six Sigma Black Belt, Asst. Director, Lean Sigma Deployment
© The Johns Hopkins University, The Johns Hopkins Hospital, and Johns Hopkins Health System
June 12, 2019
Armstrong Institute for Patient Safety and QualityJohns Hopkins Medicine
2
MissionWe partner with patients, their loved ones and all interested parties to end preventable harm, to continuously improve patient outcomes and experience, and to eliminate waste in health care.
Disclosures
3
None
Background
4
• Johns Hopkins Bayview Medical Center sought to prepare healthcare leaders for their role managing in a lean management system environment
Identified the principle-based behaviors we wanted from managers & staffReviewed the literature for existing lean assessment tools
Review of Literature
5
• Majority of lean assessments were at the organization level• Focused on implementation of Lean tools and methods
Lean Assessment Tool Industry
Lean Production Checklist Manufacturing
Lean Construction Maturity Model Construction
Lean Manufacturing: Performance Evaluation Audit Manufacturing
Lean Index Wood products industry
Lean Enterprise Self-Assessment Tool (LESAT) Aerospace
Shingo Prize for Manufacturing Excellence Manufacturing
Shingo Prize for Operational Excellence Expanded to other industries in 2008
Study Aims
6
• Aim 1– Develop and validate a tool to assess adoption of ideal
Lean behaviors at an individual level
• Aim 2– Pilot the validated tool at Johns Hopkins Bayview Medical
Center.
Setting
7
Acute Care Medical Centers within the Johns Hopkins Health System in Maryland.
Methods: Tool DevelopmentDrawing from Lean Management Thought Leaders
8
• Shigeo Shingo– Industrial engineer, expert in Toyota Production System
• John Toussaint– Physician expert in Lean Management in healthcare
• David Mann– Organizational psychologist, expert in Lean Management Systems
Methods: Tool Development Shingo Guiding Principles for Enterprise Excellence
9
Shingo Institute, https://shingo.org
Methods: Tool Development Toussaint & Ehrlich Lean Behavioral Dimensions
10
Toussaint JS, Ehrlich, SP. Five Changes Great Leaders Make to Develop an Improvement Culture. NEJM Catalyst, April 27, 2017.
Methods: Tool Development Toussaint & Berry Lean Principles
11
6 principles:• Lean is an Attitude of Continuous Improvement• Lean is Value-Creating• Lean is Unity of Purpose• Lean is Respect for the People Who Do the Work• Lean is Visual• Lean is Flexible Regimentation
Toussaint JS, Berry LL. The Promise of Lean in Health Care. Mayo Clinic Proceedings 2013; 88(1):74-82.
Methods: Tool Development Mann’s Managerial Elements
12
• Leader Standard Work• Visual Controls• Daily Accountability• Discipline
Mann D. Creating a Lean Culture. 3rd edn. New York: CRC Press, 2015..
Methods: Tool Development Cross Walk of Themes
13
Shingo
Toussaint & Ehrlich
Toussaint & Berry
Mann
Continuous Improvement
Seek PerfectionEmbrace Scientific
Thinking Focus on Process
PerseveranceCuriosity
Attitude of Continuous
ImprovementVisual
Visual ControlsDaily Accountability
Create Value
Flow & Pull Value Assure Quality at the
SourceCreate Value for the
Customer
Value-Creating
Unity of Purpose
Think SystemicallyCreate Constancy of
Purpose
Unity of Purpose
Respect for People
Respect Every Individual
Lead with Humility
Humility
Respect for the People Who Do the
Work
Flexible Regimentation
Self-DisciplineWillingness
Flexible Regimentation
Leader Work StandardDiscipline
Methods: Tool Development Lean Leader Principle-Based Behavioral Domains
14
• Promote Continuous Improvement• Create Value• Inspire Unity of Purpose• Demonstrate Respect for People• Embrace Flexible Regimentation
Methods: Tool Development Constructing questions
15
• Brainstormed specific behaviors– JHBMC leaders and staff – Health system lean experts
• Combined lists• Categorized behaviors by domain • Paired down to the essence of each domain• Resulted in 27 items
Methods: Tool Development
16
• Section 1: You, Your Peers, and Your Leadership– 27 items (81 responses)
• Section 2: Your Organization– 5 items
• Section 3: You (Demographics)
Methods: Tool DevelopmentContent Validity
17
• Requested participation from 9 Lean experts– Upon agreement to participate, emailed excel version of the tool
• Received 8 responses– 2 Health Services Researchers– 1 Expert in Lean Manufacturing– 5 Experts in Lean Healthcare
• Responses collated for analysis
Methods: Tool DevelopmentContent Validity
18*Polit et al.: For 6-10 reviewers, I-CVI is valid if at least 0.78. I-CVI used by Marsteller et al. at 0.75 threshold.
• Relevancy assessment of each item using the following four-point Likert scale:
4 -Highly Relevant3 -Quite Relevant 2 -Somewhat Relevant1 -Not Relevant
• Calculated Item-level Content Validity Indices (I-CVI) – I-CVI = # in agreement / # responses– Included items with I-CVI of at least 0.75*
In Agreement
Results & Data AnalysisSection 1: You, Your Peers, Your Leadership
19
• In this section, think about the frequency with which YOU, YOUR PEERS, and YOUR LEADERSHIP exhibit the following behaviors.
• 5 Domains– Continuous Improvement– Create Value– Unity of Purpose– Respect for People– Flexible Regimentation
Results & Data Analysis: Content Validity
20
Initial VersionPost-Content
Validity ChangeSection 1: You, Your Peers, and Your Leadership 27 26 -1
Continuous Improvement 7 4 -3Create Value 5 5 0Unity of Purpose 4 4 0Respect for People 8 9 +1Flexible Regimentation 3 4 +1
Section 2: Your Organization 5 5 0Section 3: You (Demographics) n/a n/a n/a
# of items
Results & Data Analysis: Scale Content Validity
21
• Calculated Scale-level Content Validity Indices (S-CVI)– S-CVI = average of I-CVIs = 0.956*
*Polit et al.: S-CVI is valid if at least 0.90.
Strengths & Limitations
22
Strengths• Private collection of the content validity feedback from experts
Limitations• Selection Bias: Experts chosen may have opinions that differ
from others
Next Steps
23
• Face Validity– 2 focus groups of acute care leaders identified by Senior Director of Innovation &
Continuous Improvement• “More” Lean exposed• “Less” Lean exposed
– Feedback on usability and sensibility• Do the questions make sense to you?• What do you think about how we are trying to measure a culture of continuous
improvement?• Is there a concept you think does not apply to you?
• Application of tool at Acute Care Medical Center
Future Research
24
• Based on results from the pilot, consider opportunities to validate the tool in other settings
References
25
• Mann D. Creating a Lean Culture. 3rd edn. New York: CRC Press, 2015.
• Marsteller JA, Hsu YJ, Chan KS, Lubomski LH. Assessing content validity and user perspectives on the Tea Check-up Tool: expert survey and user focus groups. BMJ Quality & Safety Published Online First: 12 April 2016 doi:10.1136/bmjqs-2015-004635.
• Nightingale D, Abdimomunova L, Shields T, Perkins N, Srinivasan J, Valerdi R. LAI Enterprise Self-Assessment Tool LESAT Version 2.0 February 2012. Massachusetts Institute of Technology. 2012.
• Polit DF, Beek CT, Owen SV. Is the CVI an acceptable indicator of content validity? Appraisal and recommendations. Res Nurs Health 2007;30:459–67.
• Shingo Application Guidelines. Behavior Assessment Scale. https://shingo.org
• Toussaint JS, Berry LL. The Promise of Lean in Health Care. Mayo Clinic Proceedings 2013;88(1):74-82.
• Toussaint JS, Ehrlich, SP. Five Changes Great Leaders Make to Develop an Improvement Culture. NEJM Catalyst, April 27, 2017.
• Urban W. The Lean Management Maturity Self-Assessment Tool Based on Organisational Culture Diagnosis. Procedia –Social and Behavioral Sciences. 2015;213:728-733.
26
Your Questions