Educational Video-based Scenarios and Evaluative InstrumentFor Handoff Communication
Saba Berhie1; Vineet M Arora1; Leora Horwitz2; Mark Saathoff1; Paul Staisiunas1; Jeanne M. Farnan1 1University of Chicago, Chicago, IL; 2Yale University, New Haven ,CT
Background
• Single institution and small sample size• Future work planned at partner institutions
With generous funding from: • AHRQ R03 Grant # 1R03HS018278-01• NIA T35 Grant #5T35AG029795-02
Instrument Development: Handoff CEX
Communication Professionalism Setting
Superior “Check post-transfusion HgB at 2pm. If <10, transfuse 2 U“
No inappropriate comments, not hurried
Satisfactory
“the only to do is check CBC at 2pm”
“I’ve had a rough day, lets be quick. I’ll start with the sickest first, she’s a train wreck”
Un-satisfactory
“Oh yeah, can you check labs?”
“the ER could not intubate to save their life”
Research Funding
Limitations
Workshop Objectives & Structure
Conclusions
• Utilize videos which highlight varying levels of handoff performance.
• Rate and assess each video utilizing Handoff CEX instrument
WORKSHOP OUTLINE• Two 60-minutes workshops offered • Six videos watched and rated
independently by faculty participants using Handoff CEX • Post-workshop roundtable held after viewing the videos
• 3-item Handoff CEX shows promise as a reliable and valid tool to assess varying levels of videos depicting handoff performance
• Raters had difficulty distinguishing between satisfactory and superior levels• Correlation between communication and professionalism
• Future work in real-time clinical scenarios
Results
• 14 faculty from 2 departments participated• 73 of a possible 90 (82%) handoff observations were captured
• Reliability•Cronbach’s alpha: 0.81•Kendall’s coefficient of concordance: 0.59
• Based on “Mini- CEX” instrument widely used in internal medicine (Norcini,2003)
Peer Evaluation Pilot Testing
• Internal Medicine residents at the University of Chicago: 366 handoff observations
• Factor analysis revealed that three domains captured 82% of the variance of overall sign-out quality
• Communication, Professionalism and Setting
02
46
810
Unsatisfactory Satisfactory Superior
Communication
p <0.001
•Validity • Test of trend across ordered groups: p<0.001• Two-way ANOVA
• No rater bias; results significant only for difference between “superior” & “unsatisfactory” levels of performance
• July 2011– ACGME sets new limits for resident duty hours
• Unintended consequence is increase in number of handoffs
• Few trainees receive education on effective handoff communication (Horwitz, 2006)
• Lack of validated instruments to measure handoff performance
HOW ACCOMPLISH THIS?
Specific Aims
• To create video-based examples of varying levels of handoff performance for education, adapting the approach in Arora et al MedEd Portal:
“Handoffs: A Typical Day on the Wards”
• To validate an assessment instrument: the Handoff CEX
• 6 scripts developed & reviewed• Changing handoff quality in the 3 domains of performance• De-brief instrument development
VIDEO DEVELOPMENT
MedEd Portal Submission
• Arora V, Farnan J, Paro J, Vidyarthi A, Johnson J. Teaching Video:"Handoffs: A Typical Day on the Wards” MedEdPORTAL; 2011
Figure 1: Faculty Ratings of Communication
02
46
810
Unsatisfactory Superior
Setting
p <0.001
Figure 2: Faculty Ratings of Setting
02
46
81
0
Unsatisfactory Satisfactory Superior
Professionalism
p <0.001
Figure 3: Faculty Ratings of Professionalism
• Ability to detect level of performance for communication impacted by level of professionalism depicted
Top Related