Transforming EHR Interactions Using Voice Assistants...1 Transforming EHR Interactions Using Voice...
Transcript of Transforming EHR Interactions Using Voice Assistants...1 Transforming EHR Interactions Using Voice...
![Page 1: Transforming EHR Interactions Using Voice Assistants...1 Transforming EHR Interactions Using Voice Assistants Session 164, February 13, 2019 Yaa Kumah-Crystal, MD MPH Timothy Coffman](https://reader035.fdocuments.us/reader035/viewer/2022070923/5fbb8b4ffa3f7d6f340df4c5/html5/thumbnails/1.jpg)
1
Transforming EHR Interactions Using Voice Assistants
Session 164, February 13, 2019
Yaa Kumah-Crystal, MD MPH
Timothy Coffman
Vanderbilt University Medical Center
![Page 2: Transforming EHR Interactions Using Voice Assistants...1 Transforming EHR Interactions Using Voice Assistants Session 164, February 13, 2019 Yaa Kumah-Crystal, MD MPH Timothy Coffman](https://reader035.fdocuments.us/reader035/viewer/2022070923/5fbb8b4ffa3f7d6f340df4c5/html5/thumbnails/2.jpg)
2
Yaa Kumah-Crystal, MD MPH
Discloses the following relevant relationship with commercial interests:
• Research partnership with Nuance Communications
• Use of NLP libraries for research and development
Conflict of Interest
![Page 3: Transforming EHR Interactions Using Voice Assistants...1 Transforming EHR Interactions Using Voice Assistants Session 164, February 13, 2019 Yaa Kumah-Crystal, MD MPH Timothy Coffman](https://reader035.fdocuments.us/reader035/viewer/2022070923/5fbb8b4ffa3f7d6f340df4c5/html5/thumbnails/3.jpg)
3
Timothy Coffman
Discloses the following relevant relationship with commercial interests:
• Research partnership with Nuance Communications
• Use of NLP libraries for research and development
Conflict of Interest
![Page 4: Transforming EHR Interactions Using Voice Assistants...1 Transforming EHR Interactions Using Voice Assistants Session 164, February 13, 2019 Yaa Kumah-Crystal, MD MPH Timothy Coffman](https://reader035.fdocuments.us/reader035/viewer/2022070923/5fbb8b4ffa3f7d6f340df4c5/html5/thumbnails/4.jpg)
4
• Principles employed in creating a voice user interface for the electronic health record
• Challenges faced in data capture
• Challenges faced in data presentation
• Assessment
• Implementation and future plans
Agenda
![Page 5: Transforming EHR Interactions Using Voice Assistants...1 Transforming EHR Interactions Using Voice Assistants Session 164, February 13, 2019 Yaa Kumah-Crystal, MD MPH Timothy Coffman](https://reader035.fdocuments.us/reader035/viewer/2022070923/5fbb8b4ffa3f7d6f340df4c5/html5/thumbnails/5.jpg)
5
• Discuss the value of artificial intelligent (AI)-enabled natural language processing and understanding technologies in the advancement of healthcare
• Discuss the limitations of current keyboard and mouse workflows in the EHR
• Describe the usability design considerations, information request mapping and artificial intelligence/natural language understanding platform built to process voice requests
• Discuss the provider cohort experience and outcomes around satisfaction
Learning Objectives
![Page 6: Transforming EHR Interactions Using Voice Assistants...1 Transforming EHR Interactions Using Voice Assistants Session 164, February 13, 2019 Yaa Kumah-Crystal, MD MPH Timothy Coffman](https://reader035.fdocuments.us/reader035/viewer/2022070923/5fbb8b4ffa3f7d6f340df4c5/html5/thumbnails/6.jpg)
6
Informal Poll
Own a smart phone
Use your smart phone voice assistant
• Siri, Google assistant, Bixby
Own a voice assistant device
• Alexa, Google home, Apple Homepod
![Page 7: Transforming EHR Interactions Using Voice Assistants...1 Transforming EHR Interactions Using Voice Assistants Session 164, February 13, 2019 Yaa Kumah-Crystal, MD MPH Timothy Coffman](https://reader035.fdocuments.us/reader035/viewer/2022070923/5fbb8b4ffa3f7d6f340df4c5/html5/thumbnails/7.jpg)
7
Why Voice?
![Page 8: Transforming EHR Interactions Using Voice Assistants...1 Transforming EHR Interactions Using Voice Assistants Session 164, February 13, 2019 Yaa Kumah-Crystal, MD MPH Timothy Coffman](https://reader035.fdocuments.us/reader035/viewer/2022070923/5fbb8b4ffa3f7d6f340df4c5/html5/thumbnails/8.jpg)
8
Documentation Modalities
Writing
~13
WPM
Typing
~40
WPM
Speech
~150
WPM
![Page 9: Transforming EHR Interactions Using Voice Assistants...1 Transforming EHR Interactions Using Voice Assistants Session 164, February 13, 2019 Yaa Kumah-Crystal, MD MPH Timothy Coffman](https://reader035.fdocuments.us/reader035/viewer/2022070923/5fbb8b4ffa3f7d6f340df4c5/html5/thumbnails/9.jpg)
9
Why not Voice?
![Page 10: Transforming EHR Interactions Using Voice Assistants...1 Transforming EHR Interactions Using Voice Assistants Session 164, February 13, 2019 Yaa Kumah-Crystal, MD MPH Timothy Coffman](https://reader035.fdocuments.us/reader035/viewer/2022070923/5fbb8b4ffa3f7d6f340df4c5/html5/thumbnails/10.jpg)
10
Macros
Physical exam
Insert normal physical exam
![Page 11: Transforming EHR Interactions Using Voice Assistants...1 Transforming EHR Interactions Using Voice Assistants Session 164, February 13, 2019 Yaa Kumah-Crystal, MD MPH Timothy Coffman](https://reader035.fdocuments.us/reader035/viewer/2022070923/5fbb8b4ffa3f7d6f340df4c5/html5/thumbnails/11.jpg)
11
Vanderbilt Innovations Portfolio Research
![Page 12: Transforming EHR Interactions Using Voice Assistants...1 Transforming EHR Interactions Using Voice Assistants Session 164, February 13, 2019 Yaa Kumah-Crystal, MD MPH Timothy Coffman](https://reader035.fdocuments.us/reader035/viewer/2022070923/5fbb8b4ffa3f7d6f340df4c5/html5/thumbnails/12.jpg)
12
Given
• Installed EHR
• EHR vendor has existing partnership with natural language
services provider (including dictation)
• Existing in-house innovation capacity– Infrastructure supporting custom development
– Team in place with necessary SW development skills
– Mission to develop expertise in new ways to interact with the EHR
Decision
• Develop new software to integrate EHR with additional natural
language services
Solutions
![Page 13: Transforming EHR Interactions Using Voice Assistants...1 Transforming EHR Interactions Using Voice Assistants Session 164, February 13, 2019 Yaa Kumah-Crystal, MD MPH Timothy Coffman](https://reader035.fdocuments.us/reader035/viewer/2022070923/5fbb8b4ffa3f7d6f340df4c5/html5/thumbnails/13.jpg)
13
Well really, how how hard can it be?
13
![Page 14: Transforming EHR Interactions Using Voice Assistants...1 Transforming EHR Interactions Using Voice Assistants Session 164, February 13, 2019 Yaa Kumah-Crystal, MD MPH Timothy Coffman](https://reader035.fdocuments.us/reader035/viewer/2022070923/5fbb8b4ffa3f7d6f340df4c5/html5/thumbnails/14.jpg)
14
Oh...
![Page 15: Transforming EHR Interactions Using Voice Assistants...1 Transforming EHR Interactions Using Voice Assistants Session 164, February 13, 2019 Yaa Kumah-Crystal, MD MPH Timothy Coffman](https://reader035.fdocuments.us/reader035/viewer/2022070923/5fbb8b4ffa3f7d6f340df4c5/html5/thumbnails/15.jpg)
15
Demonstration
![Page 16: Transforming EHR Interactions Using Voice Assistants...1 Transforming EHR Interactions Using Voice Assistants Session 164, February 13, 2019 Yaa Kumah-Crystal, MD MPH Timothy Coffman](https://reader035.fdocuments.us/reader035/viewer/2022070923/5fbb8b4ffa3f7d6f340df4c5/html5/thumbnails/16.jpg)
16
![Page 17: Transforming EHR Interactions Using Voice Assistants...1 Transforming EHR Interactions Using Voice Assistants Session 164, February 13, 2019 Yaa Kumah-Crystal, MD MPH Timothy Coffman](https://reader035.fdocuments.us/reader035/viewer/2022070923/5fbb8b4ffa3f7d6f340df4c5/html5/thumbnails/17.jpg)
17
VEVA Processing
NLP
User Query
Speech to text
Intent Fulfillment Text to speech
Results Display
What is her A1C?
Sally’s A1C is 8.6
![Page 18: Transforming EHR Interactions Using Voice Assistants...1 Transforming EHR Interactions Using Voice Assistants Session 164, February 13, 2019 Yaa Kumah-Crystal, MD MPH Timothy Coffman](https://reader035.fdocuments.us/reader035/viewer/2022070923/5fbb8b4ffa3f7d6f340df4c5/html5/thumbnails/18.jpg)
18
VEVA Architecture
![Page 19: Transforming EHR Interactions Using Voice Assistants...1 Transforming EHR Interactions Using Voice Assistants Session 164, February 13, 2019 Yaa Kumah-Crystal, MD MPH Timothy Coffman](https://reader035.fdocuments.us/reader035/viewer/2022070923/5fbb8b4ffa3f7d6f340df4c5/html5/thumbnails/19.jpg)
19
• EHR’s FHIR implementation does not yet support all desired data
• Additional development required for each new query
• New phrasings of questions requiring updates to language model
• Microphone and speaker issues in a virtual desktop environment
• Latency
Technical Limitations
![Page 20: Transforming EHR Interactions Using Voice Assistants...1 Transforming EHR Interactions Using Voice Assistants Session 164, February 13, 2019 Yaa Kumah-Crystal, MD MPH Timothy Coffman](https://reader035.fdocuments.us/reader035/viewer/2022070923/5fbb8b4ffa3f7d6f340df4c5/html5/thumbnails/20.jpg)
20
Focus on developing demonstration of capabilities supporting
• One (1) specialty clinic
• Handful of general responses
• Aligned with FHIR (DSTU2) resources
• Some EHR-proprietary services
Build in capability to extend with
• Additional capabilities for other specialties
• General responses, discovered through user feedback
• FHIR (STU3) and FHIR (R4) resources as available
• FHIR-based alternatives to proprietary services
Planning for Extension
![Page 21: Transforming EHR Interactions Using Voice Assistants...1 Transforming EHR Interactions Using Voice Assistants Session 164, February 13, 2019 Yaa Kumah-Crystal, MD MPH Timothy Coffman](https://reader035.fdocuments.us/reader035/viewer/2022070923/5fbb8b4ffa3f7d6f340df4c5/html5/thumbnails/21.jpg)
21
• MVP
• Rapid Iteration
• Vanderbilt as a Design Shop
Technical Considerations
![Page 22: Transforming EHR Interactions Using Voice Assistants...1 Transforming EHR Interactions Using Voice Assistants Session 164, February 13, 2019 Yaa Kumah-Crystal, MD MPH Timothy Coffman](https://reader035.fdocuments.us/reader035/viewer/2022070923/5fbb8b4ffa3f7d6f340df4c5/html5/thumbnails/22.jpg)
22
Integrating voice response involves real-time services
• Audio streaming capabilities
– Virtual desktops introduce challenges
• Automatic connections
• Latency
– Varying hardware installed at point-of-use
• Individual laptop microphones
• High-end dictation microphones
• All of the above
Infrastructure Challenges
![Page 23: Transforming EHR Interactions Using Voice Assistants...1 Transforming EHR Interactions Using Voice Assistants Session 164, February 13, 2019 Yaa Kumah-Crystal, MD MPH Timothy Coffman](https://reader035.fdocuments.us/reader035/viewer/2022070923/5fbb8b4ffa3f7d6f340df4c5/html5/thumbnails/23.jpg)
23
• “While voice is a natural interface for humans, this may be the first
time we are using it as an interface to have a computer
communicate back to us like this. Different rules may apply.”
Design Considerations
![Page 24: Transforming EHR Interactions Using Voice Assistants...1 Transforming EHR Interactions Using Voice Assistants Session 164, February 13, 2019 Yaa Kumah-Crystal, MD MPH Timothy Coffman](https://reader035.fdocuments.us/reader035/viewer/2022070923/5fbb8b4ffa3f7d6f340df4c5/html5/thumbnails/24.jpg)
24
• Brevity vs. Clarity
• “Blood pressure is 120 millimeters of mercury over 80 millimeters of mercury ”
• “Blood pressure is 120 over 80”
• Uncanny Valley of Words
• I ate three big red apples
• I ate red big three apples
• Quantity, Opinion, Size, Age, Shape, Color, Material, Purpose
Design Considerations
![Page 25: Transforming EHR Interactions Using Voice Assistants...1 Transforming EHR Interactions Using Voice Assistants Session 164, February 13, 2019 Yaa Kumah-Crystal, MD MPH Timothy Coffman](https://reader035.fdocuments.us/reader035/viewer/2022070923/5fbb8b4ffa3f7d6f340df4c5/html5/thumbnails/25.jpg)
25
Use Cases
![Page 26: Transforming EHR Interactions Using Voice Assistants...1 Transforming EHR Interactions Using Voice Assistants Session 164, February 13, 2019 Yaa Kumah-Crystal, MD MPH Timothy Coffman](https://reader035.fdocuments.us/reader035/viewer/2022070923/5fbb8b4ffa3f7d6f340df4c5/html5/thumbnails/26.jpg)
26
• Preparing to see a patient in a provider workroom
– Large screen
– PHI private
Use Case A
![Page 27: Transforming EHR Interactions Using Voice Assistants...1 Transforming EHR Interactions Using Voice Assistants Session 164, February 13, 2019 Yaa Kumah-Crystal, MD MPH Timothy Coffman](https://reader035.fdocuments.us/reader035/viewer/2022070923/5fbb8b4ffa3f7d6f340df4c5/html5/thumbnails/27.jpg)
27
• Reviewing information with patient present
– Large screen
– PHI with patient 3rd party
Use Case B
![Page 28: Transforming EHR Interactions Using Voice Assistants...1 Transforming EHR Interactions Using Voice Assistants Session 164, February 13, 2019 Yaa Kumah-Crystal, MD MPH Timothy Coffman](https://reader035.fdocuments.us/reader035/viewer/2022070923/5fbb8b4ffa3f7d6f340df4c5/html5/thumbnails/28.jpg)
28
• In transition in a private environment (commute)
– Small Screen (mobile)- limited visibility
– Primarily verbal navigation
– PHI private
Use Case C
![Page 29: Transforming EHR Interactions Using Voice Assistants...1 Transforming EHR Interactions Using Voice Assistants Session 164, February 13, 2019 Yaa Kumah-Crystal, MD MPH Timothy Coffman](https://reader035.fdocuments.us/reader035/viewer/2022070923/5fbb8b4ffa3f7d6f340df4c5/html5/thumbnails/29.jpg)
29
• In transition in a public environment (hallway)
– Small screen (mobile)
– PHI considerations
(aloud only with headphones?)
Use Case D
![Page 30: Transforming EHR Interactions Using Voice Assistants...1 Transforming EHR Interactions Using Voice Assistants Session 164, February 13, 2019 Yaa Kumah-Crystal, MD MPH Timothy Coffman](https://reader035.fdocuments.us/reader035/viewer/2022070923/5fbb8b4ffa3f7d6f340df4c5/html5/thumbnails/30.jpg)
30
Usability Study
![Page 31: Transforming EHR Interactions Using Voice Assistants...1 Transforming EHR Interactions Using Voice Assistants Session 164, February 13, 2019 Yaa Kumah-Crystal, MD MPH Timothy Coffman](https://reader035.fdocuments.us/reader035/viewer/2022070923/5fbb8b4ffa3f7d6f340df4c5/html5/thumbnails/31.jpg)
31
• Setting: Vanderbilt University Medical Center
• Cohort: 14 Pediatric Endocrinology Providers
• Female (10) | Male (4)
• Mean age 39, range 29-65
Study Design
![Page 32: Transforming EHR Interactions Using Voice Assistants...1 Transforming EHR Interactions Using Voice Assistants Session 164, February 13, 2019 Yaa Kumah-Crystal, MD MPH Timothy Coffman](https://reader035.fdocuments.us/reader035/viewer/2022070923/5fbb8b4ffa3f7d6f340df4c5/html5/thumbnails/32.jpg)
32
• Usability Assessment
• 6 Skills Evaluated
– Training: Hello / What can you do
– Summary
– A1C
– Weight
– Blood Pressure
– Health Maintenance
Study Design
![Page 33: Transforming EHR Interactions Using Voice Assistants...1 Transforming EHR Interactions Using Voice Assistants Session 164, February 13, 2019 Yaa Kumah-Crystal, MD MPH Timothy Coffman](https://reader035.fdocuments.us/reader035/viewer/2022070923/5fbb8b4ffa3f7d6f340df4c5/html5/thumbnails/33.jpg)
33
• Value of information communicated
• Length of content communicated
• Value of graphical display
• Overall thoughts about voice interactions
Feedback
![Page 34: Transforming EHR Interactions Using Voice Assistants...1 Transforming EHR Interactions Using Voice Assistants Session 164, February 13, 2019 Yaa Kumah-Crystal, MD MPH Timothy Coffman](https://reader035.fdocuments.us/reader035/viewer/2022070923/5fbb8b4ffa3f7d6f340df4c5/html5/thumbnails/34.jpg)
34
System Usability Scale: Positively Worded
Strongly Disagree
Mean 4.12 (SD .46) 3.93 (SD 1.20)
4.50 (SD .52)
3.79 (SD .70)
4.00 (SD .79)
4.43 (SD .51)
Strongly Agree
![Page 35: Transforming EHR Interactions Using Voice Assistants...1 Transforming EHR Interactions Using Voice Assistants Session 164, February 13, 2019 Yaa Kumah-Crystal, MD MPH Timothy Coffman](https://reader035.fdocuments.us/reader035/viewer/2022070923/5fbb8b4ffa3f7d6f340df4c5/html5/thumbnails/35.jpg)
35
System Usability Scale: Negatively Worded
Mean 1.67 (SD .66)
Strongly Disagree Strongly Agree
1.36 (SD .84)
1.71 (SD .83)
1.79 (SD 1.19)
1.64 (SD .74)
1.86 (SD 1.03)
![Page 36: Transforming EHR Interactions Using Voice Assistants...1 Transforming EHR Interactions Using Voice Assistants Session 164, February 13, 2019 Yaa Kumah-Crystal, MD MPH Timothy Coffman](https://reader035.fdocuments.us/reader035/viewer/2022070923/5fbb8b4ffa3f7d6f340df4c5/html5/thumbnails/36.jpg)
36
• Mean System Usability Scale score 80.7 (highly usable)
• Scores of >68 are considered above average
• Summary
• Relevant medications, other problems
• Weight
• Growth chart
• Blood pressure
• Percentile information
User Feedback
![Page 37: Transforming EHR Interactions Using Voice Assistants...1 Transforming EHR Interactions Using Voice Assistants Session 164, February 13, 2019 Yaa Kumah-Crystal, MD MPH Timothy Coffman](https://reader035.fdocuments.us/reader035/viewer/2022070923/5fbb8b4ffa3f7d6f340df4c5/html5/thumbnails/37.jpg)
37
Willingness to Use System
36%64%
NoYes
4 count9 count
2
1
1
![Page 38: Transforming EHR Interactions Using Voice Assistants...1 Transforming EHR Interactions Using Voice Assistants Session 164, February 13, 2019 Yaa Kumah-Crystal, MD MPH Timothy Coffman](https://reader035.fdocuments.us/reader035/viewer/2022070923/5fbb8b4ffa3f7d6f340df4c5/html5/thumbnails/38.jpg)
38
• Incorporation of User Feedback
• Desktop Time Comparison
• Scalability
• Alerts and Decision Support
Next Steps
![Page 39: Transforming EHR Interactions Using Voice Assistants...1 Transforming EHR Interactions Using Voice Assistants Session 164, February 13, 2019 Yaa Kumah-Crystal, MD MPH Timothy Coffman](https://reader035.fdocuments.us/reader035/viewer/2022070923/5fbb8b4ffa3f7d6f340df4c5/html5/thumbnails/39.jpg)
39
• Understand and measure provider frustration with EHR
• Create the business case for optimizing their experience and saving providers time within their EHR workflow
• Assemble a cross-functional team to build the overarching model taking into consideration the providers’ pain points, workflow, and information needs
• Build and iterate a voice assistant prototype while users test and provide feedback
• Understand information theory and map queries and content to satisfy users’ needs
Best Practices/Recommendations
![Page 40: Transforming EHR Interactions Using Voice Assistants...1 Transforming EHR Interactions Using Voice Assistants Session 164, February 13, 2019 Yaa Kumah-Crystal, MD MPH Timothy Coffman](https://reader035.fdocuments.us/reader035/viewer/2022070923/5fbb8b4ffa3f7d6f340df4c5/html5/thumbnails/40.jpg)
40
An Interesting Study About Voice
Nationwide Children's Hospital in Columbus, Ohio
![Page 41: Transforming EHR Interactions Using Voice Assistants...1 Transforming EHR Interactions Using Voice Assistants Session 164, February 13, 2019 Yaa Kumah-Crystal, MD MPH Timothy Coffman](https://reader035.fdocuments.us/reader035/viewer/2022070923/5fbb8b4ffa3f7d6f340df4c5/html5/thumbnails/41.jpg)
41
It looks like you have a question.
Yes Go away Clippy!
Questions
Please remember to complete the online session evaluation
![Page 42: Transforming EHR Interactions Using Voice Assistants...1 Transforming EHR Interactions Using Voice Assistants Session 164, February 13, 2019 Yaa Kumah-Crystal, MD MPH Timothy Coffman](https://reader035.fdocuments.us/reader035/viewer/2022070923/5fbb8b4ffa3f7d6f340df4c5/html5/thumbnails/42.jpg)
42
Contact Us