Usability and Human Factors - Remote-Learner user’s interactions in their normal workplace ... •...
-
Upload
nguyenphuc -
Category
Documents
-
view
217 -
download
0
Transcript of Usability and Human Factors - Remote-Learner user’s interactions in their normal workplace ... •...
Usability and Human Factors
Electronic Health Records and Usability
Lecture c
This material (Comp15_Unit6c) was developed by Columbia University, funded by the Department of Health and Human
Services, Office of the National Coordinator for Health Information Technology under Award Number 1U24OC000003.
Electronic Health Records and Usability
Learning Objectives
2
• 1. Explain how user-centered design can enhance adoption of EHRs (Lecture c)
• 2. Discuss the role of usability testing, training and implementation of electronic health records (Lecture c)
• 3. Describe Web 2.0 and novel concepts in system design (Lecture c)
• 4. Identify potential methods of assessing and rating EHR usability when selecting an appropriate EHR system (HIMSS document) (Lecture c)
Health IT Workforce Curriculum
Version 3.0/Spring 2012
Usability and Human Factors
EHRs and Usability
Lecture c
Usability Challenges in Healthcare
3Health IT Workforce Curriculum
Version 3.0/Spring 2012
Usability and Human Factors
EHRs and Usability
Lecture c
Complex needs, time pressure
Vary from setting to setting, specialty
50 specialties; other professions• (OT, PT, pharmacists,
respiratory therapists etc.)
Team-based work
Clinician mobility, primary focus on patient (should
be)
Legal, ethical, errors, high-
stakes
Multiple standards,
evidence-based Medicine
Hard to get clinician feedback
Confidentiality makes in situ observation,
testing difficult
Cost of change (time, vendor,
consensus, cost)
Interruptions, time pressure,
institution policies
More issues
4Health IT Workforce Curriculum
Version 3.0/Spring 2012
Usability and Human Factors
EHRs and Usability
Lecture c
• Clauses mean institution liable
Hold harmless
• Clinician responsible even if no choice or access to guts of software
Learned intermediary
• 25% medication errors (2006) involved computer technology
• 82% of these CPOE/data entry
Patient Safety
Basic Minimums
5Health IT Workforce Curriculum
Version 3.0/Spring 2012
Usability and Human Factors
EHRs and Usability
Lecture c
Usability = Efficiency of use + usefulness + ease of use?
For EHRs, we want to:
• Minimize likelihood of user error
• Provide cognitive support – guide interaction to foster good work, no errors
Avoid:
• Errors of commission: e.g. wrong patient chart
• Errors of omission: e.g. fail to notice abnormal result
• Failure to complete task (due to interruption and no handling of interruption)
Evidence-based Usability/Cognitive Support
6Health IT Workforce Curriculum
Version 3.0/Spring 2012
Usability and Human Factors
EHRs and Usability
Lecture c
• Usability is not just screen or software
design
– Affected by workflow, time pressure, physical
space layout, lighting, policies for use, and
even user experience during implementation
• Design needs to be evidence based, and
evidence is available» Karsh, 2010
Evidence-based Usability/Cognitive Support (cont.)
7Health IT Workforce Curriculum
Version 3.0/Spring 2012
Usability and Human Factors
EHRs and Usability
Lecture c
Fact:
• Healthcare is complex, training will be required
Myths:
• No training needed with good usability
Evidence-based Usability/Cognitive Support (cont.)
8Health IT Workforce Curriculum
Version 3.0/Spring 2012
Usability and Human Factors
EHRs and Usability
Lecture c
Fact:
• Users not trained in science of usability and cognition
• What they want may be wrong, mis-specified, or inarticulate
Myth:
• User-centered design = give users what they want
Evidence-based Usability/Cognitive Support (cont.)
9Health IT Workforce Curriculum
Version 3.0/Spring 2012
Usability and Human Factors
EHRs and Usability
Lecture c
Myth:
• Health IT should integrate into workflow
Fact:
• Healthcare workflow is emergent
• what is needed is more flexible availability of data
Evidence-based Usability/Cognitive Support (cont.)
• Usability is not a fixed target
• Don’t think we have the answers
• Ongoing research is needed» Karsh, 2010
10Health IT Workforce Curriculum
Version 3.0/Spring 2012
Usability and Human Factors
EHRs and Usability
Lecture c
Evidence-based Design Exists: AHRQ Resources
Electronic Health Record Usability:
– Interface Design Considerations Recommended actions to support development
of an objective EHR usability evidence base, formative policies
– Evaluation and Use Case Framework literature and best practices regarding the
usability of EHRs, use cases for primary care IT evaluation
– Vendor Practices and Perspectives: current usability processes, practices, and
perspectives of certified EHR vendors
•Available at: http://healthit.ahrq.gov
Redish, et al., (2010).
11Health IT Workforce Curriculum
Version 3.0/Spring 2012
Usability and Human Factors
EHRs and Usability
Lecture c
Usability Testing
12Health IT Workforce Curriculum
Version 3.0/Spring 2012
Usability and Human Factors
EHRs and Usability
Lecture c
“In routine handling, the Sorrento feels responsive
in corners, with nicely weighted, quick steering… The gated zigzag shifter is
awkward to use”
“It posted a commendable speed through our
avoidance maneuver. Avoidance maneuver,
max. spd: 51.5 mph. 0 to 60 mph: 7.6 sec”
Two Voices in Consumer Reports Car Reviews
Subjective v. Objective
13Health IT Workforce Curriculum
Version 3.0/Spring 2012
Usability and Human Factors
EHRs and Usability
Lecture c
Subjectivist/Qualitative (“art criticism”)
• Not everything of importance can be quantified
• Differences of opinion are okay
• The value is in the “thick description”
• Rigorous methods exist (one is formal criticism)
Objectivist/Quantitative
• Believable knowledge derives from measurement of attributes that are inherent in objects
• All observers should agree on measurement results (inter-subjectivity)
• On what result is “better” (polarity)
• Accuracy
• Response time
• Time to identify / Time spent searching
• Eye gaze
Heuristic Evaluation
14Health IT Workforce Curriculum
Version 3.0/Spring 2012
Usability and Human Factors
EHRs and Usability
Lecture c
Expert(s) evaluate system according to heuristics (Norman)
• 1. Visibility of system status
• 2. Match between system and the real world
• 3. User control and freedom
• 4. Consistency and standards
• 5. Error prevention
• 6. Recognition rather than recall
• 7. Flexibility and efficiency of use
• 8. Aesthetic and minimalist design
• 9. Help users recognize, diagnose, and recover from errors
• 10. Help and documentation
10 main axes:
Focus Groups
15Health IT Workforce Curriculum
Version 3.0/Spring 2012
Usability and Human Factors
EHRs and Usability
Lecture c
• Find out information designers can use
• What should the system do?
• What are users worst problems?
• What is their workflow?
• How do they do the task now?
Use for formative evaluation
Use for summative evaluation (after system is built)
• Group similar users together
• Don’t put supervisors with staff (people should feel free to speak)
Method: get typical users to view and discuss system and related ideas
Focus Groups (cont.)
16Health IT Workforce Curriculum
Version 3.0/Spring 2012
Usability and Human Factors
EHRs and Usability
Lecture c
Prepare open-ended questions, not yes/no
Provide fixed time, food
• Clinicians are busy and highly paid
• Compensate appropriately, e.g. $100/hour
Compensate users
• Institutional Review Board
Get permission in writing
• e.g. participants will be anonymous in any publications, talks
Define privacy policy
Focus Groups (cont.)
17Health IT Workforce Curriculum
Version 3.0/Spring 2012
Usability and Human Factors
EHRs and Usability
Lecture c
Record meeting (two digital or tape recorders); obtain permission of subjects
Transcribe and code thematically
Usability TestingThink aloud, in-lab
• Subject (typical user) uses system in quiet office, software captures video of their screen actions and voice (and face, if desired)
• User is told to think aloud while using software for typical tasks
• Video is coded and analyzed for themes, action patterns, problems, time for tasks…
• Morae software is a common tool– http://www.morae.com
– Requires webcam with sound, can do remote testing
• Video can be edited for administrators, clients, decision-makers, programmers
18Health IT Workforce Curriculum
Version 3.0/Spring 2012
Usability and Human Factors
EHRs and Usability
Lecture c
Field Usability Testing
19Health IT Workforce Curriculum
Version 3.0/Spring 2012
Usability and Human Factors
EHRs and Usability
Lecture c
Examine user’s interactions in their normal workplace setting
• Important method to establish conditions of work, workflow
Field observation
• What are time constraints, interruptions, noise, information sharing with colleagues, information flow, information sources, needs, team members?
Answer questions such as
• Morae testing will be the main methods
Observation, log file analysis, user interviews, and perhaps remote
Problems not detected in a laboratory test likely to come to light after deployment
Monitor usage closely soon after deployment
Cognitive Walkthrough
• Method of inspecting software for
problems
• Have goal, sub-goal, actions taken,
system response, potential problems
• Classify problems:
20Health IT Workforce Curriculum
Version 3.0/Spring 2012
Usability and Human Factors
EHRs and Usability
Lecture c
1. Cosmetic
• Need not be fixed unless time available
2. Minor usability
• Low priority for fix
3. Major usability
• High priority
4. Usability catastrophe
• Imperative to fix
Scenes from a Walkthrough
Senathirajah & Bakken, (2008).
21Health IT Workforce Curriculum
Version 3.0/Spring 2012
Usability and Human Factors
EHRs and Usability
Lecture c
Context: Patient Presents with a medical problem to a Family Physician’sOffice
Task: Diagnostic Reasoning
Goal Structure and Action SequenceGoal: Represent chief complaintChief Complaint: Patient complains that she has been feeling pretty tired
for the past 6 months
Subgoal 1: Characterize Patient’s Observation of TirednessAction 1: Open Top-Level Category General ConditionSystem Response: Displays Findings Organized by CategoryPotential Problem: System has sluggish response
Subgoal 2: Enter Finding FatigueSubgoal 3: Locate FindingAction 2: Open Category/Select Finding "Fatigue"
Subgoal 4: Describe Severity/QualitySubgoal 5: Translate quality "Pretty Tired" into Quantity indicating
SeverityAction 3: Enter severity: Three +++Action 4: Selects Quality: Lack of energy
Subgoal 6: Indicate ChronologyAction 5: Selects chronology duration of 6 monthsSystem Response: Displays chronology in "years“Potential Problem: “years” is not appropriate unit for 6 months
The Special Case of CPOE
• Poor CPOE design can facilitate medical
error
– e.g. overdose, wrong patient given drug
• Quantitative and qualitative studies show
flaws leading to errors
• Many adverse drug events due to poor
interface
• Heavy cognitive demands22
Health IT Workforce Curriculum
Version 3.0/Spring 2012
Usability and Human Factors
EHRs and Usability
Lecture c
Koppel:EHR Interface flaws
• Can’t clearly identify the patient
• Can’t view all meds on single screen
• Log-in/log-out failures
• Extra steps required to ‘activate’ orders
• Automatic cancellation of pre-surgical orders
• Downtime delays
• Orders near midnight interpreted as ‘tomorrow’
• Cumbersome interface – charting difficult
23Health IT Workforce Curriculum
Version 3.0/Spring 2012
Usability and Human Factors
EHRs and Usability
Lecture c
CPOE-related Usability Problems
24Health IT Workforce Curriculum
Version 3.0/Spring 2012
Usability and Human Factors
EHRs and Usability
Lecture c
• Deep navigational structures requiring multiple clicks
• Too close screen elements -> errors in selections
• Same color for data entry fields and others
• Long drop-down menus requiring scrolling
• Documentation templates different from clinician cognitive model of ordering
• Use of string sensitive data fields for abbreviations
• Excessive alerts, alerts which do not appear at appropriate time (when clinician would look for that information)
• Excess screen density, or too many screens
• Obscure hierarchies of orders or order sets
Avoid:
CPOE Design Recommendations
25Health IT Workforce Curriculum
Version 3.0/Spring 2012
Usability and Human Factors
EHRs and Usability
Lecture c
View complete medication record on one screen, avoid scrolling/other screens/fragmentation
Avoid subtle differences in layout, forms, labels for large functional differences
Explicit time indications
Map interface to ordering workflow
Maximum 3 layers of screens
Use consistent terms, organize elements into logical groups, separated by space, alignment
Distinguish active and passive elements
Consistent, sparing color
Technology Effects
26Health IT Workforce Curriculum
Version 3.0/Spring 2012
Usability and Human Factors
EHRs and Usability
Lecture c
• How performance changes when one uses a system
Effects with technology
• Longer-term effects of technology on cognition, even when the technology is no longer being used
Effects of technology
Information Gathering Strategies
27Health IT Workforce Curriculum
Version 3.0/Spring 2012
Usability and Human Factors
EHRs and Usability
Lecture c
• Requests for information guided by clinician’s hypothesis independent of the screen displays
Hypothesis-driven strategy:
• Guided by the ordered sequence of information on the computer screenScreen-driven:
• Novice changed from hypothesis-driven strategies to screen-driven
With experience:
Study Results
• Paper-based records– Narrative form, with connected and linked text and
sentences
• EMRs– More info on patient’s past medical history, lifestyle, and
primary diagnosis
– Information entered in point form, not linked in narrative
– Followed structure and sequence of system
– Time course not adequately captured
• Post EMR paper-based record– Closely resembled EMR in structure and format
– No connecting narrative
– Limited info on time course
28Health IT Workforce Curriculum
Version 3.0/Spring 2012
Usability and Human Factors
EHRs and Usability
Lecture c
Web 2.0 and Modern Approaches
29Health IT Workforce Curriculum
Version 3.0/Spring 2012
Usability and Human Factors
EHRs and Usability
Lecture c
• Give the user more control
‘Web 2.0’ is a change in internet approaches
Both philosophical approaches & technical approaches
• e.g. Facebook
Social networking applications
• Obtain information or judgment from a large group of users
Crowd sourcing:
Web 2.0 and EHRs
• Facilitate user control, a better user
experience, new forms of interactive
information display, and social networking
• Address problems of clinician
collaboration, optimal design of EHRs,
flexibility to meet rapid change, and other
problems
30Health IT Workforce Curriculum
Version 3.0/Spring 2012
Usability and Human Factors
EHRs and Usability
Lecture c
Electronic Health Records and Usability Summary
• EHR usability is a complex area in which we do not yet
have standards, but best practices are being intensively
studied
• Usability is one of the most important factors affecting
adoption, satisfaction, and optimal use of EHRs
• Usability should be an important factor in selection and
deployment of a system
• In the next two years much research will be done in this
area; it is important to keep abreast of developments
31Health IT Workforce Curriculum
Version 3.0/Spring 2012
Usability and Human Factors
EHRs and Usability
Lecture c
Electronic Health Records and Usability
References – Lecture cReferences
1. Karsh, B.TB. (2010). Health IT Design and Usability: Myths and Realities. AHRQ/NIST EHR Usability Conference;
2010; Washington DC.
2. Koppel , R., Metlay, J.P., Cohen, A., Abaluck, B., et al. (2005). Role of computerized physician order entry
systems in facilitating medical errors. JAMA 293(10): 1197-1203.
3. Patel V, Kushniruk A. Cognitive and usability engineering methods for the evaluation of clinical information
systems. Journal of Biomedical Informatics. 2004;37(1):56-76.
4. Patel VL, Kushniruk, A.W., Yang, S., & Yale, J.F. . Impact of a computerized patient record system on medical data
collection, organization and reasoning. J of the American Medical Informatics Association. 2000;7(6):569-85.
5. Shabot M. Ten commandments for implementing clinical information systems. Proc (Bayl Univ Med Cent).
2004;17(3):265-9.
6. Staggers N, Mills ME. Nurse-Computer Interaction: Staff Performance Outcomes. Nursing Research.
1994;43(3):144-50.
7. Zhu X, Gold SA, Lai A, Hripcsak G, Cimino J, editors. Using Timeline Displays to Improve Medication
Reconciliation. 2009 International Conference on eHealth, Telemedicine, and Social Medicine; 2009.
8. Belden J. EHR usability: an illustrated guide. AHRQ/NIST EHR Usability Conference; 2010; Washington DC:
National Institute of Standards and Technology.
9. Friedman, C. (2010). Usability in Health IT: technical strategy, research, and implementation: summary workshop.
National Institute of Standards and Technology, U.S., Department of Commerce. Retrieved on September 4th,
2012 from http://www.nist.gov/itl/hit/upload/U-HIT_Workshop_Report_Publication_Version.pdf
10. Khajouei, R., Jaspers, MWM. CPOE System Design Aspects and Their Qualitative Effect on Usability. eHealth
Beyond the Horizon.
32Health IT Workforce Curriculum
Version 3.0/Spring 2012
Usability and Human Factors
EHRs and Usability
Lecture c
Electronic Health Records and Usability
References – Lecture cImages
Slide 11: Redish, J.G., Lowry, S.Z., Locke, G., Gallagher, P.D., Friedman, C. (2010). Usability in Health IT: technical
strategy, research, and implementation: summary workshop. National Institute of Standards and Technology, U.S.,
Department of Commerce. Retrieved on September 4th, 2012 from http://www.nist.gov/itl/hit/upload/U-
HIT_Workshop_Report_Publication_Version.pdf
Slide 21: Senathirajah Y, Bakken, S., editor. A User-Configurable EHR using Web 2.0 Approaches. AMIA Spring 2008;
2008; Phoenix, AZ: AMIA.
33Health IT Workforce Curriculum
Version 3.0/Spring 2012
Usability and Human Factors
EHRs and Usability
Lecture c