Usability and Human Factors Unit 5a Usability Evaluation Methods.
-
Upload
elvin-bennett -
Category
Documents
-
view
234 -
download
4
Transcript of Usability and Human Factors Unit 5a Usability Evaluation Methods.
Overview
• Usability
• Evaluation methods
• Interviews
• Surveys / questionnaires
• Usability inspection methods– Heuristic evaluation– Cognitive walkthrough
• Usability testing
Component 15/Unit 5aHealth IT Workforce Curriculum
Version 1.0/Fall 20102
Why Usability Matters: The Clinical Story
Component 15/Unit 5aHealth IT Workforce Curriculum
Version 1.0/Fall 20104
Why Usability Matters: The Story from Patients and Consumers
Component 15/Unit 5aHealth IT Workforce Curriculum
Version 1.0/Fall 20105
Usability Evaluation Methods
Component 15/Unit 5aHealth IT Workforce Curriculum
Version 1.0/Fall 20106
Interviews:Four Main Types
Component 15/Unit 5aHealth IT Workforce Curriculum
Version 1.0/Fall 20108
Interviews:Four Main Types (cont.)
Component 15/Unit 5aHealth IT Workforce Curriculum
Version 1.0/Fall 20109
Usability Interview Example
Component 15/Unit 5aHealth IT Workforce Curriculum
Version 1.0/Fall 201011
Usability Interview # 15 (Upstate 4)I: So I am going to ask you a few questions, if I may. I
may start by asking you your age?P: 68I: Have you lived in this area all your life?P: YesI: The greater Pulaski area, Syracuse region?P: I have been here 15 years, cross the road 30
years, and was born about 4 miles country side.I: Pretty close. What do you do?P: I worked as a toolmaker; I am retired now.I: What was your highest level of education?P: I got some college; I went to night school for a
while,I: Was this your first computer?P: YesI: Do you monitor your blood sugar daily?P: Yes.I: Prior to receiving this computer, did you have the
home monitoring kit?P: Yes. I have had monitor ever since I have had it.I: So you pretty well been monitoring it?P: Yes, for the last 8 years.
Profile: 68 years of age Male English Speaker Originally from Pulaski area Retired toolmaker Works part-time at saw sharpening shop Some college education System was his first computer, later
bought another one Lives out on the route, greater Pulaski
area Lives with wife Nice, comfortable house Has exercise machine in house Avid hunter Computer in bedroom-easy access, but
no chair. Keeps devices disconnected, keeps
them in a different room. Keeps diabetes diary of results and
values
Focus Groups• Commonly used method in marketing and social
science research• Typically 3 to 10 participants representative of the
sample population– For example: attending physicians, residents, nurses,
pharmacists & administrators• Discussion led by trained facilitator• Preset agenda to guide discussion• Sufficient flexibility to pursue emergent issues• Facilitator guides and prompts discussion
– Encourages less talkative participants to speak up– Discourages more verbose participants from dominating
Component 15/Unit 5aHealth IT Workforce Curriculum
Version 1.0/Fall 201012
Questionnaire / Surveys
• Common method for collecting user demographic data and soliciting opinions
• Can reach a rather large group of people
• Questions are typically closed, though open comments may be solicited
• Some use a mix of negative and positive questions– Require respondent to be more attentive
Component 15/Unit 5aHealth IT Workforce Curriculum
Version 1.0/Fall 201013
Rating Scales: Likert
• Mostly commonly used• Measure opinions, attitudes and beliefs• Widely used for evaluating user satisfaction• Statement representing a range of opinions
– I found the system to be very easy to use
• Rate response on a continuum from one extreme (e.g., highest or strongest agreement) to another (lowest or strongest disagreement)
Component 15/Unit 5aHealth IT Workforce Curriculum
Version 1.0/Fall 201014
Sample Usability Likert Questions
Component 15/Unit 5aHealth IT Workforce Curriculum
Version 1.0/Fall 201015
Rating Scales: Semantic Differential
Component 15/Unit 5aHealth IT Workforce Curriculum
Version 1.0/Fall 201016
Online Surveys:Two Types
Component 15/Unit 5a 17Health IT Workforce Curriculum
Version 1.0/Fall 2010
Survey Monkey• http://www.surveymonkey
.com• Illustration of online
survey• Focuses on user
experience• Different forms of
interaction– Check boxes for multiple
selection– Radio buttons for choosing
among alternatives or rating items on Likert scale
• Configurable and flexible
Component 15/Unit 5aHealth IT Workforce Curriculum
Version 1.0/Fall 201018
Usability Inspection Methods:Expert Reviews
Component 15/Unit 5aHealth IT Workforce Curriculum
Version 1.0/Fall 201019
Heuristic Evaluation• Developed by Jakob Nielsen and Rolf Molich• Method for structuring the critique of a system using
simple & general heuristics• Several analysts critique a system to identify &
diagnose usability problems• Display based• Each evaluator assesses the system & notes
violations of usability principles• In a second pass, analyst rates severity of problems• Can be used in all phases of design including
storyboards, prototypes and fully functioning systems
Component 15/Unit 5aHealth IT Workforce Curriculum
Version 1.0/Fall 201020
Usability Principles (Nielsen, 1993)
• Visibility of system status• Match between system and the real world• User control and freedom• Consistency and standards• Help users recognize, diagnose, and recover
from errors• Minimize memory load
– Emphasize recognition rather than recall• Flexibility and efficiency• Motivation and engagement
Component 15/Unit 5aHealth IT Workforce Curriculum
Version 1.0/Fall 201021
More Specific Heuristics (Gerhardt-Powals, 1996)
• Automate unwanted workload: – Free cognitive resources for high-level tasks – Eliminate mental calculations, estimations,
comparisons, & thinking
• Reduce uncertainty; display data in a clear and obvious manner
• Reduce cognitive load by bringing together lower level data into a higher-level summation
• Present new information with meaningful aids to interpretation: – Use a familiar framework, everyday terms, metaphors
Component 15/Unit 5aHealth IT Workforce Curriculum
Version 1.0/Fall 201022
More Specific Heuristics (cont.)
• Use names that are conceptually related to function – Context-dependent– Attempt to improve recall and recognition
• Group data consistently meaningful ways to decrease search time
• Limit data-driven tasks: – Reduce time assimilating raw data, appropriate
use of color & graphics• Include only information needed by the user
at a given time
Component 15/Unit 5aHealth IT Workforce Curriculum
Version 1.0/Fall 201023
Heuristic EvaluationProblem Description, Recommended Solution & Commentary Heuristics Violated
P. Description is clear, key points should be more visually prominent.S. Use bullets or bolded Text
Consistency Standards
P. Picture is extraneous and eats up valuable space. S. Summary positioned above the picture.
Minimalism
P. An excess of textS. Convey points more succinctly, use bullets and eliminate sentences
Minimalism
P. User is not fully aware of scope of session and perhaps programS. Perhaps an advance organizer and a timeline would be helpful
Visibility
P. Can’t return to session from Discussion by clicking back arrow. Control/Flexibility
P. Change Program is a misleading label for editing preferences. Match
P. Help is not sufficiently informative and context sensitive.S. Useful if system provides some context-sensitive guidance.
Help
P. Can’t return to English after switching to Spanish Reversible action/Control
Component 15/Unit 5aHealth IT Workforce Curriculum
Version 1.0/Fall 201024