Usability Evaluation of a Personal Health Record Noa Segall1, Jeffrey G. Saville2, Pete L’Engle2, Boyd Carlson2, Melanie C. Wright3, Kevin Schulman4, James E. Tcheng1,2 1Duke University Medical Center, Durham, NC 2Duke University Health System, Durham, NC 3St. Alphonsus Medical Center, Boise, ID 4Duke Clinical Research Institute, Durham, NC

Introduction Electronic personal health records (PHRs) can

facilitate patient-centered care Usability and utility to patients have received

little attention Outcome-based PHR studies

Mixed findings Little information to support design and

functionality decisions Used human-centered design methods to

evaluate a PHR

HealthView

Methods 20 participants 9 tasks Interview Surveys

Participant background

Usability Functionality

Measures Effectiveness Efficiency Satisfaction Qualitative

Tasks Finding lab results

Tasks Making an appointment

Tasks Making an appointment

Questions Interview Questions

What do you think are the most useful features in HealthView that can improve the quality of care you receive?

If you could make one significant change to the HealthView website, what change would you make?

Would you consider opening a HealthView account? Why or why not?

Survey Questions How often do you use

the internet to look up information about medications?

How interested would you be in using the internet to make a doctor appointment?

Organization of information is… Confusing 1 2 3 4 5 Very Clear

Results 23 participants

Age 27 to 84 (average, 53) 48% males 70% Caucasian 78% had cardiovascular disease 74% had education/training beyond high

school 70% beginner/average computer skills 61% spent <10 hours/week using computers 61% had a HealthView account

How often do you use the internet for the following tasks?

How interested would you be in using the internet to do any of the following tasks?

Functionality: Interviews “I have high blood pressure, a batch of things… I try to take control. HealthView… helps me buy into my health, my wellbeing, better. So I feel really good about it.

“I think it's a very informative and useful website. I think it can help you a lot… it keeps a lot of information for you.

Functionality: Surveys

Usability: Surveys

Usability: Surveys

Usability: Observations Task Errors/Gave Up Help Requests Find and interpret 4 lab results 17.5% of labs 41.3% of labs Make an appointment and reschedule it

31.9% of subtasks 14.5% of subtasks

Describe how to take 7 medications 13.5% of medications

0.8% of medications

Interpret trends in vital signs data 22.4% of subtasks 12.2% of subtasks Determine allergy documentation 5.3% of subtasks 5.3% of subtasks Find and print a payment history 25% of subtasks 0% of subtasks Update personal information 11.9% of subtasks 5.1% of subtasks Add a child to the health record 5% of subtasks 15% of subtasks Find an introduction video 5% of subtasks 25% of subtasks

Usability: Interviews “The website is very good, it’s very informative… Once you get used to where you’re going… just like anything else, you have to learn it. But once you learn it, it’s a piece of cake.

“I received my own report back and was very, very confused by what everything meant. And while it said, no cause for alarm, you're OK, there were items in this report that I didn't know what they were.

“The intent is there, but the user interface needs a lot of work.

Design Recommendations Current Display

Normal value 14.6 g/dL [12-15.5] Abnormal value *16.7 g/dL [12-15.5]

Recommended Display

Normal value

Abnormal value 12 15.5

14.6

12 15.5

16.7

Design Recommendations Current Display Recommended Displays

Previous Next Back to Lab Reports

Design Recommendations Current Display Recommended Display

Appointments

Appointments

Schedule Appointment Request Appointment

Appointments >

New Appointment View/Change Appointments

Conclusions Study participants were pleased with

HealthView functionality Participants encountered many usability

problems Difficulty navigating the website Long task completion times Misleading interfaces leading to errors Unclear medical terminology

Conclusions Human-centered design and evaluation

Improved performance Increased system acceptance Improved usage and satisfaction

Employing HCD methods throughout the design process may have prevented many usability problems

Discrepancy between survey data and interview and observation data highlights need to apply multiple methods during evaluation