Considering the ‘Human Factor’ in Medical Device Procurement

Thomas LovedaySnr Manager, Organisational Development and Human Factors

October 2016

Presentation OverviewThis presentation will:• A brief review of why ‘usability’ is a core concern

in medical device procurement• How Human Factors provides a framework for

evaluating usability• Designing for usability OR evaluating usability• The usability evaluation processes/tools being

developed within HealthShare NSW

Clinical Engagement in Procurement• Healthcare staff are now formally involved in:

– Specification writing

– Ongoing contract management

– Desk top evaluation products

– Local site trials

3

Includes one usability item

Why is usability critical?

4

What is Human Factors?• Human Factors is the practice of studying and

designing systems so that they fit the peoplewithin them. It is also known as:

• User-Centred Design (UCD)

• Ergonomics (physical, cognitive, organisational)

The Human Factor in Health• Doctors

• Nurses

• Pharmacists

• Clerical staff

• Cleaners

• Patients

Human Factors• Physiological characteristics

– including vision, strength, flexibility, height

• Psychological characteristics– Including perception, memory, intuition, stress,

fatigue, values

• Proprioceptive characteristics– Including hand-eye coordination, spatial awareness,

balance

System Factors• Communication and interaction between

individuals, teams and services

• The formal structure and the informal culture of the organisation

• Interactions with equipment and physical environment

Change the System or the Human?• Hard (impossible) to change many human factors

– Universal; and/or

– Fixed (i.e. height, perception, memory)

• We can procure devices that accommodate the “human factor” and reduce the incidence of use error.

Comprehensive Device Usability Evaluation Process

10

(The Goal)

HF Specifications in Contracts• Request compliance with:

– AAMI/ANSI HE75:2009;

– ISO/IEC 62366-1:2015;

– ANSI/AAMI/ISO 14971:2007; AND/OR

– ISO 60601-1-8:2006.

• Additional HF specifications identified by:– HF Specialist

– Human Factors Advisory Committee

Heuristic Evaluation• A type of usability inspection method, where

3+ evaluators examine a device for consistency with usability principles

• Designed for interfaces, but has been adapted for devices (Zhang et al, 2004)

Zhang et al (2004) Heuristics1. Consistency and standards – Users should be informed about what is going on

with the system

2. Visibility of system state. Users should be informed about what is going on with the system.

3. Match between system and world – Clear conceptual link between controls and outcomes

4. Minimalist – Any extraneous information is a distraction and a slow-down.

5. Minimize memory load – Users should not have to memorize a lot of information to carry out tasks.

6. Informative feedback – Users should be given prompt and informative feedback about their actions.

7. Flexibility and efficiency – Allow users to create shortcuts to accelerate their performance (where appropriate).

Zhang et al (2004) Heuristics8. Good error messages – Device messages/alerts should be informative enough that

users can understand the nature of errors, learn from errors, and recover from errors.

9. Prevent errors – It is always better to design interfaces that prevent errors from happening.

10. Clear closure – Every task has a beginning and an end.

11. Reversible actions – Users should be allowed to recover from errors.

12. Use users’ language – The language should be always presented in a form understandable by the intended users.

13. Users in control – Do not give users that impression that they are controlled by the systems.

14. Help and documentation – Always provide help when needed.

Method• Evaluator completes key device tasks several

times to identify key user interactions

– For ‘complex’ devices ask an expert to help

• Compares each ‘interaction’ against usability principles (if they apply)

Heuristic Evaluation Cheap

Fast

Requires minimal training

Not specific

Subjective

No user input/involvement

Effectiveness dependent on HF expertise

Simulation-based usability testing

• Control risk;• Observe rare events;• Provide experimental control;• Incorporate qualitative AND

quantitative metrics.

17

Usability Testing can identifying impediments to “ease of use”

and “goal completion” by testing it on real users.

What happens in a usability test?

Courtesy of Dr Cate McIntosh

What are the results of a test?• Identify impediments to ease of use based on:

– Errors

– Time to completion

– Verbal statements during task

– Where users got “stuck”

– Etc

• There metrics provide a basis for comparing devices (comparative testing)

Usability testing Otherwise difficult to predict what a first-time user will experience

Provides more detailed information

As few as five ‘users’ can reveal >90% of design issues

Scenario development requires information gathering

Comparative testing is resource intensive

(2 facilitators x 5 sessions x No. of devices in a simulation suite)

Local site trials

Relatively resource light Real-world evidence May not encounter rare critical events Present risk to patients

Requires rigour and metrics for comparison of devices

Which devices?High-risk categories?

• Infusion Pumps

• PCA Pumps

• Syringe drivers

• Defibrillators

• Beds

The NSW Health Approach

23

(the reality)

Raj Ratwani quote“Summative testing requirements occur at the end of development of the product and any design flaws that are identified are unlikely to be addressed since the product has already been fully developed.”Testimony of Raj M. Ratwani, PhD to U.S. Senate Committee on Health, Education, Labor and Pensions

24

Vendor self-assessment

Vendor asked to identify user-centred design (UCD) methods used during development of device

2.1With which of the following Human Factors standard(s) does the device design and/or evaluation comply?

2.2Has the device or a similar device (see Note 2) ever undergone formal Human Factors assessment as part of the design process?

2.2.1Did the Human Factors Assessment include a review of prior assessments and studies of similar devices to identify and manage potential usability risks?

2.2.2Did the Human Factors Assessment include a 'Heuristic Evaluation' (i.e. a panel of experts reviews the device and judges its compliance with recognised usability principles)?

2.2.3Did the Human Factors Assessment include 'Usability Testing?'

Vendor self-assessment Feasible and sustainable

Increase pressure on manufacturers to adopt UCD practices

UCD processes may be superficial

May not capture unique local critical tasks.

26

Thank you

Questions

For further information:

Thomas.loveday@health.nsw.gov.au