reactive… ”oops I forgot the manual” proactive…. …€¦ · · 2018-02-09Human Factors...

Copyright © 2015 BSI. All rights reserved.

Usability Where we’ve been…

reactive… ”oops I forgot the manual” Where are we going… proactive…. ”Understanding with minimal use of manual”

Usability: BSI Roadshow 2015 Richard Stein

3 Copyright © 2015 BSI. All rights reserved.

What is Usability?

• The ability for a human to interact easily and relatively error-free with a system, product or procedure

• What is Human Factors? A synonym for usability engineering

• Assume the system or product is a medical device for this presentation

08/12/2015


Users

08/12/2015

Who* Where*

Patients On the go Labs

Caregiver Home Clinics

Clinicians (doc, nurse, tech) Emergency transport In the field

EMTS ORs ERs

Hospital room Nursing home

*consider age, dexterity, education/training, context

This presentation doesn’t focus on the function


What types of devices may apply: implantable devices

Devices pictured are examples obtained from the Web, not an endorsement, nor assessed by BSI.

08/12/2015


What types of devices may apply: implantable devices


08/12/2015

small


What types of devices may apply: Robotic Surgery


08/12/2015

Users?


What types of devices may apply: CPAP

Devices pictured are examples obtained from the Web, not an endorsement, nor assessed by BSI. 08/12/2015


What types of devices may apply: Ventilator


08/12/2015


What types of devices may apply: Medical Apps with Sensors


08/12/2015


What types of devices may apply • CPAP

• nebulizers

• compression therapy

• EKGs

• Blood Pressure monitors

• Infusion pumps

• Implantable devices

• Catheters

• LVADs

• Surgical equipment (manual and powered)

• Endoscopes

• Apps, some with sensors

• Robotic surgery

• Ultrasound

• Ventilator

• Defibrillator

• Infant incubator

• Combination devices

• Software

• Expect sensors to facilitate the “human OS”

• More…….

15/09/2015


Why implement usability in Your medical device design? • Understandable • Useful device • Successful • Safer • Sales • Directives (EU) • Standards (IEC) • Guidance (FDA) • Serious Adverse Events, serious injury or death

08/12/2015


Why the concern? In the past:

• Often, there has not been enough time to correct the usability shortcomings in the medical device that was found in validation, so the manual was re-written.

• If manual was not enough to address shortcomings, then training was provided. • If training did not address the remainder of the shortcomings, then complaints may

have occurred which were addressed via instructions on the phone, etc. Now:

• It is expected to provide a safe product that the user understands, errors minimized Still:

• SAE’s have occurred, see MAUDE, some due to Use Errors • Complaints to the manufacturers are still occurring

15/09/2015

http://google2.fda.gov/search?as_sitesearch=www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfmaude/detail.cfm&q=advese+events+for+medical+devices&client=FDAgov&proxystylesheet=FDAgov&output=xml_no_dtd&site=FDAgov&requiredfields=-archive:Yes&sort=date:D:L:d1&filter=1&ie=UTF-8&ulang=en&&access=p&entqr=1&entqrm=0&wc=200&wc_mc=1&oe=UTF-8&ud=1


Some history

• Human Factors (Usability) first addressed by Peter Carstensen of FDA • Initiatives to make anesthesia equipment safer through application of human factors principles • Motivated the creation of an international committee to create standards

• ANSI/AAMI HE-48* Human Factors Engineering Guidelines and Preferred Practices for the Design of Medical Devices

o *Much of which adopted from MIL-STD-1472 Human Engineering

• ANSI/AAMI HE-74 Human factors design process for medical devices

• IEC 60601-1-6 Medical electrical equipment – Part 1-6: General requirements for basic safety and essential performance – Collateral standard: Usability

• IEC 62366:2007 Application of usability engineering to medical devices

• ANSI/AAMI HE-75 Human Factors Engineering – Design of Medical Devices (a Tutorial to HE-74)

• Today • IEC 62366-1:2015 Part 1: Application of usability engineering to medical devices

• Soon • IEC/TR 62366-2:2 Part 2: Guidance on the application of usability engineering to medical devices

08/12/2015


Injuries associated with anesthesia* “The anaesthetized patient is at risk of complications resulting from the actions, or inaction, of the anesthetist, from the actions of the surgeon, and from failure or malfunction of anesthetic equipment.” One of many of the root causes attributable to usability:

• Failure to check equipment, lack of vigilance, distraction, and carelessness were the most frequently associated human factors

• More see FDA CDRH, MAUDE Adverse Events Report

• Why?

*British Journal of Anesthesia Volume 95, Issue 1, pages 95-109

08/12/2015

http://google2.fda.gov/search?as_sitesearch=www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfmaude/detail.cfm&q=Injuries+associated+with+anesthesia&client=FDAgov&site=FDAgov&lr=&proxystylesheet=FDAgov&requiredfields=-archive:Yes&output=xml_no_dtd&getfields=*&ie=UTF-8&ulang=en&access=p&sort=date:D:L:d1&entqr=1&entqrm=0&wc=200&wc_mc=1&oe=UTF-8&ud=1&


Injuries associated with Infusion Pumps* • From 2005 through 2009, FDA received approximately 56,000 reports of Adverse Events associated

with the use of infusion pumps, including numerous injuries and deaths • These adverse event reports and device recalls have not been isolated to a specific manufacturer, type

of infusion pump, or use environment; rather, they have occurred across the board. Some examples: • Infusion pump screen confuses the user • Infusion pump screen.. Which units of measurement to enter.. Decimal point error

• pump labels…or...become damaged under routine use…labels worn • IFU or cues for mechanical set-up are not specific or clear enough • Infusion pump screen design is clunky or confusing to users, cause delays

• IFU are confusing, inadequate, outdated, or unavailable.

• In 2010 the FDA announced three steps it would take to improve infusion pump safety. These steps were to:

• (1) increase user awareness, • (2) proactively facilitate device improvements, and • (3) publish new guidance for industry

*See FDA CDRH Infusion Pumps

08/12/2015

http://www.fda.gov/MedicalDevices/ProductsandMedicalProcedures/GeneralHospitalDevicesandSupplies/InfusionPumps/


Examples of Different screens

Devices pictured are examples obtained from the Web, not an endorsement, nor assessed by BSI 08/12/2015


Why IEC 62366-1:2015 we already have 62366:2007?

• Created to be more “usable”, easier to understand than original 62366 • Original 62366 required an expert to understand • Implements practices by developers • Attempts to harmonize methods to meet FDA and international regulators • Contains the “what” requirements, the “how” is in 62366-2 • Closer ties to risk management, EN ISO 14971 • Hazard related use scenarios • No reference to usability goals, however goals may still have a need • Errors and/or shortcomings in the design to be addressed through risk evaluation • Reduced emphasis on functional analysis • Closer to FDA guidance • Legacy still addressed in Annex C (UOUP)

*Annexes are very helpful in implementation

08/12/2015


Major activities (shalls*) in IEC 62366-1:2015 • Usability Engineering activities shall be planned • To reduce risk do: safe design, protective measures, and/or information on safety

• Establish a usability engineering process • Documents Usability activities to a usability file • Use specification • ID user interface** characteristics • ID hazards & situations (foreseeable, the unforeseeable is gone) • ID hazard use scenarios for summative evaluation • User interface spec • Prepare user interface evaluation plan (summative & formative)

• Perform the design & summative evaluations

* Most of the requirements

**what the user looks at, uses

08/12/2015


What’s in the new EN 62366-2:2016?

This is the “how” document or “tutorial” • How safety relates to usability • Reasons to invest in usability • How to implement a usability program • Overall usability process • Prepare the use specification • Identify characteristics related to safety and potential use errors • Identify hazard-related use scenarios • Select hazard-related use scenarios for summative evaluation • Establish user interface specification • Establish user interface evaluation plan • Design/implement user interface • Perform formative evaluations • Perform summative evaluations

*note: the above is paraphrased by the author and the final version is not yet released

08/12/2015


Isn’t the usability process an addition to my development?

When you think about it, the activities are part of the development process

08/12/2015

Design process Usability process

Concept development Preliminary analysis

Design User interface design

Prototype development Formative evaluation

Verification & validation

Summative evaluation


Challenges

• Process to include usability • Conduct adequate formative evaluation • Conduct adequate summative evaluation • Address and reduce use risks • Just an IFU to address usability may no longer be enough

08/12/2015


Notified Body expectations of manufacturers

• Address usability in the Essential Requirements, in the Directives • Asking a couple of clinicians if they like the product is not enough……be careful of KOLs • Follow a process, ideally the harmonised standard or rationale, to meet ERs • Specifications should include the intended user, environment and procedure (as applicable) • Include usability in the requirements of the device/system and risk management • Verify the usability requirements of the device/system • Reduce usability risks as low as possible • Legacy products will need to be addressed, (See Annex 3)

Expectations are rising in Europe. For example: MHRA has task force on HF that is starting to look deeper into usability and how it can be improved to reduce use errors. A presumption is that the needs will require the NB’s to audit for usability.

FDA’s expectations have also increased.

8/12/2015


When does BSI apply EN62366-1:2015?

• Now? Auditors may use EN62366-1:2015 as a benchmark, BSI will use the Directives. • Auditors need training • Auditors will look for evidence of compliance • Legacy is addressed • If I do not have high risk, must I apply it? Remember the shalls and risk based in

standard • If EN 62366-2 (TR) is guidance, then must I apply it? What is in it for me?

08/12/2015


Case study: Abbott Symbiq™ IV Infusion System

Abbott’s Process: • Literature Review • User Analysis • Use Scenarios • “paper prototypes” use screens (GUI) • Formative evaluation

• Simulations & models • Design • Summative evaluation (some simulated)

08/12/2015

Case study supplied by usability consultant, not assessed by BSI


Case study: Intelliject Epicard

Consultant’s Process • Development motivated by needs, mistakes, complaints • Conduct preliminary evaluations to understand mistakes

• Observed on how used • Learned from use errors

• Create the mock-up design • Conduct formative evaluation with mock-ups


08/12/2015


Case study: Intelliject epicard Multiple concepts were evaluated in formative, resulted in refine design


08/12/2015


Case study: Intelliject epicard

• Final design then evaluated in summative evaluations (simulated, many users)


08/12/2015


Case study: Drug injection for Rheumatoid Arthritis

• Previous systems

• Difficult to use with hands • Time • Multiple steps • Mistakes • Know the user • Understand the context


08/12/2015



Consultant’s Process • Created the mock-up design • Conducted formative evaluation with mock-ups, simulated, many users • Created design based on learning from formative evaluation


08/12/2015



• Summative design, many users


08/12/2015


Case Study: Mobile Apps

• Mobile apps may be largest increasing segment in medical devices

Consultant’s Process • Formative evaluation, simulated, many users

• What was learned • Simplify • Minimize • Don’t repeat unnecessarily • Use regular “English” • Visually appealing • Show progress • Provide actionable choices • Encourage action

08/12/2015




08/12/2015



• Summative evaluation, not simulated, many users,


08/12/2015


Where can I learn more?

• AAMI HF Committee: • AAMI Website: for webinars and training • ANSI/AAMI HE-74 • ANSI/AAMI HE-75 • IEC 62366-1:2015 • IEC/TR 62366-2 • Handbook of Human Factors Medical Device Design, Matthew Weinger

et.al. ISBN 978-0-8058-5627-9

• FDA Guidance, http://www.fda.gov/medicaldevices/deviceregulationandguidance/guidancedocuments/ucm259748.htm

• The psychology of everyday things, Donald A. Norman ISBN 0-465-06709-3

• Set Phasers on stun, Steven Casey, ISBN 0-936178-8-5

• Usability testing of medical devices, Michael Wicklund et. al. ISBN 978-1-4398-1183-2

• The growing role of human factors and usability engineering for medical device, http://medicaldevices.bsigroup.com/en-GB/resources/Whitepapers-and-articles/

08/12/2015

http://www.fda.gov/medicaldevices/deviceregulationandguidance/guidancedocuments/ucm259748.htm

http://medicaldevices.bsigroup.com/en-GB/resources/Whitepapers-and-articles/


Conclusion

• Usability has become a vital part of the medical device development process • Usability activities should be conducted throughout all phases of the development

process, include it not a separate process • Usability should be part of the overall risk management process

08/12/2015

reactive… ”oops I forgot the manual” proactive…. …€¦ · · 2018-02-09Human Factors...

Documents

Transcript of reactive… ”oops I forgot the manual” proactive…. …€¦ · · 2018-02-09Human Factors...