HIMSS EHR Usability Task Force Report on Defining and Testing EMR Usability
-
Upload
shahid-n-shah -
Category
Documents
-
view
219 -
download
0
Transcript of HIMSS EHR Usability Task Force Report on Defining and Testing EMR Usability
-
8/14/2019 HIMSS EHR Usability Task Force Report on Defining and Testing EMR Usability
1/42
2009HealthcareInformationandManagementSystemsSociety(HIMSS)
DefiningandTestingEMR
Usability:
Pr inc ip les and Proposed Met hodsof EMR Usabi l i t y Evaluat ion and
Rat ing
HIMSSEHRUsabilityTaskForce
June2009
-
8/14/2019 HIMSS EHR Usability Task Force Report on Defining and Testing EMR Usability
2/42
2009HealthcareInformationandManagementSystemsSociety(HIMSS)
CONTENTS
EXECUTIVESUMMARY.......................................................................................................................................1
INTRODUCTION..................................................................................................................................................2
WHATISUSABILITY?.......................................................................................................................................................3
THECHALLENGESOFEMRDESIGN....................................................................................................................................
3
SCOPEOFTHISREPORT....................................................................................................................................................4
USABILITYPRINCIPLES........................................................................................................................................4
SIMPLICITY....................................................................................................................................................................5
NATURALNESS...............................................................................................................................................................6
CONSISTENCY................................................................................................................................................................6
MINIMIZINGCOGNITIVELOAD..........................................................................................................................................6
EFFICIENTINTERACTIONS.................................................................................................................................................7
FORGIVENESSANDFEEDBACK...........................................................................................................................................7
EFFECTIVEUSEOFLANGUAGE...........................................................................................................................................7
EFFECTIVEINFORMATIONPRESENTATION............................................................................................................................
8
PRESERVATIONOFCONTEXT...........................................................................................................................................10
USABILITYEVALUATIONANDRATINGMETHODS...............................................................................................10
EVALUATIONMETHODSANDMETRICS.............................................................................................................................12
SELECTINGTASKSFOREVALUATION.................................................................................................................................17
5STARUSABILITYRATINGSYSTEM..................................................................................................................................18
CERTIFICATIONANDEMRUSABILITYRATING....................................................................................................................18
RECOMMENDATIONS........................................................................................................................................19
REFERENCES......................................................................................................................................................20
CONTRIBUTORS................................................................................................................................................24
AUTHORS...................................................................................................................................................................24
USABILITYPRINCIPLESWORKGROUP...............................................................................................................................24
EDITORS.....................................................................................................................................................................25
HIMSSEHRUSABILITYTASKFORCE................................................................................................................................25
APPENDICES......................................................................................................................................................27
A. TESTTASKANDSCENARIOEXAMPLES.......................................................................................................................27
B. BENCHMARKEXAMPLES........................................................................................................................................35
C. PATIENTSAFETYCHECKLISTEXAMPLES.....................................................................................................................36
D.
USABILITYPRINCIPLESWORKGROUPBIOS...............................................................................................................
38
-
8/14/2019 HIMSS EHR Usability Task Force Report on Defining and Testing EMR Usability
3/42
2009HealthcareInformationandManagementSystemsSociety(HIMSS)
EXECUTIVE SUMMARY
Electronicmedicalrecord(EMR)adoptionrateshavebeenslowerthanexpectedintheUnitedStates,
especiallyincomparisontootherindustrysectorsandotherdevelopedcountries.Akeyreason,aside
frominitialcostsandlostproductivityduringEMRimplementation,islackofefficiencyandusabilityof
EMRscurrentlyavailable.AchievingthehealthcarereformgoalsofbroadEMRadoptionand
meaningfulusewillrequirethatefficiencyandusabilitybeeffectivelyaddressedatafundamental
level.
Weconductedaliteraturereviewofusabilityprinciples,especiallythoseapplicabletoEMRs.Thekey
principlesidentifiedweresimplicity,naturalness,consistency,minimizingcognitiveload,efficient
interactions,forgivenessandfeedback,effectiveuseoflanguage,effectiveinformationpresentation,
andpreservationofcontext.
Usabilityisoftenmistakenlyequatedwithusersatisfaction,whichisanoversimplification.Wedescribe
methodsof
usability
evaluation,
offering
several
alternative
methods
for
measuring
efficiency
and
effectiveness,includingpatientsafety.Weprovidesamplesofobjective,repeatableandcostefficient
testscenariosapplicabletoevaluatingEMRusabilityasanadjuncttocertification,andwediscussrating
schemaforscoringtheresults.
1
-
8/14/2019 HIMSS EHR Usability Task Force Report on Defining and Testing EMR Usability
4/42
2009HealthcareInformationandManagementSystemsSociety(HIMSS)
INTRODUCTION
TherateatwhichEMRshavebeenadoptedinclinicandhospitalsettingswithintheUnitedStateshas
laggedbehindtheadoptionofinformationtechnologythathasbeencommoninotherindustriesfor
morethan20years.19
Multiplecauseshavebeensuggestedincludingcost,resistancetochange,fearoravoidanceof
technology,andingrainedpatternsofbehavior.Increasingly,however,usabilityhasbeenacknowledged
asadeterrenttoadoption40,51,35,23,andonethatmustbeaddressed.
Wesubmitthatusabilityisoneofthemajorfactorspossiblythe
mostimportantfactorhinderingwidespreadadoptionofEMRs.
Usabilityhasastrong,oftendirectrelationshipwithclinical
productivity,errorrate,userfatigueandusersatisfactioncritical
factorsforEMRadoption.Cliniciansloseproductivityduringthe
trainingdays
and
for
months
afterward
as
they
adapt
to
the
new
tools
andworkflow.Someproductivitylossesaresustained,mostlydueto
longertimeneededforencounterdocumentationincomplex
patients31.
Effectivetrainingandimplementationmethodsaffectuseradoption
ratesaswell,buttrainingisbothharderandmorecostly,andimplementationismorecomplexand
difficultwhenusabilityislacking.
IthasproveddifficultforclinicianstoevaluateEMRusabilityaspartofthepurchaseprocessforseveral
reasons.Properevaluationbypurchasersrequiresindepthstudyusingunfamiliarskills.Mostusersof
oneEMR
often
have
not
experienced
other
EMRs,
so
single
product
ratings
are
less
helpful
than
those
whichcomparesystems.9Therehasbeenworkdonebythirdpartyconsultinggroupstosurveycurrent
users,buttheseresultsarerarelyprovideddirectlybyclinicalendusers.Industrysurveyinstrumentsare
generallynotconstructed toprovidereliableusabilitydata;theyalsoonlyprovideusersatisfaction
ratingsasinglecomponentofusability.
Thispaperwill:
1. DescribeanddefineusabilityasitpertainstotheEMR.
2. Identifyasetofwellestablishedprinciplesofusabilityanddesign.
3. Offer
potential
methods
of
assessing
and
rating
EMR
usability.
Wesubmitthattheseprinciplesandmethodscouldbeusedbycertificationorganizationstotestand
rateproductsforusability.Requiringthisadjuncttocertificationmayspurdevelopmentofmoreusable
EMRproducts,andallowdecisionmakersmoreconfidenceinchoosingaproductthatwillbenefit
clinicians.
InBrief:Adoptionratesby
physiciansandhospitalshavebeen
slowerthanexpectedintheUS,in
part,duetopoorefficiencyand
usability.Weexplorewell
establishedusability
principles
and
testingmethodsandpropose
methodstotestandrateEMRsfor
efficiency,effectivenessandsafety.
2
-
8/14/2019 HIMSS EHR Usability Task Force Report on Defining and Testing EMR Usability
5/42
2009HealthcareInformationandManagementSystemsSociety(HIMSS)
What is Usabi l i t y?
Usabilityistheeffectiveness,efficiencyandsatisfactionwithwhichspecificuserscanachieveaspecificsetoftasksinaparticularenvironment.39Inessence,asystemwithgoodusabilityiseasytouseandeffective.Itisintuitive,forgivingofmistakesandallowsonetoperformnecessarytasksquickly,
efficientlyandwithaminimumofmentaleffort.Taskswhichcanbeperformedbythesoftware(suchas
dataretrieval,organization,summary,crosschecking,calculating,etc.)aredoneinthebackground,
improvingaccuracyandfreeinguptheuserscognitiveresourcesforothertasks.
Usabilityevaluationisfarbroaderthanthesimpleprocessofmeasuringusersatisfaction.Justas
importantly,usabilitymetricsincludemeasuresofefficiency,effectiveness,cognitiveloadandeaseof
learning.Usabilityemergesfromunderstandingtheneedsoftheusers,usingestablishedmethodsof
iterativedesign,andperformingappropriateusertestingwhenneeded.Thereareawiderangeof
designandevaluationmethodologies,bothsubjectiveandobjective,whicharecontinuallygrowingin
sophistication.BuiltinwebcamsonmodernlaptopPCs,robustwirelessnetworking,remotetesting
software,andcompact,inexpensivevideorecordersmakeitincreasinglyeasiertotestinliveclinical
settings.
The Challeng es of EMR Design
ItisparticularlychallengingtodevelopexcellentusabilityinEMRsystems.Thereisawiderangeof
complexinformationneeds,whichvaryfromsettingtosetting,amongdifferentadministrative,financial
andcliniciangroups,andfromtasktotaskwithinagroup.Thereareover50physicianspecialties(AMA
specialtycodes2)eachwithitsownsoftwareneeds,aswellasthesoftwareneedsofotherclinical
groupssuchasnurses,pharmacists,physicaltherapists,respiratorytherapists,medicaldieticiansand
others.Eachdisciplinemayhaveseveraldifferenttaskscenariosinaworkingday,witheachscenario
demandingadifferentsoftwareinterfacedesign.
Cliniciansareoftenmobile,goingfromroomtoroom,hospitaltoclinic.Theyseldomgivetheirfull
attentiontothesoftware.Theirprimaryfocusshouldbeonthepatient,andcliniciansareoftentalking,
listeningorthinkingwhileusingthesoftware.Theyoftenhaveafrequentlychangingagendaduringa
singlepatientworkflow,andinterruptionsarecommon.
Administrativeandfinancialissuescomplicateevenroutinetasks(providingbillingcodes,discovering
drugformularycoverage,pursuingpriorauthorization)andvarywidelywithdifferentinsurers.Thereisa
burgeoningimpetustomeasurequalityofcare,complicatedbymultiplestandards.
ItmaybechallengingforEMRdeveloperstogetaccesstoclinicianusersforfeedbackortesting.Busy
physiciansallow
only
limited
access
for
user
centered
design
work.
Clinicians
have
other
significant
constraintsthatcomplicateusabilityevaluations,suchasconfidentialityconcernsinalltheirencounters,
theneedtotestintheactualworkenvironment,andfrequentinterruptionsintheirworkflow.
3
-
8/14/2019 HIMSS EHR Usability Task Force Report on Defining and Testing EMR Usability
6/42
2009HealthcareInformationandManagementSystemsSociety(HIMSS)
Scope of th is Report
WediscusstheusabilityoftheEMRfromtheperspectiveofclinicianusers(physicians,nurses,
pharmacists,physicaltherapists,respiratorytherapistsandothers)intheambulatory,inpatientand
acutecareenvironments.Weconfinedourselvestoissuesofusercentereddesignandusability
evaluation.Theseconceptsapplytovendorproductdevelopmentprocesses,publicproductusability
ratingmethodsandvendorselectioncriteriaforhealthcareorganizations.Inaddition,thesemethods
shouldbeappliedintheconfigurationofhighlyadaptablesystemsduringimplementation.Wedonot
otherwiseaddressconcernsofimplementation,usertrainingorchangemanagement,thoughthese
issuesdoaffectuseradoptionsuccessrates.
Terminologynote:AnEMRisacomputersystemcomposedofmultiple,integratedapplicationsenabling
clinicianstoorder,documentandstorepatientinformation.Thetermelectronichealthrecord(EHR)is
sometimes,andincorrectly,usedinterchangeably.Incontrast,anEHRispatienthealthinformationfrom
multiplecaredeliveryorganizationsEMRs,comprisingapatientcentric,longitudinalviewofapatients
encounterswithhealthcareproviders.13Forthepurposeofthispaper,thetermEMRwillbeused,aswe
areaddressing
systems
vs.
data.
USABILITY PRINCIPLES
Inrecentyears,usabilityhasbecomeanincreasinglyprevalenttopicinthehealthinformation
technology(HIT)literatureandmedia.ManyHITprofessionals,healthcareinformaticiansand
researchershaveclearlyarticulateddesignproblemsinthecurrentgenerationofclinical
applications.40,20,8,50TheNationalResearchCouncil(NRC)hasassertedthattodaysclinicalsystems
provide
poor
support
for
the
cognitive
tasks
and
workflow
of
clinicians.
28
These
problems
can
dramaticallyimpactuseracceptanceandproductivity.
Patientsafetyisaprominentconcernintheliterature.TheJoint
Commission(formallyknownasJointCommissiononAccreditationof
HealthcareOrganizations)recentlyissuedSentinelEventAlert42
regardingtechnologyrelatedadverseevents(TheJointCommission,
2008).ThissafetyalertincludedEMRs,computerphysicianorderentry
(CPOE)andclinicaldecisionsupport(CDS)systems.Theyreportedthat
approximately25percentofmedicationerrorsincludedinthe2006
PharmacopeiaMEDMARXinvolvedcomputertechnologyasacontributing
cause.Theoverwhelmingmajorityofthese(82percent)stemmedfrom
CPOEandotherdataentryfunctions.Manystudieshavedocumentedthe
issuesofalertfatigue,screenfragmentation,terminologyconfusionandlackofappropriatedefaultsin
CPOEandCDSsystems.8,6,21
InBrief:Wellknownprinciples
suchassimplicity,naturalness,
consistency,protectionagainst
cognitiveoverloadandothers,
definegoodusability.
Incorporatedeepknowledgeof
theuserstasks,contextand
workflows.
4
-
8/14/2019 HIMSS EHR Usability Task Force Report on Defining and Testing EMR Usability
7/42
2009HealthcareInformationandManagementSystemsSociety(HIMSS)
Notallauthorsdiscusstheseissuesdirectlyintermsofusability.Farfewerpresenttheseproblemsin
termsoftheunderlyingdesignprinciplesbeingviolated.Itisthislevelthatmustbeaddressedinorder
todesignapplicationsthatwillachievedesiredefficiency,broadusage(aprerequisiteformeaningful
use)andavertsystemfacilitatedsafetyerrors.
Expertsin
usability
and
Human
Factors
have
published
many
compilations
of
principles
and
guidelines
to
aidindesigningthemosteffectiveuserinterfaces.30,32,46,49Mostoftheselistssharecertaincoreideas.
Principlesmustbeevaluatedfortheirimportancetotheparticularcontextofuse.Thosethatareofkey
importancetothedesignofanEMRsystemarediscussedbelow.Theseprincipleswereselectedfor
discussionbasedontheircontributiontotwoessentialfactorsforclinicianacceptanceandsystem
success:
1)Efficiencyofuse.
2)Minimizinglikelihoodofusererror.
Usererrorshaveadirectrelationshiptopotentialpatientsafety.Usererrorsmaybeeithererrorsof
commissionor
errors
of
omission:18
Example,errorsofcommission:
o Selectingthewrongpatient,wrongmedication,wrongdosageorwrongencounter.
Example,errorsofomission:
o Overlookingormisinterpretingkeydataduetopoorinformationdisplay(e.g.,overlooking
criticallyabnormallabresult,orroutinelydismissingacriticallyharmfuldrugdrug
interactionwarning).
o Failingtocompleteatask(perhapsduetointerruption)suchastransmittingordersor
signingdocumentation.
Testingmethodswhichmeasureefficiency,effectiveness,easeoflearningandusersatisfactionhave
beendevelopedtotaketheseusabilityprinciplesintoaccount.Tousethemethodsproperly,the
principlesbehindthemshouldbewellunderstood.
Simpl ic i ty
Simplicityindesignreferstoeverythingfromlackofvisualclutterandconciseinformationdisplayto
inclusionofonlyfunctionalitythatisneededtoeffectivelyaccomplishtasks.Alessismorephilosophy
isappropriate,withemphasisbeinggiventoinformationneededfordecisionmaking.29,26Themore
complexan
application,
the
more
important
this
principle
becomes.
Clinical
systems
are
complex
as
well
asinformationdenseitisessentialforefficiencyaswellaspatientsafetythatdisplaysareeasytoread,
thatimportantinformationstandsout,andthatfunctionoptionsarestraightforward.Simplicityasa
principleshouldnotbeinterpretedassimple.Clear,cleanscreendesignrequiressubstantiallymore
effortthancluttereddisplays;italsomaymeanthatsomecomplexityhasbeenremovedfromthe
5
-
8/14/2019 HIMSS EHR Usability Task Force Report on Defining and Testing EMR Usability
8/42
2009HealthcareInformationandManagementSystemsSociety(HIMSS)
surfaceandmovedunderthehood.Simplicityappliestoanydesignregardlessoftheexperiencelevel
ofthetargetuser.
Natura lness
Naturalnessreferstohowautomaticallyfamiliarandeasytousetheapplicationfeelstotheuser.
Factorsthat
contribute
to
this
feeling
include
terminology
used
in
the
interface
and
how
well
the
design
andscreenflowsmaptotheuserstasksandexpectations.30Thisiscriticaltoclinicalapplicationssinceit
isextremelydifficulttoprovideextensivetraining,especiallytoclinicianswhosetimeislimitedand
fragmented.Goodworkflowdesigncancontributesignificantlytoefficiencyandreducecognitiveload.
Naturalworkflowcanvarydramaticallyfromonespecialtytoanotherorinanacutesetting,fromone
departmenttoanother.AnEmergencyDepartmentsworkflowisverydifferentfromthatofaninpatient
medicalsurgicalunit.Likesimplicity,naturalnessalsocontributestoerrorreduction.
Cons is tency
Externalandinternalconsistencyareimportanttothedesignofanyapplication.Externalconsistency
primarilyhas
to
do
with
how
much
an
applications
structure,
interactions
and
behaviors
match
ausers
experiencewithothersoftwareapplications.Themoreausercanapplypriorexperiencetoanew
system,thelowerthelearningcurve,themoreeffectivetheirusage,andthefewertheirerrors.An
internallyconsistentapplicationusesconcepts,behavior,appearanceandlayoutconsistently
throughout.30,46,49Predictabilityisanotherimportantfactorinenablingefficientuseandreducingerrors.
Minim iz ing Cogni t ive Load
Whilethisprinciplemaysoundabitesoteric,itisessentialforacomplex,informationdenseclinical
application.Clinicians,inparticular,arealmostalwaysperformingundersignificanttimepressureandin
environmentsburstingwithmultipledemandsfortheirattention.Combinedwiththestaggering
informationload
faced
by
todays
providers,
this
can
be
arecipe
for
cognitive
overload,
which
could
negativelyimpactpatientsafety.
Presentingalltheinformationneededforthetaskathandreducescognitiveload.Forexample,when
reviewingresultsofalipidprofile,theproviderwillwanttoseethepatientslatestandpriorresults,the
medicationlist,theproblemlistandallergylistallinthesamevisualfieldsothatdecisionsand
subsequentactionsmaybeperformedwithoutchangingscreens.Displayinginformationorganizedby
meaningfulrelationshipsisonemethodofprovidingcognitivesupporttotheuser.50,28
AnEMRmustnotonlyassistwithtaskperformanceanddecisionmaking,butstrivefortransparency.
Indesignterms,transparencymeansthatuseofthesoftwareapplicationdoesnotcreatetoomany
intrusivethoughtsfortheuserlikeHowdoI?,Whatdoesthisdo?orWhereis...?These
mentalinterruptionscancausetheusertolosetheirthoughtprocessaboutthetaskordecisionmaking
processinwhichtheyareengaged.Inotherwords,theusershouldnothavetothinktoomuchabout
theapplicationitself.22
6
-
8/14/2019 HIMSS EHR Usability Task Force Report on Defining and Testing EMR Usability
9/42
2009HealthcareInformationandManagementSystemsSociety(HIMSS)
Cognitiveloadisincreasedbyanyaspectsofadesignthatdonotfollowtheprinciplesofsimplicity,
naturalnessandconsistency.Itisalsoincreasedifauserisrequiredtorelyonmemory(recall)rather
thanvisualrecognition,ifausermusttrytorememberinformationfromonescreentoanother,whata
buttonreallydoes,orwhatnamesomethingiscalledasinanorderableslist.Highinformationdensity,
poorfeedbacktotheuserandinadequatecuesfordataentryfieldsalsoaffectcognitiveload.
Ef f ic ient In terac t ions
Oneofthemostdirectwaystofacilitateefficientuserinteractionsistominimizethenumberofstepsit
takestocompletetasksandtoprovideshortcutsforusebyfrequentand/orexperiencedusers.46While
thisissomewhatstatingtheobvious,itisincludedherebecauseofitsimportancetotheuser
acceptanceofaclinicalapplication.Otherexamplesofdesigningforefficientinteractionsincludeauto
tabbing;gooddefaultvalues;largeenoughlistandtextboxestolimitscrolling;andpreventingtheneed
forfrequentswitchingbetweenkeyboardandmouse.49Somewhatlessobviousfactorsincludeattention
tominimizingtheamountofvisualsearchingrequiredtolocateinformationandthedistancethecursor
musttraveltomakeselections.Excessivecursormovementandvisualscanningbothcontributetouser
fatigueand
frustration.
Forgiveness and Feedback
Forgivenessmeansthatadesignallowstheusertodiscoveritthroughexplorationwithoutfearof
disastrousresults.Thisapproachaccelerateslearningwhilebuildinginprotectionsagainstunintended
consequences.30,46Thisisespeciallyhelpfuliftrainingislimited.Goodfeedbacktotheusersupportsthis
goalbyinformingthemabouttheeffectsoftheactionstheyareabouttotake.Campbelletal.8provide
ananalysisofthetypesofunintendedconsequencesrelatedtoCPOE.Forgivenessandfeedbackwork
togethertoreduceusererrorsandprovidegracefulrecoverywhenmistakesaremade.Goodfeedback
alsoreassurestheuserthattheiractionshavehadthedesiredeffect.Likeconsistency,theseprinciples
arestandard
in
the
design
of
any
application,
but
of
special
importance
in
aclinical
information
system
duetotheimpacttheycanhaveonusererrorsaswellascognitiveload.
Effect ive Use of Language
AlllanguageusedinanEMRshouldbeconciseandunambiguous.Terminologyusedalsomustbethat
whichisfamiliarandmeaningfultotheendusersinthecontextoftheirwork;notermsrelatedto
computers,technology,HL7,databases,etc.shouldappearintheuserinterface.Thisappliesto
everything:labels,descriptions,picklistsanderrormessages.
Textshouldneverbedisplayedinalluppercase;thisisconsideredshouting.Itismoredifficultand
takeslonger
to
read,
and
increases
perceived
density.
Even
iflists
of
orderables
or
terms
are
received
by
theEMRinuppercase,theyshouldbetranslatedtotitlecasebeforedisplayintheinterface.Rare
exceptionsincludeoneortwowordmessagesthatareintendedtodrawtheattentionoftheuser.26,49,38
Abbreviationsandacronymsshouldonlybedisplayedwhentheyarecommonlyunderstoodand
unambiguous.49,50Informationthatmustbespelledoutbuttakesmorespacethanavailableshouldhave
ellipsesinsertedtoindicatethereismorewiththefulltextavailableonmouseover.Thisisinparta
7
-
8/14/2019 HIMSS EHR Usability Task Force Report on Defining and Testing EMR Usability
10/42
2009HealthcareInformationandManagementSystemsSociety(HIMSS)
patientsafetyissue.Forefficiency,however,alargernumberofcommonabbreviationsandsynonyms
shouldbeavailabletotheuserforthepurposesofdataentryandsearching,expandingifnecessaryfor
display.
AlanguageissuespecifictoEMRdesignistheneedtocapturestructured(discrete)clinicaltermsfrom
providerdocumentation
such
as
visit
notes,
allergies
and
problem
and
medication
lists.
This
data
is
used
toidentifyclinicalrelationshipsinpatientrecords,drivedecisionsupportfunctions,eliminateredundant
dataentryandsupplycodeddataelementstoadministrativeandreportingfunctions.Meaningfuluse
criteriaforhealthreformwilllikelyincluderequirementsfortheroutinecaptureofcodedclinicaldata.
Thechallengewithdiscreteclinicaldataentryisthepresentationofstructuredterminologyintheuser
interface.Vocabularymustbeefficienttonavigate,presentedintermsfamiliartoclinicalpractice(e.g.,
insteadofbilling)andattheappropriatelevelofgranularity.Interfaceterminologyisacomplexissue
andanactiveareaofMedicalInformaticsresearch.36
Ef fec t ive In format ion Presenta t ion
Appropr ia te Densi ty
Whiledensityofinformationonascreenisnotcommonlymeasured(thoughitcanbe),itisavery
importantconcepttobecognizantofwhendesigningEMRscreens.Inclinicalapplications,therecanbe
somuchrelevantinformationtodisplayitcanbetemptingtopackasmuchaspossibleontoascreen.
However,visualsearchtimesandusererrorsincreaseinproportiontodensity.Itischallengingto
balanceprovidingallthenecessaryinformationandlimitthenumberofscreenchangeswhile
maintaininganappropriatescreendensity.Testingactualuserswillrevealwhenthebalancehasbeen
reached.
Charactercount,resolution,font,fontsizeandgroupingtechniquesimpactvisualdensity.Screen
elementssuchaslines,buttons,controls,scrollbarsandiconsalsocontributetodensity,whichisyet
anotherreasonthatsimplicityissoimportant.
Accordingtoergonomicrecommendationsforinformationpresentationoncomputerscreens,15
an
upperlimitof40percentdensityisappropriateforcharacterbaseddisplays(thepercentageofpotential
characterpositionsactuallyfilledbycharacters).Graphicaluserinterfacesmustbeevenlessduetothe
otherelementscontributingtoperceptionofdensity.
Animportantmeansofreducingdensityisviewingdataatasummarylevelbeforedrillingdownto
detail.Roughly,the80/20ruleappliestosummaryscreens80percentofthetimetheinformationat
thesummary
level
is
sufficient
for
decision
making
and
is
the
most
frequently
needed
information;
20
percentofthetimetheuserwillneedtodelvedeeper.
Meaningful Use of Color
Colorisoneofseveralattributesofvisualcommunication.Itissingledoutherefordiscussiondueto
howpoorlyithasbeenutilizedinmanysystemdesignstodate.Skillfuluseofcolorcertainlycontributes
toauserinterfacethatispleasinginappearance.However,aestheticsshouldbethelastconsideration
8
-
8/14/2019 HIMSS EHR Usability Task Force Report on Defining and Testing EMR Usability
11/42
2009HealthcareInformationandManagementSystemsSociety(HIMSS)
forusingcolorinanytaskorientedapplication.Firstandforemost,colorshouldbeusedtoconvey
meaningtotheuser.Thisincludesallaspectsofinformationpresentation,navigation,differentiationof
screenareasandstaterepresentationofcontrols.Everythingintheuserstaskareaofthescreen,
includingnavigationbars,needstoobeyameaningfulcolorscheme.Purelyaestheticuseofcolorshould
belimitedtodesignoficons,logosandbannerareas.
Simplicityandconsistencyarebothkeyprinciplesintheuseofcolor.Forcolortoconveymeaning,there
cannotbealargernumberofcolorsusedthantheusercanremember,andtheymustbeused
consistentlythroughouttheapplication.Forinstance,ifbrightyellowisusedasahighlightercolorto
emphasizethenameofthepatientwhoseordersarecurrentlybeingentered,thenbrightyellowshould
onlybeusedasahighlightcolorforkeyinformation. Inconsistentorgratuitoususeofcolorincreases
thelikelihoodofusererrorduetomisinterpretationoroversightofimportantdetails;themeaningwill
belost.
Toaccommodateuserswithcolorblindness,allmeaningconveyedwithcolormustalsobe
differentiatedwithasecondvisualmechanism(redundantencoding)suchasfontcharacteristicsorfill
pattern.Forexample,ifredisusedtodisplaycriticallabvaluesthenthecharactersshouldalsobe
bolded,increasedinsizeorsomeothercharacteristic.Itishighlyrecommendedthatdisplaysbe
designedingrayscalepriortoaddingcolortoensurethatallmeaningisrepresented.Ifnot,theinability
todifferentiatecolorsalsomayleadtousererrorsthathavepatientsafetyconsequences.
Naturalnessisaccomplishedbyadheringtoculturalconventionsofcolormeaning.IntheUnitedStates,
thefollowingcolorinterpretationsarecommonlyunderstood.Comprehensiveguidelinesonuseofcolor
haverecentlybeendevelopedbyHFES15;seealsoHHS49andAccessibilityForum.1
Red: Stop, Hot, Danger, Error, Extreme Warning, Severe Alert, Emergency, Alarm
Yellow: Caution, Potential or Mild Warning, Requires Attention, Slow, Moderate Alert
Green: Go, Safe, Normal, Good, Proceed
Blue: Cold, Advisory
Readabi l i ty
Screenreadabilityalsoisakeyfactorinobjectivesofefficiencyandsafety.Clinicalusersmustbeableto
scaninformationquicklywithhighcomprehension.Thepaceandfrequentinterruptionsinclinical
workflowguaranteethatdecisionswillsometimesbemadebaseduponcursoryscreenreview.
Simplicity,naturalness,languageuse,densityandcolorallcontributetoreadability.Inaddition,
guidelinesrecommendusingafontsizeofnosmallerthan12pointforimportantcontentandnever
smallerthan
9point
as
defaults.
Differences
in
visual
acuity
make
it
necessary
to
allow
users
to
modify
textsizeasneeded.Systemsettingsforcolor,fontsandfontsizeshouldalwaysberespected.1,26,49,15San
seriffontscanbereadmoreeasilyincomputerdisplaysthanseriffonts.Highcontrastbetweentextand
backgroundisalsoimportant;blackonwhiteisthemostreadable.
9
-
8/14/2019 HIMSS EHR Usability Task Force Report on Defining and Testing EMR Usability
12/42
2009HealthcareInformationandManagementSystemsSociety(HIMSS)
Preservat ion of Contex t
Preservationofcontextisaveryimportantaspectofdesigningatransparentapplication.Inpractical
terms,thismeanskeepingscreenchangesandvisualinterruptionstoaminimumduringcompletionofa
particulartask.Visualinterruptionsincludeanythingthatforcestheusertoshiftvisualfocusawayfrom
theareaonthescreenwheretheyarecurrentlyreadingand/orworkingtoaddresssomethingelse,and
thenreestablishfocusafterward.Themostfrequentviolatoristhedialogbox,whichalsotendsto
obscureasignificantpartofthescreen.Dialogboxesshouldbekepttoaminimum.Forinstance,whena
dialogormessageboxistriggered,itshouldappearincontext(adjacenttoorjustbelowthecontrolthat
triggeredit).Thislimitsvisualsearchingandmakesitfeellikeitisanaturalpartofthecurrenttask.Allof
theseboxesshouldalsobeassmallaspossiblewithoutcompromisingtheirusability.
Anotherimportantguidelineassociatedwithpreservationofcontextisthatofdirectness.Inpart,thisis
acomponentofthewhatyouseeiswhatyougetphilosophyifyouchangesomethingonthescreen,
youshouldseethechangeimmediatelyandintheformatexpected.Anaspectofdirectnessthat
sometimesfallsthroughthecracksistoavoidmodes.Indataentry,thissometimesoccursintheform
ofviewing
vs.
entry
modes;
these
should
not
be
separate.30
If
auser
is
viewing
information
on
a
formthattheyhavepermissiontoedit,theyshouldbeabletodoso,incontext.Thisdoesnotmeanthat
informationcollectedviaaparticularform(e.g.,allergies)shouldntbedisplayedelsewhereinthe
systemasviewonly.However,anydatapresentedthatispotentiallyusereditable,shouldhavea
mechanismfortakingtheuserdirectlytotheappropriateentryformifupdatingisdesired.
SeeANSI/HFES200:HumanFactorsEngineeringofSoftwareUserInterfaces15,WindowsUserExperience
InteractionGuidelines26andResearchBasedWebDesign&UsabilityGuidelines49forcurrent,
comprehensiveguidelinesondesigningforoptimalusability.Thislastworkisuniqueinthatitsinclusion
criteriaforguidelineswasresearchbasedevidenceratherthanexpertopinion(paralleltothe
evidencebased
medicine
concept).
While
targeted
at
Web
site
design,
most
of
these
guidelines
apply
equallywelltoWebbasedanddesktopapplications.Eachguidelineisaccompaniedbyresearch
referencesandtheequivalentof5starscoresfora)relativeimportance,andb)strengthofthe
evidence.
HavingreviewedtheessentialprinciplesofEMRusability,wenowmovetoadiscussionofevaluation
methods.
USABILITY EVALUATION AND RATING METHODS
ThepeoplewhoselectanduseEMRsoftwarearemakingacommitmentthatcannoteasilybereversed.
Costsofimplementationaretypicallyhigh,andthecostsofabandoninganimplementationorswitching
toanotherproductarevastlyhigher.Reliableusabilityratingschemesofferproductpurchasersatool
forcomparingproductsbeforepurchaseorimplementation.Thesemethodscanfostercompetitionand
innovationbymakingexcellentusabilityvisibletotheentirecommunityofpurchasersandusers.47
10
-
8/14/2019 HIMSS EHR Usability Task Force Report on Defining and Testing EMR Usability
13/42
2009HealthcareInformationandManagementSystemsSociety(HIMSS)
Organizationsevaluateandcommunicateusabilityinformationfora
numberofreasons.Commercialorganizationsevaluateusabilityaspart
ofproductimprovement,andasameansofdifferentiatingtheirproducts
fromcompetitors.Thegovernmentevaluatesusabilitytoassuresafety
standardsarenotcompromised.IndependentgroupssuchasConsumer
Reportsprovide
evaluation
data
to
help
consumers
make
informed
purchases.Thissectionwilldescribetwosuchprogramsthatprovide
potentialEMRratingorganizationswithperspectivesonevaluating
usabilityinproductsthatmustmaintainhighsafetystandards.
1. Nat iona l Highw ay Tra f f ic Safe ty Admin is t ra t ion Chi ld Safe ty Seat Usabi l i ty
Rat ing Program
TheNationalHighwayTrafficSafetyAdministration(NHTSA)EaseofUseratingprogramwasdesignedto
createmarket
forces
that
encourage
child
seat
manufacturers
to
include
user
friendly
features,
labeling,
andinstructionmanualsandtoprovideconsumersadditionaldataastheymakechildsafetyseat
purchasedecisions.TheinitialmandateforsucharatingprogramwaspartoftheTransportationRecall
Enhancement,Accountability,andDocumentation(TREAD)Actin2000.NHTSAhadtherequirements
thattheprogrammustberepeatableandmustbeobjective.41NHTSAsEaseofUseratingprogramevolvedoverseveralyears.Thechosenapproachusestrained
evaluatorstoassignratingstofeaturesconsideredtoimpacttheusabilityofinstallingchildsafetyseats.
Weightedaveragesforeachcategoryandoverallusabilityarecommunicatedtothepublicthroughthe
useofa5starratingsystem.41
Severalkey
aspects
of
the
NHTSA
rating
program
may
provide
insight
to
EMR
certification
organizations.
First,ittooktwoyearsfromthemandatetothefirstimplementationofNHTSAsratingprogram.
Second,NHTSAcollecteddatatoevaluatetheeffectivenessoftheirprogramfromtheinitiationofthe
program.Third,NHTSArecognizesthattheEaseofUseratingprogramneedstobeflexibleandchange
asthemarketchanges.In2008NHTSAupdatedtheirEaseofUseratingprogramto:
Makeimprovementswheretheinitialprogramwasweak.
Makeupdateswheretheinitialprogramhadbecomeobsolete.
Accommodatenewsafetyseatfeaturesthatdidnotexistwhentheratingprogramwas
initiated.
Addapre
evaluation
program
to
encourage
continued
innovation.
2. FDA and Human Fact ors Regulat ion and Guidel ines for Device
Manufacturers
TheFoodandDrugAdministration(FDA)requiresdevicemanufacturerstofollowHumanFactors
regulationsandprovidesguidancetoensuresafeuseofmedicaldevices.In1997theFDApresentedthe
InBrief:Objectivemethodsare
availabletoevaluateEMR
usability.Wefocusparticularlyon
efficiency,effectivenessAND
SAFETY,butalsodiscussuser
satisfactionand
cognitive
workload. A5starratingsystem
canhelpinformEMRpurchasers
aboutanEMRsusability.
11
-
8/14/2019 HIMSS EHR Usability Task Force Report on Defining and Testing EMR Usability
14/42
2009HealthcareInformationandManagementSystemsSociety(HIMSS)
finalruleforGoodManufacturingPractice.Aspartofthisqualitycontrolmandate,medicaldevice
manufacturesarerequiredtodemonstrateadherencetogooddesignandmanufacturingprocesses.The
objectiveoftheHumanFactorsaspectoftheregulationisaimedatminimizingusererrorsthatcould
causepatientinjuryordeath.Theregulationcallsfordesigninput,designverificationanddesign
validation.17
TheessenceoftheregulationisthatHumanFactorsactivitiesaretobeconductedthroughoutthe
designanddevelopmentofamedicaldevice.Designinputcallsformanufacturerstoestablishand
maintainproceduresthatensuredesignrequirementsareappropriateandaddresstheintendeduseof
thedevice,userneedsandpatientneeds.Designverificationrequiresmanufacturestoestablishand
maintainproceduresforverifyingthedesigninput.HumanFactorsactivitiesmayincludetaskanalysis,
functionalanalyses,userstudies,prototypetestsandmockupreviews.Designvalidationrequiresthat
thedeviceconformstothedefineduserandpatientneeds,andassuressafeuseinbothintendedand
unintendedusesofthedevice.Ariskanalysisaimedatminimizingusererrorthatcanleadtopatient
injuryordeathmustbeincludedaspartofdesignvalidation.16
KeyaspectsoftheFDAsregulationandguidanceusefultoanEMRcertificationprocessincludethe
following:First,theFDAprovidesguidance,placingtheresponsibilityonthemanufacturertobe
educatedinHumanFactorsandtoselectappropriatemethodstomeettheFDAsregulations.Second,
theFDAendorsesandrequiresmanufacturestoadheretostandardsdevelopedbyotherstandards
organizationsinadditiontotheirown.Third,theFDAholdsmanufacturersaccountabletoHuman
Factorsregulationsthroughfieldinspections,premarketreviews,andpostmarketsurveillance.Ineach
situationtheFDAinstructsmanufacturerstoprovideevidenceforappropriateHumanFactorsanalyses
andtestsfortheproductunderreview.17
Evaluat ion Methods and Met r ics
Dependingonthereasonfortesting(e.g.,earlydesignvs.differentiationbetweeninterfaces)
measurementmethodsdiffer.Usabilityexpertsapproachproductevaluationasaprocess.Assuch,
therearespecificgoalsforeachphaseoftheprocessandtherearespecificactivitiesappropriateto
addressphasespecificgoals.42Usabilityevaluationmethodsareoftendescribedasbeingprimarily
formativeorsummativeinnature.Formativeevaluationisusedtoinformandimprovetheproduct
designduringthedevelopmentprocess.Summativeusabilitytestingisavalidationexercisetoevaluate
aproductattheendofthedevelopmentprocess.
Usabilityistheresultofcarefuldesignandevaluationthroughoutproductdevelopment.Duringthe
designanddevelopmentprocess,formativeusabilityactivitiesarecarriedoutinsupportofdefiningthe
application,understanding
the
user
and
user
workflow,
and
making
iterative
improvements
to
the
product.Thedatagatheredduringtheseactivitiestendtobemorequalitativeanddescriptive.The
findingsfromformativeusabilityactivitiesaremeanttodescribeanddefineusersanduserneedsand
productfeatures,aswellashaveanimpactonthedesignoftheproductsuserinterface.
12
-
8/14/2019 HIMSS EHR Usability Task Force Report on Defining and Testing EMR Usability
15/42
2009HealthcareInformationandManagementSystemsSociety(HIMSS)
Thisphaseisclearlytheresponsibilityofthesoftwarevendor.Inaddition,thesemethodsshouldbe
appliedintheconfigurationofhighlyadaptablesystemsduringimplementation;configurationcan
involveahighdegreeofscreendesignandworkflowengineering.Formativeusabilityactivitiesinclude
butarenotlimitedto:
ContextualInquiry
FocusGroups
StakeholderMeetings
AffinityDiagramming
TaskAnalysis
RiskAssessment
ExpertReview
Oneononeusabilitytesting
Laterinthedevelopmentprocess,summativeusabilityactivitiesarecarriedouttorefinetheproduct.
Theyalso
may
be
done
after
product
completion
to
validate
the
usability
of
the
product,
or
compare
it
withcompetitorproducts.Recommendedusabilityratingactivitiesclearlyfallafterproductcompletion
andshouldbesummativeinnature.Summativeusabilityactivitiesincludebutarenotlimitedto:
ExpertReview
PerformanceTesting
RiskAssessment
Oneononeusabilitytesting
Summativeusabilityactivitieseachhavespecificgoalswhichtheyappropriatelyaddress.Thedata
gatheredduringtheseactivitiestendtobemorequantitativeandobjective.However,somesummative
researchactivities
are
subjective.
Expert
reviews
as
ameans
to
validate
usability
introduce
subjective
expertinput.Thefindingsfromsummativeactivitiesaremeanttovalidateandconfirmusability.Ifa
vendorhasemployedaniterativeusercentereddesignprocessthroughtheproductdevelopment
process,thereshouldbefewsurprisesthatariseinsummativeusabilitytesting.
Animportanttradeofftoconsiderinanyusabilityevaluationisthetestingenvironment.Assoonasthe
usabilityevaluationismovedoutoftheactualenvironmentandintoatestenvironment,muchofthe
complexitycausedbytheenvironmentisremoved.Assuch,asystemthatisratedhighinusabilityina
testenvironmentmaynotbeeasytouseinthecontextoftheactualenvironmentrifewithinterruptions
andchangingworkpriorities.Usabilitytestingbestpracticessubmitsthatitisalwaysbesttoconduct
formativeusability
testing
in
the
environment
that
is
closest
to
the
users
actual
environment.
Software
vendorsshouldensurethattheirdesignersanddevelopershavetheopportunitytoexperiencetheirend
productinuseinaclinicalsetting.Thisprocesscouldhelpmaketremendousstridestowardminimizing
disconnectbetweenwhattheuserneedsanapplicationtodoandwhatitactuallydoesorhowit
doesit.
13
-
8/14/2019 HIMSS EHR Usability Task Force Report on Defining and Testing EMR Usability
16/42
2009HealthcareInformationandManagementSystemsSociety(HIMSS)
Usabilityspecialistshavestarteddevelopingautomatedmethodsforusabilitytesting.Automated
usabilitytesttoolstypicallyevaluateuserinterfacesagainstdesignrequirements.TheWebsite
http://usability.govprovidesanautomatedusabilitytooltofederalagencies(AutomatedUsability).In
addition,commercialproductsareavailableforautomatedusabilitytestingofsoftwareapplicationsand
Webbasedsolutions.
Evaluat ing Eff ic ienc y
Efficiency,asatestmetric,isthespeedwithwhichausercansuccessfullyaccomplishthetaskathand.
Researchactivitiesaimedatevaluatingefficiencyincludesexpertreviewandefficiencystudies.Expert
ReviewisaHumanFactorsexpertreviewoftheproduct.Aspartofthereview,theusabilityspecialist
identifiesareasintheproductwheretheproductconformsorfailstoconformtoHumanFactorsbest
practices.Thereissomeamountofsubjectivejudgmentinvolvedinanexpertreview.25
Thereareanumberofvariantsononeoneoneusabilitytestsaimedatevaluatingefficiency.Atypical
efficiencystudycallsforanexpert,intermediateornoviceusertocompletespecifickeytaskswiththe
application.
Performance
data
is
collected.
Sessions
are
frequently
recorded
with
special
software
that
capturesinteractionswiththegraphicaluserinterfacesandmatchestheinteractionswithtimestamps.
Theresultsareusedtoevaluatetheefficiencyoftheproduct.25
Themostcommonmeasuresofefficiency:
Timetoperformaparticulartask.
Numberofkeypressesorinteractionstoachievetask.
Numberofscreensvisitedtocompleteaspecificworkflowscenario.
NumberofBackbuttonuses.
Timetoexecuteaparticularsetofinstructions.
Evaluat ing Effect iveness
Effectivenessistheaccuracyandcompletenesswithwhichausercanachievetaskgoals.Riskanalysisis
acollectionoftechniquesforidentifyingthemostlikelyhumanerrorpointsinasystem.A
comprehensiveriskanalysiswillidentify,quantifyandmitigateriskswithiterativeassessmentand
implementationthroughoutproductdevelopment.Thesetechniqueshavebeenusedformanyyearsin
numerousindustriessuchasthespaceprogram,shippingandnuclearenergy.Earlyonitwaslearned
thathumanfailuresweremuchmoredifficulttopredictthanmechanicalorelectroniccomponents.24
TheJointCommission,VeteransAdministration,theFDA,andtheDepartmentofDefensehavespenta
greatdeal
of
time
and
effort
developing
variants
of
Failure
Modes
and
Effects
Analysis
(FMEA)
to
identifyandanalyzerisksinhospitalandotherhealthcareprocesses,medicaldevicedevelopmentand
othercomplexsystems.FMEAisoneofthemostwidelyusedformsofbottomupriskanalysisandis
theprevalentformofriskanalysisintheautomotiveandaviationindustries.43AnFMEAforanalyzing
humanerrorisasystematicprocessexaminingtheusersworkflowforpointswhereerrorcouldoccur.
14
-
8/14/2019 HIMSS EHR Usability Task Force Report on Defining and Testing EMR Usability
17/42
2009HealthcareInformationandManagementSystemsSociety(HIMSS)
Theprobabilityandseverityoflikelyerrorsareevaluated,andappropriatemitigationforeachpotential
errorisidentified.
InadditiontoFMEA,whichhasawelldevelopedformalism,amoregenericformofriskanalysisisa
topologicalriskanalysis.Thetopologicalriskanalysisisaviablefirstanalysissteptoexaminethe
processin
enough
depth
to
define
its
layout
(or
topology).
A
topological
analysis
can
help
identify
a
numberofriskelementssuchassinglepointfailuresandcommonmodefailures.
Asinglepointfailurewouldbeanyactionbytheclinicianthatresultsinharm,injuryordeathtothe
patientwithoutaredundantsafetycheckinplace.Anexamplewouldbeprescribingadrugthepatient
isallergictobecausedrugallergieswerenotdisplayedontheorderingscreen,anddecisionsupportfor
drugallergycheckingwasnotyetimplemented.Acommonmodefailureiswhenmultipleactionsbythe
clinicianstemfromasinglecausethatresultsinharm,injury,ordeathtothepatient.Forexample,a
prescribingerrorcausedbyforcingtheprovidertoaccessmultiplescreensandholddetailsinmemory
tocompletetheprescribingprocess.Asrisksareidentified,detailedanalysisandmitigationefforts
shouldconcentrateatthesepoints.24
Ausabilityratingprocesscanbedevelopedbyadaptingriskassessmentmethodologiestoobjectively
evaluatethepotentialforusererror.Certaindesignfactorscanleadtousererrorwhichwouldhave
patientsafetyimplications.AssessinganEMRuserinterfaceforthepresence/absenceofthesedesign
factorsprovidesanimportantmeansofevaluatingEMReffectiveness.Examplesofusererrorswith
patientsafetyimplicationsareprovidedinAppendixC.Indepthworkonexamininghowuserinterface
designchoicescancompromisepatientsafetyisbeingconductedbytheePrescribing&
CommonUserInterfaceprogramsoftheNationalHealthService(NHS)intheU.K.Thisgrouphas
designedasafetyfocusedusabilityevaluationmethodbasedonerrortrapssimilartothepatient
safetychecklistconceptproposedinthisdocument.11TheNHSalsoisintheprocessofdeveloping
specificguidelines
for
safe
on
screen
display
of
medication
information.
Effectivenessstudiesareaclassofoneononeusabilityteststhatinvolvecollectingmeasuresof
effectivenesswhenuserscompletespecifickeytaskswiththeapplication.
Commonmeasuresofeffectivenessincludebutarenotlimitedto:
Numberorrateoferrors
Pathtakentocompletetask
Severityoferrors
Requestsforhelp
Evaluat ing Ease of Learning
Improvingusabilityhasbeenshowntoimproveeaseoflearningorlearnability.37Themoreausercan
applypriorexperiencetoanewsystemandthegreatertheinternalconsistency(useofconsistence
concepts,behaviors,layout,etc.)thelowerthelearningcurve.Whenasystemisforgivingofmistakes
15
-
8/14/2019 HIMSS EHR Usability Task Force Report on Defining and Testing EMR Usability
18/42
2009HealthcareInformationandManagementSystemsSociety(HIMSS)
andallowsdiscoverythroughexploration,itfostersfasterlearningbyreducingtheusersfearof
unintendedconsequences.Errors,pathtakentocompletetasksandrequestsforhelpeachcorrelate
withhowwellauserknowsthesystem.
Easeoflearningcanbeevaluatedintermsofthetimeittakestheusertoreachaspecifiedlevelof
proficiencyand
in
terms
of
the
time
it
takes
auser
who
has
never
seen
the
system
interface
to
successfullyaccomplishbasictasks.Itisimportanttoconsiderlearningthroughoutthelifetimeofuseof
aproduct.33
Possiblemeasuresofeaseoflearningincludebutarenotlimitedto:
Timetoachieveexpertperformance.
Numberoficonsrememberedaftertaskcompletion.
Timespentusingmanual.
Timetoperformaparticulartaskafteraspecifiedperiodoftimeawayfromtheproduct.
Timetoperformtaskcomparedtoanexpert.
Numberof
times
the
Help
function
is
accessed.
Foramorecompletereviewofmetrics,methodologies,andguidelinesregardingusabilityandeaseof
learningseeGrossman,etal.14
Evaluat ing Cognit ive Load
Manymethodsformeasuringcognitiveloadinvolvecomplextestingthatrequiretheskillsofcognitive
psychologistsorexperiencedHumanFactorsengineers.However,thereareafewwelldevelopedand
validatedinstrumentsthatareadministeredassimplequestionnaires.TwoexamplesaretheNASATask
LoadIndex(NASATLX)andtheSubjectiveWorkloadAssessmentTechnique(SWAT).Itispossibleto
simplifythesemethodssuchthattheymaybeadministeredfairlyeasily.10Cognitiveloadissuchan
importantissueforcliniciansthatthisshouldbeconsideredforinclusioninanEMRusabilityrating
program.
Evaluat ing User Sat is fact ion
Thedefinitionofusabilitytypicallyincludesreferencetousersatisfaction.Usersatisfactionisapersons
subjectiveresponsetotheirinteractionwithasystem.Whenevaluatingusability,satisfactioncanbe
addressedinseveralways.AcommonapproachusesLikertscalequestionnairesaskinguserstorate
theirsatisfactionwithvariousaspectsoftheproduct(e.g.,onascaleofoneto10).Typicallythisisdone
immediatelyafterhandsonusabilitytaskperformanceandattheendofausabilitytestsession.Whatis
weakaboutthisapproachisthatthemethodhasnotbeendevelopedunderscientificscrutiny.Other
researchersusemorescientificratingtoolssuchastheSystemUsabilityScale.7,48,5,44
Thesescalesare
strongerbecausethetoolsareaccompaniedbymeasuresofreliabilityandvalidity.Researchsuggests
thatusersatisfactiondoesnotcorrelatewellwithothermorerigorouslyobtainedmeasuresofusability
suchaseffectivenessandefficiency.4,12
16
-
8/14/2019 HIMSS EHR Usability Task Force Report on Defining and Testing EMR Usability
19/42
2009HealthcareInformationandManagementSystemsSociety(HIMSS)
Weagreethatusersatisfactionisonecomponentofusability.However,becauseofthesubjective
natureofevaluatingusersatisfaction,wewillnotproviderecommendationsconcerningmeasuringuser
satisfactionaspartofausabilityratingprogram.WedohighlyrecommendthatEMRpurchasers
performstructuredusersatisfactiontestingaspartoftheirEMRselectionprocess.Aggregationofuser
satisfactiondatafromcurrentEMRenduserswouldalsobeofvalueifcollectedusingappropriate
instrumentsand
methodology.
Select i ng Tasks for Evaluat ion
Testtasksshouldbeselectedbasedonhowthedataaregoingtobeused.Whenevaluatingefficiency,
tasksfocusedonuserandsystemefficiencyshouldbeconsidered:thatis,tasksthatwillbefrequently
performedbyusersandtasksthatareknowntobeinefficient.Whenevaluatingeffectiveness,tasks
focusedonusereffectivenessshouldbeconsidered:thatis,tasksthataredeemeddifficulttoperform
andtasksthatareknowntobeatriskforusererrorbasedonpriorevaluations.Userobservations,
workflowanalysisandtaskanalysisaremethodsusedtoidentifyfrequentanderrorpronetasks.
Surveysandinterviewsaremethodsusedtosolicitinformationfromusers;however,userbehaviors
shouldalways
be
directly
observed
when
possible
because
users
are
not
always
accurate
in
describing
whattheyactuallydo.
OneofthechallengesofevaluatingusabilityinEMRsisthecomplexityofusertasks,workflowsandthe
userenvironment.Considerthetasksimplystatedasrefillamedication.Theactualclinicalworkflow
includesacombinationofelements.Specifically,theproviderneedstoconsiderthefollowing:
1. Pastdatapoints(e.g.,medicationhistory,lastvisitdate,relevantlabvalues,lastclinicnote)
2. Futuredatapoints(e.g.,nextlaborvisitdate).
3. Medicalevidencepersonalizedforthepatient(e.g.,whatisthegoalcholesterolforthispatient,
howoftendolabsneedtobecheckedonthismedication).
4. Contextualrelevanceof#13:Whereisthepatientinthelifecycleofthismedication(e.g.,did
theyjuststartitorhavetheybeenonitforfiveyears,reasonforrefill).
5. Taskofformallycreating/approvingtherefill.
6. Considerationsofcostandformularycoverage,andpossiblealternativeproductswithbetter
formularycoverage.
7. Communicatingwiththeirassistantorthepharmacy.
Forthepurposesofusabilityrating,werecommendselectingtesttasksthatencompassentire
workflows(scenarios).EMRsarecomplexsystemsandusabilityofcomplexsystemsmustincludethe
interactionsbetweentheinformationandusersubtasksthatmakeuptheactualwork.
Oncetasksareselected,successfulcompletioncriteria(suchaserrorstates,deviationsandothers)
mustbedefinedforeachtask.ExamplesoftesttasksandtestscenariosareprovidedinAppendixA.
17
-
8/14/2019 HIMSS EHR Usability Task Force Report on Defining and Testing EMR Usability
20/42
2009HealthcareInformationandManagementSystemsSociety(HIMSS)
5-Star Usabi l i t y Rat ing System
Thefivestarratingsystemisreadilyrecognized,sinceitisacommonschemeusedwithconsumer
productsinmanycommercialWebsites.Developmentworkisneededtodefineausabilityrating
system(e.g.,5star=excellent,4star=good,etc.)thatcanbeusedtocommunicatetheresultsofa
usabilityratingprogramtoEMRpurchasers.
Considerationmustbegiventodefinitionofthescaleandassigningmetricintervalcutoffs.Asan
example,shouldtheratingscaleconsistofequalintervals(e.g.,5staris059seconds;4staris6069
seconds,etc.)orshouldtheratingscaleconsistofintervalsbasedonthenormaldistribution(e.g.,3star
isthemiddle50thpercentile).
Anotherconsiderationisthegranularityofthereportingsystem.Oneoptionistocombinethescores
fromthetasksineachcategory(efficiencyandeffectiveness)andreportoneglobalstarrating.Asecond
optionistoreportseparatestarratingsforeachcategoryofmeasure.Initiallythismayonlymean
separateratingsforefficiencyandeffectiveness;ratingsforfactorssuchascognitiveloadandeaseof
learningshould
also
be
considered.
Themostimportantaspectofdevelopingastarratingsystemisdefiningthebenchmarkmetricsforeach
measure.Forefficiencymeasures,goaltasktimesshouldbebasedontheneedsofthecliniciansin
actualclinicalpractice.Expectationsshouldbebothhighandattainable.AppendixBincludesbenchmark
examples.
Cert i f ic at ion and EMR Usabi l i t y Rat i ng
Duringoursurveyoftheliterature,welearnedthattheCertificationCommissionforHealthcare
InformationTechnology(CCHIT)wasactivelyconsideringaddingusabilityasacriteriontoitsEMR
certificationprocess.Subsequently,wecontactedthemtounderstandtheiroverallprogram
requirements,andtobetterunderstandtheneedsofcertificationorganizations.CCHITisaprivate
nonprofitorganizationwiththesolepublicmissionofacceleratingtheadoptionofrobust,interoperable
healthinformationtechnologybycreatingacredible,efficientcertificationprocess.Theydescribedthe
followingcharacteristicstheyfounddesirableinthedevelopmentofausabilityprogram:
Theprogrammustbeobjective.
Theprogrammustberepeatable.
Theprogrammustbecostefficienttoimplement.
Theprogramshouldfocusonevaluatingefficiencyandpatientsafety.
Theprogramshouldevaluateproductsthatarereadyformarket.
Agood
approach
would
be
to
rate
usability
on
ascale
similar
to
star
programs
seen
in
consumer
products.
Usabilityratingshouldbeanadjuncttoproductcertificationwithoutaffectingcertification
outcome.
Theusabilityratingsystemadoptedshouldnotbeapass/failmodel
18
-
8/14/2019 HIMSS EHR Usability Task Force Report on Defining and Testing EMR Usability
21/42
2009HealthcareInformationandManagementSystemsSociety(HIMSS)
RECOMMENDATIONS
Asthecertifyingorganizationimplementsitsusabilityratingprogram,
basedonourresearchandpriorknowledgeofusabilityprinciplesand
practiceweproposethefollowingapproach:
Star t smal l .
Beginusabilityratingwithafocusonsimpleefficiencyandeffectiveness
measures,includingsomeinitialpatientsafetycheckpoints.Dontletit
becomeamultiyearprocesstosetupaninitialprogram.Learnfromand
adaptHumanFactorsprocessesestablishedbytheFDA,NHTSAandother
organizations.
Develop measurement s.
Deviseobjectivemeasuresofefficiencythattakesintoaccounttimeontaskandnumberofuser
interactions.Develop
objective
measures
of
effectiveness
that
takes
into
account
system
characteristics
thatimpactpatientsafety.Werecommendinitiallytestingeffectivenessusingapredefinedchecklistof
systeminteractionsthathavepatientsafetyimplicationsifnothandledwellbythesystem.AppendixC
illustratespatientsafetychecklistexamples;alsoseeFoneandLewis.11
Create a 5-s tar ra t ing system.
Definetheusabilityratingsystemusing5starsbasedonanabsolutestandardagainstbenchmarks.An
absolutestandardallowseachproducttostandonitsownanddemonstrateprogressovertimerather
thanincomparisontootherproducts.Thisrequiresthatbenchmarksbeestablishedinadvancedefining
targetscoresforeachmeasurement.Benchmarksshouldreflectuserneedsinactualclinicalpractice.
SeeAppendix
B
for
sample
benchmarks.
Initially,reportstarratingsonafewmeasures.Insubsequentyears,additionalmeasurescanbeadded.
Def ine the process.
Selectasetoftesttasksforevaluatingefficiencythata)arefrequentlyperformedbyproviders;b)areat
riskofbeinginefficient;andc)allowevaluationoftasksandworkflow.Beginwithsimple(butcarefully
planned)scenariobasedusertestingsimilartothediscountusabilityengineeringmethodsdescribed
byNielsen.30Recall,however,thattheprocessesshouldbeengineeredtobesummativeinnature.
Forefficiencyevaluationwerecommendamultistepapproach:
1)Thevendordoesawalkthroughexplanationtotheratingorganizationsselectedintermediateor
expertclinicalusers.
2)Thevendorperformsthetesttasks;tasktimesarerecordedasmeasuresofexpertperformance.
nBrief:Weofferspecific
recommendationsforactiontorating
organizations:
Startsmall.
Developmeasurements.
Definetheprocess.
Createa5starratingsystem.
Improvewithtime.
Encourageotherstodotheirpart.
19
-
8/14/2019 HIMSS EHR Usability Task Force Report on Defining and Testing EMR Usability
22/42
2009HealthcareInformationandManagementSystemsSociety(HIMSS)
3)Theratingorganizationsselectedusersperformtesttasksforintermediateusermeasurement.As
muchaspossible,thegroupofusertestparticipantsshouldrepresentamixtureofroletypes(e.g.,
physician,nurse,medicalassistant,physicaltherapist)performingtasksappropriatetotheirrole.The
ratingorganizationsselectedusersshouldbeexperiencedintwoormoreEMRs,butnottheonebeing
tested.Thenumberofuserparticipantsnecessarytoproducemeaningfulresultswillneedtobe
evaluated.In
general,
summative
testing
requires
more
participants
than
formative
testing.
Foreffectivenessevaluation,anevaluatorwillneedtodeterminethepresenceorabsenceofpatient
safetyitemsfromthechecklistdeveloped.SeeAppendixCforexampleeffectivenesstestsforpatient
safety.
Improve w i th t ime .
Usabilityratingprogramsshouldevolveinsophisticationoveramultiyearperiod.Eventuallythey
shouldincludemeasuresofcognitiveloadandeaseoflearning.Infutureyears,consideralsotesting
naveusers.Evaluatetheeffectivenessoftheprogramitselfonanannualbasis.Updatescenarios,tasks,
methods
and
measures
to
reflect
any
needed
improvements
as
well
as
evolution
in
the
EMR
marketplaceandusabilitybestpractices.
Itmaybehelpfultotheconsumertobreakthestarsystemintocategoriesasitbecomesmorecomplex,
e.g.,aproductmayscore4starsforefficiency,3starsforeffectivenessand3starsforeaseoflearning.
Duetoitssubjectivenature,werecommendthatusersatisfactionbeleftforpotentialcustomersand
thirdpartiestoevaluate.
Encourage o thers t o do the i r par t .
Encouragevendorstoutilizeiterativedesignwithformativeuserbasedresearchthroughoutthedesign
anddevelopmentprocesswithsummativeusabilityevaluationbeforelaunch.Atthesametimeeducate
clinicaldecision
makers
to
assess
EMR
usability
as
part
of
their
EMR
purchase
and
system
configuration
processes.
REFERENCES
1.AccessibilityForum.QuickReferenceGuidetoSection508ResourceDocuments.September15,2003.
AccessedJune8,2009.Availableat:www.accessibilityforum.org/paper_tool.html.
2.AmericanMedicalAssociation.AMAspecialtycodes.(n.d.).AccessedApril23,2009.Availableat:
www.ehealthlist.com/ama_specabbrev.asp.
3.Usability.gov.AutomatedUsabilityTestEnvironment(UTE)ToolFactSheet.(n.d.).AccessedJune7,
2009.Availableat:www.usability.gov/refine/UTEfactsheet.html.
4.BaileyB.GettingtheCompletePicturewithUsabilityTesting.March2006.AccessedJune7,2009.
Availableat:www.usability.gov/pubs/030106news.html.
20
-
8/14/2019 HIMSS EHR Usability Task Force Report on Defining and Testing EMR Usability
23/42
2009HealthcareInformationandManagementSystemsSociety(HIMSS)
5.BangorA,KortumPT,MillerJA.AnempiricalevaluationoftheSystemUsabilityScale(SUS).IntJHu
CompInteract.2008;24(6):574594.
6.BatesDW,KupermanGJ,WangS,GandhiT,KittlerA,VolkLetal.Tencommandmentsforeffective
clinicaldecisionsupport:makingthepracticeofevidencebasedmedicineareality.JAMIA.
2003;10(6):523530.
7.BrookeJ.SUS:aquickanddirtyusabilityscale.InJordanPW,ThomasP,WeerdmeesterBA,
McClellandAL(Eds.).UsabilityEvaluationinIndustry.London:TaylorandFrancis.1996.
8.CampbellEM,SittigDF,AshJS,GuapponeKP,DykstraRH.Typesofunintendedconsequencesrelated
tocomputerizedproviderorderentry.JAMIA.2006;13(5):547556.
9.EdsallRL,AdlerKG.UsersatisfactionwithEHRs:reportofasurveyof422familyphysicians.FamPrace
Manage.2008;15:2532.
10.EmbreyD,BlackettC,MarsdenP,PeacheyJ.DevelopmentofaHumanCognitiveWorkload
AssessmentTool.July2006.AccessedJune8,2009.Availableat:
www.mcga.gov.uk/c4mca/research_report_546.pdf.
11.FoneJ,LewisK.Safety&usabilitymethodsforevaluatingtheuserinterface.April28,2009.
AccessedJune19,2009.Availableat:www.mit.edu/~juhan/nhs_public/Safetyfocusedusability
evaluation_28Apr2009_KitLewis_for_HC_v2.pdf.
12.FrkjrE,HertzumM,HornbkK.Measuringusability:areeffectiveness,efficiency,andsatisfaction
reallycorrelated?ProceedingsoftheSIGCHIconferenceonHumanFactorsincomputing
systems,(pp.345352).NewYork,NY,USA.2000.
13.GaretsD,DavidM.Electronicmedicalrecordsvs.electronichealthrecords:yes,thereisadifference.
HIMSSAnalytics.January26,2006.AccessedJune17,2009.Availableat:
www.himssanalytics.org/docs/WP_EMR_EHR.pdf.
14.GrossmanT,FitzmauriceG,AttarR.Asurveyofsoftwarelearnability:metrics,methodologiesand
guidelines.Proceedingsofthe27thinternationalConferenceonHumanFactorsinComputing
Systems(Boston,MA,USA,April04 09,2009)(pp.649658).NewYork,NY:CHI'09.ACM.
15.HumanFactorsandErgonomicsSociety.2008.ANSI/HFES200:HumanFactorsEngineeringof
SoftwareUserInterfaces.SantaMonica,CA:HumanFactorsandErgonomicsSociety.
16.HumanFactorsImplicationsoftheNewGMPRuleOverallRequirementsoftheNewQualitySystem
Regulation.2009.AccessedJune6,2009.Availableat:
www.fda.gov/MedicalDevices/DeviceRegulationandGuidance/PostmarketRequirements/Human
Factors/ucm119215.htm.
21
-
8/14/2019 HIMSS EHR Usability Task Force Report on Defining and Testing EMR Usability
24/42
2009HealthcareInformationandManagementSystemsSociety(HIMSS)
17.InformationforManufacturersandDistributors.2009.AccessedJune7,2009.Availableat:
www.fda.gov/MedicalDevices/DeviceRegulationandGuidance/PostmarketRequirements/Human
Factors/ucm119190.htm#1.
18.InstituteofMedicine.PatientSafety:Achievinganewstandardforcare.2004.(AspdenP,Corrigan
JM,Wolcott
J,
Erickson
SM,
Eds.)
Washington,
D.C.:
The
National
Academies
Press.
19.JhaAK,DesRochesCM,CampbellEG,DonelalnK,RaoSR,FerrisTG,etal.Useofelectronichealth
recordsinU.S.hospitals.NEnglJMed.2009;360:16281638.
20.KhajoueiR,JaspersMWCPOEsystemdesignaspectsandtheirqualitativeeffectonusability.Studies
inHealthTechnologyInformatics.2008;136:30914.
21.KrallMA,SittigDF.(2002).Clinician'sassessmentsofoutpatientelectronicmedicalrecordalertand
reminderusabilityandusefulnessrequirements.AMIA2002SymposiumProceedings,(pp.400
404).
22.KrugS.(2006).Don'tMakeMeThink!(2nded.).Berkeley,CA:NewRiders.
23.KushnirukA,BoryckiE,AndersonJG,AndersonMM.(2008).Combiningtwoformsofsimulationto
predictthepotentialimpactofinterfacedesignontechnologyinducederrorinhealthcare.
Proceedingsofthe2008Springsimulationmulticonference(pp.497504).SanDiego,CA:The
SocietyforComputerSimulation,International.
24.MaddoxME,BarnesHJ.(2004).RiskyBusiness:AnalysisToolsforMedicalDevicesandProcesses.
Workshoppresentedatthe48thMeetingoftheHumanFactorsandErgonomicsSociety.New
Orleans,LA.
25.MayhewDJ.KeystrokeLevelModelingasaCostJustificationTool.InBiasRG,MayhewDJ(Eds.),
CostJustifyingUsability,AnUpdatefortheInternetAge.SanFrancisco,CA:MorganKaufmann
Publishers.2005.
26.MicrosoftCorporation.WindowsUserExperienceInteractionGuidelines.November6,2008.
AccessedJune8,2009.Availableat:msdn.microsoft.com/enus/library/aa511258.aspx.
27.NationalInstituteofStandardsandTechnology.CommonIndustrySpecificationforUsability
Requirements(NISTIR7432).June2007.AccessedJune7,2009.Availableat:
zing.ncsl.nist.gov/iusr/documents/CISURIR7432.pdf.
28.NationalResearchCouncil.(2009).ComputationalTechnologyforEffectiveHealthCare:Immediate
stepsandstrategicdirections.(SteadWW,LinHS,Eds.)Washington,D.C.:TheNational
AcademiesPress.
29.NielsenJ.DesigningWebUsability:ThePracticeofSimplicity.Indianapolis:NewRidersPublishing.
1999.
22
-
8/14/2019 HIMSS EHR Usability Task Force Report on Defining and Testing EMR Usability
25/42
2009HealthcareInformationandManagementSystemsSociety(HIMSS)
30.NielsenJ..UsabilityEngineering.ChestnutHill,MA:AcademicPress.1993.
31.PoissantL,PereiraJ,TamblynR,KawasumiY.Theimpactofelectronichealthrecordsontime
efficiencyofphysiciansandnurses:asystematicreview.JAMIA.2005;12:505516.
32.PrinciplesforUsableDesign.(n.d.).AccessedJune8,2009.Availableat:
www.usabilitybok.org/design/p287.
33.QuesenberyW.(2001).WhatDoesUsabilityMean:LookingBeyondEaseofUse'.Proceedingsofthe
48thAnnualConference,SocietyforTechnicalCommunication.Availableat:
www.wqusability.com/articles/morethaneaseofuse.html.
35.RoseAF,SchnipperJL,ParkER,PoonEG,LiQ,MiddletonB.Usingqualitativestudiestoimprovethe
usabilityofanEMR.JBioInform.2005;38:5160.
36.RosenbloomST,MillerRA,JohnsonKB,ElkinPL,BrownSH.Amodelforevaluatinginterface
terminologies.JAMIA,2008;15:6576.
37.SaleemJJ,PattersonES,MilitelloL,AschSM,DoebbelingBN,RenderML.(2007).UsingHuman
FactorsMethodstoDesignaNewInterfaceforanElectronicMedicalRecord.AMIA2007
SymposiumProceedings,(pp.640644).
38.SandersMS,McCormickEJ.HumanFactorsinEngineeringandDesign.NewYork,NY:McGrawHill.
1993.
39.SchoeffelR.Theconceptofproductusability.ISOBulletin.2003;34:67.
40.SmelcerJB,MillerJacobsH,KantrovichL.Usabilityofelectronicmedicalrecords.JUsabilityStudies.
2009;4:7084.
41.SmithCS,RockwellTE,CollinsLA,KimSD,ParkBT.(2009).NHTSASchildsafetyseatusabilityrating
program.AccessedJune8,2009.Availableat:
www.lifesaversconference.org/handouts2009/Smith2.pdf.
42.StepbyStepUsabilityGuide.(n.d.).AccessedJune7,2009.Availableat:www.usability.gov/.
43.StephansRA.SystemSafetyforthe21stCentury:TheUpdatedandRevisedEditionofSystemSafety
2000.Hoboken,NJ:JohnWiley&Sons,Inc.2004.
44.System
Usability
Scale.
(2009).
Accessed
June
8,
2009.
Available
at:
http://en.wikipedia.org/wiki/System_Usability_Scale_(SUS).
45.TheJointCommission.Safelyimplementinghealthinformationandconvergingtechnologies.
December11.2008.Availableat:
www.jointcommission.org/SentinelEvents/SentinelEventAlert/sea_42.htm.
23
-
8/14/2019 HIMSS EHR Usability Task Force Report on Defining and Testing EMR Usability
26/42
2009HealthcareInformationandManagementSystemsSociety(HIMSS)
46.TognazziniB.(n.d.).FirstPrinciplesofInteractionDesign.AccessedJune6,2009.Availableat:
www.asktog.com/basics/firstPrinciples.html.
47.Tullis,T.S.,&Albert,W.MeasuringtheUserExperience:Collecting,Analyzing,andPresenting
UsabilityMetrics.Burlington,MA:MorganKaufmann.2008.
48.TullisTS,StetsonJN.(2004).AComparisonofQuestionnairesforAssessingWebsiteUsability.
UsabilityProfessionalAssociationConferenceProceedings.
49.U.S.DepartmentofHealthandHumanServices.(2006).ResearchBasedWebDesign&Usability
Guidelines.AccessedJune8,2009.Availableat: www.usability.gov/pdfs/guidelines.html.
50.WalkerJ.Usability.InWalkerJM,BieberEJ,RichardsF.(Eds.),ImplementinganElectronicHealth
RecordSystem(pp.4759).Springer.2005.
51.ZhengK,PadmanR,JohnsonMP,DiamondHS.Aninterfacedrivenanalysisofuserinteractionswith
anelectronichealthrecordssystem.JAMIA.2009;16:228237.
CONTRIBUTORS
Authors
JefferyL.Belden,MDChair,HIMSSEHRUsabilityTaskforce
RebeccaGrayson
JaneyBarnes,PhD
Usabi l i t y Pr inc ip les Workgr oup
PatriciaAlafaireet,MHA
JaneyBarnes,PhD
JefferyL.Belden,MDChair,HIMSSEHRUsabilityTaskforce
EdnaBoone,MASS,CPHIMSHIMSSSeniorStaffLiaison
JonDuke,MD
RebeccaGrayson
AndrewHutson
JasminPhua
24
-
8/14/2019 HIMSS EHR Usability Task Force Report on Defining and Testing EMR Usability
27/42
2009HealthcareInformationandManagementSystemsSociety(HIMSS)
TianaThomasLeadershipCouncil,andCochairEHRUsabilityPrinciplesWorkgroup
JuhanSonin
PennWhite,MD LeadershipCouncilandCochairEHRUsabilityPrinciplesWorkgroup
Editors
MelanieBrodnik,PhD,RHIAViceChair,HIMSSEHRUsabilityTaskforce
HelenVolger,MSHA,CPHIMS
InAcknowledgement:
WewouldliketothankthemanyHIMSSEHRUsabilityTaskForcemembersandotherswhoprovided
theirtimeandexpertiseforreviewandcommentonthefinaldrafts.Appreciationisalsoextendedto
theTIGER
Initiative
for
permitting
us
use
of
the
compendium
of
material
organized
by
their
Usability
and
ClinicalApplicationDesignWorkgroup.Thisdocumentisthesynthesisofnumerousexperienced
professionalsintheHealthcareITandHumanFactorscommunities.
HIMSS EHR Usabi l i ty Task forc e
PatriciaAlafaireet,MHA
CeciliaBackman,MBA,RHIA,CPHQ LeadershipCouncil
JaneyBarnes,
PhD
JefferyL.Belden,MD Chair
LyleBerkowitz,MD,FHIMSS
JudiBinderman,MD,MBA
EdnaBoone,MASS,CPHIMSHIMSSSeniorStaffLiaison
MelanieBrodnik,PhD,RHIA ViceChair,HIMSSEHRUsabilityTaskforce
JonDuke,
MD
RobertDuthe,MBA LeadershipCouncil
RebeccaGrayson
DoronGutkind
25
-
8/14/2019 HIMSS EHR Usability Task Force Report on Defining and Testing EMR Usability
28/42
2009HealthcareInformationandManagementSystemsSociety(HIMSS)
ShannonHouser,PhD,MPH,RHIA
AndrewHutson
RonRibitzky,MD LeadershipCouncil
JuhanSonin
MaryAnneSterling LeadershipCouncilandCochairEHRUsabilityUserPainPointsWorkgroup
SandraStork
CarolynSwanson LeadershipCouncilandCochairEHRUsabilityUserPainPointsWorkgroup
TianaThomas LeadershipCouncilandCochairEHRUsabilityPrinciplesWorkgroup
JuanitaThreat HIMSSCoordinator
RoyceUehara
MichaelVanOrnum,RPh,RN,BCPSViceChair,HIMSSEHRUsabilityTaskforce
AlbertVillari,MD LeadershipCouncilandChairEHRUsabilityVendorBestPracticesWorkgroup
HelenVolger,MSHA,CPHIMS
PennWhite,MD LeadershipCouncilandCochairEHRUsabilityPrinciplesWorkgroup
26
-
8/14/2019 HIMSS EHR Usability Task Force Report on Defining and Testing EMR Usability
29/42
2009HealthcareInformationandManagementSystemsSociety(HIMSS)
APPENDICES
A.Test task and scenar io examples
Toplevel:
Clinical
Scenarios
These
are
entire
workflows
consisting
of
anumber
of
associated
componenttesttasks.Thesescenariosaresufficientlycomplextorepresentaclinicianworkflow
worthyoftesting.
Nextlevel:TestTasksThesearecomponenttasksthatoccurfrequentlyinclinicalsettings,orare
tasksthatareatriskforusererror.Individually,theywouldbetoosimpletoconstituteatest
workflow.
Thissetofexamplesisnotmeanttobeexhaustive,buttoserveasastartingpointfortypesof
scenariosandtasksthatmightbeapartofusabilitytesting.
27
-
8/14/2019 HIMSS EHR Usability Task Force Report on Defining and Testing EMR Usability
30/42
2009HealthcareInformationandManagementSystemsSociety(HIMSS)
TableA1.ClinicalScenarios
Theseareentireworkflowsconsistingofanumberofassociatedcomponenttesttasks.Thesescenariosaresuffi
clinicianworkflowworthyoftesting.
D
No. ScenarioName ScenarioDescription Features/Rationale
Simplicity
Naturalness
Consistency
A B C
1
AcuteUTI
with
heydocrashNew
patient
presents
with
3days
of
dysuria,hematuria,urgencyand
frequency.Nofever,chills,orback
pain.Lateron,sheremembersthat
shehasanitchyrashbetweenthe
toesofherleftfoot.Doctororders
BactrimDS1tabbidx3days,
phenazopyridine200mgtidafter
mealsandrecommendsOTC
terbinafinecreamtoapplybidx10
days.
Ohby
the
way
complaint
notincludedintheinitial
reasonforvisit.
Demonstrateshowprogram
handlesmulticomplaint
visitsandeaseofchartinga
lastminuteaddition.
+
28
-
8/14/2019 HIMSS EHR Usability Task Force Report on Defining and Testing EMR Usability
31/42
2009HealthcareInformationandManagementSystemsSociety(HIMSS)
2 Chroniccomplex
diabeticwithLDL
elevation
ChronicpatientwithHTN,Obesity,
Type2DiabetesandelevatedLDL
comesinforarecheckofhisweight
anddiabetes.DoctorwantsfastingBS
(inoffice),Lipidpanel&HbA1c(sent
out),VSincludingweight,diabetic
footexam,andinterveninghistory
beforeseeingpatient.Atendofvisit,
doctorincreasesglipizidefrom5mg
bidto10mgbid.
Careforchronicdisease.How
doesprogramhandle
instructionstostaffbefore
seeingthepatient?How
easilyareoutsidelaborders
handled?Howefficientlyare
medicationorderschanged?
Doessystemofferdecision
supportfortargetLDL,
aspirintherapyindications,
remindersforperiodic
testingandimmunizations?
+
3 Medicationrefill
request
Respondtoamedicationrefill
request.Checkmedicationhistory,
patientproblemlist,drugprescribing
informationandlabtrackingstudies
recommendedforthismedication.
Doesdesignofdisplay
providenecessary
informationinaterse,
aggregatedfashion?
+ +
4
Depressioninitial
visit
Established53
yr
old
male
with
3
monthsofdepressionsymptoms.Not
suicidalorpsychotic.Orderlabtests
tolookformedicalcausesof
depression.Initiatetreatmentwith
SSRI.Printoutapatienteducation
handoutforthepatient.
OrderTSH,
CMP
or
BMP.
Efficientaccesstopatient
educationmaterials,and
wayfindingtotheproper
handout.LinkingtheEHR
diagnosistothePatient
Educationresourcewould
savetime.
+ + +
29
-
8/14/2019 HIMSS EHR Usability Task Force Report on Defining and Testing EMR Usability
32/42
2009HealthcareInformationandManagementSystemsSociety(HIMSS)
5 Labresultletter&
orders
Sendaletter(oremail)tothepatient
reportingonherabnormalthyroid
testresult,orderthyroidmedicine,
andschedulerepeattestingforsix
weeksfromnow.
Effectiveuseoflanguage
appropriateforthepatient
receivingthecommunication.
Pagelayoutintheletterthat
makescommunication
effective.Clinician
efficiency.
Simplifieddatadisplayfor
clinician.Nonintrusive
decisionsupportforselecting
theproperthyroidtest.How
isreceiptofthenotification
tothepatientverified?What
happensifthepatientmisses
herretest?
+
30
-
8/14/2019 HIMSS EHR Usability Task Force Report on Defining and Testing EMR Usability
33/42
2009HealthcareInformationandManagementSystemsSociety(HIMSS)
TableA2.TestTasks.
Thesearecomponenttasksthatoccurfrequentlyinclinicalsettings,oraretasksthatareatriskforusererror.In
simpletoconstituteatestworkflow.
No.
Task
Name
Task
Description
Features
/
Rationale
Simplicity
Naturalness
Consistency
A B C1 FindLDL FindthepatientslatestLDLresult. Dontmaketheclinician
calculatetheLDLresult.
2 CountCADrisk
factors
Howmanycoronaryarterydisease
riskfactorsdoesthepatienthave?
Doesthesystemaggregate
riskfactordataandpresentit
conciselyandappropriately
forthetaskathand?Reduce
cognitiveload.Simpledata
presentation.
+
31
-
8/14/2019 HIMSS EHR Usability Task Force Report on Defining and Testing EMR Usability
34/42
2009HealthcareInformationandManagementSystemsSociety(HIMSS)
3 CADriskscore Whatisthepatientsriskofhavinga
coronarydiseaserelatedeventinthe
next10years?
Doesthesystemaggregate
riskfactordataandpresentit
conciselyandappropriately
forthetaskathand?
PresentingFraminghamrisk
scorecan
guide
clinician
in
makingdecisionsaboutlipid
reductiontherapy.
+ +
4 Changedefault
pharmacy
Howdoyouchangeapatients
pharmacyofchoice?Whathappensif
thenewpharmacyisnotapprovedby
thepatientsinsuranceplan?
ForgivenessandFeedbackin
eventoferrorindataentry.
Appropriatesystemdefaults. + + +
5 Druginteraction
alert
&
response
Prescribingnewdrugbringsupdrug
interactionwarning.
Physician
reviewswarning,completes
prescriptionorder,andmakeschange
indefaultsettingforDIseveritylevel
threshold.
Avoidsalertfatigue.Patient
safetyis
at
stake.
Is
informationterseand
actionable?Isseverity
thresholdeasilyadjustable?
+ + +
32
-
8/14/2019 HIMSS EHR Usability Task Force Report on Defining and Testing EMR Usability
35/42
2009HealthcareInformationandManagementSystemsSociety(HIMSS)
TableA3.PotentialTasksthatNeedAdditionalWorktobeconsideredaTestTask.
Thesearetoovaguetobecomponenttasks,orhavecomponentsthathavenoclinicalconsensusastoappropria
No. TaskName TaskDescription Rationalefornoninclusion
Simplicity
Naturalness
Consistency
A B C
1 Neworders. Placenewordersonacomplex
patient
Thistaskneedsmoredetailin
ordertodefinethetaskasa
testtask.+ +
2 Manage
hypertension
Reviewyourplanformanagingthe
patientshypertension.
Thistaskistoovague.In
ordertobeatesttask,the
taskwouldneedtohave
moredetail.
3 Whatkindof
penicillinallergy?
Whatisthenatureofthepatients
penicillinallergy?
Thistaskistoosimpleas
statedandneedscontext
regardingthe
need
for
the
informationandhowthe
informationisgoingtobe
usedinaclinicaldecision.
+ + +
33
-
8/14/2019 HIMSS EHR Usability Task Force Report on Defining and Testing EMR Usability
36/42
2009HealthcareInformationandManagementSystemsSociety(HIMSS)
4 Orthopedicconsult
order
Orderanorthopedicsconsult,with
appropriateprevisittesting.
Thistaskneedsmoredetail
asthereisnotastandard
approachforordering
consults.Shouldtheclinician
ordertheMRIoftheknee,or
letthe
orthopedic
surgeon
decideifitisneeded?
+ +
34
-
8/14/2019 HIMSS EHR Usability Task Force Report on Defining and Testing EMR Usability
37/42
2009HealthcareInformationandManagementSystemsSociety(HIMSS)
B.Benchmark examp les
Forusabilityratingstooffermorethancomparativeinformation,usabilitybenchmarkmetricsneedto
beestablished.Benchmarkmetricscanbedevelopedbymeasuringclinicianusersinactualclinical
environmentsperformingeachtaskorscenario.Clinicianuserpanelswouldthencomparethebest
actualperformancesagainstusersperceivedidealperformance,inordertodevelopatargetscorethat
betterreflectsactualuserneeds,asopposedtothecurrentstateoftheartEMRperformance.Some
targetcriteriawouldbemorestraightforward,aseitherpresentorabsentfeatures.
SeeNationalInstituteofStandardsandTechnology(2007)foradetaileddescriptionforbenchmarking
usabilitycriteria.
Table1presentsexamplesoftargetcriteriaformeasuringeffectivenesstiedtopatientsafety.
Table 1. Target Cri t er ia for Evaluat ing Pat i ent Safety.
Pat ient Safe ty Check l is t Effect iveness: Pass/Fai l of Pat ient
Safe ty I t em
MedicationlistdisplayedinTallmanlettering Pass/Fail
Patient'sdrugallergiesdisplayedonmedication
orderingscreen
Pass/Fail
Table2presentsexamplesoftargetcriteriaformeasuringtheefficiencyofanEMR.Inthisexample
efficiencyisdefinedastheaveragetimefortestparticipantstocompleteeachspecifictaskorscenario.
Usertimeandsystemresponsetime(e.g.,downloadtimes)shouldbeincludedinthetasktime.
Table 2 . Target Cr i te r ia for Measur ing Ef f ic iency.
Task or Scenar io Eff ic ienc y: Maxim um
accep tab le task t ime
Scenario1.AcuteUTIwithheydocrash minutes
Scenario2.ChroniccomplexdiabeticwithLDLelevation minutes
35
-
8/14/2019 HIMSS EHR Usability Task Force Report on Defining and Testing EMR Usability
38/42
2009HealthcareInformationandManagementSystemsSociety(HIMSS)
C. Pat ient sa fe ty check l is t examples
Thislistprovidessamplesofthetypesofdesignfactorsthatcouldleadtousererrorswhichwouldhave
patientsafetyimplications.TheyareintendedtobestraightforwardPass/Failtests.Theresulting
effectivenessmeasurecouldsimplybehowmanyofthechosenteststheproductpassed(e.g."8outof
10").Opportunitiesforcheckingsomeofthesemayneedtobebuiltintotestingscenarios,butmostcan
bescenarioindependent.
These"tests"focusonpreventionofthefollowingusererrorsorpractices:
1. Selectionofthewrongpatientorpatientencounter.
2. Selectionofthewrongmedicationordosage.
3. Steppingawayfromaterminalwithoutloggingoutorsuspendingthesession.
4. Steppinguptoaterminalandtakingactionwithinsomeoneelse'sactivesession.
5. Overlookingorbeingunawareofcriticalpatientinformation.
Pat ient Se lect ion and Ident i f ic a t ion
Patient'sfullname,uniqueID,age(orDOB)andgenderareprominentlydisplayedonallchart
screens.
Patient'sfullname,uniqueID,DOBandgenderistheminimumsetofidentifiersdisplayedwhen
selectingapatienttoaccesstheirchart.
CPOE/CDS/ePrescribing
Patient'sdrugallergiesaredisplayedonthemedicationorderingscreen.
Patient'scurrentmedicationsaredisplayedonasinglescreen.
Viewingofthepatientscurrentmedicationlistisatmostoneclickawayfromthemedication
orderingscreen.
Actionstorenew,discontinueorcancelaredonedirectlyfromthecurre