EHR Usability Test Report Product: [email protected] Introduction The EHR tested for...

9265SkyParkCourtSanDiego,CA92123Corp619-488-9800Sales800-852-1771Fax619-528-0050Sales@chirotouch.com www.chirotouch.com

EHRUsabilityTestReport

Product:ChiroTouch

Version:7.0

Reportbasedon(NISTIR7742)CustomizedCommonIndustryFormatTemplateforElectronicHealthRecordUsabilityTesting

DateofUsabilityTest: October11,2017toOctober13,2017

DateofReport: October16,2017

ReportPreparedBy:

TheUsabilityPeople,LLC.

703-687-6300 [email protected] 4000LegatoRoad,Suite1100 Fairfax,VA22033

Note:ThefollowingstudywasdevelopedusingtheNISTIR7742templateasaguideforreportingourfindings:CustomizedCommonIndustryFormatTemplateforElectronicHealthRecordUsabilityTesting.


ExecutiveSummaryAusabilitytestofChiroTouchVersion7.0,acompleteEHRwasconductedonOctober11toOctober13th,2017InavirtualenvironmentbyIntegratedPracticeSolutions,Inc.Thepurposeofthistestwastotestandvalidatetheusabilityofthe2015EHRuserinterface,andtoprovideevidenceofthisusabilityintheChiroTouchsoftware.

Duringtheusabilitytest,8chiropracticprovidersand2healthcareprofessionals(EHRusers)servedasparticipantsandusedtheEHRinsimulated,butrepresentativetasks.

Thisstudycollectedperformancedataon18taskstypicallyconductedonanEHR.Thetasksfocusedonthetwelve(12)2015EditioncertificationcriteriaspecifiedbyONC:

• Section170.315(a)(1)Computerizedproviderorderentry–medications• Section170.315(a)(2)Computerizedproviderorderentry–laboratory• Section170.315(a)(3)Computerizedproviderorderentry–diagnosticimaging• Section170.315(a)(4)Drug-drug,drug-allergyinteractionchecks• Section170.315(a)(5)Demographics• Section170.315(a)(6)Problemlist• Section170.315(a)(7)Medicationlist• Section170.315(a)(8)Medicationallergylist• Section170.315(a)(9)Clinicaldecisionsupport• Section170.315(a)(14)Implantabledevicelist• Section170.315(b)(2)Clinicalinformationreconciliationandincorporation• Section170.315(b)(3)Electronicprescribing

Duringthe60to90-minuteone-on-oneusabilitytest,eachparticipantwasgreetedbytheadministratorandaskedtoreviewandsignaninformedconsent/releaseform;theywereinstructedthattheycouldwithdrawatanytime.

MostparticipantshadpriorexperiencewiththeEHR.Theywerealsoprovidedtrainingmaterials(usermanualdocumentation)priortothetesttointroducethemtonewconcepts.

Theadministratorintroducedthetest,andinstructedparticipantstocompleteaseriesoftasks(givenoneatatime)usingtheChiroTouchsoftware.Duringthetesting,theadministratortimedthetestand,alongwiththedatalogger(s)recordeduserperformancedataonpaperandelectronically.Theadministratordidnotgivetheparticipantassistanceinhowtocompletethetask.Eachsessionwasrecordedforsubsequentanalysis.


TestLogisticsAllparticipantsweretestedontheChirotouchsystemduringremoteconferencingsessionsusingGotoMeeting.

TheGotoMeetingusabilitytestsessionwasconductedbyatestadministratorfromthetestingorganization(TheUsabilityPeople,LLC)workingfromasmallconferenceroomatTheUsabilityPeople’sFairfax,VAlocation.Adataloggerfromthetestorganizationalsotookdetailednotesoneachsession,includingusercommentsandotherratingsfollowingeachtask.Duringasessionboththetestadministratorandthedatalogger(s)couldseeonlytheparticipant’sscreenandheartheparticipant’scomments,questions,andresponses.

ChiroTouchwasevaluatedforeffectiveness,efficiencyandsatisfaction(ISO9241-11)asdefinedbythefollowingmeasurescollectedandanalyzedforeachparticipant:

• Numberoftaskssuccessfullycompletedwithintheallottedtimewithoutassistance• Timetocompletethetasks• Numberandtypesoferrors• Pathdeviations• Participant’sverbalizations

Participant’ssatisfactionratingsofthesystem.

ThefollowingisasummaryoftheperformanceandratingdatacollectedontheEHR.

Deviations Satisfaction

Task Participants TaskSuccess

TimetoComplete(Avg)

Total AverageAverage(outof5)

Task1DemographicInformation10 100% 1:49

60.60 4.20

Task2ProblemListView/Intervention10 100% 0:42

111.10 4.20

Task3ProblemListIntervention-Remove10 100% 0:57

40.40 3.40

Task4ClinicalInfoRecon/Incorporation10 70% 1:49

191.90 3.10

Task5MedicationAllergy-Add10 90% 1:33

171.70 3.50

Task6Drug-AllergyInteractionCheck10 100% 1:02

50.50 3.50

Task7ElectronicPrescribe10 100% 1:49

20.20 3.80

Task8eRXCancel/Change10 90% 2:46

111.10 3.90

Task9eRX-Renew10 100% 0:45

40.40 4.60

Task10MedOrder;Drug-DrugIntervention10 100% 1:39

10.10 4.30

Task11Medicationlist-Stop10 100% 0:21

00.00 4.80

Task12MedAllergyList-Remove10 100% 0:27

00.00 4.70


TheresultsfromtheSystemUsabilityScalescoredtheoverallsubjectivesatisfactionwiththesystembasedonperformancewiththesetaskstobe:53.

Inadditiontotheperformancedata,thefollowingqualitativeobservationsweremade:Inadditiontotheperformancedata,thefollowingqualitativeobservationsweremade:

1. MajorFindingsTheoveralltaskperformanceforthisstudywasveryhigh,withanoveralltaskcompletionrateofninety-seven(97)percent.Althoughtheperformancedatawasveryhigh,andtheindividualtaskperformanceratingwasalsohigh,thesubjectiveusabilityratingscollectedviatheSUSscalewerelowerthanexpected.

2. AreasforImprovementThissystem,likemanyotherEHRswouldbenefitfromseveralminorenhancementstotheuserexperience.ThetaskthatcreatedthemostnumberofissuesfortheparticipantswastheClinicalInformationReconciliationtask.Thistask,andseveralothers,couldbemadetobeeasiertoperformbyadditionsomeadditionalworkflowindicationtohelp“guide”theuserstowardscompletionofthetask.Severalotherareasforimprovementarediscussedlaterinthisdocument

Task13CPOEDiagImaging10 100% 1:44

10.10 4.10

Task14CPOEDiagImaging-Modify10 100% 1:18

60.60 4.30

Task15CPOELabOrders10 100% 1:36

10.10 4.20

Task16CPOELabOrders-Modify10 100% 0:59

30.30 4.70

Task17ImplantableDevice-Add10 100% 0:59

90.90 4.20

Task18ImplantableDevice-Modify10 100% 0:48

101.00 4.20


IntroductionTheEHRtestedforthisstudywasChiroTouchVersion7.0,acompleteEHR.Thissystemisdesignedtorepresentacompleteandcomprehensivedigitalversionofapatient’spaperchartwithreal-time,patient-centeredrecordsandpresentmedicalinformationtohealthcareprovidersinanambulatorysetting.ThissystemcomplieswithallONCcertificationcriteriaandisbeingsubmittedforverificationofcompliancewith2015certificationcriteria.Theusabilitytestingwasconductedwitheveryefforttorepresentrealisticexercisesandconditions.

Thepurposeofthisstudywastotestandvalidatetheusabilityofthecurrentuserinterface,andprovideevidenceofusabilityintheChiroTouchEHRsoftware.Tothisend,measureofeffectivenessandefficiency(timetoperformtasks;totalnumberofdeviations;totalnumberoferrors;etc.)werecapturedduringtheusabilitytesting.

Method

ParticipantsAtotalof10participantsweretestedontheChiroTouchEHRSoftware.Participantsinthetestwereindividualsthatworkwithinanambulatoryhealthcareenvironment.ParticipantswerecontactedbyChiroTouchdirectlyandfromadatabaseofmedicalprofessionalsmaintainedbythetestingorganization(TheUsabilityPeople,LLC).Inaddition,participantshadnodirectconnectiontothedevelopmentofororganizationproducingtheChiroTouchEHRSoftware.ParticipantswerenotfromIntegratedPracticeSolutions,Inc.Participantsweregiventheopportunitytohavethesameorientationandleveloftrainingastheactualenduserswouldhavereceived.

Forthetestpurposes,end-usercharacteristicswereidentifiedandtranslatedintoarecruitmentscreenerusedtosolicitpotentialparticipants.AnexampleofascreenerisprovidedinAppendix1.

Recruitedparticipantshadamixofbackgroundsanddemographiccharacteristicsconformingtotherecruitmentscreener.Thefollowingisatableofparticipantsbycharacteristics,includingdemographics,professionalexperience,userrole,userneedsforassistivetechnology,andproductexperience.ParticipantnameswerereplacedwithParticipantIDssothatanindividual’sdatacannotbetiedbacktoindividualidentities.

ParticipantID

Age Education Gender ProfessionalExperience

AssistiveTechnologyNeeds

UserRole ComputerExperience(years)

ProductExperience(Years)

1 P0101 50to59

Bachelor’sdegree Female 20

None Officemanager

353

2 P010250to59

Somecollegecredit,nodegree Female 30

None

Officemanager

20

23 P0103

40to49

Tradetechnicalvocationaltraining Female 22

None

Officemanager

15

44 P0104 50to

59Doctorate

degree(e.g., Female 35None Chiropractor

154


Atotaloftwelve12participantsrecruitedforthetestshoweduptoparticipateinthetest.Two(2)participantsfailedtoshowforthestudy.

Participantswerescheduledforsixty(60)toninety(90)minutesessionswithuptothirty(30)minutesinbetweeneachsessionfordebriefbytheadministrator(s)anddatalogger(s),andtoresetsystemstopropertestconditions.Aspreadsheetwasusedtokeeptrackoftheparticipantschedule,andincludedeachparticipant’sdemographiccharacteristicsasprovidedbythetestingfirm.

StudyDesignOverall,theobjectiveofthistestwastouncoverareaswheretheapplicationperformedeffectively,efficiently,andwithsatisfaction,andareaswheretheapplicationfailedtomeettheneedsoftheparticipants.ThedatafromthistestmayserveasabaselineforfuturetestswithanupdatedversionofthesameEHRand/orcomparisonwithotherEHRsprovidedthesametasksareused.Inshort,thistestingservesasbothameanstorecordorbenchmarkcurrentusability,butalsotoidentifyareaswhereimprovementsmustbemade.

Duringtheusabilitytest,participantsinteractedwithone(1)EHR.Eachparticipantwasprovidedthesamesetofinstructions.Thesystemwasevaluatedforeffectivenessandefficiencyasdefinedbymeasurescollectedandanalyzedforeachparticipant.

• Timetocompletethetask• Numberandtypesoferrors• Pathdeviations• Participantsverbalizations• Taskscompletedintheallottedtime• Participant’ssatisfactionratingsofthesystem• Numberoftaskscompletedsuccessfullywithintheallottedtimewithoutassistance

MD,DNP,DMD,PhD)

5 P0105 20to29

Bachelor’sdegree Female 5

None OfficeManager

152

6 P0106 40to49

Associatedegree Female 12

None

ChiropracticAssistant

202

7 P0107

50to59

Doctoratedegree(e.g.,MD,DNP,DMD,PhD) Male 27

None

Owner/chiropractor

15

38 P0108 20to

29 NursingDegree Female 9None RN

150

9 P0109

60to69

Highschoolgraduate,

diplomaortheequivalent Female 7

None

Ins.ClerkandCA

17

310 P0110 40to

49MastersDegree Male 4

None HealthITInstructor

150


AnumberoftaskswereconstructedthatwouldberealisticandrepresentativeofthekindsofactivitiesausermightdowiththisEHR,including:

• ModifyPatientDemographicinformation• Addanitemtopatientproblemlist• Removeanitemfrompatientproblemlist• Reconcileandintegrateinformationfromanotherprovider• AddamedicalAllergy• Prescribeadrug/ViewCDS• Prescribeadrugw/oCDSIntervention• CancelaneRX• RenewaneRX• Orderamedication;viewCDS• Stopamedication• Removeanallergy• Createimagingorders• ModifyImagingorder• CreateLaborder• ModifyLaborder• AddanImplantableMedicalDevice• ModifyanImplantableMedicalDevice

Taskswereselectedbasedontheirfrequencyofuse,criticalityoffunction,andthosethatmaybemosttroublesomeforusers.

ProceduresUponcommencementoftest,participantsweregreeted,remotelyloggedin,andtheiridentitywasverified.ParticipantswerethenassignedaparticipantID.Allparticipantssignedaninformedconsentformpriortothetesting(SeeAppendix1).Thetestadministratorprovidedtheinstructionsforeachtest,notedallcommentsfromtheparticipants;whilethedataloggernotedalltimes,pathdeviations,comments,successes,anderrors.

Participantswereinstructedtoperformthetasks:

• Afterlisteningtotheinstructionsfromthetestingadministrator• Asquicklyaspossiblewithasfewdeviationsorerrorsaspossible• Withoutassistance–participantswereallowedtogiveclarificationontasks,butnotguidanceonuse• Withoutusingathinkaloudtechnique

Foreachtask,theparticipantswereshownacopyofthetaskonthescreenandwerefreetoswitchback-and-forthbetweenthetaskandtheEHR.Tasktimingbeganafterthecompletionoftheverbalinstructionsfromtheadministrator;andafteranacknowledgementfromtheparticipantthattheywerereadytobegin.Thetasktimewasstoppedoncetheparticipantindicatedtheyhadsuccessfullycompletedthetask.

Followingthetest,theadministratorgavetheparticipantthepost-testquestionnaire;andthenthankedthemfortheirtime.Participants'demographicinformation,tasksuccessrate,timeontask,errors,deviations,verbalresponses,andpost-testquestionnairewererecordedintoaspreadsheet.


TheUsabilityPeople,LLCstaffmemberswereresponsibleforloggingdatarecordedallparticipants’demographicinformation,tasksuccessrate,timeontask,errors,comments,anddeviationsintoaspreadsheet.

TestLocationThetestwasadministeredinaremotesettingwhereparticipantswereisolatedfromotherparticipantsinthestudy.Onlythetestadministratorandloggerwereonthecallwiththeparticipantswhilethestudywasbeingadministered.

TestEnvironmentTheChiroTouchEHRSoftwarewouldbetypicallybeusedinaprivatepractice,andinthisinstancethetestingwasconductedremotelyineachparticipant’sactualpractice.Althoughthetypeofcomputer,operatingsystemanddisplayresolutionoftheremoteparticipantsystemwasunknown,thesystemthatwasusedbythetestadministratorandcontrolledbytheremoteparticipantwasaDellInspironLaptoprunningtheWindows10operatingsystemataresolutionof1366x768pixels.UsersalsousedamouseandkeyboardwhileinteractingwiththeEHR.

TheChiroTouchapplicationisaprimarilyclient-serversolutionwithsomewebcomponents;andallcomputerswererunningonhighspeedinternetconnections.Allparticipantsindicatedthatsystemperformanceduringthetestwaswhattheywereusedtoseeingduringtheirtypicalworkday.Additionally,participantswereinstructednottochangeanyofthedefaultsystemsettings(suchascontroloffontsize).

TestFormsandToolsDuringtheusabilitytest,variousinstrumentsanddocumentswereused,including:

• Informedconsent• ModeratorGuide• Post-testquestionnaire• Study

ExamplesofthesedocumentscanbelocatedintheAppendixsection.Theparticipant’sinteractionwiththeChiroTouchEHRSoftwarewascapturedandrecordeddigitallywithscreencapturesoftwarerunningonthemoderator’smachine.

ParticipantInstructionsTheAdministratorreadthefollowinginstructionsaloudtoeachparticipant:

“Thankyouforparticipatingintoday’susabilitystudyofChiroTouch.Inafewminutes,youwillbeaskedtoperformaseriesoftasksandcompleteausersurvey.Pleaseattempttocompletethesetasksasquicklyaspossible.ThepurposeofthisstudyisforChiroTouchtoobtaininformationonwhereenhancementsareneededintheapplicationbasedonhowquicklyandeasilythetaskisbeingperformedintheChiroTouchEHRsystem.

Pleasenotethatwearenottestingyou;wearetestingthesystem.Thereforeifyouhaveanydifficultythismaymeanthatsomethingneedstobeimprovedinthesystem.Iwillbehereincaseyouneedspecifichelp,butIamnotabletoinstructyouorprovidehelpinhowtousetheapplication.

Overall,weareinterestedinhoweasy(orhowdifficult)thissystemistouse,whatinitwouldbeusefultoyou,andhowwecouldimproveit.Ididnothaveanyinvolvementinitscreation,sopleasebehonestwithyouropinions.


Alloftheinformationthatyouprovidewillbekeptconfidentialandyournamewillnotbeassociatedwithyourcommentsatanytime.Shouldyoufeelitnecessaryyoumaywithdrawatanytimeduringtesting.”

Participantswerethengiven18taskstocomplete.TasksarelistedintheDataAppendix3.


UsabilityMetricsAccordingtotheNISTGuidetotheProcessesApproachforImprovingtheUsabilityofElectronicHealthRecords,EHRsshouldsupportaprocessthatprovidesahighlevelofusabilityforallusers.Thegoalisforuserstointeractwiththesystemeffectively,efficiently,andwithanacceptablelevelofsatisfaction.Tothisend,metricsforeffectiveness,efficiencyandusersatisfactionwerecapturedduringtheusabilitytesting.

Thegoalsofthistestweretoassess:

1. TheefficiencyofChiroTouchbymeasuringthelengthoftimeittakesforausertocompletethetask;andthesuccessoftaskcompletion.

2. TheefficiencyofChiroTouchbymeasuringthepathdeviationstakenbytheuserduringthetask.3. TheefficiencyoftheChiroTouchbymeasuringtheaveragetasktimeandpathdeviations.4. TheeffectivenessofChiroTouchbymeasuringthenumberandtypesoferrorsexperiencedbytheuserduringthetask.5. ThesatisfactionoftheuserwithChiroTouchbyloggingtheircommentsonthetask.

DataScoringThetablebelowdetailshoweachtaskwasscored.

Measure RationaleandScoring

TaskTime Timingstartedwhentheadministratorsaid‘Begin’.Thetimeendedwhentheparticipantsaid‘Done’.Intheeventthattheparticipantfinished,anddidnotsay‘Done’,theadministratorstoppedtheclockwhenitwascleartheparticipanthadcompletedthetask.Tasktimewasonlycountediftheparticipantcompletedthetaskintheallottedtime.Averagetimepertaskwascalculatedforeachtask.

Errors Thetaskresultedinanerroriftheparticipant:failedtofinishthetaskintheallottedtime;or,iftheybecame‘stuck’andcouldnotproceedwithoutaskingforassistance.Tasktimewasnotcountedwhenthetaskresultedinanerror.

PathDeviations Pathdeviationswererecordedasactionstakenduringthetaskthatwerenotpartofthenecessaryactionsneededtocompletethetask.

Wecalculatedpathdeviationsbytakingthetotalnumberofobserveddeviationsanddividingthatnumberbythetotalnumberofstepstakenusinganoptimalpathtoprovidearatioofpathdeviation.

TaskSuccess Thetaskwasconsideredasuccessiftheparticipantcompletedthetaskintheallottedtimewithoutassistance.

TaskRating Participant’ssubjectiveimpressionoftheeaseofusewasmeasuredbyadministeringapost-sessionquestionnaireandapost-taskquestion.Theparticipantswereaskedtoratetheoveralleaseofeachtask on a scale of 1 (Very Difficult) to 5 (Very Easy) To measure participants’ confidence in andlikeabilityoftheoverallsystem,thetestingteamadministeredapost-testquestionnaire.Questionsincludedthoseaddressingtheeaseofuseofthesystemandparticipants’thoughtsonhowquicklyausercouldlearnthesystem.SeefullquestionnaireinAppendix2.


Results

DataAnalysisandReportingTheresultsoftheusabilitytestwerecalculatedaccordingtothemethodsspecifiedintheUsabilityMetricssectionabove.

AsummaryofthetestingresultsforChiroToucharedetailedbelow.Thetablebelowidentifiesthetasksperformedandtheperformancelevelandsatisfactionforeachtask.

Task MeanTaskTime

SD

CompletionRate(%)

Mean#PathDeviations

SD

MeanTaskSatisfaction

SD

Task1DemographicInformation 1:49 0:54 100% 0.60 0.66 4.20 0.98

Task2ProblemListView/Intervention 0:42 0:10 100% 1.10 0.70 4.20 0.75

Task3ProblemListIntervention-Remove 0:57 0:37 100% 0.40 0.92 3.40 1.20

Task4ClinicalInfoRecon/Incorporation 1:49 1:01 70% 1.90 1.81 3.10 1.45

Task5MedicationAllergy-Add 1:33 0:40 90% 1.70 1.62 3.50 1.02

Task6Drug-AllergyInteractionCheck 1:02 0:21 100% 0.50 0.81 3.50 1.20

Task7ElectronicPrescribe 1:49 0:26 100% 0.20 0.40 3.80 1.08

Task8eRXCancel/Change 2:46 0:40 90% 1.10 1.87 3.90 0.94

Task9eRX-Renew 0:45 0:42 100% 0.40 0.92 4.60 0.92

Task10MedOrder;Drug-DrugIntervention 1:39 0:22 100% 0.10 0.30 4.30 0.64

Task11Medicationlist-Stop 0:21 0:07 100% 0.00 0.00 4.80 0.40

Task12MedAllergyList-Remove 0:27 0:12 100% 0.00 0.00 4.70 0.64

Task13CPOEDiagImaging 1:44 0:28 100% 0.10 0.30 4.10 0.83

Task14CPOEDiagImaging-Modify 1:18 0:21 100% 0.60 0.66 4.30 0.64

Task15CPOELabOrders 1:36 0:43 100% 0.10 0.30 4.20 0.75

Task16CPOELabOrders-Modify 0:59 0:15 100% 0.30 0.64 4.70 0.46

Task17ImplantableDevice-Add 0:59 0:24 100% 0.90 0.83 4.20 0.60

Task18ImplantableDevice-Modify 0:48 0:20 100% 1.00 1.18 4.20 0.75

Asthetableaboveshows,relativetooptimalperformancestandardsasdefinedbyChiroTouchandTheUsabilityPeople,

participantperformanceintheChiroTouch EHRusabilitytestwasquitesatisfactory.Theoverallaveragetaskcompletionrate

wasninety-seven(97)percent.


ReliabilityDuringtheentiredatacollectionphase,itwasobservedthatthesystemprovidedaconsistentandreliableinterfacetoeachparticipantastheycompletedtheirtasks.Aseachparticipantcompletedtheirassignedtasks,thesystemprovidedthesameinformationandrespondedtotheirinputwiththesameverbiageandusingthesamemodeofcommunication(e.g.Pop-upmessage,orembeddedassistance).

RiskAssessmentThefollowingtablepresentsaprioritizedlistoftasks(alongwithadescriptionofthetaskandanidentificationofthecriteria

evaluated)prioritizedbytheriskoferrorasobservedduringthetesting.

TaskCriteria

Evaluated DescriptionPercentComplete RiskStatus

Task1DemographicInformation a(5) ModifyPatientDemographicinformation 100% None

Task2ProblemListView/Intervention a(6) Addanitemtopatientproblemlist 100% None

Task3ProblemListIntervention-Remove a(6) Removeanitemfrompatientproblemlist 100% Low

Task4ClinicalInfoRecon/Incorporation b(2)Reconcileandintegrateinformationfromanotherprovider 70% Moderate

Task5MedicationAllergy-Add a(8) AddamedicalAllergy 90% Low

Task6Drug-AllergyInteractionCheck a(1);a(9) Prescribeadrug/ViewCDS 100% None

Task7ElectronicPrescribeb(3);a(4);

a(9) Prescribeadrugw/oCDSIntervention 100% None

Task8eRXCancel/Change b(3) CancelaneRX 90% Low

Task9eRX-Renew b(3) RenewaneRX 100% None

Task10MedOrder;Drug-DrugInterventiona(7);a4;a(9) Orderamedication;viewCDS 100% None

Task11Medicationlist-Stop a(7) Stopamedication 100% None

Task12MedAllergyList-Remove a(8) Removeanallergy 100% None

Task13CPOEDiagImaging a(3) Createimagingorders 100% None

Task14CPOEDiagImaging-Modify a(3) ModifyImagingorder 100% None

Task15CPOELabOrders a(2) CreateLaborder 100% None

Task16CPOELabOrders-Modify a(2) ModifyLaborder 100% None

Task17ImplantableDevice-Add a(14) AddanImplantableMedicalDevice 100% None

Task18ImplantableDevice-Modify a(14) ModifyanImplantableMedicalDevice 100% None

EffectivenessParticipantsinthestudyexperiencederrorsinallbuttwo(2)tasks.However,thebulkoftheerrorsoccurredinthreeareas:“ClinicalInformationReconciliation”,“AddanAllergy”,and“Cancelanelectronicprescription.”ErrorsfoundintheseareaswereofnosurprisetoChiroTouch.TheClinicalInformationReconciliationapplicationisanewercomponenttoChiroTouchusers.Thisfeature,aswellastheallergyandeRXfeaturesarenotusedtypicallybythethoseintheChiropracticprofession.


EfficiencyParticipantswhosuccessfullycompletedtasksgenerallycompletedthosetaskswithinanacceptabletime.Sometaskswerecompletedmorequicklythanthecalculatedoptimaltime,whileseveraltaskstookslightlylongerthanexpected.Thetasksthattookthelongestrequiredtheparticipantstonavigatetoaparticularunfamiliarportionofapage,interactwithaworkflow,locateandselectspecificactions.

Someparticipantsmadeerrorswhenattemptingtonavigatetowardsolvingtheirassignedtasks.TheseerrorsmaybeassociatedwiththoseparticipantsnotbeingfamiliarandnotunderstandingthepresentedinformationarchitectureoftheEHRsystem.Asnotedabove,priorexperiencewithEHRsystemswasrelatedtosuccessfultaskcompletion.Similarly,experienceandpracticewiththegivensystemmayhavepositiveeffectswithregardtouserefficiency.

SatisfactionIndividualTaskSatisfaction

Participantsverballyindicatedtheirsatisfactionwiththeeaseofuseforeachtaskusingascaleof“1”(“VeryDifficult”)to“5,”

(“VeryEasy”).Asthefigurebelowshowsindividualtasksatisfactionrangedfromalowof3.1outof5onTask4(Clinical

InformationReconciliation)toahighof4.8outof5onTasks11(Medicationlist–Stopamedication).Theoverallaveragetask

satisfactionwas4.09indicatingthatoveralltheparticipantswerequitesatisfiedwiththeirtasks.


SystemUsabilityScale

TheSystemUsabilityScale(SUS)isasimple,10-itemLikert-typeattitudescaleprovidingaglobalsubjectiveassessmentof

usabilityfromtheuser’sperspective(JohnBrookeatDigitalEquipmentCompanydevelopedtheSUSin1986).TheSUSscaleis

scoredfrom0to100;scoresunder60representsystemswithlessthanoptimalusability,scoresover80areconsideredbetter

thanaverage.

ThemeantotalSUSscorefortheChiroTouch EHRwasfifty-three(53)andrangedfromalowofthirty(30)andahighofninety

(68).Overall,participant-usersratedtheirsatisfactionwiththeChiroTouch EHRsystemtobelowerthantheexpectedrange

giventhehighindividualsatisfactionratingsandexcellenttaskperformancedata.

ThefollowinggraphshowstheSUSscorebyeachparticipant:

0.00 1.00 2.00 3.00 4.00 5.00

Task1DemographicInformationTask2ProblemList…Task3ProblemList…Task4ClinicalInfo…

Task5MedicationAllergy- AddTask6Drug-AllergyInteraction…

Task7ElectronicPrescribeTask8eRXCancel/Change

Task9eRX- RenewTask10MedOrder;Drug-Drug…Task11Medicationlist- Stop

Task12MedAllergyList-…Task13CPOEDiagImagingTask14CPOEDiagImaging-…

Task15CPOELabOrdersTask16CPOELabOrders- ModifyTask17ImplantableDevice- Add

Task18ImplantableDevice-…

4.204.20

3.403.103.503.503.803.904.604.304.804.70

4.104.304.204.70

4.204.20

AverageTaskSatisfaction


Onthepost-studyquestionnaire4out8ofexperiencedChirotouchusersexpressedtheywere“VeryorSomewhatSatisfied”withtheChiroTouchsystem.


MajorFindings

Theoveralltaskperformanceforthisstudywasveryhigh,withanoveralltaskcompletionrateofninety-seven(97)percent.Althoughtheperformancedatawasveryhigh,andtheindividualtaskperformanceratingwasalsohigh,thesubjectiveusabilityratingscollectedaspartoftheSystemUsabilityScale(SUS)weremuchlowerthanexpected.Thisseemstoimplythatalthoughthesystemiseasytouse,andthatparticipantsareabletosuccessfullycompletetheirtasks,somethingisnegativelyinfluencingtheiroverallsatisfactionwiththesystem.

AreasforImprovement

Theoveralltaskperformancerateforthistestwasveryhigh,atninety-seven(97)percent.Thisindicatesthatamajorityofthesystemishighlyusableanddoesnotrequireanysignificantimprovement.Therearealways,however,severalareasthatcouldbeenhancedtoachieveevenhighertaskcompletionrates,andperhapscouldincreaseoverallparticipantsatisfaction.

• Indicationoftaskworkflowo Inseveralareasoftheproduct,theusermustnavigateacrossseveraltabsorotheruserinterfaceelementinto

ordertocompletetheirtask.Insteadofpresentingtheseitemsacrosscommontabs,theworkflowcouldbeindicatedusinga1->2->3typearrangementthatwouldservetoguidetheusertocompletionofthetask.

o Inaddition,someembeddedassistancecouldbeprovidedtohelpusersunderstandandcompletetheirtasks.

• Buttons;PrimaryandSecondarybuttonso Acrossthesystemtherewasaninconsistentuseofuseractionbuttons,links,buttonnamesandoccasionally

buttonarrangement.Thesystemcouldbenefitfromprovidingamoreconsistentuseofbuttonsorlinksandperhapsamoresalientindicationofprimaryvssecondaryactionbuttons.

• Betterintegrationof3rdpartytoolso Thesystemtakesadvantageofseveral3rdpartytoolsinordertoachieverequiredfunctionality.Unfortunately,

eachofthesetoolspresentsadifferentuserinterface,andthiscanhavenegativelyimpactedtheperceptionoftheoveralluserexperience.


Appendix1:SampleConsentForm

ChiroTouchUsabilityStudyPracticeConsent

ChiroTouchwouldliketothankyouforconsideringparticipationinthisstudy.ThepurposeofthestudyistoevaluatetheusabilityoftheChiroTouchEHRsystem.YourparticipationinthisstudywillincludeperformingaspecifictaskwithinChiroTouch;andcompletingashortsurveyfollowingthestudy.Thestudyshouldtakeapproximately60-90minutes.TheinformationcollectedbyChiroTouchduringthestudyisforresearchpurposesonly.Yourparticipationinthisstudyisvoluntary,soyouarefreetowithdrawatanypointduringthestudy.

Pleaseanswerthefollowingscreeningquestions:

YesNo

☐☐ Doyou,oranyoneinyourhome,haveacommercialorresearchinterestinanEHRsoftwareorconsultingcompany?☐☐ Doyou,ordoesanyoneinyourhome,workinmarketingresearch,usabilityresearch,webdesign[…etc.]?☐☐ Doyourequireanyassistivetechnologiestouseacomputer?☐☐ Areyouactivelypracticingasalicensedmedicalproviderwithinyourclinic?

NameofPractice

NameofParticipant

Age ☐23to39|☐40to59|☐60-to74|☐75andolder

Gender ☐Male☐Female☐Other

YearsinPractice(ifDC)

Position ☐Provider☐Other ☐FrontOffice☐BackDesk/ProviderAssistants

Experiencew/ChiroTouch(Yrs)

Location

Bysigningbelow,Iagreetoparticipateinthestudy.


SignatureDate PrintedNameDate

Appendix2:SamplePost-TestQuestionnaire

ChiroTouchUsabilityStudyPost-TestQuestionnaire

1. Howwouldyoupreferwecommunicatenewenhancementstoyourpractice?�Email �MembersAreaPage �Newsletter �Wedonotcaretobenotified

2. WhatisyourpreferredmethodofworkingwiththeChiroTouchSupportdepartment?�Phone�Email �OnlineChat(ifthiswereavailable)

3. Whatisyourpreferredmethodforcreatingaprescription?�DrugSearch �Favorites

4. HowlongdoesittaketoenteranewpatientinChiroTouch(Demographicdataonly)?�0-3minutes �3-5minutes �>5Minutes

5. IfyoucouldchangeonepartofChiroTouch,whatwouldyouchange

6. DoyoufeelthattheChiroTouchsystemmakesreportingandmaintainingcomplianceeasy?

7. Whatcompliancefeature(s)wouldyouaddtotheChiroTouchreportingsystemtomakeiteasierforyoutoreport?

8. Onascaleof1-5,howwouldrateyouroverallsatisfactionwithChiroTouch?�1(VeryDissatisfied) �2(SomewhatDissatisfied) �3 �4(SomewhatSatisfied) �5(VerySatisfied)

9. WhichofthefollowingfunctionsareavailablethroughChiroTouch?(Youmaycheckmorethan1option)�PatientEducationMaterials�ElectronicallyPrescribeControlledSubstances�PrescriptionCouponsfordistributingtoPatients�Secureemailmessaging

10. DidyoufindtheChiroTouchsystemoranypartofitunnecessarilycomplexorcumbersome?Pleaseexplain.

11. IfyoucouldaddonepieceoffunctionalitytoChiroTouch,whatwouldyouadd?


Appendix3Data

1. TasksPerformedDuringStudyPreparation Task DesignatedTaskTime OptimalPaths

Task1DemographicInformation

ModifyPatientDemographicinformation

120(seconds)7

Task2ProblemListView/Intervention

Addanitemtopatientproblemlist 30(seconds)

5Task3ProblemListIntervention-Remove

Removeanitemfrompatientproblemlist

30(seconds)

2Task4ClinicalInfoRecon/Incorporation

Reconcileandintegrateinformationfromanotherprovider

180(seconds)

11Task5MedicationAllergy–Add

AddamedicalAllergy 60(seconds)9

Task6Drug-AllergyInteractionCheck

Prescribeadrug/ViewCDS 120(seconds)

5Task7ElectronicPrescribe Prescribeadrugw/oCDSIntervention 180(seconds) 6Task8eRXCancel/Change CancelaneRX 180(seconds) 12Task9eRX-Renew RenewaneRX 30(seconds) 2Task10MedOrder;Drug-DrugIntervention

Orderamedication;viewCDS 60(seconds)6

Task11Medicationlist-Stop Stopamedication 30(seconds) 2Task12MedAllergyList–Remove

Removeanallergy 30(seconds)2

Task13CPOEDiagImaging Createimagingorders 120(seconds) 7Task14CPOEDiagImaging-Modify

ModifyImagingorder 60(seconds)

6Task15CPOELabOrders CreateLaborder 60(seconds) 7Task16CPOELabOrders–Modify

ModifyLaborder 60(seconds)

6Task17ImplantableDevice–Add

AddanImplantableMedicalDevice 60(seconds)

5Task18ImplantableDevice–Modify

ModifyanImplantableMedicalDevice 30(seconds)4


2. ParticipantsDemographicsGender

Male 2Female 8Total 10

ChiroTouchUserRoleProvider 2BackDesk/Prov.Asst. 1FrontDesk 5Other 2

YearsofExperiencew/ChiroTouch<1 2<2 3>2 5

YearsinPractice<1 02-5 25-10 2>10 6


Appendix4:ModeratorGuide

ChiroTouchUsabilityStudyModeratorGuide

Prior to testing § Confirm schedule with Participants § Ensure the client’s ChiroTouch software is running properly

Prior to each participant:

§ Reset application § Start session recordings with Morae

Prior to each task:

§ Reset application to starting point for next task

After each participant: § End session recordings with Morae

After all testing

§ Back up all video and data files

Orientation (5 minutes)

Thank you for participating in this study. Our session today will last 60 to 90 minutes. I will ask you to complete a few tasks using this system and answer some questions. We are interested in how easy (or how difficult) this system is to use, what in it would be useful to you, and how we could improve it. You will be asked to complete these tasks on your own trying to do them as quickly as possible with the fewest possible errors or deviations. Do not do anything more than asked. If you get lost or have difficulty I cannot help you with anything to do with the system itself. Please save your detailed comments until the end of a task or the end of the session when we can discuss freely.

I did not have any involvement in its creation, so please be honest with your opinions.

The product you will be using today is describe the state of the application, i.e., production version, early prototype, etc. Some of the data may not make sense as it is placeholder data.

We are recording our session today. All of the information that you provide will be kept confidential and your name will not be associated with your comments at any time.

Do you have any questions or concerns?


TASK1:DemographicInformation (120 Seconds)

Direct the participant to the starting point for the task.

Yourpatient,KerryKennedy,hasarrivedfortheirscheduledappointment.Beforebeginningtheirtreatment,youneedtoverifyorentersomeofthedemographicinformationthatisstoredwithinthesystem.NavigatetotheDemographicinformationsectionfromtheFrontDeskapplicationandmakesurethatthefollowinginformationisstoredintheEHR:

Asyoureviewthepatienthistory,theymentionthattheyprefertospeakSpanishasherfirstlanguageandsheisnotascomfortablereadingEnglish.

• SetthepatientLanguageto‘Spanish’• SetthepatientRaceas‘African-American’• SetthepatientEthnicityas‘HispanicorLatino’

Asyoucontinuethroughtheirsocialhistory,theyinformyouthatwhilehersexassignedatbirthwasmale,shehaslegallychangedhernametoRandyandnowidentifiesasafemale.Documentthefollowinginformationinthechart.

• SexualOrientationas'Choosenottodisclose'• IndicatetheGenderIdentityas‘Male-to-Female(MTF)transsexual'

EnterandsavethisinformationintotheEHR.

Success:

¨ Easilycompleted¨ Completedwithdifficultyorhelp::Describebelow¨ NotcompletedComments:

Task Time: Seconds

Comments:

Observed Errors and Verbalizations:

Rating: Overall, this task was:

Ask participant to rate the task from : “Very Difficult” (1) to “Very Easy” (5)


Administrator / Notetaker Comments:


TASK2:ProblemListViewandIntervention. (30 Seconds)


Yourassistanthasmeasuredthepatient’sbloodpressureandrecordedthefollowingvalues:150/90.Consideringtheseresults,youhavediagnosedyourpatientwiththefollowing:

401.9-Hypertension

Success: ¨ Easilycompleted¨ Completedwithdifficultyorhelp::Describebelow¨ NotcompletedComments:

Task Time: Seconds

Comments:






TASK3:ProblemListIntervention--Remove. (30 Seconds)


Youalsonoticethattheyhavealisteddiagnosisof“(M54.5)LowBackPain“ofwhichtheyclaimthattheynolongersufferfrom.

• Removethe“(M54.5)LowerBackPain“diagnosis.”


Task Time: Seconds

Comments:






TASK4:ClinicalInformationReconciliation/Incorporation (180 Seconds)


ThepatienthascometoyourofficewithaCCDAdocumentwithmedicalinformationfromanotherprovider.Thisinformationhasalreadybeenpre-loadedintotheChiroTouchsystem.UsingtheReconciliationfeaturesforCCDAlocatedonthepatientsearchscreenonthePatientsapplicationpleasedothefollowing:

• AccesstheCCDDocumentforyourpatient• ReconciletheMedications• ReconciletheMedicationAllergies• ReconciletheProblems.



Task Time: Seconds

Comments:






TASK5:MedicationAllergyList-Addanallergy.(60 Seconds)


Asyoubeginyourconsultationthepatienttellsyouthattheyrememberthataftertakingamedicationthatcontained“sulfa”awhilebacktheyhadexperiencedsevereurticaria(hives).

• AddanallergyforSulfadixineorSulfa(SulfonamideAntibiotics)totheirrecord.o Youmayseeawarningmessage.



Task Time: Seconds

Comments:






TASK6:Drug-AllergyInteractionCheck.(120 Seconds)


Yourpatientsaidthatthemainreasontheycameintoyourofficeisthattheystartedtofeelpoorlyaftereatingasaladbowlfromalocalfast-foodMexicanrestaurantchain.Youthinkthattheymayhavecontractedane-Coliinfection.

YoudecidetoprescribeKeflex(cephalexin)capsule250MG.However,onaddingKeflexyounoticeadrug-allergyinteractionalertsoyoudecidetostopthisprescription.

EntertheinformationaboveintotheEHRandidentify/viewthedrug-allergyalert.


Task Time: Seconds

Comments:






TASK7:ElectronicPrescribing. (180 Seconds)


Youstillneedtoprescribeamedicationtowhichthepatientisnotallergic.YoudecidetoorderDoxycyclineinstead.

Pleaseprescribe:

• DoxycyclineHyclate20MGtabletstobetakenonetablettwiceadayaftermealsfor5days.

• SendtheRXtothechosenpharmacyusingthefollowingSIG:1234


Task Time: Seconds

Comments:






Task8.ElectronicPrescribingCancel/Change. (180 Seconds)


Oops,youjustrealizedthatyouprescribedandsentelectronicallytothepharmacytheantibioticaboveat20mgdose,butyoureallywantedtoprescribethe100mgDoxycyclinetablet.Youneedtomakemodificationssuchthatthepharmacydispensesthecorrectdosage,andmakesurethepatientsmedicationlistisalsocorrect.

• Navigatetoareportoftheprescriptionsthatweresentelectronicallyandcancelthe20mgtablet.• PrescribeDoxycyclineHyclate100MGtabletstobetakenonetablettwiceadayaftermealsfor5days.

• MakesuretheMedicationlistreflectsthischange.


Task Time: Seconds

Comments:






Task9.ElectronicPrescribing–renew. (30 Seconds)


YoupatienthasbeentakingLipitor(atorvastatin)forseveralyearswithoutanyproblemsandithasbeendoingagoodjobcontrollingtheircholesterollevels.Theyhaveaskedthatyourenewtheircurrentprescription.

• ElectronicallyrenewtheexistingRXforLipitor(atorvastatin)


Task Time: Seconds

Comments:






TASK10:MedicationOrder,ViewDrug-DrugInteractions,MedicationListInterventions (60 Seconds)


SeveraltasksagoyoudiagnosedyourpatientwithHypertension.Inordertotreatyourpatient’selevatedbloodpressure,youdecidetoaddAccupriltotheirhypertensiondiagnosis.

Enteraprescriptionfor:

• AccuprilOralTablet10MGtotakeonetablettwiceadayfor30days.


Task Time: Seconds

Comments:






Task11:Stopamedication. (30 Seconds)


Uponexaminationofthepatient’scurrentmedicationlist,younoticethatthedrugNaproxenislistedasanactivemedication.YoumayalsonoticethatthisdrugisassociatedwithareactiontoapatientAllergy.YouneedtostopthepatientfromtakingNaproxenandindicatethisstopintheirrecord.

• EnteramedicationstopforNaproxen.


Task Time: Seconds

Comments:






TASK12:MedicationAllergyList-Removeanallergy. (30 Seconds)


Duringtheconsultation,yourpatienthastoldyouthattheynolongerhaveanallergytoAmoxicillin.Youdecidetoremovethisallergyfromthepatientrecord.

• RemovetheAmoxicillinallergyfromthepatientrecordintheEHR.


Task Time: Seconds

Comments:






Task13:RecordanoutpatientRadiologyorder. (120 Seconds)


Yourpatienthasbeencomplainingofanklepainafterinjuringthemselveswhileplayingbasketball.Uponfurtherexamination,youdecidetoorderanx-ray,usingtheCPOEfeaturesofthesystem.EnterthefollowingintotheEHR:Diagnosis:ICD-10code:S93.01XASubluxationofRightAnkleJointTest:Radiologicexaminationankle2viewsReview:RequestingPhysicianCraigPalmer


Task Time: Seconds

Comments:






Task14:ModifyoutpatientRadiologyorder. (60 Seconds)


YourPatienthasalsocomplainedofKneepainafterinjuringthemselveswhileplayingbasketball.Youdecidethatyouwanttoaddanadditionalitemtotheradiologyorderthatyouplacedabove.Modifytheorderyourplacedinthetaskabovetoincludethefollowingtest:

• Radiologicexaminationknee3views


Task Time: Seconds

Comments:






TASK15:LaboratoryOrders. (60 Seconds)


Yourpatienthasbothhighbloodpressureandelevatedcholesterollevels.Youwanttoordersomeotherbasiclabworktomakesurethatthepatientdoesn’thaveanyotherunderlyingproblems,suchasakidneyissue.UsingtheCPOEsystempleaseenteranorderforlabworkasfollows:

• Diagnosis:EssentialPrimaryHypertension• Tests:

o Creatinine,24-Hoururineo Fastingglucoseinserumorplasma

EnterthelaborderintotheEHR.


Task Time: Seconds

Comments:






TASK16:LaboratoryOrders–Modify (60 Seconds)


.

Uponreviewofthelaborderthatyoujustcompletedintheprevioustask,youdecidetochangethislaborderandaddanadditionaltest.Addthefollowinglabtesttothepreviousorder.

• Urobilinogen,Semi-Qn

Makethenecessarychangessothattheadditionaltestisalsoordered.


Task Time: Seconds

Comments:






TASK17:ImplantabledevicelistTASK7:ElectronicPrescribing. (60 Seconds)


.

Thepatienttellsyouthatafewyearsbacktheyhada“Calciumcompoundbonevoidfiller”implantedintotheirleftarm.UponexaminationoftheimplantabledevicelistpresentedintheEHR,thereisnorecordofthismedicaldevice.

• AddamedicaldevicewiththefollowingUDIcode.

=/W4146EB0010T0475=,000025=A99971312345600=>014032=}013032&,1000000000000XYZ123(Note:thisinformationisstoredonyourclipboardandsavedasa.txtfileontheWindowsDesktop)

• Parsetheinformationassociatedwiththeimplantedmedicaldeviceandverballystatetheexpirationdate.


Task Time: Seconds

Comments:






TASK18:Implantabledevice-Modify. (30 Seconds)


Theimplanteddevicethatyouenteredintheprevioustaskisexpiredandshouldnolongerclassifiedas“active.”ModifytheinformationfortheBoneVoidFillersothattheDeviceStatusissettoINACTIVE.


Task Time: Seconds

Comments:






Appendix5.SpecificTaskData

ParticipantNumber

Task1DemographicInformation

TaskTime Outcome#Path

DeviationsTask

SatisfactionP0101 1:54 Success 1 4P0102 1:23 Success 0 3P0103 2:08 Success 0 5P0104 2:42 Success 2 2P0105 1:06 Success 0 5P0106 1:45 Success 0 5P0107 1:47 Success 0 5P0108 0:20 Success 1 4P0109 3:48 Success 1 4P0110 1:18 Success 1 5

ExpectedTimeonTask 2:00 (SD) AverageTimeonTask 1:49 0:54 AverageTaskSatisfaction 4.20 0.98 Average#PathDeviations 0.60 0.66 PercentSuccess 100%

ParticipantNumber

Task2ProblemListView/Intervention


DeviationsTask


ExpectedTimeonTask 0:30 (SD)

AverageTimeonTask 0:42 0:10 AverageTaskSatisfaction 4.20 0.75 Average#PathDeviations 1.10 0.70 PercentSuccess 100%


ParticipantNumber

Task3ProblemListIntervention-Remove


DeviationsTask




ParticipantNumber

Task4ClinicalInfoRecon/Incorporation


DeviationsTask

SatisfactionP0101 4:23 Success 3 1P0102 1:50 Success 1 4P0103 3:15 Fail 3 1P0104 2:09 Fail 6 1P0105 1:17 Success 0 4P0106 0:44 Success 0 5P0107 1:16 Success 0 4P0108 1:42 Success 2 3P0109 2:06 Fail 3 4P0110 1:31 Success 1 4




ParticipantNumber

Task5MedicationAllergy-Add


DeviationsTask

SatisfactionP0101 1:20 Success 1 4P0102 1:48 Success 2 3P0103 1:13 Success 1 5P0104 2:29 Success 3 2P0105 0:58 Success 1 5P0106 1:33 Success 1 2P0107 1:38 Success 0 4P0108 1:31 Success 1 3P0109 3:20 FAIL 6 3P0110 1:24 Success 1 4



ParticipantNumber

Task6Drug-AllergyInteractionCheck


DeviationsTask





ParticipantNumber

Task7ElectronicPrescribe


DeviationsTask




ParticipantNumber

Task8eRXCancel/Change


DeviationsTask

SatisfactionP0101 2:28 Success 0 3P0102 3:48 Fail 6 3P0103 2:13 Success 0 5P0104 3:26 Success 0 2P0105 1:43 Success 0 4P0106 3:27 Success 1 4P0107 2:25 Success 0 5P0108 2:23 Success 0 4P0109 3:15 Success 3 5P0110 3:32 Success 1 4




ParticipantNumber

Task9eRX-Renew


DeviationsTask




ParticipantNumber

Task10MedOrder;Drug-DrugIntervention


DeviationsTask





ParticipantNumber

Task11Medicationlist-Stop


DeviationsTask




ParticipantNumber

Task12MedAllergyList-Remove


DeviationsTask





ParticipantNumber

Task13CPOEDiagImaging


DeviationsTask




ParticipantNumber

Task14CPOEDiagImaging-Modify


DeviationsTask


ExpectedTimeonTask 1:00 (SD) AverageTimeonTask 1:18 0:21 AverageTaskSatisfaction 4.30 0.64 Average#PathDeviations 0.60 0.66 PercentSuccess 100%


ParticipantNumber

Task15CPOELabOrders


DeviationsTask




ParticipantNumber

Task16CPOELabOrders-Modify


DeviationsTask





ParticipantNumber

Task17ImplantableDevice-Add


DeviationsTask




ParticipantNumber

Task18ImplantableDevice-Modify


DeviationsTask




EHR Usability Test Report Product: [email protected] Introduction The EHR tested for...

Documents

Transcript of EHR Usability Test Report Product: [email protected] Introduction The EHR tested for...