Amia Pres Oct 26 2011 Final

October 26 th, 2011

Brad Doebbeling, MD, MSc

VA HSR&D Center on Implementi ng Evidence-Based Practi ce

Regenstrief Insti tute

IU School of Medicine, IUPUI School of Informati cs & Purdue University

Development of a Workflow Integration Survey (WIS) for Implementing Computerized Clinical Decision Support

Co-authors:

Mindy Flanagan, PhD (1st author)

Nicole Arbuckle, BA

David A. Haggstrom, MD, MAS

Laura G. Militello, MA

Jason J. Saleem, PhD

Supported by:• Agency for Healthcare Quality and Research (AHRQ)• HSA2902006000131• VA Consortium for Healthcare Informatics Research (CHIR) travel

2

Acknowledgements

• Computerized clinical decision support (CDS) can improve clinical decision making, support adherence to evidence-based guidelines, and ultimately improve quality of care

• Integration of CDS into workflow critical

• Metrics for assessing workflow integration in clinical settings not well established

3

Background

“Benchmark institutions” for CDS (Chaudhry, Ann Intern Med, 2006)• Regenstrief Institute, Partners Healthcare, Veterans Health

Administration, (Intermountain Healthcare)• Early-adopter institutions that implemented internally

developed EHR systems, including CDS•Widely implemented CDS in these institutions• Ideal settings to study integration of CDS in workflow

4

Institution Selection

Part 1: Qualitative field study• Objective: Identify barriers to CDS workflow integration• Direct observation (with opportunistic interviews)• Key Informant Interviews• Focus Groups

Part 2: Survey Development• Objective: Develop survey to assess CDS integration into

workflow• Item generation and revision• Pilot test in simulated patient encounter

5

Methods

11 outpatient clinics from:• 2 VA Medical Centers (VAMCs)• Clinics affiliated with Regenstrief Institute• Partners Healthcare

2-4 observers experienced in ethnographic observation

Participants: •Observation & opportunistic interviews:

-120 providers (physicians, NPs, PAs, nurses) and staff (MAs, clerks)• 11 Key informant interviews• 2 Focus groups: 11 physicians

6

Part 1 Study Sites and Participants

7

VA Computerized Patient Record System

VAMC 1

Qualitative analysis: top-down vs. bottom-up coding

Coding template based on the sociotechnical model•Social, technical, and external subsystems

Findings integrated across sites

Focused on themes that emerged as part of the technical subsystem for survey development

8

Analysis

Navigation•How logically information is organized•How easily information is located

Functionality•Extent to which computer system has tools or

operations available to complete necessary tasks (e.g., order tests)

Usability•How easy or hard it is to use information system

9

Technical Subsystem Emergent Themes

Paper Workarounds•Persistence of paper-based documentation or paper-

based notes when EHR available

Workload•Degree to which the computer system increases effort

(e.g., data entry, number of clicks) to perform necessary actions

10

Technical Subsystem Emergent Themes (cont.)

Generated 19 items for initial scale for five dimensions

3 content experts reviewed items and rated for clarity

Revisions resulted in 15-item scale

Nine primary care physicians completed 15-item scale and provided feedback

Scale revised and the Paper Workaround items removed• Physicians did not routinely use CDS during patient encounter

Workflow Integration Survey consists of 12 items

11

Part 2 Survey Development

Navigation• Patient information is easy to find in CPRS. • Patient information is easily accessed with CPRS.• With CPRS, it is difficult to search for patient information during face-

to-face encounters.*

Functionality• CPRS has all of the functions (e.g., order entry, medication list)

needed to complete face-to-face patient encounters.• CPRS helps you perform the tasks (e.g., order entry, progress notes,

record review) you need to during face-to-face patient encounters.• The same information is entered into CPRS multiple times during

face-to-face patient encounters.*

12

Workflow Integration Survey

Usability• CPRS is challenging to use.*• CPRS is easy to use.• CPRS is frustrating to use.*

Workload• Using CPRS during face-to-face patient encounters adds effort (e.g.,

typing, clicks).*• Using CPRS during face-to-face patient encounters increases workload.*• CPRS helps you complete face-to-face patient encounters efficiently.*Indicates items are reverse-scored; Response set: 1=strongly disagree,

5=strongly agree

13

Workflow Integration Survey (cont.)

New design features for a clinical reminder were developed

Participants tested new design features in simulated patient exams

Within-subject design• Each participant completed 2 patient scenarios X 2 versions of CDS• Counterbalanced order of CDS versions

Participants completed WIS twice (after both patient scenarios for given CDS version complete)

14

Simulation Study

Scores for the Workflow Integration Survey were compared across the two versions of CDS using Wilcoxon signed-rank test

Within-subject comparisons were made between the survey subscales

Statistical tests were two-tailed with a 0.05 level of significance

15

Analysis

PCP Participants -- 10 physicians, 2 NPs, from 5 outpatient clinics)

Excellent internal reliability • for CDS, α = 0.93; for enhanced CDS, α = 0.80

Enhanced CDS version rated significantly higher (better) than current CDS for each subscale (p < 0.05)

16

Results

Workflow Integration Survey assesses multiple dimensions of workflow (specific to healthcare settings)

Initial tests encouraging

Easily adaptable to assessing other specific CDS tools

Further validation (Workaround subscale excluded) needed

17

Conclusions

Haggstrom, D.A., Saleem, J.J., Militello, L.M., Arbuckle, N., Flanagan, M., and Doebbeling, B.N. Examining the Relationship Between Clinical Decision Support and Performance Measurement. AMIA Symposium 2009; 2009:223-227.

Saleem, J.J., Militello, L.M., Arbuckle, N., Flanagan, M., Haggstrom, D.A., and Doebbeling, B.N. Provider Perceptions of Clinical Decision Support at Three Benchmark Institutions. AMIA Symposium 2009; 2009:558-62.

Doebbeling, B.N., Militello, L.G., Flanagan, M., Haggstrom, D., Arbuckle, N.B., Kiess, C.L., Saleem, J.J. Integration of Computerized Decision Support into Clinical Workflow: Investigating Social, Technical and Contextual Factors, JAMIA (In revision), 2011.

Saleem, J.J., Haggstrom, D.E., Militello, L.G., Flanagan, M.E., Arbuckle, C.L., Doebbeling BN. Redesign of a Computerized Clinical Reminder for Colorectal Cancer Screening: A Human-Computer Interaction Evaluation. BMC Medical Informatics and Decision Making, (In revision), 2011.

Militello, L.G., Arbuckle, N.B., Saleem, J.J., Patterson, E, Flanagan, M., Haggstrom, D., Doebbeling, B.N. Sources of Variation in Clinical Workflow: Implications for the Design of Cognitive Support, Applied Clinical Inform. (In review), 2011.

18

Publications from Study

Mindy Flanagan, [email protected]

Nicole Arbuckle, [email protected]

Jason Saleem, [email protected]

Brad Doebbeling, [email protected]

19

Thank-you!

mailto:[email protected]




VA Medical Center 1• Computerized clinical reminder for CRC screening

VA Medical Center 2• Set of computerized clinical reminders for screening, follow-up, and surveillance

Regenstrief Institute• Paper encounter form reminder for CRC screening

Partners Healthcare• Electronic, template health maintenance list

20

Form of Colorectal Cancer Screening CDS at Study Sites





21


Printed paper encounter form with clinical reminders at the bottom.

Paper reminders automatically generated by the CDS rules.

* Annual FOBT and periodic sigmoidoscopy are recommended for all persons aged 50 or over to screen for colorectal cancer. If screening FOBT is positive, colonoscopy is recommended.

* HEMOCCULT 1)Pt refused 2) Done Today (results: ___________________)

22

Regenstrief Medical Record System





23


24

Partners – Longitudinal Medical Record

Amia Pres Oct 26 2011 Final

Documents

Transcript of Amia Pres Oct 26 2011 Final