Impact of the Institute for Impact of the Institute for Healthcare Improvement Healthcare Improvement

Global Trigger Tool Compared Global Trigger Tool Compared to Current Adverse Event to Current Adverse Event

Monitoring Systems at a Large Monitoring Systems at a Large Teaching HospitalTeaching Hospital

Tania Vila, PharmDTania Vila, PharmDDrug Information Specialty Drug Information Specialty

ResidentResidentDuke University HospitalDuke University Hospital

Co-Investigators: Heidi Cozart, R.Ph.; Lynn Eschenbacher, PharmD, MBA;

Diana Brown, RN; William Richardson, MD

Background: Adverse Event Background: Adverse Event Detection MethodsDetection Methods

44,000 to 98,000 deaths per year 44,000 to 98,000 deaths per year occur in U.S. hospitals as a result of occur in U.S. hospitals as a result of errorserrors

Three-pronged approach to adverse Three-pronged approach to adverse event monitoringevent monitoring Voluntary ReportingVoluntary Reporting Computerized MonitoringComputerized Monitoring Global Trigger Tool – New methodologyGlobal Trigger Tool – New methodology

Kohn LT, Corrigan JM, Donaldson MS, editors. To err is human: building a safer health system. Washington, DC: National Academy Press; 2000.

Background: Adverse Event Background: Adverse Event Detection MethodsDetection Methods

Voluntary ReportingVoluntary Reporting Qualitative data informs and guides safety Qualitative data informs and guides safety

and quality (sentinel events, near misses)and quality (sentinel events, near misses) First-hand account of event from reportersFirst-hand account of event from reporters Anonymous & accessible to all hospital Anonymous & accessible to all hospital

employees (available online)employees (available online) Under-reportingUnder-reporting

Computerized MonitoringComputerized Monitoring Automated surveillance for possible ADEs Automated surveillance for possible ADEs

based on logic-based rules followed by based on logic-based rules followed by chart reviewchart review Scans demographic, laboratory, pharmacy Scans demographic, laboratory, pharmacy

system, and other clinical databasessystem, and other clinical databases Quantitative data allows for trendingQuantitative data allows for trending Standardized scoring system for severity Standardized scoring system for severity

and causality (Naranjo Scale)and causality (Naranjo Scale) Only high risk medication rules are Only high risk medication rules are

evaluatedevaluatedKillbridge P, Classen. Surveillance for adverse drug events: history, methods and current Killbridge P, Classen. Surveillance for adverse drug events: history, methods and current issues. VHA’s 2002 Research series;3:1-44.issues. VHA’s 2002 Research series;3:1-44.

Background: Adverse Event Background: Adverse Event Detection MethodsDetection Methods

Example of Computerized Example of Computerized Monitoring (ADE-S) RulesMonitoring (ADE-S) Rules

Antidotes or combinations of Antidotes or combinations of medications and laboratory valuesmedications and laboratory values Naloxone IVNaloxone IV Dextrose 50% IV Dextrose 50% IV ANDAND low blood glucose low blood glucose

(<50 mg/dL)(<50 mg/dL) 2 consecutive aPTT values > 100 2 consecutive aPTT values > 100 OROR one one

>150 seconds in patient receiving >150 seconds in patient receiving heparinheparin

IHI Global Trigger ToolIHI Global Trigger Tool™™

New methodologyNew methodology Trigger-based manual chart reviewTrigger-based manual chart review

Review team minimum of 3 peopleReview team minimum of 3 people 20 charts per month recommended20 charts per month recommended 20 minutes/chart20 minutes/chart

Less resource intensiveLess resource intensive

Griffin FA, Resar RK. IHI Global Trigger Tool for Measuring Adverse Events. IHI Innovation Series white paper. Cambridge, MA: Institute for Healthcare Improvement; 2007.

Background: Adverse Event Background: Adverse Event Detection MethodsDetection Methods

Purpose: Purpose: Primary ObjectivePrimary Objective

To determine the frequency and To determine the frequency and types of adverse events in urology types of adverse events in urology and orthopedics services as and orthopedics services as identified by the IHI Global Trigger identified by the IHI Global Trigger ToolTool™™ Frequency defined as follows:Frequency defined as follows:

Adverse events/1,000 patient daysAdverse events/1,000 patient days

Purpose:Purpose:Secondary ObjectivesSecondary Objectives

To compare the frequency and To compare the frequency and characteristics of AEs detected by characteristics of AEs detected by the IHI Global Trigger Toolthe IHI Global Trigger Tool™™, , computerized monitoring, and the computerized monitoring, and the voluntary reporting systemvoluntary reporting system

To identify potential quality To identify potential quality improvement opportunities and to improvement opportunities and to recommend enhancements to the recommend enhancements to the computerized monitoring system computerized monitoring system based on resultsbased on results

DesignDesign Retrospective chart review of patients Retrospective chart review of patients

admitted between January 1, 2007 and admitted between January 1, 2007 and June 30, 2007June 30, 2007

Inclusion CriteriaInclusion Criteria ≥≥18 years18 years Patients admitted to urology or orthopedicsPatients admitted to urology or orthopedics Chart must be closed and completeChart must be closed and complete

Approved by the Duke University Approved by the Duke University Institutional Review BoardInstitutional Review Board

Methods: IHI Global Trigger Methods: IHI Global Trigger Tool™Tool™

120 patients admitted to urology and 120 patients admitted to urology and orthopedic services were randomly selected orthopedic services were randomly selected (5/service) in 2 week blocks(5/service) in 2 week blocks

Training: Nurse, pharmacist, and physician Training: Nurse, pharmacist, and physician completed IHI training processcompleted IHI training process Phase I: IHI standardized practice charts with keyPhase I: IHI standardized practice charts with key Phase II: IHI trigger tool with Duke-specific chartsPhase II: IHI trigger tool with Duke-specific charts

Data Collection: Nurse and pharmacist Data Collection: Nurse and pharmacist reviewed each chart; MD reviewed reviewed each chart; MD reviewed discordant casesdiscordant cases

IHI Chart Review Process Data Collection Process

Discharge summary Laboratory results Physician orders/MAR Nursing notes/flow sheets and progress

notes Documentation

All triggers documented whether or not an All triggers documented whether or not an adverse event occurredadverse event occurred

Event severity was scored according to an Event severity was scored according to an internal scale and a nationally validated internal scale and a nationally validated scalescale

Methods: Severity ScoresMethods: Severity Scores Severity Index, (internal scale; 0 through 6)Severity Index, (internal scale; 0 through 6)

Only 3 through 6 represent patient harmOnly 3 through 6 represent patient harm

33 Transient adverse patient effects occurred which Transient adverse patient effects occurred which required some corrective therapy, increased length required some corrective therapy, increased length of stay (LOS) 1-2 days or resulted in lab values, of stay (LOS) 1-2 days or resulted in lab values, vital signs or medication effects outside the vital signs or medication effects outside the desirable parametersdesirable parameters

44 Significant adverse patient effects occurred which Significant adverse patient effects occurred which required aggressive intervention such as code, required aggressive intervention such as code, intubation, transfer to ICU, antidote, interventional intubation, transfer to ICU, antidote, interventional drug therapy or increased LOS > 2 daysdrug therapy or increased LOS > 2 days

55 Permanent adverse patient effects occurred such Permanent adverse patient effects occurred such as paralysis, brain damage, disability or loss of as paralysis, brain damage, disability or loss of limb, organ, or bodily functionlimb, organ, or bodily function

66 Patient deathPatient death

Methods: Severity ScoresMethods: Severity Scores National Coordinating Council for Medication Error National Coordinating Council for Medication Error

Reporting and Prevention (NCC MERP), (A through Reporting and Prevention (NCC MERP), (A through I)I) Only E through I represent patient harmOnly E through I represent patient harm

EE Temporary harm to the patient and required Temporary harm to the patient and required interventionintervention

FF Temporary harm to the patient and required initial Temporary harm to the patient and required initial or prolonged hospitalizationor prolonged hospitalization

GG Permanent patient harmPermanent patient harm

HH Intervention required to sustain lifeIntervention required to sustain life

II Patient deathPatient death

Patient Population Definitions

IHI Global Trigger Tool™ Random sample of 120 patients from all

orthopedics and urology admissions between Jan. 1 to June 30, 2007

Voluntary Reporting and Computerized Monitoring Unique admissions to orthopedics or

urology services on the 2 patient care units most likely to care for these patients between Jan. 1 to June 30, 2007

Results: Demographics of Results: Demographics of Adverse Event PopulationAdverse Event Population

Detection Method(admissions with

AE*)

IHI (n = 50)

Computerized

Monitoring

(n = 17)

Voluntary Reporting(n =11)

Mean Age, years ± Mean Age, years ± SDSD

RangeRange 18-64 years, n 18-64 years, n

(%)(%) ≥ ≥65 years, n (%)65 years, n (%)

59.9 ± 15.559.9 ± 15.521-9021-9029 (58%)29 (58%)21 (42%)21 (42%)

69.5 ± 14.269.5 ± 14.237-9037-905 (29%)5 (29%)12 (71%)12 (71%)

51.2 ± 13.151.2 ± 13.130-7430-7410 (91%)10 (91%)1 (9%)1 (9%)

GenderGender Male, n (%)Male, n (%) 31 (62%)31 (62%) 7 (41%)7 (41%) 5 (46%)5 (46%)

Admit ServiceAdmit Service Orthopedics, n Orthopedics, n

(%)(%) Urology, n (%)Urology, n (%)

24 (48%)24 (48%)26 (52%)26 (52%)

9 (53%)9 (53%)8 (47%)8 (47%)

9 (73%)9 (73%)3 (27%) 3 (27%)

Median LOS Median LOS ††, Days, DaysRangeRange

44

1-231-23881-441-44

553-143-14

*unique admissions to orthopedics or urology with an adverse event identified†LOS = Length of stay

Results: Adverse Event Results: Adverse Event Detection MethodsDetection Methods

16

10

1 2

1

5719 events

13 events

59 events

1294 unique 1294 unique admissionsadmissions

Random sample Random sample of 120 unique of 120 unique

admissionsadmissions

IHI IHI Trigger Trigger

ToolTool

IHI Trigger ToolIHI Trigger Tool289 positive triggers

Voluntary Voluntary ReportingReporting

Voluntary Voluntary ReportingReporting

85 reports

Computerized Computerized MonitoringMonitoring

ComputerizedComputerizedMonitoringMonitoring222 rules fired

Results: Adverse Event Results: Adverse Event FrequenciesFrequencies

0

20

40

60

80

100

120

140

High risk medication related adverse

events/1000 patientsAdverse events/1000 patients

IHI Global Trigger Tool

Computerized Monitoring

Voluntary Reporting

0

20

40

60

80

100

120

140

136

4 323

4 2

Primaryendpoint

Results: IHI Global Trigger Tool™ Results: IHI Global Trigger Tool™ Adverse Event by CategoryAdverse Event by Category

Fall, 1.7%

Hospital-associate

d infection,

13.6%

Adverse drug reaction, 13.6%

Transfusion/blood loss anemia, 23.7%

Surgical complications,

22%

Thrombosis/ Anticoagulatio

n, 5.1%

Chest pain, 3.4%

Narcotics/Benzodiazepines, 11.9%

Miscellaneous†, 5.1%

†Miscellaneous includes chemo-induced neutropenia (n = 1), hyponatremia/delirium (n = 2)

Care Module (47.5%)

MedicationModule(30.6%)

Surgical Module(22%)

59 adverse events

detected

Results: Computerized Results: Computerized Monitoring Adverse Event by Monitoring Adverse Event by

CategoryCategory

Anticoagulants, 52.6%

Narcotics/ Benzodiazepine

s, 26.3%

Hypoglycemia, 21.1%

19 adverse events

detected

5.1% captured by IHI

0% captured by IHI

11.9% captured by IHI

Compared to IHI…

Results: Voluntary Reporting Results: Voluntary Reporting Adverse Events by CategoryAdverse Events by Category

Narcotics/ Benzodiazepines,

38.5%

Anticoagulants, 15.4%

Antibiotics, 23.1%

Anesthetics, 7.7%

Electrolyte Replacement

Therapy (potassium), 7.7%

Cathartics/Laxatives, 7.7%

13 adverse events

detected

Compared to IHI…

More high risk medication

adverse events

No hypoglycemia events

Results: Adverse Event Results: Adverse Event SeveritySeverity

0102030405060708090

100

<3 3 4

IHI Global Trigger ToolComputerized MonitoringVoluntary Reporting

0

10

20

30

40

50

60

70

80

90

100

E F

Nationally validated

NCC MERP

Internal Severity Index

79%

21%

66%

34%

67%

33%

<3 = did not reach patient; 3 = temporary harm, required corrective therapy OR resulted in labs/vitals outside desirable range; 4 = required aggressive treatment, prolonged LOS > 2 days OR resulted in initial hospitalization

Perc

enta

ge (

%)

15.3%

84.7%

Future Enhancements

Expand computerized monitoringExpand computerized monitoring Discharge notes and nursing notes were Discharge notes and nursing notes were

valuable in adverse event identification valuable in adverse event identification in IHIin IHI

As technology advances and nursing As technology advances and nursing notes become electronic, free-text notes become electronic, free-text scanning may become a valuable scanning may become a valuable implementationimplementation

Example “oversedation” search versus IV Example “oversedation” search versus IV naloxone rulenaloxone rule

LimitationsLimitations

Retrospective chart review Retrospective chart review Not generalizable to other servicesNot generalizable to other services Causality and preventability not Causality and preventability not

assessed on IHI Global Trigger Tool™assessed on IHI Global Trigger Tool™ Physician reviewer was an orthopedic Physician reviewer was an orthopedic

surgeon which may introduce biassurgeon which may introduce bias

Limitations

Subjective judgment of adverse events Subjective judgment of adverse events (e.g. anemia due to blood loss from (e.g. anemia due to blood loss from surgery)surgery)

Not 1:1 comparisons with IHINot 1:1 comparisons with IHI Computerized monitoring and voluntary Computerized monitoring and voluntary

reporting data extracted by the units most reporting data extracted by the units most likely to care for these orthopedics and urology likely to care for these orthopedics and urology patientspatients

IHI Trigger Tool™ used a random sample rather IHI Trigger Tool™ used a random sample rather than all admissions to orthopedics and urologythan all admissions to orthopedics and urology

ConclusionsConclusions

IHI Trigger Tool™ identifies the greatest IHI Trigger Tool™ identifies the greatest number of events overallnumber of events overall Computerized monitoring and voluntary Computerized monitoring and voluntary

reporting identify greater proportion of reporting identify greater proportion of medication-related eventsmedication-related events

Voluntary systems good at capturing Voluntary systems good at capturing near misses/rare events whereas near misses/rare events whereas surveillance (IHI and computerized surveillance (IHI and computerized monitoring) better at capturing harmmonitoring) better at capturing harm

All 3 adverse event detection methods All 3 adverse event detection methods are complementaryare complementary

Acknowledgements Acknowledgements

Department of PharmacyDepartment of Pharmacy Residency Research CommitteeResidency Research Committee Computerized Patient Safety Computerized Patient Safety

Initiatives, Duke Health Technology Initiatives, Duke Health Technology SolutionsSolutions Julie Eckstrand, Pharm.D.Julie Eckstrand, Pharm.D. Monica Horvath, Ph.D.Monica Horvath, Ph.D. Andrea Long, Pharm.D.Andrea Long, Pharm.D. Julie Whitehurst, Pharm.D.Julie Whitehurst, Pharm.D.

QuestionsQuestions

ADE-S SystemADE-S System

Clinical Rules Engine

Laboratory

Demographic Data

Pharmacy

List of Triggers (daily)

Health-system clinical pharmacists

Computerized Patient Safety Initiative (CPSI)

pharmacists

ImmediateIntervention

required?

Yes

NoInvestigate trigger(e.g. chart review)

Intervene and document intervention

Score and document

ADE causality,severity, and

narrative

Kappapharmacist

Kappapharmacist

Are there previous conclusive reports on this reaction? +1 0 0

Did the adverse event appear after the drug was administered?

+2 -1 0

Did the adverse reaction improve when the drug was discontinued or a specific antagonist was administered?

+1 0 0

Did the adverse reaction reappear when the drug was re-administered?

+2 -1 0

Are there alternative causes (other than the drug) that could, on their own, have caused the reaction?

-1 -2 0

Did the reaction re-appear when a placebo was given? -1 +1 0

Was the drug detected in the blood (or other fluids) in concentrations known to be toxic?

+1 0 0

Was the reaction more severe when the dose was decreased?

+1 0 0

Did the patient have a similar reaction to the same or similar drugs in any previous exposure?

+1 0 0

Was the adverse event confirmed by an objective evidence? +1 0 0

YesYes NoNo Do Do Not Not

KnowKnow

Adverse Drug Reaction Probability Adverse Drug Reaction Probability ScaleScale

≥≥9 Definite; 8 to 5 Probable; 1 to 4 Possible; ≤0 Doubtful9 Definite; 8 to 5 Probable; 1 to 4 Possible; ≤0 Doubtful

Naranjo CA, Busto U, Sellers EM, et al. A method for estimating the probability of adverse drug Naranjo CA, Busto U, Sellers EM, et al. A method for estimating the probability of adverse drug reactions. Clin Pharmacol Ther 1981;30:239-245.reactions. Clin Pharmacol Ther 1981;30:239-245.

Sample Size CalculationSample Size Calculation

87% power at 120 charts to detect a 87% power at 120 charts to detect a 4-fold difference in rates between IHI 4-fold difference in rates between IHI Trigger Tool™ and ADE-S during a 6 Trigger Tool™ and ADE-S during a 6 month periodmonth period

IHI Surgical Trigger ToolIHI Surgical Trigger Tool™™

Surgical Trigger ToolSurgical Trigger Tool™™ contains 28 contains 28 triggerstriggers Surgical triggers have corresponding code Surgical triggers have corresponding code

on global trigger tool (not limited to surgical on global trigger tool (not limited to surgical module)module)

193 surgical triggers vs. 289 global 193 surgical triggers vs. 289 global triggers (67% correspondence)triggers (67% correspondence)

Results: IHI Triggers by Frequency

Other triggers 20%

Transfusion10.1%

Anti-emetic use

18.4%

Abrupt drop in Hct of 25%

13.9%

Oversedation/ hypotension

7.6%

Benadryl use 6.9%

Readm

ission

8%

X-r

ay in

tra-o

p 5.

6%

op-t

ime

>6 h

3.1%

Abr

upt m

ed s

top

3.5%

Fall

2.8

%

• Return to surgery

•Admission to ICU post-op

• X-ray or doppler studies for emboli

• Any procedure complication

• SCr > 2 times baseline

• Consult requested in PACU

• C. diff positive culture

•Romazicon (flumazenil) use

•Time in ED > 6 h

•Narcan (naloxone) use

289 triggers