Cullen Presentation

Introduction to Patient Safety ResearchIntroduction to Patient Safety Research

Presentation #9 - Understanding Causes: Cohort StudyPresentation #9 - Understanding Causes: Cohort Study

WORLD ALLIANCE FOR PATIENT SAFETWORLD ALLIANCE FOR PATIENT SAFETY

Presentation # - Understanding Causes: Cohort StudyPresentation # - Understanding Causes: Cohort Study

2 OverviewOverview ObjectivesObjectives

To compare the frequency and preventability of adverse drug To compare the frequency and preventability of adverse drug events (ADE) and potential ADEs in intensive care units (ICUs) events (ADE) and potential ADEs in intensive care units (ICUs) and non-ICUs. and non-ICUs.

To evaluate systems factors involved in each ADR by comparing To evaluate systems factors involved in each ADR by comparing ICUs with non-ICUs and medical ICUs with surgical ICUs. ICUs with non-ICUs and medical ICUs with surgical ICUs.

MethodsMethods Prospective cohort study involving all 4,031 adult admissions Prospective cohort study involving all 4,031 adult admissions

to a stratified, random sample of 11 medical and surgical units to a stratified, random sample of 11 medical and surgical units in 2 hospitals over 6 months in 2 hospitals over 6 months

Measures: rate of preventable ADEs and potential ADEs, length Measures: rate of preventable ADEs and potential ADEs, length of stay, charges, costs, and measures of the unit's of stay, charges, costs, and measures of the unit's environment. environment.

Incidents detected by self-report by nurses and pharmacists Incidents detected by self-report by nurses and pharmacists and by daily chart review. Individuals involved in ADEs and by daily chart review. Individuals involved in ADEs underwent detailed interviews. underwent detailed interviews.

ResultsResults Rate of preventable ADEs and potential ADEs in ICUs was 19 Rate of preventable ADEs and potential ADEs in ICUs was 19

events per 1000 patient days, nearly twice that rate of non-events per 1000 patient days, nearly twice that rate of non-ICUs. ICUs.

However, when adjusted for number of drugs ordered, there However, when adjusted for number of drugs ordered, there was no greater likelihood of preventable and potential ADEs in was no greater likelihood of preventable and potential ADEs in ICUs than in non-ICUs.ICUs than in non-ICUs.

Conclusion Conclusion Preventable adverse drug events and potential adverse drug Preventable adverse drug events and potential adverse drug

events occurred in units that functioned normally and involved events occurred in units that functioned normally and involved caregivers who were working under reasonably normal caregivers who were working under reasonably normal circumstances, not at the extremes of workload, stress, or a circumstances, not at the extremes of workload, stress, or a difficult environment.difficult environment.



3 Introduction: Study DetailsIntroduction: Study Details

Full ReferenceFull ReferenceCullen DJ, Sweitzer BJ, Bates DW, Burdick E, Edmondson Cullen DJ, Sweitzer BJ, Bates DW, Burdick E, Edmondson A, Leape LL. Preventable adverse drug events in A, Leape LL. Preventable adverse drug events in hospitalized patients: a comparative study of intensive hospitalized patients: a comparative study of intensive care and general care units. Crit Care Med, 1997, care and general care units. Crit Care Med, 1997, 25;1289-129725;1289-1297

Link to Abstract (HTML)Link to Abstract (HTML) Link to Full TextLink to Full TextCan be ordered Can be ordered

online at: online at: http://www.ccmjournal.comhttp://www.ccmjournal.com



4 Introduction: Research Team Introduction: Research Team

Head researcher – David J. Cullen, MD, MSHead researcher – David J. Cullen, MD, MS Former Chairman of Department of Anaesthesiology and Former Chairman of Department of Anaesthesiology and

Pain Medicine (1996 – 2005)Pain Medicine (1996 – 2005) St. Elizabeth's Medical Center and Tufts University St. Elizabeth's Medical Center and Tufts University

Medical School in Boston, Massachusetts, USA Medical School in Boston, Massachusetts, USA Area of expertise: anaesthesiology and critical care Area of expertise: anaesthesiology and critical care

medicine medicine Other team members:Other team members:

Bobbie Jean Sweitzer, MDBobbie Jean Sweitzer, MD David W. Bates, MDDavid W. Bates, MD Elisabeth Burdick, MSElisabeth Burdick, MS Amy Edmondson, PhDAmy Edmondson, PhD Lucian Leape, MDLucian Leape, MD



5 Background: Opening PointsBackground: Opening Points

Medical treatment is estimated to accidentally injure Medical treatment is estimated to accidentally injure 1.3 million people each year in the US1.3 million people each year in the US Harvard Medical Practice Study found that medications Harvard Medical Practice Study found that medications

are the most common cause of adverse eventsare the most common cause of adverse events Patients in intensive care units (ICUs) are at Patients in intensive care units (ICUs) are at

especially high risk of an adverse drug event related especially high risk of an adverse drug event related to human error because:to human error because: Workload is intenseWorkload is intense Many interactions between patients and caregiversMany interactions between patients and caregivers Critical illness reduces both the patients' natural Critical illness reduces both the patients' natural

resilience and ability to defend themselves from resilience and ability to defend themselves from consequences of human errorconsequences of human error

Patients in the ICU receive twice as many drugs as Patients in the ICU receive twice as many drugs as patients in general care unitspatients in general care units



6 Background: Study RationaleBackground: Study Rationale

Many adverse drug events (ADEs) are preventableMany adverse drug events (ADEs) are preventable Understanding how errors in drug use occur is Understanding how errors in drug use occur is

essential for reducing injuries and developing essential for reducing injuries and developing prevention strategiesprevention strategies "Our intent was to study human errors leading to ADEs, "Our intent was to study human errors leading to ADEs,

looking for systemic problems and designing system looking for systemic problems and designing system wide solutions and then testing their efficacy."wide solutions and then testing their efficacy."



7 Background: Setting Up a Research TeamBackground: Setting Up a Research Team

Part of a larger study of adverse drug eventsPart of a larger study of adverse drug events Selecting collaborators:Selecting collaborators:

""Those whom I chose were based on interest, motivation Those whom I chose were based on interest, motivation and ambition."and ambition."

Obtaining funding:Obtaining funding: Federal Grant from US Agency for Health Care Policy Federal Grant from US Agency for Health Care Policy

and Research and Research Smaller grants obtained from Harvard malpractice Smaller grants obtained from Harvard malpractice

insurer insurer



8 Methods: Study Design Methods: Study Design

DesignDesign: prospective cohort study: prospective cohort study ObjectivesObjectives::

To compare the frequency and preventability of adverse To compare the frequency and preventability of adverse drug events and potential adverse drug events in ICUs drug events and potential adverse drug events in ICUs and non-ICUsand non-ICUs

To evaluate systems factors involving the individual To evaluate systems factors involving the individual caregivers, care unit teams, and patients involved in caregivers, care unit teams, and patients involved in each adverse drug event by comparing: each adverse drug event by comparing:

• ICUs with non-ICUsICUs with non-ICUs• Medical ICUs with surgical ICUsMedical ICUs with surgical ICUs



9 Study Design: Population and SettingStudy Design: Population and Setting

PopulationPopulation: 4,031 adult patients admitted to 11 ICU : 4,031 adult patients admitted to 11 ICU and general care units in two tertiary care hospitals and general care units in two tertiary care hospitals in the US between Feb. and July 1993in the US between Feb. and July 1993 Two medical ICUsTwo medical ICUs Three surgical ICUsThree surgical ICUs Four medical general care unitsFour medical general care units Two surgical general care unitsTwo surgical general care units

SamplingSampling: Stratified, random sample of patients : Stratified, random sample of patients admitted to medical and surgical unitsadmitted to medical and surgical units Patients eligible to be in study more than oncePatients eligible to be in study more than once When patients had more then one adverse drug event, When patients had more then one adverse drug event,

only the first episode in that admission was evaluatedonly the first episode in that admission was evaluated



10 Methods: Data CollectionMethods: Data Collection

Incidents were Incidents were identifiedidentified in three ways: in three ways: Unit personnel asked to report incidents to nurse Unit personnel asked to report incidents to nurse

investigatorsinvestigators Nurse investigator visited each unit and solicited Nurse investigator visited each unit and solicited

information from nurses, pharmacists, and clerical information from nurses, pharmacists, and clerical personnel concerning all actual or potential drug-related personnel concerning all actual or potential drug-related incidentsincidents

Nurse investigator reviewed all charts daily on Nurse investigator reviewed all charts daily on weekdays and once on weekendsweekdays and once on weekends

All incidents evaluated independently by two All incidents evaluated independently by two physician reviewers and physician reviewers and classifiedclassified according to: according to: Whether they represented actual or potential ADEs Whether they represented actual or potential ADEs Severity and preventability of the eventSeverity and preventability of the event



11 Methods: Data Collection (2)Methods: Data Collection (2)

Each preventable or potential ADE Each preventable or potential ADE investigatedinvestigated to to determine if there was an error and if so, the determine if there was an error and if so, the circumstances, apparent causes, and profiles of the circumstances, apparent causes, and profiles of the persons involvedpersons involved Individuals involved in the preventable actual or Individuals involved in the preventable actual or

potential ADE underwent detailed interviewspotential ADE underwent detailed interviews Interviews conducted by peer case-investigators Interviews conducted by peer case-investigators

(physician to physician, nurse to nurse, and pharmacist (physician to physician, nurse to nurse, and pharmacist to pharmacist)to pharmacist)



12 Methods: InterviewsMethods: Interviews

Case-investigators used structured forms to seek Case-investigators used structured forms to seek details about the circumstances surrounding the details about the circumstances surrounding the incidentincident E.g.experience with the drug, treatment plan, stress E.g.experience with the drug, treatment plan, stress

factors, external distractions, sleep deprivation, etc.factors, external distractions, sleep deprivation, etc. Interviewees were asked to self-assess:Interviewees were asked to self-assess:

Competency and skill Competency and skill Decision-making styleDecision-making style Openness to change Openness to change Duration on the service or job Duration on the service or job Amount and quality of supervision Amount and quality of supervision Relationship of the incident to the timing of their shift Relationship of the incident to the timing of their shift

Respondents asked to discuss their perception of why Respondents asked to discuss their perception of why the event occurred, graded on a 1 to 5 Likert scalethe event occurred, graded on a 1 to 5 Likert scale



13 Methods: Data Analysis and Interpretation Methods: Data Analysis and Interpretation

Results of each preventable ADE analyzed by a Results of each preventable ADE analyzed by a multidisciplinary team of physicians, pharmacists, multidisciplinary team of physicians, pharmacists, nurses and systems analystsnurses and systems analysts

Analyses performed:Analyses performed: Comparison of difference between units in rates of Comparison of difference between units in rates of

adverse drug events (Chi-square, analysis of variance, adverse drug events (Chi-square, analysis of variance, and unpaired t-tests)and unpaired t-tests)

Univariate comparison of resource utilization (Wilcoxon Univariate comparison of resource utilization (Wilcoxon rank-sum test)rank-sum test)

Multivariate comparisons of post-event length of stay Multivariate comparisons of post-event length of stay and resource utilization for ICU vs. non-ICU patients and and resource utilization for ICU vs. non-ICU patients and medical vs. surgical patients (multiple linear regression)medical vs. surgical patients (multiple linear regression)



14 Results: Key FindingResults: Key Finding

Study identified 247 adverse drug events in 206 Study identified 247 adverse drug events in 206 admissions admissions

236 persons involved in the preventable and 236 persons involved in the preventable and potential ADEs interviewed by a peer case-potential ADEs interviewed by a peer case-investigatorinvestigator



15 Results: Key Findings (2)Results: Key Findings (2)

Rate of preventable ADEs and potential ADEs in ICUs was Rate of preventable ADEs and potential ADEs in ICUs was 19 events per 1000 patient days, nearly twice that rate of 19 events per 1000 patient days, nearly twice that rate of non-ICUsnon-ICUs (blue)(blue)

However, when adjusted for the number of drugs used, no However, when adjusted for the number of drugs used, no statistically significant differences in rates between ICUs statistically significant differences in rates between ICUs and non-ICUs and non-ICUs (green)(green)

Table 1. Combined event rates, unadjusted and adjusted for drug use: Table 1. Combined event rates, unadjusted and adjusted for drug use: Intensive care unit (ICU) vs. non-ICUIntensive care unit (ICU) vs. non-ICU




Medical ICU rate (25 events per 1000 patient days) was Medical ICU rate (25 events per 1000 patient days) was significantly higher than the surgical ICU rate (14 events significantly higher than the surgical ICU rate (14 events per 1000 patient days)per 1000 patient days)

Table 2. Combined event rates by unit typeTable 2. Combined event rates by unit type




Length of stay and severity of the adverse drug Length of stay and severity of the adverse drug event were greater in ICUs than non-ICUs, but there event were greater in ICUs than non-ICUs, but there were no differences between medical ICU and were no differences between medical ICU and surgical ICU patientssurgical ICU patients

Structured interviews indicated almost no differences Structured interviews indicated almost no differences between ICUs and non-ICUs for many characteristics between ICUs and non-ICUs for many characteristics of the patient, patient care team, systems, and of the patient, patient care team, systems, and individual caregivers.individual caregivers.



18 Conclusion: Main PointsConclusion: Main Points

Unadjusted rate of preventable and potential Unadjusted rate of preventable and potential adverse drug events was twice as high in ICUs adverse drug events was twice as high in ICUs compared with non-ICUscompared with non-ICUs

However, when adjusted for the number of drugs However, when adjusted for the number of drugs ordered, there was no greater likelihood for ordered, there was no greater likelihood for preventable ADEs and potential ADEs to occur in ICUs preventable ADEs and potential ADEs to occur in ICUs than in non-ICUsthan in non-ICUs

Preventable adverse drug events and potential Preventable adverse drug events and potential adverse drug events occurred in units that adverse drug events occurred in units that functioned normallyfunctioned normally Involved caregivers who were working under normal Involved caregivers who were working under normal

circumstances, not at the extremes of workload, stress, circumstances, not at the extremes of workload, stress, or a difficult environmentor a difficult environment



19 Conclusion: DiscussionConclusion: Discussion

Reducing the number of drugs used in the ICU may Reducing the number of drugs used in the ICU may decrease the incidence of adverse drug eventsdecrease the incidence of adverse drug events Even if only a small fraction of errors result in injury, Even if only a small fraction of errors result in injury,

this rate can be substantial in the ICU because of the this rate can be substantial in the ICU because of the intensity of treatmentintensity of treatment

Systems failures may be far more important Systems failures may be far more important contributing factors to ADEs than the obvious causes contributing factors to ADEs than the obvious causes of fatigue and stressof fatigue and stress Study did not confirm conventional wisdom that serious Study did not confirm conventional wisdom that serious

errors are made primarily by overworked and exhausted errors are made primarily by overworked and exhausted individuals working with complex patients in an individuals working with complex patients in an environment filled with distractions environment filled with distractions

Common systems failures include poor communication, Common systems failures include poor communication, lack of standardization and insufficient labellinglack of standardization and insufficient labelling



20 Conclusion: Discussion (2)Conclusion: Discussion (2)

Study limitationsStudy limitations Study included only two tertiary care hospitals that Study included only two tertiary care hospitals that

managed relatively similar patientsmanaged relatively similar patients Methods for detecting ADEs undoubtedly missed some Methods for detecting ADEs undoubtedly missed some

of these events – may result in underestimation of of these events – may result in underestimation of incidence of ADEsincidence of ADEs

Interviewers could not be blinded to the purposes of the Interviewers could not be blinded to the purposes of the studies – may have introduced interview biasstudies – may have introduced interview bias

Interviewees self-assessment of sleep status may have Interviewees self-assessment of sleep status may have underestimated degree of fatigueunderestimated degree of fatigue



21 Conclusion: Study ImpactConclusion: Study Impact

Academic impactAcademic impact More than 20 major articles published in leading general More than 20 major articles published in leading general

medical, critical care and anaesthesiology journalsmedical, critical care and anaesthesiology journals Extensive citations, lay and professional press media Extensive citations, lay and professional press media

interviewsinterviews Practice impactPractice impact

Promoted the development of a patient safety culture Promoted the development of a patient safety culture based on scientific studies, not subjective opinionbased on scientific studies, not subjective opinion

Highlighted the important role that clinical pharmacists Highlighted the important role that clinical pharmacists may play in reducing ADEsmay play in reducing ADEs



22 Conclusion: Study Impact (2)Conclusion: Study Impact (2)

Policy impactPolicy impact Increased awareness of medical errors and the need to Increased awareness of medical errors and the need to

fix problems, not fire peoplefix problems, not fire people Highlighted potential for cost savings through reducing Highlighted potential for cost savings through reducing

errorserrors Led to the formation of the National Patient Safety Led to the formation of the National Patient Safety

FoundationFoundation Patient impactPatient impact

Led to studies of ADEs in the outpatient settings and of Led to studies of ADEs in the outpatient settings and of comparable human errors in medicine (e.g. blood banks)comparable human errors in medicine (e.g. blood banks)



23 Conclusion: Practical Considerations Conclusion: Practical Considerations

Study durationStudy duration 60 months from conception to write-up 60 months from conception to write-up

CostCost Over $1 million USD for the whole study effort (not just Over $1 million USD for the whole study effort (not just

this paper)this paper) Competencies neededCompetencies needed

High level statistician support High level statistician support Extensive data managementExtensive data management Expertise from multiple disciplines: psychology, Expertise from multiple disciplines: psychology,

pharmacy, etc.pharmacy, etc. Ethical approvalEthical approval

Took several months to obtain, and some difficulties Took several months to obtain, and some difficulties were encountered at one of the two hospitalswere encountered at one of the two hospitals



24 Author Reflections: Overcoming BarriersAuthor Reflections: Overcoming Barriers

Need for informed consent?Need for informed consent? "The Human Studies Committee wanted informed "The Human Studies Committee wanted informed

consent from each patient, even through we never consent from each patient, even through we never interacted with any patient. However, we eventually interacted with any patient. However, we eventually convinced them to back off."convinced them to back off."

Reassuring participants about confidentiality:Reassuring participants about confidentiality: "Also, those who made the errors were scared to talk "Also, those who made the errors were scared to talk

privately about it with our interviewers. We had to privately about it with our interviewers. We had to reassure them about the confidentiality issues and it reassure them about the confidentiality issues and it worked most of the time." worked most of the time."



25 Author Reflections: Lessons and AdviceAuthor Reflections: Lessons and Advice

If you could do one thing differently in this study If you could do one thing differently in this study what would it be? what would it be? "If we had the resources, study many more hospitals of "If we had the resources, study many more hospitals of

different types, cultures and locations to show different types, cultures and locations to show generalizability."generalizability."

Would this research be feasible and applicable in Would this research be feasible and applicable in developing countries? developing countries? "No, far too many resources needed.""No, far too many resources needed."



26 Author Reflections: Ideas for Future ResearchAuthor Reflections: Ideas for Future Research

What message do you have for future researchers What message do you have for future researchers from developing countries? from developing countries? "Focus on a clear question and don’t try to do too much "Focus on a clear question and don’t try to do too much

in any one study."in any one study." What would be an important research project you What would be an important research project you

recommend that they do? recommend that they do? "Test a suggested solution to one human error problem "Test a suggested solution to one human error problem

and see if it reduces the specific error." and see if it reduces the specific error."



27 Additional References and ResourcesAdditional References and Resources

InstitutionsInstitutions National Patient Safety FoundationNational Patient Safety Foundation Lucien Leape FoundationLucien Leape Foundation

ReferencesReferences Kohn LT, Corrigan JM, Donalson MS, eds. To Err Is Kohn LT, Corrigan JM, Donalson MS, eds. To Err Is

Human. Institute of Medicine, Washington, DC: National Human. Institute of Medicine, Washington, DC: National Academy Press; 1999.Academy Press; 1999.

Cullen DJ, Bates DW, Leape LL, and the Adverse Drug Cullen DJ, Bates DW, Leape LL, and the Adverse Drug Event Prevention Study Group. Prevention of adverse Event Prevention Study Group. Prevention of adverse drug events: a decade of progress in patient safety. J drug events: a decade of progress in patient safety. J Clin Anesth. 2000;12:600-614.Clin Anesth. 2000;12:600-614.

Cullen Presentation

Health & Medicine

Transcript of Cullen Presentation