IBM Medical Records Text Analytics Solution Helps UNC Healthcare Improve

the Quality of Hospital DischargesSession Number ECA-1419A

Carlton Moore, MDUNC Healthcare

Fiodar ZboichykIBM

2

Overview

• Hospital Readmission Rates

– Medical and Economic Impact

• Reasons for High Readmission Rates

– Importance of discharge summary

• Proposed NLP solution

– Development issues (example, unstructured, inconsistent)

• Results (sensitivity, specificity)

• Future directions

3

30-Day Hospital Readmission Rates by State

Jencks S, Williams M, Coleman E. N Engl J Med 2009; 360 (14): 1418-28

Estimated annual cost to Medicare = $17.4B

4

Economic Impact on Hospitals

• In 2013 Medicare will start applying financial penalties to hospital with higher than expected readmission rates

• Other health insurers are likely to follow Medicare’s lead!!

Condition #of Patients

Average Reimbursement

%Higher than Expected

Potential Penalty

Heart Failure 600 $5,000 20% $600,000

Heart Attack 400 $4,000 20% $320,000

Pneumonia 350 $3,000 15% $157,500

$1,107,500

Sample Hospital

Potential Penalty = (# of patients with condition) x (Avg. reimbursement for condition) x (% Higher than expected)

5

Why are Hospital Readmission Rates So High?

6

Conceptual Framework

Discharge instructions not carried out

Adverse Event

Hospital Readmission

Patient discharged with unresolved medical issues that need to be addressed after leaving hospital

follow-up physician visitsfollow-up tests and procedures

Discharge Instructions a concise action plan

describing what needs to occur after a patient

leaves the hospital

Definition: condition worsens because of inappropriate or inadequate medical care

Only 50% of discharge summaries are ever received by patients’

physicians

Poor communication of discharge instructions

7

Discharge Instructions

Diagnostic Pro-cedures

Physician Referrals Lab Tests0%

10%

20%

30%

40%

50%

48%

35%

17%

Types of Discharge Instructions(693 hospital discharges)

50% not completed 27% no completed 15% not completed

Moore C, McGinn T, Halm E. Arch Intern Med. 2007;167:1305-1311

8

Examples of Discharge Instructions not Completed

Types of Procedure Reasons for Procedures

CT of Chest Lung mass found on previous x-ray

CT scan of the abdomen Abdominal abscess and kidney mass

Chest x-ray Lung nodule on admission chest x-ray

Colonoscopy Gastrointestinal bleeding

Physician Referrals Reasons for Referrals

Psychiatry Suicidal Ideation

Neurology Seizures

Nephrology Kidney failure

Surgery Infected wound

Moore C, McGinn T, Halm E. Arch Intern Med. 2007;167:1305-1311

9

Adverse Events after Hospital Discharge

• 1 in 5 (20%) patients has an adverse event shortly after hospital discharge

Forster AJ, Murff HJ, Peterson JF, Gandhi TK, Bates DW. Ann Intern Med. 2003

ADE Procdeure Related

Other Infection Fall0%

10%

20%

30%

40%

50%

60%

70%62%

16% 14%

5% 4%

Types of Adverse Events, %

ADE: adverse drug eventOther: incorrect treatment and/or missed diagnosis

10

Example of an Adverse Event

• A patient with heart failure started receiving spironolactone in the hospital. The patient was sent home with a prescription for this medication in addition to previous use of ramipril and potassium supplements.

• Blood tests were not monitored after hospital discharge even though it was clearly documented in the discharge summary that the patient needed follow-up blood tests.

• Two weeks later the patient developed extreme weakness and went to the emergency room. Blood tests revealed a potassium level >7.5 mmol/L (normal = 4.5 mmol/L).

Forster AJ, Murff HJ, Peterson JF, Gandhi TK, Bates DW. Ann Intern Med. 2003

11

Purpose of Project

• Extract key elements of the discharge instructions:

– Discharge medications

– Discharge diagnosis

– Follow-up appointments

• Convert the extracted data into structured format that can be:

– electronically transmitted to healthcare providers responsible for care after hospital discharge

– used to generate reminders and alerts to healthcare providers

12

Discharge Instructions

Clinic Physician Scheduled Date/Time

Internal Medicine

Joseph Morgan

Yes 8/10/200916:10

Cardiology - EP Null No Null

Anticoagulation Null Yes 8/4/20098:45

13

Study Design

• Discharge instructions (Name, Type/Location, Time Frame) were extracted from free-text hospital discharge summaries:

– Manual review (physician)

– IBM Content Analytics (ICA)

• Accuracy of ICA was calculated using manual physician review as the “gold standard”

– Sensitivity, specificity

– Positive predictive value, negative predictive value

14

Measurement

• Overall Accuracy = (TP +TN)/(Total)

• Sensitivity

– % of records containing follow-up elements that were identified via text analytics.

• Specificity

– % of records lacking follow-up elements that were not flagged via text analytics.

• Positive Predictive Value (PPV)

– % of records flagged as containing follow-up elements using text analytics that actually contained follow-ups

15

Results: Accuracy of Text Analytics in Identifying Follow-up Appointments and Diagnoses

Element Overall Accuracy

Precision Sensitivity (Recall)

Specificity PPV

Diagnoses 78% 90% 80% 68% 90%

Followup 79% 95% 74% 91% 95%

16

IBM

Co

nte

nt

An

aly

tic

s Document Server

(UIMA Pipeline)

Extended ICA JDBC Crawler

IBM InfoSphere Guardium Data Redaction

UNC Health Care Clinical Data Warehouse

Apache Lucene Search Engine

ICA-Text Miner Web Application

LuceneIndex

JDB

C U

IMA

CA

S

Con

sum

er

Pathology ReportsDischarge Summary ReportsEchocardiogram Reports

Med

ical

Ann

otat

ors

ICA

Ann

otat

ors

ICA-LanguageWare Resource Workbench

Health Language Inc.Language Engine

SNOMED, RxNorm, ICD-9ICD-10, CPT-4

Medical Terminology

UNC Health CareTerminology

UNC Health Care Solution Component Architecture

Discharge Follow-up Reporting

Business Intelligence Tool

17

Project Lessons Learned

• Medical texts are more complicated than we thought… again.

• Standard terminology (RxNorm, SNOMED CT, ICD9, …)

– Absolutely required, but not good enough for dictionary matches

– “tick-born disease”, but not “tick borne illness”.

• Diagnoses

– Negation is actually just part of the range – “rule out”, “possible”.

– “Left femur fracture” and “fracture, left femur”.

– “Discharge diagnosis: same as above”.

• Follow-ups. Sometimes just “fup”.

– Usually “Dr. Good”, but sometimes “her cardiologist”.

– Usually “Vascular Surgery Clinic”, but sometimes “heme-onc”.

18

Summary

• NLP will improve communication of discharge instructions:

– Improve patient care (reduce hospital readmissions)

– Reduce risk of Medicare penalties to the hospital

19

Future Directions

• Cohort identification for researchers and quality improvement specialists

• Cancer diagnoses in pathology reports

• Findings in radiology reports

• Extracting quality measure data for the hospital

• Researchers

– 156 current NIH-funded grants ($75M) utilizing NLP