Secondary Use of Patient Data forSecondary Use of Patient Data for Research and Quality Improvements:

Tips, Tricks, Tools, Troubles, Triumphs ps, c s, oo s, oub es, u p sand other Topics

Deborah H Batson The Children’s Hospital DenverMichael G Kahn, MD, PhD University of Colorado, Denver

Supported by The Children’s Hospital Research Institute and the NIH/NCRR Colorado CTSI Grant Number UL1 RR025780. Its contents are the authors’ sole responsibility and do not necessarily represent official NIH views

Secondary Use

• Use of health care data in addition to direct patient carepatient care– Monitor and analyze quality of care– Promote public health surveillancePromote public health surveillance– Improve patient care through research– Monitor quality of health care data

• Aggregation and analysis of data collected during routine patient care across patient sets

Some Use Cases

• Data quality analysis• Health care quality and outcomes metrics• Research aggregates• Public health surveillance• Public health securityPublic health security• Disease registry participation• Market trending and analysis• Patient access to health care data• Patient access to health care data• Unstructured data mining• Phenotypic/genotypic correlation analysis

TCH CI Use Cases

• Data quality analysis• Health care quality and outcomes metrics• Health care quality and outcomes metrics• Research aggregates• Public health surveillance• Public health security/bio-surveillance• Disease registry participation• Market trending and analysisMarket trending and analysis• Patient access to health care data• Mining of unstructured data• Phenotypic/genotypic correlation analysis• Phenotypic/genotypic correlation analysis

Rich sources of data

• Clinical data– EMR Electronic Medical Record

LIS L b t I f ti S t– LIS Laboratory Information Systems– DSS Decision Support Systems– National data collections

• Registries• Registries • Collaboratives• Public datasets

– Research data collection– Radiology and Pathology Annotations

• Administrative data – Finance and Billing• Demographic data - EMRg p

Data Requests at TCH

Distribution of Data Requests 2002-2008

researchfinance-operations

other-operations

quality

finance

operations

T’s

• Troubles• Tips• Tricks• Tools• Triumphsp

Big Trouble• Standard terminologies

– What’s a ‘census’?– What’s an ‘encounter’?

U it f ? L b ll ti diff ? M di ti d h ?– Units of measure? Lab collection differences? Medication dose changes?• Patient identification

– List all clinic patients– Who are my research patients?y p

• EMR has one identifier• LIS may have another• Research records may have distinct identifier

• Data Quality– Married 6 year olds!– Missing data, nulls, invalid dates…

• Impact of work flow assumptions– How we collect data impacts the interpretation of the data– How we collect data impacts the interpretation of the data

Trouble: Four Examples

1. Too much data• Insufficiently bounded query returns too many y q y y

patients2. Bad assumptions

• “That’s not what I wanted AT ALL!”3. Alternate realities

• Finance vs Clinical definitions of “complex patient”4. Data in text notes

• Clinical systems built for provider adoption use different workflows than research systems

Hematology

• Trouble: Too much data• Use Case: Hematology: DVT cases 2007-09• Use Case: Hematology: DVT cases 2007-09

– First pass: many thrombosis dx - 3000 pts– Second pass: deep vein thrombosis - 345 ptsSecond pass: deep vein thrombosis 345 pts– Third pass: acquired during inpatient stay - 84 pts

• TOOLS– i2b2 iterative data research– Seek data from other databases when one isn’t

enough – Chart review is sometimes the ONLY & right tool

ED: The Big Dip

• Trouble: Bad assumptions– Client assumptions based on unknown data p

relationships– Analyst assumptions based on large request

• Use Case: Emergency Department– Which ED crowding indicators predict delayed care in

ED treatment?ED treatment?– Large, complex data set in several dimensions

• TIP• TIP– Work together in increments

Integrated Service Discovery • Trouble: Alternate realities• Use Case: find patients across three p

subspecialties to create new medical service– Clinical criteria: 145 patients– Financial criteria: 70 patients– Finance preferred the larger number!

Missed cases because of billing assumptions on really• Missed cases because of billing assumptions on really REALLY expensive patients

• After removing financial outliers, 100 patients met criteria

• TRIUMPH– New integrated service is in planning stage

Urology• Trouble: Data in Text• Use Case: Urology Outcomesgy

– Data request for surgical outcomes expressed in post-surgical clinical follow-up notes

• TRICK– Concept enabled notes

Near discrete nstr ct red data “going for ard”• Near-discrete unstructured data “going forward”• Prospective-only solution, need NLP

• TRIUMPHTRIUMPH– Urologist became an advocate for structured data

entry

Triumph: Result of Urology Report

• Requests now look like this:– “Please generate a report on ANY pt. that has one of

the below listed clinical enabled concepts for Urology:19918199182148621491…”

Summary

Troubles T’s

Too Much Data Tools: iterative research into data; iterative visits with investigator

Bad Assumptions Tip: “small dips” of dataBad Assumptions Tip: small dips of data

Alternate Realities Triumph!

Data in Text Trick: concept-enabled notesTriumph: structured data championTriumph: structured data champion

Terribles

• Trudging through the data tundra• Queries that never endQueries that never end

– Poorly formed hypothesis• Poor scientific method in the study design• NO STUDY DESIGN!

– Non-testable measuresEver expanding fishing expeditions– Ever-expanding fishing expeditions

• “That’s nice, but now can we look at…”

•• S t r e t c h i n gS t r e t c h i n g IRB and analyst boundariesS t r e t c h i n gS t r e t c h i n g IRB and analyst boundaries

Other TCH Tools

• Business intelligence– Reporting softwareReporting software– Aggregating software– Research-specific tools

• Human intelligence– Data request triage– Marriage: clinical / technical backgrounds

Thursdays

– Collaboration• Clinical Informatics (Research and Quality)( y)• Clinical Applications Services (Operations)• Surgical Services• Ambulatory Services• Finance

HIM• HIM• Security• IRB• IRB

Thursday Meetings

– Team sharing• Review all open requests for the weekp q• Share techniques and data sources• Unused meeting time

– Review interesting or difficult queries– Introduce new knowledge

D t d t» Data source updates» Tools » Tricks of the trade» Tricks of the trade

Thursday Meetings

– Connect clinical “data silos”• Analysts are distributed throughout the hospitalAnalysts are distributed throughout the hospital• Some IS, many not in IS• Departments have independent analysts / domain y

experts• Weekly sessions around a table benefit all

F ti ll t t li d BI t ff• Functionally operates as a centralized BI staff

Example

• User’s request for clinical notes AND billing data– Team approachpp

• Operations• Physician billing• Hospital finance• Hospital finance• Clinical

Resource “library”

• Dense body of knowledge in complete reports to draw on– Library of reports– Human resources

Data Validation

• Second set of eyes– SOP: data isn’t delivered without review

• Pro: better data validation• Challenge: scarcity of time and people

Describe assumptions of the problem and strategy of– Describe assumptions of the problem and strategy of solution

• Knowledge sharing

– Beware: analysts share a set of assumptions and a data source bias

Discussion

• Use cases• Unique techniques• Troubles• Tips• Tricks• Tools• TriumphsTriumphs

Contacts

Deborah H. BatsonClinical Research Data Warehouse ArchitectDepartment of Clinical Informatics

Michael G. Kahn MD, PhDAssociate Professor

Department of Clinical InformaticsThe Children's Hospital13123 East 16th Avenue Box 400Aurora, CO 80045

Section of Pediatric EpidemiologyDepartment of PediatricsCo-Director, Colorado Clinical and Translational Sciences InstituteCore Director, CCTSI Biomedical InformaticsUniversity of Colorado Denver

Batson.Deborah@tchden.org

(720) 777 5704(720) 777 7300 fax

University of Colorado Denver

Director, Clinical InformaticsQuality and Patient SafetyThe Children's Hospital

(720) 777 7300 faxAurora, Colorado

Kahn.Michael@tchden.org720-777-6407