Vojtech huser-data-warehouse-evaluation-2010-04-idr-snapshot014c

23
A Methodology for Quantitative Measurement of Quality and Comprehensiveness of a Research Data Repository (IDR Snapshot) Vojtech Huser MD PhD

description

 

Transcript of Vojtech huser-data-warehouse-evaluation-2010-04-idr-snapshot014c

Page 1: Vojtech huser-data-warehouse-evaluation-2010-04-idr-snapshot014c

A Methodology for Quantitative Measurement of Quality and Comprehensiveness of a Research Data Repository

(IDR Snapshot)

Vojtech Huser MD PhD

Page 2: Vojtech huser-data-warehouse-evaluation-2010-04-idr-snapshot014c

Vojtech Huser, MD, PhDVojtech Huser, MD, PhD

2

Acknowledgement Wisconsin Institute for discovery

Private branch: Morgridge Institute CTSA grant (NIH (NCRR) 1UL1RR025011)

U of Wisconsin-Madison + Marshfield Clinic Marshfield Clinic Research foundation

marshfieldclinic.org/birc

Page 3: Vojtech huser-data-warehouse-evaluation-2010-04-idr-snapshot014c

Vojtech Huser, MD, PhDVojtech Huser, MD, PhD

3

Agenda Introduction Methods

Design requirements Measure components

Results Marshfield Clinic

Discussion/Conclusion

marshfieldclinic.org/birc

Page 4: Vojtech huser-data-warehouse-evaluation-2010-04-idr-snapshot014c

Vojtech Huser, MD, PhDVojtech Huser, MD, PhD

4marshfieldclinic.org/birc

Clinical Informatics Subfield of Biomedical Informatics

Background

Biomedical and Health Informatics

Bioinformatics(cellular and molecular)

Informatics = Technology + Information + People

Public Health Informatics

(population)

Medical (Clinical) Informatics

(person)

Translational InformaticsResearch Informatics

Hersh BMC Medical Informatics and Decision Making (2009) 9:24 doi:10.1186/1472-6947-9-24

Background

Page 5: Vojtech huser-data-warehouse-evaluation-2010-04-idr-snapshot014c

Vojtech Huser, MD, PhDVojtech Huser, MD, PhD

5marshfieldclinic.org/birc

Clinical Informatics Subfield of Biomedical Informatics

Background

Biomedical and Health Informatics

Bioinformatics(cellular and molecular)

Informatics = Technology + Information + People

Public Health Informatics

(population)

Medical (Clinical) Informatics

(person)

Translational InformaticsResearch Informatics

Hersh BMC Medical Informatics and Decision Making (2009) 9:24 doi:10.1186/1472-6947-9-24

Background

Page 6: Vojtech huser-data-warehouse-evaluation-2010-04-idr-snapshot014c

Vojtech Huser, MD, PhDVojtech Huser, MD, PhD

6

Introduction Advances

Federated research data warehouse Future data growth

Conduct of research RCT vs. cohort and case-control studies

IDR = Integrated Data Repository Choice of multiple institutions Quick evaluation of an IDR

marshfieldclinic.org/birc

Frueh FW. Back to the future: why randomized controlled trials cannot be the answer to pharmacogenomics and personalized medicine. Pharmacogenomics 2009;10(7):1077-81.

TRIAD

Page 7: Vojtech huser-data-warehouse-evaluation-2010-04-idr-snapshot014c

Vojtech Huser, MD, PhDVojtech Huser, MD, PhD

7

SHRINE: Query federation example

Weber at al. (2009) Application of Information Technology: The Shared

Health Research Information Network (SHRINE): A Prototype Federated Query Tool for Clinical Data Repositories

JAMIA 2009;16:624-63

Fusaro et al. Electronic Medical Record Analysis Using Cloud

Computing AMIA CRI summit, 2010

marshfieldclinic.org/birc

Page 8: Vojtech huser-data-warehouse-evaluation-2010-04-idr-snapshot014c

Vojtech Huser, MD, PhDVojtech Huser, MD, PhD

8

Motivation Motivation

10 institutions with IDRs; budget allows only 3; How do you choose?

Tracking improvement within an organization

Assumptions (limitations) Goal: lifetime and complete EHR (+genetics, +behavior, +environment) Improvement limits General measures (rather then project specific) Simple evaluation, pragmatic approach Dichotomous approach to data sources (not profiling them) Early stage (community interest solicitation)

linkedin.com/in/vojtechhuser

Page 9: Vojtech huser-data-warehouse-evaluation-2010-04-idr-snapshot014c

Vojtech Huser, MD, PhDVojtech Huser, MD, PhD

9

Measure Measures

Qualitative description Quantitative: Simple counts, Composite measure

Composite scores: APGAR, FICO, Google PageRank Score: A short-hand way to communicate about a complex concept

Data warehouse quality Size (1M vs. 10M), quality (billing data vs. structured EHR data)

Possible requirements Design issues

Completeness, Concision (simple), Measurability (objective, reliable), Independence

Our requirements Intuitive to interpret Facilitates improvement Fairness

linkedin.com/in/vojtechhuser

Page 10: Vojtech huser-data-warehouse-evaluation-2010-04-idr-snapshot014c

Vojtech Huser, MD, PhDVojtech Huser, MD, PhD

10

Data Structures (VDW)

linkedin.com/in/vojtechhuser

Page 11: Vojtech huser-data-warehouse-evaluation-2010-04-idr-snapshot014c

Vojtech Huser, MD, PhDVojtech Huser, MD, PhD

11

Data Structures (i2b2)

linkedin.com/in/vojtechhuser

Page 12: Vojtech huser-data-warehouse-evaluation-2010-04-idr-snapshot014c

Vojtech Huser, MD, PhDVojtech Huser, MD, PhD

12

Data Structures (HealthFlow)

linkedin.com/in/vojtechhuser

healthcareworkflow.wordpress.com

Page 13: Vojtech huser-data-warehouse-evaluation-2010-04-idr-snapshot014c

Vojtech Huser, MD, PhDVojtech Huser, MD, PhD

13

Data structures (data schemas) VDW

Several tables Data domain specific

structures Optimized for queries,

users

i2b2, HealthFlow One event table (+ attributes)

Generic data structures Flexibility, extensibility

Page 14: Vojtech huser-data-warehouse-evaluation-2010-04-idr-snapshot014c

Vojtech Huser, MD, PhDVojtech Huser, MD, PhD

14

Event table (IDR Snapshot v1)

code.google.com/p/IDRSnapshot

(Event-DOB) + ‘3000-01-01’

Page 15: Vojtech huser-data-warehouse-evaluation-2010-04-idr-snapshot014c

Vojtech Huser, MD, PhDVojtech Huser, MD, PhD

15

Results Events table (ev_measure01)

Generation took 2.5 hours Size: 43 GB (includes additional info)

code.google.com/p/IDRSnapshot

Page 16: Vojtech huser-data-warehouse-evaluation-2010-04-idr-snapshot014c

Vojtech Huser, MD, PhDVojtech Huser, MD, PhD

16

Code

code.google.com/p/IDRSnapshot

Page 17: Vojtech huser-data-warehouse-evaluation-2010-04-idr-snapshot014c

Vojtech Huser, MD, PhDVojtech Huser, MD, PhD

17

Code

code.google.com/p/IDRSnapshot

Page 18: Vojtech huser-data-warehouse-evaluation-2010-04-idr-snapshot014c

Vojtech Huser, MD, PhDVojtech Huser, MD, PhD

18

Code

code.google.com/p/IDRSnapshot

Page 19: Vojtech huser-data-warehouse-evaluation-2010-04-idr-snapshot014c

Vojtech Huser, MD, PhDVojtech Huser, MD, PhD

19

Cross institution comparison

code.google.com/p/IDRSnapshot

Page 20: Vojtech huser-data-warehouse-evaluation-2010-04-idr-snapshot014c

Vojtech Huser, MD, PhDVojtech Huser, MD, PhD

20

Monitoring

code.google.com/p/IDRSnapshot

Page 21: Vojtech huser-data-warehouse-evaluation-2010-04-idr-snapshot014c

Vojtech Huser, MD, PhDVojtech Huser, MD, PhD

21

What’s in it for me? Use it

Download our SQL code Use it to evaluate your IDR

Collaborate with us Paper in writing (3 institutions involved so far)

IDR Snapshot version 2

code.google.com/p/IDRSnapshot

IRB note: aggregate numbers

Page 22: Vojtech huser-data-warehouse-evaluation-2010-04-idr-snapshot014c

Vojtech Huser, MD, PhDVojtech Huser, MD, PhD

22

Future work Learn from IDRs of other institutions IDR Snapshot version 2

Event table structure HL7 vMR (lite, SQL format)

Delphi method to extend measures Qualitative measures (transfers, insurance, PHR, family history) Institutional context Quantitative measures (general level, more complex “average patient“ pattern)

Structured data Beyond claims data (e.g. nursing homes, decision support audit trails)

Specialty snapshot Prioritizing CDS interventions CKD example (dialysis data)

code.google.com/p/IDRSnapshot

Page 23: Vojtech huser-data-warehouse-evaluation-2010-04-idr-snapshot014c

Vojtech Huser, MD, PhDVojtech Huser, MD, PhD

23

Thank you [email protected] http://code.google.com/p/idrsnapshot

Slides: http://www.linkedin.com/in/vojtechhuser http://marshfieldclinic.org/birc Questions?