Area 4 SHARP Face-to-Face Conference
Phenotyping Team – Centerphase Project
Assessing the Value of Phenotyping Algorithms
June 30, 2011
Topics
Centerphase Background Project Overview Hypothesis Research Design Results to-date Next Steps
Centerphase: Background
CENTERPHASE SOLUTIONS, INC. is a technology-driven services company formed through a collaboration with Mayo Clinic in 2010
The goal is to leverage electronic medical records (EMRs) and clinical expertise from academic medical centers and other research sites to address a broad array of healthcare opportunities
The initial focus is to support enhanced design, planning and execution of clinical trials
Future areas include comparative effectiveness, pharmacoeconomics, compliance and epidemiological studies
Centerphase’s role on the Phenotyping Team is to evaluate the effectiveness (cost and time) of using phenotyping algorithms for identifying patient cohorts
It’s About Speed AND Accuracy…
Hypothesis
The development of phenotyping algorithms and tools can reduce time and cost while maintaining or enhancing quality, associated with identifying patient cohorts for multiple secondary uses including clinical trials and care management.
1. Choose a use case that can provide valuable insights into a real world application
2. Develop a phenotyping methodology (“flowchart”) to identify the patient cohort
3. Generate a random sample of patients from Mayo EMR system based on ICD9-code
4. Conduct algorithm-driven and manual processes in parallel on the sample
5. Compare the time, cost and accuracy of results from the algorithm-driven to manual process
Approach
Type II Diabetes Mellitus (T2DM): 90-95% of all adult cases of diabetes
Multi-stage phenotype representing a combined adaptation of:
– The eMERGE Northwestern T2DM algorithm for clinical trial selection and
– The group practice reporting options (GPRO) as defined under NCQA for population management under the Southeast Minnesota Beacon project
Diagnosed and Undiagnosed Diabetes Source: 2005–2008 National Health and Nutrition Examination Survey
Diabetes is a growing epidemic in this country: 25.8 million (8.3% of population) have diabetes. Last year, 1.9 million new cases alone in population of 20 years or older (CDC).
Initial Use Cases
Use Cases
Case 1: Care Management Identify all high risk patients in a pool of 500 cases
Case 2: Clinical TrialIdentify patients that are good candidates for a study
Phenotype MethodologyT2DM
ICD9 Code
Screen 1: Age
Screen 2: Medications
Screen 3:Labs & Vitals
Patient Cohort
eMERGEAlgorithm for T2DM
Beacon Criteria for categorizing patient risk Identified as high risk or
“RED” patient
Note: All screens based on two-year measurement period 1/1/09 – 12/31/10
Blood Glucose: HbA1c > or = 9
Cholesterol: LDL > 130
Blood pressure: Systolic > 160 & Diastolic > 100
If ANY of the most recent values exceed allowable levels OR ANY of these elements has not been captured in the measurement period, patient is classified as high risk or RED
Identified as T2DM patient
Randomly generate ONE sample set of patient records from database:Based on T2DM ICD9 codes from at least 2 visits during measurement period
Sample Patient Records
Screens 1 -3 Screens 1 -3
Patient Result Set
Patient Result Set
Manual Process
Algorithm-DrivenProcess
Compare time, cost and accuracy of results
Study coordinator (SC) conducts manual review of patient charts, and monitors activity time
Programmer develops and runs algorithm to query records, and monitors development and run time
Research Design
Validation and Evaluation Process
“Dry Run”20 Charts
500 Charts
Step 1
• Review each chart for manual and algorithm processes to identify any screening errors
• Confirm approaches are consistent
• Refine procedures as appropriate
Step 2
• Start with 50; review results and adjust if necessary
• Complete manual reviews
• Collect time, cost and patient result sets
• Conduct data queries
• Analyze results and evaluate / compare performance of methods
Step 1 Completed Step 2 Underway
And How Did We Do…
Initial Results• 50 Charts reviewed
• Manual process*• Identified 10 “Red” (high risk) patients• Required 11.5 total hours**
• Algorithm-driven process*• Identified 8 “Red” patients
All 8 were identified in manual process Missed 2 patients (false negatives)
• Required 7.4 hours**
• For the purposes of this presentation, the following analysis extrapolated these results to evaluate the impact on 500 patients…. Actual findings will be reported upon completion of 500 charts
* Currently evaluating accuracy of both manual and algorithm-driven processes** Includes time for manual validation of all Red charts
Preliminary Analysis: Case 1 - Care Management Extrapolated to 500 Charts based upon Initial 50 Charts
Algorithm
80%Less Costly
Algorithm
90%Faster
Note: Costs and hours reflect time for secondary manual validation of all Red charts identified through both processes
Preliminary Analysis: Case 2 – Clinical TrialsExtrapolated to 500 Charts based upon Initial 50 Charts
Manual Method
Algorithm Method
80% fewer charts to review Over 30 hours saved
Preliminary Comparison: Algorithm-driven to Manual processAlmost 50% cost savings
Note: Costs and hours reflect time for secondary manual validation of all Red charts identified through both processes
Preliminary Conclusions and Next Steps
Initial takeaways:If extrapolated results are validated….
•Applying algorithms to identify subsets of patients can save time and be cost effective
•Algorithms can be most effective when search for larger numbers of patients
•More work needs to evaluate relative accuracy
Sample Patient Records
Screens 1 -3 Screens 1 -3
Patient Result Set
Patient Result Set
Manual Process
Algorithm-DrivenProcess
Next steps:•Complete review of 500 charts•Document results in white paper or manuscript
Top Related