Ambulatory Safety & Quality Initiative: Enabling Quality Measurement

Rebecca Roper Session 91

September 10, 2012

Agency for Healthcare Research and Quality– Long-term and system-wide improvement of

health care quality Only federal agency with a focus on health services

research With an expanding focus on implementation and system

change

– Not a policy-making or regulatory agency

Ambulatory, Safety, and Quality (ASQ) Initiative

Scope of ambulatory care, increasing, as volume and complexity of care are expanding

Institute of Medicine Report, Patient Safety: Achieving a New Standard for Care (IOM, 2004). Priority Areas

Cornerstone of ASQ is to explore and demonstrate how health information technology (health IT) can improve quality of care provided in ambulatory care setting and for transitions in care settings

See: http://healthit.ahrq.gov/ASQ

ASQ Grant Initiatives

Includes four health IT-focused request for applications (RFAs): 1) Enabling Quality Measurement through Health IT

(EQM), HS-07-0022) Improving Quality through Clinician Use of Health

IT (IQHIT), HS-07-0063) Enabling Patient-Centered Care through Health IT

(PCC), HS-07-0074) Improving Management of Individuals with

Complex healthcare Needs through Health IT (MCP), HS-08-002

RFA-Specific Summary Products

RFA-specific ReportLinks to final grant reportsCross link to stories and webinar

RFA-specific Exemplary storiesWritten exemplary storiesVideo exemplary stories

RFA-Specific National Webinar

Goals of RFA-Specific Report

1) Summarizes the extent to which these projects addressed the research foci of RFA

2) Identifies practical insight3) Presents illustrative initial findings to:

– Inform research discussion – Provide guidance to other entities implementing

health IT systems for quality measurement and improvement

Health IT Ambulatory, Safety & Quality: Enabling Quality Measurement (EQM)

Key findings and lessons from the 17 grants of the EQM grant initiative – Helps researchers understand

the realities and complexities in quality measurement through health IT

http://healthit.ahrq.gov/ASQEQMRPT2012.pdf

EQM Investigators and Projects

  PI Name Project Title

 Bailey, Thomas (Kilbridge, Peter) Surveillance for Adverse Drug Events in Ambulatory Pediatrics

  Berner, Eta Closing the Feedback Loop to Improve Diagnostic Quality

  Davidson, Arthur Colorado Associated Community Health Information Exchange

Hazlehurst, Brian Automating Assessment of Asthma Care Quality

Kaushal, Rainu Developing and Using Valid Clinical Quality Metrics for HIT with HIE

  Kmetik, Karen Cardio-Hit Phase II

  Lazarus, Ross Electronic Support for Public Health - Vaccine Adverse Event Reporting System

 Lehmann, Christoph Medication Monitoring for Vulnerable Populations via IT

Logan, Judith Improving Quality In Cancer Screening: The Excellence Report For Colonoscopy

EQM Investigators and Projects

  PI Name Project Title

  McColm, Denni Standardization and Automatic Extraction of Quality Measures in an Ambulatory EHR

  Schneider, Eric Massachusetts Quality E-Measure Validation Study

  Selby, Joe Feedback of Treatment Intensification Data to Reduce Cardiovascular Disease Risk

Thomas, Eric Using Electronic Records to Detect and Learn from Ambulatory Diagnostic Errors

Turchin, Alexander Monitoring Intensification of Treatment for Hyperglycemia and Hyper

lipidemia

 Vogt, Thomas(Williams, Andrew)

Crossing the Quality Assessment Chasm: Aligning Measured and True Quality of Care

  Weiner, Mark Using IT to Improve the Quality of CVD Prevention & Management

 Wu, Winfred(Mostashari, Farzad)

Bringing Measurement to the Point of Care

Enabling Quality Measurement (EQM) Initiative

Strategies for the development, deployment and export of quality measures from electronic health record systems.

1) Development of retooled quality measures via health IT2) Development of de novo quality measures via health IT

Issues addressed include:– Measure development across episodes of care– Clinical data needs for quality measurement

export and reporting– Reporting of quality data for improvement

http://grants.nih.gov/grants/guide/rfa-files/RFA-HS-07-002.html

EQM Foci and Associated Projects

Total of 17 Projects with a variety of foci

* Some projects had multiple focus areas

Two EQM Foci were not explicitly addressed:

  FOA Focus Number of Projects*

1. Developing new measures 5

2. Accuracy of measurement 10

3. Capturing and integrating data 12

4. Feedback to clinicians 6

5. Efficiency of measurement 3

6. Interoperable data systems to measure quality and safety for episodes of care across settings

7. HIE as a data source for quality and safety measurement, including public reporting

Counts of Type of Health IT for EQM Grantees

Clinical Information Systems

Natural Language Processing

Automated Surveillance System

Electronic Health Records

Data Warehouse/Data Repository

Clinical/Medication Reminders

Results Reporting

Disease Registry

Interface

Quality of Care Decision Support

Interactive Voice Response/Telephony

HIE/Regional Health Information Organization

Standards

Clinical Decision Support

0 2 4 6 8 10 12 14 16

Counts of EQM Grantees by IOM Priority Area

Asthma

Care Coordination

Children With Special Needs

Diabetes

Cancer Screening

Hypertension

Immunization

Heart Disease

Major Depression

Obesity

Tobacco Treatment

0 1 2 3 4 5 6 7 8

Counts of EQM Grantees by Type of Ambulatory Care Setting

*FQHC (Federally Qualified Health Center) CHC (Community Health Clinic)

FQHC or CHC*

Primary Care

Outpatient Clinic

Specialty Practice

Other

0 1 2 3 4 5 6 7 8 9 10

Developing New Measures

Projects focusing on developing new measures– Berner– Kaushal– Thomas– Turchin– Vogt and Williams

For a summary of findings from all projects that addressed “Developing New Measures”, see “Findings and Lessons from the Enabling Quality Measurement Through Health IT Grant Initiative”

http://healthit.ahrq.gov/EQMReport2012.pdf

Developing New Measures, Selected Findings

Berner: new measures were calculated through patient feedback collected via telephone and interactive voice response. – The topics of measures included patient-reported

problem resolution, medication adherence, and followup activity

Kaushal: expert panel reviewed measures related to the quality of ambulatory care.– 18 existing measures were prioritized to be

generated by EHRs, and 14 new measures were identified in underrepresented measurement areas.

Vogt and Williams: developed EHR-based indices for the quality of cardiovascular disease management services in primary care.

Accuracy of Measurement

Projects focusing on accuracy of measurement

– Bailey and Kilbridge– Hazlehurst– Kaushal– Kmetik– Lehmann

– McColm– Thomas– Turchin– Weiner– Wu and Mostashari

For a summary of findings from all projects that addressed “Accuracy of Measurement”, see “Findings and Lessons from the Enabling Quality Measurement Through Health IT Grant Initiative”

http://healthit.ahrq.gov/EQMReport2012.pdf

Accuracy of Measurement, Selected Findings

Kmetik tested the accuracy of patient, medical, and system-related reasons for excluding patients from measure denominators. – Frequency of exclusions was relatively low and the

health IT systems identified them accurately compared to manual chart review.

Bailey and Kilbridge used NLP to search clinical, demographic, encounter, laboratory, and pharma cy data to identify ADEs in children with cystic fibrosis, sickle cell disease, and cancer.– The system did not perform as well as chart review.– The system, however, identified 4 to 20 times more

ADEs than the typical voluntary reporting system.

Accuracy of Measurement, Selected Findings

Hazlehurst tested an NLP approach to the measurement of 18 measures related to the quality of outpatient asthma care. – Most health IT-enabled measures gave results

comparable to manual chart review – Sensitivity rates above 60 percent for 16 of the 18

measures Kaushal tested the reliability of electronic

generation of 11 established measures at a local FQHC.– Sensitivity of 88 percent and specificity of 89 percent

compared to manual chart review. – Reliability varied con siderably across measures, with

measures relying on data from both structured fields and unstructured notes tending to be less reliable.

Accuracy of Measurement, Selected Findings

Lehmann implemented flags in the EHR to identify patients in need of medication monitoring according to measures developed by the National Committee for Quality Assurance. – This was significantly more accurate than manual

chart review with higher PPV, sensitivity, and specificity.

McColm compared manual coder performance with electronic extraction and coding of data from the EHR.– Electronic extraction and coding was highly accurate

for case identification for blood pressure, hemoglobin A1c, and low-density lipoprotein data elements.

Capturing and Integrating Data

Projects focusing on accuracy of measurement– Bailey and Kilbridge– Davidson– Hazlehurst– Lazarus– Lehmann– Logan

– McColm– Schneider– Turchin– Vogt and Williams– Weiner– Wu and Mostashari

For a summary of findings from all projects that addressed “Capturing and Integrating Data”, see “Findings and Lessons from the Enabling Quality Measurement Through Health IT Grant Initiative”

http://healthit.ahrq.gov/EQMReport2012.pdf

Capturing and Integrating Data, Selected Findings

Logan implemented and evaluated a set of 15 measures of the quality of colonoscopy procedures. – This confirmed the feasibility of the generating

measures using data captured at the point of care through custom data entry screens in an EMR.

Turchin researchers developed NLP software to extract information on insulin dosing to identify patients for whom medication therapy was intensified.

Davidson worked with nine local CHCs to collaboratively define requirements for a shared quality information sys tem – Team developed business requirements for templates

for capturing data re lated to diabetes and smoking cessation.

Capturing and Integrating Data, Selected Findings

Vogt and Williams developed EHR-based quality indices for 11 cardiovascular primary care services.– Even though the indices were imple mented in Kaiser

Permanente sites that had substantial experience using the same EHR, investigators had to create an extensive process for extracting, cleaning, and coding the data.

Weiner integrated EHR data from two institu tions that cared for some of the same patients to test a method of risk adjusting physician-level diabetes quality of care rankings. – Using a linkage between database tables of

demographics and patient identifiers from the two systems, the researchers were able to find patients with visit activity in both locations and conduct a descrip tive analysis of their patterns of care.

Capturing and Integrating Data, Selected Findings

Wu and Mostashari created health IT tools to assist primary care physicians in small practices in measuring the quality of care. – The software “hard-coded” 34 existing measures

into the EHR, making them easily accessible to the provider.

Schneider attempted to develop a measurement approach that integrated data from primary care practices participating in three community-wide, multi-payer HIE efforts. – Several barriers prevented a successful evaluation

of the adequacy of these data sources for performance measure ment

Feedback to Clinicians

Projects focusing on feedback to clinicians– Berner– Davidson– Lehmann– Logan– Selby– Wu and Mostashari

For a summary of findings from all projects that addressed ‘Feedback to Clinicians”, see “Findings and Lessons from the Enabling Quality Measurement Through Health IT Grant Initiative”

http://healthit.ahrq.gov/EQMReport2012.pdf

Feedback to Clinicians, Selected Findings

Selby offered feedback to staff responsible for population management on the need to intensify medication treatment for those patients with out-of-range values. – Modest impact on treatment intensification rates for

patients with elevated systolic blood pressure and low-density lipoprotein levels.

– No observed impact on proportions of patients with levels in the target range

Logan posted monthly, physician-specific performance reports – Based on feedback from participating physicians, the

researchers stream lined the reports and integrated them into the EHR (analysis of the impact of the reports is in progress).

Feedback to Clinicians, Selected Findings

Wu and Mostashari provided patient-specific clini cian reminders and decision support at the point of care and real-time reports on a provider’s overall performance on the quality measures. – Provider performance on nearly all measures exhibited statis

tically significant improvements, ranging from 5 to 20 percentage points per measure.

Davidson found that use of built-in templates to support clinicians providing and documenting care led to improvements in measures related to smoking cessation at some sites.

Lehmann gave primary care providers an EHR-generated paper bulletin listing patients due for therapeutic monitoring tests related to one or more medications. – Patients appearing on the bulletins were somewhat more likely

to receive monitoring within 2 months.

Efficiency of Measurement

Projects focusing on efficiency of measurement– Bailey and Kilbridge– Lazarus– Thomas

For a summary of findings from all projects that addressed ‘Feedback to Clinicians”, see “Findings and Lessons from the Enabling Quality Measurement Through Health IT Grant Initiative”

http://healthit.ahrq.gov/EQMReport2012.pdf

Efficiency of Measurement, Selected Findings

Bailey and Kilbridge used NLP to search clinical, demographic, encounter, laboratory, and pharma cy data to identify pediatric ADEs. – Time-savings en abled the researchers to identify a greater

number of serious ADEs. Lazarus created a method for prospectively inte grating

multiple types of EHR data with the goal of identifying potential adverse events related to vaccinations. – Data showed that 2.6 percent of vaccinations resulted in

possible reactions. Thomas applied electronic triggers that might be

associated with a diagnostic error – This methodology was more efficient than conducting

random record reviews and identified errors that were more consequential than many routine errors.

Using Electronic Health Records To Measure and Improve Quality for Colonoscopy Procedures

Judith Logan, OHSU Effectiveness of colonoscopy screening

procedures, typically done in an ambulatory setting, depends on providing high quality examinations that result in accurate diagnoses and few complications.

Story and video highlight how the investigators were able to use data from electronic medical records for quality measurement for colonoscopy procedures. Investigators discuss the lessons learned as they formulated, implemented, presented measures the clinicians.

http://healthit.ahrq.gov/EQMStoryLogan2012.pdfhttp://healthit.ahrq.gov/EQMLoganVideo

LOGAN (continued)

Logan and her team created and evaluated an electronic quality measurement and feedback program—known as excellence report-- for colonoscopies.

Findings– Physicians receptive to feedback as a way to improve

effectiveness and safety of their procedures. – Point-of-care data entry was not seen as overly burdensome. – Physicians wished to have feedback shared broadly.

Continued Use– Excellence report is now delivered in the EHR.

http://healthit.ahrq.gov/EQMStoryLogan2012.pdfhttp://healthit.ahrq.gov/EQMLoganVideo

Developing and Testing Quality Measures for Interoperable Electronic Health Records

Rainu Kaushal, Weil Cornell Demonstrate effective use of both

electronic health records (EHRs) and health information exchange (HIE) to electronically measure quality of care delivered in ambulatory settings.

Story and video highlight the methods used to identify quality measures that could be supported and impacted by EHRs and HIE; results of reliability testing of the quality measures; and subsequent the impact of the investigators’ work on national health IT policy and “Meaningful Use”.

http://healthit.ahrq.gov/EQMStoryKaushal2012.pdf http://healthit.ahrq.gov/EQMKaushalVideo

Kaushal (continued)

Dr. Rainu Kaushal and team pursued the identification, prioritization, development, and reliability testing of quality measures using an interoperable EHR in a primary care setting.

Process: – With the assistance of an expert panel, they applied a four-part conceptual

framework to identify 18 prioritized measures of chronic disease management and preventive services.

Findings:– Electronic reporting correctly identified 88 percent of the patients who received

recommended care and 89 percent of the patients who did not receive recommended care compared to manual chart review.

Sustainability: – Fourteen new HIE-enabled measures were developed in five important categories:

test ordering, medication management, referrals, followup after discharge, and revisits.

http://healthit.ahrq.gov/ASQStoryKaushal2012.pdf http://healthit.ahrq.gov/EQMKaushalVideo

Standardization and Automatic Extraction of Quality Measures in an Ambulatory EHR,

McColm Exemplary Story - Written

The lack of standards for clinical documentation in an EHR is a major barrier to automated quality measurement – Denni McColm’s team established standards for

clinical documentation and demonstrated the efficiency and accuracy of using data extraction and reporting to perform quality measurement in the ambulatory care setting.

Story highlights the methods used to establish standards and findings from the implementation of an automated system for data extraction of quality measures in the ambulatory setting, including valid, reliable reports that provide actionable insight for the measurement and analysis of care.

http://healthit.ahrq.gov/EQMStoryMcColm2009.pdf

Use of Natural Language Processing to Improve Quality Measurement,

a National Web Conference

Purpose– To address the existing gap between a health care and

a public health practitioner's competencies as it relates to the health IT environment.

– This specific webinar illustrated how new methods of analyzing free text data stored in electronic health records can impact quality measurement.

Learning Objectives– Discuss the principles of NLP design and

implementation.– Describe how NLP is used to operationalize the

assessment of quality measurement in asthma care.– Explain how NLP is used in monitoring intensification of

treatment for patients with diabetes.

http://healthit.ahrq.gov/nlp-eqmwebinar

Additional EQM Products and Links

ASQ Web site: http://healthit.ahrq.gov/ASQ

EQM Report: http://healthit.ahrq.gov/EQMReport2012.pdf

NLP Webinar: http://healthit.ahrq.gov/nlp-eqmwebinar

Kaushal Video: http://healthit.ahrq.gov/EQMKaushalVideo

Kaushal Story: http://healthit.ahrq.gov/EQMStoryKaushal2012.pdf

Logan Video: http://healthit.ahrq.gov/EQMLoganVideo

Logan Story: http://healthit.ahrq.gov/EQMStoryLogan2012.pdf

McColm Story: http://healthit.ahrq.gov/EQMStoryMcColm2009.pdf