New and Improved: Registries for Evaluating Patient

Outcomes and HIT

Elise Berliner, Ph.D.

Director, Technology Assessment Program

Center for Outcomes and Evidence

Agenda

Introduction to the Second Edition

– Elise Berliner, Ph.D., Task Order Officer

Review of major changes and new sections

– Richard Gliklich, M.D., Senior Editor

– Nancy Dreyer, Ph.D., M.P.H., Senior Editor

Planning for the Third Edition

– Audience discussion, led by Dr. Berliner

Background

First edition, published in 2007, has been widely used as a reference for designing, operating, and evaluating patient registries.

– Numerous citations (>60) in literature and in significant government publications (e.g., Federal Register, FCC Report to the President, etc.)

As registries continue to evolve, many new methodological and practical issues have arisen.

Purpose of the Second Edition

Update and expand the first edition of the User’s Guide with new information gathered from recent publications and reported experiences encountered by researchers and other professionals who utilize registries for research.

Expound on selected topics in the original guide and add new topics that deserve further in-depth discussion.

Process for Creating the Guide

Topics identified based on public comments received for the first edition.

New sections: – Author and reviewer teams assembled with similar balance

to first edition.– Posted for public comment and revision.

Original chapters:– Original authors and reviewers were invited to participate.

Additions made for new topic areas when necessary.– Full, revised handbook posted for public comment.

Open call for case examples.

Second Edition Collaborators

55 contributors from industry, academia, health plans, physician societies, and government.

49 invited peer reviewers, plus public comment.

38 case studies illustrate challenges and solutions (including 20 new examples).

Senior Editors: R. Gliklich; N. Dreyer. Managing Editor, M. Leavy. Outcome DEcIDE Center.

Second Edition: Table of Contents

1. Patient Registries (Overview)

2. Planning a Registry

3. Registry Design (includes Planning for the End of a Patient Registry)

4. Use of Registries in Product Safety Assessment

5. Data Elements for Registries

6. Data Sources for Registries

7. Linking Registry Data: Technical and Legal Considerations

8. Principles of Registry Ethics, Data Ownership, and Privacy

9. Recruiting and Retaining Participants in the Registry

10. Data Collection and Quality Assurance

11. Interfacing Registries With Electronic Health Records

12. Adverse Event Detection, Processing, and Reporting

13. Analysis and Interpretation of Registry Data To Evaluate Outcomes

14. Assessing Quality

©2010 Outcome Sciences, Inc.

Registries for Evaluating Patient Outcomes: A User’s GuideSecond Edition

Richard Gliklich, M.D., Senior EditorNancy Dreyer, Ph.D., M.P.H., Senior Editor

• Changes to Sections on Creating and Operating Registries:

– All chapters were updated. Some chapters had more significant additions.

– Multiple new case studies.

• Review of “Linking Registry Data.”

• Review of “Interfacing Registries with EHRs.”

• Highlighted Case Examples.

Review of Major Changes

• “Planning a Registry” (Chapter 2) now discusses public-private partnerships.

• “Registry Design” (Chapter 3) and “Analysis and Interpretation” (Chapter 13) were reorganized to align more closely.

• “Data Elements” (Chapter 5) was expanded to discuss new data standards.

Updates to 1st Edition Chapters

• “Principles of Registry Ethics, Data Ownership, and Privacy” (Chapter 8) addresses new legislation:– Patient Safety and Quality Improvement Act of 2005

(PSQIA)

– Genetic Information Nondiscrimination Act of 2008 (GINA)

– Health Information Technology for Economic and Clinical Health Act (HITECH Act)

• “Adverse Event Detection, Processing, and Reporting” (Chapter 12) discusses risk evaluation and mitigation strategies (REMs).

Updates to 1st Edition Chapters

13. Identifying and Responding to Adverse Events Found in a Registry Database

14. Selecting Data Elements for a Registry

Case Examples (new examples highlighted)

1. Using Registries To Understand Rare Diseases2. Creating a Registry To Fulfill Multiple Purposes and Using a Publications Committee To Review Data Requests3. Using a Registry To Track Emerging Infectious Diseases4. Using a Collaborative Approach To Plan and Implement a Registry5. Using a Scientific Advisory Board To Support Investigator Research Projects6. Determining When To Stop an Open-Ended Registry7. Designing a Registry for a Health Technology Assessment8. Assessing the Safety of Products Used During Pregnancy9. Designing a Registry To Study Outcomes10. Analyzing Clinical Effectiveness and Comparative Effectiveness in an Observational Study11. Using a Registry To Assess Long-Term Product Safety12. Using a Registry To Monitor Long-Term Product Safety13. Using Performance Measures To Develop a Dataset14. Developing and Validating a Patient-Administered Questionnaire15. Understanding the Needs and Goals of Registry Participants16. Using Validated Measures To Collect Patient-Reported Outcomes17. Integrating Data From Multiple Sources With Patient ID Matching18. Linking Registries at the International Level19. Linking a Procedure-Based Registry With Claims Data To Study Long-Term Outcomes20. Linking Registry Data To Examine Long-Term Survival21. Considering the Institutional Review Board Process During Registry Design22. Issues With Obtaining Informed Consent23. Building Value as a Means To Recruit Hospitals

Case Examples (new examples highlighted)

26. Using Registry Tools To Recruit Sites27. Using Proactive Awareness Activities To Recruit Patients for a Pregnancy

Exposure Registry 28. Using Reimbursement as an Incentive for Participation29. Data Collection Challenges in Rare Disease Registries30. Managing Care and Quality Improvement for Chronic Diseases31. Developing a Performance-Linked Access System32. Using Audits To Monitor Data Quality33. Challenges in Creating Electronic Interfaces Between Registries and Electronic

Health Records34. Creating a Registry Interface To Incorporate Data From Multiple Electronic Health

Records35. Technical and Security Issues in Creating a Health Information Exchange36. Developing a New Model for Gathering and Reporting Adverse Events37. Using Registry Data To Evaluate Outcomes by Practice38. Using Registry Data To Study Patterns of Use and Outcomes

Case Examples (new examples highlighted)

Rationale: Increasingly, statistical methods are used to link data from multiple de-identified sources, including registries. For these projects:

– What is the risk of identifying patients by combining data from multiple registries?

– What are the legal and ethical requirements for researchers to ensure patient privacy and confidentiality?

• Linking Registry Data: Overview– Chapter is divided into 2 sections: Technical Aspects and Legal Considerations.

– Technical Aspects: What is a feasible technical approach to linking the data?

– Legal Considerations: Is the linkage legally feasible under the permissions, terms, and conditions that applied to the original compilations of each data set?

New Chapter: Linking Registry Data

1616

Linking Registry Data: Case Example

• Goal: Study long-term patient outcomes for diagnostic cardiac catheterizations and percutaneous coronary interventions (PCI).

• Data Sources: CathPCI registry lacks long-term follow-up. Medicare data lacks detailed procedure information.

• Legal Approach: After review of HIPAA, Common Rule, project team determined that linkage required use of limited datasets only and IRB approval for linkage project.

• Technical Approach: CathPCI data were linked with Medicare data using probabilistic matching techniques.

Rationale: With national efforts to invest in EHRs, and to advance the evidence base in areas such as effectiveness, safety, and quality through registries and other studies, it is clear that interfacing registries with EHRs will become increasingly important to reduce data collection burden.

This chapter explores issues of interoperability and a pragmatic “building-block approach” towards a functional, open-standards based solution.

New Chapter: Interfacing Registries with Electronic Health Records (EHRs)

Interfacing Registries & EHRs: Overview

• Chapter covers 4 main topics:

– Role of EHRs and patient registries in health care.

– Vision of EHR-registry interoperability.

– Interoperability challenges.

– Partial and potential solutions.

Interfacing Registries & EHRs: Definitions

• EHR: an electronic record of health-related information on an individual that conforms to nationally recognized interoperability standards and that can be created, managed, and consulted by authorized clinicians and staff, across more than one health care organization.

– Individual focused

• Patient registry: an organized system that uses observational study methods to collect uniform data (clinical and other) to evaluate specified outcomes for a population defined by a particular disease, condition, or exposure, and that serves one or more predetermined scientific, clinical or policy purposes.

– Population focused

• Syntactic interoperability (communication): the ability of heterogeneous health information systems to exchange data.– Wiring, application protocol, standard messaging protocol

– Most easily solved

• Semantic interoperability (content): implies that the systems understand the data that has been exchanged at the level of defined domain concepts.– Depends on standard vocabulary and shared data elements

– More difficult to solve; efforts include CDASH, ASTM Continuity of Care Record (CCR), HL7 Continuity of Care Document

• Other issues: managing patient identifiers and authenticating users across multiple applications.

Interoperability Challenges

• We lack a single, complete interoperability model, but, several open standard components provide a reasonably good start which can be improved with further development, testing, and adoption of additional open standard building blocks.

Partial and Potential Solutions

Building-Block Approach: Retrieve Form for Data Capture/HITSP-TP50

• EHR–registry interoperability will be increasingly important as adoption of EHRs and patient registries increases significantly.

• Interoperability requires accurate and consistent data exchange and use of the information that has been exchanged.

• While a complete interoperability solution does not yet exist, enough open standard building blocks do exist today to enable ‘functional interoperability’ which while imperfect, significantly improves workflow and reduces duplication of effort.

Summary

• A registry focused on effectiveness in pain management was made interoperable with a commercial EHR using RFD communication.

• ASTER project: interoperability was achieved for the purpose of reporting adverse event information to the FDA.

• A commercial EHR was made interoperable with a quality reporting initiative for the American College of Rheumatology and with a Physician Quality Reporting Initiative Registry for reporting data to the Centers for Medicare and Medicaid Services.

Interfacing Registries & EHRs: Case Examples

• Review of “Planning for the End of a Registry.”

• Review of “Use of Registries for Product Safety Assessment.”

• Highlighted Case Examples.

• Changes to Section III, Evaluating Registries.

Review of Major Changes

Rationale: Registries may be developed without specific endpoints, making it difficult to determine when to end data collection. A more clear explanation of recommendations for stopping a particular registry is useful from several perspectives, including costs.

This section addresses how to determine when a registry has met its objectives and should be closed.

New Section: When to End a Registry

• Registry protocol should include specific and measurable endpoints against which to judge whether the project should continue or stop.

• Measurable goals for endpoints will make it possible to determine whether the registry has achieved a core purpose and may lead to a reasonable stopping point.

• Conversely, a registry that fails to meet measurable goals and appears to be unable to meet them in a reasonable time is also a candidate to be stopped.

• Other reasons for stopping:– Incomplete or poor quality data.– Registry has outlived the question it was created to answer.– Funding and/or staffing is no longer available.

Stopping Decisions and Registry Goals

• Stopping a registry could mean:

– ceasing all data collection and issuing a final report.

– ceasing to accrue new patients while continuing to collect data on existing patients.

• Regardless, archiving rules should be checked and followed.

• No clear ethical obligation to participants to continue a registry that has outlived its scientific usefulness.

What Happens When a Registry Stops?

• Registries with one sponsor who has decided to stop the registry should be encouraged to engage other stakeholders in discussions of potential transitioning of the registry to other owners.

– Issues of data ownership, property, confidentiality, and patient privacy would need to be satisfactorily addressed to make such transitions possible.

• Reasons to consider preserving registry data:

– If data may be capable of producing a recognized public health benefit that will continue if the registry does.

– If registry has historical importance, such as a registry that tracks the outbreak of a novel infectious disease that may provide insight into the transmission of the disease if not now, then sometime in the future.

– If longitudinal data may be useful for hypothesis generation.

What Happens When a Registry Stops?

Ending a Registry: Case Example

• Bupropion Pregnancy Registry:

– Open-ended, observational exposure-registration and followup study to monitor prenatal exposure to bupropion and detect any major teratogenic effect.

• Registry collected data on over 1,500 exposed pregnant women over 10 years.

• In 2007, advisory committee recommended discontinuation of the registry, based on its conclusion that sufficient information had accumulated to meet the scientific objective of the registry.

Rationale: Registry populations often include patients that are significantly different from patients studied in clinical trials (age, comorbidities, etc.) and therefore are of particular interest in post-approval safety monitoring.

– How should data be monitored for adverse events?

– How is the “expected” rate calculated?

– How often should the data be analyzed?

– What is the legal and ethical responsibility of the registry for reporting, informing physicians and institutions of potential problems?

New Chapter: Use of Registries for Product Safety Assessments

Use of Registries for Product Safety Assessments: Overview

• Chapter divided into 4 sections:

– Registries specifically designed for safety assessment.

– Registries designed for purposes other than safety.

– Signal detection in registries.

– Potential obligations for registry developers in reporting safety issues.

• Registries are well suited to identify effects that can only be observed in a large and diverse population over an extended period of time (drug-drug interactions, genetic differences in drug metabolism, etc.).

• When designing a registry for the purposes of safety, the size of the registry, the enrolled population, and duration of follow-up are all critical to ensure validity of the inferences made based on the data collected.

• Some other challenges:

– Recruiting a meaningful patient population.

– Evaluating the utility of a registry when the entire population-at-risk has not been included (a common scenario.)

– Understanding timing of treatments, switching of treatments, multiple therapies, dose effects, delayed effects, and patient compliance.

Registries Designed for Safety Assessments

• Registries examining comparative effectiveness, the natural history of a disease, evidence in support of national coverage decisions, or quality improvement efforts may gather and report AE data, but may not be able to reliably detect all events.

– Facilitate safety reporting rather than evaluate it.

Registries Designed for Purposes Other than Safety

Use of Registries for Product Safety Assessment: Case Example

• British Society for Rheumatology Biologics Register (BSRBR):

– Prospective observational study to monitor the routine clinical use and long-term safety of biologics in patients with severe rheumatoid arthritis and other rheumatic conditions.

• Registry enrolls patients on anti-TNF agents and a control cohort of patients on DMARDs.

• Data from registry have been used to determine whether an increased risk of tuberculosis existed in patients treated with anti-TNF therapy. Analysis found a differential risk among the three anti-TNF agents (Dixon et al., Annal Rheum Dis 2010).

Evaluating Registries

• Tables in this chapter were re-organized to group the good practices by concept.– Research Quality: Basic Elements and Potential

Enhancements of Good Practice for Establishing and Operating Registries

– Evidence Quality: Indicators of Good Evidence Quality and Indicators of Enhanced Good Evidence Quality for Registries

• Chapter was updated to include good practices related to the new handbook sections.

43

Plan for the Third Edition

Development begins in Fall 2010.

Plan is to include 11 new chapters, plus case examples.

7 topics identified through public comments on second edition.

Topics for the Third Edition

1. Patient identity management

2. Protection of data from litigation

3. Data protection concerns

4. Public-private partnerships

5. Statistical techniques for analyzing combined data

6. Pregnancy registries

7. Registry transitions

Additional Topics

For discussion: What other topics should be addressed in the third edition?

Contact

Elise Berlinerelise.berliner@ahrq.hhs.gov

301-427-1612