Putting It All Together: Informatics Challenges and Opportunities in Health Outcomes ResearchMichelle Erin Johnson

June 2019 – CHIP Clinical Information Sciences Certificate Practicum

Key Findings

Clinical research can be inefficient due to scientific, technical, policy, and governance challenges

It is complex to develop solutions from clinical research findings that can be implemented into clinical practice

Clinical research informatics (CRI) tools and approaches boost research efficiency, innovativeness, and implementation

Health outcomes research can benefit substantially from CRI tools and approaches

My Background

Research ethics & compliance

Technology transfer

Innovation support

Research project management

Data collection & management

Outcomes Research

Goals of the internship

• Develop comprehensive understanding of clinical research informatics approaches, tools, and methods.

Develop

• Identify common informatics challenges and opportunities faced by health outcomes researchers

Identify

• Develop best practices for data collection and data management in health outcomes research

Develop

• Identify new sources of outcomes data and develop best practices for API integration

Identify

Research with human subjects that is:

1) Patient-oriented research. Research conducted with human subjects (or on human tissues, specimens, and cognitive phenomena) for which an investigator directly interacts with human subjects. Excludes in vitro studies that use human tissues that cannot be linked to a living individual. Includes (a) mechanisms of human disease, (b), therapeutic interventions, (c) clinical trials, or (d) development of new technologies.

2) Epidemiological and behavioral studies.

3) Outcomes research and health services research

*Studies falling under 45 CFR 46.101(b) (4) are not considered clinical research by this definition

More: https://www.hhs.gov/ohrp/regulations-and-policy/decision-charts/index.html

NIH Definitionof ClinicalResearch

Clinical Research InformaticsClinical research informatics focuses on developing theories, tools, and solutions to accelerate the translation of research findings from basic science to clinical research to implementation in health practice

“Clinical researchers are faced with significant and increasingly complex workflow and information management challenges… Due to the preceding characteristics …and the recognition that effective and efficient information access is critical to any solution to the many challenges faced by the domain, there has been a corresponding and rapid evolution of the biomedical informatics methods and tools specifically designed to address clinical research information management requirements.”1

Health Outcomes Research

Health outcomes research seeks to identify and improve the end results (outcomes) of the health care delivery system, including its impact on the well-being of patients and populations.

Value-Based Care Model Measures Outcomes

Outcomes Research: Many Data Sources

INSURANCE CLAIMS

EHR PATIENTS CAREGIVERS

CLINICAL RESEARCH

STAFF

DEVICES AND HEALTH APPS

RESEARCH DATA

REPOSITORIES

METADATA

Outcomes Research: Data Collection Modes

Collected Directly from the Patient

via web-based forms

via direct phone calls

via Interactive Voice Response (IVR) calls

via text messages

via smart phone applications

via in-person interviews

via tablet during clinic visits

via paper forms in the mail

via EHR patient portal

Collected Indirectly by Research Staff

via electronic data management systems

via data entry

Collected Indirectly by APIs

via device API integration

via web application integration

Outcomes Research: Integration with Clinical Workflows

• Outcomes research data EHR

• EHR data research data collection systems

• Both directions?

EHR integration

• Patient recruitment in clinic

• Symptom alerts to clinical staff

• Clinician reports

Clinical workflow

integration

Outcomes Research: Integration with Devices

• Fitbit

• Garmin

• Apple Watch

Wearables / fitness trackers

• Pill dispensers

• Blood pressure monitors

• Sleep trackers

• Glucometers and insulin delivery systems

• Smart scales

• Medical alert devices

Personal health devices

• Fertility apps

• Weight loss apps

• Medication management apps

• Mental health and wellbeing apps

Health applications

• Environmental sensors

• Amazon Alexa

• Robotic assistants

Smart home devices

Outcomes Research: Complex Processes

Data collection schedule and frequency varies

Study arms / randomization / interventions

Device management Multi-site studies and large research teams

Outcomes Research: CRI Opportunities

Recruitment tools

Retention & adherence tools

Electronic data capture

Electronic data management systems

Data quality and monitoring

Secure & compliant data sharing across sites and teams

Integration across different data sources, devices, applications, modes

Outcomes Research: Technological Challenges

• Status discrepancies

• Mid-study data collection instrument changes

• Multi-site studies where “go-live” requirements differ by site

Data systems integration challenges

Spam filtering (affects IVR, phone call, and text messaging interventions)

• Sync errors

• Internet connectivity

• Inexplicable data loss

Device issues

Development time / costs

System user access

Outcomes Research: Scientific Challenges

Using multiple data collection modes and methods leads to data

inconsistencies and errors if SOPs are unclear / protocol

gaps

Misleading or confusingly worded survey questions

Survey questions in which participant responses are

subjective and/or contextual

Sampling bias

• E.g. studies using Apple iPhone apps

• E.g. studies enrolling patients at major academic medical centers

Response bias

• Population not represented in the responses

Device data - Rigor, reproducibility and scientific

validity of devices is not always guaranteed

Outcomes Research: Policy Challenges

Survey instrument copyrights

HIPAA and allowable methods for PHI data transfer/storage

Lack of institutional clarity around research data retention / data destruction policies

Gaps in information security policies

Data access restrictions

No institutional guidance or standardized templates for data collection

No guidance or standardized templates for data storage/management

Outcomes Research: Governance Challenges

Implementation of outcomes solutions is highly dependent on institutional risk-benefit analysis and strategic priorities

Solutions must be scientifically validated and demonstrably feasible prior to implementation

Outcomes solutions in clinical practice tend to have limited scope: clinician administered, department administered, disease and/or project specific

How do I build an electronic system to

do this more efficiently?

How do I work with leadership and stakeholders to

implement a solution into

practice?

What policy and

compliance issues do I

need to keep in mind?

What research methods should I

use? Will my evidence be valid and reliable? Do I

understand the problem I’m trying

to solve?

What tools do I need? What data do I need? How do I put

it all together?

Implementing a CRI Solution: Seeing the Trees instead of the ForestIs this even a problem that needs fixing?

Does it require a high-tech solution?

Forest of Effective Outcomes Projects

Governance Gorge

Feasibility Forest

Hypothesis HollowPolicy Preserve

Solution Summit

Technological Timberline

Workflow Woodlands

Scientific Approach

Pros to this approach:Start with the clinical problem you are trying to solve

Well-defined problem and scope

Apply available technology to try to solve it

Build out an evidence-based solution over time

Cons to this approachLess likely to be aware of available features

Less likely to be familiar with technological limitations or challenges

More likely to use “tried and true” / traditional methods rather than innovative methods

Not knowing what is possible means less likely to consider how new approaches could be useful

Technological Approach

Pros to this approach:“Build it and they will come”

Develop for the market, validate afterwards

Focus on first-to-market

Make a platform that can be used to address many problems

Cons to this approachSolutions in search of a problem

Not always evidence-based

Doesn’t know what the clinical problems are

Scientific Approach Recommendations

Have an understanding of the CRI tools and technologies available to your study

Understand that technologies often progresses faster than research

Understand the complexity of integrating different systems/technologies – especially with the EHR

Meet with systems/technological domain experts early in the research design part of your study

Seek out funding that supports and targets innovative approaches to research questions

Technological Approach Recommendations

Look for the device most likely to have patient impact

Focus attention on developing technology for those devices first

Health insurance may pay for development / implementation if improves patient care

Understand organizational change needed to use your technology

Collaborate with clinical/healthcare domain experts

Governance Challenges: Applicable to Both Approaches

Governance is biggest hurtle to implementing healthcare solutions and effecting change, setting new standards for care, and seeing improved outcomes

Identify institutional stakeholders and strategic priorities

Identify institutional barriers to implementation

What must be done before your solution can be implemented?

Are there any technology limitations/barriers at your institution?

Internship Deliverable: Guide to Data Collection & Management for Health Outcomes Research

What’s next?

Write technical and end user documentation for PRO Core platformWrite

Build new PRO Core APIs to interface with consumer devices and other data collection systemsBuild

Find new ways to integrate outcomes research findings into clinical workflowsFind

Develop predictive algorithms (e.g. chatbot) based on outcomes research data Develop

With gratitude for all the support!

• Antonia Bennett, Faculty Director, PRO Core

• Mattias Jonsson, Director of Systems, PRO Core

References

• Basch E, Abernethy AP, Mullins CD, et al. Recommendations for incorporating patient-reported outcomes into clinical comparative effectiveness research in adult oncology. J Clin Oncol 2012;30:4249–4255. PMID: 23071244. doi:10.1200/JCO.2012.42.5967

• Butte AJ. Translational bioinformatics: coming of age. J Am Med Inform Assoc. 2008;15(6):709–714. doi:10.1197/jamia.M2824

• Embi PJ, Payne PR. Clinical research informatics: challenges, opportunities and definition for an emerging domain. J Am Med Inform Assoc. 2009;16(3):316–327. doi:10.1197/jamia.M3005

• Embi PJ, Payne PR, Kaufman SE, Logan JR, Barr CE. Identifying challenges and opportunities in clinical research informatics: analysis of a facilitated discussion at the 2006 AMIA Annual Symposium. AMIA Annu Symp Proc. 2007;2007:221–225. Published 2007.

• Johnson SB. Clinical Research Informatics: Supporting the Research Study Lifecycle. Yearb Med Inform. 2017;26(1):193–200. doi:10.15265/IY-2017-022

• Lucila Ohno-Machado, Clinical research informatics: a growing subspecialization of biomedical informatics, Journal of the American Medical Informatics Association, Volume 25, Issue 3, March 2018, Page 223, https://doi.org/10.1093/jamia/ocy008

• Matkar S, Gangawane A. An outline of data management in clinical research. Int J Clin Trials 2017;4(1):1-6.

• Richesson RL, Horvath MM, Rusincovitch SA. Clinical research informatics and electronic health record data. Yearb Med Inform. 2014;9(1):215–223. Published 2014 Aug 15. doi:10.15265/IY-2014-0009

• Saczynski JS, McManus DD, Goldberg RJ. Commonly used data-collection approaches in clinical research. Am J Med. 2013;126(11):946–950. doi:10.1016/j.amjmed.2013.04.016

• Segal, Courtney; Holve, Erin; and Sabharwal, Raj, "Collecting and Using Patient-Reported Outcomes (PRO) for Comparative Effectiveness Research (CER) and Patient-Centered Outcomes Research (PCOR): Challenges and Opportunities" (2013). Issue Briefs and Reports. Paper 10. http://repository.academyhealth.org/edm_briefs/10

• Zerhouni EA. Translational and clinical science—Time for a new vision N Engl J Med 2005;353(15):1621-1623Oct 13.

References (2)

• Huser V, Sastry C, Breymaier M, Idriss A, Cimino JJ. Standardizing data exchange for clinical research protocols and case report forms: An assessment of the suitability of the Clinical Data Interchange Standards Consortium (CDISC) Operational Data Model (ODM). J Biomed Inform. 2015;57:88–99. doi:10.1016/j.jbi.2015.06.023

• Kahn MG, Weng C. Clinical research informatics: a conceptual perspective. J Am Med Inform Assoc. 2012;19(e1):e36–e42. doi:10.1136/amiajnl-2012-000968

• Richesson RL, Nadkarni P. Data standards for clinical research data collection forms: current status and challenges. J Am Med Inform Assoc. 2011;18(3):341–346. doi:10.1136/amiajnl-2011-000107

• Whitaker TJ, Mayo CS, Ma DJ, et al. Data collection of patient outcomes: one institution's experience. J Radiat Res. 2018;59(suppl_1):i19–i24. doi:10.1093/jrr/rry013