Some Thoughts

Progress takes time:

– It has been more than 10 years since the ISPOR Board of

Directors commissioned the RWDTF in the Spring of 2004

– I began work on benefit-risk modeling in 2006 and

performance-based risk-sharing arrangement in 2007.

We have made progress, but it’s difficult to know

when we might reach a tipping point or acceleration,

but there are increasing pressures to demonstrate

value post-launch.

Use of CEA for guidelines development (ACC/AHA) is

very encouraging.

Incentives drive decisions more than information.

Registries

Omar Dabbous, MD, MPH

Agenda

Introduction

Definitions

Strategic rationale

Design considerations

Operational (implementation) considerations

Examples

Other issues and future considerations

Conclusions

Introduction Section 1:

Fixed treatment options

Few patients and short follow up

Limited types of patients

Delay in publication (end of trial)

Limitations of clinical trials

Efficacy: “is the extent to which medical

interventions achieve health improvements

under ideal circumstances”

Does it work?

Effectiveness: “is the extent to which medical

interventions achieve health improvements in

real practice setting”

Does it work in the real world?

Efficacy vs. Effectiveness “Lab vs Field”

Gold et al, Cost Effectiveness in Health and Medicine, New York NY,

Oxford University Press. 1996 392 - 411

Definition and Perspective Section 2:

Registry Definition

A patient registry is an organized system that

uses observational study methods to collect

uniform data (clinical and other) to evaluate

specified outcomes for a population defined by a

particular disease, condition, or exposure, and

that serves a predetermined scientific, clinical, or

policy purpose(s).

Gliklich RE, Dreyer NA: Registries for Evaluating Patient Registries: A User’s Guide:

AHRQ publication No. 07-EHC001. Rockville, MD. April 2007

Defining Characteristics

• Observational, naturalistic, “real world”

• Not driven by protocol (per se) nor randomization

• Not driven by hypothesis (usually) but not purely exploratory

• Large, multi-center, long-term (typically) • To benefit from geographic and patient variability, and

sheer numbers

• Scientific Advisory Panel (recommended) • Steering committee

• Publication committee

• Active practitioner participants (generally)

Defining Characteristics (cont’d)

• IRB/EC approval, informed consent, privacy protected (always)

• On-going analysis, reporting, publication – But, (typically) finite timeframe

• A prospective study that requires a customized operational approach – Data collection

– “Monitoring”

– Site management

– Project management

– “Program” management

Universe of Registries: Who Needs Real World Data

Government

Registry

Physicians Patient

Public

policy Industry

Employer Payer/

insurers

Accreditors

http://effectivehealthcare.ahrq.gov/decide/registryOutline.cfm#chapter2

My Perspective: Patient registries Can Achieve Strategic and Scientific goals…

Commercial

Medical R&D

–…but must be designed collaboratively and

“operationalized” appropriately

Strategic Rationale Section 3:

Motivations

• Commercial – Accelerate acceptance

– Optimize utilization

– Recertification

• Safety Surveillance – AE/SAE driven

– Pregnancy exposure

– Risk management plan

Epidemiological

• Natural history of disease

Outcomes Research

• Economic (Value Documentation, Modeling)

• Humanistic (Outcomes, Validation)

• Pricing justification

• Model validation and populating

Registries are Needed Because…

• …RCTs can be too artificial in intent and design, and therefore poorly reflective of actual medical practice – Tight inclusion criteria

– Experimental protocol

– Tight procedural control

– Randomization, blinding, placebo, etc.

– Short in duration

– Homogeneous sites

Registries are Needed Because… • Real world data…we

need to know how a product is used and how it “performs” under real world conditions – Safety : mandated by

regulator and now by payers

– Value • Clinical outcomes (CER)

• Economic value

• Humanistic value

What Do we Get from Registries: Real World Data

Real-world clinical practice patterns

Natural history of the disease

Provides data on key accounts

Long term clinical safety & effectiveness

Comparative effectiveness & product differentiation

Cost effectiveness

Quality measurement/improvement

Hypotheses for future clinical trials

Benefits of Registries: Real World Data

• Benefits of RW data: Effectiveness vs. efficacy

Multiple interventions

Long-term benefits and harms

Diverse population

Broader range of outcomes

Resource use

Dosing, compliance, adherence

When RCT not possible

Confirmatory of RCTs

Urgent, life-threatening situation

Interim evidence in absence of RCT

Strategic Considerations

Registries can be expensive

Use within a strategically important indication and in a competitive market

Conduct when payers request validation of trials data and findings

Conduct when mandated by a regulatory authority

When disease/product data is lacking or missing

In support of patients programs and managed entry agreements

Registries create a multifaceted communication platform

Deliverables Analysis/Publication

Case Studies

CME Monographs &

Case Studies

Manuscripts Registry Reports

Abstracts Presentations

Aggregate Data

Subset Data and Patient Types

Site Specific Data

Website

Registry Summit

Meetings

Consumer Awareness

Design Considerations Section 4:

Consider….

• Product versus disease registry?

• Strategic, scientific goal – Consider “working

backwards” from what you want to achieve

• Tangible output and “theme”

• Analytical structure

• Design

• Data requirements and processes

• Everything else

Strategic Planning: Building the Registry from the Ground Up

What are the research

goals?

What are the strategic

goals underlying the

study?

– Direct impact on how the

project/study should be

‘operationalized’

• Direct impact on budget

and ROI

How will you know if you’ve

achieved success?

– Performance metrics

(impacted by strategic

goals)

REPORTS

LEGAL, REGULATORY, IRB REVIEW

MATERIALS PRODUCTION AND DISTRIBUTION SITE RECRUITMENT AND TRAINING

PATIENT ENROLLMENT,

OUTCOMES TRACKING,

DATA COLLECTION

SITE

SUPPORT

ANALYSES

NEWSLETTERS

PUBLICATIONS ABSTRACTS, PRESENTATIONS

MEETINGS

STRATEGY

ANALYSIS PLAN COMMUNICATIONS PLAN

DATA COLLECTION FORMS,

PROCESSES, AND LOGISTICSSCIENTIFIC ADVISORY PANEL

SITE IDENTIFICATION (FIELD

INVOLVEMENT)

Implications for Design

What do you want to do?

What question do you want to

answer?

What issues do you want to explore?

Will the findings be useful and

informative?

What are your strategic (business)

goals?

– A vs. B? A vs. D?

– A vs. all? A vs. other Rx products only?

– All?

e

A

B

F

d

b C

D

Implications for Design (cont’d)

Can we “control for” ….?

– Patient variation and comorbidities

– Physician experience, preference and

constraints

– Product availability

– Patient compliance

– Etc.

Not perfectly…

– Goal: minimize bias

– Control for confounders

– Employ advanced statistical techniques

e

A

B

F

d

b C

D

Implications for Design (cont’d)

• Relative to RCTs, a patient registry is

imperfect BUT still essential;

therefore…

– Set expectations

– Set expectations realistically

e

A

B

F

d

b C

D

Operational (implementation) considerations

Section 5:

Different Conditions Require Different Operational Processes

Operational Issues and Challenges

• Site selection

• Site training and start-up

• Regulatory considerations

– Approval/notification

– Legal review

• Contracts versus agreements

– IRB/EC approval

• Privacy

– Informed consent

Site “interaction” (monitoring) and management

• Site motivation

• Protocol “adherence”

- Inclusion

- Procedures

Data Management

• Accommodating multiple measures

• EDC issues

• Data quality

- SDV

Operational Issues and Challenges (cont’d)

Analysis

• Biases, etc.

• Representative of treated patient population

• Association versus causality

• Findings

• Ensure use of robust statistical methodology

• Accurate interpretation of the data

• Reporting (communications)

Operational Issues and Challenges (cont’d)

Other issues and future considerations

Section 6:

Other Issues…..

• Distinct SOPs?

• Listing as clinical trials? • Mandated to register observational research

• Organizational “ownership”? • HEOR

• Epidemiology

• Medical Affairs

• Marketing / Product Management

• Clinical Operation

• R&D

• Safety/PV

• Other

Future Considerations

More “provisional” approvals, requiring the

submission of registry data

– US version of NICE?

– FDA, CMS, PCORI, etc.

Payers (in the US) will fuel the push for CER,

causing progressive pharma companies to be

proactive and collaborative

CER will embrace a variety of research methods

– ‘Hybrid’ prospective observational research

– Retrospective database research

– Meta-analyses and systematic reviews

– RCTs

Future Considerations (cont’d)

Digital Health:

– EMRs will be critical and will change the nature of patient

registries

Combination:

– Trials + registry

– EMR + Claims data

– EMR + claims data + survey

– EMR + claims data + survey + Patient reported

outcomes

Benefits of registries

Global and Local Registries

Global Program

• Benefits – “Engine” with

customizable “body” • Opportunity for staged

participation by smaller affiliates

• Challenges – Value of aggregate

analysis may be questionable

– Local adaptation may be too extreme

– Local regulations may be too constraining

Local Program

Benefits • Customized specifically for local conditions

Challenges

• Expensive to develop and implement

• Limited benefit beyond local market

• Limited population may constrain scientific value

Strategic objectives: Industry

Transform customers into scientific partners

Can serve as a platform for common

research project

Provides data on key accounts

Post-marketing safety surveillance

Used to develop clinical guidelines, and

evidence based medicine publications

Strategic objectives: Sponsors

Post-marketing data on clinical practices – how is a product being used?

– validation of trial results in real-world practice

– evaluating whether MDs are adhering to guidelines

– comparison with competitor’s products

– who is prescribing the procedures, drugs “Dr’s specialty”

– which groups of patients are using your product?

Marketing & building brand identity & loyalty – plan competitive future marketing strategy

– make effective business decisions

– tools for marketing team “new relationships”

Strategic objectives: Sponsors (cont’d)

Medical practice patterns

Understand geographic practice patterns

Aid healthcare technology development

Guide & help the design of new clinical trials

Supportive data in sponsored symposia

Deliverables: Direct Benefits to Sponsors

Product-specific utilization report (confidential)

Aggregate report (quarterly or twice yearly)

Special reports (on request)

Scientific support (on request)

Strategic objectives: Investigators

Educational feedback

Quarterly reports

News letters

Regular scientific meetings

Quality assurance

Physician self-study audit tool

Publications and exposure

Strategic objectives: Patients

Identify opportunities to improve the quality of

care/life for patients

Help identify new patient populations

Tools for patients engagement and education

Tools for patients triage

Conclusions

• Best practices

• Questions

Best Practices

Be realistic in registry planning

Set appropriate expectations (internally and

externally)

– Registries are part of an overall “portfolio”

Strive for organizational inclusiveness and

consensus

Develop guidelines addressing study design and

SOPs addressing unique operational requirements

Best Practices (cont’d)

Interact with KOLs and all stakeholders during

planning stages (and concurrently)

Maintain a collaborative stance with research

partners

– Minimize operational constraints

Expect change: “shift” happens

Everything is Connected

REPORTS

LEGAL, REGULATORY, IRB REVIEW

MATERIALS PRODUCTION AND DISTRIBUTION SITE RECRUITMENT AND TRAINING

PATIENT ENROLLMENT,

OUTCOMES TRACKING,

DATA COLLECTION

SITE

SUPPORT

ANALYSES

NEWSLETTERS

PUBLICATIONS ABSTRACTS, PRESENTATIONS

MEETINGS

STRATEGY

ANALYSIS PLAN COMMUNICATIONS PLAN

DATA COLLECTION FORMS,

PROCESSES, AND LOGISTICSSCIENTIFIC ADVISORY PANEL

SITE IDENTIFICATION (FIELD

INVOLVEMENT)

Registries Experience: Examples NDB – (RA)

CORRONA (RA and PsA)

TREAT (Crohn’s Disease)

Pregnancy registry (Sweden, Denmark) – (IBD, RA, PsA, AS, & PsO )

Psoriasis registry (f/u for 8 years)

Pediatric IBD registry (f/u for 20 years)

GRACE registry

Hip and Knee replacement surgery

Gioblastoma registry

ALS

Veinus Thromboembolism

Capsure Prostate cancer

Irritable Bowel Syndrome

Registry of registries (lymphoma)

*Regulatory requirement upon approval

Guidelines for Quality – Observational Studies

GRACE principles for observational studies of comparative

effectiveness. Am J Man Care 2010;16(6):21-24

– ENcEPP Checklist for methodologic studies, 2010. www.encepp.eu

– ISPOR Good Research Practices for CER I, II, III . Value in Health

2009; 1044-1072

– GPP: Guidelines for good pharmacoepidemiology practices

Pharmacoepidemiology & Drug Safety 2008:17:200-208

– STROBE: Strengthening the Reporting of Observational Studies in

Epidemiology, Epidemiology 2007;18(6): 805-835

– AHRQ REGISTRIES HANDBOOK: Gliklich RE, Dreyer NA, eds. :

Registries for Evaluating Patient Outcomes: A User's Guide. Prepared

by Outcome DEcIDE Center. AHRQ Publ. No. 07-EHC001-1. Rockville,

MD. 2007. 2nd edition, in press, 2010.

50

Limitations of RWD

• Most significant concern is bias

• Despite sophisticated statistical adjustment

techniques, real world data don’t meet the rigor of

an RCT

• Can be costly to conduct

Major Challenge:

Striking the right balance regarding the value of RCTs versus the value

of RW data

Competing Scientific Perspectives on the Value of RCTs vs. RWE

– There is a subset for health outcomes researchers

who hold strongly to the view that RCTs are far

superior to real-world data collection.

– There is a subset who think that RCTs are not

particularly helpful to understand what economic

impacts and PROs are going to show in real world

settings for coverage and reimbursement.

• This tension is reflected in real-world institutions:

– IQWiG and DERP focus on RCT reviews

– NICE uses integrated modeling (Bayesian) approach

– AMCP guidelines allow modeling approach

Real-World Evidence: An Economic Perspective on Key Facts

Information has many public good aspects:

– Implies tendency for suppliers (pharma and payers) to free

ride

– Implies tendency for consumers (pharma and payers) to free

ride

– A free (unregulated) market will tend to undersupply.

Evidence generation is costly.

The patent system for innovative medicines provides

a powerful incentive for minimizing time to market.

Policy tools include regulation (public or private),

intellectual property, and subsidies (e.g., NIH or

PCORI).

Robust Evidence

• Innovative clinical and outcomes study design • Clinically and payer relevant

• Disease/conditions, treatments, comparators, target population

• Measures and magnitude of efficacy, effectiveness, safety and

tolerability

• Quality of life and patient satisfaction

• Resource utilization and cost

• Apply sophisticated analytic methods to proactively

address anticipated payers’ questions: • Data collection, including handling of missing data

• Appropriate methods to address confounding

• Comparison to patients with similar likelihood of treatment and

benefit

• Consideration of alternative explanations

• Accurate data interpretation and validation

Questions?