2/20/2012

1

Pragmatic Clinical TrialsPart IIPart II

Christopher B. Granger, MD

John P. Vavalle, MD

Duke University Medical Center

Pragmatic Clinical Trials• Part I

– The importance of large clinical trials

– The need for evidence from randomized studies (RCT)

– Limitations with current large clinical trials

• Part II– Possible solutions to current limitations and the role for pragmatic 

clinical trials (PCT)

– Opportunities for improving efficiencies in clinical trials

– Doing more with less, the realities of the current funding structure

– Examples of successful pragmatic trials

General Research Constructs

• Validity

– Internally consistent

– Would you get the same answer if the study was repeated?was repeated?

• Generalizability

– Can the findings be extended to settings beyond the specific experiment

• Often there is a trade off between these 2

2/20/2012

2

Pragmatic Clinical Trial Objectives(1) Increase the number of effective therapies available 

to patients (number of trials)

(2) Reduce the time to peak therapeutic value (timely trials)

(3) Increase the chance of incorporation into guidelines and regulatory approval (trial rigor)

(4) Increase the reliability and generalizability of clinical trial results (pragmatic)

(5) Reduce patient morbidity and mortality, and improve quality of life

CER Definition (HHS 2009)

Research comparing different interventions and strategies to prevent, diagnose, treat andmonitor health conditions in order to inform patients providers and decision makerspatients, providers and decision‐makers, responding to their expressed needs, about which interventions are most effective for which patients under which circumstances.

Clinical Research In the US: Some thoughts on what’s needed to reverse the decline

• Role for Academic Medical Centers and Integrated Health Care Systems

– Alignment of clinical care with research activities

– Acknowledgement of the professionalism required to perform human subjects research

– Re‐thinking of the roles of IRBs, contracts, issues around relationships with industry

– Clearly defined academic homes, training, mentoring and career development for clinical investigators (clinical and quantitative scientists)

– Investment in information technology

– Research opportunities in health care reform

2/20/2012

3

• Societal Needs

– Investment in national research infrastructure (CTSAs, NIH networks); commitment to collaboration and team science

– Commitment to data standards, informatics and EHRs (HL7 CDISC professional societies)

Clinical Research In the US: Some thoughts on what’s needed to reverse the decline

(HL7, CDISC, professional societies)

– National disease registries and databases (NIH, professional societies); integration with RCTs

– National debate/discussions on relationships with industry; “rules” for public‐private partnerships and co‐investments

– Improve national clinical research literacy

July 24, 2011Rule Changes Proposed for Research on HumansBy Andrew Pollack

The government is proposing sweeping changes in the rules covering research involving human subjects, an effort officials say would strengthen protections while reducing red tape that can impede studies.

The officials said the changes were needed to deal with a vastly altered research climate whose new features include genomics studies using

The New York Times

research climate, whose new features include genomics studies using patients’ DNA samples, the use of the Internet and a growing reliance on studies that take place at many sites at once.

…a proposed change would allow a single institutional review board to oversee studies that take place at multiple sites.

Right now, the institutional review board at each location generally must endorse a trial, which can lead to long delays. Federal officials said that besides eliminating redundancy and delays, having a single reviewer that is truly accountable for its decisions might actually strengthen oversight.

Improving the Research Process:

• Short‐term

– practice‐friendly protocols (collaborate) 

– improve/invest in training (meaningful training beyond GCP)

– improve/simplify contracts, regulatory

– limit and focus data collection

– re‐evaluate concept of monitoring (on site vs central; 100% vs focused)

– adequate reimbursement for work at sites

2/20/2012

4

• Longer‐term

– invest in national research infrastructure (NIH; AMCs; investigator networks)

– research methods (regs; IRBs; AE reporting)

– form relationships not just project‐specific work

Improving the Research Process:

– form relationships not just project‐specific work (master contracts; investment; training)

– commit to uniform data standards (CDISC; HL7; professional societies)

– invest in communication tools for investigators and coordinators

– modulate/reduce role of commercial CROs

• Longer‐term

– Build a global network to bring academic/clinical  leadership to clinical research

– Use social media tools to link investigators /coordinators for education and research

Improving the Research Process:

/

– Transform research using electronic health records 

• Identifying patients

• Collecting baseline descriptors

• Capturing clinical outcomes and follow‐up status

– Engage with FDA’s Clinical Trials Transformation Initiative (CTTI)

Part II

Vavalle

2/20/2012

5

Identifying Specific Opportunities to Improve Clinical Trials

• Hypothesis:

There are tangible obstacle that can be– There are tangible obstacle that can be improved to reduce regulatory complexity and delays in initiating large clinical trials

Duley L Clin Trials 2008; 5: 40-48

Duley L Clin Trials 2008; 5: 40-48

2/20/2012

6

Duley L Clin Trials 2008; 5: 40-48

Areas for Improvement:(1) Increase ability of sites to become top 

performers

(2) Use electronic resources to improve site management and monitoringmanagement and monitoring

(3) Streamline and enhance clinical trial operations

(1) Increasing Site Capabilities: Adopt a site‐focus to the trial

Design trials, case report forms, and electronic data capture systems around clinical practice routines

Limit case report form length

Grant appropriate compensation to sites

Competition for Patients

Allow patients to have multiple enrollments

Make trials more attractive to sites 

2/20/2012

7

(1) Increasing Site Capabilities• Increase performance

– Select Sites that best meet protocol requirements

• ‐ Based upon performance in similar trials

• ‐ Run eligibility lists from clinical databases

– Site Development

E l i i• ‐ Evaluate sites over time

• ‐ Set expectations using education programs 

• ‐Monitor performance with feedback reports

• ‐Mentoring in clinical research networks

(2) Computer Systems for Site Monitoring

• Centralize source document verification

• Use statistical programs to monitor data lianomalies

• Remote monitoring via conference calls/webinars.

Range of options for on‐site monitoring

Arrangements for site visiting may vary:

• Routine visits to all sites

• Visits to random selection of sites

• Targeted visits to less experienced sites, or those for which central monitoring suggests possible problems

MRC/DH joint project (www.cl-toolkit.ac.uk)

2/20/2012

8

Prevention of misconduct by better trial design (rather than by more policing)

• Relax eligibility criteria: Excessively restrictive entry criteria may lead to entry data being altered

• Assess compliance crudely: Detailed pill counts may be unnecessary (& random sampling better)

• Limit data collected: Important adverse events may be under‐reported if data collection is excessive

• Accept missing values: Undue pressure for complete data may lead to values being invented

More cost‐effective design allows much larger numbers to be randomised, yielding smaller random errors

(3) Streamline and Enhance Operations

• Develop one level of evidence standard for trials with similar purposes (government and commercial) 

• Adopt current levels of evidence in government sponsored trials

• Evaluate cost‐effectiveness of current practices with further research

Efficiencies

There is still room to consider more efficiencies:

– Specifically, the types of patients enrolled

– Enroll patient with characteristics that give them higher risk of events.  

C b i b l i hi i l– Can be genomic, metabolomic, historical. 

Some populations (Examples):

• People who have an event despite good lipids, BP, secondary prevention therapy.

• People with prognostic indicators: CRP, ACS with elevated troponin

• Cancer patients with genetic predictors of recurrence

2/20/2012

9

Part III

Granger

Large‐scale RCTs

• The term “practical” or “pragmatic” clinical trial attempts to capture more than the size of trials

• The size of the trial should be LARGE ENOUGH to answer the question posed in terms of health outcomes (live longer, feel better, spend less ( g , , pmoney)

– But a PCT has other characteristics that are frequently missing in trial design

– Relevant comparisons, relevant populations, relevant time period, appropriate background therapy

First Alternative: Do trials right!• Questions should be framed by those who use the information rather than by companies aiming to advantage their products through clever design

• Infrastructure to do trials should be supported• Infrastructure to do trials should be supported by the enterprise rather than having each study as a “one off” experience

• Trials should be embedded in a nodal network of health systems with Electronic Health Records and specialty registries cutting across health systems

2/20/2012

10

Randomization is a critical scientific and policy tool when:

• The likely effects of the intervention are modest (RR or HR of < 3 or so)

• Because

– There is ALWAYS bias in who gets which treatment

• However …

Many argue that non‐randomized analyses are needed, because

• Every question cannot be answered by a PCT and decisions need to be made

• An increasing number of practices and health systems have large data repositoriessystems have large data repositories

• Methods are available to attempt to adjust for measured confounders

• But beware, large datasets may simply result in a more precise estimate of a biased and misleading finding, and there is no method to adjust for unmeasured confounders !

Data Repositories

• Decisions are being made every day by

– Administrators 

– Clinicians

• The question will be:

– Is it better to combine evidence from PCT’s with opinion

– Or is it better to use a layered approach with PCT’s for critical big questions, non‐randomized analyses to fill in the gaps, and opinion where neither is available?

2/20/2012

11

Trials Registries/ Observational Studies

Str

engt

hs How are treatments applied?

Population outcomes

Defining treatment effect

Defining safety

Long term effects

Lack generalizability

Too expensive

Safety in high-risk populations

Wea

knes

ses

Unreliable inferences about treatment effects

Lower data quality

Second Alternative—Cluster RCT’s

• Randomize practices or systems to one practice or another

• Issues:

– Adjusting for non‐independence of observations within practice

– Preventing “contamination”

– Consent—really the big issue

Randomization

All SitesN=399

Intervention #2:IMA Use in

Elderly N=120

Intervention #1:Preop ß-blockade

N=124

Control:No Intervention

N=115

Regional QI:Both Interventions

N=40

2/20/2012

12

Third Choice—Observational Treatment Comparisons

• Not for amateurs

• Must consider (always)

– Confounding by indication

Inception time bias– Inception time bias

• Must also consider 

– Missing data at baseline to adjust for differences

– Missing data during follow‐up

– Characterization of outcomes

Observational Treatment Comparisons Must Include

• Adjustment for known prognostic factors

• Adjustment for propensity 

– Consider inverse weighted probabilityConsider inverse weighted probability estimators for chronic treatments

• Time adjusted covariates when inception time is variable

– To get really fancy use inverse weighted probability estimates

Typical NIH NetworkAcademic Health Center Sites& Data Coordinating Center

2/20/2012

13

Integration of Clinical Research Networks

• Link existing networks so clinical studies and trials can be conducted morecan be conducted more effectively

• Ensure that patients, physicians, and scientists form true  “Communities of Research”

Sensible Approaches for Reducing for Reducing

Clinical Trial Costs

Sensibly Reducing Clinical Trial Costs:  Objectives

• To assess the value of practices commonly employed in the conduct of large‐scale clinical trialsclinical trials.

• To identify areas where costs could be reduced without compromising scientific validity.

2/20/2012

14

Sensibly Reducing Clinical Trial Costs: Methods

• Qualitative Phase:–Modifications of large‐scale trial designs and operations to maximize their value (cost vs scientific benefit tradeoff)(cost vs. scientific benefit tradeoff)

• Quantitative Phase:– Large trial economic model used to assess financial implications of panel’s recommendations.

Full Cost Model Parameters

Trial Type Chronic Disease

Number of Patients 20,000

Number of Sites 1000

Months Duration 48Months Duration 48

CRF Pages 60

Site Monitor Visits 24

Site Payment $10,000

Eisenstein EL. Clin Trials. 2008;5:75-84.

Full Cost Model Results

Category Costs Percent

Direct Labor $167 40%

Non-labor $255 60%

Site payments $202 48%

Other (air hotel etc ) $53 12%Other (air, hotel, etc.) $53 12%

Total $421

$US in 2007 millions

Eisenstein EL. Clin Trials. 2008;5:75-84.

2/20/2012

15

Site Management Assumptions

Full Cost Industry

StreamlinedIndustry

Evaluation visits 50% 10%

Site visits per site 24 4

Close-out visits 100% 0%

Source document verification

100% 10%

More Streamlined Trial Assumptions

• Assumed previous work with all sites

– Limit to 100 sites

– Eliminate on‐site evaluation, close‐out visits, and source document verification.

• Focused case report form (5 pages)

– Enrollment / baseline data (1 page)

– Follow‐up (4 pages, 3 questions)

• Site payment 

– $650 ($250 baseline, $100 follow‐up)

Eisenstein EL. Clin Trials. 2008;5:75-84.

Clinical Trial Cost Estimates

$250

$300

$350

$400

$450

To tal

Coo rd inatin g Center

S ite Paym ents

Oth er

$0

$50

$100

$150

$200

Full Cost Industry

Streamlined Industry

More Streamlined

$ In US 2007 Millions

Eisenstein EL. Clin Trials. 2008;5:75-84.

2/20/2012

16

Inputs: The National LandscapeGrowth from 1994 to 2003: 7.8%Growth from 2003 to 2008: 3.4%

Percent of total health costs: 4.5%Percent on health services

research: 0.1%No increase in drugs or devices

Major sources:

Industry: 58%Federal: 33%

Dorsey ER et al. JAMA 2010;303:137-43

2011 Buying Power = 2000 Dollars

$40,000

$35,000

$30,000

NIH Budget in Current and Constant Dollars

)

1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011CR

2012Pres

Budget

Current Appropriation

Constant 2010

With Supplemental Appropriation (ARRA)

$25,000

$20,000

$15,000

$10,000

$5,000

Dol

lars

(M

illio

ns)

Research Project Grants Applications, Awards, and Success Rates

50,000

45,000

40,000

35,000

30,000

100%

90%

80%

70%

60%

Aw

ards S

ucce

Applications Awards Success Rate (%)

25,000

20,000

15,000

10,000

5,000

0

50%

40%

30%

20%

10%

0%

App

licat

ions

/ A ess R

ate (%)

1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010Fiscal Year

http://report.nih.gov/NIHDatabook/Charts/Default.aspx?showm=Y&chartId=20&catId=2

2/20/2012

17

Outputs: The National Landscape

49

1995 – 2006: ~117,000 world cardiovascular articles• 37,000 from the United States (31%)• 10,000 received NIH funding (28%)

Type of science (based on US articles) receiving NIH funding• Basic 40%• Clinical trials 20%•Multicenter RCTs 12%

Lyubarova R et al. PLoS One 2009;4:e6425

Part IV

Vavalle

Conventional Evidence Development

Clinical Trials

Evidence DevelopmentIdentification of theScientific Question

Industry need

Investigators 

Scientific leadership; NIH

‘Expert’ Care Guidelines

2/20/2012

18

NHLBI Workshop on Cardiovascular CER How to embed maximal scientific efficiency within an established 

enterprise

SystematicReview

Decision Analysis;Markov modeling

Cochrane and national Clinical TrialsObservational Data;

CER Evidence and Data Portfolio ManagementIdentification of theScientific Question

National Research Priorities

Inclusive clinician + methodology leadership 

Cochrane and national Evidence Based Practice 

Centers

Clinical TrialsRegistries

Collaborative Care Guidelines

Periodic Review(3‐5 years)

ImprovedClinical Care, Cost Effectiveness and Patient  Outcomes

Patient and community questions

National Recognition of the Need for New Research Paradigms: NIH

• 7/08 NHLBI Workshop:

Outcomes Research in CV Imaging

• Is an outcomes paradigm feasible for p gimaging research?

• Highest priority study was Imaging CER

– Evaluation of chest pain/angina 

– Functional vs anatomic testing strategy

Douglas JACC Img 2009

Optimal Use of Imaging to Diagnose CAD:What Can CER Teach Us?

1. Is there a ‘best’ initial noninvasive test for a 

patient with new CAD symptoms?

2 Is ischemia (stress testing) or anatomy (CTA)2. Is ischemia (stress testing) or anatomy (CTA) 

more valuable in evaluating stable chest pain?

3. Should new imaging tests be required to 

demonstrate improved effectiveness through 

better clinical outcomes?

2/20/2012

19

PROMISECER Pragmatic Trial

The PROMISE Trial:PROspective Multicenter Imaging Study

for Evaluation of Chest Pain

• Perform a randomized controlled pragmatic trial of diagnostic strategies in stable CAD symptoms

• Use CER trials methods to assess clinical outcomes and cost endpoints

Randomization

Anatomic strategy Functional strategy

Stable symptoms suspicious for significant CAD, Requiring non‐emergent noninvasive testing

PROMISE Design: 10,000 pts; 200 sites

1º = 36 mo death, MI, complications, UA hosp

2º = MACE components, Costs, QOL; Safety: Rad exp

64+ slice CTAExercise ECG or Exercise Imaging

PharmacologicStress imaging

Clinical results immediately available to care team;Subsequent testing/mgmt per care team per best practices

CER Clinical Trials Methodology to Test Imaging Strategies for Dx of CAD

• Effectiveness, not efficacy (drug or device performance) or utility (test performance or accuracy) 

• PROMISE is a pragmatic trial– Large, simple, efficient 

– Generalizable patient population

– Real world standard of care

• Implementation is complex: ‘Best practices; Usual care’– Site read images but rigorous qualification and ongoing QA

• Pragmatic imaging trial methodology is working – Sites reflect broad spectrum of specialties and care patterns

– Enrollment is well ahead of predicted

2/20/2012

20

Thrombus Aspiration in ST-Elevation myocardial infarction

in Scandinavia (TASTE trial)

Why SCAAR? • The most complete PCI registry in the world!

– 100% complete, nationwide

– Validated

– Complete follow‐up

– Unique opportunity for prospective observational studiesq pp y p p

– Merge with other national registries‐Mortality, CV diagnoses, hospitalizations, PCI

• Possibility to establish evidence in an area other trials will not or cannot 

• Possibility to complete the world’s largest randomized trials during the shortest period of time

Patients with suspected STEMI referred to primary PCIN = 5000

STEMI diagnosis confirmed at coronary angiography. Informed consent obtained

Online 1:1 randomization in SCAAR

TASTE Trial Flow Chart

Thrombus aspiration and PCI PCI alone

Immediately after PCI: TIMI flow grade

30 days: all-cause death

1, 2, 5 and 10 years: all-cause death and additional secondary endpoints

2/20/2012

21

PREPARE

VAMPIRE

TAPAS

TASTE

TASTE compared with previous thrombectomy studies

TASTE

0 1000 2000 3000 4000 5000

REMEDIA

DEAR-MI

EXPIRA

X AMINE ST

Number of patients

Worlds highest buildings

Woolworth Building

Chrysler Building

Empire State Building

World Trade Center (North tower)

Willis Tower (formerly Sears Tower)

Taipei 101

Burj Khalifa

0 100 200 300 400 500 600 700 800 900

Equitable Life Building

Auditorium Building

New York World Building

Manhattan Life Insurance Building

Park Row Building

Philadelphia City Hall

Singer Building

Met Life Tower

Height in meters

Inclusion Rate

Included

All primary PCI:sNumber

Possible candidates Included in TASTE

Date

2/20/2012

22

Inclusion Rate Per HospitalHospital

Proportion Included

Potential Engagement

29%5% Very likely

Please consider your own possible participation as a volunteer in clinical research. How likely would you be to participate in a clinical research study?

47%

19%

Very likely

Somewhat likely

Not likely

Would not participate

Courtesy of Michael Lauer

Where’s The Engagement?

Has your doctor ever suggested that you participate in a clinical research study?

6%

94%

Yes

No

Courtesy of Michael Lauer

2/20/2012

23

Conclusions1. Many opportunities exist for improving the ways in which clinical 

trials are performed.

2. Costs are increasing while funding is decreasing.  This must change and there is no sign of increased funding in the near future.

3. Comparative Effectiveness Research in the form of Pragmatic Clinical Trials is what is needed to inform healthcare decisions

4. One size doesn’t fit all.  Need trial designs based upon scientific objectives and stage of development.

5. There are examples of successful clinical trials using a pragmatic design, and others should learn from them

6. Substantial reductions in the costs of large‐scale clinical trials can be achieve without compromising the scientific validity of results.

7. We need to find better ways to reduce the barriers for sites and patients to participate in clinical trials.

References & Resources1. Dorsey, E. R., de Roulet, J., Thompson, J. P., Reminick, J. I., Thai, A., White‐Stellato, Z., . . . Moses, H., 3rd. 

(2010). Funding of US biomedical research, 2003‐2008. [Research Support, N.I.H., Extramural]. JAMA, 303(2), 137‐143. doi: 10.1001/jama.2009.1987

2. Douglas, P. S., Taylor, A., Bild, D., Bonow, R., Greenland, P., Lauer, M., . . . Udelson, J. (2009). Outcomes research in cardiovascular imaging: report of a workshop sponsored by the National Heart, Lung, and Blood Institute. [Congresses

3. Research Support, Non‐U.S. Gov't]. JACC Cardiovasc Imaging, 2(7), 897‐907. doi: 10.1016/j.jcmg.2009.01.018

4. Duley, L., Antman, K., Arena, J., Avezum, A., Blumenthal, M., Bosch, J., . . . Yusuf, S. (2008). Specific barriers to the cond ct of randomi ed trials Clin Trials 5(1) 40 48 doi 10 1177/1740774507087704to the conduct of randomized trials. Clin Trials, 5(1), 40‐48. doi: 10.1177/1740774507087704

5. Eisenstein, E. L., Collins, R., Cracknell, B. S., Podesta, O., Reid, E. D., Sandercock, P., . . . Diaz, R. (2008). Sensible approaches for reducing clinical trial costs. Clin Trials, 5(1), 75‐84. doi: 10.1177/1740774507087551

6. Lauer, Michael. Personal communication with Christopher B. Granger, & John P. Vavalle. Engagement in research.

7. Lyubarova, R., Itagaki, B. K., & Itagaki, M. W. (2009). The impact of National Institutes of Health funding on U.S. cardiovascular disease research. PLoS One, 4(7), e6425. doi: 10.1371/journal.pone.0006425

8. Pollack, A. (2011). Rule Changes Proposed for Research on Humans, The New York Times. Retrieved from http://www.nytimes.com/2011/07/25/health/research/25research.html?_r=1

9. National Insitutes of Health (2011). Research Grants Research Project Grants: Applications, awards, and success rates NIH IMPAC, Success Rate File: National Insititutes of Health.