Getting a Clinical Research Protocol Started

The CTSA Response

Daniel E Ford, MD, MPHVice Dean for Clinical Investigation

Director, Institute for Clinical and Translational ResearchJohns Hopkins School of Medicine

Current State

• Time to initiate studies longer for US than most countries

• Steps required to start study are increasing (ex, billing for services provided in clinical trial)

• Recent NCI analysis found that for cooperative group phase 3 trials time from original idea to activation of study was 808 days (481 distinct processes)

June 22-23, 2008

Enhancing Clinical Investigation by

Improved Management

A Workshop Convened by the

Clinical and Translational Science Award (CTSA) Consortium/

Regulatory Knowledge Workgroup/

Clinical Research Management Taskforce

Organizing Committee

• Teshia Johnson, MBA Yale

• Michael Joyner, MD Mayo• Robert Califf, MD Duke

• Daniel Ford, MD, MPH Johns Hopkins

• Daniel Rosenblum, MD NIH NCRR

Premeeting Survey

• Three surveys similar

to each of 24 CTSA institutions

– CTSA PI (representing researcher perspective)

– IRB Director– Research Administration (contracts) Director

PI Survey Results

• 4 Institutions post performance measures for IRB and contracting

• IRB barriers– Incomplete applications from research teams– Infrequent meetings of IRB or subcommittees– IRB committees too picky about wording

PI Survey Results

• Major barriers for contracting– Even if Master agreements in place,

they are not followed by companies

– Legal review too slow– Difficult negotiations related to

indemnification, publication rights and data transfer

PI Survey Results

• 84% contracting longer than IRB

• 5% IRB longer than contracting

• 11% equal time for both

• Where delays occur:– 36% sponsors– 48% academic centers

– 21% PIs and research teams

Research Administration Survey

• Huge variation in organizational structure

• 36% of staff have left in past three years

• 78% of contracts tracked electronically– 34% of those faculty can view status of

contracts

• 50% report not meeting with one sponsor in 2007 to discuss contracting

IRB Survey

• Average of 2025 protocols approved in 2007 (682 convened)

• Average of 18 staff– 50% turnover since 2005

• 50% of protocols tracked electronically

• 50% of institutions do not pay their IRB members

• 55% met with no sponsors/NIH in last year

Agenda for Workshop

• Stakeholder perspective on problem– Walter Koroshetz, MD NINDS– Nancy Trapp, Eli Lilly Sponsor

– Tony Perez, JD IRB– Gary Chadwick, PhD Alternative IRB– James Moran, JD Contracts– Sara Rockwell, PhD Faculty

Agenda for Workshop

• Discussion groups addressing specific questions

• Poster session where everyone could share best practices and recent quality improvement programs

Why do we need research study initiation performance measures?

• Perception of sponsors and research faculty we are inefficient

• No other accreditation organization has taken on this role– Association of Human Resource Protection Programs (AHRPP)– Office of Human Research Protections (OHRP/HHS)– Federal Drug Administration (FDA)

• Opportunity for “the doers” to be proactive and lead the way

Why do we need research study initiation performance measures?

• Measurement is the first step to understanding the process

• Same metrics across multiple academic sites allows comparisons and identification of best processes (ask for more resources if not performing as expected)

• Measurement of performance metrics tends to reduce variability

Why do we need research study initiation performance measures?

• Posting metrics for sponsors and public demonstrates responsibility and accountability

• Once we demonstrate accountability, can ask for same from sponsors and research teams

Challenges to Measuring Performance

• Selecting the right performance measures– Are study protocols included in the performance

measure representative or informative?– Are the performance measures easy to measure?– Can the performance measures be measured with

reliability?• Does the performance measure vary too much by type of

study to be informative? (pediatric vs. adult, first site versus 50th site)

• Do the performance measures provide hints about the process or what the path to improvement might be?

Challenges to Measuring Performance

• Is speed the only measure of performance? What about quality of IRB review? What about quality of a contract that is clear and covers multiple potential problems?

• Are there ways to “game” the performance measures?

First Performance Metric

• Duration in days from receipt of first “ready” study protocol/site agreement received to time both the clinical investigational site and sponsor have executed the agreement

Performance Metric Definitions

• What studies are included?– Multicenter clinical trials

• Easier to assign date when site was able to start study initiation/approval process

• Include all sponsors (NIH and commercial)

– Dates• 2007 chosen as most study protocols will have

had opportunity to be activated

Performance Metric Definitions

• When is start date?– Date agreement sent from sponsor– Date agreement is received– Research teams may not have recorded

this date

Performance Metric Definitions

• Time to executed site agreement– Total days or “business” days?– Is it date when one party signed agreement or

when all parties signed agreement?– Possible to “game” this time by signing agreements

quickly and then need multiple amendments to really have study ready to enroll participants

Performance Metric Definitions

• Days to execution of agreements – How to count protocols that never were

completed?– What about outliers?

– Report median or 90th percentile?– Report 25, 50 and 75 percentile?– Report percent of protocols that met

some performance standard

First Performance Metric

• Duration in days from receipt of first “ready” study protocol/site agreement received to time both the clinical investigational site and sponsor have executed the agreement

Days to Enrollment Ready by Percentiles

132 209 265 352 372

113164

196263

307

050

100150200250300350400

0.25 0.5 0.75 0.9 1

No. o

f Day

s

Commercial Funding - Days from protocol receipt to contract execution

Federal Funding - Days from protocol receipt to IRB approval

Days to Contract Execution by Percentiles (Start date when contract presented to Research Office)

919

28

53

143

0

20

40

60

80

100

120

140

160

25%

50%

75%

90%

100%

Number of Agreements Completed

Nu

mb

er

of

Da

ys

IRB Performance Metric

• Duration in days from PI receipt of protocol to site IRB approval- Measure protocol receipt to initial (department

or IRB) submission

- Measure protocol receipt to IRB submission complete

- Measure protocol receipt to first IRB review

IRB Performance Metric

50 Multi-center, Sponsored, Clinical Trials Submitted in 2007

0

77

106

123

157

0

20

40

60

80

100

120

140

160

180P

roto

col

rece

ive

d b

y si

te

Initi

al

IRB

/De

pa

rtm

en

tsu

bm

issi

on

IRB

su

bm

issi

on

com

ple

te

Fir

st I

RB

re

vie

w

IRB

ap

pro

val

Num

ber

of D

ays

IRB Outcome Measures

50 Multi-center, Sponsored Clinical Trials submitted in 2007

7

14

29

02

0

5

10

15

20

25

30

35

Ap

pro

ved

at

first

me

etin

g

Mo

difi

catio

ns

req

uir

ed

at

first

me

etin

g

De

ferr

ed

at

first

me

etin

g

Dis

ap

pro

ved

at

first

me

etin

g

Ave

rag

en

um

be

r o

fm

ee

ting

s p

er

stu

dy

Nu

mb

er

of

Stu

die

s

Insights from Workshop

• Valuable forum to bring together IRB and research administration professionals

• Need to consider the entire study initiation process

• Need to bring general counsels into the quality improvement process

• Agree that establishing metrics will aid in quality improvement

• Not certain if goal is to standardize study management across sites or maximize efficiency within each site with own solutions

Process For Implementing Study Initiation Metrics

Post Metrics on Performance on www.CTSAweb.org

Performance report on 2007 multi-center studies, due Sept 1st.

Report plan to CCOC

Present metrics, request comments from sites, general counsels at sites, NIH and others by Feb 1st

Review the recommendations from each task force and propose combined set of metrics by Jan 1st

CTSA Clinical Research Management Taskforce

Report data to CRM TF

3 CTSA institutions to volunteer to gather data on performance metrics

Contracts TF site – collect data on multi-center studies for itself and 4 other sites

Propose 3 core and 2 optional metrics by Nov 15th

Contracts Task Force

Report data to CRM TF

3 CTSA institutions to volunteer to gather data on performance metrics

IRB TF site – collect data on multi-center studies for itself and 4 other sites

Propose 3 core and 2 optional metrics by Nov 15th

IRB Task Force

Sept/10Sept/093/092/091/0912/0811/08Item/Month