RESEARCH Open Access

Comparison of randomized controlled trialsdiscontinued or revised for poorrecruitment and completed trials with thesame research question: a matchedqualitative studyMatthias Briel1,2* , Benjamin Speich1, Erik von Elm3 and Viktoria Gloy1

Abstract

Background: More than a quarter of randomized controlled trials (RCTs) are prematurely discontinued, mostly dueto poor recruitment of patients. In this study, we systematically compared RCTs discontinued or revised for poorrecruitment and completed RCTs with the same underlying research question to better understand the causes ofpoor recruitment, particularly related to methodological aspects and context-specific study settings.

Methods: We compared RCTs that were discontinued or revised for poor recruitment to RCTs that were completedas planned, matching in terms of population and intervention. Based on an existing sample of RCTs discontinued orrevised due to poor recruitment, we identified matching RCTs through a literature search for systematic reviewsthat cited the discontinued or revised RCT and matching completed RCTs without poor recruitment. Based onextracted data, we explored differences in the design, conduct, and study settings between RCTs with and withoutpoor recruitment, separately for each research question using semi-structured discussions.

Results: We identified 15 separate research questions with a total of 29 RCTs discontinued or revised for poorrecruitment and 48 RCTs completed as planned. Prominent research areas in the sample were cancer and acutecare. The mean number of RCTs with poor recruitment per research question was 1.9 ranging from 1 to 4suggesting clusters of research questions or settings prone to recruitment problems. The reporting quality of therecruitment process in RCT publications was generally low. We found that RCTs with poor recruitment often hadnarrower eligibility criteria, were investigator- rather than industry-sponsored, were associated with a higher burdenfor patients and recruiters, sometimes used outdated control interventions, and were often launched later in timethan RCTs without poor recruitment compromising uncertainty about tested interventions through emergingevidence. Whether a multi- or single-center setting was advantageous for patient recruitment seemed to dependon the research context.

(Continued on next page)

© The Author(s). 2019 Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, andreproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link tothe Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver(http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

* Correspondence: matthias.briel@usb.ch1Basel Institute for Clinical Epidemiology and Biostatistics, Department ofClinical Research, University of Basel and University Hospital Basel, Basel,Switzerland2Department of Health Research Methods, Evidence, and Impact, McMasterUniversity, Hamilton, ON, CanadaFull list of author information is available at the end of the article

Briel et al. Trials (2019) 20:800 https://doi.org/10.1186/s13063-019-3957-4

(Continued from previous page)

Conclusions: Our study confirmed previously identified causes for poor recruitment, i.e., narrow eligibility criteria,investigator sponsorship, and a reduced motivation of patients and recruiters. Newly identified aspects were thatresearchers need to be aware of all other RCTs on a research question so that compromising effects on therecruitment can be minimized and that a larger number of centers is not always advantageous.

Keywords: randomized controlled trials , early termination of clinical trials, poor recruitment, systematic reviews,qualitative analysis

BackgroundEvidence-based health care relies on high-quality clinicalresearch. Randomized controlled trials (RCTs) are themethod of choice to assess preventive and therapeuticinterventions and are a cornerstone in the final phase ofdrug development and in comparative effectiveness re-search. Conducting high-quality RCTs, however, is chal-lenging. More than a quarter of RCTs do not reach theplanned sample size, mostly due to poor recruitment ofpatients, and are prematurely discontinued [1, 2].Investigator-initiated RCTs are particularly prone [1].Implications of poor patient recruitment and prematurediscontinuation of RCTs are that up to 70% of such tri-als remain unpublished, root causes of recruitment diffi-culties are not shared with the scientific community andmay therefore be repeated in the future, research ques-tions remain unanswered, and substantial amounts ofscarce research resources are wasted [1, 3].To better understand the causes of poor recruitment,

previous quantitative approaches using RCT protocolsand registry information [1, 2] as well as qualitative ana-lyses from published reports and semi-structured inter-views with trialists and other stakeholders in clinicalresearch have already provided important insights [3, 4].That is, for instance, that investigator-initiated RCTs orRCTs in the acute care setting were found to be at muchhigher risk for discontinuation due to poor recruitmentthan industry-sponsored RCTs or RCTs in non-acutecare settings [1, 2]; or that insufficient preparation,overly narrow eligibility criteria, and prejudiced views ofrecruiters and patients on trial interventions are com-mon reasons for poor recruitment [3]. As suggested byothers previously [5], we undertook a systematic com-parison of RCTs discontinued or revised for poor re-cruitment and RCTs completed as planned with thesame underlying research question. We aimed to provideadditional evidence on potential causes for poor recruit-ment specifically related to design aspects and context-specific study settings of RCTs.

MethodsThis is a matched qualitative study comparing RCTs thatdid not reach 90% of their originally planned sample sizedue to poor recruitment (cases; discontinued or revised

RCTs with poor recruitment) to RCTs completed asplanned (controls without poor recruitment) matchingin terms of patient population and experimental inter-vention. The study is reported according to the Stan-dards for Reporting Qualitative Research (SRQR, http://www.equator-network.org/reporting-guidelines/srqr/) asdescribed in Additional file 1.

Identification of discontinued and matching completedRCTsIn a previous study, we included 20 RCTs that were dis-continued or revised for poor recruitment with theplanned sample size being reported in a publication [1].For these studies, we aimed to find matching RCTsreaching at least 90% of their originally planned samplesize. We searched for systematic reviews that cited thediscontinued or revised RCT using the “times cited”function in Web of Science (times cited “view all of thearticles that cite this one” https://apps.webofknowledge.com) in January 2016. One reviewer (VG) screened thetitles, abstracts, and full texts of potentially eligible sys-tematic reviews for relevance. An eligible systematic re-view had to describe a literature search conducted in atleast one electronic database (e.g., Medline), includeRCTs, and had to have a research question similar tothat of the discontinued or revised RCT. If more thanone systematic review were eligible, we chose the mostup-to-date, comprehensive systematic review (frequentlya Cochrane review). If no systematic review was identi-fied, we searched by the same means for similar narra-tive reviews. In case VG was in doubt about theeligibility of a systematic review, she involved a secondreviewer (MB) for discussion and consensus decision.From each eligible systematic review we retrieved the

full text articles of included RCTs (i.e., potentially eli-gible matching RCTs) and collected a small set of pre-liminary data. These included whether the RCT wasdiscontinued or revised for poor recruitment, plannedand actually achieved sample size, the patient popula-tion, and the experimental and control intervention.This was done by two methodologically trained investi-gators (VG, BS) working independently and in duplicate.Any disagreements were resolved by consensus and, ifneeded, through involving another investigator (MB).

Briel et al. Trials (2019) 20:800 Page 2 of 12

The matching criteria (inclusion criteria) were the testedintervention and included patient population, whichneeded to be similar enough so that the RCTs could beincluded in the same meta-analysis. Other trial charac-teristics such as comparator interventions, outcomes ofinterest, and trial settings already qualified as factors po-tentially associated with poor recruitment.

Data collectionThe following data were collected from the eventually in-cluded RCTs: study design (e.g., superiority or non-inferiority trial, factorial or parallel design, allocation ratio),sample size calculation, allocation concealment, blinding,reporting quality of the recruitment process, eligibility cri-teria, trial sponsor, country and place of patient recruitment,recruitment period, reporting of recruitment networks, sup-port from a clinical trial unit or contract researchorganization, study population, interventions, comparators,and primary outcome. In addition, we also extracted self-reported reasons (if any) for poor recruitment of thediscontinued or revised RCTs. Data were collected by one in-vestigator (VG or BS) and checked by another (VG, BS orMB). Any disagreements were resolved by consensus.

Data analysisWe explored differences in the design, conduct. and studysettings between RCTs with and without poor recruit-ment, separately for each research question, thus context-specific, using semi-structured discussions (MB, BS, andVG). In a first round, we gathered differences betweenRCTs with and without poor recruitment, based on ourextracted data; we systematically went through a pre-specified checklist of items potentially associated withpoor recruitment, including eligibility criteria, trial spon-sor, single versus multiple centers, etc. (Additional file 2);and discussed each item in turn followed by observationsbeyond the checklist items (e.g., the chronology of RCTsbuilding up the evidence base for a specific research ques-tion). When we reached agreement among us, the identi-fied differences and potentially relevant observations werecaptured by VG in free-text form. In a second round, wereflected on the identified differences between RCTs withand without poor recruitment and other relevant observa-tions for each research question and integrated the rele-vant information in order to come up with an explanationwhy for a certain research question one or more RCTshad serious recruitment problems while others did nothave such problems. In addition, we considered the self-reported reasons for poor recruitment (if any).

Researchers’ reflexivityThe three researchers who carried out the analysis have di-verse disciplinary background and training such as medicine/clinical epidemiology (MB), nutrition/health technology

assessment (VG), and epidemiology/public health (BS). Tobetter understand specific characteristics of oncology trials,we involved a practicing oncologist in the respective discus-sions. For the other topics, the analysis team members feltthat they had sufficient knowledge to assess trial characteris-tics, and their methodological expertise with clinical trialslikely strengthened the analysis. None of us knew any of theincluded RCTs or their investigators. During analysis, all re-searchers worked together as a team and extensively dis-cussed the data interpretation to minimize bias.

ResultsStudy sampleIn a previous study, there were 20 published RCTs discon-tinued or revised for poor recruitment that reported theplanned sample size [1]. For five of these RCTs our litera-ture search could not identify a systematic or narrative re-view that cited the discontinued or revised RCT (Fig. 1).For the remaining 15 RCTs discontinued or revised due topoor recruitment representing 15 different research ques-tions, we identified 110 matching RCTs. We excluded 48RCTs because the planned sample size was not reported(n = 31); the RCT was discontinued for another reasonthan poor recruitment (n = 9); the article was not pub-lished in English (n = 3); the research question was notsimilar enough (n = 2); there was no full-text publicationavailable (n = 2); or the same results were published mul-tiple times (n = 1). Of the 62 matching RCTs, 48 werecompleted as planned and 14 were newly identified RCTsdiscontinued or revised for poor recruitment. Hence, for15 separate research questions a total of 77 RCTs (29 withand 48 without poor recruitment) were available for com-parative analyses. Twenty-five of the 29 RCTs with poorrecruitment explicitly reported that they were discontin-ued due to recruitment problems; the remaining fourRCTs [6–9] had recruitment problems but revised theiroriginal target sample sizes during the trial (reduction ofthe originally planned number of patients in each trial byabout 50%) and met the revised targets. References of allincluded RCTs are provided in Additional file 3.

Research questions, recruitment, and reporting qualityMedical areas of the 15 included research questions werecancer research (n = 5), research in acute care (n = 8; in-cluding preterm infants [n = 2] and laboring women [n =1]), surgery (n = 1), and infectious diseases (n = 1). Themean number of RCTs with poor recruitment was 1.9 perresearch question ranging from 1 to 4. Table 1 summa-rizes the recruitment characteristics for the RCTs withand without poor recruitment across all research ques-tions. In general, RCTs with poor recruitment recruitedfewer patients per year per recruiting center compared toRCTs that were completed as planned. More detailed re-cruitment characteristics for each research question and

Briel et al. Trials (2019) 20:800 Page 3 of 12

for all included RCTs, as well as general study characteris-tics are provided in Additional files 4, 5 and 6.The reporting of the recruitment process was generally

in included RCT publications with little detail. None ofthe articles reported on who actually recruited patientsor the anticipated prevalence of eligible patients. Only6% (5/77) of the RCTs reported on the anticipated re-cruitment duration, 51% (39/77) reported the locationwhere patients were recruited, 27% (21/77) provided adetailed patient flow, and 90% (69/77) reported the ac-tual recruitment period or duration (Additional file 7).

Comparison of RCTs with and without poor recruitmentTable 2 summarizes the differences observed betweenRCTs with and without poor recruitment for each researchquestion as well as our context-specific conclusions on thepossible reasons for poor recruitment. The most recurrenttheme across research questions was that, in RCTs withpoor recruitment, eligibility criteria were substantially nar-rower than in RCTs without poor recruitment (researchquestions #1, #2, #3, #4, #9, #12, #13 in Table 2).There was no consistent pattern as to whether an

international or national multicenter setting or a single-

Fig. 1 Selection of included randomized controlled trials. *Five of the original 20 discontinued RCTs for poor recruitment that were publishedand reported a sample size calculation were excluded, because we did not identify a systematic review or narrative review citing the RCT.**Including four RCTs with recruitment problems but revised original target sample sizes during the trial (reduction of the originally plannednumber of patients in each trial by about 50%) and meeting their revised targets. RCT randomized controlled trial

Table 1 Recruitment characteristics of included randomized controlled trials with and without poor recruitment across researchquestions

RCTs with poor recruitment, n = 29 RCTs without poorrecruitment, n = 48

Originally planned number of patients (median; IQR) 395 (72–600) 272 (22–500)

Number of patients randomized (median; IQR) 150 (41–329) 306 (23–559)

Duration of recruitment period in months (median; IQR) 39 (10–47) 24 (7–45)

Number of patients recruited per year and centera (median; IQR) 3 (1–5) 7 (1–25)

Trial sponsor (industry n (%) / investigator n (%)) 12 (41%) / 17 (59%) 20 (42%) / 27 (56%) /NR 1 (2%)

IQR interquartile range (25th and 75th percentile), NR not reported, RCT randomized controlled trialaRough estimate for recruitment speed based on own calculations (number of patients recruited divided by recruitment duration and number of study centers);not adjusted to the time a site was actually open for recruitment, because this was not reported in the publications of included trials)

Briel et al. Trials (2019) 20:800 Page 4 of 12

Table

2Differen

cesbe

tweenrand

omized

controlledtrialswith

andwith

outpo

orrecruitm

entandconclusion

son

thepo

ssiblereason

sforpo

orrecruitm

ent

Group

sof

RCTs

with

acommon

research

questio

nObserveddifferences

with

respectto

stud

ycharacteristics

(design,cond

uct,spon

sor)be

tweenRC

Tswith

andwith

outpo

orrecruitm

ent

Con

text-spe

cific

conclusion

son

thepo

ssiblereason

sforpo

orrecruitm

ent

#1:Surgicaltreatmen

tcomparedto

proton

-pum

pinhibitorsin

gastroesop

hage

alrefluxdisease

RCTs

with

poor

recruitm

entn=2

RCTs

with

outpo

orrecruitm

entn=2

1)Stud

yspon

sor:RC

Tswith

poor

recruitm

entwereinvestigator-

spon

soredwhe

reas

RCTs

with

outpo

orrecruitm

entwere

indu

stry-spo

nsored

Absen

ceof

indu

stry

spon

sorin

gcouldmeanless

profession

altrial

organizatio

nandlim

itedfund

ingin

gene

ral;narrow

ereligibility

criteria

may

correlatewith

overestim

ationof

recruitm

entrates

andwith

slow

recruitm

ent;andahigh

erbu

rden

forpatients

durin

gfollow-upmay

have

lowered

themotivationof

reflux

patientsto

participate

2)Eligibility

criteria:m

orerestrictivein

theRC

Tswith

poor

recruitm

entwith

respectto

diseasedu

ratio

nanddu

ratio

nof

priorproton

-pum

pinhibitorusage

3)Patient

burden

durin

gfollow-up:

patient

assessmen

tdu

ring

follow-upmorefre

quen

tin

oneof

theRC

Tswith

poor

recruit-

men

t(Anvarietal.2011)

than

intheRC

Tswith

outpo

orrecruit-

men

t(e.g.,assessmen

tof

quality

oflifeevery3mon

thsvs.every

12mon

ths)

#2:A

nthracyclines

with

orwith

outanytaxane

scomparedto

anthracyclines

plus

anyanticancertreatm

entor

comparedto

sing

le-age

nttaxane

sin

metastatic

breastcancer

RCTs

with

poor

recruitm

entn=3

RCTs

with

outpo

orrecruitm

entn=3

1)National/internatio

nal:RC

Tswith

poor

recruitm

entwere

cond

uctedin

onecoun

try(national)whe

reas

theRC

Tswith

out

poor

recruitm

entwereinternational(1no

trepo

rted

)

Absen

ceof

aninternationaln

etworkof

recruitin

gcentersmay

beassociated

with

limitedrecruitm

entcapacitiesforpatients

with

metastatic

breastcancer;narrower

eligibility

criteria

may

correlatewith

overestim

ated

recruitm

entratesandwith

slow

recruitm

ent;andthefact

that

eviden

ceon

theeffectiven

ess

ofanthracyclinecombinatio

ntherapyfro

mprevious

RCTs

was

alreadyavailablemay

have

comprom

ised

themotivation

ofrecruitersin

theRC

Tswith

poor

recruitm

ent

2)Eligibility

criteria:m

orerestrictivein

2of

3RC

Tswith

poor

recruitm

ent(Bon

neterreet

al.2004andBo

nten

baletal.2005)

with

respectto

adjuvant

therapypriorto

enrolm

ent,i.e.,adjuvant

therapyhadto

bestop

pedat

least12

mon

thsbe

fore

3)Pu

blicationchrono

logy/availableeviden

ce:RCTs

with

poor

recruitm

entwerepu

blishe

d(years:2004,2005,and

2010)after2

ofthe3RC

Tswith

outpo

orrecruitm

ent(Biganzoliet

al.2002and

Jassem

etal.2001);the

othe

rRC

Twith

outpo

orrecruitm

entwas

publishe

din

2005

(Nabho

ltzet

al.2005)

#3:First-linetreatm

entwith

thearom

ataseinhibitorexem

estane

comparedto

anastrozolein

postmen

opausalw

omen

with

ad-

vanced

breastcancer

RCTs

with

poor

recruitm

entn=1

RCTs

with

outpo

orrecruitm

entn=1

Eligibility

criteria:m

orerestrictivein

theRC

Twith

poor

recruitm

entwith

respectto

metastasesanddiseasestage

(visceralm

etastaseson

lyvs.advancedstageor

anymetastasesin

theRC

Twith

outpo

orrecruitm

ent)

Narrower

eligibility

criteria

forpo

stmen

opausalw

omen

with

advanced

breastcancer

may

correlatewith

overestim

ated

recruitm

entratesandwith

slow

recruitm

ent

#4:A

ntiarrhythmicagen

tscomparedto

placeb

oor

backgrou

ndtherapywith

beta-blocker

inventricular

arrhythm

iaRC

Tswith

poor

recruitm

entn=1

RCTs

with

outpo

orrecruitm

ent

n=2

1)Interven

tiontested

:RCTs

with

outpo

orrecruitm

enttested

ane

winterven

tion,i.e.,celivaron

e(Kow

eyet

al.2011)

orazim

ilide

(Dorianet

al.2004)

whe

reas

theRC

Twith

poor

recruitm

ent[8]

tested

availabledrug

s(beta-blocker,am

iodarone

,and

sotalol)

Thefact

that

alreadyavailableantiarrhythmicdrug

sandno

new

drug

sweretested

may

have

comprom

ised

themotivation

ofrecruitersor

patients;recruitm

entcapacitiesmight

have

been

insufficien

tdu

eto

toofew

stud

ycenters;andnarrow

ereligibility

criteria

forpatientswith

ventricular

arrhythm

iamay

correlate

with

overestim

ated

recruitm

entratesandwith

slow

recruitm

ent

2)Num

berof

stud

ycenters:3to

4tim

eslower

inRC

Twith

poor

recruitm

ent

3)Eligibility

criteria:m

orerestrictivein

theRC

Twith

poor

recruitm

ent[8]with

respectto

usageof

antiarrhythmicagen

tspriorto

enrolm

entandtheim

plantablecardioverter

defib

rillator

device

(onlyon

especifictype

allowed

)

#5:Proph

ylactic

antib

ioticscomparedto

placeb

oor

usualcare

inacutene

crotizingpancreatitis

RCTs

with

poor

recruitm

entn=3

RCTs

with

outpo

orrecruitm

entn=0

Nodifferences

observed

becauseon

lyRC

Tswith

poor

recruitm

entwereiden

tified

Thefact

that

3RC

Tswerediscon

tinueddu

eto

poor

recruitm

ent

inthisresearch

area

sugg

eststhat

itseem

sge

nerally

prob

lematic

torecruitpatientswith

acutene

crotizingpancreatitis,which

isassociated

with

ahigh

mortalityrate

(vulne

rablepatients)

Briel et al. Trials (2019) 20:800 Page 5 of 12

Table

2Differen

cesbe

tweenrand

omized

controlledtrialswith

andwith

outpo

orrecruitm

entandconclusion

son

thepo

ssiblereason

sforpo

orrecruitm

ent(Con

tinued)

Group

sof

RCTs

with

acommon

research

questio

nObserveddifferences

with

respectto

stud

ycharacteristics

(design,cond

uct,spon

sor)be

tweenRC

Tswith

andwith

outpo

orrecruitm

ent

Con

text-spe

cific

conclusion

son

thepo

ssiblereason

sforpo

orrecruitm

ent

#6:Latepo

stnatal(>7days)corticosteroid

treatm

entcompared

toplaceb

oor

usualcarein

preterm

neon

ates

RCTs

with

poor

recruitm

entn=3

RCTs

with

outpo

orrecruitm

entn=7

1)Num

berof

stud

ycenters:allR

CTs

with

outpo

orrecruitm

ent

weresing

le-cen

terstud

ies,whe

reas

2(Karietal.1993andDoyle

etal.2006)

ofthe3RC

Tswith

poor

recruitm

entweremulticen

ter

stud

ies

Thefact

that

3RC

Tsrecruitedpo

orlyin

thisresearch

area

sugg

eststhat

itseem

sge

nerally

prob

lematicto

recruitpreterm

neon

ates

(vulne

rablepatients).Inasettingwith

vulnerable

patients,asing

le-cen

terRC

Tmay

proveadvantageo

usdu

eto

moremotivated

andbe

tter

prep

ared

/trained

recruitersand

closer

mon

itorin

gof

recruitm

ent).

Particularlyprob

lematic

appe

ared

thechoice

ofprim

aryou

tcom

ein

Doyleet

al.2006;

tofocuson

apo

tentialsideeffect

oftheinterven

tionmay

have

comprom

ised

themotivationof

recruitersandparentsto

includ

echildrenin

such

anRC

T.Equipo

isemay

becomprom

ised

forRC

Tscond

uctedlaterin

agrou

pof

RCTs

addressing

thesameresearch

questio

n(leadingto

lower

motivationof

recruitersandparents),b

ecause

eviden

ceon

the

effectiven

essof

aninterven

tionfro

mprevious

RCTs

was

already

available

2)Prim

aryou

tcom

e:theprim

aryou

tcom

echosen

inon

eRC

Twith

poor

recruitm

ent(Doyleet

al.2006)

focusedon

apo

tential

side

effect

oftheinterven

tion(m

ajor

neurosen

sory

disability),

whe

reas

theothe

rtrialshadan

effectiven

essou

tcom

e

3)Pu

blicationchrono

logy/availableeviden

ce:a

largeRC

Twith

poor

recruitm

ent(Doyleet

al.2006)

was

thelastpu

blishe

dRC

Tin

thisarea

ofresearch

whe

reas

allo

ther

RCTs

(discontinuedor

completed

)werepu

blishe

dearlier

andalreadyshow

edbe

nefits

interm

sof

better

respiratio

nof

infants

#7:Ven

tilatorygaswith

nitricoxidecomparedto

ventilatorgas

with

outnitricoxidein

preterm

neon

ates

RCTs

with

poor

recruitm

entn=2

RCTs

with

outpo

orrecruitm

entn=5

1)Num

berof

stud

ycenters:Thenu

mbe

rof

stud

ycenterswas

consistentlysm

alleram

ongtheRC

Tswith

outpo

orrecruitm

ent

Inasettingwith

vulnerablepatients(preterm

neon

ates)a

smallernu

mbe

rof

stud

ycenterswith

in1coun

trymay

prove

advantageo

usdu

eto

less

complex

trialo

rganizationand

recruitm

entmon

itorin

gandpo

tentially

moremotivated

and

better

prep

ared

/trained

recruiters.

2)National/internatio

nal:whe

reas

allR

CTs

with

poor

recruitm

ent

wereinternationalstudies,allbu

ton

eRC

Twith

outpo

orrecruitm

ent(Hascoet

etal.2005)

hadanatio

nalsettin

g

#8:Prim

aryangiop

lastycomparedto

on-site

thrombo

lytic

ther-

apyin

acutemyocardialinfarction(with

in12

h)RC

Tswith

poor

recruitm

entn=3

RCTs

with

outpo

orrecruitm

entn=3

Target

samplesize:the

numbe

rof

patientsplanne

dto

been

rolledwas

consistentlyhigh

erin

RCTs

with

poor

recruitm

ent

Thefact

that

3RC

Tsrecruitedpo

orlyin

thisresearch

area

sugg

eststhat

itseem

sge

nerally

prob

lematicto

recruitacute

care

patients.Thecomparison

ofangiop

lasty(rapidpatient

transfer

tocenter

necessary)to

on-site

thrombo

lytic

therapy

was

logisticallyvery

challeng

ingat

thetim

e.AllRC

Tswith

poor

recruitm

entweremeant

toinclud

emore

patientsthan

thecompleted

RCTs,con

sequ

ently

requ

iring

morecentersandofteninternationalcollabo

ratio

n,which

additio

nally

increasedchalleng

eswith

triallog

istics

#9:M

oxifloxacin

comparedto

othe

rantib

ioticsin

patientswith

pneumon

iaRC

Tswith

poor

recruitm

entn=2

RCTs

with

outpo

orrecruitm

entn=4

1)Patient

popu

latio

n:bo

thRC

Tswith

poor

recruitm

entinclud

edpatientswith

hospital-acquiredpn

eumon

ia,atleastin

partmech-

anicallyventilated(poten

tially

sicker/m

orevulnerablepatients),

whileRC

Tswith

outpo

orrecruitm

entinclud

edpatientswith

commun

ity-acquiredpn

eumon

iawith

outmechanicalven

tilation

Recruitm

entof

sicker/m

echanically

ventilatedpatients

appe

ared

morechalleng

ing(inform

edconsen

tetc.);

patientswho

acqu

iredpn

eumon

iain

aho

spitalm

ayhave

been

less

motivated

toparticipatein

anRC

Tcond

uctedat

that

hospital,particularlyifseverelyaffected

(mechanically

ventilated)

Inadditio

n,all6

RCTs

werede

sign

edas

non-inferio

ritytrials,i.e.,thetested

antib

ioticswereno

thypo

thesized

tohave

supe

rioreffectsbu

tmay

have

been

cheape

ror

easier

toapply,so

patientswith

hospital-acquired

pneumon

iacouldno

texpe

ctamoreefficacious

treatm

ent

andmight

have

thereforebe

enless

motivated

toparticipate.

Forthesamereason

s(sickerpatients,ho

spital-acquired

infection,no

n-inferio

rityde

sign

)recruitersmay

have

been

less

motivated

,too

.Theeligibility

criteria

requ

iredtheapplicationof

acomplex

scoringsystem

(e.g.,APA

CHEscore)

tode

term

inepatient

eligibility,thismay

have

increasedthebu

rden

forrecruiters

2)Eligibility

criteria:o

neof

theRC

Tswith

poor

recruitm

entused

theAPA

CHE(Acute

Physiology

And

Chron

icHealth

Evaluatio

n)scoreto

defineeligiblepatients(Höffken

etal.the

RCTs

with

out

poor

recruitm

entdidno

t

Briel et al. Trials (2019) 20:800 Page 6 of 12

Table

2Differen

cesbe

tweenrand

omized

controlledtrialswith

andwith

outpo

orrecruitm

entandconclusion

son

thepo

ssiblereason

sforpo

orrecruitm

ent(Con

tinued)

Group

sof

RCTs

with

acommon

research

questio

nObserveddifferences

with

respectto

stud

ycharacteristics

(design,cond

uct,spon

sor)be

tweenRC

Tswith

andwith

outpo

orrecruitm

ent

Con

text-spe

cific

conclusion

son

thepo

ssiblereason

sforpo

orrecruitm

ent

andredu

cedtheirmotivation

#10:Temozolom

ide(che

mothe

rape

uticagen

t)alon

eor

incombinatio

nwith

radiothe

rapy

comparedto

nochem

othe

rapy

(e.g.,radiothe

rapy),no

n-temozolom

ide-basedchem

othe

rapy

ortemozolom

ideat

different

dosesin

gliomapatients

RCTs

with

poor

recruitm

entn=1

RCTs

with

outpo

orrecruitm

entn=3

Inthisgrou

pof

RCTs

with

gliomapatients,alltrialswere

publishe

dafterthecompleted

andpu

blishe

dRC

Tcond

ucted

byStup

pet

al.2005.Subseq

uent

tothispu

blication,itwas

prob

ablydifficultto

recruitpatientsin

RCTs

that

didno

tinclud

etheinterven

tionthat

was

show

nto

besupe

riorin

Stup

pet

al.2005(com

binatio

nof

radiothe

rapy

and

chem

othe

rapy).How

ever,neither

theRC

Twith

poor

recruitm

ent(M

almström

etal.2012)

noranothe

rRC

Twith

outpo

orrecruitm

ent(W

icket

al.2012)

includ

edsuch

acombinatio

ntherapy,bo

thRC

Tswerevery

similar

with

respectto

patient

popu

latio

n(elderly)andcompared

interven

tions

(tem

ozolom

idevs.stand

ardradiothe

rapy).

Theon

lyothe

rRC

Twith

outpo

orrecruitm

entcompared

temozolom

ideto

anothe

rchem

othe

rape

utictreatm

entin

patientsno

tsuitableforradiothe

rapy

(Brada

etal.2010)

#11:Capecitabine

-based

chem

othe

rapy

comparedto

non-

cape

citabine

chem

othe

rapy

inmetastatic

breastcancer

RCTs

with

poor

recruitm

entn=4

RCTs

with

outpo

orrecruitm

entn=5

Interven

tiontested

:excep

tfor1RC

Twith

poor

recruitm

ent

(Bache

lotet

al.2011)

theRC

Tswith

poor

recruitm

entallocated

patientsto

either

anoralmon

othe

rapy

with

cape

citabine

oran

intraven

ousapplicationof

acape

citabine

-free

chem

othe

rapy

ei-

ther

asmon

othe

rapy

oras

combinatio

ntherapy.In

contrast,inall

but1RC

Twith

outpo

orrecruitm

ent(O’Shaug

hnessy

etal.2001),

therouteof

administrationwas

iden

ticalforbo

thtreatm

entarms

(alwaysinclud

edan

intraven

ousapplication)

andcape

citabine

was

(with

thesameexceptionof

O′Shaug

hnessy

etal.2001)

givenas

combinatio

ntherapy.TheRC

Twith

outpo

orrecruitm

ent

O′Shaug

hnessy

etal.2001,similarto

theRC

Tswith

poor

recruit-

men

talso

allocatedpatientseither

tointraven

ousor

oraltreat-

men

t,bu

tat

thetim

eof

trialcon

duct,oralm

onothe

rapy

with

cape

citabine

was

relativelyne

wanda2:1rand

omizationwas

done

tomakeitmorelikelyforpatientsto

getallocatedto

the

anticipated

patients’preferredtreatm

entop

tion

Patients’preferen

cesregardingtheform

ofapplication

might

have

played

arolein

thisresearch

area,i.e.,patient

recruitm

entin

RCTs

comparin

gdifferent

interven

tions

that,

inadditio

n,usedifferent

routes

ofapplicationishampe

red

ifpatientshave

preferen

cesabou

ttherouteof

application

andthereforeareless

willingto

berand

omized

.Tw

oothe

rreason

sforpo

orrecruitm

entrelatedto

patients’

orrecruiterpreferen

cesarepo

ssible,b

utcouldbe

specific

fortherespectiveRC

T.(1)In

1RC

Twith

poor

recruitm

ent

(Talbo

tet

al.2002)

thetested

drug

was

appliedcontinuo

usly

whe

reas

interm

itten

tlymight

have

been

preferredby

patients.

(2)In

anothe

rRC

Twith

poor

recruitm

ent(Stockleret

al.2011)

thecontroltreatmen

t(cycloph

osph

amide,metho

trexate,and

fluorou

racil,CMF)

was,atthetim

e,ou

tdated

andtherefore

themotivationof

recruitersandpatientsmight

have

been

redu

ced.

Inthediscussion

authorsstated

:“Themain

limitatio

nsof

ourtrialare

theam

bigu

ityof

broad,

pragmatic

eligibility

criteria

andtheuseof

anun

fashionablecontrol

regimen

.”(Stockler

etal.2011)

#12:Prim

arythrombo

prop

hylaxis(hep

arin)comparedto

placeb

oor

usualcarein

ambu

latory

cancer

patientsreceiving

chem

othe

rapy

RCTs

with

poor

recruitm

entn=2

RCTs

with

outpo

orrecruitm

entn=2

1)Stud

yspon

sor:bo

thRC

Tswith

poor

recruitm

entwere

investigator

initiated

whe

reas

both

RCTs

with

outpo

orrecruitm

entwereindu

stry-spo

nsored

Testingthrombo

prop

hylaxiswith

heparin

incancer

patientsmay

bechalleng

ingin

gene

ral,be

causetheinterven

tionwas

not

focusedon

combatin

gthecancer,and

thereforerecruiters’and

patients’interestin

such

anRC

Tmay

beredu

cedandseen

asbu

rden

;com

petin

gRC

Tstestingantitum

oragen

tsmight

bepreferred.

Inadditio

n,theabsenceof

indu

stry

spon

sorin

gcouldmeanless

profession

altrialo

rganizationandlim

ited

fund

ingin

gene

ral,andafocuson

only1type

ofcancer

(narrow

eligibility

criteria,Perry

etal.2002)

may

have

contrib

uted

topo

orrecruitm

ent

2)Eligibility

criteria:o

nlygliomapatientswereeligiblein

1of

the

RCTs

with

poor

recruitm

ent(Perry

etal.2002)

whilebo

thRC

Tswith

outpo

orrecruitm

entinclud

edpatientswith

vario

ustype

sof

cancer

Briel et al. Trials (2019) 20:800 Page 7 of 12

Table

2Differen

cesbe

tweenrand

omized

controlledtrialswith

andwith

outpo

orrecruitm

entandconclusion

son

thepo

ssiblereason

sforpo

orrecruitm

ent(Con

tinued)

Group

sof

RCTs

with

acommon

research

questio

nObserveddifferences

with

respectto

stud

ycharacteristics

(design,cond

uct,spon

sor)be

tweenRC

Tswith

andwith

outpo

orrecruitm

ent

Con

text-spe

cific

conclusion

son

thepo

ssiblereason

sforpo

orrecruitm

ent

#13:Recombinant

tissueplasminog

enactivator

(rtPA

)compared

toplaceb

oin

acuteischem

icstroke

RCTs

with

poor

recruitm

entn=1

RCTs

with

outpo

orrecruitm

entn=5

1)Patient

popu

latio

nandeligibility

criteria:the

RCTwith

poor

recruitm

entfocusedon

patientswho

didno

tfulfilthe

licen

secriteria

ofrtPA

(i.e.,elderlypatients>80

years),w

hereas

RCTs

with

outpo

orrecruitm

entdid(fo

cuson

patients<80

years)

Thefact

that

theRC

Twith

poor

recruitm

entfocusedon

elde

rlypatients(>

80years;i.e.,narrow

eligibility

criteria,vulne

rable

patients)to

berecruitedin

anacutecare

settingmay

have

contrib

uted

topo

orrecruitm

ent.Themorecomplex

subg

roup

hypo

theses

intheRC

Twith

poor

recruitm

entcouldhave

made

thetrialm

oredifficultto

explainto

colleaguesandpatients.

Finally,itispo

ssiblethat

themotivationof

recruitersand

patientswas

redu

ced,

becauseeviden

ceon

theeffectiven

ess

from

previous

RCTs

was

alreadyavailable(com

prom

ised

equipo

ise)

2)Target

samplesize:the

numbe

rof

patientsplanne

dto

been

rolledwas

high

erin

theRC

Twith

poor

recruitm

entdu

eto

planne

dsubg

roup

analyses

(morecomplex

design

/hypothe

ses)

3)Pu

blicationchrono

logy/availableeviden

ce:the

RCTwith

poor

recruitm

ent(Sande

rcocket

al.2012)

was

thelastRC

Tcond

ucted

forthisresearch

questio

nto

date

#14:Transdermalnitrog

lycerin

comparedto

placeb

oor

usual

care

inlabo

ringwom

en(gestatio

nalage

betw

een24

and32

weeks)

RCTs

with

poor

recruitm

entn=1

RCTs

with

outpo

orrecruitm

entn=1

1)Patient

burden

durin

gfollow-up:

long

erfollow-upwith

more

complex

assessmen

tsof

thene

onates

intheRC

Twith

poor

recruitm

ent

Labo

ringwom

enin

both

RCTs

hadto

beat

high

risk

ofpreterm

deliverybu

talso

nottooadvanced

sothat

they

couldstillbe

rand

omized

;how

ever,the

RCTwith

poor

recruitm

entim

posedahigh

erbu

rden

onpatientswith

long

erfollow-upandamorecomplex

assessmen

tof

thene

onates,

prob

ablyredu

cing

themotivationof

patientsandrecruitersto

participate;in

contrast,the

prim

aryou

tcom

e(i.e.,num

berof

days

from

rand

omizationto

delivery)in

theRC

Twith

outpo

orrecruitm

entdidno

trequ

ireanyeffortfro

mpatients.Afew

well-p

ickedstud

ycentersmay

have

been

advantageo

usin

thisacutecare

setting.

Themotivationof

recruitersand

patientswas

prob

ablyfurthe

rredu

cedin

thetrialw

ithpo

orrecruitm

ent,whe

neviden

ceon

theeffectiven

essof

the

tested

interven

tion(nitrog

lycerin

)fro

motherRC

Tsbe

came

available(com

prom

ised

equipo

ise,Sm

ithet

al.2007).

Only2RC

Tswereconsidered

inthisgrou

pof

RCTs,b

ecause

themajority

(n=7)

ofRC

Tsinclud

edin

thesamesystem

atic

review

didno

trepo

rtanysamplesize

calculation/target

samplesize

potentially

hiding

recruitm

entprob

lemsin

thesetrials

2)Prim

aryou

tcom

e:theRC

Twith

poor

recruitm

enthadamore

complex

prim

aryou

tcom

e(neo

natalm

orbidity

associated

with

long

-term

morbidity

andpe

rinatalmortalityvs.num

berof

days

from

rand

omizationto

delivery)

3)Num

berof

stud

ycenters:theRC

Twith

poor

recruitm

enthad

over

3tim

esmorestud

ycentersthan

theRC

Twith

outpo

orrecruitm

ent

4)Pu

blicationchrono

logy/availableeviden

ce:the

RCTwith

poor

recruitm

entwas

stillon

goingwhe

ntheRC

Twith

outpo

orrecruitm

entwas

publishe

d

#15:Vasopressin-containing

regimen

comparedto

epinep

hrine

incardiacarrest

RCTs

with

poor

recruitm

entn=1

RCTs

with

outpo

orrecruitm

entn=5

Num

berof

stud

ycenters:theRC

Twith

poor

recruitm

entwas

amulticen

terRC

Twhe

reas

allb

uton

eRC

Twith

outpo

orrecruitm

ent(Gueug

niaudet

al.2008)

weresing

le-center

RCTs.

Inthisacutecare

settingwith

vulnerablepatientsasing

lecenter

RCTmay

beadvantageo

usdu

eto

high

ermotivation

andbe

tter

prep

ared

/trained

recruitersandtrialteam

and

closer

mon

itorin

gof

recruitm

ent).

TheRC

Twith

poor

recruitm

entwas

cond

uctedin

44commun

ity-based

emerge

ncyroom

s.One

othe

rRC

Twith

outpo

orrecruitm

ent(Gueug

niaudet

al.2008)

hadasimilar

numbe

rof

stud

ycentersas

thediscon

tinuedRC

T,bu

tauthorsrepo

rted

that

thene

tworkof

centerswas

well

stablishe

dandexpe

rienced

inrunn

ingacutecare

RCTs.

RCTrand

omized

controlledtrial;Allof

thearticlescitedin

Table2arereferenced

inAdd

ition

alfile3

Briel et al. Trials (2019) 20:800 Page 8 of 12

center setting was advantageous for patient recruitmentin RCTs. In research question #4, for instance, investi-gating antiarrhythmic drugs, the RCT with poor recruit-ment had three to four times fewer study centers thanRCTs without poor recruitment; or in research area #2(metastatic breast cancer therapy) the RCTs with poorrecruitment were all restricted to a national setting,while the RCTs without poor recruitment were all donein large international collaborations. On the other hand,single-center RCTs or settings with only a few, carefullychosen study centers may have worked better in settingswith particular logistical challenges (e.g., question #8 onprimary angioplasty versus onsite thrombolysis andquestion #15 testing therapies for resuscitation) or theinclusion of particularly vulnerable patients (e.g., re-search questions #6 and #7 focusing on the recruitmentof preterm neonates) in the absence of well-establishedand experienced trial networks.We observed that investigator-sponsored RCTs are as-

sociated with a higher risk for poor recruitment thanindustry-sponsored RCTs; in research questions #1 and#12 all RCTs with poor recruitment were investigator-sponsored, while all RCTs without poor recruitmentwere industry-sponsored.The chronology of RCTs, e.g., when a RCT is launched

while other RCTs on the same research question havealready been completed, appears to impact on recruit-ment. RCTs with poor recruitment were often initiatedand published later, after other RCTs had already beensuccessfully completed (research questions #2, #6, #10,#13, #14); i.e., evidence on the potential benefits andharms of a certain intervention was already available atsome point during the conduct of such RCTs. This mayhave compromised the uncertainty about the testedtreatments as an ethical precondition (equipoise), andthe motivation of patients and recruiters for furtherrandomization. In research questions #10 and #11 thecontrol intervention(s) were already outdated, when theRCTs with poor recruitment were launched. In addition,in some RCTs with poor recruitment the burden for pa-tients, such as numerous or invasive assessments duringfollow-up (research questions #1, #12, #14), or the burdenfor recruiters, such as the need to apply a complex scoringsystem in order to include patients (research question #9),was higher than in corresponding RCTs without poor re-cruitment. Furthermore, it happened that the tested inter-ventions (already available drugs in RCTs with poorrecruitment versus new drugs in RCTs without poor re-cruitment; research question #4) or the study design withside effects of the experimental drug being the primaryoutcome (research question #6) were less attractive inRCTs with poor recruitment than in corresponding RCTswithout poor recruitment. If interventions to be comparedin an RCT not only differed in the administered substance

or drug but, in addition, in the route (e.g., intravenous ororal administration) or timing of application, then pa-tients’ preferences could have compromised their willing-ness to be randomized (research question #11 in Table 2).

DiscussionMain findingsIn this qualitative comparison between RCTs that didnot achieve their originally planned sample size due torecruitment problems and RCTs that were completed asplanned, we identified several reasons for poor recruit-ment. We found that RCTs with poor recruitment oftenhad narrower eligibility criteria than RCTs without poorrecruitment, were investigator-sponsored rather thanindustry-sponsored, and were less attractive for patientsand recruiters due to higher burden, outdated controlinterventions, or already existing or accumulating evi-dence on benefits and harms of interventions from otherRCTs. An existing network of study centers experiencedin the conduct of RCTs is instrumental for successful re-cruitment, but whether one or a few study centers or aninternational multicenter design was more advantageousseemed to depend on the research context. With chal-lenging settings such as acute care, vulnerable patients,or complex logistics, one or a few carefully chosen cen-ters may be preferable due to closer monitoring of re-cruitment, potentially better prepared and motivatedstudy staff, established procedures and, perhaps mostimportant, efficient communication among the trialteam. Prominent research areas in our sample were can-cer research and research in acute care with, on average,more than one RCT with poor recruitment per researchquestion suggesting that there are clusters of researchareas typically prone to recruitment problems.

Strengths and limitationsThe strengths of our study include a new systematic ap-proach to qualitatively compare RCTs discontinued orrevised due to poor recruitment and RCTs completed asplanned, that had previously been suggested [5] but, toour knowledge, never been used to date. We included 77RCTs from a broad range of settings and research topicsand analyzed them specifically in their context, therebystrengthening the applicability of our results.Our study has several limitations. First, the present

qualitative analysis was limited to the information pro-vided in publications of RCTs and did not include infor-mation from other sources such as study protocols orinterviews with trialists. This might have constituted a se-lection because the majority of RCTs discontinued forpoor recruitment were not published in a peer-reviewedjournal [1]. Second, we were not able to assess reasons forpoor recruitment that were not described (e.g., lack offunding, a theme recurrently coming up in an interview

Briel et al. Trials (2019) 20:800 Page 9 of 12

study on the topic [4]). Third, we did not comprehensivelysearch the literature for reports of RCTs discontinued orrevised due to poor recruitment, but pragmatically startedout with a sample of discontinued or revised RCTs identi-fied in a previous study [1], and we used existing system-atic and narrative reviews to find matching RCTs. Inaddition, we excluded 31 RCTs from our analysis becausearticles did not report a planned sample size, and there-fore we were unable to judge whether the originallyplanned sample size was achieved or not. Fourth, thereporting of the patient recruitment process in includedRCT publications provided little detail (irrespective ofwhether the RCT struggled with recruitment or not) com-promising the effectiveness of our qualitative analysis. Par-ticularly the fact that many articles did not report thenumber of patients screened for eligibility, the numbernot meeting eligibility criteria, and the number of patientsdeclining to participate, often limited a better understand-ing of potential recruitment problems. Fifth, one re-searcher extracted all relevant information from includedRCT publications and another checked this informationrather than two researchers extracting relevant data inde-pendently and in duplicate. We chose this approach forfeasibility reasons risking a higher rate of extraction errors.However, information directly relevant for our interpret-ation of reasons underlying recruitment problems were ac-tually verified by two methodologically trained researchers.Sixth, although our study captured a broad range of clinicalareas, 12 of the 15 research questions were related to drugtherapy, leaving uncertainty whether our findings areequally applicable to other interventions such as surgery,behavior change, or complex interventions. Finally, al-though we found evidence for saturation in our qualitativeanalysis, the size of our study sample was mainly deter-mined by practical issues of our approach.

Comparison with other studies investigating poorrecruitment in RCTsOur results confirm several findings of previous studiesusing different methods. Investigator-sponsored (in thesense of investigator-initiated) RCTs, for instance, werefound to be at higher risk for discontinuation due topoor recruitment than industry-sponsored RCTs by twostudies using a quantitative approach with multivariableregression analysis [1, 2]. The common interpretation isthat industry sponsorship is associated with sufficientfunding and better planning and conduct, factors that fa-cilitate successful patient recruitment. Moreover, theacute care setting (e.g., emergency rooms, intensive careunits, care for preterm neonates) seems particularlyprone to insufficient recruitment [10].Similar to the present study, a systematic review of

published reports of RCTs discontinued due to poor re-cruitment found that overly narrow eligibility criteria

and prejudiced views of recruiters and patients on trialinterventions were the most frequent reasons for poorrecruitment [3]. Prejudiced views of patients and re-cruiters may come from different sources. Our studyfound that RCTs with poor recruitment were oftenlaunched relatively late in the sequence of RCTs on thesame research question. As evidence accumulates overtime, the uncertainty about the benefits and harms of acertain intervention or about the superiority of oneintervention over another (equipoise) may become in-creasingly compromised. Some control interventionswere even considered outdated right from the start of anRCT (research questions #10 and #11 in Table 2), whichconfirms the observation by Habre et al. [11]. In someinstances, the route of application of an intervention orthe more complex logistics associated with an interven-tion were less appealing to patients or recruiters. This isalso consistent with the finding by Bernardez-Pereiraet al. that single-arm clinical trials were less prone todiscontinuation due to poor recruitment compared tomultiple-arm trials. That is because in single-arm trialsall patients receive the same intervention and, thus, arenot confronted with the fact that they could be random-ized to a different, maybe less preferred, treatment [2].Another common finding in RCTs with poor recruit-

ment was a high burden or inconvenience for patients orrecruiters due to trial procedures (e.g., many follow-upvisits, blood draws, lengthy questionnaires or case reportforms). This was mentioned previously as a problem inseveral other qualitative studies [3, 12–14].Our study highlights two aspects about recruitment

challenges to RCTs that are, so far, not prominent in thepublished literature. First, investigators need to be awareof all other RCTs on a research question, their timing,and the accumulating evidence base, so that potentiallycompromising effects on the recruitment to their owntrial can be minimized. Second, apart from the notionthat well-established networks of collaborating studycenters are an asset for the successful recruitment of pa-tients to RCTs [4], it seems that a larger number of cen-ters is not always advantageous. The performance of astudy center typically depends on its commitment to-ward an RCT, the enthusiasm, training, and quality ofcommunication of staff; therefore, challenging settingssuch as acute care, vulnerable patients, or complex logis-tics require careful selection and close monitoring ofparticipating centers.Finally, our study documents the urgent need for a more

detailed reporting of participant recruitment in RCTs, whichis in line with previous reports [3, 15, 16]. Furthermore, wedid not find that poor reporting of the recruitment processwas a particular issue of RCTs discontinued or revised forpoor recruitment. Indicating that, if they do indeed get pub-lished, the reporting quality of the recruitment process is

Briel et al. Trials (2019) 20:800 Page 10 of 12

similarly poor as in RCTs that were completed as planned.The current Consolidated Standards of Reporting Trials(CONSORT) statement [17] explicitly recommends report-ing the number of patients assessed for eligibility, number ofeligible patients, and number of consenting patients. In thecontext of RCT discontinuation due to poor recruitment,however, investigators should additionally describe how theyprojected the number of eligible and consenting patients;whether a pilot study including informed consent was done;whether recruitment was closely monitored; which measureswere undertaken to improve recruitment; and the specificroot causes for recruitment failure in their case, so that fu-ture recruitment failures in that area of research can effi-ciently be prevented.Given its magnitude and global presence of the problem,

the evidence base on what actually works to improve re-cruitment in RCTs is still astonishingly thin [18, 19]. Basedon the numerous analyses about the nature, extent, andcauses for recruitment failure, it is time for internationalcollaborative efforts to overcome the problem. Specifically,we need more randomized ‘studies within a trial’ (SWATs),i.e., promising interventions to improve recruitment needto be empirically evaluated within a host trial as propagatedby the Trial Forge initiative, the Medical Research Council’sNetwork of Hubs for Trials Methodology Research in theUK, and the Health Research Board’s Trials MethodologyResearch Network in Ireland [20, 21].

ConclusionsThis qualitative comparison of RCTs discontinued or re-vised due to poor recruitment and RCTs completed asplanned on the same research question complementsprevious efforts to identify risk factors for recruitmentfailure in RCTs, and to better understand the underlyingmechanisms. Our study confirms previously identifiedcauses such as narrow eligibility criteria, investigator-sponsorship, and a high burden of trial procedures forpatients and recruiters, but also stresses the importanceof considering the accumulating evidence and the timingof other RCTs on the same topic as well as carefullyselecting and closely monitoring participating centers forRCTs in challenging settings. A more detailed reportingof patient recruitment in RCTs is urgently needed sothat recruitment failure can provide lessons for other re-searchers in the future.

Supplementary informationSupplementary information accompanies this paper at https://doi.org/10.1186/s13063-019-3957-4.

Additional file 1. Standards for Reporting Qualitative Research (SRQR)*

Additional file 2. Pre-specified checklist of items potentially associatedwith poor recruitment

Additional file 3. References of included randomized controlled trials(RCTs)

Additional file 4. Recruitment characteristics of included randomizedcontrolled trials by research question

Additional file 5. Recruitment characteristics of included randomizedcontrolled trials (RCTs)

Additional file 6. General characteristics of included randomizedcontrolled trials (RCTs)

Additional file 7. Reporting quality of the recruitment process inincluded randomized controlled trials

AbbreviationsCONSORT: Consolidated Standards of Reporting Trials; RCT: Randomizedcontrolled trial; SRQR: Standards for Reporting Qualitative Research;SWAT: Study within a trial

AcknowledgmentsWe thank PD Dr. Dr. Benjamin Kasenda (oncologist at the University HospitalBasel) for providing expert knowledge about chemotherapy regimens inincluded RCTs with cancer patients.

Authors’ contributionsMB and EvE conceived of the study, wrote the protocol, and acquiredfunding. VG and BS conducted the literature searches and extracted datafrom relevant articles. MB, BS, and VG qualitatively analyzed the data. MB andVG wrote the first draft of the manuscript. BS and EvE critically revised themanuscript. All authors approved the final version before submission.

FundingThis study was funded by the Swiss National Science Foundation (grant no.320030_149496/1). The funder had no role in the study design, datacollection and analysis, decision to publish, or preparation of this manuscript.

Availability of data and materialsThe data used for and/or analyzed during the current study is available inAdditional files 1, 2, 3, 4, 5, 6, and 7.

Ethics approval and consent to participateNot applicable.

Consent for publicationNot applicable.

Competing interestsThe authors declare that they have no competing interests.

Author details1Basel Institute for Clinical Epidemiology and Biostatistics, Department ofClinical Research, University of Basel and University Hospital Basel, Basel,Switzerland. 2Department of Health Research Methods, Evidence, and Impact,McMaster University, Hamilton, ON, Canada. 3Cochrane Switzerland, Centerfor Primary Care and Public Health (Unisanté), University of Lausanne,Lausanne, Switzerland.

Received: 22 December 2018 Accepted: 4 December 2019

References1. Kasenda B, von Elm E, You J, Blumle A, Tomonaga Y, Saccilotto R, Amstutz

A, Bengough T, Meerpohl JJ, Stegert M, et al. Prevalence, characteristics, andpublication of discontinued randomized trials. JAMA. 2014;311(10):1045–51.

2. Bernardez-Pereira S, Lopes RD, Carrion MJ, Santucci EV, Soares RM, deOliveira AM, Laranjeira LN, Ikeoka DT, Zazula AD, Moreira FR, et al.Prevalence, characteristics, and predictors of early termination ofcardiovascular clinical trials due to low recruitment: insights from theClinicalTrials.gov registry. Am Heart J. 2014;168(2):213–9 e211.

3. Briel M, Olu KK, von Elm E, Kasenda B, Alturki R, Agarwal A, Bhatnagar N,Schandelmaier S. A systematic review of discontinued trials suggested that

Briel et al. Trials (2019) 20:800 Page 11 of 12

most reasons for recruitment failure were preventable. J Clin Epidemiol.2016;80:8–15.

4. Briel M, Elger B, von Elm E, Satalkar P. Insufficient recruitment andpremature discontinuation of clinical trials in Switzerland: qualitative studywith trialists and other stakeholders. Swiss Med Wkly. 2017;147:w14556.

5. Prescott RJ, Counsell CE, Gillespie WJ, Grant AM, Russell IT, Kiauka S, ColthartIR, Ross S, Shepherd SM, Russell D. Factors that limit the quality, numberand progress of randomised controlled trials. Health Technol Assess. 1999;3(20):1–143.

6. Anvari M, Allen C, Marshall J, Armstrong D, Goeree R, Ungar W, GoldsmithC. A randomized controlled trial of laparoscopic Nissen fundoplicationversus proton pump inhibitors for the treatment of patients with chronicgastroesophageal reflux disease (GERD): 3-year outcomes. Surg Endosc.2011;25(8):2547–54.

7. Grant AM, Wileman SM, Ramsay CR, Mowat NA, Krukowski ZH, Heading RC,Thursz MR, Campbell MK, Group RT. Minimal access surgery compared withmedical management for chronic gastro-oesophageal reflux disease: UKcollaborative randomised trial. BMJ. 2008;337:a2664.

8. Connolly SJ, Dorian P, Roberts RS, Gent M, Bailin S, Fain ES, Thorpe K,Champagne J, Talajic M, Coutu B, et al. Comparison of beta-blockers,amiodarone plus beta-blockers, or sotalol for prevention of shocks fromimplantable cardioverter defibrillators: the OPTIC Study: a randomized trial.JAMA. 2006;295(2):165–71.

9. Sandercock P, Wardlaw JM, Lindley RI, Dennis M, Cohen G, Murray G, InnesK, Venables G, Czlonkowska A, Kobayashi A, et al. The benefits and harms ofintravenous thrombolysis with recombinant tissue plasminogen activatorwithin 6 h of acute ischaemic stroke (the third international stroke trial [IST-3]): a randomised controlled trial. Lancet. 2012;379(9834):2352–63.

10. Schandelmaier S, von Elm E, You JJ, Blumle A, Tomonaga Y, Lamontagne F,Saccilotto R, Amstutz A, Bengough T, Meerpohl JJ, et al. Prematurediscontinuation of randomized trials in critical and emergency care: aretrospective cohort study. Crit Care Med. 2016;44(1):130–7.

11. Habre C, Tramer MR, Popping DM, Elia N. Ability of a meta-analysis toprevent redundant research: systematic review of studies on pain frompropofol injection. BMJ. 2014;348:g5219.

12. Spaar A, Frey M, Turk A, Karrer W, Puhan MA. Recruitment barriers in arandomized controlled trial from the physicians' perspective: a postalsurvey. BMC Med Res Methodol. 2009;9:14.

13. Mills E, Wilson K, Rachlis B, Griffith L, Wu P, Guyatt G, Cooper C. Barriers toparticipation in HIV drug trials: a systematic review. Lancet Infect Dis. 2006;6(1):32–8.

14. Kaur G, Smyth RL, Williamson P. Developing a survey of barriers andfacilitators to recruitment in randomized controlled trials. Trials. 2012;13:218.

15. Garde A, O'Hearn K, Nicholls S, Menon K. Reporting of consent rates incritical care studies: room for improvement. J Clin Epidemiol. 2016;74:51–6.

16. Toerien M, Brookes ST, Metcalfe C, de Salis I, Tomlin Z, Peters TJ, Sterne J,Donovan JL. A review of reporting of participant recruitment and retentionin RCTs in six major journals. Trials. 2009;10:52.

17. Schulz KF, Altman DG, Moher D, Group C. CONSORT 2010 statement:updated guidelines for reporting parallel group randomised trials. BMJ.2010;340:c332.

18. Treweek S, Pitkethly M, Cook J, Fraser C, Mitchell E, Sullivan F, Jackson C,Taskila TK, Gardner H. Strategies to improve recruitment to randomisedtrials. Cochrane Database Syst Rev. 2018;2:MR000013.

19. Gardner HR, Fraser C, MacLennan G, Treweek S. A protocol for a systematicreview of non-randomised evaluations of strategies to improve participantrecruitment to randomised controlled trials. Syst Rev. 2016;5(1):131.

20. Treweek S, Altman DG, Bower P, Campbell M, Chalmers I, Cotton S, Craig P,Crosby D, Davidson P, Devane D, et al. Making randomised trials moreefficient: report of the first meeting to discuss the Trial Forge platform.Trials. 2015;16:261.

21. Treweek S, Bevan S, Bower P, Campbell M, Christie J, Clarke M, Collett C,Cotton S, Devane D, El Feky A, et al. Trial Forge Guidance 1: what is a StudyWithin A Trial (SWAT)? Trials. 2018;19(1):139.

Publisher’s NoteSpringer Nature remains neutral with regard to jurisdictional claims inpublished maps and institutional affiliations.

Briel et al. Trials (2019) 20:800 Page 12 of 12