The James Lind Alliance Tackling treatment uncertainties together
Open access Research The James Lind Alliance process ...
18
1 Nygaard A, et al. BMJ Open 2019;9:e027473. doi:10.1136/bmjopen-2018-027473 Open access The James Lind Alliance process approach: scoping review Agnete Nygaard, 1,2 Liv Halvorsrud, 1 Siv Linnerud, 2 Ellen Karine Grov, 1 Astrid Bergland 1 To cite: Nygaard A, Halvorsrud L, Linnerud S, et al. The James Lind Alliance process approach: scoping review. BMJ Open 2019;9:e027473. doi:10.1136/ bmjopen-2018-027473 ► Prepublication history for this paper is available online. To view these files, please visit the journal online (http://dx.doi. org/10.1136/bmjopen-2018- 027473). Received 05 November 2018 Revised 11 July 2019 Accepted 16 July 2019 1 Faculty of Health Sciences, Oslo Metropolitan University, Oslo, Norway 2 Center for Development of Institutional and Home Care, Lørenskog, Akershus Correspondence to Agnete Nygaard; [email protected] Research © Author(s) (or their employer(s)) 2019. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ. Strengths and limitations of this study ► This is the first scoping review of published studies using the James Lind Alliance (JLA) approach avail- able with involvement of patients, carers and the public in the setting the research agenda. ► The weakest voices often lack representation, which could limit the generalisability of these priorities to these populations. ► Because a scoping review approach was used, the quality of the articles was not assessed prior to inclusion. ► We were not in contact with the JLA Coordinating Centre and search in all relevant literature, such as grey literature and studies, which do not described all steps of the JLA process, might have limited our results. ► A limitation of this scoping review was our inclusion of only English-language articles. ABSTRACT Objective To summarise study descriptions of the James Lind Alliance (JLA) approach to the priority setting partnership (PSP) process and how this process is used to identify uncertainties and to develop lists of top 10 priorities. Design Scoping review. Data sources The Embase, Medline (Ovid), PubMed, CINAHL and the Cochrane Library as of October 2018. Study selection All studies reporting the use of JLA process steps and the development of a list of top 10 priorities, with adult participants aged 18 years. Data extraction A data extraction sheet was created to collect demographic details, study aims, sample and patient group details, PSP details (eg, stakeholders), lists of top 10 priorities, descriptions of JLA facilitator roles and the PSP stages followed. Individual and comparative appraisals were discussed among the scoping review authors until agreement was reached. Results Database searches yielded 431 potentially relevant studies published in 2010–2018, of which 37 met the inclusion criteria. JLA process participants were patients, carers and clinicians, aged 18 years, who had experience with the study-relevant diagnoses. All studies reported having a steering group, although partners and stakeholders were described differently across studies. The number of JLA PSP process steps varied from four to eight. Uncertainties were typically collected via an online survey hosted on, or linked to, the PSP website. The number of submitted uncertainties varied across studies, from 323 submitted by 58 participants to 8227 submitted by 2587 participants. Conclusions JLA-based PSP makes a useful contribution to identifying research questions. Through this process, patients, carers and clinicians work together to identify and prioritise unanswered uncertainties. However, representation of those with different health conditions depends on their having the capacity and resources to participate. No studies reported difficulties in developing their top 10 priorities. INTRODUCTION Over the past decade, patient and public involvement (PPI) has been highlighted worldwide in both health research agendas and the development of next-step research projects. 1 PPI has been defined as ‘experi- menting with’ as opposed to ‘experimenting on’ patients or the public. 2 PPI allows patients to actively contribute, through discussion, to decision-making regarding research design, acceptability, relevance, conduct and gover- nance from study conception to dissemina- tion. 3 However, PPI may also involve active data collection, analysis and dissemination. 4 Researchers have noted that involving healthcare service users, the public and patients improves research quality, relevance, implementation and cost-effectiveness; it also improves researchers’ understanding of and insight into the medical and social conditions they are studying, 1 5 although such evidence is still relatively limited. 4 The James Lind Alliance (JLA) is a UK-based non-profit initiative that was estab- lished in 2004. The JLA process is focused on bringing patients, carers and clinicians together, on an equal basis, in a priority setting partnership (PSP) to define and prior- itise uncertainties relating to a specific condi- tion. 6 Hall et al 7 note that the JLA aims to raise awareness among research funding groups about what matters most to both patients and clinicians, in order to ensure that clinical research is both relevant and beneficial to end users. According to the JLA Guidebook, 6 on January 31, 2022 by guest. Protected by copyright. http://bmjopen.bmj.com/ BMJ Open: first published as 10.1136/bmjopen-2018-027473 on 30 August 2019. Downloaded from
Transcript of Open access Research The James Lind Alliance process ...
1Nygaard A, et al. BMJ Open 2019;9:e027473. doi:10.1136/bmjopen-2018-027473
Open access
The James Lind Alliance process approach: scoping review
Agnete Nygaard,1,2 Liv Halvorsrud,1 Siv Linnerud,2 Ellen Karine Grov,1 Astrid Bergland1
To cite: Nygaard A, Halvorsrud L, Linnerud S, et al. The James Lind Alliance process approach: scoping review. BMJ Open 2019;9:e027473. doi:10.1136/bmjopen-2018-027473
► Prepublication history for this paper is available online. To view these files, please visit the journal online (http:// dx. doi. org/ 10. 1136/ bmjopen- 2018- 027473).
Received 05 November 2018Revised 11 July 2019Accepted 16 July 2019
1Faculty of Health Sciences, Oslo Metropolitan University, Oslo, Norway2Center for Development of Institutional and Home Care, Lørenskog, Akershus
Correspondence toAgnete Nygaard; s98209@ oslomet. no
Research
© Author(s) (or their employer(s)) 2019. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.
Strengths and limitations of this study
► This is the first scoping review of published studies using the James Lind Alliance (JLA) approach avail-able with involvement of patients, carers and the public in the setting the research agenda.
► The weakest voices often lack representation, which could limit the generalisability of these priorities to these populations.
► Because a scoping review approach was used, the quality of the articles was not assessed prior to inclusion.
► We were not in contact with the JLA Coordinating Centre and search in all relevant literature, such as grey literature and studies, which do not described all steps of the JLA process, might have limited our results.
► A limitation of this scoping review was our inclusion of only English-language articles.
AbStrACtObjective To summarise study descriptions of the James Lind Alliance (JLA) approach to the priority setting partnership (PSP) process and how this process is used to identify uncertainties and to develop lists of top 10 priorities.Design Scoping review.Data sources The Embase, Medline (Ovid), PubMed, CINAHL and the Cochrane Library as of October 2018.Study selection All studies reporting the use of JLA process steps and the development of a list of top 10 priorities, with adult participants aged 18 years.Data extraction A data extraction sheet was created to collect demographic details, study aims, sample and patient group details, PSP details (eg, stakeholders), lists of top 10 priorities, descriptions of JLA facilitator roles and the PSP stages followed. Individual and comparative appraisals were discussed among the scoping review authors until agreement was reached.results Database searches yielded 431 potentially relevant studies published in 2010–2018, of which 37 met the inclusion criteria. JLA process participants were patients, carers and clinicians, aged 18 years, who had experience with the study-relevant diagnoses. All studies reported having a steering group, although partners and stakeholders were described differently across studies. The number of JLA PSP process steps varied from four to eight. Uncertainties were typically collected via an online survey hosted on, or linked to, the PSP website. The number of submitted uncertainties varied across studies, from 323 submitted by 58 participants to 8227 submitted by 2587 participants.Conclusions JLA-based PSP makes a useful contribution to identifying research questions. Through this process, patients, carers and clinicians work together to identify and prioritise unanswered uncertainties. However, representation of those with different health conditions depends on their having the capacity and resources to participate. No studies reported difficulties in developing their top 10 priorities.
IntrODuCtIOnOver the past decade, patient and public involvement (PPI) has been highlighted worldwide in both health research agendas and the development of next-step research projects.1 PPI has been defined as ‘experi-menting with’ as opposed to ‘experimenting on’ patients or the public.2 PPI allows patients
to actively contribute, through discussion, to decision-making regarding research design, acceptability, relevance, conduct and gover-nance from study conception to dissemina-tion.3 However, PPI may also involve active data collection, analysis and dissemination.4
Researchers have noted that involving healthcare service users, the public and patients improves research quality, relevance, implementation and cost-effectiveness; it also improves researchers’ understanding of and insight into the medical and social conditions they are studying,1 5 although such evidence is still relatively limited.4
The James Lind Alliance (JLA) is a UK-based non-profit initiative that was estab-lished in 2004. The JLA process is focused on bringing patients, carers and clinicians together, on an equal basis, in a priority setting partnership (PSP) to define and prior-itise uncertainties relating to a specific condi-tion.6 Hall et al7 note that the JLA aims to raise awareness among research funding groups about what matters most to both patients and clinicians, in order to ensure that clinical research is both relevant and beneficial to end users. According to the JLA Guidebook,6
on January 31, 2022 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2018-027473 on 30 August 2019. D
ownloaded from
2 Nygaard A, et al. BMJ Open 2019;9:e027473. doi:10.1136/bmjopen-2018-027473
Open access
Figure 1 Flow diagram. JLA, James Lind Alliance; PSP, priority setting partnership; RCT, randomised controlled trial.
Table 1 Criteria for inclusion and exclusion
Inclusion criteria Exclusion criteria
► All steps from JLA ► List of top 10 priorities ► Adults (aged >18 years or older)
► Unpublished literature ► Articles not written in English
► PSP without JLA ► JLA without PSP ► Protocols ► Errata ► Editorial ► Thesis ► Comments ► Review ► Guidelines ► Randomised controlled trials
JLA, James Lind Alliance; PSP, priority setting partnership.
uncertainties and how to prioritise these are key features of the JLA process. The process begins by defining unan-swered questions (ie, ‘uncertainties’) about the effects of treatment and healthcare—questions that cannot be adequately answered based on existing research evidence, such as reliable, up-to-date systematic reviews—and then prioritises the uncertainties based on their importance. The most recent version of the JLA Guidebook explains that many PSPs interpret the definition of treatment uncertainties broadly. They may interpret ‘treatments’ to include interventions such as care, support and diagnosis. This approach has been an important development and one that helps the JLA adapt to the changing health and care landscapes, as well as to the changing needs of its users.6
The JLA provides facilitation and guidance in the identification and prioritisation processes. This process forms part of a widening approach to PPI in research. The characteristics of the PSP process are (1) setting up a steering group to supervise all aspects of the study; (2) establishing a PSP; (3) assembling potential research questions; (4) processing, categorising, and summarising those research questions; and (5) determining the top 10 research priorities through an interim process and a final priority setting workshop using respondent ranking and consensus discussion. To ensure that all voices in the work-shop are heard, the JLA supports an adapted nominal group technique (NGT) for PSPs when choosing their priorities. NGT is a well-established and well-documented approach to decision-making.6
To our knowledge, there is a gap in existing research given that no review has yet been published describing how the JLA approach is used to establish steering groups, set up PSPs, gather uncertainties, summarise uncertain-ties and determine the lists of top 10 priorities. Thus, the objective of this scoping review was to summarise study descriptions of the JLA approach to the PSP process, and how this process is used to identify uncertainties and develop lists of top 10 priorities.
► How do the studies describe the characteristics of the PSPs and, elaborating on aspects, how have they oper-ationalised the JLA methods?
► How do the studies describe involvement of different user groups?
► What processes are used to gather and verify uncertainties?
MethODSIdentifying relevant studiesA systematic search was conducted up to October 2018 using five databases: Embase, Medline (Ovid), PubMed, CINAHL and the Cochrane Library. The search strategy in each database was «james lind*» OR «priorit* setting part-nership*». We also searched in JLA website. This search identified 746 records and 431 potentially relevant cita-tions. After removing duplicates and screening titles and abstracts based on our inclusion and exclusion criteria, the full text of 171 studies was examined in greater detail. A total of 37 studies met all criteria for review and were subsequently investigated. These numbers were verified by a university librarian (see flowchart, figure 1).
Selecting relevant studiesA prescreening process included reviewing the search results and excluding all articles that were not research studies, that were unavailable in full text or that clearly did not involve the JLA PSP approach. At least two authors screened the remaining articles using the inclusion and exclusion criteria presented in table 1.
on January 31, 2022 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2018-027473 on 30 August 2019. D
ownloaded from
3Nygaard A, et al. BMJ Open 2019;9:e027473. doi:10.1136/bmjopen-2018-027473
Open access
Charting dataA data extraction sheet was created to collect studies’ demographic details, aims, samples and patient groups. The sheet was used to collect methodological details about the studies’ PSPs, including descriptions of stake-holders, lists of top 10 priorities, descriptions of the roles of JLA facilitators and PSP stages.
ProcedureIn addition to the first author, one of the other authors evaluated each article, and individual and comparative appraisals were discussed among the authors until agree-ment was reached. At least two authors were involved in each of the study selection procedures. A predefined procedure was developed for consulting a third author, or the whole research team, in cases of discrepancies; however, this was never necessary (ie, decisions to accept or reject unclear articles were based on a dyad consensus). The first author and one other author extracted the char-acteristics and findings of each study.
Quality appraisalThe most recent JLA Guidebook6 served as the context for investigating the descriptions of the studies’ methods. A quality assessment was not included in the remit of this scoping review.8
Patient and Public InvolvementNo patient was involved.
Collating, summarising and reporting resultsFindings related to the scoping review’s research ques-tions, based on the JLA approach, were extracted and documented. The information shown in table 2 includes the studies’ aims, suggested uncertainties and—depending on the version of the JLA guidelines used—how these uncertainties were determined. We also collected information on the stakeholders (including members of the PSP), whether a JLA advisor/facilitator was used, and the JLA process stages: (1) setting up a PSP, (2) gathering uncertainties, (3) data processing and veri-fying uncertainties, (4) interim priority setting and (5) final priority setting. The results are presented based on the JLA Guidebook steps, which have remained consistent across versions.6 9–11
reSultSIn total, 37 studies met the inclusion criteria; their charac-teristics are summarised in table 2.
The publication years ranged from 2010 to 2018. The number of studies using this process has increased annu-ally, with 12 published in 2017. In our sample, 27 of the studies were from the UK,1 5 7 12–35 8 were from Canada,36–43 and 1 each was from India44 and Spain.45
The JLA process participants were patients, carers and clinicians aged ≥18 years. The studies collectively repre-sented patient groups with heterogeneous ages and health conditions/diseases, with later studies generally
more focused on symptoms and function than on diseases (table 2). Totally, 15 of the studies gave information about ethnicity.13 14 16 19 21 23 25–27 32 33 35 36 40 42 One of the studies also gave information about socioeconomic status.26 Another study gave only information about socioeco-nomic status.44
Three of studies described that patient and carers submitted more questions on psychosocial issues, psycho-social stress, depression and anxiety compered with clinicians.13 23 40 No studies described disagreement in the prioritisation stages. However, 24 other studies also mentioned psychosocial issues without noting who had done so.1 7 14–19 25–27 29 31–39 41–43 Ten studies did not mention psychosocial issues.5 12 20–22 24 26 28 44 45 The types of health conditions that were addressed included gastrointestinal,26–28 neurological,1 5 7 16 21 38 dermato-logical,13 15 17 29 34 45 endocrinal14 32 42 and cancer22 25 41 43 conditions.
Setting up a PSPThe JLA steering group is made up of key organisations and individuals who can collectively represent all or the majority of issues related to the PSP, either individually or through their networks.6
All included studies had a steering group, although they were described differently. Nineteen studies1 5 12 14–17 19 20 22 23 25 31 36 37 39–41 45 included patients, carers and clinicians in their steering groups; 16 studies7 13 18 24 26–29 32–35 38 42–44 did not include carers in their steering group (ie, only patients and clinicians). In one study,30 the titles of the members on the steering group were not reported; in another,21 the steering group did not specifically include patients, carers or clinicians, but rather stated that representation from all stakeholders was ensured.
The number of JLA steps in the PSP process varied across studies from four steps1 32 33 37 39 42 to eight steps.20 22 44 Five steps, corresponding to JLA Guidebook V.4, V.5 and V.6, were most common,12 13 15–19 23 24 26–29 31 34 36 38 40 41 43 45 with step 1, initiation; step 2, collection of uncertainties; step 3, collation of uncertainties; step 4, interim priority setting; and step 5, final priority workshop.
Gathering uncertaintiesPSPs aimed to gather uncertainties from as wide a range of potential contributors as possible, ensuring that patients were equally confident and empowered compared with clinicians in submitting their perspectives on uncertainties.6
With regard to recruitment, various partner organisa-tions, local advertisements, social media, patients, carers and clinicians were PSP information targets. In addition to an online and paper survey, two studies also used face-to-face methods to reach and facilitate involvement by their identified groups.5 42
The questions were usually deliberately open-ended to encourage full responses regarding the experi-ences of patients, carers and clinicians. One of the 37
on January 31, 2022 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2018-027473 on 30 August 2019. D
ownloaded from
4 Nygaard A, et al. BMJ Open 2019;9:e027473. doi:10.1136/bmjopen-2018-027473
Open access
Tab
le 2
C
hara
cter
istic
s of
incl
uded
stu
die
s
Year
Aut
hor
Co
untr
yA
im o
f th
e st
udy
1. U
ser
gro
up*
2. JLA
Guideb
ook,
yea
r an
d
vers
ion
3. A
ge
of
pat
ient
†4.
Hea
lth
cond
itio
n/d
isea
se5.
Num
ber
of
init
ial u
ncer
tain
ties
an
d p
arti
cip
ants
or
retu
rned
su
rvey
s o
r up
load
ed r
esea
rch
pri
ori
ties
Ste
erin
g g
roup
‡ id
enti
fica
tio
n an
d m
anag
emen
t o
f p
artn
ers/
stak
eho
lder
s
JLA
The
ro
le o
f th
e fa
cilit
ato
r/ad
viso
r
PS
PN
umb
er o
f st
eps
Des
crip
tio
n o
f st
ages
NG
T
2010
Buc
kley
et
al12 U
K
To id
entif
y an
d p
riorit
ise
‘clin
ical
un
cert
aint
ies’
rel
atin
g to
tre
atm
ent
of U
I.
1.
NR
.2.
A
ge ≥
40 y
ears
.3.
U
I.4.
In
tot
al, 4
94, ‘
raw
’ tre
atm
ent
unce
rtai
ntie
s.5.
P
atie
nts,
car
ers,
clin
icia
ns.
Org
anis
atio
ns w
ere
iden
tified
, whi
ch r
epre
sent
ed o
r co
uld
ad
voca
te fo
r p
atie
nts,
the
ir in
form
al c
arer
s an
d
clin
icia
ns in
volv
ed in
the
tre
atm
ent
or m
anag
emen
t.
NR
Five
ste
ps+
NG
T1.
In
itiat
ion.
2.
Con
sulta
tion.
3.
Col
latio
n.4.
P
riorit
isat
ion.
5.
Dis
sem
inat
ion.
2011
Ele
fthe
riad
ou
et a
l13
UK
To s
timul
ate
and
ste
er fu
ture
re
sear
ch in
the
fiel
d o
f viti
ligo
trea
tmen
t b
y id
entif
ying
the
10
mos
t im
por
tant
res
earc
h ar
eas
for
pat
ient
s an
d c
linic
ians
.
1.
Pat
ient
s, c
arer
s, c
linic
ians
and
re
sear
cher
s.2.
JL
A G
uid
eboo
k 20
10, V
.4.
3.
NR
.4.
V
itilig
o.5.
In
tot
al, 6
60 t
reat
men
t un
cert
aint
ies
wer
e su
bm
itted
b
y 46
1 p
artic
ipan
ts.
Pro
fess
iona
l bod
ies
and
pat
ient
sup
por
t gr
oup
s;
stee
ring
grou
p in
clud
ed 1
2 m
emb
ers
with
kn
owle
dge
and
inte
rest
in v
itilig
o.
The
Viti
ligo
PS
P a
dop
ted
th
e m
etho
ds
advo
cate
d b
y th
e JL
A, w
hich
wer
e re
fined
to
mee
t th
e ne
eds
of t
his
par
ticul
ar P
SP.
Five
ste
ps
1. In
itiat
ion.
2. C
onsu
ltatio
n.3.
Col
latio
n.4.
Ran
king
exe
rcis
e (In
terim
p
riorit
isat
ion
exer
cise
).5.
Fin
al P
riorit
isat
ion
Wor
ksho
p.
2012
Gad
sby
et a
l14
UK
To c
olle
ct u
ncer
tain
ties
abou
t th
e tr
eatm
ent
of t
ype
1 d
iab
etes
fr
om p
atie
nts,
car
ers
and
hea
lth
pro
fess
iona
ls, a
nd t
o co
llate
and
p
riorit
ise
thes
e un
cert
aint
ies
to
dev
elop
a li
st o
f top
10
of r
esea
rch
prio
ritie
s.
1.
Pat
ient
s, c
arer
s an
d c
linic
ians
.2.
JL
A G
uid
eboo
k 20
10, V
.4.
3.
NR
.4.
Ty
pe
1 d
iab
etes
.5.
In
tot
al, 1
141
trea
tmen
t un
cert
aint
ies
wer
e su
bm
itted
b
y 58
3 p
artic
ipan
ts.
Mem
ber
s w
ith p
ersp
ectiv
es in
pae
dia
tric
s an
d
prim
ary
care
, use
rs o
f typ
e 1
dia
bet
es s
ervi
ces,
in
clud
ing
pat
ient
s an
d c
arer
s;a
stee
ring
grou
p o
f rep
rese
ntat
ives
from
the
se
orga
nisa
tions
(n=
9 p
lus
an in
dep
end
ent
info
rmat
ion
spec
ialis
t) an
d p
artn
er o
rgan
isat
ions
.
JLA
, bei
ng r
epre
sent
ed o
n th
e st
eerin
g gr
oup
Six
ste
ps
1.
Set
ting
up t
he p
artn
ersh
ip/
surv
ey.
2.
Col
lect
ing
unce
rtai
ntie
s.3.
C
olla
tion
activ
ity.
4.
Inte
rim p
riorit
y se
ttin
g.5.
Fi
nal p
riorit
y se
ttin
g w
orks
hop
.6.
R
evie
w.
2013
Bat
chel
or e
t al
15
UK
To id
entif
y th
e un
cert
aint
ies
in e
czem
a tr
eatm
ent
that
are
im
por
tant
to
pat
ient
s w
ho h
ave
ecze
ma,
the
ir ca
rers
and
the
he
alth
care
pro
fess
iona
ls w
ho t
reat
th
em.
1.
Pat
ient
s, c
arer
s an
d c
linic
ians
.2.
JL
A G
uid
eboo
k 20
10, V
.4.
3.
NR
.4.
E
czem
a.5.
In
tot
al, 1
070
trea
tmen
t un
cert
aint
ies
wer
e su
bm
itted
b
y 49
3 p
artic
ipan
ts.
The
stee
ring
grou
p c
omp
rised
four
pat
ient
s an
d
care
rs, i
nclu
din
g a
rep
rese
ntat
ive
from
the
Nat
iona
l E
czem
a S
ocie
ty, f
our
clin
icia
ns, t
wo
der
mat
olog
ists
, a
der
mat
olog
y nu
rse
spec
ialis
t an
d a
GP
and
th
ree
rese
arch
ers⁄
adm
inis
trat
ors
at t
he C
entr
e of
E
vid
ence
-Bas
ed D
erm
atol
ogy.
The
PS
P w
as
coor
din
ated
from
the
C
entr
e of
Evi
den
ce-
Bas
ed D
erm
atol
ogy
in
Not
tingh
am, w
ith o
vers
ight
b
y a
rep
rese
ntat
ive
of J
LA,
who
was
the
ind
epen
den
t ch
air
of t
he P
SP
ste
erin
g gr
oup
.
Five
ste
ps
1. In
itiat
ion.
2. C
onsu
ltatio
n—co
llect
ion
of
trea
tmen
t un
cert
aint
ies.
3. C
olla
tion
and
tre
atm
ent
unce
rtai
ntie
s.4.
Ran
king
of t
reat
men
t un
cert
aint
ies.
5. W
orks
hop
to
dev
elop
re
sear
ch q
uest
ions
. Con
tinue
d
on January 31, 2022 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2018-027473 on 30 August 2019. D
ownloaded from
5Nygaard A, et al. BMJ Open 2019;9:e027473. doi:10.1136/bmjopen-2018-027473
Open access
Year
Aut
hor
Co
untr
yA
im o
f th
e st
udy
1. U
ser
gro
up*
2. JLA
Guideb
ook,
yea
r an
d
vers
ion
3. A
ge
of
pat
ient
†4.
Hea
lth
cond
itio
n/d
isea
se5.
Num
ber
of
init
ial u
ncer
tain
ties
an
d p
arti
cip
ants
or
retu
rned
su
rvey
s o
r up
load
ed r
esea
rch
pri
ori
ties
Ste
erin
g g
roup
‡ id
enti
fica
tio
n an
d m
anag
emen
t o
f p
artn
ers/
stak
eho
lder
s
JLA
The
ro
le o
f th
e fa
cilit
ato
r/ad
viso
r
PS
PN
umb
er o
f st
eps
Des
crip
tio
n o
f st
ages
NG
T
2013
Dav
ila-S
eijo
et
al45
Sp
ain
To d
escr
ibe
and
prio
ritis
e th
e m
ost
imp
orta
nt u
ncer
tain
ties
abou
t d
ystr
ophi
c ep
ider
mol
ysis
bul
losa
tr
eatm
ent
shar
ed b
y p
atie
nts,
ca
rers
and
hea
lthca
re p
rofe
ssio
nals
in
ord
er t
o p
rom
ote
rese
arch
in
thos
e ar
eas.
1.
Pat
ient
s, c
arer
s an
d c
linic
ians
.2.
JL
A G
uid
eboo
k 20
10, V
.4.
3.
Age
21–
54 y
ears
.4.
D
ystr
ophi
c ep
ider
mol
ysis
b
ullo
sa.
5.
In t
otal
, 323
tre
atm
ent
unce
rtai
ntie
s w
ere
sub
mitt
ed
by
58 p
artic
ipan
ts.
The
stee
ring
grou
p c
omp
rised
eig
ht p
eop
le,
incl
udin
g p
atie
nts/
care
rs, a
rep
rese
ntat
ive
from
th
e D
ystr
ophi
c E
pid
erm
olys
is B
ullo
sa R
esea
rch
Ass
ocia
tion
Sp
ain,
a c
linic
ian;
der
mat
olog
ists
, nu
rses
and
res
earc
hers
; and
the
Sp
anis
h A
cad
emy
of D
erm
atol
ogy
and
Ven
ereo
logy
.
Wor
ksho
p a
dvo
cate
d b
y th
e JL
AFi
ve s
tep
s+N
GT
1.
Initi
atio
n.2.
C
onsu
ltatio
n su
rvey
: co
llect
ion
of t
reat
men
t un
cert
aint
ies.
3.
Ran
king
exe
rcis
e.4.
R
anki
ng e
xerc
ise.
5.
Fina
l prio
ritis
atio
n w
orks
hop
.
2013
Hal
l et
al7
UK
To d
escr
ibe
the
tinni
tus
PS
P in
pro
vid
ing
a p
latf
orm
fo
r p
atie
nts
and
clin
icia
ns t
o co
llab
orat
e, t
o id
entif
y an
d
to p
riorit
ise
unce
rtai
ntie
s or
‘u
nans
wer
ed q
uest
ions
’.
1.
Pat
ient
s an
d c
linic
ians
.2.
JL
A G
uid
eboo
k 20
10, V
.4.
3.
NR
.4.
Ti
nnitu
s.5.
In
tot
al, 2
483
trea
tmen
t un
cert
aint
ies
wer
e su
bm
itted
b
y 82
5 p
artic
ipan
ts.
Mem
ber
ship
of t
he s
teer
ing
grou
p p
rovi
ded
a
bro
ad r
epre
sent
atio
n of
peo
ple
from
the
fiel
d o
f tin
nitu
s, in
clud
ing
pro
fess
iona
l bod
ies,
cha
ritie
s an
d
advo
cato
rs fo
r p
eop
le w
ith t
inni
tus.
The
wid
er w
orki
ng p
artn
ersh
ip in
clud
ed 5
6 m
ajor
U
K s
take
hold
ers,
incl
udin
g in
div
idua
l ad
voca
tors
fo
r p
eop
le w
ith t
inni
tus,
sup
por
t gr
oup
s, h
osp
ital
cent
res
and
com
mer
cial
org
anis
atio
ns.
Ind
epen
den
t ch
airp
erso
n re
pre
sent
ing
the
JLA
Sev
en s
tep
s1.
E
stab
lishi
ng a
wor
king
p
artn
ersh
ip.
2.
Gat
herin
g su
gges
tions
fo
r re
sear
ch o
n th
e as
sess
men
t, d
iagn
osis
an
d t
reat
men
t of
tin
nitu
s.3.
C
heck
ing
and
cat
egor
isin
g su
bm
itted
unc
erta
intie
s.4.
P
riorit
isin
g th
e un
cert
aint
ies.
5.
Dev
elop
ing
a co
nsen
sus.
6.
Top
10
clin
ical
res
earc
h q
uest
ions
.7.
R
ecom
men
dat
ions
for
futu
re r
esea
rch
stra
tegy
.
2014
Dea
ne e
t al
16
UK
To id
entif
y an
d p
riorit
ise
the
top
10
evid
entia
l unc
erta
intie
s th
at im
pac
t on
eve
ryd
ay c
linic
al p
ract
ice
for
the
man
agem
ent
of P
arki
nson
’s
dis
ease
.
1.
Pat
ient
s, c
arer
s, fa
mily
, frie
nds,
cl
inic
ians
.2.
JL
A G
uid
eboo
k 20
13, V
.5.
3.
NR
.4.
P
arki
nson
’s d
isea
se.
5.
In t
otal
, 410
0 tr
eatm
ent
unce
rtai
ntie
s w
ere
sub
mitt
ed
by
1000
par
ticip
ants
.
The
stee
ring
grou
p c
onsi
sted
of r
epre
sent
ativ
es
from
Par
kins
on’s
UK
(n=
8) a
nd t
he C
ure
Par
kins
on’s
Tr
ust
(n=
1), p
atie
nts
(n=
2), c
arer
s (n
=2)
, clin
ical
co
nsul
tant
s (n
=2)
and
a P
arki
nson
’s d
isea
se
nurs
e sp
ecia
list
(n=
1). T
hose
from
Par
kins
on’s
U
K in
clud
ed r
epre
sent
ativ
es w
ith e
xper
tise
in
rese
arch
dev
elop
men
t, p
olic
y an
d c
amp
aign
s (n
=5)
, in
form
atio
n an
d s
upp
ort
wor
ker
serv
ices
(n=
1),
advi
sory
ser
vice
s (n
=1)
and
res
ourc
es a
nd d
iver
sity
(n
=1)
.
The
JLA
pro
vid
ed a
n in
dep
end
ent
chai
r, ad
vise
d
on t
he m
etho
dol
ogy,
and
fa
cilit
ated
the
pro
cess
.
Five
ste
ps+
NG
T1.
In
itiat
ion.
2.
Con
sulta
tion.
3.
Unc
erta
intie
s su
rvey
.4.
C
olla
tion.
5.
Prio
rizat
ion.
Tab
le 2
C
ontin
ued
Con
tinue
d
on January 31, 2022 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2018-027473 on 30 August 2019. D
ownloaded from
6 Nygaard A, et al. BMJ Open 2019;9:e027473. doi:10.1136/bmjopen-2018-027473
Open access
Year
Aut
hor
Co
untr
yA
im o
f th
e st
udy
1. U
ser
gro
up*
2. JLA
Guideb
ook,
yea
r an
d
vers
ion
3. A
ge
of
pat
ient
†4.
Hea
lth
cond
itio
n/d
isea
se5.
Num
ber
of
init
ial u
ncer
tain
ties
an
d p
arti
cip
ants
or
retu
rned
su
rvey
s o
r up
load
ed r
esea
rch
pri
ori
ties
Ste
erin
g g
roup
‡ id
enti
fica
tio
n an
d m
anag
emen
t o
f p
artn
ers/
stak
eho
lder
s
JLA
The
ro
le o
f th
e fa
cilit
ato
r/ad
viso
r
PS
PN
umb
er o
f st
eps
Des
crip
tio
n o
f st
ages
NG
T
2014
Ingr
am e
t al
17
UK
To g
ener
ate
a to
p 1
0 lis
t of
hi
dra
den
itis
sup
pur
ativ
a re
sear
ch
prio
ritie
s, fr
om t
he p
ersp
ectiv
es
of p
atie
nts
with
hid
rad
eniti
s su
pp
urat
iva,
car
ers
and
clin
icia
ns,
to t
ake
to fu
ndin
g b
odie
s.
1.
Pat
ient
s, c
arer
s an
d c
linic
ians
.2.
JL
A G
uid
eboo
k 20
13, V
.5.
3.
NR
.4.
H
idra
den
itis
sup
pur
ativ
a.5.
In
tot
al, 1
495
trea
tmen
t un
cert
aint
ies
wer
e su
bm
itted
b
y 37
1 p
artic
ipan
ts.
The
stee
ring
com
mitt
ee in
clud
ed fi
ve p
atie
nts
and
car
ers,
incl
udin
g tw
o re
pre
sent
ativ
es o
f th
e H
idra
den
itis
Sup
pur
ativ
a Tr
ust
UK
pat
ient
or
gani
satio
n; s
ix d
erm
atol
ogis
ts, i
nclu
din
g tw
o tr
aine
es, t
wo
der
mat
olog
y sp
ecia
list
nurs
es,
a p
last
ic s
urge
on, a
gen
eral
pra
ctiti
oner
; the
JL
A r
epre
sent
ativ
e an
d a
n ad
min
istr
ator
and
st
akeh
old
ers
from
var
ious
Roy
al C
olle
ge-r
elat
ed
grou
ps.
Thre
e JL
A fa
cilit
ator
s or
fo
ur fa
cilit
ator
sFi
ve s
tep
s+N
TG1.
Id
entif
y st
akeh
old
ers.
2.
Invi
tatio
n to
sub
mit
unce
rtai
ntie
s.3.
G
ener
ate
‘ind
icat
ive
unce
rtai
ntie
s’.
4.
Ran
k un
cert
aint
ies.
5.
Fina
l wor
ksho
p.
2014
Man
ns e
t al
36
Can
ada
To im
pro
ve u
nder
stan
din
g of
ki
dne
y fu
nctio
n an
d d
isea
se,
incl
udin
g th
ose
for
spec
ific
area
s,
such
as
dia
lysi
s th
erap
ies.
1.
Pat
ient
s, c
arer
s an
d c
linic
ians
.2.
JL
A G
uid
eboo
k 20
13, V
.5.
3.
Age
18
to >
80 y
ears
.4.
P
atie
nts
on o
r ne
ar d
ialy
sis.
5.
In t
otal
, 182
0 tr
eatm
ent
unce
rtai
ntie
s w
ere
sub
mitt
ed
by
317
resp
ond
ents
.
The
prio
rity
sett
ing
pro
cess
was
initi
ated
with
th
e fo
rmat
ion
of a
n 11
-per
son
stee
ring
grou
p,
whi
ch in
clud
ed p
atie
nts,
a c
areg
iver
, clin
icia
ns, a
n em
plo
yee
of t
he K
idne
y Fo
und
atio
n of
Can
ada
and
an
exp
ert
in t
heJL
A a
pp
roac
h.
Exp
erie
nced
faci
litat
ors
Five
ste
ps+
NG
T1.
S
urve
y.2.
C
olla
tion.
3.
Com
bin
ing.
4.
Inte
rim p
riorit
isat
ion.
5.
Fina
l wor
ksho
p.
2014
Pol
lock
et
al5
UK
To id
entif
y th
e to
p 1
0 re
sear
ch
prio
ritie
s re
latin
g to
life
aft
er s
trok
e,
as a
gree
d b
y st
roke
sur
vivo
rs,
care
rs a
nd c
linic
ians
.
1.
Pat
ient
s, c
arer
s, c
linic
ians
2.
JLA
Gui
deb
ook
2010
, V.4
.3.
N
R.
4.
Life
aft
er s
trok
e.5.
In
tot
al, 5
48 t
reat
men
t un
cert
aint
ies.
A s
teer
ing
grou
p c
omp
risin
g a
stro
ke s
urvi
vor,
care
rs, a
nur
se, a
phy
sici
an, a
llied
clin
icia
ns, a
re
sear
cher
and
rep
rese
ntat
ives
from
key
nat
iona
l st
roke
cha
ritie
s/p
atie
nt o
rgan
isat
ions
and
from
the
JL
A; t
he S
cott
ish
Gov
ernm
ent’s
Nat
iona
l Ad
viso
ry
Com
mitt
ee fo
r S
trok
e. T
his
pro
ject
was
com
ple
ted
in
par
tner
ship
with
Che
st H
eart
& S
trok
e S
cotla
nd a
nd
the
Str
oke
Ass
ocia
tion
in S
cotla
nd.
The
faci
litat
ors
wer
e b
riefe
d
by
mem
ber
s of
the
JLA
on
the
imp
orta
nce
of e
nsur
ing
equi
tab
le p
artic
ipat
ion
of a
ll gr
oup
mem
ber
s
Six
ste
ps+
NG
T1.
Fo
rm P
SP.
2.
Gat
her
trea
tmen
t un
cert
aint
ies.
3.
Che
ck t
reat
men
t un
cert
aint
ies.
4.
Inte
rim p
riorit
isat
ion.
5.
Fina
l prio
rity
sett
ing.
6.
Rep
ortin
g an
d
dis
sem
inat
ion.
2014
Row
e et
al18
UK
To Id
entif
y re
sear
ch p
riorit
ies
rela
ting
to s
ight
loss
and
vis
ion
thro
ugh
cons
ulta
tion
with
pat
ient
s,
care
rs a
nd c
linic
ians
.
1.
Pat
ient
s, c
arer
s an
d c
linic
ians
.2.
JL
A G
uid
eboo
k 20
13, V
.5.
3.
Ave
rage
age
of
par
ticip
ants
=65
.7 y
ears
.4.
S
ight
loss
or
an e
ye c
ond
ition
.5.
In
tot
al, 4
461
trea
tmen
t un
cert
aint
ies
wer
e su
bm
itted
b
y 22
20 p
artic
ipan
ts.
The
stee
ring
com
mitt
ee in
clud
ed p
atie
nt
rep
rese
ntat
ives
and
eye
hea
lth p
rofe
ssio
nals
.A
ste
erin
g co
mm
ittee
and
dat
a as
sess
men
t gr
oup
co
mp
risin
g th
e au
thor
s of
thi
s ar
ticle
ove
rsaw
th
e p
roce
ss a
nd s
take
hold
ers
from
var
ious
Roy
al
Col
lege
-rel
ated
gro
ups.
The
Ste
erin
g C
omm
ittee
al
so in
clud
ed p
atie
nt r
epre
sent
ativ
es a
nd e
ye h
ealth
p
rofe
ssio
nals
.
Rep
rese
ntat
ive
from
the
JL
A c
onve
ned
mee
tings
of
the
stee
ring
com
mitt
ee
Five
ste
ps+
NG
T1.
E
stab
lishi
ng t
he S
ight
Lo
ss V
isio
n P
SP.
2.
Sur
vey.
3.
Dat
a as
sess
men
t.4.
In
terim
prio
ritis
atio
n.5.
Fi
nal p
riorit
isat
ion.
Tab
le 2
C
ontin
ued
Con
tinue
d
on January 31, 2022 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2018-027473 on 30 August 2019. D
ownloaded from
7Nygaard A, et al. BMJ Open 2019;9:e027473. doi:10.1136/bmjopen-2018-027473
Open access
Year
Aut
hor
Co
untr
yA
im o
f th
e st
udy
1. U
ser
gro
up*
2. JLA
Guideb
ook,
yea
r an
d
vers
ion
3. A
ge
of
pat
ient
†4.
Hea
lth
cond
itio
n/d
isea
se5.
Num
ber
of
init
ial u
ncer
tain
ties
an
d p
arti
cip
ants
or
retu
rned
su
rvey
s o
r up
load
ed r
esea
rch
pri
ori
ties
Ste
erin
g g
roup
‡ id
enti
fica
tio
n an
d m
anag
emen
t o
f p
artn
ers/
stak
eho
lder
s
JLA
The
ro
le o
f th
e fa
cilit
ato
r/ad
viso
r
PS
PN
umb
er o
f st
eps
Des
crip
tio
n o
f st
ages
NG
T
2014
Uhm
et
al19
UK
To d
isco
ver
the
rese
arch
que
stio
ns
for
pre
term
birt
h an
d t
o gr
ade
them
ac
cord
ing
to t
heir
imp
orta
nce
for
infa
nts
and
fam
ilies
.
1.
Pat
ient
s, c
arer
s an
d c
linic
ians
.2.
N
R.
3.
NR
.4.
P
rete
rm b
irth.
5.
In t
otal
, 593
res
earc
h q
uest
ions
w
ere
sub
mitt
ed b
y 38
6 p
eop
le.
Pot
entia
l par
tner
s w
ere
iden
tified
thr
ough
a p
roce
ss
of p
eer
know
led
ge a
nd c
onsu
ltatio
n, s
teer
ing
grou
p
mem
ber
s’ n
etw
orks
and
JLA
’s e
xist
ing
regi
ster
of
affil
iate
s. S
take
hold
ers
from
var
ious
Roy
al C
olle
ge-
rela
ted
gro
ups.
Two
faci
litat
ors
from
the
JL
AFi
ve s
tep
s+N
GT
1.
Initi
atio
n of
the
p
artn
ersh
ip.
2.
Iden
tifyi
ng t
reat
men
t un
cert
aint
ies.
3.
Col
latio
n: r
efini
ng
que
stio
ns a
nd
unce
rtai
ntie
s.4.
P
riorit
isat
ion—
inte
rim a
nd
final
sta
ges.
5.
Pub
licity
and
pub
lishi
ng
resu
lts.
2015
Bar
nieh
et
al37
Can
ada
To a
sses
s th
e re
sear
ch p
riorit
ies
of p
atie
nts
on o
r ne
arin
g d
ialy
sis
with
in C
anad
a an
d t
heir
care
rs a
nd
clin
icia
ns.
1.
Pat
ient
s ca
rers
and
clin
icia
ns.
2.
JLA
Gui
deb
ook
2013
, V.5
.3.
N
R.
4.
On
or n
earin
g d
ialy
sis.
5.
In t
otal
, 182
0 tr
eatm
ent
unce
rtai
ntie
s an
d n
umb
er o
f p
artic
ipan
ts w
ere
not
rep
orte
d.
The
11-p
erso
n st
eerin
g gr
oup
com
pris
ed fo
ur
pat
ient
s, o
ne c
arer
, thr
ee c
linic
ians
, an
emp
loye
e of
the
Kid
ney
Foun
dat
ion
of C
anad
a (a
n im
por
tant
fu
nder
of k
idne
y re
sear
ch in
Can
ada)
, an
exp
ert
in
the
JLA
ap
pro
ach
and
a r
esea
rche
r. Th
e st
eerin
g gr
oup
incl
uded
ind
ivid
uals
from
acr
oss
Can
ada
and
d
iffer
ent
stak
ehol
der
s.
Faci
litat
ors
with
exp
erie
nce
in t
he J
LA m
etho
ds
lead
th
e w
orks
hop
Four
ste
ps+
NG
T1.
For
m P
SP.
2. G
athe
r re
sear
ch
unce
rtai
ntie
s.3.
Pro
cess
and
col
late
su
bm
itted
res
earc
h un
cert
aint
ies.
4. F
inal
prio
rity
sett
ing
wor
ksho
p.
2015
Bon
ey e
t al
20
UK
To id
entif
y re
sear
ch p
riorit
ies
for
anae
sthe
sia
and
per
iop
erat
ive
med
icin
e.
1.
Pat
ient
s, c
arer
s an
d c
linic
ians
.2.
JL
A G
uid
eboo
k 20
13, V
.5.
3.
NR
.4.
A
naes
thes
ia a
nd p
erio
per
ativ
e m
edic
ine.
5.
In t
otal
, 142
0 tr
eatm
ent
unce
rtai
ntie
s w
ere
sub
mitt
ed
by
623
par
ticip
ants
.
The
stee
ring
grou
p c
omp
rised
rep
rese
ntat
ives
of t
he
fund
ing
par
tner
org
anis
atio
ns, p
atie
nts
and
car
ers,
an
d t
he J
LA.
Alm
ost
2000
sta
keho
lder
s co
ntrib
uted
the
ir vi
ews
rega
rdin
g an
aest
hetic
and
per
iop
erat
ive
rese
arch
p
riorit
ies.
Sta
keho
lder
s w
ere
defi
ned
as
‘any
per
son
or o
rgan
isat
ion
with
an
inte
rest
in a
naes
thes
ia a
nd
per
iop
erat
ive
care
’.
Ste
erin
g gr
oup
cha
ired
by
the
JLA
ad
vise
rE
ight
ste
ps
1.
Enr
ol p
artn
er
orga
nisa
tions
.2.
Id
entif
y re
sear
ch
que
stio
ns.
3.
Cla
ssify
and
refi
ne
rese
arch
que
stio
n.4.
S
hort
-lis
ting.
5.
Lite
ratu
re r
evie
w.
6.
Inte
rim p
riorit
isat
ion.
7.
Fina
l prio
ritis
atio
n.8.
P
ublic
atio
n an
d
dis
sem
inat
ion
of r
esul
ts.
Tab
le 2
C
ontin
ued
Con
tinue
d
on January 31, 2022 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2018-027473 on 30 August 2019. D
ownloaded from
8 Nygaard A, et al. BMJ Open 2019;9:e027473. doi:10.1136/bmjopen-2018-027473
Open access
Year
Aut
hor
Co
untr
yA
im o
f th
e st
udy
1. U
ser
gro
up*
2. JLA
Guideb
ook,
yea
r an
d
vers
ion
3. A
ge
of
pat
ient
†4.
Hea
lth
cond
itio
n/d
isea
se5.
Num
ber
of
init
ial u
ncer
tain
ties
an
d p
arti
cip
ants
or
retu
rned
su
rvey
s o
r up
load
ed r
esea
rch
pri
ori
ties
Ste
erin
g g
roup
‡ id
enti
fica
tio
n an
d m
anag
emen
t o
f p
artn
ers/
stak
eho
lder
s
JLA
The
ro
le o
f th
e fa
cilit
ato
r/ad
viso
r
PS
PN
umb
er o
f st
eps
Des
crip
tio
n o
f st
ages
NG
T
2015
Kel
ly e
t al
21
UK
To id
entif
y un
answ
ered
que
stio
ns
arou
nd t
he p
reve
ntio
n, t
reat
men
t,
dia
gnos
is a
nd c
are
of d
emen
tia,
with
the
invo
lvem
ent
of a
ll st
akeh
old
ers;
to
iden
tify
a to
p 1
0 p
riorit
ised
list
of u
ncer
tain
ties.
1.
Pat
ient
s, c
arer
s/re
lativ
es, a
nd
clin
icia
ns.
2.
JLA
Gui
deb
ook
2013
, V.5
.3.
N
R.
4.
Dem
entia
.5.
In
tot
al, 1
563
uplo
aded
sur
veys
.
Pot
entia
l par
tner
org
anis
atio
ns w
ere
iden
tified
th
roug
h th
e ne
twor
ks o
f the
Alz
heim
er’s
Soc
iety
and
th
e st
eerin
g gr
oup
, ens
urin
g re
pre
sent
atio
n fr
om a
ll st
akeh
old
ers.
Pat
ient
s, c
arer
s an
d c
linic
ians
wer
e no
t in
volv
ed in
the
ste
erin
g gr
oup
.
The
Dem
entia
PS
P w
as
guid
ed a
nd c
haire
d b
y an
ind
epen
den
t JL
A
rep
rese
ntat
ive.
Six
ste
ps+
NG
T1.
In
volv
emen
t of
pot
entia
l p
artn
er o
rgan
isat
ions
.2.
Id
entif
ying
unc
erta
intie
s.3.
Q
uest
ion
man
agem
ent
and
ana
lysi
s.4.
Ve
rifyi
ng u
ncer
tain
ties.
5.
Inte
rim p
riorit
isat
ion.
6.
Fina
l prio
ritis
atio
n w
orks
hop
.
2015
Ste
phe
ns e
t al
22
UK
To id
entif
y th
e to
p 1
0 re
sear
ch
prio
ritie
s re
latin
g to
mes
othe
liom
a (p
leur
al o
r p
erito
neal
), sp
ecifi
cally
, to
iden
tify
thos
e un
answ
ered
q
uest
ions
tha
t in
volv
ed a
n in
terv
entio
n.
1.
Pat
ient
s, c
urre
nt a
nd b
erea
ved
ca
rers
, and
clin
icia
ns.
2.
JLA
Gui
deb
ook
2013
, V.5
.3.
N
R.
4.
Mes
othe
liom
a.5.
In
tot
al, 4
53 in
itial
sur
veys
.
Ste
erin
g gr
oup
com
pris
ed t
wo
pat
ient
s, o
ne
ber
eave
d c
arer
, nin
e cl
inic
ians
(inc
lud
ing
nurs
es,
surg
eons
, onc
olog
ists
, che
st p
hysi
cian
s an
d
pal
liativ
e ca
re e
xper
ts) a
nd fo
ur r
epre
sent
ativ
es
of p
atie
nt a
nd fa
mily
sup
por
t gr
oup
s (o
ne o
f the
re
pre
sent
ativ
es w
as a
lso
a b
erea
ved
car
er);
in t
otal
, 16
par
ticip
ants
.
The
stee
ring
grou
p w
as
chai
red
by
a JL
A fa
cilit
ator
.E
ight
ste
ps
1.
Est
ablis
hing
a s
teer
ing
grou
p.
2.
Initi
al s
urve
y q
uest
ionn
aire
.3.
R
evie
win
g th
e su
rvey
re
spon
ses.
4.
Sea
rchi
ng.
5.
Inte
rim p
riorit
isat
ion.
6.
Fina
l prio
rity
sett
ing.
7.
Iden
tified
una
nsw
ered
q
uest
ions
.8.
A
n ad
diti
onal
PS
P.
2016
Kni
ght
et a
l23
UK
To id
entif
y un
answ
ered
res
earc
h q
uest
ions
in t
he fi
eld
of k
idne
y tr
ansp
lant
atio
n fr
om e
nd-s
ervi
ce
user
s (p
atie
nts,
car
ers
and
he
alth
care
pro
fess
iona
ls).
1.
Pat
ient
s, c
arer
s an
d c
linic
ians
.2.
JL
A G
uid
eboo
k 20
13, V
.5.
3.
NR
.4.
K
idne
y tr
ansp
lant
atio
n.5.
In
tot
al, 4
97 t
reat
men
t un
cert
aint
ies
wer
e su
bm
itted
b
y 18
3 p
artic
ipan
ts.
The
stee
ring
grou
p in
clud
ed t
rans
pla
nt s
urge
ons,
ne
phr
olog
ists
, tra
nsp
lant
rec
ipie
nts,
livi
ng d
onor
s an
d c
arer
s. A
dd
ition
al p
artn
er o
rgan
isat
ions
wer
e in
vite
d t
o ta
ke p
art
in t
he p
roce
ss b
y in
volv
ing
thei
r m
emb
ers
in t
he s
urve
ys a
nd h
elp
ing
to p
rom
ote
the
pro
cess
.N
atio
nal p
atie
nt a
nd p
rofe
ssio
nal o
rgan
isat
ions
and
ch
ariti
es in
volv
ed in
kid
ney
tran
spla
ntat
ion
wer
e co
ntac
ted
ab
out
the
pro
ject
and
wer
e in
vite
d t
o co
ntrib
ute
to a
ste
erin
g gr
oup
.
The
stee
ring
grou
p w
as
chai
red
by
an e
xper
ienc
ed
advi
sor
from
the
JLA
.
Five
ste
ps+
NG
T1.
O
rgan
isat
ion
and
sco
pe.
2.
Iden
tifica
tion
of p
oten
tial
rese
arch
que
stio
ns.
3.
Refi
nem
ent
of q
uest
ions
an
d id
entifi
catio
n of
ex
istin
g lit
erat
ure.
4.
Inte
rim p
riorit
isat
ion.
5.
Fina
l prio
ritis
atio
n w
orks
hop
.
Tab
le 2
C
ontin
ued
Con
tinue
d
on January 31, 2022 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2018-027473 on 30 August 2019. D
ownloaded from
9Nygaard A, et al. BMJ Open 2019;9:e027473. doi:10.1136/bmjopen-2018-027473
Open access
Year
Aut
hor
Co
untr
yA
im o
f th
e st
udy
1. U
ser
gro
up*
2. JLA
Guideb
ook,
yea
r an
d
vers
ion
3. A
ge
of
pat
ient
†4.
Hea
lth
cond
itio
n/d
isea
se5.
Num
ber
of
init
ial u
ncer
tain
ties
an
d p
arti
cip
ants
or
retu
rned
su
rvey
s o
r up
load
ed r
esea
rch
pri
ori
ties
Ste
erin
g g
roup
‡ id
enti
fica
tio
n an
d m
anag
emen
t o
f p
artn
ers/
stak
eho
lder
s
JLA
The
ro
le o
f th
e fa
cilit
ato
r/ad
viso
r
PS
PN
umb
er o
f st
eps
Des
crip
tio
n o
f st
ages
NG
T
2016
Ran
gan
et a
l24
UK
To r
un a
UK
-bas
ed J
LA P
SP
for
‘sur
gery
for
com
mon
sho
uld
er
pro
ble
ms’
1.
Pat
ient
s, c
arer
s an
d c
linic
ians
.2.
JL
A G
uid
eboo
k 20
13, V
.5.
3.
NR
.4.
S
houl
der
sur
gery
5.
In t
otal
, 652
tre
atm
ent
unce
rtai
ntie
s w
ere
sub
mitt
ed
by
371
par
ticip
ants
.
The
stee
ring
grou
p w
as m
ade
up o
f the
mos
t re
leva
nt s
take
hold
ers
and
incl
uded
pat
ient
s,
phy
siot
hera
pis
ts, G
Ps,
sho
uld
er s
urge
ons,
an
aest
hetis
ts a
nd p
ain
cont
rol e
xper
ts, o
rtho
pae
dic
nu
rses
and
aca
dem
ic c
linic
ians
; nat
iona
l net
wor
ks
and
inte
rest
org
anis
atio
ns
A J
LA a
dvi
ser
Five
ste
ps
1.
Iden
tifica
tion
and
invi
tatio
n of
pot
entia
l par
tner
s.2.
In
itial
mee
ting/
aw
aren
ess
rais
ing.
3.
Iden
tifyi
ng t
reat
men
t un
cert
aint
ies.
4.
Refi
ning
que
stio
ns a
nd
unce
rtai
ntie
s.5.
P
riorit
isat
ion
inte
rim
and
fina
l prio
ritis
atio
n w
orks
hop
.
2016
van
Mid
den
dor
p
et a
l1
UK
To id
entif
y a
list
of T
op 1
0 p
riorit
ies
for
futu
re r
esea
rch
into
sp
inal
cor
d
inju
ry.
1.
Pat
ient
, sp
ouse
/par
tner
and
cl
inic
ians
.2.
JL
A G
uid
eboo
k 20
13, V
.5.
3.
Age
s 18
–80
year
s.4.
S
pin
al c
ord
inju
ry.
5.
In, t
otal
, 784
tre
atm
ent
unce
rtai
ntie
s w
ere
sub
mitt
ed
by
403
par
ticip
ants
.
The
stee
ring
grou
p c
omp
rised
rep
rese
ntat
ives
fr
om e
ach
stak
ehol
der
org
anis
atio
n, in
clud
ing
an
ind
epen
den
t in
form
atio
n m
anag
er. S
take
hold
ers
incl
uded
con
sum
er o
rgan
isat
ions
, clin
icia
n so
ciet
ies
and
car
ers
rep
rese
ntat
ives
.
Sup
por
t an
d g
uid
ance
wer
e p
rovi
ded
by
the
JLA
Four
ste
ps
1. G
athe
ring
of r
esea
rch
que
stio
ns.
2. C
heck
ing
of e
xist
ing
rese
arch
evi
den
ce.
3. In
terim
prio
ritis
atio
n.4.
Fin
al c
onse
nsus
mee
ting.
2016
,W
an e
t al
25
UK
To e
stab
lish
a co
nsen
sus
rega
rdin
g th
e to
p 1
0 un
answ
ered
res
earc
h q
uest
ions
in e
ndom
etria
l can
cer.
1.
Pat
ient
s, c
arer
s an
d c
linic
ians
.2.
JL
A G
uid
eboo
k 20
13, V
.5.
3.
NR
.4.
E
ndom
etria
l can
cer.
5.
In t
otal
, 786
ind
ivid
ual
sub
mis
sion
s fr
om 4
13
par
ticip
ants
.
As
par
t of
the
JLA
pro
cess
, all
orga
nisa
tions
tha
t co
uld
rea
ch a
nd a
dvo
cate
for
pat
ient
s, c
arer
s an
d
clin
icia
ns w
ere
invi
ted
to
bec
ome
invo
lved
in a
PS
P.
A s
teer
ing
grou
p c
omp
osed
of r
epre
sent
ativ
es fr
om
thes
e gr
oup
s w
as t
hen
form
ed t
o en
sure
the
stu
dy
rem
aine
d in
clus
ive
and
fulfi
lled
its
aim
to
del
iver
and
p
ublic
ise
a lis
t of
sha
red
res
earc
h p
riorit
ies.
A g
roup
of 2
3 st
akeh
old
ers
was
con
stitu
ted
but
was
no
t d
escr
ibed
in d
etai
l.
An
ind
epen
den
t ad
viso
r fr
om t
he J
LA w
as c
hair
of
the
stee
ring
grou
p
Six
ste
ps+
NG
T1.
Est
ablis
hing
a s
teer
ing
grou
p.
2. C
onsu
ltativ
e p
roce
ss. 3
. G
athe
ring
unce
rtai
ntie
s.4.
Dat
a an
alys
is a
nd v
erify
ing
unce
rtai
ntie
s.5.
Inte
rim p
riorit
y se
ttin
g. 6
. Fi
nal p
riorit
y se
ttin
g.
2017
,B
ritto
n et
al26
UK
Faci
litat
e b
alan
ced
inp
ut in
th
e p
riorit
y se
ttin
g p
roce
ss fo
r B
arre
tt’s
oes
opha
gus
and
gas
tro-
oeso
pha
geal
refl
ux d
isea
se a
nd t
o re
ach
a co
nsen
sus
on t
he T
op 1
0 un
cert
aint
ies
in t
he fi
eld
1.
Pat
ient
s, c
arer
s an
d c
linic
ians
.2.
JL
A G
uid
eboo
k 20
13, V
.5.
3.
NR
.4.
G
astr
o-oe
sop
hage
al r
eflux
d
isea
se a
nd B
arre
tt’s
oe
sop
hagu
s.5.
In
tot
al, 6
29 t
reat
men
t un
cert
aint
ies
wer
e su
bm
itted
b
y 17
0 p
artic
ipan
ts.
Pro
fess
iona
ls, p
atie
nts
and
cha
rity
rep
rese
ntat
ives
fo
rmed
a s
teer
ing
com
mitt
ee. T
he s
teer
ing
com
mitt
ee, w
hich
iden
tified
the
bro
ader
prio
ritie
s,
The
Brit
ish
Soc
iety
of G
astr
oent
erol
ogy,
Nat
iona
l H
ealth
Ser
vice
, the
Uni
vers
ity o
f Man
ches
ter,
the
Ass
ocia
tion
of U
pp
er G
astr
oint
estin
al S
urge
ons
and
th
e P
rimar
y S
ocie
ty fo
r G
astr
oent
erol
ogy.
NR
.Fi
ve s
tep
s+N
GT
1. In
itial
sur
vey.
2. In
itial
res
pon
se li
st.
3. L
ong-
list
gene
ratio
n an
d
verifi
catio
n.4.
Inte
rim p
riorit
isat
ion
surv
ey.
5. F
inal
wor
ksho
p.
Tab
le 2
C
ontin
ued
Con
tinue
d
on January 31, 2022 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2018-027473 on 30 August 2019. D
ownloaded from
10 Nygaard A, et al. BMJ Open 2019;9:e027473. doi:10.1136/bmjopen-2018-027473
Open access
Year
Aut
hor
Co
untr
yA
im o
f th
e st
udy
1. U
ser
gro
up*
2. JLA
Guideb
ook,
yea
r an
d
vers
ion
3. A
ge
of
pat
ient
†4.
Hea
lth
cond
itio
n/d
isea
se5.
Num
ber
of
init
ial u
ncer
tain
ties
an
d p
arti
cip
ants
or
retu
rned
su
rvey
s o
r up
load
ed r
esea
rch
pri
ori
ties
Ste
erin
g g
roup
‡ id
enti
fica
tio
n an
d m
anag
emen
t o
f p
artn
ers/
stak
eho
lder
s
JLA
The
ro
le o
f th
e fa
cilit
ato
r/ad
viso
r
PS
PN
umb
er o
f st
eps
Des
crip
tio
n o
f st
ages
NG
T
2017
,Fi
tzch
arle
s et
al
38
Can
ada
Prio
ritie
s of
unc
erta
intie
s fo
r th
e m
anag
emen
t of
fib
rom
yalg
ia (F
M)
that
cou
ld p
rop
el fu
ture
res
earc
h.
1.
Pat
ient
s, c
arer
s an
d c
linic
ians
.2.
JL
A G
uid
eboo
k 20
13, V
.5.
3.
Age
s 18
to
>70
yea
rs.
4.
Fib
rom
yalg
ia.
5.
In t
otal
, 455
7 tr
eatm
ent
unce
rtai
ntie
s w
ere
sub
mitt
ed
by
550
par
ticip
ants
.
The
stee
ring
com
mitt
ee w
as c
omp
osed
of fi
ve
pat
ient
s (o
ne p
atie
nt w
as a
pra
ctis
ing
pha
rmac
ist),
fiv
e he
alth
care
pro
fess
iona
ls (o
ne fa
mily
phy
sici
an,
two
rheu
mat
olog
ists
, one
psy
chol
ogis
t an
d o
ne
inte
rnis
t), a
n in
tern
ist
with
pre
viou
s ex
per
ienc
e of
the
JL
A p
roce
ss b
ut w
ithou
t sp
ecifi
c in
tere
st in
FM
, and
a
rheu
mat
olog
ist.
Faci
litat
ors
with
exp
erie
nce
of t
he J
LA p
roce
ssFi
ve s
tep
s1.
S
urve
y re
sults
.2.
In
sco
pe
unce
rtai
ntie
s.3.
C
odin
g un
cert
aint
ies.
4.
Inte
rim p
riorit
isat
ion.
5.
Fina
l wor
ksho
p.
2017
,H
art
et a
l27
UK
To d
evis
e a
list
of t
he k
ey r
esea
rch
prio
ritie
s re
gard
ing
trea
tmen
t of
in
flam
mat
ory
bow
el d
isea
se, a
s se
en b
y cl
inic
ians
, pat
ient
s an
d
thei
r su
pp
ort
grou
ps,
usi
ng a
st
ruct
ure
esta
blis
hed
by
the
JLA
.
1.
Pat
ient
s, c
arer
s an
d c
linic
ians
.2.
JL
A G
uid
eboo
k 20
13, V
.5.
3.
NR
.4.
In
flam
mat
ory
bow
el d
isea
se.
5.
In t
otal
, 163
6 tr
eatm
ent
unce
rtai
ntie
s w
ere
sub
mitt
ed
by
531
par
ticip
ants
.
A s
teer
ing
com
mitt
ee w
as e
stab
lishe
d fo
llow
ing
an in
itial
exp
lana
tory
mee
ting
and
incl
uded
tw
o p
atie
nts,
tw
o ga
stro
ente
rolo
gist
s, t
wo
infla
mm
ator
y b
owel
dis
ease
sp
ecia
list
nurs
es, t
wo
colo
rect
al
surg
eons
, tw
o d
ietit
ians
, a r
epre
sent
ativ
e fr
om t
he
UK
infla
mm
ator
y b
owel
dis
ease
cha
rity
orga
nisa
tion
Cro
hn’s
and
Col
itis
UK
, a r
epre
sent
ativ
e of
the
JLA
an
d a
n ad
min
istr
ator
.
A J
LA fa
cilit
ator
Five
ste
ps
1. In
itiat
ion
and
set
ting
up t
he
com
mitt
ee.
2. C
olle
ctio
n of
tre
atm
ent
unce
rtai
ntie
s.3.
Col
latio
n of
tre
atm
ent
unce
rtai
ntie
s.4.
Ran
king
of t
reat
men
t un
cert
aint
ies.
5. D
evel
opm
ent
of a
list
of t
op
10 p
riorit
ies.
2017
,H
emm
elga
rn
et a
l39
Can
ada
To id
entif
y th
e m
ost
imp
orta
nt
unan
swer
ed q
uest
ions
(or
unce
rtai
ntie
s) a
bou
t th
e m
anag
emen
t of
CK
D, t
hat
is, i
n te
rms
of d
iagn
osis
, pro
gnos
is a
nd
trea
tmen
t.
1.
Pat
ient
s, c
arer
s, c
linic
ians
and
p
olic
ymak
ers.
2.
JLA
Gui
deb
ook
2013
, V.5
.3.
A
ge ≥
65 y
ears
.4.
N
on-d
ialy
sis
CK
D.
5.
In t
otal
, 224
1 tr
eatm
ent
unce
rtai
ntie
s w
ere
sub
mitt
ed
by
439
par
ticip
ants
.
The
prio
rity
sett
ing
pro
cess
with
the
form
atio
n of
a
12-p
erso
n st
eerin
g gr
oup
from
acr
oss
Can
ada,
in
clud
ing
pat
ient
s w
ith n
on-d
ialy
sis
CK
D, a
car
er,
clin
icia
ns (n
ephr
olog
ists
), re
sear
cher
s an
d a
n em
plo
yee
of t
he K
idne
y Fo
und
atio
n of
Can
ada
(non
-p
rofit
org
anis
atio
n fo
r p
atie
nts
with
kid
ney
dis
ease
).
Join
tly o
rgan
ised
PS
P
bro
adly
ad
herin
g to
the
JLA
G
uid
eboo
k
Four
ste
ps+
NG
T1.
Id
entifi
catio
n an
d in
vita
tion
of p
oten
tial p
artn
ers.
2.
Col
lect
ion
of r
esea
rch
unce
rtai
ntie
s th
roug
h a
natio
nal s
urve
y.3.
R
efine
men
t an
d
prio
ritis
atio
n.4.
P
riorit
y se
ttin
g w
orks
hop
.
2017
,K
han
et a
l40
Can
ada
To id
entif
y th
e 10
mos
t im
por
tant
re
sear
ch p
riorit
ies
of p
atie
nts,
ca
rers
and
clin
icia
ns fo
r hy
per
tens
ion
man
agem
ent.
1.
Pat
ient
s, c
arer
s an
d c
linic
ians
.2.
JL
A G
uid
eboo
k 20
13, V
.5.
3.
NR
.4.
H
yper
tens
ion.
5.
In t
otal
, 673
ind
ivid
ual r
esea
rch
que
stio
ns w
ere
sub
mitt
ed b
y 38
6 p
artic
ipan
ts.
Ste
erin
g co
mm
ittee
of 1
5 vo
lunt
eer
pat
ient
s, c
arer
s an
d c
linic
ians
from
acr
oss
Can
ada.
Sta
keho
lder
not
rep
orte
d in
det
ail.
JLA
faci
litat
or fr
om t
he U
KFi
ve s
tep
s1.
E
stab
lishi
ng a
ste
erin
g gr
oup
.2.
Fo
rmin
g P
SP
s.3.
C
olle
ctin
g p
oten
tial
rese
arch
que
stio
ns.
4.
Pro
cess
ing,
cat
egor
isin
g,
and
sum
mar
isin
g th
ose
rese
arch
que
stio
ns.
5.
Sel
ectin
g th
e to
p 1
0 r e
sear
ch p
riorit
ies.
Tab
le 2
C
ontin
ued
Con
tinue
d
on January 31, 2022 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2018-027473 on 30 August 2019. D
ownloaded from
11Nygaard A, et al. BMJ Open 2019;9:e027473. doi:10.1136/bmjopen-2018-027473
Open access
Year
Aut
hor
Co
untr
yA
im o
f th
e st
udy
1. U
ser
gro
up*
2. JLA
Guideb
ook,
yea
r an
d
vers
ion
3. A
ge
of
pat
ient
†4.
Hea
lth
cond
itio
n/d
isea
se5.
Num
ber
of
init
ial u
ncer
tain
ties
an
d p
arti
cip
ants
or
retu
rned
su
rvey
s o
r up
load
ed r
esea
rch
pri
ori
ties
Ste
erin
g g
roup
‡ id
enti
fica
tio
n an
d m
anag
emen
t o
f p
artn
ers/
stak
eho
lder
s
JLA
The
ro
le o
f th
e fa
cilit
ato
r/ad
viso
r
PS
PN
umb
er o
f st
eps
Des
crip
tio
n o
f st
ages
NG
T
2017
,Jo
nes
et a
l41
Can
ada
To id
entif
y un
answ
ered
que
stio
ns
enco
unte
red
dur
ing
man
agem
ent
of k
idne
y ca
ncer
agr
eem
ent
by
cons
ensu
s on
a p
riorit
ised
list
of
the
top
10
shar
ed u
nans
wer
ed
que
stio
ns a
nd t
o es
tab
lish
corr
esp
ond
ing
rese
arch
prio
ritie
s.
1.
Pat
ient
s, c
arer
s, a
nd c
linic
ians
.2.
JL
A G
uid
eboo
k 20
13, V
.5.
3.
NR
.4.
P
atie
nts
with
kid
ney
canc
er.
5.
In t
otal
, 200
4 tr
eatm
ent
que
stio
ns w
ere
sub
mitt
ed b
y 22
5 p
artic
ipan
ts.
A 1
5-p
erso
n st
eerin
g gr
oup
was
form
ed w
ith 7
p
atie
nts/
care
rs a
nd 7
exp
ert
clin
icia
ns fr
om a
cros
s C
anad
a. In
res
pon
se, t
he K
idne
y C
ance
r R
esea
rch
Net
wor
k of
Can
ada,
in c
olla
bor
atio
n w
ith t
he J
LA,
Kid
ney
Can
cer
Can
ada,
the
Kid
ney
Foun
dat
ion
of
Can
ada,
was
form
ed
The
grou
p a
lso
incl
uded
an
ad
viso
r fr
om t
he J
LA
(UK
) who
pro
vid
ed s
upp
ort
and
ad
vice
thr
ough
out
the
pro
cess
.
Five
ste
ps
1.
Form
atio
n of
ste
erin
g gr
oup
.2.
Id
entif
ying
tre
atm
ent
que
stio
ns.
3.
Col
latin
g q
uest
ions
.4.
In
terim
ran
king
of
que
stio
ns.
5.
Fina
l prio
rity
sett
ing
wor
ksho
p.
2017
,Lo
mer
et
al28
UK
To p
rovi
de
a co
mp
rehe
nsiv
e su
mm
ary
of t
he r
esea
rch
prio
rity
find
ings
rel
atin
g to
die
t in
the
tr
eatm
ent
of in
flam
mat
ory
bow
el
dis
ease
1.
Pat
ient
s, c
arer
s an
d c
linic
ians
.2.
JL
A G
uid
eboo
k 20
16, V
.6.
3.
NR
.4.
D
ieta
ry t
reat
men
t of
in
flam
mat
ory
bow
el d
isea
se.
5.
In t
otal
, 167
1 tr
eatm
ent
unce
rtai
ntie
s w
ere
sub
mitt
ed
by
531
par
ticip
ants
.
Ste
erin
g co
mm
ittee
com
pris
ing
two
pat
ient
s, t
wo
gast
roen
tero
logi
sts,
tw
o in
flam
mat
ory
bow
el d
isea
se
spec
ialis
t nu
rses
, tw
o co
lore
ctal
sur
geon
s, t
wo
die
titia
ns, a
rep
rese
ntat
ive
from
the
UK
infla
mm
ator
y b
owel
dis
ease
cha
rity
orga
nisa
tion,
Cro
hn’s
and
C
oliti
s U
K, a
rep
rese
ntat
ive
of t
he J
LA a
nd a
n ad
min
istr
ator
(ie,
13-
per
son
stee
ring
com
mitt
ee).
Sta
keho
lder
s fr
om v
ario
us r
oles
, age
s an
d e
thni
c gr
oup
s.
A r
epre
sent
ativ
e of
the
JLA
an
d a
n ad
min
istr
ator
on
the
stee
ring
com
mitt
ee.
Five
ste
ps
1.
Ste
erin
g co
mm
ittee
.2.
Q
uest
ionn
aire
sur
vey.
3.
Rem
aini
ng u
ncer
tain
ties
wer
e re
view
ed.
4.
Unc
erta
intie
s w
ere
det
erm
ined
.5.
Fi
nal w
orks
hop
of t
he
stee
ring
grou
p.
2017
,M
acb
eth
et a
l29
UK
To id
entif
y un
cert
aint
ies
in a
lop
ecia
ar
eata
man
agem
ent
and
tre
atm
ent
that
are
imp
orta
nt t
o b
oth
serv
ice
user
s, p
eop
le w
ith h
air
loss
, car
ers/
re
lativ
es a
nd c
linic
ians
.
1.
Pat
ient
s, p
artn
ers/
par
ents
/ ca
rers
and
clin
icia
ns.
2.
JLA
Gui
deb
ook
2016
, V.6
.3.
N
R.
4.
Alo
pec
ia a
reat
a.5.
In
tot
al, 2
747
trea
tmen
t un
cert
aint
ies
wer
e su
bm
itted
b
y 91
2 p
artic
ipan
ts.
Four
peo
ple
rep
rese
ntin
g va
rious
pat
ient
sup
por
t gr
oup
s, fo
ur d
erm
atol
ogis
ts a
nd t
wo
furt
her
ind
ivid
uals
to
rep
rese
nt t
he B
HN
S a
nd t
he E
urop
ean
Hai
r R
esea
rch
Soc
iety
; an
acad
emic
psy
chol
ogis
t;
a re
gist
ered
tric
holo
gist
and
a G
P, a
nd a
JLA
re
pre
sent
ativ
e. T
wo
sep
arat
e st
eerin
g gr
oup
s.
A J
LA r
epre
sent
ativ
e p
rovi
ded
ind
epen
den
t ov
ersi
ght
of t
heP
SP
and
cha
ired
the
st
eerin
g gr
oup
.
Five
ste
ps+
NG
T1.
Iden
tifica
tion
and
invi
tatio
n of
pot
entia
l par
tner
s.2.
Invi
tatio
n to
sub
mit
unce
rtai
ntie
s.3.
Col
latio
n.4.
Ran
king
of t
reat
men
t un
cert
aint
ies.
5. F
inal
wor
ksho
p.
2017
,N
arah
ari e
t al
44
Ind
ia
To s
umm
aris
e th
e p
roce
ss o
f ly
mp
hoed
ema
PS
P, d
iscu
ssio
n d
urin
g th
e fin
al p
riorit
isat
ion
wor
ksho
p a
nd r
ecom
men
dat
ion
on
the
top
sev
en p
riorit
ies
for
futu
re
rese
arch
in ly
mp
hoed
ema
and
a
brie
f roa
d m
ap.
1.
Pat
ient
s, t
heor
ist
and
nur
ses.
2.
JLA
Gui
deb
ook
2013
, V.5
.3.
N
R.
4.
Lym
pho
edem
.a5.
In
tot
al, 1
37 r
esp
ond
ents
up
load
ed r
esea
rch
prio
ritie
s.
The
Facu
lty o
f Ap
plie
d D
erm
atol
ogy
and
the
Cen
tral
U
nive
rsity
of K
eral
a p
artic
ipat
ed in
the
coo
rdin
atin
g co
mm
ittee
NR
Eig
ht s
tep
s1.
In
itiat
ion
and
set
ting
up a
co
ord
inat
ing
com
mitt
ee.
2.
Lite
ratu
re s
earc
h.3.
C
onta
ctin
g st
akeh
old
ers.
4.
List
ing
prio
ritie
s fo
r re
sear
ch.
5.
Ran
dom
col
latio
n of
p
riorit
ies.
6.
Ran
king
exe
rcis
es.
7.
Free
lym
pho
edem
a m
edic
al c
amp
.8.
Fi
nal p
riorit
isat
ion
wor
ksho
p.
Tab
le 2
C
ontin
ued
Con
tinue
d
on January 31, 2022 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2018-027473 on 30 August 2019. D
ownloaded from
12 Nygaard A, et al. BMJ Open 2019;9:e027473. doi:10.1136/bmjopen-2018-027473
Open access
Year
Aut
hor
Co
untr
yA
im o
f th
e st
udy
1. U
ser
gro
up*
2. JLA
Guideb
ook,
yea
r an
d
vers
ion
3. A
ge
of
pat
ient
†4.
Hea
lth
cond
itio
n/d
isea
se5.
Num
ber
of
init
ial u
ncer
tain
ties
an
d p
arti
cip
ants
or
retu
rned
su
rvey
s o
r up
load
ed r
esea
rch
pri
ori
ties
Ste
erin
g g
roup
‡ id
enti
fica
tio
n an
d m
anag
emen
t o
f p
artn
ers/
stak
eho
lder
s
JLA
The
ro
le o
f th
e fa
cilit
ato
r/ad
viso
r
PS
PN
umb
er o
f st
eps
Des
crip
tio
n o
f st
ages
NG
T
2017
Prio
r et
al
UK
To id
entif
y an
d p
riorit
ise
imp
orta
nt
rese
arch
que
stio
ns fo
r m
isca
rria
ge.
1.
Pat
ient
s, p
artn
ers,
fam
ily
mem
ber
s, fr
iend
s or
col
leag
ues
and
clin
icia
ns.
2.
JLA
Gui
deb
ook
2016
, V.6
.3.
N
R.
4.
Mis
carr
iage
.5.
In
tot
al, 3
279
que
stio
ns w
ere
sub
mitt
ed b
y 21
22 p
artic
ipan
ts.
The
stee
ring
grou
p w
as a
bal
ance
d c
omp
ositi
on
of w
omen
cha
ritie
s th
at r
epre
sent
ed t
hem
and
cl
inic
ians
. Som
e m
emb
ers
rep
rese
ntin
g ch
ariti
es o
r cl
inic
ians
als
o ha
d p
erso
nal e
xper
ienc
e of
pre
gnan
cy
loss
.
The
wor
ksho
p w
as c
haire
d
by
an in
dep
end
ent
JLA
fa
cilit
ator
.
Six
ste
ps
1.
Initi
atio
n.2.
C
onsu
ltatio
n.3.
Id
entif
ying
unc
erta
intie
s.4.
R
efini
ng u
ncer
tain
ties.
5.
Inte
rim p
riorit
isat
ion.
6.
Fina
l wor
ksho
p.
2017
Ree
s et
al42
Can
ada
Eng
agin
g p
atie
nts
and
clin
icia
ns in
es
tab
lishi
ng r
esea
rch
prio
ritie
s fo
r ge
stat
iona
l dia
bet
es m
ellit
us
1.
Pat
ient
s, fr
iend
s an
d r
elat
ives
an
d c
linic
ians
.2.
JL
A G
uid
eboo
k 20
13, V
.5.
3.
Age
s18–
69 y
ears
.4.
G
esta
tiona
l dia
bet
es m
ellit
us.
5.
In t
otal
, 389
tre
atm
ent
unce
rtai
ntie
s w
ere
sub
mitt
ed
by
75 p
artic
ipan
ts.
A s
teer
ing
com
mitt
ee c
onsi
stin
g of
thr
ee p
atie
nts
and
thr
ee c
linic
ians
(one
fam
ily p
hysi
cian
who
p
ract
ises
intr
apar
tum
car
e, a
n en
doc
rinol
ogis
t an
d a
neo
nato
logi
st);
a fa
cilit
ator
fam
iliar
with
the
JL
A p
roce
ss a
nd a
pro
ject
man
ager
. The
Dia
bet
es
Ob
esity
and
Nut
ritio
n S
trat
egic
Clin
ical
Net
wor
k w
ith
the
Alb
erta
Hea
lth S
ervi
ces
sup
por
ted
thi
s re
sear
ch.
Sta
keho
lder
s w
ere
not
rep
orte
d.
A fa
cilit
ator
fam
iliar
with
the
JL
A p
roce
ssFo
ur s
tep
s+N
GT
1.
Sur
vey.
2.
Pro
cess
and
col
late
.3.
In
terim
ran
king
.4.
P
riorit
y se
ttin
g w
orks
hop
.
2017
Sm
ith e
t al
30
UK
Prio
ritis
e re
sear
ch q
uest
ions
in
em
erge
ncy
med
icin
e in
a
cons
ensu
s p
roce
ss t
o d
eter
min
e th
e To
p 1
0 q
uest
ions
1.
Pat
ient
s, c
arer
s an
d c
linic
ians
.2.
JL
A G
uid
eboo
k 20
13, V
.5.
3.
NR
.4.
E
mer
genc
y m
edic
ine.
5.
In t
otal
, 214
initi
al u
ncer
tain
ties.
The
stee
ring
grou
p m
emb
ers
wer
e no
t re
por
ted
with
tit
les
but
con
sist
ed o
f 16
mem
ber
s.Th
e R
oyal
Col
lege
of E
mer
genc
y M
edic
ine.
NR
.S
ix s
tep
s1.
O
nlin
e su
bm
issi
ons.
2.
Wor
king
gro
up r
evie
ws.
3.
Min
i sys
tem
atic
rev
iew
s.4.
W
orki
ng g
roup
p
riorit
isat
ion
exer
cise
.5.
P
ublic
prio
ritis
atio
n ex
erci
se.
6.
Face
-to-
face
fina
l p
riorit
isat
ion.
2018
Fern
and
ez e
t al
31
UK
To e
stab
lish
the
rese
arch
prio
ritie
s fo
r ad
ults
with
frag
ility
frac
ture
s of
the
low
er li
mb
and
pel
vis
that
re
pre
sent
the
sha
red
inte
rest
s an
d
prio
ritie
s.
1.
Pat
ient
s, c
arer
s an
d c
linic
ians
.2.
JL
A G
uid
eboo
k 20
16, V
.6.
3.
Age
≥60
yea
rs.
4.
Frag
ility
frac
ture
s of
the
low
er
limb
and
pel
vis.
5.
In t
otal
, 963
tre
atm
ent
unce
rtai
ntie
s w
ere
sub
mitt
ed
by
365
par
ticip
ants
.
The
stee
ring
grou
p c
onsi
sted
of p
atie
nt
rep
rese
ntat
ives
, hea
lthca
re p
rofe
ssio
nals
and
ca
rers
with
est
ablis
hed
link
s to
rel
evan
t p
artn
er
orga
nisa
tions
to
ensu
re t
hat
a ra
nge
of s
take
hold
er
grou
ps
wer
e re
pre
sent
ed.
A J
LA a
dvi
ser
sup
por
ted
an
d g
uid
ed t
he P
SP
Five
ste
ps
1.
Firs
t su
rvey
.2.
S
cree
ning
.3.
Th
emat
ic a
naly
sis,
orig
inal
un
cert
aint
ies
turn
ed in
to
over
arch
ing
ind
icat
ive
que
stio
ns.
4.
Evi
den
ce s
earc
h in
terim
p
riorit
isat
ion.
5.
Fina
l wor
ksho
p.
Tab
le 2
C
ontin
ued
Con
tinue
d
on January 31, 2022 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2018-027473 on 30 August 2019. D
ownloaded from
13Nygaard A, et al. BMJ Open 2019;9:e027473. doi:10.1136/bmjopen-2018-027473
Open access
Year
Aut
hor
Co
untr
yA
im o
f th
e st
udy
1. U
ser
gro
up*
2. JLA
Guideb
ook,
yea
r an
d
vers
ion
3. A
ge
of
pat
ient
†4.
Hea
lth
cond
itio
n/d
isea
se5.
Num
ber
of
init
ial u
ncer
tain
ties
an
d p
arti
cip
ants
or
retu
rned
su
rvey
s o
r up
load
ed r
esea
rch
pri
ori
ties
Ste
erin
g g
roup
‡ id
enti
fica
tio
n an
d m
anag
emen
t o
f p
artn
ers/
stak
eho
lder
s
JLA
The
ro
le o
f th
e fa
cilit
ato
r/ad
viso
r
PS
PN
umb
er o
f st
eps
Des
crip
tio
n o
f st
ages
NG
T
2018
Fine
r et
al32
UK
To d
escr
ibe
pro
cess
es a
nd
outc
omes
of a
PS
P a
nd t
o id
entif
y th
e to
p 1
0 re
sear
ch p
riorit
ies’
inty
pe
2 d
iab
etes
.
1.
Pat
ient
s, c
arer
s an
d c
linic
ians
.2.
JL
A G
uid
eboo
k 20
16, V
.6.
3.
NR
.4.
Ty
pe
2 d
iab
etes
.5.
In
tot
al, 8
227
trea
tmen
t un
cert
aint
ies
wer
e su
bm
itted
b
y 25
87 p
artic
ipan
ts.
The
stee
ring
grou
p c
omp
rised
five
peo
ple
livi
ng w
ith
typ
e 2
dia
bet
es (m
anag
ing
thei
r co
nditi
on in
diff
eren
t w
ays)
, five
clin
icia
ns (i
nclu
din
g a
die
ticia
n,
dia
bet
es s
pec
ialis
t nu
rse,
GP
and
tw
o co
nsul
tant
d
iale
ctol
ogis
ts),
an in
form
atio
n sp
ecia
list,
sev
en
mem
ber
s of
the
Dia
bet
es U
K R
esea
rch
and
the
se
nior
lead
ersh
ip t
eam
, and
a J
LA s
enio
r ad
viso
r. Th
e st
eerin
g gr
oup
(47%
men
and
53%
wom
en a
nd
26%
from
bla
ck a
nd m
inor
ity e
thni
c gr
oup
s) m
et
12 t
imes
dur
ing
the
PS
P p
roce
ss, i
n p
erso
n or
by
tele
conf
eren
ce.
The
wor
ksho
p w
as
faci
litat
ed b
y tr
aine
d J
LA
advi
sors
.
Four
ste
ps+
NG
T1.
Gat
herin
g un
cert
aint
ies.
2. O
rgan
isin
g th
e un
cert
aint
ies.
3. In
terim
prio
rity
sett
ing.
4. F
inal
prio
rity
sett
ing.
2018
Lech
elt
et a
l43
Can
ada
To id
entif
y th
e to
p 1
0 tr
eatm
ent
unce
rtai
ntie
s in
hea
d a
nd n
eck
canc
er fr
om t
he jo
int
per
spec
tive
of p
atie
nts,
car
egiv
ers,
fam
ily
mem
ber
s an
d t
reat
ing
clin
icia
ns.
1.
Pat
ient
s, c
arer
s, fa
mily
m
emb
ers,
and
clin
icia
ns.
2.
JLA
Gui
deb
ook
2013
, V.5
.3.
N
R.
4.
Pat
ient
s w
ith h
ead
and
nec
k ca
ncer
.5.
In
tot
al, 8
18 t
reat
men
t un
cert
aint
ies
wer
e su
bm
itted
b
y 16
1 p
artic
ipan
ts.
The
stee
ring
com
mitt
ee in
clud
ed fi
ve p
atie
nts
with
hea
d a
nd n
eck
canc
er w
ho w
ere
from
3 t
o 25
yea
rs s
ince
dia
gnos
is; s
even
clin
icia
ns in
volv
ed
in t
he t
reat
men
t an
d m
anag
emen
t of
hea
d a
nd
neck
can
cer
(max
illa-
faci
al p
rost
hod
ontis
t,
rad
iatio
n on
colo
gist
, sp
eech
lang
uage
pat
holo
gist
cl
inic
ian-
rese
arch
er, i
nfec
tious
dis
ease
sp
ecia
list,
an
apla
stol
ogis
t, a
nd t
wo
head
and
nec
k on
colo
gica
l an
d r
econ
stru
ctiv
e su
rgeo
ns).
How
ever
, a s
ixth
in
div
idua
l (fa
mily
mem
ber
) was
invo
lved
info
rmal
ly
thro
ugho
ut t
he p
roje
ct, d
esp
ite b
eing
una
ble
to
com
mit
to r
egul
ar p
artic
ipat
ion.
Alb
erta
Can
cer
Foun
dat
ion
and
the
Inst
itute
for
Rec
onst
ruct
ive
Sci
ence
s in
Med
icin
e
The
wor
ksho
p w
as
led
by
an in
dep
end
ent
faci
litat
or w
ith e
xten
sive
ex
per
ienc
e on
JLA
PS
P
pro
ject
s, s
upp
orte
d b
y tw
o co
faci
litat
ors,
all
of
who
m w
ere
brie
fed
by
the
JLA
sen
ior
advi
sor
on r
ecom
men
ded
JLA
p
roto
cols
.
Five
ste
ps+
NG
T1.
In
itial
sur
vey
dev
elop
men
t an
d d
eplo
ymen
t.2.
Id
entif
ying
unc
erta
intie
s th
roug
h su
rvey
dat
a p
roce
ssin
g.3.
Ve
rifyi
ng u
ncer
tain
ties.
4.
Inte
rim p
riorit
isat
ion.
5.
Fina
l wor
ksho
p.
2018
Loug
h et
al33
UK
To id
entif
y th
e sh
ared
prio
ritie
s fo
r fu
ture
res
earc
h of
wom
en a
ffect
ed
by
and
clin
icia
ns in
volv
ed w
ith
pes
sary
use
for
the
man
agem
ent
of
pro
lap
se.
1.
Pat
ient
s, c
arer
s an
d c
linic
ians
.2.
JL
A G
uid
eboo
k 20
16, V
.6.
3.
Age
s 30
–89
year
s.4.
P
essa
ry u
se in
wom
en w
ith
pro
lap
se.
5.
In t
otal
, 669
que
stio
ns w
ere
sub
mitt
ed b
y 21
0 p
artic
ipan
ts.
The
stee
ring
grou
p c
omp
rised
thr
ee w
omen
with
p
essa
ry e
xper
ienc
e, t
hree
clin
icia
ns e
xper
ienc
ed in
m
anag
ing
pro
lap
se w
ith p
essa
ries,
tw
o re
sear
cher
s an
d a
pes
sary
com
pan
y re
pre
sent
ativ
e, t
he P
SP
w
ith g
uid
ance
from
the
JLA
ad
vise
r an
d p
roje
ct
lead
er. T
he J
LA P
essa
ry P
SP
was
par
tially
fund
ed
by
a U
K C
ontin
ence
Soc
iety
res
earc
h gr
ant,
tw
o gr
ants
from
the
Pel
vic
Ob
stet
ric a
nd G
ynae
colo
gica
l P
hysi
othe
rap
y gr
oup
of t
he C
hart
ered
Soc
iety
of
Phy
siot
hera
py
and
a fu
nded
stu
den
tshi
p fr
om
Gla
sgow
Cal
edon
ian
Uni
vers
ity.
The
stee
ring
grou
p a
gree
d
on t
he t
erm
s of
ref
eren
ce
and
pro
toco
l for
the
JLA
ad
vise
r an
d p
roje
ct le
ader
.
Four
ste
ps+
NG
T1.
G
athe
ring
que
stio
ns/
unce
rtai
ntie
s.2.
R
efini
ng t
he q
uest
ions
and
ch
ecki
ng t
he e
vid
ence
.3.
P
riorit
isin
g/ra
nkin
g th
e q
uest
ions
.4.
C
hoos
ing
the
top
10
prio
ritie
s b
y co
nsen
sus.
Tab
le 2
C
ontin
ued
Con
tinue
d
on January 31, 2022 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2018-027473 on 30 August 2019. D
ownloaded from
14 Nygaard A, et al. BMJ Open 2019;9:e027473. doi:10.1136/bmjopen-2018-027473
Open access
Year
Aut
hor
Co
untr
yA
im o
f th
e st
udy
1. U
ser
gro
up*
2. JLA
Guideb
ook,
yea
r an
d
vers
ion
3. A
ge
of
pat
ient
†4.
Hea
lth
cond
itio
n/d
isea
se5.
Num
ber
of
init
ial u
ncer
tain
ties
an
d p
arti
cip
ants
or
retu
rned
su
rvey
s o
r up
load
ed r
esea
rch
pri
ori
ties
Ste
erin
g g
roup
‡ id
enti
fica
tio
n an
d m
anag
emen
t o
f p
artn
ers/
stak
eho
lder
s
JLA
The
ro
le o
f th
e fa
cilit
ato
r/ad
viso
r
PS
PN
umb
er o
f st
eps
Des
crip
tio
n o
f st
ages
NG
T
2018
Mac
bet
h et
al34
UK
Iden
tify
unce
rtai
ntie
s in
hai
r lo
ss m
anag
emen
t, p
reve
ntio
n,
dia
gnos
is a
nd t
reat
men
t th
at a
re
imp
orta
nt t
o b
oth
peo
ple
with
hai
r lo
ss a
nd c
linic
ians
1.
Pat
ient
s, c
arer
s re
lativ
es a
nd
clin
icia
ns.
2.
JLA
Gui
deb
ook
2016
, V.6
.3.
N
R4.
H
air
loss
(exc
lud
ing
alop
ecia
ar
eata
)5.
In
tot
al, 2
747
trea
tmen
t un
cert
aint
ies
wer
e su
bm
itted
b
y 91
2 p
artic
ipan
ts.
The
stee
ring
grou
p c
omp
rised
four
peo
ple
re
pre
sent
ing
vario
us p
atie
nt s
upp
ort
grou
ps,
fo
ur d
erm
atol
ogis
ts, a
psy
chol
ogis
t, a
reg
iste
red
tr
icho
logi
st a
nd a
GP.
A J
LA r
epre
sent
ativ
e en
sure
d
key
stak
ehol
der
s w
ere
iden
tified
thr
ough
a p
roce
ss
of c
onsu
ltatio
n an
d p
eer
know
led
ge, b
uild
ing
on
stee
ring
grou
p m
emb
ers’
net
wor
ks a
nd e
xist
ing
JLA
af
filia
tes.
The
pro
cess
was
faci
litat
ed
by
the
JLA
to
ensu
re
fairn
ess,
tra
nsp
aren
cy a
nd
acco
unta
bili
ty.
Five
ste
ps+
NG
T1.
Id
entifi
catio
n an
d in
vita
tion
of p
oten
tial p
artn
ers.
2.
Invi
tatio
n to
sub
mit
unce
rtai
ntie
s.3.
C
olla
tion.
4.
Ran
king
of t
reat
men
t un
cert
aint
ies.
5.
Fina
l wor
ksho
p.
*Use
r gr
oup
mea
ns t
he p
artic
ipan
ts w
ho a
re in
volv
ed in
the
PS
P p
roce
ss, n
ot o
nly
the
surv
ey.
†Age
ref
ers
to a
ge o
f pat
ient
s w
ho a
re in
volv
ed in
the
sur
vey.
‡Ste
erin
g gr
oup
, ste
erin
g co
mm
ittee
and
coo
rdin
atin
g co
mm
ittee
are
defi
ned
as
equa
l con
cep
ts.
BH
NS
, Brit
ish
Hai
r an
d N
ail S
ocie
ty; C
KD
, chr
onic
kid
ney
dis
ease
; GP,
gen
eral
pra
ctiti
oner
; JLA
, Jam
es L
ind
Alli
ance
; NG
T, n
omin
al g
roup
tec
hniq
ue; N
R, n
ot r
epor
ted
; PS
P, p
riorit
y se
ttin
g p
artn
ersh
ip; U
I, ur
inar
y in
cont
inen
ce.
Tab
le 2
C
ontin
ued
studies44 used an online survey to collect uncertainties; patients and clinicians were invited via email to endorse their priorities based on a table that had been devel-oped from abstracts collected in a literature search. Among the other 36 studies, 12 used open-ended ques-tions1 15 18 23 25 32 35 40–43 45 such as ‘What questions about the management of hypertension or high blood pressure would you like to see answered by research?’ In seven studies, participants (patients, carers and clinicians) were asked to submit three to five research ideas.16 17 20 21 27 28 33 In eight studies, no limit was placed on the types of ques-tions that could be submitted.5 13 24 30 31 36 37 39 One study asked about eight open-ended questions requesting a narrative answer.38 Close-ended questions were used in three studies,22 29 34 such as ‘Do you have questions about the prevention, diagnosis or treatment of hair loss that need to be answered by research?’ Five studies did not report their question format.7 12 14 19 26
The number of submitted uncertainties ranged from 8227, submitted by 2587 participants,32 to 323, submitted by 58 participants.45 All studies except two7 44 reported involving patients, carers and clinicians in the initial survey. Two of the studies addressed veri-fying uncertainties for example by content experts or librarians.40 43 The steering group or researchers were involved in addressing verifying uncertainties in 22 of the studies and5 7 14–16 18 20 21 23–27 30 31 33 35 37 39 41 44 45 in 13 of the studies not describing verifying the uncertain-ties.1 12 13 17 19 22 28 29 32 34 36 38 42
Data processing and verifying uncertaintiesUnlike most surveys that are designed to collect answers, JLA PSP surveys are designed to collect questions. The survey responses must then be reviewed, sorted and turned into a list of ‘indicative’ questions, all of which are unanswered uncertainties.6
According to Lechelt et al,43 uncertainties are organ-ised through coding, with natural clusters emerging. During this step, duplicates such as similar and related uncertainties are identified. Clinician–patient dyads consolidate and rephrase each cluster of related ques-tions into a single indicative uncertainty, written in lay language using a standard format. Lomer et al28 specified that similar uncertainties are combined to create indicative uncertainties. Among our included studies, 20 described refining questions into indicative uncertainties,5 13–15 17 19 20 23 24 27–29 31–34 38 39 42 43 while 17 did not describe a concept of indicative uncertain-ties.1 7 12 16 18 21 22 25 26 30 35–37 40 41 44 45
In total, 16 of the studies described directly ranking and assessing survey-generated uncertain-ties from a long list ranging from 43 to 226 uncertain-ties.1 5 13 14 19–21 23 24 26 27 30 38 41 43 44
The wording of the long list of uncertainties was reviewed by the steering group, and, in some cases, wording was altered to make the uncertainties more understandable and to explain complex words not gener-ally well known to the public.1
on January 31, 2022 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2018-027473 on 30 August 2019. D
ownloaded from
15Nygaard A, et al. BMJ Open 2019;9:e027473. doi:10.1136/bmjopen-2018-027473
Open access
Interim priority settingInterim prioritisation is the stage at which the long list of uncertainties (indicative questions) is reduced to a short list for the final priority setting workshop.6
All studies described an interim stage, using the terms: interim priority setting14 32; interim prioritisation1 5 42; and ranking exercise.13 44
Their short lists varied from 2226 to 30 uncertain-ties.12 17–19 22 25 30 36 39 Sixteen of the studies used an interim prioritisation of their top 25 uncertainties that were taken to a final prioritisation workshop, where the participants agreed on their top 10 priorities.1 7 13 20 21 23 24 28 29 31 33–35 37 38 40 Three of the studies did not describe the number of short-listed treatment uncertainties.15 27 44
To reduce the number of uncertainties, an interim prioritisation exercise was conducted by email or by post.5 18 32 Patients, carers and health professionals were initially invited to examine the long list18; 14 of the studies used a second online survey,1 19 20 23 25 28 29 31–35 38 40 and in one study, the steering group members facilitated an interim ranking exercise.39
Final priority settingThe JLA’s final stage is a rank ordering of the uncer-tainties, with a particular emphasis on the lists of top 10 priorities. For JLA PSPs, a final face-to-face priority setting workshop is conducted with both small group and whole group discussions. The NGT can be used by groups, with voting to ensure that all opinions are considered6; 21 of the studies reported using the NGT in the final priority setting workshop.5 12 16–19 21 23 25 26 29 32–34 36 37 39 41–43 45
All of the studies implemented a final priority setting workshop to agree on their top 10 priorities. In most of the studies, these final workshops included patients, carers and clinicians; nine studies mentioned including only patients and clinicians.7 23 28 29 33 34 42–44
DISCuSSIOnTo our knowledge, this is the first scoping review of published studies using the JLA approach, although the number of steps used by PSPs differed and not all papers describe in detail every aspect of the JLA approach. However, overall they incorporated the same proce-dural content, which indicates no implications or small implications for our findings. Thus, this scoping review provides unique insight into a broad and varied range of perspectives on PPI using the JLA approach. Interest-ingly, there were some differences between the questions submitted by patients and carers compared with those submitted by clinicians. The patients focused more on symptoms and function than on disease, while clinicians focused on general treatment. Compared with clinicians, patients submitted more questions about psychosocial issues, psychosocial stress, depression and anxiety.13 23 40 There were no studies that described disagreement in the prioritisation steps. The health conditions addressed in these studies were primarily somatic diseases, although
one study was about life after stroke and included mental health.5 Thus, the JLA approach is an appropriate and important method for defining research from the perspectives of end users that is, patients and carers.46
A key value that informs such partnerships is often described as equality. Equitable partnerships might be defined as a gradation of shared responsibility negoti-ated in a collaborative and cooperative decision-making environment. Whether such values always align within the JLA process is an open question. Thus, reflecting on and clarifying values about involvement before starting collab-orative work might enhance the positive impacts while avoiding the negative impacts of public involvement.47
The number of priority setting exercises in health research is increasing,48 and our review indicates that the use of the JLA approach is also growing . This approach facilitates broad stakeholder involvement, and it is trans-parent and easy to replicate. This is consistent with find-ings by Yoshida,48 who argues that there is a clear need for transparent, replicable, systematic and structured approaches to research priority setting to assist policy-makers and research funding agencies in making invest-ments. Increased public involvement can lead to a wider range of identified and prioritised research topics that are more relevant to service users.49 A key strength of involving the public and patients, rather than only academics, throughout the partnership process is described in these studies, including having a project led by representatives of a wider range of consumer and clinician organisa-tions.1 The number of resulting uncertainties reflects this breadth. The studies examined tended to conclude that the JLA principles were welcomed, but consistently empha-sised the need for an even broader understanding, better conceptualisation and improved processes to incorporate the results into research. However, few studies focused on how to reach the weakest voices for survey participa-tion. After critically reading these studies, one might ask whether they included the lowest socioeconomic groups and most vulnerable patients. Many respondents, particu-larly those associated with charity organisations, are likely to be white and middle class and to have high education attainment levels. Yet it is the individuals who are more difficult to reach, such as those in low socioeconomic groups and those who are vulnerable patients, who may have the greatest unmet needs and stand to gain the most from improved treatment.25 26 35 42 Given that the JLA is designed to identify shared research priorities, such indi-viduals and their needs may not be reflected in what is typically reported studies. In one case, to better facilitate patient and carer involvement, and to reach those who may not receive and/or respond to email or postal infor-mation, a steering group member visited existing support groups and arranged the distribution of information leaflets at local meetings.5 Although great efforts were reportedly made25 to include participants from black and minority ethnic groups and care home populations, they were not particularly successful. Lough et al33 reported that the use of an online survey may introduce a bias in
on January 31, 2022 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2018-027473 on 30 August 2019. D
ownloaded from
16 Nygaard A, et al. BMJ Open 2019;9:e027473. doi:10.1136/bmjopen-2018-027473
Open access
favour of patients who use the internet and social media. It is also likely that those with literacy issues will not participate.16 Three of the studies5 18 42 attempted to facil-itate participation among those with language barriers and literacy issues, which implies that efforts need to be made to enable minority groups and learning disabilities to participate in the PSP process. Stephens et al22 note another major challenge to involving users in research and patients in the steering group who have incapaci-tating symptoms and short expected survival durations. Another important issue is that all but two studies44 45 were from English-speaking countries and thus represent a relatively limited global population.
According to the JLA Guidebook,6 PSPs usually report their process and methods, the participants involved, results, reflections on successes, lessons learnt or limita-tions, and the next steps. It is important that these reports be written in a language understandable to everyone with an interest in the topic, not just to clinicians. Lough et al33 explained that all of the unanswered questions generated by their PSPs would be available on the JLA website and widely disseminated to research commissioners, public health and research funders. However, such reports can be difficult to obtain by those without ready online access or by those with literacy issues. Eleftheriadou et al13 included implementation of a feasibility study as one of their top 10 priorities; the authors hoped that, following its publication, along with their list of the most important uncertainties, relevant studies would be developed.
Running a PSP and involving the relevant stakeholders in deciding which research should be funded seem to be an effective and sustainable model.24 Without doubt, the essential advantage is integration of this involvement in both research and healthcare. Identifying research priorities is perhaps where the PSP’s greatest effect can be achieved.26 Nevertheless, one might ask whether PSPs emphasise basic research less than applied research. Abma et al50 have argued that the international literature describes corresponding challenges in research agenda setting and follow-up; patient involvement is limited to actual agenda setting, and there is limited understanding of what happens next and how to shape patient involve-ment activities in follow-up phases. This scoping review process gathered a large number of research priorities from a diverse set of respondents.32 34 There has been a clear paradigm shift from a reactive to a more proactive approach described as ‘predictive, personalised, preven-tative and participatory’.25 It is expected that the JLA process will have a clinical impact by driving relevant research studies based on PPI. Crowe et al51 reported that a critical mismatch between the treatments that patients and clinicians want to have evaluated and the treatments actually being evaluated by researchers. This apparent mismatch should be taken into account in future research.
Strengths and limitationsA major strength of this paper is the application of a rigorous and robust scoping review method, including
independent screening and data extraction. The search strategy was carefully performed in conjunction with a research librarian. To strengthen the review’s validity, several databases were used, and we have reported them with complete transparency. The studies selected for inclusion were manually searched. Although we searched multiple databases for the period since their inception, we may not have identified all relevant studies. We did not search the grey literature, assuming that empir-ical research using the JLA approach would be found in indexed databases. As a scoping review, the findings describe the nature of research using JLA’s approach and provide direction for future research; hence, this review cannot suggest how to operationalise the JLA process or how to use it in a given context. Another strength is that several of the researchers contributing to this project also work in the clinical areas represented in the studies. In addition, while a quality analysis was beyond the scope of this paper, we have noted varying descriptions within the selected studies (ie, sample sizes, health status and age of groups). Finally, the included studies do not provide information about the impact of involvement, regarding development of consensus, the discussions among all those who took part, the distribution of power and the politics. In future work, it may be important to evaluate how much influence patient/public partners had during the process, besides the impact of the number of partici-pants in the respective groups. Another limitation might involve our inclusion criteria with respect to requirement for peer-reviewed publications, which by definition will use more academic language and may not be readily accessible to the layperson. Lastly, the cost and time involved in a PSP are described in one publication only.24 According to the JLA Guidebook, the PSP process will last approximately 12–18 months.6
COnCluSIOnSJLA-based PSP makes a useful contribution to identifying research questions. A range from 327 to 8227 uncer-tainties were published, with 27 studies from UK. The number of reported steps varied from four to eight. In total, 33 studies mentioned the involvement of a JLA facilitator. Twenty-four included studies that addressed methods for verifying uncertainties, and the use of NGT was reported in 21 studies. Finally, it is important that the results of these studies, including the top 10 priori-ties, reach those who answered the survey, including the vulnerable groups. Online publishing might contribute to this. Future studies should focus on factors influencing patient and carer involvement in priority setting projects.
Acknowledgements We thank research librarian Malene Wøhlk Gundersen for her helpful and knowledgeable assistance.
Contributors AN, LH, SL, EKG and AB designed the study. AN coordinated the project and is the guarantor. AN, LH, SL, EKG and AB screened articles and performed data extraction. AN conducted the literature search. AN, LH, SL, EKG and AB interpreted the data. AN drafted the manuscript and all authors critically reviewed it. All authors read and approved the manuscript.
on January 31, 2022 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2018-027473 on 30 August 2019. D
ownloaded from
17Nygaard A, et al. BMJ Open 2019;9:e027473. doi:10.1136/bmjopen-2018-027473
Open access
Funding This work was supported by the Research Council of Norway (grant number OFFPHD prnr 271870), Lørenskog Municipality and Oslo Metropolitan University. The funders had no role in the study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests None declared.
Patient consent for publication Not required.
Provenance and peer review Not commissioned; externally peer reviewed.
Data availability statement All data relevant to the study are included in the article.
Open access This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http:// creativecommons. org/ licenses/ by- nc/ 4. 0/.
reFerenCeS 1. van Middendorp JJ, Allison HC, Ahuja S, et al. Top ten research
priorities for spinal cord injury: the methodology and results of a British priority setting partnership. Spinal Cord 2016;54:341–6.
2. Hanley B, Bradburn J, Barnes M, et al. Involving the public in NHS public health, and social care research: Briefing notes for researchers. UK: Involve 2004;2:1–61.
3. Hoddinott P, Pollock A, O'Cathain A, et al. How to incorporate patient and public perspectives into the design and conduct of research. F1000Res 2018;7.
4. Price A, Albarqouni L, Kirkpatrick J, et al. Patient and public involvement in the design of clinical trials: an overview of systematic reviews. J Eval Clin Pract 2018;24:240–53.
5. Pollock A, St George B, Fenton M, et al. Top 10 research priorities relating to life after stroke – consensus from stroke survivors, caregivers, and health professionals. International Journal of Stroke 2014;9:313–20.
6. National Institute for health research, the James Lind alliance Guidebook: version 7, 2018. Available: http://www. jla. nihr. ac. uk/ jla- guidebook/ downloads/ Print- JLA- guidebook- version- 7- March- 2018. pdf
7. Hall DA, Mohamad N, Firkins L, et al. Identifying and prioritizing unmet research questions for people with tinnitus: the James Lind alliance tinnitus priority setting partnership. Clin Investig 2013;3:21–8.
8. Tricco AC, Lillie E, Zarin W, et al. A scoping review on the conduct and reporting of scoping reviews. BMC Med Res Methodol 2016;16:15.
9. National Institute for health research, the James Lind alliance Guidebook: version 6, 2016. Available: http:// jla. nihr. ac. uk/ jla- guidebook/ downloads/ JLA- Guidebook- Version- 6- February- 2016. pdf
10. Cowan K, Oliver S. The James Lind alliance Guidebook: version 5, 2013. Available: http://www. jlaguidebook. org/ pdfguidebook/ guidebook. pdf
11. Cowan K, Oliver S. James Lind alliance Guidebook: version 4, 2010. Available: http://www. bvsde. paho. org/ texcom/ cd045364/ guidebook. pdf
12. Buckley BS, Grant AM, Tincello DG, et al. Prioritizing research: patients, carers, and clinicians working together to identify and prioritize important clinical uncertainties in urinary incontinence. Neurourol Urodyn 2010;29:708–14.
13. Eleftheriadou V, Whitton ME, Gawkrodger DJ, et al. Future research into the treatment of vitiligo: where should our priorities lie? results of the vitiligo priority setting partnership. Br J Dermatol 2011;164:no–6.
14. Gadsby R, Snow R, Daly AC, et al. Setting research priorities for Type 1 diabetes. Diabetic Medicine 2012;29:1321–6.
15. Batchelor JM, Ridd MJ, Clarke T, et al. The eczema priority setting partnership: a collaboration between patients, carers, clinicians and researchers to identify and prioritize important research questions for the treatment of eczema. Br J Dermatol 2013;168:577–82.
16. Deane KHO, Flaherty H, Daley DJ, et al. Priority setting partnership to identify the top 10 research priorities for the management of Parkinson's disease. BMJ Open 2014;4:e006434.
17. Ingram JR, Abbott R, Ghazavi M, et al. The hidradenitis suppurativa priority setting partnership. Br J Dermatol 2014;171:1422–7.
18. Rowe F, Wormald R, Cable R, et al. The sight loss and vision priority setting partnership (SLV-PSP): overview and results of the research prioritisation survey process. BMJ Open 2014;4:e004905.
19. Uhm S, Crowe S, Dowling I, et al. The process and outcomes of setting research priorities about preterm birth — a collaborative partnership. Infant 2014;10:178–81.
20. Boney O, Bell M, Bell N, et al. Identifying research priorities in anaesthesia and perioperative care: final report of the joint National Institute of academic Anaesthesia/James Lind alliance research priority setting partnership. BMJ Open 2015;5:e010006.
21. Kelly S, Lafortune L, Hart N, et al. Dementia priority setting partnership with the James Lind alliance: using patient and public involvement and the evidence base to inform the research agenda. Age Ageing 2015;44:985–93.
22. Stephens RJ, Whiting C, Cowan K, et al. Research priorities in mesothelioma: a James Lind alliance priority setting partnership. Lung Cancer 2015;89:175–80.
23. Knight SR, Metcalfe L, O’Donoghue K, et al. Defining priorities for future research: results of the UK kidney transplant priority setting partnership. PLoS One 2016;11:e0162136.
24. Rangan A, Upadhaya S, Regan S, et al. Research priorities for shoulder surgery: results of the 2015 James Lind alliance patient and clinician priority setting partnership. BMJ Open 2016;6:e010412.
25. Wan YL, Beverley-Stevenson R, Carlisle D, et al. Working together to shape the endometrial cancer research agenda: the top ten unanswered research questions. Gynecol Oncol 2016;143:287–93.
26. Britton J, Gadeke L, Lovat L, et al. Research priority setting in Barrett's oesophagus and gastro-oesophageal reflux disease. Lancet Gastroenterol Hepatol 2017;2:824–31.
27. Hart AL, Lomer M, Verjee A, et al. What are the top 10 research questions in the treatment of inflammatory bowel disease? A priority setting partnership with the James Lind alliance. ECCOJC 2017;11:204–11.
28. Lomer MC, Hart AL, Verjee A, et al. What are the dietary treatment research priorities for inflammatory bowel disease? a short report based on a priority setting partnership with the James Lind alliance. J Hum Nutr Diet 2017;30:709–13.
29. Macbeth AE, Tomlinson J, Messenger AG, et al. Establishing and prioritizing research questions for the treatment of alopecia areata: the alopecia areata priority setting partnership. Br J Dermatol 2017;176:1316–20.
30. Smith J, Keating L, Flowerdew L, et al. An emergency medicine research priority setting partnership to establish the top 10 research priorities in emergency medicine. Emerg Med J 2017;34:454–6.
31. Fernandez MA, Arnel L, Gould J, et al. Research priorities in fragility fractures of the lower limb and pelvis: a UK priority setting partnership with the James Lind alliance. BMJ Open 2018;8:e023301.
32. Finer S, Robb P, Cowan K, et al. Setting the top 10 research priorities to improve the health of people with type 2 diabetes: a diabetes UK-James Lind alliance priority setting partnership. Diabetic Medicine 2018;35:862–70.
33. Lough K, Hagen S, McClurg D, et al. Shared research priorities for pessary use in women with prolapse: results from a James Lind alliance priority setting partnership. BMJ Open 2018;8:e021276.
34. Macbeth A, Tomlinson J, Messenger A, et al. Establishing and prioritizing research questions for the prevention, diagnosis and treatment of hair loss (excluding alopecia areata): the hair loss priority setting partnership. Br J Dermatol 2018;178:535–40.
35. Prior M, Bagness C, Brewin J, et al. Priorities for research in miscarriage: a priority setting partnership between people affected by miscarriage and professionals following the James Lind alliance methodology. BMJ Open 2017;7:e016571.
36. Manns B, Hemmelgarn B, Lillie E, et al. Setting research priorities for patients on or nearing dialysis. CJASN 2014;9:1813–21.
37. Barnieh L, Jun M, Laupacis A, et al. Determining research priorities through partnership with patients: an overview. Semin Dial 2015;28:141–6.
38. Fitzcharles M-A, Brachaniec M, Cooper L, et al. A paradigm change to inform fibromyalgia research priorities by engaging patients and health care professionals. Canadian Journal of Pain 2017;1:137–47.
39. Hemmelgarn BR, Pannu N, Ahmed SB, et al. Determining the research priorities for patients with chronic kidney disease not on dialysis. Nephrol Dial Transplant 2017;32:847–54.
40. Khan N, Bacon SL, Khan S, et al. Hypertension management research priorities from patients, caregivers, and healthcare providers: a report from the hypertension Canada priority setting partnership group. J Clin Hypertens 2017;19:1063–9.
41. Jones J, Bhatt J, Avery J, et al. The kidney cancer research priority-setting partnership: identifying the top 10 research priorities as defined by patients, caregivers, and expert clinicians. Can Urol Assoc J 2017;11:379–87.
on January 31, 2022 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2018-027473 on 30 August 2019. D
ownloaded from
18 Nygaard A, et al. BMJ Open 2019;9:e027473. doi:10.1136/bmjopen-2018-027473
Open access
42. Rees SE, Chadha R, Donovan LE, et al. Engaging patients and clinicians in establishing research priorities for gestational diabetes mellitus. Canadian Journal of Diabetes 2017;41:156–63.
43. Lechelt LA, Rieger JM, Cowan K, et al. Top 10 research priorities in head and neck cancer: results of an Alberta priority setting partnership of patients, caregivers, family members, and clinicians. Head Neck 2018;40:544–54.
44. Narahari SR, Aggithaya M, Moffatt C, et al. Future research priorities for morbidity control of lymphedema. Indian J Dermatol 2017;62:33–40.
45. Davila-Seijo P, Hernández-Martín A, Morcillo-Makow E, et al. Prioritization of therapy uncertainties in dystrophic epidermolysis bullosa: where should research direct to? an example of priority setting partnership in very rare disorders. Orphanet J Rare Dis 2013;8:61.
46. Chalmers I, Ignorance CT. Confronting therapeutic ignorance. BMJ 2008;337:a841–7.
47. Gradinger F, Britten N, Wyatt K, et al. Values associated with public involvement in health and social care research: a narrative review. Health Expectations 2015;18:661–75.
48. Yoshida S. Approaches, tools and methods used for setting priorities in health research in the 21st century. J Glob Health 2016;6.
49. Barber R, Boote JD, Parry GD, et al. Can the impact of public involvement on research be evaluated? a mixed methods study. Health Expectations 2012;15:229–41.
50. Abma TA, Pittens CACM, Visse M, et al. Patient involvement in research programming and implementation: a responsive evaluation of the dialogue model for research agenda setting. Health Expect 2015;18:2449–64.
51. Crowe S, Fenton M, Hall M, et al. Patients', clinicians' and the research communities' priorities for treatment research: there is an important mismatch. Res Involv Engagem 2015;2.
on January 31, 2022 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2018-027473 on 30 August 2019. D
ownloaded from
