Incorporating patient-preference evidence into regulatory ......indicating their relative value for...
Transcript of Incorporating patient-preference evidence into regulatory ......indicating their relative value for...
Incorporating patient-preference evidence into regulatorydecision making
Martin P. Ho • Juan Marcos Gonzalez • Herbert P. Lerner •
Carolyn Y. Neuland • Joyce M. Whang • Michelle McMurry-Heath •
A. Brett Hauber • Telba Irony
Received: 5 September 2014 / Accepted: 9 November 2014
� Springer Science+Business Media New York (outside the USA) 2015
Abstract
Background Patients have a unique role in deciding what
treatments should be available for them and regulatory
agencies should take their preferences into account when
making treatment approval decisions. This is the first study
designed to obtain quantitative patient-preference evidence
to inform regulatory approval decisions by the Food and
Drug Administration Center for Devices and Radiological
Health.
Methods Five-hundred and forty United States adults
with body mass index (BMI) C30 kg/m2 evaluated trade-
offs among effectiveness, safety, and other attributes of
weight-loss devices in a scientific survey. Discrete-choice
experiments were used to quantify the importance of
safety, effectiveness, and other attributes of weight-loss
devices to obese respondents. A tool based on these mea-
sures is being used to inform benefit-risk assessments for
premarket approval of medical devices.
Results Respondent choices yielded preference scores
indicating their relative value for attributes of weight-loss
devices in this study. We developed a tool to estimate the
minimum weight loss acceptable by a patient to receive a
device with a given risk profile and the maximum mortality
risk tolerable in exchange for a given weight loss. For
example, to accept a device with 0.01 % mortality risk, a
risk tolerant patient will require about 10 % total body
weight loss lasting 5 years.
Conclusions Patient preference evidence was used make
regulatory decision making more patient-centered. In
addition, we captured the heterogeneity of patient prefer-
ences allowing market approval of effective devices for
risk tolerant patients. CDRH is using the study tool to
define minimum clinical effectiveness to evaluate new
weight-loss devices. The methods presented can be applied
to a wide variety of medical products. This study supports
the ongoing development of a guidance document on
incorporating patient preferences into medical-device pre-
market approval decisions.Electronic supplementary material The online version of thisarticle (doi:10.1007/s00464-014-4044-2) contains supplementarymaterial, which is available to authorized users.
M. P. Ho � H. P. Lerner � C. Y. Neuland �J. M. Whang � M. McMurry-Heath � T. Irony (&)
Center for Devices and Radiological Health, U.S. Food and Drug
Administration, 10903 New Hampshire Ave, Building 66, Room
2232, Silver Spring, MD 20993-0002, USA
e-mail: [email protected]
M. P. Ho
e-mail: [email protected]
H. P. Lerner
e-mail: [email protected]
C. Y. Neuland
e-mail: [email protected]
J. M. Whang
e-mail: [email protected]
M. McMurry-Heath
e-mail: [email protected]
J. M. Gonzalez � A. Brett Hauber
RTI Health Solutions, Durham, USA
e-mail: [email protected]
A. Brett Hauber
e-mail: [email protected]
M. McMurry-Heath
FaegreBD Consulting, Washington, USA
123
Surg Endosc
DOI 10.1007/s00464-014-4044-2
and Other Interventional Techniques
Keywords Patient preferences � Weight-loss devices �Obesity treatment � FDA � Benefit-risk assessment �Regulatory-approval decisions
In the past, the United States (US) Food and Drug
Administration (FDA) did not have a scientific way to
consider a representative opinion of patients when weigh-
ing the benefits and risks of a medical product to approve it
to be marketed in the United States [1, 2]. Since the release
of its guidance document on benefit-risk determinations in
2012, the FDA Center for Devices and Radiological Health
(CDRH) has made clear that patient preference is an
important factor to consider when evaluating medical
devices for market approval. The guidance explicitly states
that
Risk tolerance will vary among patients, and this will
affect individual patient decisions as to whether the
risks are acceptable in exchange for a probable ben-
efit. …FDA would consider evidence relating to
patients’ perspective of what constitutes a meaningful
benefit.
For example, when evaluating weight-loss devices for
market approval, CDRH seeks to identify appropriate
levels of benefit that outweigh different levels of risk [3].
Patients have a unique role in deciding what treatments
should be available for them. Their perspectives about the
value of benefits and the impact of risks of medical treat-
ments are essential because only patients live with their
medical conditions and consequences of the choices they
make for their own care. Moreover, their perspectives can
be different from those of regulators and care providers. As
the principle of patient-centered heath care has been widely
accepted, regulatory authorities are very interested in
obtaining input on patient perspectives on benefits and
tolerance for risk for making regulatory evaluations,
knowing that unless a treatment is approved, it is not an
option for patients regardless of their risk tolerance [4–6].
This study is the first step in demonstrating how to
implement this idea and offers a quantitative approach to
bridging the gap between what CDRH reviewers and
patients regard as acceptable benefit-risk tradeoffs for
weight-loss devices. Although numerous efforts of regu-
latory agencies have been made to actively reach out and
listen to the opinion of patients with various conditions, the
response received has been qualitative in nature. While
these contributions can be valuable for other purposes, they
cannot adequately address the approval decisions because
they lack measurability, representativeness, accuracy, and
inclusiveness. In contrast, this study provides quantitative
evidence of patient benefit-risk tradeoff preference that
includes a wide spectrum of patients. It quantifies patients’
perspectives on benefits, providing the minimum benefit
they expect from a weight-loss device to tolerate a given
level of risk and other device attributes. Patient tolerance
for risk is quantified as the maximum device-related mor-
tality risk patients are willing to tolerate for a given weight
loss and other device attributes.
Because patients’ perspectives on benefits and risks may
be very heterogeneous within a patient group, CDRH needs
to consider the whole spectrum of patient preferences.
Some patients in the top quartile of the risk-tolerance dis-
tribution could be ‘‘early adopters,’’ meaning they will
tolerate higher risks to gain quick access to innovative
treatments. CDRH will consider approving a medical
device that demonstrates meaningful benefits even though
its benefit-risk profile would be acceptable only to a subset
of patients who are risk-tolerant. Such a device’s Indication
for Use will explain that the benefit-risk profile may be
suitable to only a subgroup of patients who should consult
with their care providers in a context of shared medical
decision making.
The objectives of this study were to (1) apply well-
established methods for quantifying the relative importance
of effectiveness, safety, and other attributes of weight-loss
devices to patients; (2) derive measures of patients’ pref-
erences to inform benefit-risk assessments; and (3) incor-
porate such measures into regulatory decision making.
This is the first patient-preference study designed by
regulatory reviewers and used to inform regulatory deci-
sion making. The results are providing direction for
designing clinical trials for premarket approval, informing
benefit-risk assessments once the trial results are being
evaluated, and guiding post-approval decisions. The con-
ceptual framework and quantitative methods can be gen-
eralized to a wide variety of medical treatments and are
particularly relevant when patients and regulators face
difficult decisions when weighing potential treatment
benefits against serious adverse-event risks.
Materials and methods
Design overview
Choice-experiment surveys ask respondents to choose the
most-preferred alternative from a set of two or more con-
structed virtual alternatives in a series of questions. Sta-
tistical analysis of the pattern of these choices reveals the
implicit relative importance of the attributes of the health
intervention that influence respondents’ choices among the
alternatives offered. The result is an estimate of the per-
ceived value of an intervention as a weighted sum of the
intervention attributes, where the weights reflect the mean
Surg Endosc
123
relative importance of each factor. In this study we call
these weights ‘‘preference scores.’’
Setting and participants
A Web-enabled, cross-sectional survey was administered
between September and November 2012 to a stratified sample
of English-speaking US residents with a current or previous
BMI of at least 30 kg/m2 who were willing to lose weight. The
sample was drawn from the online GfK KnowledgePanel�,
divided equally among three groups: 30 B BMI\35,
35 B BMI\40, and 40 B BMI.
Respondents who did not meet the BMI inclusion cri-
teria were screened out. Respondents who did not answer
any question or who always picked the same alternative
were excluded from the sample. Because we compared the
available choices with a no-device alternative, data that did
not include an answer to the follow-up device-acceptance
question or for which the answer was ‘‘Don’t know/not
sure’’ were excluded.
The consent form, survey instrument, and data-collec-
tion protocol were approved by the RTI International
Office of Research Protection and Ethics and the Office of
Information and Regulatory Affairs in the US Office of
Management and Budget.
Survey instrument
Survey content was developed based on published clinical
research on obesity, regulatory knowledge of weight-loss
device technology, and interviews with obese individuals.
Eight weight-loss device attributes were selected based on
knowledge about devices that are likely to be reviewed by
CDRH in the near future. The attributes were of primary
concern to physicians, regulators, and obese subjects con-
sulted during survey development: (1) total body weight loss
(TBWL) as percentage of current weight, (2) duration of
weight loss, (3) duration of mild-to-moderate side effects,
(4) mortality risk, (5) chance of a side effect requiring hos-
pitalization, (6) recommended dietary restrictions, (7)
reduction in chance of comorbidity or reduction in pre-
scription dosage for existing comorbidity, and (8) type of
surgery. Each attribute had three to five levels. The levels of
the continuous attributes covered a wide range of values to
allow for interpolation of estimates, whereas the levels of the
categorical attributes included several plausible categories.
Clinical-outcome attributes were described as average
results for each device profile. Respondents were asked to
assume that all costs were covered by insurance. All the
device attributes and levels used in the study are listed in
Appendix A in Supplementary material.
The study design included several internal-validity tests,
with particular attention to verifying that respondents
understood basic risk concepts. Before answering the
choice questions, respondents received a risk tutorial and
answered a question to verify their understanding.
The choice questions employed a standard format for
choice-experiment surveys [7–9]. Each respondent evalu-
ated eight pairs of virtual weight-loss device profiles and
were asked ‘‘Which weight-loss device do you think is
better for people like you?’’ Figure 1 is an example choice
question. Respondents were asked in a separate follow-up
question whether they personally would accept the better
device if it were available or they would prefer no device.
Device profiles in the choice questions varied according
to a predetermined experimental design with known sta-
tistical properties. An experimental-design algorithm yiel-
ded 120 choice questions [10–12]. Good-practice
guidelines recommend 8–12 questions per respondent to
limit measurement error due to fatigue. Because of the
complexity of a choice task that includes a probabilistic
outcome, the design was divided into 15 survey versions of
8 questions each, and respondents were randomized to one
of the versions. More information on the final experimental
design is included in Appendix B in Supplementary
material.
A draft of the survey instrument was pretested in
90-min, face-to-face, semistructured interviews with nine
obese subjects in accordance with US Office of Manage-
ment and Budget regulations. Following final ethics and
regulatory review, the Web-enabled instrument was further
field-tested by drawing 86 respondents from the GfK Web
panel before its full release.
Results
Data quality
Of 1,057 panel members randomly drawn from GfK’s
KnowledgePanel�, a large Web panel that matches the
demographics of the general US population, 710 responded
to the invitation and 568 qualified for the survey: 1.2 % did
not answer any tradeoff question, 1.9 % always picked the
same alternative, and 14.3 % did not answer the follow-up
device-acceptance question. After these exclusions, 540
respondents were available for the final analysis, yielding a
qualification rate of 80.0 % and a final-stage completion
rate of 67.2 %. Additional details on response rates are
provided in Appendix C in Supplementary material.
Results of internal validity tests indicated the data sat-
isfied high quality standards. For example, only 34
respondents (6.3 %) failed a quiz question following the
risk tutorial. Additional details on internal validity are
provided in Appendices D and E in Supplementary
material.
Surg Endosc
123
The demographic characteristics of respondents who
were included and excluded in the analysis were similar.
Although the sample was stratified by BMI, the analysis
sample on average was about 2 years older and about 8.5 %
heavier than the general US obese population. Additional
details on comparisons among included respondents,
excluded respondents, and the general obese population are
provided in Appendix F in Supplementary material.
Fig. 1 Example tradeoff
question
Surg Endosc
123
Preference-score estimates
The analysis provides estimates of the preference scores
that best explain the pattern of observed choices in the data.
Additional details on parameter estimates are provided in
Appendix G in Supplementary material.
Figure 2 shows the preference-score estimates for all
attribute levels. They are on a scale from -10 (least pre-
ferred), to 10 (most preferred), where -10 is the perceived
value of 5 % mortality risk. The difference between the
best and worst preference scores for each attribute indicates
how influential that attribute is in explaining device choi-
ces. Mortality risk was the most important attribute, fol-
lowed by weight loss, weight-loss duration, and side-effect
duration. However, no side effect requiring hospitalization
and a 5 % risk of a side effect requiring hospitalization
with no surgery were considered much less important.
Table 1 shows the estimated preference scores for the
attributes and their levels, indicating their relative impor-
tance. For example, the importance of an increase in
mortality risk from 0 to 1 % can be quantified by the
preference-score difference between these two levels; it
was about -3.5 (6.5 - 10). In comparison, the importance
of a 30 % TBWL was about 4.3 (4.3 - 0). Therefore, if
other device attributes are kept the same, 30 % TBWL
more than compensates for having to bear a 1 % risk of
death from the device.
Benefit-risk tradeoff comparisons
In this study, we used the estimated preference scores to
calculate the minimum acceptable benefit (MinB), i.e.,
minimum weight loss respondents expect from a device to
tolerate a specific level of risk and other device attributes.
Fig. 2 Relative-importance estimates for attributes of weight-loss
devices preference parameters were rescaled relative to 10, corre-
sponding to the absolute value of the most important outcome—5 %
mortality risk. Attributes with connecting lines were modelled as
nonlinear Box–Cox transformed continuous variables. Other attri-
butes were modelled as categorical variables. Vertical bars around
preference scores denote 95 % confidence intervals. Benefits have
positive scores and harms have negative scores. Differences between
the best and worst level in each attribute indicate relative attribute
importance over the level ranges in the study design. Attributes are
sorted from most important benefit to most important harm. Mortality
risk is the most important outcome or feature, followed by weight
loss, weight-loss duration, and side-effect duration. The relative
importance of a 1 % increase in risk is -3.5, in comparison with the
relative importance of a 30 % total body weight loss of ?4.3
Surg Endosc
123
We also calculated the maximum acceptable risk (MaxR),
i.e., maximum device-related mortality risk respondents are
willing to tolerate for a given weight loss and other device
attributes.
The statistical analysis accounted for heterogeneity in
patients’ preferences and allowed quantitative segmenta-
tion of obese respondents according to their risk tolerance.
The MinB and MaxR estimates in the middle 50 % and the
upper 25 % of the sample indicate the quantitative benefit-
risk tradeoff preferences of average patients and risk-tol-
erant patients, respectively.
Table 2 compares the overall acceptability of three
devices relative to the no-device option indicated by MinB
and MaxR for the gastric-band and for Devices A and B,
which are worse and better than the gastric band, respec-
tively. Preference-score estimates predict that only 5.0 %
of respondents would judge Device A better than the no-
device alternative, while 27.5 % of respondents would
judge Device B better than the no-device alternative.
Devices with less attractive attributes require either larger
weight loss or smaller mortality risks in compensation. For
example, the gastric band offers only 13 % TBWL, but the
Table 1 Estimates of preference scores by attributes and levels
Attribute Level Preference score (SE)
Average amount of weight loss (TBWL) 30 % ?4.3 (0.52)
20 % ?2.0 (0.11)
10 % ?0.6 (0.15)
5 % ?0.2 (0.23)
0 % Reference level
Weight loss duration 60 months ?4.3 (0.47)
12 months ?2.0 (0.01)
6 months ?1.4 (0.1)
0 months Reference level
Side effect duration 0 months Reference level
1 months -1.0 (0.11)
12 months -2.0 (0.09)
60 months -3.2 (0.31)
Chance of side effects requiring hospitalization None Reference level
5 % chance of hospitalization, no surgery -0.2 (0.39)
20 % chance of hospitalization, no surgery -0.5 (0.35)
5 % chance of hospitalization, with surgery -0.6 (0.36)
Dietary restrictions Eat 1/4 cup of food at a time Reference level
Wait 4 h between eating -0.1 (0.29)
Can’t eat sweets or foods that are hard to digest -2.2 (0.33)
Average reduction in dose of prescription drugs for comorbiditya Eliminate need/risk ?3.2 (0.37)
50 % dose/risk ?2.2 (0.29)
No change Reference level
Type of operation Laparoscopic surgery Reference level
Endoscopic surgery -0.5 (0.3)
Open surgery -2.5 (0.31)
Chance of dying from getting the device 0 % Reference level
1 % -3.5 (0.13)
3 % -7.1 (0.15)
5 % -10 (0.37)
Receiving a weight-loss device through open surgery versus endoscopic surgery had a preference-score difference of -2 (-2.5 minus -0.5).
Maintaining weight loss for 12 months versus no months had a preference-score difference of ?2 (2 - 0). Holding everything else constant,
receiving a device placed through open surgery instead of endoscopically requires at least 12 months of weight loss to compensate. The same
metric can be used to measure patient tolerance for risks of adverse events
TBWL total body weight lossa Average reduction in dose of prescription drugs for the current primary comorbid condition or chance of getting the most feared comorbid
condition at the lower weight
Surg Endosc
123
middle 50 % of respondents require a mean TBWL of
more than 30 % given the attributes of the gastric band.
Similarly, the gastric band has a 1 % mortality risk while
the middle 50 % of respondents would tolerate only a
0.16 % risk for the attributes of the gastric band. On the
other hand, for risk-tolerant early adopters, the mean
minimum acceptable TBWL is only 13.0 % and the mean
maximum acceptable risk is 7.1 %, which compare favor-
ably to the gastric band’s profile. Overall, estimates imply
that only 11.6 % of respondents would judge the gastric
band better than no device, which is consistent with the fact
that only a small percentage of obese patients have chosen
gastric-band surgery [13, 14].
The tool
One of the main issues faced by regulatory reviewers when
designing clinical studies and analyzing their results is
determination of the ‘‘minimum clinical effectiveness’’ that
is sufficient to offset the risks and inconveniences posed by
the treatments under review. This value is used to size the
clinical studies and, when results are available, to decide
Table 2 Preference relative to a no-device alternative, minimum acceptable benefit, and maximum acceptable risk of a 243 pound respondent
for three weight-loss device profiles
Weight loss device profile % Respondents who
judged better than
a no-device
alternative
Risk
Tolerance
Minimum
acceptable
TBWLa
Maximum
acceptable
mortality riskb
Virtual device A 5.0 % Middle 50 % [30 %c 0.1 % (0.04–0.26)
Benefit: 5 % TBWL
Risk: 1 % chance of death Upper 25 % 15.1 %
(11.63–19.05)
3.8 % (1.85–7.77)
Type of surgery: laparoscopic surgery
Dietary restriction: eat 1/4 cup of food at a time
Weight-loss duration: 36 months
Minor side-effect duration: 36 months
Chance of hospitalization with surgery: 5 %
Comorbidity: no improvement
Gastric band 11.6 % Middle 50 % [30 %c 0.16 % (0.07–0.38)
Benefit: 13 % TBWL
Risk: 1 % chance of death Upper 25 % 13.0 % (9.76–16.90) 7.1 % (3.82–13.50)
Type of surgery: laparoscopic surgery
Dietary restriction: eat 1/4 cup of food at a time
Weight-loss duration: 5 years
Minor side-effect duration: 5 years
Chance of hospitalization for side effects requiring surgery:
5 %
Comorbidity: no improvement
Virtual device B 27.5 % Middle 50 % [30 %c 0.6 % (0.29–1.12)
Benefit: 20 % TBWL
Risk: 1 % chance of death Upper 25 % 12.8 % (9.85–16.45) 10.9 %
(6.54–18.78)Type of surgery: endoscopic surgery
Dietary restriction: eat 1/4 cup of food at a time
Weight-loss duration: 60 months
Minor side-effect duration: 12 months
Chance of hospitalization but no surgery: 5 %
Comorbidity: 50 % improvement
95 % confidence intervals in parentheses
TBWL total body weight lossa For indicated mortality risk and other device attributesb For indicated weight-loss benefit and other device attributesc Exceeds upper limit of weight-loss levels included in the study design
Surg Endosc
123
whether the benefits of the treatment outweigh the risks for
market approval. There is an enormous amount of discus-
sion among device makers, regulators and experts about
this value.
To include patient preference input in these discussions,
we use the preference-score estimates to build a MaxR–
MinB calculator for weight-loss devices (‘‘the tool’’). The
MinB obtained from the tool is used to inform CDRH
reviewers in establishing the minimum clinical effective-
ness (weight loss) of devices under review to size clinical
studies. In addition, once the results of the study are
available, reviewers use the minimum clinical effective-
ness as a reference to evaluate the benefit of a device that is
being considered for premarket approval. Given the profile
of the weight-loss device, the tool can answer three dif-
ferent but related questions:
1. What percentage of subjects would prefer getting the
device over a no-device alternative?
2. What is the MaxR of an average subject or of an early
adopter for a device that provides a given weight loss?
3. What is the MinB of an average subject or of an early
adopter for a device that poses a given mortality risk?
Development of the tool went through several evalua-
tions and iterations with the regulatory reviewers to make it
usable, understandable, and relevant to the review work.
Examples of the tool’s appearance, input fields, and func-
tion are shown in Fig. 3.
Discussion
This is the first study designed to provide regulators with
quantitative data on patient preferences in support of reg-
ulatory benefit-risk tradeoff determinations. Previous
information on patient preferences largely has consisted of
qualitative, anecdotal testimony obtained in advisory-
committee proceedings and public meetings. In contrast,
this study provides quantitative data obtained in a con-
trolled experiment from a cross-sectional sample of US
obese respondents. FDA regulators and the investigators
jointly developed the survey instrument to ensure the study
design provided relevant tradeoff information for devices
likely to be reviewed by the agency in the foreseeable
future. Moreover, these attributes were important to obese
subjects, and their levels are clinically meaningful and
relevant to regulatory decision making.
Patients’ perspectives on benefits and tolerance for risks
are likely to be diverse within a patient population and, as a
whole, may also differ from clinicians’ perspectives. It is
important that a regulatory agency takes into account
variations in risk tolerance of the whole spectrum of pos-
sible users of a medical treatment because unless a
treatment is approved, it is not available to physicians and
patients, regardless of their risk tolerance. Patients who are
more tolerant of higher risks are likely to be ‘‘early
adopters,’’ meaning they could be willing to accept higher
risks to gain faster access to innovative treatments. Risk-
tolerant early adopters also are likely to participate in
clinical trials of a novel technology and play a critical role
in postapproval evaluations by accepting the novel tech-
nology once it is on market. If only risk-tolerant patients
would accept a certain device profile, the FDA might
consider approving such a device only for risk-tolerant
patients. Such indication for use will be explained in the
device label.
CDRH is using the MaxR–MinB calculator ‘‘the tool,’’
to inform reviewers in establishing the minimum clinical
effectiveness of weight-loss devices under review. Similar
decision-aid calculators can be developed for other medical
products. For a given medical product profile, the tool can
assist regulatory reviewers to determine the minimum
clinical effectiveness to be used in trial design and inter-
pretation of trial results. In addition, opinions among cli-
nicians and patients may be mixed when evaluating a new
medical product that offers an incremental improvement in
effectiveness but a slightly worse safety profile than
existing treatment options. Such medical products could be
preferred by some but rejected by others, and the FDA
should take this into account.
This study may have been subject to some technical
limitations. Choices among virtual weight-loss devices do
not have the same clinical and emotional consequences as
actual choices. However, clearly defined and validated
definitions of device attributes helped increase the reli-
ability of the study results by ensuring that respondents
were well informed. The potential for hypothetical bias was
further reduced by first eliciting judgments about which
combination of plausible attributes was better and subse-
quently eliciting the stated choice between device and no-
device alternatives.
The inclusion criteria relied on patient-reported height
and weight, not measured BMI. Assurance of anonymity
reduced incentives to understate weight and overstate
height. In fact, the mean self-reported weight in the study
cFig. 3 The MaxR–MinB Tool. 3.1 Using the tool to estimate
minimum acceptable benefit (25 %TBWL). 3.2 Using the tool to
estimate maximum acceptable risk (0.5 % mortality). As shown in 3.1and 3.2, the tool provides an input area at the top where the device
attributes, including mortality risk (0.05 %) for (3.1) and benefit
(15 % TBWL) for (3.2) are entered. The green cells in the center
display the estimated MinB (25 %) and MaxR (0.5 %) respectively.
Results can be shown for average patients (middle quartile of the
distribution) or for early adopters (upper quartile of the distribution)
and for different base weights (Color figure online)
Surg Endosc
123
Surg Endosc
123
sample was greater than the national average for the same
age cohort. Furthermore, we found no significant differ-
ences in preferences between respondents with BMI less
than or greater than 40.
Whereas the Web panel matches the demographics of
the general U.S. population, our stratified sample of obese
panel members does not need to match the demographics
of the general obese population. Nevertheless, we found
only a 2-year difference in mean age. Because we did not
find significant differences in preferences between
respondents by BMI, the 8.5 % difference in mean weight
between the study and national sample has no effect on
estimates.
Despite these limitations, the estimated preference
scores for treatment attributes and their levels provide a
quantitative solution to the problem of comparing different
outcomes in benefit-risk assessments. They also provide
information for making patient-centered, evidence-based
regulatory decisions. This study has demonstrated the
practical feasibility of eliciting and using evidence on
patient benefit-risk tradeoff preferences to inform regula-
tory decisions. The experience acquired from this study
serves as a proof of principle to support the ongoing
development of a CDRH guidance document on including
evidence on patient preferences in medical-device sub-
missions for premarket approval. We believe that the
approach used here is superior to making regulatory deci-
sions without an understanding of the patients’ values and
their heterogeneity. This study is a significant step in the
direction of incorporating patients’ benefit preferences and
risk tolerance into the existing evidence-based regulatory
process.
Acknowledgments We acknowledge Jeffrey Shuren, MD, JD for
his support of the study and insightful advice on the manuscript. We
also acknowledge Priya Venkataraman-Rao, MD; Megan Shackel-
ford, MS; Rebecca Nipper; Richard Kotz, MS; Kathleen Olvey; and
Martin Golding, MD, for their regulatory input in the development of
the survey instrument. We are grateful to the FDA CDRH Obesity
Devices Working Group for their comments on interpretation of study
results and their feedback on using the MinB–MaxR calculator in
regulatory reviews of weight-loss device submissions. The members
of that group include Jeffrey Cooper, DVM; Megan Shackelford, MS;
Irene Bacalocostantis, PhD; Brandan Reid, PhD; Martha Betz, PhD;
Elizabeth Katz, PhD; David Pudwill; Martin Golding, MD; Priya
Venkataraman-Rao, MD; and Benjamin Fisher, PhD. The acknowl-
edged persons above are of CDRH, FDA.
Disclosures Drs. Hauber and Gonzalez received compensation for
their work through a contract with the Center for Devices and
Radiological Health (CDRH) of the U.S. Food and Drug Adminis-
tration (FDA). Ho, Lerner, Neuland, Whang, McMurry-Heath, and
Irony have no conflicts of interest or financial ties to disclose.
References
1. http://www.fda.gov/downloads/MedicalDevices/DeviceR
egulationandGuidance/GuidanceDocuments/UCM296379.pdf.
Accessed 2 Sept 2014
2. Section 513(a) of the Federal Food, Drug, and Cosmetic Act [21
U.S.C. 360c(a)]
3. Lerner H, Whang J, Nipper R (2013) Benefit-risk paradigm for
clinical trial design of obesity devices: FDA proposal. Surg En-
dosc 27(3):702–707
4. Shuren J, Woodcock J (2013) Reviewing FDA’s implementation
of FDASIA. http://www.fda.gov/NewsEvents/Testimony/ucm374
544.htm. Accessed 2 Sept 2014
5. McMurry-Heath M (2013) FDA brings patients into the process.
http://blogs.fda.gov/fdavoice/index.php/2013/09/fda-brings-
patients-into-the-process/. Accessed 2 Sept 2014
6. http://www.ema.europa.eu/docs/en_GB/document_library/
Report/2013/10/WC500153276.pdf. Accessed 2 Sept 2014
7. Marshall D, Bridges JFP, Hauber B et al (2010) Conjoint analysis
applications in health: how are studies being designed and
reported? Patient 3(4):1178–1653
8. Bridges JFP, Hauber AB, Marshall D et al (2011) Conjoint
analysis applications in health—a checklist: a report of the IS-
POR good research practices for conjoint analysis task force.
Value Health 14(4):403–413
9. Johnson FR, Lancsar E, Marshall D et al (2013) Constructing
experimental designs for discrete-choice experiments: report of
the ISPOR conjoint analysis experimental design good research
practices task force. Value Health 16(1):3–13
10. Kuhfeld WF, Tobias RD, Garratt M (1994) Efficient experimental
design with marketing research applications. J Mark Res
31:545–557
11. Kuhfeld WF (2010) Marketing research methods in SAS. http://
support.sas.com/techsup/technote/mr2010.pdf. Accessed 2 Sept
2014
12. Ogden CL, Carroll MD, Kit BK, Flegal KM (2012) Prevalence of
obesity in the United States, 2009–2010. NCHS Data Brief No. 82
13. Nguyen NT, Masoomi H, Magno CP, Nguyen XMT, Laugenour
K, Lane J (2011) Trends in use of bariatric surgery, 2003–2008.
J Am Coll Surg 213(2):261–266
14. Callegaro M, DiSogra C (2008) Computing response metrics for
online panels. Public Opin Q 72(5):1008–1032
Surg Endosc
123