NETWORK META-ANALYSIS USING DATA FROM PUBLISHED TRIALS

AND DATA FROM THE FOOD AND DRUG ADMINISTRATION

MEDICAL REVIEWS: A CASE EXAMPLE OF FIRST LINE MEDICATIONS

FOR OPEN ANGLE GLAUCOMA

by

Qiyuan Shi

A thesis submitted to Johns Hopkins University in conformity with the

requirements for the degree of Master of Health Science

Baltimore, Maryland

May, 2015

ii

Abstracts

Background

Network meta-analysis (NMA) is a method that incorporates both direct evidence

within trials and indirect evidence across trials. The impact of different data sources

on NMAs is not widely examined.

Objectives

The objective of this study was to conduct and compare the results from NMAs using

clinical trial data available in the Food and Drug Administration (FDA) medical

reviews (FDA trials) and those published in the medical literature (published trials) on

first line medications for open angle glaucoma.

Methods

We searched the Cochrane Central Register of Controlled Trials, MEDLINE,

EMBASE in March 2014 and the US Food and Drug Administration’s website in

April 2014 for randomized, parallel group trials on the first line topical medications

for primary open-angle glaucoma and ocular hypertension. Eligible trials had to

compare a single active treatment with no treatment/placebo or another single active

topical medical treatment. Two individuals independently assessed trial eligibility,

abstracted data, and assessed the risk of bias. Using the FDA trials and the published

trials, we performed pair-wise meta-analyses and Bayesian network meta-analyses on

intraocular pressure at 3 months, our pre-specified primary outcome. We compared

the results from these analyses.

Results

iii

We included 16 FDA trials and 105 published trials. The network based on FDA trials

had 10 nodes (nine active drugs and placebo/vehicle) with a total sample size of 6183.

The network based on published trials had 14 nodes (13 active drugs and

placebo/vehicle) with a total sample size of 16898. There were 9 common treatments

between the FDA trials and the published trials in the network meta-analysis, resulting

in 36 comparisons. We estimated mean difference in IOP of 36 pairs of these

treatments. The median relative difference of the two networks was 14% (interquartile

range: 7% to 42%). The relative differences were greater than 25% in 14 pairs. Point

estimates from two pairs showed opposite directions. Generally, the results of the

published trials had better precision than the results of the FDA trials because of the

larger sample size of the published trials. In terms of relative rankings, bimatoprost,

travoprost, and latanoprost are the best ranking drugs regardless of the data source,

although 4/9 (44%) drugs had different ranking.

Conclusions

Systematic reviewers should consider FDA medical reviews because of the amount of

the information they can provide. The data source did not seem to change the

inference of the relative effectiveness of most of the drugs studied. The relative

rankings changed for the middle-ranked drugs but not for the top-ranked drugs.

Whether reporting bias has a role in the differences we observed needs to be further

evaluated.

Advisor: Tianjing Li, MD, MHS, PhD

Second reader: Mark L. Van Natta, MHS

iv

Acknowledgements

Before proceeding to the contents of this study, I would like to make some

acknowledgements to thank the people who contributed to this study or gave me help

of all kinds. Foremost, I would like to express my sincere gratitude to my advisor Dr.

Tiangjing Li for the continuous support of this study and research, for her patience,

motivation, enthusiasm, and immense knowledge. Her guidance helped me in all the

time of research and writing of this thesis. Second, I want to thank Mark L. Van Natta

for being my second reader and for his great advice and instructions. I would also like

to thank my dear colleague Benjamin Rouse for his support and help.

I would like to thank Hwanhee Hong, Sueko Ng, Nan Guo Cesar Ugarte-Gil, Manuele

Michelessi, Lijuan Zeng, and Kexin Jin for their help with screening, data abstraction,

and analysis.

Finally I would like to thank my parents for their support and love for all the time.

v

Table of Contents

1 Introduction .............................................................................................. 1

2 Methods ................................................................................................... 5

2.1 Eligibility Criteria ................................................................................................ 5

2.2 Retrieving Medical Reviews from the FDA Website .......................................... 5

2.3 Retrieving Published Trials .................................................................................. 6

2.4 Data Abstraction and Management ...................................................................... 6

2.5 Outcomes .............................................................................................................. 6

2.6 Pairwise and Network Meta-analysis ................................................................... 6

2.7 Evaluation of Clinical and Methodological Heterogeneity .................................. 8

2.8 Evaluation of Statistical Heterogeneity and Inconsistency .................................. 8

2.9 Comparison between Results from Different Sources ......................................... 9

3 Results .................................................................................................... 11

3.1 Identification of Trials ........................................................................................ 11

3.2 Analysis of FDA trials........................................................................................ 11

3.3 Analysis of Published Trials .............................................................................. 12

3.4 Comparison between FDA Trials and Published Trials ..................................... 14

3.5 Heterogeneity ..................................................................................................... 16

3.6 Inconsistency ...................................................................................................... 17

4 Discussions ............................................................................................ 18

5 References: ............................................................................................ 22

6 Appendices ............................................................................................ 25

7 Tables ..................................................................................................... 31

Table 1. Summary estimates for intraocular pressure at 3 months derived from

pair-wise meta-analysis based on 13 direct comparisons from 16 FDA trials ......... 31

vi

Table 2. Summary estimates for intraocular pressure at 3 months derived from

network meta-analysis of 16 FDA trials .................................................................. 32

Table 3. Summary estimates for intraocular pressure at 3 months derived from

pair-wise meta-analysis based on 36 direct comparisons from 105 published trials33

Table 4. Summary estimates for intraocular pressure at 3 months derived from

network meta-analysis of 105 published trials ......................................................... 34

Table 5. Comparisons between the results of direct pairwise meta-analyses based on

FDA trials and published trials................................................................................. 35

8 Figures ................................................................................................... 36

Figure 1. Identification of trials................................................................................ 36

Figure 2. Network graphs ......................................................................................... 37

Figure 3. Ranking probabilities for any drug at any position in the FDA trials network

.................................................................................................................................. 38

Figure 4. Ranking probabilities for any drug at any position in the published trials

network ..................................................................................................................... 39

Figure 5. Scatterplot of relative effect size between two treatments based on both

FDA and published trials.......................................................................................... 40

Figure 6. Relative effect size based on FDA trials and published trials .................. 41

Figure 7. Relative effect size between active drugs and placebo/timolol based on

FDA trials and published trials................................................................................. 42

Figure 8. Percentage ‘SUCRA’ of drugs in the two networks ................................. 43

1

1 Introduction

Since the concept of evidence-base medicine was introduced, systematic reviews are

often regarded as the highest-level of evidence [1, 2]. Because systematic reviews use

‘explicit and systematic methods to minimize bias’, they are thought to provide more

reliable findings than individual research studies from which conclusions and decision

can be made [3]. However, a typical systematic review focuses on the synthesis of

direct, pair-wise comparisons that are available from at least one study. If no such

study is available, for example, if no study has examined the comparative

effectiveness of treatment A and treatment B, it becomes a limitation for conventional

systematic reviews. Such needs lead to the development of network meta-analysis

(NMA).

NMA, as is indicated by its name, aims to build a network where nodes represent

interventions while the edges represent direct comparisons (that is available from at

least one included study) [4]. If two interventions A and B have one intermediate

comparator C, A and B can be compared indirectly generating indirect evidence. By

weighting both direct and indirect evidence, a mixed effect can be estimated. Thus,

NMAs can accomplish the task of “all-way” comparisons with more information [4].

A unique feature of NMAs is that it also can generate ranking probabilities of all the

interventions in terms of their efficacy and/or potential for harm[5]. Conclusions on

the comparative effectiveness of all interventions included in a network could be

drawn based on both the estimates of relative effects and ranking probabilities.

2

Systematic reviews are often subject to reporting bias, that is “systematic differences

between reported and unreported findings” [3]. Two common types of reporting bias

are publication bias and selective outcome reporting bias. Publication bias refers to

whether a study is published is dependent on the direction and magnitude of the

results; selective outcome reporting bias refers to bias in published studies due to

partial or distorted reporting of the outcomes [3]. Numerous studies have shown that

reporting bias is prevalent in clinical trials and has become a threat to the validity of

systematic reviews [6-8].

Because NMA is a newly developed methodology, the impact of reporting bias on

NMA is not well understood. One study by Trinquart et al., examining an

antidepressants network based on clinical trial data obtained from the Food and Drug

Administration (FDA), found that reporting bias affecting one drug could change the

relative rankings of drugs in the whole network [9]. In this study, medical reviews of

trials prepared by the FDA (we will use FDA trials for simplicity) were considered as

the reference (‘truth’). After conducting a sensitivity analysis on each drug, the

authors found that if the FDA trials were partially reported in the scientific journals,

the relative rankings of drugs based on the incomplete data set would differ from the

‘truth’, where all FDA trials were used.

In another study, Song et al. found that if all placebo-controlled trials were biased in

3

favor of “new” or “sponsored” treatments to the same extent, the ‘indirect comparison

will counterbalance such bias’ [10]. This finding suggests that NMAs might be more

robust to reporting bias under certain circumstances than pair-wise meta-analysis. The

same phenomenon was observed in a study of citalopram and escitalopram in which

the indirect evidence shows no evidence of difference while the direct evidence favors

the new, sponsored drug (in this case escitalopram)[11]. The authors speculated that

there was bias in the direct evidence.

Investigating reporting bias requires unique methods and sources of data. Registry

data such as data submitted to the regulatory agencies for gaining marketing approval

have become a popular source. Using the FDA in the United States as an example,

manufactures must submit clinical trial data including protocols and raw data from the

trials to the FDA before a drug can be approved. Scientists at the FDA review these

data and prepare reports that summarize their evaluation and decisions. Most of these

documents prepared by the FDA, especially the more recent ones are posted on the

FDA’s website for public viewing and use

(http://www.accessdata.fda.gov/scripts/cder/drugsatfda/).

These same trials sponsored by drug companies also may be published in scientific

journals, although not all of them are published. Trials with negative and null results

are less likely to be published than trials with positive results [12]. Studies also have

shown that the trial data submitted to the FDA and those published in the literature

4

sometimes disagree in sample size, effect size, and in the analysis plan [13].

To further investigate the impact of reporting bias on NMA, in particular, the value of

using FDA trials for NMA, we need to know the effect estimates and rankings of

interventions based on (1) the FDA trials alone; (2) published trials alone; (3) all

trials.

For the purpose of this project, we focused on analyses (1) and (2). We built upon an

ongoing systematic review and network meta-analysis [14] on the comparative

effectiveness and safety of first-line intraocular pressure (IOP) lowering drugs for

patients with primary open angle glaucoma (POAG) or ocular hypertension (OH).

Our study is a case example of examining the impact of reporting bias on NMA.

The objectives of this study were:

1. To identify and abstract data from FDA reviews of trials on first-line IOP lowering

drugs for patients with POAG or OH.

2. To conduct pairwise meta-analyses and NMAs using (1) data abstracted from the

FDA trials; (2) data abstracted from all published trials.

3. To compare the results from the two analyses above and evaluate the impact of

different data sources on the findings from NMA.

5

2 Methods

2.1 Eligibility Criteria

Trials were eligible for our NMA if they (1) recruited more than 60% patients

diagnosed with POAG or OH, using any definition specified in the trial; (2) compared

a first-line topical IOP lowering drug to placebo/no treatment or another IOP lowering

drug; (3) randomized controlled trial (RCT) that used a parallel design.

Trials were excluded if it (1) enrolled fewer than 10 patients in any study arm; (2)

evaluated combination medical therapies for POAG or OH; (3) had a follow-up

duration shorter than 28 days after randomization.

2.2 Retrieving Medical Reviews from the FDA Website

As described previously by Turner et al.[15], FDA medical reviews of approved drugs

are available from the FDA website

(http://www.accessdata.fda.gov/scripts/cder/drugsatfda/). We searched drug names

(active ingredients) and downloaded medical reviews in electronic form, including

‘Approval History’, ‘Letters’, ‘Reviews’, and ‘Related Documents’. For those

medical reviews that are not available from the FDA website, which are usually

medical reviews for drugs approved before 1996, we filed a request to FDA. We did

not receive any response from the FDA by the timeline allowed for this project. We

proceeded with medical reviews available for download from the FDA website. The

list of drugs we searched is available in Appendix 1.

6

2.3 Retrieving Published Trials

We used trials identified from an ongoing systematic review and NMA [14]. In brief,

Li et al. searched MEDLINE, EMBASE and Cochrane Register of Controlled Trials

(CENTRAL) for eligible trials up to March 2014. The search strategies are available

in Appendix 1.

2.4 Data Abstraction and Management

Two individuals working independently abstracted data from included trials into an

online database developed in the Systematic Review Data Repository

(http://srdr.ahrq.gov/) [16]. We adjudicated discrepancies through discussion.

2.5 Outcomes

IOP was typically used as the primary outcome in both published trials and FDA trials

because it is the endpoint upon which the hypotensive glaucoma drugs is approved

[17]. Thus, for all of the analyses in this study, we used mean IOP at 3 months as the

outcome measurement. If the 3 months IOP data were not available, we used the time

point that was closest to 3 months.

2.6 Pairwise and Network Meta-analysis

We conducted both pairwise meta-analyses and NMAs using data from the above

described sources (i.e., FDA trials and published trials). For pairwise meta-analysis,

7

we performed a random-effects meta-analysis for all comparisons with two or more

trials using the DerSimonian and Laird’s method [18], implemented in STATA

package ‘metan’ [19-21]. We assumed both comparison-specific heterogeneity and a

common heterogeneity across all comparisons [18].The latter assumption in theory

allows a larger degree of heterogeneity than the comparison-specific heterogeneity

approach and would results in less precise estimates.

We used Lu and Ade’s model under a Bayesian hierarchical framework with

random-effects for NMA [22, 23]. We conducted the analysis using the R package

‘gemtc’ and JAGS for Monte Carlo Markov Chain sampling [24, 25]. The analysis

estimated the mean difference in IOP for each pair of interventions. We also estimated

the probability for each intervention being ranked at one of the possible positions.

Such probabilities were used for plotting the ‘SUCRA’ (surface under the cumulative

ranking curve) values [19]. SUCRA value is the cumulative ranking probability,

which in theory accounts for the uncertainly in ranking better than a crude ranking

probability [19, 26]. Because the numbers of treatments (nodes) in each network were

different, the absolute ‘SCURA’ values from the two networks would have different

ranges. We therefore used the percentage of ‘SUCRA’, which was defined as ‘SUCRA’

values divided by maximum possible ‘SUCRA’ value in a specific network [26]. A

percentage ‘SUCRA’ of 100% means that the treatment has the best overall rankings.

After rescaling the ‘SUCRA’ values, we were able to compare the relative rankings of

treatments in the two networks regardless of the numbers of treatments in each

8

network. The larger the SUCRA value the better ranking a drug.

2.7 Evaluation of Clinical and Methodological Heterogeneity

Heterogeneity referred to any variability in the studies included in systematic reviews

[3]. We qualitatively evaluate clinical and methodological heterogeneity by

comparing the differences in characteristics of the patients, interventions, and the

nature of study designs.

2.8 Evaluation of Statistical Heterogeneity and Inconsistency

Statistical heterogeneity is a quantitative measurement of the effect size differences

due to variability. It is often measured by I2, tau2 and Q statistics in conventional

pairwise meta-analysis [27]. Q statistics are based on chi2 tests for differences

between observed data and expected data. I2 represents the “the percentage of the

variability in effect estimates that is due to heterogeneity rather than sampling

error”[28]. Tau2 is the parameter that captures the between-study variance when using

random-effect models.

In the pairwise meta-analysis, we first assumed comparison-specific heterogeneity

and estimated the tau2 for each comparison. Then we assumed that there was a

common heterogeneity across all the comparisons, regardless of the interventions

compared. Thus, a common tau2 was estimated to present the heterogeneity of all the

studies. In the NMA, we assumed common heterogeneity across the entire network

9

and estimated the tau2.

In the NMA, inconsistency refers to the disagreement in the estimated effect size

derived from direct and indirect comparisons. Several methods have been developed

to address inconsistency. We used ‘loop-specific’ method and ‘node-splitting’ method

to explore inconsistency [29-32]. ‘Loop-specific’ approach checks the triangular or

quadratic loops formed by three or four comparators. Inconsistency factors, defined as

the difference between estimates derived from the direct and indirect evidence, and

their 95% confidence intervals are commonly used to indicate the significance of

local inconsistency under the ‘loop-specific’ approach [23, 33]. Another approach,

‘node-splitting’ method, takes out direct comparisons one at a time and conducts a

NMA based on the remaining studies to give an indirect estimate for the same

comparison. Then the direct and indirect estimates are compared to quantify the

amount of inconsistency [29].

2.9 Comparison between Results from Different Sources

We conducted the above described analyses using the FDA trials and trials published

in the literature (referred to as published trials) separately. We compared the following

results obtained from these two data sources:

1. The point estimates and 95% confidence intervals based on pairwise

meta-analyses;

2. The point estimates and 95% credible intervals (intervals where the true parameter

10

lies in with a probability of 0.95 based on the posterior distribution in Bayesian

statistics) based on NMAs;

3. The percentage ‘SUCRA’ values based on NMAs.

11

3 Results

3.1 Identification of Trials

We identified 26 FDA medical reviews describing results from 72 studies (some are

not RCTs). Thirty (42%) RCTs were eligible; however only 16 of 30 (53%) trials

reported sufficient data (point estimates and confidence intervals) for our analysis

(Figure 1). The network based on FDA trials had 10 nodes (nine active drugs and

placebo/vehicle) with a total sample size of 6183 (Figure 2). Timolol was the most

commonly used comparator in these trials and was used as the reference group in our

analysis.

We identified 105 published trials eligible for analysis from the search results of

10936 studies. The reasons of exclusions are shown in Figure 1. The network based

on published trials had 14 nodes (13 active drugs and placebo/vehicle) with a total

sample size of 16898 (Figure 2). Similar to the network described above, timolol was

the most commonly used comparators (Figure 2).

3.2 Analysis of FDA trials

Table 1 shows the results from direct, pairwise meta-analyses of FDA trials.

Dorzolamide, bimatoprost, unoprostone, levobetaxolol, travoprost were compared to

timolol in at least two trials. Timolol lowered IOP more than dorzolamide,

unoprostone, and levobetaxolol; the mean differences (95% confidence intervals) in

IOP were 1.32 (0.63; 2.00), 2.00 (1.59; 2.41), and 1.25 (0.27; 2.23), respectively.

Bimatoprost and travoprost, the two prostaglandins, on the other hand, lowered IOP

12

more than timolol; the mean differences (95% confidence intervals) were 2.26 (1.70;

2.82) and 0.98 (0.52; 1.45), respectively.

Table 2 shows the results from the NMA of FDA trials. All active treatments except

unoprostone and betaxolol were more effective than placebo/vehicle (i.e., 95%

credible interval does not include 0) in lowering IOP at 3 months. Compared to

placebo, the mean differences (95% credible intervals) in IOP at 3 months, ordered by

the magnitude of IOP reduction, were: bimatoprost 6.03 (4.17; 7.85), travoprost 4.75

(2.95; 6.50), latanoprost 4.60 (2.61; 6.56), timolol 3.76 (2.09; 5.40), brinzolamide

2.66 (0.67; 4.59), levobetaxolol 2.63 (0.75, 3.98), unoprostone 1.77 (-0.06; 3.55), and

betaxolol 1.01(-0.96; 2.95).

Figure 3 shows the probability of each drug being ranked at every possible position

and the cumulative ranking probabilities. The ranking results were consistent with the

effect estimates obtained from the NMA. Bimatoprost was most likely to be ranked as

the best (probability=0.97). Placebo was most likely to be ranked as the worst

(probability=0.83). The descending order of ranking, in terms of effectiveness in

lowering IOP at 3 months, based on percentage SUCRA value was bimatoprost (9.97),

travoprost (8.64), latanoprost (8.31), timolol (7.03), brinzolamide (5.35), dorzolamide

(4.70), levobetaxolol (4.57), unoprostone (3.02), betaxolol (2.05) and placebo (0.35).

3.3 Analysis of Published Trials

13

Table 3 shows the results from direct, pairwise meta-analyses of published trials.

Apraclonidine, brimonidine, betaxolol, carteolol, levobunolol, brinzolamide,

dorzolamide, bimatoprost, latanoprost, travoprost and unoprostone were compared to

timolol in at least two trials. Among these comparisons, six were statistically

significant (i.e., 95% confidence interval does not include 0). Timolol lowered IOP

more than betaxolol, brinzolamide, dorzolamide; the mean differences (95%

confidence intervals) in IOP were 1.58 (0.87; 2.29), 1.10 (0.50; 1.70) and 0.76 (0.13;

1.39), respectively. Bimatoprost, latanoprost and travoprost, the three prostaglandins,

on the other hand, lowered IOP more than timolol; the mean differences (95%

confidence intervals) were 2.07 (1.49; 2.64), 1.32 (0.88; 1.77) and 1.22 (0.24; 2.20),

respectively.

Table 4 shows the results from the NMA of published trials. All active treatments

were more effective than placebo/vehicle (i.e., 95% credible interval does not include

0). Compared to placebo, the mean differences (95% credible intervals) in IOP at 3

months, ordered by the magnitude of reduction, were: bimatoprost 5.65 (4.89; 6.40),

travoprost 5.02 (4.23; 5.80), latanoprost 4.88 (4.22; 5.54), levobunolol 4.54 (3.77;

5.31), tafluprost 4.41 (2.88; 5.97), timolol 3.70 (3.11; 4.29), brimonidine 3.62 (2.87;

4.37), carteolol 3.44 (2.36; 4.52), dorzolamide 2.56 (1.86; 3.27), apraclonidine 2.55

(0.89; 4.23), brinzomalide 2.46 (1.61; 3.31), unoprostone 1.98 (1.13; 2.84), and

betaxolol 2.34 (1.63; 3.05).

14

Figure 3 shows the probability of each drug being ranked at every possible position

and the cumulative ranking probabilities. The ranking results were consistent with the

effect estimates gained from the NMA. Bimatoprost was most likely to be ranked as

the best (probability=0.924). Placebo was most likely to be ranked as the worst

(probability=0.999). The descending order in ranking, in terms of effectiveness in

lowering IOP at 3 months, based on percentage SUCRA value was bimatoprost

(13.92), travoprost (12.35), latanoprost (11.89), levobunolol (10.82), tafluprost

(10.46), timolol (8.42), brimonidine (8.07), carteolol (7.50), dorzolamide (4.79),

apraclonidine (4.78), brinzolamide (4.35), betaxolol (3.92), unoprostone (2.74), and

placebo (0.00).

3.4 Comparison between FDA Trials and Published Trials

We first compared the results of direct, pairwise meta-analyses based on the two data

sources. There were 10 common comparisons between the FDA trials and the

published trials. Using results from FDA trials as the reference, we found that,

compared to timolol, the results from published trials increased the point estimates of

the relative effectiveness of five drugs and decreased the point estimate of the relative

effectiveness of one drug. The percentage differences, defined as (estimate of

published trials minus estimate of FDA trials) divided by estimate of FDA trials, were

42% for dorzolamide, 8% for unoprostone, 33% for travoprost, 13% for latanoprost, 7%

for betaxolol, and -8% for bimatoprost.

15

We then compared the results derived from NMAs based on the two data sources.

There were 9 common treatments between the FDA trials and the published trials,

resulting in 36 comparisons. We estimated mean difference in IOP of 36 pairs of these

treatments. Figure 5 presents the point estimates from the two data sources. Point

estimates from two pairs (light blue dots in Figure 5) showed opposite directions,

although the credible intervals all included the null value. Analysis of FDA trials

revealed that unoprostone was better than betaxolol by 0.76 (-1.09; 2.57) and

brinzolamide was better than dorzolamide by 0.21 (-0.55; 0.97). Analysis of published

trials revealed that betaxolol was better than unoprostone by 0.36 (-0.48; 1.20) and

dorzolamide was better than brinzolamide by 0.1 (-0.95; 0.74).

The median relative difference of the two networks was 14% (interquartile range: 7%

to 42%). The relative differences were greater than 25% in 14 pairs. To account for

uncertainty, we also compared the credible intervals (Figure 6). Generally, the results

of the published trials had better precision than the results of the FDA trials because

of the larger sample size of the published trials (Figure 2). Three comparisons were

statistically different based on published trials: unoprostone was better than placebo

by 1.98 (1.13; 2.84); latanoprost was better than timolol by 1.18 (0.79; 1.57), and

betaxolol was better than placebo by 2.34 (1.63; 3.05). The same three comparisons

were not statistically significant based on FDA trials. The relative estimates were 1.77

(-0.06; 3.55) between unoprostone and placebo; 0.84 (-0.23; 1.94) between

latanoprost and timolol, and 1.01 (-0.93; 2.95) between betaxolol and placebo.

16

Figure 7 shows a subset of the 36 comparisons derived from NMAs based on the two

data sources using either placebo or timolol as the comparator. Compared to placebo,

the point estimates are similar regardless of the data source, although two of the

comparisons based on the FDA trials did not reach statistical significance. Compared

to timolol, the point estimates are also similar regardless of the data sources, and one

comparison based on the FDA trials did not reach statistical significance.

Figure 8 illustrates the differences in percentage ‘SUCRA’ values between two

networks. We found that the relative rankings changed for dorzolamide, brinzolamide

unoprostone, and betaxolol. The percentage ‘SUCRA’ values, sorted in descending

order, for the treatments in the network of FDA trials were: bimatoprost 99.69%,

travoprost 86.45%, latanoprost 83.10%, timolol 70.34%, brinzolamide 53.52%,

dorzolamide 47.00%, levobetaxolol 45.69%, unoprostone 30.21%, betaxolol 20.48,

and placebo 3.54%. The percentage ‘SUCRA’ values, sorted in descending order, for

the treatments in the network of published trials were: bimatoprost 99.42%, travoprost

88.18%, latanoprost 84.95%, levobunolol 77.23, tafluprost 74.72%, timolol 60.17%,

brimonidine 57.67%, carteolol 53.55%, dorzolamide 34.24%, apraclonidine 34.12%,

brinzolamide 31.05%, betaxolol 27.97%, unoprostone 19.56%, and placebo 0.02%.

3.5 Heterogeneity

In general, FDA trials are more homogenous than published trials. The I2 values for

comparisons based on FDA trials were less than 30%, indicating low level of

17

statistical heterogeneity [3]. For published trials, we identified two comparisons

(betaxolol and placebo, unoprostone and timolol) in which the I2 was greater than

75%, suggesting considerable statistical heterogeneity [3].

3.6 Inconsistency

The network of FDA trials had a “start” shape and did not include comparisons that

could be both directly and indirectly estimated. Thus, there was no estimable

inconsistency. As for the published trials, using the ‘node-splitting’ approach, we

found one node with evidence of statistical inconsistency. Using the ‘loop-specific’

approach, we found 6 out of 34 triangle loops (17.6%) had significant inconsistency

(p value < 0.05). We speculated that funding source and extreme effect size could be

the reasons introducing inconsistency.

18

4 Discussions

In this study, we conducted conventional pairwise meta-analyses and NMAs using

data from FDA trials and published trials. We found that FDA medical reviews are an

important source of clinical trial data that could be used for NMA. The data source

does not seem to change the inference about the drug’s relative effectiveness. The

relative rankings changed for the middle-ranked drugs but not for the top-ranked

drugs.

Several studies have examined the impact of reporting bias using the FDA trials.

Turner et al. used the FDA trials and their published versions to describe the selective

reporting bias in antidepressants and antipsychotics trials. Based on Turner’s work,

Trinquart et al. further examined the reporting bias on NMAs. In both Turner and

Trinquart’ studies, they used all FDA trials as one data source and published FDA

trials as the second data source, and compared the relative effectiveness and rankings

of drugs. However, it is worth noting that non-FDA trials published in the medical

literature, the traditional source of trials used for systematic review and meta-analysis,

could also be biased. Thus, we framed our question in a broader context than what has

been done previously. We used the FDA trials as the ‘reference’ and examined the

extent of discrepancies by using all published trials. Because published trials still

serve as the main source of data, we answered a pragmatic question: is it worth to put

resources in looking for FDA trials when conducting NMA?

19

Based on our case example, we believe that FDA medical reviews serve as an

important source of trial data; systematic reviewers should look for these data for

inclusion in their analysis. However, the FDA trial data were incomplete through the

public portal. Medical reviews of older drugs are not always available and for one half

of the available ones, no precision estimates were provided in the medical reviews.

The impact of incomplete FDA trial data on our analysis was threefold. First, the

numbers of nodes in the two networks were different. Second, the sample size of

some treatments was small in the FDA network (e.g., betaxolol and placebo),

resulting in less precise estimates and lower power to detect the differences between

the two data sources if there were any. Third, the ‘star’ shape of the FDA network

limits the ability to assess inconsistency.

As treatment decisions are increasingly relying on the synthesis of all trials through

systematic reviews and meta-analyses, the value of trial data could not be fully

realized if there were missing parts, either missing the entire trial or missing

information on effect size. We strongly urge the FDA to consider the downstream

products of trials and how public is already using the medical reviews to address

medical and public health questions. This implies that medical reviews for all

approved drugs should be made available. Furthermore, the medical reviews should

be prepared following a consistent format and the data on treatment effect should be

reported completely. In particular, the precision estimates for the primary and

secondary outcomes (or endpoints) should be provided.

20

We found that the point estimates are similar but the relative rankings are different for

a small number of drugs included in our networks. The estimates based on the FDA

trials are less precise, which could contribute to the unstable rankings we observed. It

is important to interpret the two types of measures (relative effectiveness and rankings)

jointly in the context of NMA.

A subset of the FDA trials included in our analyses was also published. One might be

interested in comparing the details of the FDA trials with its matching publication to

explore disagreement, a traditional approach to examine selective outcome reporting.

For the same reason, when we compared the point estimates and relative rankings

between the two data sources, we did not account for the precisions because the two

data sources are not independent. Future work could focus on developing statistical

tests to facilitate such comparisons.

Our study has several strengths. First, this study provided an informative comparison

between the results based on published trials and FDA trials in the context of NMA,

which was not empirically examined before. We included head-to-head as well as

placebo-controlled trials. We answered a pragmatic question of how different the

results would be by using different data sources. In addition, NMA allowed the

examinations of the impact of data sources and the likelihood of reporting bias

between any two treatments in the networks regardless of whether the treatments were

21

compared directly or indirectly in individual studies.

In conclusion, FDA medical reviews provide a good amount of trial data that should

be considered for NMA. The point estimates for the relative effectiveness were

similar for most drugs regardless of whether FDA or published trial data are used in

the NMA, although the relative rankings changed for some drugs. Whether reporting

bias has a role in the differences we observed needs to be further evaluated.

22

5 References:

1. Guyatt, G., et al., Evidence-based medicine: A new approach to teaching the practice of

medicine. JAMA, 1992. 268(17): p. 2420-2425.

2. Guyatt, G.H., et al., Users&#39; guides to the medical literature: Ix. a method for grading

health care recommendations. JAMA, 1995. 274(22): p. 1800-1804.

3. Higgins JPT, G.S.e., Cochrane Handbook for Systematic Reviews of Interventions. The

Cochrane Collaboration, 2011.

4. Cipriani, A., et al., Conceptual and technical challenges in network meta-analysis. Ann Intern

Med, 2013. 159(2): p. 130-7.

5. Salanti, G., Indirect and mixed-treatment comparison, network, or multiple-treatments

meta-analysis: many names, many benefits, many concerns for the next generation evidence

synthesis tool. Research Synthesis Methods, 2012. 3(2): p. 80-97.

6. Dwan, K., et al., Comparison of protocols and registry entries to published reports for

randomised controlled trials. Cochrane Database Syst Rev, 2011(1): p. Mr000031.

7. Hopewell, S., et al., Publication bias in clinical trials due to statistical significance or

direction of trial results. Cochrane Database Syst Rev, 2009(1): p. Mr000006.

8. Scherer, R.W., P. Langenberg, and E. von Elm, Full publication of results initially presented in

abstracts. Cochrane Database Syst Rev, 2007(2): p. Mr000005.

9. Trinquart, L., A. Abbe, and P. Ravaud, Impact of reporting bias in network meta-analysis of

antidepressant placebo-controlled trials. PLoS One, 2012. 7(4): p. e35219.

10. Song, F., I. Harvey, and R. Lilford, Adjusted indirect comparison may be less biased than

direct comparison for evaluating new pharmaceutical interventions. J Clin Epidemiol, 2008.

61(5): p. 455-63.

11. Alkhafaji, A., et al., Impact of evergreening on patients and health insurance: a meta analysis

and reimbursement cost analysis of citalopram/escitalopram antidepressants. BMC Medicine,

2012. 10(1): p. 142.

12. Turner, E.H., et al., Selective Publication of Antidepressant Trials and Its Influence on

Apparent Efficacy. New England Journal of Medicine, 2008. 358(3): p. 252-260.

13. Turner, E.H., D. Knoepflmacher, and L. Shapley, Publication bias in antipsychotic trials: an

analysis of efficacy comparing the published literature to the US Food and Drug

Administration database. PLoS Med, 2012. 9(3): p. e1001189.

14. Li, T., et al., Comparative Effectiveness of First-line Medications for Patients with Primary

Open Angle Glaucoma or Ocular Hypertension – A Systematic Review and Network

Meta-analysis (in process).

15. Turner, E.H., How to access and process FDA drug approval packages for use in research.

BMJ, 2013. 347: p. f5992.

16. Li, T., et al., Innovations in data collection, management, and archiving for systematic reviews.

Ann Intern Med, 2015. 162(4): p. 287-94.

17. Weinreb, R.N. and P.L. Kaufman, Glaucoma Research Community and FDA Look to the

Future, II: NEI/FDA Glaucoma Clinical Trial Design and Endpoints Symposium: Measures of

Structural Change and Visual Function. Investigative Ophthalmology & Visual Science, 2011.

52(11): p. 7842-7851.

23

18. DerSimonian, R. and N. Laird, Meta-analysis in clinical trials. Control Clin Trials, 1986. 7(3):

p. 177-88.

19. Chaimani, A., et al., Graphical tools for network meta-analysis in STATA. PLoS One, 2013.

8(10): p. e76654.

20. Harris, R., et al., metan: fixed- and random-effects meta-analysis. Stata Journal, 2008. 8(1): p.

3-28.

21. White, I.R., Multivariate random-effects meta-regression: Updates to mvmeta. Stata Journal,

2011. 11(2): p. 255-270.

22. Lu, G. and A.E. Ades, Combination of direct and indirect evidence in mixed treatment

comparisons. Statistics in Medicine, 2004. 23(20): p. 3105-3124.

23. Lu, G. and A.E. Ades, Assessing Evidence Inconsistency in Mixed Treatment Comparisons.

Journal of the American Statistical Association, 2006. 101(474): p. 447-459.

24. van Valkenhoef, G., et al., Algorithmic parameterization of mixed treatment comparisons.

Statistics and Computing, 2012. 22(5): p. 1099-1111.

25. van Valkenhoef, G., et al., Automating network meta-analysis. Research Synthesis Methods,

2012. 3(4): p. 285-299.

26. Salanti, G., A.E. Ades, and J.P. Ioannidis, Graphical methods and numerical summaries for

presenting results from multiple-treatment meta-analysis: an overview and tutorial. J Clin

Epidemiol, 2011. 64(2): p. 163-71.

27. Higgins, J.P., et al., Measuring inconsistency in meta-analyses. Bmj, 2003. 327(7414): p.

557-60.

28. Higgins, J.P. and S.G. Thompson, Quantifying heterogeneity in a meta-analysis. Stat Med,

2002. 21(11): p. 1539-58.

29. Dias, S., et al., Checking consistency in mixed treatment comparison meta-analysis. Stat Med,

2010. 29(7-8): p. 932-44.

30. Dias, S., et al., Evidence synthesis for decision making 4: inconsistency in networks of

evidence based on randomized controlled trials. Med Decis Making, 2013. 33(5): p. 641-56.

31. Higgins, J.P.T., et al., Consistency and inconsistency in network meta-analysis: concepts and

models for multi-arm studies. Research Synthesis Methods, 2012. 3(2): p. 98-110.

32. Lumley, T., Network meta-analysis for indirect treatment comparisons. Stat Med, 2002.

21(16): p. 2313-24.

33. Dias, S., et al., Evidence Synthesis for Decision Making 4: Inconsistency in Networks of

Evidence Based on Randomized Controlled Trials. Medical Decision Making, 2013. 33(5): p.

641-656.

34. Tianjing Li, P.K.L., MS; Benjamin Rouse; Hwanhee Hong, PhD; Qiyuan Shi; David S.

Friedman, MD; Richard Wormald, FRCOphth; Kay Dickersin, PhD, Comparative

Effectiveness of First-line Medications for Patients with Primary Open Angle Glaucoma or

Ocular Hypertension – A Systematic Review and Network Meta-analysis (in process).

24

25

6 Appendices

Appendix 1. Search strategies

Cochrane Library #1 MeSH descriptor: [Glaucoma, Open-Angle] explode all trees

#2 MeSH descriptor: [Ocular Hypertension] explode all trees

#3 (open near/2 angle near/2 glaucoma*)

#4 (POAG or OHT)

#5 (((increas* or elevat* or high*) near/3 (ocular or intra-ocular)) and pressure)

#6 [34-#5]

#7 MeSH descriptor: [Adrenergic beta-Antagonists] explode all trees

#8 MeSH descriptor: [Timolol] explode all trees

#9 Timolol*

#10 MeSH descriptor: [Metipranolol] explode all trees

#11 Metipranolol*

#12 MeSH descriptor: [Carteolol] explode all trees

#13 Carteolol*

#14 MeSH descriptor: [Levobunolol] explode all trees

#15 Levobunolol*

#16 MeSH descriptor: [Betaxolol] explode all trees

#17 Betaxolol*

#18 MeSH descriptor: [Carbonic Anhydrase Inhibitors] explode all trees

#19 (Carbonic near/2 Anhydrase near/2 Inhibitor*)

#20 MeSH descriptor: [Acetazolamide] explode all trees

#21 Acetazolam*

#22 Brinzolamide*

#23 Dorzolamide*

#24 MeSH descriptor: [Prostaglandins, Synthetic] explode all trees

#25 latanoprost*

#26 travoprost*

#27 bimatoprost*

#28 unoprostone*

#29 tafluprost*

#30 MeSH descriptor: [Antihypertensive Agents] explode all trees

#31 MeSH descriptor: [Pilocarpine] explode all trees

#32 Pilocarpin*

#33 MeSH descriptor: [Epinephrine] explode all trees

#34 epinephrine*

#35 dipivefrin*

#36 MeSH descriptor: [Adrenergic alpha-2 Receptor Agonists] explode all trees

#37 (adrenergic near/2 alpha* near/3 agonist*)

#38 apraclonidin*

26

#39 brimonidine*

#40 (drug* or medic* or pharmacologic*) near/3 (treat* or therap* or intervent*)

#41 {or #7-#40}

#42 #6 and #41

MEDLINE (OVID) 1. exp clinical trial/ [publication type]

2. (randomized or randomised).ab,ti.

3. placebo.ab,ti.

4. dt.fs.

5. randomly.ab,ti.

6. trial.ab,ti.

7. groups.ab,ti.

8. or/1-7

9. exp animals/

10. exp humans/

11. 9 not (9 and 10)

12. 8 not 11

13. exp glaucoma open angle/

14. exp ocular hypertension/

15. (open adj2 angle adj2 glaucoma$).tw.

16. (POAG or OHT).tw.

17. (((increas$ or elevat$ or high$) adj3 (ocular or intra-ocular)) and pressure).tw.

18. or/13-17

19. exp adrenergic beta antagonists/

20. exp timolol/

21. timolol$.tw.

22. exp metipranolol/

23. metipranolol$.tw.

24. exp carteolol/

25. carteolol$.tw.

26. exp levobunolol/

27. levobunolol$.tw.

28. exp betaxolol/

29. betaxolol$.tw.

30. exp carbonic anhydrase inhibitors/

31. (carbonic adj2 anhydrase adj2 inhibitor$).tw.

32. exp Acetazolamide/

33. acetazolamide$.tw.

34. brinzolamide$.tw.

35. dorzolamide$.tw.

36. exp Prostaglandins, Synthetic/

37. latanoprost$.tw.

38. travoprost$.tw.

27

39. bimatoprost$.tw.

40. unoprostone$.tw.

41. brimonidine$.tw.

42. exp antihypertensive agents/

43. exp pilocarpine/

44. pilocarpin$.tw.

45. exp epinephrine/

46. epinephrin$.tw.

47. dipivefrin$.tw.

48. exp Adrenergic alpha-2 Receptor Agonists/

49. ((adrenergic adj2 alpha$ adj2 receptor$) or (adrenergic adj2 alpha$ adj2 agonist$)).tw.

50. apraclonidin$.tw.

51. tafluprost$.tw.

52. ((drug$ or medic$ or pharmacologic$) adj3 (treat$ or therap$ or intervent$)).tw.

53. or/19-52

54. 18 and 53

55. 12 and 54

Embase.com #1 'randomized controlled trial'/exp

#2 'randomization'/exp

#3 'double blind procedure'/exp

#4 'single blind procedure'/exp

#5 random*:ab,ti

#6 #1 OR #2 OR #3 OR #4 OR #5

#7 'animal'/exp OR 'animal experiment'/exp

#8 'human'/exp

#9 #7 AND #8

#10 #7 NOT #9

#11 #6 NOT #10

#12 'clinical trial'/exp

#13 (clin* NEAR/3 trial*):ab,ti

#14 ((singl* OR doubl* OR trebl* OR tripl*) NEAR/3 (blind* OR mask*)):ab,ti

#15 'placebo'/exp

#16 placebo*:ab,ti

#17 random*:ab,ti

#18 'experimental design'/exp

#19 'crossover procedure'/exp

#20 'control group'/exp

#21 'latin square design'/exp

#22 #12 OR #13 OR #14 OR #15 OR #16 OR #17 OR #18 OR #19 OR #20 OR #21

#23 #22 NOT #10

#24 #23 NOT #11

#25 'comparative study'/exp

28

#26 'evaluation'/exp

#27 'prospective study'/exp

#28 control*:ab,ti OR prospectiv*:ab,ti OR volunteer*:ab,ti

#29 #25 OR #26 OR #27 OR #28

#30 #29 NOT #10

#31 #30 NOT (#11 OR #23)

#32 #11 OR #24 OR #31

#33 'open angle glaucoma'/exp

#34 'intraocular hypertension'/exp

#35 (open NEAR/2 angle):ab,ti AND (angle NEAR/2 glaucoma*):ab,ti

#36 poag:ab,ti OR oht:ab,ti

#37 ((increas* OR elevat* OR high*) NEAR/3 (ocular OR 'intra ocular')):ab,ti AND

pressure:ab,ti

#38 #33 OR #34 OR #35 OR #36 OR #37

#39 'beta adrenergic receptor blocking agent'/exp

#40 'timolol'/exp

#41 timolol*:ab,ti

#42 'metipranolol'/exp

#43 metipranolol*:ab,ti

#44 'carteolol'/exp

#45 carteolol*:ab,ti

#46 'levobunolol'/exp

#47 levobunolol*:ab,ti

#48 'betaxolol'/exp

#49 betaxolol*:ab,ti

#50 'carbonate dehydratase inhibitor'/exp

#51 (carbonic NEAR/2 anhydrase):ab,ti AND (anhydrase NEAR/2 inhibitor*):ab,ti

#52 'acetazolamide'/exp

#53 acetazolamide*:ab,ti

#54 brinzolamide*:ab,ti

#55 dorzolamide*:ab,ti

#56 'latanoprost'/exp

#57 latanoprost*:ab,ti

#58 'travoprost'/exp

#59 travoprost*:ab,ti

#60 'bimatoprost'/exp

#61 bimatoprost*:ab,ti

#62 'unoprostone isopropyl ester'/exp

#63 unoprostone*:ab,ti

#64 'brimonidine'/exp

#65 brimonidine*:ab,ti

#66 'antihypertensive agent'/exp

#67 'pilocarpine'/exp

#68 pilocarpin*:ab,ti

29

#69 'adrenalin'/exp

#70 epinephrin*:ab,ti

#71 dipivefrin*:ab,ti

#72 'alpha 2 adrenergic receptor stimulating agent'/exp

#73 (adrenergic NEAR/2 alpha*):ab,ti AND (alpha* NEAR/2 agonist*):ab,ti

#74 apraclonidin*:ab,ti

#75 'tafluprost'/exp

#76 tafluprost*:ab,ti

#77 ((drug* OR medic* OR pharmacologic*) NEAR/3 (treat* OR therap* OR intervent*)):ab,ti

#78 #39 OR #40 OR #41 OR #42 OR #43 OR #44 OR #45 OR #46 OR #47 OR #48 OR #49 OR

#50 OR #51 OR #52 OR #53 OR #54 OR #55 OR #56 OR #57 OR #58 OR #59 OR #60 OR #61

OR #62 OR #63 OR #64 OR #65 OR #66 OR #67 OR #68 OR #69 OR #70 OR #71 OR #72 OR

#73 OR #74 OR #75 OR #76 OR #77

#79 #38 AND #78

#80 #32 AND #79

PubMed #1 ((randomized controlled trial[pt]) OR (controlled clinical trial[pt]) OR (randomised[tiab] OR

randomized[tiab]) OR (placebo[tiab]) OR (drug therapy[sh]) OR (randomly[tiab]) OR (trial[tiab])

OR (groups[tiab])) NOT (animals[mh] NOT humans[mh])

#2 (open[tw] AND angle[tw] AND glaucoma*[tw]) NOT Medline[sb]

#3 (POAG[tw] OR OHT[tw]) NOT Medline[sb]

#4 (((increase*[tw] OR elevat*[tw] OR high*[tw]) AND (ocular[tw] OR intra-ocular[tw])) AND

pressure[tw]) NOT Medline[sb]

#5 #2 OR #3 OR #4

#6 timolol*[tw] NOT Medline[sb]

#7 metipranolol*[tw] NOT Medline[sb]

#8 carteolol*[tw] NOT Medline[sb]

#9 levobunolol*[tw] NOT Medline[sb]

#10 betaxolol*[tw] NOT Medline[sb]

#11 (carbonic[tw] AND anhydrase[tw] AND inhibitor*[tw]) NOT Medline[sb]

#12 acetazolamide*[tw] NOT Medline[sb]

#13 brinzolamide*[tw] NOT Medline[sb]

#14 dorzolamide*[tw] NOT Medline[sb]

#15 latanoprost*[tw] NOT Medline[sb]

#16 travoprost*[tw] NOT Medline[sb]

#17 bimatoprost*[tw] NOT Medline[sb]

#18 unoprostone*[tw] NOT Medline[sb]

#19 brimonidine*[tw] NOT Medline[sb]

#20 pilocarpin*[tw] NOT Medline[sb]

#21 epinephrin*[tw] NOT Medline[sb]

#22 dipivefrin* NOT Medline[sb]

#23 ((adrenergic[tw] AND alpha*[tw] AND receptor*[tw]) OR (adrenergic[tw] AND alpha*[tw]

AND agonist*[tw])) NOT Medline[sb]

30

#24 apraclonidin*[tw] NOT Medline[sb]

#25 tafluprost*[tw] NOT Medline[sb]

#26 ((drug*[tw] OR medic*[tw] OR pharmacologic*[tw]) AND (treat*[tw] OR therap*[tw] OR

intervent*[tw])) NOT Medline[sb]

#27 #6 OR #7 OR #8 OR #9 OR #10 OR #11 OR #12 OR #13 OR #14 OR #15 OR #16 OR #17

OR #18 OR #19 OR #20 OR #21 OR #22 OR #23 OR #24 OR #25 OR #26

#28 #5 AND #27

#29 #1 AND #28

FDA (Drugs@FDA) #1. Acetazolamide

#2. Brimonidine

#3. Betaxolol

#4. Dichlorphenamide

#5. Levobunolol

#6. Timolol

#7. Brinzolamide

#8. Dorzolamide

#9. Bimatoprost

#10. Latanoprost

#11. Travoprost

#12. Unoprostone

#13. Acetazolamide

#14. Methazolamide

#15. Pilocarpine

#16. Carbachol

#17. Echothiophate

#18. Epinephrine

#19. Dipivefrin

#20. Metipranolol

#21. Demecarium

#22. Apraclonidine

#23. Carteolol

#24. Tafluprost

31

7 Tables

Table 1. Summary estimates for intraocular pressure at 3 months derived

from pair-wise meta-analysis based on 13 direct comparisons from 16 FDA

trials

Comparison-specific heterogeneity

Num. of trials Mean difference 95% CI, lower 95% CI, upper Tau-squared I-squared Mean difference 95% CI, lower 95% CI, upper

Placebo vs.

1 -3.00 -4.53 -1.47 NA NA -4.32 -6.56 -2.08

1 -1.00 -2.76 0.76 NA NA -2.32 -4.84 0.19

1 -2.70 -4.44 -0.96 NA NA -4.02 -6.52 -1.53

Timolol vs.

3 1.32 0.63 2.00 0.12 24% 1.32 0.49 2.15

2 -2.26 -2.82 -1.70 0.00 0% -3.58 -4.75 -2.41

2 2.00 1.59 2.41 0.00 0% 0.67 -0.39 1.73

3 1.25 0.27 2.23 0.52 73% 0.14 -0.95 1.22

3 -0.98 -1.45 -0.52 0.03 19% -2.31 -3.35 -1.27

1 -1.10 -1.99 -0.21 NA NA -2.42 -3.95 -0.90

Betaxolol vs.

1 -2.00 -3.54 -0.46 NA NA -3.32 -5.57 -1.07

1 -1.70 -3.45 0.05 NA NA -3.02 -5.53 -0.52

Brinzolamide vs.

2 0.22 -0.37 0.80 0.06 32% -1.11 -2.21 0.00

Latanoprost vs.

1 -0.30 -1.03 0.43 NA NA -1.62 -2.99 -0.25

Tau square: between-study variance in random-effect models; I square: proportion of variance due to heterogeneity

*Estimated tau square is 0.039; estimated I square is 36.03%

Common heterogeneity*

Levobetaxolol

Betaxolol

Timolol

Dorzolamide

Bimatoprost

Dorzolamide

Travoprost

Unoprostone

Levobetaxolol

Travoprost

Latanoprost

Levobetaxolol

Timolol

32

Table 2. Summary estimates for intraocular pressure at 3 months derived

from network meta-analysis of 16 FDA trials

Placebo

1.01 Betaxolol

(-0.93;2.95)

3.76 2.75 Timolol

(2.09;5.40) (1.05;4.42)

2.36 1.35 -1.40 Levobetaxolol

(0.75;3.98) (-0.3;3.03) (-2.06;-0.62)

2.66 1.65 -1.10 0.30 Brinzolamide

(0.67;4.59) (-0.35;3.59) (-2.15;-0.06) (-1.02;1.51)

2.45 1.44 -1.31 0.09 -0.21 Dorzolamide

(0.61;4.24) (-0.42;3.24) (-2.04;-0.58) (-0.98;1.06) (-0.97;0.55)

6.03 5.02 2.27 3.67 3.37 3.58 Bimatoprost

(4.17;7.85) (3.13;6.87) (1.45;3.07) (2.53;4.68) (2.05;4.69) (2.5;4.67)

4.60 3.59 0.84 2.24 1.94 2.15 -1.43 Latanoprost

(2.61;6.56) (1.58;5.56) (-0.23;1.94) (0.91;3.49) (0.44;3.46) (0.86;3.46) (-2.75;-0.06)

4.75 3.74 0.99 2.39 2.09 2.30 -1.28 0.15 Travoprost

(2.95;6.5) (1.92;5.53) (0.36;1.62) (1.37;3.28) (0.87;3.31) (1.35;3.26) (-2.30;-0.25) (-0.91;1.18)

1.77 0.76 -1.99 -0.59 -0.89 -0.68 -4.26 -2.83 -2.98 Unoprostone

(-0.06;3.55) (-1.09;2.57) (-2.71;-1.26) (-1.67;0.36) (-2.16;0.39) (-1.70;0.34) (-5.33;-3.17) (-4.14;-1.55) (-3.94;-2.02)

Grey

Green

Red

Blue

Beta-blocker

Carbonic anhydrase inhibitor

Prostaglandin analog

Unit: mmHg; Color coding: drug class

Mean difference < 0 favors the drug in the coloum

Mean difference > 0 favors the drug in the row

Reported numbers are calculated by column - row under the Lu and Ades homogeneous random effects model assuming consistency

Reported posterior means and 95% Bayesian credible intervals

Placebo/vehicle/no treatment

33

Table 3. Summary estimates for intraocular pressure at 3 months derived

from pair-wise meta-analysis based on 36 direct comparisons from 105

published trials

Comparison-specific heterogeneity

Num. of trials Mean difference 95% CI, lower 95% CI, upper Tau-squared I-squared Mean difference 95% CI, lower 95% CI, upper

Brimonidine 1 -2.30 -3.99 -0.61 NA NA -0.41 -2.86 2.04

Betaxolol 2 -3.90 -5.29 -2.52 1.30 81% -0.89 -2.48 0.69

Levobunolol 2 -7.52 -8.50 -6.50 0.00 0% -5.56 -7.36 -3.75

Timolol 4 -3.91 -5.12 -2.69 0.85 57% -2.00 -3.47 -0.52

Brinzolamide 2 -2.17 -3.23 -1.10 0.00 0% -0.29 -2.09 1.51

Dorzolamide 4 -1.91 -2.92 -0.90 0.51 51% -1.89 -2.90 -0.88

Bimatoprost 1 -4.60 -5.60 -3.60 NA NA -2.71 -4.70 -0.72

Unoprostone 1 -0.20 -1.56 1.16 NA NA 1.69 -0.52 3.90

Apraclonidine vs.

Timolol 2 -0.84 -3.75 2.08 3.73 84% 0.66 -1.12 2.45

Brimonidine vs.

Betaxolol 1 1.94 0.84 3.04 NA NA 3.83 1.78 5.88

Timolol 4 0.17 -0.70 1.03 0.55 81% 2.06 0.76 3.37

Brinzolamide 2 1.01 0.50 1.53 0.00 0% 2.89 1.40 4.39

Latanoprost 5 -1.36 -2.21 -0.50 0.73 78% 0.51 -0.74 1.77

Travoprost 1 -1.20 -3.77 1.37 NA NA 0.69 -2.49 3.87

Levobunolol 2 -2.00 -3.54 -0.46 NA NA -1.41 -3.54 0.72

Timolol 8 -1.58 -2.29 -0.87 0.43 48% 0.31 -0.95 1.57

Dorzolamide 2 -0.30 -0.96 0.36 0.00 0% 1.54 -0.02 3.11

Latanoprost 2 -1.06 -2.62 0.51 0.33 25% 0.84 -1.15 2.83

Unoprostone 1 0.60 0.09 1.11 NA NA 2.49 0.71 4.26

Levobunolol 1 -2.90 -4.59 -1.22 NA NA -1.01 -3.46 1.44

Timolol 4 0.03 -0.61 0.68 0.11 24% 1.95 0.58 3.31

Levobunolol vs.

Timolol 11 -0.03 -0.44 0.39 0.01 3% 1.88 0.68 3.08

Brinzolamide 3 1.10 0.50 1.70 0.00 0% 2.82 1.32 4.33

Dorzolamide 5 0.76 0.13 1.39 0.24 47% 2.65 1.38 3.93

Bimatoprost 5 -2.07 -2.64 -1.49 0.15 35% -0.28 -1.57 1.01

Latanoprost 14 -1.32 -1.77 -0.88 0.40 64% 0.56 -0.56 1.68

Travoprost 5 -1.22 -2.20 -0.24 0.79 67% 0.67 -0.66 1.99

Tafluprost 1 -0.30 -0.72 0.12 NA NA 1.59 -0.16 3.34

Unoprostone 2 0.94 -0.43 2.31 0.85 87% 2.86 1.37 4.34

Brinzolamide vs.

Dorzolamide 2 -0.58 -1.15 0.00 0.00 0% 1.30 -0.23 2.82

Dorzolamide vs.

Latanoprost 1 -2.90 -3.70 -2.10 NA NA -1.01 -2.90 0.88

Bimatoprost vs.

Latanoprost 6 0.87 0.01 1.73 0.82 76% 2.75 1.50 3.99

Travoprost 8 0.59 -0.13 1.30 0.73 74% 2.48 1.28 3.67

Latanoprost vs.

Travoprost 7 -0.22 -0.86 0.41 0.33 48% 1.68 0.44 2.91

Tafluprost 1 -0.90 -3.40 1.60 NA NA 0.99 -2.14 4.11

Unoprostone 6 3.07 2.51 3.63 0.01 2% 4.85 3.53 6.16

Tau square: between-study variance in random-effect models; I square: proportion of variance due to heterogeneity

*Estimated tau squared is 0.4374; estimated I-squared is 59.63%

Common heterogeneity*

Placebo vs.

Betaxolol vs.

Carteolol vs.

Timolol vs.

34

Table 4. Summary estimates for intraocular pressure at 3 months derived from network meta-analysis of 105 published trials

Placebo

2.55 Apraclonidine

(0.89;4.23)

3.62 1.07 Brimonidine

(2.87;4.37) (-0.60;2.72)

2.34 -0.21 -1.28 Betaxolol

(1.63;3.05) (-1.89;1.45) (-2.01;-0.54)

3.44 0.89 -0.18 1.09 Carteolol

(2.36;4.52) (-0.94;2.7) (-1.26;0.89) (0.02;2.17)

4.54 1.99 0.92 2.19 1.10 Levobunolol

(3.77;5.31) (0.31;3.65) (0.11;1.73) (1.43;2.97) (0.06;2.15)

3.70 1.15 0.08 1.36 0.27 -0.84 Timolol

(3.11;4.29) (-0.42;2.7) (-0.48;0.65) (0.79;1.93) (-0.65;1.18) (-1.43;-0.25)

2.46 -0.09 -1.16 0.11 -0.98 -2.08 -1.25 Brinzolamide

(1.61;3.31) (-1.84;1.64) (-1.97;-0.35) (-0.79;1.02) (-2.16;0.21) (-3.03;-1.14) (-2.01;-0.48)

2.56 0.01 -1.06 0.22 -0.87 -1.98 -1.14 0.10 Dorzolamide

(1.86;3.27) (-1.67;1.68) (-1.83;-0.28) (-0.52;0.95) (-1.97;0.22) (-2.80;-1.16) (-1.74;-0.53) (-0.74;0.95)

5.65 3.10 2.03 3.30 2.21 1.11 1.94 3.19 3.08 Bimatoprost

(4.89;6.40) (1.43;4.75) (1.29;2.76) (2.54;4.07) (1.14;3.28) (0.3;1.9) (1.39;2.5) (2.27;4.1) (2.29;3.88)

4.88 2.33 1.26 2.54 1.44 0.34 1.18 2.42 2.32 -0.76 Latanoprost

(4.22;5.54) (0.71;3.93) (0.69;1.84) (1.9;3.18) (0.45;2.44) (-0.36;1.04) (0.79;1.57) (1.61;3.23) (1.64;2.99) (-1.32;-0.21)

5.02 2.47 1.40 2.67 1.58 0.48 1.31 2.56 2.45 -0.63 0.13 Travoprost

(4.23;5.80) (0.80;4.12) (0.65;2.14) (1.89;3.46) (0.49;2.66) (-0.35;1.29) (0.74;1.89) (1.63;3.48) (1.64;3.26) (-1.2;-0.05) (-0.43;0.7)

4.41 1.86 0.79 2.07 0.97 -0.13 0.71 1.95 1.85 -1.24 -0.47 -0.61 Tafluprost

(2.88;5.97) (-0.26;3.98) (-0.73;2.34) (0.54;3.63) (-0.72;2.68) (-1.67;1.43) (-0.72;2.15) (0.34;3.58) (0.31;3.42) (-2.75;0.31) (-1.92;1.00) (-2.13;0.94)

1.98 -0.57 -1.64 -0.36 -1.45 -2.55 -1.72 -0.47 -0.58 -3.66 -2.90 -3.03 -2.43 Unoprostone

(1.13;2.84) (-2.29;1.14) (-2.48;-0.79) (-1.20;0.48) (-2.61;-0.30) (-3.47;-1.65) (-2.43;-1.01) (-1.48;0.53) (-1.47;0.31) (-4.51;-2.82) (-3.58;-2.21) (-3.88;-2.18) (-4.03;-0.85)

Grey

Gold

Green

Red

Blue

Alpha-2 adrenergic agonist

Placebo/vehicle/no treatment

Beta-blocker

Carbonic anhydrase inhibitor

Prostaglandin analog

Unit: mmHg; Color coding: drug class

Mean difference < 0 favors the drug in the coloum

Mean difference > 0 favors the drug in the row

Reported numbers are calculated by column - row under the Lu and Ades homogeneous random effects model assuming consistency

Reported posterior means and 95% Bayesian credible intervals

35

Table 5. Comparisons between the results of direct pairwise meta-analyses based on FDA trials and published trials

N Effect size (95%CI) Tau-squared I-squared N Effect size (95%CI) Tau-squared I-squared

1 -1.00 (-2.76; 0.76) NA NA 2 -3.90 (-5.29;-2.52) 1.30 81%

1 -2.70 (-4.44;-0.96) NA NA 4 -3.91 (-5.12;-2.69) 0.85 57%

3 1.32 (0.63; 2.00) 0.12 24% 5 0.76 (0.13;1.39) 0.24 47%

2 -2.26 (-2.82;-1.70) 0.00 0% 5 -2.07 (-2.64;-1.49) 0.15 35%

2 2.00 (1.59;2.41) 0.00 0% 3 1.35 (0.42;2.27) 0.57 86%

3 -0.98 (-1.45;-0.52) 0.03 19% 5 -1.22 (-2.20; -0.24) 0.79 67%

1 -1.10 (-1.99;-0.21) NA NA 14 -1.32 (-1.77; -0.88) 0.40 64%

1 1.70 (0.05;-3.45) NA NA 8 1.58 (0.87;-2.29) 0.43 48%

Brinzolamide vs.

2 0.22 (-0.37;0.80) 0.06 32% 2 -0.58 (-1.15;0.00) 0.00 0%

Latanoprost vs.

1 -0.30 (-1.03;0.43) NA NA 7 -0.22 (-0.86; 0.41) 0.33 48%

Dorzolamide

Travoprost

Dorzolamide

Bimatoprost

Unoprostone

Travoprost

Latanoprost

Betaxolol

Pairwise comparisons based on FDA trials Pairwise comparisons based on published trials

Placebo vs.

Betaxolol

Timolol

Timolol vs.

36

8 Figures

Figure 1. Identification of trials

37

Figure 2. Network graphs

38

Figure 3. Ranking probabilities for any drug at any position in the FDA trials network

Bimatoprost Travoprost Latanoprost Timolol Brinzolamide Dorzolamide Levobetaxolol Betaxolol Unoprostone Placebo

10 0 0 0 0 0.002 0.001 0 0.148 0.016 0.832

9 0 0 0 0 0.013 0.011 0.008 0.645 0.175 0.148

8 0 0 0 0 0.049 0.068 0.098 0.124 0.646 0.014

7 0 0 0.001 0 0.124 0.27 0.461 0.036 0.106 0.003

6 0 0 0.002 0.001 0.224 0.508 0.2 0.025 0.039 0.002

5 0 0.001 0.006 0.02 0.564 0.142 0.23 0.02 0.016 0.001

4 0 0.003 0.048 0.925 0.019 0.001 0.003 0.001 0 0

3 0.005 0.367 0.57 0.053 0.004 0 0.001 0 0 0

2 0.022 0.621 0.354 0.001 0.001 0 0 0 0 0

1 0.972 0.008 0.02 0 0 0 0 0 0 0

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

10

9

8

7

6

5

4

3

2

1

39

Figure 4. Ranking probabilities for any drug at any position in the published trials network

40

Figure 5. Scatterplot of relative effect size between two treatments based on both FDA and published trials

41

Figure 6. Relative effect size based on FDA trials and published trials

42

Figure 7. Relative effect size between active drugs and placebo/timolol based on FDA trials and published trials

43

Figure 8. Percentage ‘SUCRA’ of drugs in the two networks

44

Curriculum vitae

PERSONAL

Qiyuan Shi was born on Oct. 21st 1990 in Shanghai, China.

EDUCATION

Johns Hopkins University School of Public Health

09/2013-05/2015

Degree: Master of Health Science

Concentration: Epidemiology, Clinical trials and Evidence Synthesis

Relevant Coursework: Epidemiology, Biostatistics, Systematic Review and

Meta-analysis, Statistical Computing, Clinical Trials Management,

Pharmaco-epidemiology, Survival Analysis, Longitudinal Analysis.

Fudan University School of Pharmacy

09/2009-06/2013

Major: Pharmacy

Degree: Bachelor of Science

PROFESSIONAL DEVELOPMENT

Software & Languages: Microsoft Office Suite, SAS (Certified Base Programmer),

R, STATA, ArcGIS

Typesetting: Latex

Language: English & Mandarin

PUBLIC HEALTH EXPERIENCE

Research Assistant, Cochrane Eyes and Vision Group Johns Hopkins University

Center for Clinical Trials, Department of Epidemiology, School of Public Health,

Johns Hopkins University

02/2014—present

Search, collect, manage and screen clinical research studies from different

databases

Extract and manage data from studies

Conduct qualitative and quantitative analysis

Research Assistant, Cochrane Hypertension Group Fudan University

Department of Clinical Pharmacology, School of Pharmacy, Fudan University

06/2012—06/2013

Search, collect, manage and screen clinical research studies from different

databases

45

Extract and manage data from studies

Conduct qualitative and quantitative analysis

TEACHING EXPERIENCE

Teaching assistant and lab instructor, Department of Epidemiology &

Department of Biostatistics, Johns Hopkins University

09/2014—12/2014

INTERNSHIP EXPERIENCE

Pharmacist

Zhongshan Hospital, Shanghai, China

07/2012-12/2012