HAL Id: hal-00477858https://hal.archives-ouvertes.fr/hal-00477858

Submitted on 30 Apr 2010

HAL is a multi-disciplinary open accessarchive for the deposit and dissemination of sci-entific research documents, whether they are pub-lished or not. The documents may come fromteaching and research institutions in France orabroad, or from public or private research centers.

L’archive ouverte pluridisciplinaire HAL, estdestinée au dépôt et à la diffusion de documentsscientifiques de niveau recherche, publiés ou non,émanant des établissements d’enseignement et derecherche français ou étrangers, des laboratoirespublics ou privés.

Hazard ratio funnel plots for Survival ComparisonsPaul Silcocks

To cite this version:Paul Silcocks. Hazard ratio funnel plots for Survival Comparisons. Journal of Epidemiology andCommunity Health, BMJ Publishing Group, 2009, 63 (10), pp.856-n/a. �10.1136/jech.2008.075069�.�hal-00477858�

Address for correspondence: Paul Silcocks, Medical Advisor Trent Cancer Registry 5 Old Fulwood Road Sheffield S10 3TG England Tel 0114 226 3563 Fax 0114 226 3561 e-mail paul.silcocks@trentcancer.nhs.uk Key words: survival analysis, funnel plots, over-dispersion Word count: 2751 Running title: Hazard ratio funnel plot

Hazard ratio funnel plots for Survival Comparisons

Paul Silcocks Trent Cancer Registry,

And Trent RDSU, Nottingham

The Corresponding Author has the right to grant on behalf of all authors and does grant on behalf of all authors, an exclusive licence (or non exclusive for government employees) on a worldwide basis to the BMJ Publishing Group Ltd and its Licensees to permit this article (if accepted) to be published in (Journal name) and any other BMJPGL products to exploit all subsidiary rights, as set out in our licence (http://heart.bmjjournals.com/ifora/licence.pdf).

ABSTRACT

Objectives: Description of Cox regression model for using hazard ratios to compare

survival between institutions with adjustment for covariates and allowance for over-

dispersion.

Design: Analysis of simulated and real survival data

Setting: Applicable to evaluation of clinician or institutional performance, but illustrated

using populations at Primary Care Trust & Local Authority level.

Results: It describes how centred hazard ratio estimates adjusted for covariates can be

obtained from a Cox regression and gives details of the necessary programming in Stata.

Allowance for over-dispersion can be made by multiplying the standard errors by a factor

based on either the model or the log-rank chi-squared statistics. Simulated results and a

real example are presented.

Conclusion: Funnel plots based on Hazard ratios are easier to interpret than multiple

Kaplan-Meier survival plots, and in contrast to funnel plots based on survival at, say, 5

years, are less open to accusations of bias and use more information. The interpretation

of such plots may be enhanced by using standard meta-analysis methods. Hazard ratio

comparisons may now be added to the repertoire of techniques used by Cancer Registries,

Primary Care Trusts, and other commissioners of Health Care

INTRODUCTION

A range of statistical procedures exists to ensure quality control of manufacturing

processes. One of the simplest is a control chart, in which a measure of some critical

aspect of quality for each item manufactured is plotted against time, with boundaries to

indicate when quality has become unacceptable – that is, when the process is “out of

control”. Control charts are closely related to hypothesis tests, in that the control chart

tests the null hypothesis that the process is in statistical control.

Although funnel plots have been used as evidence in assessing publication bias since the

mid 1980s [1], their value as a form of control chart for graphically comparing

institutional performance was suggested by Spiegelhalter only relatively recently [2].

The difference between this method and the usual control chart is that the funnel plot

gives a snapshot of many institutions at a particular moment in time, as opposed to a

measuring performance of a single institution at many moments in time over a defined

period. In his paper Spiegelhalter described how control limits could be set for binomial

proportions (including various functions of proportions including changes, ratios and

odds ratios), standardised rates, ratios of SMRs and continuous response data, but not for

survival data.

When survival is to be compared between institutions, comparison at only one time point

involves picking an arbitrary time (eg at 5 years) which may hide a real difference that

would be obvious if the whole survival curves were compared [3, 4]. Moreover the

proportion surviving at a particular time has a relatively large standard error, whereas an

overall comparison that reflects the whole survival curve, such as one based on a

proportional hazards model, might detect a significant difference [3]. A point mentioned

by a reviewer is that some restriction of follow-up, for example to 5 years post-diagnosis

(or less than this for some purposes), is advisable, to avoid potential biased (or

diluted/attenuated) comparisons if later follow-up is available for some groups. While a

proportional hazards model may not be suitable for detecting every kind of discrepancy

between survival curves, it is reasonable for a control chart in which the null assumption

is that the process is “in control” and therefore the survival curves for each institution

should differ simply by chance (in addition with a restricted follow-up the proportional

hazards model is more likely to hold at least approximately).

While a Cox regression could be used to estimate and formally compare the hazard ratios,

two points need to be considered. Firstly, such a comparison uses one of the institutions

as a reference category, so to obtain a hazard ratio for each institution the hazard ratios

(or rather their logarithms) need to be centred – that is, estimated with reference to a

common value. This transformation will also require a transformation of the covariance

matrix of the regression coefficients corresponding to the log hazard ratios for each

institution. Secondly the control limits correspond to confidence limits assuming the

null hypothesis is true (ie that survival curves differ only by chance), and therefore the

reported covariance matrix of the regression coefficients is inappropriate.

METHOD

If there are three centres each represented by a 0/1 indicator variable, the proportional

hazards model can be represented as:

332211)ratio hazardln( DDD βββ ++=

In which D3 - the indicator for the third (and reference) centre - can effectively be

ignored provided its corresponding regression coefficient β3 is constrained to be zero.

The log hazard ratios are to be shifted so that they sum to a value of zero (which, if the

process is in control, should also be close to their common mean). The principle

involved in shifting them is as follows:

Define:

3

)( 321 βββγ ++=

Then the shifted coefficients are:

γγββγββγββ

−=−=

−=

−=

3*3

2*2

1*1

(the last because β3 is assumed to be constrained to be zero). However it is also

necessary to make a corresponding transformation to the variance-covariance matrix, and

in particular the variance-covariance matrix evaluated under the null hypothesis of no

between-centre differences. This is because the funnel plot can be thought of as a way of

displaying significance test results.

These processes are explained at greater length in the appendix, using matrix algebra

together with some additional tricks required to extract the required values from

computer output.

Covariates

Typically allowance needs to be made for covariates that affect case-mix - age, stage and

treatment being likely candidates. While in the Cox regression these variables can be

adjusted for by inclusion as covariates, in the log-rank test it is necessary to adjust for

these variables by stratification. Because the covariance matrix under the null hypothesis

is obtained using the log-rank test, for consistency the same stratification is employed in

the Cox regression to estimate the log hazard ratios.

Estimating and handling overdispersion

If overdispersion is present, the variance of the hazard ratios will be greater than

predicted by the above model. The consequence is that at first sight a higher proportion

of centres may appear to be out of control than is really the case. Spiegelhalter [6]

suggested a number of approaches to handling overdispersion – the simplest being

estimation of a dispersion factor that multiplies the variance of the quantity for which

control limits are to be found. This dispersion factor φ (a single value common for all k

institutions), may be estimated as the value of the logrank chi-squared calculated in the

usual way, divided by its degrees of freedom. Note that the over-dispersion factor

modifies the control limits, not the log hazard ratios actually observed and plotted.

The standard approach is to regard values of φ less than one as equal to one (that is,

under-dispersion is ignored) – the same assumption as is made in the Der Simonian and

Laird [7] estimate of between-centre variance employed in meta-analysis, so that the

basic estimate of overdispersion is given by:

>=

otherwise

if

1

ˆ2

2

νχνχ

φ νν

where υ is the degrees of freedom for the chi-squared test.

One approach to guard against assuming overdispersion when this is just a chance finding

is to set the estimate φ̂ >1 only if the 2χ is “statistically significant”. However “absence

of evidence is not evidence of absence” and in addition the statistical significance will

reflect both the extent of overdispersion and the precision of the hazard ratio estimates

within each centre.

A better approach suggested by Spiegelhalter [6] is to use a Winsorised estimate of φ .

This is a method of robust estimation that is less sensitive to outliers. For this the upper

and lower most extreme x% of the values in an observed set of values are replaced by the

xth and 100-xth percentiles to produce a 2x% Winsorised estimate. The estimated

overdispersion parameter φ̂ can be written as: 1

ˆ

1

1

2

−=∑

−

=

k

zk

ii

φ where k-1 is the degrees of

freedom and the 2iz denote the squared standard Normal deviates for each regression

coefficient (excluding the kth, reference, value). Spiegelhalter suggested Winsorising the

z-scores, so that the Winsorised estimate of φ is given by: 1

ˆ

1

1

2*

*

−=∑

−

=

k

zk

ii

φ where the

asterisk indicates that the Winsorised values are used.

Clearly this process must be used cautiously and typically 10% (5th and 95th percentiles)

or at most 20% (10th and 90th percentiles) Winsorising is applied. If only the two extreme

observations are Winsorised then 20% Winsorising implies that there must be at least 10

centres, which is close to the advice on the minimum number of degrees of freedom

(around 12) needed to estimate a variance reliably [8].

Note that the Winsorising is only needed to estimate φ and the un-Winsorised but centred

log hazard ratio estimates are displayed in the funnel plot.

It is arguable that a Winsorised estimate of φ should be used routinely because one

cause of overdispersion is the existence of unmeasured prognostic variables that are

consequently not adjusted for in the analysis. Such an inadequate case-mix adjustment is

probably the norm in the context of routinely available statistics. One prognostic

variable that it is typically possible to allow for is age at diagnosis. When stratifying for

age however, the definition of the age bands used should be declared in advance because,

for instance, the resulting funnel plot based on equal width age bands may be very

different from one in which age bands are defined on the basis of equal numbers of

events.

The main distinction to be made on the basis of the funnel plot is between a) general

variation caused by factors common to all centres such as case-mix (and which may be

addressed by allowing for overdispersion) b) the presence of one or two extreme outliers

that may require investigation and which may be detected by Winsorising so that such

observations don’t influence the estimated standard error, c) skewness in the distribution

of random effects, for which Winsorising may be ineffective as the effects are spread

across many centres. For the latter, one possibility would be to use tests for asymmetry

in funnel plots devised for meta-analysis, [9, 10].

Practical implementation

In Stata, the covariance matrix of the score statistic can be accessed by using sts test, with

the logrank option.

1. Assuming that the institutions being compared are labelled 1 to k, with the reference

category being the first - delete the first row and column (this is necessary because the

covariance matrix is singular)

2. invert the resulting matrix

3. augment this matrix with a new first row and column of zeroes, to give the covariance

matrix V0 of the regression coefficients under the null hypothesis

4. transform V0 as described above

5. take the square root of each diagonal element of V0 to give the standard error of each

regression coefficient under the null hypothesis,

6. identify extent of overdispersion and adjust standard errors if necessary

7. calculate control limits

8. apply tests for bias in funnel plot

Code fragments to perform these steps are given in the appendix.

.

Once the control limits have been calculated it is easy to plot these with the centre-

specific log hazard ratios, using the reciprocal of the standard error under the null

hypothesis as a measure of precision, and incorporating Stata meta-analysis add-on

commands.

RESULTS

Table 1 displays the centred log hazard ratios for simulated survival data for ten centres,

roughly mimicking “old” Health Districts, with stratification for a case-mix covariate in

seven bands. The standard errors (of the point estimates under H1), plus 95% and 99%

control limits are also given, with and without adjustment for overdispersion (which was

induced by including the effect of a centre-level random variable not allowed for in the

stratified analysis). For this small data set the Winsorisation did not alter phi. An average

of 10 deaths per centre was used in accordance with the 10:1 number of events/variable

rule [8].

Without over-dispersion With over-dispersion

Centre Centred log HR

Se|H1 LCL (95%)

UCL (95%)

LCL (99%)

UCL (99%)

LCL (95%)

UCL (95%)

LCL (99%)

UCL (99%)

1 -0.739

0.553 -0.803 0.803 -1.055 1.055 -1.583 1.583 -2.081 2.081

2 -0.005

0.333 -0.624 0.624 -0.820 0.820 -1.231 1.231 -1.618 1.618

3 0.904

0.274 -0.724 0.724 -0.951 0.951 -1.427 1.427 -1.876 1.876

4 -0.808

0.389 -0.532 0.532 -0.699 0.699 -1.049 1.049 -1.379 1.379

5 -0.490

0.362 -0.563 0.563 -0.740 0.740 -1.111 1.111 -1.460 1.460

6 -0.140

0.310 -0.562 0.562 -0.739 0.739 -1.108 1.108 -1.457 1.457

7 0.261

0.310 -0.663 0.663 -0.871 0.871 -1.307 1.307 -1.717 1.717

8 -0.074

0.304 -0.564 0.564 -0.742 0.742 -1.113 1.113 -1.463 1.463

9 1.087

0.280 -0.782 0.782 -1.027 1.027 -1.542 1.542 -2.026 2.026

10 0.005

0.296 -0.563 0.563 -0.740 0.740 -1.111 1.111 -1.460 1.460

Unadjusted for

casemix Adjusted for

casemix

2χ (9 df)

37.1 P <0.0001

35.0 P =0.0001

φ = 3.89

Figure 1a shows corresponding conventional survival curves by centre, while in figure 1b

these have been adjusted for casemix. The adjusted curves are closer together than the

unadjusted ones, but in fact the chi-squared test is not much less significant.

Figures 2a and 2b display funnel plots for the data in table 1 (without and with allowance

for overdispersion). Figure 3 on the other hand displays results allowing for

overdispersion when this has arisen from a positively skewed distribution for the

between-centre random effect. While figure 3 superficially resembles figure 2b, the

statistics for the asymmetry tests are non-significant for the data in table 1 (continuity

corrected Begg’s test P = 0.118; Egger’s test P = 0.129) but are both significant for figure

3 (continuity corrected Begg’s test P = 0.029; Egger’s test P = 0.001). Figure 4

displays the Egger plot corresponding to figure 3.

Note that the measure of precision used for these analyses is the reciprocal of the

standard error under H0. The justification is that, as described by Tang and Liu, the

observed standard error is a function of both risk and sample size. Tang and Liu

suggested that funnel plots using sample size as a precision measure might be preferable;

for log hazard ratios under H0 the standard error is inversely related to the total number

of events (and is thus independent of the hazard ratio estimate).

Figures 5a and 5b give a real example, using breast cancer survival data from Local

Authorities (LAs) in the East Midlands. In both cases the results are age-adjusted with a

20% Winsorised allowance for over-dispersion. In figure 5a the five age-bands used

have equal numbers of cases, whereas in figure 5b they have equal numbers of deaths.

When age-bands have equal numbers of cases, then LA number 6 lies outside the upper

99% boundary, LA number 7 lies on lower 99% boundary, and LAs 10 & 8 lie between

the upper 95% & 99% bounds. However if the age bands have equal numbers of deaths,

then LAs 30 and 34 lie just on the lower 99% limits, but no LA lies beyond the 99%

limits.

For figure 5a, asymmetry tests fail to show evidence of skewness of the random effect

(continuity corrected Begg’s test; P = 0.280; Egger’s test; P = 0.10) – this with 20%

Winsorising and allowance for overdispersion after omitting the LA with centred

coefficient closest to zero to ensure independence of the estimates.

DISCUSSION

It is not easy to detect outliers in survival by inspection of cumulative survival plots as is

clear from figure 1, let alone in the presence of over-dispersion. A funnel plot, by

converting the survival data into a summary value (the centred hazard ratio) together with

defined boundaries, does make this task easier and much less subjective. Covariates that

affect survival if unevenly distributed across centres may induce additional variation

between institutions or centres; their effect can be allowed for directly by performing a

stratified analysis, but if these are not measured, then indirectly by incorporating over-

dispersion in the significance limits. If both analyses are available then possible to see to

what extent the stratified analysis has accounted for all the overdispersion (and if not, it

implies that still other, unmeasured, variables are having an effect).

Care is necessary not only in choosing the stratification variables for which adjustment is

to be made but also in how these are defined. For example, although age-bands defined

on equal numbers of deaths might be preferred as having equal precision, this choice is

debatable and the method of constructing age bands and other strata must be made before

performing the analysis and inspecting the results to avoid accusations of massaging the

data.

The funnel plot allowing for overdispersion should also be assessed as to whether the

factor(s) inducing the overdispersion are common to all centres (symmetric plot),

whether the distribution of the factors is uneven (asymmetric plot) or whether just one or

two centres are outliers – which may be detected by employing a Winsorised estimate

for the boundaries. Assessment of asymmetry can employ the same methods as used in

meta-analysis to assess publication bias. The results for figure 5 are consistent with the

fact that the Egger regression asymmetry test is said to suggest the presence of

publication bias more frequently than the Begg rank correlation test, that is, the former

test is more sensitive. For meta-analyses these tests lack power, but it is likely that for

institutional comparisons there will be more data available than in many meta-analyses.

Regardless of covariates used for adjustment or stratification, casemix for institutional

comparisons may still be problematic if selection criteria of cases for treatment (such as

surgery) vary or if there are variations in referral patterns, treatment plans and so forth

which may or may not be detectable through overdispersion or funnel plot asymmetry,

depending on the size and frequency of the effect in question, and the size of the sample

being studied. If planning a comparative study, as with any formal investigation,

consideration should also be given to the eligibility criteria for cases – for instance

whether cases diagnosed elsewhere but referred on would be excluded, which would help

increase between-institution homogeneity, but also possibly negate the value of the

comparison in highlighting such differences.

The method described here requires relatively little programming to implement, and is

capable of being incorporated into more sophisticated routines or batch files for routine

analyses. The example .do file displays funnel plots both unadjusted and adjusted for

over-dispersion in the estimated hazard ratios, with an additional plot incorporating user-

specified extent of Winsorising.

An example data set can be downloaded together with the Stata routine to perform the

calculations and plots.

ACKNOWLEDGEMENTS

All views expressed are personal and do not necessarily reflect Registry policy

WHAT THIS PAPER ADDS

What is already known on this topic

Funnel plots are increasingly becoming a standard tool for comparing institutional

performance and for comparisons of survival, a summary measure based on the hazard

ratio reflects the whole survival experience is preferable.

What this study adds

This paper explains the theory of how centred hazard ratio estimates can be obtained

from a Cox regression, with funnel plot control limits obtained from the log-rank test,

with or without adjustment for overdispersion. The method for obtaining robust

(Winsorised) estimates of the overdispersion parameter is also explained and advice is

given on stratification for covariates.

Stata code is given for practical implementation of the methods, and it is suggested that

standard meta-analysis tools be used to assess asymmetry as an aid to interpretation of

outliers in the funnel plot.

POLICY IMPLICATIONS

Hazard ratio comparisons may be added to the repertoire of techniques used by Cancer

Registries, Primary Care Trusts, and other commissioners of Health Care.

REFERENCES

1 Light RJ, Pillemer DB. Summing Up: The Science of Reviewing Research. Cambridge,

Massachusetts: Harvard University Press, 1984.

2 Spiegelhalter DJ. Funnel plots for comparing institutional performance.

Stats in Med 2005; 24: 1185-1202

3 Esteve J, Benhamou E, Raymond L. Statistical methods in cancer research volume IV.

Descriptive Epidemiology. IARC Scientific Publications. Lyon. 1994

4 Peto R, Pike MC, Armitage P, Breslow NE, Cox DR, Howard SV, et al. Design and

analysis of randomised clinical trials requiring prolonged observation of each patient. II:

analysis and examples. Br J Cancer 1977; 35: 1-39

5 Le CT. Applied Survival Analysis. John Wiley, New York. 1997

6 Spiegelhalter DJ. Handling overdispersion of performance indicators. Qual Saf Health

Care 2005; 14: 347-351

7. DerSimonian R, Laird N. Meta-analysis in clinical trials Cont Clin Trials 1986; 7:

177-188

8. van Belle, G. Statistical rules of thumb. John Wiley, New York. 2002

9. Begg, C. B., Mazumdar, M.. Operating characteristics of a rank

correlation test for publication bias Biometrics 1994; 50: 1088-1101.

10. Egger, M., Smith, G. D., Schneider, M., Minder, C. Bias in

meta-analysis detected by a simple, graphical test. British Medical

Journal 1997; 315: 629-634.

11. Tang JL, Liu JL. Misleading funnel plot for detection of bias in meta-analysis. J Clin

Epidemiol. 2000; 53: 477-84

Appendix 1: Algebraic details

In order that the log hazard ratios sum to zero (corresponding to the null hazard ratio of

1), the mean of the regression coefficients must be subtracted from each one:

With k institutions, the shifted ith regression coefficient is given by:

k

k

ii

ii

∑=−= 1*

βββ

With kβ (for the reference category) being zero.

This transformation can be written in matrix form as ββ T=* where the kxk matrix T

has the form:

T =

−−

−−

k

k

k

k

/11000

0/1100

00/110

000/11

If the variance-covariance matrix of the untransformed regression coefficients under the

null hypothesis is V0, then the transformed coefficients have covariance matrix given by:

TTVV ′= 0*

0

Where T’ is the transpose of T.

Under the null hypothesis, the covariance matrix of the test statistic has elements of the

form [5]:

∑= −

−t

j jj

jjijj

NN

dNNd

12 )1(

)( on the leading diagonal,

with off-diagonal elements of the form:

∑= −

−t

j jj

kjijj

NN

NNd

12 )1(

where the ijN , kjN are the number of subjects alive at time j in groups i and k; jd is the

number of deaths occurring at time j, and jN is the total number of subjects alive at time

j.

The subscript j runs from1 to t, reflecting each death time, while subscripts i, k label

different institutions. The Cox model assumes there are no ties on death times, so that all

the dj are equal to 1.

It can be shown that the covariance matrix for the regression coefficients is given by

minus the inverse of the covariance matrix of the score statistic after deleting the row and

column corresponding to the reference category (because the covariance matrix of the

score statistic is singular). After inversion, the dropped row and column can be “added

back” with entries of zero, because the regression coefficients for the reference category

– being a constant (zero) – have variances equal to zero also. After this the covariance

matrix of the regression coefficients (under H0) can be transformed as described above.

Appendix 2: Stata Code

/* APPENDIX: Stata example code to create centred log hazard ratios and obtain funnel plots. In the example data set the institutions to be compared are denoted by the variable "centre", and the observations have been age-stratified into the variable "group". The stratification variable in the example dataset is ageband The basic, overdispersed and overdispersed (Winsorised) plots are produced and saved automatically in the current directory The metabias add-on needs to be installed for evaluation of asymmetry */ clear set mem 50m set more off display "Enter path and name of data file (omit quotes)" _request(filename) display "Enter name of stratification variable (omit quotes)" _request(stratname) local Wval "junk" capture confirm number `Wval' while _rc!=0 { display "Enter extent of Winsorisation (usual = 10%), as decimal eg 0.1 " _request(_Wval) capture confirm number `Wval' } quietly { use "$filename", clear local group = "$stratname" /* perform Cox regression */ xi: stcox i.centre, strata(`group') /* extract coefficient vector and transpose into column vector */ matrix B = e(b) matrix B = B' /* create augmented coefficient vector to include reference category with value equal to zero */ local rows = rowsof(B)+1 matrix Bplus = J(`rows',1,0) matrix Bplus[2,1]=B svmat Bplus /* create transformation matrix to centre regression coefficients on zero */ matrix Jt = J(`rows',`rows',1)/`rows' matrix I = I(`rows') matrix T = (I - Jt) /* centre regression coefficients and transform covariance matrix */ matrix newB = T*Bplus svmat newB rename newB1 CtrB /* CtrB is the vector of centred log hazard ratios ) */ /* extract covariance matrix |H1 */ matrix V = e(V)

/* likewise create augmented covariance matrix |H1 with first row and column values equal to zero */ matrix Vplus= J(`rows',`rows',0) matrix Vplus[2,2] = V /* create transformation matrix to centre regression coefficients */ matrix Jt = J(`rows',`rows',1)/`rows' matrix I = I(`rows') matrix T = (I - Jt) /* Transform covariance matrix |H1 */ matrix newV = T*Vplus*T' matrix newB = T*Bplus /* extract variances|H1 of centred coefficients and save original and new coefficients */ matrix S2 = vecdiag(newV)' svmat S2 replace S21 = sqrt(S21) /* needed for meta-analysis plots */ /* obtain covariance matrix under H0 using logrank test (option gives covariance covariance matrix of the score under H0) */ sts test centre, strata(`group') mat(U V0) /* estimate overdispersion */ scalar df = r(df) scalar phi = r(chi2)/df scalar phi = max(1, phi) /* Obtain covariance matrix |H0 for regression coefficients */ matrix V0=inv(V0[2..`rows',2..`rows']) /* NB VLR is singular - need to drop first row and column before inverting */ matrix V0plus= J(`rows',`rows',0) matrix V0plus[2,2] = V0 /* transform H0 covariance matrix */ matrix newV0 = T*V0plus*T' /* extract H0 variances of centred coefficients and save */ matrix S02 = vecdiag(newV0)' svmat S02 /* get z score & estimate overdispersion from Winsorised z score */ matrix U0 = U[1,2..`rows'] matrix U0 = U0' matrix rtV = cholesky(V0) matrix c2 = rtV'*U0 svmat c2 /* into variable c21 */ rename c21 z_score summ z_score local Wlo = 100*`Wval'/2 local Whi = 100*(1-`Wval'/2) _pctile z_score, percentile(`Wlo', `Whi') /* these are the percentiles for Winsorising */ gen z_score_W = z_score replace z_score_W = r(r2) if z_score_W>r(r2) & z_score ~=. replace z_score_W = r(r1) if z_score_W<r(r1) & z_score ~=. gen z_score_W_sqd = z_score_W^2 egen S_zW2 = sum(z_score_W_sqd) scalar phiW = S_zW2/df scalar phi = max(1, phi) scalar phiW = max(1, phiW) /* =================================== */

/* Calculate control limits */ gen LCtrlL95 = -1.96*sqrt(S021) gen UCtrlL95 = 1.96*sqrt(S021) gen LCtrlL99 = -2.576*sqrt(S021) gen UCtrlL99 = 2.576*sqrt(S021) gen ODLCtrlL95 = -1.96*sqrt(phi*S021) gen ODUCtrlL95 = 1.96*sqrt(phi*S021) gen ODLCtrlL99 = -2.576*sqrt(phi*S021) gen ODUCtrlL99 = 2.576*sqrt(phi*S021) gen ODLWCtrlL95 = -1.96*sqrt(phiW*S021) gen ODUWCtrlL95 = 1.96*sqrt(phiW*S021) gen ODLWCtrlL99 = -2.576*sqrt(phiW*S021) gen ODUWCtrlL99 = 2.576*sqrt(phiW*S021) keep if CtrB~=. replace centre = _n /* lines to exclude observation for meta-analysis */ gen abs_dev = abs(CtrB) egen minabs_dev = min(abs_dev) gen use = 1 replace use = 0 if abs_dev == minabs_dev drop abs_dev minabs_dev gen Precision = 1/S021 /* precision measure for plotting */ } /* <----- end of quietly loop */ /* ============ Plotting =============== */ twoway (scatter CtrB Precision , sort mcolor(black) mlabcolor(black) /// mlabel(centre) ylabel(#10) yscale(range(-1 +1))) /// (line LCtrlL95 Precision , sort lcolor(black) lpattern(solid)) /// (line UCtrlL95 Precision , sort lcolor(black) lpattern(solid)) /// (line LCtrlL99 Precision, sort lcolor(black) lpattern(dash)) /// (line UCtrlL99 Precision , sort lcolor(black) lpattern(dash)), /// scheme(s2mono) saving(LA_Funnel_Basic, replace) twoway (scatter CtrB Precision , sort mcolor(black) mlabcolor(black) /// mlabel(centre) ylabel(#10) yscale(range(-1 +1))) /// (line ODLCtrlL95 Precision , sort lcolor(black) lpattern(solid)) /// (line ODUCtrlL95 Precision , sort lcolor(black) lpattern(solid)) /// (line ODLCtrlL99 Precision, sort lcolor(black) lpattern(dash)) /// (line ODUCtrlL99 Precision , sort lcolor(black) lpattern(dash)), /// scheme(s2mono) saving(LA_Funnel_OD, replace) twoway (scatter CtrB Precision , sort mcolor(black) mlabcolor(black) /// mlabel(centre) ylabel(#10) yscale(range(-1 +1))) /// (line ODLWCtrlL95 Precision , sort lcolor(black) lpattern(solid)) /// (line ODUWCtrlL95 Precision , sort lcolor(black) lpattern(solid)) /// (line ODLWCtrlL99 Precision, sort lcolor(black) lpattern(dash)) /// (line ODUWCtrlL99 Precision , sort lcolor(black) lpattern(dash)), /// scheme(s2mono) saving(LA_Funnel_ODWin, replace) /* meta-analysis for bias: omits centre with coefficient closest to zero (to ensure independence) */ gen var_H0 = phi*S021 metabias CtrB var_H0 if use ==1, var graph( egger) saving(egger_plot, replace)