1

Session 3Design and Analysis of Trials of Therapist Treatments

Chris RobertsUniversity of ManchesterEmail: Chris.Roberts@manchester.ac.uk

Methodology Research Group

Research funded by:MRC Methodology Grants G0600555

G0900678, G0800606, G0802418MHRN Methodology Research Group

2

Intervention Trials with clustering

• Cluster randomised trials• Health professional activities - therapist trials• Group therapies/ group administered treatments trials

In all three types of trial patients can be considered to be in clusters.

Clusters may be:• Experimental units define by randomisation.• Observational units related to the delivery of treatment.

3

Overview of presentation

• Cluster randomised trials – Design – Analysis –Sample Size – Advantage & Disadvantages

• Therapist and Group Therapy Trials– Design – Analysis -

In all three types of trial patients can be considered to be in clusters.

Clusters may be:• Experimental units define by randomisation.• Observational units related to the delivery of treatment.

4

Cluster Randomised Trials (CRTs)

5

Cluster Randomised Trials

Interventions are randomised to groups of patients (clusters) rather than to patients individually.

In mental health CRTs are typically used to test:

•IMPLEMENTATION of new procedures where the intervention applies to complete clinical units.

•PROFESSIONAL BEHAVIOUR CHANGE where the intervention involves training of health professionals.

For these types of intervention once the intervention has taken place it is difficult if not impossible for patients to receive the comparator without changing clinical unit or health professional.

6

Mental Health Cluster Randomised Trials

ExampleSystematic assessments of need and care planning in severe mental illness Priebe et al BJP, (2007) 191: 420-426. Aims To test a computer-mediated intervention structuring patient–clinician dialogue (DIALOG). Methods 134 key workers (clusters) were allocated to DIALOG or treatment as usual.507 people with schizophrenia or related disorders were included.Outcome MeasuresPrimary outcome was subjective quality of life, and secondary outcomes were unmet needs and treatment satisfaction.

7

Illustration of Cluster Randomised Trials

Clus1

P11 P12 P13

Clus2

P21 P22 P23

Clus3

P31 P32 P33

Intervention A

Clus4

P41 P42 P43

Clus5

P51 P52 P53

clus6

P61 P62 P63

Intervention B

CLUSTER

PATIENT

Intervention A

CLUSTER

PATIENT

Intervention B

Schematic Unit Diagram

Classification Diagram

8

A statistical model for a cluster randomised trial

Treatment effect

Baseline covariates x

and coefficient

Variation between clusters

Patient variation

(2) (1)( ). [ 1]i i i cluster i iy t x u e E

0/1 variable indicating treatment cluster(i) indicates the

cluster id of the ith

patient

9

Methods of statistical analysis

Summary measure analysis

Use each cluster as the unit of analysis.Obtain an average for each cluster then compare treatments use a t-

test applied to the cluster averages.

Modelling Methods

Maximum Likelihood (ML)REstrict Maximum Likelihood (REML)

Generalised Estimating Equation (GEE)Survey Estimators (SVY in Stata)

10

Measuring the effect of clustering

2 2 2/u u Between Cluster

VariancePatient

Variance

Intra-cluster correlation (ICC or ρ)

= Proportion of Variance due to cluster

ICC=0 implies no variation between clusters.

ICC=1 implies all patients in each cluster have the same outcome.

Whether the ICC should be allowed to have a value less than one is debated.

Typically, the ICC takes values <0.05 for clinical outcomes

in large samples.

11

Sample Size and Power for a Cluster Randomised Trial

As well as effect size, test size () and s.d., power depends on:• number of clusters (n)• numbers of patients per cluster (m) • intra-cluster correlation coefficient (ρ)

Design Effect (DE) = Ratio of the number of subjects required where there is clustering to the number required assuming no clustering.

Assuming clusters are of equal size

1 ( 1)m DE

Cluster size (m) ICC (ρ)

12

Power if sample size ignored clustering

ICC (ρ)Clusters

/ArmSubjects/Cluster

Total Subjects

Power

Ignoring Clustering 128 80%

0 5 13 130 68%

0.025 5 13 130 56%

0.05 5 13 130 48%

(Assuming standardised effect size = 0.5, =0.05)

If the the power calculation in takes account of numbers of cluster, power is reduced even if ICC (ρ) is assumed to be zero due to the degrees of freedom.

13

ICC (ρ)Clusters /

ArmSubjects/Cluster

Total Subjects

Power 1/ρ

Ignoring Clustering 128 80% -

0 5 18 180 81% -

0.025 5 30 300 80% 40

0.05 5 130 1300 80% 20

(Assuming standardised effect size = 0.5, =0.05)

Numbers of subjects required increases substantial as ICC increases particularly once it exceeds 1/ρ.

Increasing the number of Subjects/Cluster

14

Increasing number of clusters/arm

ICC (ρ)Clusters

/ArmSubjects/Cluster

Total Subjects

Power

Ignoring Clustering 128 80%

0 7 13 182 86%

0.025 8 13 208 83%

0.05 9 13 234 80%

(Assuming standardised effect size = 0.5, =0.05)

Required numbers of subjects more feasible.

15

Advantages of Cluster Randomised Trial

• No contamination between the interventions being compared.

• External validity as intervention can be made to correspond more closely to Usual Care as patients not being individually randomised.

• Administrative and logistic convenience.• Acceptability to health service staff as less disruptive than

individual patient randomisation.• Potential for greater recruitment of patients and groups.• Patient consent may be simpler.

16

Disadvantages of Cluster Randomised Trial

Any two patients within the same cluster are likely to be more similar than two patients from different clusters due to:•Characteristics of patients in a cluster.•Patient interaction within a cluster.•Measured and unmeasured characteristics of health care delivery of a cluster.

This effect is called clustering or more formally intra-cluster correlation.

Valid statistical methods need to take account of the implications of intra-cluster correlation. This requires:•More complex methods of statistical analysis.•A larger sample size of patients is required to achieve the same power and precision as the corresponding individually randomised trial.

Internal validity depends on the selection of patients being independent of randomisation.

17

Summary: Cluster RCTs

Use of cluster randomisation may have advantages including external validity but:

• Precision is reduced.• Validity depends on patient recruitment being

independent of the cluster randomisation.

18

Therapist and Group Administered Treatments Trials

19

Therapist Variation and Clinical Trials

Outcome for patients treated by the same therapist may be more similar than outcomes for patients treated by different therapists due to Therapist characteristics such as:

•Experience•Training•Competence•Empathy

Patients can be considered to be clustered by therapist.

This variation has implications for the Precisions and Validity of psycho-therapy trials.

20

Consort Guidance for Non-pharmacological Trials

A recently published extension of the Consort Guidelines for Non-pharmacological Treatment Trials has drawn attention to the issue.

They recommend that trial of Non-pharmacological Treatment reports how clustering by care-provider has been considered in relation to:

•Selection of care providers.•Sample size calculation.•Allocation of care providers to each trial arm.•Statistical analysis of outcome.

21

Type of Comparisons involving Care Providers

TechniquesDifferent treatment methods delivered by the same care provider.Face-2-Face and Telephone delivered CBT therapy for patients with OCD.Different surgical methods.

Care Provider CharacteristicsComparison of nurse practitioners and general practitioners in primary care.

Packages

Different techniques and different characteristics combined.CBT delivered by a clinical psychologist with Non-directive counselling delivered by a counsellor.

22

Group Administered Treatments

• Examples: Group CBT, Substance misuse, Anger management

• Outcome may be more similar for subjects in the same class or group as patients may interact and this may be a component of the treatment.

• The clustering effect of group deliver only applies to group therapy arm if trial patients are randomised individually.

• Groups may be closed, open, or rolling.

• Where groups are closed, each therapy group can be considered as a cluster.

23

Care Providers and Trial Designs

24

Trial Designs for Therapist Treatment

• Nested (Therapist) Design also called Hierarchical (Therapist) Design.

• Partially Nested (Therapist) Design.

• Crossed (Therapist) Design also called a Stratified (Therapist) Design.

25

Nested Therapist Design

Schematic Unit Diagram

THERAPIST

PATIENT

Therapy A

THERAPIST

PATIENT

Therapy A

Treatment Classification Diagram

T1

P11 P12 P13

T2

P21 P22 P23

T3

P31 P32 P33

Therapy A

T4

P41 P42 P43

T5

P51 P52 P53

T6

P61 P62 P63

Therapy B

26

Partially Nested Therapist Design

THERAPIST

PATIENT

Therapy A

PATIENT

Therapy A

DRUG

Schematic Unit Diagram

Treatment Classification Diagram

P1 P2 P3 P4 P5 P6 P7 P8 P9

Therapy B

T1

P11 P12 P13

T2

P21 P22 P23

T3

P31 P32 P33

Therapy A

27

Crossed Therapist Design

THERAPIST

PATIENT

Therapy A

PATIENT

Therapy B

THERAPIST

CROSSED

P11 P12

A

P13 P14

B

P21 P22

A

P23 P24

B

P31 P32

A

P33 P34

B

P41 P42

A

P43 P44

B

T1 T2 T3 T4

Schematic Unit Diagram

Treatment Classification Diagram

28

Crossed Therapist Design

Same health professional delivers both interventions.

Example: Treatment delivered face-to-face or by phone

Similar to:

• Multi-centre study: Therapist Centre/Study• Meta-analysis: Therapist Study• Crossover cluster randomised trial: Therapist Cluster• Matched pairs cluster randomised: Therapist Matched pair

Treatment effect can be estimated within therapists

See Appendix for details of regarding analysis and sample size.

29

Statistical Analyses for Nested and Partially Nested Designs

• Statistical analysis of Nested and Partially nested designs are similar to cluster randomised trials.

• There is nevertheless added complexity due to the cluster being defined by treatment rather pre-specified for the purpose of randomisation.

30

Comparison of Cluster Randomised, Therapist & Group Therapy Trials

Trial TypeCluster

RandomisedTrial

Nested Therapist Trial

Group TherapyTrial

Randomisation ClusterIndividual /

ClusterIndividual /

Cluster

Between Cluster Variation

Same in both arms

May differ between arms

May differ between arms

Cluster SizeSimilar due to randomisation

May differ between arms

May differ between arms

Cluster Membership

Defined at randomisation

May be poorly defined

Should be well defined for closed group treatments

ValiditySelection of

patients

Selection of therapist for treatments

Selection of therapist for

group treatments

31

Care Providers and Statistical Analysis

32

Models for Nested Therapist Design

Treatment Effect

Therapist (or Group) Variation

Patient Variation

(2) (1)( ) [ 2]i i therapist i iy t u E

Therapist (or Group)-Treatment Interaction

Random Effect

(2) (2) (1)( ) ( ) [ 3]i i therapist i therapist i i iy t u v t E

Covariance between utherapist(i) and vtherapist(i) not identified.

33

Level Heteroscedasticity in a Nested Therapist Design

In cluster randomised trial between arm heteroscedasticity generally ignored and not important

Treatment Effect Therapist or Group

VariationPatient

Variation

Heteroscedasticity in patient variation between treatments (level 1) may bias estimates of therapist variances (level 2) if cluster size differ.

(2) (2) (1) (1)( ) ( ) [ 4]i i therapist i therapist i i i i iy t u v t t E

34

Level Heteroscedasticity in a Nested Therapist Design (Alternative Parameterization)

In cluster randomised trial between arm heteroscedasticity generally ignored and not important

Treatment Effect Therapist or Group

VariationPatient

Variation

Heteroscedasticity in patient variation between treatments (level 1) may bias estimates of therapist variances (level 2) if cluster size differ.

(2) (2) (1) (1)1, ( ) 2, ( ) 1, 2,. 1 . 1 . [ 5]i i therapist i i therapist i i i i i iy t u t u t t t E

35

Intra-cluster Correlation for Therapist (ρ)

2 2 2/u u

Therapist Variance

Patient Variance

ICC (ρ)= Proportion of Variance due to Therapist

2 2 2 2 2 22 u v u v

ICC for second treatment where differential clustering is considered is

36

Partially Nested Design

Subjects in control arm are treated as clusters of size 1[ref Roberts & Roberts 2005]

Treatment Effect

Therapist or Group Variation

Patient Variation

Example: No Drug vs. Drug TherapyComparison of group therapy with individual therapy.

(2) (1)( ) [ 6]i i therapist i i iy t v t E

37

Heteroscedasticity and the Partially Nested Design

Failure to take account of heteroscedasticity can bias estimates of therapist variation and can effect test size

[ref Roberts & Roberts 2005]

Treatment Effect

Therapist or Group Variation

Patient Variation

Example: No Drug vs. Drug TherapyComparison of group therapy with individual therapy.

(2) (1) (1)( ) [ 7]i i therapist i i i i iy t v t t E

38

Crossed Design

2 2 2/ 2v v

Variance of treatment with therapist interaction

(vtherapist(i) )

Pooled Variance of patient within

treatment

How should clustering effect be considered in a crossed design?

(2) (2) (1)( ) ( ) [ 3]i i i i therapist i therapist i i iy t x u v t E

Loss of precision and design effect depends on variance of vtherapist(i) but not utherapist(i) . A parameter for sample size calculation could be

39

Crossed Design (Alternative parameterization)

2 2 2/q q q

Variance of treatment with therapist interaction

(qtreat(i) )

Pooled Variance of patient within

treatment

Three level model with patients nested in treatments nested in therapist

(3) (2) (1)( ) ( ) [ 8]i i i i therapist i treat i iy t x p q E

qtreat(i) is the variation between treatments within therapist

Loss of precision and design effect depends on variance of qtreat(i) but not ptherapist(i) .

40

Internal and External Validity

41

Internal Validity and Therapist Variation

Randomisations Possible in Trials of Therapeutic Approaches

PATIENTS

NESTED DESIGN

THERAPISTS

TREATMENTS

• Random allocation of treatment to therapist to prevent confounding of therapist characteristics with the treatment effect. Example: Schnurr et al.

• Random allocation of Therapist to Patient to would prevent confounding of therapist variation with patient characteristics. Example: Schnurr et al.

42

External Validity: Selection of Therapists

Where interventions cannot be randomised to therapists (trials of Therapist Characteristics and Psychotherapy Packages) selection of therapists has implications for external validity.

THERAPIST

PATIENT

Therapy A

THERAPIST

PATIENT

Therapy B

TherapistPopulation Therapist

Population

43

External Validity of Therapist

Particularly for trials of Therapist Characteristics and Psychotherapy Packages selection of therapist has implications for external validity.

Need to consider how representative trial therapists are of non-trial therapists.

Just as for patients in any trial inclusion and exclusion criteria are need for therapist.

Recruiting therapists directly from clinical practice using broad eligibility criteria might be more representative

but this could also increase the intra-cluster correlation.

44

External Validity: Therapists and Patient characteristics

T1

P11 P12 P13

T2

P21 P22 P23

T3

P31 P32 P33

Therapy A

T4

P41 P42 P43

T5

P51 P52 P53

T6

P61 P62 P63

Therapy B

Randomisation of treatment to patients balances patients characteristics across arms.

Since it is rare to be able to randomise to therapists to patients, therapist variation may be confounded by patient characteristics.

Two possibilities for analysis are:• Estimating the ICC by therapist for patient baseline characteristics.• The effect of baseline covariates on the ICC for therapist for the

outcome measures.

45

Could ICC for therapist be induced by unobserved confounding?

Suppose we have

• Outcome Y and unobserved confounder Z

• Suppose ICC for Z is Z

• Suppose rYZ is the correlation between Y and Z

• Induced ICC is Z rYZ2

Given that rYZ is unlikely to exceed 0.5 Z would need to be

large for an ICC of Y to be induced by unobserved confounding.

46

Internal Validity and Therapist Variation in a Crossed Design.

PATIENTS

THERAPISTS

PATIENTSSTRATIFIED BY

THERAPIST

CROSSED DESIGN

• Patients stratified by therapist.• Treatment effect estimated within therapist.• Random allocation of Therapist to Patient to could in theory

prevent confounding of treatment effect by the therapist and patient characteristics.

• Randomisation may not be desirable as in usual care certain types of therapist may treat certain type of patient.

47

Treatment Comparisons in Psychotherapy Trials

Therapeutic ApproachesEither Crossed Design or Nested Design.

Therapist CharacteristicsNested Design

Psychotherapy Packages combining different approaches and different characteristics

Nested Design

Whether a trial is of Therapeutic Approaches, Therapist Charactistics or a Package depends on the counter factual treatment. What would/should a patient receive if they did not receive the intervention?

48

An Example

Comparison of CBT & NDC for Depression in Primary Care

Data from Ward et al BMJ 2000;321:1383-8

49

Trial: CBT v NDC for Depression in Primary Care

Participants: 262 patients presenting with depression or mixed anxiety + depression.

Interventions: Up to 12 sessions of therapy•cognitive-behaviour therapy (12 psychologist) •non-directive counselling (14 counsellors)

Primary outcome measures: Beck Depression Inventory scores at 4 and 12 months.

Original published analysis ignored clustering by therapist.

Reanalyzed including random effect for therapist.

50

Trial: CBT v NDC for Depression in Primary Care

CLINICALPSCHOLOGIST

PATIENT

CBT

ROGERIANCOUNCELOR

PATIENT

NDC

Treatment Classification Diagram

51

Summary Stats by Treatment Arm

Cognitive Behavioural Therapy

Non-directive Counselling

Mean sd n mean sd n

AGE 35.5 12.0 134 36.1 11.8 126

Male 25% (34/134) 28% (35/126)

BDI mean sd n mean sd n

At Rand. 27.6 7.9 134 27.6 9.0 126

4 months 12.5 10.0 117 12.3 8.5 112

12 months

9.9 10.2 107 11.2 9.1 102

52

Intra-cluster Correlation of Baseline Characteristics

ICC() 95% c.i.

AGE 0.014 (0.000, 0.119)

Male 0.019 (0.000, 0.125)

BDI 0 (0.000, 0.102)

If therapists were randomised to patients, one would expected these to be zero.

53

Analysis of Outcome BDI

• Random effect models fitted using REML (STATA:xtmixed) or RIGLS (ML-WIN)

• Outcome BDI at 4 months.

• Covariates Age, Gender, BDI at baseline.

• BDI squared transformed to improve normality.

Alternative analysis not presented:

• Survey estimators (STATA:svy)

• Generalized Estimating Equations (STATA:xtgee)

• Bayesian Random Effects Models (MCMC: Winbugs/R/MLWin)

54

Numbers of Patients per Therapist

Number of patients

Number of therapists

NDC CBT

52 1 1

39 1

17 1

15 1

8 1

6 1

5 1

4 1 1

3 2

2 3 2

1 4 5

No therapy 10 10

Therapist not known 8 7

Total 127 135

55

Psychotherapy Therapy Sessions

Psychotherapy Therapy Sessions per Patient CBT NDC

0 10 (7) 10 (8)

1-3 29 (22) 24 (20)

4-7 52 (39) 22 (17)

8-13 40 (31) 64 (51)

Not known 2 (1) 5 (4)

N 134 126

.

56

Missing Therapist ID

• Non-recording of Therapist ID for some patients.• Patient non-compliance.

Implications

• Treat missing values as clusters of size 1 in the analysis.• This form of analysis can be biases if compliant and non-

compliant patients have very different outcomes.• Ideally randomly allocate therapists to patient and apply ITT

using the allocated therapist ID.• Purposively assign a patient’s therapist at randomization of

patients and analyze accordingly.

57

Problems caused by large variation in cluster size

• Maximum cluster size places lower limit on ICC equal to -1/(max cluster size -1).

• Variable cluster size leads to loss of power.

• Problem for GEE, survey estimators, MCMC, and ML if negative variances are allowed.

Important that numbers of patients treated by each therapist do no vary greatly

58

Treatment Effect Estimates for BDI (4 Months)

Treatment effect*

SEICC Design

Effect

OLS (assuming no clustering)

-0.008 0.189 - 1

REML(STATA:xtmixed)

Combined -0.041 0.215 0.013 1.29

Separate -0.045 0.214 0.010 1.28

0.014

RIGLS (MLWin)

Combined -0.053 0.222 0.018 1.37

Separate -0.045 0.221 0.010 1.28

0.014

* adjusted for Age Gender and baseline BDI

59

Treatment Effect Estimates for BDI (12 Months)

Treatment effect*

SEICC Design

Effect

OLS (assuming no clustering)

0.219 0.207 - 1

REML (STATA:xtmixed)

Combined 0.219 0.210 <0.0001 1.026

Separate 0.219 0.210 <0.0001 1.027

<0.0001

RIGLS (MLWin)

Combined 0.219 0.207 0.0 1.000

Separate 0.219 0.207 0.0 1.000

0.0

* adjusted for Age Gender and baseline BDI

60

Multiple Membership

Patient may receive of treatment from more than one therapist.

This creates a multiple-membership relationship as each patient is then clustered within two therapist.

This conflicts with the hierarchy of standard multilevel models.

Options for analysis• Define a “primary” cbt therapist (the therapist giving most sessions)

then analyze by primary therapist using hierarchical models described above

• Fit a multiple membership model.

61

Pre-testing for therapist variation

Two-stage analysis has been proposed1. Test for therapist variation. 2. Only model therapist variation if the null hypothesis is rejected.3. Otherwise do an analysis disregarding clustering.

This approach has several flaws:

• The test lacks power unless the number of therapists is large.• It is the point estimate that determines the loss of precision due

to clustering.• The analysis strategy misuses hypothesis testing as

failure to reject the null hypothesis of no therapist variation does not justify its acceptance.

62

Conclusions regarding modelling

• Pattern of clustering in therapist trials may be complex than in cluster randomised trials

• Heterogeneity of clustering needs to be consided

• Clusters due to therapist or therapy group may be poorly defined non-adherence with treatment

• Multiple membership.

63

Other options for dealing with therapist clustering

• Ignore it .

• Standardize treatment to remove between-therapist variation.

64

Ignore it

• It may be plausible to believe there is no clustering.

• Trials with small numbers of therapists are still justified as they can

demonstrate efficacy or potential effect.

• Generalisation will be theoretical based on the assumption of no

variation between therapist (“no therapist effect”) .

• External validity of such a trial is weaker having only a small

numbers of therapist.

65

Standardize treatment

• Treatment manuals.• Harmonisation of procedures.• Supervision and observation of treatment delivery.

Has been shown to reduce therapist variation, but not completely.

This approach may be fine for explanatory pilot trials considering efficacy or developing new treatments.

butStatistical analyses that ignore therapist variation are conditional on there being none.

This may not be acceptable where a treatment is to be rolled our across health services as the treatment may not be generalisable

Efficacy vs Effectiveness

66

References

• Roberts C. The implications of variation in outcome between health professionals for the design and analysis of randomized controlled trials. Statistics in Medicine 1999; 18(19):2605-2615.

• Hoover DR. Clinical trials of behavioural interventions with heterogeneous teaching subgroup effects. Statistics in Medicine 2002; 21(10):1351-1364.

• Schnurr PP, Friedman MJ, Lavori PW, Hsieh FY. Design of Department of Veterans Affairs Cooperative Study No. 420: Group treatment of posttraumatic stress disorder. Controlled Clinical Trials 2001; 22(1):74-88.

• Schnurr PP, Friedman MJ, Engel CC, Foa EB, Shea MT, Resick PM, James KE, Chow BK. Issues in the design of multisite clinical trials of psychotherapy: VA Cooperative Study No. 494 as an example. Contemporary Clinical Trials 2005; 26(6):626-636.

• Lee KJ, Thompson SG. Clustering by health professional in individually randomised trials. British Medical Journal 2005; 330(7483):142-144.

67

References II

• Roberts C, Roberts SA (2005) Design and analysis of clinical trials with clustering effects due to treatment Clinical Trials, 2:152-162

• Lee KJ, Thompson SG. The use of random effects models to allow for clustering in individually randomized trials. Clinical Trials 2005; 2(2):163-173.

• Moerbeek M, Wong WK. Sample size formulae for trials comparing group and individual treatments in a multilevel model. Statistics in Medicine 2008; 27(15):2850-2864.

• Boutron I, Moher D, Altman DG, Schulz KF, Ravaud P, for the CONSORT Group. Extending the CONSORT statement to randomized trials of nonpharmacolgic treatment: Explanation and elaboration. Annals of Internal Medicine 2008; 148:295-309. (www.consort-statement.org)

• Walwyn, R. & Roberts, C. (2010). Therapist variation in randomised trials of psychotherapy: Implications for precision, internal and external validity. Statistical Methods in Medical Research 19:291-315.

68

References III

• Martindale C. The therapist-as-fixed-effect fallacy in psychotherapy research. Journal of Consulting & Clinical Psychology 1978; 46(6):1526-1530.

• Crits-Christoph P, Mintz J. Implications of therapist effects for the design and analysis of comparative studies of psychotherapies. Journal of Consulting & Clinical Psychology 1991; 59(1):20-26.

• Wampold BE, Serlin RC. The consequence of ignoring a nested factor on measures of effect size in analysis of variance. Psychological Methods 2000; 5(4):425-433.

• Elkin I, Falconnier L, Martinovich Z, Mahoney C. Therapist effects in the National Institute of Mental Health Treatment of Depression Collaborative Research Program. Psychotherapy Research 2006; 16(2):144-160.

69

References IV

• Kim D-M, Wampold BE, Bolt DM. Therapist effects in psychotherapy: A random-effects modeling of the National Institute of Mental Health Treatment of Depression Collaborative Research Program data. Psychotherapy Research 2006; 16(2):161-172.

• Goodyer I, Dubicka B, Wilkinson P, Kelvin R, Roberts C, Byford S, Breen S, Ford C, Barrett B, Leech A, Rothwell J, White L, Harrington R (), Selective serotonin reuptake inhibitors (SSRIs) and routine specialist care with and without cognitive behaviour therapy in adolescents with major depression: randomised controlled trial. BMJ 2007 335(7611):142

• Ward E, King M, Lloyd M, Bower P, Sibbald B, Farrelly S, et al. Randomised controlled trial of non directive counselling, cognitive behaviour therapy and usual general practitioner care for patients with depression. I: Clinical effectiveness. BMJ 2000;321:1383 8.

70

APPENDIX I

SAMPLE SIZE &

POWER CALCULATION

71

SAMPLE SIZE & POWER CALCULATION

• Asymptotic approximation are available to calculate sample size for specified cluster sizes and numbers of clusters.

• Where sample size of therapist or groups is small (say <20 per group) non-asymptotic approximations can be used to calculate power for a given number of clusters, cluster size and intra-cluster correlations coefficient.

• With similar syntax to sampsi a user written STATA routine cluspower can calculates power were cluster size, numbers of clusters and the intra-cluster correlation coefficient differs between treatment arms. This can be used iteratively to calculate sample size.

Available from http://www.medicine.manchester.ac.uk/healthmethodology/

research/biostatistics/data/cluspower/ • clsampsi coming soon calculates sample sizes for given

power

72

APPENDIX II

STATA Code

73

STATA xtmixed syntax

Cluster Randomised Trial

xtmixed y {linear predictor including treatment} ||cluster: [E1]

Nested Design

xtmixed y {linear predictor including treatment} ||therapist: [E2]

xtmixed y {linear predictor including treatment} ||therapist:treat ,cov(independent) [E3]

xtmixed y {linear predictor including treatment} ||therapist:treat notreat , nocons cov(independent) [E4]

xtmixed y {linear predictor including treatment} ||therapist:treat notreat, nocons cov(independent) residual(inde,by(treat)) [E5]

74

STATA xtmixed syntax

Partially Nested Design

xtmixed y {linear predictor including treatment} ||therapist:

xtmixed y {linear predictor including treatment} ||therapist:treat, nocons [E6]

xtmixed y {linear predictor including treatment} ||therapist:treat, nocons residual(inde,by(treat)) [E7]

treat and notreat are indicator variables for treatment and no treatment

Crossed Design as a 3 level model

xtmixed y {linear predictor including treatment} ||therapist:treat ,cov(unstru) [E3]

xtmixed y {linear predictor including treatment} ||therapist:||ther_treat:[E7]

which is equivalent to

xtmixed y {linear predictor including treatment} ||therapist:treat notreat , nocons cov(exch) [E4]

Note differences in random effect covariance matrix for Nested and Crossed Designs