10

Developing measures of surgeons’equipoise to assess the feasibility ofrandomized controlled trials invascular surgeryJane Young, MBBS, MPH, PhD, FAFPHM, James Harrison, BSc (Hons), Geoffrey White, MBBS,FRACS, James May, MBBS, MS, MD, FRACS, and Michael Solomon, MB, BCh, MSc (Clin Epi),FRACS, Sydney, Australia

Background. Equipoise is defined medically as a state of genuine uncertainty about the relative benefitsof alternative treatment options. This study investigated individual and collective equipoise amongvascular surgeons for controversial clinical questions to assess the feasibility of conducting randomizedcontrolled trials.Methods. Vascular surgeons throughout Australia and New Zealand received a survey by mail.Results. Vascular surgeons (n = 146, 77% response fraction) were able to quantify the strength of theirtreatment preferences and did so differentially between clinical scenarios using a simple scale. Almost onequarter (24%; 95% CI, 18%-32%) were completely undecided about whether carotid endarterectomyor carotid stenting was preferable to treat carotid stenosis in high-risk patients, indicating individualequipoise. In contrast, the vast majority of respondents (89%; 95% CI, 82%-93%) favored carotidendarterectomy over carotid stenting for average-risk patients, suggesting lack of community equipoise forthis patient group. Similarly, there was lack of community equipoise for treatments for abdominal aorticaneurysm in high-risk patients with 88% (95% CI, 81%-92%) favoring a minimally invasiveapproach. Older respondents were consistently less willing to take part in randomized trials, withstrength of treatment preference also independently predicting willingness to participate in 4 of 6 trials.Conclusions. Individual and community equipoise can be measured in a representative sample ofsurgeons as part of the feasibility assessment for future randomized controlled trials. (Surgery2004;136:1070-6.)

From the Surgical Outcomes Research Centre, Central Sydney Area Health Service, the University of Sydney, theDepartment of Vascular Surgery, Royal Prince Alfred Hospital, and the University of Sydney, Sydney, Australia

THE PAST 3 DECADES HAVE WITNESSED rapid technolog-ical advances in vascular surgery with the develop-ment of minimally invasive surgical techniques,such as angioplasty, stenting, and stent grafting.1 Asin all fields of surgery, careful evaluation is neededto determine whether such innovative therapeuticinterventions actually lead to improved healthoutcomes for patients with vascular disease.Randomized controlled trials (RCTs) are widelyrecognized as the optimal study design to evaluate

Accepted for publication April 4, 2004.

Reprint requests: Jane Young, PhD, Executive Director, SurgicalOutcomes Research Centre, PO Box M157, Missenden Rd NSW2050, Sydney, Australia.

0039-6060/$ - see front matter

� 2004 Elsevier Inc. All rights reserved.

doi:10.1016/j.surg.2004.04.012

70 SURGERY

health care interventions, yet they remain un-common in surgery.2-5 Vascular surgery is noexception.2 For example, among 202 RCTs in thefield of general surgery published in leadingacademic surgical journals in 1990, only 35 (17%)trials were performed in vascular surgery.3

While methodologic issues such as difficultieswith blinding, lack of placebo controls, anddifficulties with standardizing operative techniquesbetween surgeons and institutions undoubtedlyhave hindered RCTs in surgery,3-9 surgeons may bereluctant to randomize patients to different treat-ment groups in the absence of genuine clinicaluncertainty about the relative merits of thetreatments being compared.10-13 The term equipoiseis often used to reflect a state of genuine clinicaluncertainty, when preferences for alternative ther-apies are ‘‘poised’’ or equally balanced, and neither

SurgeryVolume 136, Number 5

Young et al 1071

treatment option is clearly superior.14 Equipoisecan work at the level of the individual clinician(‘‘clinical equipoise’’) or within the clinical com-munity (‘‘collective equipoise’’).14-15

Recently, an investigation into reasons for poorrecruitment to a RCT of different treatmentoptions for men with localized prostate cancerhighlighted the importance of equipoise, with theinvestigators noting: ‘‘We found it essential thatrecruiting staff were able to express confidentlythat men were eligible for all three treatments, thatthe most effective treatment was unknown, thata trial was urgently needed, and that randomizationcould provide a plausible way of reaching a de-cision.’’16 Others have argued that ‘‘community’’ or‘‘collective’’ equipoise, in which there is a lack ofconsensus in the expert clinical community, ratherthan individual clinician equipoise provides theethical grounds for conduct of a trial.14,15 Toquantify the level of community equipoise requiredfor a trial to be considered ethical, Johnson andcolleagues17 interviewed people from a variety ofbackgrounds in the United Kingdom. They foundthat 50% of people perceived a trial would beunethical when the community equipoise wasdisturbed beyond 70:30 (in other words, when70% or more of expert clinicians favored 1 of thetreatment options).

Before embarking on large-scale RCTs, it isimportant also to assess the clinical significance ofthe hypothesis, clinicians’ treatment preferences,and their willingness to take part in a trial.11,18

Patients’ treatment preferences and their willing-ness to take part in RCTs can be readily assessed.19

Measuring surgical equipoise and preferences incombination with patient preferences can helpdetermine not only the important clinical out-comes but also the feasibility of performing a RCTin terms of likely recruitment. In addition, assess-ment of preferences can identify the magnitude ofbenefit or harm that must be demonstrated for theresults of a RCT to be of practical significance toclinicians and patients.20

To investigate individual and community equi-poise, we sought to determine whether vascularsurgeons could quantify their current level ofclinical certainty about alternative treatmentapproaches in controversial clinical scenarios,using a simple self-administered questionnaire.We also sought to identify those clinical questionsperceived by vascular surgeons as priorities forclinical research and to assess their willingnessto take part in such studies in order to investigatethe feasibility of conducting RCTs in vascularsurgery.

METHODS

Development of questionnaire. We developedthe following 6 clinical scenarios each describing 2alternative treatment approaches to a vascularsurgical problem:

� Open or endoluminal repair for abdominal aortic

aneurysm in average-risk patients

� Open or endoluminal repair for abdominal aortic

aneurysm in high-risk patients

� Surgery or observation for small abdominal aortic

aneurysm (4-5.5 cm) in average-risk patients

� Femoro-popliteal bypass or angioplasty for peripheral

vascular disease

� Carotid endarterectomy or carotid stent for carotid

stenosis in average-risk patients

� Carotid endarterectomy or carotid stent for carotid

stenosis in high-risk patients

These scenarios were considered controversial,either because of lack of scientific evidence ofeffectiveness or because of difficulties in balancingdifferential benefits and risks of each treatment.The scenarios, described exactly as above, wereincluded in a self-administered questionnaire thatfirst asked respondents to rate the importance ofeach scenario for future clinical research (‘‘ex-tremely important,’’ ‘‘very important,’’ ‘‘somewhatimportant,’’ or ‘‘not at all important’’). Next,respondents were asked to rate their level of clinicalcertainty about which of the 2 treatment options ineach scenario was their preferred option. To dothis, respondents were presented with a linearanalog scale with 1 treatment option anchored ateach end of the scale. The scale was centered onzero to represent ‘‘completely undecided’’ andmarked from ‘‘1’’ to ‘‘5’’ toward each end torepresent increasing certainty in the treatmentapproach. Respondents were asked to circle thenumber on the scale that best represented theirlevel of certainty. For each clinical scenario,participants were also asked whether they wouldparticipate in an RCT or nonrandomized follow upstudy (‘‘yes’’ or ‘‘no’’), assuming that funding andresearch support were available to conduct a mul-ticenter study. Demographic information also wasobtained.

Administration of survey. The ANZ Society forVascular Surgery is the professional organizationfor specialist vascular surgeons in Australasia. Thecontact details for all members of the Society wereobtained, with permission, from the society’sdatabase. Clinicians were considered ineligible ifthey had died, retired, were on extended leave ofmore than 6 weeks, or were not a resident inAustralia or New Zealand. To improve the response

SurgeryNovember 2004

1072 Young et al

rate,21 each surgeon was mailed an advance letternotification 1 week before the mailing of a coverletter, questionnaire, and reply-paid envelope inNovember 2002. Up to 3 follow-up reminders weremailed to nonresponders.

Statistical analyses. Descriptive statistics (pro-portions and 95% confidence intervals) werecalculated. To assess community equipoise, wecategorized surgeons’ responses on the linearanalog scale into 1 of 3 groups: preferred treatmentA; completely undecided; preferred treatment B.For each scenario, logistic regression modeling wasconducted to identify independent predictors ofrespondents who indicated individual equipoise(completely undecided). If more than 70% ofrespondents favored 1 of the treatment options, weconsidered that community equipoise was lackingfor that scenario.17

For each scenario, the McNemar test for pairedproportions was used to compare willingness toparticipate in an RCT with willingness to participatein a nonrandomized follow-up study. Logisticregression modeling was conducted to identifyindependent predictors of willingness to partici-pate in a RCT and nonrandomized follow-up studyfor each of the scenarios. Analyses were carried out

Table I. Characteristics of respondents (n = 146)

Characteristic n %

Age< 35 years 14 1035-44 years 41 2845-54 years 43 3055-64 years 40 2765+ years 7 5

GenderMale 139 95Female 7 5

Location of practiceCapital city 105 72Urban area 30 20Rural 10 7Other 1 < 1

Type of appointmentPrivate practitioner 95 65Salaried medical officer 23 16Salaried university academic 5 3Registrar/fellow 8 5

Type of hospitalTertiary teaching hospital 88 60District general hospital 18 12Private hospital 30 20District general & private 2 1Tertiary & private 3 2

with the use of SAS statistical software (SASInstitute Inc, Cary, NC).22

RESULTS

Of 203 surgeons of the ANZ Society for VascularSurgery, 13 were ineligible (9 retired, 4 could notbe contacted). We received 146 completedquestionnaires from 190 eligible surgeons (77%response rate). Personal and professional charac-teristics of respondents are shown in Table I. Themean age of participants was 48 years.

Measurement of equipoise. For each of the 6scenarios, surgeons’ treatment preferences areshown in the figure. The scenario for which thehighest proportion of respondents indicated theywere completely undecided about which treatmentoption was preferable (individual equipoise) per-tained to carotid endarterectomy or carotid stentfor carotid stenosis in high-risk patients. Almostone quarter of respondents (24%; 95% CI, 18%–32%) indicated uncertainly about this treatmentoption. For 2 scenarios, namely open orendoluminal repair for abdominal aortic aneurysmin high-risk patients and carotid endarterectomy orstent for carotid stenosis in average-risk patients,there was no evidence of community equipoise as 1of the treatment options was preferred by well over70% of respondents (Table II).

Predictors of individual equipoise. For 5 of the6 scenarios, there were no significant associationsbetween individual equipoise and demographic orpractice characteristics of respondents. For thesixth scenario, namely open or minimally invasivesurgery for high-risk patients with carotid stenosis,respondents were significantly less likely to indicateindividual equipoise as their age increased(b = �0.06, P = .01).

Perceived importance of clinical questions.Overall, the scenario rated as ‘‘extremely impor-tant’’ by the most respondents (51%; 95% CI, 43%-60%) pertained to open or minimally invasivesurgery for carotid stenosis in average-risk patients;the rating was significantly higher than for thesame scenario in high-risk patients (McNemar test,v2 = 6.4, df = 1, P = .01) (Table II).

Willingness to participate in randomized andnonrandomized studies. Respondents’ willingnessto participate in RCTs or nonrandomized follow-upstudies to address these clinical scenarios are shownin Table II. The highest proportion of respondents(67%; 95% CI, 59%–74%) indicated they would bewilling to participate in an RCT for carotidendarterectomy or carotid stent for carotid stenosisin high-risk patients; this response was significantlyhigher than the 36% (95% CI, 29%–45%) who

SurgeryVolume 136, Number 5

Young et al 1073

Figure. Respondents’ level of certainty for treatment options in each of 6 surgical scenarios. A, Open orendoluminal repair for AAA in average-risk patients. Equipoise ratio (%) : Open, 65; Undecided, 6;Endoluminal, 26. B, Open or endoluminal repair for AAA in high-risk patients. Equipoise ratio (%) : Open,6; Undecided, 6; Endoluminal, 88. C, Surgery or observation for small (4-5.5 cm) in average-risk patients.Equipoise ratio (%) : Surgery, 25; Undecided, 8; Observation, 63. D, Femoro-popliteal bypass or angioplastyfor peripheral vascular disease. Equipoise ratio (%) : Fem-pop, 39; Undecided, 20; Angioplasty, 34. E, Carotidendarterectomy or carotid stent for carotid stenosis in average-risk patients. Equipoise ratio (%) :Endarterectomy, 89; Undecided, 8; Stent, 2. F, Carotid endarterectomy or carotid stent for carotidstenosis in high-risk patients. Equipoise ratio (%) : Endarterectomy, 46; Undecided, 24; Stent, 29.

SurgeryNovember 2004

1074 Young et al

Table II. Respondents’ ratings of importance, willingness to participate in research, and communityequipoise for each of 6 clinical scenarios

Rated‘‘extremelyimportant’’n (%)

Wouldparticipatein RCTn (%)

Wouldparticipatein non-RCT

n (%)

Level of clinical certainty

Evidence ofcommunityequipoise?17

Preferredtrmt An (%)

Completelyundecidedn (%)

Preferredtrmt Bn (%)

Open or endoluminal repair forAAA in average-risk patients

44 (30) 84 (58) 116 (80) 95 (65) 9 (6) 38 (26) Yes

Open or endoluminal repair forAAA in high-risk patients

41 (28) 75 (51) 115 (79) 8 (6) 8 (6) 128 (88) No

Surgery or observation for smallAAA (4-5.5 cm) in average-riskpatients

19 (13) 79 (54) 102 (70) 37 (25) 11 (8) 92 (63) Yes

Femoro-popliteal bypass orangioplasty for peripheral vasculardisease

22 (15) 73 (50) 105 (72) 57 (39) 30 (20) 49 (33) Yes

Carotid endarterectomy or carotidstent for carotid stenosis inaverage-risk patients

75 (51) 53 (36) 89 (61) 130 (89) 12 (8) 3 (2) No

Carotid endarterectomy or carotidstent for carotid stenosis forhigh-risk patients

59 (40) 98 (67) 106 (73) 67 (46) 35 (24) 43 (29) Yes

RCT, Randomized controlled trial; trmt, treatment; AAA, abdominal aortic aneurysm.

would be willing to participate in the same trialamong average-risk patients (McNemar test,v2 = 31.74, df = 1, P < .001).

In 5 of 6 clinical scenarios, respondents weresignificantly more likely to indicate willingness toparticipate in a nonrandomized follow-up studythan in the corresponding RCT (all P values <.001). For the sixth scenario, namely carotidendarterectomy or carotid stent for carotid stenosisin high-risk patients, there was no significantdifference in the proportion who would be willingto take part in either type of study (McNemar test,v2 = 1.17, df = 1, P = .28).

Predictors of surgeons’ willingness to partici-pate in RCTs. For all 6 clinical scenarios,respondents were significantly less likely to indicatewillingness to participate in a RCT as their ageincreased. For 4 of the 6 scenarios, increasing levelof certainty for 1 of the 2 treatment options alsoindependently predicted unwillingness to partici-pate (Table III).

DISCUSSION

This survey is the first, to our knowledge, tomeasure equipoise in the surgical community forcontroversial clinical questions to assess the feasi-bility of conducting RCTs. Vascular surgeons wereable to quantify the strength of their treatmentpreferences and did so differentially betweenscenarios using a simple scale. Our study identified2 areas in vascular surgery, namely endoluminal

repair for abdominal aortic aneurysm in high-riskpatients and carotid stenting in average-riskpatients, where it may be difficult to justify a trialon ethical grounds as there was overwhelmingpreference for 1 of the treatment options (ie, nocommunity equipoise). Overall, there was strongsupport for the conduct of RCTs, with more thanhalf of respondents indicating they would take partin a trial in all but 1 scenario. Support forobservational research was stronger still with sur-geons significantly more likely to be willing to par-ticipate in a nonrandomized follow-up study than inthe corresponding RCT for 5 of the 6 scenarios.

The questionnaire developed in this studyappears to provide a useful and novel means ofassessing community equipoise among a represen-tative group of surgeons before the instigation ofRCTs in surgery. When asked to indicate theirpreference for treatment options outlined in thescenarios, the vast majority indicated a preferencefor endoluminal rather than open repair forabdominal aortic aneurysm in high-risk patientsand a preference for carotid endarterectomy ratherthan carotid stenting for carotid stenosis inaverage-risk patients. Because of the apparent lackof equipoise, RCTs to address these clinicalquestions may not be ethically justifiable or feasibleas surgeons are likely to be unwilling to randomizetheir patients. In 2 other scenarios, namelytreatment options for abdominal aortic aneurysmin average-risk patients and for small aneurysms,

SurgeryVolume 136, Number 5

Young et al 1075

there was majority support for 1 of the treatmentoptions, but the support did not quite reach the70% threshold for community equipoise. Thescenario for which the largest number ofrespondents (24%) indicated that they werecompletely undecided pertained to the questionof carotid endarterectomy or carotid stenting forcarotid stenosis in high-risk patients. Furthermore,in this scenario, respondents who did indicatea preference were fairly evenly divided between the2 surgical approaches, suggesting this is an areawith high levels of both individual and communityequipoise for which an RCT would be both valuableand feasible.

While our scale to measure level of uncertaintywas exploratory, surgeons did not appear to havedifficulty with the concept because there were fewmissing data for these questions and surgeons’responses did appear to discriminate betweenscenarios. To explore the issue of equipoise as partof a mailed survey, we necessarily adopted a prag-matic, simple definition that encompassed onlycurrent uncertainty about which of 2 alternativetreatment options was considered preferable.Others contest that equipoise is a dynamic ratherthan static construct, modified by ongoing dialoguebetween clinicians, patients, ethics committees, andother stakeholders, and has ramifications for de-cisions about when it is ethical to stop randomizingpatients in an ongoing trial.23 Our focus was toinvestigate current levels of clinical uncertaintyamong vascular surgeons in order to assess thefeasibility of undertaking future RCTs. Becausefailure to recruit sufficient patients is a major reasonfor the failure of many trials,24 a simple measure toscreen out proposed trials on the basis of lack ofcommunity equipoise could prevent waste of res-ources and help focus efforts on successful conductof those surgical trials most likely to achieve theiraims. However, further research is needed toestablish the face validity and reliability of our scalebefore advocating its widespread use.

Overall, the question of carotid endarterectomyor carotid stent for average-risk patients was rated as‘‘extremely important’’ by the most respondents(51%). This is despite the fact that there waswidespread agreement among surgeons in favor ofcarotid endarterectomy as the preferred surgicalapproach in this patient group, suggesting thatratings of importance elicited in this study were notsolely based on treatment uncertainty. Further-more, we found that perceived importance of theclinical question was unrelated to surgeons’ willing-ness to participate in either RCTs or observationalresearch.

A substantial proportion of respondents in-dicated they would be willing to participate ina RCT for each of the scenarios. While self-reportedwillingness to participate in a hypothetical trial mayoverestimate actual participation in a real trial,comparison of responses between scenarios pro-vides some interesting insights. Two-thirds (67%)of surgeons would take part in a trial of carotidendarterectomy or carotid stenting for carotidstenosis in high-risk patients, the scenario withthe greatest level of uncertainty about which of thetreatment options was preferable. In contrast,significantly fewer (36%) would take part in thesame trial involving average-risk patients. Re-spondents’ age consistently predicted their willing-ness to take part in a trial, with younger surgeonsbeing more willing to do so. Our findings also areconsistent with previous studies suggesting thatconcerns about randomization are major barriersto RCTs in surgery9,11,12 because, for 5 of the 6scenarios, surgeons were significantly more likely totake part in a nonrandomized follow-up study thanin the corresponding RCT.

CONCLUSION

This study has developed a means to assessequipoise in the surgical community using a scale

Table III. Independent predictors of willingnessto participate in an RCT

Scenario Independent predictor b* P value

AAA repair inaverage-riskpatients

Age -0.06 .007Level of certainty -0.6 < .001Type of hospital -1.0y .02

AAA repair inhigh-riskpatients

Age -0.06 .002Level of certainty -0.6 < .001

Management ofsmall AAA

Age -0.04 .03

Peripheral vasculardisease

Age -0.07 < .001

Carotid stenosis inaverage-riskpatients

Age -0.06 .01Level of certainty -0.8 .001

Location -1.1z .03Carotid stenosis in

high-riskpatients

Age -0.04 .03Level of certainty -0.06 < .001

RCT, randomized controlled trial; AAA, abdominal aortic aneurysm.*–ve values indicate decreasing willingness to participate in a RCTwith increasing value of predictor (ie, increasing age or increasingcertainty in 1 treatment option).yRespondents outside tertiary centers were less likely to indicatewillingness to participate in an RCT than those in tertiary hospitals.zRespondents outside capital cities were less likely to indicatewillingness to participate in an RCT than those in capital cities.

SurgeryNovember 2004

1076 Young et al

to measure individual treatment preferences am-ong a representative sample of surgeons. Becauseassessment of equipoise, clinical importance, andfeasibility are important prerequisites for theconduct of any RCT, use of a simple questionnairesuch as ours holds potential to focus future clinicalresearch in surgery.

We are grateful to the vascular surgeons who took partin this survey. Our thanks also go to Mary Bourke andChristine Westbury for assistance with mail-outs and dataentry. This study was approved by the institutional ethicscommittees of the Central Sydney Area Health Service(RPAH zone) and the University of Sydney.

REFERENCES

1. Weiss VJ, Lumsden AB. Minimally invasive vascular surgery:a review of current modalities. World J Surg 1999;23:406-14.

2. Barnes RW. Understanding investigative clinical trials. J VascSurg 1989;9:609-18.

3. Solomon MJ, Laxamana A, Devore L, McLeod RS.Randomized controlled trials in surgery. Surgery 1994;115:707-12.

4. Haines SJ. Randomized clinical trials in the evaluation ofsurgical innovation. J Neurosurg 1979;51:5-11.

5. Pollock AV. The rise and fall of the randomised controlledtrial in surgery. Theor Surg 1989;4:163-70.

6. Fielding LP, Stewart-Brown S, Dudley HAF. Surgeon-relatedvariables and the clinical trial. Lancet 1978;ii:778-9.

7. Lawrence W Jr. Some problems with clinical trials. ArchSurg 1991;126:370-8.

8. Van der Linden W. Pitfalls in randomized surgical trials.Surgery 1980;87:258-62.

9. McCulloch P, Taylor I, Sasako M, Lovett B, Griffin D.Randomised trials in surgery: problems and possiblesolutions. BMJ 2002;324:1448-51.

10. Solomon MJ, McLeod RS. Surgery and the randomisedcontrolled trial: past, present and future. MJA 1998;169:380-3.

11. Solomon MJ, McLeod RS. Should we be performing morerandomised controlled trials evaluating surgical operations?Surgery 1995;118:459-67.

12. Gattellari M, Ward JE, Solomon MS. Randomized, con-trolled trials in surgery: perceived barriers and attitudes ofAustralian colorectal surgeons. Dis Colon Rectum 2001;44:1413-20.

13. Ellis PM, Butow PN, Simes RJ, Tattersall MH, Dunn SM.Barriers to participation in randomized controlled trials forearly breast cancer among Australian cancer specialists.ANZ J Surg 1999;69:486-91.

14. Freedman B. Equipoise and the ethics of clinical research.N Engl J Med 1987;317:141-5.

15. Weijer C, Shapiro SH, Cranley Grass K. Clinical equipoiseand not the uncertainty principle is the moral underpin-ning of the randomised controlled trial. BMJ 2000;321:756-7.

16. Donovan J, Mills N, Smith M, et al. Quality improvementreport: improving design and conduct of randomised trialsby embedding them in qualitative research: Protect T(prostate testing for cancer and treatment) study. BMJ 2002;325:766-70.

17. Johnson N, Lilford RJ, Brazier W. At what level of collectiveequipoise does a clinical trial become ethical? J Med Ethics1991;17:30-4.

18. Lilford R. for the Fetal Compromise Group. Formalmeasurement of clinical uncertainty: prelude to a trial inperinatal medicine. BMJ 1994;308:111-3.

19. Solomon MJ, Pager C, Keshava A, Findlay M, Butow P,Salkeld G, Roberts R. What do patients want? Patientpreferences and surrogate decision-making in the treat-ment of colorectal cancer. Dis Colon Rectum 2003;46:1351-7.

20. Simes RJ, Coates AS. Patient preferences for adjuvantchemotherapy of early breast cancer: how much benefit isneeded? J Natl Cancer Inst Monogr 2001;30:146-52.

21. Edwards P, Roberts I, Clarke M, et al. Increasing responserates to postal questionnaires: systematic review. BMJ 2002;324:1183-91.

22. SAS Institute Inc. The SAS System for Windows [computerprogram] Version 8.01. Cary, NC: SAS Institute Inc; 1999.

23. Little M. �Better than numbers. . .� A gentle critique ofevidence-based medicine. ANZ J Surg 2003;73:177-82.

24. Taylor KM, Margolese RG, Soskolne CL. Physicians’ reasonsfor not entering eligible patients in a randomized clinicaltrail of surgery for breast cancer. N Engl J Med 1984;10:1363-7.