Comparison of directed and self-directed learning in evidence-based medicine: a randomised...

Comparison of directed and self-directed learning inevidence-based medicine: a randomised controlledtrialPeter Bradley,

1

Christina Oterholt,2

Jeph Herrin,3

Lena Nordheim4

& Arild Bjørndal4

OBJECTIVES To compare 2 educational pro-grammes for teaching evidence-based medicine(EBM).

DESIGN Prospective randomised controlled trialaccompanied by a qualitative evaluation.

SETTING University of Oslo, Norway, 2002–03.

PARTICIPANTS A total of 175 students entered thestudy. All tenth semester medical students from 3semesters were eligible for inclusion if they comple-ted baseline assessment and consent forms and eitherattended teaching on the first day of the semester orgave reasons for their absence on the first day inadvance.

INTERVENTIONS One intervention was based oncomputer-assisted, self-directed learning (self-directedintervention), whilst the other was organised as work-shops based on social learning theory (directed inter-vention). Both educational interventions consisted of5 half-day sessions.

MAIN OUTCOME MEASURES The primaryoutcomes were knowledge about EBM and skills incritical appraisal. A secondary outcome measuredattitudes to EBM. Outcomes were compared on anintention-to-treat basis using a stratified Wilcoxonrank-sum test.

RESULTS There were no differences in outcomes forthe 2 study groups in terms of EBM knowledge (meandeviation 0.0 [95% confidence interval ) 1.0, 1.0],P ¼ 0.8), critical appraisal skills (MD 0.1 [95% CI) 0.9, 1.1], P ¼ 0.5), or attitudes to EBM (MD ) 0.3[95% CI ) 1.4, 0.8], P ¼ 0.5). Follow-up rates were96%, 97% and 63%, respectively.

CONCLUSIONS This trial and its accompanyingqualitative evaluation suggest that self-directed,computer-assisted learning may be an alternativeformat for teaching EBM. However, further researchis needed to confirm this and investigate alternativeeducational models.

KEYWORDS randomised controlled trial (publica-tion type); teaching ⁄ *methods; education, medical,undergraduate ⁄ *methods; comparative studies;evidence-based medicine; prospective studies.

Medical Education 2005; 39: 1027–1035doi:10.1111/j.1365-2929.2005.02268.x

INTRODUCTION

Comparative educational trials have beenrecommended to find out how best to teach evi-dence-based medicine (EBM).1 This randomisedcontrolled trial (RCT) evaluated 2 educational pro-grammes in EBM for medical students in the tenthsemester of studies at the University of Oslo, Norway.One programme was based on computer-assisted(CAL), self-directed learning (self-directed interven-tion), whilst the other was organised as workshopsbased on social learning theory (directed intervention).The purpose of this trial was to investigate whetherthe 2 interventions transferred EBM knowledge,skills, and attitudes equally.

education studies

1Norwegian Medicines Agency, Oslo; Norway2Department of Medicine, University of Oslo, Oslo, Norway3Flying Buttress Associates, Charlottesville, Virginia, USA4Norwegian Centre for Health Services Research, Oslo, Norway

Correspondence: Peter Bradley, Director of Public Health and HealthImprovement, Thingoe House, Cotton Lane, Bury St. Edmonds,Suffolk IP33 1YJ. UK. Tel: 00 44 1284 829650; Fax: 00 44 1284 829567;E-mail: [email protected]

� Blackwell Publishing Ltd 2005. MEDICAL EDUCATION 2005; 39: 1027–1035 1027

The RCT was accompanied by a qualitative evalua-tion,2 as recommended by other authors.3–5 Inaddition, a systematic review6 of existing quantitativeand qualitative research was conducted to complete amultistage model for evaluation. Both accompanyingstudies were used to plan the intervention andinterpret the trial’s results.

The practice of EBM has been defined elsewhere7 ascomprising several steps, which include the ability toformulate appropriate questions, retrieve and critic-ally appraise research (evidence), apply evidence topractice and evaluate subsequent changes in prac-tice.

Systematic reviews have repeatedly shown that short,classroom-based courses in EBM successfully transferEBM knowledge and often improve critical appraisalskills and attitudes to EBM for medical students andnewly qualified doctors, compared with no interven-tion or a �traditional medical syllabus�.1,6,8,9 Specific-ally, the directed learning intervention was evaluated

in a previous RCT by Taylor et al.,8 where knowledgeand critical appraisal skills improved significantly inthe intervention group compared with the non-intervention group. In this trial, a pretest of studentsin a non-compulsory examination was judged to belikely to result in low compliance. Therefore, someimprovement in knowledge and critical appraisalwere assumed for the direct learning group, and thisstudy group functioned as a control comparison(R. Taylor et al., National NHS Research andDevelopment Programme, 1999, unpublishedresults).

This assumption allowed the interventions to becompared only by a direct post-test comparison andalthough this approach may have been slightly lessrobust (as opposed to pre- and post-test), it avoidedthe consequences of a potential �learning effect�where outcome measures become invalid becauseparticipants have learned how to respond to themthrough pre-intervention exposure.

The CAL-based, self-directed intervention, was chosenfor comparison, as such educational approaches arebecoming increasingly popular.10,11 Systematic reviewshave suggested that few trials have been conductedevaluating these approaches12 (and see R. Enock,University of Oxford, UK, 2000, unpublished results).

METHODS

Participants, setting and inclusion criteria

Three cohorts of tenth semester medical students atthe University of Oslo, Norway took part in the study.The first cohort received the intervention in spring2002 and the last in spring 2003. All students(n ¼ 281) in the 3 cohorts were eligible for trialinclusion, if they:

1 attended day 1 of the tenth semester or gave areason for their absence on day 1 in advance, and

2 completed consent and baseline characteristicsforms.

Students received information on the course and trialverbally and by e-mail before written consent formswere signed. Person identifiers were used to anony-mise data in a password-protected network system.

The educational interventions

The educational interventions were held at theMedical Faculty at the University of Oslo, the nearby

education studies

Overview

What is already known on this subject

Systematic reviews have repeatedly shown thatshort, classroom-based courses in EBM suc-cessfully transfer knowledge and often im-prove critical appraisal skills and attitudesamongst medical students and newly qualifieddoctors, compared to no intervention or a�traditional medical syllabus�.

What this study adds

This study suggests that computer-assisted,self-directed learning offers a realistic alter-native to an interactive workshop format forteaching EBM. A multistage evaluation meth-od involving a systematic literature review,quantitative and qualitative evaluation proveduseful in interpreting the study’s results.

Suggestions for further research

More research is needed which compareseducational models for EBM courses, consid-ers other professional groups and investigateslongterm effects

1028

� Blackwell Publishing Ltd 2005. MEDICAL EDUCATION 2005; 39: 1027–1035

university hospital and the Norwegian Institute forPublic Health within the first 2 weeks of the 20-weektenth semester and consisted of 5 half-day sessions.Attendance was not compulsory. All studentsreceived 1 of the 2 educational interventions inEBM which were designed to cover the EBM stepsdescribed above. The rest of the time in the first 2weeks was used for the teaching of unrelated topics.After the 2 weeks, students continued with 12 weeksof clinical attachments, split equally between localhospitals and general practice. This was followed by5 weeks of teaching and 1 final week of compulsoryexaminations when primary outcomes were assessed(Fig. 1).

The comparison of the directed and self-directedinterventions aimed to compare 2 distinct educa-tional methods, not just educational media. Theself-directed students were encouraged to beautonomous in their learning and to choose thetime, place and speed of learning, whereas thedirected students used a predetermined model forlearning based on social learning theory. Thesemodels have been described in detail elsewhere.2

Both interventions were pilot-tested with a studentcohort in early 2001.

Outcomes

Primary outcomes

Knowledge about EBM

Knowledge about critical appraisal and searchingtechniques was measured using 1 distinct multiple-choice questionnaire per student cohort (3 in all),consisting of 6 stem questions with 3 sub-questionsper stem. Negative marking was employed for incor-rect answers, so scores ranging from ) 18 to + 18 werepossible, with higher scores indicating better know-ledge.

Critical appraisal skills

Each student critically appraised a scientific paper inan allotted examination time of 60 minutes duringthe end-of-semester examination. An appropriateapproved checklist for critical appraisal was given tostudents. Students were asked to make short notes infree format describing:

1 the study question addressed and the suitability ofthe study design chosen;

Week 20

Directed

Hospital/general practice Clinical attachments

Randomisation of eligiblestudents

Randomisation of students who;1) attended day 1 of 10th semester,

or gave reason for absence in advance 2) completed consent/baseline characterstics

forms

Completion of baseline characteristicsand consent forms

Day 1 of 10th semester(or before)

Day 1 of 10th

semester

Self-directedintervention intervention

University-based teaching (not related to EBM)

Compulsoryexaminations

Attitude questionnaire completed

Knowledge and skills assessment in examination

Intervention conducted

Weeks 3 to 14

Weeks 1 and 2

Weeks 15 to 19

Figure 1 Overview of trial and tenth semester.

1029


2 the methodological quality of the paper (internalvalidity);

3 the results, and4 the generalisability of the results (external valid-

ity) to a specific scenario.

The study was assessed by 2 blinded outcome asses-sors who could allot 0–5 marks to each of the 4sections. Possible scored thus ranged from 0 to 20,with higher scores indicating better levels of skill.

Secondary outcome

Attitudes to EBM

The attitude questionnaire was used to assess atti-tudes about EBM generally. It consisted of 7 state-ments, which were scored by participants on a 5-pointLikert scale in weeks 3–17 of the semester. The scoresfrom the individual statements were totalled, withpotential scores ranging from 0 to 35, where a higherscore indicated a more positive attitude.

The outcomes measures were based on those used inthe trial conducted by Taylor et al. (1999, unpublishedresults), including the validated knowledge and atti-tude questionnaires.13 The questionnaires were trans-lated into Norwegian and then back to English tocheck for inconsistencies. All outcome measures werepiloted with a cohort of tenth semester medicalstudents in 2001, which led to only minor changes. Asthe knowledge questionnaire included new questions(e.g. on searching), it was revalidated.14 In addition,the scale used to score critical appraisal skills differedto that used by Taylor et al. Two outcome assessorsindependently marked students’ responses to thequestionnaires, checking that the same scores wereobtained (LN, PB). For the critical appraisal skillsassignment, a sample answer was prepared (LN, PB).Where scores differed by more than 3 points, assign-ments were independently re-marked and scoresaveraged.

Sample size determination

There were no estimates of variance for the primaryoutcomes. Therefore, detectable standardised effectsizes as units of standard deviation were estimated. Anon-normal distribution was anticipated, so the useof non-parametric statistical methods was assumed.We needed to recruit 66 students per group toachieve 80% power in detecting a difference of 0.50standard deviations. For normally distributedresponses the detectable effect size would be 0.49. Aneffect size of 0.5 was considered �moderate� and

would require, for example, a student currently onthe median centile of an outcome distributionmoving to the top third.15,16 It was decided thatrecruitment would include all consenting students inany cohort, but would stop in the cohort where 75students had been recruited in each study group (toallow for 10% lost to follow-up).

Randomisation, allocation concealment and blinding

For each cohort, author CO created an allocationsequence using random number tables sealed inopaque envelopes, which were ordered numericallybefore the day of randomisation. Another investi-gator, PB, completed a list of students meeting theinclusion criteria on the first day of each semester.CO used this list of students and numerically orderedenvelopes to sequentially allocate the students to theintervention groups on the same day. The allocationsequence for each cohort was known only to CO untilallocation was completed.

Blinding of tutors and students was not possible, butoutcome assessors were blinded by use of study codesand examination candidate numbers, known only toCO. The codes were broken only when data analysiswas completed.

Statistical methods

We summarised the characteristics of all includedsubjects, and compared these across study groupsusing standard t-tests and chi-squared tests. To assessthe internal consistency of the knowledge scale wecalculated Cronbach’s alpha for the 6 items, both bystudy cohort and including all study cohorts.

We summarised outcomes by study group and acrossstudy group. Because outcomes were not normallydistributed, comparisons were made using Wilcoxonrank-sum tests, stratified by cohort. We also calcula-ted 95% confidence intervals for mean outcomes andintervention effects.

RESULTS

Recruitment and participant flow

Students were recruited to the trial for 2 monthsbefore each of 3 semesters, beginning in December2001 and ending in January 2002. A total of 38% ofthe eligible students did not meet the inclusioncriteria. The participant flow and reasons for ineligi-bility are shown in Fig. 2.

education studies1030


Attendance rates were considered to provide indirectmeasures of intervention compliance for the directedlearning group. However, this measure was inappro-priate for the self-directed students, who wereencouraged to choose the pace, time and site of theirlearning. Actual attendance rates per teaching ses-sion varied, at 20)80% for self-directed and 50–85%for directed students. Overall, 35% of self-directed

and 73% of directed students attended at least 3 of 5teaching sessions.

Compliance and contamination

There was an inevitable risk of contamination in thistrial. Subsequently, student attendance registers werekept and measures implemented to prevent students

Eligible study participants (n=281)All 10th semester medical students starting spring 2001, autumn 2001 and spring 2002

Lost to follow-up Knowledge questionnaire (n=2)* Critical appraisal assignment (n=3)*Attitude questionnaire (n=15)**

* Non -attendance at examination, or failure to answer question in examination or answer uninterpretable** Non-response despite repeated requests, reason unknown

AllocatedDirected intervention (n=90, 51%)

Allocated Self-directed intervention (n=85, 49%)

Lost to follow-up Knowledge questionnaire (n=5)* Critical appraisal assignment (n=2)*Attitude questionnaire (n=34)**

* Non-attendance at examination, failure toanswer question in examination or answeruninterpretable.** Non-response despite repeated requests, reason unknown

AnalysedKnowledge questionnaire n=85 (94%)Critical appraisal skills assignment n=88 (98%)Attitude questionnaire n=54 (60%)

AnalysedKnowledge questionnaire n=83 (98%)Critical appraisal skills assignment n=82 (96%)Attitude questionnaire n=57 (67%)

Students not randomised (n=106, 38%) Excluded by authors (n=1 of 106, 1%)*Did not answer invitation (n=52 of 106, 49%)Answered, but refused consent (n=53 of 106, 50%)*did not complete baseline characteristics form

Eligible for randomisation(n=175, 62%)

Figure 2 Participant flow.

1031


attending teaching sessions for the opposing studygroup. Despite this, 2 self-directed students attended1 of the 5 directed-learning sessions, but were notallowed further access to these sessions. This was notconsidered to be a major protocol breach.

The use by directed-learning students of the self-directed students’ CD-ROMs was more difficult toprevent or assess. Students interviewed in theaccompanying qualitative evaluation did not cite anyspecific cases of this having occurred.2 However, thisdid not preclude the use of all self-learning tech-niques. In any case, directed-learning students couldnot access tutor support promoting self-directedlearning, as this was only available at scheduled timesand attendance registers were kept.

In the first cohort, the University of Oslo inadvert-ently distributed the marking framework (intendedfor assessors) to the students during the examination.This potentially serious threat to internal validity wasassessed by a sensitivity analysis (see below).

Baseline data and numbers analysed

Before randomisation, baseline characteristics for allstudy participants were measured. These are shown inTable 1, summarised by intervention group. The onlysignificant differences between the groups concernedgender, with a higher percentage of females in thedirected group (55.6% versus 40.5%).

As the primary end-points were measured by acompulsory examination, loss to follow-up was low. Atotal of 168 of 175 students (96%) completed theknowledge questionnaire and 170 of 175 (97%) thecritical appraisal skills assignment. Only 111 of 178students (62%) completed the attitude question-naire – the secondary outcome, despite repeatedrequests. Reasons for loss to follow-up are shown inFig. 2.

The data were analysed using an intention-to-treatanalysis, where participants partially or not receivingthe interventions were analysed in the group to whichthey had originally been randomised. Internal con-sistency of the knowledge scale ranged from 0.52 to0.61 across the 3 cohorts with a Cronbach’s alpha of0.57 for the 3 cohorts combined.

Outcomes and estimates

The results are shown in Table 2. The interraterkappa score for the critical appraisal skills assign-ment was 0.70, indicating a good level of agreementbetween assessors. There were no statistical differ-ences between the study groups for any of theoutcomes. The scores obtained by medical studentswere generally higher than those obtained post-intervention in the trial by Taylor et al., despite thetrials having used the same scoring indices(Table 3), suggesting that learning had actuallytaken place.

education studies

Table 1 Baseline characteristics of 175 included students

Characteristic Directed (n) Self-directed (n) Total (n) P-value*

n 90 85 175Female 55.6% (50) 40.5% (34) 48.3% (84) 0.05Age (years) �

20–29 90.0% (81) 89.3% (75) 89.7% (156)30–39 8.9% (8) 10.7% (9) 9.8% (17)39 + 1.1% (1) 0.0% (0) 0.6% (1)

Previously attended course in EBM 8.9% (8) 11.8% (10) 10.3% (18) �Skills in using Internet 77.8% (70) 83.5% (71) 80.6% (141) �Access to Internet at home 50.0% (45) 57.6% (49) 53.7% (94) �Performed search in biomedical

database in last 6 months 81.1% (73) 78.6% (66) 79.9% (139) �Ever received training in:

(a) research methodology 25.6% (23) 17.6% (15) 21.7% (38) �(b) epidemiology 1.1% (1) 3.5% (3) 2.3% (4) �(c) statistics 25.6% (23) 22.4% (19) 24.0% (42) �(d) literature searching 14.4% (13) 10.6% (9) 12.6% (22) �

* Fisher’s exact test.� P > 0.25.

1032


Tutors’ and students’ experiences of the programmesas evaluated in the qualitative evaluation are sum-marised in Fig. 3 and reported in detail elsewhere.2

Ancillary analyses

To improve precision and to account for protocolbreaches and variations across groups in studentcharacteristics, supplementary, pre-specified regres-sion models (1 for each outcome) were estimatedadjusting for gender and cohort. This did not lead toany appreciable changes in the results. Exclusion ofthe cohort that inadvertently saw the marking frame-work produced similar results (P ¼ 0.31 for know-ledge, P ¼ 0.32 for critical appraisal skills and P ¼ 0.48

for attitudes); moreover, the test of heterogeneity ofeffect across the cohorts was non-significant for all 3outcomes (P ¼ 0.07 for knowledge, P ¼ 0.52 forcritical appraisal skills and P ¼ 0.44 for attitudes).

DISCUSSION

Interpretation

Overall, the results suggest that computer-assisted,self-directed learning offers a realistic alternative toan interactive workshop format in teaching EBM,although a single trial is insufficient to draw any firmconclusions about the intervention’s relative quality.

This trial overcame some of the previously reportedmethodological limitations by using adequate rand-omisation and allocation procedures, using validatedoutcome measures, achieving high follow-up rates,using intention-to-treat analysis and successfullyblinding outcome assessors. The interpretation of theresults was facilitated by an accompanying qualitativeevaluation, systematic review and by explicitly basingthe interventions on educational theory.

The accompanying qualitative evaluation2 stronglysuggested that the 2 educational programmes weresufficiently distinct to allow a true comparison andwere implemented (to a large extent) according tothe project plans based on educational theory. Moststudents and tutors evaluated the programmes pos-itively, and more highly than teaching received on

Table 2 Results of intervention

Outcome

Group Difference inDirected Self-directed Total Means P-value*

EBM knowledge�Mean score (SE) 12.5 (0.4) 12.6 (0.3) 12.5 (0.3) 0.0 0.80195% CI (11.8, 13.3) (11.9, 13.2) (12.0, 13.0) () 1.0, 1.0)n 85 83 168

Critical appraisal skills�Mean score (SE) 11.8 (0.4) 11.7 (0.3) 11.7 (0.3) 0.1 0.53695% CI (11.1, 12.5) (11.1, 12.3) (11.3, 12.2) () 0.9, 1.1)n 88 82 170

Attitudes to EBM�Mean score (SE) 27.7 (0.4) 28.0 (0.4) 27.9 (0.3) ) 0.3 0.50295% CI (27.0, 28.4) (27.2, 28.8) (27.3, 28.4) () 1.4, 0.8)n 54 57 111

* P-value based on Wilcoxon rank-sum test.� Higher scores indicate better knowledge, skills or more positive attitudes.

Table 3 Comparison of results from this trial with theTaylor et al. (1999) trial

Outcome Bradleyet al.*

Tayloret al.�

EBM knowledge(mean score)

12.5 9.7

Critical appraisal skills(mean score)

� �

Attitudes to EBM(mean score)

27.9 25.0

* Average of scores from both interventiongroups.� Score from intervention group.� Scales and scores not directly comparable.

1033


other topics, suggesting they were of adequatequality. There was no evidence of major study group�contamination�, as measures were taken to preventthis, as reported elsewhere.2 Nonetheless, the direc-ted intervention did not preclude the use of self-directed techniques and the impact of contaminationis difficult to fully assess. As well as this, varying courseattendance prevented �full� implementation.2 Thisreflects the reality of evaluating and teaching non-obligatory courses.

Uncertainty remains, because the results of this trialare still dependent on multiple interactions, whichmay ultimately be context-specific (see generalisabil-ity below).15,17 In addition, some authors havesuggested that medical students are often so highlymotivated that they will learn effectively whatever thequality of teaching.17 Moreover, the questions used toassess knowledge showed less internal consistencythan when the questionnaires were originally piloted,with an overall Cronbach’s alpha of 0.57. This mayhave reduced the validity of 1 of the primary outcomemeasures and made real differences in knowledgemore difficult to detect.

Generalisability

In addition to the points above, several issues maylimit the generalisability of the study results. Firstly,38% of the students were excluded from the trial. Onthe other hand, this recruitment rate is higher thanin many other trials1 and limited data (n ¼ 5) showsincluded and excluded students generally displayedsimilar background characteristics (not reported).

Secondly, behavioural outcomes were not consid-ered, as the educational interventions were judged

to be too short. This view is supported by theconclusions of the accompanying systematic review6

and the views of interviewed students.2 Nonetheless,longterm effects on patient outcomes, professionalskills and behaviour are of prime interest for suchtrials.

Thirdly, the accompanying qualitative evaluationdescribed the educational processes observed2 butdid not identify which factors or combination offactors were most important for student learning. Onthe other hand, there was no reason to suggest thatsimilar courses could not be introduced at othermedical schools, except that the Oslo students hadprior experience in problem-based learning (PBL),which may have facilitated small-group working in thedirected learning group.

Fourthly, the comparison did not include a formaleconomic evaluation. Two tutors were present ateach teaching session and general administrativetasks (e.g. booking rooms) were almost identical forboth groups. Therefore, the differences in costsconcerned the price of teaching materials andcourse material preparation. The directed learningstudents received 2 teaching packs, developed andproduced in-house (which had taken approximately40 hours to develop and took 10 hours to admin-ister each semester). The self-directed studentsgained access to an Internet site, which had takenabout 100 working hours to develop, and received 1copy of an externally produced CD-ROM, atapproximately £30.00 (UK) per copy, depending onthe quantity purchased. The time spent administer-ing these latter teaching resources was minimal. Inour trial, teaching venues were provided free ofcharge by the University of Oslo. In other settings,

education studies

Data collection began with participant observation during teaching sessions and continued with semistructured interviews, focus groups and a written questionnaire for the first 2 cohorts of students in the trial.

Overall 98% of self-directed and 97% of directed students described the courses as useful or very useful and a good use of time.Participant observation and student interviews suggested the programmes were sufficiently distinct to allow a true comparison and were implemented according to the project plans based on distinct educational theories. However, there was varying uptake of the programmes because of non-attendance. Students unanimously stated that EBM should be taught earlier in their studies. In all, 9% of self-directed and 3% of directed students felt the programmes had been poorly organised.

There were only a limited number of reports of ‘contamination’ between the study groups and no specific cases were identified. Most students felt that neither course organisation nor being a research participant had adversely affected learning.

Tutors and a majority of students suggested that personality, learning style and previous learning experience determined preference for learning methods,whereas gender was felt to be unimportant. Despite this, the majority of students felt the programme they had been allocated was best for them. Nonethe less, many students were unsure about their ability to translate their new EBM skills to clinical practice.

Figure 3 Main results from the qualitative evaluation.

1034


these might lead to cost differences. The self-directed students received 5 separate 3-hour com-puter room sessions with 1 computer per person,whereas the directed students used the computerrooms in only 2 of 5 sessions, otherwise usingordinary classrooms.

The interpretation of results comparing the 2educational programmes in the RCT was facilitatedby planning the intervention on the results ofexisting research and theory and simultaneouslyevaluating the programmes qualitatively. Theauthors suggest that this step-wise evaluation modelcould be useful in evaluating other educationalinterventions.

Finally, this trial has only compared 2 of manypossible methods for teaching EBM and focused onmedical students. More research comparing educa-tional models for EBM courses, considering otherprofessional groups and investigating longtermeffects is still needed.6

Contributors: PB was responsible for project co-ordination, the conception and design of the trial, datainterpretation and drafting the article. CO contributed tothe conception and design of the trial, and revision of thearticle. JH was responsible for data analysis and contributedto data interpretation and drafting the article. LNcontributed to the conception and design of the trial, datainterpretation and revision of the article. AB contributed tothe conception and design of the trial and revision of thearticle. All authors approved the final version of the paper.Acknowledgements: we would like to thank Amanda Burls,and Josie Sandercock at the University of Birmingham forcommenting on the protocol; Julia Bradley-Norman at theNorwegian Centre for Health Services for editorial adviceand the course tutors and administrative staff at theMedical Faculty at the University of Oslo for supporting theteaching programme.Funding: none.

Conflicts of interest: PB and LN are actively involved inthe CASP International Network (linked to the CriticalAppraisal Skills Programme [CASP], which developed andfinancially subsidised the use of the educational materials).PB has received payment from CASP for workshopfacilitation on several occasions during the last 5 years.Ethical approval: this research was approved by the localethical committee in 2001.

REFERENCES

1 Hyde C, Parkes J, Deeks J, Milne R. Systematic Reviewof Effectiveness of Teaching Critical Appraisal. Oxford:ICRF ⁄NHS Centre for Statistics in Medicine 2000.

2 Bradley P, Oterholt C, Nordheim L, Bjørndal A. Med-ical students’ and tutors’ experiences of directed andself-directed learning programmes in evidence-basedmedicine: a qualitative evaluation accompanying arandomised controlled trial. Evaluation Rev2004;29:1–77.

3 Hatala R, Guyatt G. Evaluating the teaching of evi-dence-based medicine. JAMA 2000;288(9):1110–1.

4 Stacey R, Spencer J. Assessing the evidence in qualit-ative medical education research. Med Educ 2000;34(7):498–500.

5 Buckley G. Partial truths – research papers in medicaleducation. Med Educ 1998;32:1–2.

6 Bradley P, De la Harpe D, Nordheim L, Innvær S,Thompson C. Educational interventions for evidence-based practice (EBP). A review of the qualitative lit-erature. http://www.mysite.wanadoo-members.co.uk/Cochrane_Qual_Method/srmodule.htm. [AccessedApril 2004.]

7 Sackett DL, Rosenberg WMC, Gray JAM, Haynes RB,Richardson WS. Evidence-based medicine: what it isand what it isn’t. BMJ 1996;312:71–2.

8 Taylor R, Reeves B, Ewings P, Binns S, Khan PE, Keast J,Mears R. A systematic review of the effectiveness of crit-ical appraisal skills training for clinicians. Med Educ2000;34:120–5.

9 Parkes J, Hyde C, Deeks J, Milne R. Teaching criticalappraisal skills in health care settings. (CochraneReview.) In: The Cochrane Library. Issue 3. Oxford:Update Software.

10 Campbell JK, Johnson C. Trend spotting: fashions inmedical education. BMJ 1999;318:1272–5.

11 Spencer JA, Jordan RK. Learner-centred approaches inmedical education. BMJ 1999;318:1280–3.

12 Greenhalgh T. Computer-assisted learning in under-graduate medical education. BMJ 2001;322:40–4.

13 Taylor R, Reeves B, Mears R, Keast J, Binns S, Khan PE.Development of a questionnaire to evaluate theeffectiveness of evidence practice teaching. Med Educ2001;35(6):544–7.

14 Bradley P, Herrin J. Development and validation of aquestionnaire to measure knowledge on evidence-based practice and searching skills. Med Educ Online2004. Available at http://www.med-ed-online.org/volume9.htm

15 Colliver J. Effectiveness of problem-based learningcurricula: research and theory. Acad Med 2000;75(3):259–66.

16 Cohen J. Statistical Power Analysis for the Behavioral Sci-ences. Hillsdale, New Jersey: Lawrence Erlbaum Associ-ates 1988.

17 Albanese M. Problem-based learning: why curricula arelikely to show little effect on knowledge and clinicalskills. Med Educ 2000;34:729–38.

Received 25 August 2004; editorial comments to authors 26August 2004, 21 October 2004; accepted for publication 8December 2004


1035

Comparison of directed and self-directed learning in evidence-based medicine: a randomised...

Documents

Transcript of Comparison of directed and self-directed learning in evidence-based medicine: a randomised...