Perceptions of how well graduates are prepared for the role of pre-registration house officer: a...

Perceptions of how well graduates are prepared for the roleof pre-registration house of®cer: a comparison of outcomesfrom a traditional and an integrated PBL curriculum

Alison Jones, Patricia J McArdle & Paul A O'Neill

Introduction Most medical schools in the UK have been

engaged in major curriculum reform based on their

premises of what might improve undergraduate medical

education. In 1994 the course at the medical school of

the University of Manchester changed to an integrated

course using problem-based learning throughout and

with increased emphasis on community-based medical

education. This study explores whether the new curri-

culum has produced any differences in perceptions of

how well graduates are prepared for the role of pre-

registration house of®cer.

Methods A postal questionnaire was used to survey 1998

Manchester graduates (traditional course) and 1999

Manchester graduates (new course), three months into

their ®rst pre-registration house of®cer placement. A

similar questionnaire was sent to the educational

supervisors who were supervising the graduates. The

questionnaire was designed to measure perceptions of

levels of preparedness for the role of pre-registration

house of®cer, using a list of broad areas of competence

and speci®c skills listed in the General Medical Council's

`The New Doctor'.1

Results Graduates rated the new course signi®cantly

more effective for 12 of the 19 broad competences and

eight of the 13 speci®c skills that were listed. The `new'

graduates rated their understanding of disease pro-

cesses lower than the `traditional' graduates, but there

was no difference in the ratings given by the educational

supervisors for this. Overall the educational supervisors

rated the new course as better preparing graduates in

®ve of the competences.

Conclusions Overall, the evaluation shows that a major

change in curriculum approach has changed the pro®le

of the perceived preparedness of graduates for entering

professional practice.

Keywords Comparative study; Curriculum; Education,

medical, undergraduate/*standards; Great Britain;

problem-based learning/methods; professional compe-

tence; questionnaires.

Medical Education 2002;36:16±25

Introduction

During the last decade, recommendations on medical

education have been published that have strongly

in¯uenced undergraduate curriculum development

in the UK.1±3 The course at Manchester has been

reformed to re¯ect these proposals and also to re¯ect the

body of research on the best ways to facilitate under-

graduate learning. In 1994 the new undergraduates at

Manchester embarked on a course that was markedly

different to that of their predecessors. The traditional

pre-clinical/clinical curriculum was replaced by one that

is integrated across disciplines, more learner-centred,

with increased emphasis on the delivery of medical

education in the community rather than in the hospital

setting, and uses problem-based learning (PBL) as the

method for delivering core knowledge and under-

standing. Students work in small groups to study written

descriptions of clinical situations, working with a sys-

tem-based rather than discipline-based approach. They

use a series of steps as a guide to their discussion of the

clinical scenarios, meeting two or three times each week

in small groups facilitated by a tutor. In between tuto-

rials, the students use a variety of resources to meet their

learning objectives; in year three onwards these

resources include clinical experience.

There is limited evidence in the literature on differ-

ences in outcomes between traditional and PBL courses

at the level of newly graduated doctor. The ®ndings from

Medical Education Unit, University of Manchester, Faculty of

Medicine, Dentistry, Nursing and Pharmacy

Correspondence: Alison Jones, Medical Education Unit, Ist Floor,

Rusholme Health Centre, Walmer Street, Manchester M14 5NP, UK.

Tel.: 0161 256 3015 x 289; Fax: 0161 256 1070; E-mail: alisonj@

fs1.with.man.ac.uk

Preparing for the job

16 Ó Blackwell Science Ltd MEDICAL EDUCATION 2002;36:16±25

published studies have varied from better `clinical

functioning' by PBL students, better performance in

knowledge tests by traditional course students, greater

awareness of recent guidelines by PBL students, through

to little or marginal bene®ts.4±6 In these studies, the

schools using PBL have mostly con®ned the use of this

method of learning to the early part of the curriculum.

The use of clinical supervisor rating forms to com-

pare perceptions of the performance of graduates from

traditional and PBL schools has shown that PBL

graduates perform better in respect of interpersonal

relationships, reliability and self-directed learning. It

was found that graduates from one of the two tradi-

tional schools were rated higher for teaching, diagnostic

skills and understanding of basic mechanisms.7

A common element among such studies is that the

comparisons are between students or graduates of dif-

ferent schools and in doing so there are dif®culties in

drawing conclusions on separating out the effect of

curriculum design from the overall context of the

school.5,7,8 At Manchester, with the radical change in

methods of curriculum delivery, we had the oppor-

tunity to compare graduates of two different courses

within the same institution.

Our hypothesis was that there would be differences

in perceptions of graduates' levels of competence in

some of the speci®c skills or more broad areas as de-

®ned by the General Medical Council as the aims of

general clinical training for the `New Doctor'.1

Amongst other things, our study looked at whether

changing the methods of teaching and learning could

improve the preparedness of graduates for important

areas such as communication and team-working,

which are of topical interest given the media interest in

the performance of doctors and emphasis on

performance review.

Undergraduate medical educationat Manchester

Manchester is a large UK medical school admitting

(in 1994) over 240 students in the ®rst year, with an

additional 90 students joining the course in the third

year from St Andrews University. Up until 1994 the

students followed a traditional ®ve year course with two

years of pre-clinical and three years of almost entirely

hospital-based clinical teaching. For the 1994 intake

onwards, the school introduced PBL in small groups as

the major method for delivering core knowledge and

understanding, with the curriculum integrated across

disciplines and the learning being systems-based.

We have described the new curriculum in detail

elsewhere, thus only a brief description is given.9,10 The

outline of the new ®ve-year curriculum is shown in

Figure 1. SSMs are `special study modules' in which

students have wide choice in what they study or gain

experience in.2 Assessment has also changed from

being departmental to being integrated across disci-

plines and has been designed to match the course

objectives closely.

Distinct from almost all of schools using PBL, the

students continue to use this method of learning when

they move from the medical school to a hospital base

for years three and four.9 The PBL approach used in

Manchester is high on the hierarchy set out by

Barrows.11 In years three and four, a group of students

meet for one hour with a tutor to discuss a case using a

series of steps. The students set group learning objec-

tives (questions) at the end of the ®rst discussion ses-

sion and then meet again after one week for 90 minutes

to discuss their ®ndings.

As the students in year three onwards are working in

a clinical environment, the students can seek out clin-

ical experience as well using, for example, books, arti-

cles and lectures to answer their learning questions. We

encourage the students in group discussions to make

connections between the paper case and the patients

they have seen.

The PBL cases consist of clinical vignettes that

integrate diagnosis, investigation and management as

well as the relevant basic and clinical sciences and the

wider psychosocial aspects of clinical medicine. Over

the four years, the cases are organised into modules

lasting for one semester each, and each module has an

overarching theme. During these semesters/modules

the students cover the core curriculum based on a list of

`index clinical situations' (ICSs ± consisting of 61

presentations and 154 diseases or syndromes) drawn

from repeated consultations with hospital consultants

and general practitioners in the region.9

Key learning points

Surveying house of®cers and educational super-

visors was used for generating outcome evaluation

data in order to compare `products' of different

undergraduate courses.

Graduate ratings of their levels of preparedness

were affected by a change in curriculum, with

innovative course graduates providing more

favourable ratings for several items.

Educational supervisors perceived some change

in the preparedness of house of®cers in the set

of competences.

Ó Blackwell Science Ltd MEDICAL EDUCATION 2002;36:16±25

How well prepared are graduates for the role of pre-registration house of®cer? · A Jones et al. 17

In addition to the use of PBL, there are other major

differences between the new and the old curriculum. In

the new course, whilst students are attached to clinical

®rms in hospitals, there is also a much greater emphasis

on using the community as part of their experience.2

We wanted to give students a better appreciation of the

Figure 1 Diagram of course.

How well prepared are graduates for the role of pre-registration house of®cer? · A Jones et al.18


context of medical care, including the importance of

broad health determinants and the relevance of the

interactions between health problems and social issues.

In order to achieve these aims, one day per week in

the core modules in the third and fourth year is set

aside for students to make use of community res-

ources,which are usually based around a particular GP

practice.12

In year ®ve, the students rotate through teaching

hospital, district general hospital, community and elec-

tive placements. In the ®rst three of these there is weekly

group work in which the approach to PBL is extended by

the students setting the agenda through discussing

paients that they have encountered in their clinical

attachments rather than having set paper cases. The ®nal

few weeks in the ®nal year on the new course are spent

shadowing the pre-registration house of®cer whose post

they will be ®lling.

After graduating from the course, students take up a

pre-registration house of®cer position, usually staying in

the Northwest region for their ®rst post. During their ®rst

year they have an allocated educational supervisor who is

responsible for monitoring their progress. These super-

visors are NHS consultants who have been on an

appropriate development course. Supervisors are res-

ponsible for a varying number of graduates and some will

have more contact with their PRHOs than others.

Methods

In order to obtain comparative data on the perceptions

of the knowledge, skills and attitudes of the Manchester

graduates we surveyed the graduates and their educa-

tional supervisors over a two-year period.

We surveyed the last group of students to graduate

from the traditional course (in 1998) and the ®rst group

to graduate from the new PBL course (in 1999). For

these two cohorts selection to the medical course was

based on their `A' level grades or equivalent and a

reference from their school. Manchester did not routi-

nely interview its applicants until 1995, so neither

cohort in the present study had been interviewed before

admission. For ease of tracking non-respondents, gra-

duates whose home address was overseas were excluded

from the study. The two groups of graduates were

surveyed at the same point, i.e. three months into their

®rst pre-registration house of®cer placement.

Educational supervisors were selected for the survey

if they were working in the North-west region and were

supervising Manchester graduates. Again, two cohorts

were surveyed; they comprised those supervising the

last group of graduates from the traditional course and

those supervising the ®rst group of graduates from the

new course. Most educational supervisors of house

of®cers carry out the role for several years and so almost

all of those in the second group would have been the

same as those surveyed the previous year.

In order to obtain data from a large sample, a ques-

tionnaire was designed. We used the New Doctor1 as

the basis for de®ning the competencies required of a

graduating medical student rather than using the

objectives of the new Manchester curriculum, as we

considered that these might have biased the results in

favour of the new course. We also wanted to evaluate

the curriculum against national recommendations. The

New Doctor lists broad areas of competence, e.g.

`communicating effectively' as well as speci®c skills e.g.

`suturing', which in turn represent composites of

knowledge, skills and attitudes that should be built on

in `general clinical training'.

There were two sections on the questionnaire, the

®rst asking graduates to consider `How well did the course

prepare you for¼?' followed by a list of broad areas of

competence as de®ned in the New Doctor. In relation

to competence in more speci®c procedures, the ques-

tion was `How well did the course provide you with a

competence in¼?'. Respondents were required to rate

their answers on a ®ve point scale, with `very well

prepared/competent' and `not at all well prepared/

competent' as the range; the mid-point label was `quite

well prepared/competent'.

A similar questionnaire was sent to the educational

supervisors. The question asked of educational super-

visors was `Please rate the Manchester PRHOs on their

competence in the following¼', with the same list of items

as those on the graduate questionnaire. As educational

supervisors tend to supervise more than one PRHO at a

time they were asked to respond on the basis of a

general perception of graduates rather than on ratings

of an individual. Supervisors were asked to leave an

item blank if they felt unable to provide a rating.

Follow up letters and copies of the questionnaire

were sent to non-respondents, both graduates (four

follow-ups) and supervisors (one follow-up). Graduate

non-respondents were followed up ®rst at their home

address and then at their PRHO placement.

Analysis of the data was done using SPSS for

Windows version 7. Raw data were used for the ana-

lyses of variances between groups, although for ease of

presentation some results have been combined into

fewer groups. Non-parametric tests (Mann±Whitney

(U) were used to explore any signi®cant differences

between groups. As slightly different wordings of

questions were used for graduates and supervisors, it

was not appropriate to test for statistically signi®cant

differences between the two groups.



Results

Graduates

The response from the traditional graduates was 171/256

(67%) and 163/267 (61%) from the graduates of the new

course. For the purpose of comparing traditional and

new course outcomes those graduates who had taken

more than ®ve years to complete their studies, for

example those who had done an intercalated degree,

were excluded. This was because those who graduated in

1999 and had commenced before 1994 would have done

part traditional and part new course. In order to avoid

any bias from reducing the new course cohort, the same

rule was adopted to exclude respondents from the tradi-

tional cohort. Consequently, the analysis was carried out

on the data from 123 (48%) traditional course graduates

and 138 (52%) new course graduates.

Solely for the purpose of presentation, the data were

recoded into the number rating above the mid-point on

the ®ve-point scale, those rating at the mid-point and

those below the mid-point, i.e. reduced to three groups.

All statistical tests were based on the raw data. Statis-

tical signi®cance was set at P ³ 0á01 because of the

multiple comparisons.

Table 1 shows the comparative data for graduate

responses to how well they felt prepared in broad

areas of competence. Responses to 12 of the 19 items

were signi®cantly different between the two groups.

All but one of these (`understanding disease pro-

cesses') were rated more favourably by the new course

graduates.

Table 1 Graduate ratings of broad competencies. `How well did the course prepare you for¼?'

% rating themselves as

% responding

to item

More than quite

well prepared

Quite well

prepared

Less than quite

well prepared

Item 1999 1998 1999 1998 1999 1998 1999 1998 P-value

History taking, clinical examination &

selection & interpretation of diagnostic tests

99á3 99á2 66á4 77á9 29á9 22á1 3á6 ± ns

Understanding disease processes 99á3 99á2 40á1 77á9 35á8 21á3 24á1 0á8 0á000

Communicating effectively 99á3 99á2 83á2 58á2 13á9 32á8 2á9 9á0 0á000

Being aware of your own limitations 99á3 99á2 69á3 57á4 25á5 38á5 5á1 4á1 ns

Working in a team 99á3 99á2 83á2 50á8 12á4 34á4 4á4 14á8 0á000

Recognition of the social & emotional factors

in illness and treatment

97á8 99á2 68á9 43á4 25á2 37á7 5á9 18á9 0á000

Keeping accurate records 98á6 99á2 37á5 37á7 45á6 36á1 16á9 26á2 ns

Using opportunities for disease prevention &

health promotion

98á6 99á2 33á8 33á6 52á9 41á0 13á2 25á4 ns

Managing time effectively 97á8 99á2 36á3 28á7 43á7 41á8 20á0 29á5 ns

Making the best use of laboratory & other

diagnostic services

98á6 99á2 30á9 32á8 49á3 49á2 19á9 18á0 ns

Understanding the relationship between

primary & social care & hospital care

97á8 99á2 68á1 28á7 28á1 45á9 3á7 25á4 0á000

Developing appropriate attitudes towards

personal health & wellbeing

97á1 99á2 43á3 27á0 35á8 41á8 20á9 31á1 0á006

Understanding the principles of evidence-

based medicine

97á8 97á6 53á3 26á7 33á3 35á8 13á3 37á5 0á000

Diagnosis, decision making & the provision

of treatment including prescribing

98á6 99á2 35á3 25á4 42á6 43á4 22á1 31á1 ns

Coping with uncertainty 98á6 99á2 50á0 23á8 28á7 35á2 21á3 41á0 0á000

Understanding the purpose & practice of

audit, peer review and appraisal

98á6 99á2 41á9 22á1 39á7 35á2 18á4 42á6 0á000

Providing appropriate care for people of

different cultures

99á3 99á2 51á1 18á9 30á7 31á1 18á2 50á0 0á000

Using informatics as a tool in medical practice 97á1 95á9 48á5 19á5 32á8 25á4 18á7 55á1 0á000

Being aware of legal & ethical issues 98á6 99á2 46á3 18á0 44á1 32á8 9á6 49á2 0á000

ns � not signi®cant.



The responses from the two cohorts of graduates

in response to the question about preparedness for

performing speci®c skills are shown in Table 2.

Responses to eight of the 13 items were signi®cantly

different between the two groups, with responses from

the new course graduates being more favourable.

Educational supervisors

There were 194 educational supervisors in the North-

west region who were supervising Manchester gradu-

ates from the traditional course and 218 supervisors of

the new course graduates. All were sent questionnaires,

but ®ve supervisors of the traditional course graduates

and 12 of the new course graduates responded stating

that they could not complete a questionnaire either

because they were no longer supervising graduates or

felt unable to generalise having had too few graduates

under their supervision. In total, 151 (80%) completed

questionnaires were received from supervisors of tradi-

tional course graduates and 169 (82%) from new course

graduates.

As with the graduate questionnaire, data were

recoded into just three groups for clarity of presenta-

tion, with statistical tests based on the raw data and

signi®cance set at P ³ 0á01.

The responses of the supervisors as to how well they

thought graduates were prepared in broad areas of

competence are shown in Table 3. There was a statis-

tically signi®cant difference between the ratings of the

traditional and new course graduates for ®ve of the

18 listed competencies, with the graduates from the

new course being rated as more prepared.

There were no signi®cant differences for any of the

speci®c skills (Table 4).

Discussion

Results from graduates tended to favour the new course,

although the less favourable rating for one of the broad

areas of competence, understanding disease processes,

is of some concern. The results from supervisors were

similar for the two courses in terms of speci®c skills, but

were supportive of the new course for broad areas of

competence. These data suggest that in part the aims of

the innovative course have been met although the per-

ceived reduction in preparedness for `understanding

disease processes' needs more exploration.

The measurement of outcomes of undergraduate

medical education raises speci®c methodological issues.

This study attempts to measure perceptions of the

preparedness of graduates and compare those percep-

tions across courses. We do not claim to have measured

or reported differences in performance once graduates

are working as pre-registration house of®cers. It is

known that self-assessment may not be closely corre-

lated to actual performance.13

In our comparative analysis, we picked one point in

the graduate's career at which to collect the data on

perceptions of preparedness. It could be argued that

Table 2 Graduate ratings of speci®c skills. `How well did the course provide you with a competence in¼?'

% rating themselves as

% responding

to item

More than quite

competent

Quite

competent

Less than quite

competent

Item 1999 1998 1999 1998 1999 1998 1999 1998 P-value

Venepuncture 98á6 99á2 89á7 70á5 7á4 23á0 2.9 6.6 0á001

Basic CPR 99á3 99á2 70á8 65á6 24á8 23.0 4á4 11á5 ns

Arterial blood sampling 99á3 99á2 54á0 50á0 21á2 30á3 24á8 19á7 ns

Administering oxygen therapy safely 98á6 99á2 30á9 27á0 29á4 29á5 39á7 43á4 ns

Urinary catheterisation 99á3 99á2 58á4 32á8 21á2 25á4 20á4 41á8 0á000

Obtaining valid consent 98á6 97á6 44á1 25á0 26á5 24á2 29á4 50á8 0á000

Performing an ECG 99á3 99á2 71á5 26á2 16á8 20á5 11á7 53á3 0á000

Writing a prescription 99á3 98á4 39á4 19á0 33á6 24á0 27á0 57á0 0á000

Control of haemorrhage 99á3 99á2 14á6 14á8 35á0 28á7 50á4 56á6 ns

Calculating accurate drug dosages 99á3 98á4 24á8 14á9 27á0 25á6 48á2 59á5 ns

Correctly using a nebuliser 99á3 99á2 21á2 10á7 25á5 23á0 53á3 66á4 0á002

Suturing 99á3 99á2 11á7 9á8 21á2 8á2 67á2 82á0 0á010

Inserting a nasogastric tube 99á3 99á2 8á8 4á1 16á8 8á2 74á5 87á7 0á005




differences in outcomes may be more or less apparent

at a later stage in their career and so we do not have a

true picture of the long-term effect of either the tradi-

tional or new course.

A further potential drawback is the effect of removing

from the data set the responses of graduates who had

taken more than ®ve years to complete their course. We

could not identify the educational supervisors of these

house of®cers so the supervisor ratings were based on

all Manchester graduates. However, the numbers

involved were small so it is unlikely that this had a

major effect on the results.

The pattern of responses may also have been in¯u-

enced by the educational supervisors of the traditional

course graduates basing their perceptions on Man-

chester graduates observed over a number of years,

whereas the new graduate supervisors, who may have

also replied to the ®rst questionnaire, would have

based their ratings on a small number of PRHOs they

had worked with in the previous few months. How-

ever, it is known that in assessment, supervisors base

their ratings on very recent experience of the trainee

and are heavily in¯uenced by more recent inter-

actions14 so even with the traditional course, supervi-

sors were probably basing their ratings on their house

of®cers at that time.

Interpretation of the results of this study should also

be made in the knowledge that supervisors tend to be

more lenient in their ratings than trainees and that there

is a tendency for supervisors to rate on the basis of their

overall impression of a graduate rather than speci®c

aspects of performance.15 However, others have sug-

gested that this possible `halo effect' is unlikely to cause

signi®cant bias.7

Table 3 Educational supervisor ratings of broad competencies. `Please rate the Manchester PRHOs on their competence in the following¼'

% rating PRHOs as

% responding

to item

More than quite

competent

Quite

competent

Less than quite

competent

Item 1999 1998 1999 1998 1999 1998 1999 1998 P-value

Understanding disease processes 98á8 99á3 40á1 36á0 46á7 48á7 13á2 15á3 ns

Communicating effectively 99á4 98á7 63á1 48á3 31á5 47á0 5á4 4á7 0á006

Being aware of their own limitations 97á0 97á4 64á6 59á2 29á3 38á1 6á1 2á7 ns

Working in a team 99á4 99á3 65á5 56á7 28á6 38á7 6á0 4á7 ns

Recognition of the social & emotional factors

in illness and treatment

92á9 94á0 41á4 40á1 49á0 45á8 9á6 14á1 ns

Keeping accurate records 92á3 95á4 51á9 56á9 35á9 34á7 12á2 8á3 ns

Using opportunities for disease prevention &

health promotion

81á7 81á5 21á7 16á3 52á9 45á5 25á4 38á2 ns

Managing time effectively 97á6 97á4 41á8 30á6 40á6 50á3 17á6 19á0 ns

Making the best use of laboratory &

other diagnostic services

99á4 98á0 35á1 23á6 42á9 51á4 22á0 25á0 ns

Understanding the relationship between

primary & social care & hospital care

91á7 90á7 29á7 21á9 52á3 54á7 18á1 23á4 ns

Developing appropriate attitudes towards

personal health & wellbeing

87á6 86á8 35á1 42á7 58á8 48á1 6á1 9á2 ns

Understanding the principles of evidence-

based medicine

94á1 93á4 35á2 27á7 53á5 56á0 11á3 16á3 ns

Diagnosis, decision making & the provision of

treatment including prescribing

98á8 99á3 33á5 34á7 48á5 48á7 18á0 16á7 ns

Coping with uncertainty 93á5 93á4 34á2 17á0 44á3 54á6 21á5 28á4 0á002

Understanding the purpose & practice of

audit, peer review and appraisal

96á4 92á1 30á7 20á1 46á6 40á3 22á7 39á6 0á001

Providing appropriate care for people of

different cultures

86á4 90á7 55á5 39á4 41á8 50á4 2á7 10á2 0á001

Using informatics as a tool in medical practice 84á0 77á5 38á0 23á9 45á1 51á3 16á9 24á8 ns

Being aware of legal & ethical issues 91á1 92á1 25á3 17á3 53á9 43á9 20á8 38á8 0á001


NB `History taking, clinical examination and the selection and interpretation of diagnostic tests' was not included as an item on the 1998

supervisor questionnaire.



In addition to the general aspects of supervisor rat-

ings, it is also likely that some supervisors may not have

been in a position to comment accurately on some of the

items as they may not have seen graduates performing

certain skills or demonstrating certain broad compe-

tencies. This is supported by the lower response rates

by supervisors to some items on the questionnaire,

particularly in relation to some of the speci®c skills

(Tables 3 and 4). Other supervisors may have simply

rated around the mid-point. It could be argued that

junior medical staff or nurses working with the gradu-

ates may have been better placed to comment on their

performance in speci®c areas.

Given these limitations to the methods, what are the

main ®ndings from the study? Overall, the ratings for

the new course in preparing graduates for practice may

be attributed to the substantial change in methods of

teaching and learning that occurred with curriculum

reform. For example, differences in favour of the new

course for the items `communicating effectively' and

`working in a team', may be linked to the PBL group

work. Others have reported similar results. One study

found that supervisor ratings favoured PBL graduates

in respect of communication skills.15 Similarly, the PBL

graduates in another study were rated signi®cantly

better than traditional course graduates for their rela-

tionships with patients and their families and relation-

ships with other health professionals.7

We also found that the new graduates were better

prepared in terms of `understanding the relationship

between primary and social care and hospital care' and

`providing appropriate care for people of different cul-

tures'. As encouraged by the GMC,2 we have placed a

much greater emphasis on learning in a community

setting as well as integrating this with hospital experi-

ence. Similarly, the graduates from the `New Pathway'

from Harvard medical school expressed more con®d-

ence in the wider aspects of medicine.16

We have previously reported a preliminary study

looking at the attitudes of the new course students in

Manchester compared to those of students from tradi-

tional courses.17 As well as having a more holistic

approach to care and a commitment to continued

learning, the year three students were more positive

about `coping with uncertainty'. This last attitude has

been maintained into practice and was perceived by the

educational supervisors. It has probably been rein-

forced by the nature of studying using PBL, in which

students are encouraged to discuss where sources of

information are contradictory or where there is no

reliable evidence. We believe that this approach will be

helpful to the graduates as they continue with their

postgraduate development.

The difference in ratings for `being aware of legal and

ethical issues' is likely to be a re¯ection of the increased

emphasis on this area in the new course. Others have

reported that medical ethics and law were amongst the

subjects that graduates felt there had not been enough

`factual content' in their training.18 More recently, the

teachers of ethics and law have called for these to be

Table 4 Educational supervisor ratings of speci®c skills. `Please rate the Manchester PRHOs on their competence in the following¼'

% rating PRHOs as

% responding

to item

More than quite

competent

Quite

competent

Less than quite

competent

Item 1999 1998 1999 1998 1999 1998 1999 1998 P-value

Venepuncture 92á3 94á7 50á6 60á1 38á5 33á6 10á9 6á3 ns

Basic CPR 76á9 79á5 33á8 41á7 46á2 43á3 20á0 15á0 ns

Arterial blood sampling 85á8 89á4 41á4 41á5 36á6 32á6 22á1 25á9 ns

Administering oxygen therapy safely 78á7 86á1 39á8 40á0 41á4 41á5 18á8 18á5 ns

Urinary catheterisation 75á7 82á8 39á1 31á2 45á3 45á6 15á6 23á2 ns

Obtaining valid consent 87á0 92á1 42á9 48á9 48á3 40á3 8á8 10á8 ns

Performing an ECG 75á1 78á1 40á2 42á4 44á9 40á7 15á0 16á9 ns

Writing a prescription 91á1 94á7 48á7 51á7 43á5 42á0 7á8 6á3 ns

Control of haemorrhage 61á5 71á5 28á8 22á2 51á0 50á0 20á2 27á8 ns

Calculating accurate drug dosages 85á8 91á4 41á4 47á1 48á3 49á3 10á3 3á6 ns

Correctly using a nebuliser 74á6 78á8 34á1 36á1 46á8 43á7 19á0 20á2 ns

Suturing 50á9 56á3 14á0 15á3 25á6 30á6 60á5 54á1 ns

Inserting a nasogastric tube 53á3 69á5 11á1 18á1 35á6 33á3 53á3 48á6 ns




core subjects in the undergraduate curriculum.19 As we

have an integrated course, we do not have a speci®c

ethics and law module, but we believe that the inclusion

of ethical and legal dilemmas in the PBL cases20 has

gone some way to developing awareness of these in the

Manchester graduates.

Within the broad competences, we found that the

new course graduates did not feel as well prepared for

`understanding disease processes'. However, in pub-

lished reviews, consistent differences in the knowledge

of graduates from the two types of courses have not

been found.4,21 A more recent study compared stu-

dents' performance on knowledge tests between those

who had followed a traditional course and those from a

PBL course. They concluded that the use of PBL

methods during the pre-clinical phase does not dis-

advantage students when they are tested on their basic

science and clinical knowledge.22

In relation to applying knowledge, it has been

reported that students from a PBL curriculum had

greater diagnostic accuracy.8 Similarly PBL students at

Rush Medical College were able to give better patho-

physiological explanations for patients' problems.23 In

Manchester, we found that in the ®nal examination, the

graduates from the new course scored signi®cantly

higher in identical questions in a patient management

problem paper than those from the last two years of the

traditional course (data on ®le: New course 1999: mean

score 62á3% [s.d. 8á8]; Traditional course 1998: mean

score 59á8% [s.d. 9á1]; Traditional course 1997: mean

score 58á0% [s.d. 7á7], P < 0á001).

From our data and the literature, we do not have

evidence that the difference in rating of `understanding

disease processes' does relate to a real gap in prepar-

edness. The educational supervisors in Manchester,

unlike the nurse raters15 or the supervisors of interns

surveyed elsewhere,7 did not rate the new graduates as

being less well prepared in this area. One possible

explanation for our results may be similar to the

`growing pains' or anxieties inherent in the ®rst year of a

new course discussed by Mann and Kaufman.24 They

contrasted the perceptions of the ®rst cohort of

graduates from an innovative Canadian course with

their performance in Part 1 of the Canadian Qualifying

exam and found that their perceptions that their

knowledge base was weak were unsupported.

The second part of the survey concerned speci®c

skills. Others have reported that PRHOs acquire the

necessary skills by the end of the pre-registration house

of®cer year, but that having had those skills to start with

would have made the transition easier.25 Our results

suggest that the development of a core set of skills has

had a bene®cial effect on perceptions of levels of

competence by the new course graduates, but not by

the supervisors. Other probable reasons for the higher

ratings from graduates were the implementation of

skills laboratories and the ®nal year being based entirely

in a clinical environment. At the end of this ®fth year,

the ®nal year students `shadow' the PRHO that they

will be taking over from after graduation and conse-

quently have a good opportunity to improve their skills

where necessary. Our skills curriculum continues

to develop so we hope that supervisors will perceive

improved competence in the future.

Conclusion

Overall, our study shows graduates from an innovative

course feel better prepared for the role of PRHO.

Considering the ®ndings of other studies and our recent

data, there is still a need for further research into the

effectiveness of non-traditional curricula to establish

the extent to which teaching and learning methods can

affect outcomes. The research presented here has

focused on perceptions and there is very little literature

on the actual behaviours of graduates from innovative

courses.5 Given the energy and resources that have

been put into curriculum reform in the UK since the

publication of `Tomorrow's Doctors',2 it is very

important that we seek evidence for any differences

produced. The current study is part of a much larger

programme of evaluation using quantitative and qual-

itative methods, which can be used to build a more

comprehensive picture of the outcomes of medical

teaching at Manchester.

Acknowledgements

We would like to thank the graduates and their

educational supervisors who took part in the survey,

and Professors Carl Whitehouse, Roger Green and

David Gordon for their helpful comments on early

drafts of this paper.

Con¯ict of interest

None declared.

Contributors

AJ and PO'N jointly wrote the paper in consultation

with PM. PO'N initiated, supervised and guided the

project and AJ was responsible for designing the

research instruments and analysing the results. PM

acted as external consultant to the project and assisted

with revision of the manuscript.



Funding

The project is funded by the NHS, Northwest Region.

References

1 General Medical Council. The New Doctor. London: GMC;

1997.

2 General Medical Council. Tomorrow's doctors. London: GMC;

1993.

3 British Medical Association. Report of the working party on

medical education. London: BMA; 1995.

4 Vernon DTA, Blake RL. Does Problem-based Learning

Work? A meta-analysis of evaluative research. Ac Med

1993;68:550±63.

5 Shin JH, Haynes BH, Johnston ME. Effect of problem-based,

self-directed undergraduate education on life-long learning.

Can Med Assoc J 1993;148:969±76.

6 Colliver J. Effectiveness of problem based learning curricula.

Ac Med 2000;75:259±66.

7 Rolfe IE, Andren JM, Pearson S, Hensley MJ, Gordon JJ.

Clinical competence of interns. Med Educ 1995;29:225±30.

8 Schmidt HG, Machiels-Bongaerts M, Hermans H, ten

Cate TJ, Venekamp R, Boshuizen HPA. The development

of diagnostic competence: comparison of a problem-based,

an integrated and a conventional medical curriculum. Ac Med

1996;71:658±64.

9 O'Neill PA. Problem-based learning alongside clinical

experience: Reform of the Manchester curriculum. Educ for

Health 1998;11:37±48.

10 Whitehouse C. Planning for community-oriented medical

education in Manchester. Educ for Health 1996;9:45±59.

11 Barrows HS. A taxonomy of problem-based learning methods.

Med Educ 1986;20:481±6.

12 Whitehouse CR. The community experience in the new

Manchester medical undergraduate curriculum: reactions

to the ®rst module. Educ for Health 1998;11:173±82.

13 Gordon MJ. A review of the validity and accuracy of self-

assessments in health professional training. Ac Med

1991;66:762±9.

14 Ross M. Relation of implicit theories to the construction of

personal histories. Psychol Rev 1989;96:341±57.

15 Santos-Gomez L, Kalishman S, Rezler A, Skipper B, Mennin

SP. Residency performance of graduates from a problem-

based and a conventional curriculum. Med Educ

1990;24:366±75.

16 Peters AS, Greenberger-Rosovsky R, Crowder C, Block SD,

Moore GT. Long-term outcomes of the New Pathway pro-

gram at Harvard Medical School: a randomized controlled

trial. Ac Med 2000;75:470±9.

17 Silverstone Z. Tomorrow's Doctors ± Tomorrow's Attitudes?

Presented at the Association for the Study of Medical

Education Annual Scienti®c Meeting;1997.

18 Clack GB. Medical graduates evaluate the effectiveness of

their education. Med Educ 1994;28:418±31.

19 Consensus group of teachers of medical ethics and law in

UK medical schools. Teaching medical ethics and law within

medical education. A model for the UK core curriculum.

J Med Ethics 1998;24:188±92.

20 Singer PA. Medical ethics. BMJ 2000;321:282±5.

21 Albanese MA, Mitchell SM. Problem-based learning: a review

of literature on its outcomes and implementation issues.

Ac Med 1993;68:52±81.

22 Blake RL, Hosokawa MC, Riley SL. Student performances on

Step 1 and Step 2 of the United States Medical Licensing

Examination following implementation of a problem-based

learning curriculum. Ac Med 2000;75:66±70.

23 Hmelo CE. Cognitive consequences of problem-based learn-

ing for the early development of medical expertise. Teaching

Learning Med 1998;10:92±100.

24 Mann KV, Kaufman DM. A comparative study of problem-

based and conventional undergraduate curricula in preparing

students for graduate medical education. Ac Med 1999;74:

S4±S6.

25 Calman KC, Donaldson M. The pre-registration house of®cer

year: a critical incident study. Med Educ 1991;25:51±9.

Received 23 March 2001; accepted for publication 29 March 2001



Perceptions of how well graduates are prepared for the role of pre-registration house officer: a...

Documents

Transcript of Perceptions of how well graduates are prepared for the role of pre-registration house officer: a...