How well prepared are graduates for the role ofpre-registration house of®cer? A comparison of theperceptions of new graduates and educational supervisors

Alison Jones1, Patricia J McArdle2 & Paul A O'Neill1

Objective The principal aim of undergraduate medical

education is to produce competent pre-registration

house of®cers (PRHOs). We examined and compared

the perceptions of graduates and educational supervi-

sors concerning how well prepared graduates were for

their ®rst post.

Methods A postal questionnaire was sent to house of-

®cers who had graduated from Manchester 3 months

earlier and also to educational supervisors of PRHOs in

the North-west Region. The questionnaires were based

on the competencies set out by the General Medical

Council of the United Kingdom.

Results The response rates were 66% from the gradu-

ates and 76% from the supervisors. Of the 18 broad

areas of competence listed, only four were rated more

than `quite well prepared' by at least 50% of

the graduates (`understanding disease processes',

`communicating effectively', `awareness of limitations'

and `working in a team'). Similarly, more than half of

educational supervisors rated graduates as more than

quite competent in only three areas (`awareness of

limitations', `keeping accurate records' and `working in

a team'). Within the competencies surveyed, there were

differences between the perceptions of graduates and

educational supervisors on the preparedness of gradu-

ates for the skills they may require as a pre-registration

house of®cer.

Conclusion Overall, given that most graduates and

supervisors perceived the preparedness as `quite well'

or less, the undergraduate course had only partially

met its objectives. A mismatch in ratings could be

attributed to either inappropriate expectations on the

part of the educational supervisors or the graduates or

an inaccurate assessment by either group of respond-

ents.

Keywords Education, medical, undergraduate,

*standards; faculty; clinical competence; educational

measurement; curriculum; England; questionnaires.2

Medical Education 2001;35:578±584

Introduction

In 1993, the General Medical Council of the UK1

de®ned the knowledge, skills and attitudes to be

acquired during undergraduate medical education.

These have formed the basis for reform of many

undergraduate curricula. More recently, the compe-

tencies for a pre-registration house of®cer (PRHO)

have also been set down.2 During their undergraduate

years, students are assessed in some of these compe-

tencies. These summative examinations equate to the

`can do' level within the commonly used pyramid of

competency assessment.3 However, the examinations

cannot look at how the students perform skills and use

their knowledge in day-to-day practice.

Although direct observation of performance of spe-

ci®c tasks would be ideal, this is not usually practical.

Instead, previous research has utilized ratings of

graduates on how well prepared they feel for the role

they are carrying out. Clack4 identi®ed de®ciencies in

aspects of training in medicine and concluded that

undergraduate skills training may not adequately pre-

pare students for their PRHO year. Particular de®ci-

encies highlighted included clinical pharmacology and

medical ethics.4 Others have reported a lack of basic

clinical skills.5,6

These studies have relied solely on the ratings of

graduates. Another approach would be to canvas the

opinion of the educational supervisors. Senior doctors

1 1South Manchester University Hospitals Trust, Manchester, UK2Harvard Medical School, Boston, USA

Correspondence: A Jones, 2nd Floor Research and Teaching Building,

South Manchester University Hospitals Trust, Nell Lane, Manchester

M20 2LR, UK

Pre-registration house of®cers

578 Ó Blackwell Science Ltd MEDICAL EDUCATION 2001;35:578±584

(consultants), who are responsible for appraising the

performance of new graduates, are now assigned to all

PRHOs. The supervisor should meet with the PRHO a

minimum of three times during a 6-month post and

discuss how the graduate is progressing, as judged

against the competencies listed by the GMC in The

New Doctor.2 The supervisor should not simply rely on

their own observation and judgement, but should

gather the opinions of other doctors and health

professionals working with the PRHOs. Accordingly,

the supervisor should be in a good position to rate the

preparedness for practice of a new graduate.

In Manchester, we have been engaged in signi®cant

reform of the undergraduate curriculum in line with the

recommendations of Tomorrow's Doctors.1 As part of

this change, we needed to determine how well our

current graduates feel prepared for practice and also

how they are rated by their educational supervisors.

The purpose of this study was to explore the two

perspectives on levels of preparation for the role of

PRHO, in order to draw comparisons and use those

results to inform curriculum development.

Methods

Using The New Doctor2 as the basis for de®ning the

competencies required of a graduating medical student,

a questionnaire was developed to send to all those who

graduated from the medical school of the University of

Manchester in 1998, other than the 24 whose home

addresses were overseas. Graduates (n � 256) were

sent the questionnaire 3 months into their ®rst PRHO

placement.

The questionnaire asked graduates to consider

`How well did the course prepare you for¼?', followed

by a list of broad areas of competence and, for more

speci®c procedures, `How well did the course provide

you with a competence in ¼?'. Respondents were

required to rate their answers on a 5-point scale, with

`very well prepared/competent' and `not at all well

prepared/competent' as the range; the mid-point label

was `quite well prepared/competent'. In addition to

the quantitative data collection of the questionnaire,

space was left for written comment. Four follow-up

letters with copies of the questionnaire were sent to

non-respondents.

A similar questionnaire was sent to the 194 educa-

tional supervisors in the North-west Region who

between them were responsible for the supervision of

249 Manchester graduates. The question asked of

educational supervisors was `Please rate the Manche-

ster PRHOs on their competence in the following¼',

with the same list of items as those on the graduate

questionnaire. As educational supervisors tend to

supervise more than one PRHO at a time, they

responded on the basis of a general perception of

graduates rather than on ratings of individuals. They

were asked to leave an item blank if they felt unable to

provide a rating. Space was left for comment. One

follow-up letter and a copy of the questionnaire were

sent out to non-respondents.

Quantitative data were analysed using the Statistical

Package for the Social Sciences, version 7á0 for Windows

(SPSS). Qualitative data were separated according to

whether they related to broad domains or speci®c skills

and the comments were then grouped according to the

questionnaire item to which they referred.

Results

In total, 169 graduates (66%) and 147 (76%) educa-

tional supervisors responded to the survey. Educational

supervisor respondents were drawn from both teaching

hospitals (31%) and district general hospitals (67%).

The percentage of graduate respondents who were

female (51á5%) was slightly higher than for the year

group as a whole (47á1%).

The results are presented in two sections, A and

B: ®rst those data relating to broader areas of

competence (A), e.g. providing appropriate care for

people of different cultures, and then those data

Key learning points

Pre-registration house of®cers (PRHOs) felt better

prepared for the broad areas of competence, such

as communication, than for speci®c skills such as

suturing.

Supervisors differed from graduates in their

perceptions of graduates' competence in under-

standing disease processes. Graduates rated

themselves much more favourably.

`Awareness of legal and ethical issues' and

`understanding the purpose and practice of audit,

peer review and appraisal' have been identi®ed as

areas of competence where graduates felt less well

prepared.

The undergraduate course appeared to have only

partly met its objective of preparing graduates for

the PRHO year, although there may be inappro-

priate expectations on the part of respondents for

the level of competence expected in a new grad-

uate.

Ó Blackwell Science Ltd MEDICAL EDUCATION 2001;35:578±584

Preparedness of graduates for the role of pre-registration house of®cer · A Jones et al. 579

relating to speci®c skills (B), e.g. venepuncture. The

data for both sections have been summarized into the

upper two points, the mid-point and the lower two

points on the scale.

A. Broad competence and attitudinal domains

Graduate responses

Table 1 shows the ratings from graduates for how well

they felt the course had prepared them, with the more

positive ratings ®rst. Overall, there were only four items

where more than half the graduates rated themselves at

more than the mid-point: `understanding disease pro-

cesses', `communicating effectively', `being aware of

your own limitations' and `working in a team'. One-®fth

or less of graduates thought that they were well prepared

for `providing appropriate care for people of different

cultures', `using informatics as a tool in medical

practice' or `being aware of legal and ethical issues'.

Educational supervisor responses

Responses from educational supervisors are shown in

Table 2. Similarly to the ®ndings from the graduates,

there were only three items where more than half of the

supervisors rated the PRHOs as being higher than the

mid-point: `being aware of their own limitations',

`keeping accurate records' and `working in a team'.

Comparison of groups

A total of 10 items were rated higher by graduates than

by educational supervisors. Large differences were evi-

dent in ratings for the items `understanding disease

processes' and `using opportunities for disease preven-

tion and health promotion'.

Supervisors' ratings were more favourable than those

of the graduates for eight of the items, in particular for

`providing appropriate care for people of different cul-

tures' and `keeping accurate records'. Some ratings

were very similar, for example for `being aware of legal

and ethical issues'.

Qualitative responses

Among the graduate respondents, 102 made 234

comments on the ®rst section of the questionnaire,

of which 91 related to broad competence and attitu-

dinal domains. In the questionnaires to educational

supervisors, 109 comments were made by 69 super-

visors, of which 53 comments related to the broad

domains.

These comments give insight into the data gained

from the quantitative section. One supervisor made the

comment:

Generally poor basic knowledge of disease and the

practical aspects of arriving at a diagnosis.

Table 1 `Preparedness' ratings provided by graduates

% rating as

Item

More than

mid-point Mid-point

Less than

mid-point

Understanding disease processes 75á3 23á5 1á2Communicating effectively 58á9 32á9 8á3Being aware of your own limitations 54á7 39á4 5á9Working in a team 51á7 33á5 14á7Recognition of the social and emotional factors in illness and treatment 41á2 38á2 20á6

Keeping accurate records 37á9 37á9 24á3Using opportunities for disease prevention and health promotion 35á0 39á1 26á0Managing time effectively 32á9 35á9 31á2Making the best use of laboratory and other diagnostic services 32á4 47á6 20á0Understanding the relationship between primary and social care and hospital care 31á2 44á7 24á1

Developing appropriate attitudes towards personal health and wellbeing 27á8 41á4 30á8Understanding the principles of evidence-based medicine 26á9 37á1 35á9Diagnosis, decision making and the provision of treatment including prescribing 25á9 42á4 31á8Coping with uncertainty 25á3 34á7 40á0Understanding the purpose and practice of audit, peer review and appraisal 21á2 38á2 40á6

Providing appropriate care for people of different cultures 20á0 32á4 47á6Using informatics as a tool in medical practice 19á4 29á7 50á9Being aware of legal and ethical issues 17á7 32á4 50á0

Preparedness of graduates for the role of pre-registration house of®cer · A Jones et al.580

Ó Blackwell Science Ltd MEDICAL EDUCATION 2001;35:578±584

Several graduate respondents made speci®c reference

to understanding disease processes, for example:

The traditional course put great emphasis on know-

ledge and I feel we picked up much information

regarding disease processes and diagnosing them,

and understanding the pathological events behind

the diseases.

Excellent knowledge of disease process, but not

linked well enough to clinical management.

Too much time spent on learning useless informa-

tion. What's the point of learning the detailed

histopathology of a rare disease if I don't have ade-

quate training in basic procedures, common drugs,

and the day to day things a house of®cer does.

Some comments from graduates suggested that they

felt they had had adequate exposure to people from

different cultures, for example:

Manchester serves a cosmopolitan community which

you have wide contact with as an undergraduate,

helping you to adapt easily to patients from a wide

variety of cultures.

There were no comments relating to using oppor-

tunities for disease prevention and health promotion.

B. Speci®c procedures

Table 3 shows the ratings for graduates' competence in

speci®c procedures.

Graduate responses

The most favourable ratings, from over 50% of

graduates, were for venepuncture, basic cardio-pul-

monary resuscitation (CPR)3 and arterial blood samp-

ling. Less than 10% considered that they were well

prepared for `correct use of a nebuliser', `suturing' or

`inserting a nasogastric tube'.

Educational supervisor responses

The two most favourable ratings were for venepunc-

ture and writing a prescription. The least favourable

ratings were for suturing and inserting a nasogastric

tube.

Comparison of groups

Venepuncture attracted the most positive rating from

both groups and suturing and inserting a nasogastric

tube the least positive. Graduates rated themselves

more favourably than educational supervisors for only

three of the 13 skills (venepuncture, basic CPR and

arterial blood sampling).

Table 2 Competence ratings provided by educational supervisors

% rating as

Item

More than

mid-point Mid-point

Less than

mid-point

Being aware of their own limitations 59á2 38á1 2á7Keeping accurate records 56á9 34á7 8á3Working in a team 56á7 38á7 4á7Communicating effectively 48á3 47á0 4á7Developing appropriate attitudes towards personal health and well-being 42á7 48á1 9á2

Recognition of the social and emotional factors in illness and treatment 40á1 45á8 14á1Providing appropriate care for people of different cultures 39á4 50á4 10á2Understanding disease processes 36á0 48á7 15á3Diagnosis, decision making and the provision of treatment including prescribing 34á7 48á7 16á7Managing time effectively 30á6 50á3 19á0

Understanding the principles of evidence-based medicine 27á7 56á0 16á3Using informatics as a tool in medical practice 23á9 51á3 24á8Making the best use of laboratory and other diagnostic services 23á6 51á4 25á0Understanding the relationship between primary and social care and hospital care 21á9 54á7 23á4Understanding the purpose and practice of audit, peer review and appraisal 20á1 40á3 39á6

Being aware of legal and ethical issues 17á3 43á9 38á8Coping with uncertainty 17á0 54á6 28á4Using opportunities for disease prevention and health promotion 16á3 45á5 38á2

Preparedness of graduates for the role of pre-registration house of®cer · A Jones et al. 581

Ó Blackwell Science Ltd MEDICAL EDUCATION 2001;35:578±584

Large differences in responses were seen for the items

`calculating accurate drug dosages' and `writing a

prescription'. When the two groups' responses were

compared, the percentages of ratings above the mid-

point were reasonably close, i.e. within 10%, for only

three items: `catheterisation', `suturing' and `control of

a haemorrhage'. Again, the educational supervisors

gave more positive ratings.

Qualitative responses

Of the 234 comments made by graduates in the ®rst

section of the questionnaire, 56 related to preparation

for performing speci®c skills; 14 of the 69 comments

made by educational supervisors related to graduates'

skills. Some examples of the graduates' comments

are:

In terms of clinical knowledge and skills, the

university provides ample resources and opportun-

ities for the students. Perhaps there should be an

increase in frequency for teaching practical skills such

as CPR, suturing, ECG, etc.

Basic skills were taught well at Manchester; vene-

puncture, blood gases, cannulation, catheterisation.

However, when I started I had no idea how to suture

or pass NGT [nasogastric tubes], things that are of-

ten expected of PRHOs.

One supervisor noted:

Surgical skills are decreasing rapidly ± few have ever

sutured and time in operating theatre is infrequent.

Most need to be trained to `scrub up' on arrival.

Several comments from graduates related to prepar-

ation for administering drugs to patients, often drawing

on the importance of these skills for the role of a house

of®cer:

I was the last of the year that studied the old course and

don't feel that it provided adequate training of the

essential, but perhaps mundane, duties that form the

majority of a house of®cer's workload, e.g. ¯uid

prescriptions monitoring and treating electrolyte

levels/disturbances andpaincontrol/managingnausea.

A general comment from a supervisor was:

The undergraduate course does not prepare one for

the practical day-to-day aspects of the PRHO job,

e.g. prescribing medications, commencing IV anti-

biotics, setting up a drip. The `bread and butter' of

the job!

Discussion

The ®ndings of this study are relevant to curriculum

planning and evaluation. Where there were high ratings

and agreement between the graduates and supervisors,

it is reasonable to assume that the course has prepared

students well for their ®rst professional post. However,

where the graduates and supervisors agreed on low

ratings, then the curriculum has not entirely met its

principal aim of producing graduates fully prepared for

practice. For those competencies where there is lack of

agreement between the graduates and supervisors, this

probably re¯ects different expectations of performance.

Table 3 Competence ratings provided by graduates and educational supervisors

Graduates, % rating as Educational supervisors, % rating as

Item

More than

mid-point Mid-point

Less than

mid-point

More than

mid-point Mid-point

Less than

mid-point

Venepuncture 71á8 22á4 5á9 60á1 33á6 6á3Basic CPR 62á9 25á3 11á8 41á7 43á3 15á0Arterial blood sampling 51á8 28á8 19á4 41á5 32á6 25á9Administering oxygen therapy safely 27á6 29á4 42á9 40á0 41á5 18á5Urinary catheterization 27á6 28á8 43á5 31á2 45á6 23á2Obtaining valid consent 23á2 26á2 50á6 48á9 40á3 10á8Performing an ECG 21á2 20á6 58á2 42á4 40á7 16á9Writing a prescription 17á8 24á3 58á0 51á7 42á0 6á3Control of haemorrhage 12á4 31á2 56á5 22á2 50á0 27á8Calculating accurate drug dosages 11á8 27á8 60á4 47á1 49á3 3á6Correctly using a nebulizer 8á8 21á1 70á0 36á1 43á7 20á2Suturing 8á8 10á0 81á2 15á3 30á6 54á1Inserting a nasogastric tube 2á9 6á5 90á6 18á1 33á3 48á6

CPR8 , cardio-pulmonary resuscitation

Preparedness of graduates for the role of pre-registration house of®cer · A Jones et al.582

Ó Blackwell Science Ltd MEDICAL EDUCATION 2001;35:578±584

The ®ndings of this study show that for a small

number of competencies based on what the GMC ex-

pect of a PRHO, the graduates and supervisors con-

sider that the course has prepared the students very well

for initial professional practice. Whether it is reasonable

to conclude that the curriculum overall has met its

principal aim is dependent on the interpretation of the

remaining data. If the mid-point (`quite well prepared')

is judged to be the minimum level of preparedness, then

most of the graduates' and supervisors' ratings reached

this level, at least for the broad areas of competence. In

support of this is the notion that undergraduate, post-

graduate and continuing professional development are

parts of a continuum of education that should go on

throughout professional life. However, most curri-

culum designers would be disappointed that only

approximately one-quarter of graduates felt the course

had prepared them well in, for example, `understanding

the principles of evidence-based medicine'.

There are some limitations in the methods used for

this study. The response rate for the questionnaires

was adequate, although it would have been preferable

to have a higher rate from the graduate respondents.

Several attempts to increase this rate were made, but

one obstacle was the resources required to track down

PRHOs in the hospital system.

The construction of the questionnaire was based on

a 5-point Likert scale. This probably produced a ten-

dency to record the middle value where there was

uncertainty in the mind of the respondent, and so

perhaps a 4-point scale would be preferable for future

studies. A further drawback was that the questions to

the graduates and supervisors were phrased slightly

differently and so our statistical advice was that formal

testing between the two groups was inappropriate.

The responses represented the perceptions of either

graduates or educational supervisors formed over a

period of time. For the graduates in particular, by the

time they completed the questionnaire there may have

been in¯uences on their perceived levels of competence

other than their undergraduate education. The timing

of the questionnaire distribution was chosen to allow

graduates to have settled into the job of PRHO and yet

not to be so far into the job that other in¯uences have

been signi®cant. It was impossible to tell the level of

in¯uence, for example, of the working environment and

other members of the team.

We were interested in assessing how well prepared

the graduates (and their supervisors) thought they were

to perform the duties of a PRHO. However, this was

only an indirect measure of performance.3 Neverthe-

less, 3 months into their ®rst post, the graduates were

most probably rating their preparedness after having

been faced with these tasks. They may have found this

rating process easier for the speci®c skills rather than

the broad areas of competence. Furthermore, speci®c

skills were more likely to have been assessed in under-

graduate exams, so the graduates would have had some

formal measurement to guide them.

Another area of concern was the possibility of a `halo

effect',7 which may have led respondents to rate on the

basis of an overall perception rather than competence in

each speci®c item on the list. However, the range in

responses suggested that this was not the case. Some of

the educational supervisors did note that it was dif®-

cult to re¯ect general perceptions of the Manchester

graduates as their views were often skewed by a

particularly good (or poor) house of®cer that they had

supervised in the recent past. There may also be a

dif®culty in that educational supervisors were not in a

position to be fully aware of the competence of gradu-

ates in performing all the skills listed; this was re¯ected

in the fewer responses to the question on suturing.

Some supervisors did suggest that the junior doctors

working with them were in a better position to com-

ment on the competence of PRHOs.

There are several messages for curriculum planners,

given that our results showed some degree of consensus

among graduates and educational supervisors on grad-

uate levels of preparation for the duties demanded in the

role of PRHO. The attributes of effective communica-

tion, teamwork and being aware of one's limitations are

all identi®ed as objectives of undergraduate medical

education2,8 and these were generally rated favourably

by the graduates, as were the core skills of venepuncture,

CPR and arterial blood sampling. The low rating for

awareness of legal and ethical issues is similar to that

found elsewhere, as is the ability to use informatics.4

Inserting a nasogastric tube has been identi®ed as a

core skill to be performed competently by graduates.2,8

It is therefore worrying that so many graduates report

having had little experience of this procedure; other

research has shown similar ®ndings.94 Perhaps it is an

inappropriate expectation. Suturing is another task

which graduates should be able to do under supervi-

sion,8 but few graduates or educational supervisors felt

that the course had provided students with a reasonable

level of competence in this procedure. There were

comments suggesting that these skills are quickly

acquired on busy wards and this has been shown to be

the case elsewhere6,10 but other research has suggested

that the reality of the job of a PRHO does not lend itself

to structured acquisition of these skills.4

The use of educational supervisors' ratings adds a

different perspective on the graduates' level of prepar-

edness.9 Mostly the two groups gave concordant views,

Preparedness of graduates for the role of pre-registration house of®cer · A Jones et al. 583

Ó Blackwell Science Ltd MEDICAL EDUCATION 2001;35:578±584

but there were some differences. The supervisors did

not think that that the graduates were adequately pre-

pared in relation to `understanding disease processes'.

In contrast, comments from the graduates were that too

much time had been devoted to this; time that they

thought would have been better spent learning how to

manage the illnesses which patients present with. It

appeared that for the PRHOs the pressure concerns

patient management, whilst the supervisors felt that

perhaps a better understanding of disease processes was

required.

The differing perceptions on graduates' abilities to

prescribe and calculate drug dosages raises the question

of why graduates should have felt so unprepared when

supervisors felt they were competent. Other research4,6

supports these graduate perceptions that the under-

graduate course places too little emphasis on clinical

pharmacology and therapeutics. Some of the comments

from graduates suggested that they had not appreciated

how important this competency would be for the role of

PRHO. It may be that graduates turn to other members

of the health care team, such as nursing or pharmacy

staff, for help and so the supervisors were unaware that

the PRHOs were not performing the tasks unaided. An

alternative explanation is that supervisors expect new

graduates to ask for assistance, whereas the new

PRHOs expect to be able to prescribe and calculate

drug dosages on their own.

In conclusion, the results from this study have

provided valuable baseline data to assist with curri-

culum development. Overall, there is still considerable

work to be done in preparing graduates for professional

practice. The philosophy, design and content of the

course at Manchester has changed since the PRHOs

surveyed here graduated, and it will be interesting to see

which areas are of concern for future graduates and

supervisors.

Acknowledgements

We are grateful to the graduates and educational

supervisors who took the time to complete the

questionnaires and to provide us with such constructive

comments. We also thank Julie Morris for expert

statistical advice.

Contributors5

AJ jointly wrote the paper, designed the research

instruments and analysed the results. PM was the ex-

ternal consultant and assisted with the revision of the

manuscript. PO initiated, supervised, and guided the

project and jointly wrote the paper.

Funding

The research was funded by the Department of Health

North-west Region.

References

1 General Medical Council. Tomorrow's Doctors. London: Gen-

eral Medical Council; 1993.

2 General Medical Council. The New Doctor. London: General

Medical Council; 1997.

3 Miller GE. The assessment of clinical skills/competence/per-

formance. Acad Med 1990;65(Suppl.):563±5676 .

4 Clack GB. Medical graduates evaluate the effectiveness of

their education. Med Educ 1994;28:418±31.

5 Calman KC, Donaldson M. The pre-registration house of®cer

year: a critical incident study. Med Educ 1991;25:51±9.

6 Hannon FB. A national medical education needs' assessment

of interns and the development of an intern education and

training programme. Med Educ 2000;34:275±84.

7 Thompson WG, Lipkin M, Gilbert DA, Guzzo RA,

Roberson L. Assessment of the American Board of Internal

Medicine resident evaluation form. J Gen Intern Med

1990;5:214±7.

8 University of Leeds School of Medicine. Core Clinical

Competencies in Medicine Project. Leeds: University of Leeds

School of Medicine; 1998.

9 Rolfe IE, Andren JM, Pearson S, Hensley MJ, Gordon JJ.

Clinical competence of interns. Med Educ 1995;29:225±30.

10 Bax NDS, Godfrey J. Identifying core skills for the medical

curriculum. Med Educ 1997;31:347±51.

7Received 18 February 2000; editorial comments to authors 1 August

2000; accepted for publication 1 September 2000

Preparedness of graduates for the role of pre-registration house of®cer · A Jones et al.584

Ó Blackwell Science Ltd MEDICAL EDUCATION 2001;35:578±584