ISSUES AND INNOVATIONS IN NURSING PRACTICE
Evaluating Emergency Nurse Practitioner services: a randomized
controlled trial
Mark A. Cooper BN RGN
ENP Course Co-ordinator, Accident and Emergency Departments, North Glasgow University Hospitals NHS Trust, Glasgow,
UK
Grace M. Lindsay BSc MN PhD RGN SCM
Senior Lecturer, Nursing and Midwifery School, University of Glasgow, Glasgow, UK
Sue Kinn BSc MSc PhD
Programme Leader, Nursing Research Initiative for Scotland, Glasgow Caledonian University, Glasgow, UK
and Ian J. Swann MBBS FRCS FFAEM DFM
Consultant in Accident and Emergency Medicine, Glasgow Royal Infirmary, Glasgow, UK
Submitted for publication 17 April 2002
Accepted for publication 25 September 2002
Correspondence:
Mark A. Cooper,
Accident and Emergency Departments,
Glasgow Royal Infirmary,
84 Castle Street,
Glasgow G4 0SF,
UK.
E-mail: [email protected]
COOPER MCOOPER M.AA., LINDSAY GLINDSAY G.MM., KINN SKINN S. && SWANN ISWANN I.JJ . (2002)(2002) Journal of
Advanced Nursing 40(6), 721–730
Evaluating Emergency Nurse Practitioner services: a randomized controlled trial
Background. Emergency Nurse Practitioners (ENP) are increasingly managing
minor injuries in Accident and Emergency departments across the United Kingdom.
This study aimed to develop methods and tools that could be used to measure the
quality of ENP-led care. These tools were then tested in a randomized controlled trial.
Methods. A convenience sample of 199 eligible patients, over 16 years old, and
with specific minor injuries was randomized either to ENP-led care (n ¼ 99) or
Senior House Officer (SHO)-led care (n ¼ 100) and were diagnosed, treated, re-
ferred or discharged by this lead clinician. Following treatment, patients were asked
to complete a patient satisfaction questionnaire related to the consultation. Clinical
documentation was assessed using a ‘Documentation Audit Tool’. A follow-up
questionnaire was sent to all patients at 1 month. Return visits to the department
and missed injuries were monitored.
Results. Patients were satisfied with the level of care from both ENPs and SHOs.
However, they reported that ENPs were easier to talk to (P ¼ 0Æ009); gave them
information on accident and illness prevention (P ¼ 0Æ001); and gave them enough
information on their injury (P ¼ 0Æ007). Overall they were more satisfied with the
treatment provided by ENPs than with that from SHOs (P < 0Æ001). ENPs’ clinical
documentation was of higher quality than SHOs (P < 0Æ001). No differences were
found in recovery times, level of symptoms, time off work or unplanned follow-up
between groups. Missed injuries were the same for both groups (n ¼ 1 in each group).
Conclusion. The study was sufficiently large to demonstrate higher levels of patient
satisfaction and clinical documentation quality with ENP-led than SHO-led care. A
larger study involving 769 patients in each arm would be required to detect a 2%
difference in missed injury rates. The methods and tools used in this trial could be
used in Accident and Emergency departments to measure the quality of ENP-led care.
� 2002 Blackwell Science Ltd 721
Keywords: Emergency Nurse Practitioners, Accident and Emergency, minor
injuries, randomized controlled trial, patient satisfaction, clinical documentation,
junior medical staff
Introduction
The number of patients attending Accident and Emergency
(A & E) departments in the United Kingdom (UK) each year
is rising (Audit Commission 2001) and this, combined with
changes in health service policy to reduce junior doctors’
hours (National Health Service [NHS] Management Execu-
tive 1992), has seen waiting times in A & E becoming
longer despite increases in overall numbers of A & E
doctors (Audit Commission 2001). One potential solution,
or at least a partial solution, has been the introduction of
Emergency Nurse Practitioners (ENPs) to many depart-
ments. Redesigning how A & E services are currently
delivered is expected to be one of the key features of a
modern health care system and is likely to provide many
opportunities for ENPs (Scottish Executive Health Depart-
ment 2001).
Just introducing a new service or redesigning an existing
one is, however, not enough. No longer is it acceptable just to
provide a service; it must now be demonstrated to be of
sufficient quality to justify its continued existence (Dickens
1994). A government publication has reported that:
A series of well publicised lapses in quality have prompted doubts in
the minds of patients about the overall standard of care they may
receive (Department of Health 1997).
The introduction or expansion of any ENP services must
therefore be shown to be both acceptable to patients and
provide a quality of service at least as good, if not better,
than existing services. The majority of patients who attend
A & E are managed by Senior House Officers (SHOs) who
are often in their first postregistration post (Wallis & Guly
2001). If ENPs are introduced to provide a service that
SHOs have been providing, it seems reasonable that they
should be expected to provide a service of at least equal
quality.
Emergency Nurse Practitioners
The ENPs are nurses who are ‘authorized to assess and
treat patients attending an A & E department’ (Read et al.
1992). The first department in the UK to introduce a
formal ENP service is generally acknowledged to be to
Oldchurch Hospital in Essex (Head 1988). They introduced
this following a Community Health Council Survey and
patient complaints about lengthening waiting times. How-
ever, nurses in many small community hospital casualty
units (Read & George 1994) and specialist ophthalmic A
& E units assessed and treated particular patients
independently and often unofficially (Jones et al. 1986,
Shaw et al. 1988), and hence worked as de facto ENPs
prior to this.
Since 1986, when ENPs started at Oldchurch, the
number of A & E departments using them has risen
dramatically. In a survey of all major A & E departments
in the UK in 1996, Tye et al. (1998) found that 36% of
major departments provided some kind of formal ENP
service. In 1998, in a survey of all A & E departments in
Scotland (including very small community hospitals) 47%
of departments reported having nurses who functioned as
ENPs (Cooper et al. 2001).
A growing body of research evidence appears to show that
ENPs are able to provide a safe and effective service to minor
injury patients. Most previous studies of the effectiveness of
ENPs have concentrated on particular skills, for example
ability to request and interpret X-rays (Macleod & Freeland
1992, Freij et al. 1996, Meek et al. 1998, Overton-Brown &
Anthony 1998) or administer medication under protocols
(Marshall et al. 1997). Two randomized controlled trials
have examined the package of care provide by ENPs: an
Australian study (Chang et al. 1999) which involved 169
patients and an English study (Sakr et al. 1999) with 1453
patients. Sakr et al. (1999) concluded that:
properly trained accident and emergency nurse practitioners, who
work within agreed guidelines can provide care for patients with
minor injuries that is equal or in some ways better than that provided
by junior doctors.
The ENPs in that trial had undertaken, or were undergoing,
the English National Board Autonomous Practice course
(A33), had locally agreed guidelines for practice and worked
in an A & E department with medical staff immediately
available for advice. Unfortunately, not all nurses working as
ENPs have had additional training and not all departments
have agreed working guidelines for them (Cooper et al.
2001). The type of department they work in also varies, from
large hospitals with medical staff available for referral and
advice to small community hospitals, minor injury units
(Cooper et al. 2001) and NHS Walk-in Centres (Munro et al.
2000) where medical staff may not always be on-site. The
M.A. Cooper et al.
722 � 2002 Blackwell Science Ltd, Journal of Advanced Nursing, 40(6), 721–730
types and severity of injuries managed by ENPs also vary
between departments (Dolan et al. 1997, Cooper et al.
2001). ENP services in different settings, with different local
arrangements for referral and advice, with nurses who have
undergone different types of educational preparation, and
treating different types of injuries, should be evaluated within
their specific context. However, there should be common
objectives in such evaluations. Four important areas that
should be core to any evaluation are (1) patient satisfaction,
(2) clinical documentation, (3) unplanned follow-up and (4)
missed injuries.
When any new service is introduced it is vital that it is
acceptable to patients. This is an important influence deter-
mining whether a patient seeks medical advice and complies
with treatment (Larson & Rootman 1976).
The ability to write comprehensive and accurate clinical
notes is another important aspect of practising independently
as an ENP. Written clinical documentation is often the only
way of communicating vital information about an indivi-
dual’s care to colleagues who are also involved with and
responsible for a patient. Accurate information is essential for
proper care and effective service management (Audit Com-
mission 1996). Good notes are often said to imply good
practice (Montague 1996); hence it is vital that ENPs (and
doctors) accurately record details on every patient they treat.
Clinical documentation can be called as evidence before a
court of law, health service commissioner or professional
conduct committee (United Kingdom Central Council for
Nursing [UKCC] 1998). Hospitals need good records to
defend themselves against claims of negligence (Audit Com-
mission 1995). Whilst accurate documentation can help
protect patients and staff (Read 1999), poor documentation
makes it difficult to defend a hospital in a clinical negligence
case (Tingle 1995).
Most minor injuries are managed using relatively simple
techniques and are expected to heal within a relatively short
time and without substantial input from the health services. A
small percentage of minor injuries, however, if mismanaged
may cause significant morbidity (Gwynne et al. 1997).
Identifying mismanaged cases and missed injuries can be
difficult. Formal review of X-rays following discharge (and
usually within 24 hours) is practised in many departments.
This provides a safety net and should pick up any fractures
missed at initial attendance. These patients can then be
recalled for further assessment or to alter the initial manage-
ment. Missed injuries or problems with initial management
may subsequently be picked up at follow-up clinics. Patients
may also choose to re-attend or seek advice elsewhere if
concerned and problems may be picked up at that time, i.e.
unplanned follow-up.
The study
Aim
To develop methods and tools that could be easily used, in
different A & E departments, to measure the quality of ENP-
led care (in terms of patient satisfaction, quality of clinical
documentation, unplanned follow-up and missed injuries).
Following the introduction of ENPs at the research site, these
tools were tested using a randomized controlled trial to
compare ENP-led care with the existing predominantly SHO-
provided service.
Design
The trial was conducted over a 2-month period during
December 1998 and January 1999. All patients who attended
the Glasgow Royal Infirmary A & E department were
assessed by a triage nurse (routine practice). Patients with
minor injuries were then reviewed by the researcher (MC) for
suitability for inclusion. Consecutive patients who were over
16 and had sustained an injury, which fell within the ENP
protocols at the research site, were invited to participate in
the trial. Patients were only recruited when both the
researcher and an ENP were on duty.
Following informed written consent patients were ran-
domized to either the experimental group (ENP-led care) or
the control group (SHO-led care). Opaque, sequentially
numbered sealed envelopes containing randomized assign-
ments to the two groups were provided by one of the authors
(SK), not directly involved in the clinical part of the trial.
Methods
Demographic information and types of injury
Demographic information on patients in each arm of the trial
was collected by the A & E reception staff. This was done as
part of the normal process of patient registration prior to
recruitment. Following the patient’s departure from the
department the researcher reviewed the clinical documenta-
tion and collected data on the type of injury the patient had
sustained.
Deprivation was measured using the Carstair’s Score
(McLoone 1997). This score is derived from variables from
small area Census data and uses postcode sectors. Scores
range from DEPCAT 1 (the most affluent postcode sectors) to
DEPCAT 7 (the most deprived). The scores are based on four
different variables in the Census data: number of people per
room, male unemployment, social class and car ownership.
The score is a relative measure of the deprivation or affluence
Issues and innovations in nursing practice Evaluating Emergency Nurse Practitioner services
� 2002 Blackwell Science Ltd, Journal of Advanced Nursing, 40(6), 721–730 723
which refers to the population of the postcode sector where
the patient lives and not to the patient individually.
Consultation and referral
Each ENP and SHO was asked to record on a trial ‘Treat-
ment Record’ form whether any advice on diagnosis, X-ray
interpretation or treatment was sought from any other cli-
nician. A more senior A & E doctor (usually a more experi-
enced SHO or a specialist registrar) was available for
consultation to both ENPs and SHOs, who could also
directly refer patients to specialities within the hospital for an
opinion on emergency treatment or possible admission.
Both SHOs and ENPs were able to refer patients to a
number of hospital follow-up clinics or to their general
practitioner (GP). Follow-up clinics available included an A
& E Soft Tissue Clinic, orthopaedic Fracture Clinic, Burns
Clinic run by the regional burns unit and Hand Clinic.
Information on numbers of patients referred to the various
clinics was collected from the A & E notes, as were details of
any investigations requested.
A ‘Clinic Referral Form’ was developed after discussions
with three A & E consultants and the nurse-in-charge of the
clinics. This was used to collect information on the appro-
priateness of referrals to follow-up clinics and on the clinical
management of the patient. The form was completed by the
doctor reviewing the patient at the clinic.
Patient satisfaction
A self-completion patient satisfaction questionnaire modified
from a previously validated questionnaire developed by
Jenkins and Thomas (1996) was used to measure satisfaction
with the consultation. The questionnaires were distributed to
patients at the time of recruitment. Each patient was asked to
complete the questionnaire immediately after their treatment
and prior to leaving the department. Patients were given the
opportunity to remain in the room where they had been
treated to provide privacy when completing the question-
naire. Completed questionnaires were collected via a sealed
post box in the waiting room. Although the questionnaires
were not anonymous, patients were assured that only the
researcher would see the data and no member of staff in-
volved with directly treating patients would have access to
individual questionnaires.
Quality of clinical documentation
The researcher measured the quality of each set of clinical
notes written by the ENPs and SHOs using a ‘Documentation
Audit Tool’ developed to evaluate the clinical documentation
of patients with minor injuries. Development of the tool using
an expert panel and a consensus methodology is described
elsewhere (Cooper et al. 2000). Each set of clinical notes was
given a score out of 30.
One-month follow-up
A postal questionnaire, developed from a patient diary ori-
ginally designed by Read and George (1994) for a proposed
RCT of ENPs, was posted to patients 1 month after their
attendance. Reminders were posted to nonrespondents. This
questionnaire collected information on (1) time to recovery,
(2) level and frequency of pain still being experienced,
(3) level of symptoms and activity, (4) time off work and (5)
whether any unplanned follow-up was sought.
Returns and missed injuries
Any study patients who returned to the department were
identified through its computer system and their clinical notes
were examined and the reasons for return noted. Missed in-
juries were identified by (1) monitoring return patients, (2) a
systematic search of patients through the department’s recall
register, (3) the ‘Clinic Referral Forms’ which allowed missed
injuries discovered at follow-up clinics to be reported to the
researchers, and (4) formal complaints.
Ethical approval
Ethical approval was obtained prior to the start of the trial from
the Glasgow Royal Infirmary Research Ethics Committee.
Data analysis
Data from the questionnaires were coded and entered into a
Microsoft Access 97 database created for the study. The SPSS
(Statistical Package for the Social Sciences v8Æ0) software was
used to analyse the data. Descriptive statistics were calculated
for all of the variables and histograms were plotted to ensure
that the data were normally distributed. Two-tailed t-tests
were applied to continuous variables. For categorical varia-
bles the chi-squared test for independent samples was used,
or Fisher’s exact test if values were less than 5 in any cell. The
Mann–Whitney U-test was used in the analysis of the ordinal
data from patient satisfaction questionnaires.
Analysis was undertaken comparing patients in the groups
to which they were originally assigned. Any patient who was
not seen initially by the clinician to whom they were
randomized was excluded from the final analysis.
Results
Recruitment
A total of 214 minor injury patients were invited to
participate in the trial, and 95% of these took part
M.A. Cooper et al.
724 � 2002 Blackwell Science Ltd, Journal of Advanced Nursing, 40(6), 721–730
(n ¼ 204). Five patients were subsequently withdrawn as
they were not seen in the clinician group to which they were
randomized (see Fig. 1). Patients were recruited into the trial
over 28 days during December 1998 and January 1999 (no
recruitment was conducted during the Christmas and New
Year period). Approximately 215 hours were spent in the
department recruiting patients. Eight ENPs and 12 SHOs
took part in the study. Seven of the eight ENPs had been
practising for 1 year at the time of the study. The eighth had
completed her training 3 months previously. All SHOs were
in their fifth and sixth months of their 6-month A & E
posting, and the majority were in their first SHO post.
Demographic information and types of injury
The average age of patients in the study was 36Æ3 years and
just over half were male (56Æ4%). Demographic characteris-
tics and the injuries treated in both the ENP and SHO groups
were compared and no statistical differences were found
between for age, sex, deprivation score and type of injury
(Table 1).
Consultation and referral
The average waiting time to see an SHO was significantly
longer than that for an ENP (SHO 70Æ1 minutes, ENP
48Æ6 minutes, P < 0Æ001; 95% CI, 11Æ2–31Æ8 minutes).
However, there was no significant difference in the total
consultation time (including the time for treatment) (ENP
30Æ0 minutes, SHO 24Æ9 minutes, P ¼ 0Æ115; 95% CI, �1Æ3
to 11Æ5 minutes).
ENPs and SHOs were able to seek advice from senior
medical staff for a variety of reasons including diagnosis,
X-ray interpretation and management. At the time of the trial
ENPs had to request advice on interpreting X-rays. They
sought advice on patients more often than SHOs (64Æ6%
compared with 21Æ2%, P < 0Æ001). However, when patients
who had been X-rayed were excluded, there was no
difference between the two groups (ENP 20Æ9%, SHO
11Æ5%, P ¼ 0Æ21). There was also no difference between
the groups in the numbers of X-rays requested (ENP 56Æ6%,
SHO 47Æ5%, P ¼ 0Æ2).
There was no difference between the groups in patients
admitted (ENP 2Æ0%, SHO 6Æ0%, P ¼ 0Æ279) or between
those referred to follow-up clinics (ENPs 33Æ3%, SHOs
27Æ5%, P ¼ 0Æ358). The percentage of ‘Clinic Referral Forms’
returned by the various follow-up clinics varied considerably
from 17Æ4% to 100% (Table 2). No statistical difference
was detected between the two groups in appropriateness of
referral or clinical management (Table 3). However two
patients in the ENP group were considered to have received
unsatisfactory clinical management. The first case involved a
patient with a suspected ulna collateral ligament injury to the
metacarpal-phalangeal joint of the right thumb. The patient
had been correctly diagnosed and referred to the appropri-
ate follow-up clinic. The thumb had also, correctly, been
immobilized in a thumb spica; however, the patient had not
been given a sling, which the reviewing doctor felt was
Eligible patients (n = 214)
Declined to praticipate (n = 10) Did not want to see an ENP (n = 6) Did not have time (n = 1) Did not want to participate in research (n = 2) No reason given (n =1)
SHO-led careAllocated to control intervention (n = 102)
Did not receive control allocation & withdrawn (n = 2) Seen by Middle grade Dr (n = 1) Seen by Consultant (n = 1)
ENP-led careAllocated to test intervention (n = 102)
Did not receive test allocation &withdrawn (n = 3) Seen by SHO (n = 1) Seen by Middle grade Dr (n = 2)
Returned patient satisfaction questionnaire (n = 81) following consultation
Returned follow-up questionnaire (n = 65) at one month
Clinic refenal form returned (n = 10/28)
Unexpectedly returned to department (n = 4)
Returned patient satisfaction questionnaire(n = 87) following consultation
Returned follow-up questionnaire (n = 63) at one month
Clinic refenal form returned (n = 17/34)
Unexpectedly returned to department (n = 2)
R
Figure 1 Flowchart of trial. ‘R’ indicates
randomization.
Issues and innovations in nursing practice Evaluating Emergency Nurse Practitioner services
� 2002 Blackwell Science Ltd, Journal of Advanced Nursing, 40(6), 721–730 725
unsatisfactory. At the review clinic the hand was still swollen
and thorough examination was not possible until the swelling
had subsided. The second case involved a toe fracture, which
had been correctly diagnosed and managed by strapping the
toes together; however, the reviewing doctor noticed that a
piece of gauze which should have been placed between the toes
prior to strapping was missing. This was felt to have been
unsatisfactory management which could, if not corrected, lead
to an adverse outcome. The piece of gauze is used to help
prevent the skin between the toes from becoming macerated.
Patient satisfaction
One hundred and sixty-eight patients returned satisfaction
questionnaires immediately after their treatment, giving a
response rate of 84% (ENP n ¼ 87, SHO n ¼ 81). Patients
appeared very satisfied with the level of care they received
Characteristic
ENP-led care
n ¼ 102
SHO-led care
n ¼ 102 Significance
Age, years (mean) 35Æ85 36Æ80 P ¼ 0Æ648
Sex
Male 59 56 P ¼ 0Æ672
Female 43 46
Deprivation score (Carstairs Index)
(McLoone 1997)
(Least deprived)
1 2 1 P ¼ 0Æ612
2 2 5
3 12 16
4 13 10
5 11 6
6 16 20
(Most deprived)
7 43 43
Type of injury (primary complaint)
Ankle/foot sprain 18 11 P ¼ 0Æ196
Wrist/hand sprain 9 8
Wound inc. burns and scalds 34 36
Contusion injury 8 8
Hand/wrist fracture 11 15
Ankle/foot fracture 12 10
Minor head injury 0 1
Other 7 10
Table 1 Demographic information and
types of injury of patients entered into trial
Table 2 Response rates for the clinic referral form
Clinic
No. of
patients referred
Proformas
recovered
Response
rate percentage
Soft tissue clinic 20 20 100
Fracture clinic 23 4 17Æ4Hand clinic 18 10 55Æ6Burns clinic 1 1 100
ENP-led care SHO-led care Significance
Patients referred to follow-up clinics 34 28 P ¼ 0Æ358
Patient who failed to attend clinics 4 4
Patients who attended clinic 30 24
Completed clinic forms returned 17 10 N/A
Inappropriate or borderline referrals 3 1 P ¼ 0Æ596
Unsatisfactory management 2 0 P ¼ 0Æ254
Adverse effect on clinical outcome likely,
where management was considered unsatisfactory
1 0 N/A
Table 3 Results from follow-up clinics
(clinic referral form)
M.A. Cooper et al.
726 � 2002 Blackwell Science Ltd, Journal of Advanced Nursing, 40(6), 721–730
from both the SHOs and the ENPs. However, they reported
that ENPs were easier to talk to; they were given information
on accident and illness prevention; they were given enough
information on their injury. Overall they were more satisfied
with the treatment provided by ENPs than they were with that
by SHOs (Table 4). Not all patients answered every question.
Quality of clinical documentation
The clinical documentation was audited 4 months after the
trial ended, using the previously validated Documentation
Audit Tool (Cooper et al. 2000). A total of 186 clinical notes
were audited (93Æ5%) (ENP n ¼ 94, SHO n ¼ 92), and 13
could not be found. ENPs were found to have written notes
of higher quality than SHOs (ENP 28Æ0 of 30, SHO 26Æ6 of
30 P < 0Æ001).
One-month follow-up
The patient 1-month follow-up questionnaire yielded a 64%
(ENP n ¼ 63, SHO n ¼ 65) response rate following one postal
reminder. Patients were asked how long it had taken them to
recover fully from their injury. There was no difference in time
to recovery (P ¼ 0Æ96), level of symptoms (swelling, P ¼ 0Æ92
and stiffness, P ¼ 0Æ80), level of activity (looking after
themselves, P ¼ 0Æ58; ability to go to work/school, P ¼ 0Æ40;
sleep pattern, P ¼ 0Æ87), and time off work (P ¼ 0Æ14).
Patients were asked if they required further medical or
nursing advice in the month following their attendance in
A & E, excluding any follow-up appointments made or
suggestions to visit with their GP (i.e. unplanned follow-up).
A fifth of patients (20Æ0%) who replied reported the need to
seek this (ENP 18Æ3%, SHO 21Æ5% P ¼ 0Æ654). No statistical
difference was found between the groups.
Returns and missed injuries
Ten patients (5Æ0%) re-attended the department, and they
returned for a variety of reasons including new injuries (ENP
n ¼ 1, SHO n ¼ 1), concern about their injury (ENP n ¼ 2,
SHO n ¼ 1), problems complying with treatment (ENP
n ¼ 2, SHO n ¼ 1) and problems with treatment (ENP
n ¼ 1, SHO n ¼ 1).
A total of three patients entered into the trial had injuries
missed by the reviewing clinician. There was one in each of the
treatment groups and a further one amongst the five patients
withdrawn from the study. No formal complaints were
received by the hospital about any patient entered into the trial.
Discussion
The study has demonstrated that using the tools it has tested,
it is possible to evaluate patient satisfaction with ENP-led
care and to measure the quality of documentation in the real
life situation of an A & E department.
This study has also demonstrated that patients, when they
leave A & E, were more satisfied with the treatment provided
by ENPs. In particular, they were more satisfied with how
easy it was to discuss their injury with the ENP, the amount
of information ENPs gave them concerning their injury, and
the advice ENPs gave them on avoiding injury or illness in the
future. The results also show that the quality of ENP
documentation was higher and, importantly, levels of missed
injuries appear to be very low; however, large numbers of
patients need to be monitored to determine the actual rate.
Only two other RCTs of ENP- compared with SHO-led
care have been published (Chang et al. 1999, Sakr et al. 1999)
and neither found a significant difference in patient satisfac-
tion. The larger trial (Sakr et al. 1999), involving 1453
Table 4 Patient satisfaction questionnaire results
Percentage agreeing or strongly
agreeing with statementStatistics
SignificanceStatement (Statistically significant statements in bold) ENP-led care SHO-led care
I feel the doctor/nurse practitioner listened to me 97Æ7 (n ¼ 87) 86Æ4 (n ¼ 81) P ¼ 0Æ089
I feel the doctor/nurse practitioner gave me enough information
about my injury/condition
95Æ2 (n ¼ 83) 82Æ5 (n ¼ 80) P ¼ 0Æ007
I felt able to ask questions about my injury/condition 94Æ0 (n ¼ 84) 83Æ8 (n ¼ 80) P ¼ 0Æ123
I feel the doctor/nurse practitioner gave me enough time 995Æ3 (n ¼ 86) 82Æ5 (n ¼ 80) P ¼ 0Æ12
The doctor/nurse practitioner gave me advice on how to avoid illness/injuries 75Æ3 (n ¼ 81) 45Æ2 (n ¼ 73) P ¼ 0Æ001
I felt it easy to tell the doctor/nurse practitioner about my injury/condition 97Æ6 (n ¼ 85) 84Æ0 (n ¼ 81) P ¼ 0Æ009
I understood the advice the doctor/nurse practitioner gave me 94Æ1 (n ¼ 85) 84Æ6 (n ¼ 78) P ¼ 0Æ080
I am satisfied with the treatment the doctor/nurse practitioner gave me 98Æ8 (n ¼ 85) 87Æ7 (n ¼ 81) P < 0Æ001
n ¼ Total number of responses to each statement in each group.
Issues and innovations in nursing practice Evaluating Emergency Nurse Practitioner services
� 2002 Blackwell Science Ltd, Journal of Advanced Nursing, 40(6), 721–730 727
patients, only found a nonsignificant trend in favour of ENPs.
Neither trial specifically examined clinical documentation.
Byrne et al. (2000) conducted a study which examined
minor injury patients attending three different types of centre
(1) an A & E department, (2) a minor accident treatment
service (MATS) located within an A & E department, and (3)
a minor injuries unit (MIU). Patients received care from
junior doctors (SHOs) in the A & E department and from
ENPs in the MATS unit and MIU. One hundred and sixty-
three patients completed questionnaires (A & E department
n ¼ 43; MATS n ¼ 57; MIU n ¼ 63) and overall were
satisfied with their care. Those who were managed by ENPs
were: (1) more likely to have had health and first aid advice
(P ¼ 0Æ05); (2) more likely to have been told who to contact
for advice (P ¼ 0Æ01); (3) more likely to have been given
written instructions (P ¼ 0Æ01); and (4) less likely to be
worried about their health (P ¼ 0Æ05).
Whilst it is acknowledged that patient satisfaction surveys
tend to show uniformly high ratings (McColl et al. 1996), the
findings from Byrne et al. (2000) support our results and
suggest that ENPs are more likely to provide health education
advice and be better at providing information to patients than
SHOs.
Chang et al. (1999) demonstrated that ‘appropriately
prepared emergency nurse practitioners can assist in the
management of nonurgent cases’. Sakr et al. (1999) conclu-
ded that ‘properly trained accident and emergency nurse
practitioners, who work within agreed guidelines can provide
care of patients with minor injuries that is equal or in some
ways better than that provided by junior doctors’. The results
from our study support both these conclusions.
Attempting to evaluate any form of longer-term outcome
following treatment in A & E for a minor injury was found to
be a complicated undertaking, as Read and George (1994) also
identified. Patients may be discharged with no anticipated
follow-up being thought necessary, advised to seek a consul-
tation with their GP or given an appointment to return to a
hospital follow-up clinic. However, they have a much wider
range of options: they can follow the advice about follow-up,
may decide to self-refer to another service provider, may return
to A & E, seek an appointment with their GP, go to another A
& E department, self-manage their condition, attend their
occupational health service, or seek a private consultation with
a physiotherapist or private doctor, for example.
Therefore, if an injury was missed at first presentation or
mismanaged, this ability to seek unplanned follow-up from a
large number of different health care providers often means
that it is unlikely for feedback to be forwarded to the initial A
& E department. This makes it difficult for departments to
monitor adverse outcomes in minor injury patients.
As the majority of patients are not expected to return to
hospital for any form of follow-up, it is difficult to evaluate
any longer term follow-up. As in Sakr et al.’s (1999) study,
we had to mainly rely on self-reported outcomes through a
postal questionnaire at 28 days.
Study limitations
Self-completion questionnaires have a number of limitations.
The key difficulty often involves the refusal of respondents to
complete or return the questionnaire (Barker 1991), leading to
bias if nonresponders differ from responders. Subjects may also
ask other people either to assist in completing the questionnaire
or even complete if on their behalf, prejudicing the sample
(Barker 1991). This may occur where subjects may have
injured their dominate hand and are unable to write without
help or in their haste have not brought reading glasses with
them to hospital and cannot read the questionnaire properly.
Lack of reading ability may also contribute to poor
completion or nonresponse. Whilst very few adults are
totally illiterate, approximately 23% of the Scottish popula-
tion would have difficulty identifying the correct amount of
medicine to give a child from the information given on the
medicine package (OECD and Statistics Canada 2000). This
may account for some of the nonresponders.
Analysis
In line with several other studies (Daoud et al. 1997, Fazekas
et al. 1997, Jacobson et al. 1997, Spruance et al. 1997),
patients who did not start the intended intervention were
excluded from the final analysis. This was felt unlikely to lead
to bias, as the intended effects of the intervention could only
occur if an ENP or SHO saw a patient.
This problem could perhaps have been avoided if random-
ization had occurred immediately before a patient was seen.
As the research was being conducted in a ‘real-life’ situation it
was necessary to randomize a patient whilst they were
waiting to be seen, and it was impossible to guarantee that
neither the ENP nor SHO would get called away or get
caught up with another case. Two of the five patients were
seen accidentally by more senior medical staff, and in both
these cases Clinical Assistants who were unaware or had
forgotten about the study saw patients randomized to the
SHOs. A second researcher, observing compliance with
randomization, may have been able to prevent this.
Recommendations for further research
Problems with initial mismanagement or missed injuries can
only be identified if reported to the A & E department.
M.A. Cooper et al.
728 � 2002 Blackwell Science Ltd, Journal of Advanced Nursing, 40(6), 721–730
However, patients can self-refer to many different places. In
this study one in five patients reported needing to seek
additional advice in the month after initial attendance.
Further research is needed to investigate the reasons for
seeking unplanned follow-up, where patients seek advice and
what measures could be taken to reduce this additional
burden on the health service.
In this study no difference was detected in missed injury
rates (one in each group); however, two patients in the
ENP-led care group were felt to have received unsatisfactory
management when reviewed in follow-up clinics. Therefore a
2% difference could be said to exist between the two groups.
Using Sampsize v2Æ0 (Machin et al. 1997), the sample size for
a full-scale trial was calculated. In order to detect a 2%
difference in the missed injury and mismanaged rates between
the two groups, with a power of 80% and at a 5% level of
significance, a trial would need to be undertaken with 1538
patients (769 patients in each arm).
Implications of the study for professional practice
Departments wishing to evaluate their own ENP services are
faced with many difficulties. Not least of which is the expense
and time-consuming nature of designing or replicating a
suitable study. The opportunity to use tools that have been
tested in a rigorous study design and are easy to administer
may allow more departments to evaluate and monitor their
services.
Conclusions
The study was sufficiently large to demonstrate higher levels
of satisfaction and clinical documentation quality with
ENP-led care. The patient satisfaction questionnaire and
Documentation Audit Tool are simple to use and sensitive
enough detect differences in the quality of care provided by
ENPs and SHOs. Both tools can be easily incorporated into
local evaluations to measure important aspects of the quality
of minor injury care. Monitoring recalls and other returns to
a department is an important measure of quality of care in A
& E; however, caution must be exercised in interpreting these
results as patients are at liberty to seek second opinions and
other unplanned follow-up from different health care pro-
viders. Further research is required to examine unplanned
follow-up in minor injury patients.
Acknowledgements
We would like to thank the ENPs and SHOs who took part in
the study, the Chief Scientist Office, Scottish Executive and
Glasgow Royal Infirmary for funding the study, and Dr Tracy
Ibbotson for her guidance on aspects of the study design. We
would also like to thank Dr M. Jenkins and Dr Sue Read for
kind permission to use and adapt their respective question-
naires.
References
Audit Commission (1995) Setting the Records Straight. HMSO,
London.
Audit Commission (1996) By Accident or Design: Improving A & E
Services in England and Wales. HMSO, London.
Audit Commission (2001) Acute Hospital Portfolio: Review of
National Findings – Accident and Emergency. Audit Commission
Publications, London.
Barker P.J. (1991) Questionnaire. In The Research Process in Nur-
sing (Cormack D.F.S. ed.), 2nd edn. Blackwell Scientific, Oxford,
pp. 215–227.
Byrne G., Richardson M., Brunsdon J. & Patel A. (2000) Patient
satisfaction with emergency nurse practitioners in A & E. Journal
of Clinical Nursing 9, 83–93.
Chang E., Daly J., Hawkins A., McGirr J., Fielding K., Hemmings L.,
O’Donoghue A. & Dennis M. (1999) An evaluation of the nurse
practitioner role in a major rural emergency department. Journal of
Advanced Nursing 30, 260–268.
Cooper M., Kinn S., Ibbotson T., Lindsay G. & Swann I. (2000)
Emergency nurse practitioner’s documentation: development of an
audit tool. Emergency Nurse 8, 34–39.
Cooper M.A., Hair S., Ibbotson T.R., Lindsay G.M. & Kinn S.
(2001) The extent and nature of emergency nurse practi-
tioner services in Scotland. Accident and Emergency Nursing 9,
123–129.
Daoud E.G., Strickberger S.A., Man K.C., Goyal R., Deeb G.M.,
Bolling S.F., Pagani F.D., Bitar C., Meissner M.D. & Morady F.
(1997) Preoperative amiodarone as prophylaxis against atrial
fibrillation after heart surgery. New England Journal of Medicine
337, 1785–1791.
Department of Health (1997) The New NHS Modern, Dependable.
HMSO, London.
Dickens P. (1994) Quality and Excellence in Human Services. John
Wiley & Sons, Chichester.
Dolan B., Dale J. & Morley V. (1997) Nurse practitioners: role in A
& E and primary care. Nursing Standard 11, 33–38.
Fazekas F., Deisenhammer F., Strasser-Fuchs S., Nahler G. &
Mamoli B. (1997) Randomised placebo controlled trial of monthly
intravenous immunoglobulin therapy in relapsing–remitting mul-
tiple sclerosis. Austrian Immunoglobulin in Multiple Sclerosis
Study Group. Lancet 349, 589–593.
Freij R.M., Duffy T., Hackett D., Cunningham D. & Fothergill J.
(1996) Radiographic interpretation by nurse practitioners in a
minor injuries unit. Journal of Accident and Emergency Medicine
13, 41–43.
Gwynne A., Barber P. & Tavener F. (1997) A review of 105 negli-
gence claims against accident and emergency departments. Journal
of Accident and Emergency Medicine 14, 243–245.
Head S. (1988) Nurse practitioners: the new pioneers. Nursing Times
84, 27–28.
Issues and innovations in nursing practice Evaluating Emergency Nurse Practitioner services
� 2002 Blackwell Science Ltd, Journal of Advanced Nursing, 40(6), 721–730 729
Jacobson J.M., Greenspan J.S., Spritzler J., Ketter N., Fahey J.L.,
Jackson J.B., Fox L., Chernoff M., Wu A.W., MacPhail L.A.,
Vasquez G.J. & Wohl D.A. (1997) Thalidomide for the treatment
of oral aphthous ulcers in patients with human immunodeficiency
virus infection. National Institute of Allergy and Infectious
Diseases AIDS Clinical Trials Group. New England Journal of
Medicine 336, 1487–1493.
Jenkins M. & Thomas A. (1996) The assessment of general practi-
tioner registrars’ consultations by a patient satisfaction question-
naire. Medical Teacher 18, 347–350.
Jones N.P., Hayward J.M., Khaw P.T., Claoue C.M.P. & Elkington
A.R. (1986) Function of an ophthalmic accident and emergency
department: results of a six month survey. British Medical Journal
292, 188–190.
Larsen D.E. & Rootman I. (1976) Physicians’ role performance and
patient satisfaction. Social Science Medicine 10, 29–32.
Machin D., Campbell M.J., Fayer P.M. & Pinol A.P.Y. (1997)
Sample Size Tables for Clinical Studies, 2nd edn. Blackwell
Science, Oxford.
Macleod A.J. & Freeland P. (1992) Should nurses be allowed to
request X-rays in an accident and emergency department? Archives
of Emergency Medicine 9, 19–22.
Marshall J., Edwards C. & Lambert M. (1997) Administration of
medicines by emergency nurse practitioners according to protocols
in an accident and emergency department. Emergency Medicine
Journal 14, 233–237.
McColl E., Thomas L. & Bond S. (1996) A study to determine
patient satisfaction with nursing care. Nursing Standard 10,
34–38.
McLoone P. (1997) Carstairs Scores for Scottish Postal Code Sectors
from the 1991 Census. Public Health Research Unit, University of
Glasgow, Glasgow.
Meek S.J., Ruffles G., Anderson J. & Ohiorenoya D. (1998) Can
A & E nurse practitioners interpret radiographs? A multicentre
study. Journal of Accident and Emergency Medicine 15, 105–107.
Montague A. (1996) Legal Problems in Emergency Medicine. Oxford
University Press, Oxford.
Munro J., Nicholl J., Webber L. & Paisley S. (2000) Walk-in Centres:
a Review of Existing Research. Medical Care Research Unit,
University of Sheffield, Sheffield.
NHS Management Executive (1992) Junior Doctors: the New Deal.
NHS Management Executive, London.
OECD and Statistics Canada (2000) Literacy in the Information
Age: Final Report of the International Adult Literacy Survey.
Organisation of Economic Co-operation and Development (and
Statistics Canada), Paris & Ottawa.
Overton-Brown P. & Anthony D. (1998) Towards a partnership in
care: nurses’ and doctors’ interpretation of extremity trauma
radiology. Journal of Advanced Nursing 27, 890–896.
Read H. (1999) Documentation in the outpatient setting. Nursing
Standard 13, 41–43.
Read S.M. & George S. (1994) Nurse practitioners in accident and
emergency departments: reflections on a pilot study. Journal of
Advanced Nursing 19, 705–716.
Read S.M., Jones N.M.B. & Williams B.T. (1992) Nurse practi-
tioners in accident and emergency departments: what do they do?
British Medical Journal 305, 1466–1469.
Sakr M., Angus J., Perrin J., Nixon C., Nicholl J. & Wardrope J.
(1999) Care of minor injuries by emergency nurse practitioners or
junior doctors: a randomised controlled trial. Lancet 354, 1319–
1326.
Scottish Executive Health Department (2001) Caring for Scotland:
the Strategy for Nursing and Midwifery in Scotland. The Station-
ary Office, Edinburgh.
Shaw C., Hurst M. & Stone S. (1988) Towards Good Practices in
Small Hospitals – Some Suggested Guidelines. National Associ-
ation of Health Authorities, Birmingham.
Spruance S.L., Rea T.L., Yhoming C., Tucker R., Saltzman R. &
Boon R. (1997) Penciclovir cream for the treatment of herpes
simplex labialis: a randomised, multicenter, double-blind, placebo-
controlled trial. Topical Penciclovir Collaborative Study Group.
JAMA 277, 1374–1379.
Tingle J. (1995) Why hospital medical record keeping must improve.
British Journal of Nursing 4, 982–983.
Tye C.C., Ross F. & Kerry S.M. (1998) Emergency nurse practitioner
services in major accident and emergency departments: a United
Kingdom postal survey. Journal of Accident and Emergency
Medicine 15, 31–34.
UKCC (1998) Guidelines for Records and Record-Keeping. United
Kingdom Central Council for Nursing, Midwifery and Health
Visiting, London.
Wallis L.A. & Guly H.R. (2001) Improving care in accident and
emergency departments. British Medical Journal 323, 39–42.
M.A. Cooper et al.
730 � 2002 Blackwell Science Ltd, Journal of Advanced Nursing, 40(6), 721–730
Top Related