MENTORSHIP PROGRAMS AND THE NOVICE NURSE: A RAPID EVIDENCE ...
148
MENTORSHIP PROGRAMS AND THE NOVICE NURSE: A RAPID EVIDENCE ASESSMENT by STEPHANIE KYLA ERICKSON B.S.N, Kwantlen University College, 2002 A THESIS SUBMITTED IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF MASTER OF SCIENCE in The Faculty of Graduate and Postdoctoral Studies (Nursing) THE UNIVERSITY OF BRITISH COLUMBIA (Vancouver) August 2015 © Stephanie Kyla Erickson, 2015
Transcript of MENTORSHIP PROGRAMS AND THE NOVICE NURSE: A RAPID EVIDENCE ...
MENTORSHIP PROGRAMS AND THE NOVICE NURSE:
A RAPID EVIDENCE ASESSMENT
by
STEPHANIE KYLA ERICKSON
B.S.N, Kwantlen University College, 2002
A THESIS SUBMITTED IN PARTIAL FULFILLMENT OF
THE REQUIREMENTS FOR THE DEGREE OF
MASTER OF SCIENCE
in
The Faculty of Graduate and Postdoctoral Studies
(Nursing)
THE UNIVERSITY OF BRITISH COLUMBIA
(Vancouver)
August 2015
© Stephanie Kyla Erickson, 2015
ii
ABSTRACT
New graduate registered nurses (RNs) experience many challenges as they transition
from the role of student nurse to professional nurse. Mentoring can support new nurses with the
development of clinical nursing skills and competencies, and is linked to professionalism,
nursing quality improvement, self-confidence, retention, and job satisfaction. This rapid
evidence assessment (REA) addresses how new graduate mentorship programs can be effective
in improving performance, satisfaction, retention, and confidence in novice nurses’ practice. It
also reports the key elements of effective new graduate mentorship programs and reported
problems in implementing new graduate RN mentorship programs.
Seventeen research studies were selected for inclusion and examined using Bandura’s
social learning theory. The data from each research study was extracted using the EPPI-Centre
Data Extraction and Coding Tool for Education Studies to allow for mapping and analysis. Each
research study was then scored from highest level of evidence to lowest level of evidence. The
findings were then synthesized to suggest that mentorship programs can be effective in
improving performance, satisfaction, retention, and confidence in novice nurses’ practice under
the right conditions. The reported key elements of effective new graduate RN mentorship
programs include mentor-mentee matching, availability of mentors, adequate training and
preparation of mentors, commitment and support, and length of the mentoring relationship. The
reported problems in implementing new graduate RN mentorship programs include lack of
training and preparation of mentors and mentees, availability of mentors, and mentor-mentee
mismatch.
iii
PREFACE
This thesis is original, unpublished, independent work by the author, S. Erickson. There was no
ethics approval required due to the nature of this work.
iv
TABLE OF CONTENTS
Abstract ........................................................................................................................................... ii
Preface............................................................................................................................................ iii
Table of Contents ........................................................................................................................... iv
List of Tables ................................................................................................................................ vii
List of Figures .............................................................................................................................. viii
Acknowledgements ........................................................................................................................ ix
CHAPTER 1 Introduction, Background, and Research Question .................................................. 1
1.1 Introduction .................................................................................................................. 1
1.2 Background ................................................................................................................... 2
1.2.1 Challenges for Newly Qualified Nurses ............................................................. 2
1.2.2 Attrition ............................................................................................................... 4
1.2.3 The Role of Mentorship Programs ...................................................................... 4
1.3 Research Question and Sub-questions .......................................................................... 5
1.4 Chapter Summary ......................................................................................................... 5
CHAPTER 2 Mentoring Overview, Conceptual Framework, and Definitions .............................. 6
2.1 Overview of Mentoring ................................................................................................ 6
2.1.1 Attributes of an Effective Mentor ....................................................................... 7
2.1.2 Attributes of an Effective Mentee ....................................................................... 7
2.1.3 Phases of a Mentoring Relationship.................................................................... 7
2.1.4 Types of Mentoring Relationships ...................................................................... 9
2.2 Conceptual Framework ............................................................................................... 10
2.3 Definitions .................................................................................................................. 14
2.4 Chapter Summary ....................................................................................................... 15
CHAPTER 3 Approach and Methodology ................................................................................... 16
3.1 Approach .................................................................................................................... 16
3.2 Justification ................................................................................................................. 17
3.3 Methodology ............................................................................................................... 18
3.3.1 Formulating the Question.................................................................................. 18
3.3.2 Inclusion and Exclusion Criteria ....................................................................... 19
3.3.3 Search Strategy ................................................................................................. 21
3.3.4 Data Collection ................................................................................................. 22
3.3.5 Screening and Selecting Studies ....................................................................... 23
3.3.6 Scoring .............................................................................................................. 23
3.4 Analysis ...................................................................................................................... 24
3.4.1 Critical Appraisal .............................................................................................. 24
v
3.4.1.1 EPPI-Centre Data Extraction and Coding Tool for Education
Studies .................................................................................................. 25
3.4.1.2 GSRS Weight of Evidence (WoE) Tool ............................................... 26
3.4.1.3 Maryland Scale of Scientific Methods (MSSM) Tool.......................... 26
3.4.1.4 Critical Appraisal Skills Programme (CASP) Tool .............................. 26
3.4.2 Synthesis of Findings ........................................................................................ 27
3.4.3 Communicating Findings .................................................................................. 28
3.5 Chapter Summary ....................................................................................................... 29
CHAPTER 4 Results..................................................................................................................... 30
4.1 Selected Research Studies .......................................................................................... 30
4.2 Excluded Research Studies ......................................................................................... 34
4.3 Quantitative Research Studies .................................................................................... 34
4.3.1 WoE High Level Studies................................................................................... 37
4.3.2 WoE Medium Level Studies ............................................................................. 40
4.4 Mixed-Methods Research Studies .............................................................................. 42
4.4.1 WoE and CASP High Level Studies ................................................................. 45
4.4.2 WoE and CASP Medium Level Studies ........................................................... 48
4.5 Qualitative Research Studies ...................................................................................... 49
4.5.1 WoE Medium and CASP High Level Studies .................................................. 50
4.6 Chapter Summary ....................................................................................................... 54
CHAPTER 5 Discussion ............................................................................................................... 56
5.1 Synthesis of Findings .................................................................................................. 56
5.1.1 Improving Performance .................................................................................... 56
5.1.2 Improving Satisfaction ...................................................................................... 58
5.1.3 Improving Retention ......................................................................................... 61
5.1.4 Improving Confidence ...................................................................................... 62
5.2 Key Elements of Effective New Graduate RN Mentorship Programs ....................... 64
5.2.1 Mentor-Mentee Matching ................................................................................. 64
5.2.2 Availability........................................................................................................ 65
5.2.3 Training ............................................................................................................. 66
5.2.4 Commitment and Support ................................................................................. 67
5.2.5 Length of Mentoring Relationship .................................................................... 68
5.3 Reported Problems in Implementing New Graduate RN Mentorship Programs ....... 68
5.3.1 Training ............................................................................................................. 68
5.3.2 Availability........................................................................................................ 69
5.3.3 Mentor-Mentee Mismatch ................................................................................. 70
5.4 Limitations of this REA .............................................................................................. 71
5.5 Chapter Summary ....................................................................................................... 71
CHAPTER 6 Conclusion .............................................................................................................. 73
6.1 REA Summary ............................................................................................................ 73
6.1.1 Performance ...................................................................................................... 74
6.1.2 Satisfaction ........................................................................................................ 75
6.1.3 Retention ........................................................................................................... 76
vi
6.1.4 Confidence ........................................................................................................ 76
6.1.5 Key Elements for Effective Programs .............................................................. 77
6.1.6 Reported Problems ............................................................................................ 78
6.2 Recommendations for Future Research ...................................................................... 78
6.3 Chapter Summary ....................................................................................................... 80
BIBLIOGRAPHY ......................................................................................................................... 81
APPENDIX A Excel Spreadsheet ................................................................................................ 90
APPENDIX B GSRS Weight of Evidence (WoE) Tool ............................................................... 96
APPENDIXC Maryland Scale of Scientific Methods (MSSM) Tool .......................................... 97
APPENDIX D Critical Appraisal Skills Programme (CASP) Tool ............................................. 98
APPENDIX E EPPI-Centre Data Extraction and Coding Tool for Education Studies .............. 101
vii
LIST OF TABLES
Table 3.1 Categorical Ranking of Scores ..................................................................................... 24
Table 4.1 List of Selected Research Studies ................................................................................. 31
Table 4.2 Quantitative Study Details ............................................................................................ 35
Table 4.3 GSRS WoE Assessment Criteria Questions and Score ................................................ 36
Table 4.4 Mixed-Methods Study Details ...................................................................................... 43
Table 4.5 GSRS WoE Assessment Criteria Questions and Score ................................................ 44
Table 4.6 Qualitative Study Details .............................................................................................. 49
Table 4.7 GSRS WoE Assessment Criteria Questions and Score ................................................ 50
viii
LIST OF FIGURES
Figure 2.1 Bandura’s Direction of Social Learning Processes ..................................................... 12
Figure 3.1 Confidence in Review Studies..................................................................................... 17
Figure 3.2 Stages of Synthesis ...................................................................................................... 28
Figure 4.1 Research Studies Selection Process............................................................................. 31
Figure 4.2 Publication Year of Research Studies ......................................................................... 33
Figure 4.3 Research Study Sample Size Histogram ..................................................................... 34
Figure 4.4 Quantitative Critical Appraisal Scoring ...................................................................... 36
Figure 4.5 Mixed-Methods Critical Appraisal Scoring ................................................................ 44
Figure 4.6 Qualitative Critical Appraisal Scoring ........................................................................ 50
Figure 4.7 Overall WoE Level of Evidence .................................................................................. 53
Figure 4.8 Overall CASP Level of Evidence ................................................................................ 53
Figure 4.9 Overall MSSM Level of Evidence .............................................................................. 54
ix
ACKNOWLEDGEMENTS
I offer my sincerest gratitude to my thesis committee for their time, guidance, and
contributions. I owe particular thanks to Dr. Bernie Garrett for your expertise, feedback, and
always being available to answer my questions. Additional thanks to Dr. Cathryn Jackson and
Dr. Tarnia Taverner for their invaluable input and support in completing this work.
To Lila a very special thanks for your unconditional support and encouragement as I
completed this journey and fulfilled my dream.
1
CHAPTER 1: Introduction, Background, and Research Question
1.1 Introduction
Nursing shortage is a major concern both in Canada and across the world (Buchan &
Aiken, 2008; Canadian Nurses Association [CNA], 2009). According to the Canadian Institute
for Health Information (CIHI, 2013), in 2011 there were 270,274 registered nurses (RNs)
employed in Canada with 40.3% of them over the age of 50 years. When looking at British
Columbia (BC) in 2011 there were 30,151 RNs employed with 42.9% of them aged 50 years and
older (CIHI). This suggests, depending on the age of retirement that up to one third of Canada’s
nursing workforce could retire from active practice within the next ten years (Maddalena,
Kearney, & Adams, 2012). The result of these experienced nurses retiring means that
considerable pressure is being placed on new graduate RNs to fill the ensuing vacancies
(Maddalena et al.).
In 2009 the CNA estimated that Canada currently needed 11,000 full-time equivalent
(FTE) RNs to meet health care needs and anticipated that Canada will be short almost 60,000
FTE RNs by 2022. The main causes of a nursing shortage in economically developed countries
have been identified as: inadequate workforce planning and allocation mechanisms creating a
mismatch between education supply and service demand; undersupply of new staff; poor
recruitment, retention, and ‘return’ policies; and ineffective use of available nursing resources
(Buchan & Aiken, 2008). The CNA proposes six policy scenarios to deal with Canada’s
projected RN shortage including increasing RN productivity, reducing RN annual absenteeism,
increasing enrolment in nursing schools, improving the retention of practicing RNs, reducing
attrition rates in RN entry-to-practice programs, and reducing international in-migration (2009).
2
This issue has also led to pressure to take on new RNs in areas that have typically not
employed them in the past. In 2011 there were 12,645 RNs employed in operating room (OR)
practice, with 46.5% of them aged 50 years and older which means they need to ensure they are
recruiting and retaining new RNs to meet the needs of an aging workforce (CNA, 2013).
Historically the practice of hiring a new graduate RN into a specialty area, such as the OR, has
been rare but due to a nursing shortage this has become the rule, rather the exception (Baxter,
2008; Persaud, 2008). This has resulted in the new phenomenon of RNs who have only recently
graduated from nursing school entering specialty areas who have limited experience with
practice skills such as time management, organization, and applying theoretical knowledge into
practice (Chen & Lou, 2014). In a complex work environment, it has been suggested that a new
graduate RN may take up to one year to transition successfully into their new practice
environment (Persaud; Woodfine, 2011).
The stakeholders who find these issues most significant includes individuals who are
responsible for recruitment and retention within health care organizations, such as directors and
managers; and particularly staff who are responsible for the orientation and mentorship of newly
graduated RNs (such as clinical nurse educators). Therefore there is a need to explore the impact
of mentorship programs on the practice of novice nurses’.
1.2 Background
1.2.1 Challenges for Newly Qualified Nurses
New graduate RNs experience many challenges as they transition from the role of student
nurse to professional nurse (Baxter, 2010; Maddalena et al., 2012; Rheaume, Clement, & LeBel,
2011). Some of the challenges they experience include short staffing, poor communication
among colleagues, abusive or unsupportive colleagues, heavy workloads, physical and emotional
3
demands, and lack of administrative support (Maddalena et al.). Although these challenges are
not uncommon for experienced RNs, they are especially stressful for new graduate RNs and high
levels of stress during the novice period may contribute to the decision to leave their place of
employment (Maddalena et al.). Some of the reasons cited for attrition from nursing include job
dissatisfaction, inadequate training, lack of support, and “realty shock” (Baxter). Other common
factors influencing attrition include perceptions of unsafe patient care related to high patient
acuity, unacceptable nurse-to-patient ratios, lack of support and guidance in the workplace, and
unacceptable salary, benefits, or scheduling (Bowles & Candela, 2005).
The first three to six months of employment for new graduate RNs can be described as
the most stressful and the greatest challenge is putting what was learned in school into bedside
practice (Almada, Carafoli, Flattery, French, & McNamara, 2004). New graduate RNs
demonstrate stress concerning competence, confidence, making errors, and adjusting to their new
workplace environment (Almada et al.). Bowles and Candela’s (2005) research on first job
experiences of recent RN graduates revealed that the newly graduated RNs did not perceive their
work environment as safe, felt staffing levels were inadequate, and believed that there wasn’t
enough time available to spend with their patients to provide adequate care. Cho, Laschinger,
and Wong (2006) described the sources of stress for new graduate nurses as the gap between
what was learned in school and what is practiced in the workplace, the fear of making errors due
to excessive workloads and responsibilities, lack of confidence in their clinical skills, and lack of
mentorship from more experienced nurses. Hunsberger, Baumann, and Crea-Arsenio (2013)
identified sources of anxiety for new graduate RNs as not knowing what to do in unexpected
situations, interactions with physicians, and role issues such as ambiguity and work overload.
The greatest concern for a new graduate RN was reported by Craig, Moscato, & Moyce (2012)
4
as a fear of not knowing what to do in an unexpected crisis or situation, followed by concerns
about missing a key piece of information related to their patient, and their lack of experience to
provide safe patient care.
1.2.2 Attrition
It has been estimated that up to 69 % of new graduate RNs will leave their place of
employment within their first year of practice (Baxter, 2010; Persaud, 2008). O’Brien-Pallas et
al. (2008) report that the average cost of replacing a RN in Canada is $25,000 and high turnover
rates are associated with a decrease in job satisfaction, increase in likelihood of medical errors,
overtime, and environmental complexity. High turnover rates can also affect a new graduate RN
personally and professionally in addition to the associated high cost to the employer (Baxter).
Bowles and Candela’s (2005) research discovered the most frequent reason why new graduate
RNs left their first job as issues relating to patient care, such as the acuity of patients,
unacceptable nurse-to-patient ratios, and feeling patient care was unsafe. Another reason
identified included issues with the work environment such as management issues, lack of support
or guidance, and being given too much responsibility (Bowles & Candela).
1.2.3 The Role of Mentorship Programs
Mentoring can be described as a way to support new colleagues with the development of
clinical nursing skills and competencies, and is linked to professionalism, nursing quality
improvement, self-confidence, retention, and job satisfaction (Ronston, Andersson, &
Gustafsson, 2005). Mentoring has also been identified as one successful strategy to guide and
teach new graduate RNs, to develop professional growth for experienced RNs, to promote
recognition of nursing as a profession, and to increase nursing retention (Young, 2009). Mentors
can also experience benefits such as increased confidence in knowledge and skills and ability to
5
provide feedback (Block, Claffey, Korow, & McCaffrey, 2005). The presence of a mentoring
program can provide multiple benefits including developing the growth of expertise in a safe
environment, providing professional encouragement, modelling of professional values and
leadership, modelling expertise through sharing of experiences, facilitating both professional and
personal relationships, easing job transition from novice to graduate nurse, and creating support
systems (Leners, Wilson, Connor, & Fenton, 2006).
1.3 Research Question and Sub-questions
The focus of this thesis will be to answer the following research question:
What is the evidence that new graduate mentorship programs are effective in improving
performance, satisfaction, retention, and confidence in novice nurses’ practice?
This thesis will also answer the following sub-questions:
What are the reported key elements of effective new graduate RN mentorship programs?
What are the reported problems in implementing new graduate RN mentorship programs?
To answer this question and sub-questions a rapid evidence assessment (REA) will be
undertaken (Government Social Research Service [GSRS], 2010). Novice nurses in this research
inquiry are defined as RNs practicing within two years since graduating from nursing school.
1.4 Chapter Summary
This chapter has provided an introduction to nursing shortage and its resulting impact of
new graduate RNs filling ensuing vacancies. It has also provided an overview of the challenges
that newly graduated nurses’ experience, reasons for attrition from nursing, and the role of
mentorship programs. The research question and sub-questions that this thesis will answer was
also introduced. In the next chapter an overview of mentoring, the conceptual framework, and
definitions used will be discussed.
6
CHAPTER 2: Mentoring Overview, Conceptual Framework, and Definitions
In this chapter an overview of mentoring, the conceptual framework, and definitions used
in this research study are presented.
2.1 Overview of Mentoring
The concept of mentoring has been a foundation of nursing practice dating back to the
times of Florence Nightingale (Barton, Gowdy, & Hawthorne, 2005). Mentorship can be defined
as “a relationship between two nurses formed on the basis of mutual respect and compatible
personalities with the common goal of guiding the nurse towards personal and professional
growth” (Block et al., 2005, p. 134). Another definition of mentorship is “a long-term and one-
to-one interpersonal relationship that encourages the personal and professional development of
the mentee” (Chen & Lou, 2014, p. 434).
Although the concept of mentorship is a topic that is clearly defined in the nursing
literature, there is some conflict about the interchangeable use of the term with preceptorship,
especially in the international community (CNA, 2004; Yonge, Billay, Myrick, & Luhanga,
2007). Some authors feel the roles are interchangeable and have been that way since the
inception of this concept in nursing (Allen, 2006; Harvey, 2012) while others cite the main
difference is time commitment – mentorship suggests a long-term relationship whereas
preceptorship is short-term (Block et al., 2005; CNA, 2004; Wensel, 2006). Preceptorship is
more focused on assisting the novice to develop beginning practice competencies through direct
supervision over a limited time period and in Canada often refers to the relationship with nursing
students (CNA, 2004; Yonge et al.). Mentoring focuses on positively influencing personal and
professional growth over a longer time period and can occur within or outside the clinical setting
(CNA, 2004; Wagner & Seymour, 2007).
7
2.1.1 Attributes of an Effective Mentor
The nursing literature appears to be in agreement about the attributes that an effective
mentor should possess to ensure a successful mentoring relationship including personal
attributes, professional skills and abilities, and communication skills (LaFleur & White, 2010).
Some personal attributes that an effective mentor will possess include respect, honesty, patience,
openness, friendliness, enthusiasm, compassion, and flexibility (Academy of Medical-Surgical
Nurses [AMSN], 2012; Fawcett, 2002; Harvey, 2012; LaFleur & White; Wagner & Seymour,
2007). The professional skills and abilities comprise a good knowledge base, teaching and
counseling ability, competence, and ability to think critically (Harvey; LaFleur & White; Wagner
& Seymour). When looking at communication skills, an effective mentor should be diplomatic, a
storyteller, an active listener, able to provide constructive feedback, and possess strong
interpersonal skills (AMSN; CNA, 2004; Harvey; LaFleur & White; Wagner & Seymour).
2.1.2 Attributes of an Effective Mentee
Although the attributes of an effective mentee are not as widely discussed in the
literature, the mentee plays a key role in the success of a mentoring relationship (AMSN, 2012;
Kanaskie, 2006). Mentees should be open to receiving help and guidance from their mentor
while assuming responsibility for their own learning and growth (AMSN; Greene & Puetzer,
2002; Kanaskie). Some of the personal attributes that an effective mentee should possess include
respect, honesty, energy, motivation, initiative, and a strong self-identity (Greene & Puetzer;
Kanaskie).
2.1.3 Phases of a Mentoring Relationship
A mentoring relationship requires thought, time, and care (Cooper & Wheeler, 2010).
The AMSN (2012) characterizes a mentoring relationship into three phases – the beginning,
8
middle, and closing. In the beginning phase, mentors and mentees focus on interpersonal
relationship building through establishing trust, engaging in meaningful dialogue, and
determining learning goals (AMSN). The middle phase is when the mentors offer specific
suggestions about achieving goals; and the mentees experience enhanced self-esteem while
developing and confirming new skills (AMSN). Finally in the closing phase, the mentees feel
comfortable functioning independently, achieve greater autonomy, and become empowered
which results in the relationship being closed (AMSN).
Shaffer, Tallarica, and Walsh (2000) and Kopp and Hinkle (2006) describe four stages
that a mentoring relationship evolves through: initiation, cultivation, separation, and redefinition.
In the initiation stage, the mentor and mentee engage in series of conversations to clarify values
and establish shared interpersonal boundaries such as confidentiality and respect (Kopp &
Hinkle). The cultivation stage is a working phase in which the mentor teaches the mentee how to
navigate unfamiliar environments and adjust to the new responsibilities and demands (Kopp &
Hinkle). During the separation stage, the mentee begins to practice independently and the
mentor slowly takes on the role of a safety net (Kopp & Hinkle). Finally in the redefinition
stage, the mentor and mentee become equal colleagues and communication becomes peer
dialogue instead of novice-expert interactions (Shaffer, Tallarica, & Walsh).
Cooper and Wheeler (2010) developed a five-phase mentoring relationship model to help
mentors and mentees build an effective relationship: purpose, engagement, planning, emergence,
and completion. In the purpose phase, a clearly articulated intention for the mentoring
relationship is developed consisting of career vision, goals, and plans (Cooper & Wheeler). The
engagement phase focuses on the mentor and mentee determining whether their mutual goals,
learning needs, and learning styles fit and then deciding to enter into a mentoring relationship
9
(Cooper & Wheeler). During the planning phase, a mentoring action plan is developed which
includes goals, action steps, resources, timelines, and evaluation of the mentoring relationship
along with the expectations that the mentor or mentee have (Cooper & Wheeler). The
emergence phase is where the mentoring relationship evolves and the mentor facilitates the
growth and development of the mentee through supporting, encouraging, and challenging
(Cooper & Wheeler). Finally, the completion phase is a time for celebrating accomplishments,
redefining the relationship, and examining what the next steps may include (Cooper & Wheeler).
2.1.4 Types of Mentoring Relationships
A mentoring relationship may be a formal or informal arrangement (AMSN, 2012; CNA,
2004; Dunn, 2014; Tourigny & Pulich, 2005). Both formal and informal mentoring relationships
can be effective as long as the mentor and mentee are committed to the relationship and agree to
identify and meet the needs of the mentee (AMSN). Formal mentoring programs are developed
by an organization and usually involve establishing the mentoring objectives and duration,
selecting and matching mentor-mentee dyads, and determining the frequency of time spent in
mentoring activities (AMSN; Tourigny & Pulich). The advantages of a formal mentoring
program include fostering career and organization commitment along with higher levels of
involvement in the nursing profession (Tourigny & Pulich). The disadvantages may include
mentor-mentee mismatch, the potential for role conflict and ambiguity, and the effects on
personal learning could only be short-term ceasing at the duration of the contract (Tourigny &
Pulich).
Informal mentoring is unstructured, occurs spontaneously, and based upon mutual
identification and personal development needs (AMSN, 2012; CNA, 2004; Tourigny & Pulich,
2005). An informal mentoring relationship may be either hierarchical – superior to employee or
10
peer – employee to employee (Tourigny & Pulich; Yonge et al., 2007). The advantages of
informal mentoring include mutual identification, increased potential for learning as these
relationships often extend over a long period, and activities are not restricted by contracts
(Tourigny & Pulich). The disadvantages may include perceived favoritism by other coworkers,
greater potential for role conflict, and lack of recognition and control by the organization
(Tourigny & Pulich).
2.2 Conceptual Framework
There are many different conceptual frameworks that can be applied to the concept of
mentoring and could have been selected to guide this research study. Benner’s (1984) novice to
expert theory is based upon the Dreyfus Model of Skill Acquisition tool and posits that a nurse
passes through five levels of proficiency in the development of a skill: novice, advanced
beginner, competent, proficient, and expert. In the progression to the expert level of proficiency,
the reliance on rules and guidelines changes to intuition and decision making based upon past
experiences (Benner). Duchscher’s (2008) stages of transition theory which suggest that
allowing new graduates time to adjust to what ‘is’ within a context of support that allows them to
develop their thinking and practice expertise will assist them to move through the stages of
professional role transition. The initial transition to professional practice is believed to last about
twelve months and during that time a new graduate nurse evolves through three stages: doing,
being, and knowing (Duchscher). Ultimately, Bandura’s (1977) social learning theory was
selected as the conceptual framework to guide this research study.
Social learning theory is a perspective on learning that includes consideration of the
personal characteristics of the learner, behavior patterns, and the environment (Braungart &
Braungart, 2008). It is suggested that individuals learn by observing, imitating, and modeling
11
other’s behaviors, attitudes, experiences, and consequences of behaviors (Bandura, 1977).
Considerable learning occurs when individuals take note of other people’s behaviors and what
happens to them, as a result learning via role modelling (Braungart & Braungart). For example,
when applying the concept of social learning theory an experienced nurse who possesses the
characteristics of clinical competence, knowledge and expertise of their practice area, self-
confidence, and enthusiasm (Kaviani & Stillwell, 2000) could be used as a mentor for a less
experienced nurse (Braungart & Braungart). Social learning theory was selected over Benner’s
(1984) novice to expert theory and Duchscher’s (2008) stages of transition theory because its
central concept is based upon role modeling which has been shown in the literature to be an
important aspect of mentorship (Allen, 2006; AMSN, 2012; CNA, 2004). As well, another key
attribute for the selection of social learning theory is the impact that the mentoring relationship
has on the mentor and mentee and the supportive environment that is required to be successful.
A mentoring relationship works best when both the mentor and mentee are actively engaged and
collaborate to meet the goals of both individuals.
Bandura (1977) defined a four-step, largely internal process that directs social learning as
shown in Figure 2.1 below (Braungart & Braungart, 2008).
12
Figure 2.1 Bandura’s Direction of Social Learning Processes
Source: Bandura’s Direction of Social Learning Process (Braungart & Braungart, 2008).
When looking at the relationship between a new graduate RN and their mentor in the
attentional phase, the mentor is the role model that the new graduate RN observes and then
models their observed behavior. Role models with a high status and competence are more likely
to be observed, thus an important aspect of the mentoring relationship is to have the new
graduate RN think of their mentor as a role model (Braungart & Braungart, 2008). It may be
challenging for a new graduate RN to think of their mentor as a role model when a mentoring
relationship is arranged, as often found in formal arrangements, therefore the use of informal
mentoring may be preferred when applying the concept of social learning theory.
In the context of this research study, the retention phase would have the new graduate RN
retain the observed behaviors of their mentor by imaginal and verbal systems which can then be
further reinforced by rehearsal and repeated exposure (Bahn, 2001). For example, retention of
13
observed behaviors in a new graduate RN can be aided by maintaining a reflective diary of their
experiences and recording their thoughts and feelings (Bahn).
In the reproduction phase the new graduate RN will perform the tasks or actions from
memory that they previously observed their mentor performing. During this phase it is important
for feedback to be provided by the mentor since individuals cannot observe their own
performance (Bahn, 2001). Feedback should be corrective and provide the new graduate RN
with enough information that they can perform self-corrective adjustments (Bandura, 1977).
Feedback can also assist the new graduate RN with building their perception of self-efficacy
which Bandura defines as confidence in one’s ability to take action and persist in action, thus it is
an important aspect of a new RN’s practice to develop.
Learning by a new graduate RN in the motivational phase focuses on whether they are
motivated to perform a certain type of behavior and is influenced by vicarious reinforcement and
punishment (Braungart & Braungart, 2008). Vicarious reinforcement is the result of learning by
observing the mentor’s successes and failures (Bahn, 2001). Upon observation of the
appropriate and effective management of the mentor’s successes and failures can result in
valuable learning experiences for the new graduate RN whereas inappropriate or aggressive
responses can greatly reduce learning (Bahn). For example, if the new graduate RN observes
their mentor engaging in a difficult conversation with another co-worker that involves yelling,
the resulting outcome may be a negative impact on the new graduate RN’s learning. Bandura
(1977) suggests that some people respond to their own actions by self-reward, which can result
in a great sense of pride, or self-punishment. Mentors can encourage individualized learning to
meet the needs of the new graduate RN and increase their sense of achievement but should be
mindful of unrealistically high standards that can result in failure (Bahn). One suggestion to
14
prevent these unrealistically high standards is the development of a learning contract, based on
assessment of the new graduate RN’s capabilities so that realistic, measurable, and achievable
goals are set (Bahn).
2.3 Definitions
The following definitions are used in this research study:
Competence: Refers to the potential ability and/or a capability to function in a given situation
and makes one capable of fulfilling his/her job responsibilities (Schroeter, 2008).
Confidence: Part of, related to, or integral to professional identity; self-confidence and self-
esteem; self-concept; competency; clinical competence; critical thinking; and self-efficacy
(Brown et al., 2003).
Evaluation: The process of making judgments about learning and achievement, clinical
performance, and competence based upon assessment data (Oermann & Gaberson, 2009).
Mentee: Someone who chooses to be counseled, guided, and advised (McBurney, 2015).
Mentor: Someone who serves as a career role model and actively advises, guides, and promotes
another’s career and training (Mills & Mullins, 2008).
Mentoring: A nurturing process, in which a more experienced person, serving as a role model,
teaches, sponsors, encourages, counsels, and befriends a less experienced person for the purpose
of promoting the latter’s professional and/or personal development (Meier, 2013).
Mentorship program: Provides formal, structured partnering of individuals with a prescribed list
of activities or skill sets to review, orient to, or demonstrate competency in (Wilson, Andrews, &
Leners, 2006).
Novice or new graduate RN: An entry level RN who is a recent graduate from a recognized
nursing education program (College of Registered Nurses of British Columbia [CRNBC], 2014).
15
Performance: In nursing, can be measured by competencies, nursing-sensitive quality indicators,
and measures of performance on specific tasks (DeLucia, Ott, & Palmieri, 2009).
Preceptorship: A formal one-to-one relationship of pre-determined length, between an
experienced nurse and a novice designed to assist the novice in successfully adjusting to and
performing a new role (CNA, 2004).
Retention: Actions and strategies taken to keep the nurses currently employed by a hospital or
organization (Westendorf, 2007).
Satisfaction: Consists of a feeling of wellbeing, resulting from the interaction of several
occupational aspects and may influence the worker’s relationship with the organization, patients,
and family (Melo, Barbosa, & Souza, 2011).
Social learning: Behaviour is learned from the environment through the process of observational
learning (Bandura, 1977).
2.4 Chapter Summary
This chapter presented an overview of mentoring comprising the attributes of an effective
mentor and mentee, phases of a mentoring relationship, and types of mentoring relationships. As
well, Bandura’s (1977) social learning theory was introduced as the conceptual framework
guiding this research study and a list of definitions used in this research inquiry was provided.
In the next chapter the REA approach and methodology will be presented.
16
CHAPTER 3: Approach and Methodology
In this chapter an overview of the REA approach, the justification for its selection, the
REA methodology, and the REA analysis will be presented.
3.1 Approach
An REA is a systematic review method to search and critically appraise existing research
on a topic in a shorter time period, two to six months, compared with a full systematic review
that normally takes a minimum of eight to twelve months (GSRS, 2010). The REA method is
especially useful to quickly gather existing evidence in a research area and determine what future
research can be conducted (Boycott, Schneider, & McMurran, 2012). An REA involves a
specific and rigorous methodology, but is advantageous as it can be conducted by a single
researcher (Garrett, 2012). Healthcare particularly demands rapid access to current research to
ensure evidence-informed decision making and practice (Ganann, Ciliska, & Thomas, 2010;
Watt et al., 2008). The REA method may be driven by clinical urgency and intense demands for
uptake of technology, or may be determined by limited time and resources (Ganann, Ciliska, &
Thomas). When looking at an evidence hierarchy for confidence in review studies, the REA
method is found just below a full systematic review as shown in Figure 3.1 below (Garrett).
17
Figure 3.1 Confidence in Review Studies
Source: Figure 3.1 Confidence in Review Studies
http://hlwiki.slais.ubc.ca/index.php/File:Evidence-review-types.jpg
Although the REA process aims to be rigorous and explicit in method, it does have some
limitations over a full systematic review including the depth and breadth of the search process
which is limited thereby increasing the potential of introducing bias (GSRS).
3.2 Justification
The REA method was selected for this research study since it involves a specific and
rigorous methodology while possessing the ability to be conducted by a single researcher in a
shorter period of time (Garrett, 2012). A full systematic review is the most robust way to review
evidence but they are time and resource consuming, often requiring a team of researchers, and
outside the scope of this thesis (GSRS, 2010). According to the GSRS, an REA will provide a
balanced assessment about what is already known about mentorship and new graduate RNs by
using a systematic review method to search and critically appraise any existing research. The
18
REA method makes concessions to the breadth of the process by limiting particular aspects of
the systematic review process including the question, searching of literature, screening of
literature, mapping stage, data extraction, or appraisal and/or synthesis of studies (GSRS).
3.3 Methodology
3.3.1 Formulating the Question
The first step in the REA method involves formulating a research question and
determining whether it is an impact or non-impact question (GSRS, 2010). According to the
GSRS, the REA question should be the driver for all REA processes, a statement that can be
investigated rather than a subject of interest, clear and answerable, and be worth answering. An
impact question reflects a deductive approach, and addresses “what works” inquiries. It focuses
on finding studies that investigated the population of interest, and intervention one is interested
in, using a suitably rigorous method such as having one control group, and quantitatively
measuring the interested outcomes (GSRS). A non-impact question is more of an inductive
exploratory question, and appropriate to answer a range of inquiries including needs, process,
implementation, correlation, attitude, and economic questions (GSRS). An REA question will
lead the direction of the research, consequently having a significant effect on the conclusions and
a narrower focus may limit the available evidence whereas a broader question is likely to require
more extensive resources (GSRS). As well, an REA can address more than one type of question,
particularly in combining impact questions with implementation and economic questions
(GSRS). The research question posed in this research study is an impact question to identify the
impact of a mentorship program on novice nurses’ performance, satisfaction, retention, and
confidence as presented in chapter one. The sub-questions are also impact questions to identify
19
the key elements of an effective mentorship program and the reported problems in implementing
a mentorship program.
3.3.2 Inclusion and Exclusion Criteria
The next step in the REA method involves deciding on the inclusion and exclusion
criteria. Before conclusions can be drawn from the studies that have been selected for inclusion,
they need to be critically appraised to ensure relevancy and reliability of the findings (GSRS,
2010). In order to ensure the process is rapid constraints are imposed on the inclusion criteria
and may include the nature of what’s being studied, setting and population, date of research,
research methods, and language of report (GSRS). According to Gough (2007), there are three
main dimensions to be considered in the appraisal of quality and relevance of studies: the
methodological quality of the study, the relevance of the research design for answering the REA
question, and the relevance of the study focus for answering the REA question. The following
inclusion and exclusion criteria were developed utilizing these three dimensions for this research
inquiry:
Inclusion criteria
• Research studies written in the English language utilizing quantitative, qualitative, or
mixed-methods research, selected for inclusion to avoid the introduction of a language
bias and to ensure the included research studies provide a high level of evidence to
answer the REA question and sub-questions.
• Studies published within the past 15 years, selected for inclusion to include only the most
current research.
20
• Studies that involved a one-to-one mentorship program, selected for inclusion to avoid
confusion with the effect of a mentorship program if it were offered in more than one
way.
• Studies that included participants who were practicing RNs in their first two years of
work, selected for inclusion in view of the fact that this is what the literature offers as a
definition of a new graduate RNs.
• Studies that were available from an electronic bibliographic database, selected for
inclusion due to the time constraint of only three weeks being allotted for literature
searching in this REA.
Exclusion criteria
• Studies that focused on nursing students, selected for exclusion due to the fact that the
learning needs of a nursing student is vastly different than that of a new graduate RN and
grouping them together could have confused the results.
• Studies that offered a group mentoring program, selected for exclusion to avoid
confusion about the effect of a mentorship programs if it were offered in more than one
way.
• ‘Grey’ literature, selected for exclusion due to the time constraint of only three weeks
being allotted for literature searching in this REA.
• Opinion papers, abstracts, or letters to editors, selected for exclusion to ensure the
included research studies provide a high level of evidence to answer the REA question
and sub-questions.
21
3.3.3 Search Strategy
The next step in the REA method is specifying the methods utilized for conducting the
search. An REA search strategy should be principled, planned, rigorous, taken with care and
checked, explicitly reported, and grounded in the research question (GSRS, 2010). The GSRS
identifies four elements that a search strategy should employ and all elements were incorporated
in this REA.
1. The first element involves what is being searched for as defined by the inclusion criteria
and this is reported earlier in this chapter.
2. The second element involves including the sources that will be searched (GSRS). To
answer the research question and sub-questions identified in this REA the following
electronic databases will be searched: Cumulative Index to Nursing and Allied Health
Literature (CINAHL), Medline, PubMed, Embase, Web of Science, and ProQuest
Dissertations and Theses. Web of Science and ProQuest Dissertations and Theses will be
selected to search for studies that may have never been published in a journal but
involved research on mentorship and new graduate RNs.
3. The third element involves how the databases will be searched and what search terms will
be used (GSRS). The type of search strategy employed in this REA will be
comprehensive or exhaustive searching which aims to identify as much literature as
possible that meets the inclusion criteria (GSRS). Comprehensive searching will be
selected since this method has increased sensitivity, refers to the amount of literature that
is found, as opposed to increased specificity, refers to the amount of relevant versus non-
relevant literature that is found; although clearly defined search terms can help balance
sensitivity and specificity as well (GSRS). The following search terms were selected:
22
New graduate nurse AND mentor*
Novice nurse AND mentor*
Newly qualified RN and mentor*
Mentor* of novice nurses
Mentor* of graduate nurses
New graduate nurse AND preceptor*
Novice nurse AND preceptor*
Preceptor* of novice nurses
Preceptor* of graduate nurses
Benefits of mentor*
Mentorship programs
4. The final element involves writing up the actual detailed methods of the search strategy
to provide readers with the ability to see how the search was undertaken; therefore being
transparent (GSRS).
Once the search strategy was defined as above, the author proceeded to search the
literature for relevant studies for a period of three weeks. Systematic literature searching
includes electronic sources, print sources, and ‘grey’ literature (GSRS, 2010). Due to the limited
timeframe the searching for literature in this REA only included electronic sources and hand
searching of print sources from the references of relevant research studies, which is a potential
limitation that will be described further in chapter five.
3.3.4 Data Collection
The next step of the REA method is data collection which involves two main
components: the location and the description of the research studies (GSRS, 2010). Studies were
23
located through searching of the electronic databases identified earlier in this chapter and
through hand searching of the references of relevant research studies. According to the GSRS,
access to good library facilities is essential for the completion of a successful REA and the
University of British Columbia’s (UBC) library was utilized for data collection during this step.
Once the research studies have been located the references need to be recorded through standard
word-processing and spreadsheet applications or reference management software (GSRS). The
selected research studies used in this REA were catalogued using the Mendeley bibliographic
database software and key elements of the work tabulated into a Microsoft Excel spreadsheet,
grouped according to their research method: quantitative, mixed-methods, or qualitative (See
Appendix A).
3.3.5 Screening and Selecting Studies
Once the initial research studies were identified, they were screened to ensure they met
the identified inclusion and exclusion criteria. This was a two-step process that involved
reviewing the abstract and then reading the full article (GSRS, 2010). The author found that
several research studies needed to be excluded after reading the full article due to conflicting
definitions of a novice nurse. The screening process can be very time-consuming and one
strategy to keep the screening rapid was utilizing a time limit of three weeks for screening and
selecting.
3.3.6 Scoring
Each research study was scored using the GSRS Weight of Evidence (WoE) tool (EPPI-
Centre, 2007), found in Appendix B, in which each study is weighted according to three
dimensions with scores then ranked as either low evidence, medium evidence, or high evidence.
Quantitative and mixed-methods research studies are scored using the Maryland Scale of
24
Scientific Methods (MSSM) tool, found in Appendix C (Sherman et al., 1997). The MSSM is a
five-point scale for classifying the strength of methodologies and scores are ranked as either low
evidence, medium evidence, or high evidence (GSRS, 2010). For qualitative and mixed-methods
research studies, scoring was through the Critical Appraisal Skills Programme (CASP) tool,
found in Appendix D (Public Health Resource Unit, 2006). The CASP score is based upon ten
questions and ranked as either low evidence, medium evidence, or high evidence (GSRS).
Table 3.1 Categorical Ranking of Scores
WoE Score: Low = 3, Medium = 4-6, High = 7-9 (all studies)
MSSM Score: Low = 1, Medium = 2-3, High = 4-5(quantitative and mixed-methods studies)
CASP Score: Low = 1-3, Medium = 4-7, High = 8-10 (qualitative and mixed-methods studies)
3.4 Analysis
3.4.1 Critical Appraisal
The key information from each research study was systematically described using a data
extraction form and coding the information collected (GSRS, 2010). The data extraction form
allows researchers to identify, extract, and code information about each individual research study
and the tool utilized in this research inquiry was the Evidence for Policy and Practice
Information (EPPI)-Centre Data Extraction and Coding Tool for Education Studies (2007), found
in Appendix E. Using a data extraction tool allows for mapping - providing a description of each
study to build up a map of the research field, and synthesis - providing information to enable
synthesis such as how studies were undertaken for quality and relevance appraisal; the study
findings; and reporting on aspects of individual studies (GSRS).
25
The author critically appraised the selected research studies to ensure the findings are
relevant and reliable; and to separate those research studies that are higher quality from the
weaker ones (GSRS, 2010). This was a two-step process that involved the author critically
appraising and scoring all the research studies at one time and then re-critically appraising and
re-scoring each research study the following week. All seventeen research studies were only
critically appraised and scored by the author of this REA, which is a potential limitation that will
be discussed further in chapter five. The three main dimensions considered in quality and
appraisal of studies according to Gough (2007) include: the relevance of the research design in
answering the REA question, the relevance of the study focus for answering the REA question,
and the methodological quality of evidence for the research study being considered, all of which
were considered in this REA. The findings of lower quality studies are to either be excluded or
given less weight in the synthesis (GSRS). All seventeen research studies included in this REA
are categorized as medium evidence or high evidence based upon the WoE and CASP scores and
the results are tabulated in a Microsoft Excel spreadsheet, found in Appendix A.
3.4.1.1 EPPI-Centre Data Extraction and Coding Tool for Education Studies
This tool, found in Appendix E, was designed to help researchers identify, extract, and
code information from a single primary research study (EPPI-Centre, 2007). The purpose is to
help reviewers obtain all the necessary information to assess the quality of a study, identify the
relevant contextual information that may have affected the results, identify the contextual
information that is relevant to any assessment of generalizability, and identify relevant
information about the design, execution, and context of a study for the purpose of synthesizing
results (EPPI-Centre).
26
3.4.1.2 GSRS Weight of Evidence (WoE) Tool
This tool, found in Appendix B, assesses all research studies and includes four specific
criteria for scoring: A - takes into account whether the study findings can be trusted in answering
the research question(s), B - assesses the appropriateness of the research design and analysis in
addressing the research question(s), C - assesses the relevance of REA topic for answering the
research question(s), and D - an overall weight of evidence score (EPPI-Centre, 2007). WoE A,
B, and C can each receive a score of three for high evidence, two for medium evidence, and one
for low evidence and the overall score for WoE D is either three for low level of evidence, four
to six for medium level of evidence, or seven to nine for high level of evidence.
3.4.1.3 Maryland Scale of Scientific Methods (MSSM) Tool
This tool, found in Appendix C, is used to appraise methodological quality of quantitative
research studies and assists with identifying potential threats to internal validity (Sherman et al.,
1997). It is a five-level scale: 1 – the measurement of impact of a specific intervention at a single
point in time, 2 – before and after scores following an intervention, 3 – before and after scores
following an intervention with a second control group, 4 – comparison between more than two
groups with and without the intervention, and 5 – randomized controlled trials. Overall MSSM
scoring is either low level of evidence (Level 1), medium level of evidence (Level 2 & 3), or
high level of evidence (Level 4 & 5).
3.4.1.4 Critical Appraisal Skills Programme (CASP) Tool
This tool, found in Appendix D, is an appraisal method for qualitative research studies
which assesses rigour, credibility, and relevance (Public Health Resource Unit, 2006). It is based
upon ten questions that receive a score of one for yes and zero for no and with the answers
27
totaled to get a final score out of ten. Overall CASP scoring is either low level of evidence
(score of 1-3), medium level of evidence (score of 4-7), or high level of evidence (score of 8-10).
3.4.2 Synthesis of Findings
The synthesis stage generates the findings to answer the questions and sub-questions and
allow conclusions to be drawn from the selected research studies (GSRS, 2010). Synthesis of the
results occurred through the use of the following analytical tools: EPPI-Centre Data Extraction
and Coding Tool for Education Studies; WoE tool; MSSM tool; and CASP tool, described earlier
in this chapter, to provide evidence that mentorship programs are effective in improving
performance, satisfaction, retention, and confidence in novice nurses’ practice. These findings
were also linked to the conceptual framework identified in chapter two as well as discussing the
reported key elements of an effective mentorship program and problems in implementing a
mentorship program. Narrative synthesis described by Popay et al. (2006) as the “synthesis of
findings from multiple studies that relies primarily on the use of words and text to summarise
and explain the synthesis of findings of the synthesis” (p. 5) was employed in this REA as
outlined in Figure 3.2 below.
28
Figure 3.2 Stages of Synthesis
Source: Figure 3.2 Stages of Synthesis
http://www.civilservice.gov.uk/networks/gsr/resources-and-guidance/rapid-evidence-
assessment/how-to-do-a-rea
3.4.3 Communicating Findings
The final step in the REA method involves communicating the findings in a way that will
engage with users of the research evidence (GSRS, 2010). A written report is necessary to
provide transparency and enables the readers to see how the review was conducted to provide
accountability of the process of the review and its potential for replication (EPPI-Centre, 2010).
The EPPI-Centre recommends three different formats for reports: a short one page summary on
the findings of the review; a short user-friendly report about 25 pages which concentrates on the
findings and context of the review and gives a more thorough description of the findings; or a
technical report about100 pages that includes all the fine details of the methods. The findings in
this REA are communicated through the short user-friendly report format and the results of the
29
selected research studies are presented in chapter four and the results of the analysis are
presented in chapter five.
3.5 Chapter Summary
This chapter presented an overview of the REA approach; the justification for its
selection as an appropriate methodology; the REA methodology including identification of
inclusion and exclusion criteria, search strategy, data collection, screening and selecting of
studies, and scoring; and the REA analysis using quality of evidence assessment tools. In the
next chapter an overview of the included seventeen research studies along with the critical
appraisal scoring and their findings will be described.
30
CHAPTER 4: Results
In this chapter an overview of the final seventeen selected research studies along with
their Weight of Evidence, and Maryland Scale of Scientific Methods Scores.
4.1 Selected Research Studies
A total of 66 research studies were initially selected and their abstracts reviewed to screen
for meeting the inclusion criteria and answering the research question and sub-questions
identified in chapter 1. Twenty-one research studies were then selected to be included in this
REA but upon further review four had to be excluded due to their definitions of a new graduate
RN not meeting the inclusion criteria leaving a final total of seventeen research studies for this
REA. The data from each research study was extracted using the EPPI-Centre Data Extraction
and Coding Tool for Education Studies (see Appendix E), and the studies categorized according
to the research approach used (quantitative, mixed-methods, or qualitative) to allow for mapping
and analysis. Each research study was then scored using the tools described in chapter three and
are listed from highest level of evidence to lowest level of evidence, according to their GSRS
WoE level, MSSM, and CASP scores as appropriate for the type of study.
31
Figure 4.1 Research Studies Selection Process
The list of seventeen research studies is presented below in Table 4.1:
Table 4.1 List of Selected Research Studies
Author Year Country Study Title Method Sample
Size
Almada, P.,
Carafoli, K.,
Flattery, J.B.,
French, D.A., &
McNamara, M.
2004 USA Improving the retention
rate of newly graduated
nurses.
Mixed-Methods 40
Beercroft, P.C.,
Santner, S., Lacy,
M.L., Kunzman, L.,
& Dorey, F.
2006 USA New graduate nurses’
perceptions of mentoring:
Six-year programme
evaluation.
Mixed-Methods 318
Bialkowski, K. 2009 Canada Impact of mentoring on
job satisfaction and
retention.
Mixed-Methods 21
Fox, K.C. 2010 USA Mentor program boosts
new nurses’ satisfaction
and lowers turnover rate.
Evaluation
Survey
12
32
Author Year Country Study Title Method Sample
Size
Grindel, C.G. &
Hagerstrom, G.
2009 USA Nurses nurturing nurses:
Outcomes and lessons
learned.
Longitudinal
129
Haggerty, C.,
Holloway, K., &
Wilson, D.
2013 New
Zealand
How to grow our own:
An evaluation of
preceptorship in New
Zealand graduate nurse
programmes.
Mixed-Methods 1023
Hale, R. 2004 USA Mentorship of nurses: An
assessment of the first
year of licensure.
Descriptive
Exploratory
144
Halfer, D., Graf, E.,
& Sullivan, C.
2008 USA The organizational
impact of a new graduate
pediatric nurse mentoring
program.
Descriptive
Survey
234
Hardyman, R., &
Hickey, G.
2001 UK What do newly-qualified
nurse expect from
preceptorship?
Exploring the perspective
of the preceptee.
Longitudinal
Survey
1512
Hunsberger, M.,
Baumann, A., &
Crea-Arsenio, M.
2013 Canada The road to providing
quality care: Orientation
and mentorship for new
graduate nurses.
Mixed-Methods 3813
Komaratat, S., &
Oumtanee, A.
2009 Thailand Using a mentorship
model to prepare newly
graduated nurses for
competency.
Quasi-
Experimental
19
Lewis, S., &
McGowan, B.
2015 UK Newly qualified nurses’
experiences of a
preceptorship.
Qualitative 8
Lindsey, K.S. 2000 USA Perceptions of novice
nurses job satisfaction
levels related to
mentoring.
Descriptive
Comparative
Survey
163
Marks-Maran, D.,
Ooms, A., Tapping,
J., Muir, J., Phillips,
S., & Burke, L.
2013 UK A preceptorship
programme for newly
qualified nurses: A study
of preceptees’
perceptions.
Mixed-Methods 44
Navarro, J. 2009 Canada The mentoring
experiences and self-
efficacy of new graduate
nurses during transition
Descriptive
Exploratory
5
33
Author Year Country Study Title Method Sample
Size
from student to
professional nurse.
Smith, C.B. 2006 USA The influence of
mentoring on goal
attainment and role
satisfaction for registered
nurses in acute care
facilities.
Descriptive
Correlational
51
Wolak, E.S. 2007 USA Perceptions of an
intensive care unit
mentorship program.
Focus Group 5
Nine of the research studies were conducted in the United States of America, three in Canada,
three in the United Kingdom, and one each in Thailand and New Zealand
Figure 4.2 Publication Year of Research Studies
0
1
2
3
4
5
2000 2001 2004 2006 2007 2008 2009 2010 2013 2015
Nu
mb
er
of
Stu
die
s
Year
Publication Year of Research Studies
34
Figure 4.3 Research Study Sample Size Histogram
4.2 Excluded Research Studies
The 49 excluded research studies did not meet the inclusion criteria and answer the
research question and sub-questions. Several of these research studies focused on new graduate
RNs in a speciality practice area which meant they had been practicing nursing for more than
two years and others considered new graduate RNs as those who have been practicing less than
three years. Six research studies were discovered to be based upon the same bodies of research
therefore only three were selected for inclusion. Other studies focused on group mentoring
programs but the majority excluded were opinion papers rather than research studies.
4.3 Quantitative Research Studies
Eight of the research studies used quantitative methods and were critically appraised
using the following tools:
1. GSRS Weight of Evidence (WoE) tool, found in Appendix B.
2. Maryland Scale of Scientific Methods (MSSM) tool, found in Appendix C.
0
1
2
3
4
Nu
mb
er
of
Stu
die
s
Sample Sizes
Reasearch Study Sample Sizes
35
Table 4.2 Quantitative Study Details
Study Author Study Approach WoE Level
& Score
MSSM
Level &
Score
Lindsey, K.S. (2000)
Grindel, C.G., & Hagerstrom,
G. (2009)
Komaratat, S., & Oumtanee,
A. (2009)
Hale, R. (2004)
Halfer, D., Graf, E., &
Sullivan. (2008)
Descriptive Comparative Survey
Longitudinal
Quasi-Experimental
Descriptive Exploratory
Descriptive Survey
High 9
High 8
High 8
High 8
High 7
Medium 3
Medium 2
Medium 2
Low 1
Medium 3
Smith, C.B. (2006)
Fox, K.C. (2010)
Hardyman, R., & Hickey, G.
(2001)
Descriptive Correlational
Evaluation Survey
Longitudinal Survey
Medium 6
Medium 5
Medium 4
Low 1
N/A
N/A
Five quantitative research studies were scored as high WoE level of evidence and three as
medium WoE level of evidence. Four quantitative research studies were scored as MSSM
medium level of evidence, two as MSSM low level of evidence, and two scored N/A which
means they only used descriptive statistics.
36
Figure 4.4 Quantitative Critical Appraisal Scoring
Table 4.3 GSRS WoE Assessment Criteria Questions and Score
Study Author WoE A:
Methodological
Quality of Study
WoE B:
Relevance of
Research Design
for Answering
REA Question
WoE C:
Relevance of
Study Focus for
Answering REA
Question
WoE D: Overall
Score
Lindsey, K.S.
High High High High 9
Grindel, C.G., &
Hagerstrom, G.
Medium High High High 8
Komaratat, S., &
Oumtanee, A.
High Medium High High 8
Hale, R.
High High Medium High 8
Halfer, D., Graf,
E., & Sullivan.
High Medium Medium High 7
Smith, C.B.
Medium Medium Medium Medium 6
Fox, K.C.
Medium Medium Low Medium 5
Hardyman, R., &
Hickey, G.
Low Medium Low Medium 4
0
1
2
3
4
5
6
7
8
9
Sco
re
Study Author
Quantitative Critical Apprasial Scoring
WoE Score
MSSM Score
37
4.3.1 WoE High Level Studies
Lindsey (2000) used a descriptive, non-experimental comparative evaluation survey
design to investigate the level of job satisfaction of new graduate RNs who participated in a
mentoring program versus those who did not participate in a mentoring program. Random
sampling was utilized to select the 163 RNs who took part in the study. The instrumentation
used to measure the independent and dependent variables included a Job Satisfaction Survey
tool, a 20-question Likert scale, and a Quality of Mentoring tool, a 14-question Likert scale.
This researcher performed several different types of analysis to test the hypothesis
involving job satisfaction and mentored versus non-mentored RNs. An independent t-test was
run using the Job Satisfaction Survey as the dependent variable and mentored versus non-
mentored RN as the independent variable. The results were statistically significant with
mentored nurses having a higher total score on the Job Satisfaction Survey compared to non-
mentored nurses (t = 2.66; p < .01). Another independent t-test was conducted using question
21, a job satisfaction rating, as the dependent variable and mentored versus non-mentored RN as
the independent variable. The results were statistically significant with mentored RNs giving a
higher job satisfaction rating compared to non-mentored RNs (t = 2.69; p < .01). Finally, the
author performed a cross tabulation and chi-square analysis to see if there was a statistically
significant relationship between mentored versus non-mentored RNs and the question - Are you
satisfied with your job? The results of the chi-square analysis did show a statistically significant
relationship between the two variables (X2 =
16.55; p < .01).
Grindel and Hagerstrom (2009) used a longitudinal design to evaluate the effectiveness of
the Nurses Nurturing Nurses (N3) mentorship program whose goal was to enhance RNs’ job
satisfaction and intent to stay in the agency of employment. The purpose of the N3 program was
38
to examine the effect of a mentor-mentee program on job satisfaction, new RN confidence, intent
to stay, and satisfaction with both the mentorship relationship and the program. The
instrumentation used included a Job Diagnostic Survey tool, a fifteen statement 7-point Likert
scale; a Nurse Job Satisfaction Survey tool, a 26-item questionnaire using a 5-point Likert scale;
and a New Nurse Confidence Scale tool, a 26-item 5-point Likert scale. Data was collected at
four different points over a twelve-month period: two weeks into the program (Time 1), at three
months (Time 2), at six months (Time 3), and at twelve months (Time 4).
Repeated measures analysis of variance (ANOVA-RM) calculation was conducted on
mean scores for new nurse confidence from Time 1 through Time 3 and showed a significant
increase in RN confidence scores (F = 47.5; p = .000). Job satisfaction mean scores were
moderately high at Time 1 and remained stable throughout the study, therefore the ANOVA-RM
results indicated no change over the first six months (F = .195; p = .824). Intent to stay was
measured at Times 2, 3, and 4; the mean scores were moderately high throughout the first six
months and only rose slightly at Time 4. The sample size at Time 4 was small and not included
in this analysis, thus there was no difference between participant scores on intent to stay at Time
2 and Time 3 (t = -.38; p = .970).
Komaratat and Oumtanee (2009) used a quasi-experimental, one-group time series design
to study the level of nursing competency of newly graduated RNs after using a mentorship
model. Nineteen new graduate RNs took part in the program and their competency scores were
measured at three times: before the experiment (Time 1), one month later (Time 2), and after the
mentorship experience was completed (Time 3). The competency scores at Time 1 and Time 2
were baseline scores before the mentorship started to document that there were no confounding
variables affecting the scores. The instrument used was the Nursing Competency Scale which
39
consisted of twenty questions with a 5-point rating scale to evaluate four dimensions: nursing,
human relationship and communication, decision-making and problem-solving, and quality
development and assurance.
These researchers analysed the data using the Wilcoxon signed ranks test and set the
significance level at 0.05. They found that the nursing competency of new graduate RNs after
using the mentorship model produced significantly higher scores than pre-experiment time one
(Z = -3.831) and time two (Z = -3.825) which supports their hypothesis. There was no difference
reported in nursing competency of new graduate RNs pre-experiment between Time 1 and Time
2 (Z = -1.155).
Hale (2004) used a descriptive, exploratory design that explored mentorship relationships
from the perspective of new graduate RNs in their first year of licensure. Stratified random
sampling was used to select the 144 participants who agreed to take part in this study. Of the
144 participants who responded, 82% (n = 118) reported having a mentorship relationship with
33% (n = 48) reporting a formal relationship and 49% (n = 70) reporting an informal
relationship. The Hale Mentorship Assessment for Nurses instrument was developed by the
author and included 63 4-point items on a forced choice scale, four questions about mentorship,
and fifteen questions about demographics.
This researcher assessed the consequences of the mentorship relationship to determine if
the new graduate RNs reported positive benefits from the relationship. New graduate RNs in a
mentorship relationship reported increased self-confidence (94%), competence as an RN (95%),
job satisfaction (86%), and satisfaction with their nursing career (88%). One-way ANOVA was
performed by the researcher to determine whether there was any difference between formal (M =
40
157.58) and informal (M = 161.61) mentorship relationships and no statistically significant
difference was found.
Halfer, Graf, and Sullivan (2008) used a descriptive survey design to compare the job
satisfaction and retention rates of two cohorts of new graduate RNs – one before and one after
the implementation of a Pediatric RN Internship Program. The sample consisted of 84 new
graduate RNs in the pre-implementation group and 212 new graduate RNs in the post-
implementation group with 234 participants responding to the surveys. The researchers designed
a job satisfaction tool which was comprised of demographic fill-in blanks, twenty-one 4-step
Likert questions, and four open-ended questions. The job satisfaction tool was mailed to all
participants at three, six, twelve, and eighteen months corresponding with an RN’s time on the
job.
These researchers found that the RNs’ perceptions of job satisfaction was significantly
higher in the post-internship group as compared to the pre-internship group (p = .046). Analysis
on longitudinal job satisfaction was statistically significant after eighteen months of employment
(p = .02) as compared to six months of employment. Voluntary turnover rate was calculated as
12% for the post-internship group as compared to 20% for the pre-internship group and was
sustained during the two-year post-intervention study period which reflects improved retention
of new graduate nurses.
4.3.2 WoE Medium Level Studies
Smith (2006) used a descriptive, correlational design to examine the influence of
mentoring on goal attainment and role satisfaction for new graduate RNs in acute care facilities.
Both non-probability and probability sampling were utilized to select the 45 participants who
took part in this study. The instrument used was a 77-item tool developed by Bouquillon to
41
explore mentoring antecedents and functions along with measuring career outcomes (goal
attainment) and job satisfaction. Bouquillon’s tool included 73 5-point Likert questions and four
open-ended questions.
This researcher analysed the data using Pearson’s r and found that there was a strong
correlation between mentoring and goal attainment (r = .80; p < .001). There was no statistically
significant relationship found between mentoring and role satisfaction (r = .27; p = .071),
however a statistically significant correlation between the presence of mentoring antecedents and
job satisfaction was determined (r = .345; p < .05). When Pearson’s r analysis was conducted on
those RNs who have high levels of goal attainment, there was a moderate correlation found
between mentoring and role satisfaction (r = .54; p < .05).
Fox (2010) used an evaluation survey design to describe a pilot mentorship program that
was implemented at three hospital campuses in Indiana. Twelve pairs of mentor-mentees took
part in the one-year program that required face-to-face meeting and completing evaluation forms
at seven different times. The turnover rate of first-year RNs at these hospitals was 32% in 2005
before the implementation of this mentorship program and had decreased to 10.3% by 2009 (a
21.5% decrease). The author reported this mentoring program an overwhelming success as the
retention rate of the pilot group of mentored nurses was 100%. Satisfaction scores improved by
one level (from agree to strongly agree or tend to disagree to agree) in 75% of the participants,
reflecting improved satisfaction. The mentee also reported feeling more comfortable in their
roles due to the support and resources provided by the mentors. Reported key elements of an
effective mentorship program include proper training for the mentor and mentee, regular face-to-
face meetings, selection process to ensure personality type matching, mentors with excellent
communication skills, and signing a contract.
42
Hardyman and Hickey (2001) used a longitudinal survey design to explore the
expectations and experiences of mentorship from the perspectives of newly graduated nurses.
The instrumentation used was a 3-item questionnaire addressing having a mentor, the length of
mentorship, and the content of mentorship. The questionnaire was developed with a pilot cohort
and 1512 newly graduated RNs took part in this study. Data was collected via the questionnaire
when the new graduate RNs became qualified and then again six months later.
These researchers utilized descriptive statistics and found 97% of participants wanted to
have a preceptor during their first nursing job with the preferred length of the mentorship being
six months (51%) followed by four or five months in length (25%). The aspects of mentorship
found to be most important to the participants include constructive feedback on clinical skills
(99%), teaching new clinical skills (99%), confidence building (95%), and helping to settle into
the work environment (94%). A reported key element of an effective mentorship program
includes the duration of the program being from four to six months in length.
4.4 Mixed-Methods Research Studies
Six of the research studies used quantitative and qualitative methods and were critically
appraised using the following tools:
1. GSRS Weight of Evidence (WoE) tool, found in Appendix B.
2. Maryland Scale of Scientific Methods (MSSM) tool, found in Appendix C.
3. Critical Appraisal Skills Programme (CASP) tool, found in Appendix D.
43
Table 4.4 Mixed-Methods Study Details
Study Author Study Approach WoE Level
& Score
MSSM
Level &
Score
CASP
Level &
Score
Marks-Maran, D., Ooms,
A., Tapping, J., Muir, J.,
Phillips, S., & Burke, L.
(2013)
Hunsberger, M.,
Baumann, A., & Crea-
Arsenio, M. (2013)
Beercroft, P.C., Santner,
S., Lacy, M.L., Kunzman,
L., & Dorey, F. (2006)
Haggerty, C., Holloway,
K., & Wilson, D. (2013)
Almada, P., Carafoli, K.,
Flattery, J.B., French,
D.A., & McNamara, M.
(2004)
Bialkowski, K. (2009)
Mixed-Methods
Mixed-Methods
Mixed-Methods
Mixed Methods
Mixed-Methods
Mixed-Methods
High 9
High 8
High 8
High 8
Medium 6
Medium 4
Low 1
N/A
Low 1
N/A
Low 1
Medium 2
High 8
High 9
High 8
Medium 7
Medium 6
Medium 7
Three mixed-methods research studies were scored as high WoE and CASP levels of evidence,
one as high WoE level of evidence and medium CASP level of evidence, and two as medium
WoE and CASP levels of evidence. One mixed-methods research study was scored as MSSM
medium level of evidence, three as MSSM low level of evidence, and two scored N/A which
means they only used descriptive statistics.
44
Figure 4.5 Mixed-Methods Critical Appraisal Scoring
Table 4.5 GSRS WoE Assessment Criteria Questions and Score
Study Author WoE A:
Methodological
Quality of
Study
WoE B: Relevance
of Research
Design for
Answering REA
Question
WoE C: Relevance
of Study Focus for
Answering REA
Question
WoE D:
Overall Score
Marks-Maran,
D., Ooms, A.,
Tapping, J.,
Muir, J., Phillips,
S., & Burke, L.
High High High High 9
Hunsberger, M.,
Baumann, A., &
Crea-Arsenio, M.
High Medium High High 8
Beercroft, P.C.,
Santner, S.,
Lacy, M.L.,
Kunzman, L., &
Dorey, F.
Medium High High High 8
Haggerty, C.,
Holloway, K., &
Wilson, D.
High High Medium High 8
Almada, P.,
Carafoli, K.,
Medium Medium Medium Medium 6
0 1 2 3 4 5 6 7 8 9
Sco
re
Study Author
Mixed-Methods Critical Apprasial Scoring
WoE Score
MSSM Score
CASP Score
45
Study Author WoE A:
Methodological
Quality of
Study
WoE B: Relevance
of Research
Design for
Answering REA
Question
WoE C: Relevance
of Study Focus for
Answering REA
Question
WoE D:
Overall Score
Flattery, J.B.,
French, D.A., &
McNamara, M.
Bialkowski, K.
Low Medium Low Medium 4
4.4.1 WoE and CASP High Level Studies
Marks-Maran, Ooms, Tapping, Muir, Phillips, and Burke (2013) used an evaluative
mixed-methods design to evaluate a mentorship program for newly graduated RNs to determine
mentee engagement with the program and the impact, value, and sustainability of the program.
A total of 44 new graduate RNs took part in this study and both qualitative and quantitative data
was collected through questionnaires, reflective journals, and personal audio recordings. The
questionnaire contained demographic questions, 52 4-point Likert questions, and three open-
ended questions.
These researchers performed quantitative data analysis utilizing descriptive statistics, t-
tests, and Cronbach’s alpha to measure reliability. Qualitative data analysis involved thematic
analysis using the Framework Method by Ritchie and Spencer and then mapping the emerged
themes against the findings of the quantitative data. Findings from this study included 78% of
the mentees reporting an improvement in confidence with making decisions about patient care,
70% reporting that the mentorship has enhanced their role satisfaction, 68% reporting improved
competence with drug administration and 75% reporting being able to deal more confidently
with problems relating to patient care. Reported key elements of an effective mentorship
program include the expertise of the mentors and the ability for choosing one’s own mentor.
46
Reported problems for a mentorship program include time commitment, conflicting shifts, not
understanding the purpose of the program, and mentor-mentee mismatch.
Hunsberger, Baumann, and Crea-Arsenio (2013) used a longitudinal trend mixed-
methods design to examine the impact of a government-supported extended orientation and
mentorship program intended to facilitate the transition of new graduate RNs to professional
practice. Quantitative data collection involved the use of an online survey asking questions
about demographics, employment, and mentorship while qualitative data collection included
semi-structured interviews and focus groups. Convenience sampling was utilized to select the
new graduate RNs who would take part in the interviews and focus groups, which took place via
phone.
These researchers performed data analysis through the use of descriptive statistics and
thematic analysis to code the interview responses into major themes and key findings. An
average of 82% of new graduate RNs reported the mentorship they received as excellent, very
good, or good and 90% gave the mentorship program a high rating for facilitating their transition
to nursing. The new graduate RNs reported mentorship helped them to become more confident
with documentation and medication administration and were also able to integrate into the
culture of the unit’s which results in increased job satisfaction.
Beercroft, Santner, Lacy, Kunzman, and Dorey (2006) used an evaluation mixed-methods
design to determine whether mentoring was successful and if new graduate RNs were
satisfactorily matched with their mentor, received guidance and support, attained socialization
into nursing, benefited from having a role model, maintained contact with their mentor, and were
satisfied with the mentorship. A 35-item survey was completed by 318 participants but this
study was only based upon eight questions pertaining to mentorship.
47
The researchers performed quantitative data analysis using descriptive statistics and
logistic regression analysis. Qualitative data analysis involved manifest content analysis to
identify themes important for successful mentorship. 44% of all positively coded comments
showed evidence of satisfaction that the new graduate RNs had with the mentorship. Reported
key elements of an effective mentorship program include regular face-to-face meetings, timing
for the starting of the relationship, adequate training for both participants, the dedication and
commitment to the relationship, and support from managers. Reported problems for a
mentorship program include lack of connection with mentor-mentee matching, not being able to
choose one’s mentor, not meeting regularly, role inadequacy, and time or schedule constraints.
Haggerty, Holloway, and Wilson (2010) used a longitudinal fourth generation evaluation
mixed-methods design that focused on mentorship support for new graduate RNs and the nurse
entry to practice program in New Zealand. Quantitative data collection involved a questionnaire
survey while qualitative data collection comprised in-depth focus groups and individual
interviews. A total of 1023 new graduate RNs participated in this study that took place over
three years.
The researchers identified the key issues relating to mentorship as: access to mentors,
how mentors met new graduate learning needs, the importance of the mentor-new graduate
relationship, mentor preparation for their role, culture of support, and development of confidence
and competence. Descriptive statistics were used and overall satisfaction from the new graduate
RNs in relation to the mentors meeting their expectations increased from 64% to 71% over the
three years of this study. Reported key elements of an effective mentorship program include
quality of mentor, mentor-mentee matching, access to mentors, and support of nursing
48
leadership. Reported problems for a mentorship program include workload and acuity levels of
the units, lack of mentor preparation, and having multiple mentors.
4.4.2 WoE and CASP Medium Level Studies
Almada, Carafoli, Flattery, French, and McNamara (2004) used a survey mixed-methods
design to determine if a newly designed mentor program provided new graduate RNs adequate
education, support, and acceptance in their new role thereby increasing retention rates.
Convenience sampling was utilized to select the 40 participants who took part in this study. A
coded survey tool with yes/no questions, visual analogue scales, and open-ended questions was
sent at completion of the preceptor program and three months after working independently. The
survey addressed satisfaction, reasons the new graduate RNs may have considered for leaving,
and feedback for program improvements.
The researchers performed quantitative data analysis and the findings indicated a high
level of satisfaction with the program (visual analogue mean score 93.7). The overall retention
rate at this hospital was increased by 29% (from 60% to 89%) and the hospital’s vacancy rate
was decreased 9.5% (down to 3%). The new graduate RNs reported an increase in their level of
comfort at completion of the program and a higher level of confidence and satisfaction was seen
as compared to previous new graduate RNs. Reported key elements of an effective mentorship
program include the length of the program and mentor-mentee match.
Bialowski (2009) used a survey mixed-methods design to evaluate a mentoring program
by examining the impact of length of orientation on job satisfaction, organizational commitment,
and propensity to leave. A total of twenty-one participants took part in this study and data
collection took place after three and nine months of employment. The instrumentation used
included an Organizational Commitment Questionnaire, a 15-item 7-point Likert scale; a
49
Propensity to Leave Questionnaire, a 3-item 5-point Likert scale; and the McCloskey/Mueller
Satisfaction Scale, a 31-item 5-point Likert scale questionnaire.
This researcher performed repeated measures analysis of variance (ANOVA-RM) and an
independent samples t-test which found no significant difference exists between three and nine
months of employment scores for organizational commitment, propensity to leave, and job
satisfaction (F = .15; p = .70). A series of one-way ANOVA was conducted to examine the
impact of length of orientation on organizational commitment and job satisfaction and a
significant difference was found between less than one month of orientation as compared to three
to six months. No significant difference was reported between propensity to leave and length of
orientation. Bivariate correlations reported a positive relationship between organizational
commitment and job satisfaction (r = .83; p = <.001), which indicates that RNs who are very
committed to their organizations are also highly satisfied with their jobs.
4.5 Qualitative Research Studies
Three of the research studies used qualitative methods and were critically appraised using
the following tools:
1. GSRS Weight of Evidence (WoE) tool, found in Appendix B.
2. Critical Appraisal Skills Programme (CASP) tool, found in Appendix D.
Table 4.6 Qualitative Study Details
Study Author Study Approach WoE Level
& Score
CASP
Level &
Score
Lewis, S., & McGowan, B.
(2015)
Navarro, J. (2009)
Wolak, E.S. (2007)
Qualitative
Descriptive Exploratory
Focus Group
Medium 6
Medium 5
Medium 5
High 9
High 9
High 9
50
All three qualitative research studies were scored as medium WoE level of evidence and high
CASP level of evidence.
Figure 4.6 Qualitative Critical Appraisal Scoring
Table 4.7 GSRS WoE Assessment Criteria Questions and Score
Study Author WoE A:
Methodological
Quality of Study
WoE B:
Relevance of
Research Design
for Answering
REA Question
WoE C:
Relevance of
Study Focus for
Answering REA
Question
WoE D: Overall
Score
Lewis, S., &
McGowan, B.
Medium Medium Medium Medium 6
Navarro, J.
Medium Medium Low Medium 5
Wolak, E.S.
Medium Medium Low Medium 5
4.5.1 WoE Medium and CASP High Level Studies
Lewis and McGowan (2015) used a qualitative design to examine and gain insight into
what the experience of mentorship was like for newly qualified RNs. Purposive recruitment was
used to select the eight participants who took part in one-hour semi-structured, one-to-one
0
1
2
3
4
5
6
7
8
9
Lewis et al. Navarro Wolak
Sco
re
Study Author
Qualitative Critical Apprasial Scoring
WoE Score
CASP Score
51
interviews. Data collection and analysis followed Newell and Burnard’s Pragmatic Approach to
Qualitative Data Analysis and two main categories emerged – support requirements and
expectations of mentorship. Support requirements was further organized into the themes of
development of knowledge, building confidence, and time management of the process.
Expectations of preceptorship was further organized into the themes of understanding the process
and understanding the preceptors’ roles.
Most of the participants reported that the mentorship enabled them to develop their
confidence and further develop their knowledge and skills. Reported key elements of an
effective mentorship program include working closely together and the availability of the
mentor. Reported problems for a mentorship program include lack of time, working different
shifts, unclear expectations of the process, and too much additional paperwork.
Navarro (2009) used a descriptive exploratory design to describe the mentorship
experiences of new graduate RNs as they transition from student to RN, how the mentorship
experience shaped the perceived self-efficacy of new graduate RNs, and how the mentorship
experience shaped the transition from student to RN. Purposive sampling was used to select the
five participants who took part in this study. One-hour semi-structured telephone interviews
were used for data collection and took place approximately six months post graduation. Data
analysis was completed using Hsieh and Shannon’s directed approach to content analyses and
several themes were identified for each research question. The identified themes include a
formal experience, forming new relationships, the experience of relational supports, the
experience of cultural supports, being encouraged, being challenged, being inspired, reframing,
seeking opportunities, a commitment to goals, and discovering me.
52
All the participants in this study reported their mentors using positive verbal persuasion
and encouragement which resulted in increased confidence. The mentors also challenged the
new graduate RNs to perform new skills and resulted in an increased clinical competence and the
confidence to perform skills independently. The majority of the participants reported the
mentorship experience allowed for the development of their professional identity which
translated into building their sense of confidence and self-worth. Reported problems for a
mentorship program include the inability to choose one’s mentor and the lack of adequate mentor
training.
Wolak (2007) used a focus group design to examine the experiences of mentees and
mentors in a structured mentorship program. Non-random purposive sampling was utilized to
select the five mentees and six mentors who took part in this study. The mentor-mentee pairs
had been paired together at least ten months and were required to meet outside of work once a
month. Separate focus groups lasting 30 minutes were held with all the mentors and all the
mentees for data collection. Data was analyzed for specific themes using long-table
methodological analysis and the following themes emerged – availability, sense of community,
and support and knowledge.
All the participants reported that the mentors provided the mentees with valuable
knowledge and skills thus contributing to increased job satisfaction and organizational
commitment. Reported key elements of an effective mentorship program include similar work
schedules for the mentor and mentee and the mentors being accessible for questions and clinical
support. Reported problems for a mentorship program include having to meet monthly outside
of work and lack of enthusiasm from the mentor.
53
Figure 4.7 Overall WoE Level of Evidence
Figure 4.8 Overall CASP Level of Evidence
0
1
2
3
4
5
6
Quantitative Mixed Methods Qualitative
Nu
mb
er
of
Stu
die
s
Research Study Design
Overall WoE Level of Evidence
WoE High
WoE Med
0
1
2
3
4
Mixed Methods Qualitative
Nu
mb
er
of
Stu
die
s
Research Study Design
Overall CASP Level of Evidence
CASP High
CASP Med
54
Figure 4.9 Overall MSSM Level of Evidence
4.6 Chapter Summary
This chapter provided an overview of the seventeen research studies along with their
critical appraisal scoring and findings. Each research study had the data extracted using the
EPPI-Centre Data Extraction and Coding Tool for Education Studies, found in Appendix E. The
research studies were grouped according to their approach – quantitative, mixed-methods, or
qualitative and listed from highest level of evidence to lowest level of evidence, according to
their WoE level and score. Eight quantitative research studies were critically appraised using the
GSRS WoE tool, found in Appendix B and the MSSM tool, found in Appendix C. Six research
studies utilizing both qualitative and quantitative methods were critically appraised using the
GSRS WoE tool, found in Appendix B; CASP tool, found in Appendix D; and the MSSM tool,
found in Appendix C. Three qualitative research studies were critically appraised using the
GSRS WoE tool, found in Appendix B and CASP tool, found in Appendix D. In the next
chapter, the findings from the seventeen research studies are synthesized to answer the research
0
1
2
3
4
5
Quantitative Mixed Methods
Nu
mb
er
of
Stu
die
s
Research Study Design
Overall MSSM Level of Evidence
MSSM Med
MSSM Low
MSSM N/A
55
question and sub-questions presented in chapter one. As well, a discussion about the limitations
of this REA will be presented.
56
CHAPTER 5: Discussion
In this chapter the findings from the final seventeen research studies are synthesized to
explore the value of mentorship programs in more detail in the context of the literature
examined, and current practice. The reported key elements of an effective mentorship program
and problems in implementing a mentorship program will also be discussed. As well, this
chapter will end with a discussion of the limitations of this REA.
5.1 Synthesis of Findings
Eight research studies utilizing quantitative methods, six utilizing mixed-methods, and
three utilizing qualitative methods were included in this REA and their results synthesized to
answer the following research question: What is the evidence that new graduate mentorship
programs are effective in improving performance, satisfaction, retention, and confidence in
novice nurses’ practice?
5.1.1 Improving Performance
Three high WoE level studies (Hale, 2004; Komaratat & Oumtanee, 2009; Marks-Maran
et al., 2013) and four medium WoE level studies (Hardyman & Hickey, 2001; Lewis &
McGowan, 2015; Navarro, 2009; Wolak, 2007) suggested that the use of a mentorship program
improved the performance of novice nurses’ practice. Hale explored mentorship relationships
from the perspective of new graduate RN’s in their first year of licensure. The findings of this
study suggested that new graduate RN’s reported an improvement in their competence as a
nurse, as a result of being in a mentorship relationship. Komaratat and Oumtanee studied the
level of nursing competency of newly graduated RN’s after using a mentorship model. Their
findings indicated that a mentorship model improved the performance of newly graduated RN’s
in nursing skills, decision-making, and problem resolution. Marks-Maran et al. evaluated a
57
mentorship program for newly graduated RN’s and their findings suggested that new graduate
RN’s who took part in a mentorship program reported improved performance by increased
clinical competence with drug administration, meeting the nutritional needs of patients, wound
management, and other health and safety issues.
Hardyman and Hickey explored the expectations and experiences of mentorship from the
perspectives of newly graduated RN’s. This study reported that teaching new clinical skills and
help with settling into the work environment were important aspects of a mentorship program.
Lewis & McGowan sought to examine and gain insight into what the experience of mentorship
was like for newly qualified RN’s. Their findings suggested that a mentorship program enabled
the new graduate RN’s to further develop their knowledge and skills, which translated into
improved performance. Navarro described the mentorship experiences of new graduate RN’s as
they transition from student to RN. The findings of this study suggested that mentors
challenged the new graduate RN’s to perform new skills which resulted in increased clinical
competence. Wolak examined the experiences of mentees and mentors in a structured intensive
care unit mentorship program. This study reported that the mentors provided the mentees with
valuable knowledge and skills which suggested improved new graduate RN performance.
Several research studies offered similarities with how they measured improved
performance as a result of a mentorship program. Komaratat and Oumtanee scored nursing
competency based upon nursing care, human relationship and communication, decision-making
and problem-solving, and quality development and assurance while Marks-Maran et al.
measured increased competence with nursing skills development, problem-solving related to
patient care, and the positive impact on developing high standards of practice. As well,
Navarro’s findings focused on performance with the development of clinical skills and
58
knowledge. In contrast, Hale measured performance through emotional functions such as
communication, encouragement, and support; social functions such as advocacy and
socialization; and professional role functions such as guidance, intellectual stimulation, and
career direction.
Bandura’s (1977) social learning theory seems to support the improvement of
performance of novice nurses’ practice with the social learning engendered in mentorship.
Bandura suggested that individuals learn by observing, imitating, and modeling other’s
behaviors, attitudes, experiences, and consequences of behaviors. The mentor is role modelling
for the mentee, who observes and then models their observed behavior at a later time. The use of
a reflective diary to record the new graduate RN’s thoughts, feelings, and experiences also
supports social learning, and enhanced performance. The new graduate RN can use these diary
entries to reflect on their practice and develop new ways to improve their performance,
especially with performing skills. Having the mentor provide the mentee with feedback is very
important during the reproduction stage of Bandura’s social learning theory and it is suggested
that this can assist the new graduate RN with building their perception of self-efficacy. An
increased perception of self-efficacy can result in improved performance of the new graduate
RN, thus it is important for the mentor to help develop during their mentorship relationship.
5.1.2 Improving Satisfaction
Six high WoE level studies (Beercroft et al., 2006; Haggerty, Holloway, & Wilson, 2013;
Hale, 2004; Halfer, Graf, & Sullivan, 2008; Hunsberger, Baumann, & Crea-Arsenio, 2013;
Lindsey, 2000) and three medium WoE level studies (Almada et al., 2004; Bialowski, 2009;
Smith, 2006) suggested that the use of a mentorship program improved the satisfaction of novice
nurses. Beercroft et al. sought to determine whether mentoring was successful and if new
59
graduates were satisfactorily matched with a mentor; received guidance and support; attained
socialization into the nursing profession; benefitted from having a role model for acquisition of
professional behaviours, maintained contact with their mentor throughout the program; and were
satisfied with the mentorship. Their findings suggested that a mentorship relationship improved
satisfaction for some of the new graduate RN’s as indicated by appreciation, excellence, or
benefit. Haggerty, Holloway, and Wilson provided an overview of the nurse entry to practice
programs within New Zealand that focused on mentorship support for new graduate RN’s. They
suggested that the increased confidence and competence of the new graduate RN’s due to the
mentorship program was linked to increased job satisfaction, which was also reported by
participants as a result of the program. Hale’s findings demonstrated that new graduate RN’s
experienced increased job satisfaction due to being involved in a mentorship relationship.
Halfer, Graf, and Sullivan studied Pediatric new graduate RNs who took part in a
Pediatric RN Internship Program to compare their job satisfaction and retention rates. The
reported overall job satisfaction was significantly higher in the post-internship group as
compared to the pre-internship group. Hunsberger, Baumann, and Crea-Arsenio examined the
impact of a government supported extended orientation and mentorship program on the transition
of new graduate RN’s to their professional role. New graduate RN’s were provided with a
“robust opportunity to integrate into the culture of the unit” (Hunsberger, Baumann, & Crea-
Arsenio, p. 82) which resulted in increased satisfaction with their positions. It was reported that
mentorship enabled them to feel part of the team and created a supportive environment that
improved satisfaction. Lindsey investigated novice RN’s in their first two years of practice to
determine whether mentored RN’s versus non-mentored RN’s had greater job satisfaction levels.
60
Her findings reported a statistically significant result with mentored new graduate RN’s having a
higher job satisfaction than non-mentored new graduate RN’s.
Almada et al. reported the findings of a study which investigated a community hospital’s
implementation of an education-based mentorship program. Their study indicated a high
satisfaction rate with the experience of a mentorship program and a higher level of satisfaction
was seen in the mentored new graduate RN’s as compared to non-mentored new graduate RN’s.
Bialowski sought to evaluate a mentorship program, the Vermont Nurse Internship Program, by
examining the impact of length of orientation on job satisfaction, organizational commitment,
and propensity to leave. Her findings suggested an overall job satisfaction that remained
constant over the nine month period of the study. Smith examined the influence of mentoring on
goal attainment and role satisfaction for RN’s who were employed in acute care facilities for less
than two years. Her findings suggested that there was a moderate relationship between
mentoring and role satisfaction in those RN’s who scored above the mean for the goal attainment
concept. She also identified a significant correlation between the presence of mentoring
antecedents, such as integrity, trust, willingness to engage and accept, and acknowledgment of
professional responsibility, and role satisfaction.
Some of the above research studies offered differences with the length of mentorship
program that improved satisfaction was based upon. Hunsberger, Baumann, and Crea-Arsenio’s
study participants took part in a three to six month supernumerary mentorship program whereas
Bialowski’s mentorship program length varied from one month up to seven months. As well, in
Lindsey’s study the average length of mentoring relationship was eleven months. In comparison,
Almada et al. and Haggerty, Holloway, and Wilson’s study participants took part in eight week
and six week supernumerary mentorship programs before job satisfaction was measured.
61
Improving satisfaction of a novice nurses’ practice is also supported by Bandura’s (1977)
social learning theory. A key attribute of social learning theory is that a supportive environment
is required to be successful. A supportive environment may be the result of a mentorship
relationship which hopefully will increase the new graduate RN’s satisfaction with their job.
Vicarious reinforcement is very important during the motivational phase of Bandura’s social
learning theory and can influence the new graduate RN’s learning. If the new graduate RN
observes their mentor having positive interactions with other coworkers or doctors, this can serve
to influence their perceived satisfaction with their job and nursing unit.
5.1.3 Improving Retention
Two high WoE level studies (Grindel & Hagerstrom, 2009; Halfer, Graf, & Sullivan,
2008) and two medium WoE level studies (Almada et al., 2004; Fox, 2010) suggested that the
use of a mentorship program improved the retention of novice nurses. Grindel and Hagerstrom
reported the outcomes and lessons learned from a hospital-based formal mentorship program
which they called, Nurses Nurturing Nurses, in which mentor and mentees worked together for
twelve months to facilitate new RN transition. Retention was measured at three, six, and twelve
months and all participant scores were moderately high throughout the first six months and then
rose at twelve months, suggesting that most respondents were going to remain in their current
positions. Halfer, Graf, and Sullivan’s findings indicated that the new graduate RN’s who took
part in the mentorship program had a lower turnover rate that was sustained during the two year
post-intervention period.
Almada et al. suggested that an overall increase in retention of new graduate RN’s during
a fourteen month period was the direct result of their mentorship program. Their findings also
reported that the hospital’s vacancy rate was decreased due to the retention of new graduate
62
RN’s. Fox reported the findings of a one year pilot mentorship program that sought to improve
retention and turnover rates of newly graduated RN’s. Her study suggested that the mentorship
program was successful in improving the retention of new graduate RN’s due to the support and
resources that the mentors provided. Fox also reported a dramatic decrease in the turnover rate
with the initiation of the mentorship program.
Fox and Grindel and Hagerstrom offered similarities with how they measured improved
retention as a result of a mentorship program. Fox measured the turnover rate of novice nurses
after twelve months of starting a mentorship program whereas Grindel and Hagerstrom measured
intent to stay at three times over a twelve month period. In contrast, Almada et al. measured
retention approximately six months from starting a mentorship program while Halfer, Graf, and
Sullivan measured retention at four times over an eighteen month period.
Bandura’s (1977) social learning theory would seem to offer a supporting framework to
explain the improved retention of novice nurses’ in practice through a social supportive
environment. New graduate RN’s may not want to stay working on a nursing unit that is not
supportive when their mentorship program is completed. Vicarious reinforcement, the result of
learning by observing the mentor’s successes and failures, may also influence the retention of
new graduate RN’s.
5.1.4 Improving Confidence
Five high WoE level studies (Grindel & Hagerstrom, 2009; Hale, 2004; Haggerty,
Holloway, & Wilson, 2013; Hunsberger, Baumann, & Crea-Arsenio, 2013; Marks-Maran et al.,
2013) and four medium WoE level studies (Almada et al., 2004; Hardyman & Hickey, 2001;
Lewis & McGowan, 2015; Navarro, 2009) suggested that the use of a mentorship program
improved the confidence of a novice nurses’ practice. Grindel and Hagerstrom reported a
63
significant increase in RN confidence scores, particularly between initiation of the mentorship
program and three months. Hale’s findings reported new graduate RN’s increased confidence as
a result of the mentorship relationship. Moreover, Haggerty, Holloway, and Wilson’s study
suggested that the structured, supportive environment that the mentorship program provided
resulted in increased confidence of the new graduate RNs’ and Hunsberger, Baumann, and Crea-
Arsenio’s findings indicated that mentorship increased new graduate RN’s confidence which
allowed them to make decisions in a safe, protected environment. The new graduate RN’s also
reported that they became more confident about documentation as well with administering
medication according to patients’ preferences. Increased confidence was further reported in
Marks-Maran et al.’s study as one sub-theme from their qualitative data. Furthermore, their
findings indicated for a majority of new graduate RN’s a structured mentorship program had the
potential to build confidence and enabled the new graduates to achieve a level of confidence that
they did not have upon graduating.
Almada et al.’s study had new graduate RN’s reporting an increased level of comfort and
a higher level of confidence when released from their mentorship program. Hardyman and
Hickey identified confidence building as an important aspect of a mentorship program by the
majority of their new graduate RN participants. Lewis and McGowan’s findings suggest that the
majority of new graduate RNs expressed the view that mentorship enabled them to develop their
confidence and further develop their knowledge and skills in the transition from student to newly
qualified RN. Navarro’s study participants all reported that their mentor’s use of verbal
persuasion and encouragement helped foster their confidence and strengthen their belief to
succeed. Increased confidence due to the presence of the mentorship relationship was also
reported when managing stressful situations and communicating effectively.
64
There were a few differences in how the research studies measured improved confidence
as a result of a mentorship program. Navarro measured increased confidence when dealing with
stressful situations and being able to effectively communicate while Grindel and Hagerstrom
measured increased confidence with routine nursing activities such as providing patient care,
interpreting lab tests, or delegating tasks to other nursing staff. When looking at similarities,
Hunsberger, Baumann, and Crea-Arsenio measured increased confidence with decision-making
and critically thinking whereas Lewis and McGowan measured increased confidence with
nursing knowledge and skills.
Bandura’s (1977) social learning theory appears to support improved confidence of a
novice nurses’ practice. Bandura suggests that having feedback provided by the mentor can
assist the new graduate RN with building their perception of self-efficacy, which he defines as
confidence in one’s ability to take action and persist in action. Increased self-efficacy can
translate into increased confidence of the new graduate RN, which may be attributed to the
mentorship relationship when applying the concept of social learning theory. The use of a
learning contract can be important during the motivational phase of Bandura’s social learning
theory and may result in increased new graduate RN confidence. Through the use of a learning
contract, mentors can encourage individualized learning to meet the needs of the new graduate
RN and increase their sense of achievement, which in turn increases their confidence.
5.2 Key Elements of Effective New Graduate RN Mentorship Programs
What are the reported key elements of effective new graduate RN mentorship programs?
5.2.1 Mentor-Mentee Matching
Mentor-mentee matching was a key element of an effective mentorship program reported
in three high WoE level studies (Beercroft et al., 2006; Haggerty, Holloway, & Wilson, 2013;
Marks-Maran et al., 2013) and two medium WoE level studies (Almada et al., 2004; Fox, 2010).
65
In the Beercroft et al. study, having a strong connection or ‘clicking’ with one’s mentor was an
indication for satisfaction with the mentoring relationship. Some mentees indicated that being
able to choose their own mentor would be preferable to ensure connection. Participants in
Haggerty, Holloway, and Wilson’s study identified the matching of mentors and mentees as key
to successful new graduate programs despite noting that it is an area that requires further
strengthening. Marks-Maran et al. reported that 70% of mentees would prefer to choose their
own mentors. It was suggested that many mentees liked the facilitation style offered by their
mentors who made them feel comfortable sharing problems and helped them become introduced
to their new role.
Participants in the study by Almada et al. ranked mentor-mentee matching as one of the
most important aspects of a mentorship program. All mentors were required to attend a full-day
program to increase their understanding of learning styles, communication techniques,
personality traits, and conflict resolution before being matched with their mentee. Fox suggested
the selection process for mentors and mentees was one of the reasons for the success of their
mentorship program. Mentors and mentees were both matched according to their personality
types, educational degrees, and taking into account the shifts that each individual worked,
preferring to pair those that worked similar rotations.
5.2.2 Availability
Availability of mentors was a key element of an effective mentorship program reported in
two high WoE level studies (Beercroft et al., 2006; Haggerty, Holloway, & Wilson, 2013) and
two medium WoE level studies (Fox, 2010; Wolak, 2007). Beercroft et al.’s findings reported
that meeting on a regular basis had the most impact on the success of the mentorship
relationship. For those mentees meeting on a regular basis, the majority reported that their
66
mentor provided guidance and feedback while being a stress reducer. Haggerty, Holloway, and
Wilson findings suggested that when mentors were available, new graduate RN’s reported high
levels of satisfaction with meeting their learning needs. Creative solutions were required to
ensure availability of mentors such as the allocation of primary and secondary mentors or
multiple mentors job sharing the role, establishing a dedicated mentor role on a unit, or having
the mentee assigned to all the mentor’s shifts.
Participants in Fox’s study were required to sign a contract to ensure they understood the
expectation of meeting regularly. Mentor-mentee pairs met a minimum of seven times during
the twelve month program and all meetings needed to be face-to-face, which were reported a
successes of their mentorship program. Wolak’s findings identified availability as a major theme
with the context centered on being accessible for questions and clinical support. Mentees
reported that the ability to access mentors as their learning needs required rather than just during
planned meetings or while working together as instrumental to their nursing practice. The
individualized attention of the mentor helped create an environment of support and trust. It was
also suggested that mentees and mentors work similar schedules for the duration of the
mentorship relationship.
5.2.3 Training
Adequate training and preparation of mentors was a key element of an effective
mentorship program reported in two high WoE level studies (Haggerty, Holloway, & Wilson,
2013; Marks-Maran et al., 2013) and two medium WoE level studies (Fox, 2010; Navarro,
2009). Haggerty, Holloway, and Wilson identified the quality of mentors as a key element and
suggested that effective mentors need to demonstrate high levels of interpersonal and facilitation
skills. These authors also stated that the preparation of mentors was important to ensure a
67
professional relationship that supported and challenged new graduates. In another study, with a
similar focus participants indicated that the expertise of their mentor was of particular value and
an important consideration for selection (Marks-Maran et al.).
Furthermore, Fox indicated proper training was a crucial element of the success of the
initiation of her mentor program. All mentors and mentees took part in a required training
program together as an introduction and bonding process in which a structured agenda covered
key topics relating to mentorship. Navarro reported the importance of adequate training and
preparing mentors to provide better learning experiences for new graduate nurses transitioning
into practice. She proposed formal orientation or training programs providing education
regarding the impact and value of the mentor role and effective teaching, learning, and
communication strategies.
5.2.4 Commitment and Support
Commitment and support was a key element of an effective mentorship program reported
in two high WoE level studies (Beercroft et al., 2006; Haggerty, Holloway, & Wilson, 2013).
Findings from the Beercroft et al. study suggested that an important role and benefit of a mentor
is the support and guidance they offer to the mentee. To ensure success, mentors need to be
dedicated and committed to spending the time required to cultivate the mentorship relationship.
They also reported that support from mangers to allow off unit times for face-to-face meetings
was a key element for success. Haggerty, Holloway, and Wilson identified a culture of support
from nursing staff, especially nursing leaders, as a key component for a successful mentorship
program. Nurse leaders were reported as having a responsibility to create an environment of
support for the new graduate nurse and their mentor by promoting creative solutions, inspiring
colleagues, and allocating appropriate resources. This is supported by Bandura’s (1977) social
68
learning theory which suggests that a supportive environment is a key component to having a
successful relationship.
5.2.5 Length of Mentoring Relationship
Length of the mentoring relationship was a key element of an effective mentorship
program reported in three medium WoE level studies (Almada et al., 2004; Bialowski, 2009;
Hardyman & Hickey, 2001). Participants in the study by Almada et al. ranked length of time,
one of the most important aspects of a mentorship program. Their program included a minimum
of eleven weeks of mentorship with extensions available if deemed necessary. Bialowski’s
findings indicated that the new graduate RN’s who received three to six months of mentorship
were significantly more committed to their organizations and more satisfied with their jobs’ thus
she suggested six months as the ideal length of a mentorship program. Participants in the study
by Hardyman and Hickey reported that their preferred length of mentorship would be six months
with the second choice of four to five months in length.
5.3 Reported Problems in Implementing New Graduate RN Mentorship Programs
What are the reported problems in implementing new graduate RN mentorship programs?
5.3.1 Training
Lack of training and preparation of mentors and mentees was a problem reported in three
high WoE level studies (Beercroft et al., 2006; Haggerty, Holloway, & Wilson, 2013; Marks-
Maran et al., 2013) and two medium WoE studies (Lewis & McGowan, 2015; Navarro, 2009).
Beercroft et al. cited role inadequacy of both the mentor and mentee, due to inadequate training,
as a problem with mentorship programs. Their study also suggested that a number of mentees
had a narrow view of mentoring and what could be gained from the relationship, and support was
the only expectation from a mentoring relationship. Haggerty, Holloway, and Wilson report that
69
many mentors are poorly prepared and were unable to attend mentor training due to workload
and acuity of their nursing unit. They also found that new graduate RN’s reported less
satisfaction with the experience when having unprepared mentors despite having adequate access
to them. Marks-Maran et al. reported different understandings about the nature of mentorship
which may have contributed to the lack of success of the relationship.
Lewis and McGowan referred to the lack of understanding of what the process of
mentorship involved and what exactly were the mentor and mentee roles. Some mentees
assumed they would be working alongside their mentors or at least very closely together rather
than working alone with support being offered from a distance. It was also reported by some
mentors that not having a clear understanding of the documentation required for mentorship
became burdensome. Lack of mentor development and training was found to be a barrier to
mentorship programs (Navarro), which was reported by some study participants as negatively
influencing their mentorship experience.
5.3.2 Availability
Availability of mentors was a problem reported in three high WoE level studies
(Beercroft et al., 2006; Haggerty, Holloway, & Wilson, 2013; Marks-Maran et al., 2013) and two
medium WoE level studies (Lewis & McGowan, 2015; Wolak, 2007). Beercroft et al.’s study
participants reported time and schedule constraints as reasons for not fulfilling their
responsibilities, which resulted in an unsuccessful mentorship relationship. As well, some
mentors did not want to participate in the program so their dedication and commitment to the
relationship was lacking. Haggerty, Holloway, and Wilson identified the allocation of mentors
as not always appropriate. This included mentors going on planned leave at critical times,
mentors taking unplanned leave, and assigned mentors who worked part-time with no
70
replacement allocated. Furthermore, making time for meetings between the mentor and mentee
due to time constraints, conflicting shift patterns, and prioritization of the mentorship meetings
was reported as a challenge (Marks-Maran et al.).
Lewis and McGowan reported the theme of ‘time to do’ which was described by the
mentees as finding it difficult to get time together with their mentors. This was due to busy or
short-staffed wards, high patient acuity, and working different shifts. Wolak’s participants
identified the requirement of monthly meetings outside work to be inconvenient and a potential
hindrance on the relationship. Both mentors and mentees felt that impromptu meetings and
discussions in the work setting were beneficial and potentially superior to outside of work
meetings.
5.3.3 Mentor-Mentee Mismatch
Mentor-mentee mismatch was a problem reported in one high WoE level study (Marks-
Maran et al., 2013) and one medium WoE level study (Navarro, 2009). Marks-Maran et al.’s
findings suggested that some study participants did not have a valuable mentorship experience
due to personality clashes and relationship issues. Seventy percent of new graduate nurses felt
that they should be able to choose their own mentor rather than one being assigned. Navarro’s
findings suggested that some mentors lacked interest in the mentees transition and learning
experience which was discouraging and hindered the mentorship relationship. This was possibly
due to mentors being assigned by default rather than volunteering for the role. The inability to
choose your own mentor was identified as a negative by some participants and resulted in feeling
disappointed and undervalued. This is supported by Bandura’s (1977) social learning theory
which suggests that a new graduate RN may find it challenging to think of their mentor as a role
model when the mentoring relationship is arranged.
71
5.4 Limitations of this REA
Several limitations were identified in the development of this REA. The author decided
to exclude ‘grey’ literature and only searched published material written in the English language
for a period of three weeks, which may have led to a publication bias. As well, due to time
constraints, the searching for literature in this REA only included electronic sources and hand
searching of print sources from the references of relevant research studies, which again may have
led to a publication bias. Completing a full systematic review on the topic of mentorship in
novice nurses would help address this limitation but was beyond the scope of this project.
As well, all seventeen research studies were critically appraised and scored by the author
of this REA, which is a potential selection limitation due to the author’s inexperience with
research and all seventeen research studies were only selected by this author and no experts
involved, which may have resulted in a selection bias. This limitation again could be avoided by
completing a full systematic review.
Lastly, this REA only included research studies utilizing quantitative, qualitative, or
mixed-methods research, selected to ensure the included research studies provide a high level of
evidence to answer the REA question and sub-questions but this can also be a limitation. There
may be reported key elements of an effective mentorship program and problems in implementing
a mentorship program that have only been reported in opinion papers and those results will not
be captured in this REA.
5.5 Chapter Summary
This chapter presented the synthesis of the final seventeen research studies to provide
evidence that mentorship programs are effective in improving performance, satisfaction,
retention, and confidence in new graduate RNs’ practice. These findings were also linked to the
conceptual framework identified in chapter two. The reported key elements of an effective
72
mentorship program and problems in implementing a mentorship program were also discussed as
well as any limitations that were associated with writing this REA. The next chapter will
summarize conclusions from this REA and offer recommendations for future research.
73
CHAPTER 6: Conclusion
The findings of this REA reflect a synthesis of knowledge that mentorship programs are
effective in improving performance, satisfaction, retention, and confidence in novice nurses’
practice. The focus of this REA was to address the following research question:
What is the evidence that new graduate mentorship programs are effective in improving
performance, satisfaction, retention, and confidence in novice nurses’ practice?
This REA also attended to the following sub-questions:
What are the reported key elements of effective new graduate RN mentorship programs?
What are the reported problems in implementing new graduate RN mentorship programs?
6.1 REA Summary
A total of 66 research studies were initially selected and their abstracts reviewed to screen
for meeting the inclusion criteria and answering the above research question and sub-questions.
Twenty-one research studies were then selected to be included in this REA but upon further
review four had to be excluded due to their definitions of a new graduate RN not meeting the
inclusion criteria leaving a final total of seventeen research studies. Eight research studies
utilizing quantitative methods, six utilizing mixed-methods, and three utilizing qualitative
methods were included in this REA. The data from each research study was extracted using the
EPPI-Centre Data Extraction and Coding Tool for Education Studies (see Appendix E). Each
research study was then scored using the tools described in chapter three from highest level of
evidence to lowest level of evidence, according to their WoE level, MSSM and CASP scores as
appropriate for the type of study.
The findings from the final seventeen research studies were synthesized and suggest that
mentorship programs can be effective in improving performance, satisfaction, retention, and
74
confidence in novice nurses’ practice under the right conditions. Of the seventeen research
articles included in this REA, none of them examined mentorship programs and its effect on all
four aspects (performance, satisfaction, retention, and confidence) of a novice nurses’ practice.
This means that the positive conclusions described below were drawn from examining
performance, satisfaction, retention, and confidence in isolation, thus suggesting an overall
positive effect of mentorship programs on a novice nurses’ practice.
6.1.1 Performance
The findings from this REA suggest that use of a mentorship program improved the
performance of novice nurses’ practice as shown by three high WoE level studies (Hale, 2004;
Komaratat & Oumtanee, 2009; Marks-Maran et al., 2013) and four medium WoE level studies
(Hardyman & Hickey, 2001; Lewis & McGowan, 2015; Navarro, 2009; Wolak, 2007).
Although only seven of the seventeen included research studies examined the effects of a
mentorship program on performance, all seven reported positive results, therefore suggesting a
positive impact despite the evidence being marginal. It was suggested that a mentorship model
improved the performance of newly graduated RN’s with nursing skills, decision-making, and
problem resolution (Komaratat & Oumtanee) as well as with increased clinical competence with
drug administration, meeting the nutritional needs of patients, wound management, and other
health and safety issues (Marks-Maran et al.).
The application of a mentorship program on performance in the practice context could
involve the mentor helping the new graduate RN with developing a learning plan such as a
Competency, Assessment, Planning, and Evaluation (CAPE) tool. A CAPE tool can assist a new
graduate RN to meet their personal learning needs by identifying expectations which can lead to
increased performance. The mentor can also help the new graduate RN with learning nursing
75
skills through observation, role modelling, and reflection which relates to Bandura’s social
learning theory.
6.1.2 Satisfaction
The findings from this REA suggest that the use of a mentorship program improved the
satisfaction of a novice nurses’ practice as presented by six high WoE level studies (Beercroft et
al., 2006; Haggerty, Holloway, & Wilson, 2013; Hale, 2004; Halfer, Graf, & Sullivan, 2008;
Hunsberger, Baumann, & Crea-Arsenio, 2013; Lindsey, 2000) and three medium WoE level
studies (Almada et al., 2004; Bialowski, 2009; Smith, 2006). The effect of a mentorship
program on satisfaction was only examined in the above nine research studies, consequently
suggesting a positive result where satisfaction was examined. It was reported that the increased
confidence and competence of new graduate RN’s was due to participating in a mentorship
program and this was linked to increased job satisfaction (Haggerty, Holloway, & Wilson).
Furthermore, mentorship enabled novice nurses to feel part of the team and created a supportive
environment that improved satisfaction (Hunsberger, Baumann, & Crea-Arsenio).
In the practice context, the application of a mentorship program on satisfaction could
comprise having clear expectations identified at the beginning of the relationship so both the
mentor and new graduate RN are aware of their roles. This may involve the development of a
learning plan, the completion of a learning contract, or establishing explicit learning goals. It
would also be beneficial for the mentor to have an understanding of adult learning theory and
different styles of learning so they best support and meet the individual learning needs of the new
graduate RN.
76
6.1.3 Retention
The findings of this REA suggest that the use of a mentorship program improved the
retention of novice nurses as shown by two high WoE level studies (Grindel & Hagerstrom,
2009; Halfer, Graf, & Sullivan, 2008) and two medium WoE level studies (Almada et al., 2004;
Fox, 2010). Although there were only four research studies that provided evidence to support
this statement, they were the only research studies from the included seventeen that addressed
this aspect, thus where retention was examined the results seem to be positive even though the
evidence appears to be marginal. It was suggested that a mentorship program was successful in
improving the retention of new graduate RN’s due to the support and resources that the mentors
provided (Fox). Additionally, new graduate RN’s who took part in a mentorship program had
lower turnover rates hence improved retention (Halfer, Graf, & Sullivan).
The application of a mentorship program on retention in the practice context could
consist of a mentor helping the new graduate RN to assimilate into the work culture and develop
a sense of belonging. This may include introducing the new graduate RN to other team members
both on the unit and throughout the hospital, providing a guided tour of the hospital to help the
new graduate RN to become more familiar with their surroundings, and assisting the new
graduate RN to adjust to a new learning environment with factors such as unit culture,
management style, and workload issues.
6.1.4 Confidence
The findings of this REA suggest that the use of a mentorship program improved the
confidence of novice nurses’ practice as presented by five high WoE level studies (Grindel &
Hagerstrom, 2009; Hale, 2004; Haggerty, Holloway, & Wilson, 2013; Hunsberger, Baumann, &
Crea-Arsenio, 2013; Marks-Maran et al., 2013) and four medium WoE level studies (Almada et
77
al., 2004; Hardyman & Hickey, 2001; Lewis & McGowan, 2015; Navarro, 2009). The effect of
a mentorship program on confidence was only examined in nine of the seventeen included
research articles but where it was examined the results appear positive, therefore indicating a
positive effect on confidence. It was suggested that the structured, supportive environment that a
mentorship program provided resulted in increased confidence of the new graduate RNs’
(Haggerty, Holloway, & Wilson). Moreover, a majority of new graduate RN’s in a structured
mentorship program had the potential to build confidence and enabled the new graduates to
achieve a level of confidence that they did not have upon graduating (Marks-Maran et al.) as
well as mentorship increased new graduate RN’s confidence which allowed them to make
decisions in a safe, protected environment (Hunsberger, Baumann, & Crea-Arsenio).
In the practice context, the application of a mentorship program on confidence could
include the mentor assisting the new graduate RN to perform nursing tasks and skills that they
are already competent with independently and providing supervision with more challenging
activities. The mentor can provide support through frequent constructive and supportive
feedback and also encouraging the new graduate RN to self-reflect which may enhance their
confidence. Starting the mentorship program with adequate orientation that incorporates the
objectives, expected outcomes, and clearly identified roles could also assist with confidence.
6.1.5 Key Elements for Effective Programs
The reported key elements of effective new graduate RN mentorship programs include
mentor-mentee matching, availability of mentors, adequate training and preparation of mentors,
commitment and support, and length of mentoring relationship. Mentor-mentee matching
included having a strong connection, the ability to select one’s mentor, and working similar
rotations. Availability of mentors included meeting on a regular basis, allocation of primary and
78
secondary mentors, and working similar schedules. Adequate training and preparation of
mentors included high levels of interpersonal and facilitation skills and formal orientation or
training programs. Commitment and support included support from mangers to allow off unit
times for face-to-face meetings and a culture of support from nursing staff, especially nursing
leaders. Length of mentoring relationship suggested six months as the ideal length of a
mentorship program.
6.1.6 Reported Problems
The reported problems in implementing new graduate RN mentorship programs include
lack of training and preparation of mentors and mentees, availability of mentors, and mentor-
mentee mismatch. Lack of training and preparation of mentors and mentees included role
inadequacy of both the mentor and mentee, different understandings about the nature of
mentorship, and being unable to attend mentor training due to workload and acuity of the nursing
unit. Availability of mentors included time and schedule constraints as reasons for not fulfilling
their responsibilities, the allocation of mentors as not always appropriate, and making time for
meetings between the mentor and mentee. Mentor-mentee mismatch included personality
clashes and relationship issues, mentors being assigned by default rather than volunteering for
the role, and the inability to choose your own mentor.
6.2 Recommendations for Future Research
The literature search conducted for this REA revealed there is not a huge body of high
level of evidence research being conducted on mentorship programs for new graduate RNs.
Many of the original 66 research studies were excluded due to being opinion papers rather than
high evidence research studies which indicate a need for further work in this area, such as
focused evaluation studies that explore the effect of mentorship programs on performance,
79
satisfaction, retention, and confidence in the novice nurse. It was interesting to note that
although much of the opinion nursing literature reported that mentorship improves retention of
RNs, this REA only found four research studies to provide evidence. Nursing shortage is a
major concern across Canada with an anticipated shortage of almost 60,000 FTE RNs by 2022
(CNA, 2009), therefore it is recommended that future research includes further high
methodological quality research on mentorship and its effect on retention of new graduate RNs.
This could include comparative studies that study the effect of mentorship programs along with
another variable, such as an extended orientation or residency program, on the retention of new
graduate RNs.
Another recommendation for future work would be to explore the effects of a mentorship
program on all four aspects (performance, satisfaction, retention, and confidence) of a novice
nurses’ practice in one high evidence research study. The author of this REA was not able to
find any research study that examined performance, satisfaction, retention, and confidence at the
same time; thus indicating a need for further research. This could include a comparative study
that individually compares each aspect on a novice nurses’ practice to determine which variable
is most influenced by a mentorship program.
A further recommendation for future work is on mentorship and its effect on retention in
experienced RNs in a new practice area. Experienced RNs in a new practice area can also face
challenges such as short staffing, poor communication among colleagues, abusive or
unsupportive colleagues, heavy workloads, physical and emotional demands, and lack of
administrative support which may result in the decision to leave a practice area (Maddalena et
al.). The average cost of replacing a RN in Canada as $25,000 and high turnover rates are
associated with a decrease in job satisfaction, increase in likelihood of medical errors, overtime,
80
and environmental complexity (O’Brien-Pallas et al.), thus retention of experienced RNs is also
crucial to ensure adequate supply of RNs in Canada.
6.3 Chapter Summary
This final chapter has provided a summary of the evidence that suggests mentorship
programs are effective in improving performance, satisfaction, retention, and confidence in
novice nurses’ practice under the right conditions. The reported key elements of effective new
graduate RN mentorship programs and the reported problems in implementing new graduate RN
mentorship programs were also presented. Three recommendations for future research on the
topic of mentorship programs were also provided. The findings from this REA suggest that the
mentorship of new graduate RNs is valuable and should be considered by stakeholders who are
responsible for recruitment and retention within health care organizations, such as directors and
managers; and particularly staff who are responsible for the orientation and mentorship of newly
qualified RNs (such as clinical nurse educators).
81
BIBLIOGRAPHY
Allen, S. L. (2006). Mentoring: The magic partnership. Canadian Operating Room Nursing
Journal, 24(4), 30-2, 37-8.
Almada, P., Carafoli, K., Flattery, J.B., French, D.A., & McNamara, M. (2004). Improving the
retention rate of newly graduated nurses. Journal for Nurses in Staff Development, 20(6),
268-73.
Academy of Medical-Surgical Nurses. (2012). AMSN mentoring program: Mentoring guide.
Author: Pitman, NJ. https://www.amsn.org/sites/default/files/documents/professional-
development/mentoring/AMSN-Mentoring-Mentor-Guide.pdf
Bahn, D. (2001). Social learning theory: Its application in the context of nurse education. Nurse
Education Today, 21, 110-7. doi:10.1054/nedt.2000.0522
Bandura, A. (1977). Social learning theory. Englewood Cliffs, NJ: Prentice Hall.
Barton, D. S., Gowdy, M., & Hawthorne, B. W. (2005). Mentorship programs for novice nurses.
Nurse Leader, 3(4), 41-44. doi: 10.1016/j.mnl.2005.06.005
Baxter, P.E. (2010). Providing orientation programs to new graduate nurses. Journal for Nurses
in Staff Development, 26(4), 12-7.
Beercroft, P.C., Santner, S., Lacy, M.L., Kunzman, L., & Dorey, F. (2006). New graduate
nurses’ perceptions of mentoring: Six-year programme evaluation. Journal of Advanced
Nursing, 55(6), 736-47.
Benner, P. (1984). From novice to expert. The American Journal of Nursing, 82(3), 402-7.
Block, L.M., Claffey, C., Korow, M.K., & McCaffrey, R. (2005). The value of mentorship
within nursing organizations. Nursing Forum, 40(4), 134-40.
Bialkowski, K. (2009). Impact of mentoring on job satisfaction and retention. Available from
82
ProQuest Dissertations & Theses Global. (921509994). Retrieved from
http://search.proquest.com.ezproxy.library.ubc.ca/docview/921509994?accountid=14656
Bowles, C. & Candela, L. (2005). First job experiences of recent RN graduates. The Journal of
Nursing Administration, 35(3), 130-7.
Boycott, N., Schneider, J., & McMurran, M. (2012). Additional interventions to enhance the
effectiveness of individual placement and support: A rapid evidence assessment.
Rehabilitation Research and Practice, 2012, 1-8. doi:10.1155/2012/382420
Braungart, M.M. & Braungart, R.G. (2008). Applying learning theories to healthcare practice. In:
S. Bastabel (Ed.), Health professional as educator (pp. 51-89). Sudbury, MA: Jones &
Bartlett Learning.
Brown, B., O’Mara, L., Hunsberger, M., Love, B., Black, M., Carpio, B., Crooks, D., Nosegaard,
C. (2003). Professional confidence in baccalaureate nursing students. Nurse Education in
Practice, 3, 163-70. doi:10.1016/S1471-5953(02)00111-7
Buchan, J. & Aiken, L. (2008). Solving nursing shortages: A common priority. Journal of
Clinical Nursing, 17, 3262-8. doi: 10.1111/j.1365-2702.2008.02636.x
Canadian Institute for Health Information (2013). Regulated nurses: Canadian trends, 2007 to
2011. Author: Ottawa. https://secure.cihi.ca/free_products/Regulated_Nurses_EN.pdf
Canadian Nurses Association. (2004). A guide to preceptorship and mentoring. Author: Ottawa.
https://www.cna-aiic.ca/~/media/cna/page-content/pdf-
en/achieving_excellence_2004_e.pdf?la=en
Canadian Nurses Association. (2009). Tested solutions for eliminating Canada’s registered nurse
shortage. Author: Ottawa.
https://nursesunions.ca/sites/default/files/rn_shortage_report_e.pdf
83
Canadian Nurses Association. (2013). RN workplace profile by area of responsibility. Author:
Ottawa. http://cna-aiic.ca/~/media/cna/files/en/2011_rn_profiles_responsibility_e.pdf
Chen, C-M., & Lou, M-F. (2014). The effectiveness and application of mentorship programmes
for recently registered nurses: A systematic review. Journal of Nursing Management,
22(4), 433-42. doi: 10.1111/jonm.12102
Cho, J., Laschinger, H.K.S., & Wong, C. (2006). Workplace empowerment, work engagement,
and organizational commitment of new graduate nurses. Nursing Leadership, 19(3), 43-
60.
College of Registered Nurses of British Columbia. (2014). Profile of a newly graduated
registered nurse. Author: Vancouver.
https://www.crnbc.ca/Standards/Lists/StandardResources/390profilenewgrad.pdf
Cooper, M., & Wheeler, M.M. (2010). Building successful mentoring relationships. Canadian
Nurse, 106(7), 34-5.
Craig, C., Moscato, S., & Moyce, S. (2012). New BSN nurses’ perspectives on the transition to
practice in changing economic times. The Journal of Nursing Administration, 42(4), 202-
7. doi: 10.1097/NNA.0b013e31824ccd95
DeLucia, P.R, Ott, T.E., Palmieri, P.A. (2009). Performance in nursing. In: F. Durso (Ed.),
Reviews of human factors and ergonomics (pp. 1-40). Sage Publications.
Duchscher, J.B. (2008). A process of becoming: The stages of new nursing graduate professional
role transition. The Journal of Continuing Education in Nursing, 39(10), 441-50.
Dunn, D. (2014). Where, oh where, are the OR nurses: Retention. OR Nurse, 8(5), 8-11. doi:
10.1097/01.ORN.0000453335.45094.5b
EPPI-Centre. (2007). Review guidelines for extracting data and quality assessing primary
84
studies in educational research version 2.0. Author: London. Retrieved March 26, 2015
from http://www.civilservice.gov.uk/networks/gsr/resources-and-guidance
EPPI-Centre. (2010). Methods for conducting systematic reviews. Author: London. Retrieved
March 27, 2015 from
https://eppi.ioe.ac.uk/cms/LinkClick.aspx?fileticket=hQBu8y4uVwI%3D&tabid=88
Fawcett, D.L. (2002). Mentoring – What it is and how to make it work. Association of
periOperative Registered Nurses Journal, 75(5), 950-4.
Fox, K.C. (2010). Mentor program boosts new nurses’ satisfaction and lowers turnover rate. The
Journal of Continuing Education in Nursing, 41(7), 311-6.
Ganann, R., Ciliska, D., & Thomas, H. (2010). Expediting systematic reviews: Methods and
implications of rapid reviews. Retrieved March 23, 2015 from
http://www.implementationscience.com/content/pdf/1748-5908-5-56.pdf
Garrett, B. (2012). Rapid evidence assessment thesis option. UBC School of Nursing, Faculty
of Applied Science.
Gough, D. (2007). Weight of evidence: A framework for the appraisal of the quality and
relevance of evidence. Research Papers in Education, 22(2), 213-28.
Government Social Research Service. (2010). Rapid Evidence Assessment Toolkit. Retrieved
January 12, 2015 from http://www.civilservice.gov.uk/networks/gsr/resources-and-
guidance
Greene, M.T., & Puetzer, M. (2002). The values of mentoring: A strategic approach to retention
and recruitment. Journal of Nursing Care Quality, 17(1), 63-70.
Grindel, C.G. & Hagerstrom. (2009). Nurses nurturing nurses: Outcomes and lessons learned.
MedSurg Nursing, 18(3), 183-94.
85
Haggerty, C., Holloway, K., & Wilson, D. (2013). How to grow our own: An evaluation of
preceptorship in New Zealand graduate nurse programmes. Contemporary Nurse, 43(2),
162-71.
Hale, R. (2004). Mentorship of nurses: An assessment of the first year of licensure. Available
from ProQuest Dissertations & Theses Global. (305103039). Retrieved from
http://search.proquest.com.ezproxy.library.ubc.ca/docview/305103039?accountid=14656
Halfer, D., Graf, E., & Sullivan, C. (2008). The organizational impact of a new graduate
pediatric nurse mentoring program. Nursing Economic$, 26(4), 243-9.
Hardyman, R., & Hickey, G. (2001). What do newly-qualified nurse expect from preceptorship?
Exploring the perspective of the preceptee. Nurse Education Today, 21(1), 58-64.
Harvey, P. (2012). Role of the mentor in the theatre setting. Journal of Perioperative Practice,
22(7), 232-236.
Hunsberger, M., Baumann, A., & Crea-Arsenio, M. (2013). The road to providing quality care:
Orientation and mentorship for new graduate nurses. Canadian Journal of Nursing
Research, 45(4), 72-87.
Kanaskie, M.L. (2006). Mentoring – A staff retention tool. Critical Care Nursing Quarterly,
29(3), 248-52.
Kaviani, N., & Stilwell, Y. (2000). An evaluative study of clinical preceptorship. Nurse
Education Today, 20, 218-26. doi:10.1054/nedt.1999.0386
Komaratat, S., & Oumtanee, A. (2009). Using a mentorship model to prepare newly graduated
nurses for competency. The Journal of Continuing Education in Nursing, 40(10), 475-80.
Kopp, E.M., & Hinkle, J.L. (2006). Understanding mentoring relationships. Journal of
Neuroscience Nursing, 38(2), 126-31.
86
LaFleur, A.K., & White, B.J. (2010). Appreciating mentorship: The benefits of being a mentor.
Professional Case Management, 15(6), 305-11.
Leners, D.W., Wilson, V.W., Connor, P., & Fenton, J. (2006). Mentorship: Increasing retention
possibilities. Journal of Nursing Management, 14(8), 652-4.
Lewis, S., & McGowan, B. (2015). Newly qualified nurses’ experiences of a preceptorship.
British Journal of Nursing, 24(1), 40-3.
Lindsey, K.S. (2000). Perceptions of novice nurses job satisfaction levels related to mentoring.
Available from ProQuest Central; ProQuest Dissertations & Theses Global. (250693961).
Retrieved from
http://search.proquest.com.ezproxy.library.ubc.ca/docview/250693961?accountid=14656
Maddalena, V., Kearney, A.J., & Adams, L. (2012). Quality of work life of novice nurses.
Journal for Nurses in Staff Development, 28(2), 74-9.
Marks-Maran, D., Ooms, A., Tapping, J., Muir, J., Phillips, S., & Burke, L. (2013). A
preceptorship programme for newly qualified nurses: A study of preceptees’ perceptions.
Nurse Education Today, 33(11), 1428-34.
McBurney, E.I. (2015). Strategic mentoring: Growth for mentor and mentee. Clinics in
Dermatology, 33(2), 257-60. doi:10.1016/j.clindermatol.2014.12.019
Meier, S.R. (2013). Concept analysis of mentoring. Advances in Neonatal Care, 13(5), 341-5.
doi: 10.1097/ANC.0b013e3182a14ca4
Melo, M.B., Barbosa, M.A., Souza, P.R. (2011). Job satisfaction of nursing staff: Integrative
review. Revista Latino-Americana Enfermagem, 19(4), 1047-55.
Mills, J.F., & Mullins, A.C. (2008). The California nurse mentor project: Every nurse deserves a
mentor. Nursing Economic$, 26(5), 310-5.
87
Navarro, J. (2009). The mentoring experiences and self-efficacy of new graduate nurses during
transition from student to professional nurse. Available from ProQuest Dissertations &
Theses Global. (276267757). Retrieved from
http://search.proquest.com.ezproxy.library.ubc.ca/docview/276267757?accountid=14656
O’Brien-Pallas, L., Tomblin, G., Shamian, J., Li, X., Kephart, G., Laschinger, H., Smadu, M.,
McGillis, L., D’Amour, D., Gallant, M., Hayes, L., Lee, J., Lee, N., & Liu, Y. (2008).
Understanding the costs and outcomes of nurses’ turnover in Canadian hospitals. Author:
Ottawa. http://nhsru.com/publications/understanding-the-costs-and-outcomes-of-nurses-
turnover-in-canadian-hospitals/
Oermann, M.H., & Gaberson, K.B. (2009). Evaluation and testing in nursing education. New
York: Springer Publishing Company.
Persaud, D. (2008). Mentoring the new graduate perioperative nurse: A valuable retention
strategy. Association of periOperative Registered Nurses Journal, 87(6), 1173-9.
Popay, J., Roberts, H., Sowden, A., Petticrew, M., Arai, L., Rodgers, M., Britten, N., Roen, K.,
& Duffy, S. (2006). Guidance on the conduct of narrative synthesis in systematic
reviews. Retrieved March 26, 2015 from
http://www.lancaster.ac.uk/shm/research/nssr/research/dissemination/publications/NS_Sy
nthesis_Guidance_v1.pdf
Public Health Resource Unit. (2006). Critical Appraisal Skills Program (CASP). Retrieved
March 26, 2015 from http://resources.civilservice.gov.uk/wp-
content/uploads/2011/09/Qualitative-Appraisal-Tool_tcm6-7385.pdf
Rheaume, A., Clement, L., LeBel, N. (2011). Understanding intention to leave amongst new
graduate Canadian nurses: A repeated cross sectional survey. International Journal of
88
Nursing Studies, 48, 490-500. doi:10.1016/j.ijnurstu.2010.08.005
Ronsten, B., Andersson, E., & Gustafsson, B. (2005). Confirming mentorship. Journal of
Nursing Management, 13(4), 312-21.
Schroeter, K. (2008). Competence literature review. Competency & Credentialing Institute.
Retrieved April 10, 2015 from http://www.cc-institute.org/docs/default-document-
library/2011/10/19/competence_lit_review.pdf
Shaffer, B., Tallanco, B., & Walsh, J. (2000). Win-win mentoring. Dimensions of Critical Care
Nursing, 19(3), 36-8.
Sherman, L. W., Gottfredson, D., MacKenzie, D., Eck, J., Reuter, P., and Bushway, S. (1997).
Preventing crime: What works, what doesn’t, what’s promising. National Institute of
Justice. Retrieved March 26, 2015 from https://www.ncjrs.gov/pdffiles/171676.PDF
Smith, C.B. (2006). The influence of mentoring on goal attainment and role satisfaction for
registered nurses in acute care facilities. Available from ProQuest Central; ProQuest
Dissertations & Theses Global. (305327278). Retrieved from
http://search.proquest.com.ezproxy.library.ubc.ca/docview/305327278?accountid=14656
Tourigny, L., & Pulich, M. (2005). A critical examination of formal and informal mentoring
among nurses. The Health Care Manager, 24(1), 68-76.
Wagner, A.L., & Seymour, M.E. (2007). A model of caring mentorship for nursing. Journal for
Nurses in Staff Development, 23(5), 201-11.
Watt, A., Cameron, A., Sturm, L., Lathlean, T., Blamey, S., Facey, K., Hailey, D., Norderhaug,
I., & Maddern, G. (2008). Rapid reviews versus full systematic reviews: An inventory of
current methods and practice in health technology assessment. International Journal of
Technology Assessment in Health Care, 24(2), 133-9.
89
Wensel, T.M. (2006). Mentor or preceptor: What is the difference? American Journal of
Health-System Pharmacy, 63(17), 1597.
Westendorf, J.J. (2007). The nursing shortage: Recruitment and retention of current and future
nurses. Plastic Surgical Nursing, 27(2), 93-7.
Wilson, V.W., Andrews, M., & Leners, D.W. (2006). Mentoring as a strategy for retaining
racial and ethnically diverse students in nursing programs. The Journal of Multicultural
Nursing and Health, 12(3), 17-23.
Wolak, E.S. (2007). Perceptions of an intensive care unit mentorship program. Available from
ProQuest Central; ProQuest Dissertations & Theses Global. (304832833). Retrieved from
http://search.proquest.com.ezproxy.library.ubc.ca/docview/304832833?accountid=14656
Woodfine, P. (2011). Taking a novice nurse under your wing. Nursing, 41(9), 53-5.
Yonge, O., Billay, D., Myrick, F., & Luhanga, F. (2007). Preceptorship and mentorship: Not
merely a matter of semantics. International Journal of Nursing Education Scholarship,
4(1), 19-32. doi: 10.2202/1548-923X.1384
Young, L.E. (2009). Mentoring new nurses in stressful times. Canadian Operating Room
Nursing Journal, 27(2), 6-7, 14-5, 29-30.
90
APPENDIX A: Excel Spreadsheet
Author &
Year Study Design Purpose/Aims
Intervention
Length Participants Outcomes/Findings
WoE
Level &
Score
MSSM
Level &
Score
CASP
Level &
Score
Lindsey,
K.S. (2000)
Descriptive
Comparative
Survey
Is there a difference in job
satisfaction levels between
novice nurses who have been
mentored versus those who
haven't been mentored?
Varied
between
participants 163
Significant relationship
between mentoring and job
satisfaction. High 9
Medium
3
Grindel,
C.G. &
Hagerstrom.
(2009) Longitudinal
A 12-month mentorship
program wherein the mentor
and mentee would work
together to facilitate the
transition of the new nurse to
professional nursing practice
and implement career goals of
the mentee.
12 month
mentorship
program 129
Significant relationship
between mentoring and job
satisfaction. No
relationship between
mentoring and intent to
stay and confidence. High 8
Medium
2
Komaratat,
S., &
Oumtanee,
A. (2009)
Quasi-
Experimental
This research was conducted
to study the level of nursing
competency of newly
graduated nurses after using a
mentorship model.
1 month
program
working
together 19
Significant relationship
between mentoring and
nursing competence
(performance). High 8
Medium
2
Hale, R.
(2004)
Descriptive
Exploratory
To explore mentoring
relationships from the
perspectives of the new
graduate nurses and assess the
consequences of the
mentorship relationships
(positive benefits).
Varied
between
participants 144
Significant relationship
between mentoring and
confidence, competence,
and job satisfaction. High 8 Low 1
91
Author &
Year Study Design Purpose/Aims
Intervention
Length Participants Outcomes/Findings
WoE
Level &
Score
MSSM
Level &
Score
CASP
Level &
Score
Halfer, D.,
Graf, E., &
Sullivan, C.
(2008)
Descriptive
Survey
A descriptive study to
compare the job satisfaction
and retention rates of two
cohorts of new graduate
nurses: one before and one
after implementation of a
Pediatric RN Internship
Program. Not described 234
Job satisfaction higher in
post group and lower
turnover rate (sustained for
2 years). High 7
Medium
3
Smith, C.B.
(2006)
Descriptive
Correlational
This quantitative study
examined the influence of
mentoring for RNs who have
been employed in acute care
facilities for the first time for
less than 2 years.
Varied
between
participants 51
Significant relationship
between mentoring and
role satisfaction with high
levels of goal attainment
and mentoring antecedents
and job satisfaction
Medium
6 Low 1
Fox, K.C.
(2010)
Evaluation
Survey
A pilot mentoring program
was initiated to reduce the
turnover rate of newly hired
registered nurses.
1 year
program 12
Satisfaction scores
improved by one level
(from agree to strongly
agree or tend to disagree to
agree) in 75% of the
participants, reflecting
improved satisfaction. The
mentee also reported
feeling more comfortable
in their roles due to the
support and resources
provided by the mentors.
Medium
5 N/A
92
Author &
Year Study Design Purpose/Aims
Intervention
Length Participants Outcomes/Findings
WoE
Level &
Score
MSSM
Level &
Score
CASP
Level &
Score
Hardyman,
R., &
Hickey, G.
(2001)
Longitudinal
Survey
A longitudinal, questionnaire
survey exploring the
expectations of preceptorship
from the perspective of newly
qualified nurses. Not described 1512
The aspects of
preceptorship found to be
most important to the
participants include
constructive feedback on
clinical skills (99%),
teaching new clinical skills
(99%), confidence building
(95%), and helping to settle
into the work environment
(94%).
Medium
4 N/A
Marks-
Maran, D.,
Ooms, A.,
Tapping, J.,
Muir, J.,
Phillips, S.,
& Burke, L.
(2013)
Mixed-
Methods
This article presents the
evaluation of a preceptorship
programme for newly
qualified nurses to determine
preceptee engagement with
the preceptorship programme,
and the impact, value, and
sustainability of the
programme from the
preceptees' perspective. Not described 44
Findings from this study
included 78% of the
mentees reporting an
improvement in confidence
with making decisions
about patient care, 70%
reporting that the
preceptorship has enhanced
their role satisfaction, 68%
reporting improved
competence with drug
administration and 75%
reporting being able to deal
more confidently with
problems relating to patient
care. High 9 Low 1 High 8
93
Author &
Year Study Design Purpose/Aims
Intervention
Length Participants Outcomes/Findings
WoE
Level &
Score
MSSM
Level &
Score
CASP
Level &
Score
Hunsberger,
M.,
Baumann,
A., & Crea-
Arsenio, M.
(2013)
Mixed-
Methods
A trend study design was
used to examine the impact of
extended orientation and
mentorship on the transition
of new graduate nurses to
professional practice over a 3-
year period.
3-6 month
program 3813
Mentor helped to be more
confident with
documentation and
medication administration High 8 N/A High 9
Beercroft,
P.C.,
Santner, S.,
Lacy, M.L.,
Kunzman,
L., &
Dorey, F.
(2006)
Mixed-
Methods
The aims of the study were to
determine whether mentoring
was successful and if new
grads: were satisfactorily
matched with a mentor;
received guidance and
support; attained
socialization; benefitted from
having a role model;
maintained contact with
mentor; and satisfied with
mentorship. Not described 318
Appears mentors could be
instrumental in retention of
new grads by increasing
confidence High 8 Low 1 High 8
Haggerty,
C.,
Holloway,
K., &
Wilson, D.
(2013)
Mixed-
Methods
This article provides an
overview of nurse entry to
practice programmes in New
Zealand and the key findings
on new graduate's confidence
and competence.
6 weeks with
preceptor 1023
Overall satisfaction and
increased competence and
confidence due to program High 8 N/A
Medium
7
94
Author &
Year Study Design Purpose/Aims
Intervention
Length Participants Outcomes/Findings
WoE
Level &
Score
MSSM
Level &
Score
CASP
Level &
Score
Almada, P.,
Carafoli,
K., Flattery,
J.B.,
French,
D.A., &
McNamara,
M. (2004)
Mixed-
Methods
Will a newly designed
preceptor program provide
NGNs adequate education,
support, and acceptance in
their new role as staff nurses
and thereby increase retention
rates?
8 weeks with
preceptor 40
Retention rate increased
from 60 to 89% and mean
scores for satisfaction
93.7%
Medium
6 Low 1
Medium
6
Bialkowski,
K. (2009)
Mixed-
Methods
To evaluate a mentoring
program by examining the
impact of length of
orientation on job
satisfaction, organizational
commitment, and propensity
to leave.
Up to 7.5
month long
program 21
Significant difference
between
mentorship/orientation of <
1 month and 3-6 months
Medium
4
Medium
2
Medium
7
Lewis, S.,
&
McGowan,
B. (2015) Qualitative
What was the experience of
preceptorship like for newly
qualified registered nurses in
a healthcare trust? Not described 8
Preceptorship enabled them
to develop their confidence
and further develop their
knowledge and skills
Medium
6 High 9
95
Author &
Year Study Design Purpose/Aims
Intervention
Length Participants Outcomes/Findings
WoE
Level &
Score
MSSM
Level &
Score
CASP
Level &
Score
Navarro, J.
(2009)
Descriptive
Exploratory
The purpose of this study was
to: describe the mentorship
experiences of NGNs as they
transition from student to RN,
describe how the mentorship
experience shaped the
perceived self-efficacy of
NGNs, and describe how the
mentorship experiences
shaped the transition from
student to RN.
Varied
between
participants 5
All the participants in this
study reported their
mentors using positive
verbal persuasion and
encouragement which
resulted in increased
confidence. The mentors
also challenged the new
graduate nurses to perform
new skills and resulted in
an increased clinical
competence and the
confidence to perform
skills independently.
Medium
5 High 9
Wolak, E.S.
(2007) Focus Group
The purpose of this study was
to examine the experiences of
mentees and mentors in a
structured mentorship
program.
Paired
together at
least 10
months 5
All the participants
reported that the mentors
provided the mentees with
valuable knowledge and
skills thus contributing to
increased job satisfaction
and organizational
commitment.
Medium
5 High 9
96
APPENDIX B: GSRS Weight of Evidence (WoE) Tool
Weight of Evidence A: Taking account of all quality assessment issues, can the study findings be
trusted in answering the study question(s)?
High Evidence Score of 3
Medium Evidence Score of 2
Low Evidence Score of 1
Weight of Evidence B: Appropriateness of research design and analysis for addressing the
question, or sub-questions, of this specific REA
High Evidence Score of 3
Medium Evidence Score of 2
Low Evidence Score of 1
Weight of Evidence C: Relevance of particular focus of the study (including conceptual focus,
context, sample and measures) for addressing the research question, or sub-questions, of this
specific REA
High Evidence Score of 3
Medium Evidence Score of 2
Low Evidence Score of 1
Weight of Evidence D: Overall weight of evidence
High Evidence Score of 7 - 9
Medium Evidence Score of 4 - 6
Low Evidence Score of 3
Source: EPPI-Centre (2007) Review Guidelines for Extracting Data and Quality Assessing
Primary Studies in Educational Research. Version 2.0 London: EPPI-Centre, Social Science
Research Unit.http://www.civilservice.gov.uk/networks/gsr/resources-and-guidance/rapid-
evidence-ssessment/how-to-do-a-rea
97
APPENDIX C: Maryland Scale of Scientific Methods (MSSM) Tool
Increasing Methodological Quality for Impact Studies
Level 1
Observed correlation between an intervention and outcomes at a single point
in time. A study that only measured the impact of the service using a
questionnaire at the end of the intervention would fall into this level.
Level 2
Temporal sequence between the intervention and the outcome clearly
observed; or the presence of a comparison group that cannot be
demonstrated to be comparable. A study that measured the outcomes of
people who used a service before it was set up and after it finished would fit
into this level.
Level 3
A comparison between two or more comparable units of analysis, one with
and one without the intervention. A matched-area design using two locations
in the UK would fit into this category if the individuals in the research and
the areas themselves were comparable.
Level 4
Comparison between multiple units with and without the intervention,
controlling for other factors or using comparison units that evidence only
minor differences. A method such as propensity score matching, that used
statistical techniques to ensure that the programme and comparison groups
were similar would fall into this category.
Level 5
Random assignment and analysis of comparable units to intervention and
control groups. A well conducted Randomized Controlled Trial fits into this
category.
Source: Sherman et al. (1997). Preventing crime: What works, what doesn’t, what’s promising.
National Institute of Justice. Retrieved March 26, 2015 from
https://www.ncjrs.gov/pdffiles/171676.PDF
98
APPENDIX D: Critical Appraisal Skills Programme (CASP) Tool
This assessment tool has been developed for those unfamiliar with qualitative research and its
theoretical perspectives. This tool presents a number of questions that deal very broadly with
some of the principles or assumptions that characterise qualitative research. It is not a definitive
guide and extensive further reading is recommended.
How to use this appraisal tool
Three broad issues need to be considered when appraising the report of qualitative research:
Rigour: has a thorough and appropriate approach been applied to key research methods in
the study?
Credibility: are the findings well presented and meaningful?
Relevance: how useful are the findings to you and your organisation?
The 10 questions on the following pages are designed to help you think about these issues
systematically.
The first two questions are screening questions and can be answered quickly. If the answer to
both is “yes”, it is worth proceeding with the remaining questions.
A number of italicised prompts are given after each question. These are designed to remind you
why the question is important. Record your reasons for your answers in the spaces provided.
Screening Questions
1. Was there a clear statement of the aims of the research? Yes No
Consider:
– what the goal of the research was
– why it is important
– its relevance
2. Is a qualitative methodology appropriate? Yes No
Consider:
– if the research seeks to interpret or illuminate the actions and/or subjective experiences of
research participants
Is it worth continuing?
Appropriate research design
3. Was the research design appropriate to address the aims of the research?
Consider:
– if the researcher has justified the research design (e.g. have they discussed how they decided
99
which methods to use?)
Sampling
4. Was the recruitment strategy appropriate to the aims of the research?
Consider:
– if the researcher has explained how the participants were selected
– if they explained why the participants they selected were the most appropriate to provide
access to the type of knowledge sought by the study
– if there are any discussions around recruitment (e.g. why some people chose not to take
part)
Data collection
5. Were the data collected in a way that addressed the research issue?
Consider:
– if the setting for data collection was justified
– if it is clear how data were collected (e.g. focus group, semi-structured interview etc)
– if the researcher has justified the methods chosen
– if the researcher has made the methods explicit (e.g. for interview method, is there an
indication of how interviews were conducted, did they used a topic guide?)
– if methods were modified during the study. If so, has the researcher explained how and
why?
– if the form of data is clear (e.g. tape recordings, video material, notes etc)
– if the researcher has discussed saturation of data
Reflexivity (research partnership relations/recognition of researcher bias)
6. Has the relationship between researcher and participants been adequately considered?
Consider whether it is clear:
– if the researcher critically examined their own role, potential bias and influence during:
- formulation of research questions
- data collection, including sample recruitment and choice of location
– how the researcher responded to events during the study and whether they considered the
implications of any changes in the research design
Ethical Issues
7. Have ethical issues been taken into consideration?
Consider:
– if there are sufficient details of how the research was explained to participants for the reader
to assess whether ethical standards were maintained
– if the researcher has discussed issues raised by the study (e. g. issues around informed
consent or confidentiality or how they have handled the effects of the study on the participants
during and after the study)
– if approval has been sought from the ethics committee
100
Data Analysis
8. Was the data analysis sufficiently rigorous?
Consider:
– if there is an in-depth description of the analysis process
– if thematic analysis is used. If so, is it clear how the categories/themes were derived from
the data?
– whether the researcher explains how the data presented were selected from the original
sample to demonstrate the analysis process
– if sufficient data are presented to support the findings
– to what extent contradictory data are taken into account
– whether the researcher critically examined their own role, potential bias and influence
during analysis and selection of data for presentation
Findings
9. Is there a clear statement of findings?
Consider:
– if the findings are explicit
– if there is adequate discussion of the evidence both for and against the researcher’s
arguments
– if the researcher has discussed the credibility of their findings (e.g. triangulation, respondent
validation, more than one analyst.)
– if the findings are discussed in relation to the original research questions
Value of the Research
10. How valuable is the research?
Consider:
– if the researcher discusses the contribution the study makes to existing knowledge or
understanding (e.g. do they consider the findings in relation to current practice or policy, or
relevant research-based literature?)
– if they identify new areas where research is necessary
– if the researchers have discussed whether or how the findings can be transferred to other
populations or considered other ways the research may be used
Source: Public Health Resource Unit, England (2006). Critical Appraisal Skills Program
(CASP). Retrieved March 26, 2015 from http://resources.civilservice.gov.uk/wp-
content/uploads/2011/09/Qualitative-Appraisal-Tool_tcm6-7385.pdf
101
APPENDIX E: EPPI-Centre Data Extraction and Coding Tool for Education Studies
Purpose and use of this tool
This tool is designed to help those conducting systematic reviews on educational topics identify
extract and code information about a particular research study that is to be included in a
systematic review.
It is designed to help the reviewer obtain all the necessary information to
assess the quality of the study or its internal validity
Identify the relevant contextual information that may have affected the results obtained in
the specific study
Identify the contextual information about a study that will be relevant to any assessment
of the generalizability of findings in the individual study
Identify relevant information about the design , execution and context of a study for the
purpose of synthesizing (bringing together) results from all the studies that are included
in a particular review
The tool is designed to be used to extract data from a single primary study. That is the report(s)
of a piece of research i.e. not a review (systematic or otherwise), a scholarly paper, treatise or
opinion piece.
The study may be reported in more than one paper for which a single data extraction is
completed
Each separate study included in a review will require a separate data extraction
For the purposes of producing a ‘map’ review groups will usually include questions from
sections A,B,C, D, E (if relevant), G.
Questions B2 and G3 must be included in the coding questions for the map
Additional questions used will depend on the purpose of the map and the type of review. The
questions to be used should be agreed with the funder and the EPPI-Centre prior to starting
coding
Other sections and questions are completed only on studies included in the ‘in-depth review’
102
Section A: Administrative Details
Use of these guidelines should be cited as: EPPI-Centre (2007) Review Guidelines for Extracting
Data and Quality Assessing Primary Studies in Educational Research. Version 2.0 London: EPPI-
Centre, Social Science Research Unit.
A.1 Name of the reviewer A.1.1 Details
A.2 Date of the review A.2.1 Details
A.3 Please enter the details of each paper which
reports on this item/study and which is used to
complete this data extraction.
(1): A paper can be a journal article, a book, or
chapter in a book, or an unpublished report.
A.3.1 Paper (1)
Fill in a separate entry for further papers as
required.
A.3.2 Unique Identifier:
A.3.3 Authors:
A.3.4 Title:
A.3.5 Paper (2)
A.3.6 Unique Identifier:
A.3.7 Authors:
A.3.8 Title:
A.4 Main paper. Please classify one of the
above papers as the 'main' report of the study
and enter its unique identifier here.
NB(1): When only one paper reports on the
study, this will be the 'main' report.
NB(2): In some cases the 'main' paper will be
the one which provides the fullest or the latest
report of the study. In other cases the decision
about which is the 'main' report will have to be
made on an arbitrary basis.
A.4.1 Unique Identifier:
A.5 Please enter the details of each paper which
reports on this study but is NOT being used to
complete this data extraction.
NB A paper can be a journal article, a book, or
chapter in a book, or an unpublished report.
A.5.1 Paper (1)
Fill in a separate entry for further papers as
required.
A.5.2 Unique Identifier:
A.5.3 Authors:
A.5.4 Title:
103
A.5.5 Paper (2)
A.5.6 Unique Identifier:
A.5.7 Authors:
A.5.8 Title:
A.6 If the study has a broad focus and this data
extraction focuses on just one component of the
study, please specify this here.
A.6.1 Not applicable (whole study is focus of
data extraction)
A.6.2 Specific focus of this data extraction
(please specify)
A.7 Identification of report (or reports)
Please use AS MANY KEYWORDS AS APPLY. A.7.1 Citation
Please use this keyword if the report was
identified from the bibliographic list of
another report.
A.7.2 Contact
Please use this keyword if the report was
found through a personal/professional
contact.
A.7.3 Handsearch
Please use this keyword if the report was
found through handsearching a journal.
A.7.4 Unknown
Please use this keyword if it is unknown how
the report was found.
A.7.5 Electronic database
Please use this keyword if the report was
found through searching on an electronic
bibliographic database.
In addition, if the report was found on an
electronic database please use ONE OR
MORE of the following keywords to indicate
which database it was found on:
aidsline
For AIDSLINE
104
appsocscience
For Applied Social and Abstracts
artscitation
For the Arts and Humanities Citation Index
aei
For the Australian Education Index
bei
For the British Education Index
bibliomap
For the EPPI-Centre's specialist register of
research
cabhealth
For CABhealth
cei
For the Canadian Education Index
ceruk
For CERUK
cinahl
For the CINAHL
cochranelib
For the Cochrane Library
dissabs
For Dissertation Abstracts
dislearn
For the Distance Learning Database
eduabs
For Education Abstracts
105
educationline
For Education-line
embase
For EMBASE
eric
For ERIC
healthplan
For Health Planning
healthpromis
For HealthPromis
intbibsocsci
For the International Bibliography of the
Social Sciences
langbehrabs
For Linguistic and Language Behaviour
Abstracts
medline
For MEDLINE
psycinfo
For PsycINFO
regard
For REGARD
sigle
For SIGLE
socscicitation
For the Social Science Citation Index
106
socservabs
For the Social Services Abstracts
socioabs
For Sociological Abstracts
spectr
For the Social, Psychological, Educational &
Criminological Trials Register
A.8 Status
Please use ONE keyword only A.8.1 Published
Please use this keyword if the report has an
ISBN or ISSN number.
A.8.2 Published as a report or conference
paper
Please use this code for reports which do not
have an ISBN or ISSN number (eg. 'internal'
reports; conference papers)
A.8.3 Unpublished
e.g. thesis or author manuscript
A.9 Language (please specify) A.9.1 Details of Language of report
Please use as many keywords that apply
If the name of the language is specified/known
then please use the name as a keyword. For
example:
Dutch
English
French
If non-English and you cannot name the
language:
non English
Section B: Study Aims and Rationale
B.1 What are the broad aims of the study?
Please write in authors’ description if there is B.1.1 Explicitly stated (please specify)
107
one. Elaborate if necessary, but indicate which
aspects are reviewers’ interpretation. Other,
more specific questions about the research
questions and hypotheses are asked later.
B.1.2 Implicit (please specify)
B.1.3 Not stated/unclear (please specify)
B.2 What is the purpose of the study?
N.B. This question refers only to the purpose of
a study, not to the design or methods used.
A: Description
Please use this code for studies in which the
aim is to produce a description of a state of
affairs or a particular phenomenon, and/or to
document its characteristics. In these types of
studies there is no attempt to evaluate a
particular intervention programme (according
to either the processes involved in its
implementation or its effects on outcomes), or
to examine the associations between one or
more variables. These types of studies are
usually, but not always, conducted at one point
in time (i.e. cross sectional). They can include
studies such as an interview of head teachers to
count how many have explicit policies on
continuing professional development for
teachers; a study documenting student attitudes
to national examinations using focus groups; a
survey of the felt needs of parents using self-
completion questionnaires, about whether they
want a school bus service.
B: Exploration of relationships
Please use this code for a study type which
examines relationships and/or statistical
associations between variables in order to build
theories and develop hypotheses. These studies
may describe a process or processes (what goes
on) in order to explore how a particular state of
affairs might be produced, maintained and
changed.
B.2.1 A: Description
B.2.2 B: Exploration of relationships
B.2.3 C: What works?
B.2.4 D: Methods development
B.2.5 E: Reviewing/synthesising research
108
These relationships may be discovered using
qualitative techniques, and/or statistical
analyses. For instance, observations of children
at play may elucidate the process of gender
stereotyping, and suggest the kinds of
interventions which may be appropriate to
reduce any negative effects in the classroom.
Complex statistical analysis may be helpful in
modelling the relationships between parents'
social class and language in the home. These
may lead to the development of theories about
the mechanisms of language acquisition, and
possible policies to intervene in a causal
pathway.
These studies often consider variables such as
social class and gender which are not
interventions, although these studies may aid
understanding, and may suggest possible
interventions, as well as ways in which a
programme design and implementation could
be improved. These studies do not directly
evaluate the effects of policies and practices.
C: What works
A study will only fall within this category if it
measures effectiveness - i.e. the impact of a
specific intervention or programme on a
defined sample of recipients or subjects of the
programme or intervention.
D: Methods development
Studies where the principle focus is on
methodology.
E: Reviewing/Synthesising research
Studies which summarise and synthesise
primary research studies.
109
B.3 Why was the study done at that point in
time, in those contexts and with those people or
institutions?
Please write in authors’ rationale if there is
one. Elaborate if necessary, but indicate which
aspects are reviewers’ interpretation.
B.3.1 Explicitly stated (please specify)
B.3.2 Implicit (please specify)
B.3.3 Not stated/unclear (please specify)
B.4 Was the study informed by, or linked to, an
existing body of empirical and/or theoretical
research?
Please write in authors’ description if there is
one. Elaborate if necessary, but indicate which
aspects are reviewers’ interpretation.
B.4.1 Explicitly stated (please specify)
B.4.2 Implicit (please specify)
B.4.3 Not stated/unclear (please specify)
B.5 Which of the following groups were
consulted in working out the aims of the study,
or issues to be addressed in the study?
Please write in authors’ description if there is
one. Elaborate if necessary, but indicate which
aspects are reviewers’ interpretation. Please
cover details of how and why people were
consulted and how they influenced the
aims/issues to be addressed.
B.5.1 Researchers (please specify)
B.5.2 Funder (please specify)
B.5.3 Head teacher/Senior management
(please specify)
B.5.4 Teaching staff (please specify)
B.5.5 Non-teaching staff (please specify)
B.5.6 Parents (please specify)
B.5.7 Pupils/students (please specify)
B.5.8 Governors (please specify)
B.5.9 LEA/Government officials (please
specify)
B.5.10 Other education practitioner (please
specify)
B.5.11 Other (please specify)
B.5.12 None/Not stated
B.5.13 Coding is based on: Authors'
description
B.5.14 Coding is based on: Reviewers’
inference
B.6 Do authors report how the study was
funded? B.6.1 Explicitly stated (please specify)
110
B.6.2 Implicit (please specify)
B.6.3 Not stated/unclear (please specify)
B.7 When was the study carried out?
If the authors give a year, or range of years,
then put that in. If not, give a ‘not later than’
date by looking for a date of first submission to
the journal, or for clues like the publication
dates of other reports from the study.
B.7.1 Explicitly stated (please specify )
B.7.2 Implicit (please specify)
B.7.3 Not stated/unclear (please specify)
B.8 What are the study research questions
and/or hypotheses?
Research questions or hypotheses
operationalise the aims of the study. Please
write in authors' description if there is one.
Elaborate if necessary, but indicate which
aspects are reviewers' interpretation.
B.8.1 Explicitly stated (please specify)
B.8.2 Implicit (please specify)
B.8.3 Not stated/ unclear (please specify)
Section C: Study Policy or Practice Focus
C.1 What is/are the topic focus/foci of the
study? C.1.1 Assessment (please specify)
C.1.2 Classroom management (please specify)
C.1.3 Curriculum (see next question below)
C.1.4 Equal opportunities (please specify)
C.1.5 Methodology (please specify)
C.1.6 Organisation and management (please
specify)
C.1.7 Policy (please specify)
C.1.8 Teacher careers (please specify)
C.1.9 Teaching and learning (please specify)
C.1.10 Other ( please specify)
C.1.11 Coding is based on: Authors'
description
C.1.12 Coding is based on: Reviewers'
inference
111
C.2 What is the curriculum area, if any? C.2.1 Art
C.2.2 Business Studies
C.2.3 Citizenship
C.2.4 Cross-curricular
C.2.5 Design & Technology
C.2.6 Environment
C.2.7 General
C.2.8 Geography
C.2.9 Hidden
C.2.10 History
C.2.11 ICT
C.2.12 Literacy - first languages
C.2.13 Literacy - further languages
C.2.14 Literature
C.2.15 Maths
C.2.16 Music
C.2.17 PSE
C.2.18 Phys. Ed
C.2.19 Religious Ed.
C.2.20 Science
C.2.21 Vocational
C.2.22 Other
C.2.23 Coding is based on: Authors'
description
C.2.24 Coding is based on: Reviewers'
inference
C.3 What is/are the educational setting(s) of the
study? C.3.1 Community centre
C.3.2 Correctional institution
C.3.3 Government department
112
C.3.4 Higher education institution
C.3.5 Home
C.3.6 Independent school
C.3.7 Local education authority
C.3.8 Nursery school
C.3.9 Other early years setting
C.3.10 Post-compulsory education institution
C.3.11 Primary school
C.3.12 Pupil referral unit
C.3.13 Residential school
C.3.14 Secondary school
C.3.15 Special needs school
C.3.16 Workplace
C.3.17 Other educational setting
C.3.18 Coding is based on: Authors'
description
C.3.19 Coding is based on: Reviewers'
inference
C.4 In which country or countries was the study
carried out?
Provide further details where relevant e.g.
region or city.
C.4.1 Explicitly stated (please specify)
C.4.2 Not stated/unclear (please specify)
C.5 Please describe in more detail the specific
phenomena, factors, services or interventions
with which the study is concerned.
The questions so far have asked about the aims
of the study and any named programme under
study, but this may not fully capture what the
study is about. Please state or clarify here.
C.5.1 Details
113
Section D: Actual Sample
If there are several samples or levels of sample, please complete for each level
D.1 Who or what is/ are the sample in the
study?
Please use AS MANY codes AS APPLY to
describe the nature of the sample of the report.
Only indicate a code if the report specifically
characterises the sample focus in terms of the
categories indicated below
D.1.1 Learners
Please use this code if a population focus of
the study is on pupils, students, apprentices, or
other kinds of learners
D.1.2 Senior management
Please use this code if a sample foci of the
study is on those with responsibility in any
educational institution for the strategic
leadership and management of a whole
organisation. This will include the person with
ultimate responsibility for the educational
institution under study. In the school setting,
the term 'headteacher' is typically used
('principal' in the U.S.A., Canada and
Australia); the term 'principal' is often used in
a college setting, the term 'vice-chancellor' in
a university setting.
D.1.3 Teaching staff
Please use this code if a sample focus of the
study is on staff who teach (or lecture) in a
classroom/lecture-hall setting
D.1.4 Non-teaching staff
Please use this code if a population focus of
the study is on staff who do not teach, but
whose role within the educational institution is
administrative/ organisational, e.g. equal
opportunities coordinators, other support staff
D.1.5 Other educational practitioners
Please use this code if the sample focus of the
study includes representatives from other
educational bodies, including
interest/advisory groups; school governing
bodies and parent support groups
D.1.6 Government
Please use this code if the sample focus of the
114
study is on representatives from government
or governing bodies e.g. from the DfES
(Department for Education and Skills),
BECTA (British Educational Communications
and Technology Agency), LSDA (learning and
Skills Development Agency, formerly FEDA -
Further Education Development Agency) etc.
D.1.7 Local education authority officers
Please use this code if a sample focus of the
study is people who work in a local education
authority
D.1.8 Parents
Please use this code if the sample focus of the
study refers to the inclusive category of carers
of 'children' and 'young people', which may
include natural
parents/mother/father/adoptive parents/foster
parents etc
D.1.9 Governors
Please use this code if the sample focus of the
study is on members of the governing body,
which may include teachers or parents. They
play a role in the management and vision of
the educational institution
D.1.10 Other sample focus (please specify)
D.2 What was the total number of participants
in the study (the actual sample)?
if more than one group is being compared,
please give numbers for each group
D.2.1 Not applicable (e.g. study of policies,
documents etc)
D.2.2 Explicitly stated (please specify)
D.2.3 Implicit (please specify)
D.2.4 Not stated/ unclear (please specify)
D.3 What is the proportion of those selected for
the study who actually participated in the
study?
Please specify numbers and percentages if
possible.
D.3.1 Not applicable (e.g. review)
D.3.2 Explicitly stated (please specify)
D.3.3 Implicit (please specify)
D.3.4 Not stated/unclear (please specify)
115
D.4 Which country/countries are the individuals
in the actual sample from?
If UK, please distinguish between England,
Scotland, N. Ireland and Wales, if possible. If
from different countries, please give numbers
for each.
If more than one group is being compared,
please describe for each group.
D.4.1 Not applicable (e.g. study of policies,
documents, etc.)
D.4.2 Explicitly stated (please specify)
D.4.3 Implicit (please specify)
D.4.4 Not stated/unclear (please specify)
D.5 If the individuals in the actual sample are
involved with an educational institution, what
type of institution is it?
For evaluations of interventions, this will be the
site(s) of the intervention.
Please give details of the institutions (e.g. size,
geographic location mixed/single sex etc.) as
described by the authors. If individuals are
from different institutions, please give numbers
for each. If more than one group is being
compared, please describe all of the above for
each group.
D.5.1 Not applicable (e.g. study of policies,
documents, etc.)
D.5.2 Community centre (please specify)
D.5.3 Post-compulsory education institution
(please specify)
D.5.4 Government Department (please
specify)
D.5.5 Independent school (please specify age
range and school type)
D.5.6 Nursery school (please specify)
D.5.7 Other early years setting (please
specify)
D.5.8 Local education authority (please
specify)
D.5.9 Higher Education Institution (please
specify)
D.5.10 Primary school (please specify)
D.5.11 Correctional Institution (please
specify)
D.5.12 Pupil referral unit (please specify)
D.5.13 Residential school (please specify)
D.5.14 Secondary school (please specify age
range)
D.5.15 Special needs school (please specify)
116
D.5.16 Workplace (please specify)
D.5.17 Other educational setting (please
specify)
D.5.18 Coding is based on: Authors'
description
D.5.19 Coding is based on: Reviewers'
inference
D.6 What ages are covered by the actual
sample?
Please give the numbers of the sample that fall
within each of the given categories. If necessary
refer to a page number in the report (e.g. for a
useful table).
If more than one group is being compared,
please describe for each group
if follow-up study, age of entry to the study
D.6.1 Not applicable (e.g. study of policies,
documents etc)
D.6.2 0-4
D.6.3 5-10
D.6.4 11-16
D.6.5 17 to 20
D.6.6 21 and over
D.6.7 Not stated/unclear (please specify)
D.6.8 Coding is based on: Authors' description
D.6.9 Coding is based on: Reviewers'
inference
D.7 What is the sex of the individuals in the
actual sample?
Please give the numbers of the sample that fall
within each of the given categories. If necessary
refer to a page number in the report (e.g. for a
useful table).
If more than one group is being compared,
please describe for each group.
D.7.1 Not applicable (e.g. study of policies,
documents etc)
D.7.2 Single sex (please specify)
D.7.3 Mixed sex (please specify)
D.7.4 Not stated/unclear (please specify)
D.7.5 Coding is based on: Authors' description
D.7.6 Coding is based on: Reviewers'
inference
D.8 What is the socio-economic status of the
individuals within the actual sample?
If more than one group is being compared,
please describe for each group.
D.8.1 Not applicable (e.g. study of policies,
documents etc)
D.8.2 Explicitly stated (please specify)
D.8.3 Implicit (please specify)
117
D.8.4 Not stated/unclear (please specify)
D.9 What is the ethnicity of the individuals
within the actual sample?
If more than one group is being compared,
please describe for each group.
D.9.1 Not applicable (e.g. study of policies,
documents etc)
D.9.2 Explicitly stated (please specify)
D.9.3 Implicit (please specify)
D.9.4 Not stated/unclear (please specify)
D.10 What is known about the special
educational needs of individuals within the
actual sample?
e.g. specific learning, physical, emotional,
behavioural, intellectual difficulties.
D.10.1 Not applicable (e.g. study of policies,
documents etc)
D.10.2 Explicitly stated (please specify)
D.10.3 Implicit (please specify)
D.10.4 Not stated/unclear (please specify)
D.11 Please specify any other useful
information about the study participants. D.11.1 Details
Section E: Programme or Intervention Description
E.1 If a programme or intervention is being
studied, does it have a formal name? E.1.1 Not applicable (no programme or
intervention)
E.1.2 Yes (please specify)
E.1.3 No (please specify)
E.1.4 Not stated/ unclear (please specify)
E.2 Content of the intervention package
Describe the intervention in detail, whenever
possible copying the authors' description from
the report word for word. If specified in the
report, also describe in detail what the control/
comparison group(s) were exposed to.
E.2.1 Details
E.3 Aim(s) of the intervention E.3.1 Not stated
E.3.2 Not explicitly stated (Write in, as
worded by the reviewer)
E.3.3 Stated (Write in, as stated by the
118
authors)
E.4 Year intervention started
Where relevant E.4.1 Details
E.5 Duration of the intervention
Choose the relevant category and write in the
exact intervention length if specified in the
report
When the intervention is ongoing, tick 'OTHER'
and indicate the length of intervention as the
length of the outcome assessment period
E.5.1 Not stated
E.5.2 Not applicable
E.5.3 Unclear
E.5.4 One day or less (please specify)
E.5.5 1 day to 1 week (please specify)
E.5.6 1 week (and 1 day) to 1 month (please
specify)
E.5.7 1 month (and 1 day) to 3 months (please
specify)
E.5.8 3 months (and 1 day) to 6 months
(please specify)
E.5.9 6 months (and 1 day) to 1 year (please
specify)
E.5.10 1 year (and 1 day) to 2 years (please
specify)
E.5.11 2 years (and 1 day) to 3 years (please
specify)
E.5.12 3 years (and 1 day) to 5 years (please
specify)
E.5.13 more than 5 years (please specify)
E.5.14 Other (please specify)
E.6 Person providing the intervention (tick as
many as appropriate) E.6.1 Not stated
E.6.2 Unclear
E.6.3 Not applicable
E.6.4 Counsellor
E.6.5 Health professional (please specify)
E.6.6 parent
119
E.6.7 peer
E.6.8 Psychologist
E.6.9 Researcher
E.6.10 Social worker
E.6.11 Teacher/lecturer
E.6.12 Other (specify)
E.7 Number of people recruited to provide the
intervention (and comparison condition) (e.g.
teachers or health professionals)
E.7.1 Not stated
E.7.2 Unclear
E.7.3 Reported (include the number for the
providers involved in the intervention and
comparison groups, as appropriate)
E.8 How were the people providing the
intervention recruited? (Write in) Also, give
information on the providers involved in the
comparison group(s), as appropriate.
E.8.1 Not stated
E.8.2 Stated (write in)
E.9 Was special training given to people
providing the intervention?
Provide as much detail as possible
E.9.1 Not stated
E.9.2 Unclear
E.9.3 Yes (please specify)
E.9.4 No
Section F: Results and Conclusions
In future this section is likely to incorporate material from EPPI reviewer to facilitate reporting
numerical results
F.1 How are the results of the study presented?
e.g. as quotations/ figures within text, in tables,
as appendices
F.1.1 Details
F.2 What are the results of the study as reported
by the authors?
Before completing data extraction you will need
to consider what type of synthesis will be
undertaken and what kind of 'results' data is
required for the synthesis
F.2.1 Details
120
Warning! Failure to provide sufficient data
here will hamper the synthesis stage of the
review.
Please give details and refer to page numbers
in the report(s) of the study, where necessary
(e.g. for key tables)
F.3 What do the author(s) conclude about the
findings of the study?
Please give details and refer to page numbers
in the report of the study, where necessary
F.3.1 Details
Section G: Study Method
G.1 Study Timing
Please indicate all that apply and give further
details where possible
-If the study examines one or more samples but
each at only one point in time it is cross-
sectional
-If the study examines the same samples but as
they have changed over time, it is a
retrospective, provided that the interest is in
starting at one timepoint and looking
backwards over time
-If the study examines the same samples as they
have changed over time and if data are
collected forward over time, it is prospective
provided that the interest is in starting at one
timepoint and looking forward in time
G.1.1 Cross-sectional
G.1.2 Retrospective
G.1.3 Prospective
G.1.4 Not stated/ unclear (please specify)
G.2 when were the measurements of the
variable(s) used as outcome measures made, in
relation to the intervention
Use only if the purpose of the study is to
measure the effectiveness or impact of an
G.2.1 Not applicable (not an evaluation)
G.2.2 Before and after
G.2.3 Only after
G.2.4 Other (please specify)
121
intervention or programme i.e. its purpose is
coded as 'What Works' in Section B2 -
If at least one of the outcome variables is
measured both before and after the
intervention, please use the 'before and after'
category.
G.2.5 Not stated/unclear (please specify)
G.3 What is the method used in the study?
NB: Studies may use more than one method
please code each method used for which data
extraction is being completed and the
respective outcomes for each method.
A=Please use this code if the outcome
evaluation employed the design of a
randomised controlled trial. To be classified as
an RCT, the evaluation must:
i). compare two or more groups which receive
different interventions or different
intensities/levels of an intervention with each
other; and/or with a group which does not
receive any intervention at all
AND
ii) allocate participants (individuals, groups,
classes, schools, LEAs etc) or sequences to the
different groups based on a fully random
schedule (e.g. a random numbers table is used).
If the report states that random allocation was
used and no further information is given then
please keyword as RCT. If the allocation is
NOT fully randomised (e.g. allocation by
alternate numbers by date of birth) then please
keyword as a non-randomised controlled trial
B=Please use this code if the evaluation
compared two or more groups which receive
different interventions, or different
intensities/levels of an intervention to each
G.3.1 A=Random experiment with random
allocation to groups
G.3.2 B=Experiment with non-random
allocation to groups
G.3.3 C=One group pre-post test
G.3.4 D=one group post-test only
G.3.5 E=Cohort study
G.3.6 F=Case-control study
G.3.7 G=Statistical survey
G.3.8 H=Views study
G.3.9 I=Ethnography
G.3.10 J=Systematic review
G.3.11 K=Other review (non systematic)
G.3.12 L=Case study
G.3.13 M= Document study
G.3.14 N=Action research
G.3.15 O= Methodological study
G.3.16 P=Secondary data analysis
122
other and/or with a group which does not
receive any intervention at all BUT DOES NOT
allocate participants (individuals, groups,
classes, schools, LEAs etc) or sequences in a
fully random manner. This keyword should be
used for studies which describe groups being
allocated using a quasi-random method (e.g.
allocation by alternate numbers or by date of
birth) or other non- random method
C=Please use this code where a group of
subjects e.g. a class of school children is tested
on outcome of interest before being given an
intervention which is being evaluated. After
receiving the intervention the same test is
administered again to the same subjects. The
outcome is the difference between the pre and
post test scores of the subjects.
D=Please use this code where one group of
subjects is tested on outcome of interest after
receiving the intervention which is being
evaluated
E=Please use this code where researchers
prospectively study a sample (e.g. learners),
collect data on the different aspects of policies
or practices experienced by members of the
sample (e.g. teaching methods, class sizes),
look forward in time to measure their later
outcomes (e.g. achievement) and relate the
experiences to the outcomes achieved. The
purpose is to assess the effect of the different
experiences on outcomes.
F=Please use this code where researchers
compare two or more groups of individuals on
the basis of their current situation (e.g. 16 year
old pupils with high current educational
performance compared to those with average
123
educational performance), and look back in
time to examine the statistical association with
different policies or practices which they have
experienced (e.g. class size; attendance at
single sex or mixed sex schools; non school
activities etc).
G= please use this code where researchers
have used a questionnaire to collect
quantitative information about items in a
sample or population e.g. parents views on
education
H= Please use this code where the researchers
try to understand phenomenon from the point of
the 'worldview' of a particular, group, culture
or society. In these studies there is attention to
subjective meaning, perspectives and
experience'.
I= please use this code when the researchers
present a qualitative description of human
social phenomena, based on fieldwork
J= please use this code if the review is explicit
in its reporting of a systematic strategy used for
(i) searching for studies (i.e. it reports which
databases have been searched and the
keywords used to search the database, the list
of journals hand searched, and describes
attempts to find unpublished or 'grey'
literature; (ii) the criteria for including and
excluding studies in the review and, (iii)
methods used for assessing the quality and
collating the findings of included studies.
K= Please use this code for cases where the
review discusses a particular issue bringing
together the opinions/findings/conclusions from
a range of previous studies but where the
124
review does not meet the criteria for a
systematic review (as defined above)
L= please use this code when researchers refer
specifically to their design/ approach as a 'case
study'. Where possible further information
about the methods used in the case study should
be coded
M=please use this code where researchers have
used documents as a source of data e.g.
newspaper reports
N=Please use this code where practitioners or
institutions (with or without the help of
researchers) have used research as part of a
process of development and/or change. Where
possible further information about the research
methods used should be coded
O=please use this keyword for studies which
focus on the development or discussion of
methods; for example discussions of a
statistical technique, a recruitment or sampling
procedure, a particular way of collecting or
analysing data etc. It may also refer to a
description of the processes or stages involved
in developing an 'instrument' (e.g. an
assessment procedure).
P= Please use this code where researchers
have used data from a pre-existing dataset e.g.
The British Household Panel Survey to answer
their 'new' research question.
Section H: Methods-Groups
H.1 If Comparisons are being made between
two or more groups*, please specify the basis of H.1.1 Not applicable (not more than one
group)
125
any divisions made for making these
comparisons
Please give further details where possible
*If no comparisons are being made between
groups please continue to Section I (Methods -
sampling strategy)
H.1.2 Prospective allocation into more than
one group
e.g. allocation to different interventions, or
allocation to intervention and control groups
H.1.3 No prospective allocation but use of pre-
existing differences to create comparison
groups
e.g. receiving different interventions or
characterised by different levels of a variable
such as social class
H.1.4 Other (please specify)
H.1.5 Not stated/ unclear (please specify)
H.2 How do the groups differ? H.2.1 Not applicable (not in more than one
group)
H.2.2 Explicitly stated (please specify)
H.2.3 Implicit (please specify)
H.2.4 Not stated/ unclear (please specify)
H.3 Number of groups
For instance, in studies in which comparisons
are made between group, this may be the
number of groups into which the dataset is
divided for analysis (e.g. social class, or form
size), or the number of groups allocated to, or
receiving, an intervention.
H.3.1 Not applicable (not more than one
group)
H.3.2 One
H.3.3 Two
H.3.4 Three
H.3.5 Four or more (please specify)
H.3.6 Other/ unclear (please specify)
H.4 If prospective allocation into more than one
group, what was the unit of allocation?
Please indicate all that apply and give further
details where possible
H.4.1 Not applicable (not more than one
group)
H.4.2 Not applicable (no prospective
allocation)
H.4.3 Individuals
H.4.4 Groupings or clusters of individuals (e.g
classes or schools) please specify
126
H.4.5 Other (e.g individuals or groups acting
as their own controls - please specify)
H.4.6 Not stated/ unclear (please specify)
H.5 If prospective allocation into more than one
group, which method was used to generate the
allocation sequence?
H.5.1 Not applicable (not more than one
group)
H.5.2 Not applicable (no prospective
allocation)
H.5.3 Random
H.5.4 Quasi-random
H.5.5 Non-random
H.5.6 Not stated/unclear (please specify)
H.6 If prospective allocation into more than one
group, was the allocation sequence concealed?
Bias can be introduced, consciously or
otherwise, if the allocation of pupils or classes
or schools to a programme or intervention is
made in the knowledge of key characteristics of
those allocated. For example, children with
more serious reading difficulty might be seen as
in greater need and might be more likely to be
allocated to the 'new' programme, or the
opposite might happen. Either would introduce
bias.
H.6.1 Not applicable (not more than one
group)
H.6.2 Not applicable (no prospective
allocation)
H.6.3 Yes (please specify)
H.6.4 No (please specify)
H.6.5 Not stated/unclear (please specify)
H.7 Study design summary
In addition to answering the questions in this
section, describe the study design in your own
words. You may want to draw upon and
elaborate on the answers already given.
H.7.1 Details
Section I: Methods - Sampling Strategy
I.1 Are the authors trying to produce findings
that are representative of a given population? I.1.1 Explicitly stated (please specify)
I.1.2 Implicit (please specify)
127
Please write in authors' description. If authors
do not specify, please indicate reviewers'
interpretation.
I.1.3 Not stated/unclear (please specify)
I.2 What is the sampling frame (if any) from
which the participants are chosen?
e.g. telephone directory, electoral register,
postcode, school listings etc.
There may be two stages - e.g. first sampling
schools and then classes or pupils within them.
I.2.1 Not applicable (please specify)
I.2.2 Explicitly stated (please specify)
I.2.3 Implicit (please specify)
I.2.4 Not stated/unclear (please specify)
I.3 Which method does the study use to select
people, or groups of people (from the sampling
frame)?
e.g. selecting people at random, systematically -
selecting, for example, every 5th person,
purposively, in order to reach a quota for a
given characteristic.
I.3.1 Not applicable (no sampling frame)
I.3.2 Explicitly stated (please specify)
I.3.3 Implicit (please specify)
I.3.4 Not stated/unclear (please specify)
I.4 Planned sample size
If more than one group, please give details for
each group separately.
In intervention studies, the sample size will
have a bearing upon the statistical power, error
rate and precision of estimate of the study.
I.4.1 Not applicable (please specify)
I.4.2 Explicitly stated (please specify)
I.4.3 Not stated/unclear (please specify)
I.5 How representative was the achieved sample
(as recruited at the start of the study) in relation
to the aims of the sampling frame?
Please specify basis for your decision.
I.5.1 Not applicable (e.g. study of policies,
documents, etc.)
I.5.2 Not applicable (no sampling frame)
I.5.3 High (please specify)
I.5.4 Medium (please specify)
I.5.5 Low (please specify)
I.5.6 Unclear (please specify)
I.6 If the study involves studying samples
prospectively over time, what proportion of the
sample dropped out over the course of the
study?
If the study involves more than one group,
I.6.1 Not applicable (e.g. study of policies,
documents, etc.)
I.6.2 Not applicable (not following samples
prospectively over time)
128
please give drop-out rates for each group
separately. If necessary, refer to a page number
in the report (e.g. for a useful table).
I.6.3 Explicitly stated (please specify)
I.6.4 Implicit (please specify)
I.6.5 Not stated/unclear (please specify)
I.7 For studies that involve following samples
prospectively over time, do the authors provide
any information on whether, and/or how, those
who dropped out of the study differ from those
who remained in the study?
I.7.1 Not applicable (e.g. study of policies,
documents, etc.)
I.7.2 Not applicable (not following samples
prospectively over time)
I.7.3 Not applicable (no drop outs)
I.7.4 Yes (please specify)
I.7.5 No
I.8 If the study involves following samples
prospectively over time, do authors provide
baseline values of key variables, such as those
being used as outcomes, and relevant socio-
demographic variables?
I.8.1 Not applicable (e.g. study of policies,
documents, etc.)
I.8.2 Not applicable (not following samples
prospectively over time)
I.8.3 Yes (please specify)
I.8.4 No
Section J: Methods - Recruitment and Consent
J.1 Which methods are used to recruit people
into the study?
e.g. letters of invitation, telephone contact,
face-to-face contact.
J.1.1 Not applicable (please specify)
J.1.2 Explicitly stated (please specify)
J.1.3 Implicit (please specify)
J.1.4 Not stated/unclear (please specify)
J.1.5 Please specify any other details relevant
to recruitment and consent
J.2 Were any incentives provided to recruit
people into the study? J.2.1 Not applicable (please specify)
J.2.2 Explicitly stated (please specify)
J.2.3 Not stated/unclear (please specify)
J.3 Was consent sought?
Please comment on the quality of consent, if J.3.1 Not applicable (please specify)
129
relevant. J.3.2 Participant consent sought
J.3.3 Parental consent sought
J.3.4 Other consent sought
J.3.5 Consent not sought
J.3.6 Not stated/unclear (please specify)
Section K: Methods - Data Collection
K.1 Which variables or concepts, if any, does
the study aim to measure or examine? K.1.1 Explicitly stated (please specify)
K.1.2 Implicit (please specify)
K.1.3 Not stated/ unclear
K.2 Please describe the main types of data
collected and specify if they were used to (a) to
define the sample; (b) to measure aspects of the
sample as findings of the study?
Only detail if more specific than the previous
question
K.2.1 Details
K.3 Which methods were used to collect the
data?
Please indicate all that apply and give further
detail where possible
K.3.1 Curriculum-based assessment
K.3.2 Focus group interview
K.3.3 One-to-one interview (face to face or by
phone)
K.3.4 Observation
K.3.5 Self-completion questionnaire
K.3.6 self-completion report or diary
K.3.7 Examinations
K.3.8 Clinical test
K.3.9 Practical test
K.3.10 Psychological test (e.g I.Q test)
K.3.11 Hypothetical scenario including
vignettes
K.3.12 School/ college records (e.g attendance
130
records etc)
K.3.13 Secondary data such as publicly
available statistics
K.3.14 Other documentation
K.3.15 Not stated/ unclear (please specify)
K.3.16 Please specify any other important
features of data collection
K.3.17 Coding is based on: Author's
description
K.3.18 Coding is based on: Reviewers'
interpretation
K.4 Details of data collection instruments or
tool(s).
Please provide details including names for all
tools used to collect data, and examples of any
questions/items given. Also, please state
whether source is cited in the report
K.4.1 Explicitly stated (please specify)
K.4.2 Implicit (please specify)
K.4.3 Not stated/ unclear (please specify)
K.5 Who collected the data?
Please indicate all that apply and give further
detail where possible
K.5.1 Researcher
K.5.2 Head teacher/ Senior management
K.5.3 Teaching or other staff
K.5.4 Parents
K.5.5 Pupils/ students
K.5.6 Governors
K.5.7 LEA/Government officials
K.5.8 Other educational practitioner
K.5.9 Other (please specify)
K.5.10 Not stated/unclear
K.5.11 Coding is based on: Author's
description
K.5.12 Coding is based on: Reviewers'
inference
131
K.6 Do the authors' describe any ways they
addressed the repeatability or reliability of their
data collection tools/methods?
e.g. test-re-test methods
(where more than one tool was employed,
please provide details for each)
K.6.1 Details
K.7 Do the authors describe any ways they have
addressed the validity or trustworthiness of
their data collection tools/methods?
e.g. mention previous piloting or validation of
tools, published version of tools, involvement of
target population in development of tools.
(Where more than one tool was employed,
please provide details for each)
K.7.1 Details
K.8 Was there a concealment of which group
that subjects were assigned to (i.e. the
intervention or control) or other key factors
from those carrying out measurement of
outcome - if relevant?
Not applicable - e.g. analysis of existing data,
qualitative study.
No - e.g. assessment of reading progress for
dyslexic pupils done by teacher who provided
intervention
Yes - e.g. researcher assessing pupil knowledge
of drugs - unaware of whether pupil received
the intervention or not.
K.8.1 Not applicable (please say why)
K.8.2 Yes (please specify)
K.8.3 No (please specify)
K.9 Where were the data collected?
e.g. school, home K.9.1 Educational Institution (please specify)
K.9.2 Home (please specify)
K.9.3 Other institutional setting (please
specify)
K.9.4 Not stated/ unclear (please specify)
132
Section L: Methods - Data Analysis
L.1 What rationale do the authors give for the
methods of analysis for the study?
e.g. for their methods of sampling, data
collection or analysis.
L.1.1 Details
L.2 Which methods were used to analyse the
data?
Please give details (e.g., for in-depth
interviews, how were the data handled?)
Details of statistical analyses can be given next.
L.2.1 Explicitly stated (please specify)
L.2.2 Implicit (please specify)
L.2.3 Not stated/unclear (please specify)
L.2.4 Please specify any important analytic or
statistical issues
L.3 Which statistical methods, if any, were used
in the analysis? L.3.1 Details
L.4 Did the study address multiplicity by
reporting ancillary analyses, including sub-
group analyses and adjusted analyses, and do
the authors report on whether these were pre-
specified or exploratory?
L.4.1 Yes (please specify)
L.4.2 No (please specify)
L.4.3 Not applicable
L.5 Do the authors describe strategies used in
the analysis to control for bias from
confounding variables?
L.5.1 Yes (please specify)
L.5.2 No
L.5.3 Not applicable
L.6 For evaluation studies that use prospective
allocation, please specify the basis on which
data analysis was carried out.
'Intention to intervene' means that data were
analysed on the basis of the original number of
participants, as recruited into the different
groups.
'Intervention received' means data were
analysed on the basis of the number of
participants actually receiving the intervention.
L.6.1 Not applicable (not an evaluation study
with prospective allocation)
L.6.2 'Intention to intervene'
L.6.3 'Intervention received'
L.6.4 Not stated/unclear (please specify)
L.7 Do the authors describe any ways they have
addressed the repeatability or reliability of data
analysis?
L.7.1 Details
133
e.g. using more than one researcher to analyse
data, looking for negative cases.
L.8 Do the authors describe any ways that they
have addressed the validity or trustworthiness
of data analysis?
e.g. internal or external consistency, checking
results with participants.
Have any statistical assumptions necessary for
analysis been met?
L.8.1 Details
L.9 If the study uses qualitative methods, how
well has diversity of perspective and content
been explored?
L.9.1 Details
L.10 If the study uses qualitative methods, how
well has the detail, depth and complexity (i.e.
the richness) of the data been conveyed?
L.10.1 Details
L.11 If the study uses qualitative methods, has
analysis been conducted such that context is
preserved?
L.11.1 Details
Section M: Quality of Study - Reporting
M.1 Is the context of the study adequately
described?
Consider your previous answers to these
questions (see Section B):
why was this study done at this point in time, in
those contexts and with those people or
institutions? (B3)
Was the study informed by, or linked to an
existing body of empirical and/or theoretical
research? (B4)
Which groups were consulted in working out
the aims to be addressed in this study? (B5)
M.1.1 Yes (please specify)
M.1.2 No (please specify)
134
Do the authors report how the study was
funded? (B6)
When was the study carried out? (B7)
M.2 Are the aims of the study clearly reported?
Consider your previous answers to these
questions (See module B):
What are the broad aims of the study? (B1)
What are the study research questions and/or
hypothesis? (B8)
M.2.1 Yes (please specify)
M.2.2 No (please specify)
M.3 Is there an adequate description of the
sample used in the study and how the sample
was identified and recruited?
Consider your answer to all questions in
sections D (Actual Sample), I (Sampling
Strategy) and J (Recruitment and Consent).
M.3.1 Yes (please specify)
M.3.2 No (please specify)
M.4 Is there an adequate description of the
methods used in the study to collect data?
Consider your answers to the following
questions (See Section K)
What methods were used to collect the data?
(K3)
Details of data collection instruments and tools
(K4)
Who collected the data? (K5)
Where were the data collected? (K9)
M.4.1 Yes (please specify)
M.4.2 No (please specify)
M.5 Is there an adequate description of the
methods of data analysis?
Consider your answers to previous questions
(see module L)
M.5.1 Yes (please specify)
M.5.2 No (please specify)
135
Which methods were used to analysis the data?
(L2)
What statistical method, if any, were used in the
analysis? (L3)
Did the study address multiplicity by reporting
ancillary analyses (including sub-group
analyses and adjusted analyses), and do the
authors report on whether these were pre-
specified or exploratory? (L4)
Do the authors describe strategies used in the
analysis to control for bias from confounding
variables? (L5)
M.6 Is the study replicable from this report? M.6.1 Yes (please specify)
M.6.2 No (please specify)
M.7 Do the authors state where the full, original
data are stored? M.7.1 Yes (please specify)
M.7.2 No (please specify)
M.8 Do the authors avoid selective reporting
bias? (e.g. do they report on all variables they
aimed to study, as specified in their
aims/research questions?)
M.8.1 Yes (please specify)
M.8.2 No (please specify)
Section N: Quality of the Study - Weight of Evidence
N.1 Are there ethical concerns about the way
the study was done?
Consider consent, funding, privacy, etc.
N.1.1 Yes, some concerns (please specify)
N.1.2 No (please specify)
N.2 Were students and/or parents appropriately
involved in the design or conduct of the study?
Consider your answer to the appropriate
question in module B.1
N.2.1 Yes, a lot (please specify)
N.2.2 Yes, a little (please specify)
N.2.3 No (please specify)
N.3 Is there sufficient justification for why the
study was done the way it was? N.3.1 Yes (please specify)
N.3.2 No (please specify)
136
Consider answers to questions B1, B2, B3, B4
N.4 Was the choice of research design
appropriate for addressing the research
question(s) posed?
N.4.1 yes, completely (please specify)
N.4.2 No (please specify)
N.5 Have sufficient attempts been made to
establish the repeatability or reliability of data
collection methods or tools?
Consider your answers to previous questions:
Do the authors describe any ways they have
addressed the reliability or repeatability of
their data collection tools and methods (K7)
N.5.1 Yes, good (please specify)
N.5.2 Yes, some attempt (please specify)
N.5.3 No, none (please specify)
N.6 Have sufficient attempts been made to
establish the validity or trustworthiness of data
collection tools and methods?
Consider your answers to previous questions:
Do the authors describe any ways they have
addressed the validity or trustworthiness of
their data collection tools/ methods (K6)
N.6.1 Yes, good (please specify)
N.6.2 Yes, some attempt (please specify)
N.6.3 No, none (please specify)
N.7 Have sufficient attempts been made to
establish the repeatability or reliability of data
analysis?
Consider your answer to the previous question:
Do the authors describe any ways they have
addressed the repeatability or reliability of data
analysis? (L7)
N.7.1 Yes (please specify)
N.7.2 No (please specify)
N.8 Have sufficient attempts been made to
establish the validity or trustworthiness of data
analysis?
Consider your answer to the previous question:
Do the authors describe any ways they have
addressed the validity or trustworthiness of
data analysis? (L8, L9, L10, L11)
N.8.1 Yes, good (please specify)
N.8.2 Yes, some attempt (please specify)
N.8.3 No, none (please specify)
N.9 To what extent are the research design and N.9.1 A lot (please specify)
137
methods employed able to rule out any other
sources of error/bias which would lead to
alternative explanations for the findings of the
study?
e.g. (1) In an evaluation, was the process by
which participants were allocated to, or
otherwise received the factor being evaluated,
concealed and not predictable in advance? If
not, were sufficient substitute procedures
employed with adequate rigour to rule out any
alternative explanations of the findings which
arise as a result?
e.g. (2) Was the attrition rate low and, if
applicable, similar between different groups?
N.9.2 A little (please specify)
N.9.3 Not at all (please specify)
N.10 How generalisable are the study results? N.10.1 Details
N.11 In light of the above, do the reviewers
differ from the authors over the findings or
conclusions of the study?
Please state what any difference is.
N.11.1 Not applicable (no difference in
conclusions)
N.11.2 Yes (please specify)
N.12 Have sufficient attempts been made to
justify the conclusions drawn from the findings,
so that the conclusions are trustworthy?
N.12.1 Not applicable (results and conclusions
inseparable)
N.12.2 High trustworthiness
N.12.3 Medium trustworthiness
N.12.4 Low trustworthiness
N.13 Weight of evidence A: Taking account of
all quality assessment issues, can the study
findings be trusted in answering the study
question(s)?
In some studies it is difficult to distinguish
between the findings of the study and the
conclusions. In those cases, please code the
trustworthiness of these combined
results/conclusions.
N.13.1 High trustworthiness
N.13.2 Medium trustworthiness
N.13.3 Low trustworthiness
N.14 Weight of evidence B: Appropriateness of
research design and analysis for addressing the N.14.1 High
138
question, or sub-questions, of this specific
systematic review. N.14.2 Medium
N.14.3 Low
N.15 Weight of evidence C: Relevance of
particular focus of the study (including
conceptual focus, context, sample and
measures) for addressing the question, or sub-
questions, of this specific systematic review
N.15.1 High
N.15.2 Medium
N.15.3 Low
N.16 Weight of evidence D: Overall weight of
evidence
Taking into account quality of execution,
appropriateness of design and relevance of
focus, what is the overall weight of evidence
this study provides to answer the question of
this specific systematic review?
N.16.1 High
N.16.2 Medium
N.16.3 Low
Section O: This section provides a record of the review of the study
O.1 Sections completed
Please indicate sections completed. O.1.1 Section A: Administrative details
O.1.2 Section B: Study aims and rationale
O.1.3 Section C: Study policy or practice
focus
O.1.4 Section D: Actual sample
O.1.5 Section E: Programme or intervention
description
O.1.6 Section F: Results and conclusions
O.1.7 Section G: Methods - study method
O.1.8 Section H: Methods - groups
O.1.9 Section I: Methods - sampling strategy
O.1.10 Section J: Methods recruitment and
consent
O.1.11 Section K: Methods - data collection
O.1.12 Section L: Methods - data analysis
O.1.13 Section M: Quality of study - reporting
139
O.1.14 Section N: WoE A: Quality of the
study - methods and data
O.1.15 Section N: WoE B: Appropriateness of
research design for review question
O.1.16 Section N: WoE C: Relevance of
particular focus of the study to review
question
O.1.17 Section N: WoE D: Overall weight of
evidence this study provides to answer this
review question?
O.1.18 Reviewing record
O.2 Please use this space here to give any
general feedback about these data extraction
guidelines
O.2.1 Details
O.3 Please use this space to give any feedback
on how these guidelines apply to your Review
Group's field of interest
O.3.1 Details
Source: EPPI-Centre (2007) Review Guidelines for Extracting Data and Quality Assessing
Primary Studies in Educational Research. Version 2.0 London: EPPI-Centre, Social Science
Research Unit. Retrieved March 26, 2015 from
http://www.civilservice.gov.uk/networks/gsr/resources-and-guidance/rapid-evidence-
ssessment/how-to-do-a-rea
