Final Action Research Report
-
date post
14-Sep-2014 -
Category
Documents
-
view
4.578 -
download
0
description
Transcript of Final Action Research Report
Identifying Process Gaps Through Evaluation
Suzie Rose
University of Colorado, Denver
July 26, 2010
Identifying Process Gaps Through Evaluation
Introduction
I am an instructional designer in the training department of Edelweiss Communications.
Throughout the duration of this study, I held a leadership role in an initiative to improve the
extent to which approximately 2,000 existing customer-facing employees stayed abreast of
company changes. To do this, I had to analyze the extent to which employees were currently
staying abreast of changes, possible causes of low performance, and implement possible
solutions for process improvements.
First, it is necessary to define why Edelweiss Communications’ customer-facing
employees must stay abreast of company changes. Their goal is to provide exceptional customer
service experiences at all times. This requires that they have extensive knowledge and skills
about policies, internal and external procedures, products, services, and more. As the company
changes and grows with updates and new initiatives, existing employees must increase their
knowledge and skill sets. To ensure employees have the resources necessary to do this,
Edelweiss Communications distributes weekly updates. Weekly updates outline the details of
company changes and will be discussed in more detail in a subsequent section of this document.
The training department that I am part of is responsible for supporting delivery of the
information within weekly updates. It is responsible for identifying if training is needed and if
so, determining the method in which it is delivered. This study will help identify the extent to
which customer-facing employees stay abreast of company changes so the training department
can assess possible gaps and implement process improvements.
Problem Statement
Upon commencement of this study, Edelweiss Communications did not measure the
extent to which customer-facing employees stayed abreast of company changes. This lack of
measurement placed the training department in a precarious position. It was unable to validate if
the processes used to support the delivery of information in weekly updates was effective or
valuable. Without this information, the training department struggled to show its contributions
to the business organization. This situation held the potential to affect decisions regarding
resources and budget allocation to the training department.
Edelweiss Communications was also unable to identify employees who were not staying
abreast of new information. Employees who do not stay abreast of new company information
need additional coaching to prevent occurrences where they inadvertently provide
misinformation to customers. Customers who receive misinformation are likely to leave a
company which ultimately costs Edelweiss Communications valuable revenue. In fact, a study
by Frederick Reichfeld and W. Earl Sasser Jr., showed that a 5% increase in customer loyalty
can increase profits from 25% up to 85% by extending the average customer relationship lifetime
(1990). This significant difference clearly indicates that customer retention and loyalty is
valuable to any company.
Purpose
The purpose of this study was to help customer-facing employees stay abreast of
company changes. There will be multiple phases of this study. However, this phase of the study
(a) defined the problem, (b) measured the current state of employee knowledge as related to
company changes, (c) conducted a focus group to collect qualitative data from key players, (d)
analyzed the results of a knowledge based assessment to create a baseline of data, (e) compared
and contrasted findings from the focus group, and (f) determined possible causes of low
performance. It laid the foundation for Edelweiss Communications’ ultimate goals of increasing
efficiencies and improving processes; possibly through methods explored in the Literature
Review section of this document.
To put this study into action, I evaluated the extent to which employees were currently
staying abreast of company changes at the time of the study. I then analyzed the data collected
to identify gaps in processes. According to Ron Kurtus, the founder of the School for
Champions, “By organizing or re-organizing your operations and processes, you can make your
business a more effective machine” (2002). Kurtus’ viewpoint echoes that of Edelweiss
Communications and illustrates its aspiration to create an efficient and well-oiled organization.
To achieve this aspiration, Edelweiss Communications chose to utilize Level Two assessments
as defined by Donald L. Kirkpatrick (1959). These assessments served as the source of baseline
data. Throughout this study, a synthesis of Level Two assessments and process improvements
culminated into an exploration of the relationship between evaluation and process improvements
in corporate settings. It helped assess possible gaps in Edelweiss Communications’ processes
and propose improvements.
Intended Audience
This initiative had a variety of audiences. The primary audience for this study was Dr.
Jennifer VanBerschott and fellow students in the Research in Information and Learning
Technologies, ILT 6720 class. The secondary audiences included the management staff of
Edelweiss Communications’ content management and training departments including managers,
training managers, curriculum developers, content developers, project managers, vice presidents,
and presidents of both departments.
Note: Participation was not compensated beyond normal salary and benefits currently received
at the time of the study.
Research Questions
Although future areas of focus and process improvement initiatives will follow this study,
the focus of this portion of the study was identifying gaps in processes and analyzing possible
causes for them. To identify and analyze these items properly, an overarching research question
and guiding questions were developed to help keep this study focused.
Overarching Question
To what extent are customer-facing employees staying abreast of company changes?
To help customer-facing employees stay abreast of company changes, it is important to
establish a baseline of data. Focusing this phase of the study on an attempt to capture baseline
data helps the long-term goal of helping employees stay abreast of company changes.
Guiding Questions
1. What processes are involved in helping customer-facing employees stay abreast of company changes? To what extent are the processes involved?
Processes are a possible cause and distinct antecedent of the extent to which employees
stay abreast of company changes. In many cases, the processes involved have an extensive
amount of influence on the transfer of knowledge. This includes those within company
procedures and those involved in the delivery methods of the information.
2. What are the most effective methods of evaluation to help identify gaps in processes?
For the purposes of this study, Edelweiss Communications used Level Two assessments
as defined by Kirkpatrick’s model of training evaluation. This question aimed to identify if the
method is effective or if another phase of study should be implemented with a different data
collection method.
3. What are possible process improvement methodologies that may improve the state of the
situation?
Since Edelweiss Communications had not yet defined the type of process improvement
methodology it wished to implement, it was decided that research regarding industry
methodologies be explored throughout this phase of study. These findings are found in the
Literature Review section of this document.
Although the overarching question remained the same throughout the duration of the
study, the guiding questions changed. The second guiding question originally read, “What
correlations and patterns exist between employee assessment scores and the weekly update
delivery methods?” Upon further inspection, it was clear that this was an inherent feature of
data analysis and would naturally be answered throughout the analysis process.
Figure 1
Instead, the following question was added, “What are possible process improvement
methodologies that may improve the state of the situation?” Throughout the literature review
conducted during this research study, it was found that many methodologies are currently in
practice to help companies improve processes and efficiencies. Because the ultimate goal of all
phases of this study is to help Edelweiss Communications’ employees stay abreast of company
changes, methodologies on how to do so held a high level of importance. Although Edelweiss
Communications must find its own tailored methodology and further investigation will be done
in subsequent portions of this study, it is important to research alternative methodologies of
process improvements while the foundation of the study is built – in phase one.
Context of the Study
Process All company changes and initiatives funnel through Edelweiss
Communications’ content management department which creates a
report summarizing each week’s updates. This is distributed to
leadership staff at each location within the company. The leadership
staffs then redistribute the information to site supervisors and trainers
who deliver it through team meetings, instructor led training sessions,
or eLearns. The training department supports this process by defining
delivery methods and creating content if applicable. Employees have
scheduled time to participate in team meetings, instructor led
trainings, and eLearns. Participation in these activities is a job
requirement and is enforced by employees’ supervisors. Refer to
figure one for a flow chart of this process.
Demographics
To provide details about Edelweiss Communications’ demographics, a community profile
was necessary. A community profile, as described by Stringer (2007), is a snapshot of the
context in which the investigation is placed. The snapshot of Edelweiss Communications
includes communication methods, locations, company organization, and more. Employees
communicate with customers through phone conversations, emails, instant messages (IMs),
door-to-door interactions, and in-store interactions. Refer to Figure 2 for a visual representation
of this flow. They are located in eight locations within the United States. Some employees work
directly for the corporation; some are contracted through third-party agreements. Employees
include men and women of all age groups and nationalities. The median age is 28. These items
were considered throughout this study to help Edelweiss Communications identify trends and
gaps that may be related to demographics.
Background
Weekly updates have been distributed in the process described for many years. As the
company grows and responsibilities increase, Edelweiss Communications fears that attention to
these updates is dropping in priority for both employees and their supervisors. Retention of
weekly updates is assumed as a job responsibility, and may therefore receive little attention; it
Figure 2
does not seem to have a significant priority in terms of participation or enforcement of
participation. This could result in employees falling out of touch with weekly updates and could
be a cause of possible performance opportunities. By reassessing the priority and accountability
levels of the weekly updates, Edelweiss Communications may be able to stimulate change in this
situation.
Literature Review
A vital component of this study was to research existing information about the
relationship between evaluation and process improvements in corporate settings. Being familiar
with existing studies and literature helps one situate the study into existing knowledge and
hopefully contribute to it (Knopf, 2006). The key component of this familiarity for Edelweiss
Communications was the ability to situate this study into existing knowledge. The literature
review was extremely helpful in researching, identifying, and defining methodologies of process
improvements that are currently in practice in corporate environments. This insight provided a
course of action for a subsequent phase of the study: implementation of process improvements.
This comparison of viewpoints aligns with Stringer’s stance that literature reviews set the stage
for comparisons of viewpoints from existing literature with those one discovers through their
own studies (2007).
Literature Search Questions
Throughout the process of this literature review, articles and studies similar in culture and
context to those of Edelweiss Communications’ were searched for; this helped explore and
substantiate the study’s core concerns. This study explored the relationship between evaluation
and process improvements in corporate settings to help assess possible gaps in Edelweiss
Communications’ processes and propose improvements. Because this study has never before
been conducted within Edelweiss Communications, it was necessary to expand the focus of the
literature review to a broader stage, but also contain it within its core concerns. To do so,
literature search questions were created to break down and emphasize the study’s core concerns:
1. What links and correlations exist between evaluation techniques or methods and process
improvements in corporate settings?
2. What are the most effective methods of evaluation to help identify gaps in processes?
3. What are the most effective methods of implementing process improvements?
Search Procedures
All content searches started online to allow for easy duplication and cross-referencing of
resources. As a student of the University of Colorado Denver, I began my search within the
Auraria Library Database search engine. From there, I navigated to the Article Databases &
Indexes where I selected the “Companies” category. After sorting for databases related to
corporate training, I found a niche in the EBSCO database. In my search, I used the following
combinations of terms: (a) “Process Improvement + Assessment,” (b) “Process Improvement +
Experimental Design,” (c) “Process Improvement + Study,” (d) “Process Improvement +
Identify Gaps,” (e) “Process Improvement + Methodologies,” (f) “Process Improvement +
Lean,” (g) “Process Improvement + Kaizen,” (h) “Six Sigma,” (i) “DMAIC,” (j) “Evaluate +
Kirkpatrick + Knowledge.” I was not familiar with some of these terms at the beginning of the
literature review, but rather discovered them as I performed research inquiries. These include
Six Sigma, DMAIC, Lean, and Kaizen.
Literature Review Findings
Overview
As previously noted, the literature review was helpful in researching, identifying, and
defining methodologies of process improvements that are currently in practice in corporate
environments. Services are bound by time in terms of the processes that lead to the delivery of
an outcome that benefits a customer (Fraser & Fraser, 2009). As competition increases,
customers’ expectations about products and services increase as well. Statistics show that
retaining satisfied and loyal customers is more profitable than acquiring new ones (Pakdil,
Özkök, Dengiz, Kara, Selvi & Kargı, 2009). To retain customers, it is important that companies,
Edelweiss Communications included, evaluate their processes to improve flows and efficiencies
wherever possible. Becoming truly customer-centric is essential for the success of all businesses
(Naumann & Giel, 1995). As the American culture continues to shift its focus from product-
centric to customer-centric, businesses must keep pace with the shift. Therefore, companies look
for ways to improve their organizational efficiencies to meet customer expectations (Jones,
Parast, & Adams, 2010). Although some company efficiencies are not directly customer-facing,
most contribute in a large way to a customer’s experience with a company, service, or product.
A part of customer-centricity can be achieved through process improvements. In fact,
Continuous Process Improvement (CPI) has attracted significant attention from corporate and
industry environments (Li & Rajagopalan, 2008). Improving processes increases company
efficiencies, employee job satisfaction, and the delivery of products and services to customers.
Eliminating errors that result in customer dissatisfaction improves processes (Pakdil et al., 2009).
But just how do companies eliminate errors and improve processes?
Kaizen
The goal of CPIs is to evaluate and improve a current state
(Tarantino, 2009). This ties directly to Edelweiss Communications’
goal. Edelweiss Communications wants to evaluate and improve the
current state of the extent to which employees stay abreast of
company changes. This type of evaluation and improvement takes a
great deal of focus and attention. A successful corporate cultural
movement that helps support CPI is Kaizen. Carter & Carmichael
(2009) explain that Kaizen is the Japanese word for good change (see
Figure 3) and it encompasses a commitment to continual and steady
improvement for management and employees. Weaknesses are actively sought out, documented,
and remedied via dedicated discussion and actions that are part of the normal workday, rather
than being occasional or external.
Ortiz (2006) explains that the philosophy of Kaizen is the foundation and driver for
process improvement initiatives. It helps build a foundation for change that will enable a
company to sustain process improvements for its lifetime. A successful Kaizen program has a
governing committee, communication boards and newsletters, regular meetings, and event
tracking.
The following sections of this review outline the industry leaders in process improvement
methodologies that are possible because of Kaizen: Six Sigma, Lean, and Lean Six Sigma. Any
or all of these methodologies have the potential to help Edelweiss Communications improve the
extent to which employees stay abreast of company changes.
Figure 3
Six Sigma
Six Sigma is a data-driven philosophy used to drive management decisions and actions
across an organization (Caulcutt, 2001). Data is used here as a subjective term and can include
various observable information of both a quantitative and qualitative nature. Motorola generated
the concept in 1981 and pioneers using Six Sigma helped refine and develop it (Bendell, 2005).
As the first US organization to implement Six Sigma, General Electric contributed to broadening
its use across manufacturing, design and transactional areas (Grant, 2008). These companies are
examples of pioneers which implemented the ideas and concepts of the Six Sigma methodology
to establish concrete financial advantages.
Six Sigma uses a structured method to solve business problems (Jones et al., 2010)
through the use of a standard framework known as Define-Measure-Analyze-Improve-Control
[DMAIC] (Gygi, DeCarlo, & Williams, 2005). Refer to Figure 4 for a visual representation of
this flow. Although this is the most recognized framework, and some literature refers to it alone,
Six Sigma uses another framework known as Define-Measure-Analyze-Design-Verify
[DMADV] (Jones et al., 2010) – Refer to Figure 5 for a visual representation of this flow.
Figure 4. DMAIC Framework
Figure 5. DMADV Framework
The DMAIC framework is designed to help eliminate waste and defects to improve on
existing processes, whereas the DMADV framework is designed to develop new products,
services, or tools (Jones et al., 2010). Each framework follows the phases within it in a linear
manner. For detailed definitions of each phase, refer to Appendix A.
Although Six Sigma is the most recognized method of process improvement, it receives
mixed reviews from critics. Motorola and General Electric report significant savings as a result
of their Six Sigma initiatives (Pande, Neuman, & Cavanagh, 2000). Yet others fear it will fail
because of intense business competitiveness (Jones et al., 2010). In fact, one study found that
companies that adopted Six Sigma underperformed in the stock market and achieved only
incremental gains (Bernett & Nentl, 2010). However, of all the literature included in this review,
that is the only reference to less than favourable results. Bernett and Nentl (2010) also contradict
others by stating that in 1998 Motorola’s second quarter profits were minimal and the company
eliminated 15,000 jobs.
Despite this unfavourable review, Six Sigma is generally respected and considered
successful in most corporate environments. Six Sigma is appealing to corporate organizations
because it is built upon a foundation of statistical methods to improve quality, and it can define
success in a quantifiable manner (Tarantino, 2009). This is of great importance to organizations
such as Edelweiss Communications that aim to quantify return on investment and track
improvement rates.
Lean
Another prevalent CPI method is Lean. John Krafcik coined the term ‘Lean’ to describe
the new production techniques at Toyota after World War II (Bendell, 2005). Lean is defined as,
“The systematic pursuit of perfect value through the elimination of waste in all aspects of the
organization’s business processes” (Bendell, 2005). It incorporates five basic principles: (a) the
elimination of waste, (b) the identification of the Value Stream, (c) the achievement of Flow
through the process, (d) pacing by a pull (rather than push) signal, and (e) the continuous pursuit
of perfection (Womack, Jones, & Ross, 1990). Refer to Appendix B for further explanation of
Lean definitions.
Both Six Sigma and Lean are built upon the foundation of Kaizen and business process
improvement (Bendell, 2005). Unlike Six Sigma, Lean focuses on concepts and values as its
structure rather than phases of a process. Another difference is that Lean is heavily centered on
process analysis and flowcharting/mapping techniques, rather than data (Grant, 2008). Like Six
Sigma, Lean focuses on increasing efficiencies and decreasing waste.
Also like Six Sigma, Lean receives mixed reviews from critics. Bernett and Nentl (2010)
report that Lean has been well received by corporate organizations since Womack and Jones
captured organizations’ attention in 1996 with the release of their book, Lean Thinking.
However, other critics claim that Lean has a limited application range; there is general agreement
that it is well suited to companies with similar characteristics to the automotive industry rather
than corporate organizations (Lasa, de Castro, & Laburu, 2009).
However a recent contribution to the Lean model includes Value Stream Mapping (VSM)
which focuses on the analysis and improvement of discontinuous flow lines. It includes five
phases: (1) selection of product family, (2) current state mapping, (3) future state mapping, (4)
definition of working plan, and (5) achievement of working plan (Lasa et al., 2009). This adds a
more concrete plan of implementation to the concept and allows more organizations to
implement the Lean model. By comparing the current state and the identified future state of a
process, organizations are better able to define and achieve a working plan.
Lean Six Sigma
In a quest for a truly holistic approach to business and process improvement, many
corporate organizations attempt to combine Six Sigma and Lean into practice simultaneously.
Lean practices and techniques focus on streamlining processes, whereas Six Sigma practices and
techniques help identify and eliminate root causes of problems (Evans & Lindsay, 2005). An
effective combination of these methods would create an invaluable process improvement
solution for many organizations.
A case study presented by Fraser and Fraser (2009) discusses the successful
implementation of a Lean Six Sigma model by the National Grid on a project concerning
renewal prompts for vehicles. In this case, the organization used both models concurrently and
matched the phases side by side. This is one variation of combining Lean and Six Sigma.
In an attempt to uncover patterns of implementation between Lean Six Sigma, Shah,
Chandrasekaran, and Linderman (2008) conducted a research study using implementation and
performance data from a sample of 2511 industrial plants. They found that implementation of
Lean practices improved the likelihood of implementing Six Sigma and indicated a significant
difference in the performance levels between those who did and those who did not implement
these models. However, there was no report of how the organizations combined the two models.
Since there was not a structured method of how to combine the models, there could be great
variation between these organizations.
Tarantino (2009) supports the combination and explains that combining the two provides
a powerful best practice for process and customer service improvements if done so in the hands
of focused experts. However, the reviews are mixed. Bendall (2005) states, “It would be clearly
desirable if a single process-improvement-based approach was available that effectively
combined the approaches” (p. 975). He continued to explain that most attempts of combination
resulted in a situation where one of the two methods was taken as a dominate, and the other as a
subordinate. He pointed out that this is not a particularly good approach, since both methods
have unique features and benefits that only more effective combinations would retain.
Kirkpatrick
Because Edelweiss Communications began its process improvement initiative by
evaluating its current state using Kirkpatrick’s learning evaluation theory, it is necessary to
include literature regarding this facet as well. The goal of this inclusion is to benchmark
Edelweiss Communications’ choice of evaluation with other organizations. This provides the
opportunity to compare Edelweiss Communications with those considered to be best-in-class.
Benchmarking helps draw comparisons and determine which processes are adaptable to other
organizations (Carter & Carmichael, 2009).
Kirkpatrick’s learning evaluation theory consists of four levels of training evaluation: (a)
reaction, (b) learning, (c) behaviour, and (d) results (1959). Rowden (1998) writes, “These
often-cited levels describe a rubric that is often used to evaluate programs. It is the last two
levels—behaviour change and results—that ROI is concerned with” (p. 32). Level One
evaluation measures employee satisfaction with training. Level Two evaluation assesses the
acquisition and retention of declarative knowledge, procedural knowledge, and attitude changes
(Lupton, Weiss, & Peterson, 1999). Levels Three and Four, behaviour change and results, are
the measurable changes we can see in employees and the benefits a company enjoys from
employees implementing what they have learned (Rowden, 1998). Based upon these definitions,
Kirkpatrick’s model may be effective in helping execute Six Sigma’s Measure phase.
Quality of Literature
Overall, the quality of the literature used in this review was very strong. All articles were
peer reviewed and many were from popular notable publications such as the Journal of Applied
Statistics and International Journal of Production Research. The literature provided excellent
information about what is currently happening in corporate organizations. For each CPI method,
information about its origin, its core structure or framework, and how well it works or is being
received by corporate organizations was available. A broad variety of angles or points of view
were also used in various articles. For example, Grant’s article (2008), provided overviews of
Six Sigma, Lean, and a few less recognized methodologies. Jones et al. (2010) defined and
described Six Sigma and then explored best practices for implementation. Bernett and Nentl
(2010) conducted a survey about continual improvement and explored the findings and opinions
of those surveyed. Nuran and Frasier (2009) as well as Matson and Stauffer (2009) each
explored findings of case studies. This variety allowed for an exceptional sample of articles.
Combining these articles into a whole helped build a multi-faceted view of the existing
relationships between evaluations and process improvement efforts.
However, some intrinsic limitations were present due to the lack of time allotted to
perform the literature review. If more time were available, the opportunity would have been
available to expand data collection methods to include visits to libraries and searches through
tangible periodicals. Although the online search allows for easy duplication of the search
process, it also creates a limitation of data to be collected.
The literature also showed a limit in participants represented. All authors were
journalists or researchers rather than facilitators, business leaders, or other participants that may
have been able to share first-hand experiences with the success or failure of a specific CPI
method. The inclusion of these points of views would have provided a greater diversity to the
study and would have possibly brought about different results.
Yet, the largest limit of this literature review was the lack of evaluation techniques
explored. This is caused by both a lack of time to conduct research on what those techniques
may be, and a lack of founding knowledge about evaluation techniques. This created a situation
in which key search terms were not as strong as they could have been. This could have been
resolved if more time were allotted for the literature review. However, the combination of both
situations (lack of time and knowledge) results in the inability to research evaluation techniques
to a greater extent. In literature reviews that will be conducted in future phases of this research
study, a section devoted to evaluation techniques will be implemented.
Gap in Literature
Most of the literature in this review focused on describing and explaining the concepts,
methodologies, and tools of each CPI method. They also discussed strengths, weaknesses, and
success factors. However, individual company needs, goals, and context were excluded. Further
information on organizational specifics is found in strategy literature which forces business
leaders to complete in-depth analysis and cross referencing between the two areas. Because this
is a timely and arduous task, most business leaders select a methodology for their company based
on suggestions from friends, colleagues, gurus, and practitioner publications (Charlesworth,
2000). This indicates that there is a gap in the literature that would help business leaders assess
their circumstances and match it with an appropriate process improvement methodology. In fact,
Thawesaengskulthai and Tannock note that without an explicit decision support, organizations
may base their selection on subjective judgments. A process improvement methodology selected
through subjective judgements may not provide a company with all the features and benefits it
needs.
The literature also failed to explain how a company would implement a process
improvement methodology. For example, the Define-Measure-Analyze-Improve-Control
framework of Six Sigma provides a detailed breakdown of what to do, but fails to explain how to
do so. Edelweiss Communications’ study is filling this gap by providing an explicit example of
a study.
Literature Review Summary
This literature review explored existing content about how evaluation of training
initiatives helps corporate organizations identify and implement process improvements. Models
and concepts explored include Kaizen, Six Sigma, Lean, and Lean Six Sigma. The literature
review allowed for study of each methodology in terms of origin, structure or framework, how
they work, and the extent of their success.
The review also showed findings on Kirkpatrick’s model of training evaluation in an
attempt to find correlations between Edelweiss Communications’ current method of evaluation
and how that may fit into the context of process improvements. Although no articles combined
these issues within their own context, uniting these articles through this review helped build a
multi-faceted view of the existing relationships between evaluations and process improvement
efforts. The following items break down the literature review as compared to the literature
search questions introduced at the beginning of this section.
1. What links and correlations exist between evaluation techniques or methods and
process improvements in corporate settings?
As noted in the Literature Gap section of this document, the articles failed to explain
exactly how a process improvement methodology is put into action. Because of this gap,
the links and correlations between evaluation techniques or methods and process
improvements are also undocumented. However, the literature on Kirkpatrick’s learning
evaluation theory suggest that Level Two assessments are a successful method for
capturing data regarding the current state of a situation. But ultimately, links and
correlations cannot be established based on the literature review alone.
2. What are the most effective methods of evaluation to help identify gaps in
processes?
As previously noted, methods of evaluation and process improvements were not
identified in the literature. Therefore links between these two concepts were not
explored. However, the literature on Kirkpatrick’s learning evaluation theory suggest
that Level Two assessments are a successful method for capturing data regarding the
current state of a situation.
3. What are the most effective methods of implementing process improvements?
The literature review outlined the most prevalent methods of process improvements
found. This includes Six Sigma, Lean, and Lean Six Sigma. The literature suggests that
all three of these methods are effective, however, a delicately balanced combination of
Six Sigma and Lean seem to hold the most promise as a complete and holistic approach
to process improvements.
Methods
My primary data gathering method followed the protocol of statistical-observational
study. To adhere to the statistical approach of the study, I collected quantitative data from a
specific population (Roberts, 2010). I created and distributed a 30-question electronic
assessment. The questions were based on factual knowledge and molded to comply with Level
Two evaluations as defined by Kirkpatrick. This fulfills the statistical requirements of this type
of study. The analysis of this data focused on the observational aspect of my statistical-
observational study. As defined by Roberts, (2010) “In an observational study, the sample
population being studied is measured, or surveyed, as it is…Data is simply gathered and
correlations are investigated” (p. 1). I did not manipulate any data, I simply collected the
information, restructured it to ensure proper analysis, and identified patterns or correlations.
My secondary data gathering method was conducted in a focus group where I asked
interview questions to collect viewpoints and thoughts from key players in the study. This
exemplifies an action research aspect because it incorporates members of the community to help
identify and solve problems.
The data collected from this initiative contributes to Edelweiss Communications’ ability
to assess and identify gaps in additional areas such as the effectiveness of weekly updates,
training content, and delivery methods. By reassessing the priority and accountability levels of
the weekly updates, Edelweiss Communications is now better able to stimulate change in this
situation.
However, the study may cause a conflict of goals for customer-facing employees. If we
increase the accountability of their responsibilities regarding staying abreast of company
changes, they may become overwhelmed. Because they have restrictions placed upon them in
terms of time management and performance, enforcement of responsibilities (that may not have
been enforced in the past) may hinder their performance in other areas, thus causing frustration
and possible employee attrition.
Participants
The primary participants of this study were 1,918 customer-facing employees employed
by Edelweiss Communications. There was not direct communication with them because their
feedback was collected via a 30-question electronic assessment through a local learning
management system (LMS). It also included two training managers employed by Edelweiss
Communications with whom I conducted a focus group.
I led the project, created the assessment, collected and analyzed data, and compiled this
report. The content management department created the weekly updates mentioned throughout
this study and delivered weekly conference calls to communicate its content to site leaders.
Trainers and supervisors delivered the information in the weekly updates to customer-facing
employees. The training department leaders sponsored the project and communicated data
results and analysis to their superiors and the content management department up to and
including presidents and vice presidents.
Data Gathering
Quantitative
As previously mentioned, my primary data gathering method followed the protocol of
statistical-observational study. To adhere to the statistical approach of the study, I collected
quantitative data from approximately 1,918 existing customer-facing employees. I created and
distributed a 30-question electronic assessment through an LMS. Employees had 30 minutes
scheduled in their shift to complete the assessment. The 30-minute allocation may have been
scheduled anywhere within a two week span. Participation in this assessment is a job
requirement with enforcement from employees’ supervisors.
Software capabilities prevented participants from skipping questions or returning to
previous questions. Participants could however, exit the assessment and re-enter at the location
where they left.
The questions focused on factual information, were directly related to the information
within the weekly updates, and covered information about company changes spanning Quarter
One 2010 (January – March).
Qualitative
I conducted one focus group with two training managers. These training managers are
located in different sites and were selected because they oversee the largest variety of site
locations. I facilitated this over the phone on a group conference call as defined by the training
managers’ schedules for the duration of one hour. I recited disclosures to them before the focus
group began so they could understand the impact of their participation as listed in Appendix C.
I already had an established rapport with these individuals through previous working
relationships and was therefore able to help participants feel that they could say or express what
they really felt or thought (Stringer, 2007). Refer to Appendix D for focus group questions.
Ethical Procedures
For the quantitative aspect of this study, participants were notified that data from the
assessment would be collected for company informational purposes. They did not have the
option to refuse participation because the assessment scores are also used for other purposes such
as individualized coaching. However, they did have the option to exclude their assessment
results from the data used in this study. Any and all data was and will be distributed solely
within Edelweiss Communications on a need-to-know basis only. Data distributed throughout
Edelweiss Communications was provided to leadership staff only and is regulated by internal
records management policies and procedures.
Data distributed to Dr. Jennifer VanBerschott and fellow students in the class was altered
with pseudonyms or placeholders to protect the identity of the participants, the locations of the
sites, the company name and profile, etc. Due to the vast amount of raw data (2,000 participants
x 30 questions = 60,000 lines of data) and the fact that nearly all information within it was
identifying, it was excluded from the final submission for this class. Instead, I used source data,
which is the restructured data that served as my source for analysis.
As previously noted, for the qualitative aspect of this study, I provided full disclosures as
outlined in Appendix C. The disclosures listed (a) purpose of the study, (b) why the members of
the focus group were selected for participation, (c) the members of the study who would be privy
to their responses, (d) their options for participation, and (e) procedures for withdrawal.
Checks for Rigor
To ensure the outcomes of my research remain
balances (Stringer, 2007). Stringer (2007) states,
multiple sources of information are incorporated” (p. 58).
By using both quantitative and qualitat
In the statistical-observational portion of my study, I incorporate
checking (Stringer, 2007). Participants
level of involvement. Customer-
Supervisors can review the results of their own assessment and the results of their direct reports’
assessments. The training and content management departments and above can review all
results. The project lead and sponsor (and above if applicable) in the training department can
review all raw data, analysis, and reports.
To ensure the outcomes of my research remained trustworthy, I incorporated multiple checks and
Stringer (2007) states, “The credibility of a study is enhanced when
multiple sources of information are incorporated” (p. 58). This is the concept of triangulation.
By using both quantitative and qualitative data, I was able to implement this concept.
observational portion of my study, I incorporated the concept of member
checking (Stringer, 2007). Participants were, and are, able to view their scores based on their
-facing employees can view the results of their own assessment.
Supervisors can review the results of their own assessment and the results of their direct reports’
training and content management departments and above can review all
results. The project lead and sponsor (and above if applicable) in the training department can
review all raw data, analysis, and reports. Figure 6 clarifies this concept visually.
Project Lead and Sponsor
Training / Content Mgmnt
Supervisors
Participants
multiple checks and
“The credibility of a study is enhanced when
This is the concept of triangulation.
ive data, I was able to implement this concept.
the concept of member
able to view their scores based on their
facing employees can view the results of their own assessment.
Supervisors can review the results of their own assessment and the results of their direct reports’
training and content management departments and above can review all
results. The project lead and sponsor (and above if applicable) in the training department can
clarifies this concept visually.
Figure 6
I kept all records of raw data, reports, analysis, communications, and focus group notes to
provide and show documentation of the procedures followed during this study to create to fulfill
the concept of Dependability (Stringer, 2007).
Data Analysis Procedures Quantitative
I used Crystal Reports to capture data from all the employees who completed the
assessment. Crystal Reports generated data about (a) employee job titles (supervisor or
subordinate), (b) employee locations, (c) scores, and (d) which questions employees passed or
failed.
By utilizing the functions in Microsoft Excel such as sort, import, and pivot tables, I
restructured the data from Crystal Reports to emphasize the most relevant information. The
restructured data served as my source data for analysis and included (a) average scores of the
assessment (by location and company-wide), (b) pass/fail percentages of the assessment (by
location and company-wide), and (c) pass/fail percentages of each question (by location and
company-wide). Please refer to Appendices E-G for complete details. In addition to this
information, I also collected further data in various ways. I (a) searched archival scheduling
records to determine the delivery method as related to each question, (b) collected the wording of
each question as it appeared in the assessment from the development source files, and (c)
collected the correct answer for each question from the development source files. Items b and c
are directly related to identifiable company information; therefore they are listed in Appendix G
as placeholders.
The analysis of this data focused on the observational aspect of my statistical-
observational study. As previously noted, I did not manipulate any data, I simply collected the
information, restructured it to ensure proper analysis, and identified patterns or correlations. The
data I analyzed included (a) average scores of the assessment (by location and company-wide -
listed in Appendix E), (b) pass/fail percentages of the assessment (by location and company-wide
- listed in Appendix E and F), and (c) pass/fail percentages of each question (by location and
company-wide - listed in Appendix G).
To analyze the data, I identified the aspects of the information that would help my
audience understand the meaning of the data (Stringer, 2007). This falls in line with the method
of analyzing key experiences as defined by Stringer. I focused on events and trends that impact
my stakeholders. I began by categorizing and coding my data to identify events or experiences
that may have some significance. While doing this, I searched for relationships between the
following items: (a) the pass/fail percentage of the assessment and location (see E and F), (c) the
pass/fail percentage of each question and location (see Appendix G), and (d) the pass/fail
percentage of each question and delivery method (see Appendix G). This helped me find themes
in the variety of relationships listed, which in turn, helped me identify elements that compose the
experience for participants of the study.
Qualitative
To identify significant features and elements of my participants’ experiences, I analyzed
my qualitative data through means of categorizing and coding as defined by Stringer (2007). To
do this, I used the verbatim principle: using terms and concepts from the participants themselves
(Stringer, 2007). I then (a) reviewed the collected data, (b) unitized the data, (c) categorized the
data, (d) identified themes, (e) organized a category system, and (d) developed a report
framework (Stringer, 2007).
Schedule
To ensure timely completion and organization throughout the duration of this project, I
implemented and followed the schedule listed in the below table (Table 1 – Schedule).
Table 1 - Schedule Item Task Type Completion
Submit Action Research Proposal - Draft
Milestone June 21, 2010
Revise Action Research Proposal Development June 22, 2010 – June 26, 2010
Submit Action Research Proposal Milestone June 26, 2010
Write Interview Questions Data Collection and Analysis June 21, 2010
Focus Group to collect qualitative data
Data Collection and Analysis June 29, 2010
Run reports to collect quantitative data
Data Collection and Analysis June 28, 2010 – July 2, 2010
Restructure reports/create source data Data Collection and Analysis June 28, 2010 – July 2, 2010
Analyze quantitative data Data Collection and Analysis July 3, 2010 – July 9, 2010
Analyze qualitative data Data Collection and Analysis July 3, 2010 – July 9, 2010
Write Literature Review Development June 27, 2010 – July 4, 2010
Submit Literature Review – Draft Milestone July 5, 2010
Revise Literature Review Development July 6, 2010 – July 10, 2010
Submit Literature Review Milestone July 10, 2010
Write Data Analysis Report Development July 11, 2010 – July 18, 2010
Submit Data Analysis Report – Draft Milestone July 19, 2010
Revise Data Analysis Report Development July 20, 2010 – July 26, 2010
Submit Data Analysis Report Milestone July 26, 2010
Write Final Research Report Development July 20, 2010 – July 25, 2010
Submit Final Research Report - Draft Milestone July 26, 2010
Revise Final Research Report Development July 27, 2010 – July 31, 2010
Submit Final Research Report Milestone July 31, 2020
No changes to the schedule were required throughout the duration of this study.
Findings
Quantitative
Throughout the trends listed in this section, please note that Edelweiss Communications
considers a score of 80% or higher to be a passing score. The following sections citing exceeds
and fail percentages or vernacular refer to this designation.
Notable Exceeds and Fail Trends
Edelweiss Communications expects that supervisors ensure at least 80% of their teams
pass the assessment. Only 81 of 229 supervisors accomplished this goal. In fact, of 229
supervisors, 2 had 100% of their employees fail. Refer to Appendix F for additional details.
Of 229 supervisors, only 3 had a team larger than 10 and 100% of their employees passed
the assessment. They were able to communicate and reinforce the content contained in the
weekly updates despite their high employee count and were therefore recognized for their
abilities to exceed expectations. Refer to Appendix F for additional details.
These results signify that more employees are failing than exceeding and hint that
customer-facing employees may be struggling to stay abreast of company changes.
Notable Center Trends
In general, pass percentages were low. The highest center (Snapdragon) had 80% of its
employees pass. The lowest center (Rhododendron) had 50% of its employees pass. All centers
had a median average of 67% passed. This data is particularly significant in the sense that it
implies that employees are likely to be struggling to stay abreast of company changes. In fact,
further details reveal that Snapdragon had the highest average scores at 86%. Rhododendron had
the lowest average scores at 79%. These numbers reflect totals, which includes both customer-
facing employees and supervisors. It is also important to note that Snapdragon had the highest
percent passed and the highest scores. Rhododendron had the lowest percent passed and the
lowest scores. Refer to Appendix E for additional details.
Question Level Analysis – Below 80%
Of 30 questions, 8 had less than 80% answered correctly. Refer to Appendix H for an
analysis of causes listed in the order of lowest to highest average score. To protect company
confidentiality, the actual questions are not listed, but only the question number and an analysis
of possible reasons for the low score. This data should show the variety of possible causes and
themes between them.
To analyze the reasons these questions received low success rates, I used the analysis
method as defined by Stringer (2007) called Antecedents and Consequences. To complete this
process effectively, Stringer suggests that facilitators or participants identify (a) the core
problem, (b) major antecedents to the problem, (c) significant factors related to the antecedents,
(d) major negative consequences, and (e) other significant consequences. The remaining data in
this section and in Appendix H was analyzed using these methods.
A more global analysis revealed that more than 25% of the questions were answered
incorrectly. This signified that employees were not staying abreast of company changes. It also
indicated that the procedure in which weekly updates is distributed may not be effective.
Major antecedents of this problem were identified through finding patterns in the possible
causes of low scores for each question as listed in Appendix H. This includes employees (a) may
not have used the Intranet properly, (b) may have expected information to be in different
locations, (c) may not have been using the Intranet at all (d) were not receiving the information
as required in team meetings, (e) were not receiving the information in full, (f) were not
receiving it correctly (g) or may have missed some of the questions due to reading
comprehension founded on (a) improperly written questions, (b) convoluted policies and
procedures, or (c) insufficient reading comprehension skills. This analysis indicates that Level
Two evaluation may be an effective method of evaluation to help identify gaps in processes since
processes are prevalent in some of these findings. It provides numerical data which is vital to
stimulating change in a corporate environment.
Qualitative
The results of the focus group provided great insight into the inner workings of Edelweiss
Communications’ processes. My participants were selected because they have the largest
amount of visibility into possible issues. Refer to Appendix D for focus group questions.
Employees may not receive the information
Throughout the focus group, participants clarified the process of distributing information
in the weekly updates. Participant A explained that supervisors attend weekly update calls, and
are then asked to review the information in team meetings. However, supervisors (a) may not
follow the scheduled training times, (b) may not deliver the content correctly, or (c) may not
cover the information at all. She stated, “Our trainers will show up to train during a team
meeting, and no one is there.” This shows that team meetings are not happening at scheduled
times and signifies that they may not be happening at all. As a result, employees are forced to
use the Intranet as their only resource for the information, rather than learning the information in
team meetings. Participant B agreed that supervisors may not cover the information because
there is a lack of consistency in following protocol. She was only 70% confident that the
messages were communicated. This data clearly indicated that the process of releasing
information from the weekly updates was not effective.
Lack of Priority
After explaining that supervisors attend weekly update calls, and are then asked to review
the information in team meetings, Participant A voiced her concern that the weekly calls are little
more than someone reading the documents on the phone. She explained that as a result,
audiences don’t feel a sense of urgency. She added that there are disconnects between the
delivery of the call and the information reaching customer-facing employees. In fact, people
give out codes for weekly update call attendance; so many supervisors may not even listen to the
calls first-hand.
She continued to explain that there is a lift in participation if the director is enforcing the
priority of the weekly updates; but it must continue to be a priority. When the director enforces
it, it becomes a priority. But they key is that this be driven from executive leaders. Participant B
added that it really depends on the priority of the moment. Priorities of the moment make
weekly updates seem like something that can be pushed aside. She agreed that it needs to be a
consistent message from supervisors, managers, and directors.
This sparked a conversation about ownership. Both participants explained that the
training department acts as a messenger for certain information in weekly updates, but even then,
Training does not own the initiative. Participant A stated, “It is important to the corporate
training teams and operational teams, but it lacks priority and accountability on the front line.”
Not only did this portion of the focus group identify gaps in the current process, but it
also led to ideas to improve the situation. Leadership support is a vital component of making any
process work and seemed to be a gap in the existing process.
Delivery Methods
In regard to the methods of training delivery, Participant B stated that instructor led
training (ILT) has a stronger take-away if done by a trainer (rather than a supervisor) because
trainers have the ability to make sure the transfer of knowledge occurs. To illustrate this point,
she stated, “It’s more focused and intense.” Employees know it has an impact and importance
because it was chosen for “training;” in team meetings, the message doesn’t seem as important.
Since one method of delivery not only has a stronger take-away, but it sends an unspoken
message to employees about the importance, urgency, and priority of the content, a possible gap
in the process may be linked to the unspoken connotations that are tied to each method of
delivery.
Suggestions for Improvements
Participant B continued to explain that to improve the extent to which customer-facing
employees stay abreast of company changes, there must be an increase in consistency in
following protocol. Participant A agreed and stated that Edelweiss Communications as a whole
needs to focus on the weekly updates rather than the weekly update call. She recommended
creating a check and balance system in team meeting environments – a sign-off process to help
develop tracking methods for team meetings. She also mentioned the idea of an online survey
where employees could select topics that they did or did not receive training on. Participant B
agreed that there needs to be a consistent message from supervisors, managers, and directors,
“There needs to be accountability, monitoring, and measurement.” Each of these ideas isolate
and identify key components that could contribute to process improvements and are directly
linked to the research question, “What are possible processes that may improve then state of the
situation?” as found in Appendix D.
Reaction to the Assessment
In reaction to the assessment Edelweiss Communications distributed as a method of
evaluation to help identify gaps in processes, Participant A stated that she hasn’t heard a lot of
noise and that may be because she had not received the results or scores yet. Both participants
agreed that an assessment is not the best way to evaluate if customer-facing employees are
staying abreast of company changes or to find gaps in processes. Rather, they stated that we can
get the best snapshot of gaps in the process by conducting focus groups with employees.
Although my participants had the largest amount of visibility into possible issues, their
feedback is only a part of the first cycle of action research. They are considered one of the
primary stakeholders, but employees and their supervisors (secondary stakeholders) can provide
valuable feedback about the causes and effects of employees failing to stay abreast of company
changes as well. And like the training managers I interviewed, employees and their supervisors
can provide valuable insight into possible solutions and the most effective methods of evaluation
to help identify gaps in processes. The next cycle of action research will include focus groups
with them to help further clarify and add information from a larger pool of stakeholders.
Limitations
By using both quantitative and qualitative data, I was able to implement the concept of
triangulation, which states that, “The credibility of a study is enhanced when multiple sources of
information are incorporated” (Stringer, 2007, p. 58). However, complete triangulation is not
fully achieved at this time. It’s imperative to include qualitative analysis that typifies the points
of views of both primary and secondary stakeholders. At this time, I have only accounted for a
sample of primary stakeholders. There are many additional parties with whom I plan to conduct
focus groups such as employees and their supervisors, the content management team that
develops the weekly updates, and trainers. Once these items are complete and taken into
account, the triangulation of this research project will be much stronger.
Comparison of the Research to Literature Review
The literature review that I conducted explored existing content about how evaluation of
training initiatives helps corporate organizations identify and implement process improvements.
Models and concepts explored included Kaizen, Six Sigma, Lean, and Lean Six Sigma. The
literature review allowed for study of each methodology in terms of origin, structure or
framework, how they work, and the extent of their success.
The review also studied findings on Kirkpatrick’s model of training evaluation in an
attempt to find correlations between Edelweiss Communications’ current method of evaluation
and how that may fit into the context of process improvements.
My study takes into account methods of evaluation, whereas many articles within my
literature review did not. However, the method used in the study thus far, Kirkpatrick’s Level
Two assessments, is not sufficient evidence on which to identify gaps in the process properly.
This issue will be resolved by incorporating more qualitative analysis into further cycles of this
study.
My study failed to examine methods in which Edelweiss Communication can improve
the processes that may not be working in its culture. The literature I studied covered methods
such as Kaizen, Six Sigma, Lean, and Lean Six Sigma effectively and provided a valuable
resource for me to tap into once I reach that phase of the research process. This cycle of research
has clearly shown that improvements to Edelweiss Communications’ current processes are
necessary. The following cycles will gather additional data from various points of view to
reinforce this idea and explore methods of improvement. Once data is collected effectively,
Edelweiss Communications will be in a position to select a process improvement method that
meets the needs of Edelweiss Communications effectively.
Summary of Findings
Quantitative data analysis showed that the success rate for over 25% of the questions was
below expectations. This global observation signified that employees were not staying abreast of
company changes. It also indicated that the procedure in which weekly updates is distributed is
not effective enough. Causes include: (a) employees may not have used the Intranet properly,
(b) they may not have received the information as required in team meetings, and (c) reading
comprehension may have influenced the data collected. This analysis indicated that Level Two
analysis may be an effective method of evaluation to help identify gaps in processes since
processes were prevalent in some of these findings. However, the qualitative analysis revealed
that focus groups including all the primary and secondary stakeholders may provide different and
valuable insight into how and why process improvements may be necessary. The feedback
already collected qualitatively reveals a number of promising improvement ideas.
Implications for Practice
Based on these findings, it is evident that Edelweiss Communications must invest
additional time into collecting qualitative data from a more diverse group of stakeholders. It
must also (a) identify particular processes that are the causes of core problems, (b) select a
methodology of process improvement, (c) design a plan to improve the process, and (d)
implement the plan into action.
This initiative will contribute to Edelweiss Communications’ ability to assess and
identify gaps in additional areas such as the effectiveness of weekly updates, training content,
and delivery methods. By reassessing the priority and accountability levels of the weekly
updates, Edelweiss Communications may be able to stimulate improvements.
Conclusion
The quantitative findings of this study clearly indicate that there is a gap present.
However, this data does not reveal the source of the gap. An analysis of questions breaks down
the findings into more tangible possibilities. For example, the three most prevalent possibilities
found in this analysis include (a) improperly written questions, (b) convoluted policies and
procedures, or (c) insufficient reading comprehension skills. These hypotheses of causes are
helpful, but are made more complete once the qualitative data of this study is integrated into the
findings. The qualitative data gathered in this study revealed that employees may not have
received the information as required in team meetings or may not have received it correctly.
However, the results found within this study are applicable to Edelweiss Communications
only. It is possible to replicate the study in another corporate environment, and is recommended
to do so, but both quantitative and qualitative data are unique to Edelweiss Communications’
unique processes, policies, procedures, and culture. However, because of this tailored approach,
the findings are incredible valuable, if only to a small group of individuals.
An additional limitation is the lack of qualitative data. This is in part due to the (a) lack
of time available to complete this phase of the study, (b) lack of departmental involvement in the
study, and (c) lack of resources which made it difficult to identify and locate individuals who
would be able to contribute to the study. Only a small number of primary stakeholders were
interviewed in this phase of the process. Yet, this phase of the study revealed that other groups
must be brought into the process and their thoughts and ideas must be integrated into the study.
Since this is only phase one of the study, there will be ample time to put this into action, and the
data that was collected provided further justification to do so.
Ultimately, this study has proven beneficial for many parties. The information collected
throughout this study will help the training department identify the strengths and weaknesses of
their participation in the weekly update process. It will also help the content management
department identify possible the strengths and weaknesses of their participation and ownership of
the weekly update process. And finally, it will help employees explain possible low scores; it is
unreasonable to expect employees to achieve high scores on an assessment about information
that was not delivered, or not delivered correctly.
This study has helped Edelweiss Communication take its first steps to help accomplish its
ultimate goal of helping employees stay abreast of company changes.
References
Bendell, T. (2005). Structuring business process improvement methodologies. [Editorial]. [Electronic version]. Total Quality Management, Vol. 16, No. 8–9, 969-978.
Bernett, R., & Nentl, N. (2010). Opinions and expectations about continuous improvement programs. [Editorial]. [Electronic version]. The Journal For Quality & Participation, January 2010, 35-38.
Carter, L., & Carmichael , P. (2009). Best practices go beyond benchmarking. [Editorial]. [Electronic version]. Leadership Excellence, November 2009, 16-17.
Caulcutt, R. (2001). Why is Six Sigma so successful? Journal of Applied Statistics, 28, 301 306.
Charlesworth, K., A question of quality? A survey of Performance improvement initiatives, 2000 (British Quality Foundation, Institute of Management: London).
Dictionary.com. (2010). Retrieved June 21, 2010 from Correlation: http://dictionary.reference.com/browse/correlation.
Dictionary.com. (2010). Retrieved June 21, 2010 from Pattern: http://dictionary.reference.com/browse/pattern.
Evans, J.R. and Lindsay, W.M., The management and control of quality, 6th ed., 2005 (South Western College Publishers: Ohio).
Fraser, N., & Fraser, J. (2009). Lean Six Sigma applied to a customer service process within a commercial finance organization. [Editorial]. [Electronic version]. Management Services, Autumn 2009, 25-31.
Grant, V. (2008). Lean Six Sigma. [Editorial]. [Electronic version]. Management Services, Spring 2008, 20-23.
Gygi, C., N. DeCarlo, and B. Williams, Six Sigma for Dummies, Wiley Publishing, Inc. (2005).
Jones, E. C., Parast, M. M., & Adams, S. G. (2010). A framework for effective Six Sigma implementation. [Editorial]. [Electronic version]. Total Quality Management, Vol. 21, No. 4, 415-424.
Knopf, J. W. (2006). Doing a literature review. [Editorial]. [Electronic version]. PS: Political Science & Politics, January 2006, 127-132.
Kirkpatrick, D. L. (1959) Evaluating training programs, 2nd ed., Berrett Koehler, San Francisco.
Kurtus, R. (2002). Steps to improve or grow your business. Retrieved June 21, 2010 from schoolforchampions.com: http://schoolforchampions.com/business/improvement_steps.htm.
Lasa, I. S., De Castro, R., & Laburu, C. O. (2009). Extent of the use of Lean concepts proposed for a value stream mapping application. [Editorial]. [Electronic version]. Production Planning & Control, Vol. 20, No. 1, 82-98.
Lean Terms & Definitions . (2010). Retrieved April 10, 2010 from The Northwest Lean Networks: http://www.nwlean.net/leandefs.htm#7wastes.
Li, G., and Rajagopalan, S. (2008). Process improvement, learning, and real options. [Editorial]. [Electronic version]. Production and Operations Management, Vol. 17, No. 1, 61-74.
Lupton, Robert A., John E. Weiss, and Robin T. Peterson (1999), “Sales Training Evaluation Model (STEM): A Conceptual Framework,” Industrial Marketing Management, 28, 1 (January), 73-86.
Matson, E., and Stauffer, L. (2009). Developing an assessment tool for two organizations using Six Sigma principles. [Editorial]. [Electronic version]. Engineering Management Journal, Vol. 21 No. 4, 7-15.
Naumann, E., and Giel, K. (1995). Customer satisfaction measurement and management. Wisconsin: ASQ Quality Press.
Ortiz, C. (2006). All-out kaizen. [Editorial]. [Electronic version]. Industrial Engineer, April 2006, 30-34.
Pakdil, F. Özkök, O. Dengiz, B. Kara, I. Selvi, N. Kargi, A. (2009). A systematic approach to reduce human and system-related errors causing customer dissatisfaction in a production environment. [Editorial]. [Electronic version].Total Quality Management, Vol. 20, No. 1, 129-137.
Pande, P.S., Neuman, R.P., & Cavanagh, R.R. (2000). The Six Sigma way: How GE, Motorola and other top companies are honing their performance. New York: McGraw-Hill.
Reichfeld, F. and Sasser, W.E. (1990). “Zero Defections: Quality Comes to Service”. Harvard Business Review, September – October, 1-8.
Roberts, D. (2010). Statistical Studies. Retrieved June 21, 2010 from Statistical Studies: http://www.regentsprep.org/regents/math/algtrig/ats1/statsurveylesson.htm.
Rowden, R. W. (1998). The return-on-investment from training. [Editorial]. [Electronic version]. Business Forum, Volume 27 Number 1, 31-36.
Shah, R., Chandrasekaran, A., & Linderman, K. (2008). In pursuit of implementation patterns: the context of Lean and Six Sigma. [Editorial]. [Electronic version]. International Journal of Production Research, Vol. 46, No. 23, 6679-6699.
Stringer, E. T. (2007). Action Research. Thousand Oaks, California: Sage Publications.
Tarantino, A. (2009). Risk management for the next generation. [Editorial]. [Electronic version]. Industrial Management , january/february 2009, 14-17.
Thawesaengskulthai, N., & Tannock, J. (2008). A decision aid for selecting improvement methodologies. [Editorial]. [Electronic version]. International Journal of Production Research, Vol. 46, No. 23, 6721-6737.
Toronto. (2010). Retrieved June 21, 2010 from Glossary of Terms: http://www.toronto.ca/lobbying/glossary.htm.
Wayne State University. (2010). Retrieved June 21, 2010 from eLearning Glossary: http://www.eng.wayne.edu/page.php?id=1263.
Wikipedia.org. (2010). Retrieved June 21, 2010 from Customer Experience: http://en.wikipedia.org/wiki/customer_experience.
Wikipedia.org. (2010). Retrieved June 21, 2010 from Instructor-led training: http://en.wikipedia.org/wiki/instructor-led_training.
Wikipedia.org. (2010). Retrieved June 21, 2010 from Procedures: http://en.wikipedia.org/wiki/procedures.
Womack, J.P., Jones D.T., and Ross D., The Machine that Changed the World, Rawson Associates, New York, 1990, 323 PP.
Appendix A – Six Sigma Flows and Definitions
Shared Define During the Define phase, organizations establish project parameters, review and
define goals, as well as identify customer needs and processes/products to be improved.
Measure Measuring consists of evaluating the performance of the core business process and collect data needed for future comparisons.
Analyze The purpose of the Analyze phase is to identify the causes of poor performance and to determine how the process can be improved.
DMAIC DMADV Improve The purpose of the Improve
phase is to address the root causes of poor performance.
Control In the Control phase, organizations ensure that the improved procedures and tools are consistently applied.
Design In the Design phase, organizations design, develop, and implement new procedures or tools.
Verify In the Verify phase, organizations ensure that the new procedures or tools are consistently applied.
(Matson & Stauffer, 2009)
Appendix B – Lean Definitions
Principle Term Definition Elimination of waste Waste Anything a paying customer is not willing to pay
for. 7 categories of waste: • Over-production • Waiting • Transportation • Inventory • Motion • Over-processing • Defects
Identification of the Value Stream
Value Anything a paying customer is willing to pay for
Achievement of Flow through the process
Flow Continuous evaluation of inventories, batch sizes, and work methods & distribution
Pacing by a pull signal Pull Production is restricted to actual demand rather than capacity
Continuous pursuit of perfection
n/a
(NWLEAN Inc, 2010)
Appendix C – Focus Group Disclosures
I’m asking you to participate as a subject in a research project entitled "Weekly Updates
and Process Improvements.” You’ve been selected for this focus group because you oversee the
largest variety of site locations. The purpose of this study is to help customer-facing employees
stay abreast of company changes. By identifying gaps and analyzing possible causes for them,
the training department is equipped to assess the strengths and weaknesses of our current
processes.
If you choose to participate in this focus group your responses will be recorded and
shared only with myself and the project sponsor. If you have additional information that you
would like to share but not record, that is an option. We ask that you are as objective and open
as possible. There are no reasonably foreseeable risks associated with participating in this study.
There may not be direct benefits to you from participating in this study.
Participation is voluntary. You have the right to refuse to be in this study. If you decide
to be in the study and change your mind, you have the right to discontinue participation at any
time. However, any information already used up to the point when you withdraw consent, will
not be removed from the study’s data. Whatever you decide, you will not lose any benefits to
which you are otherwise entitled.
This report will not contain personal identifying information such as your name or
department. Your participation will not be identified and information you provide will be kept
confidential and not reported to others outside the research project in a way that personally
identifies you. You may ask any questions about this project of the researcher or project sponsor
at any time.
Appendix D – Focus Group Questions
1. To what extent do you feel customer-facing employees are currently staying abreast of
company changes that are outlined in weekly updates?
2. How are processes involved in the retention of weekly updates?
3. How would you describe the current situations surrounding how we release weekly
updates?
a. What is working?
b. What is not working?
4. Are things different in different locations?
a. If so, how are they different?
5. Do you see any trends between delivery methods and how well employees are retaining
the information in the weekly updates?
6. What is the current priority of the involvement of employees and supervisors as related to
the weekly updates?
7. Is there consistency among how supervisors reinforce the importance of the weekly
updates?
8. What motivates employees to care about the weekly updates?
9. Are there consequences for employees who do not stay abreast of weekly updates?
10. How did employees receive the assessment we released for Q1?
11. Do you think the assessment is an accurate way to see if employees are retaining
information from the weekly updates
a. What worked?
b. What didn’t?
Appendix E – Average Pass Percentages and Average Scores
Site Type Total
Count
Passed % Passed Ave
Score %
Forsythia Employee 265 184 69% 58%
Supervisor 11 11 100% 96%
Total 276 195 71% 83%
Gloxinia Employee 227 159 70% 82%
Supervisor 14 10 71% 87%
Total 241 169 70% 82%
Laburnum Employee 344 196 57% 80%
Supervisor 11 9 82% 90%
Total 355 205 58% 80%
Wisteria Employee 101 69 68% 84%
Supervisor 3 3 100% 96%
Total 104 72 69% 84%
Whortleberry Employee 181 116 64% 84%
Supervisor 8 8 100% 90%
Total 189 124 66% 84%
Rhododendron Employee 197 95 48% 70%
Supervisor 11 8 73% 85%
Total 208 103 50% 79%
Snapdragon Employee 227 185 81% 85%
Supervisor 7 7 100% 98%
Total 234 192 82% 86%
Valerian Employee 397 284 72% 85%
Supervisor 24 22 92% 91%
Total 421 306 73% 85%
Across All Sites 2028 1366 67% 83%
Appendix F - Percent Passed Detail – Listed by Site Location
Site
Manager
Employees Failed
Employees Passed
Grand Total
Percentage Passed
Forsythia Supervisor Name 12 2 14 14%
Supervisor Name 6 5 11 45%
Supervisor Name 5 6 11 55%
Supervisor Name 4 5 9 56%
Supervisor Name 6 8 14 57%
Supervisor Name 7 11 18 61%
Supervisor Name 5 8 13 62%
Supervisor Name 6 10 16 63%
Supervisor Name 5 11 16 69%
Supervisor Name 5 11 16 69%
Supervisor Name 3 8 11 73%
Supervisor Name 3 8 11 73%
Supervisor Name 3 9 12 75%
Supervisor Name 3 9 12 75%
Supervisor Name 3 10 13 77%
Supervisor Name 2 15 17 88%
Supervisor Name 1 11 12 92%
Supervisor Name 13 13 100%
Supervisor Name 2 2 100%
Supervisor Name 7 7 100%
Supervisor Name 9 9 100%
Supervisor Name 13 13 100%
Supervisor Name 3 3 100%
Forsythia Totals 79 194 273 71%
Gloxinia Supervisor Name 4 4 0%
Supervisor Name 1 1 0%
Supervisor Name 5 3 8 38%
Supervisor Name 3 2 5 40%
Supervisor Name 7 7 14 50%
Supervisor Name 2 2 4 50%
Supervisor Name 1 1 2 50%
Supervisor Name 2 2 4 50%
Supervisor Name 2 2 4 50%
Supervisor Name 5 7 12 58%
Supervisor Name 4 6 10 60%
Supervisor Name 5 9 14 64%
Supervisor Name 4 8 12 67%
Supervisor Name 2 4 6 67%
Supervisor Name 1 2 3 67%
Supervisor Name 1 2 3 67%
Supervisor Name 3 9 12 75%
Supervisor Name 4 13 17 76%
Supervisor Name 3 10 13 77%
Supervisor Name 2 8 10 80%
Supervisor Name 1 5 6 83%
Supervisor Name 1 9 10 90%
Supervisor Name 9 9 100%
Supervisor Name 1 1 100%
Supervisor Name 3 3 100%
Supervisor Name 16 16 100%
Supervisor Name 3 3 100%
Supervisor Name 8 8 100%
Supervisor Name 3 3 100%
Gloxinia Totals 63 154 217 71%
Laburnum Supervisor Name 4 4 0%
Supervisor Name 1 1 0%
Supervisor Name 7 2 9 22%
Supervisor Name 7 2 9 22%
Supervisor Name 3 1 4 25%
Supervisor Name 3 1 4 25%
Supervisor Name 2 1 3 33%
Supervisor Name 6 3 9 33%
Supervisor Name 4 2 6 33%
Supervisor Name 4 2 6 33%
Supervisor Name 6 3 9 33%
Supervisor Name 4 2 6 33%
Supervisor Name 14 9 23 39%
Supervisor Name 3 2 5 40%
Supervisor Name 3 2 5 40%
Supervisor Name 3 2 5 40%
Supervisor Name 3 2 5 40%
Supervisor Name 6 4 10 40%
Supervisor Name 5 4 9 44%
Supervisor Name 4 4 8 50%
Supervisor Name 1 1 2 50%
Supervisor Name 5 5 10 50%
Supervisor Name 2 2 4 50%
Supervisor Name 4 5 9 56%
Supervisor Name 3 4 7 57%
Supervisor Name 2 3 5 60%
Supervisor Name 4 6 10 60%
Supervisor Name 3 5 8 63%
Supervisor Name 3 5 8 63%
Supervisor Name 4 7 11 64%
Supervisor Name 1 2 3 67%
Supervisor Name 2 4 6 67%
Supervisor Name 2 4 6 67%
Supervisor Name 1 2 3 67%
Supervisor Name 3 6 9 67%
Supervisor Name 2 5 7 71%
Supervisor Name 2 5 7 71%
Supervisor Name 2 7 9 78%
Supervisor Name 2 7 9 78%
Supervisor Name 1 8 9 89%
Supervisor Name 1 8 9 89%
Supervisor Name 2 2 100%
Supervisor Name 2 2 100%
Supervisor Name 9 9 100%
Supervisor Name 1 1 100%
Supervisor Name 3 3 100%
Supervisor Name 4 4 100%
Supervisor Name 3 3 100%
Supervisor Name 8 8 100%
Supervisor Name 3 3 100%
Supervisor Name 6 6 100%
Supervisor Name 1 1 100%
Laburnum Totals
142 191 333 57%
Wisteria Supervisor Name 1 1 0%
Supervisor Name 7 3 10 30%
Supervisor Name 5 5 10 50%
Supervisor Name 4 7 11 64%
Supervisor Name 4 7 11 64%
Supervisor Name 3 8 11 73%
Supervisor Name 2 6 8 75%
Supervisor Name 2 6 8 75%
Supervisor Name 2 6 8 75%
Supervisor Name 1 3 4 75%
Supervisor Name 1 5 6 83%
Supervisor Name 1 9 10 90%
Supervisor Name 3 3 100%
Supervisor Name 1 1 100%
Wisteria Totals 33 69 102 68%
Whortleberry Supervisor Name 9 2 11 18%
Supervisor Name 4 3 7 43%
Supervisor Name 1 1 2 50%
Supervisor Name 4 4 8 50%
Supervisor Name 6 6 12 50%
Supervisor Name 4 5 9 56%
Supervisor Name 4 5 9 56%
Supervisor Name 7 11 18 61%
Supervisor Name 3 5 8 63%
Supervisor Name 1 2 3 67%
Supervisor Name 2 5 7 71%
Supervisor Name 1 3 4 75%
Supervisor Name 1 3 4 75%
Supervisor Name 3 10 13 77%
Supervisor Name 2 7 9 78%
Supervisor Name 2 8 10 80%
Supervisor Name 2 8 10 80%
Supervisor Name 1 8 9 89%
Supervisor Name 3 3 100%
Supervisor Name 4 4 100%
Supervisor Name 4 4 100%
Supervisor Name 2 2 100%
Supervisor Name 3 3 100%
Supervisor Name 4 4 100%
Whortleberry Totals
57 116 173 67%
Rhododendron Supervisor Name 1 1 0%
Supervisor Name 3 3 0%
Supervisor Name 3 3 0%
Supervisor Name 6 2 8 25%
Supervisor Name 8 3 11 27%
Supervisor Name 8 4 12 33%
Supervisor Name 10 5 15 33%
Supervisor Name 4 2 6 33%
Supervisor Name 7 4 11 36%
Supervisor Name 3 2 5 40%
Supervisor Name 4 3 7 43%
Supervisor Name 6 5 11 45%
Supervisor Name 6 6 12 50%
Supervisor Name 5 5 10 50%
Supervisor Name 3 3 6 50%
Supervisor Name 2 2 4 50%
Supervisor Name 4 4 8 50%
Supervisor Name 4 6 10 60%
Supervisor Name 5 9 14 64%
Supervisor Name 3 7 10 70%
Supervisor Name 3 8 11 73%
Supervisor Name 3 10 13 77%
Supervisor Name 1 4 5 80%
Supervisor Name 1 1 100%
Supervisor Name 2 2 100%
Supervisor Name 4 4 100%
Supervisor Name 2 2 100%
Rhododendron Totals
102 103 205 50%
Snapdragon Supervisor Name 4 4 8 50%
Supervisor Name 4 5 9 56%
Supervisor Name 4 7 11 64%
Supervisor Name 4 10 14 71%
Supervisor Name 4 10 14 71%
Supervisor Name 2 5 7 71%
Supervisor Name 3 8 11 73%
Supervisor Name 4 12 16 75%
Supervisor Name 2 8 10 80%
Supervisor Name 2 8 10 80%
Supervisor Name 1 5 6 83%
Supervisor Name 2 10 12 83%
Supervisor Name 1 6 7 86%
Supervisor Name 1 7 8 88%
Supervisor Name 1 11 12 92%
Supervisor Name 2 2 100%
Supervisor Name 9 9 100%
Supervisor Name 1 1 100%
Supervisor Name 13 13 100%
Supervisor Name 1 1 100%
Supervisor Name 2 2 100%
Supervisor Name 1 1 100%
Supervisor Name 1 1 100%
Supervisor Name 4 4 100%
Supervisor Name 1 1 100%
Supervisor Name 4 4 100%
Supervisor Name 2 2 100%
Snapdragon Totals
39 157 196 80%
Valerian Supervisor Name 11 6 17 35%
Supervisor Name 9 6 15 40%
Supervisor Name 7 9 16 56%
Supervisor Name 6 9 15 60%
Supervisor Name 6 9 15 60%
Supervisor Name 4 7 11 64%
Supervisor Name 5 9 14 64%
Supervisor Name 5 10 15 67%
Supervisor Name 6 13 19 68%
Supervisor Name 4 10 14 71%
Supervisor Name 5 13 18 72%
Supervisor Name 4 11 15 73%
Supervisor Name 4 11 15 73%
Supervisor Name 3 9 12 75%
Supervisor Name 4 12 16 75%
Supervisor Name 4 12 16 75%
Supervisor Name 3 10 13 77%
Supervisor Name 4 14 18 78%
Supervisor Name 3 11 14 79%
Supervisor Name 1 4 5 80%
Supervisor Name 3 12 15 80%
Supervisor Name 3 12 15 80%
Supervisor Name 3 12 15 80%
Supervisor Name 2 12 14 86%
Supervisor Name 2 13 15 87%
Supervisor Name 2 14 16 88%
Supervisor Name 1 8 9 89%
Supervisor Name 1 14 15 93%
Supervisor Name 6 6 100%
Supervisor Name 3 3 100%
Supervisor Name 1 1 100%
Supervisor Name 2 2 100%
Valerian Totals 115 304 419 73%
1918
Appendix G - Question Totals – Listed by Site Location
Question
Totals
Forsythia
Gloxinia
Laburnum
Wisteria
Whortleberry
Rhododendron
Snapdragon
Valerian Delivery
Method
Sampl
e Que
stio
n 1
(and
cor
rect ans
wer)
94%
81%
94
%
98%
98
%
93%
91
%
98%
97
%
Non
e Sa
mpl
e Que
stio
n 2
(and
cor
rect ans
wer)
69%
49%
62
%
75%
50
%
83%
74
%
89%
67
%
Non
e Sa
mpl
e Que
stio
n 3
(and
cor
rect ans
wer)
96%
92%
95
%
95%
98
%
96%
95
%
98%
95
%
Non
e Sa
mpl
e Que
stio
n 4
(and
cor
rect ans
wer)
75%
81%
65
%
69%
75
%
87%
64
%
85%
73
%
Non
e Sa
mpl
e Que
stio
n 5
(and
cor
rect ans
wer)
86%
88%
82
%
81%
86
%
86%
83
%
91%
87
%
Non
e Sa
mpl
e Que
stio
n 6
(and
cor
rect ans
wer)
92%
99%
95
%
88%
83
%
98%
83
%
94%
97
%
Non
e
Sampl
e Que
stio
n 7
(and
cor
rect ans
wer)
81%
77%
78
%
81%
89
%
83%
80
%
85%
78
%
Non
e Sa
mpl
e Que
stio
n 8
(and
cor
rect ans
wer)
70%
76%
72
%
64%
70
%
67%
65
%
82%
69
%
Tea
m M
eetin
g
Sampl
e Que
stio
n 9
(and
cor
rect ans
wer)
93%
96%
93
%
91%
92
%
99%
90
%
96%
90
%
Tea
m M
eetin
g
Sampl
e Que
stio
n 10
(a
nd cor
rect ans
wer)
97%
98%
96
%
94%
98
%
98%
95
%
99%
97
%
Non
e Sa
mpl
e Que
stio
n 11
(a
nd cor
rect ans
wer)
83%
84%
84
%
83%
80
%
83%
79
%
87%
82
%
Non
e Sa
mpl
e Que
stio
n 12
(a
nd cor
rect ans
wer)
86%
84%
91
%
85%
90
%
87%
72
%
92%
88
%
Non
e
Sampl
e Que
stio
n 13
(a
nd cor
rect ans
wer)
77%
74%
77
%
78%
73
%
77%
73
%
89%
77
%
Non
e Sa
mpl
e Que
stio
n 14
(a
nd cor
rect ans
wer)
65%
54%
72
%
61%
65
%
62%
55
%
77%
72
%
Non
e Sa
mpl
e Que
stio
n 15
(a
nd cor
rect ans
wer)
81%
65%
78
%
80%
96
%
69%
87
%
85%
89
%
Non
e Sa
mpl
e Que
stio
n 16
(a
nd cor
rect ans
wer)
95%
98%
96
%
84%
98
%
99%
92
%
93%
97
%
Non
e Sa
mpl
e Que
stio
n 17
(a
nd cor
rect ans
wer)
73%
70%
78
%
67%
71
%
76%
70
%
79%
75
%
Non
e Sa
mpl
e Que
stio
n 18
(a
nd cor
rect ans
wer)
95%
94%
96
%
93%
97
%
92%
98
%
95%
97
%
10 M
inutes
Tea
m M
eetin
g
Sampl
e Que
stio
n 19
(a
nd cor
rect ans
wer)
54%
51%
56
%
58%
55
%
51%
44
%
68%
51
%
10 M
inutes
Tea
m M
eetin
g
Sampl
e Que
stio
n 20
(a
nd cor
rect ans
wer)
92%
92%
93
%
89%
89
%
91%
91
%
98%
90
%
10 M
inutes
Tea
m M
eetin
g
Sampl
e Que
stio
n 21
(a
nd cor
rect ans
wer)
94%
96%
96
%
92%
96
%
94%
89
%
90%
95
%
10 M
inutes
Tea
m M
eetin
g
Sampl
e Que
stio
n 22
(a
nd cor
rect ans
wer)
93%
97%
96
%
86%
95
%
98%
85
%
95%
95
%
Non
e Sa
mpl
e Que
stio
n 23
(a
nd cor
rect ans
wer)
90%
90%
95
%
86%
87
%
86%
90
%
93%
93
%
Non
e Sa
mpl
e Que
stio
n 24
(a
nd cor
rect ans
wer)
76%
80%
75
%
71%
78
%
76%
71
%
80%
79
%
Non
e Sa
mpl
e Que
stio
n 25
(a
nd cor
rect ans
wer)
72%
87%
77
%
62%
64
%
67%
48
%
81%
89
%
10 M
inutes
Tea
m M
eetin
g
Sampl
e Que
stio
n 26
(a
nd cor
rect ans
wer)
85%
96%
93
%
72%
74
%
94%
66
%
91%
96
%
Non
e Sa
mpl
e Que
stio
n 27
(a
nd cor
rect ans
wer)
92%
97%
90
%
84%
94
%
91%
89
%
94%
95
%
Non
e Sa
mpl
e Que
stio
n 28
(a
nd cor
rect ans
wer)
92%
97%
95
%
96%
95
%
93%
97
%
65%
96
%
Intern
al
Cen
ters
Sampl
e Que
stio
n 29
(a
nd cor
rect ans
wer)
91%
94%
93
%
84%
89
%
91%
90
%
93%
91
%
1 ho
ur IL
T
Sampl
e Que
stio
n 30
(a
nd cor
rect ans
wer)
84%
94%
89
%
75%
77
%
86%
72
%
89%
91
%
10 M
inutes
Tea
m M
eetin
g
Total
84%
84%
85
%
81%
83
%
85%
79
%
88%
86
%
Running Head: IDENTIFYING PROCESS GAPS THROUGH EVALUATION 1
Appendix H – Question Breakdown for Low Scores
Question Number
Correct Percentage
Possible Causes of Low Scores
19 54% Upon assessing the Policy and Procedure, this fact was stated clearly. It’s possible that this was a new procedure and was not given enough attention in team meetings. It’s also possible that this is a situation employees do not come across often.
14 65% If employees used Edelweiss’ Intranet for this question, it’s possible that they expected this information to be in the applicable Policy, but it is in the correlated Procedure. The Policy does refer to the Procedure properly and effectively.
2 69% Employees may have read the question incorrectly (“are not” rather than “are”). If employees in different queues have different procedures, this may have caused confusion.
8 70% It's possible that the product name caused confusion because it includes a written numeric value. However, the options available were not numerically close to the product name and would not cause further confusion. Also, only Wisteria and Snapdragon received team meeting time. This was not likely to be an area of focus for other centers.
17 73% There were numerous changes concerning this release with no training time scheduled. The date of the release changed and was announced the following week. The change of this related item may have caused employees to unconsciously add a sense of complication to the question.
25 72% This was a change of policy and was written effectively and accurately in the Policy and Procedure. It's possible this may not have been communicated effectively in team meetings.
4 75% There are four pieces of information in each response. This may have caused confusion. The possible answers to this question were extremely similar and lengthy. It's possible that some employees may have known the correct answer but selected the wrong option due to reading comprehension
24 76% This was not new information. The weekly update explained a deviation from this information for some areas. Therefore, employees may have chosen either of the two other distracters because they assumed the question would be focusing on a change.
IDENTIFYING PROCESS GAPS THROUGH EVALUATION 60
Appendix I – Definitions
Correlation - The simultaneous change in value of two numerically valued variables
(Dictionary.com, 2010)
Customer Experience – The sum of all experiences a customer has with a supplier of goods or
services, over the duration of their relationship with that supplier (Wikipedia.org, 2010)
Customer-Facing Employees – People who are paid for the performance of their duties by
Edelweiss Communications who speak directly to customers (in part from: Toronto, 2010)
eLearning – Internet-enabled learning that encompasses training, education, just-in-time
information, and communication (Wayne State University, 2010)
Instructor Led Training – (ILT) The practice of face-to-face delivery of training and learning
material between an instructor and learners, either individuals or groups (Wikipedia.org, 2010)
Pattern - A combination of qualities, acts, tendencies, etc., forming a consistent or characteristic
arrangement (Dictionary.com, 2010)
Policies – The rules, conventions, and protocols adopted by Edelweiss Communications, of
which it requires employees to adhere
Procedures – A specified series of actions executed in the same manner to always obtain the
same result under the same circumstances (Wikipedia.org, 2010)
Team Meetings – Meetings where employees collaborate in a group to share information
facilitated by employees’ direct supervisors