Internal Employee Communication Audit of JAX Mercantile Co.

in Loveland, Co.

Hannah Toole

May 6, 2011

Introduction

2 [Internal Employee Communication Audit of JAX Mercantile Co.]

In today’s industry organizations are continuously searching for ways to gain an edge

over their competitors and improve their bottom line. Research has recognized that

companies’ internal communications is the key to securing and maintaining that competitive

advantage.

A companies internal communication systems is crucial to success, however, this

critical area has often been overlooked by organizations. In this day and age, it is more

important than ever that, companies be conducting communication audits and regularly

monitoring their internal communication systems.

An internal communication audit is a thorough evaluation of a company’s ability to

transfer information. The purpose is to uncover strengths and weaknesses within the

company regarding the various ways information is transferred.

The importance of communication audits is outlined in William and Dozier’s study,

“Framing Communication Audits”. Their study sought to determine the real value of

communication audits for an organization. The most favored response from nationally

recognized communication audit consultants was that, a communication audit will help your

organization align its communication strategy with your organization strategy (William &

Dozier 2010). It is also noted that a communication audit provides advance information to

help prevent communication breakdown and provides reliable, factual information about

internal communication (William & Dozier).

When employees are more satisfied with communication within their organization

there is an increase in performance, productivity, profitability, and external customer

orientation (Koning & De Jong).

Communication is possibly the most important process in an organization.

“Employees spend tremendous amounts of time collecting and disseminating information

concerning such critical matters as company policy, placement, promotion, performance

feedback and so forth,” (Mount & Back 1993). According to Mount and Back’s study, “A

Factor-Analytic Study of Communication Satisfaction in the Lodging Industry,” researchers

have determined that there is positive relationship between communication satisfaction and

organizational effectiveness. “In fact that relationship has been shown to be fairly strong

when it comes to the relationship between communication satisfaction and job satisfaction

(William & Dozier).

In order to determine the relationship between communication and job satisfaction,

Downs and Hazen developed the Communication Satisfaction Questionnaire [CSQ] (Downs

1977). The CSQ estimates communication satisfaction of employees by asking them

evaluative statements over eight communication factors. Respondents indicate their answers

on a seven point Likert Scale. This method, developed by Downs in 1977, provides great

qualitative data for assessing communication satisfaction.

CSQ has become the most popular method for determining communication

satisfaction of employees. However, few studies have been done regarding the advantages

and limitations of it.

In the study, “Measurement of Communication Satisfaction,” Kong and De Jong,

compared CSQ against the Critical Incident Technique (CIT), in order to gain insight into the

merits and restrictions of each communication analysis technique.

CIT, “focuses on positive and negative communication experiences, which are

assumed to affect or represent employees overall level of communication satisfaction”

4 [Internal Employee Communication Audit of JAX Mercantile Co.]

(Koning & De Jong). Employees are asked to recall and describe in detail actual

communicative experiences. This method provides rich qualitative data and reveals issues

that the CSQ does not focus on.

Research has revealed that there is not just one method that is appropriate for a

reliable and valid communication audit. It is recommended that multiple techniques be used

together. For this reason, I am proposing an internal communication audit of JAX Mercantile

Co, which will include both the CSQ and CIT methods.

The focuses for this communication audit will be to assess the effectiveness of

communication within JAX Mercantile, determine the overall level of employee satisfaction,

and finally to gather employee recommendations regarding any improvements that can be

made within the company.

Through this study I hope to determine the internal communication strengths and

weakness within the company, levels of employee satisfaction, and if communication

supports the organizational objectives.

Method

Sample:

This research proposal sought to conduct an intensive study of all employees at the

JAX Mercantile store in Loveland, CO. The population for the study was relatively small, so

an intensive approach suited the purpose of this proposal in order to get the most reliable data

regarding communication practices within the company. A list of participants was selected

using a purposive sampling strategy, not a probability sampling technique.

The original sampling frame consists of 3 upper managers, 1 of which is a store

manager and the other 2 are assistant store managers. All 15 department managers and the

100 other store employees will be studied. Out of the 100 store employees, 2 participants

from each department, totaling 30 people, will volunteer themselves to participate in a series

of interviews. All employees involved in the study will have worked for JAX Mercantile for

at least 4 months.

One-on-one interviews will be conducted with the store manager, 2 assistant

managers, all department managers, and 30 sales staff (2 from each department). The sales

staff for the interviews will be selected through volunteer sampling. All store employees will

complete an online questionnaire. There will be separate questionnaires for department

managers and for sales staff.

Design:

This communication audit will be an intensive study involving correlations between

numerous sets of variables. Our ultimate criterion variable is the level of communication

satisfaction. Predictor variables responsible for estimating the criterion include: Job

satisfaction, organizational identification, organizational integrity, satisfaction with

supervisors, satisfaction with co-workers, quality of communication channels, recognition

levels, and performance and productivity levels.

The predictor variables will be operationalized through a series of intensive questions

for each category. They will be measured on a seven point Likert Scale, the interviewer will

interpret responses from the CIT interviews and a relevant scale will be developed.

The criterion variable will be operationalized after an extensive analysis of the CSQ

and CIT responses. From the data an overall level of employee communication satisfaction

will be determined.

6 [Internal Employee Communication Audit of JAX Mercantile Co.]

Pretesting:

The questionnaire and interview questions will be pretested prior to the initial study.

Participants for the pretest will be from the JAX store in Fort Collins. Two sets of focus

groups, each with ten participants will be conducted. All of the participants will collectively

complete the questionnaire and as a group will discuss confusions and questions they had.

Following discussion of the questionnaire, interviewers will run through the interview

questions and discuss each question as they go along. The purpose of the pretest will be to

uncover any flaws in the questionnaire design or confusion in question wording. The pretest

will also help us discover any useful questions that had not been considered.

Procedures:

Orientation

One of the most important steps will be to orientate the employees of JAX to the

communication audit that they will be participating in. It is essential for employee

cooperation, participation, and patience to explain the importance of the communication

audit. Employees will be informed that participation in any of the communication audit

activities is voluntary, however it is strongly encouraged. We want them to understand that

their efforts and input will directly influence changes that will be made around the company

and that the intentions of this study are to make improvements that will increase their job

satisfaction.

Managers will explain that there will be a questionnaire for each employee to fill out,

and that there will also be a series of interviews involving upper management and any sales

employees that volunteer to participate. The managers will stress the confidentiality of the

questionnaires and interviews and ask that participants be completely honest with their

answers.

Employees will be introduced to the communication audit during one of the monthly

store meetings, in which attendance is required. It will be the responsibility of the store

manager to explain exactly what will be happening, how they will be affected, the purpose of

the study, and to answer any questions and concerns that employees may have.

Communication Satisfaction Questionnaire

A comprehensive communication survey will be administered to all store employees.

There will be a different survey for managers and for department sales staff. The survey will

be administered through an online program and will be completed at the store. The survey

will be confidential and results will not be accessible to any store employees or managers.

The purpose of the questionnaire as well as a reminder that participation is voluntary will be

outlined in the Communication Satisfaction Questionnaire, refer to appendix.

According to Relationship Between Communication Satisfaction and Organizational

Identification, the Communication Satisfaction Questionnaire (CSQ), “remains the dominant

measure of communication satisfaction in applied and basic research. It has been confirmed

as having a high degree of validity and reliability across a number of organizations” (Nakra

2006).

Koning, K. H. and De Jong, M. D. measured the merits and restrictions of the

Communication Satisfaction Questionnaire. Their results determined that CSQ is one of the

most reliable and valid methods for measuring communication satisfaction and

communication issues within an organization (Konig & De Jong 2006).

8 [Internal Employee Communication Audit of JAX Mercantile Co.]

For the above reasons, a modified version of the Communication Satisfaction

Questionnaire will be used to determine the communication satisfaction of JAX employees.

“This scale measures the employees perceptions about the communication occurring within

the organization and has eight distinct dimensions.” ( Nakra 2006).

There will be two separate versions of this questionnaire, one for all lower level

employees and the other for all supervisors and managers. Participants indicate on a 7-point

Likert Scale, how satisfied they are with the eight communication dimensions of the

company. These include communication climate, supervisory communication, organizational

communication, organizational integration, media quality, co-worker communication,

organizational perspective, personal feedback, and relationship with subordinates. The last

dimension, relationship with subordinates will only be included in the supervisor and

managers survey (Nakra 2006). Refer to appendix for survey.

Employees will have a three-week period to fill out the survey, which will not take

more than a half hour to complete. Store managers and department managers will be

responsible for reminding employees to complete the survey on time. Reminder e-mails will

also be sent out to each employee twice a week until they have submitted their answers.

Participants will fill out the survey while they are on the clock, on a store computer.

Critical Incident Technique Interviews

A series of in depth interviews will also be conducted for this study. The interviews

will involve: the store manager, assistant store managers, department managers, and thirty

sales-level employees (two from each department).

The focus will be to get a greater in-depth understanding of communication within

the organization and obtain valuable information from employees regarding suggestions they

have for the company. The employees will volunteer themselves for the interviews. This

method is used in hopes that employees with the greatest opinions, concerns, and ideas that

they want to share will come forward.

The Critical Incident Technique (CIT) will be used to obtain in-depth information

from participants in the interviews. The CIT method focuses on positive or negative

communication incidents that have occurred within the organization. Participants are asked to

describe in detail, specific events regarding communication in the organization that they had

strong positive or negative feelings about. They will describe the experience, explaining

exactly what happened, who was involved, and what affects the experience had on them

(Koning, K. H. and De Jong 2006).

The article Measurement of Communication Satisfaction states, “earlier studies have

shown that this technique provides detailed insight in the nature of the communication

process within the organization. It gives insight in the extent to which information is

exchanged adequately, the way decisions are made and what communication channels are

used to communicate important organizational issues to employees” (Koning, K. H. and De

Jong 2006).

The interviews will also be used to gather ideas, suggestions, and comments for

improvement from employees. We hope that any concerns they may have with the company

will be brought up within the interviews.

All of the interviews will be administered at the store during paid company time in

the staff meeting room. With the permission of each employee, the interviews will be

10 [Internal Employee Communication Audit of JAX Mercantile Co.]

recorded for purposes of analysis. All interviews will be kept confidential and only those

conducting the communication audit will have access to them. Interviewers will once again

explain to participants that the purpose of the interview and explain that participation is

completely voluntary. See appendix for Critical Incident Form.

Interviewers will be trained beforehand through a series of mock interviews. In

training they will practice using the critical incident technique. Interviewers will practice

asking open questions which allow for elaborate answers, getting participants to describe

what led up to each incident, details about the actual experience, and the outcome of the

incident, and getting information regarding how participants dealt with incident and how it

could have been avoided. All questions and possible answers will be covered in the training

sessions. They will become comfortable with carrying the interview in different directions

and coming up with their own questions if important, interesting information is brought up.

They will also go through training to eliminate interviewer bias, which will include voice

training, body language training, and word choice training.

The only equipment needed for the interviews is a voice recorder that will be supplied

by the communication audit team. Refreshments and snacks will also be provided as

incentive during the interviews.

Data Analysis

Data will be collected using an online survey, with an embedded link sent to each

participant. Results will be confidential and upper management will not have access to

individual questionnaires. Data will be downloaded from the online survey and transferred

into a data file for statistical analysis, SPSS 11 for Macs.

One of the first steps in the data analysis of the CSQ will be to preform a reliability

test by running an alpha test. A Chi-Square test will be ran to test proportions of the data.

This test will help us determine if; 1. The frequencies of all the categories are the same and 2.

Are two sets of categories independent of one another or related? Other correlation tests will

also be ran for the different types of data gathered.

For quantitative data we will arrange data in a frequency distribution, ordering data

according to magnitude and frequency. Central tendency summary statistics will then be used

to make data even more manageable.

Examples of variables that will be measured include; Job satisfaction vs. level of

satisfaction with co-workers and supervisors, organizational identification and amount of

time spent working for JAX, satisfaction with supervisors and the communication between

supervisors and subordinates, job satisfaction and recognition, productivity and effective

communication, attitude towards the organization and knowledge of company goals and

objectives. All variables will be measured against each other.

All of the interviews will be transcribed verbatim. All initial ideas and findings will

be gathered and formulated into categories. Data analysis will be conducted using emergent

coding. The category system will be based on common factors or themes that emerge from

the data. The data will categorize the answers according to central questions and produce an

importance weighting for each. The categories will be given frequencies of occurrence. There

will also be a noted insight between facts/insights and potential/recommendations. (Hettlage

& Steinlin 2006).

Data will be examined by defining the categories with the highest frequencies. The

most reoccurring categories will be presented to management. The subjects presented

12 [Internal Employee Communication Audit of JAX Mercantile Co.]

depending on the data may either outline issues where there is room for improvement or

suggest communication efforts that the company is doing well on. A summative evaluation of

all the answers will be presented to upper management for discussion. All the information

will be presented however, there will be an emphasis on categories with the highest

frequencies.

In order to achieve intercoder reliability there will be three separate coders. Each

individually will analyze the data developing their own category system. The three coders

then will meet to review and discuss each of the category systems. A final category system

will be put together and the three coders will work together to cohesively decide what

category each incident belongs in.

Discussion

By preforming a communication audit of JAX Mercantile, we will be able to identify

problems and issues regarding their internal communication process. With this information

they will be able to make organizational changes that will benefit the success of the

company, improve employee satisfaction, and prevent future crises and issues that may arise.

The CIT gives valuable insight into employees’ perspective of strengths and

weaknesses within the company, as does the CSQ. Both methods offer rich information and

when combined they make for a valid and reliable communication audit. For communication

audits it is not recommended that any single method or technique be the sole analysis for a

study. Numerous studies have deemed the CIT and CSQ methods as effective instruments for

measuring aspects of communication satisfaction. The combination of both techniques for

this study helps eliminate limitations and issues with validity.

CSQ covers eight categories of communication satisfaction and provides us with

great quantitative data. For a communication audit, it is important that we are also provided

with rich, qualitative data. The CIT method does exactly that, giving us insight of specific

issues within the organization. The CIT method also uncovers categories and issues that the

CSQ does not. Therefor, the combination of the two methods provides us with incredibly

reliable and valid data.

With that said, there are a few limitations to this study. The first limitation is a matter

of finances. In the long run the communication audit will save the organization a great deal

of money and wasted time trying to fix problems that could have been prevented. However,

the issue we face is that this organization is incredibly frugal with their finances and is

hesitant to spend so much on a communication audit when they believe that the company is

doing well, with no foreseeable problems. Due to the lack of effective communication within

the organization, they are unaware of numerous ground level problems. They also believe

that there are more important things to be dealt with before a communication audit is

considered. The duration of the audit will also take up a decent amount of time, which is a

limitation to the study.

Another limitation is that respondents often provide answers that they believe

interviewers are searching for. This would make the study less valid. Correlation analysis

will be done on the CSQ, which presents another limitation, since we cannot make causal

conclusions from correlation findings because we cannot rule out alternative explanations.

The size of the study is also small enough that it can create false-positive results or

provide us with inaccurately represented results. Our last limitation is the exclusion of the

home office and the other JAX locations from the study. Because of this we are limited to

14 [Internal Employee Communication Audit of JAX Mercantile Co.]

issues only within the one location and are not able to determine issues that may corporate

wide or stem from the head office.

Threats to validity in this study may include a few different factors. For internal

validity, an issue with history may arise. If extremely positive or negative communication

events happen just prior to the study or during it, then respondents answers will be biased and

will not truly reflect the general communication satisfaction levels within the company.

Instrumentation may also be a threat to validity. Extensive sets of interviews are being

conducted for this study. If the interviewer grows tired over the course of the day, or is not

well trained and knowledgeable of the subject at hand, then the results will not be valid.

Sample selection is also an issue. For the interviews we hope to have two sales staff from

each department, however since the interviews are voluntary, it is possible that there will be a

clumping in specific departments, making the study less generalizable. There will be an issue

with external validity if the sample is not representative of the entire store.

References

Agency Group, 0. (n.d). Latest Survey Shows DOT Employee Satisfaction on the

Rise. FDCH Regulatory Intelligence Database, Retrieved from EBSCOhost.

Critical Incidents Technique. (n.d.). UCC home page. Retrieved May 6, 2011, from

http://www.ucc.ie/hfrg/emmus

Downs, C. W., & Adrian, A. D. (2004). Introduction. Assessing organizational

communication: strategic communication audits (pp. 2). New York: Guilford Press.

Downs, C., & Hazen, M.D. 1977. A Factor-Analytic Study of Communication

Satisfaction. Journal of Business Communication, 14(3):63-73.

Greenbaum, H., & White, N. (1976). Biofeed At the Organizational Level: the

Communication Audit. Journal of Business Communication, 13(4), 3-16.

Hettlage, R. & Steinlin, M. (2006). The Critical Incident Technique in Knowledge

Management-Related Contexts. Ingenious Peoples Knowledge. Retrieved from http://www.i-

p-k.ch/files/CriticalIncidentTechnique_in_KM.pdf

Koning, K. H. and De Jong, M. D. , 2006-06-16 "Measurement of Communication

Satisfaction: Evaluating the Communication Satisfaction Questionnaire as a Communication

Audit Tool" Paper presented at the annual meeting of the International Communication

Association, Dresden International Congress Centre, Dresden, Germany Online <PDF>.

2011-03-14 from http://www.allacademic.com/meta/p91378_index.html

Mount, D., & Back, K. (1993). A Factor-Analytic Study of Communication

Satisfaction in the Lodging Industry. Journal of Hospitality & Tourism Research, 23(401), 1-

19.

Nakra, R. (2006). Relationship Between Communication Satisfaction and

Organizational Identification: An Empirical Study. The Journal of Business Perspective,

10(2), 1-12.

Newell, E. (2010). The Happiness Factor. Government Executive, 42(13), 53-54.

Retrieved from EBSCOhost.

Survey Example Company Communications Evaluation of communication

information and strategy within the company. (n.d.). Survey Software - Online Survey

Software - Customer Satisfaction Surveys - Employee Survey - Web Survey Software.

Retrieved May 6, 2011, from http://www.questionpro.com/akira/showSurveyLibrary.do?

surveyID=255&mode=1

16 [Internal Employee Communication Audit of JAX Mercantile Co.]

Williams, Louis, & Dozier, David., 2010-3-10 “Framing Communication Audits to

Create Positive Outcomes for Organizations” Paper presented at the meeting of the 13th

International Public Relations Research Conference, Miami, Florida. <PDF>. 2011-4-28

fro

mwww.themeasurementstandard.com/issues/410/FramingCommunicationAuditsFINAL.doc

+Framing+Communication+Audits,+Louis+WIlliams

Worrell, J. (2010). Blazer Communications: A Procurement Audit Simulation. Issues

in Accounting Education, 25(3), 527-546.