Brief 122749

Evaluate the role of Mystery Shoppers in Delivering Quality Service

Introduction

The use of participant observation, also called situation research and mystery-shopping, where the

researcher interacts with the subject or subjects being observed stems from the field of cultural

anthropology (Wilson and Gutmann, 1998). Anthropologists would take part in a tribe's daily life in

order to understand the norms, attitudes and behaviours that were neither documented nor

communicable via language. Anthropology tended to use unstructured observation with researchers

immersing themselves in a situation and discovering 'reality'. Covert participant observation, where

the observation is undertaken in a concealed manner to ensure that the experience is natural and not

contrived for the sake of the observer, has also been used in the social sciences, for example in the

study of race and sex discrimination in recruitment procedures (Esmail & Everington, 1993).

Additionally, the Consumers' Association in the UK have been using covert participant observation

in their tests of service performance since 1960 (Brown, et al., 1997). Even Government

Departments have used the technique to monitor areas such as postal deliveries or the testing

undertaken in MOT test centres (garages which test the roadworthiness of cars, COT, (1996)).

In a services context, observational methods are able to provide information of the service

experience as it unfolds (Grove and Fisk, 1992). In particular, participant observation helps to

develop a richer knowledge of the experimental nature of services. The participant can identify

dimensions of the service encounter unlikely to be discerned by a distant or non-participant

observer. Concealment of this observation, although raising ethical issues can ensure that the

experience is natural and not contrived for the sake of the observer.

The need for specific performance information stems from the increasing emphasis on service

performance by service mangers. Whilst head office staff and senior management invariably set

service standards, the task of delivering these standards falls to individual staff member who face

customers. Variations in service performance can have a major impact on customer satisfaction.

This is emphasised by Bateson (1995) who suggested that the customer's experience during the

delivery of a service is as important to customer satisfaction as is the benefit that the service

provides. Grönroos, (1983) in his paper on 'interactive marketing' also stressed that 'how' you

perform a service is often as important as 'what' you perform.

Emerging academic and other literature shows that mystery-shopping measurements tend to be used

for one or more of the following purposes:

1. To act as a diagnostic tool identifying failings and weak points in an organisation's service

delivery (Wilson, 1998; Wilson and Gutmann, 1998). As a diagnostic tool, respondents use

mystery-shopping to track key elements of their organisation's service delivery process and

help identify where capital, technical and human resource requirements to be allocated.

2. To encourage, develop and motivate service teams by linking with appraisal, training and

reward mechanisms (Dorman, 1994). Definitive information from mystery-shopping is also

seen as being essential for appraising, developing, motivating and rewarding service teams

and potentially individual service personnel. In terms of motivation, mystery-shopping

results can be used to reward those service teams who are performing well against the set

service standards. Financial rewards and incentives are seen as becoming more common

in almost all sectors but particularly in the financial services sector (Wilson,1998; Dorman,

1994)

3. To assess the competitiveness of an organisation's service provision by benchmarking it

against the offerings of others in an industry (Wilson, 1998;

Wilson and Gutmann, 1998). Mystery shopping can also be used as a bench mark comparison with

the service performance of competitors. Where the research is undertaken on a syndicated basis, this

is relatively straightforward but where an organisation commissions a mystery-shopping

programme which includes an assessment of the competitors, certain ethical issues become

important (Dawson and Hillier, 1995). The Market Research Society (MRS) Code of Conduct

specifies that mystery-shopping should not involve an unreasonable amount of time or expense on

behalf of the organisation being researched.

Participant observation has tended to be used by each of these bodies as it allows the researcher to

overcome some of the potential weaknesses of interviewing and survey research.

These are highlighted by Fried.richs & Ludtke (1975).

1. There is often a discrepancy between real and reported behaviour. Occasionally

statements are made in interviews which are not in accordance with the factual behaviour of

the interviewed persons.

2. Often facts are brought to light by means of natural settings only. The interviewee is

not conscious of them and they are therefore not easy to get at by questioning.

3. The verbal capabilities of the interviewed person can limit the quality and quantity of

information gathered.

Comparative Approaches

Mystery shopping became established during the 1 970s, with approximately 25 percent to 35

percent of all banks with over $300 million in deposits conducting some type of mystery-shopping

program (Leeds, 1995). What prompted the growing interest in mystery-shopping in the '70s was

the realization by marketers and bankers of the importance of developing a sales culture (Reed and

Miles, 1995). Because sales professionalism became increasingly important, a device had to be

developed to monitor sales skills, as well as changes in service behaviours in the sales culture.

Mystery shopping began to be used as a monitoring device for sales culture development,

specifically for tracking sales behaviours and skills (Leeds, 1995; Reed and Miles, 1995). This

phenomenon then led to the use of mystery-shopping to not only monitor, but to motivate

performance, set goals or standards and reward performance. Some of the more progressive and

sales-oriented banks began rewarding employees based upon the sales achieved (Dorman, 1994). In

the 1980s, the financial industry's new catch phrase was "service quality" and, once again, mystery-

shopping (along with consumer and customer satisfaction surveys) was used extensively for

evaluating, monitoring and motivating performance. It was the combination of these two research

methodologies that changed the basic mystery-shopping methodology to one of a predictor of

customer satisfaction. By determining what customers' wants and needs are and what satisfies

customers most, checking for and reinforcing specific behaviours can be built into the mystery

shopper program (Leeds, 1995; Dwek, 1996)

While the proponents of the methodology were deriving benefit from the mystery shopping

technique, it was not without critics. Dwek (1996) states that mystery-shopping used to be regarded

as corporate spying. It was seen as an underhand activity, probably prompted by management

accountants advising clients to axe hundreds or thousands of jobs. It was, in short, an excuse for

firing people (Dwek, 1996)

In a services context, observational methods are able to provide information of the service

experience as it unfolds (Grove and Fisk, 1992). In particular, participant observation helps to

develop a richer knowledge of the experimental nature of services. The participant can identify

dimensions of the service encounter unlikely to be discerned by a distant or non-participant

observer. Concealment of this observation, although

raising ethical issues can ensure that the experience is natural and not contrived for the

sake of the observer.

In terms of ethics, observing people without their knowledge may violate their rights to

privacy and freedom from exploitation (Jorgensen, 1989). However, services are often

performed in public settings where their delivery can often be observed by members of

the public other than the specific obligation to the service provider in a concealed

observation than in normal everyday life circumstances.

Mystery shopping is a clear example of concealed participant observation in a public

setting. However, it differs from original anthropological approach to observation in

terms of its structured and systematic format. Cultural anthropology tended to use

unstructured observation with researchers immersing themselves in a situation and

discovering 'reality'. Whereas mystery-shopping uses a structured approach of checklists

and codes to gather and measure specific information about service performance in

everyday conditions (Grove and Fisk, 1992).

The need for specific performance information stems from the increasing emphasis on

service performance by service mangers. Whilst head office staff and senior management

invariably set service standards, the task of delivering these standards falls to individual

staff member who face customers. Variations in service performance can have a major

impact on customer satisfaction. This is emphasised by Bateson (1995) who suggested

that the customer's experience during the delivery of a service is as important to

customer satisfaction as is the benefit that the service provides. Grönroos, (1983) in his

paper on 'interactive marketing' also stressed that 'how' you perform a service is often as

important as 'what' you perform.

Theories and Models

In the UK, mystery-shopping is the smallest area of specialisation covered by major research

companies (Dawson and Hillier, 1995). However, in a survey conducted by the MRS in 1995, it was

found that the use of mystery-shopping as a research methodology is gaining widespread

acceptance (Dawson and Hillier, 1995). There are few published figures on the scale of mystery-

shopping activity, Wilson (1998) gave an estimate of between £20-3Omillion per annum and

Dawson and Hillier (1995) identified 187 agencies providing this form of research. Apart from

using mystery-shopping as a part of performance monitoring programmes, competitor mystery-

shopping is relatively widespread (Dawson and Hillier, 1995). However there are also a number of

areas that lead to tension among companies with regard to competitor mystery-shopping. In a

survey carried out by the MRS in 1995, the following were cited by the respondents to be

problematic issues in competitor mystery shopping: mystery shoppers take up too much time,

should not do anything out of ordinary, should make a purchase, it is ethically wrong to place orders

and then cancel, the taping of conversations is not acceptable unless prior permission is gained, etc.

(Dawson and Hillier, 1995). As a response to these problems, and also to reduce the costs of

research for individual organisations, syndicated studies involving a number of clients

are also in operation. The syndicated studies enable the client organisation to assess their

performance relative to industry norms and comprise groups of organisations. The financial services

in particular have been engaged in a syndicated shopping since 1992. Subscribers such as the

Midland, NatWest, Royal Bank of Scotland, Bank of Scotland, Bradford and Bingley, Nationwide,

Yorkshire Building Society and Leeds Permanent get comparisons on their own and competitors'

performances on a whole range of services (Miles, 1993).

Mystery Shopping is a technique increasingly used in service organizations to measure intangible

service experiences. Since a service experience exists only as a memory in the person who had the

experience, measuring how good it was is a challenge for service organizations. Instead of being

able to rely on the quality inspectors that manufacturing firrms use to ensure product standards are

met, services must invent other measurement techniques to assess the quality of their customers’

service experience. Of the many techniques service organizations might use, one of the most useful

is the mystery shopper. Mystery shoppers go to an organization to experience a service ‘‘incognito.’’

They act as a typical customer to assess predetermined service standards of service quality are

being met by employees delivering the service.

The mystery shopper visits a specific organization at a predetermined time and then completes a

systematic assessment form detailing all aspects of that service experience. Rather than relying on

unpredictable customers’ willingness to fill out comment cards, their vague memories when

responding to random surveys, or a supervisor’s chance opportunities to observe customer-

employee interactions, a mystery shopper can ‘‘catch the experience’’ while it is happening.

Whether it is a bank, health care facility, restaurant, hotel or any other organization providing an

intangible service experience, a mystery shopper provides an assessment of both objective and

subjective aspects of the quality and value of a service. The assessment forms are designed to

measure an organization’s specific service standards. Because of their comparatively high cost,

mystery shoppers typically visit organizational locations infrequently. Mystery shopper programs

using only one visit per unit per year spread across several hundred organizational locations as

would be typical for a bank,restaurant, or hotel, provide management with an overall assessment of

the organization’s ability to meet its quality standards. Analysing these data over time allows

managers to discover whether service quality delivery is improving or not for the overall

organization. Because of this limited frequency of shopper visits, the data are generally considered

useful only for assessing organizational level performance and not for individual units or a specific

employee.

Benefits of Frequent Mystery Shops

Advantages of mystery-shopping can be summarised as follows:

1. In contrast to attitudinal surveys of customers, the mystery-shopping approach is

being used to measure the process rather than the outcomes of a service

encounter (Wilson, 1998). The emphasis is on the service as it unfolds, looking

at which activities and procedures do or do not happen rather than gathering

opinion about the service experience (Miles, 1993).

2. The literature review reveals that consumers are more interested in functional qualities, as

opposed to technical qualities. Hence mystery-shopping is a very suitable methodology as it

provides more objective measures of service delivery.

Weaknesses of mystery-shopping in Internet banking services can be summarised as

follows.

1. The entire outcome of the exercise depends on the quality of the data collection

instrument. Careful design is required to give comprehensive coverage. The

omission of a crucial factor may yield spurious results.

2. Because the methodology does not solicit views from a sample of respondents, it

lacks the diversity of open-ended answers and suggestions.

3. Inevitably, some measures in a mystery-shopping exercise produce subjective evaluations. While

it is impossible to remove the subjective element, it can be reduced by formulating appropriate

measures that have objective underpinnings.

Because of the data limitations of the typical mystery shopper program, some organizations have

increased their frequency to gain better feedback. Cheesecake Factory and Restaurant Partners

Incorporated, for example, employ mystery shoppers a minimum of ten times monthly for each

location. These data offer several important advantages for organizations willing to spend the

money to obtain them. First, mystery shoppers can make a relatively objective assessment of all

aspects of the service experience, especially an employee’s performance in the customer encounter.

Because the mystery shopper is asked to assess the experience against preset service standards, a

related benefit is that using a mystery shopper requires organizations to take the time to identify

exactly what they expect to deliver to customers in their service offering. There is considerable

value for the organization in defining what the service experience should be for its customers by

identifying the aspects of the service experience that are key to its customers’ satisfaction, re-

patronage, and positive word of mouth.

These key drivers can be used to ensure that the service standards assess delivery of the service

experience expected by the customer. Second, once service standards are defined, a mystery

shopper can use a customized form to assess the organization’s effectiveness. Every shopper uses

the same measures for every shop. These data also allow the organization to make performance

comparisons across time periods, employees, and organizational units. While mystery shoppers are

effective in assessing the more subjective standards of service, such as employee friendliness or

helpfulness, they are especially effective in evaluating the objective aspects of the experience, such

as the cleanliness of the facility or the number of seconds before being greeted at a particular

location. A third, and perhaps the most important but underutilized, advantage of this technique is

that it provides third-party data for managers to use in coaching their employees. Feedback from a

mystery shopper provides a relatively objective basis for a manager to praise good and correct

poor performance. between the employee and the assessor are minimized, as

both are strangers to one another. And, the employee is unaware that his or her performance is being

formally assessed. A large private school group, for example, uses mystery phone calls to individual

school offices to reveal how enrolment inquiries are handled. An international resort organization

uses mystery audits of travel agents to learn how often agents recommend their properties. In each

case, the feedback is used to coach individuals about their performance.

A fourth advantage of more frequent use of mystery shoppers is that the data they provide remove

much of the perceived subjectivity of the appraisal process. Because mystery shop feedback is

provided anonymously on the performance of a particular server, mystery shop data are less

subject to traditional appraisal errors made by supervisors than the typical performance review

process. The manager can offer employees specific feedback anchored to specific performance

standards without appearing to be making ad hominen attacks.

Fifth, most mystery shop programs can provide rapid, usually overnight, feedback. This information

can be shared with an employee while the customer encounter assessed by the mystery shopper is

still fresh in the employee’s mind. A sixth benefit of using a mystery shopper is based on the

proven psychological value of knowledge of one’s performance. Knowledge of results almost

always leads to improved employee performance because it improves learning, as well as

motivation through subsequent goal setting. Finally, because employees do not know when their

performance will be assessed by a mystery shopper, the feedback provided allows managers to

coach on a ‘‘variable interval (VI) schedule.’’ Since employees, can’t predict when or who

is assessing their performance, they can’t engage in impression management by ‘‘dressing up’’ their

performance for an evaluator. While employees know that both the performance observation and the

coaching feedback will happen sometime, they don’t know when. Feedback is expected but

unpredictable. Mystery shopper programs allow managers to gain the power of variable interval

schedules in ways that have not previously been possible outside the laboratory.

Weaknesses of Mystery-Shopping

Weaknesses of mystery-shopping in can be summarised as follows;

1. The entire outcome of the exercise depends on the quality of the data collection

instrument. Careful design is required to give comprehensive coverage. The

omission of a crucial factor may yield spurious results.

2. Because the methodology does not solicit views from a sample of respondents, it

lacks the diversity of open-ended answers and suggestions.

3. Inevitably, some measures in a mystery-shopping exercise produce subjective

evaluations. While it is impossible to remove the subjective element, it can be

reduced by formulating appropriate measures that have objective underpinnings.

Conclusion

A mystery shopper intervention accomplishes at least three important aspects of any performance

appraisal system. First, it requires the specification of the standards of employee behaviour in clear

and measurable terms. Second, it provides performance data that can be used to coach employees.

Third, the performance assessments are based on observations from a relatively dispassionate third

party. This means that the conversation in a coaching session starts off with the comment ‘‘you were

evaluated, and here are the results’’ rather than ‘‘here is what I think of your performance.’’ It is less

stressful in an evaluation process when the data used come from a neutral source. This enables the

manager to focus on the implications of the feedback, and coach improvements in performance

instead of focusing on the impact the feedback might have on the emotions and of the person being

evaluated. It is impractical to constantly coach all employees on desired behaviour. Hence, the

obvious lesson learned from our research is that mystery shopping is a powerful technique

for managers to use in providing employee third-party feedback on a variable interval schedule in a

service organization.

Bibliography and References

Bateson, J. (1977), Do We Need Service Marketing?, Marketing Consumer Services: New Insights, Report 77-115, Marketing Science Institute, Boston

Brown, C., Sopp, L. and Gould, 5. (1997), Consumers' Association Goes Undercover, Consumer Policy Review, 7(1), pp. 2-6

Dawson, J. and Hilliar, J. (1995), Competitor Mystery Shopping: Methodological Considerations and Implications for the MRS Code of Conduct, Journal of the Market Research Society, (October), pp. 4 17-427

Dorman, K. G. (1994), Results Can Shape Your Future, Bank Marketing, (August), pp. 17-21

Dwek, R. (1996), Magic of Mystery Shopping, Marketing, 17(October), pp. 41-44

Esmail, A. and Everington, S. (1993), Racial Discrimination Against Doctors From Ethnic Minorities, British Medical Journal, 306, 13(March), pp. 69 1-692

Friedrichs, J. and Ludtke, H. (1975), Participant Observation, Lexington, MA, USA

Grönroos, C. (1993), Toward a Third Phase in Service Quality Research: Challengesand Future Directions, in Teresa A. Swartz, David E. Bowen, and Stephen W. Brown (Eds.), Advances in Services Marketing and Management: Research and Practice, Vol. 2, pp. 49-64, JAI Press, Greenwich, CT

Grove, S.J. and Fisk, R. (1992), Observational Data Collection Methods for ServicesMarketing: An Overview, Journal of the Academy of Marketing Science, 20(3),pp. 217-24

Jorgensen, D. L. (1989), Participant Observation: A Methodology for Human Studies,Sage Publications, Newbury Park, CA

Leeds, B. (1995), Mystery Shopping: From Novelty to Necessity, Bank Marketing,(June), pp. 17-23

Reed, K. and Miles, L.R. (1995), How Mystery Shoppers Support 'The Brenton Way' ofCreating and Defining a Service Culture, Journal of Retail Banking Services,Winter, 17(4), pp. 21-28

Wilson, A and Gutman, J. (1998), Public Transport: The Role of Mystery Shopping inInvestment Decisions, Journal of the Market Research Society, Oct. 40(4),pp.285-95

Wilson, A. (1998), The Use of Mystery Shopping in the Measurement of Service Delivery, Service Industries Journal, 18(3), pp. 148-163