When Rewards Encourage Professional Skepticism (And When They Do Not)

Joseph F. Brazel

North Carolina State University

jfbrazel@ncsu.edu

Justin Leiby

University of Georgia

jleiby@uga.edu

Tammie J. Schaefer

University of Missouri – Kansas City

schaefertj@umkc.edu

April 2017

We thank Shana Clor-Proell, Rick Hatfield, Steve Kachelmeier, Chad Proell, Dick Riley, Ben

van Landuyt, Brian White and workshop participants at the Spring 2015, Fall 2015, Fall 2016

Fall Meetings of the Institute for Fraud Prevention (IFP), Texas Christian University, and the

University of Texas. Part of this research was supported by a grant from the IFP. All results,

interpretations, and conclusions expressed are those of the authors alone, and do not necessarily

represent the views of the IFP. We also thank the audit professionals who participated in our

experiments and surveys and Sadie Rockefeller for research assistance.

When Rewards Encourage Professional Skepticism (And When They Do Not)

Abstract

There are widespread concerns about insufficient professional skepticism. At the same time,

auditor incentive systems often do not reward the application of appropriate skepticism,

especially in high-pressure settings in which skepticism has concrete costs, but does not identify

a misstatement (“costly skepticism”). We examine two mechanisms through which rewarding

costly skepticism could affect subsequent skepticism: (1) increasing skepticism by increasing the

motivation to be skeptical or (2) causing auditors to believe the reward improves their rank

relative to others and then, to preserve this benefit, be less skeptical in subsequent audit tasks.

We find evidence of the second effect. Specifically, those receiving positive evaluations for

costly skepticism interpret the positive evaluation as a substantial benefit that they wish to

preserve and, in turn, subsequently engage in less skepticism to avoid losing the benefit. This

effect also decreases the likelihood that auditors either adjust skepticism to underlying

misstatement risk or inform their supervisor about risky items, thus compromising audit quality.

Encouragingly, we do find an “increased motivation” effect among auditors who report high

levels of personal experience being rewarded for costly skepticism, though this experience does

not negate the aforementioned negative effect. We conduct a supplemental survey demonstrating

that supervisors are likely to reward costly skepticism when their own supervisors encourage the

behavior and promote consultative relationships with subordinates. This suggests that firms can

benefit from a culture shift in performance evaluations by promoting consistent rewards for

costly skepticism. Otherwise, audit supervisors face the dysfunctional possibility that they may

not get what they reward.

Keywords: performance evaluation, professional skepticism, motivation, strategic behavior

Data Availability: Contact the authors.

1

I. INTRODUCTION

Professional skepticism improves audit quality, and enhancing skepticism is of great

concern to audit regulators, practitioners, and scholars (e.g., Nelson 2009; IAASB 2012; PCAOB

2012; IAASB 2015; KPMG 2016). However, auditors’ incentives can constrain skepticism

because it often has intangible benefits but concrete costs – as appropriate skepticism often does

not identify misstatements and can cause delays, budget overages, and strained client relations

(Nelson 2009; Peecher et al. 2013). We use the term costly skepticism to describe appropriate

skeptical behavior that generates such costs, but does not ultimately identify a misstatement.1 It

is important to reward appropriate skepticism, regardless of outcome, in order to encourage

future skepticism in high-pressure settings (Hurtt et al. 2013). Yet, audit supervisors typically

punish such behavior (Brazel et al. 2016). Given this problem, theory offers a simple solution;

rewarding subordinates’ costly skepticism should help motivate further skeptical behavior

(Prendergast 1999).

Nevertheless, we argue that there are conditions in which rewards are likely to increase

or decrease further skepticism. We focus on two causal mechanisms of rewards. First, we

examine whether rewards increase auditors’ motivation to engage in skepticism and, in turn,

increase skeptical behavior. However, we also examine if rewards for costly skepticism may

decrease subsequent professional skepticism. Over the course of their careers, auditors work for

a variety of supervisors who likely vary in the extent to which they positively evaluate costly

skepticism. On-average, though, auditors have strong priors that supervisors will not reward

1 Brazel et al. (2016) specifically test and illustrate that skepticism that incurs costs and does yield a misstatement is

viewed by superiors as beneficial to the audit team and framed as a “normal cost.” On the other hand, skeptical

behavior that does not identify a misstatement is perceived as not being beneficial and “lost time.” So while all

skepticism bears costs, what we refer to as “costly skepticism” bears those costs without any tangible benefit.

2

costly skepticism (Brazel et al. 2016). Moreover, auditors are effective at managing impressions

with supervisors (Rich et al. 1997; Mayhew 2001; Tan and Jamal 2006). Accordingly, they may

react strategically to one-time rewards for costly skepticism, interpreting them as unanticipated

benefits that subsequent skepticism (if it ends up being costly skepticism) may jeopardize. As a

result, auditors may be less willing to act skeptically after a supervisor rewards their costly

skepticism. Despite the adage, you may not always get what you reward. Audit firm cultures and

evaluation systems may inadvertently create dysfunctional incentives for skepticism.

We tested our hypotheses in an experiment with 112 audit seniors who completed an

experimental case study related to a test of a hypothetical client’s year-end sales account. At the

onset of the case, the auditors were informed that this was their first year serving on the

engagement (i.e., they had no prior year experience with the supervisor). The auditors were told

that they engaged in an act of costly skepticism during interim testing. Half the auditors were

told they received a higher performance evaluation for this work, while the other half were told

they received a lower evaluation. The case then provided the auditors with extensive client

information, including financial data and non-financial measures (NFMs, such as number of

customer accounts, patents, and employees). We held sales growth constant at 9%, while NFM

growth for the client was negative in all cases, creating a potential red flag/evidence

inconsistency. We manipulated the trend in NFMs to be increasingly negative, specifically as a

low, moderate, or severe red flag, to vary misstatement risk (Brazel et al. 2009). This enabled us

to test the appropriateness of auditor skepticism (i.e., more skepticism as the red flag/evidence

inconsistency becomes more substantial). Our primary measure of skepticism was whether or not

the auditors, upon completion of their analytical procedure, concluded that the sales account

warranted additional investigation.

3

Our predictions are two oppositely-signed indirect effects, with rewards increasing

skepticism through increased motivation and rewards decreasing skepticism through the

preservation of an unanticipated benefit. We do not find a motivation effect. However, our

evidence is consistent with an indirect effect in which higher (as opposed to lower) evaluations

of costly skepticism inflate auditors’ perceptions of their ranking relative to peers and, in turn,

decrease auditor skepticism. When auditors receive higher evaluations of costly skepticism, we

also observe that there is a disconnect between the recognition of red flags and the exercise of

skepticism. When auditors receive lower evaluations, on the other hand, there is the expected

positive relation between the recognition of the evidence inconsistency and the exercise of

skepticism. This evidence points to a strategic effect, as auditors receiving higher supervisor

evaluations recognize the heightened risk of the red flag, but choose not to act on it.

To gain more insight into audit quality implications, we conduct supplemental tests

revealing that the theorized indirect effect reduces auditors’ willingness to hierarchically escalate

the NFM red flag to their audit manager. Failing to bring conditions that indicate a possible

misstatement to the attention of their supervisor impairs firms’ quality controls and increases

audit risk (Brazel et al. 2014; Nelson et al. 2016; Lightle et al. 2017). Further, auditors who

believe higher evaluations for costly skepticism inflate their ranking are less likely to exercise

appropriate skepticism (i.e. not inclined to be more skeptical as the risk of misstatement

increases).

Identifying this indirect effect allows us to better understand the relation between

performance evaluation and skepticism and the causal mechanisms at play. This, in turn, allows

us to develop more effective interventions to encourage skepticism and understand the conditions

under which they are likely to work. We do observe a positive effect of higher evaluations after

4

controlling for the mediating variables, suggesting that positive evaluations have the potential to

increase the exercise of skepticism. For example, our results suggest that one-time rewards for

costly skepticism do not increase auditors’ motivations to be skeptical because auditors are wary

that subsequent costly skepticism will be punished. This raises the question of whether long-term

experiences with supervisors that consistently reward costly skepticism can produce staff that are

more apt/motivated to be skeptical?

To examine this question, we measured our experimental participants’ own personal

experiences being rewarded by their supervisors for costly skepticism. While experience with

supervisors that consistently reward costly skepticism may not completely reverse the

dysfunctional response we observed, such experience may increase motivation for skepticism

and offset the dysfunctional response to some extent. Indeed, we find that auditors with a high

level of experience vis-à-vis rewards for costly skepticism are more motivated to exercise

skepticism, regardless of the evaluation they receive in the case materials. While this experience

is associated with higher skepticism, it does not completely counteract the negative, ranking-

induced strategic effect that decreases skepticism. Given the prevalence of supervisors who do

not reward costly skepticism, auditors most likely still consider their own best interest and forego

exercising skepticism if they are unsure about their supervisor. This result suggests that

skepticism is a behavior developed over time and likely fostered by consistent rewards for costly

skepticism. Overall, our results highlight the necessity for a culture shift where rewarding costly

skepticism is the norm, not an anomaly.

Finally, given the importance of auditors’ personal experience being rewarded for costly

skepticism, we conducted a case-based survey of 127 auditors to explore the attributes of

supervisors who are likely to consistently reward costly skepticism. In the survey, supervisors

5

read a brief case in which a subordinate discovers and investigates a red flag similar to the

aforementioned NFM red flag. The discovery and investigation of the red flag by the subordinate

led to a budget overage and impaired client relations, but ultimately did not identify a

misstatement. Supervisors were then asked to evaluate the subordinate. We observe that higher

evaluations of costly skepticism are associated with experiences in more consultative work

environments. Related to the importance of “tone at the top,” supervisors who believe that their

own audit partner would reward them for a subordinate’s costly skepticism provide higher

evaluations. This is consistent with our experimental findings that long-term experiences with

supportive management can motivate professional skepticism.

As a whole, our results suggest that auditor performance evaluation systems prompt

strategic, dysfunctional responses from auditors, creating a vicious cycle in which positive

evaluations of professional skepticism can discourage skepticism even further. This cycle is

likely to become more crucial to audit quality as data analytics and investigating anomalies begin

to play a larger role in the audit process. However, our results also suggest a solution: a culture

shift towards a consistent emphasis on positively evaluating skepticism, regardless of whether it

ultimately leads to the identification of a misstatement. Subordinates who regularly experience

such evaluations are more motivated to engage in skepticism.

Further, evaluators who experience consultative work environments give higher

evaluations to subordinates who engage in costly skepticism. As a result, a culture supportive of

costly skepticism and consultation when red flags are encountered may create a virtuous cycle in

which supervisors promote skepticism, subordinates exhibit skepticism, and the skepticism is

evaluated positively. Thus, it is critical for firms and engagement management to provide

assurance that appropriate levels of skepticism will be rewarded, regardless of outcome, and that

6

audit firms craft quality control systems, mentoring programs, and training mechanisms to

promote consistent rewards for costly skepticism.

Our counterintuitive finding that positive evaluations of skepticism can lead to less

skeptical behavior also provides insights into how subordinates respond to pervasive, anticipated

evaluator bias. Countering such biases is critical for any professional organization attempting to

achieve more efficient evaluations – evaluations that direct attention towards and reward

desirable behavior. This extends the stream of research investigating, for example, the

unintended effects of auditor supervision and review (e.g., Tan and Jamal 2001; Wilks 2002;

Peecher et al. 2010). Last, our empirical findings should inform those concerned with motivating

skepticism in other low base rate contexts (e.g., audit committees questioning management about

financial reporting issues, manufacturers’ quality controls for detecting defects, analysts or

agents screening for security threats).

II. THEORY AND HYPOTHESES DEVELOPMENT

Exercising professional skepticism is fundamental to auditing and increases audit quality

(e.g., PCAOB 2012; IAASB 2012; IAASB 2015; KPMG 2016). However, regulators

consistently note insufficient auditor skepticism on audit engagements and deficiencies in the

degree to which firms’ systems of quality control encourage skepticism (e.g., PCAOB 2012;

IAASB 2015). Exercising professional skepticism can be costly (to the firm and to the individual

auditor) if skepticism leads to increased testing that causes budget overruns and strained client

relations (Houston 1999; Nelson 2009; Peecher et al. 2013).2 Given low base rates of material

2 Illustrating the costs of skepticism, Lambert et al. (2017) provide the following quote from a retired audit partner:

“I was very concerned about the risk that long hours might adversely impact the degree of professional

skepticism maintained by the staff. Our auditors were very busy and they recognized that pushing the client for more

answers in areas being audited today would only delay the client's delivery of schedules needed for audit areas

scheduled to be started tomorrow.”

7

misstatements, many skeptical actions are directed towards fairly-stated accounts and can

generate immediate, concrete costs. By contrast, it is uncommon for skepticism to yield the

tangible benefit of detecting a material misstatement.3 Thus, the benefits of skepticism are often

intangible. This study takes particular interest in conditions involving budget/client pressures and

the presence of evidence inconsistencies, which represent the conditions in which skepticism is

most likely to be costly and in which the failure to exercise skepticism is a substantial threat to

audit quality.

Individual auditors confront the reality that skepticism is not likely to yield the tangible

benefit of detecting a misstatement. In addition, Brazel et al. (2016) find evidence of outcome

bias among audit supervisors who view costly skepticism as appropriate, but nonetheless

evaluate it negatively relative to skepticism that detects a misstatement. The study also provides

evidence that staff auditors anticipate negative evaluations when they exercise skepticism, incur

costs, and ultimately do not identify a misstatement. For the individual auditor, there is a

relatively low likelihood that skeptical actions will detect a misstatement, and in conditions of

budget or client pressure, there is a relatively high likelihood that skeptical actions will turn out

to be costly. Thus, under conditions where skepticism should be exercised (e.g. a red flag or

evidence inconsistency is present) and pressures exist, auditors will consider it likely that their

skepticism will be evaluated negatively (Brazel et al. 2016). As a result, performance evaluation

systems may not motivate the application of skepticism.4

3 Consistent with this argument, 26% of the auditors in our experiment reported that they have never detected a

material misstatement and over 50% reported to have detected two material misstatements or fewer. Skepticism also

tends to decrease as auditors gain experience (Shaub and Lawrence 1999), which is likely due in part to auditors

experiencing settings with no misstatements, thus auditors are both motivated and skilled at not setting off “false

alarms” (Solomon et al. 1999).

4 Brazel et al. (2016) also illustrate that corporate managers view the time spent by their employees in response to a

skeptical auditor as lost time if the investigation does not identify a misstatement. In turn, managers are more likely

to convey negative information about the audit staff to the audit partner.

8

However, theory suggests that a simple solution to increase skepticism would be to

evaluate costly skepticism positively. Performance evaluations can serve as a key mechanism to

provide auditors with incentives to exercise skepticism. More specifically, recognizing

skepticism and evaluating auditors based on their level of skepticism can motivate auditors to

allocate effort towards skepticism. This is consistent with the idea that rewarding such behavior

should encourage more of that behavior (e.g., Prendergast 1999).

In other words, positively evaluating the appropriate application of costly skepticism and

tying rewards to the positive evaluations should increase auditors’ motivations to engage in the

behavior (e.g., Prendergast 1999). By rewarding costly skepticism, the supervisor signals to the

auditor that applying skepticism, regardless of the outcome, is a necessary component of a high

quality audit. Accordingly, similar to the adage “you get what you reward,” we expect auditors

who receive positive evaluations of costly skepticism on a given engagement to be more

motivated to exercise skepticism in subsequent tasks. This motivation should, in turn, result in

the auditors being more willing to engage in further skepticism on a given engagement. This

leads to our first hypothesis, stated formally:

H1: A supervisor rewarding costly skepticism increases the auditor’s motivation to

be skeptical and, in turn, increases the auditor’s subsequent skepticism on the

engagement.

How Positive Evaluations Can Decrease Skepticism

Hypothesis 1 follows the logic that positively evaluating costly skepticism should instruct

auditors about the importance of skepticism and motivate auditors to exercise skepticism on

future tasks. However, it is also possible that positive evaluations could decrease auditors’

willingness to engage in subsequent skeptical behavior, as auditors likely have strong priors that

they will confront negative evaluations of costly skepticism. That is, auditors may view a

9

positive evaluation as an unanticipated benefit likely to be jeopardized by subsequent skeptical

behavior.

To better understand this possibility, we must first understand how auditors interpret

evaluations of costly skepticism in the broader context of auditor incentive systems. Auditors

typically work on multiple engagement teams for varying amounts of time, receiving evaluations

from multiple supervisors, and sometimes receiving multiple evaluations for the same

engagement. Audit firms typically combine the various evaluations for a given auditor into a

single, yearly evaluation and then rank-order the auditor against others at the same experience

level. Firms then use these rankings to sort auditors into groups to determine rewards such as

raises, bonuses, promotions, client assignments, etc. Ranking order is often determined by slight

variations in individual performance because firms have high expectations and most auditors are

relatively competent. This is consistent with prior research that finds, in ranking systems, the

sign and not the magnitude of a performance difference is critical to employee motivations (e.g.,

Holmstrom 1981; Cichello et al. 2009). In other words, it matters that you are ahead of the next

person, not necessarily how far ahead.

Given the importance of performance evaluations and auditor beliefs that penalties for

costly skepticism are common (Brazel et al. 2016), positive evaluations for costly skepticism

may actually provide disincentives to engage in skepticism in subsequent audit tasks. Auditors

are effective at strategically cultivating positive impressions with supervisors (Rich et al. 1997;

Mayhew 2001; Tan and Jamal 2006). If auditors typically expect penalties and not rewards for

costly skepticism, then an unexpected positive evaluation likely leads auditors to interpret their

approach to subsequent tasks from a “gain frame” (e.g., Kahneman and Tversky 1979).

10

Accordingly, auditors will act cautiously to avoid jeopardizing this unexpected benefit.5 In this

situation, a consideration of the effort-to-reward ratio suggests auditors who act skeptically have

more to lose than they have to gain, as the effort required for subsequent skeptical action is

unlikely to translate into rewards. That is, because misstatements are infrequent, it is quite

possible that subsequent skepticism will again be costly and evaluated negatively (Brazel et al.

2016).

Although a positive evaluation of costly skepticism may signal that the supervisor wants

the auditor to exercise skepticism where appropriate, people are sensitive to base rates in

determining how to respond to evaluation systems (Berger et al. 2013). Base rates of rewards for

costly skepticism are not high. Thus, a one-time positive evaluation is not likely to overcome the

general perception that costly skepticism will be penalized. In Bayesian terms, auditors have

weak priors that skeptical behavior will ultimately be rewarded. Although an initial positive

evaluation of costly skepticism from a supervisor could signal that the supervisor will also

reward subsequent costly skepticism behavior, this signal is noisy. Auditors who receive an

unanticipated benefit (e.g., are initially rewarded when exercising costly skepticism) will likely

adopt a more strategic mindset and deem the evaluation signal not sufficiently credible to

warrant updating their beliefs. This, in turn, diminishes the auditor's willingness to engage in

subsequent skepticism that risks jeopardizing the ranking benefit. Stated formally,

H2: A supervisor rewarding costly skepticism increases the auditor’s perceived

ranking and, in turn, decreases the auditor’s subsequent skepticism on that

engagement.

The Appropriateness of Skepticism

5 From a practical standpoint, the risk averse auditor will likely search for, evaluate, and document evidence that is

consistent (vs. inconsistent) with a client’s reported balance. As such, the experimental instrument used to test our

hypotheses contains both consistent and inconsistent evidence for the auditor to consider (see Section III).

11

We consider skepticism to be more appropriate when the auditor directs skepticism

towards evidence that is a relatively stronger, as opposed to weaker, indicator of heightened

misstatement risk. Nelson (2009) observes that a critical aspect of skepticism is the auditor’s

ability to appropriately direct presumptive doubt towards a valid indicator of heightened risk.

Though we hypothesize effects of performance evaluations on skepticism, our theory does not

provide a directional prediction for whether or not our posited effects are likely to affect the

appropriateness of skepticism. Accordingly, we discuss the implications of our theory when cues

of misstatement risk vary in severity.

In general, auditors can assess a broad range of evidence items and adjust skepticism

accordingly (e.g., Bonner et al. 1997; Bell et al. 2005; O’Donnell and Schultz 2005).

Consequently, if motivation increases as a result of positive evaluations (as predicted in H1),

then auditors should process evidence more effectively and better differentiate between low and

high risk items. This would enhance the appropriateness of professional skepticism. On the other

hand, it is common for auditors to approach tasks expecting there to be no misstatement. If this is

the case, then they are unlikely to adjust skepticism in response to cues (Earley 2001). Indeed,

concern over proper relations between assessed risks and the level of skepticism is evident in

Glover and Prawitt’s (2014) proposal of a “professional skepticism continuum.” The continuum

links the appropriate application of professional skepticism to relevant risks. Over time, auditors

also develop mental models for reasoning that there are non-misstatement explanations for

evidence inconsistencies (Kaplan and Reckers 1989; Kaplan et al. 1992). As a result, increased

motivation may have no effect on the tendency to discriminate between high and low risk items.

Recall that H2 predicts that auditors will act strategically to preserve an unanticipated

benefit, resulting in decreased skepticism. On one hand, auditors are prone to motivated

12

reasoning and rationalize that biased judgments are actually objective (Kadous et al. 2003).

However, as risks of misstatement increase, auditors may be less able to rationalize that

preserving one’s ranking is a reasonable excuse for diminished skepticism (e.g., Kadous et al.

2003). Conversely, given the low rate of misstatements, it is also plausible that auditors could

reason that there is a low probability that diminished skepticism will result in an undetected

misstatement, regardless of the apparent risk. Thus, the indirect effect predicted in H2 could lead

auditors to not discriminate between low and high risk items.

This leads to a research question, for which we do not have a directional hypothesis,

RQ: Does a supervisor rewarding costly skepticism affect the appropriateness of

auditors’ further skepticism on that engagement?

III. AUDITOR EXPERIMENT

Purpose

The purpose of this experiment was to test whether auditors respond to a supervisor’s

evaluation rewarding costly skepticism by increasing or decreasing their skepticism on a

subsequent task on that engagement.

Participants

The participants were 112 audit seniors from an international accounting firm. We

administered the experiment during a firm-sponsored training session. The task was a substantive

analytical procedure related to sales, which is a task appropriate for our participants’ level of

experience (Trompeter and Wright 2010; Brazel et al. 2014). The average completion time was

34 minutes. On average, our participants had 34 months of audit experience and conducted

analytical procedures for sales four times during their careers. Three participants did not

complete at least one of the variables necessary for our hypothesis tests, thus our final sample is

109 auditors.

13

We randomly assigned participants to one of six experimental conditions. Our experiment

was a 2 (Evaluation: positive, negative) X 3 (Red Flag: low, moderate, severe) between-

participants design. We describe the variables in more detail below.

Description of the Experimental Context

The experimental materials informed participants that they were the audit senior for the

hypothetical audit of Madison, Inc., a publicly traded manufacturing company, and that this was

the participant’s first year serving on the Madison engagement (i.e., there was no prior

evaluation history with the engagement or the audit manager). Participants were then told that

they had engaged in costly skepticism while conducting interim procedures at Madison.

Specifically, they had been auditing PP&E and, when recalculating depreciation expense, had

noted inconsistencies in the useful lives employed by the company for several classes of PP&E.

Although the client provided supporting internal documentation, participants had deemed it

appropriate to investigate further by: (1) requesting data from the PP&E manufacturer to support

the changes in useful lives (with company management having to serve as a liaison) and (2)

obtaining industry/competitor data to support the change.

After this testing, the changes to useful lives were deemed reasonable and no material

misstatement was identified. However, the additional investigation caused friction between

management and the engagement team and led to the audit of PP&E being significantly over

budget (i.e., costly skepticism). Participants were then informed of the evaluation they received

from their manager on Madison for their interim testing (see below for the description of the

EVAL manipulation).

Next, participants were told that their primary task was a substantive analytical procedure

related to the sales account for one of Madison’s operating units, Madison Sporting Goods.

14

Participants were informed that additional testing of sales would go over budget and that client

management was likely to react negatively to any unwarranted testing of sales. All financial

measures available to develop an expectation for the account were consistent with the client’s

sales account balance (e.g., industry data, prior year balances, ratios, budgets). Participants were

also provided with non-financial measures (NFMs) for Madison Sporting Goods such as the

number of customer accounts, patents, and employees. We manipulated between participants the

extent to which these NFMs were inconsistent with the client’s recorded sales balance (see below

for the description of the RED FLAG manipulation).

Dependent Variable

After completing the analytical procedure (i.e., calculating the difference between their

expectation and the reported balance), participants provided the study’s primary dependent

variable: SKEPTICISM. Specifically, participants chose one of three options for their conclusion

about whether or not the sales account warranted additional investigation: (1) “The difference is

IMMATERIAL and the balance appears reasonable,” (2) “The difference is IMMATERIAL, but

additional work would be required related to this analytical procedure before concluding the

balance appears reasonable” or (3) “The difference is MATERIAL and more work is required

related to this analytical procedure.”

We coded SKEPTICISM as “1” if participants concluded that more work would be

required to conclude the balance was reasonable (i.e., options (2) or (3) above), and “0” if

participants concluded the balance appeared reasonable and required no additional work (i.e.,

option (1) above).6 Coding responses that suggest additional work as higher skepticism is

6 We dichotomized this variable because only nine participants chose option (3) and interpretation is easier with two

levels of the variable. Our inferences are the same if we use a multicategorical dependent variable with three levels

instead of two.

15

consistent with the literature related to professional skepticism stressing the importance of

evaluating inconsistent evidence (e.g., IAASB 2004; Nelson 2009).

Supervisor Evaluation Rewarding Costly Skepticism (H1 and H2)

In this study, we test dual mediation hypotheses. Hypothesis 1 predicts that a positive

supervisor evaluation of costly skepticism (EVAL) increases auditors’ motivation to engage in

skeptical behavior (MOTIVATION) and, in turn, indirectly increases SKEPTICISM. Hypothesis 2

predicts that a positive supervisor evaluation of costly skepticism (EVAL) increases auditors’

beliefs about their own ranking (RANKING) and, in turn, indirectly decreases SKEPTICISM. We

manipulated EVAL after participants read the description of their costly skepticism related to

PP&E testing at interim (i.e., before participants performed the year-end analytical procedure

described above). Participants were randomly assigned to receive an interim evaluation of either

“Met Expectations” (LOWER) or “Exceeded Expectations” (HIGHER) from their supervisor.7

MOTIVATION is measured with the following question:

To what extent were you motivated to exercise professional skepticism when auditing the

12/31/12 Madison sporting goods sales account?

Participants responded on a scale ranging from 1 (Not at all motivated) to 10 (Extremely

motivated).

RANKING is measured with the following question:

How would you rank in your class (compared to other seniors in your office) if you

consistently received the above evaluation?

7 Bretz et al. (1992) find that most organizations use a five-level evaluation system, but tend to only use the top three

levels, with two-thirds of employees in the top two levels. Brazel et al. (2016) describe how auditor performance

evaluations are similarly positively skewed such that “Below Expectations” ratings are rarely given and auditors who

typically receive “Met Expectations” are likely ranked low in their class.

16

Participants responded on a scale ranging from 1 (Bottom of My Class) to 10 (Top of My Class).

Appropriateness of Skepticism – Severity of Red Flags

To examine the appropriateness of SKEPTICISM, we manipulated the severity of the

NFM RED FLAG (evidence inconsistency) in the case. Sales growth was constant at 9% in all

conditions, and all conditions included NFMs (e.g., number of employees, customer accounts)

with negative growth, inconsistent with reported sales. We manipulated the NFM trend to create

three levels of RED FLAG: LOW, MODERATE, and SEVERE. NFM growth for the LOW,

MODERATE, and SEVERE conditions was -1%, -21%, and -41%, respectively.

Primary Results

Manipulation Checks

For EVAL, in a post-test we asked participants whether they received a “Meets

Expectations” or “Exceeds Expectations” evaluation for their prior work, and 97 of the 112

participants (86%) correctly recalled the evaluation. As a manipulation check for RED FLAG, a

post-experimental question asked participants to assess the trend in NFMs from 1 (“very

negative”) to 10 (“very positive”). Lower values indicate a more negative trend. Consistent with

an effective manipulation, there is a significant effect for RED FLAG (F2, 105 = 16.46; p < 0.01)

on NFM trend assessments (F2, 105 = 16.46; p < 0.01), and the assessments decrease

monotonically from the low to moderate to severe conditions.

Tests of Hypotheses

Table 1, Panel A reports univariate values of MOTIVATION, RANKING, and

SKEPTICISM across EVAL conditions, and then broken out across RED FLAG conditions in

Panel B. Related to H1, it does not appear that that HIGHER EVAL leads to greater

MOTIVATION, but consistent with H2, we do see a higher RANKING in the HIGHER EVAL

17

condition. For our research question related to APPROPRIATENESS, in Panel B we do see the

appropriate reaction (higher percentages for SKEPTICISM as RED FLAG becomes more severe)

in the LOWER EVAL condition. However, the SKEPTICISM of the RED FLAG appears to be

more constant with HIGHER EVAL. Of particular note is that, under a SEVERE RED FLAG,

67% of auditors exhibit SKEPTICISM under LOWER EVAL, while only 44% of auditors exercise

SKEPTICISM under HIGHER EVAL.

Table 2 columns (i-iv) report the results of our tests of H1 and H2. H1 predicts that a

higher supervisor evaluation for costly skepticism increases auditors’ motivation to engage in

skepticism and, in turn, increases auditors’ subsequent skepticism. H2 predicts that a higher

supervisor evaluation for costly skepticism increases auditors’ beliefs about their rank and, in

turn, decreases auditors’ subsequent skepticism. In tandem, H1 and H2 predict that EVAL has

two, opposite-signed indirect effects. To test these hypotheses, we use the following regression

equations:

MOTIVATION = δ1 + β1EVAL+ ε (1)

RANKING = δ2 + β2EVAL+ ε (2)

SKEPTICISM (1 or 0) = δ3 + β3EVAL + β4MOTIVATION + β5RANKING + ε (3)

We use the Preacher and Hayes (2008) bootstrapping approach to test the indirect effects

predicted by H1 and H2. We use 5,000 bootstrap re-samples with replacement to estimate 90%

confidence intervals for each indirect effect, with significance indicated by intervals that do not

include zero. The indirect effect predicted by H1 is the product of β1EVAL*β4MOTIVATION,

which tests whether EVAL increases MOTIVATION and MOTIVATION increases SKEPTICISM.

We expect this product to be positive. The indirect effect predicted by H2 is the product of

18

β2EVAL*β5RANKING, which tests whether EVAL increases RANKING and RANKING decreases

SKEPTICISM. We expect this product to be negative.8

As shown in Table 2 and Figure 1, the results do not support H1. Auditors do not

perceive higher MOTIVATION in response to a higher EVAL. Table 2 (column iii) and Figure 1

report that the product of β1 EVAL*β4 MOTIVATION is insignificant (Lower CI = -0.09, Upper

CI = 0.25 (i.e., the CI includes zero)).

As shown in Table 2 and Figure 1, the results support H2. Auditors believe that a higher

EVAL for costly skepticism inflates their RANKING (p < 0.01) (column ii). Then, as shown in

column (iii) of Table 2, the coefficient on RANKING is significantly negative (p < 0.05).9 Table

2 and Figure 1 report that the product of β2EVAL*β5RANKING is also negative and significant

(Lower CI = -2.08, Upper CI = -0.44 (i.e., the CI does not include zero)). Auditors believe that a

higher evaluation of costly skepticism provides a positional advantage over peers, and are less

willing to engage in subsequent skepticism that could jeopardize this benefit.10 Column (iv)

demonstrates that our H1 and H2 results are robust to including our other manipulated

independent variable (RED FLAG) in the model.

Also, note in Table 2 that EVAL has a marginally positive coefficient after controlling for

the mediators (column iii), which indicates a positive direct effect of EVAL (Preacher and Hayes

2008) on SKEPTICISM. Situations in which the direct and indirect effects have opposite signs

are known as “competitive mediation.” The positive direct effect of EVAL suggests that positive

8 Equations (1) and (2) are OLS regressions and equation (3) is a logistic regression. Thus, we use standardized

coefficients, because the indirect effects involve multiplying coefficients from OLS with coefficients from logit.

9 In equation (3), variance inflation factors for MOTIVATION, EVAL and RANKING are less than or equal to 2.4.

We re-perform individual regressions with only MOTIVATION and RANKING as predictors of SKEPTICISM and

find that the inferences do not change, thus the coefficients used to compute the indirect effect are reliable.

10 We also measure and control for other determinants suggested by Nelson (2009) including trait skepticism (Hurtt

2010) and various dimensions of knowledge such as industry experience, experience with analytical procedures, and

experience with NFMs. Our inferences do not change when controlling for these variables.

19

evaluations for costly skepticism have the potential to increase the exercise of skepticism, if one

could remove the negative RANKING effect.

Related to this notion, Pearl (2005; 2010) suggests a procedure that quantifies “what

could have been” if the negative effects of the mediators would not occur.11 Untabulated results

related to this procedure suggest SKEPTICISM would be 0.75 at HIGHER EVAL if the negative

effects of the mediator would not occur, versus 0.47, a difference of 0.28 (referred to as a

“natural indirect effect”). Thus, the natural indirect effect of RANKING is a decrease in

SKEPTICISM of approximately 0.28. As such, a change in firm culture that reduces the

dysfunctional incentives/reactions to positive evaluations of costly skepticism could create a

setting where HIGHER EVALs are assured to foster greater SKEPTICISM or “you get what you

reward.”

Additional Analyses

In an untabulated analysis, we examine additional audit quality implications—

specifically, how these effects link to the auditor’s intention to communicate the evidence

inconsistency to their manager. Lightle et al. (2017) note that an auditor has not exercised

professional skepticism if s/he, for fear of being wrong, fails to bring conditions that indicate a

possible misstatement to the attention of his/her supervisor. Just before auditors made a

conclusion in the experiment, they wrote down issues related to the account balance that they

would discuss with their manager. We coded a variable INFORM MANAGER that equals “1” if

the auditor explicitly notes that they would discuss the NFM issue with their manager, and “0”

otherwise. We then conduct an additional mediation analysis in which we add INFORM

11 This involves estimating values of SKEPTICISM at HIGHER EVAL that adjust for the indirect effects, and then

estimating the effects on SKEPTICISM attributable to the effects of HIGHER EVAL on MOTIVATION and

RANKING. More specifically, it estimates SKEPTICISM at HIGHER EVAL (taking into account the mediators) and

what SKEPTICISM at HIGHER EVAL would be with the mediators at the level they take at LOWER EVAL.

20

MANAGER as the dependent variable, testing whether EVAL increases RANKING, a higher

RANKING decreases SKEPTICISM, and lower SKEPTICISM decreases INFORM MANANGER.

These analyses confirm that our hypothesized indirect effect through RANKING decreases

INFORM MANAGER. Thus, the indirect effect of H2 decreases the likelihood that auditors

inform their manager about the potential red flag, compromising audit quality.

We next provide evidence that, related to H2, we observe a strategic response by auditors

by providing evidence of divergence between auditors’ recognition of the red flag and their

response to that recognition. This rules out the possibility that auditors’ application of skepticism

stems from differences in their recognition of inconsistent evidence (i.e., the NFM red flag). To

do so, we compare the correlation between this recognition and SKEPTICISM across EVAL

conditions. Auditors assessed RECOGNITION on a 10-point scale as the degree of inconsistency

the auditor perceived between reported sales and NFMs, with endpoints of 1 = “Very small” and

10 = “Very large.” Table 3, Panel A, reports the cell means of RECOGNITION. As expected, in

Panel B we first find that RECOGNITION is positively correlated with SKEPTICISM in the

LOWER EVAL condition (ρ = +0.37). However, RECOGNITION and SKEPTICISM are not

correlated in the HIGHER EVAL condition (ρ = - 0.06). To the extent that auditors in the

HIGHER EVAL condition recognized an inconsistency in the audit evidence, they ignored it and

chose not to exercise skepticism. This is consistent with a strategic effect.

Appropriateness of Skepticism (Research Question)

We next address our research question about whether the indirect effect of EVAL affects

the appropriateness of skepticism. Reducing skepticism towards high risk areas or increasing

skepticism towards low risk areas has the potential to reduce audit quality (under- and over-

auditing, respectively). We create the variable APPROPRIATENESS to indicate situations in

21

which auditor SKEPTICISM is more appropriate given the underlying RED FLAG in the

evidence. Recall that there were three levels of RED FLAG: LOW, MODERATE, and SEVERE.

APPROPRIATENESS equals “1” if either (1) SKEPTICISM equals 1 when the RED FLAG is

MODERATE or SEVERE or (2) SKEPTICISM equals 0 when the RED FLAG is LOW, and equals

“0” otherwise. That is, we code skepticism as more appropriate if auditors are skeptical towards

moderate/severe red flags or are not skeptical towards less severe red flags.12

We then re-conduct our hypothesis tests to evaluate whether EVAL increases RANKING

and MOTIVATION and, in turn, whether RANKING or MOTIVATION respectively affect

APPROPRIATENESS. To do so, we re-estimate equation (3) using APPROPRIATENESS as the

dependent measure. See column (v) of Table 2. We do not observe a positive effect of EVAL on

APPROPRIATNESS that is mediated by MOTIVATION (i.e., the CI includes zero). However, the

results are consistent with a significant indirect effect of EVAL through higher RANKING, which

in turn decreases APPROPRIATENESS (i.e., the CI is negative and does not include zero). That

is, this indirect effect not only decreases SKEPTICISM but also makes auditors less apt to

differentiate how they apply skepticism between higher and lower risk settings. Overall, our

results illustrate that audit firms’ evaluation systems lead to dysfunctional reactions in which

positive evaluations for costly skepticism can undermine the appropriate application of

professional skepticism, which can undermine audit quality.

Supplement: Auditors’ Own Experiences Being Rewarded for Costly Skepticism

12 Less appropriate skepticism could manifest as either under-auditing high-risk areas or over-auditing low risk

areas. Our inferences related to H2 are the same if we conduct our tests using only the MODERATE and SEVERE

RED FLAG conditions, which suggests that diminished skepticism due to RANKING may lead to under-auditing.

Under-auditing is arguably a bigger issue because regulator concerns about auditor skepticism focus primarily on

too little skepticism, rather than too much (e.g., PCAOB 2012). We do not run a separate regression for over-

auditing because there are too few observations in which over-auditing occurred in our experiment.

22

To this point, our findings have been discouraging in that rewards have not caused an

increase in skepticism. This is consistent with our assumption and the findings of Brazel et al.

(2016) that, on average, auditors believe costly skepticism is unlikely to result in rewards. Still,

differences in this belief can develop through experience. Auditors work on multiple

engagements for a variety of supervisors in any given year. Moreover, supervisors vary in their

evaluation biases and preferences (e.g., Tan and Jamal 2001). It is possible that auditors are more

willing to engage in skepticism if they have stronger priors that costly skepticism will be

rewarded. Such auditors may be motivated to engage in skepticism when they observe evidence

inconsistencies, disregarding concerns about the eventual outcome of their tests (i.e., the likely

scenario that no misstatement will be detected). While stronger beliefs that costly skepticism will

be rewarded may not completely reverse the strategic responses to positive supervisor

evaluations of costly skepticism, these beliefs may increase motivation for skepticism and at

least offset this effect to an extent.13

As a result, we test whether consistent experience being rewarded for costly skepticism

modifies the indirect effects observed in H1 and H2. We measure the variable EXPERIENCED

REWARDS post-experimentally by asking participants the extent to which the managers they

work for would reward them for exercising professional skepticism when:

You are over budget, the relationship with management is strained, and your skeptical

behavior DID NOT identify a misstatement.

Participants responded on a scale ranging from 1 (Would not reward skepticism) to 10

(Would definitely reward skepticism). As shown in Panel A of Figure 2, there is a wide range of

13 Although experience with a supervisor consistently rewarding costly skepticism would likely prompt the auditor

to exercise skepticism on that supervisor’s engagement, we do not expect such experience to necessarily eliminate

the indirect effect with other supervisors because of the strong priors that rewarding costly skepticism is relatively

uncommon.

23

responses to EXPERIENCED REWARDS, indicating a wide range of experiences. Note also that

the distribution for EXPERIENCED REWARDS for costly skepticism is skewed more to the left

(not reward) that the distribution for rewards for GENERAL SKEPTICISM (Panel B). To ease

interpretation and the estimation of cell means, we create a dichotomous variable partitioning our

sample on EXPERIENCED REWARDS in the top 25% versus bottom 75%.14

As depicted in Figure 3, we test whether EXPERIENCED REWARDS moderates the

relations between EVAL and our two mediators MOTIVATION and RANKING. Insofar as

EXPERIENCED REWARDS moderates either or both of these relations, then it would be logical

that it would also affect SKEPTICISM. In simple terms, we are interested in whether the good

news effect through MOTIVATION is strengthened, and/or the bad news effect through

RANKING is lessened, if we consider natural differences in EXPERIENCED REWARDS.

We estimate the following equations:

MOTIVATION = δ + β6EVAL+ β7EXPERIENCED REWARDS +

β8EVAL*EXPERIENCED REWARDS ε (4)

RANKING = δ + β9EVAL+ β10EXPERIENCED REWARDS + β11 EVAL*EXPERIENCED

REWARDS + ε (5)

SKEPTICISM (1 or 0) = δ + β12EVAL + β13MOTIVATION + β14RANKING + ε (6)

We provide descriptive statistics partitioned by EXPERIENCED REWARDS and EVAL in

Table 4, Panel A. The results are encouraging, as those high, as opposed to low, in

EXPERIENCED REWARDS have higher MOTIVATION. As shown in Table 4, Panel B, column

(i), EXPERIENCED REWARDS is significantly positively associated with MOTIVATION (p <

0.01). There is also a marginally significant EVAL*EXPERIENCED REWARDS interaction term

(p < 0.10), indicating that those who have experienced consistent rewards for costly skepticism

were relatively highly-motivated to exercise professional skepticism in our study, regardless of

14 Our inferences are identical if we use the continuous measure, but estimating cell means and interpreting marginal

effects is more straightforward when using a dichotomous measure.

24

their evaluation. Those without this experience are more inclined to perceive an increase in

motivation after receiving a positive evaluation for costly skepticism. More importantly, the

index of moderated mediation is marginally significant, indicating that the indirect effect of

EVAL on SKEPTICISM via MOTIVATION depends on the level of EXPERIENCED REWARDS.

The indirect effect of EVAL on SKEPTICISM via RANKING is not affected by EXPERIENCED

REWARDS.

These findings suggest that having more experience being rewarded for costly skepticism

positively affects the motivation to be skeptical, but it does not completely eliminate the negative

effect we observed in our test of H2. In particular, EXPERIENCED REWARDS does not

attenuate the RANKING effect, which is logical. Even if auditors have been consistently

rewarded for costly skepticism, they still realize that a good evaluation is a benefit they want to

preserve. More broadly, this highlights the benefits of a shift in evaluation culture in which

costly skepticism is consistently rewarded. Auditors whose supervisors reward costly skepticism

are motivated to be skeptical, but auditors work with several managers on multiple engagements

for various amounts of time. Given the prevalence of supervisors who do not reward costly

skepticism, auditors are likely to look out for their own best interests, especially if there is any

uncertainty about whether the manager will consistently reward costly skepticism.

Notably, we also measure and find no relation between auditors’ GENERAL

EXPERIENCE being rewarded for skepticism and SKEPTICISM. Thus, increased skepticism

does not stem from the general experience of being rewarded for skepticism, but is rather

motivated by the experience of being rewarded specifically for costly skepticism. If this type of

experience is important for encouraging skepticism, then a natural follow up question is what

25

leads a supervisor to provide positive evaluations for costly skepticism? We address this question

in our follow up survey.

IV. AUDITOR SURVEY – WHO REWARDS COSTLY SKEPTICISM?

Professional skepticism is encouraged by the profession, but skeptical behavior does not

always lead to the same outcome (i.e., sometimes they detect misstatements, but often they do

not). In our experiment, we find that auditors who have more experience being rewarded for

costly skepticism are more motivated to engage in skepticism. The purpose of this case-based

survey is to explore the attributes of audit supervisors (e.g., traits, experiences) who may

consistently reward costly skepticism. Identifying the factors that are associated with supervisors

that provide positive evaluations of costly skepticism can provide insights into how these

supervisors can be recruited and/or developed.

Participants and Procedures

Participants were 127 practicing auditors from two Big Four and two non-Big Four firms.

We collected responses from two of the firms at firm-sponsored training sessions and from the

other two firms via an online survey hosted by Qualtrics. There are no significant effects for firm

or data collection medium on our variables of interest. Participants had 6.3 years of auditing

experience on-average, with 95% indicating experience evaluating the performance of

subordinates.

All participants received the exact same case-based content in their research materials.

The materials asked auditors to evaluate the performance of a third-year staff member under

26

their supervision at a hypothetical audit client from the manufacturing industry.15 The materials

then described the staff member’s performance on a substantive analytical procedure related to a

division’s sales. Similar to our auditor experiment, the information sources used by the staff

auditor in prior-year testing (the division’s own past financial performance and industry financial

data) were consistent with the sales growth reported by the division in the current year.

Nonfinancial measures (NFMs) for the division (e.g., employees, square footage of facilities)

were not considered in prior years.

Participants were informed that the staff auditor incorporated NFMs into his analytical

procedures in the current year, noted an inconsistency between sales growth and related NFMs,

and chose to investigate the inconsistency. The identification and investigation of the

inconsistency caused the staff auditor to go over budget and strain relations with management.

The materials noted that the staff auditor determined that the inconsistency was the result of the

division outsourcing some operations overseas. Participants were then told that the staff auditor

made several inquiries into the matter and collected additional audit evidence, which eventually

led to a conclusion that there were no misstatements in this revenue account. Participants were

then asked to evaluate the staff auditor, who had engaged in appropriate, yet costly, skepticism.16

Measures

The primary dependent measure is participants’ EVALUATION of the staff member:

Based on the information presented on the prior pages, how would you evaluate [the

staff auditor’s] overall performance?

Participants evaluated performance on an 11-point scale ranging from -5 to +5. Verbal

anchors were “Below Expectations” for the left endpoint, “Above Expectations” for the right

15 The materials are adopted from Brazel et al. (2016). 16 See Brazel et al. (2016) for evidence supporting the notion that the skepticism described in our case is deemed

“appropriate” by audit supervisors.

27

endpoint, and “Met Expectations” for the midpoint. As discussed earlier, evaluations in audit

firms and in other settings are positively skewed such that “Met Expectations” would be viewed

as an indication of relatively low performance (Bretz et al. 1992; Brazel et al. 2016). The mean

evaluation for our sample is 1.52, which is similar to the mean observed by Brazel et al. (2016)

when costly skepticism was evaluated by their sample of auditors (1.06).

We follow the Nelson (2009) model to identify and categorize measures that could

influence skeptical behavior/the evaluation of skepticism, which we group into incentives, traits,

and knowledge.17 For incentives, we measure beliefs about how the staff member’s actions will

affect the evaluator’s own performance evaluation received from their audit partner with

AFFECT OWN EVAL. We collected this measure on the same 11-point scale as our primary

dependent measure, with a left endpoint of “Below Expectations” and a right endpoint of “Above

Expectations.” We also measured GENERAL PS REWARDS, how participants’ own managers in

the past had rewarded general skepticism, measured on an 11-point scale with endpoints 0 =

“Did not reward” and 10 = “Always rewarded.” Finally, we measured EXPERIENCED

REWARDS for costly skepticism on an 11-point scale, asking the extent to which participants’

managers would reward skepticism when the participant went over budget, strained relations

with management, and did not detect a misstatement, with endpoints 0 = “Would Not reward”

and 10 = “Would Definitely Reward.”

For traits, we measured the TRAIT SKEPTICISM of participants using the Hurtt (2010)

scale. We also measured participants’ beliefs about their own general abilities by measuring MY

RANK (on an 11-point Likert scale) relative to others at the same experience level. Unlike the

experiment, this measure is not conditioned on anything in the research materials.

17 Nelson (2009) also includes “evidential input” as an additional determinant, but note that our research materials

hold constant the evidential input related to the analytical procedure.

28

Nelson (2009) describes how relevant knowledge is developed through experiences and

training. Accordingly, we measured multiple potentially relevant dimensions of experience that

could influence knowledge related to our setting. We measured EXPERIENCE (years of auditing

experience), NFM (percentage of the time auditors used NFMs when performing analytical

procedures) and INDUSTRY (percentage of hours charged to clients in the manufacturing

industry). We also measured FRAUD TRAINING (number of hours of fraud training), as

skepticism can develop through training, and auditors’ perceptions of the misstatement base rate

MM BASERATE, as knowledge of base rates likely influences skepticism (Bonner et al. 1997).

Finally, evaluators’ own experiences with consultative audit supervisors and supportive

managers can affect how they evaluate skepticism (Glover and Prawitt 2014; Nelson et al. 2016).

We thus include the measure CONSULTATIVE to capture the extent to which participants

consulted with their own supervisors while exercising skepticism, measured on an 11-point

Likert scale with endpoints 0 = “Never” and 10 = “Always.”

Results

Table 5 presents the results of multivariate analyses in which we run separate OLS

regressions for each group of determinants, as specified in Nelson (2009), as well as a full model

of determinants on EVAL. In column (i), we examine the effects of evaluators’ incentives. We

observe that AFFECT OWN EVAL is positively associated with rewarding costly skepticism.

Thus, we provide evidence highlighting the importance of “tone from the top,” as audit partners

who endorse costly skepticism on their engagements are apt to develop supervisors who reward

the behavior in their staff. This is consistent with the strategic effects we observe in our

experiment, as evaluators appear to act strategically and provide positive evaluations based on

the evaluations that they themselves expect to receive. Unlike our experimental results related to

29

audit staff applying skepticism, evaluators of skeptical behavior do not appear to affected by

their own experiences being rewarded for applying skepticism (either skepticism in general or

costly skepticism in particular).18

In columns (ii) and (iii), we examine the effects of traits and knowledge, respectively. In

column (ii), note that we find no effect of TRAIT SKEPTICISM.19 Those with more skeptical

dispositions are not more apt to evaluate costly skepticism positively. Those who have higher

general ability tend to reward costly skepticism, as evidenced by the significant and positive

coefficient for MY RANK. If auditors ranked higher in their class are more influential at their

firms, then it is possible that they could be effective in sharing/training “best practices” such as

consistently rewarding costly skepticism.

In column (iii), we find CONSULTATIVE has a positive and significant sign, consistent

with the idea that more positive evaluations of costly skepticism stem from auditors who

experience more supportive and open environments in which auditors feel comfortable raising

issues. Finally, we find two knowledge effects in which total EXPERIENCE and NFM are

positively associated with evaluations of costly skepticism.

In sum, our survey findings are consistent with the intuition of our experimental findings.

Beliefs that superiors will reward costly skepticism are associated with increased motivation for

skepticism among subordinates and more positive evaluations of costly skepticism by

supervisors. Similarly, among evaluators, experiencing more consultative supervisors is

associated with higher evaluations of skepticism. Just as subordinates were more willing to act

18 The inconsistent results we find in relation to incentives may be attributable to AFFECT OWN EVAL measuring

how the participant’s audit partner would evaluate them based on costly skepticism exercised by their audit staff,

while GENERAL PS REWARDS and EXPERIENCED REWARDS measure how their audit managers rewarded their

costly skepticism. As such, the source of the incentive differs between the measures.

19 We also measured and find no effect of trait skepticism in our experiment.

30

skeptically when they have experienced consistent rewards for costly skepticism, evaluators

appear more willing to reward costly skepticism when they have experienced audit teams that

value a questioning mind/consultation within the team. Overall, there appears to be a reciprocal,

“pay it forward” mentality cultivated by strong mentoring environments that reward costly

skepticism.

V. CONCLUSIONS

We examine auditors’ incentives related to professional skepticism. We begin with the

widely-held premise that there is insufficient skepticism in practice and with recent evidence that

auditor incentive systems do not consistently reward skepticism. In an experiment, we

manipulate the performance evaluation that auditors have received on a given engagement.

Those receiving positive evaluations for costly skepticism interpret the positive evaluation as a

substantial benefit that they wish to preserve and, in turn, are not apt to exercise skepticism in a

subsequent task. On the positive side, those who have experienced consistent rewards for costly

skepticism are more motivated to apply skepticism. A supplemental survey finds that evaluators

most likely to reward costly skepticism are those who have experienced supportive/consultative

supervisors vis-à-vis the application of skepticism. Consequently, audit firms are likely to benefit

from cultivating supportive cultures in which skepticism is consistently encouraged and

rewarded.

Our findings suggest that performance evaluation systems in audit firms may create

dysfunctional incentives in which audit subordinates may not heighten skepticism even when

skepticism is rewarded. The idea that auditors are strategic is consistent with the review process

literature showing that auditors actively engage in stylization to manage impressions with their

31

reviewers (e.g., Rich et al. 1997). In turn, this literature finds that reviewers have developed

mechanisms to cope with stylization (e.g., Tan and Trotman 2003). However, there is not an

immediately obvious strategic response to our theorized effect in performance evaluation. Given

evidence that audit supervisors tend to have poor insights into the capabilities of their

subordinates (Kennedy and Peecher 1997; Messier et al. 2008; Peecher et al. 2010), it is unlikely

that supervisors have particularly good insights into these motivations either. Future research

could examine the conditions under which audit supervisors anticipate strategic behavior by their

subordinates vis-à-vis skepticism and, if they do, the mechanisms they employ to cope with this

behavior. In addition, when skeptical behavior yields a misstatement the outcome is documented

in the audit adjustment file. When the outcome of skepticism is not a misstatement, it is uncertain

as to if and how the application of skepticism is evidenced by auditors. Future research can

examine ways in which costly skepticism can be documented (e.g., in the budget file) such that

acts of costly skepticism can be effectively conveyed in audit the documentation.

32

REFERENCES

Bell, T., M.E. Peecher, and I. Solomon. The 21st Century Public Company Audit: Conceptual

Elements of KPMG’s Global Audit Methodology. (2005) Montvale, NJ: KPMG

International.

Berger, J., C. Harbring, and D. Sliwka. 2013. Performance appraisals and the impact of forced

distribution: An experimental investigation. Management Science 59 (1): 54 – 68.

Bonner, S.E., R. Libby, and M. Nelson. 1997. Audit category knowledge as a precondition to

learning from experience. Accounting, Organizations and Society 22 (5): 387–410.

Brazel, J. F., K. L. Jones, and M. F. Zimbelman. 2009. Using nonfinancial measures to assess

fraud risk. Journal of Accounting Research 47 (December): 1135–1166.

Brazel, J.F., K. L. Jones, and D. Prawitt. 2014. Auditors’ reactions to inconsistencies between

financial and nonfinancial measures: The interactive effects of fraud risk assessment and

a decision prompt. Behavioral Research in Accounting 26 (1): 131-156.

Brazel, J.F., S. B. Jackson, T. J. Schaefer, and B. W. Stewart. 2016. The outcome effect and

professional skepticism. The Accounting Review 91 (6): 1577 – 1599.

Bretz, R., G. Milkovich, and W. Read 1992. The current state of performance appraisal research

and practice: Concerns, directions, and implications. Journal of Management 18 (2):

321–352.

Cichello, M. S., C. E. Fee, C. J. Hadlock, and R. Sonti. 2009. Promotions, turnover, and

performance evaluation: Evidence from the careers division managers. The Accounting

Review 84 (4): 1119–1143.

Earley, C. E. 2001. Knowledge acquisition in auditing: Training novice auditors to recognize cue

relationships in real estate valuation. The Accounting Review 76 (1): 81–97.

Gibbins, M., and K.T. Trotman. 2002. Audit review: Managers’ interpersonal expectations and

conduct of the review. Contemporary Accounting Research 19 (3): 411 – 444.

Glover, S. M., D. F. Prawitt. 2014. Enhancing auditor professional skepticism: The professional

skepticism continuum. Current Issues in Auditing, 8 (2): 1-10.

Gold, A., U. Gronewold, and S. E. Salterio. 2014. Error management in audit firms: Error

climate, type, and originator. The Accounting Review 89 (1): 303–330.

Holmstrom, B. 1981. Contractual models of the labor market. The American Economic Review

71 (2): 308 – 313.

Houston, R. W. 1999. The Effects of Fee Pressure and Client Risk on Audit Seniors' Time

Budget Decisions. Auditing: A Journal of Practice and Theory 18: 70-86.

Hurtt, R. K. 2010. Professional skepticism: An audit specific model and measurement scale.

Auditing: A Journal of Practice and Theory 29 (1):149-171.

Hurtt, R. K., H. Brown-Liburd, C. E. Earley, and G. Krishnamoorthy. 2013. Research on auditor

professional skepticism: Literature synthesis and opportunities for future research.

Auditing: A Journal of Practice & Theory 32 (Supplement): 45–97.

International Auditing and Assurance Standards Board (IAASB). 2004. Objective and general

principles governing an audit of financial statements. ISA 200. New York, NY: IFAC.

International Auditing and Assurance Standards Board (IAASB). 2004. Objective and general

principles governing an audit of financial statements. ISA 200. New York, NY: IFAC.

International Auditing and Assurance Standards Board (IAASB). 2012. Professional Skepticism

in an Audit of Financial Statements. New York, NY: IFAC.

33

International Auditing and Assurance Standards Board (IAASB). 2015. Invitation to comment:

Enhancing audit quality in the public interest: A focus on professional skepticism, quality

control and group audits. New York, NY: IFAC.

Kadous, K., J. Kennedy, and M.E. Peecher. 2003. The effect of quality assessment and

directional goal commitment on auditors' acceptance of client-preferred accounting

methods. The Accounting Review 78 (3): 759-778.

Kahneman, D. and A. Tversky. 1979. Prospect theory: Am analysis of decision under risk.

Econometrica 47 (2): 263-291

Kaplan, S., and P. Reckers. 1989. An examination of information search during initial audit

planning. Accounting, Organizations and Society 14: 337 – 345.

Kaplan, S., C. Moeckel, and J. Williams. 1992. “Auditors’ hypothesis plausibility assessments in

an analytical review setting.” Accounting, Organizations and Society 11: 50 – 65.

Kennedy, J., and M.E. Peecher. 1997. Judging auditors’ technical knowledge. Journal of

Accounting Research 35(2): 279 – 293.

KPMG, LLP (KPMG). 2016. Transparency Report 2016. KPMG Professional Services.

Lambert, T. A., K. L. Jones, J. F. Brazel, and D. S. Showalter. 2017. Audit time pressure and

earnings quality: An examination of accelerated filings. Accounting, Organizations and

Society (forthcoming).

Lightle, S., J. Catellano, and B. Baker. 2017. Why audit teams need the confidence to speak up.

Journal of Accountancy (January): 46-51.

Mayhew, B. 2001. Auditor reputation building. Journal of Accounting Research 39 (3): 599 –

617.

Messier, W., V. Owhoso, and C. Rakovski. 2008. Can audit partners predict subordinates’ ability

to detect errors? Journal of Accounting Research 46(5): 1241 – 1264.

Nelson, M.W. 2009. A model and literature review of professional skepticism in auditing.

Auditing: A Journal of Practice & Theory 28 (2): 1-34.

Nelson, M.W., C.A. Proell, and A. Randel. 2016. Team-oriented leadership and auditors’

willingness to raise audit issues. The Accounting Review 91(6): 1781 – 1805.

O’Donnell, E., and J. J. Schultz, Jr. 2005. The halo effect in business risk audits: Can strategic

risk assessment bias auditor judgment about accounting details? The Accounting Review

80 (3): 921–939.

Pearl, J. 2005. Direct and indirect effects. Proceedings of the American Statistical Association

Joint Statistical Meeting. R-273: 1572 – 1581.

Pearl, J. 2010. An introduction to causal inference. International Journal of Biostatistics 6(2):

Peecher, M. E., M. D. Piercey, J. S. Rich, and R. M. Tubbs. 2010. The effects of a supervisor’s

active intervention in subordinates’ judgments, directional goals, and perceived technical

knowledge advantage on audit team judgments. The Accounting Review 85 (5): 1763–

1786.

Peecher, M. E., I. Solomon, and K. T. Trotman. 2013. An Accountability Framework for

Financial Statement Auditors and Related Research Questions. Accounting,

Organizations and Society 38: 596-620.

Preacher, K., and A. Hayes. 2008. Asymptotic and resampling strategies for assessing and

comparing indirect effects in multiple mediator models. Behavioral Research Methods 40

(3): 879-891.

Prendergast, C. 1999. The provision of incentives in firms. Journal of Economic Literature 37

(1): 7–63.

34

Public Company Accounting Oversight Board (PCAOB). 2012. Maintaining and Applying

Professional Skepticism in Audits. Staff Audit Practice Alert No. 10 (SAPA 10).

Washington, DC: PCAOB.

Rich, J.S., I. Solomon, and K.T. Trotman. 1997. The audit review process: A characterization

from the persuasion perspective. Accounting, Organizations and Society 22 (5): 481 –

505.

Shaub, M. K., and J. E. Lawrence. 1999. Differences in Auditors' Professional Skepticism

Across Career Levels in the Firm. Advances in Accounting Behavioral Research 2: 61-83.

Solomon, I., M. Shields, and R. Whittington. 1999. What do industry specialist auditors know?

Journal of Accounting Research 37: 191-208.

Tan, H.-T., and K. Jamal. 2001. Do auditors objectively evaluate their subordinates’ work? The

Accounting Review 76 (January): 99–110.

Tan, H.T., and K.T. Trotman. 2003. Reviewers’ responses to anticipated stylization by preparers

of audit workpapers. The Accounting Review 78 (2): 581 – 604.

Tan, H.T., and K. Jamal. 2006. Managing perceptions of technical competence: How well do

auditors know how others view them? Contemporary Accounting Research 23 (3): 761 –

787.

Trompeter, G., and A. Wright. 2010. The world has changed—Have analytical procedure

practices? Contemporary Accounting Research 27 (2): 669–700.

Wilks, T. J. 2002. Predecisional distortion of evidence as a consequence of real-time audit

review. The Accounting Review 77 (1): 51–71.

35

APPENDIX – VARIABLE DESCRIPTIONS

Variable Description

EXPERIMENT EVAL Manipulated between subjects as an interim evaluation of “Met

Expectations” (LOWER) or “Exceeded Expectations” (HIGHER).

RED FLAG Manipulated at three levels between subjects LOW, MODERATE, and

SEVERE. In all conditions, sales growth was constant at positive 9%. In the

LOW conditions, NFM growth was -1%. In the MODERATE and SEVERE

conditions, NFM growth was -21% and -41%, respectively.

MOTIVATION Self-assessments of how motivated participants were to exercise

professional skepticism during the task. Participants responded on a scale

ranging from 1 (Not at all motivated) to 10 (Extremely motivated).

RANKING

Self-assessments of how participants would rank in their class (compared to

other seniors in your office) from receiving the evaluation in EVAL,

collected on a scale ranging from 1 (Bottom of My Class) to 10 (Top of My

Class).

SKEPTICISM

Coded as “1” if participants concluded that more work would be required in

the task and “0” if participants concluded the balance appeared reasonable.

INFORM MANAGER

Coded as “1” if participants explicitly noted the NFM trend in

communicating their findings to their manager.

RECOGNITION Assessments of the extent to which the NFM trend is inconsistent with the

growth in recorded sales, collected on a scale ranging from 1 (Very small)

to 10 (Very large).

APPROPRIATENESS

Coded as “1” if either (1) SKEPTICISM equals 1 and the RED FLAG is

MODERATE or SEVERE or (2) SKEPTICISM equals 0 and the RED

FLAG is LOW. Coded “0” otherwise.

EXPERIENCED REWARDS Self-assessment of likelihood that managers in participants’ own personal

experience would reward costly skepticism, collected on a scale ranging

from 1 (Would not reward skepticism) to 10 (Would definitely reward

skepticism). For analyses, we partition into HIGH (Top 25%) and LOW

(Bottom 75%).

GENERAL SKEPTICISM Self-assessment of likelihood that managers in participants’ own personal

experience would reward skepticism, collected on a scale ranging from 1

(Would not reward skepticism) to 10 (Would definitely reward skepticism).

SURVEY EVALUATION Assessments of the staff member’s performance on an 11-point scale

ranging from -5 to +5. There are verbal anchors “Below Expectations” for

the left endpoint, “Above Expectations” for the right endpoint, and “Met

Expectations” for the midpoint.

AFFECTS OWN EVAL Assessment of how the partner will evaluate the participant’s own

performance, on an 11-point scale ranging from -5 to +5. There are verbal

anchors “Below Expectations” for the left endpoint, “Above Expectations”

for the right endpoint, and “Met Expectations” for the midpoint.

GENERAL PS REWARDS Self-reported experience being rewarded by managers for any skepticism,

collected on a scale ranging from 1 (Would not reward skepticism) to 10

(Would definitely reward skepticism).

EXPERIENCED REWARDS Self-assessment of likelihood that managers in participants’ own personal

experience would reward costly skepticism, collected on a scale ranging

from 1 (Would not reward skepticism) to 10 (Would definitely reward

skepticism). For analyses, we partition into HIGH (Top 25%) and LOW

(Bottom 75%).

TRAIT SKEPTICISM Trait skepticism measured via the Hurtt (2010) scale.

MY RANK

Self-assessments of how participants currently rank in their class collected

on a scale ranging from 1 (Bottom of My Class) to 10 (Top of My Class).

36

CONSULTATIVE Self-reported extent to which participants consult their supervisors while

exercising skepticism, 0 (Never) to 10 (Always)

TOTAL EXPERIENCE Total self-reported experience in years & months (converted to decimals).

INDUSTRY Percentage of chargeable hours in manufacturing during the past three years

(the industry in our case)

NFM Percentage of substantive analytical procedures conducted by the participant

that have involved NFMs.

FRAUD TRAINING Number of hours of training on financial statement fraud

MM BASERATE The percentage of pre-audit financial statements that contain a material

misstatement.

FIGURE 1 – INDIRECT EFFECTS HYPOTHESIS TESTS

This figure depicts the coefficients of the indirect effect of EVAL on SKEPTICISM, via MOTIVATION and

RANKING. The coefficients are standardized to allow for computation of the indirect effects, and are estimated

using:

MOTIVATION = δ1 + β1EVAL+ ε (1)

RANKING = δ2 + β2EVAL+ ε (2)

SKEPTICISM (1 or 0) = δ3 + β3EVAL + β4MOTIVATION + β5RANKING + ε (3)

Significance of coefficients is indicated with *** for p < 0.01, ** for p < 0.05, and * for p < 0.10.

The indirect effects of EVAL on SKEPTICISM are the products of β1EVAL*β4MOTIVATION for the MOTIVATION

mediator and β2EVAL*β5RANKING for the RANKING mediator. Confidence intervals are bias-corrected intervals

for the estimate of the indirect effect, which are estimated using 5,000 bootstrapped re-samples of the data with

replacement. Significance of the indirect effect is indicated if the intervals do not include zero.

β3is from the path c’ and is the direct effect of EVAL on SKEPTICISM controlling for the effects of MOTIVATION

and RANKING.

38

FIGURE 2 – EXPERIMENT: AUDITORS’ OWN EXPERIENCES WITH REWARDS FOR

SKEPTICISM

Panel A: Rewards for Costly Skepticism

Panel B: Rewards for Skepticism in General

This Figure depicts auditors’ assessments of the extent to which they have been rewarded for costly skepticism

(Panel A) and for skepticism in general (Panel B). See the Appendix for variable descriptions.

12.8

6.4

11.912.8

9.2

0.9

11.9

15.6

12.8

2.8 2.8

1 2 3 4 5 5.5 6 7 8 9 10

Percentage of

Responses

Extent to Which Auditors' Own

Managers Would Reward

EXPERIENCED REWARDS for Costly

Skepticism

Would Not

Reward

Skepticism

Definitely

Would Reward

Skepticism

5.5 5.54.6

8.3

6.4

0.9

13.8

16.5

22

6.4

10.1

1 2 3 4 5 5.5 6 7 8 9 10

Percentage of

Responses

Extent to Which Auditors' Own

Managers Would Reward

Rewards for GENERAL SKEPTICISM

Would Not

Reward

Skepticism

Definitely

Would Reward

Skepticism

39

FIGURE 3 – EXPERIMENT: CONCEPTUAL MODEL OF INDIRECT EFFECTS

CONDITIONAL ON EXPERIENCED REWADS

This figure depicts the conceptual relations tested in our supplemental analysis of EXPERIENCED REWARDS.

40

TABLE 1 – EXPERIMENT: DESCRIPTIVE STATISTICS

Panel A – Descriptive Statistics by EVAL for MOTIVATION, RANKING, & SKEPTICISM

MEDIATORS OUTCOME MEASURE

MOTIVATION RANKING SKEPTICISM

LOWER EVAL

7.43

(2.01)

n = 53

5.03

(1.71)

n = 55

0.53

(0.50)

n =55

HIGHER EVAL

7.56

(2.12)

n = 57

8.65

(1.63)

n = 57

0.47

(0.50)

n = 57

MEAN

7.50

(2.06)

n = 110

6.87

(2.46)

n = 112

0.50

(0.50)

n = 112

Panel B – Cell Means by EVAL & RED FLAG for MOTIVATION, RANKING, & SKEPTICISM

RED FLAG

MEDIATORS OUTCOME MEASURE

MOTIVATION RANKING SKEPTICISM

LOW 7.76 4.68 0.35

LOWER EVAL MODERATE 6.94 5.00 0.53

SEVERE 7.55 5.33 0.67

LOW 7.45 8.70 0.45

HIGHER EVAL MODERATE 7.84 8.53 0.53

SEVERE 7.39 8.72 0.44

This table depicts descriptive statistics for MOTIVATION, RANKING, and SKEPTICISM across experimental

conditions. See the Appendix for variable descriptions

41

TABLE 2 – EXPERIMENT: SUPERVISOR REWARDS FOR COSTLY SKEPTICISM MEDIATORS OUTCOME MEASURE RQ

MOTIVATION

(i)

RANKING

(ii)

SKEPTICISM

(iii)

SKEPTICISM

(iv)

APPROPRIATENESS

(v)

CONSTANT 7.43

(26.14)

*** 4.99

(21.94)

*** 0.26

(0.29)

-0.52

(0.29)

0.81

(0.91)

EVAL 0.13

(0.32)

3.66

(11.58)

*** 1.08

(1.69)

* 2.11

(2.00)

** 1.01

(1.59)

RANKING

-0.34

(-2.52)

** -0.39

(2.43)

*** -0.33

(-2.46)

**

MOTIVATION

0.20

(1.85)

** 0.22

(2.00)

** 0.17

(1.67)

*

RED FLAG 0.83

(2.23)

**

EVAL*RED FLAG -0.82

(-1.62)

N 109 109

109

109 109

Confidence Intervals for Indirect Effect

MOTIVATION (H1) (-0.09, 0.25) (-0.10, 0.26) (-0.08, 0.19)

RANKING (H2) (-2.08, -0.44) (-2.26, -0.50) (-2.09, -0.42)

*** p < 0.01, ** p < 0.05, * p < 0.10

Panel A presents standardized coefficients from results of regressions for our hypothesis tests (i.e., H1 and H2), as

well as our research question related to the appropriateness of skepticism. The dependent measure is indicated in the

column heading. MOTIVATION and RANKING are continuous measures, thus the coefficients are standardized OLS

coefficients and test statistics are t-scores. SKEPTICISM and APPROPRIATENESS are dichotomous measures, thus

the coefficients are standardized logit coefficients and test statistics are Z-scores. See the Appendix for variable

descriptions.

42

TABLE 3 – EXPERIMENT: RECOGNITION OF INCONSISTENCY

Panel A: Descriptive Statistics for RECOGNITION – Means, (Standard Deviation), Number of

Observations

RED FLAG

Low Moderate Severe Mean

LOWER EVAL

4.44

(2.19)

n =16

6.53

(2.58)

n = 16

7.90

(2.51)

n = 20

6.41

(2.79)

n = 52

HIGHER EVAL

5.75

(1.86)

n = 20

7.74

(1.76)

n = 19

7.22

(2.31)

n = 18

6.88

(2.31)

n = 57

Mean

5.17

(2.09)

n = 36

7.19

(2.22)

n = 35

7.58

(2.41)

n = 38

6.66

(2.41)

n = 109

Panel B: Correlations between RECOGNITION & SKEPTICISM

LOWER EVAL

0.37***

n = 52 HIGHER EVAL

-0.06

n = 57

Mean 0.17

n = 109

*** p < 0.01, ** p < 0.05, * p < 0.10

Panel A depicts descriptive statistics for the RECOGNITION measure. Panel B depicts correlations between

RECOGNITION and SKEPTICISM. See the Appendix for variables descriptions.

43

TABLE 4 – EXPERIMENT: PERSONAL EXPERIENCE WITH REWARDS FOR COSTLY

SKEPTICISM

Panel A: Descriptive Statistics – Means, (Standard Deviation), Number of Observations EXPERIENCED

REWARDS

MOTIVATION RANKING SKEPTICISM

LOWER EVAL LOW 6.83 4.90 0.49

HIGH 8.53 5.18 0.59

HIGHER EVAL LOW 7.46 8.54 0.43

HIGH 7.75 8.85 0.55

Panel B: Regression for Conditional Indirect Effect

MOTIVATION RANKING SKEPTICISM

(i) (ii) (iii)

CONSTANT 6.83

(20.15)

*** 4.90

(17.32)

*** 0.21

(0.23)

EVAL 0.63

(1.33) 3.64

(9.23)

*** 1.05

(1.64)

EXPERIENCED

REWARDS

1.70

(2.86)

*** 0.28

(0.56)

EVAL*EXPERIENCED

REWARDS

-1.41

(-1.73)

* 0.03

(0.05)

MOTIVATION 0.22

(1.98)

**

RANKING -0.35

(-2.56)

**

N 105 105

105

Index of Moderated Mediation

(confidence intervals) -0.31

(-0.82, 0.00) -0.01

(-0.47, 0.40)

*** p < 0.01, ** p < 0.05, * p < 0.10. See Appendix for variable descriptions.

Panel A depicts descriptive statistics for the MOTIVATION, RANKING, and SKEPTICISM across levels of the EVAL

and EXPERIENCED REWARDS variables.

Panel B presents standardized coefficients from results of regressions for our supplemental analyses. The dependent

measure is indicated in the column heading. MOTIVATION and RANKING are continuous measures, thus the

coefficients are standardized OLS coefficients and test statistics are t-scores. SKEPTICISM is a dichotomous

measure, thus the coefficients are standardized logit coefficients and test statistics are Z-scores.

44

TABLE 5 – SURVEY: DETERMINANTS OF REWARDS FOR COSTLY SKEPTICISM

EVAL

(i) EVAL

(ii)

EVAL

(iii)

EVAL

(iv)

Incentives AFFECT OWN EVAL 0.37 ***

0.36 ***

GENERAL PS REWARDS 0.02

-0.16

EXPERIENCED REWARDS 0.04

0.04

Traits TRAIT SKEPTICISM < 0.01

0.07

MY RANK 0.19 **

0.08

Knowledge CONSULTATIVE

0.21 ** 0.26 **

TOTAL EXPERIENCE

0.19 * 0.20 **

INDUSTRY

-0.11

-0.15

NFM

0.17 * 0.17 *

FRAUD TRAINING

-0.06

-0.05

MM BASERATE

-0.10

-0.08

R-squared 0.14 0.04

0.09

0.23

N 127 127 127 127

*** p < 0.01, ** p < 0.05, * p < 0.10.

The dependent measure is EVAL, which is participants’ evaluation of a staff member’s costly skepticism, measured

on an 11-point scale. See Appendix for variable descriptions.