S U B L I M I N A L O R N O T ?

Anders Sand

Subliminal or not? An appraisal of semantic processing in the near absence of visual awareness

Anders Sand

©Anders Sand, Stockholm University 2016

ISBN 978-91-7649-454-7

Printed in Sweden by Holmbergs, Malmö 2016

Distributor: Department of Psychology, Stockholm University

To Hugo.

Abstract

Stimuli that cannot be perceived (i.e., that are subliminal) can still elicit neu-

ral responses in an observer, but can such stimuli influence behavior and

higher-order cognition? Empirical evidence for such effects has periodically

been accepted and rejected over the last six decades. Today, many psycholo-

gists seem to consider such effects well-established and recent studies have

extended the power of subliminal processing to new limits. In this thesis, I

examine whether this shift in zeitgeist is matched by a shift in evidential

strength for the phenomenon.

This thesis consists of three empirical studies involving more than 250

participants, a simulation study, and a quantitative review. The conclusion

based on these efforts is that several methodological, statistical, and theoreti-

cal issues remain in studies of subliminal processing. These issues mean that

claimed subliminal effects might be caused by occasional or weak percepts

(given the experimenters’ own definitions of perception) and that it is still

unclear what evidence there is for the cognitive processing of subliminal

stimuli. New data are presented suggesting that even in conditions tradition-

ally claimed as “subliminal”, occasional or weak percepts may in fact influ-

ence cognitive processing more strongly than do the physical stimuli, possi-

bly leading to reversed priming effects. I also summarize and provide meth-

odological, statistical, and theoretical recommendations that could benefit

future research aspiring to provide solid evidence for subliminal cognitive

processing.

List of studies

This doctoral thesis is based on the following studies:

I. Sand, A., Westerlund, J., & Nilsson, M. E. (2016). Semantic priming

goes both ways: Semantic content of subsequent target stimuli interferes

with the discrimination of preceding priming stimuli. (Submitted to

journal.)

II. Sand, A., & Nilsson, M. E. (2016). Subliminal or not? Comparing null-

hypothesis and Bayesian methods for testing subliminal priming. Con-

sciousness and Cognition, 44, 29–40.

http://doi.org/10.1016/j.concog.2016.06.012

III. Sand, A. (2016). Reversed priming effects may be driven by mispercep-

tion rather than subliminal processing. Frontiers in Psychology, 7, 198.

http://doi.org/10.3389/fpsyg.2016.00198

IV. Sand, A., & Nilsson, M. E. (2016). When perception trumps reality:

Perceived, not objective, meaning of primes drives Stroop priming.

(Submitted to journal.)

Contents

Abstract ........................................................................................................ iv

List of studies .............................................................................................. ix

Contents ....................................................................................................... xi

1. Introduction ........................................................................................... 13 Subliminal advertising ............................................................................................ 14 Subliminal influences on higher-order cognition ............................................... 15 Theoretical importance of subliminal processing ............................................... 16

2. Aims ......................................................................................................... 18

3. Subliminal processing .......................................................................... 19 Vocabulary ................................................................................................................ 19 How subliminal processes are investigated ........................................................ 21

Rendering stimuli subliminal ............................................................................ 21 Measuring perception ........................................................................................ 22 The disassociation paradigm ............................................................................ 24

4. Perceptual decisions under high uncertainty .................................. 25 Signal detection theory .......................................................................................... 25 Defining subliminal in the wake of SDT............................................................... 28 Discussion ................................................................................................................. 30

5. Subliminal semantic processing ........................................................ 33 Evidence for subliminal semantic processing ..................................................... 35

Semantic or motor priming? ............................................................................ 36 Evidence against subliminal semantic processing? ........................................... 37

6. Validity of measures of sensitivity: Study 1 ................................... 39 Background ............................................................................................................... 39 Method....................................................................................................................... 39 Results and discussion............................................................................................ 40

Underestimations of sensitivity ....................................................................... 41

7. Validity of statistical tests of lack of perception: Study 2 ........... 43 Background ............................................................................................................... 43 Method....................................................................................................................... 44 Results and discussion............................................................................................ 44

Bayesian statistics ............................................................................................. 46 Post hoc sorting .................................................................................................. 47

8. Review of demonstrations of lack of perception ............................ 49 Method....................................................................................................................... 50 Results ....................................................................................................................... 52 Discussion ................................................................................................................. 57

9. Regression approach to testing subliminal effects: Study 3 ....... 60 Inter-individual variability ...................................................................................... 60

False-positive rate of regression analysis ...................................................... 61 Background of Study 3 ........................................................................................... 62 Method....................................................................................................................... 62 Results and discussion............................................................................................ 63

10. Perception-dependent priming: Study 4 ....................................... 65 Reversed priming: Study 4 .................................................................................... 65

Background ......................................................................................................... 65 Method ................................................................................................................. 65 Results and discussion ...................................................................................... 66

Validity of zero sensitivity: a reanalysis of Study 4 .......................................... 68 Discussion ............................................................................................................ 72

11. General discussion ............................................................................. 73 Has perception been ruled out? ............................................................................ 73 Do residual percepts matter? ................................................................................ 75 Future directions ...................................................................................................... 76 Does subliminal cognition matter? ....................................................................... 77 Limitations ................................................................................................................ 78 Concluding remarks ................................................................................................ 79

Summary in Swedish ................................................................................ 81

Acknowledgements ................................................................................... 83

Appendix: Review references ................................................................. 84

References .................................................................................................. 92

13

1. Introduction

In 1973 cinemagoers were scared out of their wits when watching The Exor-

cist. In its wake, The Exorcist, like many truly great films, prompted debate

about morality and moviemaking. One controversy concerned director Wil-

liam Friedkin’s decision to incorporate briefly flashed images of a demon1 to

sustain the creepy mood of the movie in its quieter parts. The controversy

concerned the claim that using subliminal (i.e., impossible to perceive) im-

ages was unethical because these are scarier than clearly perceived images.

In an interview, the author of The Exorcist, William Peter Blatty, defended

the film by stating “There are no subliminal images [in the film]. If you can

see it, it’s not subliminal.”2

Today there is resurgent interest in delimiting what influence subliminal

stimuli have on cognition and behavior, and several recent studies suggest

that such effects can be substantial. However, to demonstrate subliminal

effects one must first be able to demonstrate that the stimuli truly were un-

perceivable. The topic of this thesis is how to validly measure and demon-

strate such a lack of perception. Mr. Blatty’s stance has been my stance

when working on this project: for something to be truly subliminal, it cannot

be perceived—not even slightly or occasionally.

This thesis is divided into three parts followed by a general discussion.

The first part (Chapters 1–5) reviews what subliminal processing is and the

current stand on subliminal semantic processing (the cognitive function em-

pirically investigated here). The second part (Chapters 6–9) treats methodo-

logical and statistical issues in this field and presents recommendations for

how these can be rectified. The third part (Chapters 9–10) contributes new

empirical evidence suggesting that occasional, fragmentary percepts may be

as important for semantic processing as the physical stimuli themselves. This

journey from potentially subliminal effects to a new perceptual effect is con-

cluded in the general discussion, where the evidence for subliminal pro-

cessing is evaluated once again.

Before exploring the details of operationalization, methodology, and sta-

tistics, let us briefly review the everyday importance of subliminal messages.

1 It is probably the face of Pazuzu: the evil spirit that possesses the poor girl in the film. 2 Page 138 in The Exorcist: Out of the Shadows (2000) by Bob McCabe.

14

Subliminal advertising

“If I had a $ for every time I’ve had to convince someone that they are more likely to find BIGFOOT than produce a working subliminal ad, I would be a very rich man.”

Tom O’Guinn (an advertising guru; quoted in Broyles, 2006, p. 403)

What Tom O’Guinn omits, of course, is that if he had come up with a work-

ing subliminal ad he would have been an even richer man! The possibility

that subliminal advertisements can influence people into buying products

without the consumer being aware of what is happening scares people per-

haps even more than The Exorcist did.

This possibility became publicly well-known in 1957 through the actions

of a Mr. James M. Vicary. Mr. Vicary claimed to have imbedded advertise-

ments in movies that increased cinema sales of popcorn and cola by 58%.

The amazing part was that the advertisements were shown so fast that no one

noticed them. Fortunately for cinemagoers, after both commercial and aca-

demic research projects failed to replicate the original results, Mr. Vicary

eventually admitted that he had invented the experimental results in an effort

to save his failing advertising company (Rogers, 1992). Even so, his widely

publicized “study” is probably the main reason why subliminal messages

continue to pique the public’s interest and fear.

Following Mr. Vicary’s hoax, a lot of actual empirical research on sub-

liminal advertising was done. De Fleur and Petranoff (1959) conducted one

of the perhaps most ecologically valid experiments: For five weeks, five

nights a week, they incorporated two different subliminal messages in actual

TV shows broadcast to a wide audience. The messages were single frames

comprising white letters on a black background superimposed over frames of

the feature broadcast. In an experimental setting it was demonstrated that the

identity of the stimuli could be guessed better than chance but that partici-

pants reported not perceiving them.3

One subliminal message was “Watch the news,” and the dependent varia-

ble was how many participants continued to watch the news immediately

following the broadcast. The other message was “Buy product x,” x being a

widely available preserved meat or vegetable oil, and the dependent variable

was the sales of these house hold products in the region.

The result was that the subliminal messages had not the slightest effect in

persuading the mass audience to perform any of these simple and routine

behaviors. In contrast, a control condition with conventional ads for product

x, containing no subliminal messages, stimulated substantial increases in

sales.

3 In other words, the stimuli were above the objective discrimination threshold but below the

subjective detection threshold.

15

There are also several reviews and meta-analytical assessments of the ex-

perimental literature on subliminal advertisements. A review by Theus

(1994, p. 282) citing 128 references concluded that “brand choice behavior,

per se, seems to be subject to little or no influence by subliminal sugges-

tion.” A more recent review by Broyles (2006, p. 405) concluded that “the

bottom line is that consumers need not worry about psychological manipula-

tion from subliminal advertising.” These reviewers’ somewhat vague con-

clusions could be explained by the difficulty of demonstrating an absolute

lack of effect. Trappey (1996), after meta-analyzing a combined sample of

3565 participants, found the effect size (Cohen’s d) of subliminal advertising

to be 0.0045 (95% confidence interval = –0.06 to 0.07). Unfortunately, I

have yet to see a Bayesian meta-analysis of the same data, which makes it

difficult to assess whether it supports absolutely no effect or only a

miniscule effect.

Subliminal influences on higher-order cognition

Although subliminal messages perhaps do not stimulate shopping, studies in

experimental psychology suggest that relatively complex information can,

without passing through the processing stages related to perception, influ-

ence even higher-order cognitive functions. It has been suggested that sub-

liminal stimuli exert an influence on, for example, conflict monitoring (van

Gaal & Lamme, 2012), task switching (Ansorge, Kunde, & Kiefer, 2014),

learning (Pessiglione et al., 2008), decision making (Ortells, Kiefer, Castillo,

Megías, & Morillas, 2016; Simon van Gaal, de Lange, & Cohen, 2012), goal

pursuit (Custers & Aarts, 2010), social inferences (Mumenthaler & Sander,

2015), and doing arithmetic (Sklar et al., 2012). This has led some psycholo-

gists to question whether perception plays any role in cognition (Bargh &

Morsella, 2008; Hassin, 2013; Lau & Rosenthal, 2011; Rosenthal, 2008) and

other psychologists to question the findings (Eriksen, 1960; Holender, 1986;

Holender & Duscherer, 2004; Miller & Schwarz, 2014; Newell & Shanks,

2014; Stafford, 2014; Vadillo, Konstantinidis, & Shanks, 2016).

Perhaps the most astounding are results published by Hassin et al. in Pro-

ceedings of National Academy of Sciences and Psychological Science

(Carter, Ferguson, & Hassin, 2011; Hassin, Ferguson, Shidlovski, & Gross,

2007). These authors subliminally exposed their participants to their national

flags. These perceptually degraded stimuli, 16 ms long and backward

masked, apparently primed the participants to change their political stance

(as measured by a questionnaire) as well as their actual voting behavior

months later. The authors concluded that the results “show that the sublimi-

nal presentation of a national flag can bring about significant changes not

only in a citizen’s expressed political opinions within an experimental set-

16

ting but also in their ‘real-life’ overt political behavior, perhaps the most

significant political behavior of all, voting” (Hassin et al., 2007, p. 19760).

Other experimenters have failed to replicate Hassin et al.’s results (“Data

from investigating variation in replicability,” 2014). An ambitious review

(Newell & Shanks, 2014) of the influence of different types of nonconscious

and perceptually weak information on decision making concluded that “evi-

dence for the existence of robust unconscious influences on decision making

and related behaviors is weak, and many of the key research findings either

demonstrate that behavior is under conscious control or can be plausibly

explained without recourse to unconscious influences” (Newell & Shanks,

2014, p. 19).

In light of how a subliminal and direct message (“Watch the news”; Fleur

& Petranoff, 1959) could not influence routine behavior, i.e., staying on the

couch, Hassin et al.’s findings are almost as surprising as the recently pub-

lished support for clairvoyance (Bem, 2011). One suggested take-home mes-

sage from Bem’s report is not that clairvoyance exists but that psychologists

must change how they analyze data (Wagenmakers, Wetzels, Borsboom, &

van der Maas, 2011). As we shall see, this may be relevant to the field of

subliminal cognition too.

Theoretical importance of subliminal processing

Although subliminal messages may not have a big impact on decision mak-

ing in ecological settings, subliminal processing has nevertheless been the

focus of extensive research in experimental psychology, psychology of cog-

nition, social psychology, and neuroscience. A search on Web of Science

(http://apps.webofknowledge.com/) using “subliminal” as the search term

identified 256 papers published just during the 2014–2015 period. Almost all

of these papers discuss or provide positive evidence of subliminal pro-

cessing.

Subliminal processing has been suggested to be a way of studying the ar-

chitecture underlying different cognitive and neural functions. For example,

the possibility that words and numbers may be differently processed sublim-

inally has been used to infer how these are stored in short- and long-term

memory (Greenwald, Abrams, Naccache, & Dehaene, 2003). In addition, the

apparently subliminal integration of sight and sound has been used to make

statements as to the neural underpinnings of multi-modal integration (Noel,

Wallace, & Blake, 2015).

Subliminal processing has also been used as a lens for studying the nature

and limit of consciousness, and reports of such effects have played a large

role in the development of models of consciousness (Dehaene, Kerszberg, &

Changeux, 1998; Dehaene & Changeux, 2011; Del Cul, Baillet, & Dehaene,

2007; Koch, 2004; Lamme, 2006; Lau & Rosenthal, 2011; van Gaal &

17

Lamme, 2012). Finally, research on subliminal processing is one of the fron-

tiers in studying phenomenological experience. To study the processing of

stimuli that cannot be perceived, perception must first to be operationalized:

because without a proper operationalization of perception, the question of

subliminal processing lies outside the scope of scientific inquiry.

18

2. Aims

Currently, subliminal processing seems to be a widely accepted phenome-

non. Two influential reviews report that “subliminal priming has now been

convincingly demonstrated at visual, semantic, and even motor levels”

(Dehaene & Changeux, 2011, p. 202) and that “nowadays, the existence of

subliminal perception is largely acknowledged” (Van den Bussche, Van den

Noortgate, & Reynvoet, 2009, p. 453).

This thesis focuses on the methodological and statistical challenges in-

volved in validly measuring and demonstrating a lack of perception. The

overall aim of this thesis is to determine to what extent residual percepts

have been ruled out as potential alternative explanations in claims of sublim-

inal processing. This issue sits at the heart of research in this field, because

the claim of subliminal effects requires that the stimulus of interest is truly

unperceivable.

Specifically, the following questions were investigated.

1) How valid are measures of lack of perception and how can these

be improved? (Study 1, 4)

2) How false-positive prone are statistical procedures commonly

used to demonstrate lack of perception and how can these be

improved? (Study 1, 2, 3)

3) What is the evidential strength for lack of perception in studies

of subliminal processing? (Quantitative review in Chapter 8)

4) How valid are operationalizations of lack of perception and how

can these be improved? (reanalysis of Study 4)

5) How can the effects of possible residual percepts be tested?

(Study 3, 4)

Although the main focus is on the methods used to demonstrate subliminal

processing, the specific focus of the empirical studies has been on subliminal

semantic processing. By answering the above questions, I hope that the cur-

rent evidence for subliminal processing can better be evaluated.

19

3. Subliminal processing

“I didn’t see anything but a lot of racing cars.” “Well, that’s the whole idea, Lieutenant. You don’t see it with your eye. You see it with your subconscious mind, which is quicker than the eye! That’s why it’s called subliminal.” From the episode “Double Exposure”; Columbo (Season 3, Episode 4, aired on 24

th of November 1974)

Classical models of cognitive architecture describe sensory information be-

ing processed through a hierarchy of representations, ranging from lower

levels (e.g., energy, geometric elements, and color) to higher levels (e.g.,

letters, words, faces, and meaning) before being translated into action (Craik

& Lockhart, 1972; Mesulam, 1998). Somewhere between transduction and

motor output, the neural processing often seems to co-occur with a percept—

the observer becomes conscious of the result of those processes. Studies in

neurobiology suggest that as sensory information passes through these pro-

cessing stages, it displays progressively more correspondence to perception:

whereas activity in primary sensory areas corresponds to the physical stimuli

independently of whether or not observers perceive the stimuli (Hulme, Fris-

ton, & Zeki, 2009; Romo & de Lafuente, 2013), activity in secondary senso-

ry areas and beyond corresponds to percepts whether or not these accurately

portray the stimuli (Grill-Spector, Knouf, & Kanwisher, 2004; Hulme et al.,

2009; Pessoa & Padmala, 2005; Romo & de Lafuente, 2013). Researchers

investigating subliminal processing try to delimit which processes always

co-occur with percepts and which processes can occur independent of per-

ception.

Vocabulary

Perception

I suggest that “perception” can be separated from the perceptual processes

that caused it and instead take “perception” to be synonymous with con-

scious experience. I therefore refrain from a trend in the field of calling the

perceptual processing of subliminal stimuli “perception” (e.g., “subliminal

perception,” “perception without awareness,” or “nonconscious perception”).

As Klotz and Neumann (1999, p. 976) put it, “perceiving is something a

20

person or an animal does, not something that can be properly ascribed to

stages, subsystems, brain areas or the like.”

Subliminal

Subliminal comes from the Latin “sub” meaning “under” or “close to” and

the Latin “limen” meaning “threshold,” that is, “under or close to the thresh-

old.” In the research of interest, the threshold refers to the threshold of per-

ception. Subliminal will be operationalized rigorously in Chapter 5, but a

tentative definition of a subliminal stimulus is a stimulus that cannot be per-

ceived, not even slightly and no matter how hard one tries.4 Stimuli that can

be perceived, on the other hand, are sometimes called supraliminal stimuli

(“over the threshold”). Of course, whether a particular stimulus is subliminal

or supraliminal may differ between persons and stimuli settings.

Nonconscious processes

Nonconscious processes are processes that we are not aware of. Many per-

ceptual or cognitive processes are nonconscious: The transduction occurring

in the retina or the information transfer in the thalamus elicited by clearly

perceivable stimuli (e.g., the computer screen I’m staring at) are noncon-

scious processes (otherwise neuroscience would be a rational rather than

empirical science). But sometimes nonconscious processing results in a per-

cept. (I use “nonconscious” rather than “unconscious” as the latter to me

suggests a vegetative state.)

Subliminal processing

I take subliminal processing to be a special class of nonconscious processing

in which the stimulus being processed is subliminal—in other words, sub-

liminal processing never results in a percept (given the definition of percep-

tion presented in Chapter 5). Subliminal processing is to some extent clearly

possible: a stimulus too weak to be perceived may still evoke transduction

responses in the retina, for example. But as the interest here is what effects

such stimuli may have on cognition, I use subliminal processing and sublim-

inal cognition as shorthand terms for psychologically interesting subliminal

processing (e.g., semantic processing and higher-order cognitive functions).

4 This seems to be a fair interpretation of what researchers in the field intend, for example: “A

subliminal stimulus is one in which the bottom–up, stimulus-driven information is so reduced

as to make it undetectable, even with focused attention” (Dehaene & Changeux, 2011, p. 200)

and “Unconscious perception at the objective detection threshold exists” (the title of Snod-

grass, Bernat, & Shevrin, 2004).

21

How subliminal processes are investigated

A key feature of many studies of subliminal processing is the contrasting of

perception and non-perception as separate all-or-nothing states. The general

experimental method comprises three steps: (1) The non-perception state is

created using stimuli that are difficult to detect or highly ambiguous: stimuli

perceptually degraded to the point, it is claimed, that they cannot be per-

ceived. (2) Perception, or lack-of-perception, is operationalized and meas-

ured. (3) A process of interest is operationalized, often as a behavioral re-

sponse, and measured. It may seem to be a simple process of elimination to

test what processes occur when there is a lack of perception, but each step

involves methodological problems that first must be overcome.

Rendering stimuli subliminal

The visual system is extremely reactive to brief and weak signals: Hecht,

Shlaer, and Pirenne (1942) estimated that for vision to occur, only 5–14

quanta need to be absorbed by the retinal pigment rhodopsin. Sperdin,

Spierer, Becker, Michel, and Landis (2015) recently demonstrated that

checkerboard patterns flashed for only 250 µs may be perceived5 and flashed

for 500 µs are definitely perceived. As such, one methodological challenge

when investigating visual subliminal processing is simply that of creating

subliminal stimuli.

Many methods are used to render stimuli subliminal (reviewed in

Breitmeyer, 2015; Kim & Blake, 2005), and they all probably achieve the

same goal in different ways in terms of visual processing (Kanai, Walsh, &

Tseng, 2010; Peremen & Lamy, 2014). The most common method used in

the experiments reviewed in this thesis, and the one used in its empirical

studies, is backward masking.

In a backward-masking paradigm, a perceptually weak stimulus is briefly

shown and immediately followed by a backward mask (a forward mask also

sometimes precedes the stimulus). If the backward mask is not shown, the

stimulus would have been clearly or weakly perceived because it would have

remained as an “iconic memory trace” or been strengthened by recurrent

processing. Different combinations of stimulus parameters (e.g., contrast and

duration), inter-stimulus intervals (ISIs) between stimulus and mask, and

mask parameters (e.g., contrast and size) can render the stimulus subliminal.

However, masking is often fragile: even small changes in experimental pa-

5 Detection sensitivity for the 250 µs stimulus was at a mean d’ of 0.33 (± SEM 0.18). Alt-

hough the authors suggested that this indicates that the stimulus was subliminal (that the

participants’ performance did not differ from chance; p = 0.078), calculating a Bayes factor

instead suggests that the stimulus was perceived (BH(0,0.2) = 3.1). A personal communication

revealed that the first author agreed that fragmentary or occasional percepts of the 250-µs

stimulus were possible. Throughout, I use Dienes’ (2015) notation to report Bayes factors.

22

rameters (e.g., background illumination) may greatly influence the effective-

ness of masking, and an efficient mask for one stimulus is not necessarily an

efficient mask for another stimulus (Wiens, 2006; Wiens et al., 2004).

Based on behavioral data, Kanai et al. (2010) have suggested that lack of

perception through backward masking stems from degraded perceptual in-

formation rather than attentional failures. In explaining the neural mecha-

nism of backward masking, responses in primary visual areas are typically

decomposed into two steps: (1) a transient feedforward activation elicited by

the masked stimulus, and (2) a sustained activation elicited by feedback from

later visual areas (reentrant processing). It is thought that the second step is

necessary for perception to occur and that backward masking disrupts this

second step only (Breitmeyer, 2007; Fahrenfort, Scholte, & Lamme, 2007;

see also Macknik & Martinez-Conde, 2007 for another interpretation of the

data). That the first step is not suppressed suggests that information con-

tained in the feedforward activation may pass from primary visual areas to,

for example, semantic processing areas or even motor output without percep-

tion occurring (Breitmeyer, 2007; Fahrenfort et al., 2007). Backward mask-

ing is therefore thought to allow the possibility of higher-order processing

occurring independently of perception (i.e., subliminal processing).

Measuring perception

How perception (i.e., whether or not a stimulus is perceived) should be

measured is far from obvious and there is ongoing debate on the proper

measurement methods (Overgaard & Sandberg, 2012; Sandberg, Timmer-

mans, Overgaard, & Cleeremans, 2010; Snodgrass, Bernat, & Shevrin,

2004a; Wiens, 2007). In his seminal review of the field, Holender (1986, p.

51) claims that “it is fundamental that an indicator of awareness must be

intentional,” defining an awareness measure as “any voluntary discrimina-

tive response that can be elicited on the basis of the meaning of a stimulus.”

Although many researchers probably agree with the first statement, many

disagree with Holender’s definition of an awareness measure.

Reingold and Merikle (1988) described two criteria that a proper measure

of perception must fulfill: a measure of perception (1) must be exclusive in

that it is affected only by perception and is to no extent influenced by sub-

liminal processing, and (2) must be exhaustive in that it measures all per-

cepts and can register the slightest change in perception. Holender’s (1986)

definition might not be exclusive because although discriminative responses

(often keyboard presses) are intentional, we cannot know whether those spe-

cific responses are motivated by percepts.

Most researchers seem to agree with Reingold and Merikle’s (1988) crite-

ria of exclusiveness and exhaustiveness, but are undecided as to what meas-

ure fulfills both criteria. Broadly, researchers can be categorized as those

23

recommending subjective and those recommending objective measures of

perception.

Percepts are inherently subjective in nature and proponents of subjective

measures of perception (e.g., Kunimoto, Miller, & Pashler, 2001; Merikle &

Cheesman, 1986; Overgaard, Rote, Mouridsen, & Ramsøy, 2006) would

argue that only subjective reports can validate that percepts occurred. Re-

searchers arguing against subjective measures claim that such measures can-

not be demonstrated to be exhaustive because participants may always

choose not to report (or fail to report) weak or fragmentary percepts about

which they are not confident (Björkman, Juslin, & Winman, 1993; Chees-

man & Merikle, 1984). These researchers therefore claim that subjective

measures cannot be used to convincingly demonstrate subliminal processing.

Although percepts are subjective in nature, these often result in behavioral

outcomes that can be measured objectively. Proponents of objective

measures (e.g., Finkbeiner, 2011; Holender, 1986; Snodgrass et al., 2004a;

Vermeiren & Cleeremans, 2012) argue that performance in forced alterna-

tive choice tasks can be used as a measure of perception. (To measure per-

formance separate from response biases, the signal detection construct d' is

necessary, Chapter 5.) Researchers arguing against objective measures claim

that objective measures cannot be demonstrated to be exclusive as sublimi-

nal processing may influence performance. For example, rapid responses to

subliminal stimuli may be driven by subliminally elicited motor responses

(but see Vermeiren & Cleeremans, 2012, for a rebuttal of that point).

These two broad categorizations of measures of perception also lead to

two broadly separate thresholds. The subjective threshold is the critical stim-

ulus strength at which participants report not perceiving the stimulus. The

objective threshold is the critical stimulus strength at which participants ac-

tually cannot detect or discriminate the stimulus. The objective threshold is

lower than the subjective threshold; in fact, above-chance-level objective

performance at the subjective threshold has in itself been taken as evidence

of subliminal processing (Miller, 1939). After Holender’s (1986) seminal

review, many researchers embraced the challenge of demonstrating sublimi-

nal processing at the objective threshold (Kouider & Dehaene, 2007).

Within these broad categories experimenters use a range of tasks to meas-

ure perception. In general, whatever measure researchers use, it should ad-

here to Newell and Shanks’ (2014) four criteria of an adequate measure of

awareness: a measure of perception (1) must be unaffected by factors that do

not influence the measure of subliminal processing, (2) must target only

information relevant to the subliminal processing, (3) should be made con-

currently or immediately after the measure of subliminal processing, and (4)

should be made under optimal retrieval conditions.

24

The disassociation paradigm

A range of experimental methods is used to investigate whether a process of

interest can occur independently of perception. Perhaps the simplest and

most commonly used method in the experiments reviewed in this thesis, and

the one used in its empirical studies, is the disassociation paradigm (re-

viewed in Schmidt & Vorberg, 2006).

In the disassociation paradigm, participants must complete two tasks ei-

ther concurrently or in separate blocks. Following Cheesman and Merikle

(1984) and Reingold and Merikle (1988), I will call one task the direct task

and the other the indirect task. The direct task is assumed to measure wheth-

er or not the stimulus was perceived. The indirect task is used to measure

whether or not some stimulus attribute of interest was processed, and is as-

sumed to be independent of perception. The indirect task can be, for exam-

ple, semantic priming, decision making, learning, or neural activation.

Demonstrating subliminal processing entails demonstrating that the indirect

task indicates some level of information being extracted from the stimulus,

whereas the direct task indicates no perception.

25

4. Perceptual decisions under high uncertainty

I previously defined a subliminal stimulus as a stimulus that cannot be per-

ceived, not even slightly and no matter how hard one tries. However, a

wealth of data suggests that behavior in situations of high stimulus uncer-

tainty is not well described by all-or-none stimulus thresholds that separate

stimuli into those that can be perceived and those that cannot. Instead, be-

cause neural responses to constant physical stimulation vary (Dean, 1981;

Mathewson, Gratton, Fabiani, Beck, & Ro, 2009; Scholvinck, Saleem, Be-

nucci, Harris, & Carandini, 2015; Shadlen & Newsome, 1998), there is only

a probabilistic relationship between stimulation and elicited perceptual states

(Macmillan, 1986; Swets, 1961).

Variability in bottom-up signals means that the null stimulus can some-

times evoke stronger neural responses than the actual stimuli (Ress & Hee-

ger, 2003) and that different stimuli can evoke overlapping neural responses

(Grill-Spector et al., 2004). Top down expectations (or prior information)

can also influence perceptual decisions, leading to response variability given

constant physical stimulation (Craik & Lockhart, 1972; Gold & Shadlen,

2007; Pouget, Beck, Ma, & Latham, 2013). As such, there are in principle no

stimuli that will never be reported as perceived, so in this sense there are no

subliminal stimuli.

Signal detection theory

Signal detection theory (SDT; Gescheider, 1997; Green & Swets, 2000;

Macmillan & Creelman, 2005) provides a conceptual framework for describ-

ing how sensory input is translated into decisions. Because of spontaneous

fluctuations in the nervous system, a detection situation must be described as

always occurring against a background of noise. A signal (or stimulation)

may elicit in an observer a sensory state of some magnitude, and the observ-

er must decide whether that sensory state was due to the signal (together

with noise) or due to noise alone. This can be a difficult decision because

noise alone can sometimes elicit strong sensory states and stimulation can

sometimes elicit weak sensory states.

Figure 1 illustrates a detection situation in which either only noise (catch

trials; S1) or a signal added to the background noise is presented (S2). Noise

alone (S1) and signal plus noise (S2) will elicit sensory states of varying

26

Figure 1. An SDT model of a detection situation: An observer must decide whether sensory state s is due to noise alone (S1) or signal plus noise (S2). She makes her decision based on whether s falls to the right or left of her detection criterion. The exemplified observer has a conservative criterion (i.e., is reluctant to respond “de-tected”), as is often the case in situations with difficult-to-perceive stimuli.

magnitudes (x-axis). Because of noise in the biological system, the sensory

magnitudes elicited by S1 and S2 are random variables in this case described

by Gaussian (i.e., normal) distributions. If the signal strength is strong, the

means of these distributions will differ in magnitude. The difference between

the means of the two distributions describes the observer’s sensitivity, d'.6

A particular trial elicits a sensory state, s, of some magnitude. The ob-

server must now decide whether s was due to noise alone (S1) or signal plus

noise (S2). The observer bases this decision on the internal evidence that s

arose from S1 or S2, i.e., the conditional likelihoods P(s|S1) versus P(s|S2).

When the observer is asked to make a categorical decision, she applies a

criterion that separates the continuous sensory magnitude into two catego-

ries. The criterion can be described as the distance (in standard deviations)

from zero and is called c. It can also be described as the likelihood ratio of s

given S2 and S1 at the criterion, i.e., P(s|S2)/P(s|S1), and is then called β.

The decision rule for the observer becomes report S2 if s ≥ c. The observer is

free to flexibly set her criterion depending on motivation to avoid misclassi-

fying noise as signals (false alarms) or signals as noise (misses) or based on

a priori probabilities favoring one or the other stimulation.

6 d' can be calculated as Φ–1(phits) – Φ–1(pfa), c as –½×[Φ–1(phits) + Φ–1(pfa)], and β as ecd',

where Φ–1 is the inverse of the standard normal cumulative distribution function. In other

words, Φ–1 converts probabilities into z-scores and d' is the difference between the distribu-

tions measured in z-scores. See Macmillan (1986) or Stanislaw and Todorov (1999) for more

information.

27

Figure 2. An illustration of the energy threshold: As the stimulus intensity increases (x-axis in the top-left panel) so does sensitivity (y-axis in top-left panel; the illustrat-ed psychophysical function is a rough approximation). The energy threshold is the stimulus intensity necessary for the S1 and S2 distributions not to overlap complete-ly (i.e., d' > 0). The other panels illustrate various degrees of sensitivity.

Note that there are no “detect” or “undetect” states in this conceptualiza-

tion; there are only sensory states falling above or below the observer’s crite-

rion. That is, there are no “subliminal sensations” – because there is no sen-

sory threshold in SDT (Swets, 1961). Note also that the decision situation is

described without referring to perception (or conscious experience). SDT

takes no stand as to whether any particular sensitivity d’ implies perception

or whether any other construct (e.g., c) overlaps with perception.

In addition to detection, participants can also be asked to perform dis-

crimination tasks. In this case S1 and S2 are two stimuli differing on some

attribute. The principal task of the observer is the same, to decide which

distribution s stems from. A lack of discrimination sensitivity, i.e., d' = 0,

indicates that the two stimuli are equally likely to be confused with each

other.

A final important concept here is the energy threshold, illustrated in Fig-

ure 2. Although in SDT there is no stimulus threshold that separates stimuli

into those that are and are not detected, there can be psychophysical energy

thresholds (Gescheider, 1997). The energy threshold is simply a critical

stimulus intensity that must be exceeded before the means of the S1 and S2

distributions become different (i.e., d' > 0). As mentioned previously, the

energy threshold for detecting unmasked visual stimulation is very low

(Hecht et al., 1942; Sperdin et al., 2015). The energy thresholds for discrimi-

nating pairs of stimuli can be much higher than the detection thresholds

(Fisk & Haase, 2011; Snodgrass et al., 2004a). There are, of course, inter-

28

individual differences in energy thresholds for different detection and dis-

crimination tasks (Chapter 9).

Defining subliminal in the wake of SDT

A wealth of empirical data suggests that there is no stimulus threshold in the

sense of a stimulus intensity that will never evoke detect responses (Ges-

cheider, 1997; Green & Swets, 2000; Macmillan & Creelman, 2005), as even

the null stimulus can occasionally lead to detect responses (unless the ob-

server has decided always to respond “no detect,” no matter what stimulus is

provided). Following Holender’s (1986) critical review of the field and

Macmillan's (1986) suggestions for possible definitions of subliminal stimuli

in terms of SDT, many researchers have operationalized perception, or lack

of perception, in terms of sensitivity, d'. Many subjective measures can also

be described in SDT in relation to the detection criterion, c.

Note that SDT has no claim on being a theory of perception. No value of

d' or any other particular construct is claimed to be related to perception.

When researchers choose to define a particular SDT construct as indexing

perception, this goes beyond SDT. However, one reason for claiming that

any level of sensitivity (i.e., d' > 0) is related to perception is that this is

more parsimonious than interpreting some sensitivities as related to percep-

tion and others as not (Snodgrass et al., 2004a).

Possible definitions of subliminal in SDT are as follows:

(a) a stimulus that is at the energy threshold of detection, i.e., detection d' = 0;

(b) a stimulus that is below some level of detection sensitivity, e.g., detection

d' < 0.5;

(c) a stimulus pair that is at the energy threshold of discrimination, i.e., dis-crimination d' = 0;

(d) a stimulus that below a certain proportion of trials leads to detect re-

sponses, i.e., a stimulus that elicits sensory states that often fall below de-tection c; and

(e) a stimulation that is reported as not perceived, i.e., an elicited sensory

state that falls below detection c.

Definitions (a)–(c) are objective operationalizations and definitions (d) and

(e) are subjective operationalizations of lack of perception. Various re-

searchers have operationalized subliminal according to these definitions in

various experiments.

29

Arguably, the definitions are organized in terms of conservativeness (or

exhaustiveness) in relation to perception. Zero detection sensitivity, i.e.,

definition (a), is a conservative definition as it implies that the stimulus elic-

its detect responses with equal probability as does the null stimulus. Demon-

strating that such stimuli have an impact on a cognitive process (when meas-

urement errors listed in Chapter 6 have been accounted for) would provide

persuasive evidence that the processing did not co-occur with percepts.

Detection sensitivity below a certain empirical level, i.e., definition (b),

has rarely been (explicitly) used by researchers (see, however, Marcel,

1983). It seems difficult to argue that one level of sensitivity (e.g., d' = 0.5)

is an index of no perception whereas another (e.g., d' = 0.6) indicates percep-

tion.

Zero discrimination sensitivity, i.e., definition (c), is more commonly

used than definition (a) as it can be argued that only percepts relevant to the

potentially subliminal process is of interest: “the candidate measure need not

be sensitive to absolutely all conscious perception, but rather only to rele-

vant conscious perception—namely, to the kind(s) of conscious perception

that would be necessary, at a minimum, for the effects of interest to occur”

(Snodgrass et al., 2004a, p. 850; see also Dienes & Seth, 2010). If a stimulus

satisfies definition (a), it necessarily satisfies definition (c) as well, as a de-

tection sensitivity of zero implies that the observer has no information about

any characteristics of the stimulus. But the opposite does not hold: a pair of

stimuli that cannot be discriminated from each other could still be discrimi-

nated from noise alone (Fisk & Haase, 2011; Snodgrass et al., 2004a).

Neither of the objective definitions (a) or (c) has to be satisfied for the

subjective definitions (d) or (e) to be satisfied. When definition (d), i.e., de-

tection responses below a certain proportion of trials, is used alone, it is al-

most exclusive measured in a debriefing session. When using definition (d),

a certain target proportion needs to be specified that the detect responses

must be below. What target proportion should this be? If an observer reports

the stimulus as detected on 50% of the trials, should the stimulus be con-

cluded to be subliminal rather than the observer simply being regarded as

uncertain about the stimulus? If 50% detect responses are too much, are 30%

few enough or 10%? It seems that researchers who use definition (d) normal-

ly use a cut-off somewhere between “never reported” and “almost never

reported” (cf. Pessiglione et al., 2007).

Compared with the other definitions, the “no detect” response, i.e., defini-

tion (e), relates subliminal to specific stimulations rather than stimuli and is

naturally measured on a trial-by-trial basis. Although the subjective defini-

tions (d) and (e) have no theoretical relationship to the objective definitions

(a) and (c), some kind of “block” measure of perception is needed to inter-

pret (e) (Schmidt, 2015). If an observer reports detecting stimuli in 90% of

trials, should the last 10% of the stimuli be seen as subliminal? Assuming

that the observer correctly detects the stimulus in 90% of the trials (d' =

30

2.56), should the minority of trials reported as not detected be classified as

“subliminal”? If we want to adhere to subjective measures only, we are back

to finding a target proportion for (d) in order to interpret (e). In a study by

van den Bussche et al. (2013), for example, participants reported not detect-

ing the stimulus in 25% of the trials. The experimenters interpreted these

trials as subliminal and as such their target proportion was 75%. This is not

an unusually high proportion when (e) is used as a definition of subliminal

(e.g., Del Cul et al., 2007).

Discussion

Some would claim that cognitive processing of stimuli at the energy thresh-

old of detection is impossible, whereas the same for stimulations reported as

not detected is unsurprising (Macmillan, 1986). As such, separating findings

using the different definitions is important. In Chapter 5, however, I follow

the field in general in not separating results found using the different defini-

tions. Otherwise, Chapters 6–9 involve the objective definitions (a) and (c)

while Chapter 10 involves definitions (a), (c), and the subjective definition

(e). I have focused on objective rather than subjective definitions of percep-

tion because the latter have already been criticized for not being exhaustive

(and thus for allowing residual percepts to explain the results).

Note that claiming that subjective measures are not exhaustive is not the

same as claiming that participants are lying when they report actual percepts

as “not perceived” or that participants can have perceptions of which they

are unaware (Cohen & Dennett, 2011; Seth, Dienes, Cleeremans, Overgaard,

& Pessoa, 2008). One must distinguish which definition of perception would

be valid (or exhaustive) given error-free measures and which definition

would be exhaustive given measurement error. What it comes down to is that

participants in subliminal processing experiments are performing a very

difficult task: they must make decisions under high uncertainty and there is a

conflict between the contradictory demands of achieving a high hit rate

while maintaining a low false alarm rate.

Imagine a participant in a backward-masked experiment who must dis-

criminate the stimuli and subjectively report her awareness. She must en-

dure, say, 50 trials under extreme uncertainty, may perform objectively bet-

ter than chance (56% correct; d' = 0.3), but reports not perceiving the stimu-

li. Proponents of objective measures would suggest that, among the trials she

reports not perceiving the stimulus, there may in fact be a few trials in which

she did (weakly) perceive the stimulus and that those few trials may be what

drive the often very small “subliminal” effects (e.g., three more correct trials

than expected). Proponents of the objective measure suggest that what looks

like a subliminal effect could actually just be an infrequent perceptual effect.

The strength of this explanation of course depends entirely on whether or not

31

Figure 3. Inter-individual differences in detection sensitivity and detection criteria for the participants in Study 4 are shown in the left panel. Two participants with similar detection sensitivities but different criteria are illustrated as black symbols and in the right panels.

residual percepts have been excluded in experiments reporting subliminal

effects.

There is also a practical problem with subjective measures in that the con-

servativeness of the criterion may depend on parameters outside the stimulus

used: task instructions, variable proportions of catch or stronger signal trials,

and inter-individual differences in conservativeness are examples of parame-

ters that affect the criterion (Haase & Fisk, 2015; Sandberg et al., 2010;

Supèr, Spekreijse, & Lamme, 2001; Wiens, 2007). This means that if the

subjective definitions (d) or (e) are used, whether or not a “subliminal effect”

is obtained in an experiment may depend on parameters outside the stimulus

itself.

To illustrate this I have re-analyzed the data from Study 4. From the data

collected in Study 4, the detection sensitivity and criterion could be estimat-

ed and are illustrated in Figure 3.7 Two participants with nearly the same

detection sensitivity but different detection criteria are illustrated. According

to SDT, these participants are experiencing sensory states of similar magni-

tude (relative to the mean of the S1 and S2 distributions), for example, state

s. But given definition (e) (i.e., whether or not a stimulation is reported as

7 It might be surprising to note that the detection sensitivity and criterion were negatively

correlated. However, although the detection sensitivity and criterion are mathematically inde-

pendent of each other, there is no reason to assume that they are inter-individually unrelated.

The mechanism underlying the correlation is that participants who find it difficult to distin-

guish signal from noise trials generally use conservative criteria so as not to falsely classify

noise as signal.

32

detected), those same sensory states will be claimed to be “subliminal” for

one participant but “supraliminal” for another. Based on their equal ability to

detect the stimuli, this seems like a difficult position to advocate. Especially

when we consider that if the two participants were to do the experiment

again, but one participant was stimulated with many catch trials and the oth-

er with many strong signal trials, those criteria would probably shift and

what was previously subliminal for one participant would become supralim-

inal for the same participant, and vice versa.

Some experimenters have suggested “criterion-free” subjective measures

of perception (e.g., Dienes, 2007; Dienes & Seth, 2010; Kanai et al., 2010;

Kunimoto et al., 2001; Persaud, McLeod, & Cowey, 2007). These measures

are very interesting, although it still seem unclear whether they are truly

“bias-free” (Evans & Azzopardi, 2007; Fleming & Lau, 2014) and, to some

extent, what they measure (Jachs, Blanco, Grantham-Hill, & Soto, 2015;

Zehetleitner & Rausch, 2013). As these measures are still quite rarely used

in subliminal processing experiments, I will not be treating them further

here.

33

5. Subliminal semantic processing

Reading words involves processing at several hierarchical levels: energy and

visual features build up graphemes (i.e., letters or combinations of letters);

graphemes build up orthographic forms and whole words; and whole words

are in turn linked to meaning and phonemes (Coltheart, Rastle, Perry, Lang-

don, & Ziegler, 2001; Harm & Seidenberg, 2004; Perry, Ziegler, & Zorzi,

2010).

One interesting property of semantic processing is that when word mean-

ing has been extracted, activation seems to spread to “nearby” concepts in an

automatic manner (Collins & Loftus, 1975; Masson, 1995). This “spreading

of activation” means that after reading or hearing a certain word (e.g., dog),

other “nearby” words (e.g., cat) will shortly be more likely to be freely gen-

erated or more quickly responded to. This “spreading of activation” effect on

behavior is often called semantic priming. Both semantic processing in gen-

eral and semantic “spreading of activation” are nonconscious processes (i.e.,

we do not notice them happening).

To exemplify semantic priming, read the following:8

“Imagine yourself driving in a yellow car along a curvy road in Costa Rica.”

Now say a fruit.

If you immediately thought of a banana, you were just the victim of non-

conscious semantic priming of supraliminal stimuli (as you can clearly per-

ceive the example above).

Let us instead say that the example sentence was shown extremely briefly

and backward masked. The sentence was so perceptually degraded that you

did not have the slightest perception of anything being shown except the

mask. In that case, if you also came to think of a banana, you would have

been the victim of (subliminal) semantic processing of subliminal stimuli.

8 I thank Leo Derkert for this excellent example of semantic priming.

34

Figure 4. An illustration of a standard experimental setup used in experiments exam-ining subliminal semantic processing. First a prime word (“dog”) is shown briefly and after an inter-stimulus interval (ISI) a backward mask is then shown. Following this, a target word (“CAT”) is shown. Participants are to respond quickly to the target word in some manner. Reaction time to the target is usually faster if it is se-mantically congruent with the prime than if the prime and target are semantically incongruent (e.g., “dog moon”). In a separate block or following the target response, participants are to respond to the prime word.

The automatic nature of semantic priming means that it can be used as an

index of whether or not semantic information was processed, independent of

what participants report. The possibility of subliminal semantic processing is

often investigated through semantic congruency priming in a disassociation

paradigm. A representative experimental setup is shown in Figure 4. In the

indirect task, participants are asked to respond quickly to a target word (e.g.,

sheep). Participants are often quicker to respond and more accurate in their

responses if the target word is preceded by a semantically congruent prime

word (e.g., dog) than if the target word is preceded by a semantically incon-

gruent prime word (e.g., house). In a subliminal condition, the prime word is

presented briefly and backward masked. In the direct task, participants are

asked to respond in some manner to the prime and these responses are used

as a measure of whether or not the prime word was perceived. Any amount

of semantic congruency priming (i.e., a reaction time difference between

incongruent – congruent prime target pairs) is taken as indicating that se-

35

mantic information was extracted from the prime word. If there is semantic

congruency priming in the indirect task, but no evidence of perception in the

direct task, this would demonstrate subliminal semantic processing.

The possibility of subliminal semantic processing is of particular im-

portance as (a) logically, if subliminal word meaning is to be able to influ-

ence decision making or other higher-order cognitive functions, word mean-

ing must first have been able to be subliminally extracted, and (b) semantic

processing in itself is arguably a higher-order cognitive function and as such

whether or not it can occur sets an upper limit on the power of subliminal

processing.

Evidence for subliminal semantic processing

After some rocky beginnings when subliminal semantic processing was rou-

tinely questioned (Eriksen, 1960; Holender, 1986), recent reviews suggest

that subliminal semantic processing is a real phenomenon: “our quantitative

review confirms that subliminal information can be processed semantically”

(Van den Bussche, Van den Noortgate, & Reynvoet, 2009, p. 469), and

“subliminal priming has now been convincingly demonstrated at visual, se-

mantic, and even motor levels” (Dehaene & Changeux, 2011, p. 202). A

review by Kouider and Dehaene (2007) reached the same conclusion and

provides an excellent historical overview of the field. Snodgrass’s (2004a)

review also reached the same conclusion as does a more recent review by

Ansorge et al. ( 2014).

Since these reviews, several additional studies have also reported sublim-

inal semantic processing (e.g., Adams & Kiefer, 2012; Axelrod, Bar, Rees,

& Yovel, 2015; Bell, Forster, & Drake, 2015; de Wit & Kinoshita, 2015;

Finkbeiner, 2011; González-García, Tudela, & Ruz, 2015; Ortells et al.,

2016; Van den Bussche et al., 2013; Xiao & Yamauchi, 2015).

The reported effect of subliminal semantic primes is generally quite

small: reaction time to the targets is typically 400–600 ms. Whereas supra-

liminal semantic priming effects can typically be around 100–150 ms (an

increase of about 30%), subliminal semantic priming effects are typically

much smaller, ranging between 10 and 50 ms (an increase of about 5%)

(e.g., Bell et al., 2015; Finkbeiner, 2011; Ortells et al., 2016; Van den Buss-

che et al., 2013; Van den Bussche, Van den Noortgate, et al., 2009).

Of some importance, I would claim, is that except for the review by

Snodgrass et al. (2004a), none of the reviews takes any stand on what consti-

tutes an adequate measure of perception, so the results of semantic priming

at various subjective and objective thresholds are intermixed. That is, none

of the reviews provides a criterion for determining what behavioral perfor-

mance should be taken as indicating that the priming was indeed subliminal.

36

To demonstrate this point, although the meta-analysis by van den

Bussche, et al. (2009) explicitly included only studies with subliminal stimu-

li (criteria 2, p. 455), the authors found that variability in objective perfor-

mance between the studies explained 20% of the variance of the variability

in semantic priming effects between the studies (p. 459). Thus, although

some experimenters operationalized a lack of perception according to objec-

tive measures, these same studies could not all have been conducted at the

objective threshold.

In summary, today the field in general seems to regard subliminal seman-

tic processing as a well-accepted phenomenon. Current concerns in the field

of subliminal processing seem more focused on investigating how top down

signals can modulate subliminal semantic processing and on explaining the

mechanism behind subliminal semantic priming (discussed below). But there

are skeptics, of course (e.g., Haase & Fisk, 2015; Holender & Duscherer,

2004; Moors, Boelens, van Overwalle, & Wagemans, 2016).

Semantic or motor priming?

Above I explained congruency priming as resulting from the “spreading of

activation” between semantically “nearby” concepts. This implies that the

extraction of the meaning of the prime words is what drives congruency

priming. There is debate, however, as to whether this is the most parsimoni-

ous explanation of masked congruency priming in many experiments.

Another explanation could be that participants associate the prime and

target response categories based on individual letters. Congruency priming

could in this case occur because of a response competition elicited by the

letters of the prime and the target stimulus without the meaning of the prime

words being extracted (Ansorge et al., 2014; Kiefer & Martens, 2010;

Klinger, Burton, & Pitts, 2000; Kunde, Kiesel, & Hoffmann, 2005; Van

Opstal, Reynvoet, & Verguts, 2005).

The latter explanation gains support from results in which congruency

priming occurred only for prime words that had been practiced throughout

the experiment and not for (the semantic category of) novel prime words

(Damian, 2001). This would suggest that what looks like semantic congru-

ency priming in some experiments should perhaps instead be labeled motor

priming (Klotz & Neumann, 1999). Arguments against this explanation

come from studies demonstrating that the magnitude of congruency priming

is related to how strongly or weakly semantically related the prime and tar-

gets are to each other (Ansorge et al., 2014; Ortells et al., 2016).

In many experiments (e.g., the empirical studies in this thesis; Abrams,

Klinger, & Greenwald, 2002), individual letters are not specifically linked to

target responses, but to semantic categories. This muddles the possible ex-

planation of congruency priming somewhat as, on one hand, only letters

need to be extracted from the prime words, whereas, on the other hand, those

37

letters map to semantic categories in conflict with the target words. Study 1

in this thesis provides some suggestive evidence that in those cases, word

meaning may indeed be what is extracted. As researchers seem undecided as

to when exactly congruency effects should be viewed as semantic priming or

motor priming (Ansorge et al., 2014; Kunde et al., 2005; Van Opstal et al.,

2005) and semantic priming does not seem to be completely explained as

motor priming (Kouider & Dehaene, 2007; Van den Bussche, Van den

Noortgate, et al., 2009), in this thesis, I have ignored the debate and taken

claims of semantic processing at face value.

Evidence against subliminal semantic processing?

There are fewer reports of semantic processing not occurring for subliminal

stimuli (but see e.g., Brintazzoli, Soetens, Deroost, & Van den Bussche,

2012; Cheesman & Merikle, 1984; de Wit & Kinoshita, 2015; Pratte & Rou-

der, 2009; Tzelgov, Porat, & Henik, 1997). In one sense, this is quite natural.

After all, when a result is not statistically significant (p > α) we cannot con-

clude that the null hypothesis is correct; in fact we cannot conclude anything

at all (see Chapter 7; Dienes, 2011; Gallistel, 2009). As such, it is natural

that null findings of subliminal semantic priming end at piloting or in the file

drawer.

Even if a lack of subliminal priming were to be supported by Bayesian

statistics (which can support the null hypothesis; Dienes, 2014; Gallistel,

2009), we still learn little, because the particular signal strength used in the

experiment can always explain the result. On one hand, lack of processing

when there is no perception is uninteresting because slightly stronger stimuli

could have led to processing but not perception. On the other hand, pro-

cessing and perception of very weak stimuli is uninteresting because slightly

weaker stimuli could have excluded perception but not processing.

This evaluation is common in the literature (e.g., Finkbeiner, 2011; Snod-

grass, Bernat, & Shevrin, 2004b). When reviewing a study, i.e., that of Pes-

soa (2005), that did not find subliminal processing, Kouider and Dehaene

(2007) suggested that “before making such a strong assumption [i.e., that

subliminal processing did not occur], one would need to check that … mask-

ing was not too strong to prevent any form of processing” (p. 866). As such,

it is doubtful whether null reports of subliminal processing can ever be used

to generate acceptable negative evidence against the hypothesis. In Chapter

10, I suggest a possible way of testing whether semantic processing is inde-

pendent of perception that does not entail demonstrating a lack of pro-

cessing.

Some suggestive evidence against the possibility of subliminal semantic

processing can be derived from findings regarding analogous higher-order

processing. Reading is generally seen as a higher-order cognitive function

38

and orthographical, phonological and lexical processing are all relatively late

cortical processes (i.e., occurring further along the visual processing path-

ways; Binder, Desai, Graves, & Conant, 2009; Price, 2012; Taylor, Rastle, &

Davis, 2013). As mentioned, studies in neurobiology suggest that activity in

similarly “late” cortical processes (e.g., secondary sensory areas and beyond)

seems to correspond to perceptual decisions rather than the physical stimuli

(Grill-Spector et al., 2004; Hulme et al., 2009; Pessoa & Padmala, 2005;

Romo & de Lafuente, 2013). If there is any merit in comparing separate

processes at similar “stages” in the neural hierarchy, this would suggest that

semantic processing could be similarly related to perception.

Another suggestive piece of evidence against subliminal semantic pro-

cessing is the fact that subliminal priming effects are often much smaller

than the corresponding supraliminal effects (e.g., 30 vs 130 ms). If the sub-

liminal effect is truly driven by an automatic process, the effects should ar-

guably not be this much smaller. As such, this opens up the possibility that

the small “subliminal” effects could actually be artifacts caused by residual

percepts.

39

6. Validity of measures of sensitivity: Study 1

Background

The aim of Study 1 was to evaluate the validity of objective performance in

the direct task in the disassociation paradigm as a measure of perception. In

the disassociation paradigm there are two measures of stimulus processing: a

direct measure of perception and an indirect measure of the cognitive pro-

cess of interest. If the indirect measure indicates some degree of stimulus

information but the direct measure does not, then this is taken to demonstrate

that the information has been subliminally processed.

For this conclusion to hold, it is important that prime perceptibility be

equal in both tasks. Reingold and Merikle (1988) suggested that the direct

and indirect tasks must be performed under exactly the same circumstanc-

es—the primes and targets must be identical and equally visible in both

tasks—with the only difference between the two tasks being in the task in-

struction. Since Holender’s (1986) critical review of the methodology of

subliminal processing experiments, many experiments have implemented

Reingold and Merikle’s (1988) recommendations.

However, there are reasons to suspect that even when these recommenda-

tions are implemented, objective performance in direct tasks might be under-

estimated, i.e., objective performance may not be a valid measure of percep-

tion. One potential source of underestimation in the direct task comes from

the fact that in standard paradigms (see Figure 4), prime perception respons-

es do not occur immediately following the prime stimulus. Instead, a target

stimulus is presented between the prime and the prime response. This target

could act as a distractor that interferes with prime responses. This would

underestimate perception as an interference effect on prime responses do not

imply that the prime was not perceived when it was presented. Study 1 eval-

uated whether this can be the case when discrimination sensitivity, i.e., defi-

nition (c), is used as a measure of perception.

Method

One-hundred and thirty-eight participants participated in four experiments

using stimuli very similar to that shown in Figure 4 (adopted from Fink-

beiner, 2011). Participants had two principal tasks: in the first block of each

40

experiment, participants performed an indirect task in which they were to

quickly discriminate a target word in Swedish. In this task, whether or not a

backward-masked prime word had been semantically processed was indi-

rectly measured as a congruency effect on the reaction time of the target

responses.

Following this block, participants performed the direct task in which they

were to discriminate the prime word. This task directly measured whether or

not the prime word had been perceived. There were different conditions in

the direct task. In all experiments, a “standard” condition was used in which

the target stimuli (following the prime word) were the same Swedish words

as in the indirect task. In experiments 2–4 new conditions of the direct task

were introduced. In these conditions, the target stimuli were replaced by

stimuli semantically unrelated to the prime words (i.e., colored rectangles,

backward masks, or words in Swahili). The key question was whether the

discrimination sensitivity for the prime words would improve under these

conditions. If it did, this would suggest that the semantic content of the Swe-

dish word targets interfered with the prime discrimination responses.

In experiments 1–3 participants dual-tasked in the direct task: they were

to discriminate both the prime word (a very difficult task) and the color of

the prime word (a very easy task). In experiment 4, participants discriminat-

ed only the prime word. This was done to test a previous suggestion that

inclusion of an easy task might improve performance in the difficult task

(Finkbeiner, 2011).

Results and discussion

In the indirect task in all four experiments participants were congruency

primed by the semantic category of the prime stimuli. In the standard condi-

tion of the direct task, participants had poor but better-than-chance discrimi-

nation sensitivity. The congruency priming was therefore not suggested to be

subliminal (but see Chapter 7).

The main result of Study 1 was that prime discrimination sensitivity im-

proved when the Swedish word targets were replaced by other stimuli. Criti-

cally, prime discrimination sensitivity improved substantially when ortho-

graphically similar stimuli that lacked semantic content (for our Swedish

participants) were used as targets. This suggests that prime discrimination

sensitivity in the standard condition of the direct task may be underestimated

due to interference effects from the semantic content of the target stimuli.

That is, semantic priming goes both ways.

There is debate concerning whether congruency priming from difficult-to-

perceive stimuli reflects word meaning extraction or only orthographic pro-

cessing (Abrams et al., 2002; Ansorge et al., 2014; Klinger et al., 2000;

Ortells et al., 2016). As it could be demonstrated in Study 1 that it was the

41

semantic content of the target stimuli that interfered with prime discrimina-

tion responses, this suggests that, at least in this experiment, the meanings of

the primes were extracted. This provides suggestive evidence that word

meaning could drive congruency effects in some other experiments as well.

Underestimations of sensitivity

Measures of sensitivity in the direct task of standard paradigms do not gen-

erally adhere to Newell and Shanks’ (2014) criteria of adequate measures of

awareness. Instead, several potential sources of underestimation of sensitivi-

ty have now been reported: interference effects from the semantic content of

the target stimuli (Study 1), interference effects from prime–target similarity

(Vermeiren & Cleeremans, 2012), task-difficulty artifacts (Pratte & Rouder,

2009), and task-instruction artifacts (Lin & Murray, 2014). The magnitude

of underestimation can (in this application) be quite substantial, as is shown

in Table 1. Many of these underestimations may occur at the same time, but

of course in a non-linear manner. These underestimations of sensitivity im-

ply that weak percepts, as indexed by sensitivity, may occur even though the

mean measured sensitivity is zero.

Finkbeiner (2011) suggested a procedure to control for the task-difficulty

artifact—namely, including a very easy task (prime color discrimination) to

improve performance in the very difficult task (prime word discrimination).

This procedure was not tested by Finkbeiner (2011) and our results in Study

1 do not substantiate this suggestion. Including the easier, prime color task

therefore may not control for task-difficulty artifacts in studies of subliminal

processing (Pratte & Rouder, 2009).

Another source of underestimation of sensitivity comes from the fact that

sensitivity is not constant throughout an experiment. As motivation and at-

tention fluctuate, so does sensitivity. A participant may have a strong task

focus and thus strong sensitivity in some trial bins but be poorly motivated

and thus have weak sensitivity in other trial bins. The average block-level

sensitivity will underestimate the true sensitivity in certain trial bins. This is

important, as periods of concentration in which the prime is perceived may

drive the often small “subliminal” priming effects in certain experiments

(Van den Bussche, et al., 2009). Support for this notion is presented in Chap-

ter 10. One way of alleviating this problem would be to measure the direct

and indirect measures concurrently in the same trials, rather than in separate

blocks, as fluctuations in attention would then perhaps more similarly affect

the two tasks.

To control for underestimation when sensitivity is used as a measure of

perception, primes and targets should be visually or semantically different

(Study1; Vermeiren & Cleeremans, 2012), easy and difficult primes should

be intermixed (Lin & Murray, 2014; Pratte & Rouder, 2009), and the direct

and indirect tasks should be measured concurrently (Studies 3 and 4). It is

42

difficult to evaluate whether subliminal semantic priming has been demon-

strated when all underestimation problems are taken into account.

Table 1. Impact of different sources of underestimation on sensitivity.

Source of

underestimation

Impact

(with–without) Source

d' p(c)

Semantic content of

target 0.31–0.59 56–62% Study 1

Task-difficulty artifact 0.20–0.56 54–61% Figure 2 in Pratte and

Rouder (2009)

Task-instruction artifact 0.15–0.51 53–60% Figure 2 in Lin and Mur-

ray (2014)

Prime–target similarity 0.20–0.70 54–64%

Figure 3 in

Vermeiren and

Cleeremans (2012)

Note. Estimates from the published studies are approximated from the relevant figures. Con-

versions between d' and the proportion correct, p(c), assume unbiased criteria.

43

7. Validity of statistical tests of lack of perception: Study 2

“… no testimony is sufficient to establish a miracle, unless the testimony be of such a kind, that its falsehood would be more miraculous, than the fact, which it endeavours to establish …” (Hume, 1748, Chapter 10)

Background

The aim of Study 2 was to evaluate how false-positive-prone statistical pro-

cedures used to demonstrate that a stimulus is subliminal can be. In Study 1,

138 participants were tested across the four experiments. Across experi-

ments, this large sample was clearly not subliminal (see manuscript). How-

ever, based on conventional statistical approaches, one experiment (n = 26)

could be seen as supporting subliminal processing. Furthermore, resampling

from the larger sample based on sample sizes often used in the field of sub-

liminal priming would, based on conventional statistical analyses, have led

to false support of subliminal priming up to 25% of the time. This suggests

that the statistics used in this field may capitalize on sampling and measure-

ment error.

Indeed, several questionable statistical approaches are used in the field of

subliminal priming: a non-significant difference is often used to support the

null hypothesis (that the stimuli were subliminal), regression analysis is of-

ten based on samples with substantial measurement error but restricted range

in the regressor (to support priming at zero sensitivity), and post hoc selec-

tion of participants who fulfill the hypothesis (those that perform near

chance levels) is often used to test the hypothesis.

In Study 2, a simulation approach was used to evaluate the error rate of

falsely supporting subliminal priming when chance-level objective perfor-

mance is used as a measure of perception, i.e., definition (a) or (c). Conven-

tional analytical methods (based on null hypothesis significance testing)

were compared with Bayesian analytical methods (Dienes, 2008).

44

Method

Because of the mathematical complexity involved in estimating power based

on non-normal distributions of sensitivity (true sensitivity < 0 is unrealistic),

we used a simulation approach to estimate the error rate of falsely supporting

subliminal priming.

Based on published studies of subliminal processing, we simulated sam-

ples with 20–100 subjects and 50–600 trials in the direct and indirect tasks

and various distributions of true sensitivity and indirect effects. We assumed

in most simulations that only participants who could perceive the primes

(sensitivity > 0) manifested any indirect effects. We then applied t-tests,

calculated Bayes factors, used regression analysis, and conducted post hoc

sorting of participants to estimate how often these analytical approaches

would lead to false support of subliminal priming.

To calculate Bayes factors to evaluate sensitivity, we defined the a priori

distribution (or the alternative hypothesis) as a half-normal distribution with

a mean of 0 and a standard deviation of 0.2. That is, the alternative hypothe-

sis was that the true sensitivity was slightly above 0, with greater sensitivity

being less probable and sensitivity greater than 0.4 being improbable. This

alternative hypothesis was based on sensitives in various experiments in

which the stimuli were difficult to perceive but not subliminal.

Results and discussion

The overall finding was that conventional statistical approaches applied to

sample sizes (and numbers of trials used to sample sensitivity) common in

the field of subliminal processing have a high error rate of falsely supporting

subliminal effects (i.e., 30–50%). In comparison, calculating Bayes factors

to test for subliminal processing results in a substantially lower false-positive

rate.

In Study 2, the assessed outcome of the different statistical approaches

was that “subliminal processing occurred,” this outcome being a combina-

tion of (1) lack of a direct effect, and (2) an indirect effect. This might have

obscured the main problem in conventional approaches, namely, that p > α

does not indicate that the stimuli was subliminal (i.e., does not support the

null hypothesis). I therefore here performed new simulations and analyses to

clarify the issue. The results are shown in Figure 5.

The upper row shows various distributions of true sensitivity. Although

these distributions include some participants with a true sensitivity of zero

(i.e., participants for whom the stimuli is subliminal), they also include many

participants with a sensitivity > 0. The second row is the probability that a t-

test applied to samples drawn from this distribution would (incorrectly) sug-

gest to researchers that their stimuli are subliminal (i.e., p > α; y-axis).

45

Figure 5. Upper row: various distributions of sensitives that include participants with sensitives above zero. Second row: Proportions of samples (of 10,000) that lead to false support for the stimuli being subliminal based on p > α given a t-test (y-axes) applied to a variable number of participants (lines) and trials used to estimate sensi-tivity (x-axes). Third row: The same but concluding that the stimuli are subliminal based on a Bayes factor (<1/3) using the alternative hypothesis suggested in Study 2. Bottom row: The same but concluding that the stimuli are subliminal based on a Bayes factor (<1/3) using a “default prior” (Cauchy distribution with an SD of 0.707).

As can be seen, given common sample sizes and numbers of trials used to

assess sensitivity applying a null hypothesis significance test is not a reliable

diagnostic regarding whether or not the stimuli is subliminal for the sample.

Importantly, this estimate of the false positive rate is based on measures

of sensitivity that are affected only by measurement error arising from the

binomial nature of the forced-choice approach. That is, underestimations of

sensitivity based on various artifacts (Chapter 6) are not taken into account.

Another factor not taken into account in Study 2 is that the direct and indi-

46

rect measures might be differently affected by residual perception in many

experiments (leading to different effect sizes; Vadillo et al., 2016).

Bayesian statistics

A Bayesian approach to evaluating whether or not stimuli are subliminal (to

support the sensitivity being zero, the null hypothesis) was demonstrated to

be substantially better than conventional tests. To exemplify this on the new

simulations in Figure 5, a Bayes factor was calculated based on the alterna-

tive hypothesis used in Study 2. The results are shown in the third row in

Figure 5. As can be seen, this approach leads to much less false support for

the stimulus being subliminal.

A complication (and strength) of Bayesian data evaluation is that the al-

ternative hypothesis needs to be clearly specified. In Study 2 we modeled

this based on previous reports of difficult but not subliminal stimuli (see

manuscript for details). To evaluate whether the low false-support rate was

not due to an overly conservative hypothesis, we tested how the alternative

hypothesis fared when true sensitivity was assumed to be zero: Based on 40

participants who all truly are subliminal and 100 trials used to measure sen-

sitivity, there is a 72% probability that calculating a Bayes factor based on

the alternative hypothesis used in Study 2 will support the null hypothesis.

This corresponds to a null hypothesis significance power level of 72%, a

power level I believe many psychologists find acceptable.

Another way of evaluating the alternative hypothesis used in Study 2

could be to compare it with the “default prior” suggested by Rouder, Speck-

man, Sun, Morey, and Iverson (2009) to be used when it is difficult to know

how to model the alternative hypothesis: the “default prior” is a Cauchy dis-

tribution (Student’s t-distribution with one degree of freedom) with a stand-

ard deviation of 0.707. This is the default setting used in the new and user-

friendly software for Bayesian statistics, JASP, Version 0.7.5.5 (JASP Team,

2016). To compare how this “default prior” compares with the prior used in

Study 2, the new simulations in Figure 5 were subjected to a new analysis

using this prior. The results are shown in the bottom row of Figure 5.

The “default prior” has a slightly higher false-positive rate than does the

prior used in Study 2. This is because the “default prior” is wider though

most importantly, symmetrical around zero. Using a symmetrical prior is

somewhat unreasonable in this application, as true sensitivity below zero is

improbable. But I believe the take home-message is that using Bayesian

statistics improves data evaluation, not that any one specific alternative hy-

pothesis is that important. Indeed, the influence of the specification of the

alternative hypothesis on the outcome of Bayesian statistics is often vastly

overestimated.

47

Another improvement of the Bayesian approach, compared with null-

hypothesis significance testing, is that it is not sensitive to stopping rules. As

illustrated in Study 2 and discussed at length before (e.g., Dienes, 2011), if

the data are insensitive regarding whether or not the stimuli is subliminal,

more participants can simply be run until a decisive Bayes factor is reached.

Perhaps the most important advantage of calculating a Bayes factor, com-

pared with a null-hypothesis significance test, is not the low false-error rate,

but rather that the statistics can actually be used to support the hypothesis

researchers are interested in.

Post hoc sorting

A common approach used in studies of subliminal processing when experi-

menters suspect that the stimuli are not subliminal (e.g., p < α) is to redo the

analysis only on the subjects whose sensitivities are near zero. In Study 2 we

demonstrate that this procedure substantially increases the risk of falsely

concluding that subliminal priming occurs, independently of whether the

evaluation is based on null hypothesis significance testing or a Bayesian

approach.

The reason for this is regression to the mean. Let us assume, as we do in

Study 2, that priming effects are directly related to sensitivity. A participant

with a true sensitivity of some value greater than zero thus also has a prim-

ing effect greater than zero. That participant may get “unlucky” and perform

worse or much worse than her true sensitivity in the direct test, but it is un-

likely that the same participant will perform equally poorly in the indirect

test. The participant may have a poor measured sensitivity and still have a

measured priming effect greater than zero.

This problem is compounded when few trials are used to measure sensi-

tivity (Merikle, 1982). In Figure 6, various true sensitivities (dotted lines) are

plotted against the number of trials used to measure these (x-axis), and the

probability that the measured sensitivity will score below the 95% upper

limit of chance performance (y-axis). The message is that an extremely large

number of trials is needed to conclude anything about a single participant’s

true sensitivity.

The post hoc sorting of participants is a procedure that clearly capitalizes

on chance. If some participants have sensitivities substantially greater than

zero, this in itself should alert the experimenter that the stimuli to which they

were exposed simply may not have been subliminal. Analyzing only the

participants with poor sensitivity and concluding that the priming effects

were subliminal “is analogous to testing a new medication and then discard-

ing all those patients who die from it, concluding that all ‘suitable’ patients

do fine under the new drug” (Schmidt, 2015, p. 36).

48

Figure 6. The dotted lines are various true values of sensitivity (corresponding to proportions of correct scores of 0.54, 0.58, 0.62, 0.66, and 0.69 assuming unbiased criteria). The left y-axis is the probability that the measured sensitivity will be below the 95% upper limit of zero sensitivity based on a given number of trials (x-axis). The solid line illustrates the 95% upper limit of measured zero sensitivity (right y-axis).

49

8. Review of demonstrations of lack of perception

“Ahhh, this porridge is just right.”—Goldilocks

The aim of the quantitative review included here was to evaluate the strength

of evidence in studies of subliminal processing that the stimulus was sublim-

inal when objective measures are used to gauge perception (i.e., that perfor-

mance was at chance level). Experimenters interested in subliminal pro-

cessing want to find a signal strength that is “just right”—not so weak that it

excludes potential subliminal processes, but not so strong that the stimulus is

perceived. Perhaps the biggest practical problem is that it is difficult (if not

impossible) to know beforehand whether the signal strength is “just right”

for a particular group of participants (see Rouder et al., 2007, for a discus-

sion of threshold estimation procedures). Only after the experiment can the

experimenter establish whether or not the stimulus was subliminal, i.e.,

whether the signal strength was “just right.”

In his 1986 review, Holender concluded that few studies at that time

could demonstrate that their stimuli truly were subliminal according to ob-

jective definitions. After Holender’s review, many experimenters embraced

the challenge of demonstrating subliminal processing at the objective thresh-

old, i.e., definition (a) or (c) (Kouider & Dehaene, 2007). The experimental

methodology that Holender criticized has improved substantially, but the

statistical demonstration has remained almost uniformly based on a ques-

tionable interpretation of null hypothesis significance testing: a non-

significant effect (p > α) is seen as providing evidence that the sensitivity is

zero (the null hypothesis). In the previous chapter, it was demonstrated that

this approach is prone to falsely “supporting” that the stimuli were sublimi-

nal.

However, despite this questionable analytical procedure, it could still be

the case that experimenters have succeeded in using subliminal stimuli. I

therefore wanted to assess to what extent modern studies could be said to

have passed Holender’s challenge of demonstrating processing at the objec-

tive threshold, and reviewed studies published in 2014 and 2015 in which the

experimental stimuli were claimed to be at the objective threshold.

In the studies included in this review, participants often performed two

different tasks, a direct task that assessed prime perception and an indirect

50

task that assessed whether the prime was processed to some degree of inter-

est. In a study by Kiefer, Sim, and Wentura (2015), for example, participants

in the direct task had to discriminate whether a prime word was emotionally

pleasant or unpleasant. Chance performance in this task was taken to indi-

cate that the prime words were subliminal. In the indirect task, participants

instead had to discriminate whether a target word was emotionally pleasant

or unpleasant. A congruency priming effect on these responses was taken to

indicate whether or not the prime was (subliminally) processed.

In this review, I focused solely on the direct measures and whether per-

formance in the direct task supported the claim that the stimuli were sublim-

inal. Many studies measured performance in the direct task as d'; when other

measures of performance (e.g., proportion correct) were used these were

transformed to d' (see Methods). The indirect measures in the studies con-

cerned a range of potentially subliminal effects, for example, face processing

(Sato, Kubota, & Toichi, 2014), cognitive control (Ocampo, Al-Janabi, &

Finkbeiner, 2015), and brain activation (Hoffmann, Mothes-Lasch, Miltner,

& Straube, 2015). The experiments mainly used backward masking or con-

tinuous flash suppression to render the stimuli difficult to perceive.

The focus of this quantitative review was slightly different from that of a

meta-analysis, as an aggregated “meta-estimate” of sensitivity was not the

aim. The aim was instead to evaluate whether subliminal processing experi-

ments as a group have demonstrated that they only included participants who

cannot perceive the stimuli (as indexed by objective performance).

Method

Retrieval of studies

Studies were located using the Web of Science search engine (formerly Web

of Knowledge; http://apps.webofknowledge.com/). A search was conducted

on the 10th of November 2015 for papers published in 2014–2015 that in-

cluded “subliminal” in their titles, abstracts, author keywords, and/or “Key-

words Plus”© (“subliminal” was used as the search term for “Topic”). Using

this search term, 256 results were found. The first 100 experimental studies

investigating effects from allegedly subliminal stimuli were included in this

review.

Coding of studies

Empirical studies were included in an initial review if the authors explicitly

or implicitly claimed that their stimuli were subliminal (e.g., in the discus-

sion section, results section, title, or abstract). These studies were then coded

51

according to the following criteria to determine inclusion in the quantitative

analyses.

(1) Did the study use a measure to support the claim that the stimulus

was subliminal?

If the authors only claimed that stimuli were subliminal (e.g., based on pre-

vious studies), these studies were excluded from the quantitative analyses.

Twenty-two studies were excluded from the quantitative analyses for this

reason.

(2) Did the study use an objective measure to support the claim that the

stimulus was subliminal?

If the authors only used a subjective definition of perception or if the authors

tested things other than perception (e.g., if the stimuli were remembered in a

later experimental session), these studies were excluded from the quantita-

tive analyses. Twenty-one studies were excluded from the quantitative anal-

yses for this reason.

(3) Did the study estimate sensitivity in a way that allows comparison of

the estimated sensitivity with zero sensitivity?

If the authors, for example, only estimated thresholds and then did not meas-

ure sensitivity at these thresholds, the studies were excluded from the quanti-

tative analyses. Eight studies were excluded from the quantitative analyses

for this reason.

(4) Otherwise, all studies were at least partly included in the quantitative

review.

In total, forty-nine studies were included in at least parts of the quantitative

analyses. Many of these studies included several experiments or several

stimuli claimed to be subliminal. Thus, ninety-seven estimates of sensitivity

could be included in the quantitative analyses. The appendix “Review refer-

ences” includes all 100 studies reviewed; references marked with an asterisk

were included in the quantitative analyses.

Data transformation

The included studies measured m-alternative-forced-choice detection or dis-

crimination performance as a measure of perception. Different scales (e.g.,

d', proportion correct, and area under the curve) were used in the different

studies. To compare studies using different scales, all scales were trans-

formed to d'.

Most studies measured 2-alternative-forced-choice proportion correct. In

these cases, proportion correct was transformed to d' via 2Φ–1

(pcorrect), which

assumes an unbiased criterion. When the measure was 3- or 4-alternative-

forced-choice proportion correct, the proportion correct was transformed to

d’ using the relevant tables supplied by Macmillan and Creelman (2005)

which also assume unbiased criteria.

52

Conversion of standard deviation measured in proportion correct to stand-

ard deviation measured in d' values was done in the following way: the pro-

portion correct at z = 0 and z = 1 (i.e., mean + 1 SD) was transformed to d'.

Standard deviation in d' values was estimated as the absolute difference be-

tween d' at z = 1 and z = 0. Simulations suggested that this procedure under-

estimates standard deviation in d' values by about 2% (assuming unbiased

criteria).

Results

Ninety-seven experiments from forty-nine studies were included in the anal-

yses. In these experiments, objective performance was used as a measure of

perception and lack of perception was operationalized as chance perfor-

mance (zero sensitivity). In most experiments (93 %), the direct measure was

discrimination, rather than detection, performance. Many articles reported

scarce information on the direct measure. In those cases, the corresponding

author for the article was contacted in order to complete the data. However,

even after this, some experiments could not be included in all analyses below

because of some missing information.

A complication in this quantitative review is that in about half of the ex-

periments (48%) participants or trials were sorted post hoc: Experimenters

who suspected that some participants could perceive the stimuli—because

the participants performed too well on the direct task or subjectively reported

perceiving the to-be-subliminal stimulus—excluded those outlying “perceiv-

ers.” Other studies sorted trials post hoc based on subjective reports of per-

ception and interpreted the stimulus as subliminal or not based on sensitivity

only in those trials. As discussed in the previous chapter, sorting participants

post hoc makes sensitivity estimates unreliable (as do sorting trials; Schmidt,

2015). Nevertheless, the quantitative analyses here were conducted on the

restricted samples from which “perceivers” were removed. This was done

because (1) data for the full sample could not be obtained for all experiments

and (2) in personal correspondence, many authors claimed that the stimuli

were subliminal only for the restricted sample, not the full sample. As such,

analyzing the restricted sample seems more fairly to evaluate experimenters’

claims of having demonstrated that their stimuli were subliminal.

Estimated sensitivity

Experimenters measured objective performance because they operational-

ized lack of perception as chance-level performance and used null hypothe-

sis significance testing to test whether there was a lack of perception. How-

ever, in 17% of the experiments, objective performance was statistically

different from zero (p < 0.05, two-tailed). The stimuli in these experiments

53

Figure 7. Distribution of estimated mean sensitivities. The gray area under the curve is based only on the experiments for which p > 0.05 (83% of the experiments). The thicker black line indicates all experiments. For reference, zero sensitivity and the expected distribution if all samples had zero sensitivity are shown in dashed lines. Sensitivity in individual experiments is indicated by the lines at the bottom (black lines are experiments for which p > 0.05 and gray lines are experiments for which p < 0.05).

therefore prima facie did not fulfill an objective operationalization of being

subliminal. In these experiments, the claim that the effect was subliminal

was therefore instead either supported through the use of regression analysis

(see next chapter) or based on subjective reports of no perception in debrief-

ing sessions.

In the other 83% of experiments, mean sensitivity did not differ statisti-

cally significantly from zero (p > 0.05). Figure 7 illustrates the distribution

of the mean sensitivities in these experiments. As can be seen, the estimated

sensitivities are slightly shifted away from zero: the mean sensitivity across

all experiments (in which p > 0.05) was 0.06, which is greater than what can

be expected by chance alone. This indicates that not all experiments could

have used stimuli that were subliminal for all participants.

When experimenters claim that their stimuli were subliminal for all par-

ticipants, they imply that all participants were drawn from a population with

a single value of sensitivity, i.e., zero. If the true sensitivity is zero, there is a

50–50 probability that a single observed sensitivity will be greater or smaller

than zero. If a sample’s true sensitivity is zero, there is also a 50–50 proba-

54

Figure 8. Left: Scatterplot of numbers of participants and trials included in the ex-periments. Right: Data are individual experiments. The y-axis is the ratio of estimat-ed standard deviation to expected standard deviation (based on the experiments’ numbers of participants and trials). Dashed lines are the 95% limits of what can be expected by chance. Gray circles are experiments for which p > 0.05 and white cir-cles are experiments for which p < 0.05.

bility that the observed mean sensitivity will be greater or smaller than zero.

However, only 18% of the samples had negative mean sensitivities (16% of

all samples independent of p-value).

Finally, the expected standard deviation for chance performance given a

certain number of trials and participants can be calculated. If a sample’s true

sensitivity is zero, there is a 50–50 probability that the observed standard

deviation will be greater or smaller than expected. However, 68% of the

samples (for which p > 0.05) had standard deviations greater than can be

expected by chance alone. Furthermore, 89% of the experiments that used

more than 100 trials (for which chance variability is smaller) had standard

deviations greater than can be expected by chance alone. These are three

indications that not all experiments could have used stimuli that were sub-

liminal for all participants.

Power

A non-significant result can of course result from an underpowered sample

rather than a lack of effect. The above considerations suggest that the lack of

a significant difference from zero could in some studies have been due to a

lack of power (in the sense of 1 – β).

Two factors that influence power are the number of participants and the

measurement error in an experiment. In the m-alternative-forced-choice pro-

cedures used in the experiments, measurement error partly stems from the

55

number of trials used: a small number of trials leads to a large standard devi-

ation and makes it almost impossible to separate chance performance from

above-chance performance. The left panel in Figure 8 illustrates the number

of participants and trials used in the experiments. Four of the experiments

included relatively large numbers of participants, i.e., n = 77, 96, 100, and

106. In the remaining experiments, the median sample size was 21 (range 9–

42). Thirty-one percent of the experiments used more than 100 trials; in the

remaining experiments, the median number of trials was 50 (range 24–96).

A large standard error makes it more difficult to reject the null hypothesis.

A complicating factor in this context is that this means that a large standard

error makes it easier to “support” the experimenter’s hypothesis (the null

hypothesis, i.e., finding p > 0.05). As mentioned above, many experiments

had larger standard deviations than can be expected by chance. This is

shown in the right panel of Figure 8. What this suggests is that in some ex-

periments, a large standard deviation—itself a sign that not all participants

were subliminal—might have been the reason why the experimenter con-

cluded that the stimuli were subliminal.

Together, these factors resulted in a median power of 13% to reject the

null hypothesis (1 – β) across experiments. That is, the experiments had a

13% chance of falsifying that the stimulus was subliminal, if in fact it was

not. As such, it seems that the experimenters often did not subject their hy-

pothesis to a fair trial.

Confidence intervals

Confidence intervals can be informative when interpreting non-significant

results. Figure 9 illustrates confidence intervals for the experiments included

in this quantitative review. By examining these confidence intervals, we can

see how non-significant differences led to misleading interpretations of the

data in many experiments: the reason the null hypothesis is not rejected is

because the data contain so much variability that they are consistent with a

wide range of possible true sensitivities, not because they are particularly

consistent with zero sensitivity. In null hypothesis significance terms, a con-

fidence interval between –0.2 and 0.4, for example, indicate that all values

within that interval are equally consistent with the observed mean; not that

the true mean is likely to be zero (Dienes, 2008; Morey, Hoekstra, Rouder,

Lee, & Wagenmakers, 2016).

56

Figure 9. 95% confidence intervals around mean sensitivity, sorted by the upper confidence limit. Gray circles are experiments for which p > 0.05 and white circles are experiments for which p < 0.05.

Bayes-factor analysis

To make an overall assessment of how many experiments had data that

could support the stimuli being subliminal, I calculated a Bayes factor for the

experiments (based on the alternative hypothesis used in Study 2; BH(0,0.2)).

The results are shown in Figure 10. Independent of p-value, 26% of the ex-

periments had data that supported their stimuli being subliminal (B < 1/3).

However, only one experiment had data that strongly supported the stimuli

being subliminal (B < 1/10). Most experiments (63%) had data that were

inconclusive. (Note that that this procedure has a 72% true-positive rate

based on 40 participants and 100 trials; see Chapter 7).

57

Figure 10. The plotted data are individual experiments and on the y-axis shows the resulting Bayes factor based on BH(0,0.2). Bayes factors below 1/3 and 1/10 “weakly” and “strongly” support the null hypothesis, respectively. Bayes factors between 1/3 and 3 indicate that the data are inconclusive. Gray circles are experiments for which p > 0.05 and white circles are experiments for which p < 0.05.

Discussion

The conclusions of this review is that, given the experimenters’ definitions

of subliminal, (a) most experiments lack data that support that the stimuli

were subliminal and (b) many experiments have data that are incompatible

with the stimuli being subliminal for all participants (e.g., the right panel in

Figure 7).

These conclusions are in line with a recent review by Vadillo et al. (2016)

in the field of unconscious learning. Using an approach similar to the present

one, those authors concluded that, in most studies, the conclusion that learn-

ing was unconscious was driven by a lack of power rather than a complete

lack of effects. These results are also, somewhat counter-intuitively, in line

with a meta-analysis that concluded that subliminal semantic processing has

substantial empirical support (Van den Bussche, et al., 2009). Although that

meta-analysis explicitly only considered experiments demonstrating sublim-

inal processing, across experiments sensitivity was positively correlated with

priming effects (Van den Bussche, et al., 2009, p. 459). That correlation

could only occur if there was a substantial range in sensitivity.

58

It seems unlikely that publication bias drives the result of the present re-

view. When a stimulus turns out to be not “just right” (p < 0.05), publication

bias would probably prompt the experimenter to rerun the experiment using

a slightly weaker signal strength (e.g., Muscarella, Brintazzoli, Gordts,

Soetens, & Van den Bussche, 2013) or excluding participants who perceive

the stimuli. Furthermore, journals seem more likely to accept reports of ef-

fects from subliminal stimuli than effects from near-subliminal stimuli (es-

pecially high-impact journals; Vadillo et al., 2016). As such, if there is a

publication bias, it probably tends in the opposite direction.

The experiments included in this quantitative review vary in methodolog-

ical quality. Therefore, to compare the results of these articles with those of

articles that one of the most-cited reviews refers to as supporting subliminal

semantic processing (Dehaene & Changeux, 2011, p. 202), in Box 1, I eval-

uate the strength of evidence that the stimuli used in those studies were sub-

liminal. In comparison, the experiments included in the present quantitative

review do not seem to constitute an unusual sample.

Together, the statistics compiled here indicate that the field as a whole has

not yet successfully stepped up to Holender’s challenge of demonstrating

that their stimuli are at the objective threshold (although, of course, individ-

ual experiments may have done this). Most experiments therefore use data

that do not exclude residual percepts as the cause of the claimed subliminal

effects.

59

Box 1. On page 202, Dehaene and Changeux (2011) review eight studies

that they, presumably, regard as strongly demonstrating subliminal semantic

processing. Here I focus on the studies’ demonstrations that the stimuli are

subliminal. Bayes factors were calculated by me based on the alternative

hypothesis used in Study 2.

Pessiglione et al. (2007): The stimuli were concluded to be sublimi-nal as “subjects reported seeing almost no stimuli” in a debriefing session (p. 906).

Dehaene et al. (1998): Six participants’ mean detection d' was 0.3 and seven participants’ mean discrimination d' was 0.2. As these mean estimates did not differ statistically significantly from zero, the stimuli were concluded to be subliminal. Insufficient data were re-ported for a Bayes factor calculation to be possible.

Lau and Passingham (2007): Two participants were excluded be-cause they were too good at discriminating the would-be-subliminal stimuli. For the remaining ten participants, the mean d' was 0.05 (SD = 0.33); as this did not differ statistically significantly from zero, the stimuli were concluded to be subliminal. Calculating a Bayes factor suggests that the data are inconclusive, i.e., BH(0,0.2) = 0.67.

van Gaal, Ridderinkhof, Fahrenfort, Scholte, and Lamme (2008): Fif-teen participants’ mean discrimination d' of 0.05 (SD = 0.17) did not differ statistically significantly from zero, so the stimuli were con-cluded to be subliminal. Calculating a Bayes factor suggests that the data are inconclusive, i.e., BH(0,0.2) = 0.69.

Naccache and Dehaene, 2001: In Experiment 1, the prime detection d' varied from 0.1 to 1.3 (mean 0.6) among eighteen subjects. In Ex-periment 2, eighteen participants discriminated numbers as smaller or larger than 5 at a mean d' level of 0.01. Because these values do not differing statistically significantly from zero, the authors conclude “that the primes were largely invisible” (p. 221). Insufficient data were reported for a Bayes factor calculation to be possible.

Van den Bussche, Notebaert, and Reynvoet (2009): Twenty-one par-ticipants discriminated prime words after the main experiment. As their mean discrimination d' was –0.1 (SE = 0.048=, the stimuli were concluded to be subliminal. Calculating a Bayes factor supports this conclusion, i.e., BH(0,0.2) = 0.08.

Leuthold and Kopp (1998): The discrimination energy threshold was found for each participant based on a staircase procedure of 40 trials, a procedure since suggested to be unreliable (Rouder et al., 2007).

Gaillard et al. (2006): based on supraliminal emotional words being correctly identified and subjectively reported as perceived more often than supraliminal neutral words, it was concluded that the emotional words were subliminally semantically processed.

60

9. Regression approach to testing subliminal effects: Study 3

Inter-individual variability

In research into the potentially subliminal processing of faces, faces were

often claimed to be subliminal as long as they were backward masked and

presented for only 33 ms. Many experimenters never tested whether their

stimuli were actually subliminal but instead relied on “standard parameters”.

However, Pessoa et al. (2005) demonstrated that there are large inter-

individual differences in the perception of masked faces: in fact, 64% of the

participants in their study were capable of detecting stimuli widely claimed

to be subliminal.

Similarly, studies of subliminal semantic processing often evaluate

whether or not a prime is perceived based on statistics applied to the means

of the group of participants. Implicit in this analytical approach is that the

mean is representative of the group. Some studies assume this more explicit-

ly in that they base their claim the the stimuli are subliminal on how a group

other than the experimental group behaved in previous studies (e.g., (Cham-

bon, Moore, & Haggard, 2015; Roberts, Gibbons, Kingsbury, & Gerrard,

2014). However, these approaches can be questioned as there are large inter-

individual differences in the energy thresholds of detection and discrimina-

tion. In other words, given a constant signal strength there will be large inter-

individual differences in sensitivity.

In Figure 11, I illustrate the inter-individual differences found in Studies

1, 3, and 4 of this thesis. The range is in line with several previous reports

(Albrecht & Mattler, 2012; Dagenbach, Carr, & Wilhelmsen, 1989; Haase &

Fisk, 2015; Muscarella et al., 2013; Zhicheng Lin & Murray, 2015). This

inter-individual variability in sensitivity must be taken into account some-

how when testing for subliminal processing.

61

Figure 11. Kernel-density estimates of inter-individual differences in discrimination (left) and detection (right) sensitivity. In each study the same prime signal strength was always used. For Studies 3 and 4, the d' for the 25-ms ISI primes is shown. Note that for Study 1, discrimination sensitivity in the “target word” condition is shown. This condition was demonstrated to underestimate sensitivity but is a stimulus set-ting representative of that used in many studies of subliminal processing; see manu-scripts for details.

False-positive rate of regression analysis

A natural approach to taking inter-individual differences in sensitivity into

account would be to use regression analysis, that is, to predict the indirect

priming effect based on the direct measure of perception. If the variables are

correlated, but the intercept is substantially above zero this could be inter-

preted as processing occurring when sensitivity is zero, (i.e., that subliminal

processing occurred). Many experimenters use this analytical approach to

support claims that the priming effects in their experiments are subliminal

(e.g., Jusyte & Schönenberg, 2014; Ocampo, 2015; Schoeberl et al., 2015;

Xiao & Yamauchi, 2014).

Greenwald et al. (1996) perhaps made regression analysis a common ana-

lytical approach. The approach has been criticized on statistical grounds

(Dosher, 1998; Miller, 2000) and in Study 2 we demonstrate that it has a

high error rate of falsely supporting subliminal processing. The high false-

positive rate of regression analysis stems from measurement error in the

regressor. Measurement error in the regressor means that the correlation will

be underestimated and if the correlation is underestimated, this will lead to

an inflated intercept. (In Study 2 we also demonstrate that in this application

orthogonal, Deming, regression does not substantially alleviate the problem.)

62

Basically, regression is a false-positive-prone analytical method if the

measurement error of sensitivity is similar in size to the range of the meas-

ured sensitivities. As many experimenters want to use stimuli that are sub-

liminal for all participants, this results in small ranges of sensitivities that,

together with measurement error, make regression an unreliable analytical

method. Two potential approaches to alleviating this problem are (a) to use a

stronger prime signal strength so that a greater sensitivity range will be

measured and (b) to use a large number of trials to decrease the measurement

error of sensitivity.

Background of Study 3

In Study 3, a Stroop congruency priming paradigm was used to investigate

subliminal semantic priming. The main aim of this experiment was to use

stronger signal strengths than those used in many experiments in order to

obtain a large inter-individual range of sensitivity, as this may permit regres-

sion as a statistical analysis test for subliminal processing.

The experiment was similar to one conducted by Van den Bussche et al.,

(2013). In that experiment, the authors used subjective measures to index

various sensory states and then used these to separate trials into “conscious,”

“uncertain,” and “unconscious” trial bins. I similarly reasoned that discrimi-

nation responses could be used to index perceptual states, and used these to

separate trials into those in which the prime was “correctly perceived” and

“misperceived.” My hypothesis was that semantic priming effects should be

stronger in trials in which the prime discrimination response was correct than

the average priming effects for the same participant. For incorrect discrimi-

nation responses, it was not clear to me whether priming effects would be

attenuated or even reversed (as congruent prime–target relationships were

misperceived as incongruent).

Method

Fifty-three observers participated in a Stroop congruency priming experi-

ment. The experimental setup was similar to that shown in Figure 4, except

that the prime stimuli were words denoting color and the targets were col-

ored rectangles. Direct (i.e., prime word discrimination sensitivity) and indi-

rect (i.e., Stroop priming) measures of prime stimulus processing were

measured in the same trials.

Several steps were taken to potentially increase the validity of the sensi-

tivity measure: three prime signal strengths (i.e., very difficult, difficult, and

easy-to-perceive prime–mask inter-stimulus intervals) were intermixed to

control for task difficulty; prime perception and priming effects were meas-

63

ured concurrently to control for non-constant sensitivity; and primes and

targets were visually different (i.e., words vs. rectangles) to control for inter-

ference effects from the targets. Although some degree of underestimation of

prime discrimination sensitivity might have been introduced by the prime

word and target stimuli being semantically related, this was necessary in

order to measure priming and perception in the same trials. To increase the

reliability of the statistical analysis, relatively strong prime signal strengths

were used to obtain a wide range of inter-individual variability in sensitivity.

Two hypotheses were tested: (1) inter-individual variability in sensitivity

should have an impact on priming effects; and (2) inter-trial variability in

sensory states (i.e., correct vs. incorrect prime discrimination responses)

should have an impact on priming effects.

Results and discussion

At the two stronger prime signal strengths, there were substantial congruen-

cy priming effects. At a mean level, the primes were perceivable at all signal

strengths; more importantly, there were large inter-individual differences in

prime sensitivity. The regression analyses suggested that prime discrimina-

tion sensitivity indeed had a strong influence on priming effects at both the

very difficult and the difficult prime signal strengths. At neither signal

strength did the regression analyses indicate subliminal congruency priming

(i.e., non-significant intercepts). This suggests that if there is a wide inter-

individual range in sensitivity, regression analysis can be a valid approach

for testing for priming effects at zero sensitivity.

This result was also replicated in Study 4. The result for the difficult

prime signal strength (i.e., 25-ms inter-stimulus-interval) is shown collapsed

across Studies 3 and 4 in Figure 12. Applying the regression analysis to the

combined sample also did not indicate subliminal priming (i.e., non-

significant intercept). As this result is based on a sample of 118 participants,

if the lack of significant subliminal Stroop priming is based on power, any

subliminal priming effects in these experiments must have been small in-

deed.

Several reviewers suggested that the prime signal strength (i.e., the short

duration, 6 ms) was perhaps was too weak to induce subliminal congruency

priming. Of course, it is true that had another prime signal strength been

used, there might have been subliminal priming effects. However, the signal

strength in an experiment should not be evaluated based on just one particu-

lar attribute of the prime stimulus (e.g., the duration), as the signal strength

depends on a combination of prime mask parameters. A better way of evalu-

ating the prime signal strength in an experiment is to look at the behavioral

effects: in this experiment, discrimination sensitivity had a larger mean and

64

Figure 12. Left: Regression analysis applied to the collapsed samples from Studies 3 and 4 for the 25-ms ISI prime signal strength. The solid lines illustrate the regression slope, the gray polygon is its 95% confidence interval, and the gray error bar at sensitivity = 0 is the 95% confidence interval for the intercept. Participants in Study 3 are shown as white symbols and participants in Study 4 as black symbols. One outlier (d' = 2, congruency priming = 305 ms) is not shown. That outlier and the other two crossed over circles were excluded based on regression diagnostics (e.g., Q-Q location and leverage). Right: Regression analysis applied to only a subset of data (i.e., those whose sensitivity is below the 95% upper limit of chance perfor-mance); no correlation can be found in this restricted range. The thinner line illus-trates the regression slope estimated from the full sample.

larger range than in many previous experiments. This indicates that the sig-

nal strength could not have been weaker than in those experiments.

Interestingly (and somewhat surprisingly), discrimination responses great-

ly affected the congruency priming effects at both the very difficult and dif-

ficult prime signal strengths. When discrimination responses were correct,

normal congruency priming occurred. When discrimination responses were

incorrect (i.e., congruent primes were reported to be incongruent), reversed

priming effects occurred. I interpreted the reversed priming effects as being

caused by the participants misperceiving the primes. This finding was repli-

cated and investigated more extensively in Study 4.

65

10. Perception-dependent priming: Study 4

Reversed priming: Study 4

Background

Perceptual outcomes can sometimes be inaccurate portrayals of a stimulus

(e.g., a happy face might be misperceived as sad). Intuitively, in these cases

although very early sensory processes might have coded the stimuli with

high verisimilitude, somewhere in the processing stream the information

became distorted, leading to an inaccurate perceptual outcome. By testing

which neural activity corresponds to the physical input and which corre-

sponds to the perceptual outcome in these situations, we may learn where in

the neural hierarchy the perceptual decision is made (Grill-Spector et al.,

2004; Hulme et al., 2009; Pessoa & Padmala, 2005; Romo & de Lafuente,

2013).

The main idea in Study 4 was that the reversed priming effects suggested

in Study 3 may indicate that semantic processing occurs after a perceptual

decision has been made. However, that the reversed priming effects in Study

3 were accompanied by incorrect percepts was only inferred based on the

fact that the effects occurred only for participants who on average perceived

the primes. In Study 4, I wanted to directly test whether reversed priming

effects were indeed accompanied by mispercepts of the prime stimuli. Partic-

ipants in Study 4 were therefore asked both to discriminate the prime word

and to (subjectively) rate their prime perception in each trial.

This procedure may also allow for testing whether semantic priming de-

pends on perception in a way that does not entail demonstrating a lack of

processing: it could be reasoned that if priming effects correspond to weak

but incorrect percepts, rather than the physical stimuli, then automatic pro-

cessing of the physical stimuli can hardly be the mechanism behind the

priming effects. Therefore I specifically focused on the trials that traditional

approaches would have considered being “subliminal.”

Method

Sixty-six participants participated in a Stroop congruency priming experi-

ment that was similar to Study 3 and the paradigm shown in Figure 4. The

66

experiment differed from Study 3 in that the most difficult-to-perceive prime

signal strengths were excluded, catch trials in which the prime was not

shown were included, and participants rated their (subjective) perception of

the primes.

The rating scale for prime perception was similar to the perceptual aware-

ness scale (Overgaard et al., 2006; Sandberg et al., 2010) and included three

steps: “no percept” indicated no percept of the stimulus; “unclear percept”

indicated a percept of something being presented, though it was unclear

what; and “clear percept” indicated a percept of the prime stimulus.

In each trial, reaction time to the target as well as discrimination response

to and perception rating of the prime were collected. Discrimination re-

sponses (correct vs. incorrect) and perception ratings were used to sort trials

into six bins. Congruency priming effects were calculated for each bin of

trials.

Results and discussion

The main result was a replication and extension of the reversed priming ef-

fects found in Study 3: participants were primed in accordance with their

prime discrimination response, regardless of whether these responses were

correct. That is, in trials in which participants gave correct discrimination

responses, they had normal congruency priming effects, but in trials in which

participants gave incorrect discrimination responses, they had reversed prim-

ing effects. These respective priming effects increased (or decreased) with

subjective prime perception ratings: in a minority of trials, participants re-

ported “clearly” perceiving the wrong prime, and in those trials they were

more strongly reverse primed than in the trials in which they reported “un-

clear” or “no” percept of the wrong prime (and vice versa for correct re-

sponses). This increase in reversed priming effect with perception rating

suggests that the effects were driven by perception rather than by an inhibi-

tory effect of the subliminal stimulus (Eimer & Schlaghecken, 1998). This

result suggests that semantic processing occurred after the weak physical

input had been transformed into a perceptual decision and thus is not inde-

pendent of perception.

Conventional approaches would have labeled some trials as subliminal:

trials in which participants reported “no percept” of the primes, i.e., defini-

tion (e), and in which participants also could not discriminate the primes

better than chance, i.e., definition (c). If semantic priming is driven purely by

the physical stimulus (i.e., is a subliminal process), then participants should

in these trials be primed by the physical stimulus. However, priming effects

even in these trials corresponded to the participants’ discrimination response.

I interpreted the congruency and reversed priming effects occurring in these

trials as suggesting that weak percepts and mispercepts may be intermixed

with non-percepts. That is, the same mechanism that explains the reversed

67

priming effects when participants report misperceiving the prime (“unclear”

or “clear percept”) also occur, but simply less frequently, when participants

report “not perceiving” the stimuli.

An alternative explanation of the (reversed) priming effects in trials rated

“no percept” is that participants were primed based on conscious or noncon-

scious expectations of the prime words, not on percepts of the prime words.

Separating these two possible explanations using empirical data seems diffi-

cult. However, I believe that this type of explanation is radically different

from an explanation that relies on subliminal processing. If expectations

alone can trump subliminal processing, then it must be concluded that sub-

liminal processing is less strong than often claimed (Bargh & Morsella,

2008; Hassin, 2013; Lau & Rosenthal, 2011; Rosenthal, 2008).

One could perhaps wonder whether the reported reversed priming effects

are somehow specific to the experimental parameters used here. Why, for

example, have such effects not been extensively reported before? One expla-

nation may be that experimenters have seldom measured discrimination re-

sponses in the same trials as the priming effects, a procedure necessary for

making the comparison. Another explanation is that there are previous re-

ports of reversed (negative) congruency priming (Carr & Dagenbach, 1990;

Goodhew, Visser, Lipp, & Dux, 2011; Kahan, 2000; Wentura & Frings,

2005), but that these effects have been interpreted as subliminal stimuli

sometimes inhibiting rather than enhancing response times to their own se-

mantic category (a “negativity compatibility effect”; Eimer & Schlaghecken,

1998). Perhaps some of these previous reports reflect priming from misper-

ceived rather than subliminal stimuli.

In the study by Goodhew et al. (2011), for example, the reversed congru-

ency effects were found for incorrect discrimination responses (based on the

same analysis as conducted in Studies 3 and 4). The prime stimulus in that

study was identified above chance (72% correct vs. chance 50%), which

indicates that the primes were perceived at least to some degree in some

trials. As such, participants in that study could perhaps simply have misper-

ceived the stimuli when they made incorrect discrimination responses rather

than not perceived the stimuli at all. Those misperceptions could have led to

the reversed priming effects, rather than the reversed priming effects stem-

ming from the subliminal inhibition of the primes’ semantic category.

Note that in Studies 3 and 4, as in most other studies of subliminal prim-

ing, the prime perception response was collected after the target had been

shown. This means that the prime perception responses may be based on

reaction times to the target stimuli, or other (nonconscious) prime target

cues, rather than on prime percepts (i.e., prime responses may not be exclu-

sive to percepts; Reingold & Merikle, 1988). This issue introduces a possible

collider bias (Elwert & Winship, 2014) into the analytical strategy in Studies

3 and 4, as trials were stratified based on prime perception responses. In

other words, there is a possibility of a causal relationship the reverse of that

68

proposed in Studies 3 and 4: target reaction time may influence answers on

the prime perception scale, not vice versa. This threatens the validity of the

conclusions of studies in which prime perception responses (measured after

the effect of interest) are used in a stratified analysis (as in, e.g., Goodhew et

al., 2011; Haase & Fisk, 2015; Lamy, Salti, & Bar-Haim, 2009; Muscarella

et al., 2013).

However, control analyses of catch trials (in which no prime was shown)

did not support the supposition that participants based their prime perception

responses on reaction time (see manuscript). Furthermore, if participants did

base their prime perception responses on reaction time and there were con-

gruency priming effects, this response strategy would lead to a better dis-

crimination sensitivity compared with guessing. However, in Study 1 when

the target was replaced with semantically neutral targets (thus removing

congruency priming effects), prime sensitivity improved rather than deterio-

rated. The empirical data therefore do not seem to support this alternative

explanation.

Validity of zero sensitivity: a reanalysis of Study 4

In many studies, a discrimination or detection sensitivity of zero is interpret-

ed as the stimulus of interest not being perceivable. Many experimenters

suggest that the objective definitions, i.e., (a) and (c), are overly conservative

measures of perception (perhaps not exclusive). Here I take the opposite

position: zero sensitivity is too liberal (not exhaustive). However, what may

at first glance look like an extreme position is actually not so extreme.

Sensitivity is usually measured as the average detection or discrimination

performance across all experimental trials. That a participant cannot on av-

erage detect or discriminate a stimulus (at a block level) is often interpreted

as indicating that the participant cannot perceive the stimulus in any trial.

This is not what an average sensitivity of zero implies, however. An average

sensitivity of zero only implies that to the extent participants have percepts,

these are equally likely to be correct as incorrect. That participants experi-

ence strong correct or incorrect percepts in a few trials is therefore fully

compatible with their having a true sensitivity of zero.

To test this, I performed a reanalysis of Study 4. I first calculated discrim-

ination sensitivity in the bins of trials in which participants used different

prime perception ratings (as in Haase & Fisk, 2015). I could then compare

each participant’s discrimination sensitivity in the full experiment with their

sensitivity in those different bins of trials. The results are shown in Figure

13. If discrimination sensitivity is unrelated to prime perception, then sensi-

tivity should fall on the major diagonal. However, as can be seen, discrimi-

nation sensitivity was strongly related to the reported perceptual state: inde-

pendent of how well participants could discriminate the primes across the

69

Figure 13. In all panels, each symbol represents one participant. The x-axis is the same in all panels and illustrates participants’ block-level discrimination sensitivity (the average over the experiment). The different panels illustrate discrimination sensitivity in the trials in which participants reported not experiencing the primes (left), having some experience of the primes (centre), or having clear experiences of the primes (right). The y-axis is the participants’ sensitivity in those respective bins of trials. For reference, the dotted line is the major diagonal and the solid line is the regression slope. Boxplots of sensitivity for each bin of trials are superimposed (using the same y-axis). The number of participants (N) varies between panels be-cause not all participants had enough trials in each bin for sensitivity to be calculat-ed. The number of trials (n) in each bin varied between participants. The dot size is proportional to the number of trials each participant contributed to each bin.

whole experiment (x-axis), they were much poorer at discriminating the

primes when they reported not perceiving them (left panel). More important-

ly, some participants had zero sensitivity (x-axis) across the whole experi-

ment, but still reported perceiving the primes in a minority of trials. In the

trials in which they reported perceiving the primes, they were better at dis-

criminating the primes (centre and right panels). As such, even if mean dis-

crimination sensitivity is zero, participants can still experience strong per-

cepts in a few trials (Haase & Fisk, 2015).

If semantic priming is dependent on perception, participants with zero

sensitivity may still be primed in a minority of trials in which they (correctly

or incorrectly) perceive the primes. To test this, I divided the sample in

Study 4 into two groups based on their discrimination sensitivity in relation

to the 95% upper limit of chance performance.9 Let us call the group of par-

ticipants below this limit participants with poor sensitivity and the partici-

pants above this limit, participants with better sensitivity. These two groups’

9 As participants under this limit are often claimed to be “subliminal,” a median split leads to

the same result.

70

Figure 14. A: Inter-individual correlation between overall discrimination sensitivity and overall priming effects (for the 25-ms ISI primes). The dotted vertical line indi-cates the 95% upper limit of the binomial distribution based on 240 trials (d' = 0.27). The group with poor sensitivity is shown by the black symbols to the left of this line, and the group with better sensitivity is shown by the white symbols to the right of this line. B and C: Proportions of trials in each perception rating bin for the 25-ms (red) and 100-ms (blue) ISI primes for the group with poor sensitivity (B) and the group with better sensitivity (C). D and E: Congruency priming for correct discrim-ination responses in each perception rating bin for the 25-ms (red) and 100-ms (blue) ISI primes for the group with poor sensitivity (D) and the group with better sensitivi-ty (E). Note that the group with better sensitivity seldom reported “no percept” for the 100-ms ISI primes, which explains the noise in this bin of trials. In panel D and E, participants with congruency priming two standard deviations from the mean effects in each bin of trials have been removed.

71

overall discrimination sensitivities and priming effects are shown in panel A

in Figure 14 and as can be seen were strongly correlated. (The main finding

in Study 4 was that congruency priming was related to discrimination re-

sponse. I will ignore this here and only focus on trials in which participants

made correct discrimination responses.)

The performance of participants with poor sensitivity is shown in the left

panels of Figure 14. These participants were poor at discriminating the 25-

ms ISI primes (mean block-level d' = 0.07) and seldom reported perceiving

the 25-ms ISI primes (panel B, red boxes). These participants were also

overall weakly primed by the 25-ms ISI primes (across detection ratings, a

mean effect of 0.86 ms). One could therefore claim that the stimuli were

“subliminal” for these participants and did not elicit priming effects.

The performance of participants with better sensitivity is shown in the

right panels of Figure 14. These participants were better at discriminating the

25-ms ISI primes (mean block-level d' = 1.12) and often reported perceiving

the 25-ms ISI primes (panel C, red boxes). These participants were also

overall primed by the 25-ms ISI primes (across detection ratings, a mean

effect of 39.91 ms).

Based on average effects, the stimuli could be claimed to be “subliminal”

and not eliciting priming effects in the first group and claimed to be “supra-

liminal” and eliciting priming effects in the second group. However, if we go

beyond the average effects, we can see that this is not an accurate description

of the effects. If we compare the red boxes in panels D and E, both groups

had priming effects of similar magnitude in the trials in which they reported

perceiving the 25-ms ISI prime. What seems to matter for the priming effect

is the instantaneous perceptual state, not overall sensitivity. The inter-

individual correlation between priming and sensitivity in panel A seems to

be driven by a difference in the proportion of different perceptual states, not

possible perceptual states; zero discrimination sensitivity allows for the pos-

sibility of strong perceptual states.

The same result is apparent if we focus on stimuli rather than participants:

Compare how the participants with poor sensitivity responded to the 25-ms

compared with the 100-ms ISI primes also included in the experiment. These

participants could discriminate the 100-ms ISI primes (block-level mean d' =

1.95 vs. 0.07 for the 25-ms ISI primes) and on a mean level were primed by

them (block-level mean effect of 58.28 ms vs. 0.86 ms for the 25-ms ISI

primes). Again, based on average effects, the 25-ms ISI primes could be

labeled “subliminal” and unable to elicit priming effects and the 100-ms ISI

primes could be labeled “supraliminal” and able to elicit priming effects.

However, this description based on average effects is simply not accurate. If

we compare the red and blue boxes in panel C, we can see that both stimuli

elicited priming effects of similar magnitude in the trials in which they elic-

72

ited strong percepts. What seems to matter for the priming effect is the in-

stantaneous perceptual state, not overall stimulus strength.

Discussion

A key feature of many investigations of subliminal processing is the con-

trasting of stimuli claimed not to elicit percepts and stimuli that do elicit

percepts. Often the claim regarding these two stimuli types are based on

objective performance (i.e., sensitivity). However, sensitivity measures per-

ception as an average over many trials, including trial-to-trial variability in

sensory responses due to bottom up and top down variability (Dean, 1981;

Mathewson et al., 2009; Scholvinck et al., 2015; Shadlen & Newsome,

1998).

This is explicit in the SDT model of performance: there is only a proba-

bilistic relationship between stimulus strength and sensory states. Given a

constant stimulus strength, the probability function between stimulation and

sensory states will vary between participants with different energy thresh-

olds. However, the possible sensory states will not vary between participants

(remember that the probability function moves asymptotically toward zero

magnitude). Similarly, possible sensory states will not vary between stimuli.

In other words, participants with poor sensitivity should (in principle) in a

few trials experience the same sensory states (or sensory magnitudes) as

participants with better sensitivity do in most trials. Similarly, weak stimuli

will in a few trials elicit the same sensory states, as strong stimuli do in most

trials.

In general, a specific signal strength or specific sensitivity across an ex-

periment cannot be said to produce archetypical responses; there is always a

probability that strong perceptual states will be produced. Thus, sensitivity

measured at an experimental level (or subjective reports in a debriefing)

needs to be supplemented with trial-by-trial subjective reports of perceptual

states, in order for small fluctuations in perception to be measured (Haase &

Fisk, 2015; Sandberg et al., 2010; Van den Bussche et al., 2013). However,

that subjective reports (or perception ratings) can be used to sort instantane-

ous perceptual states does not imply that there is a one-to-one mapping be-

tween the two (Schmidt, 2015). Because the detection situation in subliminal

processing experiments is so difficult, it seems natural that a few fragmen-

tary percepts might fall below a participant’s detection criterion. As such,

“no percept” reports need not indicate only sensory states of zero magnitude

(if there is such a thing; Swets, 1961).

73

11. General discussion

“A scientific career is peculiar in some ways. Its raison d’être is the increase of natural knowledge. Occasionally, therefore, an increase in natural knowledge occurs. But this is tactless, and feelings are hurt. For in some de-gree it is inevitable that views previously expounded are shown to be obsolete or false.” Sir Ronald Fisher (1890–1962)

There have been periods of acceptance and rejection in the history of sublim-

inal semantic processing (Eriksen, 1960; Holender, 1986; Kouider & Dehae-

ne, 2007). Today the pendulum seems to have swung to acceptance (Ansorge

et al., 2014; Dehaene & Changeux, 2011; Kouider & Dehaene, 2007; Van

den Bussche, Van den Noortgate, et al., 2009). Here I consider whether this

shift in zeitgeist also corresponds to a shift in evidential strength for the phe-

nomenon.

Has perception been ruled out?

The validity of measures of perception is critical in studies claiming to have

demonstrated subliminal processing, because this claim requires that the

stimulus of interest truly be unperceivable. The overall aim of this thesis was

to determine to what extent modern studies have indeed ruled out residual

percepts as potential alternative explanations of claimed “subliminal” ef-

fects.

That stimuli subjectively reported as not perceived (i.e., subliminal ac-

cording to subjective definitions) can influence behavior (e.g., lead to better-

than-chance objective performance; Macmillan, 1986; Miller, 1939) seems

well substantiated. However, subjective measures may always be questioned

as not being exhaustive measures of perception (Chapter 3). In comparison,

objective definitions of “subliminal” stimuli (i.e., stimuli that cannot be de-

tected or discriminated better than chance) are often claimed to be exhaus-

tive measures of perception. As such, behavioral effects from stimuli that

fulfill this definition would be much more convincing demonstrations of

subliminal processing. This thesis mainly concerned itself with claims of the

latter kind and has addressed three potential issues when using objective

measures of perception: (1) the validity of the measures of performance,

74

(2) the validity of the statistics used to demonstrate lack of perception, and

(3) the validity of the operationalization of perception as average objective

performance.

To exclude residual percepts—i.e., to be valid indicators of lack of per-

ception—measurements of perception must be made under optimal retrieval

conditions and, when possible, immediately after the stimuli of interest has

been presented (Newell & Shanks, 2014; Reingold & Merikle, 1988). This is

not the case in many experiments using objective performance as measures

of perception, however; instead target stimuli are usually presented between

the prime and the prime responses. In Study 1, it was found that interference

effects from such stimuli may lead to the underestimation of objective per-

formance. This, together with several other potential sources of underestima-

tion previously reported (Lin & Murray, 2014; Pratte & Rouder, 2009; Ver-

meiren & Cleeremans, 2012), calls into question the validity of such

measures as indicators of lack of perception.

To claim a lack of perception based on objective measures, chance per-

formance must be statistically established. In Study 2 it was demonstrated

that conventional statistical procedures used to support this claim have a

high false-positive rate (even if we assume no systematic underestimation).

The evidential strength for lack of perception in studies using these statistical

procedures was evaluated in the quantitative review in Chapter 8. The con-

clusion was that questionable statistical procedures have led to a situation in

which many claimed demonstrations of chance-level performance actually

do not have data that are conclusive with regard to their critical hypothesis,

i.e., that objective performance was at chance levels.

Finally, objective performance (i.e., chance-level detection or discrimina-

tion sensitivity) measured across an experiment was shown in the reanalysis

of Study 4 not to be a valid operationalization of lack of perception: occa-

sional, even clear, percepts of stimuli (as indicated by subjective reports) are

fully compatible with an estimated overall inability to discriminate stimuli.

One way to test the effect of possible residual percepts was suggested in

Studies 3 and 4 based on the idea of reversed priming effects when stimuli

are misperceived. Indeed, under conditions that fulfilled conventional crite-

ria of being subliminal (i.e., zero discrimination ability and subjective re-

ports of no percepts), reversed priming effects remained. Combined, these

considerations indicate that occasional percepts have not been ruled out as

potential explanations in many claimed demonstrations of subliminal pro-

cessing (given the experimenters’ own definitions of subliminal).

75

Do residual percepts matter?

One could argue that if subliminal processing effects are substantial, a few

fragmentary percepts would not greatly affect the conclusion of subliminal

processing. (To be clear, the effects, e.g., semantic priming, are not at stake,

only their “subliminal” nature.) However, claimed subliminal effects are

often much smaller than the corresponding supraliminal effects (Van den

Bussche, Van den Noortgate, et al., 2009). If semantic priming is an auto-

matic process (Collins & Loftus, 1975; Masson, 1995) that can be driven by

subliminal information, it is not clear why the priming effects should be so

much smaller under subliminal than supraliminal conditions. On the other

hand, small priming effects under “subliminal” conditions are exactly what

one would expect if the effects were actually driven by residual percepts.

As the statistical approaches used in many studies allow for a few indi-

viduals with above zero sensitivity to be included in the sample, these partic-

ipants may well be the only ones who drive the “subliminal” effects (Chapter

9). Furthermore, the operationalizations and measures of perception used in

many studies allow for a few trials with percepts to be included in the sam-

ple and these may very well be the only ones that drive the effects (reanaly-

sis of study 4; Chapter 10). In fact, occasional and fragmentary percepts may

actually be more strongly related to the effects of interest than the physical

stimuli, as suggested by the reversed priming effects found in Studies 3 and

4 (Chapters 9 and 10). Because it is not unreasonable that a few fragmentary

percepts may explain the often small (potentially) subliminal effects, it does

indeed matter whether or not occasional weak percepts have truly been ex-

cluded.

In this thesis I have tried to evaluate experiments based on the experi-

menters’ own (strict, objective) operationalizations of subliminal stimuli.

Several commentators on Holender’s (1986) critical review of the field stat-

ed that whether or not objective performance was exactly at chance in their

experiments missed the point: the point was that the participants could not

perceive the stimuli! I think many can agree that chance objective perfor-

mance is perhaps not a perfect operationalization of our naïve idea of per-

ception. However, it is problematic if researchers choose to operationalize

perception as objective performance but then do not adhere to it, as in this

case they have effectively not operationalized perception at all.

Here, on the shoulders of other researchers, I have made some methodo-

logical suggestions, mainly on how to measure and demonstrate chance-level

performance. These suggestions do not, however, bring us much closer to

our naïve idea of perception. Anyone is free to argue that the operationaliza-

tions discussed here are not valid with regard to their idea of perception.

However, if subliminal processing is to be a question of empirical science,

perception must be operationalized in some way, however crudely. In gen-

eral, because perception is such an elusive concept and can be operational-

76

ized in a range of ways, operationalizations that make the possibility of sub-

liminal processing more or less surprising, experimenters in this field should

try to abandon the habit of reporting and collapsing effects found using the

various definitions of “subliminal.” Instead, effects should be related specif-

ically to the definition used. Combining results found under such widely

different conditions does not bring us closer to our naïve idea of perception,

but only obfuscates the data.

Future directions

For researchers explicitly interested in processing at the objective threshold,

it is important that the challenges summarized and presented herein be ad-

dressed. I have provided and summarized some suggestions that may allevi-

ate the problems: For sensitivity measures to be valid (i.e., not suffer from

underestimations), primes and targets should, to the extent possible, be visu-

ally or semantically different so that the target stimuli do not interfere with

prime responses (Study 1; Vermeiren & Cleeremans, 2012), and easier and

more difficult stimuli settings should inter-mixed to ensure task motivation

(Pratte & Rouder, 2009) and completion (Lin & Murray, 2014). It is also

important that block-level measures of perception (often average objective

performance) be supplemented with trial-to-trial subjective reports of per-

ception so that fluctuations in instantaneous perceptual states are taken into

account (reanalysis of Study 4; Haase & Fisk, 2015).

If experimenters want to support chance-level performance (i.e., the null

hypothesis) at a group level, a Bayesian statistical approach, less prone to

false positives and able to support the hypothesis of interest, should be used

to evaluate the data (Study 2; Dienes, 2015). If experimenters want to use a

regression approach to take inter-individual differences in energy thresholds

into account, they should try to reduce the measurement error of sensitivity

and increase the inter-individual range of sensitivities (Studies 2 and 3).

It is also important that the hypothesis of subliminal processing be formu-

lated in such a way that it is possible to generate convincing positive and

negative evidence. After six decades of research (Eriksen, 1960; Holender,

1986; Kouider & Dehaene, 2007; Newell & Shanks, 2014), little progress

has been made in evaluating the hypothesis of subliminal processing: Re-

ports of subliminal processing are not convincing to skeptics because wheth-

er or not residual percepts have been ruled out is often an open question.

Reports of lack of subliminal processing are not convincing to believers

because other stimulus settings might have produced subliminal processing.

As such, there is a risk that the hypothesis may always linger as an unverifi-

able and unfalsifiable possibility.

I have suggested one way to reformulate the hypothesis so that it can es-

cape this fate: does the outcome of a cognitive process correspond to the

77

perceptual decision or not, ignoring where the perceptual state falls in rela-

tion to the participant’s detection criterion. Potential subliminal and noncon-

scious processing has had a large impact on theories of cognition and con-

sciousness (Newell & Shanks, 2014). Perhaps some of the impact should be

shifted back to perception: People’s perceptions do not necessarily corre-

spond to their physical environment and, in these situations, people’s percep-

tions may actually more strongly impact cognition than what people cannot

see.

Does subliminal cognition matter?

The importance of effects claimed as “subliminal” truly being free from

residual percepts or not depends on the stimuli and context the researchers

are actually interested in. Some researchers are explicitly interested in sub-

liminal stimuli (Ansorge et al., 2014; Carter et al., 2011; Dehaene &

Changeux, 2011; Finkbeiner, 2011; Haase & Fisk, 2015; Sandberg et al.,

2010; Snodgrass et al., 2004b), and for them, whether or not lack of percep-

tion can be demonstrated is the heart of the matter. But I believe many other

researchers publishing about apparently “subliminal effects” are actually not

interested in subliminal stimuli as such.

I think that the scenario some researchers (e.g., Aranyi et al., 2014;

Cetnarski, Betella, Prins, Kouider, & Verschure, 2014) are actually interest-

ed in is something like the following: An ad for product x is placed on the

handles of a shopping cart. A customer pushing that cart does not attend the

ad but the ad nevertheless influences the customer to buy product x. The ad

is clearly not subliminal; if the customer were to attend it, she would be able

to detect and discriminate it. But whether or not the ad is subliminal is not

the point; the point is that the ad was not attended.

Behavioral effects stemming from unattended stimuli are clearly interest-

ing, but such effects are, in my terms, the result of nonconscious processing

of supraliminal stimuli, not the result of subliminal processing. The problem

is that in the absence of a better operationalization, some researchers seem to

use chance-level detection or discrimination performance when the stimulus

is attended as an (invalid) operationalization of inattention. This may explain

why some researchers seem to measure these strict operationalizations in

such lenient ways (Holender, 1986).

One possible explanation for why researchers use this (invalid) operation-

alization is that inattention is difficult to operationalize. Another possible

explanation is provided by Vadillo et al. (2016): whether or not studies in-

vestigating the same phenomenon mention “awareness,” “unconscious,” and

similar terms is correlated with the impact factor of the journal in which they

are published (Figure 7, p. 97). I suggest that the processing of unattended

stimuli is simply a lot less surprising than the processing of subliminal stim-

78

uli, and that strategic conduct in psychology research is to describe effects as

more surprising than they actually are.

There are several reasons why this is problematic for the research field as

a whole: (1) The theoretical impact of results suggesting the cognitive pro-

cessing of unattended stimuli is completely different from that of results

suggesting the cognitive processing of subliminal stimuli. (2) As the terms

become confused, the support for subliminal processing becomes inflated as

claims that go beyond the data become widespread. (3) Interesting behavior-

al effects from unattended stimuli may never be reported because the authors

could not demonstrate that their stimuli were subliminal (which was never

the point in any case).

Limitations

A weakness when prime perception responses are collected after the target

has been presented (as in the empirical studies here) is that (nonconscious)

information about the prime–target relationship (e.g., reaction time) could

drive prime perception responses—i.e., prime perception responses are not

necessarily exclusive to prime percepts (Reingold & Merikle, 1988). That

participants’ base their prime perception responses on such strategies did not

gain support from the data in Study 4 (or in Study 1). However, it cannot be

completely ruled out and, as such, it threatens the validity of the conclusion

that prime percepts influence priming (rather than priming influencing prime

perception responses).

Note that this problem does not only affect (objective) prime discrimina-

tion responses collected after the target stimuli: subjective ratings, for exam-

ple, confidence ratings, may similarly be based on information other than

percepts. It is not clear to me how this problem should be controlled for.

Kahan (2000), for example, tried to do this by collecting prime-perception

responses before the targets were presented. Although this procedure elimi-

nates one problem, it instead introduces the problem that participants may be

primed (or reverse primed, as in Kahan’s study) by their prime responses

and not their prime percepts. Future investigations are needed that more

clearly disentangle the causal relationship between potential priming effects

and prime perception responses.

Another weakness of the empirical studies is that the extent to which the

congruency priming effects reflect semantic processing is (at least to me)

somewhat unclear. Perhaps the congruency effects are not driven by word

meaning but rather only by orthographical information (i.e., individual let-

ters). It should be noted, however, that in the empirical studies, orthograph-

ical information alone was not related to target responses but rather to se-

mantic categories (Abrams et al., 2002). To the extent that only orthograph-

ical information was extracted in individual trials, this information must then

79

be related to semantic categories. Future studies that more clearly measure

either a spreading of semantic activation or the motor priming mechanism

behind congruency priming should test whether both or only one priming

effect is related to the prime discrimination response (reversed priming ef-

fects).

There are two clear problems associated with using reaction time as a

measure of whether or not a process occurred (e.g., semantic processing).

One problem is that perceptual states, as, for example, in study 4, are meas-

ured on a trial-by-trial level, but the indirect effect, congruency priming, is

measured as the difference between two groups of trials (i.e., incongruent

minus congruent). This means that measurement error in the indirect meas-

ure stems from two groups of sometimes only a few trials (as in the binning

procedure in Study 4) and can thus be compounded. It would clearly be bet-

ter to measure the indirect effects as absolute values in specific trials. Again,

how this should be done is not clear to me.

Another problem is that reaction time provides only a single data point in

each trial and is not informative regarding how behavior unfolds within that

time. Xiao and Yamauchi (2015) provide a promising improvement in this

regard. In their study the indirect measure is mouse movement trajectories

toward target locations on a screen. Semantic congruency effects are record-

ed by measuring the influence of masked primes on those mouse trajectories.

In this way, not only is the end point of the behavior recorded (as in reaction

time) but also the dynamics of that behavior (e.g., partial movement toward

unintended targets).

Concluding remarks

A wealth of studies (included in the reviews, Ansorge et al., 2014; Stanislas

Dehaene & Changeux, 2011; Kouider & Dehaene, 2007; Van den Bussche,

Van den Noortgate, et al., 2009, and in the quantitative review in Chapter 8)

suggests that stimuli at the objective threshold can influence cognition and

behavior in interesting ways. However, the many methodological and statis-

tical problems presented and reviewed in this thesis have led to a situation in

which such effects are easy to “replicate” without having data that actually

supports that the effects are subliminal. My conclusion is therefore that it is

still unclear what evidence there is for the cognitive processing of objective-

ly subliminal stimuli.

However, nothing in this thesis proves that subliminal cognition is not a

real phenomenon. The empirical studies presented here (Studies 1, 3, and 4)

that does not find subliminal semantic processing does not demonstrate that

subliminal semantic processing cannot happen. The discussions of methodo-

logical problems presented here (Studies 1, 2, and 4) only suggest potential

alternative explanations for previous results, but do not invalidate them.

80

Note however, that many of the methodological problems enumerated in

this thesis have been raised previously (e.g., Eriksen, 1960; Holender, 1986;

Macmillan, 1986; Merikle, 1982). Despite this, experiments suffering from

these problems have now been taken as demonstrating subliminal pro-

cessing. I believe a paradigm shift is needed in which we raise the bar as to

what counts as evidence for the phenomenon. Perhaps the hypothesis also

needs to be re-formulated so as to allow for convincing positive or negative

evidence.

An impression I have from being a PhD student is that, as a group, we

psychologists are an intolerant bunch. We often investigate difficult and

deep questions about higher-order cognitive functions and consciousness

without having settled fundamental and prerequisite questions. I believe that

cognition without perception is today considered a demonstrated phenome-

non without the question of how to measure it first having been settled. In

this thesis I have provided methodological and statistical suggestions that

could be used in future studies to properly test the limits of processing with-

out perception.

81

Summary in Swedish

Små detaljer i vår omgivning kan påverka oss på olika vis: Till exempel kan

en affisch vi knappt uppmärksammar och snabbt går förbi väcka minnen hos

oss eller föra in en konversation på andra banor. Men kan vi bli påverkade av

saker som visas så svagt eller så snabbt att vi inte alls kan se dem? Kan vi bli

påverkade av subliminala stimuli? James Vicary är nog en av de som är

skyldiga till att väcka allmänhetens nyfikenhet och rädsla inför frågan. Han

påstod sig ha lagt in reklambudskap i biofilmer som visades så snabbt att

publiken inte kunde märka dem och att reklamen ändå ökade försäljningen

av popcorn och cola med 58%. Påståendet väckte stor uppmärksamhet men

efter att många privata och akademiska forskningsprojekt inte kunde åter-

skapa resultatet erkände Mr Vicary att det hela var en bluff.

I Vicarys spår har frågan om ifall subliminala stimuli kan påverka oss va-

rit en av psykologins mest flitigt undersökta. Saker som vi inte kan se kan

självklart ändå processas i nervsystemet – i näthinnan och i tidiga visuella

hjärnområden – men till vilken gräns? Kan till exempel skriven text tolkas

av nervsystemet och påverka beteenden trots att vi inte upplever texten? Det

empiriska stödet för att subliminala stimuli kan påverka oss har periodiskt

accepterats och avfärdats under de senaste 60 åren. Idag verkar många psy-

kologer anse det väletablerat att stimuli vi inte kan se kan påverka oss och

nya fynd tycks tyda på att de till och med kan påverka avancerade kognitiva

beslut (t.ex. politisk ståndpunkt). I den här avhandlingen undersöker jag om

den nuvarande positiva attityden gentemot fenomenet också har stöd i star-

kare ny empiri.

Avhandlingen består av tre empiriska studier med mer än 250 försöksper-

soner, en simuleringsstudie, och en kvantitativ litteraturöversikt. Dessa arbe-

ten uppmärksammar validitetsproblem med mätningar och operational-

iseringar av subliminala stimuli, problem med statistiska test och resone-

mang som använts för att styrka att stimuli är subliminala, samt teoretiska

problem i undersökningsproceduren. Dessa problem innebär att det ännu inte

är uteslutet att effekter som påstås vara subliminala faktiskt drivs av enstaka

eller svaga upplevelser. Ny empirisk data presenteras också som tyder på att

till och med i betingelser som kallas ”subliminala” så kan svaga upplevelser

och föreställningar kanske påverka oss mer än det objektiva budskapet.

Min slutsats är att de många metodologiska och statistiska problemen har

lett till att det är lätt att ”replikera” subliminala effekter utan att ha data som

faktiskt stödjer att effekten är subliminal och att det därför fortfarande är

82

oklart till vilken grad det finns stöd för subliminal påverkan. Men jag sam-

manfattar och presenterar också metodologiska, statistiska och teoretiska

förbättringar som framtida studier som strävar efter att mer robust bevisa

subliminal påverkan skulle dra nytta av.

83

Acknowledgements

I am extremely grateful to Mats E. Nilsson, who at age twelve also was in-

terested in subliminal processes and then grew up. Thank you for believing

in me enough to let me change the topic of my dissertation one year prior to

the deadline and for always having your door open for questions (if I was not

your highest ranked time-thief in the department questionnaire, you're obvi-

ously lying).

My gratitude also goes to Joakim Westerlund — who truly values scien-

tific rigor —, Pehr Granqvist — who hired me to program a subliminal ex-

periment (in which I perceived the prime in a third of the trials) that inspired

this thesis —, and Stefan Wiens who first introduced me to experimental

psychology and lend me his equipment for the data collections in this thesis.

Big thanks to all my dear friends and colleagues at Gösta Ekmans Lab

and the department at large without whom science truly would be a bore.

I also want to thank my parents and siblings for allowing (or forcing) me

to become independent and critical in a family with lots of different attitudes

and personalities.

Finally, thank you Nathalie. Thank you for your endless well of patience

and inspiring attitude to life. Except for Mats, no one has been the discussion

Object for my ideas in this project as much as you. Barring none, you have

been the biggest support against my anxiety during this project. (I think all

PhD students suffer some existential angst and as I score highly on neuroti-

cism to begin with you can imagine what her last five years have been like.)

I'm sorry for being a pain. I love you.

84

Appendix: Review references

This appendix includes all 100 studies reviewed in Chapter 8. References marked

with an asterisk were included in the quantitative analyses.

Alkozei, A., & Killgore, W. D. S. (2015). Emotional intelligence is associated with

reduced insula responses to masked angry faces: NeuroReport, 26(10), 567–571. http://doi.org/10.1097/WNR.0000000000000389

* Aranyi, G., Kouider, S., Lindsay, A., Prins, H., Ahmed, I., Jacucci, G., … Cavaz-za, M. (2014). Subliminal Cueing of Selection Behavior in a Virtual Environ-ment. Presence: Teleoperators and Virtual Environments, 23(1), 33–50. http://doi.org/10.1162/PRES_a_00167

* Axelrod, V., Bar, M., Rees, G., & Yovel, G. (2015). Neural Correlates of Sublimi-nal Language Processing. Cerebral Cortex, 25(8), 2160–2169. http://doi.org/10.1093/cercor/bhu022

* Axelrod, V., & Rees, G. (2014). Conscious awareness is required for holistic face processing. Consciousness and Cognition, 27, 233–245. http://doi.org/10.1016/j.concog.2014.05.004

Bailey, P. E., Slessor, G., Rendell, P. G., Bennetts, R. J., Campbell, A., & Ruffman, T. (2014). Age differences in conscious versus subconscious social perception: The influence of face age and valence on gaze following. Psychology and Ag-ing, 29(3), 491–502. http://doi.org/10.1037/a0036249

Bareither, I., Chaumon, M., Bernasconi, F., Villringer, A., & Busch, N. A. (2014). Invisible visual stimuli elicit increases in alpha-band power. Journal of Neuro-physiology, 112(5), 1082–1090. http://doi.org/10.1152/jn.00550.2013

* Bijleveld, E., Custers, R., Van der Stigchel, S., Aarts, H., Pas, P., & Vink, M. (2014). Distinct neural responses to conscious versus unconscious monetary re-ward cues: Conscious vs. Unconscious Monetary Reward Cues. Human Brain Mapping, 35(11), 5578–5586. http://doi.org/10.1002/hbm.22571

Blanchfield, A., Hardy, J., & Marcora, S. (2014). Non-conscious visual cues related to affect and action alter perception of effort and endurance performance. Fron-tiers in Human Neuroscience, 8. http://doi.org/10.3389/fnhum.2014.00967

Boy, F., & Sumner, P. (2014). Visibility predicts priming within but not between people: A cautionary tale for studies of cognitive individual differences. Journal of Experimental Psychology: General, 143(3), 1011–1025. http://doi.org/10.1037/a0034881

Bustin, G. M., Jones, D. N., Hansenne, M., & Quoidbach, J. (2015). Who does Red Bull give wings to? Sensation seeking moderates sensitivity to subliminal adver-tisement. Frontiers in Psychology, 6. http://doi.org/10.3389/fpsyg.2015.00825

Camors, D., Jouffrais, C., Cottereau, B. R., & Durand, J. B. (2015). Allocentric coding: Spatial range and combination rules. Vision Research, 109, 87–98. http://doi.org/10.1016/j.visres.2015.02.018

* Cetnarski, R., Betella, A., Prins, H., Kouider, S., & Verschure, P. F. M. J. (2014). Subliminal Response Priming in Mixed Reality: The Ecological Validity of a

85

Classic Paradigm of Perception. Presence: Teleoperators and Virtual Environ-ments, 23(1), 1–17. http://doi.org/10.1162/PRES_a_00171

* Chaillou, A.-C., Giersch, A., Bonnefond, A., Custers, R., & Capa, R. L. (2015). Influence of positive subliminal and supraliminal affective cues on goal pursuit in schizophrenia. Schizophrenia Research, 161(2-3), 291–298. http://doi.org/10.1016/j.schres.2014.10.052

Chambon, V., Moore, J. W., & Haggard, P. (2015). TMS stimulation over the infe-rior parietal cortex disrupts prospective sense of agency. Brain Structure and Function, 220(6), 3627–3639. http://doi.org/10.1007/s00429-014-0878-6

* Chiesa, P. A., Liuzza, M. T., Acciarino, A., & Aglioti, S. M. (2015). Subliminal perception of others’ physical pain and pleasure. Experimental Brain Research, 233(8), 2373–2382. http://doi.org/10.1007/s00221-015-4307-8

Chow, H. M., & Tseng, C. (2015). Invisible collinear structures impair search. Con-sciousness and Cognition, 31, 46–59. http://doi.org/10.1016/j.concog.2014.10.004

* Czekala, C., Mauguière, F., Mazza, S., Jackson, P. L., & Frot, M. (2015). My Brain Reads Pain in Your Face, Before Knowing Your Gender. The Journal of Pain, 16(12), 1342–1352. http://doi.org/10.1016/j.jpain.2015.09.006

de Haan, B., Stoll, T., & Karnath, H.-O. (2015). Early sensory processing in right hemispheric stroke patients with and without extinction. Neuropsychologia, 73, 141–150. http://doi.org/10.1016/j.neuropsychologia.2015.05.011

* Diao, L., Ding, C., Qi, S., Zeng, Q., Huang, B., Xu, M., … Yang, D. (2014). Influ-ence of Supraliminal Reward Information on Unconsciously Triggered Re-sponse Inhibition. PLoS ONE, 9(9), e108530. http://doi.org/10.1371/journal.pone.0108530

* Doi, H., & Shinohara, K. (2015). Unconscious Presentation of Fearful Face Modu-lates Electrophysiological Responses to Emotional Prosody. Cerebral Cortex, 25(3), 817–832. http://doi.org/10.1093/cercor/bht282

Era, V., Candidi, M., & Aglioti, S. M. (2015). Subliminal presentation of emotional-ly negative vs positive primes increases the perceived beauty of target stimuli. Experimental Brain Research, 233(11), 3271–3281. http://doi.org/10.1007/s00221-015-4395-5

* Faivre, N., Mudrik, L., Schwartz, N., & Koch, C. (2014). Multisensory Integration in Complete Unawareness Evidence From Audiovisual Congruency Priming. Psychological Science, 25(11), 2006–2016.

Farooqui, A. A., & Manly, T. (2015). Anticipatory Control Through Associative Learning of Subliminal Relations Invisible May Be Better Than Visible. Psy-chological Science, 0956797614564191.

* Freeman, J. B., Stolier, R. M., Ingbretsen, Z. A., & Hehman, E. A. (2014). Amyg-dala Responsivity to High-Level Social Information from Unseen Faces. Jour-nal of Neuroscience, 34(32), 10573–10581. http://doi.org/10.1523/JNEUROSCI.5063-13.2014

Greenlees, I., Figgins, S., & Kearney, P. (2014). Can Achievement Goals be Primed in Competitive Tasks? Journal of Human Kinetics, 40(1). http://doi.org/10.2478/hukin-2014-0026

* Grotegerd, D., Stuhrmann, A., Kugel, H., Schmidt, S., Redlich, R., Zwanzger, P., … Dannlowski, U. (2014). Amygdala excitability to subliminally presented emotional faces distinguishes unipolar and bipolar depression: An fMRI and pattern classification study: An fMRI and Pattern Classification Study. Human Brain Mapping, 35(7), 2995–3007. http://doi.org/10.1002/hbm.22380

* Hasegawa, K., & Takahashi, S. (2014). The role of visual awareness for conflict adaptation in the masked priming task: comparing block-wise adaptation with

86

trial-by-trial adaptation. Frontiers in Psychology, 5. http://doi.org/10.3389/fpsyg.2014.01347

Hesselmann, G., Darcy, N., Sterzer, P., & Knops, A. (2015). Exploring the boundary conditions of unconscious numerical priming effects with continuous flash sup-pression. Consciousness and Cognition, 31, 60–72. http://doi.org/10.1016/j.concog.2014.10.009

* Hoffmann, M., Mothes-Lasch, M., Miltner, W. H. R., & Straube, T. (2015). Brain activation to briefly presented emotional words: Effects of stimulus awareness: Awareness, Amygdala, and Words. Human Brain Mapping, 36(2), 655–665. http://doi.org/10.1002/hbm.22654

Huang, T.-H., Yeh, S.-L., Yang, Y.-H., Liao, H.-I., Tsai, Y.-Y., Chang, P.-J., & Chen, H. H. (2015). Method and experiments of subliminal cueing for real-world images. Multimedia Tools and Applications, 74(22), 10111–10135. http://doi.org/10.1007/s11042-015-2804-1

* Hüttermann, S., Bock, O., & Memmert, D. (2014). Subliminal primes for global or local processing influence judgments of vehicular traffic. Consciousness and Cognition, 29, 230–234. http://doi.org/10.1016/j.concog.2014.08.007

Hu, X., & Kaplan, S. (2015). The Effects of Unconsciously Derived Affect on Task Satisfaction and Performance. Journal of Business and Psychology, 30(1), 119–135. http://doi.org/10.1007/s10869-013-9331-8

* Ito, A., Abe, N., Kawachi, Y., Kawasaki, I., Ueno, A., Yoshida, K., … Fujii, T. (2015). Distinct neural correlates of the preference-related valuation of supra-liminally and subliminally presented faces: Preference-related Valuation of Fac-es. Human Brain Mapping, 36(8), 2865–2877. http://doi.org/10.1002/hbm.22813

Jensen, K. B., Kaptchuk, T. J., Chen, X., Kirsch, I., Ingvar, M., Gollub, R. L., & Kong, J. (2015). A Neural Mechanism for Nonconscious Activation of Condi-tioned Placebo and Nocebo Responses. Cerebral Cortex, 25(10), 3903–3910. http://doi.org/10.1093/cercor/bhu275

* Jensen, K., Kirsch, I., Odmalm, S., Kaptchuk, T. J., & Ingvar, M. (2015). Classical conditioning of analgesic and hyperalgesic pain responses without conscious awareness. Proceedings of the National Academy of Sciences, 112(25), 7863–7867. http://doi.org/10.1073/pnas.1504567112

Jessen, S., & Grossmann, T. (2014). Unconscious discrimination of social cues from eye whites in infants. Proceedings of the National Academy of Sciences, 111(45), 16208–16213.

Jessen, S., & Grossmann, T. (2015). Neural signatures of conscious and unconscious emotional face processing in human infants. Cortex, 64, 260–270. http://doi.org/10.1016/j.cortex.2014.11.007

Jiménez-Ortega, L., García-Milla, M., Fondevila, S., Casado, P., Hernández-Gutiérrez, D., & Martín-Loeches, M. (2014). Automaticity of higher cognitive functions: Neurophysiological evidence for unconscious syntactic processing of masked words. Biological Psychology, 103, 83–91. http://doi.org/10.1016/j.biopsycho.2014.08.011

* Jusyte, A., & Schönenberg, M. (2014). Subliminal cues bias perception of facial affect in patients with social phobia: evidence for enhanced unconscious threat processing. Frontiers in Human Neuroscience, 8. http://doi.org/10.3389/fnhum.2014.00580

Kagerer, F. A., Viswanathan, P., Contreras-Vidal, J. L., & Whitall, J. (2014). Audi-tory–motor integration of subliminal phase shifts in tapping: better than auditory discrimination would predict. Experimental Brain Research, 232(4), 1207–1218. http://doi.org/10.1007/s00221-014-3837-9

87

Kawakami, N., & Yoshida, F. (2015a). How do implicit effects of subliminal mere exposure become explicit? Mediating effects of social interaction. Social Influ-ence, 10(1), 43–54. http://doi.org/10.1080/15534510.2014.901245

* Kawakami, N., & Yoshida, F. (2015b). Perceiving a story outside of conscious awareness: When we infer narrative attributes from subliminal sequential stimu-li. Consciousness and Cognition, 33, 53–66. http://doi.org/10.1016/j.concog.2014.12.001

* Kido, K., & Makioka, S. (2015). Serial order learning of subliminal visual stimuli: evidence of multistage learning. Frontiers in Psychology, 6. http://doi.org/10.3389/fpsyg.2015.00076

* Kiefer, M., Sim, E.-J., & Wentura, D. (2015). Boundary conditions for the influ-ence of unfamiliar non-target primes in unconscious evaluative priming: The moderating role of attentional task sets. Consciousness and Cognition, 35, 342–356. http://doi.org/10.1016/j.concog.2015.01.010

Krendl, A. C., Ambady, N., & Kensinger, E. A. (2015). The dissociable effects of stereotype threat on older adults’ memory encoding and retrieval. Journal of Applied Research in Memory and Cognition, 4(2), 103–109. http://doi.org/10.1016/j.jarmac.2015.02.001

Levy, B. R., Pilver, C., Chung, P. H., & Slade, M. D. (2014). Subliminal Strengthen-ing Improving Older Individuals’ Physical Function Over Time With an Implic-it-Age-Stereotype Intervention. Psychological Science, 25(12), 2127–2135.

* Lierde, E. V., Desender, K., & Bussche, E. V. den. (2015). Is conflict adaptation triggered by feature repetitions? An unexpected finding. Frontiers in Psycholo-gy, 5. http://doi.org/10.3389/fpsyg.2014.01358

* Lin, Z., & Murray, S. O. (2015). Automaticity of unconscious response inhibition: Comment on Chiu and Aron (2014). Journal of Experimental Psychology: Gen-eral, 144(1), 244–254. http://doi.org/10.1037/xge0000042

* Lipka, J., Hoffmann, M., Miltner, W. H. R., & Straube, T. (2014). Effects of Cog-nitive-Behavioral Therapy on Brain Responses to Subliminal and Supraliminal Threat and Their Functional Significance in Specific Phobia. Biological Psychi-atry, 76(11), 869–877. http://doi.org/10.1016/j.biopsych.2013.11.008

* Lipp, O. V., Kempnich, C., Jee, S. H., & Arnold, D. H. (2014). Fear Conditioning to Subliminal Fear Relevant and Non Fear Relevant Stimuli. PLoS ONE, 9(9), e99332. http://doi.org/10.1371/journal.pone.0099332

Li, T.-T., & Lu, Y. (2014). The subliminal affective priming effects of faces display-ing various levels of arousal: An ERP study. Neuroscience Letters, 583, 148–153. http://doi.org/10.1016/j.neulet.2014.09.027

Liu, P., Chen, X., Dai, D., Wang, Y., & Wang, Y. (2014). A subliminal inhibitory mechanism for the negative compatibility effect: a continuous versus threshold mechanism. Experimental Brain Research, 232(7), 2305–2315. http://doi.org/10.1007/s00221-014-3925-x

Liu, Z., Wu, L., & Hou, C. (2015). Social Identity: The Cause of Distinction Be-tween Group-Reference and Self-Reference Effects. Social Behavior and Per-sonality: An International Journal, 43(9), 1409–1418. http://doi.org/10.2224/sbp.2015.43.9.1409

Mahoney, M. B., Saunders, B. A., & Cain, N. M. (2014). Priming Mortality Sali-ence: Supraliminal, Subliminal and “Double-Death” Priming Techniques. Death Studies, 38(10), 678–681. http://doi.org/10.1080/07481187.2013.839586

Małecki, M. (2015). FAMILIARITY TRANSFER AS AN EXPLANATION OF THE DÉJÀ VU EFFECT

1. Psychological Reports, 116(3), 955–982.

http://doi.org/10.2466/19.04.PR0.116k25w1

88

* Manly, T., Fish, J. E., Griffiths, S., Molenveld, M., Zhou, F. A., & Davis, G. J. (2014). Unconscious Priming of Task-Switching Generalizes to an Untrained Task. PLoS ONE, 9(2), e88416. http://doi.org/10.1371/journal.pone.0088416

* Mastropasqua, T., & Turatto, M. (2015). Attention is necessary for subliminal instrumental conditioning. Scientific Reports, 5, 12920. http://doi.org/10.1038/srep12920

* Mattiassi, A. D. A., Mele, S., Ticini, L. F., & Urgesi, C. (2014). Conscious and Unconscious Representations of Observed Actions in the Human Motor System. Journal of Cognitive Neuroscience, 26(9), 2028–2041. http://doi.org/10.1162/jocn_a_00619

* Montoro, P. R., Luna, D., & Ortells, J. J. (2014). Subliminal Gestalt grouping: Evidence of perceptual grouping by proximity and similarity in absence of con-scious perception. Consciousness and Cognition, 25, 1–8. http://doi.org/10.1016/j.concog.2014.01.004

Motomura, Y., Kitamura, S., Oba, K., Terasawa, Y., Enomoto, M., Katayose, Y., … Mishima, K. (2014). Sleepiness induced by sleep-debt enhanced amygdala ac-tivity for subliminal signals of fear. BMC Neuroscience, 15(1), 1.Mumenthaler, C., & Sander, D. (2015). Automatic integration of social information in emotion recognition. Journal of Experimental Psychology: General, 144(2), 392–399. http://doi.org/10.1037/xge0000059

* Nakajima, K., Minami, T., & Nakauchi, S. (2015). Effects of facial color on the subliminal processing of fearful faces. Neuroscience, 310, 472–485. http://doi.org/10.1016/j.neuroscience.2015.09.059

* Noguchi, Y., Kimijima, S., & Kakigi, R. (2015). Direct behavioral and neural evidence for an offset-triggered conscious perception. Cortex, 65, 159–172. http://doi.org/10.1016/j.cortex.2015.01.017

Nuske, H. J., Vivanti, G., Hudry, K., & Dissanayake, C. (2014). Pupillometry re-veals reduced unconscious emotional reactivity in autism. Biological Psycholo-gy, 101, 24–35. http://doi.org/10.1016/j.biopsycho.2014.07.003

* Ocampo, B., Al-Janabi, S., & Finkbeiner, M. (2015). Direct evidence of cognitive control without perceptual awareness. Psychonomic Bulletin & Review, 22(4), 1083–1088. http://doi.org/10.3758/s13423-014-0766-3

Ong, H. H., Mullette-Gillman, O. A., Kwok, K., & Lim, J. (2014). Moral judgment modulation by disgust is bi-directionally moderated by individual sensitivity. Frontiers in Psychology, 5. http://doi.org/10.3389/fpsyg.2014.00194

Pascucci, D., Mastropasqua, T., & Turatto, M. (2015). Monetary Reward Modulates Task-Irrelevant Perceptual Learning for Invisible Stimuli. PLOS ONE, 10(5), e0124009. http://doi.org/10.1371/journal.pone.0124009

* Pas, P., Custers, R., Bijleveld, E., & Vink, M. (2014). Effort responses to subopti-mal reward cues are related to striatal dopaminergic functioning. Motivation and Emotion, 38(6), 759–770. http://doi.org/10.1007/s11031-014-9434-1

Peraza, J., Cservenka, A., Herting, M. M., & Nagel, B. J. (2015). Atypical parietal lobe activity to subliminal faces in youth with a family history of alcoholism. The American Journal of Drug and Alcohol Abuse, 41(2), 139–145. http://doi.org/10.3109/00952990.2014.953251

Peremen, Z., & Lamy, D. (2014). Comparing unconscious processing during contin-uous flash suppression and meta-contrast masking just under the limen of con-sciousness. Frontiers in Psychology, 5. http://doi.org/10.3389/fpsyg.2014.00969

* Pine, A., Mendelsohn, A., & Dudai, Y. (2014). Unconscious learning of likes and dislikes is persistent, resilient, and reconsolidates. Frontiers in Psychology, 5. http://doi.org/10.3389/fpsyg.2014.01051

89

Podea, D. (2014). When does Subliminal Affective Image Priming Influence the Ability of Schizophrenic Patients to Perceive Face Emotions? Medical Science Monitor, 20, 2788–2798. http://doi.org/10.12659/MSM.893118

Posten, A.-C., Ockenfels, A., & Mussweiler, T. (2014). How activating cognitive content shapes trust: A subliminal priming study. Journal of Economic Psychol-ogy, 41, 12–19. http://doi.org/10.1016/j.joep.2013.04.002

Prete, G., Capotosto, P., Zappasodi, F., Laeng, B., & Tommasi, L. (2015). The cere-bral correlates of subliminal emotions: an eleoencephalographic study with emotional hybrid faces. European Journal of Neuroscience, 42(11), 2952–2962. http://doi.org/10.1111/ejn.13078

Rabellino, D., Tursich, M., Frewen, P. A., Daniels, J. K., Densmore, M., Théberge, J., & Lanius, R. A. (2015). Intrinsic Connectivity Networks in post-traumatic stress disorder during sub- and supraliminal processing of threat-related stimuli. Acta Psychiatrica Scandinavica, 132(5), 365–378. http://doi.org/10.1111/acps.12418

Ramanathan, P., Kennedy, M. R. T., & Marsolek, C. J. (2014). Implicit Memory Influences on Metamemory During Verbal Learning After Traumatic Brain In-jury. Journal of Speech Language and Hearing Research, 57(5), 1817. http://doi.org/10.1044/2014_JSLHR-L-13-0204

Reed, P., & Clarke, N. (2014). Effect of religious context on the content of visual hallucinations in individuals high in religiosity. Psychiatry Research, 215(3), 594–598. http://doi.org/10.1016/j.psychres.2014.01.006

Riener, A., Chalfoun, P., & Frasson, C. (2014). The Potential of Subliminal Infor-mation Displays to Change Driver Behavior. Presence: Teleoperators and Vir-tual Environments, 23(1), 51–70. http://doi.org/10.1162/PRES_a_00170

Roberts, M. E., Gibbons, F. X., Kingsbury, J. H., & Gerrard, M. (2014). Not intend-ing but somewhat willing: The influence of visual primes on risky sex decisions. British Journal of Health Psychology, 19(3), 553–565. http://doi.org/10.1111/bjhp.12055

Rohr, M., Degner, J., & Wentura, D. (2015). The “emotion misattribution” proce-dure: Processing beyond good and bad under masked and unmasked presenta-tion conditions. Cognition and Emotion, 29(2), 196–219. http://doi.org/10.1080/02699931.2014.898613

* Rohr, M., & Wentura, D. (2014). Spatial frequency filtered images reveal differ-ences between masked and unmasked processing of emotional information. Consciousness and Cognition, 29, 141–158. http://doi.org/10.1016/j.concog.2014.08.021

Sandberg, K., Del Pin, S. H., Bibby, B. M., & Overgaard, M. (2014). Evidence of weak conscious experiences in the exclusion task. Frontiers in Psychology, 5. http://doi.org/10.3389/fpsyg.2014.01080

* Sato, W., Kubota, Y., & Toichi, M. (2014). Enhanced subliminal emotional re-sponses to dynamic facial expressions. Frontiers in Psychology, 5. http://doi.org/10.3389/fpsyg.2014.00994

* Schoeberl, T., Fuchs, I., Theeuwes, J., & Ansorge, U. (2015). Stimulus-driven attentional capture by subliminal onset cues. Attention, Perception, & Psycho-physics, 77(3), 737–748. http://doi.org/10.3758/s13414-014-0802-4

Schoth, D. E., Williams, S., & Liossi, C. (2015). Attentional bias for pain- and peri-od-related symptom words in healthy women who experienced a recent painful period: Attentional bias in women who experienced a recent painful period. Eu-ropean Journal of Pain, 19(6), 745–751. http://doi.org/10.1002/ejp.597

90

* Silverstein, B. H., Snodgrass, M., Shevrin, H., & Kushwaha, R. (2015). P3b, con-sciousness, and complex unconscious processing. Cortex, 73, 216–227. http://doi.org/10.1016/j.cortex.2015.09.004

* Sperdin, H. F., Spierer, L., Becker, R., Michel, C. M., & Landis, T. (2015). Sub-millisecond unmasked subliminal visual stimuli evoke electrical brain respons-es: VEPs to Submillisecond Unmasked Visual Stimulation. Human Brain Map-ping, 36(4), 1470–1483. http://doi.org/10.1002/hbm.22716

* Stenner, M.-P., Bauer, M., Heinze, H.-J., Haggard, P., & Dolan, R. J. (2015). Par-allel processing streams for motor output and sensory prediction during action preparation. Journal of Neurophysiology, 113(6), 1752–1762. http://doi.org/10.1152/jn.00616.2014

* Ulrich, M., Adams, S. C., & Kiefer, M. (2014). Flexible establishment of func-tional brain networks supports attentional modulation of unconscious cognition: Dynamic Functional Networks. Human Brain Mapping, 35(11), 5500–5516. http://doi.org/10.1002/hbm.22566

Villepoux, A., Vermeulen, N., Niedenthal, P., & Mermillod, M. (2015). Evidence of fast and automatic gender bias in affective priming. Journal of Cognitive Psy-chology, 27(3), 301–309. http://doi.org/10.1080/20445911.2014.1000919

* Weichselbaum, H., Fuchs, I., & Ansorge, U. (2014). Oculomotor capture by supra-liminal and subliminal onset singletons: The role of contrast polarity. Vision Re-search, 100, 1–7. http://doi.org/10.1016/j.visres.2014.03.014

Wetherill, R. R., Childress, A. R., Jagannathan, K., Bender, J., Young, K. A., Suh, J. J., … Franklin, T. R. (2014). Neural responses to subliminally presented canna-bis and other emotionally evocative cues in cannabis-dependent individuals. Psychopharmacology, 231(7), 1397–1407. http://doi.org/10.1007/s00213-013-3342-z

* Wildegger, T., Myers, N. E., Humphreys, G., & Nobre, A. C. (2015). Supraliminal but not subliminal distracters bias working memory recall. Journal of Experi-mental Psychology: Human Perception and Performance, 41(3), 826–839. http://doi.org/10.1037/xhp0000052

* Xiao, K., & Yamauchi, T. (2015). Subliminal semantic priming in near absence of attention: A cursor motion study. Consciousness and Cognition, 38, 88–98. http://doi.org/10.1016/j.concog.2015.09.013

* Xue, X., Zhou, X., & Li, S. (2015). Unconscious reward facilitates motion percep-tual learning. Visual Cognition, 23(1-2), 161–178. http://doi.org/10.1080/13506285.2014.981625

* Ye, X., He, S., Hu, Y., Yu, Y. Q., & Wang, K. (2014). Interference between Con-scious and Unconscious Facial Expression Information. Retrieved from http://dx.plos.org/10.1371/journal.pone.0105156

* Yoshimoto, S., Imai, H., Kashino, M., & Takeuchi, T. (2014). Pupil Response and the Subliminal Mere Exposure Effect. PLoS ONE, 9(2), e90670. http://doi.org/10.1371/journal.pone.0090670

Young, K. A., Franklin, T. R., Roberts, D. C. S., Jagannathan, K., Suh, J. J., Wether-ill, R. R., … Childress, A. R. (2014). Nipping Cue Reactivity in the Bud: Baclo-fen Prevents Limbic Activation Elicited by Subliminal Drug Cues. Journal of Neuroscience, 34(14), 5038–5043. http://doi.org/10.1523/JNEUROSCI.4977-13.2014

Zhang, D., Sessa, S., Kong, W., Cosentino, S., Magistro, D., Ishii, H., … Takanishi, A. (2015). Development of subliminal persuasion system to improve the upper limb posture in laparoscopic training: a preliminary study. International Journal of Computer Assisted Radiology and Surgery, 10(11), 1863–1871. http://doi.org/10.1007/s11548-015-1198-x

91

* Züst, M. A., Colella, P., Reber, T. P., Vuilleumier, P., Hauf, M., Ruch, S., & Henke, K. (2015). Hippocampus Is Place of Interaction between Unconscious and Conscious Memories. PloS One, 10(3). Retrieved from http://www.ncbi.nlm.nih.gov/pmc/articles/PMC4380440/

92

References

Abrams, R. L., Klinger, M. R., & Greenwald, A. G. (2002). Subliminal words acti-vate semantic categories (not automated motor responses). Psychonomic Bulle-tin & Review, 9(1), 100–106.

Adams, S. C., & Kiefer, M. (2012). Testing the attentional boundary conditions of subliminal semantic priming: the influence of semantic and phonological task sets. Frontiers in Human Neuroscience, 6. http://doi.org/10.3389/fnhum.2012.00241

Albrecht, T., & Mattler, U. (2012). Individual differences in subjective experience and objective performance in metacontrast masking. Journal of Vision, 12(5), 5–5. http://doi.org/10.1167/12.5.5

Ansorge, U., Kunde, W., & Kiefer, M. (2014). Unconscious vision and executive control: How unconscious processing and conscious action control interact. Consciousness and Cognition, 27, 268–287. http://doi.org/10.1016/j.concog.2014.05.009

Aranyi, G., Kouider, S., Lindsay, A., Prins, H., Ahmed, I., Jacucci, G., … Cavazza, M. (2014). Subliminal Cueing of Selection Behavior in a Virtual Environment. Presence: Teleoperators and Virtual Environments, 23(1), 33–50. http://doi.org/10.1162/PRES_a_00167

Axelrod, V., Bar, M., Rees, G., & Yovel, G. (2015). Neural Correlates of Subliminal Language Processing. Cerebral Cortex, 25(8), 2160–2169. http://doi.org/10.1093/cercor/bhu022

Bargh, J. A., & Morsella, E. (2008). The Unconscious Mind. Perspectives on Psy-chological Science, 3(1), 73–79. http://doi.org/10.1111/j.1745-6916.2008.00064.x

Bell, D., Forster, K., & Drake, S. (2015). Early semantic activation in a semantic categorization task with masked primes: Cascaded or not? Journal of Memory and Language, 85, 1–14. http://doi.org/10.1016/j.jml.2015.06.007

Bem, D. J. (2011). Feeling the future: Experimental evidence for anomalous retroac-tive influences on cognition and affect. Journal of Personality and Social Psy-chology, 100(3), 407–425. http://doi.org/10.1037/a0021524

Binder, J. R., Desai, R. H., Graves, W. W., & Conant, L. L. (2009). Where Is the Semantic System? A Critical Review and Meta-Analysis of 120 Functional Neuroimaging Studies. Cerebral Cortex, 19(12), 2767–2796. http://doi.org/10.1093/cercor/bhp055

Björkman, M., Juslin, P., & Winman, A. (1993). Realism of confidence in sensory discrimination: The underconfidence phenomenon. Perception & Psychophys-ics, 54(1), 75–81. http://doi.org/10.3758/BF03206939

Breitmeyer, B. G. (2007). Visual masking: past accomplishments, present status, future developments. Advances in Cognitive Psychology, 3(1-2), 9–20.

Breitmeyer, B. G. (2015). Psychophysical “blinding” methods reveal a functional hierarchy of unconscious visual processing. Consciousness and Cognition, 35, 234–250. http://doi.org/10.1016/j.concog.2015.01.012

93

Brintazzoli, G., Soetens, E., Deroost, N., & Van den Bussche, E. (2012). Conscious, but not unconscious, logo priming of brands and related words. Consciousness and Cognition, 21(2), 824–834. http://doi.org/10.1016/j.concog.2012.03.006

Broyles, S. J. (2006). Subliminal Advertising and the Perpetual Popularity of Play-ing to People’s Paranoia. Journal of Consumer Affairs, 40(2), 392–406. http://doi.org/10.1111/j.1745-6606.2006.00063.x

Carr, T. H., & Dagenbach, D. (1990). Semantic priming and repetition priming from masked words: evidence for a center-surround attentional mechanism in percep-tual recognition. Journal of Experimental Psychology: Learning, Memory, and Cognition, 16(2), 341.

Carter, T. J., Ferguson, M. J., & Hassin, R. R. (2011). A Single Exposure to the American Flag Shifts Support Toward Republicanism up to 8 Months Later. Psychological Science, 22(8), 1011–1018. http://doi.org/10.1177/0956797611414726

Cetnarski, R., Betella, A., Prins, H., Kouider, S., & Verschure, P. F. M. J. (2014). Subliminal Response Priming in Mixed Reality: The Ecological Validity of a Classic Paradigm of Perception. Presence: Teleoperators and Virtual Environ-ments, 23(1), 1–17. http://doi.org/10.1162/PRES_a_00171

Chambon, V., Moore, J. W., & Haggard, P. (2015). TMS stimulation over the infe-rior parietal cortex disrupts prospective sense of agency. Brain Structure and Function, 220(6), 3627–3639. http://doi.org/10.1007/s00429-014-0878-6

Cheesman, J., & Merikle, P. M. (1984). Priming with and without awareness. Per-ception & Psychophysics, 36(4), 387–395. http://doi.org/10.3758/BF03202793

Cohen, M. A., & Dennett, D. C. (2011). Consciousness cannot be separated from function. Trends in Cognitive Sciences, 15(8), 358–364. http://doi.org/10.1016/j.tics.2011.06.008

Collins, A. M., & Loftus, E. F. (1975). A spreading-activation theory of semantic processing. Psychological Review, 82(6), 407–428. http://doi.org/10.1037/0033-295X.82.6.407

Coltheart, M., Rastle, K., Perry, C., Langdon, R., & Ziegler, J. (2001). DRC: A dual route cascaded model of visual word recognition and reading aloud. Psychologi-cal Review, 108(1), 204–256. http://doi.org/10.1037/0033-295X.108.1.204

Craik, F. I. M., & Lockhart, R. S. (1972). Levels of processing: A framework for memory research. Journal of Verbal Learning and Verbal Behavior, 11(6), 671–684. http://doi.org/10.1016/S0022-5371(72)80001-X

Custers, R., & Aarts, H. (2010). The Unconscious Will: How the Pursuit of Goals Operates Outside of Conscious Awareness. Science, 329(5987), 47–50. http://doi.org/10.1126/science.1188595

Dagenbach, D., Carr, T. H., & Wilhelmsen, A. (1989). Task-induced strategies and near-threshold priming: Conscious influences on unconscious perception. Jour-nal of Memory and Language, 28(4), 412–443. http://doi.org/10.1016/0749-596X(89)90020-X

Damian, M. F. (2001). Congruity effects evoked by subliminally presented primes: Automaticity rather than semantic processing. Journal of Experimental Psy-chology: Human Perception and Performance, 27(1), 154–165. http://doi.org/10.1037//0096-1523.27.1.154

Data from Investigating Variation in Replicability: A “Many Labs” Replication Project. (2014). Journal of Open Psychology Data, 2(1), e4. http://doi.org/10.5334/jopd.ad

Dean, A. F. (1981). The variability of discharge of simple cells in the cat striate cortex. Experimental Brain Research, 44(4). http://doi.org/10.1007/BF00238837

94

Dehaene, S., & Changeux, J.-P. (2011). Experimental and Theoretical Approaches to Conscious Processing. Neuron, 70(2), 200–227. http://doi.org/10.1016/j.neuron.2011.03.018

Dehaene, S., Kerszberg, M., & Changeux, J.-P. (1998). A neuronal model of a glob-al workspace in effortful cognitive tasks. Proceedings of the National Academy of Sciences, 95(24), 14529–14534. http://doi.org/10.1073/pnas.95.24.14529

Dehaene, S., Naccache, L., Le Clec’H, G., Koechlin, E., Mueller, M., Dehaene-Lambertz, G., … Le Bihan, D. (1998). Imaging unconscious semantic priming. Nature, 395(6702), 597–600.

Del Cul, A., Baillet, S., & Dehaene, S. (2007). Brain Dynamics Underlying the Non-linear Threshold for Access to Consciousness. PLoS Biology, 5(10), e260. http://doi.org/10.1371/journal.pbio.0050260

de Wit, B., & Kinoshita, S. (2015). The masked semantic priming effect is task de-pendent: Reconsidering the automatic spreading activation process. Journal of Experimental Psychology: Learning, Memory, and Cognition, 41(4), 1062–1075. http://doi.org/10.1037/xlm0000074

Dienes, Z. (2007). Subjective measures of unconscious knowledge. In Progress in Brain Research (Vol. 168, pp. 49–269). Elsevier. Retrieved from http://linkinghub.elsevier.com/retrieve/pii/S0079612307680054

Dienes, Z. (2008). Understanding psychology as a science: an introduction to scien-tific and statistical inference. New York: Palgrave Macmillan.

Dienes, Z. (2011). Bayesian Versus Orthodox Statistics: Which Side Are You On? Perspectives on Psychological Science, 6(3), 274–290. http://doi.org/10.1177/1745691611406920

Dienes, Z. (2014). Using Bayes to get the most out of non-significant results. Fron-tiers in Psychology, 5. http://doi.org/10.3389/fpsyg.2014.00781

Dienes, Z. (2015). How Bayesian statistics are needed to determine whether mental states are unconscious. In Behavioural Methods in Consciousness Research. (Vol. 2015, pp. 199–220). Oxford: Oxford University Press.

Dienes, Z., & Seth, A. (2010). Gambling on the unconscious: A comparison of wa-gering and confidence ratings as measures of awareness in an artificial grammar task. Consciousness and Cognition, 19(2), 674–681. http://doi.org/10.1016/j.concog.2009.09.009

Dienes, Z., & Seth, A. K. (2010). Measuring any conscious content versus measur-ing the relevant conscious content: Comment on Sandberg et al. Consciousness and Cognition, 19(4), 1079–1080. http://doi.org/10.1016/j.concog.2010.03.009

Dosher, B. A. (1998). The response-window regression method—Some problematic assumptions: Comment on Draine and Greenwald (1998). Journal of Experi-mental Psychology: General, 127(3), 311–317. http://doi.org/10.1037/0096-3445.127.3.311

Eimer, M., & Schlaghecken, F. (1998). Effects of masked stimuli on motor activa-tion: Behavioral and electrophysiological evidence. Journal of Experimental Psychology: Human Perception and Performance, 24(6), 1737–1747. http://doi.org/10.1037/0096-1523.24.6.1737

Elwert, F., & Winship, C. (2014). Endogenous Selection Bias: The Problem of Con-ditioning on a Collider Variable. Annual Review of Sociology, 40(1), 31–53. http://doi.org/10.1146/annurev-soc-071913-043455

Eriksen, C. W. (1960). Discrimination and learning without awareness: A methodo-logical survey and evaluation. Psychological Review, 67(5), 279–300. http://doi.org/10.1037/h0041622

Evans, S., & Azzopardi, P. (2007). Evaluation of a “bias-free” measure of aware-ness. Spatial Vision, 20(1), 61–77. http://doi.org/10.1163/156856807779369742

95

Fahrenfort, J. J., Scholte, H. S., & Lamme, V. A. F. (2007). Masking Disrupts Reen-trant Processing in Human Visual Cortex. Journal of Cognitive Neuroscience, 19(9), 1488–1497. http://doi.org/10.1162/jocn.2007.19.9.1488

Finkbeiner, M. (2011). Subliminal priming with nearly perfect performance in the prime-classification task. Attention, Perception, & Psychophysics, 73(4), 1255–1265. http://doi.org/10.3758/s13414-011-0088-8

Fisk, G. D., & Haase, S. J. (2011). The relationship between the objective identifica-tion threshold and priming effects does not provide a definitive boundary be-tween conscious and unconscious perceptual processes. Consciousness and Cognition, 20(4), 1221–1231. http://doi.org/10.1016/j.concog.2011.03.016

Fleming, S. M., & Lau, H. C. (2014). How to measure metacognition. Frontiers in Human Neuroscience, 8. http://doi.org/10.3389/fnhum.2014.00443

Fleur, M. L. D., & Petranoff, R. M. (1959). A Televised Test of Subliminal Persua-sion. Public Opinion Quarterly, 23(2), 168. http://doi.org/10.1086/266862

Gaillard, R., Del Cul, A., Naccache, L., Vinckier, F., Cohen, L., & Dehaene, S. (2006). Nonconscious semantic processing of emotional words modulates con-scious access. Proceedings of the National Academy of Sciences, 103(19), 7524–7529.

Gallistel, C. R. (2009). The importance of proving the null. Psychological Review, 116(2), 439–453. http://doi.org/10.1037/a0015251

Gescheider, G. A. (1997). Psychophysics: the fundamentals (3rd ed). Mahwah, N.J: L. Erlbaum Associates.

Gold, J. I., & Shadlen, M. N. (2007). The Neural Basis of Decision Making. Annual Review of Neuroscience, 30(1), 535–574. http://doi.org/10.1146/annurev.neuro.29.051605.113038

González-García, C., Tudela, P., & Ruz, M. (2015). Unconscious biases in task choices depend on conscious expectations. Consciousness and Cognition, 37, 44–56. http://doi.org/10.1016/j.concog.2015.08.001

Goodhew, S. C., Visser, T. A. W., Lipp, O. V., & Dux, P. E. (2011). Implicit seman-tic perception in object substitution masking. Cognition, 118(1), 130–134. http://doi.org/10.1016/j.cognition.2010.10.013

Green, D. M., & Swets, J. A. (2000). Signal detection theory and psychophysics (Repr. ed). Los Altos Hills, Calif: Peninsula Publ.

Greenwald, A. G., Abrams, R. L., Naccache, L., & Dehaene, S. (2003). Long-term semantic memory versus contextual memory in unconscious number processing. Journal of Experimental Psychology: Learning, Memory, and Cognition, 29(2), 235–247. http://doi.org/10.1037/0278-7393.29.2.235

Greenwald, A. G., Draine, S. C., & Abrams, R. L. (1996). Three Cognitive Markers of Unconscious Semantic Activation. Science, 273(5282), 1699–1702. http://doi.org/10.1126/science.273.5282.1699

Grill-Spector, K., Knouf, N., & Kanwisher, N. (2004). The fusiform face area sub-serves face perception, not generic within-category identification. Nature Neu-roscience, 7(5), 555–562. http://doi.org/10.1038/nn1224

Haase, S. J., & Fisk, G. D. (2015). Awareness of “Invisible” Arrows in a Metacon-trast Masking Paradigm. The American Journal of Psychology, 128(1), 15–30. http://doi.org/10.5406/amerjpsyc.128.1.0015

Harm, M. W., & Seidenberg, M. S. (2004). Computing the Meanings of Words in Reading: Cooperative Division of Labor Between Visual and Phonological Pro-cesses. Psychological Review, 111(3), 662–720. http://doi.org/10.1037/0033-295X.111.3.662

96

Hassin, R. R. (2013). Yes It Can: On the Functional Abilities of the Human Uncon-scious. Perspectives on Psychological Science, 8(2), 195–207. http://doi.org/10.1177/1745691612460684

Hassin, R. R., Ferguson, M. J., Shidlovski, D., & Gross, T. (2007). Subliminal expo-sure to national flags affects political thought and behavior. Proceedings of the National Academy of Sciences, 104(50), 19757–19761.

Hecht, S., Shlaer, S., & Pirenne, M. H. (1942). Energy, quanta, and vision. The Journal of General Physiology, 25(6), 819–840.

Hoffmann, M., Mothes-Lasch, M., Miltner, W. H. R., & Straube, T. (2015). Brain activation to briefly presented emotional words: Effects of stimulus awareness: Awareness, Amygdala, and Words. Human Brain Mapping, 36(2), 655–665. http://doi.org/10.1002/hbm.22654

Holender, D. (1986). Semantic activation without conscious identification in dichot-ic listening, parafoveal vision, and visual masking: A survey and appraisal. Be-havioral and Brain Sciences, 9(01), 1–23.

Holender, D., & Duscherer, K. (2004). Unconscious perception: The need for a paradigm shift. Perception & Psychophysics, 66(5), 872–881. http://doi.org/10.3758/BF03194980

Hulme, O. J., Friston, K. F., & Zeki, S. (2009). Neural Correlates of Stimulus Re-portability. Journal of Cognitive Neuroscience, 21(8), 1602–1610. http://doi.org/10.1162/jocn.2009.21119

Jachs, B., Blanco, M. J., Grantham-Hill, S., & Soto, D. (2015). On the independence of visual awareness and metacognition: A signal detection theoretic analysis. Journal of Experimental Psychology: Human Perception and Performance, 41(2), 269–276. http://doi.org/10.1037/xhp0000026

Jusyte, A., & Schönenberg, M. (2014). Subliminal cues bias perception of facial affect in patients with social phobia: evidence for enhanced unconscious threat processing. Frontiers in Human Neuroscience, 8. http://doi.org/10.3389/fnhum.2014.00580

Kahan, T. A. (2000). Negative priming from masked words: Retrospective prime clarification or center-surround inhibition? Journal of Experimental Psychology: Learning, Memory, and Cognition, 26(6), 1392–1410. http://doi.org/10.1037/0278-7393.26.6.1392

Kanai, R., Walsh, V., & Tseng, C. (2010). Subjective discriminability of invisibility: A framework for distinguishing perceptual and attentional failures of awareness. Consciousness and Cognition, 19(4), 1045–1057. http://doi.org/10.1016/j.concog.2010.06.003

Kiefer, M., & Martens, U. (2010). Attentional sensitization of unconscious cogni-tion: Task sets modulate subsequent masked semantic priming. Journal of Ex-perimental Psychology: General, 139(3), 464–489. http://doi.org/10.1037/a0019561

Kiefer, M., Sim, E.-J., & Wentura, D. (2015). Boundary conditions for the influence of unfamiliar non-target primes in unconscious evaluative priming: The moder-ating role of attentional task sets. Consciousness and Cognition, 35, 342–356. http://doi.org/10.1016/j.concog.2015.01.010

Kim, C.-Y., & Blake, R. (2005). Psychophysical magic: rendering the visible “invis-ible.” Trends in Cognitive Sciences, 9(8), 381–388. http://doi.org/10.1016/j.tics.2005.06.012

Klinger, M. R., Burton, P. C., & Pitts, G. S. (2000). Mechanisms of unconscious priming: I. Response competition, not spreading activation. Journal of Experi-mental Psychology: Learning, Memory, and Cognition, 26(2), 441–455. http://doi.org/10.1037//0278-7393.26.2.441

97

Klotz, W., & Neumann, O. (1999). Motor activation without conscious discrimina-tion in metacontrast masking. Journal of Experimental Psychology: Human Perception and Performance, 25(4), 976–992. http://doi.org/10.1037/0096-1523.25.4.976

Koch, C. (2004). The quest for consciousness: a neurobiological approach. Denver, Colo: Roberts and Co.

Kouider, S., & Dehaene, S. (2007). Levels of processing during non-conscious per-ception: a critical review of visual masking. Philosophical Transactions of the Royal Society B: Biological Sciences, 362(1481), 857–875. http://doi.org/10.1098/rstb.2007.2093

Kunde, W., Kiesel, A., & Hoffmann, J. (2005). On the masking and disclosure of unconscious elaborate processing. A reply to Van Opstal, Reynvoet, and Verguts (2005). Cognition, 97(1), 99–105. http://doi.org/10.1016/j.cognition.2005.03.005

Kunimoto, C., Miller, J., & Pashler, H. (2001). Confidence and Accuracy of Near-Threshold Discrimination Responses. Consciousness and Cognition, 10(3), 294–340. http://doi.org/10.1006/ccog.2000.0494

Lamme, V. A. F. (2006). Towards a true neural stance on consciousness. Trends in Cognitive Sciences, 10(11), 494–501. http://doi.org/10.1016/j.tics.2006.09.001

Lamy, D., Salti, M., & Bar-Haim, Y. (2009). Neural correlates of subjective aware-ness and unconscious processing: an ERP study. Journal of Cognitive Neurosci-ence, 21(7), 1435–1446.

Lau, H. C., & Passingham, R. E. (2007). Unconscious Activation of the Cognitive Control System in the Human Prefrontal Cortex. Journal of Neuroscience, 27(21), 5805–5811. http://doi.org/10.1523/JNEUROSCI.4335-06.2007

Lau, H., & Rosenthal, D. (2011). Empirical support for higher-order theories of conscious awareness. Trends in Cognitive Sciences, 15(8), 365–373. http://doi.org/10.1016/j.tics.2011.05.009

Leuthold, H., & Kopp, B. (1998). Mechanisms of priming by masked stimuli: Infer-ences from event-related brain potentials. Psychological Science, 9(4), 263–269.

Lin, Z., & Murray, S. O. (2014). Priming of awareness or how not to measure visual awareness. Journal of Vision, 14(1), 27–27. http://doi.org/10.1167/14.1.27

Lin, Z., & Murray, S. O. (2015). Automaticity of unconscious response inhibition: Comment on Chiu and Aron (2014). Journal of Experimental Psychology: Gen-eral, 144(1), 244–254. http://doi.org/10.1037/xge0000042

Macknik, S. L., & Martinez-Conde, S. (2007). The role of feedback in visual mask-ing and visual processing. Advances in Cognitive Psychology, 3(1-2), 125–152.

Macmillan, N. A. (1986). The psychophysics of subliminal perception. Behavioral and Brain Sciences, 9(1), 38–39.

Macmillan, N. A., & Creelman, C. D. (2005). Detection theory: a user’s guide (2nd ed). Mahwah, N.J: Lawrence Erlbaum Associates.

Marcel, A. J. (1983). Conscious and unconscious perception: Experiments on visual masking and word recognition. Cognitive Psychology, 15(2), 197–237. http://doi.org/10.1016/0010-0285(83)90009-9

Masson, M. E. J. (1995). A distributed memory model of semantic priming. Journal of Experimental Psychology: Learning, Memory, and Cognition, 21(1), 3–23. http://doi.org/10.1037/0278-7393.21.1.3

Mathewson, K. E., Gratton, G., Fabiani, M., Beck, D. M., & Ro, T. (2009). To See or Not to See: Prestimulus Phase Predicts Visual Awareness. Journal of Neuro-science, 29(9), 2725–2732. http://doi.org/10.1523/JNEUROSCI.3963-08.2009

Merikle, P. M. (1982). Unconscious perception revisited. Attention, Perception, & Psychophysics, 31(3), 298–301.

98

Merikle, P. M., & Cheesman, J. (1986). Consciousness is a “subjective” state. Be-havioral and Brain Sciences, 9(01).

Mesulam, M.-M. (1998). From sensation to cognition. Brain, 121(6), 1013–1052. Miller, J. (2000). Measurement error in subliminal perception experiments: Simula-

tion analyses of two regression methods. Journal of Experimental Psychology: Human Perception and Performance, 26(4), 1461–1477. http://doi.org/10.1037//0096-1523.26.4.1461

Miller, J. G. (1939). Discrimination without Awareness. The American Journal of Psychology, 52(4), 562. http://doi.org/10.2307/1416469

Miller, J., & Schwarz, W. (2014). Brain signals do not demonstrate unconscious decision making: An interpretation based on graded conscious awareness. Con-sciousness and Cognition, 24, 12–21. http://doi.org/10.1016/j.concog.2013.12.004

Moors, P., Boelens, D., van Overwalle, J., & Wagemans, J. (2016). Scene Integra-tion Without Awareness: No Conclusive Evidence for Processing Scene Con-gruency During Continuous Flash Suppression. Psychological Science, 27(7), 945–956. http://doi.org/10.1177/0956797616642525

Morey, R. D., Hoekstra, R., Rouder, J. N., Lee, M. D., & Wagenmakers, E.-J. (2016). The fallacy of placing confidence in confidence intervals. Psychonomic Bulletin & Review, 23(1), 103–123. http://doi.org/10.3758/s13423-015-0947-8

Mumenthaler, C., & Sander, D. (2015). Automatic integration of social information in emotion recognition. Journal of Experimental Psychology: General, 144(2), 392–399. http://doi.org/10.1037/xge0000059

Muscarella, C., Brintazzoli, G., Gordts, S., Soetens, E., & Van den Bussche, E. (2013). Short- and long-term effects of conscious, minimally conscious and un-conscious brand logos. PLoS ONE, 8(5), e57738. http://doi.org/10.1371/journal.pone.0057738

Naccache, L., & Dehaene, S. (2001). Unconscious semantic priming extends to novel unseen stimuli. Cognition, 80(3), 215–229.

Newell, B. R., & Shanks, D. R. (2014). Unconscious influences on decision making: A critical review. Behavioral and Brain Sciences, 38(01), 1–19. http://doi.org/10.1017/S0140525X12003214

Noel, J.-P., Wallace, M., & Blake, R. (2015). Cognitive Neuroscience: Integration of Sight and Sound outside of Awareness? Current Biology, 25(4), R157–R159. http://doi.org/10.1016/j.cub.2015.01.007

Ocampo, B. (2015). Unconscious manipulation of free choice by novel primes. Con-sciousness and Cognition, 34, 4–9. http://doi.org/10.1016/j.concog.2015.03.007

Ocampo, B., Al-Janabi, S., & Finkbeiner, M. (2015). Direct evidence of cognitive control without perceptual awareness. Psychonomic Bulletin & Review, 22(4), 1083–1088. http://doi.org/10.3758/s13423-014-0766-3

Ortells, J. J., Kiefer, M., Castillo, A., Megías, M., & Morillas, A. (2016). The se-mantic origin of unconscious priming: Behavioral and event-related potential evidence during category congruency priming from strongly and weakly related masked words. Cognition, 146, 143–157. http://doi.org/10.1016/j.cognition.2015.09.012

Overgaard, M., Rote, J., Mouridsen, K., & Ramsøy, T. Z. (2006). Is conscious per-ception gradual or dichotomous? A comparison of report methodologies during a visual task. Consciousness and Cognition, 15(4), 700–708. http://doi.org/10.1016/j.concog.2006.04.002

Overgaard, M., & Sandberg, K. (2012). Kinds of access: different methods for report reveal different kinds of metacognitive access. Philosophical Transactions of

99

the Royal Society B: Biological Sciences, 367(1594), 1287–1296. http://doi.org/10.1098/rstb.2011.0425

Peremen, Z., & Lamy, D. (2014). Comparing unconscious processing during contin-uous flash suppression and meta-contrast masking just under the limen of con-sciousness. Frontiers in Psychology, 5. http://doi.org/10.3389/fpsyg.2014.00969

Perry, C., Ziegler, J. C., & Zorzi, M. (2010). Beyond single syllables: Large-scale modeling of reading aloud with the Connectionist Dual Process (CDP++) mod-el. Cognitive Psychology, 61(2), 106–151. http://doi.org/10.1016/j.cogpsych.2010.04.001

Persaud, N., McLeod, P., & Cowey, A. (2007). Post-decision wagering objectively measures awareness. Nature Neuroscience, 10(2), 257–261. http://doi.org/10.1038/nn1840

Pessiglione, M., Petrovic, P., Daunizeau, J., Palminteri, S., Dolan, R. J., & Frith, C. D. (2008). Subliminal Instrumental Conditioning Demonstrated in the Human Brain. Neuron, 59(4), 561–567. http://doi.org/10.1016/j.neuron.2008.07.005

Pessiglione, M., Schmidt, L., Draganski, B., Kalisch, R., Lau, H., Dolan, R. J., & Frith, C. D. (2007). How the brain translates money into force: a neuroimaging study of subliminal motivation. Science, 316(5826), 904–906.

Pessoa, L. (2005). Target Visibility and Visual Awareness Modulate Amygdala Responses to Fearful Faces. Cerebral Cortex, 16(3), 366–375. http://doi.org/10.1093/cercor/bhi115

Pessoa, L., Japee, S., & Ungerleider, L. G. (2005). Visual Awareness and the Detec-tion of Fearful Faces. Emotion, 5(2), 243–247. http://doi.org/10.1037/1528-3542.5.2.243

Pessoa, L., & Padmala, S. (2005). Quantitative prediction of perceptual decisions during near-threshold fear detection. Proceedings of the National Academy of Sciences, 102(15), 5612–5617. http://doi.org/10.1073/pnas.0500566102

Pouget, A., Beck, J. M., Ma, W. J., & Latham, P. E. (2013). Probabilistic brains: knowns and unknowns. Nature Neuroscience, 16(9), 1170–1178. http://doi.org/10.1038/nn.3495

Pratte, M. S., & Rouder, J. N. (2009). A task-difficulty artifact in subliminal prim-ing. Attention, Perception, & Psychophysics, 71(6), 1276–1283. http://doi.org/10.3758/APP.71.6.1276

Price, C. J. (2012). A review and synthesis of the first 20years of PET and fMRI studies of heard speech, spoken language and reading. NeuroImage, 62(2), 816–847. http://doi.org/10.1016/j.neuroimage.2012.04.062

Reingold, E. M., & Merikle, P. M. (1988). Using direct and indirect measures to study perception without awareness. Perception & Psychophysics, 44(6), 563–575. http://doi.org/10.3758/BF03207490

Ress, D., & Heeger, D. J. (2003). Neuronal correlates of perception in early visual cortex. Nature Neuroscience, 6(4), 414–420. http://doi.org/10.1038/nn1024

Roberts, M. E., Gibbons, F. X., Kingsbury, J. H., & Gerrard, M. (2014). Not intend-ing but somewhat willing: The influence of visual primes on risky sex decisions. British Journal of Health Psychology, 19(3), 553–565. http://doi.org/10.1111/bjhp.12055

Rogers, S. (1992). How a Publicity Blitz Created The Myth of Subliminal Advertis-ing. Public Relations Quarterly, 37(4), 12–17.

Romo, R., & de Lafuente, V. (2013). Conversion of sensory signals into perceptual decisions. Progress in Neurobiology, 103, 41–75. http://doi.org/10.1016/j.pneurobio.2012.03.007

Rosenthal, D. M. (2008). Consciousness and its function. Neuropsychologia, 46(3), 829–840. http://doi.org/10.1016/j.neuropsychologia.2007.11.012

100

Rouder, J. N., Morey, R. D., Speckman, P. L., & Pratte, M. S. (2007). Detecting chance: A solution to the null sensitivity problem in subliminal priming. Psy-chonomic Bulletin & Review, 14(4), 597–605.

Rouder, J. N., Speckman, P. L., Sun, D., Morey, R. D., & Iverson, G. (2009). Bayes-ian t tests for accepting and rejecting the null hypothesis. Psychonomic Bulletin & Review, 16(2), 225–237. http://doi.org/10.3758/PBR.16.2.225

Sandberg, K., Timmermans, B., Overgaard, M., & Cleeremans, A. (2010). Measur-ing consciousness: Is one measure better than the other? Consciousness and Cognition, 19(4), 1069–1078. http://doi.org/10.1016/j.concog.2009.12.013

Sato, W., Kubota, Y., & Toichi, M. (2014). Enhanced subliminal emotional respons-es to dynamic facial expressions. Frontiers in Psychology, 5. http://doi.org/10.3389/fpsyg.2014.00994

Schmidt, T. (2015). Invisible Stimuli, Implicit Thresholds: Why Invisibility Judg-ments Cannot be Interpreted in Isolation. Advances in Cognitive Psychology, 11(2), 31–41. http://doi.org/10.5709/acp-0169-3

Schmidt, T., & Vorberg, D. (2006). Criteria for unconscious cognition: Three types of dissociation. Perception & Psychophysics, 68(3), 489–504. http://doi.org/10.3758/BF03193692

Schoeberl, T., Fuchs, I., Theeuwes, J., & Ansorge, U. (2015). Stimulus-driven atten-tional capture by subliminal onset cues. Attention, Perception, & Psychophysics, 77(3), 737–748. http://doi.org/10.3758/s13414-014-0802-4

Scholvinck, M. L., Saleem, A. B., Benucci, A., Harris, K. D., & Carandini, M. (2015). Cortical State Determines Global Variability and Correlations in Visual Cortex. Journal of Neuroscience, 35(1), 170–178. http://doi.org/10.1523/JNEUROSCI.4994-13.2015

Seth, A. K., Dienes, Z., Cleeremans, A., Overgaard, M., & Pessoa, L. (2008). Meas-uring consciousness: relating behavioural and neurophysiological approaches. Trends in Cognitive Sciences, 12(8), 314–321. http://doi.org/10.1016/j.tics.2008.04.008

Shadlen, M. N., & Newsome, W. T. (1998). The variable discharge of cortical neu-rons: implications for connectivity, computation, and information coding. The Journal of Neuroscience, 18(10), 3870–3896.

Sklar, A. Y., Levy, N., Goldstein, A., Mandel, R., Maril, A., & Hassin, R. R. (2012). Reading and doing arithmetic nonconsciously. Proceedings of the National Academy of Sciences, 109(48), 19614–19619.

Snodgrass, M., Bernat, E., & Shevrin, H. (2004a). Unconscious perception: A mod-el-based approach to method and evidence. Perception & Psychophysics, 66(5), 846–867. http://doi.org/10.3758/BF03194978

Snodgrass, M., Bernat, E., & Shevrin, H. (2004b). Unconscious perception at the objective detection threshold exists. Perception & Psychophysics, 66(5), 888–895. http://doi.org/10.3758/BF03194982

Sperdin, H. F., Spierer, L., Becker, R., Michel, C. M., & Landis, T. (2015). Submil-lisecond unmasked subliminal visual stimuli evoke electrical brain responses: VEPs to Submillisecond Unmasked Visual Stimulation. Human Brain Mapping, 36(4), 1470–1483. http://doi.org/10.1002/hbm.22716

Stafford, T. (2014). The perspectival shift: how experiments on unconscious pro-cessing don’t justify the claims made for them. Frontiers in Psychology, 5. http://doi.org/10.3389/fpsyg.2014.01067

Stanislaw, H., & Todorov, N. (1999). Calculation of signal detection theory measures. Behavior Research Methods, Instruments, & Computers, 31(1), 137–149. http://doi.org/10.3758/BF03207704

101

Supèr, H., Spekreijse, H., & Lamme, V. A. (2001). Two distinct modes of sensory processing observed in monkey primary visual cortex (V1). Nature Neurosci-ence, 4(3), 304–310.

Swets, J. A. (1961). Is There a Sensory Threshold?: When the effects of the observ-er’s response criterion are isolated, a sensory limitation is not evident. Science, 134(3473), 168–177. http://doi.org/10.1126/science.134.3473.168

Taylor, J. S. H., Rastle, K., & Davis, M. H. (2013). Can cognitive models explain brain activation during word and pseudoword reading? A meta-analysis of 36 neuroimaging studies. Psychological Bulletin, 139(4), 766–791. http://doi.org/10.1037/a0030266

Theus, K. T. (1994). Subliminal advertising and the psychology of processing un-conscious stimuli: A review of research. Psychology and Marketing, 11(3), 271–290. http://doi.org/10.1002/mar.4220110306

Trappey, C. (1996). A meta-analysis of consumer choice and subliminal advertising. Psychology and Marketing, 13(5), 517–530. http://doi.org/10.1002/(SICI)1520-6793(199608)13:5<517::AID-MAR5>3.0.CO;2-C

Tzelgov, J., Porat, Z., & Henik, A. (1997). Automaticity and Consciousness: Is Per-ceiving the Word Necessary for Reading It? The American Journal of Psycholo-gy, 110(3), 429. http://doi.org/10.2307/1423567

Vadillo, M. A., Konstantinidis, E., & Shanks, D. R. (2016). Underpowered samples, false negatives, and unconscious learning. Psychonomic Bulletin & Review, 23(1), 87–102. http://doi.org/10.3758/s13423-015-0892-6

Van den Bussche, E., Notebaert, K., & Reynvoet, B. (2009). Masked primes can be genuinely semantically processed: A picture prime study. Experimental Psy-chology, 56(5), 295–300. http://doi.org/10.1027/1618-3169.56.5.295

Van den Bussche, E., Van den Noortgate, W., & Reynvoet, B. (2009). Mechanisms of masked priming: A meta-analysis. Psychological Bulletin, 135(3), 452–477. http://doi.org/10.1037/a0015329

Van den Bussche, E., Vermeiren, A., Desender, K., Gevers, W., Hughes, G., Verguts, T., & Reynvoet, B. (2013). Disentangling conscious and unconscious processing: a subjective trial-based assessment approach. Frontiers in Human Neuroscience, 7. http://doi.org/10.3389/fnhum.2013.00769

van Gaal, S., de Lange, F. P., & Cohen, M. X. (2012). The role of consciousness in cognitive control and decision making. Frontiers in Human Neuroscience, 6. http://doi.org/10.3389/fnhum.2012.00121

van Gaal, S., & Lamme, V. A. F. (2012). Unconscious High-Level Information Processing: Implication for Neurobiological Theories of Consciousness. The Neuroscientist, 18(3), 287–301. http://doi.org/10.1177/1073858411404079

van Gaal, S., Ridderinkhof, K. R., Fahrenfort, J. J., Scholte, H. S., & Lamme, V. A. F. (2008). Frontal Cortex Mediates Unconsciously Triggered Inhibitory Control. Journal of Neuroscience, 28(32), 8053–8062. http://doi.org/10.1523/JNEUROSCI.1278-08.2008

Van Opstal, F., Reynvoet, B., & Verguts, T. (2005). How to trigger elaborate pro-cessing? A comment on Kunde, Kiesel, and Hoffmann (2003). Cognition, 97(1), 89–97. http://doi.org/10.1016/j.cognition.2004.12.011

Vermeiren, A., & Cleeremans, A. (2012). The validity of d′ measures. PLoS ONE, 7(2), e31595. http://doi.org/10.1371/journal.pone.0031595

Wagenmakers, E., Wetzels, R., Borsboom, D., & van der Maas, H. L. J. (2011). Why psychologists must change the way they analyze their data: The case of psi: Comment on Bem (2011). Journal of Personality and Social Psychology, 100(3), 426–432. http://doi.org/10.1037/a0022790

102

Wentura, D., & Frings, C. (2005). Repeated Masked Category Primes Interfere With Related Exemplars: New Evidence for Negative Semantic Priming. Journal of Experimental Psychology: Learning, Memory, and Cognition, 31(1), 108–120. http://doi.org/10.1037/0278-7393.31.1.108

Wiens, S. (2006). Current concerns in visual masking. Emotion, 6(4), 675–680. http://doi.org/10.1037/1528-3542.6.4.675

Wiens, S. (2007). Concepts of visual consciousness and their measurement. Advanc-es in Cognitive Psychology, 3(1), 349–359. http://doi.org/10.2478/v10053-008-0035-y

Wiens, S., Fransson, P., Dietrich, T., Lohmann, P., Ingvar, M., & Arne, O. (2004). Keeping It Short: A Comparison of Methods for Brief Picture Presentation. Psy-chological Science, 15(4), 282–285. http://doi.org/10.1111/j.0956-7976.2004.00667.x

Xiao, K., & Yamauchi, T. (2014). Semantic priming revealed by mouse movement trajectories. Consciousness and Cognition, 27, 42–52. http://doi.org/10.1016/j.concog.2014.04.004

Xiao, K., & Yamauchi, T. (2015). Subliminal semantic priming in near absence of attention: A cursor motion study. Consciousness and Cognition, 38, 88–98. http://doi.org/10.1016/j.concog.2015.09.013

Zehetleitner, M., & Rausch, M. (2013). Being confident without seeing: What sub-jective measures of visual consciousness are about. Attention, Perception, & Psychophysics, 75(7), 1406–1426. http://doi.org/10.3758/s13414-013-0505-2