Lakens - Abstract Concepts in Grounded Cognition

8/14/2019 Lakens - Abstract Concepts in Grounded Cognition

1/147

ABSTRACT

CONCEPTS

IN

GROUNDED

COGNITION

DANIL LAKENS


2/147

Abstract Concepts in Grounded Cognition

Danil Lakens


3/147


4/147

Abstract concepts in grounded cognition

Abstracte concepten in gegronde cognitie

(met een samenvatting in het Nederlands)

Proeschri

ter verkrijging van de graad van doctor

aan de Universiteit Utrechtop gezag van de rector magnicus, pro. dr. J. C. Stoo,

ingevolge het besluit van het college van promoties,

in het openbaar te verdedigen op

19 januari 2010 des middags te 2.30 uur

door

Danil Lakens

geboren op 18 juli 1980 te Rotterdam


5/147

Promotor: Pro. dr. G. R. Semin


6/147

Contents

Introduction 5

Chapter 1 Grounded Cognition 11

Chapter 2 On the Grounding o Morality in

Perceptual Symmetry 55

Chapter 3 Te Sound o ime 71

Chapter 4 But or the Bad, Tere Would not

be Good 84

Footnotes 112

Appendices 113Reerences 115

Summary in Dutch 133

Curriculum Vitae 139


7/147


8/147

5

Introduction

M

orality, time, and valence are highly abstract concepts. Exactlyhow people are able to think and talk about abstract conceptsis a question that has intrigued philosophers and scientists or

centuries. In this thesis I ocus on the question how people represent themeaning o abstract concepts, by investigating whether abstract conceptsare understood through the use o metaphors. Metaphors allow peopleto think about abstract concepts in terms o concrete experiences. Forexample, the scales o Lady Justice are a metaphor or justice. Peoplemight not be able to see the uture, but it can be represented as beingahead o us. Black knights in airytales are usually evil, and the color o

their armor is a metaphoric representation o the relation they have withthe knights in shining armor. Tese and similar metaphors are not onlyused to intentionally communicate the meaning o abstract concepts,but these metaphoric representations also underlie abstract conceptualthought. How the meaning o abstract concepts is grounded in concreteexperiences is the topic o this dissertation.

Broadly speaking, two views exist on how abstract concepts arerepresented in our mind. On the one hand, amodal views share the commontheme that people think in a purely symbolic language, where concepts


9/147

6

are represented as articial symbols, and computational operations areperormed upon these symbols during conceptual processing (Dennett,

1969; Fodor, 1975; Pylyshyn, 1984; Jackendo, 2002). Tese articialsymbols are inherently non-perceptual; none o the original sensoryor motor inormation rom any modality (e.g., vision, touch, sound,smell, etc) which created these symbols is present when the nal amodalsymbols are used to represent concepts. On the other hand, several viewson the representation o concepts have emerged over the last decenniumwhich all share the idea that conceptual processing is perceptual in

nature (e.g., Barsalou, 1999; Gallese&

Lako, 2005; Glenberg&

Kaschak,2002; Jeannerod, 2001; Pulvermller, 1999; Zwaan, 2004). According tothese grounded approaches to cognition, conceptual thought consists orepresentations built on concrete sensorimotor inormation. For example,understanding the word kick is assumed to depend at least in part on theactivation o the muscles used to perorm a kicking motion.

As a result o the growing number o empirical studies thathave investigated the role o sensorimotor activation in language

comprehension, a consensus is emerging that sensorimotor areas in thebrain are activated when people read words related to concrete concepts(Zwaan, 2009; Mahon &Caramazza, 2009; Vigliocco &Meteyard, 2008).For example, thinking about the color o a banana activates brain areasin the visual cortex (Simmons, Ramjee, Beauchamp, McRae, Martin, &Barsalou, 2007). Tinking about the color o a banana leads to remarkablysimilar patterns o brain activation as actually seeing a banana.

Te extent to which abstract conceptual thought is grounded inperceptual inormation is an important theoretical question or groundedapproaches to cognition due to the lack o perceptual characteristics oabstract concepts. Which visual, motor or other sensory inormationcould in any way acilitate the understanding o abstract concepts suchas morality? Several possible explanations have been put orward toaccommodate the perceptual representation o abstract concepts in

grounded cognition (Barsalou, 1999; Boroditsky, 2000; Glenberg&

Robertson, 2000; Lako&Johnson, 1980, 1999), but whether abstract


10/147

7

conceptual processing can be accounted or by grounded approachesto cognition remains highly disputed (Arbib, 2008; Barsalou; 2008;

Boroditsky&Prinz, 2008; Lako, 2008; Mahon &Caramazza, 2008).Abstract concepts play important roles in human cognition as well

as in social interaction. Te empirical investigation o conceptual thoughthas ocused primarily on concrete concepts (or reviews, see Fischer&Zwaan, 2008; Vigliocco &Meteyard, 2008), and the question howabstract concepts are grounded and especially which process underliesthe perceptual representation o abstract concepts remains debated. Given

that many abstract concepts constitute those things in lie that peoplecare greatly about (e.g., morality, compassion, power, knowledge) it isworthwhile to investigate how people think about these concepts. Howpeople represent abstract concepts will both structure and constrain howthese concepts are understood (Lako&Johnson, 1999), and as such,dierent representations might have proound consequences or peoplesbelies and decisions. For example, Lako (1996) describes the dierentmetaphors used by liberals and conservatives to think about morality, and

highlights the consequences o these dierent representations o moralityin politics.

Overview

In the present dissertation, I examine the role o sensory inormationor abstract conceptual thought. In Chapter 1, the central assumptions

in grounded approaches to cognition are introduced and contrastedwith more traditional views on the representation o concepts expressedby cognitive researchers since the cognitive revolution in psychology.Aer detailing the dierent theoretical views on cognition, I present anoverview o studies investigating whether and how abstract conceptsare represented perceptually (including the studies reported in the threeempirical chapters o this thesis). Tis review is concluded by a discussion

o what these empirical ndings reveal about how the meaning o abstractconcepts is grounded in concrete experiences. Alternative explanations


11/147

8

or the research ndings, based on structural and linguistic eatures othe paradigms, are discussed. Finally, I provide my own interpretation o

what the ndings on the relationship between perception and cognitionreveal about abstract conceptual thought.

Te empirical investigation o how abstract concepts are groundedin perceptual inormation is gaining popularity, but constitutes a relativelynew eld o interest. Many studies have aimed to demonstrate thatperceptual inormation plays a role in conceptual processing, but there areonly a ew theories to explain these ndings, and an integrative ramework

to account or the empirical results is currently not available. Researchershave noted that grounded theories o cognition need urther developmentto account more precisely or the eects observed in the literature (Zwaan,2009). In addition, the mechanisms which are assumed to underlieperceptual representations should be specied in more detail to be able toempirically test the predictions made by these theories (Boroditsky, 2000).Investigating the possible limitations o grounded approaches to cognition,such as whether abstract concepts can be represented perceptually, in

which modalities people can represent abstract concepts, as well as theinuence o the context in which perceptual inormation is presented, areall important steps to achieve a more detailed understanding o whetherand how perceptual inormation structures abstract concepts. Te goalo the present dissertation is to investigate these questions empirically,thereby contributing to the development o an integrative theoreticalramework o the role o sensorimotor inormation in the grounding o

abstract concepts.Te three empirical chapters o this thesis ocus on how morality,

time, and valence are grounded in perceptual symmetry, le-right auditoryspace, and brightness, respectively. Te aim o Chapter 2 is to show thatmorality, one o the hallmark abstract concepts assumed to lack theperceptual characteristics to be grounded in sensorimotor inormation,can be represented perceptually. Based on the assumption that equality is

an important eature o morality, and the equality component o moralityis argued to be metaphorically represented in balance and symmetry, I


12/147

9

predicted and ound that moral concepts show a stronger associationwith perceptually symmetrical stimuli compared to immoral concepts.

Chapter 3 increases the scope o perceptual grounding byproviding support or the theoretical prediction that abstract conceptscan be structured in the auditory modality. Earlier research into thegrounding o abstract concepts has ocused predominantly on the visualmodality. Since the auditory modality shows a similar sensitivity to spatialinormation as the visual modality, the previously observed structuring otime in space should extend to the auditory domain. ime is represented

rom le to right visual space, with the past to the le and the uture to theright, and the same structuring was expected to be observed in auditoryspace. Participants were asked to indicate whether words were presentedlouder in the le or the right ear while passively listening to time relatedwords. Critical experimental trials were presented equally loud to bothears. Future words were more oen judged to be louder in the right earthan past related words were, supporting the le to right representationo time in auditory space. Furthermore, the visual representation o time

and the auditory representation o time were shown to be signicantlyrelated, providing additional support or the multimodal grounding oabstract concepts.

In Chapter 4, two process explanations concerning themetaphorical representation o abstract concepts were tested againsteach other. Tese studies provide a rare theoretical comparison o twoprocesses that have been suggested to underlie the grounding o abstract

concepts. Whereas previous researchers (Meier, Robinson, & Clore,2004; Sherman &Clore, 2009) have argued that visual inormation (e.g.,the color white) will automatically and obligatorily activate an abstractmeaning (e.g., positivity), these studies support a more active metaphoricstructuring process. Te metaphoric representation o positivity as whitedepends on whether the conceptual opposition between good and badand the perceptual opposition between white and black are activated.

Tus, the shared relational structure between the concrete dimension andthe abstract dimension is crucial or the metaphoric representation o


13/147

10

valence in brightness to emerge.ogether, these chapters show that perceptual inormation

inuences abstract conceptual processing, even or highly abstractconcepts that lack perceptual characteristics. Some abstract concepts,such as time, seem to consist o multimodal representations, and judgments in the visual and the auditory domain are inuenced bythese metaphoric representations in a similar way. Nevertheless, sharedrelational structures (such as polar oppositions) between the abstract andconcrete dimension can be an essential pre-requisite or the metaphoric

representation o abstract concepts to become activated. Further researchshould explore the circumstances under which the activation o sharedrelational structures is necessary or metaphoric mappings to emerge, andthese attentional processes should be taken into account in uture studiesthat aim to investigate how abstract concepts are grounded.


14/147

Chapter 1

Grounded Cognition


15/147


16/147

13

Wenn wir der Krperwelt, welche unmittelbar nur in unsererVorstellung dasteht, die grte uns bekannte Realitt beilegen

wollen; so geben wir ihr die Realitt, welche r Jeden seineigener Leib hat: denn der ist Jedem das Realste.

Schopenhauer, WWR 1 105.

When people think about highly abstract concepts, they draw

upon concrete experiences to structure their thoughts.Due to the increasing popularity o grounded approachesto cognition, the question how concrete experiences shape abstractthoughts has received renewed attention over the last decennium.Grounded approaches to cognition argue that sensorimotor experiencesplay an important role when people think about concepts. Sensorimotorinormation stored in visual, auditory, and motor areas in the brain whenpeople directly experience the world is activated when they think about

the world at a later time. For example, when you think about what theearth looks like rom outer space, visual areas become activated in asimilar way as when you would see a blue and green ball covered withwhite dots o clouds. Even though people cannot directly see, hear ortouch abstract concepts, and thereore have not directly experiencedsensory inormation that is related to the meaning o abstract conceptssuch as morality, perceptual inormation is theorized to play an essential

role in abstract conceptual thought (Barsalou, 1999; Glenberg&Kaschak,2002; Gallese &Lako, 2005).

Such a grounded approach to cognition stands in stark contrastto the dominant views expressed by researchers since the cognitiverevolution in psychology. raditional theories o conceptual processinggenerally do not assume that conceptual thought is perceptual in nature,but argue or (or implicitly assume) a purely symbolic language o thought

(Anderson, 1990; Dennett, 1969; Fodor, 1975; Pylyshyn, 1984; Jackendo,2002), which consists o articial symbols. Tese symbolic (or amodal)


17/147

14

theories draw a sharp distinction between perception and cognition, andposit that perceptual inormation processing and conceptual thought are

undamentally dierent processes. Essentially, thoughts are independentrom our experiences with the world. A symbolic representation o thecolor red is not the equivalent o the sensory experience o the color red(Pylyshyn, 1984).

Symbolic representations are theorized to exist independentlyrom sensory experiences. Concrete sensory inormation is encoded intosemantically interpretable syntactic structures that do not longer retain

the perceptual inormation rom which they originate. Tis encodingis not unlike what happens when I press the d key on my keyboard.Although the letter that appears on the screen resembles the letter d,this letter is represented in a completely articial and symbolic logicalcode by the computer. Tis becomes apparent when this le is openedin a simple text editor, and the letter d is represented somewhere in thisstring o symbols: m?6UK8. Not only is the internalrepresentation o the input completely symbolic, but the relation o the

perceptual input (d) and the internal representation is arbitrary. Anysymbol can be used to represent the letter d. Tis becomes apparentwhen the ont is changed rom imes to Wingdings, where the input othe letter d is mapped to the symbol d.

According to proponents o symbolic symbol systems,computations perormed upon symbolic representations are the essence omental processes (Fodor, 1980; Pylyshyn, 1984). An important advantage

o amodal theories is that they allow dierent modalperceptual processesto be integrated at the symbolic level o representation. Te term modalis used to indicate that these perceptual processes are domain specic: orexample, visual, auditory, and haptic perceptual inormation is processedin dierent brain areas. Modal perceptual processes are ast, bottom-upand isolated rom much o our knowledge about the world (Fodor, 1985).Higher cognitive processes, on the other hand, are assumed to be slow,

neither bottom-up nor top-down (inormation can ow either way) andwell-connected, so these processes can integrate sensory inormation


18/147

15

rom dierent modal domains.One o the most important (and most dicult to answer)

challenges or symbolic symbol systems is how symbolic representationsare connected to their reerents, objects in the world. Tis challenge isknown as the symbol grounding problem (Harnad, 1990; Searle, 1980).Where do amodal symbols get their meaning rom, i their only sourceso meaning are other meaningless amodal symbols? Te meaning osymbolic representations has to be grounded in the perceptual experiencesthey reer to (e.g., the symbolic representation o the color red has to be,

in some way, related to the perceptual experience o seeing the color red),but so ar, no ully satisactory solution to the symbol grounding problemhas been provided. A second open question or symbolic symbol systemsis a detailed process explanation that describes the mechanisms throughwhich symbolic representations are created rom perceptual inormation.How exactly is the sensory experience o the color red transduced,interpreted and encoded into an amodal symbol? Finally, the absence oempirical support or the presence o amodal symbols in the brain has

been mentioned as a problematic issue or symbolic symbol systems(Barsalou, 1999). ogether, these three open questions warrant the careulconsideration o alternative approaches to cognition.

Harnad (1990) proposes a hybrid account oiconic representationso entities (which consist o analogue transormations o the sensory-motor experiences when encountering the entity) complemented bycategoricalrepresentations (which consist o symbolic abstractions rom

all iconic representations people have). Importantly, both iconic andcategorical representations are non-symbolic. Te original sensory-motorinormation that was activated when a concrete entity was encounteredis preserved due to analogue transormations. Although such a hybridsystem, consisting o both perceptual memories and amodal symbols,provides a solution to the symbol grounding problem by proposing thatamodal symbols activate perceptual memories during comprehension,

one might wonder whether there is still anything le to do or amodalsymbols in such a model.


19/147

16

Barsalou (1999, 2008a) proposes an exclusively perceptual symbolsystem where conceptual knowledge consists o modal representations.

Tis grounded approach to the representation o concepts assumes thatwhen concrete objects are perceived, the sensory-motor activation inthe brain is captured in local association areas, which are subsequentlyintegrated into multi-modal representations by higher-order cross-modal associations (or convergence zones, Damasio, 1989). Convergencezones can integrate inormation rom dierent modalities. Contraryto symbolic representations, the inormation in convergence zones has

retained the perceptual inormation o the concrete experiences themodal representations reer to, thus perorming the integrative unctiono symbolic representations while retaining the connection to theperceptual input. Te multimodal associations in convergence zones (orsimulators) can be partially activated during language comprehension,leading to neural reenactments (or simulations) that activate the samesensory and motor areas in our brain as were active when we directlyexperienced its reerent. For example, the representation o the color red

has recently been shown to activate the same visual areas in the brain whenwe directly perceive the color red, as when we think about it (Simmons etal., 2007). Studies like these provide support or the a-priori predictionsderived rom grounded theories o cognition. I a computer would workas our brain works, the letter d on a computer screen would be storedas a picture and a vinyl record, and the computer could play the soundrecording on a record player or take the picture out o a drawer when it

is needed.Similar theoretical views have appeared in recent years (e.g.,

Gallese &Lako, 2005; Glenberg &Kaschak, 2002; Jeannerod, 2001;Pulvermller, 1999; Zwaan, 2004) which all share the idea that conceptualprocessing is built upon and activates sensory-motor processes. Accordingto these approaches to cognition, conceptual thought is undamentallyperceptual. More traditional approaches to cognition have acknowledged

the possibility that people use imagery to acilitate conceptual processes(e.g., Fodor, 1975), but these theories share the basic assumption that


20/147

17

conceptual thought is not perceptual in nature. Perceptual imagery canhelp people to think about concepts, but perceptual inormation is not

necessary. Tereore, the real debate between modal and amodal symbolsystems is not about whether perception and cognition are related, buthow they are related. As Fodor (1985) summarizes: No one in his rightmind doubts that perception interacts with cognition somewhere. Whatsat issue [] is the locus o this interaction. According to amodal symbolsystems, perceptual inormation might play a marginal role in conceptualthought, but these images are generated based on symbolic representations

(Kosslyn, 1980). According to modal symbol systems, conceptual thoughtconsists o the reactivation o multimodal representations stored duringdirect experiences with its reerents (Barsalou, 1999; Glenberg&Kaschak,2002).

With the emerging popularity o grounded approaches tocognition, the question where and how perception and cognition interacthas received an increasing amount o attention rom a variety o scienticdisciplines such as linguistics, cognitive science, and social psychology.

Te main goal o these investigations is to show that conceptual processingactivates experientially consistent sensorimotor areas. For example, Hauk,Johnsrude and Pulvermller (2004) show that reading the words pick,kick and lick do not only activate areas in the brain related to languageprocessing, but that these words also activate motor areas in the brainrelated to hand movements, leg movements, and mouth movements,respectively. Te empirical support or intimate relationship between

sensorimotor inormation and the processing o concrete concepts ismounting (or a recent review, see Meteyard &Vigliocco, 2008).

Research based on a grounded approach to cognition has providedalternative ways to think about conceptual thought. First o all, ourminds and bodies are not undamentally distinct, as symbolic theorieso cognition presuppose. Te widespread bodily inuences on socialcognitive processes (or reviews, see Niedenthal, Barsalou, Winkielman,

Krauth-Gruber,&

Ric, 2005; Semin&

Smith, 2008) is directly at oddswith the claim that an examination o bodies and their workings will


21/147

18

never bring us to the subject matter o the mind at all (Dennett, 1969,p. 189). Instead, the insights rom grounded cognition are in line with

the theoretical view that bodies shape our minds (e.g., Lako&Johnson,1980; Varela, Tompson, &Rosch, 1991). Second, these studies revealthat symbolic computations are inuenced by perceptual inormation.Whereas in computational models o cognition the symbolic codesthemselves do not speciy their intended interpretation (and the modelsbehavior is not inuenced by such an interpretation) (Pylyshyn, 1984,p. 42), research has shown that perceptual inormation does inuence

cognitive processes such as language comprehension (or a review, seebelow).Despite the act that many a-priori predictions rom grounded

approaches to cognition have received empirical support, groundedcognition has not been without its criticisms. Some researchers haveargued that the activation o sensorimotor inormation does occurduring conceptual processing, but only to contribute to the ullrepresentation o a concept (see Mahon &Caramazza, 2008). As such,

sensorimotor inormation is theorized to complement amodal symbolicrepresentations. Te current empirical investigations into the relationshipbetween perceptual inormation and conceptual processing do notprovide a univocal answer to question whether conceptual processingnecessarily relies on perceptual inormation. Nevertheless, the causalinuence o manipulated bodily states and sensorimotor activation onlanguage processing (e.g., Glenberg & Kaschak, 2002; Pulvermller,

Hauk, Nikulin, &Ilmoniemi, 2005) supports the idea that the activationo perceptual inormation has a more important unction than simplyenriching amodal representations (see Fischer &Zwaan, 2008; Zwaan,2009). Concluding, neither the presence o modal symbols, nor thenecessary role o sensorimotor activation in language comprehensionshas received undisputed empirical support. Although these questionslie outside o the scope o the current thesis, the exact nature o mental

representations and the possible limitations o grounded approaches tocognition remain important questions or urther research.


22/147

19

A second source o criticism on grounded approaches tocognition has been the apparent impossibility to represent abstract

concepts perceptually which brings us to the topic o the currentdissertation. Abstract concepts (e.g., virtue, morality, or time) have noperceptual characteristics, since they cannot be seen, heard, touchedor directly perceived in any other sense. Amodal representations havethereore been argued to be the only logical answer to the question howabstract concepts are represented (Dove, 2009; Fodor, 1975; Mahon &Caramazza, 2008). wo possible approaches to accommodate perceptual

representations o abstract concepts in grounded theories o cognitionhave been put orward in the literature. Te rst approach consists ometaphorically mapping abstract target concepts on perceptual sourcedimensions (Boroditsky, 2000; Lako & Johnson, 1980, 1999). Tesecond approach suggests that abstract concepts can be represented insimulations o situations or actions (Barsalou, 2008b; Barsalou&Wiemer-Hastings, 2005; Glenberg &Robertson, 1999). Both these theoreticalapproaches propose that much o the conceptual content o abstract

concepts is provided by either metaphoric representations or perceptualsimulations, and is thus inherently perceptual. Te possible objectionsagainst symbolic approaches to cognition such as the symbol groundingproblem and the lack o empirical support or the existence o amodalsymbols, together with the increasing amount o empirical support orthe perceptual representation o concrete concepts, warrants the careulexamination o possible ways in which abstract conceptual thought can

be accommodated rom a grounded cognition perspective.

Grounding Concrete Concepts

Te literature regarding the grounding o concrete concepts is too broadto discuss in much detail in this dissertation, but in order to be able toappreciate the novel insights that grounded approaches to cognition

have provided, I will describe several studies that have urthered ourunderstanding o the nature o conceptual representations. In addition,


23/147

20

I will ocus on several studies that have provided relevant insights or thequestion how abstract concepts are grounded, and on studies that have

used paradigms that have subsequently been adapted to investigate thegrounding o abstract concepts.

Recent well-designed studies provide a good example othe empirical support or the theoretical assumption that languagecomprehension automatically activates motor processes (Zwaan &aylor, 2006; aylor &Zwaan, 2008; or a review, see Fischer &Zwaan,2008). In these studies, participants read sentences such as: Because

the music was too loud, he turned down the volume. Only one to threewords o the complete sentence were presented on the screen at one time.Participants could read the entire sentence by rotating a knob, with each 5degrees o rotation resulting in the presentation o the next segment. Tereading-rotation direction either matched or mismatched the rotationdirection implied by the sentence (e.g., turning down the volume consistso a counter-clockwise movement). When participants had to read thesentences describing counter-clockwise movement by moving the knob

clockwise, they rotated to knob more slowly compared to participants whocould read the sentence by rotating the knob counter-clockwise (Zwaan&aylor, 2006), but only on the segment o the sentence that containedthe verb. Apparently, reading a description o a movement incongruentto the movement people are currently perorming slows down readingtimes. A subsequent study revealed this eect occurs as long as the actionis within linguistic ocus, and extends to the subsequent adjectives such

as opened slowly(aylor &Zwaan, 2008).Tere have been many more studies showing that semantic

processing accesses the motor system. For example, participants wereaster to categorize sentences toward the body (such as open the drawer)or away rom the body (such as close the drawer) as sensible (vs. notsensible) when the button participants had to press required a congruentmovement toward or away rom the body (Glenberg &Kaschak, 2002).

Pulvermller (2005) showed that people were aster to identiy leg-relatedwords (compared to arm-related words) when then le-hemisphere leg


24/147

21

motor areas were stimulated using transcranial magnetic stimulation.When arm motor areas were activated, participants were aster to identiy

arm related words. Tese and similar studies all support the idea thatmotor areas in the brain are activated when people process words relatedto body movements, and words describing objects that can be touchedand manipulated.

Beyond the motor domain, several studies provide support orthe activation o modality specic brain areas when words or sentencesrelated to visual or auditory domains are processed. Pecher, Zeelenberg

and Barsalou (2003, 2004) investigated the assumption that perceptualproperties o concrete objects are represented in modality specic areasin the brain. In a series o studies, they show that switching rom arepresentation in one modality to a representation in a dierent modalitytakes time. Using a property verication task (Is a lemon yellow, yes orno?), they show that people are aster to make judgments about objectswhen the modality that is reerred to in the question (e.g., the visualmodality) is the same as in the previous question, than when the question

reers to a dierent modality (e.g., auditory: Do microwaves beep, yesor no?). People were aster to indicate a lemon was yellow aer judgingan apple is green, compared to judging whether a lemon is yellow aerjudging that leaves are rustling. Tese ndings are taken as support orthe assumption that veriying the properties o concrete objects doesnot rely on amodal symbolic representations, but requires the activationo perceptual simulations. Although property verication tasks require

explicit judgments about stimuli (which make them less conclusive aboutthe spontaneous recruitment o modality specic representations) theysupport the assumption that dierent sensory domains are activatedwhen answering questions about concrete concepts (see also Van Dantzig,Pecher, Zeelenberg, &Barsalou, 2008).

Processing concrete and abstract verbs has been shown to driveattention to specic spatial locations. In a norming study, Richardson,

Spivey, Edelman, and Naples (2001) showed that people share anagreement about how verbs should be represented in space. For example,


25/147

22

give is represented in horizontal space, whereas respect is representedin vertical space. Building on predictions rom linguistics (Lako, 1987;

Langacker, 1987; almy, 1983) and grounded cognition (Barsalou, 1999;Glenberg &Robertson, 2000), the spatial representations associated withthese words were expected to be automatically activated during languageprocessing in a set o studies by Richardson, Spivey, Barsalou, and McRae(2003). Processing a sentence containing a horizontally represented verb(e.g.,give) was expected to activate visual representations in the le-rightdimension, interering with the recognition o a stimulus presented on the

le or right side o the screen. Sentences containing vertically representedverbs (e.g., respect) were expected to interere with the recognition o asubsequently presented stimulus up or down on the screen. Te predictedndings were observed, and taken as support or the assumption that thesame visual resources are used to represent the meaning o the sentenceand to perceive the subsequent stimulus (or similar ndings, see Estes,Verges, &Barsalou, 2008).

Tese results have subsequently been replicated or concrete

verbs, but not or abstract verbs (Bergen, Lindsay, Matlock, &Narayanan,2007). Furthermore, this last set o studies revealed that words associatedwith vertical or horizontal space were not enough to lead to intererenceeects, but that the results were caused by mental imagery. Importantly,building a mental simulation can also acilitate the processing o visuallycongruent inormation (Glenberg &Kaschak, 2002; Kaschak et al., 2005;Staneld &Zwaan, 2001; Zwaan, Staneld, &Yaxley, 2002). Te critical

dierence between intererence and acilitation seems to be the length otime between the presentation o the linguistic stimulus and the perceptualstimulus. I the visual brain areas are still busy representing the linguisticstimulus (e.g., between 50 to 200 milliseconds) then intererence is morelikely, but aer the visual brain areas are no longer active, the previousvisual representation o vertical or horizontal locations might acilitaterecognition (or a discussion, see Bergen et al., 2007).

In addition to the eects o language processing driving visualattention, theoretically related studies have investigated the opposite


26/147

23

relationship between visual perception and language processing. Whenparticipants were asked to judge whether two words were semantically

related, their relative positions on the screen inuenced the speed withwhich these judgments were perormed (Zwaan &Yaxley, 2003). Moreprecisely, when the relative position o these words was congruent withtheir typical locations in the real words, judgments were aster comparedto when the words were presented in relative spatial positions incongruentwith their typical locations. When the words branch and root werepresented on the screen, participants were aster to judge the words were

related when branch was presented vertically above root, compared towhen rootwas presented above branch. Similar results have been shownwhen only one word (e.g., eagle) was being presented either on the topor the bottom o the screen (eti &Domijan, 2007). In these studies,words are assumed to activate perceptual representations, with attentionbeing ocused on the respective regions. Te relative spatial position othe words is either congruent or incongruent with the presentation o thestimulus words, thus resulting in a spatial Stroop eect (Lu &Proctor,

1995). In urther support o this reasoning, similar eects are not observedwhen pictorial (compared to verbal) stimuli o objects typically seen upor down in the world are processed, since these pictorial stimuli do nothave to be visualized to be represented (Verges &Duy, 2009). A similarreasoning and paradigm has been used extensively in abstract domains(e.g., Meier &Robinson, 2004; Schubert, 2005).

aken together, the many empirical demonstrations o

sensorimotor involvement in thoughts about concrete concepts revealthat conceptual processing and sensorimotor inormation are intimatelyrelated. Important questions still remain, such as the question whethermotor activation is necessary or language understanding, or whetherthe activation o sensorimotor systems during semantic processing isoptional (Fischer &Zwaan, 2008; Meteyard &Vilgliocco, 2008; Zwaan,2009). In addition, people can sometimes rely on purely linguistic

inormation to understand the meaning o words, without drawing onsensorimotor brain areas (see or example Andrews, Vigliocco,&Vinson,


27/147

24

2009; Boroditsky&Prinz, 2008; Simmons, Hamann, Harenski, Hu, &Barsalou, 2008). Nevertheless, the grounded cognition approach seems to

be a ruitul theoretical ramework to investigate conceptual processing both or concrete as abstract concepts.

Grounding Abstract Concepts

Highly abstract concepts such as morality, truth, and beauty cannot be seenor heard. Because such abstract concepts lack perceptual characteristics,

some researchers have argued that perceptual representations seemill-suited or representing abstract concepts (Dove, 2009). Te muchdebated question o whether and how abstract concepts are representedperceptually remains a challenge or grounded approaches to cognition(Arbib, 2008; Barsalou; 2008; Boroditsky & Prinz, 2008; Dove, 2009;Lako, 2008; Mahon & Caramazza, 2008). I grounded theories ocognition ail to account or the perceptual representation o abstractconcepts, grounded cognition would be a rather limited ramework or

conceptual thought, applying exclusively to concrete concepts (Mahon&Caramazza, 2008). Tereore, providing a theoretical account o howabstract concepts are grounded, as well as empirical support or theperceptual representation o abstract concepts are both essential or amore mature grounded cognition ramework.

Several theoretical proposals have been made to accommodateabstract concepts in grounded approaches to cognition (Barsalou, 1999,

2008; Barsalou &Wiemer-Hastings, 2005; Boroditsky, 2000; Glenberg&Robertson, 1999; Gallese &Lako, 2005; Lako& Johnson, 1980,1999; Meier & Robinson, 2004; Richardson, et al., 2003). Althoughthe perceptual representation o abstract concepts has received lessempirical attention than the perceptual representation o concreteconcepts, the predictions drawn rom these theories have been testedin a decent number o studies. Tese theoretical rameworks are oen

complementary (Glenberg, Sato, Cattaneo, Riggio et al., 2008), and ocuson the simulation o perceptual inormation, relating abstract events to


28/147

25

actions, and metaphoric representation.According to the simulation account (Barsalou, 1999, 2008a;

Richardson et al., 2003) people create images o situations, objects andintrospections to perceptually simulate specic instances o abstractconcepts. Trough selective attention, an unlimited number o simulatorscan develop or objects (e.g., car), properties (e.g., red), relations (e.g.,above), actions (e.g., driving) or introspective states (e.g., ear). Tesesimulators makes the implementation o symbolic operations possible (oran overview, see Barsalou, 2008b). Even abstract concepts such as truth,

which seems to lack any perceptual characteristics to build a simulation o,can be perceptually simulated in such a way (Barsalou, 1999). o veriy thetruth o a proposition (e.g., the balloon is above the cloud) people createand image o a balloon above a cloud and subsequently compare whetherthis image corresponds to their experience. Aer repeated experiences othis comparison process, a simulator develops which grounds the concepto truth.

Another proposal aimed at explaining how abstract concepts

are grounded ocuses on how certain abstract events can be groundedin movements. Te indexical hypothesis (Glenberg &Robertson, 1999)proposes that sentences are understood by creating a simulation o theactions that underlie them. It seems to matter little whether these actionsare concrete (e.g., giving a book) or abstract (e.g., giving inormation).Glenberg and Kaschak (2002) asked people to judge the sensibility osentences describing the transer o concrete objects such as Andy

delivered the pizza to you/You delivered the pizza to Andy and abstractinormation transer, such as Liz told you the story/You told Liz thestory. Participants responded by moving their hand rom a middle buttonto either a button arther away or closer by. Judgments were aster whenthe action implied by the sentence matched the action required to makethe response, and this was true or both the concrete and the abstracttranser sentences. A similar study showed that moving 600 beans rom

one container to the other (either away or towards the body) inuencedsentence comprehension in a subsequent task o sentences describing


29/147

26

concrete and abstract transer away rom or towards the body (Glenberg,Sato, &Cattaneo, 2008).

A nal approach to grounding abstract concepts draws upon theseminal work by Lako and Johnson (1980, 1999), who have proposedthat abstract concepts are grounded in concrete experiences throughmetaphoric representations. People use metaphors abundantly when theytalk about abstract concepts. Te Conceptual Metaphor Teory (CM)introduced by Lako and Johnson (1980, 1999) holds that metaphors arenot merely conventional ways to talk about abstract concepts, but that

abstract conceptual thought is inherently metaphorical (c. Richards,1936). According to CM, people use concrete experiences (or sources)to think about more abstract concepts (or targets). When concreteexperiences in the source domain repeatedly co-occur with more abstractexperiences in the target domain, the two brain areas that are activatedtogether will over time orm neural mappings (Lako, 2008), whichconstitute so-called primary metaphors (Grady, 1997).

An example o such a primary metaphor is happy is up. Lako

and Johnson (1980) argue that positive states are usually accompaniedby erect postures, whereas negative states coincide with a droopingposture, thereby experientially grounded the relationship between theconcrete domain o vertical space and the abstract domain o valence. Teassumption o CM that abstract thought is built on concrete experiencesis relatively undisputed (or similar reasoning, see Clark, 1973; James,1890; Piaget, 1954; Richards, 1936; Whitney, 1875). CM has inspired

many researchers to investigate whether abstract concepts are representedperceptually. Although CM provides compelling theoretical argumentsor a strong relation between concrete experiences and abstract concepts,even the updated neural theory o metaphor (Feldman, 2006; Lako, 2008)ails to clearly speciy how basic image schemas, personal experiences,contextual variations, and overall knowledge collectively determinewhich neural mappings are activated in which situation. Tis makes it

dicult to predict how abstract concepts will be grounded.Conceptual Metaphor Teory has previously been criticized or


30/147

27

lacking a more detailed model o how the relationship between concreteconcepts and abstract concepts is established (Murphy, 1996; Boroditsky,

2000; McGlone, 2007). Te recent neural theory o metaphor put orwardby Lako (2008) is the most explicit account o the processes underlyingCM so ar, and proposes that neural mappings between concreteexperiences and abstract concepts emerge through peoples experiences,and these neural mappings are combined into larger neural circuitsthat allow us to ground metaphoric language. However, two criticismsthat have been put orward previously still apply (e.g., Murphy, 1996),

namely the strong ocus on linguistic evidence (see also Casasanto, inpress; Craword, 2009) and the lack o detail o the theory, which makes itdicult or psychological researchers to derive clear testable predictions(Boroditsky, 2000). wo interpretations o CM have been put orwardthat allow the underlying assumptions to be tested empirically. Althoughboth views are derived rom CM, they highlight dierent aspects o theprocesses assumed to underlie the metaphoric representation o abstractconcepts. Below, I will introduce these two views in greater detail. Chapter

4 provides a set o studies that aim to test the assumptions o thesedierent views on the metaphorical representation o abstract conceptsagainst each other.

Metaphoric Structuring View

Te Metaphoric Structuring View was developed by Boroditsky (2000)

to provide an empirically testable theory derived rom the work on CMby Lako and Johnson (1999). Te main assumption o the MetaphoricStructuring View is that metaphors provide relational structure to thosedomains where the structure may not be obvious rom world experience

(Boroditsky, 2008, p. 3). Abstract domains are argued to lack a clearrelational structure. For example, whereas moments in time seem toappear, disappear and never return, time does not contain concrete

inormation about where moments in time come rom and where theygo to. People can move through time themselves, or time can approach


31/147

28

them. ime can ow rom the le to the right, or rom our ront to back.Metaphors allow us to structure, among others, the abstract concept o

time by mapping time onto concrete dimensions such as space. Accordingto the Metaphoric Structuring View, metaphors in language such aslooking orward to next week inuence how we structure time in space.

Boroditsky (2000) suggests that metaphors have shaped theway we think about time, but aer requent use, metaphors can lead toindependent representations that no longer rely on spatial schemas. Whichspecic metaphoric representation o time is used depends upon the on-

line mapping o spatial schemas onto the domain o time. Priming peoplewith an ego-moving spatial scenario (where objects move through space)acilitates responses or time questions ramed in an ego-moving schema(compared to a time-moving scenario, where time approaches objects).Similar exibility in the spatial representation o time was shown in across-cultural study comparing Mandarin speakers with English speakers(Boroditsky, 2001). In Manderin there is a vertical metaphor or time(comparable to the uncommon English metaphor o a meeting coming

up and handing down knowledge rom generation to generation, seealso Lindner, 1983), and thus Chinese speakers tend to represent timeas moving rom the top to the bottom. Importantly, English speakerstrained in the vertical representation o time showed a similar inuenceo spatial primes on time questions as Manderin speakers, revealing thatmetaphoric representations can be learned.

Te Metaphoric Structuring View highlights the importance o

the spatial schemas used to think about time, and shows that metaphoricrepresentations o abstract concepts are highly contextualized. Dierentrepresentations o time can be primed (Boroditsky, 2000), trained(Boroditsky, 2001) or changed depending on peoples mindset (Boroditsky&Ramscar, 2002), and subsequently inuence how time is structured inspace. Tis suggests that dierent representations o abstract conceptsexist and can be used to structure the abstract domain. Such exibility

in the spatial representation o time is dicult to account or by moreautomatic views on metaphoric representation such as the neural theory


32/147

29

o metaphor, which largely rely on stable neural connections betweenconcrete experiences and abstract concepts (Feldman, 2006; Lako, 2008).

Further support or the exibility o spatial representationscomes rom a recent set o studies revealing that whereas the ront-backrepresentation o time is the deault representation, introducing leand right response keys activates the le-right representation o time(orralbo, Santiago, &Lupiez, 2006). Tese authors argue that attentionplays an important role in the exible on-line representation o abstractconcepts such as time. Aer abstract concepts and concrete dimensions

have repeatedly been mapped onto each other, these mappings arestored in long-term memory (c. Boroditsky, 2000). Tese mappings aresubsequently used when people try to construct the most parsimoniousmental model (c. Johnson-Laird, 1983) to support conceptual processing.

Stroop-like Intererence Eects

A more recent translation o Conceptual Metaphor Teory to an

empirically testable hypothesis was initiated by Meier, Robinson andClore (2004). Tey reason that i the neural networks that are active whenpeople represent abstract concepts are intimately bound with the neuralnetworks that are active when people represent sensory experiences(Lako, 2008; Lako& Johnson, 1999), there should be an obligatoryand automatic relationship between perceptual inormation and abstractconcepts. Lako (1993) appears to endorse this view when he suggests that

the system o conceptual metaphors is used constantly and automatically,with neither eort nor awareness (pp. 227228).

Meier and colleagues (2004) investigated whether the abstractconcepts o positive and negative valence are metaphorically representedin the concrete experiences o light and dark. Teir results reveal thatpositive words presented in a white ont and negative words presented ina black ont word were evaluated aster and more accurately compared

to positive words presented in a black ont and negative words presentedin a white ont. In other words, metaphor congruent valence-brightness


33/147

30

combinations were processed more eciently than metaphor incongruentvalence-brightness combinations.

Metaphor congruency eects are argued to be the result o aStroop-like intererence eect. Meier et al., (2004, p. 86) argue that:a negative word presented in white would give rise to two responsetendencies, one to respond negative (on the basis o stimulus valence)and one to respond positive (on the basis o stimulus color). A similarreasoning is provided by Schubert (2005), who investigated the verticalrepresentation o power. He asked participants to categorize words

reerring to powerul and powerless groups on power, and ound thatwords reerring to powerul groups presented on the top o the screenwere categorized aster compared to when these words were presentedon the bottom o the screen. According to Schubert (2005, p. 18), Teresponse that a presented group is powerul or powerless has a certainthreshold, and evidence is accumulated until that threshold is reached.Te evidence comes rom multiple dimensions, and vertical position othe group label and vertical motor images are among these dimensions.

According to this Stroop-like intererence explanation ormetaphor congruency eects, which I shall reer to as the one-to-one

grounding explanation, the abstract concept o positivity is groundedin the sensory experience o seeing a light stimulus, and negativity isgrounded in the sensory experience o darkness. As such,seeing the colorwhite is assumed to automatically activate its associated valence (e.g.,positivity), which in turn can acilitate or interere with the categorization

o the conceptual meaning o the word(see also Schubert, 2005; Meier&Robinson, 2004, Sherman &Clore, 2009). Tese brightness-valenceassociations are argued to be unavoidable at the stage o word encoding,and point to the obligatory nature o aective inerences based onstimulus brightness (Meier et al., 2004, p. 85; see also Meier &Robinson,2004).

Te one-to-one grounding explanation does not yet provide a

detailed explanation o when and how perceptual inormation inuencesabstract conceptual processing, and cannot account or all observed


34/147

31

empirical results. For example, Schubert (2005) shows that words such asdictator or tyrant, which are powerul and negative, are processed aster

on the top o the screen when participants have to categorize these wordon power, but are processed aster on the bottom o the screen when thesewords are categorized on the basis o their valence. Metaphor congruencyeects seem to depend upon the current categorization goals, speakingagainst a purely automatic activation o word meaning. Relating metaphorcongruency eects to Stroop-like intererence eects is tting in thisrespect, because the assumption that Stroop-like eects are completely

automatic does not seem to be the most parsimonious explanation orthe eects observed in the literature (see MacLeod, 1991). For example,adding additional irrelevant words to the task reduces Stroop intererenceeects (Kahneman &Chajczyk, 1983), which would not be expected ithe process is completely automatic.

Although the processes underlying the metaphoric representationo abstract concepts remain debated (but see Chapter 4 or a set o studieswhich experimentally tests the one-to-one grounding approach against

the metaphoric structuring view), there is a growing amount o studiesthat have investigated the relationship between perceptual inormationand abstract conceptual processing. Below, I will review these studies,whereby I will ocus primarily on the three abstract concepts underinvestigation in the three chapters o this thesis, namely morality, time,and valence.

Morality

Morality is one o the hallmark abstract concepts or which it is arguedthat there is no sensory or motor inormation that could correspond inany reliable or direct way to their meaning (Mahon&Caramazza, 2008).Te assumed lack o perceptual characteristics o abstract concepts is oneo the most prevalent criticisms on perceptual theories o cognition. Aer

all, how can people perceptually represent abstract concepts which theyhave never seen or heard?


35/147

32

Some researchers have proposed that moral concepts cannot begrounded perceptually (Andrews, et al., 2009; Dove, 2009; Mahon &

Caramazza, 2008). Some o these researchers have proposed that moralconcepts can only be grounded in the emotional responses they cause inus (Boroditsky&Prinz, 2008; Prinz, 2005). Indeed, many studies haverevealed that immorality is strongly associated with eelings o disgust. Forexample, the olactory experience o bad smell inuences moral judgments(Schnall, Haidt, Clore, &Jordan, 2008). Other studies have revealed thatmorality and cleanliness are strongly related. Tinking about an unethical

deed makes cleaning related words more accessible and reading anunethical story increases the desire or cleaning related products (Zhong&Liljenquist, 2006). Activating the concept o cleanliness beore readingmoral dilemmas, as well as washing ones hands aer seeing a disgustingmovie clip, reduces the severity o moral judgments (Schnall, Benton,&Harvey, 2008). aken together, there is abundant support or a strongrelationship between experienced disgust and moral judgments.

Intuitive approaches to morality (Haidt, 2001; 2008) have

highlighted the importance o perceptual representations o moralitytogether with aective responses to moral situations. Building on thework by Lako & Johnson (1999), Haidt (2001, p. 285) argues thatmoral concepts are built up largely by metaphorical extensions romphysical experience. Several other researchers have argued in avoro the perceptual representation o moral concepts (Barsalou, 2008b;Gibbs, 1996; Haidt, 2001; Johnson, 1993; Meier, Robinson, &Wygant,

2007; Lako&Johnson, 1980, 1999; Sherman &Clore, 2009). Both thesimulation approach (Barsalou, 1999; Barsalou&Wiemer-Hastings, 2005)as the metaphoric representation approach (Lako & Johnson, 1999)provide possible suggestions how moral concepts might be representedperceptually. According to the metaphoric representation approach, thereare multiple concrete sources which are commonly used to representmoral concepts metaphorically (or an overview, see Lako&Johnson,

1999, chapter 14). Morality is seen as pure, straight, upright, balanced,and clean, or example, whereas immorality is seen as contaminated,


36/147

33

crooked, low, unbalanced and dirty. wo sets o studies have providedsome initial empirical support or these assumptions.

Meier, Sellbom, and Wygant (2007) investigated whether moralityis represented on a vertical dimension. Tey presented both moral andimmoral words on the top or the bottom o the screen (c. Meier andRobinson, 2004; Schubert, 2005) and ound that responses were asterin the metaphor congruent conditions (moral-up and immoral-down)compared to the metaphor incongruent conditions (moral-down andimmoral up). Te results are not exactly in line with the predictions, in

that immoral words on the bottom o the screen were responded to slowerthan all other conditions, or example. Furthermore, it is unclear to whatextent these eects dier rom the vertical representation o valence(Meier&Robinson, 2004; Craword, Margolies, Drake, &Murphy, 2006).

Sherman and Clore (2009) adapted the Stroop paradigm, andound that participants were aster to indicate the ont color o a word waswhite when the word had a moral (compared to immoral) meaning. Teirrst study is a re-analysis o Meier and colleagues (2004) Study 4, which

originally was designed to test the a-priori prediction that brightnesscategorizations would not be inuenced by the valence o the stimuli.Meier, Robinson, and Clore (2004, p. 83) assumed that the interactionbetween valence and brightness would be asymmetric, in that color wouldinteract with valence judgments (Studies 1, 2, and 3), whereas valencewould not interact with color judgments (Study 4). Such a predictionis in line with both theoretical views on the metaphoric structuring o

abstract concepts based on concrete concepts (Lako&Johnson, 1980)and empirical ndings revealing an asymmetry between the concretesource and abstract target in metaphoric representations (Boot &Pecher,in press; Boroditsky, 2000; Casasanto &Boroditsky, 2008). Nevertheless,in the studies by Sherman and Clore (2009), the abstract meaning o thewords did inuence categorizations o the color in which the words werepresented. Te authors oer no theoretical explanation to reconcile the

dierence between their ndings and previous empirical observations.In addition, it is not specied in which way these results go beyond


37/147

34

the previously reported more general valence-brightness relationship(Meier et al., 2004). Tese studies show the necessity o investigating the

relationship between perceptual inormation and abstract conceptualprocessing rom an integrative theoretical ramework. Te questionwhen and why metaphoric representations are asymmetric is surelyan interesting one, and urther insights into this question will improveour theoretical understanding o the processes underlying metaphoricrepresentations.

In Chapter 2, I take a new approach to the perceptual representation

o morality by investigating how the equality component o moralityis grounded in the balance o justice metaphor. Equality is a centraleature o morality in philosophy (e.g., Rawls, 1971), anthropology (e.g.,Fiske, 1992), olk-theories (e.g., an eye or an eye) and classical justicetheories in psychology such as equity theory (Adams, 1965). Te equalitycomponent o morality is theorized to be grounded in the balance imageschema (Gibbs, 2005; Johnson, 1987, Lako, 1987) and conceptualizedmetaphorically as balance or symmetry (Lako, 2005). Many o our

experiences with airness or justice are directly related to balanced andsymmetrical distributions, such as when parents cut a birthday cakeinto equal pieces. Given this strong experiential association betweenequal divisions and justice, moral concepts should be associated withperceptual symmetry. I peoples understanding o morality and justiceis experientially grounded in symmetrical distributions, then thinkingabout morality should activate non-verbal associations with perceptual

symmetry.Te studies in Chapter 2 investigate whether moral words are

associated with symmetrical stimuli, and immoral words are associatedwith asymmetric stimuli. Participants were asked to indicate which otwo Chinese ideographs presented on the screen correctly translatedthe Dutch stimulus word. Aer this translation task, participants ratedall ideographs on symmetry. Te results revealed that participants had

chosen more symmetric ideographs over more asymmetric ideographswhen translating moral words. Te reverse was true or immoral words.


38/147

35

A second study extended these results to judgments where the moral orimmoral meaning o the word was unrelated to the judgments participants

had to perorm. Participants were asked to indicate how many o theletters o previously presented moral and immoral words had been blue.In line with the assumption that morality is associated with symmetry,participants displayed biased recall o the number o blue and greenletters o previously presented words, indicating that the number o blueletters had been more equally distributed or moral words compared toimmoral words.

ime

Te perceptual representation o time has been investigated morethoroughly than any other abstract concept (Boroditsky, 2000; Boroditsky,2001; Boroditsky & Ramscar, 2002; Casasanto & Boroditsky, 2008;Santiago, Lupiez, Prez, & Funes, 2007; orralbo et al., 2006; versky,et al., 1991; Vallesi, Binns, & Shallice, 2008; Weger & Pratt, 2008; Zacks &

versky, 1999). Tese studies show that time can be represented in the back-ront, up-down and le-right spatial dimensions. Te overlap betweenspatial and temporal representations is also present at a neural level, withcommon brain areas being active during the processing o spatial andtemporal inormation (Basso, Nichelli, Frassinetti, & Di Pellegrino, 1996;Bjoertomt, Cowey, & Walsh, 2002; Rao, Mayer & Harrington, 200Peopleoen use space to talk about time (e.g. Te worst is behind us and We

are approaching the holidays). Boroditsky (2000) reasoned that i peopleuse space to talk about time, they might also use space to think abouttime. I this reasoning holds, then inuencing the way people thinkabout space should inuence how they think about time. Te literaturedierentiates between two reerence rames to think about time in termso space. One the one hand, people can move through time (the ego-moving rame o reerence), on the other hand, time can move toward us

(the time-moving rame o reerence). Participants were presented withan ambiguous temporal statement (e.g. `Next Wednesdays meeting has


39/147

36

been moved orward two days). In English, this statement is just as likelyto mean that the meeting has been moved to Monday, as that it has been

moved to Friday. From an ego-moving rame o reerence, where peoplemove through time, moving orward means the meeting will now be onFriday. From an object-moving perspective, where things move throughtime towards us, the meeting should now be on Monday (McGlone &Harding, 1998).

Activating these respective temporal rameworks by presentingpictures o objects moving through space toward the individual (time

moving perspective), or the individual moving through space (ego-moving perspective) makes people respond to this temporal statementin a metaphor consistent manner (Boroditsky, 2000). A subsequent set ostudies (Boroditsky & Ramscar, 2002) ound that the answer people gaveto the previously mentioned ambiguous question depended on whetherpeople had moved through space themselves (e.g., when ying in anairplane, moving through the lunch line, or on a train), with the morethey moved themselves, the more they answered the question using an

ego-moving rame o reerence.Spatial primes inuence thoughts about time dierently or

Manderin speakers than or English speakers. In Chinese, time can berepresented as moving downward through space, in addition to the le-to-right representation o time in space prevalent in Western culture. Dueto this cultural dierence, priming participants with either horizontalspatial dierences (e.g., a picture where a black worm is positioned to the

le o a white worm) or vertical spatial dierences (e.g., a picture wherea black ball is placed above a white ball) inuences temporal judgments(e.g., March comes earlier than May) in a dierent manner or Englishspeakers than or Mandarin speakers (Boroditsky, 2001).

ime can also be represented rom le (past) to right (uture).Santiago, Lupiez, Prez, and Funes (2007) manipulated both theposition (le vs. right) on the screen o past and uture related words,

as the key assignment with which these words had to be categorized.Participants had to press the le key or past words and the right key or


40/147

37

uture words, or they were instructed to use the reversed key assignment.Both spatial position and key assignment inuenced reaction times, but

these eects were notably stronger or the latter manipulation. Te actthat participants were aster to correctly categorize uture related wordsby pressing the right key, and past related words by pressing the le key,than the reverse, is a strong indication o the representation o the pastand uture rom the le to the right. Te importance o task instructionsin categorization tasks such as these was urther exemplied in two studieswhere participants responded either verbally (categorizing past and

uture related words through vocal responses) or manually (categorizingpast and uture related words by pressing a le or right response key).When answering verbally, the back (past) to ront (uture) representationo time was most active, but when answering manually, the le-rightrepresentation o time was most salient (orralbo, Santiago, &Lupiez,2006).

Te le-to-right representation o time has been investigated moreclosely in two sets o studies (Vallesi, et al., 2008; Weger &Pratt, 2008), in

which the le-to-right representation o time is approached rom a spatialstimulus-response compatibility view (or reviews, see Hommel &Prinz,1997; Umilt &Nicoletti, 1990). Weger and Pratt (2008) replicate earlierndings supporting the le-to-right representation o time, while usingless explicitly time related stimuli (namely names o actors such as CharlieChaplin or Brad Pitt). In addition, they show that past related wordspresented in the center o the screen acilitate recognition o target on the

le o the screen, whereas uture related words acilitate the recognitiono targets on the right side o the screen. Tis eect was stronger whenparticipants had to respond to the le target with their le hand, and tothe right target with their right hand (Study 2A), than when le or righttargets had to be responded to by pressing the space bar, and ller trialshad to be ignored (Study 2B). Tey conclude rom this comparison o abimanual categorization task versus a go-no-go paradigm, that the results

are primarily - although not necessarily exclusively - due to a acilitationo response codes, rather than to an impact on stimulus processing


41/147

38

(Weger &Pratt, p. 429).Vallesi and colleagues (2008) asked participants to indicate how

long a xation cross appeared on the screen (which was presented orone vs. three seconds). Inspired by the spatial representation o numbers,with small numbers being represented on the le and larger numbers onthe right, which is also known as the SNARC eect (Dehaene, Bossini,& Giraux, 1993), these researchers investigated whether short timeperiods (one second) would shi attention to le-hand responses andlonger time periods (three seconds) would shi attention to right-hand

responses. Indeed, le-hand responses were slower when participants hadto indicate the xation cross had been presented or three seconds (vs.one second), whereas right-hand responses were slower when indicatingthe xation cross had been presented or one second (vs. three seconds).Te authors conclude that this intererence emerges somewhere aerresponse selection and during response preparation. Furthermore, theauthors argue that the spatial representation o time is not easily activatedautomatically, but requires the awareness o the passage o time (or

example, by making time task-relevant).Te relationship between time and space has been shown to be

asymmetric, in that spatial primes inuence judgments about time, buttemporal primes do not inuence judgments about space (Boroditsky,2000, Study 2 and 3). Spatial primes which activate the ego-moving orobject-moving rames o reerence inuence thoughts about time, buttemporal primes did not inuence thoughts about space. Tis asymmetry

was investigated in more detail in a paradigm where participants werepresented with growing lines on a computer screen, and were asked tojudge the length o these lines or the duration the lines were presented.Te length o the lines inuenced judgments about the time the lines hadbeen presented on the screen, but the time the stimuli were presentedon the screen, did not inuence judgments about the length o the lines(Casasanto & Boroditsky, 2008). Participants were unable to ignore

irrelevant spatial inormation when they perormed judgments aboutduration, but not the other way around.


42/147

39

Research on the metaphoric representation o abstract conceptshas almost exclusively ocused on the visual modality, and when people

talk about the perceptual grounding o abstract concepts, they implicitlyseem to equate this to visualgrounding (e.g., Craword, 2009). However,there is no reason to assume that the spatial structuring o abstractconcepts is limited to the visual modality. In principle, any modality thatis sensitive to spatial inormation should be able to structure abstractconcepts. Chapter 3 provides empirical support or this assumptionby revealing how time is metaphorically structured in the le-to-right

auditory dimension. Participants listened to auditory stimulus consistingo past and uture related words over headphones. Teir task was simplyto indicate in which ear they judged the stimulus to be presented loudest.In most o the trials, there was an objective dierence in loudness, butin the critical experimental trials, the words were presented equally loudover both channels o the headphone.

As predicted, participants chose the right channel more oenor uture words than or past related words, when the auditory stimuli

were presented equally loud to both ears. Furthermore, the averagenumber o right ear judgments or each stimulus overlapped signicantlywith the spatial position o the items on a horizontal line as measuredduring a visual pilot study. Tis is strong support or the multi-modalrepresentation o time in space. Note that the temporal meaning o wordsinuenced loudness judgments even when their meaning was unrelatedto the task participants had to perorm, speaking against the view that

the temporal meaning o the words should be task relevant or the spatialrepresentation o time to be activated (Vallesi et al., 2008).

Valence

Te metaphorical representation o positivity and negativity has revealedthat the same abstract concept can be structured in many dierent concrete

dimensions (or reviews, see Craword, 2009; Meier&

Robinson, 2005).Te rst demonstration o congruency eects in the abstract domain o


43/147

40

valence was provided by Meier, Robinson and Clore (2004) who askedparticipants to evaluate positive and negative words, while presenting

these words in either a black or a white ont. Although the color o thewords was irrelevant to perorm the task correctly, Meier and colleaguesreasoned that based on the widespread use o black and white as a metaphoror evil and good the irrelevant perceptual characteristics o the stimuliwould inuence the speed and accuracy with which participants couldevaluate the words. Teir results conrmed this hypothesis: participantswere aster and/or more accurate to categorize positive words written in a

white ont and negative words in a black ont, than to categorize negativewords in a white ont and positive words in a black ont.In urther support o the relationship between valence and

brightness, priming participants with positive or negatively valencedwords has been shown to bias subsequent brightness judgments (Meier,Robinson, Craword, &Ahlvers, 2007). Identical grey cubes were judgedto be lighter when judged aer a positive (vs. negative) word had brieybeen presented on the screen. Tis result was replicated using a response-

deadline procedure, where participants were orced to judge whether agrey cube presented on the screen was the lighter or darker cube o twopreviously presented stimuli. Te act that a similar pattern o resultsis observed when using a response deadline procedure is commonlyinterpreted in support o the assumption that valence and brightnessare automatically associated. O interest is that in none o these studiesparticipants were actually asked to respond to the positive or negative

stimuli presented beore the brightness judgments were made.Other studies have shown congruency eects or positive and

negative words in dierent visual domains. In addition to the associationbetween valence and brightness, Meier and Robinson (2004) oundthat the categorization o valenced words was inuenced by the verticalposition o the stimuli. In line with the metaphoric representation ohappy being up, and sad being down (Lako&Johnson, 1980), positive

words were classied aster when presented on the top o the screen thanon the bottom, and negative words were classied aster when presented


44/147

41

on the bottom o the screen than on the top.In subsequent studies Meier & Robinson (2004) showed that

positive words direct attention upwards and negative words directattention downward (similar to related eects in concrete domains, e.g.,Bergen et al., 2007; Estes et al., 2008; Richardson et al., 2003). Peoplewere aster to indicate whether a letter presented on the top o the screenwas a p or a q aer evaluating a positive word, than aer evaluating anegative word. Te reverse was not true: aer indicating whether astimulus (+++) appeared on the top or the bottom o the screen, there

were no dierences in the speed with which positive or negative words areclassied. Although the absence o dierences is troublesome to interpret,the authors argue that the relationship between verticality and valence isasymmetrical: evaluations activate spatial positions, but spatial positionsdo not activate evaluations. Tese attentional eects were subsequentlyshown to be moderated by individual dierences in aective states, withpeople higher in neuroticism being aster in detecting stimuli in the lower(versus higher) hal o the screen (Meier &Robinson, 2006).

Further support or the relationship between verticality andvalence was observed in studies which investigated whether the valenceo pictures would inuence judgments about the location where thesepictures were previously observed (Craword, et al., 2006). Positivepictures showed an upward bias when participants tried to indicatewhere these stimuli were presented previously. Te authors suggest thatthe relationship between valence and vertical position might reect the

useulness o the vertical spatial domain to organize and structure moreabstract domains (c. Gattis, 2001), without necessarily being the result oa direct experiential associations between positive mood and an uprightposition (Lako&Johnson, 1980).

Valence can also be represented in size, ollowing the more isbetter metaphor (Lako&Johnson, 1999). Meier, Robinson and Caven(2008) presented words in a large or small ont, and asked participants

to evaluate the words as ast and accurately as possible. Participantswere slower to evaluate negative words written in a big ont compared


45/147

42

to negative words in a small ont. In another study the authors show thatwords presented in a large ont are rated more positively than words

presented in a small ont. Te authors conclude that stimulus size leadsto relatively automatic inerences concerning stimulus valence.

A last set o studies to investigate how abstract concepts aregrounded in the vertical dimension looked at representations o aectivelycharged divine concepts (Meier, Hauser, Robinson, Friesen, &Schjeldahl,2007). Tese studies showed the concept o God is strongly associatedwith up vertical locations, but gives less handholds to assume the

vertical representation o God is grounded in sensory-motor experience(e.g., looking up when praying). I the vertical representation o God isgrounded in sensory-motor experiences, the eects should be strongeror people who look up when praying (e.g., Christians) compared topeople who never pray. However, the religion o the participants did notmoderate the observed eects.

Chapter 4 ocuses on the brightness-valence metaphor. Insteado merely demonstrating this metaphoric mapping, the studies in this

chapter aim to urther our understanding o the processes through whichthe brightness-valence mapping emerges. wo process explanations, themetaphoric structuring view and the one-to-one grounding explanation(described earlier in this chapter) are tested against each other. Accordingto the one-to-one grounding account or metaphor congruency eect,brightness automatically activates positivity, and darkness automaticallyactivates negativity. According to the metaphoric structuring view, the

perceptual opposition between white and black structure the conceptualopposition between good and bad. Whereas according to the metaphoricstructuring view shared relational structures are an essential prerequisiteor the brightness-valence mapping to emerge, the polar oppositionsin the brightness and valence dimensions are irrelevant or the one-to-one grounding account. Tese two process explanations were directlycompared by manipulating the endpoints o the valence and brightness

dimensions within or between participants, thereby activating the sharedpolar oppositions in the perceptual and conceptual dimensions when


46/147

43

people perormed the study, or breaking them apart.Previous studies in the literature, as well as two pilot studies, reveal

that white is rated as a neutral color. How can a neutral color metaphoricallyrepresent the meaning o positivity? Six studies investigated whether thepositivity o white was dependent upon the presence o the negativity oblack, and that the opposition between good and bad is metaphoricallystructured in the opposition between light and dark. A Chinese translationtask, where participants were asked to indicate whether a white or blackChinese ideograph correctly translated the meaning o the word, revealed

that whereas black is strongly associated with negativity, white in isolationis not used to represent positivity. Te positivity o white emerges onlyin opposition to the negativity o black. Whereas negative words arealways translated by black ideographs above guessing average, positivewords are only translated above guessing average by white ideographswhen both the valence as the brightness dimensions are manipulatedwithin subjects. Furthermore, two subsequent studies show that thepositivity o white emerges when activating white-black positive-negative

opposition perceptually during the translation task or conceptually beoreparticipants perormed the translation task. ogether, these studies revealthat the shared relational structure in the concrete and abstract domainsis an essential eature o the brightness-valence mapping, supportingthe metaphoric structuring view. Metaphoric representations are atbest conditionally automatic, and require knowledge o the relationalstructures in abstract domains.

Linguistic Explanations

As the above review reveals, there is no shortage o empirical studies thatprovide support or the relationship between perceptual inormationand abstract conceptual processing. Studies that have examined theexact nature o this relationship, however, are substantially scarcer. Tis

criticism on the lack o process explanations within embodied cognitionis not limited to studies investigating abstract concepts (e.g., Barsalou,


47/147

44

2008b; Zwaan, 2009), but is less pressing in the case o concrete concepts,where the automatic activation o sensorimotor inormation in conceptual

processing has received behavioral (e.g., Zwaan &aylor, 2006, Glenberg,Sato, Cattaneo, et al., 2008) and neurophysiological (e.g., Hauk et al., 2004)support. Te question whether abstract conceptual processing necessarilyactivates sensorimotor areas in the brain or whether sensorimotoractivation largely plays a peripheral role in conceptual processingremains unanswered. Te relationship between perceptual inormationand abstract concepts might even be epiphenomenal, in that people

perceptually represent concepts when possible, but that the sensorimotoractivation is unrelated to conceptual processing.How conclusive are the empirical results that have been observed

so ar in the domains o morality, time and valence with regard to theperceptual nature o abstract thought? Could these ndings be explainedbased on associations between purely amodal symbols? Te short answeris yes. Tere is no empirical support or the idea that sensorimotoractivation is absolutely necessary or abstract conceptual thought, or

even that thoughts about abstract concepts are limited without relyingon perceptual representations. Tereore, it is impossible to conrm thehypothesis that abstract concepts are automatically structured in concretedimensions, and reject alternative explanations. Beside this obviouslimitation o the current empirical data, there are several other reasons totake a cautionary approach when interpreting the results o studies thathave investigated the perceptual representation o abstract concepts.

First o all, the support or metaphoric representations is almostexclusively linguistic in nature (but see Boot &Pecher, in press; Casasanto& Boroditsky, 2008; Craword et al., 2006). Tis poses a problem,because many conceptual metaphors are derived rom language use,and subsequently tested by asking people to process abstract language(Casasanto, in press). Tis circularity problem (Craword, 2009) is evenmore problematic because language encodes embodied relationships

(Louwerse, 2008). For example, remember the ndings by Zwaan&

Yaxley (2003), revealing that participants were aster to conrm that the


48/147

45

words branch and root were related when branch was presented aboveroot, compared to when these words were presented in the reverse spatial

layout. Louwerse (2008) shows that the words branch and rootoccur morerequently in the order branch-rootthan in the root-branch order in humanlanguage. Furthermore, the more requently these combinations occur inlanguage, the higher participants judge the likelihood o encounteringthe rst word (branch) above the other (root), when explicitly asked.Crucially, word order requencies predicted reaction times in thesemantic judgment task better than spatial iconicity did. Tese studies

show that ndings in reaction time studies that are currently explainedrom a grounded cognition perspective can be explained by linguisticstatistical inormation, because pre-linguistic conceptualizations such asspatial iconicity are reected in language (see also Cooper &Ross, 1975,Greenberg, 1963).

A highly related linguistic characteristic that has been argued toplay a role in studies investigating the perceptual representation o abstractconcepts is the notion olinguistic markedness. People have preerred ways

to talk and think about dimensions. For example, the question how longwill it take? is used more requently than how short will it take? whichcarries additional meaning (in this case, the expectancy that somethingwill not take long). Tereore long is called the linguistically unmarkedcategory, and short is called the marked category (Greenberg, 1963).ypically the category that gives a dimension its name is unmarked (Clark,1973), such as long (length), bright (brightness) or happy (happiness).

In addition to being used more requently, unmarked categories areprocessed aster than marked categories (Clark, 1969).

Tese dierences in markedness are not restricted to linguisticstimuli but extend to non-verbal stimuli such as arrows or le-rightresponses (or an overview, see Proctor &Cho, 2006). Finally, dierencesin markedness have also been observed or abstract relationships betweenpeople. De Soto, London and Handel (1965) were the rst to show that

the amount o errors people make when answering syllogisms about therelationship between two individuals depends on whether the words


49/147

46

better or worse are used. Although A is better than B and B is betterthan C contains the same inormation as B is worse than A and C is

worse than B, the better statements led to more accurate judgmentsabout the relationship between A and C than the worse statements.o acknowledge this more general response acilitation or th

Lakens - Abstract Concepts in Grounded Cognition

Documents

Transcript of Lakens - Abstract Concepts in Grounded Cognition