Expanding the figurative continuum to multimodal settings ... · (Zbikowski 2009, Pérez-Sobrino...

56
Paula Pérez Sobrino Francisco José Ruiz de Mendoza Ibáñez y Lorena Pérez Hernández 2014-2015 Título Director/es Facultad Titulación Departamento TESIS DOCTORAL Curso Académico Expanding the figurative continuum to multimodal settings. Patterns of interaction of multimodal metaphor and metonymy in advertising Autor/es

Transcript of Expanding the figurative continuum to multimodal settings ... · (Zbikowski 2009, Pérez-Sobrino...

Page 1: Expanding the figurative continuum to multimodal settings ... · (Zbikowski 2009, Pérez-Sobrino 2014a,b) and gestures (Müller and Cienki 2009) that provide further evidence of the

Paula Pérez Sobrino

Francisco José Ruiz de Mendoza Ibáñez y Lorena Pérez Hernández

2014-2015

Título

Director/es

Facultad

Titulación

Departamento

TESIS DOCTORAL

Curso Académico

Expanding the figurative continuum to multimodal settings. Patterns of interaction of multimodal metaphor

and metonymy in advertising

Autor/es

Page 2: Expanding the figurative continuum to multimodal settings ... · (Zbikowski 2009, Pérez-Sobrino 2014a,b) and gestures (Müller and Cienki 2009) that provide further evidence of the

© El autor© Universidad de La Rioja, Servicio de Publicaciones, 2016

publicaciones.unirioja.esE-mail: [email protected]

Expanding the figurative continuum to multimodal settings. Patterns ofinteraction of multimodal metaphor and metonymy in advertising, tesis doctoral

de Paula Pérez Sobrino, dirigida por Francisco José Ruiz de Mendoza Ibáñez y Lorena Pérez Hernández (publicada por la Universidad de La Rioja), se difunde bajo una Licencia

Creative Commons Reconocimiento-NoComercial-SinObraDerivada 3.0 Unported. Permisos que vayan más allá de lo cubierto por esta licencia pueden solicitarse a los

titulares del copyright.

Existen circunstancias excepcionales que impiden la difusión de la versión íntegra de estatesis. Por este motivo se difunden únicamente los contenidos que no están sujetos a

confidencialidad

Page 3: Expanding the figurative continuum to multimodal settings ... · (Zbikowski 2009, Pérez-Sobrino 2014a,b) and gestures (Müller and Cienki 2009) that provide further evidence of the

 Faculty of Letters and Education

Modern Philologies Department

EXPANDING THE FIGURATIVE CONTINUUM

TO MULTIMODAL SETTINGS

v Patterns of interaction of multimodal metaphor and

metonymy in advertising

INTERNATIONAL DOCTORATE

PhD Dissertation by:

Paula Pérez Sobrino

Supervised by:

Prof. Francisco José Ruiz de Mendoza Ibáñez

Dr. Lorena Pérez Hernández

2015

Page 4: Expanding the figurative continuum to multimodal settings ... · (Zbikowski 2009, Pérez-Sobrino 2014a,b) and gestures (Müller and Cienki 2009) that provide further evidence of the

  ii  

Page 5: Expanding the figurative continuum to multimodal settings ... · (Zbikowski 2009, Pérez-Sobrino 2014a,b) and gestures (Müller and Cienki 2009) that provide further evidence of the

  iii  

Creativity does not follow norms,

Rather it deliberately and knowingly exploits norms.

Patrick Hanks

Page 6: Expanding the figurative continuum to multimodal settings ... · (Zbikowski 2009, Pérez-Sobrino 2014a,b) and gestures (Müller and Cienki 2009) that provide further evidence of the

  iv  

Page 7: Expanding the figurative continuum to multimodal settings ... · (Zbikowski 2009, Pérez-Sobrino 2014a,b) and gestures (Müller and Cienki 2009) that provide further evidence of the

  v  

CONTENTS v

ACKNOWLEDGEMENTS_______________________________________ix

LIST OF FIGURES, TABLES, AND ADVERTISING EXAMPLES___xiii

LIST OF ABBREVIATIONS AND GRAPHIC CONVENTIONS_____xxi

CHAPTER 1

Introduction: Scope, need for the research, and working hypotheses_____1

1.1. Scope and need for the research………………………………………...….1

1.2. Research questions and working hypotheses……………...….…….……..5

1.3. Structure of this dissertation………………….……………………………11

CHAPTER 2

State of the art: A critical overview_________________________________15

2.1. Introduction........................................................................................15

2.2. Looking outwards: what does VSS add to the cognitive approach to

multimodal meaning construction?.............................................................18

2.3. Looking inwards: What does CL reveal about multimodal meaning

construction?.............................................................................................29

2.3.1. RT...........................................................................................30

2.3.2. CBT.........................................................................................40

2.3.3. The multimodal approach to CMT...........................................53

CHAPTER 3

Theoretical framework: Building an integrated approach to the study of

multimodal metaphor in interaction with metonymy__________________59

3.1. Introduction........................................................................................59

3.2. Patterns of conceptual interaction........................................................61

3.2.1. (MS)iT metonymy....................................................................62

Page 8: Expanding the figurative continuum to multimodal settings ... · (Zbikowski 2009, Pérez-Sobrino 2014a,b) and gestures (Müller and Cienki 2009) that provide further evidence of the

  vi  

3.2.2. Metonymic chains....................................................................64

3.2.3. Metaphtonymy........................................................................66

3.2.4. Metaphoric amalgams..............................................................69

3.2.5. Metaphoric chains....................................................................71

3.3. The figurative continuum.....................................................................73

3.4. Expanding the figurative continuum to multimodal settings: what needs

to be done..................................................................................................75

CHAPTER 4

Methodology and corpus__________________________________________81

4.1. Introduction........................................................................................81

4.2. Aspects of operationalization...............................................................84

4.2.1. The EH....................................................................................84

4.2.2. Identification of multimodal manifestations related to metaphor

and metonymy..................................................................................86

4.2.3. Identification and labelling of multimodal metaphor and

metonymy ........................................................................................89

4.2.4. Corpus and selection of data.....................................................91

4.2.5. Corpus annotation....................................................................96

CHAPTER 5

Expanding the figurative continuum to multimodal settings:

Metonymy and metonymic complexes______________________________101

5.1. Introduction......................................................................................101

5.2. Multimodal metonymy and its complexes..........................................102

5.2.1. Multimodal metonymy...........................................................102

5.2.2. Multimodal metonymic chain.................................................107

5.2.3. Multimodal (MS)iT metonymy...............................................110

5.2.4. Multimodal (MS)iT metonymic chain.....................................114

5.3. Other operations in combination with multimodal metonymy.............116

5.3.1. Hyperbole..............................................................................116

5.3.2. Paradox.................................................................................119

Page 9: Expanding the figurative continuum to multimodal settings ... · (Zbikowski 2009, Pérez-Sobrino 2014a,b) and gestures (Müller and Cienki 2009) that provide further evidence of the

  vii  

5.3.3. Onomatopoeia.......................................................................122

5.4. Interim conclusions...........................................................................124

CHAPTER 6

Expanding the figurative continuum to multimodal settings (II):

Metaphor and metaphoric complexes______________________________127

6.1. Introduction......................................................................................127

6.2. Multimodal metaphor and its complexes............................................128

6.2.1. Multimodal metaphor.............................................................128

6.2.2. Multimodal metaphtonymy....................................................134

(a) Parallel metonymic expansion in both metaphorical

domains.............................................................................135

(b) Parallel metonymic reduction in both metaphorical

domains.............................................................................138

(c) Metaphtonymy scenario.....................................................141

(d) (MS)iT metaphtonymy.......................................................147

6.2.3. Multimodal single-source metaphoric amalgam.......................152

6.2.4. Multimodal multiple-source/target metaphoric amalgam........158

6.2.5. Multimodal metaphoric chain.................................................167

6.3. Conclusions.......................................................................................171

CHAPTER 7

Conceptual complexes in advertising: A corpus-based account________173

7.1. Introduction......................................................................................173

7.2. Zooming in: A corpus-based overview of multimodal conceptual

complexes in advertising...........................................................................179

7.2.1. Conceptual operations............................................................179

7.2.2. Advertised product.................................................................182

7.2.3. Modal cues............................................................................186

7.3. Zooming out: Factors that (may) influence conceptual complexes in

advertising...............................................................................................188

7.3.1. Effect of mode on conceptual complexity................................189

Page 10: Expanding the figurative continuum to multimodal settings ... · (Zbikowski 2009, Pérez-Sobrino 2014a,b) and gestures (Müller and Cienki 2009) that provide further evidence of the

  viii  

7.3.2. Effect of product type on conceptual operations.......................192

7.4. Conclusions……………………………………………………………….198

CHAPTER 8

The comprehension of multimodal metaphor-metonymy combinations

in advertising: a cross-cultural investigation________________________201

8. 1. Introduction………………………………………………………………201

8.2. Background and rationale..................................................................204

8.3. Methodology……………………………………………………………...206

8.3.1. Research questions and working hypotheses............................206

8.3.2. Selection of participants and materials.....................................208

8.3.3. Selection of the material.........................................................208

8.3.4. Data collection.......................................................................209

8.4. Findings and discussion.....................................................................212

8.4.1. Speed of processing................................................................212

8.4.2. Complexity of interpretation...................................................213

8.4.3. Perceived persuasive potential.................................................214

8.4.4. Cross-cultural variation..........................................................215

(a) Speed of comprehension.....................................................215

(b) Complexity of interpretation...............................................216

(c) Perceived persuasive potential.............................................221

8.5. Summary and conclusions.................................................................222

CHAPTER 9

Conclusion_____________________________________________________227

9.1. Summary..........................................................................................227

9.2. Theoretical implications.....................................................................230

9.3. Practical applications.........................................................................236

9.4. Further research lines.........................................................................240

REFERENCES_________________________________________________241

APPENDIX____________________________________________________265

Page 11: Expanding the figurative continuum to multimodal settings ... · (Zbikowski 2009, Pérez-Sobrino 2014a,b) and gestures (Müller and Cienki 2009) that provide further evidence of the

CHAPTER 1: Introduction __________________________________________________________________

1

CHAPTER 1

Introduction: Scope, need for the research, and working

hypotheses

v

Alice: Where should I go?

The Chesire Cat: That depends on where you want to end up.

LEWIS CARROLL, Alice's Adventures in Wonderland

v

1.1. Scope and need for the research

As thrown in relief by the title, Expanding the figurative continuum to

multimodal environments: multimodal metaphor in interaction with metonymy in

advertising, this dissertation brings together insights from several analytical

strands of Cognitive Linguistics (CL) to approach the study of the identification

and comprehension of advertising. Although not a new enterprise, not many

interdisciplinary works end up achieving a real symbiosis between the areas

involved. In this regard, this dissertation is fully committed to providing a novel

contribution by gathering a set of analytical tools developed by cognitive

linguists to describe different creative advertising manifestations and to analyse

the conceptual structure beneath real world persuasive messages. Cognitive

linguists may thus find in this dissertation a novel way of assessing and

predicting the communicative impact of multimodal manifestations, whereas,

interestingly enough, the same set of analytical tools could be strategically

Page 12: Expanding the figurative continuum to multimodal settings ... · (Zbikowski 2009, Pérez-Sobrino 2014a,b) and gestures (Müller and Cienki 2009) that provide further evidence of the

CHAPTER 1: Introduction __________________________________________________________________

2

deployed by creative designers and marketing scholars to construe more

cognitively-effective and persuasive messages.

Meaning construction, representation and communication are

characterised nowadays by an intensified shift from verbal language toward

multimodality. The amount, variety and speed with which information takes

place at the intersection of several modes (text, pictures, moving images,

sound, music, etc.) require a new set of meaning-making practices and

analytical resources. In other words, cognitive linguists, advertisers and

creative designers must become multiliterate agents who are aware of the

affordances and limitations of each medium and mode. This challenging task

is fundamental for the successful communication of multimodal knowledge.

In this context, the major aim of this dissertation is to provide understanding

of the (re)construction of multimodal meaning through the complex interplay

of cognitive operations. More specifically, this work is concerned with a

concrete meaning-making strategy in multimodal environments: multimodal

metaphor in interaction with metonymy in advertising (both in printed

billboards and commercials).

Looking back at thirty years of CL, much has been accomplished since

Lakoff and Johnson’s (1980) ground-breaking book Metaphors We Live By. In

contrast to the traditional understanding of metaphor as a skilful tool for

literary embellishment, Lakoff and Johnson’s Conceptual Metaphor Theory

(CMT, 1980; later relabelled Contemporary Theory of Metaphor, CTM,

Lakoff 1993) made a compelling argument for the centrality of metaphor to

everyday language and thought: people make use of some concepts to

understand, talk and reason about others. Since then, CL has witnessed an

exponential growth of interest in metaphor studies (to name but a few, e.g.,

Dirven and Ruiz de Mendoza, 2010; Gibbs, 1994; Gibbs, Bogdanovich, Sykes

and Barr, 1997; Kövecses, 1990, 2000, 2002, 2005; Lakoff, 1987; Lakoff and

Turner, 1989; Lakoff and Johnson, 1999; for an assessment on these

developments, they reader may refer to Gibbs, 2011; and Ruiz de Mendoza

and Pérez-Hernández, 2011). Likewise, metonymy has yielded similar

analyses as another pervasive conceptual device (Barcelona 2000, 2011;

Page 13: Expanding the figurative continuum to multimodal settings ... · (Zbikowski 2009, Pérez-Sobrino 2014a,b) and gestures (Müller and Cienki 2009) that provide further evidence of the

CHAPTER 1: Introduction __________________________________________________________________

3

Kövecses and Radden, 1998; Lakoff and Johnson, 1980; Ruiz de Mendoza

2000, 2013).

Although it is widely acknowledged among researchers in CL that

metaphor and metonymy are conceptual devices, scholars have traditionally

restricted their studies to the exploration of verbal metaphor and metonymy,

i.e. those instances in which the conceptual mappings are realised exclusively

through linguistic means. Neglecting alternative manifestations of metaphor

ignores one of the most basic statements of CMT, namely that “metaphor is

primarily a matter of thought and action and only derivatively a matter of

language” (Lakoff and Johnson, 1980: 153). The initial nearly exclusive

attention to linguistic manifestations of metaphor necessarily limited the

development of a complete theory of thought, since it failed to account for

other levels of cognitive modeling (such as the visual, audial, olfactorial and

gestural). Furthermore, the aforementioned linguistic bias has largely

prevented researchers from making use of such powerful mechanisms of

analysis in multimodal contexts, i.e. printed advertising, commercials, films,

music or Internet sites.

The widening of scope in metaphor theorization towards multimodality

had its roots in the pioneering work carried out by Forceville (see Forceville,

1996, 2009a and the references in these papers to his own ample work on the

issue). According to Forceville (2009a: 24), multimodal metaphors are those

“whose source and target are each represented exclusively or predominantly in

different modes”. Hence, multimodal theorists hold that conceptual metaphor

manifests itself not exclusively through verbal language, but also via alternative

modes of representation. Advertising arises as a fruitful space for multimodal

metaphor production and scholarly study (cf. Forceville 2009a, Uriós-Aparisi

2009, Hidalgo and Kralievic 2011, Pérez-Sobrino 2013, 2015, for varied

accounts of verbopictorial metaphors in billboards and commercials; Velasco

and Fuertes 2006, for olfactorial metaphors in perfume billboards). The inner

logic of metaphor as a cognitive operation suits the specificities of advertising

as a discursive genre: both consist in putting in correspondence two discrete

domains (in the case of metaphor, the source and the target domain; in

Page 14: Expanding the figurative continuum to multimodal settings ... · (Zbikowski 2009, Pérez-Sobrino 2014a,b) and gestures (Müller and Cienki 2009) that provide further evidence of the

CHAPTER 1: Introduction __________________________________________________________________

4

advertising, the product or service being advertised and the corresponding

positive attributed values). There are complementary studies of multimodal

metaphor in other non-verbal genres besides advertising, such as classical music

(Zbikowski 2009, Pérez-Sobrino 2014a,b) and gestures (Müller and Cienki

2009) that provide further evidence of the existence of non-verbal

manifestations of conceptual metaphor. Unfortunately, multimodal metonymy

is still a fairly unexplored cognitive phenomenon, in much the same way as the

theorization of verbal metonymy is still a couple of decades behind that of

verbal metaphor. A few illustrative exceptions are Forceville 2009b, Villacañas

and White 2013, Pérez-Sobrino 2015 for verbopictorial metonymy in printed

advertising; Pérez-Sobrino 2014 for verbomusical metonymy in classical and

contemporary music.

In fact, the abovementioned research on multimodal metaphor and

metonymy evinces the productivity of applying tools traditionally used in the

analysis of verbal metaphor and metonymy to the study of multimodal

environments. This working assumption has been labelled the equipollence

hypothesis (EH; Ruiz de Mendoza and Mairal 2011), and it is central to the

development of the present dissertation. This methodological principle states

that the analyst should test whether conceptual processes that have been

attested in one domain of linguistic enquiry may also be (at least partially)

active in other domains. In this spirit, the extension of the EH to the

exploration of multimodal environments may prove useful to enrich current

accounts of multimodal metaphor. In this sense, this work heavily borrows

analytical tools from the existing literature on conceptual interaction between

verbal metaphor and metonymy, and applies them (with the necessary

adjustments) to the analysis of multimodal contexts. Interestingly enough, it is

expected that the EH could operate in a reverse way. It would be worthy of

examination whether the novelties arising from the study of metaphor and

metonymy in multimodal environments are replicable in verbal discourse, thus

expanding our knowledge of conceptual metaphor and metonymy.

As will be shown, the application of the analytical tools related to verbal

metaphor to the study of its multimodal manifestations is not without

Page 15: Expanding the figurative continuum to multimodal settings ... · (Zbikowski 2009, Pérez-Sobrino 2014a,b) and gestures (Müller and Cienki 2009) that provide further evidence of the

CHAPTER 1: Introduction __________________________________________________________________

5

problems. The literature shows that the work that has been so far undertaken

on multimodal metaphor emerges more as a kaleidoscopic array of personal

speculations than as a unified theoretical account due the relative youth of

multimodality as a strand of CMT. This disparity shows that, more than a

faulty attempt, the analysis of multimodal metaphor is still in need of

development of specific analytical tools.

1.2. Research questions and working hypotheses

Within this framework, this dissertation takes up the challenge and aims

at proposing the first comprehensive field guide to exploring multimodal

metaphorical and metonymic meaning construction, representation and

communication in the context of advertising. This is by no means a minor

venture. In order to ease our enterprise, we therefore break down this major

goal into six analytical inquiries. These main research questions stem directly

from the intersection of the conceptual phenomena subject of this dissertation

(i.e. metaphor-metonymy interaction in multimodal settings) and the

exploration of their communicative impact of in the context of advertising.

MRQ 1. How does multimodal metaphor interact with metonymy? This research

question investigates the potentiality of metaphor to couple with other cognitive

operations, such as metonymy and other metaphors, in order to build a finite set

of simple and complex conceptual operations in multimodal use.

MRQ 2. How is multimodal metaphor related to metonymy and the complexes

arising from their interaction? This question seeks to find out if metaphor,

metonymy, and the corresponding complexes arising from their interaction can

be placed along a cline of increasing conceptual complexity (as has been

previously postulated by Gibbs 1984, Dirven 2002, and McArthur and

Littlemore 2008). Additionally, it further investigates the order of the inventory

of different metaphor-metonymy combinations. Although there has been to date

some literature claiming that metonymy is qualitatively simpler than metaphor

Page 16: Expanding the figurative continuum to multimodal settings ... · (Zbikowski 2009, Pérez-Sobrino 2014a,b) and gestures (Müller and Cienki 2009) that provide further evidence of the

CHAPTER 1: Introduction __________________________________________________________________

6

(Dirven 2002, Klepousniotou and Baum 2007, Rundbland and Annaz 2010),

there is currently no research work on the potentiality of metaphor-metonymy

interactional patterns to trigger varying degrees of inferential activity.

MRQ 3. What are the theoretical implications offered by this dissertation? We aim

to identify how the main theoretical proposals arising from our thorough

analysis of multimodal metaphor-metonymy combinations contribute to fill

research gaps in verbal accounts of conceptual metaphor and metonymy.

MRQ 4. What is the effect of advertising variables (such as the use of modal cues

and/or the explicit representation of the product) on the potentiality of conceptual

operations to convey more or less complex persuasive messages? We wonder whether

conceptual complexity can be directly correlated with specific variables of

advertising, such as the type of promoted product, the use of modes, the explicit

or implicit representation of the product, and/or the inclusion of the product in

the conceptual operation at work. By means of a corpus-based exploration, we

aim to determine whether the current focus on multimodal metaphor

corresponds with its real weight in advertising, and whether significant

correlations can be established between the conceptual, the discursive, and the

communicative dimensions of advertising.

The complexity of the issue under scrutiny has lead us to open Chapter 7

with a set of six related research sub-questions (SRQ 4.1-4.6) in order to better

deal with MRQ4.

SRQ 4.1. What is the distribution of multimodal metaphors and

alternative conceptual operations in the corpus?

SRQ 4.2. Does the product always need to be represented?

SRQ 4.3. Does the product always coincide with the target domain of

the conceptual operation involved?

SRQ 4.4. Which are the modes chosen to represent conceptual

operations and the advertised product?

SRQ 4.5. Is the choice of one mode over another more likely to give

rise to different amounts of conceptual complexity?

Page 17: Expanding the figurative continuum to multimodal settings ... · (Zbikowski 2009, Pérez-Sobrino 2014a,b) and gestures (Müller and Cienki 2009) that provide further evidence of the

CHAPTER 1: Introduction __________________________________________________________________

7

SRQ 4.6. Is the marketing strategy underlying the promotion of a

product more or less likely to give rise to different amounts of

conceptual complexity?

MRQ 5. What is the role of conceptual complexity (in terms of metaphor-metonymy

combinations) in the interpretation of the advertisement? This question explores the

effect of conceptual complexity on the comprehension of the advertisement by

audiences in terms of speed of processing, saliency of interpretation, depth of

comprehension, and perceived appeal of a given advertisement. Similarly, we

have divided this main research question into four more specific sub-research

questions (SRQ 5.1-5.4) in Chapter 8.

SRQ 5.1. Is conceptual complexity related to speed of comprehension?

SRQ 5.2. Is conceptual complexity related to the number of responses

produced (here labelled “complexity of interpretation”)?

SRQ 5.3. Is conceptual complexity related to the perceived persuasive

potential of the advertisement?

SRQ 5.4. How do the above variables (reaction time, complexity of

interpretation, perceived appeal) vary according to the linguistic and

cultural background of the viewer?

MRQ 6. What are the practical applications of this research work? We aim to

demonstrate that this research work can serve as a starting point to frame further

applied studies. First, it makes general observations that may be useful for

advertising and marketing experts. The aim in this respect is to determine

whether there is a measurable effect of conceptual complexity in advertising that

is conditional to the way audiences understand the persuasive message. Second,

we explore whether the connection between conceptual complexity and the

various advertising variables specified in MRQ 3 positively or negatively affect

the way audiences interpret the message in terms of speed of processing, saliency

of interpretation, perceived persuasive potential of the advertisements, and the

linguistic/cultural background of the consumer.

Page 18: Expanding the figurative continuum to multimodal settings ... · (Zbikowski 2009, Pérez-Sobrino 2014a,b) and gestures (Müller and Cienki 2009) that provide further evidence of the

CHAPTER 1: Introduction __________________________________________________________________

8

The specific research inquiries that arise when considering the role of

metaphor and metonymy in relation to the purposes of advertising lead us to put

forward the following main research hypotheses that motivate and drive this

study.

MH1. This work claims that the observed patterns of interaction between

metaphor and metonymy in verbal use (following the line of research of Ruiz de

Mendoza, Pérez-Hernández, and collaborators) are powerful analytical tools of

multimodal environments. In application of the EH, we anticipate that we will

be able to replicate these verbal metaphor-metonymy combinations in our corpus

of advertisements. We still do not know whether we will find novel patterns of

interaction that are exclusive to multimodal communication; if so, a reverse

application of the EH would help to enrich verbal accounts of metaphor and

metonymy.

MH2. Metaphor and metonymy, in spite of its morphological differences, can be

placed along a scale based on the potential of each conceptual operation to

trigger inferences. We therefore adhere to the notion of figurative continuum (as

previously put forward by Gibbs 1984, Dirven 2002, McArthur and Littlemore

2008), by which metaphor and metonymy can be placed along a cline of

increasing figurativeness. We believe that the application of the figurative

continuum to multimodal settings lends further support to its conceptual status.

Furthermore, we expect to fill the gaps between metonymy, metaphor, and more

creative conceptual configurations with our inventory of multimodal complex

conceptual operations.

MH3. Concerning the theoretical implications of this dissertation, we further

hypothesise that the application of the EH in a reverse way, i.e. from non-verbal

to verbal communication, makes it possible to enhance the scope of current

language-based accounts of these two tropes in CL. Therefore, this work

contributes to expanding the body of knowledge on metaphor and metonymy by

looking at multimodal manifestations of these tropes.

Page 19: Expanding the figurative continuum to multimodal settings ... · (Zbikowski 2009, Pérez-Sobrino 2014a,b) and gestures (Müller and Cienki 2009) that provide further evidence of the

CHAPTER 1: Introduction __________________________________________________________________

9

MH4. At this point it is not possible to foresee what interactional pattern will

pervade our corpus of advertisements. We tentatively think that it will be more

likely a complex operation based on the interaction of metaphor and metonymy,

rather than a metaphor or metonymy working in isolation, precisely because

such interactional pattern will merge the highlighting power of metonymy

(useful to connect products with brands) with the cross-domain correspondences

of metaphor (a suitable way to borrow values from a well-connoted domain and

ascribe them to the advertised product). We have anticipated a series of more

specific hypotheses (SH 4.1- 4.6) in order to address in detail each of the

formulated SRQ in Chapter 5.

SH 4.1. We expect metaphtonymy to play an important role in our

corpus, given the ability of metonymy to supply a vantage point from

which access is afforded to advertisements that can combine with the

highlighting of desirable features from one domain to be ascribed to the

product via metaphorical mapping.

SH 4.2. We expect the product to be explicitly represented in most of

the cases because this is the safest procedure to make audiences

understand that advertisers intend to sell something to them.

SH 4.3. We accept that the product shall coincide with the target of one

of the conceptual operations structuring an advertisement, but certainly

we expect the occurrence of alternative conceptual operations

developing additional features of the product or the brand.

SH 4.4. We expect source domains to be mainly visual, owing to the

higher evocative power of this mode, and target domains to be either

verbopictorial or exclusively verbal. We cannot anticipate the preferred

mode to represent the promoted product, given that it may greatly

depend on the product type or the creative strategy.

SH 4.5. We have no expectation whatsoever as to whether different

modes are more or less likely to trigger different amounts of conceptual

operations.

Page 20: Expanding the figurative continuum to multimodal settings ... · (Zbikowski 2009, Pérez-Sobrino 2014a,b) and gestures (Müller and Cienki 2009) that provide further evidence of the

CHAPTER 1: Introduction __________________________________________________________________

10

SH 4.6. We expect variation in determining the degree of conceptual

complexity in the advertisement according to role of the advertised

product.

MH5. We claim that conceptual complexity is a crucial factor that greatly

influences the audience’s interpretation of advertising, and that it should be

treated as such by advertisers along other variables such as graphic design,

psychological profile, and socioeconomic status of the targeted audiences. We

expect that the formulation of a list of conceptual operations in increasing degree

of conceptual complexity will allow us to determine its specific effect on

advertising comprehension in terms of speed of processing, saliency of the

intended interpretation, depth of comprehension and perceived appeal of the

advertising. Such expectations are addressed in greater detail in five research sub-

hypotheses (SH 5.1-5.5) at the beginning of Chapter 8.

SH 5.1. We expect reaction times to increase along a continuum based

on conceptual complexity from metonymy, through metonymic chains,

metaphor, and metaphtonymy, to metaphoric amalgams. The potential

for conceptual enrichment grows as we move along this continuum

since more mappings are available.

SH 5.2. We expect greater inferential activity leading to greater

complexity in the participants’ descriptions as the advertisements move

along the figurative continuum.

SH 5.3. We do not know whether conceptual complexity will affect the

perceived persuasive power of the advertisements, as this will depend on

other variables, such as the viewer’s linguistic and cultural background.

We aim to identify which variables play a role in the triggering of

positive attitudes.

SH 5.4. We expect linguistic and cultural variation in terms of the

reaction times, complexity of responses, and appreciation of the

advertisements, but we do not know the exact form that this will take.

Page 21: Expanding the figurative continuum to multimodal settings ... · (Zbikowski 2009, Pérez-Sobrino 2014a,b) and gestures (Müller and Cienki 2009) that provide further evidence of the

CHAPTER 1: Introduction __________________________________________________________________

11

MH6. We believe that metaphor and metonymy are conceptual mechanisms

with a great but finite inferential power that guide and constrain the triggered

inferential activity, thus favouring desired assumptions about the product

while discarding faulty or misguided interpretations of the persuasive

message. The finer-grained analysis of multimodal metaphor here presented

not only explains the way in which advertisers can enhance their persuasive

messages by means of the conscious interplay of modes and tropes, but also

helps to predict consumers’ interpretation. Besides the benefits resulting from

MH 4, a contrastive investigation between audiences of different linguistic

backgrounds will help us to identify the potentiality of conceptual operations

to construct cross-culturally valid advertising. The view adopted in this

dissertation is that the conscious incorporation of the discussed inventory of

multimodal patterns of conceptual interaction at the stage of advertising

creation could ease the cross-cultural understanding of the persuasive

narrative, but that such understanding is always limited and culture-bound to

some extent.

1.3. Structure of this dissertation

This dissertation unfolds in nine chapters in which each of these inquiries

and premises are dealt with in detail.

Chapter 2 (State of the art: a critical overview) provides a bird's eye view of

the literature on figurative meaning construction in multimodal settings. Four

theoretical perspectives are surveyed in order to highlight their suitability for this

enterprise. Visual Social Semiotics (VSS; Kress and Leuween, 1996, 2001),

Relevance Theory (RT; Sperber and Wilson 1995, in its multimodal application,

as carried out by Yus 2009), Conceptual Blending Theory (CBT; Fauconnier and

Turner, 2002), and CMT (Lakoff and Johnson, 1980, in multimodal use as

theorised by Forceville and Urios-Aparisi 2009, and references therein) are put

into contrast, accompanied by a discussion of their strengths and weaknesses.

This chapter concludes by identifying a number of areas that are still in need of

Page 22: Expanding the figurative continuum to multimodal settings ... · (Zbikowski 2009, Pérez-Sobrino 2014a,b) and gestures (Müller and Cienki 2009) that provide further evidence of the

CHAPTER 1: Introduction __________________________________________________________________

12

development, and provides preliminary insights into the way in which this

dissertation will overcome such theoretical deficiencies.

Chapter 3 (Theoretical framework: Building an integrated approach to the study

of multimodal metaphor in interaction with metonymy) continues the discussion

initiated in Chapter 2 and presents the way in which this dissertation aims at

framing the description and analysis of multimodal construction processes. The

chapter presents the set of conceptual interaction patterns that have been attested

in linguistic research (Ruiz de Mendoza 2000, Ruiz de Mendoza 2007, Ruiz de

Mendoza and Pérez-Hernández 2011, Pérez-Hernández 2013), that are

subsequently placed along a continuum of increasing conceptual complexity,

thus completing Dirven’s (2002) notion of figurative continuum. The chapter

concludes by pointing out a number of aspects to take into account for

multimodal applications.

Chapter 4 (Corpus and methodology) focuses on the description of issues

related to the design of a multimodal corpus of 210 advertisements and

commercials. Several methodological steps as regards the identification,

characterization, and analysis of multimodal metaphoric and metonymic

domains are made explicit in order to establish a coherent and consistent

protocol of analysis through the next two chapters. This chapter presents and

discusses an annotation schema consisting of five categories: product type,

explicit representation of the product, coincidence of the product with the

conceptual operation target, modal cue for the conceptual operation and target,

and mode of representation of the product., It will be argued that this schema is

suitable to deal with MRQ3, and by extension, with SRQ 3.1-3.4,

Chapter 5 and 6 present the results of the qualitative analysis, which for

space reasons, has to be broken in two chapters. Chapter 5 (Expanding the

figurative continuum to multimodal settings (I): Metonymy and metonymic complexes)

provides evidence that attests to the productivity of multimodal metonymy and

its complexes (i.e. multimodal (multiple source)-in-target metonymy,

multimodal metonymic chain, multimodal (multiple source)-in-target

metonymic chain). In turn, Chapter 6 (Expanding the figurative continuum to

multimodal settings (II): Metaphor and its complexes) presents a detailed study of

Page 23: Expanding the figurative continuum to multimodal settings ... · (Zbikowski 2009, Pérez-Sobrino 2014a,b) and gestures (Müller and Cienki 2009) that provide further evidence of the

CHAPTER 1: Introduction __________________________________________________________________

13

multimodal metaphor and its complexes: multimodal metaphtonymy,

multimodal (multiple source)-in-target metaphtonymy, multimodal single-source

metaphoric amalgam, multimodal multiple-source/target metaphoric amalgam,

multimodal metaphoric chain. The direct application of the expanded figurative

continuum proposed at the end of Chapter 3 to multimodal settings yields a

series of variants and novel patterns that, combined with the verbal patterns of

interaction, allow us to build an even more comprehensive notion of the

figurative continuum. The finer-grained version of the multimodal continuum

for metonymy and metaphor (and their respective complexes) is displayed at the

end of each corresponding chapter.

Chapter 7 (Conceptual complexes in advertising: a corpus-based account)

presents the first practical application of our qualitative analysis to advertising

and marketing studies. In this chapter we report the results from the first large-

scale corpus-based study of multimodal metaphor and metonymy, and their

patterns of interaction. We first offer an overview of the composition of our

corpus by reporting frequencies of appearance of the identified conceptual

operations, the characteristics of representation of the advertised product, and

the use of modal cues. Second, we analyse the factors that may determine the

conceptual scaffolding of advertising, such as the likelihood of modal cues and

product types to trigger different amounts of conceptual complexity in terms of

conceptual operations.

Chapter 8 (The comprehension of multimodal metaphor-metonymy combinations

in advertising: a cross-cultural investigation) offers the second practical application of

our qualitative analysis to the cognitive sciences. We report the results of two

experiments carried out to measure the extent to which different degrees of

conceptual complexity (in a continuum ranging from multimodal metonymy,

metonymic complex, metaphor, metaphtonymy, and metaphoric amalgam) in

billboards play a role in (a) the time invested in the identification of the product

and its ascribed attributes, (b) the availability of all the mappings involved in the

conceptual complex structuring the billboard that determine the extension and

richness of the interpretation, and (c) the perceived persuasive potential of the

Page 24: Expanding the figurative continuum to multimodal settings ... · (Zbikowski 2009, Pérez-Sobrino 2014a,b) and gestures (Müller and Cienki 2009) that provide further evidence of the

CHAPTER 1: Introduction __________________________________________________________________

14

message. 30 participants from three linguistic and theoretical backgrounds

(English, Chinese, and Spanish) took part in this study.

Chapter 9 (Conclusion) summarises the main proposals made in this

dissertation and speculates on potential research lines to expand this work.

The subsequent section (References) provides the reader with an exhaustive list

of bibliographical material on which this dissertation relies.

Page 25: Expanding the figurative continuum to multimodal settings ... · (Zbikowski 2009, Pérez-Sobrino 2014a,b) and gestures (Müller and Cienki 2009) that provide further evidence of the

CHAPTER 9: Conclusion _______________________________________________________________________________

227

CHAPTER 9

Conclusion

v

From a certain point onward there is no longer any turning back.

That is the point that must be reached.

FRANZ KAFKA, The Castle

v

9.1. Summary

This final chapter retrieves the main proposals made in this

dissertation. Throughout this research work we have argued for the novelty of

this research work. We therefore discuss its potential implications and

applications for Cognitive Linguistics, the cognitive sciences, and advertising

studies. We firmly believe that the research lines of this work should be

further pursued; for this reason, we suggest some ways in which this can be

done.

We began this dissertation by acknowledging an intensified shift from

verbal language toward multimodality. This dissertation has laid out in detail

a comprehensive paradigm to study new meaning-making practices in

multimodal contexts such as advertising. The model propounded herein,

which is based on the recognition of the existence of complex combinations

between metaphor and metonymy, has placed metaphor and metonymy on a

continuum of increasing conceptual complexity. Such an organization carries

with itself two crucial implications. First, the great interactional dimension of

Page 26: Expanding the figurative continuum to multimodal settings ... · (Zbikowski 2009, Pérez-Sobrino 2014a,b) and gestures (Müller and Cienki 2009) that provide further evidence of the

CHAPTER 9: Conclusion _______________________________________________________________________________

228

metaphor with other operations (such as metonymy and/or other metaphors)

allows us to expand the inventory of possible meaning construction processes.

Second, the placement of simple and complex conceptual operations along a

figurative continuum (as proposed in Gibbs 1984, Dirven 2002, McArthur

and Littlemore 2008) connects with more empirical accounts to measure the

effect of conceptual complexity on other communicative features, such as

those of speed of processing, depth of understanding, or emotional response.

In order to provide this research with a rationale, in Chapter 2 we have

offered a critical review of the existing literature on multimodal meaning-

making practices. Four theoretical perspectives have been surveyed in order to

reveal their analytical strengths and weaknesses: Visual Social Semiotics

(VSS; Kress and Leuween, 1996, 2001); Relevance Theory (RT; Sperber and

Wilson 1995) and its multimodal application (Yus 2009); Conceptual

Blending Theory (CBT; Fauconnier and Turner, 2002); and Conceptual

Metaphor Theory (CMT; Lakoff and Johnson, 1980, 1999), together with its

extension into the world of multimodal communication (Forceville and Uriós-

Aparisi 2009, and the references therein). This chapter has concluded with the

identification of a number of areas that are still in need of development, and it

has advanced the way in which this dissertation has overcome such

theoretical deficiencies.

In Chapter 3 we have argued in favour of the existence, in multimodal

contexts, of complex patterns of conceptual interaction between metaphor

and metonymy that may enrich current accounts of pictorial and multimodal

metaphor. In order to illustrate this point, we have provided an overview of

the metaphor-metonymy interactional patterns identified by Ruiz de

Mendoza and his colleagues and have classified them according to the nature

(domain internal or external) and number of mappings that they involve. We

have proposed a version of the figurative continuum that goes beyond the

traditional notion as discussed in Gibbs (1984), Dirven (2002), and McArthur

and Littlemore (2008). Whereas grounded in the same terms –namely the

increasing degree of figurativeness along a continuum from literal meaning,

through metonymy and metaphor, to ad hoc conceptual configurations– the

Page 27: Expanding the figurative continuum to multimodal settings ... · (Zbikowski 2009, Pérez-Sobrino 2014a,b) and gestures (Müller and Cienki 2009) that provide further evidence of the

CHAPTER 9: Conclusion _______________________________________________________________________________

229

multimodal version includes eight conceptual complexes that fill the gaps

between metonymy, metaphor, and ad hoc figurative combinations.

We discussed in Chapter 4 a number of methodological issues, such as

the criteria for the selection of a corpus of multimodal advertisements. We

formulated a protocol of identification, characterization, and annotation of

multimodal metaphoric and metonymic domains.

Chapters 5, 6, 7, and 8, which constitute the analytical contribution of

this dissertation, deal with issues so far unexplored in multimodal studies.

The application of the verbal complexes overviews in Chapter 3 to the

analysis of advertising –which we do in application of the equipollence

hypothesis (EH)– has allowed us to expand the original version of the

figurative continuum, as analysed in detail in Chapters 5 and 6. First, we have

included in the continuum intermediate levels of conceptual complexity

between metonymy, metaphor, and ad hoc figurative combinations in

multimodal use (such as (multiple source)-in-target ((MS)iT) metonymies,

metonymic chains, metaphtonymies, single and double source metaphoric amalgams,

and metaphoric chains). Second, the thorough examination of our multimodal

corpus has additionally shed light on novel multimodal conceptual complexes

(still unidentified in verbal use) that have allowed us to enrich the new and

improved version of the figurative continuum (see Section 9.2 on the

theoretical implications of this dissertation).

In Chapter 7 we have provided the first-large scale analysis of a

multimodal corpus (210 advertisements). We have first presented the reader

with a description of our corpus in terms of the distribution of conceptual

operations, characteristics of the representation of the promoted product, and

use of modal cues. Subsequently, we have analysed the discursive and

communicative factors that have an effect on the distribution of conceptual

operations.

Chapter 8 has described an experiment designed to assess the

comprehension of varying degrees of conceptual complexity in participants of

three different nationalities (English, Chinese, and Spanish).

Page 28: Expanding the figurative continuum to multimodal settings ... · (Zbikowski 2009, Pérez-Sobrino 2014a,b) and gestures (Müller and Cienki 2009) that provide further evidence of the

CHAPTER 9: Conclusion _______________________________________________________________________________

230

To our knowledge, this dissertation is the first broad-scale research

work to explore figurative meaning construction in multimodal settings. This

project has adopted approaches and techniques from Cognitive Linguistics,

corpus linguistics, cognitive and social psychology, and marketing. We have

provided quantitative and qualitative evidence of the representation,

frequency of appearance, and workings of the seven conceptual operations

and complexes introduced above in multimodal contexts. We have

additionally reflected on the role and figurative complexity of each conceptual

operation in relation to product types and marketing strategies. Furthermore,

we have reported the results of an experiment conducted to determine how

metaphoric and metonymic content affect both the comprehension speed and

the perceived appeal of a series of advertisements by consumers of three

different linguistic and cultural backgrounds. In this connection, we believe

that, besides linguists and cognitive scientists (especially psycholinguists), this

dissertation will also benefit advertisers and branding experts.

9.2. Theoretical implications

We wondered at the beginning of the Introduction if, and if so, how,

our application of the figurative continuum to multimodal environments can

benefit the conceptual metaphor and metonymy theory. Let us overview the

main findings (MF) made over the course of this work in relation to the main

research questions (MRQ) that motivated this dissertation, so that the reader

can assess the relevance and contribution of this research.

MF1. Metaphor-metonymy combinations in multimodal use are far more

complex than the volume of inferential activity involved in a sole metaphor

or metonymy, or of metaphor in interaction with metonymy in verbal

discourse. We formulated MRQ 1 in order to explore how metaphor interacts

with metonymy in multimodal environments. To that end, we applied five

patterns of conceptual interaction between metaphor and metonymy ((MS)iT

metonymy, metonymic chain, metaphtonymy, metaphoric amalgam, and metaphoric

Page 29: Expanding the figurative continuum to multimodal settings ... · (Zbikowski 2009, Pérez-Sobrino 2014a,b) and gestures (Müller and Cienki 2009) that provide further evidence of the

CHAPTER 9: Conclusion _______________________________________________________________________________

231

chain) to advertising. Our main finding in this regard (MF 1) is that the

interplay between visuals and text has given rise to a series of variants of the

interaction patterns between metaphor and metonymy surveyed in verbal

discourse: (MS)iT) metonymic chain, ((MS)iT) metaphtonymy, parallel metonymic

developments in both domains of a metaphtonymy, metaphtonymy scenario, single-

source metaphtonymic amalgams, multiple source /target metaphoric amalgams, and

metaphtonymic chains). See Table 16 for a summary of the novelties stemming

from our incursion into the analysis of advertising.

Conceptual complex Schematic representation New developments in multimodal use (MS)iT metonymy (3.2.1)

Metonymic chain (3.2.2)

Multimodal multiple metonymic chaining (5.2.2)

A  

a  a’

a’’

Multimodal (MS)iT chain (5.2.4)

a’’’  a  a’

a’’

A  

a  m

A  A’  

a  m

A  A’  

A’’  

Page 30: Expanding the figurative continuum to multimodal settings ... · (Zbikowski 2009, Pérez-Sobrino 2014a,b) and gestures (Müller and Cienki 2009) that provide further evidence of the

CHAPTER 9: Conclusion _______________________________________________________________________________

232

Metaphtonymy (3.2.3) Paralell metonymy (expansion/reduction) developments in both source and target domains of the multimodal metaphor (6.2.2 a,b)

Metaphtonymy scenario (6.2.2 c) (MS)iT metaphtonymy (6.2.2 d)

Metaphoric amalgam (2.1.4)

Multimodal single-source metaphtonymic amalgam (6.2.3)

SD   TD  

a  

A   B  

SD   TD  

b  A  

B  

or  

A  

SD  

C  

SD  

B  

TD  

D  

TD  

Single-source

A        

SD  

C  

SD  

B        

TD  

D  

TD  

b  

SD   TD  

b  

A  

a  

B  

SD   TD  

b  

A  

a  

B  

SD   TD  

A  

a  

B  

a’  

a  

Page 31: Expanding the figurative continuum to multimodal settings ... · (Zbikowski 2009, Pérez-Sobrino 2014a,b) and gestures (Müller and Cienki 2009) that provide further evidence of the

CHAPTER 9: Conclusion _______________________________________________________________________________

233

Double source

Multimodal multiple source metaphoric amalgam / double target metaphoric amalgam (6.2.4)

Metaphoric chain (2.1.5)

Multimodal metaphtonymic chain (6.2.5)

Table 16. Summary of the novel developments in metaphor-metonymy combinations found in multimodal use

MF2. Multimodal metonymy and metaphor (and their patterns of

interaction) can be placed along a cline of increasing qualitative complexity

that results in a finer-grained account of the figurative continuum as

originally conceived by Dirven (2002). In response to MRQ2, which

investigated how multimodal metaphor can relate to metonymy, the

multimodal variants shown in Table 16 not only enrich previous accounts on

the interactional dimension between metaphor and metonymy, but can also

fill the gaps between these two tropes in the figurative continuum (see our

proposal of an expanded version in Figure 49 below). It would be worth to

explore whether these novel complexes arising from multimodal

environments are also present in verbal contexts. Furthermore, it is worth

C    IS    D  

A    IS    B  

E    IS    F  

C  A    IS    B  

D  

SD   SD  

A   B   C  

SD  

SD  

TD  

TD  

C    IS    D  

A    IS    B  

E    IS    F  

G    IS    H  

A        

B        

C        

SD  

SD  

TD  

TD  

c  b  a  

Page 32: Expanding the figurative continuum to multimodal settings ... · (Zbikowski 2009, Pérez-Sobrino 2014a,b) and gestures (Müller and Cienki 2009) that provide further evidence of the

CHAPTER 9: Conclusion _______________________________________________________________________________

234

emphasizing that the successful application of the figurative continuum to

multimodal settings lends further support to the greater explanatory power

and efficacy of the cognitive-linguistic approach to meaning construction

phenomena.

Page 33: Expanding the figurative continuum to multimodal settings ... · (Zbikowski 2009, Pérez-Sobrino 2014a,b) and gestures (Müller and Cienki 2009) that provide further evidence of the

CHAPTER 9: Conclusion _______________________________________________________________________________

235

Figure 49. Expanded version of the figurative continuum (including novel developments arisen in multimodal use)

Lit

eral

use

Ad h

oc figurative use

Multimodal metonymy and metonymic complexes Multimodal metaphor and metaphoric complexes

DS metaphoric amalgam

Metonymy

(MS)iT metonymy

   

Metaphor

Metaphtonymy

SS metaphoric amalgam

(MS)iT metonymic chain

Metaphoric chain

   

Parallel metonymic exp./red. in SD&TD

Metaphtonymy scenario (MS)iT metaphtonymy

Metaphtonymic amalgam

Multiple source metaphoric amalgam

Double target metaphoric amalgam

Metaphtonymic chain

Metonymic chain

Multiple metonymic chain

Page 34: Expanding the figurative continuum to multimodal settings ... · (Zbikowski 2009, Pérez-Sobrino 2014a,b) and gestures (Müller and Cienki 2009) that provide further evidence of the

CHAPTER 9: Conclusion _______________________________________________________________________________

236

MF 3. The study of multimodal metaphor and metonymy provides new

insights on these two tropes that lend further support to the cognitive-

linguistic contention about the conceptual status of cognitive operations.

We sought to determine in MRQ 3 the main theoretical implications of this

dissertation for the community of metaphor and metonymy scholars in

particular, and for Cognitive Linguistics in general. Our approach to meaning

construction gives pride of place to three different factors that can measure

conceptual complexity. In this order, we have first taken into account the

nature of the conceptual domain (establishing that metonymic are less

complex than metaphorical domains), the type of interactional pattern

(whereby integration is regarded as less complex than chaining), and the

number of metonymic and/or metaphoric domains involved. It is thus

demonstrated that the economic use of visuals and text in printed advertising

is not necessarily linked to the volume and complexity of the conceptual

configuration structuring the advertisement.

9.3. Practical applications

Besides the contribution to linguistics, the practical applications of this

research point directly to the design of more effective practices for tackling

cross-cultural communication. Metaphor is a highly noticeable, persuasive,

and powerful form of communication for a number of reasons: (1) metaphor

cues for the activation of mental images, which can be used to package and

convey a large amount of information in an efficient manner; (2) it allows for

indirect expression; and (3) it is a natural component of thought and, as such,

it is evident beyond language. The embodied nature of some metaphors has

proved to be more likely to provoke an emotional response than literal forms

of expression (Citron and Goldberg 2014). This may help the recipient to

develop a personal relationship with an advertisement (Chang & Yen 2013).

In particular, we outline below three main findings of our research that point

directly towards the applications of the main insights provided herein to other

disciplines, such as marketing studies and psycholinguistic experiments.

Page 35: Expanding the figurative continuum to multimodal settings ... · (Zbikowski 2009, Pérez-Sobrino 2014a,b) and gestures (Müller and Cienki 2009) that provide further evidence of the

CHAPTER 9: Conclusion _______________________________________________________________________________

237

MF4. The conceptual scaffolding of an advertisement has a significant

effect on discursive and communicative variables such as product

representation and use of modal cues. In MRQ4 it was asked whether it was

possible to correlate conceptual complexity with external advertising

variables, thus bridging the gap between the conceptual and the discursive/

communicative dimensions of advertising. As was noted at the beginning of

this chapter, the reader will find in this dissertation a great effort to overcome

the limitations of other analyses of multimodal metaphor in advertising:

• We showed that metaphtonymy is the most frequent conceptual

operation in our corpus because it combines the potential of

metonymy in providing viewers with an economic point of access to

the advertisement with the ascription of features from a positively

connoted domain to the product via metaphorical mappings.

• The inclusion of metonymy in the source or target of metaphor

serves as a cognitively economic point of access to a broader

scenario. This makes it possible for pictorial and multimodal

metaphors to refer to abstract targets that do not necessarily need to

depict the promoted commodity, or depict it as such. This

observation paves the way for the analysis of correlational

metaphors in multimodal use: To date such metaphors have

attracted scarce attention in this field of research (with the exception

of Forceville 2011, Forceville and Jeulink, 2011, and Pérez-Sobrino

2014a,b).

• The possibility of coupling metaphor with other cognitive

mechanisms gives rise to more inferential activity than a sole

metaphorical correspondence between positive values (source

domain) and the advertised product (target domain). Therefore, the

product does not compulsorily need to be represented, but rather it

can be indirectly suggested via conceptual shortcuts. The ability of

domain expansion processes to cue for metaphors makes it usual for

advertisers to represent part of the product or part of a constructed

Page 36: Expanding the figurative continuum to multimodal settings ... · (Zbikowski 2009, Pérez-Sobrino 2014a,b) and gestures (Müller and Cienki 2009) that provide further evidence of the

CHAPTER 9: Conclusion _______________________________________________________________________________

238

situation that is aimed to evoke the product. This conceptual

strategy shifts the interpretative burden to the audiences that are

responsible for deriving the more abstract scenario in which the

product is associated with its attributes.

• In consequence, it is not a condition sine qua non either that

metaphorical targets refer to concrete entities. By contrast, abstract

entities such as brands, services, or NGOs, are usually advertised to

the public at large.

• The choice of mode to convey advertisements significantly affects

the amount of conceptual complexity involved. In turn, the type of

advertised product and the marketing strategy has no significant

effect on the number and complexity of conceptual mappings in the

advertisement. That is to say, different types of marketing strategies

do not necessarily lead to different types of conceptual operations.

MF 5. The conceptual scaffolding of an advertisement has a significant

effect on variables affecting the interpretation, such as speed of processing

and depth of interpretation. MRQ5 was formulated to explore how

audiences perceive and interpret advertisements containing different

combinations of multimodal metaphor and metonymy. We found out that

conceptual complexity was significantly related to speed of processing,

complexity of interpretation, and perceived appeal of the advertisement. In

other words, when confronted with complex advertisements, our participants

demanded more time of processing, they were more likely to produce more

different interpretations, but they were not inclined to perceive them as

effective. In turn, it was shown that conceptual complexity does hinder or

facilitate reaching the most salient interpretation of the advertisement

(although salient interpretations were produced faster). We also found a

significant effect of cultural/linguistic elements on the variables mentioned

above, with the exception of the perceived persuasive potential of the

advertisement. Given that nationality does not affect the likelihood of

Page 37: Expanding the figurative continuum to multimodal settings ... · (Zbikowski 2009, Pérez-Sobrino 2014a,b) and gestures (Müller and Cienki 2009) that provide further evidence of the

CHAPTER 9: Conclusion _______________________________________________________________________________

239

perceiving an advertisement as convincing, we conjectured that it should be

worth considering alternative psychological and socioeconomic variables.

MF 6. This dissertation has a great potential for interdisciplinary research.

Finally, we explored in MRQ 6 whether this dissertation could serve as a

starting point to frame and develop other studies applied to real life issues. In

this regard, we believe that the present research will raise awareness among

advertisers on the ways in which it is possible to make use of shared

experiential knowledge for global campaigns, while selecting specific cultural

content for local campaigns. More importantly, it will help advertisers to

avoid incongruent strategies that may render their campaigns unsuccessful,

thus saving unnecessary costs. Hence, the view adopted in this dissertation is

that the conscious incorporation of the inventory of multimodal patterns of

conceptual interaction at the stage of advertisement design could improve

levels of cross-cultural understanding of the persuasive narrative, although

such an understanding is always limited and culture-bound. The strategic

exploitation of such conceptual mechanisms during the design of an

advertisement may ensure the creation of a positive image of their promoted

products, the correct interpretation of the advertisement by audiences, and the

cancellation of misguided interpretations.

This dissertation has offered a solid theoretical basis for further

empirical investigation on multimodal communication. Over the course of

Chapters 7 and 8 we have advocated for the feasibility of a “reverse

engineering” process in order to advance our knowledge of multimodal

meaning making practices, in order to draw the attention not only of cognitive

linguists, but also of marketing experts and cognitive scientists. Our proposals

on conceptual complexity, communicative impact, and multimodality have

been reformulated as testable hypotheses in a psycholinguistic study to

confirm that conceptual complexity can be quantitatively measured.

Page 38: Expanding the figurative continuum to multimodal settings ... · (Zbikowski 2009, Pérez-Sobrino 2014a,b) and gestures (Müller and Cienki 2009) that provide further evidence of the

CHAPTER 9: Conclusion _______________________________________________________________________________

240

9.4. Further research lines

It still remains to be seen whether multimodal figurative information

evokes positive or negative attitudes towards products, as some viewers may

find overt visual and verbal metaphors less appealing. Similarly, the extent to

which multimodal advertisements evoke emotion is yet to be investigated too.

Prior work has suggested an affective response to advertising, but the

direction, valence and type of emotion awaits full exploration in multimodal

messages, e.g. whether a complex combination of metaphor and metonymy

triggers a stronger emotional response than a single metaphor would, and

which combination results in a greater appreciation of the advertisement by

the viewer.

Therefore, more attention needs to be paid to the role of emotions in

advertising in static, dynamic, and viral marketing. The few studies that have

been conducted either avoid the physiological responses and adopt self-report

measures only (Dobele et al. 2007), do not adopt any real empirical measures

(Kaplan & Haenlein 2011), or use basic word counts for the number of

positive or negative words used in particular online content (Berger &

Milkman 2011), content that is neither an advertisement or dynamic. Given

the complex nature of metaphor and metonymy, as revealed by research in

linguistics and cognitive science, future research should seek to combine their

study with knowledge of marketing practices in order to determine with

precision how advertisements are actually interpreted, and what makes them

successful or not.

Page 39: Expanding the figurative continuum to multimodal settings ... · (Zbikowski 2009, Pérez-Sobrino 2014a,b) and gestures (Müller and Cienki 2009) that provide further evidence of the

REFERENCES _______________________________________________________________________________

241

v

REFERENCES

v

Ang, S. & Lim, E. (2006). The influence of metaphors and product type on

brand personality perceptions and attitudes. Journal of Advertising

35/2: 39-53.

Babarczy, A. et al. (2010). The automatic identification of conceptual

metaphors in Hungarian texts: A corpus-based analysis. In N. Bel, B.

Daille, & A. Vasiljevs (Eds.), Methods for the automatic acquisition of

language resources and their evaluation method: Proceedings of LREC

2010 Workshop (pp. 31–36). Retrieved on 13rd June 2014 from

http://www.abstract-project.eu/papers/metaphor_malta_2.2.1.pdf

Baicchi, A., & Ruiz de Mendoza, F. J. (2011). The cognitive grounding of

illocutionary constructions within the theoretical perspective of the

Lexical Constructional Model. Textus. English Studies in Italy, 23(3),

543–563.

Barcelona, A. (Ed.). (2000). Metaphor and metonymy at the crossroads.

Berlin & New York: Mouton de Gruyter.

Barcelona, A. (2005). The multilevel operation of metonymy in grammar and

discourse, with particular attention to metonymic chains. In F. J. Ruiz

de Mendoza, & S. Pe.a (Eds.), Cognitive Linguistics. Internal

dynamics and interdisciplinary interaction (pp. 313–352). Berlin &

New York: Mouton de Gruyter.

Page 40: Expanding the figurative continuum to multimodal settings ... · (Zbikowski 2009, Pérez-Sobrino 2014a,b) and gestures (Müller and Cienki 2009) that provide further evidence of the

REFERENCES _______________________________________________________________________________

242

Barcelona, A. (2011). Reviewing the properties and prototype structure of

metonymy. In A. Barcelona, R. Benczes & F. Ruiz de Mendoza (Eds.)

Defining metonymy in a Cognitive Linguistics: Towards a consensus

view. Amsterdam/Philadelphia: John Benjamins.

Barnden, J. A. (2010). Metaphor and metonymy: Making their connections

more slippery. Cognitive Linguistics, 21(1), 1–34. DOI:

10.1515/cogl.2010.001

Beijk, J. & Van Raaij, W.F. (1989) Schemata: Informatieverwerking,

Beïnvloedingsprocessen en Reclame [Schemas: Information

Processing, Persuasion Strategies and Advertising], Amsterdam: VEA.

Berger, J., & Milkman, K. L. (2012). What Makes Online Content Viral.

Journal of Marketing Research, 49(2), 192-205.

Bhattacharjee, C. (2006). Services Marketing, 1st Edition, Excel Books, New

Delhi.

Brdar-Szab., R., & Brdar, M. (2011). What do metonymic chains reveal about

the nature of metonymy? In R. Benczes, A. Barcelona, & F. J. Ruiz de

Mendoza Ib..ez (Eds.), Defining metonymy in Cognitive Linguistics:

Towards a consensus view (pp. 217–248). Amsterdam & Philadelphia:

John Benjamins.

Burgers, C. et al. (2015). Making ads less complex, yet more creative and

persuasive: the effects of conventional metaphors and irony in print

advertising, International Journal of Advertising: The Review of

Market ing Communications.

Cacciopo, J. & Petty, R. (1982). The need for cognition. Journal of

Personality and Social Psychology 42: 116-131.

Cacioppo, J. et al. (1996). Dispositional differences in cognitive motivation:

The life and times of individuals varying in need for cognition.

Psychological Bulletin 119: 197-253.

Callow, M. & Schiffman, L. (1999). A Visual Esperanto? The Pictorial

Metaphor in Global Advertising. In B. Dubois, T. Lowrey, L. Shrum

& M. Vanhuele (Eds.) E - European Advances in Consumer Research

4 (17-20). Provo, UT: Association for Consumer Research.

Page 41: Expanding the figurative continuum to multimodal settings ... · (Zbikowski 2009, Pérez-Sobrino 2014a,b) and gestures (Müller and Cienki 2009) that provide further evidence of the

REFERENCES _______________________________________________________________________________

243

Camara-Pereira, F. (2007). Creativity and artificiall intelligence: A conceptual

blending approach. Berlin/New York: Mouton de Gruyter.

Casasanto, D. (2009). Embodiment of abstract concepts: good and bad in

right- and left-handers Journal of Experimental Psychology

(General) 138: 351–367.

Chang, C.-T. & Yen, C.-T. (2013). Missing ingredients in metaphor

advertising: The right formula of metaphor type, product type, and

need for cognition,’ Journal of advertising 42/1: 80-94.

Cienki, A. (1998). Metaphoric gestures and some of their relations to verbal

metaphoric expressions. In: Jean-Pierre Koenig (ed.), Discourse and

Cognition: Bridging the Gap, pp. 189-204. Stanford, CA: Center for

the Study of Language and Information.

Citron and Goldberg (2014). Metaphorical Sentences Are More Emotionally

Engaging than Their Literal Counterparts. Journal of Cognitive

Neuroscience X:Y, pp. 1 –11 doi:10.1162/jocn_a_00654

Clark, H. & Clark, E. (1977). Psychology of Language: An Introduction to

Psycholinguistics. New York: Harcourt Brace Jovanovich.

Cook, G. (1992). The Discourse of Advertising (revised edition published in

2001). London/New York: Routledge.

Copeland, M. (1924). Principles of Merchandising. Chicago: A. W. Shaw.

Costa, P.T & McCrae, R.R. (1992). Revised NEO Personality Inventory

(NEO-PI-R) and NEO Five-Factor Inventory (NEO-FFI) manual.

Odessa, FL: Psychological Assessment Resources.

Coulson, S. (1996). The Menendez Brothers Virus: Analogical Mapping in

Blended Spaces. In A. Goldberg (Ed.) Conceptual Structure,

Discourse, and Language (67-81). Palo Alto, CA: CSLI.

Cruse, D. (1986). Lexical Semantics. (=Cambridge Textbooks in Linguistics):

Cambridge: Cambridge University Press.

Deignan, A (2005a). Metaphor and Corpus Linguistics. John Benjamins.

Deignan, A (2005b). A corpus-linguistic perspective on the relationship

between metonymy and metaphor. Style. 39(1): 72-91.

Page 42: Expanding the figurative continuum to multimodal settings ... · (Zbikowski 2009, Pérez-Sobrino 2014a,b) and gestures (Müller and Cienki 2009) that provide further evidence of the

REFERENCES _______________________________________________________________________________

244

Dens, N. & De Pelsmacker, P. (2010). Consumer responses to different

advertising appeals for new products: the moderating influence of

branding strategy and product category involvement. Journal of brand

management 18 (1): 50-65.

Dirven, R. (2002). Metonymy and metaphor: Different mental strategies of

conceptualization. In R. Dirven & R. Pörings (Eds.) Metaphor and

metonymy in comparison and contrast. Berlin/New York: Mouton de

Gruyter, 75-112.

Dirven, R., & Ruiz de Mendoza, F. (2010). Looking back at 30 years of

Cognitive Linguistics. In E. Tabakowska, M. Choiński, & L. Wiraszka

(Eds.), Cognitive Linguistics in action. From theory to application and

back (pp. 13–70). Berlin & New York: Mouton de Gruyter.

Dobele, A. et al. (2007). Why pass on viral messages? Because they connect

emotionally. Business Horizons, 50, 291-304.

Englund, A. (2010). Intermedial Topography and Metaphorical Interaction.

In L. Elleström (Ed.) Media Borders, Multimodality and

Intermediality. Basingstoke, UK: Palgrave MacMillan.

Falk, D. (2000). Hominid brain evolution and the origin of music. In N.L.

Wallin, B. Merker & S. Brown (eds) The Origins of Music. Cambridge,

MA: MIT press, 197-216.

Fauconnier, G. (1994). Mental spaces: Aspects of meaning construction in

natural language. 2nd ed. Cambridge: Cambridge University Press.

DOI: 10.1017/CBO9780511624582

Fauconnier, G. (1997). Mappings in thought and language. Cambridge:

Cambridge University Press.

Fauconnier, G. (2001). Conceptual blending. Entry for The Encyclopedia of

the Social and Behavioral Sciences (2495-2498) Retrieved on 18th

February 2015 from:

http://www.cogsci.ucsd.edu/~faucon/BEIJING/blending.pdf

Fauconnier, G., & Sweetser, E. (Eds.) (1996). Spaces, Worlds, and Grammar.

Chicago University Press.

Fauconnier, G., & Turner, M. (1998). Conceptual integration networks.

Page 43: Expanding the figurative continuum to multimodal settings ... · (Zbikowski 2009, Pérez-Sobrino 2014a,b) and gestures (Müller and Cienki 2009) that provide further evidence of the

REFERENCES _______________________________________________________________________________

245

Cognitive Science, 22 (2), 133–187. DOI:

10.1207/s15516709cog2202_1

Fauconnier, G., & Turner, M. (2002). The way we think: Conceptual

blending and the mind’s hidden complexities. New York: Basic Books

Feldman, J. (2006). From Molecule to Metaphor: A Neural Theory of

Language. Cambridge, MA: The MIT Press

Feng, D., & O'Halloran, K. (2013). The multimodal representation of

emotion in film: Integrating cognitive and semiotic approaches.

Semiotica 197: 79-100.

Forceville, Ch. (1996). Pictorial metaphor in advertising. Routledge, London

and New York.

Forceville, Ch. (1999). Educating the eye? Kress and Van Leeuwen’s Reading

Images: The Grammar of Visual Design (1996). Review article,

Language and Literature 8(2): 163-178.

Forceville, Ch. (2006). Non-verbal and multimodal metaphor in a cognitivist

framework: agendas for research. In G. Kristiansen, M. Achard, R.

Dirven, F. Ruiz de Mendoza Ibàňez (eds.) Cognitive Linguistics:

Current Applications and Future Perspectives. Mouton de Gruyter,

Berlin and New York, 379–402.

Forceville, Ch. (2007). Multimodal metaphor in ten Dutch TV commercials.

The Public Journal of Semiotics 1 (1), 15–34.

Forceville, Ch. (2009a). The role of non-verbal metaphor sound and music in

multimodal metaphor. In Ch. Forceville, & E. Urios-Aparisi (Eds.),

Multimodal metaphor (pp. 383–400). Berlin & New York: Mouton de

Gruyter.

Forceville, Ch. (2009b). Metonymy in visual and audiovisual discourse. In E.

Ventola, & A. J. Moya (Eds.), The world told and the world shown:

Multisemiotic issues (56–74). Basingstoke, UK: Palgrave-McMillan.

Forceville, Ch. (2009c). Review of Carroll (2009). Language and Literature

18, 405–408.

Forceville, Ch. (2011). A Course in Pictorial and Multimodal Metaphor.

Retrieved April 19th, 2012.

Page 44: Expanding the figurative continuum to multimodal settings ... · (Zbikowski 2009, Pérez-Sobrino 2014a,b) and gestures (Müller and Cienki 2009) that provide further evidence of the

REFERENCES _______________________________________________________________________________

246

http://semioticon.com/sio/courses/pictorial-multimodal-metaphor/

Forceville, Ch. (2014). Relevance Theory as model for analysing visual and

multimodal communication. In D. Machin (ed.) Visual

Communication (51-70). Berlin: Mouton de Gruyter.

Forceville, Ch. & Jeulink, M. (2011). The flesh and blood of embodied

understanding: the source-path-goal schema in animation film.

Pragmatics & Cognition 19(1): 37-59.

Forceville, Ch. & Urios-Aparisi, E. (eds.) (2009). Multimodal Metaphor.

Mouton de Gruyter, Berlin and New York.

Gibbons, A. (2010). Narrative worlds and multimodal figures in House of

Leaves: “-find your own words; I have no more. In M. Grishakova, &

M. Ryan (Eds.) Intermediality and Storytelling (285-311). Berlin:

Walter de Gruyter.

Gallese, V. & Lakoff, G. (2005). The brain’s concepts: the role of the sensory-

motor system in conceptual knowledge. Cognitive Neuropsychology,

1(1): 1.

Gibbons, Alison. 2012. Multimodality, Cognition, and Experimental

Literature. Routledge.

Gibbs, R. (1994). The poetics of mind: Figurative thought, language, and

understanding. New York: Cambridge University Press.

Gibbs, R. (2000). Irony in talk among friends. Metaphor and Symbol, 15, 5–

27.

Gibbs, R. (2001). Evaluating contemporary models of figurative language

understanding. Metaphor and Symbol 16(3/4): 317-333.

Gibbs, R. (2006a). Metaphor interpretation as embodied simulation. Mind

and Language 21(3): 434.

Gibbs, R. (2006b). Introspection and cofnitive linguistics: Should we trust our

own intuitions? Annual Review of Cognitive Linguistics 4: 135-151.

Gibbs, R. (2007). Why cognitive linguistic should care more about empirical

methods. In M. Gonzales, M. Spivey, S. Coulson, & I. Mittelberg

(Eds.), Empirical methods in cognitive linguistics (pp. 2–18).

Amsterdam, The Netherlands: John Benjamins.

Page 45: Expanding the figurative continuum to multimodal settings ... · (Zbikowski 2009, Pérez-Sobrino 2014a,b) and gestures (Müller and Cienki 2009) that provide further evidence of the

REFERENCES _______________________________________________________________________________

247

Gibbs, R. (2011). Evaluating Conceptual Metaphor Theory. Discourse

Processes, 48(8): 529–562.

Gibbs, R., Bogdonovic, J., Sykes, J. & Barr, D. (1997). Metaphor in idiom

comprehension. Journal of Memory and Language 37: 141-154.

Gibbs, R. & Colston, H. (1995). The cognitive psychological reality of image

schemas and their transformations. Cognitive Linguistics 6: 347-378.

Gibbs, R. & Colston, H. (2012). Interpreting Figurative Meaning. Cambridge:

Cambridge University Press.

Giora, R. (2002). On our mind: Salience, context, and figurative language.

New York: Oxford University Press.

Gkiouzepas, L. & Hogg, M. (2011). Articulating a New Framework for

Visual Metaphors in Advertising. Journal of Advertising 40 (1). 103-

120.

Gonzálvez, F. (2011). Metaphor and metonymy do not render coercion

superfluous: Evidence from the subjective-transitive construction.

Linguistics. Volume 49 (6): 1305–1358.

Goossens, L. (1990). Metaphtonymy: The interaction of metaphor and

metonymy in expressions for linguistic action. Cognitive Linguistics,

1(3), 323–340. DOI: 10.1515/cogl.1990.1.3.323Grady (1997)

Grady, J. (1999). A typology of motivation for conceptual metaphor:

correlation vs. resemblance. In R. W. Gibbs, & G. Steen (Eds.),

Metaphor in cognitive linguistics (pp. 79–100). Amsterdam &

Philadelphia: John Benjamins.

Grady, J. (2005). Primary metaphors as inputs to conceptual integration.

Journal of Pragmatics 37: 1595–1614.

Halliday, M. (1978). Language as a Social Semiotic: The Social Interpretation

of Language and Meaning. London: Edward Arnold.

Halliday, M. (1994). Introduction to Functional Grammar. London: Edward

Arnold.

Harder, P. (2003). Mental spaces: Exactly when do we need them?

Cognitive Linguistics 14: 91-96.

Haser, V. (2005). Metaphor, metonymy, and experientialist philosophy:

Page 46: Expanding the figurative continuum to multimodal settings ... · (Zbikowski 2009, Pérez-Sobrino 2014a,b) and gestures (Müller and Cienki 2009) that provide further evidence of the

REFERENCES _______________________________________________________________________________

248

Challenging cognitive semantics. Berlin, Germany, and New York,

NY: Mouton de Gruyter.

Hidalgo, L., & Kraljevic, B. (2011). Multimodal metonymy and metaphor as

complex discourse resources for creativity in ICT advertising

discourse. In F. Gonzálvez, S. Peña, & L. Pérez-Hernández (Eds.),

Metaphor and metonymy revisited beyond the Contemporary Theory

of Metaphor. Special issue of the Review of Cognitive Linguistics,

9(1), 153–178.

Ibarretxe, I. (2008). Vision metaphors for the intellect: Are they really cross-

linguistic? Atlantis 30(1): 15-33.

Jeong, S.-H. (2007). Effects of News About Genetics and Obesity on

Controllability Attribution and Helping Behavior. Health

Communication 22(3): 221-228.

Jeong, S.-H. (2008). Visual Metaphor in Advertising: Is the Persuasive Effect

Attributable to Visual Argumentation or Metaphorical Rhetoric?

Journal of Marketing Communications 14 (1): 59-73.

Jewitt, Carey (Ed.), 2009. The Routledge Handbook of Multimodal Analysis.

Routledge, London.

Johnson, M. (1987). The body in the mind: The bodily basis of meaning,

imagination, and reason. Chicago: University of Chicago Press.

Joy et al. (2009). Conceptual blending in advertising. Journal of Business

Research 62: 39–49.

Kaplan, A. M., & Haenlein, M. (2011). Two Hearts in Three-quarter Time:

How to Waltz the Social Media/Viral Marketing Dance. Business

Horizons, 54, 253-263.

Katz, A. & Ferretti, T. (2001). Moment-by-moment comprehension of

proverbs in discourse. Metaphor and Symbol 16 (3/4): 193-221.

Kertész, A & Rákosi, (2009). Cyclic vs. circular argumentation in the

Conceptual Metaphor Theory. Cognitive Linguistics 20: 703–732.

Kitchen, P. (ed.) (2008). Marketing metaphors and metamorphosis.

Basingstoke, UK: Palgrave McMillan.

Page 47: Expanding the figurative continuum to multimodal settings ... · (Zbikowski 2009, Pérez-Sobrino 2014a,b) and gestures (Müller and Cienki 2009) that provide further evidence of the

REFERENCES _______________________________________________________________________________

249

Klepousniotou, E., & Baum, S. R. (2007). Disambiguating the ambiguity

advantage effect in word recognition: An advantage for polysemous

but not homonymous words. Journal of Neurolinguistics, 20, 1–24.

Koller, V. (2009). Brand images: Multimodal metaphor in corporate branding

messages. In Ch. Forceville & E. Urios-Aparisi (eds), Multimodal

metaphor (45-71). Berlin/New York: Mouton de Gruyter.

Kotler, P. & Armstrong, G. (1997). Marketing An Introduction (4th Ed.) New

Jersey: Prentince Hall International

Kövecses, Z. (1990). Emotion Concepts. Berlin/New York: Springer-Verlag.

Kövecses, Z. (2000). Metaphor and emotion. Cambridge: Cambridge

University Press.

Kövecses, Z. (2002). Metaphor: A practical introduction. Oxford: Oxford

University Press.

Kövecses, Z. (2005). Metaphor in culture: Universality and variation.

Cambridge: Cambridge University Press.

Kövecses, Z., & Radden, G. (1998). Metonymy: Developing a cognitive

linguistic view. Cognitive Linguistics, 9, 37–77.

Kress, G. (2010). Multimodality: A Social Semiotic Approach to

Contemporary Communication. Routledge, London.

Kress, G. & Leeuwen, T. (1996). Reading Images: The Grammar of Visual

Design. Routledge, London, Revised edition published in 2006.

Kress, G. & Leeuwen, T. (2001). Multimodal Discourse: The Modes and

Media of Contemporary Communication. Arnold, London.

Kress, G. & Leeuwen, T. (2006). Reading Images: The Grammar of Visual

Design, 2nd ed. Routledge, London.

Lakoff, G. (1987). Women, fire, and dangerous things: What categories reveal

about the mind. Chicago: University of Chicago.

Lakoff, G. (1993). The contemporary theory of metaphor. In A. Ortony (Ed.),

Metaphor and thought (2nd ed.) (pp. 202–251). Cambridge:

Cambridge University Press.

Lakoff, G. (2008). The neural theory of metaphor. In R. Gibbs (Ed.),

Cambridge handbook of metaphor and thought. New York:

Page 48: Expanding the figurative continuum to multimodal settings ... · (Zbikowski 2009, Pérez-Sobrino 2014a,b) and gestures (Müller and Cienki 2009) that provide further evidence of the

REFERENCES _______________________________________________________________________________

250

Cambridge University Press.

Lakoff, G., & Johnson, M. (1980). Metaphors we live by. Chicago, IL:

University of Chicago Press.

Lakoff, G., & Johnson, M. (1999). Philosophy in the flesh. New York: Basic

Books.

Lakoff, G., & Turner, M. (1989). More Than Cool Reason: A Field Guide to

Poetic Metaphor. The University of Chicago Press, Chicago, IL.

Langacker, R. W. (1993). Reference-point constructions. Cognitive

Linguistics, 4: 1–38.

Langacker, Ronald W. 2009. Investigations in Cognitive Grammar. Berlin:

Mouton de Gruyter.

Lester, Paul Martin. (1995) Syntactic Theory of Visual Communication.

Retrieved on 21st February 2015 from:

http://commfaculty.fullerton.edu/lester/writings/viscomtheory.html

Levinson, S. C. (2000). Presumptive meanings: The theory of generalized

conversational implicature. Cambridge, MA & London, England: MIT

Press.

Levinson, S. C. (2009). Language and mind: Let's get the issues straight! In S.

D. Blum (Ed.), Making sense of language: Readings in culture and

communication (pp. 95-104). Oxford: Oxford University Press.

Lundmark, Carita (2003). Puns and blending: The case of print

advertisements. Paper presented at the 8th International Cognitive

Linguistics Conference. Logroño, 20–25 July 2003. Retrieved 21st

March 2013.

http://wwwling.arts.kuleuven.ac.be/iclc/Papers/Lundmark.pdf.

Mairal Usón, R. & Gonzálvez, F. (2010). Verbos y construcciones en el

espacio cognitivo-funcional del siglo XXI. In J. Val Álvaro & M.

Horno (Eds.) La Gramática del Sentido: Léxico y Sintaxis en la

Encrucijada. Conocimiento, Lenguaje y Comunicación / Knowledge,

Language and Communication, 3, Zaragoza, Prensas Universitarias de

Zaragoza.

Mairal, R., & Ruiz de Mendoza, F. (2009). Levels of description and

Page 49: Expanding the figurative continuum to multimodal settings ... · (Zbikowski 2009, Pérez-Sobrino 2014a,b) and gestures (Müller and Cienki 2009) that provide further evidence of the

REFERENCES _______________________________________________________________________________

251

explanation in meaning construction. In C. Butler, & J. Mart.n Arista

(Eds.), Deconstructing constructions (pp. 153–198). Amsterdam &

Philadelphia: John Benjamins.

MacArthur, F. & Littlemore, J. (2008). Exploring the Figurative Continuum:

A Discovery Approach Using Corpora in the Foreign Language

Classroom. In F.S. Boers & S. Lindstromberg (Eds.) Cognitive

Linguistic Approaches to Teaching Vocabulary and Phraseology.

Amsterdam: Mouton de Gruyter. 159-188.

McQuarrie, E. & Mick, D. (1999). Visual rhetoric in advertising: text

interpretive, experimental and reader-response analysis”. Journal of

Consumer Research 26, 37–53.

McQuarrie, E. & Mick, D. (2003). The contribution of semiotic and

rhetorical perspectives to the explanation of visual persuasion in

advertising. In L. Scott & R. Batra (eds.), Persuasive Imagery: A

Consumer Response Perspective (191–221). Mahwah, NJ: Lawrence

Erlbaum.

McQuarrie, E. & Mick, D. (2009). A laboratory study of the effect of verbal

rhetoric versus repetition when consumers are not directed to process

advertising. International Journal of Advertising, 28(2): 287-312.

McQuarrie, E.F. & Phillips, B. (2005). Indirect persuasion in advertising:

How consumers process metaphors presented in pictures and words.

Journal of Advertising 34/2: 7-20.

Mick, D.G. and Politi, L.G. (1989) ‘Consumers’ interpretations of advertising

imagery: a visit to the hell of connotation’, in E.Hirschman (ed.)

Interpretive Consumer Research, Provo, UT: Association for

Consumer Research, 85–96.

Mitchell, A. & Olson, J. (1981). Are product attribute belieft the only

mediator of advertising effects on brand attitudes? Journal of

Marketing Research 18: 318-332.

Morgan, S. & Reichert, T. (1999). The message is in the metaphor: Assessing

the comprehension of metaphors in advertisements. Journal of

Advertising 28/4: 1-12.

Page 50: Expanding the figurative continuum to multimodal settings ... · (Zbikowski 2009, Pérez-Sobrino 2014a,b) and gestures (Müller and Cienki 2009) that provide further evidence of the

REFERENCES _______________________________________________________________________________

252

Moya, A. & Pinar, M. (2008). Compositional, interpersonal and

representational meanings in a children’s narrative. A multimodal

discourse analysis. Journal of Pragmatics 40(9): 1601-1619.

Mulken, M. v. et al. (2010). The impact of perceived complexity, deviation

and comprehension on the appreciation of visual metaphor in

advertising across three European countries. Journal of Pragmatics 42:

3418–3430.

Müller, C. & Cienki, A. (2009). Words, gestures, and beyond: forms of

multimodal metaphor in the use of spoken language. In Ch. Forceville

& E. Urios-Aparisi (eds.), Multimdodal metaphor (297-328).

Berlin/New York: Mouton de Gruyter.

Murphy, G. (1996). On metaphoric representation. Cognition 60: 173-204.

Musolff, A. (2006). Metaphor scenarios in public discourse. Metaphor and

Symbol 21: 23-38.

Nikolajeva, M. & Scott, C. (2001). How Picturebooks Work (Children's

Literature and Culture). London: Garland Publishing.

Norris, S. (2009). Modal density and modal configurations: Multimodal

actions. In Jewit , C. (ed) Routledge Handbook for Multimodal

Discourse Analysis (78-90). London: Routledge.

O’Halloran, L. (2005). Mathematical Discourse: Language, Symbolism and

Visual Images. London and New York: Continuum.

O'Toole, Michael (2010) The Language of Displayed Art (Second edition).

New York: Routledge.

Oakley, T. (1996). Conceptual Blending and Counterfactual Spaces. In A.

Monaghan (Ed.) The Fifth International Conference on the Cognitive

Science of Natural Language Processing. Dublin: Natural Language

Group.

Ortiz, María J. (2011). “Primary metaphors and monomodal visual

metaphors.” Journal of Pragmatics 43: 1568-1580.

Panther, K.-U. & Radden, G. (1999). Metonymy in Language and Thought.

Amsterdam & Philadelphia: John Benjamins.

Page 51: Expanding the figurative continuum to multimodal settings ... · (Zbikowski 2009, Pérez-Sobrino 2014a,b) and gestures (Müller and Cienki 2009) that provide further evidence of the

REFERENCES _______________________________________________________________________________

253

Parry et al. (2013). ‘Shockvertising’: An exploratory investigation into

attitudinal variations and emotional reactions to shock advertising.

Journal of Consumer Behaviour 12: 112–121.

Pérez-Hernández, L. (2011). Cognitive Tools for Successful Branding.

Applied Linguistics 32/4: 369-388.

Pérez-Hernández, L. (2013a). Illocutionary constructions: (multiple-source)-

in-target metonymies, illocutionary ICMs, and specification links.

Language & Communication, 33(2), 128–149.

Pérez-Hernández, L. (2013b). Approaching the utopia of a global brand: The

relevance of image schemas as multimodal resources for the branding

industry. Review of Cognitive Linguistics 11(2): 285–302.

Pérez Hernández, L. (2014). Cognitive grounding for cross-cultural

commercial communication. Cognitive Linguistics 25(2): 203 – 247.

Pérez-Hernández, L. & Ruiz de Mendoza, F.J. (2011). A lexical-

constructional model account of illocution. VIAL 8: 98-137.

Pérez-Sobrino, P. (2013). Metaphor use in advertising: analysis of the

interaction between multimodal metaphor and metonymy in a

greenwashing advertisement. In E. Gola & F. Ervas (eds.) Metaphor in

Focus: Philosophical Perspectives on Metaphor Use. Cambridge:

Cambridge Scholars Publishing. 67-82.

Pérez-Sobrino, P. (2014a). Multimodal cognitive operations in classical

music. Vigo International Journal of Applied Linguistics 11: 137-168.

Pérez-Sobrino, P. (2014b). Conceptual disintegration and multimodal

metonymy in musical understanding. Journal of Pragmatics 70: 130-

151.

Pérez-Sobrino, P. (accepted). Conceptual complexity in advertisements:

Expanding the figurative continuum in multimodal settings. Applied

Linguistics.

Petäjäaho, E. (2012). (Non-)metaphorical meaning constructions in

advertising: a comparative study between American and Finnish beer

commercials. Doctoral dissertation. Free University of Amsterdam.

Retrieved on 12th February 2014 from:

Page 52: Expanding the figurative continuum to multimodal settings ... · (Zbikowski 2009, Pérez-Sobrino 2014a,b) and gestures (Müller and Cienki 2009) that provide further evidence of the

REFERENCES _______________________________________________________________________________

254

http://www.metaphorlab.vu.nl/en/Images/Eveliina%20thesis_tcm11

3-368039.pdf

Phillips, B. & McQuarrie, E. (2009). Impact of Advertising Metaphor on

Consumer Belief: Delineating the Contribution of Comparison Versus

Deviation Factors. Journal of Advertising 38 (1): 49-62.

Radden, Günter. 2000. How metonymic are metaphors? In Antonio

Barcelona (ed.) Metaphor and Metonymy at the Crossroads. A

Cognitive Perspective (pp. 93–108). Berlin/New York: Mouton de

Gruyter.

Ritchie, D. (2003). Argument is war—Or is it a game of chess? Multiple

meanings in the analysis of implicit metaphors. Metaphor and Symbol,

18(2), 125–146.

Ritchie, D. (2004). Lost in “conceptual space”: Metaphors of conceptual

integration. Metaphor and Symbol, 19(1), 31–50.

Ruiz de Mendoza, F. J. (1998). On the nature of blending as a cognitive

phenomenon. Journal of Pragmatics, 30, 259–274.

Ruiz de Mendoza, F. (2000). The role of mappings and domains in

understanding metonymy. In A. Barcelona (Ed.), Metaphor and

metonymy at the crossroads (pp. 109–132). Berlin & New York:

Mouton de Gruyter.

Ruiz de Mendoza, F. (2002). From semantic underdetermination, via

metaphor and metonymy to conceptual interaction. Theoria et Historia

Scientiarum. An International Journal for Interdisciplinary Studies,

1(6), 107–143.

Ruiz de Mendoza, F. (2005). Linguistic interpretation and cognition. In E.

Croitoru, D. Tuchel & M. Praisler (eds.) Cultural Matrix Reloaded.

Romanian Society for English and American Studies. Seventh

International Conference (36–64). Bucarest: Didactica Si Pedagogica.

Ruiz de Mendoza, F. J. (2007). High-level cognitive models: In search of a

unified framework for inferential and grammatical behavior. In K.

Kosecki (Ed.), Perspectives on metonymy (pp. 11–30). Frankfurt &

Main: Peter Lang.

Page 53: Expanding the figurative continuum to multimodal settings ... · (Zbikowski 2009, Pérez-Sobrino 2014a,b) and gestures (Müller and Cienki 2009) that provide further evidence of the

REFERENCES _______________________________________________________________________________

255

Ruiz de Mendoza, F. J. (2011). Metonymy and cognitive operations. In R.

Benczes, A. Barcelona, & F. J. Ruiz de Mendoza (Eds.), Defining

metonymy in Cognitive Linguistics. Towards a consensus view (pp.

103–123). Amsterdam & Philadelphia: John Benjamins.

Ruiz de Mendoza, F. J. (2013). Meaning construction, meaning

interpretation, and formal expression in the Lexical Constructional

Model. In B. Nolan, & E. Diedrichsen (Eds.), Linking constructions

into functional linguistics: The role of constructions in gramar (pp.

231–270). Amsterdam & Philadelphia: John Benjamins.

Ruiz de Mendoza, F. (2014a). Mapping concepts: Understanding figurative

thought from a cognitive-linguistic perspective. Spanish Journal of

Applied Linguistics 27(1): 187–207.

Ruiz de Mendoza, F. (2014b). On the nature and scope of metonymy in

linguistic description and explanation: towards settling some

controversies. In J. Littlemore, & J. Taylor (eds.) Bloomsbury

Companion to Cognitive Linguistics. London: Bloomsbury; 143-166.

Ruiz de Mendoza, F. & Baicchi, A. (2007). Illocutionary Constructions:

Cognitive motivation and linguistic realization. In I. Kecskes, & L.

Horn (Eds.), Explorations in pragmatics: Linguistic, cognitive, and

intercultural aspects (pp. 95–128). Berlin & New York: Mouton de

Gruyter.

Ruiz de Mendoza, F. & Díez, O. (2002). Patterns of conceptual interaction.

In R. Dirven, & R. Pörings (Eds.), Metaphor and metonymy in

comparison and contrast (pp. 489–532). Berlin & New York: Mouton

de Gruyter.

Ruiz de Mendoza, F. & Galera, A. (2011). Going beyond metaphtonymy:

Metaphoric and metonymic complexes in phrasal verb interpretation.

Language Value, 3(1),1–29.

Ruiz de Mendoza, F. & Galera, A. (2014). Cognitive modeling: A linguistic

perspective. Amsterdam & Philadelphia: John Benjamins.

Page 54: Expanding the figurative continuum to multimodal settings ... · (Zbikowski 2009, Pérez-Sobrino 2014a,b) and gestures (Müller and Cienki 2009) that provide further evidence of the

REFERENCES _______________________________________________________________________________

256

Ruiz de Mendoza, F. & Mairal, R. (2008). Levels of description and

constraining factors in meaning construction: an introduction to the

Lexical Constructional Model. Folia Linguistica 42(2): 355–400.

Ruiz de Mendoza, F. & Mairal, R. (2011). Constraints on syntactic

alternation: lexical-constructional subsumption in the Lexical

Constructional Model. In P. Guerrero (Ed.), Morphosyntactic

alternations in English. Functional and cognitive perspectives (pp. 62–

82). London, UK & Oakville, CT: Equinox.

Ruiz de Mendoza, F. & Pérez, L. (2003). Cognitive operations and pragmatic

implication. In K.-U. Panther & L. Thornburg (Eds.), Metonymy and

pragmatic inferencing (pp. 23–49). Amsterdam & Philadelphia: John

Benjamins.

Ruiz de Mendoza, F. & Pérez-Hernández, L. (2011). The contemporary

theory of metaphor: Myths, developments and challenges. Metaphor

and Symbol 26: 161–185.

Rundbland, G. & Annaz, D. (2010). Development of metaphor and

metonymy comprehension: Receptive vocabulary and conceptual

knowledge. British Journal of Developmental Psychology 28 (3): 547–

563.

Schacter, D. S., Gilbert, D. T., & Wegner, D. M. (2011). Psychology (2nd

ed.). New York: Worth.

Schalley, A. (2012). Practical theories and empirical practice – facets of a

complex interaction. In A. Schalley (Ed.), Practical Theories and

Empirical Practice. A Linguistic Perspective (1–34). Amsterdam &

Philadelphia: John Benjamins.

Sergent, J. et al. (1992) Distributed neural network underlying musical sight-

reading and keyboard performance. Sci 257: 106-109.

Seuren, P. (1988). The self-styling of relevance theory. Journal of Semantics 5:

123-143.

Sperber, D., & Wilson, D. (1985/1986). Loose talk. Proceedings of the

Aristotelian Society LXXXVI, 153–71.

Sperber, D., & Wilson, D. (1995). Relevance. Communication and cognition.

Page 55: Expanding the figurative continuum to multimodal settings ... · (Zbikowski 2009, Pérez-Sobrino 2014a,b) and gestures (Müller and Cienki 2009) that provide further evidence of the

REFERENCES _______________________________________________________________________________

257

Oxford: Basil Blackwell.

Stadler, J. (2010). AIDS ads: make a commercial, make a difference?

Corporate social responsibility and the media. Continuum 18 (4): 591-

610.

Steen, G. (2007). Finding metaphor in grammar and usage.

Amsterdam/Philadelphia: John Benjamins.

Steen, G. et al. (2010). A method for linguistic metaphor identification: From

MIP to MIPVU. Amsterdam/Philadelphia: John Benjamins.

Stefanowitsch, A. (2006). Corpus-based approaches to metaphor and

metonymy. In A. Stefanowitsch & S. T. Gries (Eds.), Corpus-based

approaches to metaphor and metonymy (pp. 1–16). Berlin, Germany,

and New York, NY: Mouton de Gruyter.

Tendahl, M. (2009). A hybrid theory of metaphor: Relevance Theory and

Cognitive Linguistics. Basingstoke, UK: Palgrave Macmillan.

Tendahl, M. & Gibbs, R. (2008). Complementary perspectives on metaphor:

Cognitive Linguistics and Relevance Theory. Journal of Pragmatics,

40, 1823–1864.

Ting, H. & de Run, E. (2012). Generations X and Y Attitude towards

Controversial Advertising. Asian Journal of Business Research 2 (2):

18-32.

Toncar, M., & Munch, J. (2001). Consumer responses to tropes in print

advertising. Journal of Advertising, 30(1), 55-65.

Turner, M. & Fauconnier, G. (1995). Conceptual integration and formal

expression. Metaphor and Symbolic Activity. 10 (3): 183-203.

Turner, M. & Fauconnier, G. (1998). Metaphor, Metonymy, and Binding.

Retrieved 13th February 2014 from:

http://markturner.org/metmet.html.

Tynan, C. et al. (2006), “Co-creating value for luxury brands”, Journal of

Business Research 63(11): 1156-63.

Tzanne, A. (2013). When the advertised product is not the target: multimodal

metaphor in Greek TV commercials. Brno Studies in English 39 (1):

107-128.

Page 56: Expanding the figurative continuum to multimodal settings ... · (Zbikowski 2009, Pérez-Sobrino 2014a,b) and gestures (Müller and Cienki 2009) that provide further evidence of the

REFERENCES _______________________________________________________________________________

258

Urios-Aparisi, E. (2009). Interaction of multimodal metaphor and metonymy

in TV commercials: Four case studies. In C. Forceville, & E. Urios-

Aparisi (Eds.), Multimodal metaphor (pp. 95–118). Berlin & New

York: Mouton de Gruyter.

Velasco, M. & Fuertes, P. (2006). Olfactory and olfactory-mixed metaphors in

print ads of perfume. Annual Review of Cognitive Linguistics 4(1):

217-252.

Ventola, E. & A. Moya (Eds) (2009). The World Told and the World Shown:

Issues in Multisemiotics. Basingstoke, UK: Palgrave MacMillan.

Vervaeke, J., & Kennedy, J. M. (1996). Metaphors in language and thought:

Falsification and multiple meanings. Metaphor and Symbolic Activity,

11(4), 273–284.

Villacañas, B. & White, M. (2013). Pictorial metonymy as creativity source in

Purificación García advertising campaigns. In L. Hidalgo & B.

Kraljevic (eds.) Metaphorical creativity across modes: Special issue of

Metaphor and the Social World 3(2): 220–239.

Waller, D. (2004). What factors make controversial advertising offensive?: A

Preliminary Study. ANZCA 2004 Proceedings: 1-10.

Perreault, W. & McCarthy, J. (2002). Basic Marketing: A Global Managerial

Approach. New York: McGraw Hill.

Yus, F. (2005). Ad hoc concepts and visual metaphor? Towards relevant ad

hoc pointers, paper delivered at the 9th International Pragmatics

Conference, Riva del Garda (Italy), July.

Yus, F. (2009). “Visual metaphor versus verbal metaphor: A unified

account.” In: Forceville and Urios-Aparisi (eds), pp. 147-172.Zatorre,

R. A. et al. (1992): Lateralization of phonetic pitch discrimination in

speech processing. Sci 256, 846-9

Zbikowski, L. (2002). Conceptualizing Music: Cognitive Structure, Theory,

and Analysis. (AMS Studies in Music.) New York: Oxford University

Press.