L 8-9 Musical Scales, Chords, and Intervals, The Pythagorean and Just Scales.
Musical Chords Influence Responses to a Non-Affective ...
Transcript of Musical Chords Influence Responses to a Non-Affective ...
ABSTRACT
BAKER, JAMES DANIEL. A Study of the Influences of Musical Chords on Responses to a
Non-Affective Visual Feature of Emotional Faces. (Under the direction of Dr. Douglas J.
Gillan.)
Under a variety of everyday circumstances, a musical stimulus may accompany a task
that requires visual information processing. There are many psychological mechanisms by
which the musical stimulus may actually influence the required visual information
processing. One such mechanism may be affective priming. The results of several studies
suggest that automatic processing of the affective connotations of musical stimuli can
interfere with the controlled processing of the affective connotations of visually displayed
words and human faces. The present experiment measured the extent to which consonant
and dissonant musical chords can prime responses to a non-affective feature of schematic
faces depicting happiness or anger. In each of the key conditions, the chord and the face
were either affect-congruent (e.g., consonant chord and happy face) or affect-incongruent
(e.g., consonant chord and angry face). Rather than responding to the affective valence of
each face, the undergraduate participants (N = 28) affirmed or denied the presence of a mole
on each face. Due to a hypothetical response conflict following a spontaneous comparison of
the affective congruence between chords and faces, task performance should have been
context-dependent. Across mole-present trials, the average speed of correct affirmations
should have been higher when the chord and face were affect-congruent compared to when
they were affect-incongruent. Conversely, across mole-absent trials, the speed of correct
denials should have been higher when the chord and face were affect-incongruent compared
to when they were affect-congruent. Statistical analyses revealed variation in the degree to
which participants exhibited the expected effects. Differences in automatic and/or strategic
deployment of attentional resources may have factored into the results.
A Study of the Influences of Musical Chords on Responses to a Non-Affective Visual
Feature of Emotional Faces
by
James Daniel Baker
A thesis submitted to the Graduate Faculty of
North Carolina State University
in partial fulfillment of the
requirements for the degree of
Master of Science
Psychology
Raleigh, North Carolina
2014
APPROVED BY:
Douglas J. Gillan
Committee Chair
James W. Kalat Donald H. Mershon
iii
Biography
James Daniel Baker grew up in Weirton, West Virginia. After graduating from Brooke High
School in 2002, he attended West Virginia University. He graduated cum laude in 2007,
earning Bachelor of Science degrees in Computer Science and Psychology. His interest in
those fields led him to pursue a graduate degree in Human Factors, a Psychology program at
North Carolina State University. He began studying music perception and cognition upon
hearing his friend play a flatted fifth on the guitar one fateful day. Throughout his life, James
has often enjoyed adding humor to mundane activities, such as walking, chewing,
autobiography-ing, and punctuating
iv
Acknowledgments
My thanks go to the professors who helped me envision and complete this project.
Dr. Douglas J. Gillan
Dr. Thomas M. Hess
Dr. James W. Kalat
Dr. Shari A. Lane
Dr. Donald H. Mershon
I am particularly grateful to Shari for her patience and sympathy.
v
Table of Contents
List of Tables ____________________________________________________________ vi List of Figures ____________________________________________________________ vii Introduction ______________________________________________________________ 1
Experiment Overview ____________________________________________________ 7 Method _________________________________________________________________ 8
Participants ____________________________________________________________ 8 Design ________________________________________________________________ 8
Materials ______________________________________________________________ 9 Apparatus _____________________________________________________________ 10
Procedure _____________________________________________________________ 11 Results __________________________________________________________________ 13
Examinations of Stimulus Ratings and Musical Sophistication ____________________ 14 Preliminary Test for a Response Mapping Issue _______________________________ 16 Exploratory Test with Neutral Stimuli Included ________________________________ 17
Affective Matching Hypothesis Test with Neutral Stimuli Excluded _______________ 19 Additional Tests for Group and Individual Differences __________________________ 20
Discussion _______________________________________________________________ 25 Applications _____________________________________________________________ 28 References _______________________________________________________________ 29
Appendices ______________________________________________________________ 36
Appendix A: Angry Faces ________________________________________________ 37 Appendix B: Neutral Faces ________________________________________________ 38 Appendix C: Happy Faces ________________________________________________ 39
vi
List of Tables
Table 1. Mean Accuracies and Speeds within Sound and Face Contexts ______________ 18
Table 2. Hierarchical Regression Results Predicting Affective Matching Effect ________ 24
vii
List of Figures
Figure 1. Affective Priming: Evidence from an Evaluative Decision Task ____________ 2
Figure 2. Affective Priming: Evidence from a Non-Evaluative Affirmation/Denial Task _ 4
Figure 3. Mean Pleasantness Ratings for Sounds and Faces ________________________ 15
Figure 4. Evidence of an Extraneous Response Conflict __________________________ 17
Figure 5. Mean Speeds within Eighteen Exploratory Conditions ____________________ 19
Figure 6. Mean Speeds within Four Experimental Conditions ______________________ 20
Figure 7. Affective Priming Effects for Neg-AME Group Members _________________ 22
Figure 8. Affective Priming Effects for Pos-AME Group Members _________________ 22
Figure 9. Affective Priming Effects for Nil-AME Group Members __________________ 23
1
Introduction
In numerous situations, a person may process a musical stimulus in parallel with a
visual stimulus. The musical stimulus can trigger affective (e.g., emotional) processes in a
listener (Hunter, & Schellenberg, 2010; Juslin & Västfjäll, 2008; Scherer, 2004; Zentner,
Grandjean, & Scherer, 2008). Consequently, the musical stimulus may momentarily
influence the attentive processing of a visual object via the phenomenon known as affective
priming.
Researchers have typically studied affective priming via experiments employing an
evaluative decision task (see Fazio, 2001). In each trial, the participant must quickly respond
as to whether a target stimulus has a positive or negative affective valence. The onset of a
task-irrelevant valenced prime stimulus precedes that of the target stimulus by a very short
interval (e.g., 200 ms). A two-way (prime valence × target valence) interaction indicates
affective priming. Performance (speed and/or accuracy) is superior under conditions in
which the prime and target express the same valence than when they express opposite
valences (see Figure 1). Fazio, Sanbonmatsu, Powell, and Kardes (1986) initially reported
affective priming effects. Their prime and target stimuli were printed words that expressed
polarized affective valence (i.e., they were deemed to be either very positive or very
negative). Since then, many other researchers have found affective priming effects via
various renditions of the same basic paradigm (e.g., Bargh, Chaiken, Govender, & Pratto,
1992; De Houwer, Hermans, & Eelen, 1998; Greenwald, Klinger, & Liu, 1989).
2
Figure 1. Affective Priming: Evidence from an Evaluative Decision Task.
Because the evaluative decision task and the classical Stroop task (Stroop, 1935) are
paradigmatically similar, a popular explanation for Stroop effects (e.g., Logan & Zbrodoff,
1979; MacLeod, 1991) has come to serve as a leading explanation for affective priming
effects (Klauer, Roßnagel, & Musch, 1997; Musch & Klauer, 2001; Wentura, 1999). In the
evaluative decision task, the participant need only attend to the affective valence of the target
stimulus. However, an automatic evaluation of the prime stimulus can momentarily bias the
participant’s response to the target. When the separate valences of the prime and target are
congruent, the superfluous evaluation of the prime can bias the participant toward providing
the correct response. Conversely, when the separate valences are incongruent, the prime can
bias the participant toward providing an incorrect response, in which case he or she will
either respond incorrectly or spend additional time overcoming the bias. Thus, the valence of
3
the target in the evaluative decision task is analogous to the ink color of the typed color-word
in the Stroop task, while the valence of the prime is analogous to the verbal color word.
The Stroop mechanism hypothesis (Klauer & Musch, 2002, 2003) has gained
additional support from experiments demonstrating a lack of affective priming effects within
tasks requiring participants to categorize targets according to some non-affective trait (De
Houwer, Hermans, Rothermund, & Wentura, 2002; Klauer & Musch, 2002; Klinger, Burton,
& Pitts, 2000). However, within non-evaluative decision tasks explicitly requiring the
participants to provide an affirmation (e.g., “yes”) or a denial (e.g., “no”) response, typical
affective priming effects have emerged when the target demands an affirmation, while
reversed affective priming effects have emerged when the target demands a denial (Klauer &
Musch, 2002; Klauer & Stern, 1992; Wentura, 2000). An affective matching mechanism
hypothesis (Klauer & Musch, 2002, 2003) can account for those peculiar results. Though a
task may require a participant to attend only to a non-affective trait of the target, the
participant may automatically compare the affective valences of the prime and target. When
the prime and target are valence-congruent, a spontaneous feeling of plausibility biases the
participant toward providing an affirmation response. Conversely, when the prime and target
are valence-incongruent, a spontaneous feeling of implausibility biases the participant toward
providing a denial response. In either case, the response bias may conflict with the correct
response regarding the non-affective trait of the target. As a result, a normal affective
priming effect can occur when the necessary response is an affirmation, while a reverse
affective priming effect can occur when the necessary response is a denial (see Figure 2).
4
Figure 2. Affective Priming: Evidence from a Non-Evaluative Affirmation/Denial Task.
Whether by the Stroop mechanism or by the affective matching mechanism, the
affective content of a task-irrelevant prime stimulus can momentarily influence selective
attention to a target stimulus. Though the evidence has come primarily from experiments
5
employing only visual stimuli, several studies have shown that the prime need not be of the
same sensory modality as the target. Using printed words as the visual targets, researchers
have demonstrated cross-modal affective priming effects by using odorants as the primes
(Hermans, Baeyens, & Eelen, 1998), by using tastants as the primes (Veldhuizen, Oosterhoff,
& Kroeze, 2010), and by using various environmental sounds as the primes (Scherer &
Larsen, 2011). Numerous other examples of auditory-visual affective priming come from
studies in which musical stimuli served as primes. However, these priming effects have been
limited to pairings of particular types of musical and visual stimuli.
Across several studies, researchers have found typical affective priming effects by
using single chords as auditory primes and single words as visual targets. The researchers
polarized the affective valences of their chords via multiple techniques that are important in
music. For example, their experiments have produced affective priming effects not only
when the primes were either consonant or dissonant chords (Sollberger, Reber, & Eckstein,
2003; Steinbeis & Koelsch, 2011), but also when the primes were either major mode or
minor mode chords (Costa, 2012; Ragozzine, 2011; Steinbeis & Koelsch, 2011).
Specifically, participants’ responses to positive target words were superior when the prime
chords were consonant or major mode, and responses to negative target words were superior
when the prime chords were dissonant or minor mode.
In other studies, researchers have examined the relationships between emotional
facial expressions and emotional speech (de Gelder & Vroomen, 2000; Horstmann, 2010;
Horstmann & Ansorge, 2011; Pell, 2005). Though these researchers were not explicitly
6
examining cross-modal affective priming by musical stimuli, their paradigms and findings
are relevant to the present study. For example, Horstmann (2010) showed that the pitch of a
single task-irrelevant contextual tone can influence not only the evaluations of facial
expressions, but also the imitations of facial expressions. Participants evaluated and imitated
happy faces more quickly when high-pitch tones accompanied the faces compared to when
low-pitch tones accompanied them. Conversely, participants responded to angry faces more
quickly when low-pitch tones accompanied the faces compared to when high-pitch tones
accompanied them.
The results of Horstmann’s (2010) experiments correspond to the results of Costa’s
(2012) experiments, in which the pitch range (octave) of single chords influenced responses
both to typed words and to pictures. Participants’ evaluated positive target words and
pictures more quickly when high-octave chords preceded the targets compared to when low-
octave chords preceded them. Conversely, participants’ evaluated negative target words and
pictures more quickly when low-register chords preceded the targets compared to when high-
register chords preceded them.
In review, a number of studies have provided evidence that, via an underlying
affective priming mechanism, a musical stimulus can exert a temporary influence on the
attentional processing of a visual stimulus. There is consistent evidence that single musical
chords can facilitate or impede responses to typed words (Costa, 2012; Ragozzine, 2011;
Sollberger et al., 2003; Steinbeis & Koelsch, 2011). There is additional evidence that tonal
properties shared by music and emotional speech can facilitate or impede responses to
7
emotionally expressive faces (de Gelder & Vroomen, 2000; Horstmann, 2010; Horstmann &
Ansorge, 2011; Pell, 2005). Taken together, these two lines of evidence imply that single
musical chords may facilitate or impede responses to emotionally expressive faces.
Experiment Overview
For the present affective priming experiment, the primes were consonant and
dissonant chords similar to those that have demonstrated affective priming capacity in past
studies (Sollberger et al., 2003; Steinbeis & Koelsch, 2011). The targets were schematic
faces expressing happiness and anger. Schematic faces have demonstrated intra-modal
affective priming capacity (Lipp, Price, & Tellegen, 2009), and compared to photographed
faces, they more readily met the demands of the participants’ task.
Rather than evaluating the affective valence of a target face, the participants quickly
decided whether or not each schematic face contained a mole (a small black spot). Because
each decision took the form of an affirmation or a denial (“yes” or “no”), I hypothesized that
the affective matching mechanism (Klauer & Musch, 2002, 2003; Klauer & Stern, 1992;
Wentura, 2000) would bias participants’ responses. I predicted that overall performance
across mole-present trials would be better (i.e., lower mean response latency and higher mean
response accuracy) when the chord and face were affect-congruent compared to when they
were affect-incongruent. Conversely, I predicted that overall performance across mole-
absent trials would be better when the chord and face were affect-incongruent compared to
when they were affect-congruent.
8
Method
Participants
Twenty-eight undergraduates (14 women and 14 men) participated, partially
satisfying a requirement within their introductory psychology course at North Carolina State
University. Their ages ranged from 16 to 34 years old, and the sample’s median age was 19.
All participants demonstrated a visual acuity of 20/30 or better via the University at Buffalo
Interactive Visual Acuity Chart (IVAC), an online Snellen-type test. Via the Hearing
Screening Inventory (Coren & Hakstian, 1992), all but three participants reported a hearing
ability that was “average” or better in both ears. No participants reported a hearing ability
that was “poor” or worse in either ear.
Design
An affective matching paradigm requires a 2 × 2 × 2 (prime valence × target valence
× response type) within-subjects design. This experiment consisted of an affective matching
paradigm embedded within a 3 × 3 × 2 design. Specifically, it included three auditory prime
conditions (dissonant chord, consonant chord, and pure tone), three facial expression
conditions (angry, happy, and neutral), and two mole presence conditions (mole-absent and
mole-present). It also included a six-level manipulation of the mole location (upper-right,
middle-right, lower-right, upper-left, middle-left, and lower-left). Participants completed six
blocks of the above design. The dependent variables were response latency and response
accuracy.
9
Materials
Primes. I used MuseScore 1.3 software (Schweer et al., 2013) to synthesize the two
chords. Each chord consisted of four superpositioned grand piano notes with fundamental
frequencies between 261.63 Hz (C4) and 523.25 Hz (C5). The consonant chord consisted of
the notes C4–E4–G4–C5, and the dissonant chord consisted of the notes C4– F♯4–B4–C5.
These chords previously demonstrated the capacity for affective priming in a study by
Steinbeis and Koelsch (2011). I used Audacity 2.0 software (The Audacity Team, 2000) to
synthesize the pure tone, which was a 261.63 Hz (C4) sine wave. All three primes were
1,400 ms in duration, with S-curve attacks of approximately 120 ms and S-curve decays of
approximately 1,000 ms. I used Audacity to adjust their amplitudes, such that when I played
them at a particular output setting, they seemed comfortable and equally loud through over-
the-ear stereo headphones. By holding a sound level meter inside the ear padding ring of
each headphone, I ensured that each sound had a peak level within a safe range of 65 to 70
dBSPL.
Targets. The visual targets were a set of three schematic faces that Öhman,
Lundqvist, and Esteves (2001) used as part of a study on threat-detection. I modified the set
to incorporate both the two-level mole presence factor and the six-level mole location factor.
Appendix A, Appendix B, and Appendix C show the angry, neutral, and happy faces,
respectively. When viewed from a distance of 24 inches, each face subtended a vertical
visual angle of approximately 7°, and each mole subtended a visual angle of approximately
0.3°.
10
Questionnaire. The questionnaire consisted of four sections. The first questionnaire
section requested a rating for each prime and target stimulus. The format was a seven-point
pleasantness scale ranging from extremely unpleasant to extremely pleasant. The second
questionnaire section was the self-report component of the Goldsmiths Musical
Sophistication Index, or Gold-MSI (Müllensiefen, Gingras, Musil, & Stewart, 2014;
Müllensiefen, Gingras, Stewart, & Musil, 2014). This 38-item inventory allows for the
calculation of a General Musical Sophistication score based upon five factors: Active
Musical Engagement, Perceptual Abilities, Musical Training, Singing Abilities, and
Sophisticated Emotional Engagement. The third questionnaire section was the Hearing
Screening Inventory, or HSI (Coren & Hakstian, 1992), which allows for estimation of
hearing loss via 12 self-report items. Upon development, the HSI demonstrated high internal
consistency (Cronbach’s = .89) and high test-retest reliability (Cronbach’s = .88), and it
showed a high correlation (r = .81) with conventional pure-tone audiometric testing. In the
present study, it served as a cost- and time-efficient alternative. The fourth and final
component of the questionnaire requested typical demographics data, including age, gender,
ethnicity, and linguistic background.
Apparatus
Participants completed the tasks via two modern Dell PCs running the Microsoft
Windows 7 operating system. The PCs were connected to 24-inch LCD monitors, which
displayed images at a resolution of 1920 × 1080p, and which produced no visually detectable
glare. One PC administered the choice reaction time (CRT) test via Affect 4.0 software
11
(Hermans, Clarysse, Baeyens, & Spruyt, 2005; Spruyt et al., 2010). The other PC
administered the questionnaire via Qualtrics online browser application. The CRT PC also
presented sounds via over-the-ear headphones, and the questionnaire PC presented sounds
via speakers located on either side of the monitor. Participants completed the CRT test via
standard keyboard, and they completed the questionnaire via both standard keyboard and
mouse. On the CRT PC’s keyboard, YES, OK, and NO labels covered the left arrow, down
arrow, and right arrow keys, respectively. The Affect software disabled all other keys on that
keyboard, with the exception of the escape key. I used IBM SPSS 22.0 software to conduct
all statistical analyses.
Procedure
Under the guidance of one researcher, each participant completed a solo session
within a quiet laboratory setting that had typical office-style fluorescent lighting. Each
session lasted approximately 40 minutes. Upon arrival, the participant read a consent form,
which described the focus of the experiment as “the human ability to make quick decisions
about different types of visual stimuli, specifically, depictions of faces.” Upon providing
consent, the participant then passed the screening for visual acuity. The participant sat in a
straight-back chair and correctly read aloud a row of letters that appeared on a computer
monitor, situated approximately 2.4 meters from the participant’s face. Given that particular
distance, the IVAC site automatically sized and spaced the letters to test for 20/30 acuity.
After passing the visual acuity screening, the participant moved to the CRT workstation. The
researcher instructed the participant to carefully read and follow all on-screen instructions.
12
The participant donned the headphones, the researcher moved to a nearby, out-of-view
location, and the Affect 4.0 software then guided the participant through the CRT test.
The participant completed one set of 18 practice trials, across which each possible
combination of the auditory prime, facial expression, and mole presence conditions appeared
exactly once and in a random order. Across the nine mole-present practice trials, the mole
randomly appeared in at least three of the six possible locations. The participant then
completed six blocks of 108 trials. Each trial block included 54 mole-present trials, across
which each possible combination of the auditory prime, facial expression, and mole location
conditions appeared exactly once. Across the 54 mole-absent trials, each possible
combination of the auditory prime and facial expression conditions appeared exactly six
times. The trial order was random within each block.
Each trial began with the display of a black fixation cross, which remained visible at
the center of the monitor for 600 ms. Immediately after the cross disappeared, the prime
sounded via the headphones for 1,400 ms. After 150 ms from the onset of the prime, the face
appeared at the center of the screen, and it remained visible for 1,250 ms or until the
participant provided a response. The software recorded the participant’s first response only.
Regardless of the participant’s response, the prime completed its full presentation, at which
point the trial ended. Thus, within each block, the trial inter-onset interval was always 2,000
ms.
Prior to initiating the set of practice trials, the software instructed the participant to
quickly enter each response “within the 1.25 seconds during which the face is displayed.”
13
Prior to initiating each trial block, the software reminded the participant to: “Look directly at
the black cross while it is present. When the cross disappears, immediately attend to the face
that appears. Press the YES key only if the face has a mole. Otherwise, press the NO key.”
The participant initiated each trial block by pressing the OK key. Thus, the procedure
allowed multiple rest periods.
After completing the CRT test, the participant moved to the questionnaire workstation
to complete the four sections of the questionnaire. The participant first rated the three
auditory stimuli in one of six possible orders. A counter-balancing and random assignment
procedure predetermined the participant’s stimulus order. Before rating each auditory
stimulus, the participant had to press a button to listen to the sound at least once (i.e., he or
she could listen to the sound multiple times, if necessary). After rating each sound, the
participant rated all 21 faces in a random order. The participant then completed the MSI,
HSI, and general demographics sections of the questionnaire, in that order. The final screen
of the questionnaire thanked the participant, and it included a debriefing that revealed the
purpose of the task-irrelevant sounds and facial expressions. The researcher addressed the
participant’s remaining questions and concerns.
Results
Prior to conducting any hypothesis tests, I filtered out all data from any trial in which
the response input was likely anticipatory (less than 200 ms latency) and from any trial in
which the time limit had expired (greater than 1,250 ms latency). This resulted in a loss of
55 (0.3%) of the 18,144 observations. Across the remaining trials, the grand mean of
14
participants’ mean accuracy scores (proportions correct) was .97 (SD = .03). In light of such
high accuracy scores, I decided to limit reports of significant accuracy statistics to tables and
figures. Thus, I focused on response latency data across only the 17,582 trials in which
responses were correct. Across those trials, the grand mean of participants’ mean latency
scores was 519 ms (SD = 66). After transforming each trial’s latency value to its log, square-
root, and inverse (speed), I selected the speed data for further inferential analyses, because
that distribution exhibited both the smallest skewness (S = 0.21, SE = 0.02) and the smallest
kurtosis (K = 0.77, SE = 0.04). Across the participants’ mean speeds, the grand mean was
2.03 responses per second (SD = 0.26).
For each analysis of variance (ANOVA), I first referred to Mauchly’s test of
sphericity. If the test revealed a sphericity violation for a particular main effect or interaction
term, I referred to the conservative Greenhouse-Geisser-corrected univariate F-test, in order
to determine significance. When conducting family-wise comparisons of means, I used
SPSS’s Bonferroni-corrected criterion to determine the significance of each mean difference.
Examinations of Stimulus Ratings and Musical Sophistication
Figure 3 shows the mean pleasantness ratings of the sounds and faces (averaged
across mole conditions). Among ratings of the sounds, pairwise comparisons showed that
the consonant chord was significantly more pleasant than the pure tone and the dissonant
chord. However, the dissonant chord was only marginally less pleasant than the pure tone.
Among ratings of the faces, pairwise comparisons showed that the happy face was
significantly more pleasant than the neutral face and the angry face, and the angry face was
15
significantly less pleasant than the neutral face.
Figure 3. Mean Pleasantness Ratings for Sounds and Faces (N = 28). A rating of 7 indicates
an “extremely pleasant” stimulus, and a rating of 1 indicates an “extremely unpleasant”
stimulus. Error bars represent standard errors. Each bold horizontal line connecting two bars
represents a significant difference between those means.
Within the General Musical Sophistication Index (Müllensiefen, Gingras, Stewart, &
Musil, 2014), the minimum achievable score is 18 (low sophistication) and the maximum is
126 (high sophistication). Müllensiefen et al. reported a mean score of 82 (SD = 21) within a
sample of over 140k participants. In the present study, the mean score was 74 (SD = 17), as
16
was the median. A MANOVA revealed that the upper 50% of participants (n = 14) did not
differ from the lower 50% with respect to their mean pleasantness ratings of the sounds and
faces (Wilks’s Λ = .809, F(6, 21) = 0.83, p = .562, = .191).
Preliminary Test for a Response Mapping Issue
During the data-collection phase, a few participants admitted that they sometimes
accidentally responded to the left-vs-right direction of a target mole rather than responding to
its mere presence. Indeed, pairwise comparisons revealed that the mean accuracies and
speeds within mole-left trials were superior to those of their mole-right counterparts (see
Figure 4). The problem was likely due to the fact that the keyboard’s left arrow key served
as the YES response, while its right arrow key served as the NO response. When a mole
appeared to the left, a direction-based response was consistent with the correct presence-
based response, but when a mole appeared to the right, a direction-based response conflicted
with the correct presence-based response. In light of this extraneous response mapping issue,
I include a dichotomized version of the mole location in later analyses.
17
Figure 4. Evidence of an Extraneous Response Conflict (N = 28). Performance scores are
means from within the mole-present condition only. Error bars represent standard errors. In
the Difference Diagrams, each line connecting two moles represents a significant difference
between the means of those trials.
Exploratory Test with Neutral Stimuli Included
The affective matching hypothesis includes no predictions regarding effects of
time/practice, valence-neutral stimuli, or the task-irrelevant locations of moles. However,
prior to testing the hypothesis, I conducted an exploratory five-way ANOVA that included
the extraneous elements of the experiment design. The within-subjects factors were trial
block (1 – 6), prime sound (dissonant chord, pure tone, or consonant chord), facial expression
(angry, neutral, or happy), mole presence (absent or present), and mole direction (right or
18
left). The main effect of prime sound was significant (F(2, 54) = 4.87, p = .011, = .153).
Pairwise comparisons showed that, for reasons unknown, participants responded with higher
speed and with lower accuracy when the prime sound was a pure tone compared to when it
was a consonant chord (see Table 1). Due to the response mapping issue, the extraneous
two-way interaction between mole presence and mole direction was significant (F(1, 27) =
142.72, p < .001, = .841), qualifying the significant main effect of mole direction (F(1,
27) = 111.62, p < .001, = .805) and that of mole presence (F(1, 27) = 8.31, p = .008,
=
.235). Along with all other main effects and interactions, the critical affective matching
interaction (prime sound × facial expression × mole presence) was not significant (F(4, 108)
= 0.54, p = .705, = .020). Thus, for the sample as a whole, neither the ability to affirm nor
the ability to deny mole presence depended upon the task-irrelevant sound and facial
expression combination (see Figure 5).
Table 1. Mean Accuracies and Speeds within Sound and Face Contexts
Accuracy (proportion correct) Speed (1 / latency in sec.)
Stimulus M SE 95% CI M SE 95% CI
Dissonant chord .973 .006 [.961, .985] 2.016 .047 [1.919, 2.113]
Pure tone .968 .007 [.955, .981] 2.032 .049 [1.931, 2.132]
Consonant chord .975 .005 [.964, .986] 2.012 .048 [1.914, 2.111]
Angry face .973 .006 [.961, .985] 2.021 .048 [1.922, 2.120]
Neutral face .971 .006 [.959, .983] 2.026 .048 [1.927, 2.124]
Happy face .972 .006 [.959, .985] 2.013 .048 [1.915, 2.111]
Note. N = 28. Compared to consonant chords, pure tones led to responses of significantly higher speed (p =
.028) and significantly lower accuracy (p = .040).
19
Figure 5. Mean Speeds within Eighteen Exploratory Conditions (N = 28). Error bars
represent standard errors. These results did not support the affective matching hypothesis.
Affective Matching Hypothesis Test with Neutral Stimuli Excluded
The affective matching hypothesis predicts a two-way interaction between a two-level
affective congruence factor (incongruent or congruent) and a two-level response type factor
(denial or affirmation). I tested the hypothesis on a reduced dataset, which excluded any trial
in which the prime was a pure tone or the target was a neutral face. I categorized each trial
as either affect-incongruent (e.g., a dissonant chord paired with a happy face) or affect-
congruent (e.g., a dissonant chord paired with an angry face), and I then conducted a four-
way ANOVA. The within-subjects factors were trial block (1 – 6), affective congruence
(incongruent or congruent), mole presence (absent or present), and mole direction (right or
20
left). Due again to the response mapping issue, the extraneous two-way interaction between
mole presence and mole direction was significant (F(1, 27) = 88.16, p < .001, = .766),
qualifying the significant main effect of mole direction (F(1, 27) = 79.52, p < .001, =
.747) and that of mole presence (F(1, 27) = 13.26, p = .001, = .329). Along with all other
main effects and interactions, the critical interaction between affective congruence and mole
presence was not significant (F(1, 27) = 0.10, p = .921, < .001). Observed power was
.051. Thus, for the sample as a whole, neither the ability to affirm nor the ability to deny
depended upon the affective congruence of the task-irrelevant sound and facial expression
(see Figure 6).
Figure 6. Mean Speeds within Four Experimental Conditions (N = 28). Error bars represent
standard errors. Each bold horizontal line connecting two bars represents a significant
difference between the means of those trials. These results did not support the affective
matching hypothesis.
Additional Tests for Group and Individual Differences
Having found no two-way interaction between affective congruence and mole
presence, I examined the priming effects of individual participants. For both the mole-
21
present trials and the mole-absent trials, I computed the magnitude of the simple affective
priming effect as the mean speed across affect-congruent trials minus the mean across affect-
incongruent trials. I then determined the affective matching effect (AME) by subtracting the
mole-absent affective priming effect (APEA) from the mole-present affective priming effect
(APEP). The affective matching hypothesis predicts positive AME scores. Most of the
participants (n = 18) exhibited mathematically positive AME scores, but some (n = 10)
exhibited mathematically negative scores.
I then used a k-means cluster analysis to divide the entire sample into three
subsamples according to AME scores. The clustering algorithm split the sample such that
the AME scores within each cluster were maximally different from those within the other
two clusters. One cluster (see Figure 7) consisted of the six participants with the highest
magnitude negative AME scores (M = -0.15, SD = 0.05). Another cluster (see Figure 8)
consisted of the eleven participants with the highest magnitude positive AME scores (M =
0.08, SD = 0.03). The third cluster (see Figure 9) consisted of the eleven participants with
AME scores closer to zero (M = 0.00, SD = 0.03). I referred to these three groups as the
Neg-AME, Pos-AME, and Nil-AME groups, respectively.
22
Figure 7. Affective Priming Effects for Neg-AME Group Members (n = 6). Each priming
effect is the mean speed across affectively congruent trials minus the mean speed across
affectively incongruent trials. The group exhibited the reverse-of-expected pattern of results.
Figure 8. Affective Priming Effects for Pos-AME Group Members (n = 11). Each priming
effect is the mean speed across affectively congruent trials minus the mean speed across
affectively incongruent trials. The group exhibited the expected pattern of results.
23
Figure 9. Affective Priming Effects for Nil-AME Group Members (n = 11). Each priming
effect is the mean speed across affectively congruent trials minus the mean speed across
affectively incongruent trials. The group exhibited neither the expected nor the reverse-of-
expected pattern of results.
I initially considered the possibility that differences in musical sophistication allowed
for differences in attentional and/or affective processing of auditory stimuli, leading to the
different patterns of affective priming among the three groups. I tested this via MANOVA,
using all five factors of the Gold-MSI as dependent variables. The AME groups did not
differ within any of the five factors (Wilks’s Λ = .797, F(10, 42) = 0.51, p = .877, = .107).
I used a separate MANOVA to test whether the groups differed in their pleasantness ratings
of the three prime sounds and the three target faces (averaged across mole conditions). The
groups did not differ with respect to their ratings of the six stimuli (Wilks’s Λ = .449, F(12,
40) = 1.64, p = .119, = .330).
24
As a final test, I used a hierarchical regression to examine the extent to which
additional extraneous variables accounted for variation in the AME scores. The first block
consisted of only the session day, coded as the day of the month of January during which
individuals participated. The second block consisted of a few participant variables: General
Musical Sophistication Index (MSI-G), the Hearing Screening Inventory (HSI) score, age,
and gender. Table 2 shows the results of the regression. By itself, session day was a
significant predictor of AME scores, due to negative scores becoming more common as the
session day increased. However, both the significance of session day and the significance of
the model as a whole were lost upon inclusion of the participant variables, none of which
were significant predictors. Nevertheless, session day was the most significant predictor,
possibly indicating a cohort effect.
Table 2. Hierarchical Regression Results Predicting Affective Matching Effect
Model 1 Model 2
Variable B SE β t p B SE β t p
Day -.007 .003 -.427 -2.41 .024 -.006 .003 -.349 -1.83 .082
MSI-G -.001 .001 -.206 -1.06 .301
HSI .005 .005 .212 1.03 .315
Age .005 .005 .195 1.00 .327
Gender .068 .045 .365 1.52 .144
R2 .18 .29
F 5.79 .024 1.76 .162
Note. N = 28. A higher MSI-G score indicates greater musical sophistication. A higher HSI score indicates
poorer hearing. Gender was coded as 0 for women and 1 for men.
25
Discussion
The affective matching paradigm differs from the typical affective priming paradigm,
in that the latter requires participants to attend to the affective dimension of the target
stimulus while the former does not. According to the affective matching hypothesis (Klauer
& Musch, 2002, 2003), participants within the present study should have spontaneously
measured the task-irrelevant affective congruence between valenced sounds and valenced
facial expressions. Affect-congruent pairs should have biased participants toward providing
affirmation responses, while affect-incongruent pairs should have biased them toward
providing denial responses. Consequently, across mole-present trials, participants should
have correctly responded “YES” more quickly within affect-congruent contexts than within
affect-incongruent contexts, and across mole-absent trials, they should have correctly
responded “NO” more quickly within affect-incongruent contexts than within affect-
congruent contexts. The results of critical statistical analyses did not support these
predictions. Furthermore, variation in the potentially relevant measures of musical
sophistication and perceived stimulus pleasantness did not account for the pattern of variation
in the affective matching effect.
In any given trial of an affective priming paradigm, the magnitude of a response
conflict may be related to the degree to which the participant automatically and/or selectively
attends to the task-irrelevant affective features of the stimuli (Spruyt, De Houwer, Everaert,
& Hermans, 2012; Spruyt, De Houwer, & Hermans, 2009; Spruyt, De Houwer, Hermans, &
Eelen, 2007). In a recent study, Gast, Werner, Heitmann, Spruyt, and Rothermund (2013)
26
examined the relationship between selective attention and affective priming effects within a
response priming paradigm that resembled the affective matching paradigm of the present
study. In two experiments, the primes and targets were affectively valenced pictures. Either
the letter “X” or the letter “Y” was present on each target picture at one of four locations.
For all participants, the primary task was letter-discrimination. In their first experiment, the
researchers instructed the experimental group of participants to attend to the valences of the
prime pictures. For those participants, an evaluative decision task followed the letter
discrimination task in some of the trials (about 17% of the total). The researchers gave no
such instructions to a control group of participants, who never encountered the evaluative
decision task. Analyses of letter discrimination latencies revealed a significant affective
priming effect within the experimental group but not within the control group. In the second
experiment, the researchers instructed the experimental group of participants to attend to the
valences of both the primes and the targets, and they occasionally tasked those participants
with categorizing the pictures as having either the same or opposite valences. Other aspects
of the design, such as the primary letter-discrimination task, were the same as in the first
experiment. Again, analyses of letter discrimination latencies revealed a significant affective
priming effect within the experimental group, but not within the control group. These results
were in line with those of other studies (e.g., Spruyt, De Houwer, Hermans, & Eelen, 2007)
that have provided evidence that affective priming effects can be greater when participants
devote greater amounts of attention to the affective properties of primes and targets, whether
or not those affective properties are relevant to the task used to measure the priming effects.
27
In the present study, it is possible that participants varied in the degree to which they
automatically and/or selectively attended to the task-irrelevant affective congruence between
prime sounds and target facial expressions. The variation in attentional deployment could
account for the peculiar patterns of affective priming effects.
In conclusion, the present study did not support the general notion that musical
stimuli can influence the attentive processing of visual stimuli via the specific mechanism of
affective priming. Any follow-up to the present study should address at least three of its
limitations. Firstly, it will be important to use a task that does not lead to extraneous
response conflicts, such as that which caused participants to erroneously respond to the left-
vs-right direction of the target mole. A possible solution would be to use special software to
recognize and record voiced “YES” and “NO” responses. That method could in fact be
optimal for inducing affective priming via the affective matching mechanism (Klauer &
Musch, 2002, 2003), given the mechanism’s dependence upon responses that are either
affirmations or denials. Secondly, in light of the results of the study by Gast, Werner,
Heitmann, Spruyt, and Rothermund (2013), it will be important to have greater control over
participants’ attentional deployment strategies. This may be achieved by simply
incorporating a version of Gast et al.’s dual task paradigm into the affective matching
paradigm. Thirdly, the design should include various types of pictorial targets (e.g.,
weapons, food, spiders, etc.) in addition to schematic faces. Such a manipulation could
reveal whether musical stimuli form better affective matches with some types of stimuli (e.g.,
social) than with other types (e.g., edible).
28
Applications
The present study served to provide a better understanding of one mechanism
(affective priming) through which a rudimentary musical stimulus can momentarily help or
hinder the concurrent processing of a visual stimulus. Evidence of cross-modal affective
priming can have implications for the design of multi-sensory displays. For example, many
scientist-practitioners are interested in improving the design of musical stimuli known as
earcons (McGookin & Brewster, 2011), which can guide users’ interactions with various
facets of a device’s visual interface. If particular musical features (e.g., consonance and
dissonance) can express/induce affect, then affective relationships may exist between
particular earcons and their referents. Some of these affective relationships might be useful,
while others may be unintentional and undesirable.
29
References
Bargh, J. A, Chaiken, S., Govender, R., & Pratto, F. (1992). The generality of the automatic
attitude activation effect. Journal of Personality and Social Psychology, 62, 893-912.
doi:10.1037/0022-3514.62.6.893
Coren, S., & Hakstian, A. (1992). The development and cross-validation of a self-report
inventory to assess pure-tone threshold hearing sensitivity. Journal of Speech &
Hearing Research, 35, 921-929.
Costa, M. (2013). Effects of mode, consonance, and register in visual and word-evaluation
affective priming experiments. Psychology of Music, 41, 713-728.
doi:10.1177/0305735612446536
de Gelder, B., & Vroomen, J. (2000). The perception of emotions by ear and by eye.
Cognition & Emotion, 14, 289-311. doi:10.1080/026999300378824
De Houwer, J., Hermans, D., & Eelen, P. (1998). Affective and identity priming with
episodically associated stimuli. Cognition & Emotion, 12, 145-169.
doi:10.1080/026999398379691
De Houwer, J., Hermans, D., Rothermund, K., & Wentura, D. (2002). Affective priming of
semantic categorisation responses. Cognition & Emotion, 16, 643-666.
doi:10.1080/02699930143000419
Fazio, R. H. (2001). On the automatic activation of associated evaluations: An overview.
Cognition & Emotion, 15, 115-141. doi:10.1080/02699930125908
30
Fazio, R. H., Sanbonmatsu, D. M., Powell, M. C., & Kardes, F. R. (1986). On the automatic
activation of attitudes. Journal of Personality and Social Psychology, 50, 229-238.
doi:10.1037/0022-3514.50.2.229
Gast, A., Werner, B., Heitmann, C., Spruyt, A., & Rothermund, K. (2013). Evaluative
stimulus (in)congruency impacts performance in an unrelated task: Evidence for a
resource-based account of evaluative priming. Experimental Psychology. Advance
online publication. doi: 10.1027/1618-3169/a000238
Greenwald, A. G., Klinger, M. R., & Liu, T. J. (1989). Unconscious processing of
dichoptically masked words. Memory & Cognition, 17, 35-47.
doi:10.3758/BF03199555
Hermans, D., Baeyens, F., & Eelen, P. (1998). Odours as affective-processing context for
word evaluation: A case of cross-modal affective priming. Cognition & Emotion, 12,
601-613. doi:10.1080/026999398379583
Hermans, D., Clarysse, J., Baeyens, F., & Spruyt, A. (2005). Affect (Version 4.0) [Computer
software]. University of Leuven, Belgium. Available from
http://fac.ppw.kuleuven.be/clep/affect4/
Horstmann, G. (2010). Tone-affect compatibility with affective stimuli and affective
responses. Quarterly Journal of Experimental Psychology, 63, 2239-2250.
doi:10.1080/17470211003687538
Horstmann, G., & Ansorge, U. (2011). Compatibility between tones, head movements, and
facial expressions. Emotion, 11, 975-980. doi:10.1037/a0023468
31
Hunter, P. G., & Schellenberg, E. G. (2010). Music and emotion. In M. R. Jones, R. R. Fay,
& A. N. Popper (Eds.), Music Perception (pp. 129-164). New York: Springer.
IVAC [Computer software]. University at Buffalo, The State University of New York.
Available from http://www.smbs.buffalo.edu/oph/ped/IVAC/IVAC.html
Juslin, P. N., & Västfjäll, D. (2008). Emotional responses to music: The need to consider
underlying mechanisms. Behavioral and Brain Sciences, 31, 559-575.
doi:10.1017/S0140525X08005293
Klauer, K. C., & Musch, J. (2002). Goal-dependent and goal-independent effects of
irrelevant evaluations. Personality and Social Psychology Bulletin, 28, 802-814.
doi:10.1177/0146167202289009
Klauer, K. C., & Musch, J. (2003). Affective priming: Findings and theories. In J. Musch &
K. C. Klauer (Eds.), The Psychology of Evaluation: Affective Processes in Cognition
and Emotion (pp. 7-49). Mahwah, NJ, US: Lawrence Erlbaum Associates Publishers.
Klauer, K. C., Roßnagel, C., & Musch, J. (1997). List-context effects in evaluative priming.
Journal of Experimental Psychology: Learning, Memory, and Cognition, 23, 246-
255. doi:10.1037/0278-7393.23.1.246
Klauer, K. C., & Stern, E. (1992). How attitudes guide memory-based judgments: A two-
process model. Journal of Experimental Social Psychology, 28, 186-206.
doi:10.1016/0022-1031(92)90038-L
32
Klinger, M. R., Burton, P. C., & Pitts, G. S. (2000). Mechanisms of unconscious priming: I.
Response competition, not spreading activation. Journal of Experimental Psychology:
Learning, Memory, and Cognition, 26, 441-455. doi:10.1037/0278-7393.26.2.441
Lipp, O. V., Price, S. M., & Tellegen, C. L. (2009). No effect of inversion on attentional and
affective processing of facial expressions. Emotion, 9, 248-259.
doi:10.1037/a0014715
Logan, G. D., & Zbrodoff, N. J. (1979). When it helps to be misled: Facilitative effects of
increasing the frequency of conflicting stimuli in a Stroop-like task. Memory &
Cognition, 7, 166-174. doi:10.3758/BF03197535
MacLeod, C. M. (1991). Half a century of research on the Stroop effect: An integrative
review. Psychological Bulletin, 109, 163-203. doi:10.1037/0033-2909.109.2.163
McGookin, D., & Brewster, S. (2011). Earcons. In Hermann, T., Hunt, A., & Neuhoff, J. G.
(Eds.), The Sonification Handbook (pp. 339-361). Berlin, Germany: Logos.
Müllensiefen, D., Gingras, B., Musil, J., & Stewart L. (2014). The musicality of non-
musicians: An index for assessing musical sophistication in the general population.
PLoS ONE, 9, e89642. doi:10.1371/journal.pone.0089642
Müllensiefen, D., Gingras, B., Stewart, L. & Musil, J. (2014). The Goldsmiths Musical
Sophistication Index (Gold-MSI): Technical Report and Documentation v1.0.
London: Goldsmiths, University of London.
33
Musch, J., & Klauer, K. C. (2001). Locational uncertainty moderates affective congruency
effects in the evaluative decision task. Cognition & Emotion, 15, 167-188.
doi:10.1080/02699930126132
Öhman, A., Lundqvist, D., & Esteves, F. (2001). The face in the crowd revisited: A threat
advantage with schematic stimuli. Journal of Personality and Social Psychology, 80,
381-396. doi:10.1037/0022-3514.80.3.381
Pell, M. D. (2005). Nonverbal emotion priming: Evidence from the “Facial Affect Decision
Task.” Journal of Nonverbal Behavior, 29, 45-73. doi:10.1007/s10919-004-0889-8
Ragozzine, F. (2011). Cross-modal affective priming with musical stimuli: Effect of major
and minor triads on word-valence categorization. Journal of ITC Sangeet Research
Academy, 25, 8-24.
Scherer, K. R. (2004). Which emotions can be induced by music? What are the underlying
mechanisms? And how can we measure them? Journal of New Music Research, 33,
239-251. doi:10.1080/0929821042000317822
Scherer, L. D., & Larsen, R. J. (2011). Cross-modal evaluative priming: Emotional sounds
influence the processing of emotion words. Emotion, 11, 203-208.
doi:10.1037/a0022588
Schweer, W., et al. (2013). MuseScore (Version 1.3) [Computer software]. Available from
http://musescore.org
34
Sollberger, B., Reber, R., & Eckstein, D. (2003). Musical chords as affective priming context
in a word-evaluation task. Music Perception, 20, 263-282.
doi:10.1525/mp.2003.20.3.263
Spruyt, A., Clarysse, J., Vansteenwegen, D., Baeyens, F., & Hermans, D. (2010). Affect 4.0:
A free software package for implementing psychological and psychophysiological
experiments. Experimental Psychology, 57, 36-45. doi:10.1027/1618-3169/a000005
Spruyt, A., De Houwer, J., Everaert, T., & Hermans, D. (2012). Unconscious semantic
activation depends on feature-specific attention allocation. Cognition, 122, 91-95.
doi:10.1016/j.cognition.2011.08.017
Spruyt, A., De Houwer, J., & Hermans, D. (2009). Modulation of automatic semantic
priming by feature-specific attention allocation. Journal of Memory and Language,
61, 37-54. doi: 10.1016/j.jml.2009.03.004
Spruyt, A., De Houwer, J., Hermans, D., & Eelen, P. (2007). Affective priming of
nonaffective semantic categorization responses. Experimental Psychology, 54, 44-53.
doi:10.1027/1618-3169.54.1.44
Steinbeis, N., & Koelsch, S. (2011). Affective priming effects of musical sounds on the
processing of word meaning. Journal of Cognitive Neuroscience, 23, 604-21.
doi:10.1162/jocn.2009.21383
Stroop, J. R. (1935). Studies of interference in serial verbal reactions. Journal of
Experimental Psychology, 18, 643-662. doi:10.1037/h0054651
35
The Audacity Team (2000). Audacity (Version 2.0) [Computer software]. Available from
http://audacity.sourceforge.net
Veldhuizen, M. G., Oosterhoff, A. F., & Kroeze, J. H. A. (2010). Flavors prime processing of
affectively congruent food words and non-food words. Appetite, 54, 71-76.
doi:10.1016/j.appet.2009.09.008
Wentura, D. (1999). Activation and inhibition of affective information: Evidence for negative
priming in the evaluation task. Cognition & Emotion, 13, 65-91.
doi:10.1080/026999399379375
Wentura, D. (2000). Dissociative affective and associative priming effects in the lexical
decision task: Yes versus no responses to word targets reveal evaluative judgment
tendencies. Journal of Experimental Psychology: Learning, Memory, and Cognition,
26, 456-469. doi:10.1037//0278-7393.26.2.456
Zentner, M., Grandjean, D., & Scherer, K. R. (2008). Emotions evoked by the sound of
music: Characterization, classification, and measurement. Emotion, 8, 495-521.
doi:10.1037/1528-3542.8.4.494