Color, Music, and Emotion - Palmer Labdynamics (soft vs. loud) orchestration (multiple instruments...
Transcript of Color, Music, and Emotion - Palmer Labdynamics (soft vs. loud) orchestration (multiple instruments...
General Methods
Relations among Color, Music, and Emotion
Background
Color, Music, and Emotion Stephen E. Palmer1, Thomas Langlois1, Tawny Tsang1, Karen B. Schloss1, & Daniel J. Levitin2
1 University of California, Berkeley, Department of Psychology 2 McGill University, Department of Psychology
Emotions Mediate Color-Music Associations
Conclusions
References and Acknowledgements
Color and Music: Stimuli
SATURATED
MUTED
LIGHT
DARK
8 Hues: Red, Yellow, Green, Blue, Orange, Chartreuse, Cyan, Purple4 Saturation/Lightness levels (”cuts”): Saturated, Light, Muted, Dark
+ 5 Achromatic Colors: White, Black, Light, Medium, & Dark Gray
Berkeley Color Project (BCP) 37 Colors 64 Mozartian Melodies
Systematic associations between classical orchestral music and color dimensions: (Schloss, Lawler & Palmer, VSS-08)
The Emotional Mediation Hypothesis: When choosing colors to “go best with” an emotional stimulus in a di�erent modality, people choose the colors that best match the emotional content of that stimulus.
The original color-music experiment used 18 recordings of orchestral selections as musical stimuli. 1. Will the color-music associations replicate with more tightly-controlled musical stimuli? 2. Which musical dimensions are most in�uential in the relation between colors and music?
The present study used single-line unaccompanied piano theme-and-variation melodies to vary: mode (major vs. minor key) note-density (quarter-notes vs. eighth-notes), tempo (fast vs. slow speed) register (high vs. low)
while controlling for (i.e., no variation in): dynamics (soft vs. loud) orchestration (multiple instruments playing together) timbre (tone color) harmonic mode (harmony of simultaneous notes)
timbretimbre
Color-Music Associations
37 colors presented simultaneously with each melody. 1) Select the 3 colors that are most consistent with the melody in order: 1st, 2nd, 3rd most consistent.
2) Select the 3 colors that are most inconsistent with the melody in order: 1st, 2nd, 3rd most inconsistent.
Music-Emotion Associations
HappySad
Each of 37 colors were rated on four bipolar emotional scales, one at a time: Agitated-Calm Happy-Sad Strong-Weak Angry-Not Angry
HappySad
Color-Music Association (CMAD) was calculated for each melody for each color appearance dimension (D) as the average D-value of the 3 colors judged to be most consistent with the melody (CD) minus the average D-value of the 3 colors judged to be most inconsistent with the melody (ID):
CD = (3c1,D+ 2c2,D+ c3,D )/3, ID = (3i1,D+2i2,D+ i3,D )/3, CMAD = CD - ID
most consistent color
4 single-line melodies adapted from Mozart piano themes
16 variations for each melody varied along four dimensions:
1) major vs. minor mode 3) sparse vs. dense notes 2) slow vs. fast tempo 4) high vs. low register
Slow-Sparse Slow-DenseFast-Sparse
Fast-Dense-150
-100
-50
0
50
100
150
200
250
CM
A (S
H)
Note Rate
Slow-Sparse Slow-DenseFast-Sparse
Fast-Dense-150
-100
-50
0
50
100
150
200
250
CM
A (A
)
Note Rate
Slow-Sparse Slow-Dense
Fast-SparseFast-Dense
-150
-100
-50
0
50
100
150
200
250
CM
A (W
S)
Note Rate
Slow-Sparse Slow-DenseFast-Sparse
Fast-Dense-150
-100
-50
0
50
100
150
200
250
CM
A (C
A)
Note Rate
Happier emotion Sadder Emotion Major (42%) Minor Faster (33%) Slower Higher (7%) LowerTotal variance 82%
Strong emotion Weak emotion Faster (60%) Slower High (5%) Low Major (2%) MinorTotal variance 67%
Agitated emotion Calm emotion Faster (69%) Slower Minor (4%) MajorTotal variance 73%
Angry emotion Not Angry Minor (73%) MajorTotal variance 73%
Yellower colors Bluer colors Faster (41%) Slower Major (22%) Minor Higher (11%) LowerTotal variance 74%
Redder colors Greener colors Fast (45%) SlowerTotal variance 45%
Lighter colors Darker colors Major (53%) Minor Higher (17%) Lower Faster (8%) SlowerTotal variance 78%
Higher saturation Lower saturation Faster (67%) Slower Major (20%) MinorTotal variance 87%
As in the �rst color, music, and emotion study (Schloss, Lawler & Palmer, VSS-08), both color and music were strongly associated with speci�c emotions: Happy-Sad, Agitated-Calm, Strong-Weak, Angry-Not Angry.
Better-controlled musical stimuli revealed the signi�cant in�uence of distinct musical dimensions (major/minor mode, slow/medium/fast note-rate, and low/high register) on color-associations and ratings of emotional content (independent of timbre, dynamics, orchestration, and harmonic mode).
When making color-music associations, subjects seem to have evaluated the emotional content of the melodies and to have chosen the colors that had the same emotional content. This �nding supports the Emotional Mediation Hypothesis.
Listen to the music
Evaluate the emotional content of the music, based on its mode, note-rate, and register.
Choose the colors whose emotional content best
matches that of the music
Schloss, K. B., Lawler, P. & Palmer, S. E. (VSS-2008). “The Color of Music.” Presented at the 8th Annual Meeting of the Vision Sciences Society, Naples, FL, May 2008.Xu, Schloss, K. B. & Palmer, S. E. (VSS-2010). “The Color of Emotionally Expressive Faces.” Presented at the 10th Annual Meeting of the Vision Sciences Society, Naples, FL, May 2010.
AcknowledgementsWe thank Rosa Poggesi, Arielle Younger, Madison Zeller, and Mathilde Heinemann (at UC Berkeley) for collecting data. We also thank Eli Strauss (also at UC Berkeley) for help with programming in Presentation, and Karle-Philip Zamor (at McGill) for generating the �nal musical stimuli from the original wave �les. We further thank the National Science Foundation (#BCS-0745820) and Google for �nancial support.
most inconsistent color
There are high correlations between the emotional ratings of each melody and the emotional ratings of colors consistent with that melody:
CE = (3c1,E + 2c2,E + c3,E )/3, IE = (3i1,E+2i2,E+ i3,E )/3, CMAE = CE - IE
most consistent color
most inconsistent color
Emotional Color-Music Association (CMAE) was calculated for each melody for each emotion dimension (E) as the average E-value of the 3 colors judged to be most consistent with the melody (CE) minus the average E-value of the 3 colors judged most inconsistent with the melody (IE):
Choose the color whose emotional content matches that of the stimulus
Evaluate emotional content of the object
Experience the stimulus to be associated with the color
Xu, Schloss & Palmer (VSS-10)
Schloss, Lawler & Palmer (VSS-08)
-100 -50 0 50 100-150
-100
-50
0
50
100
150
Happy vs. Sad
-100 -50 0 50 100-150
-100
-50
0
50
100
150
Angry vs. Not Angry
Music-Emotion Association
-100 -50 0 50 100-150
-100
-50
0
50
100
150
Strong vs. Weak
Music-Emotion Association
-100 -50 0 50 100-150
-100
-50
0
50
100
150
Agitated-Calm
Music-Emotion Association
CM
AE
HA
PPY
SAD
0
HAPPYSAD 0
Music-Emotion Association
0 ANGRYNOT-ANGRY
CM
AE
AN
GRY
NO
T-A
NG
RY
0
STR
ON
GW
EAK
0
CM
AE
STRONGWEAK 0
AG
ITAT
EDC
ALM
0
CM
AE
AGITATEDCALM 0
r = 0.92*** r = 0.82*** r = 0.63*** r = 0.88***
Slow, sparse melody
Slow, dense melody
Fast, sparse melody
Fast, dense melody
A B C D
A A’ B’ C’B C D D’
A B DC
A A’ B’ C’ D’B DC
Medium
Slow
Fast
TimeNote RateMelody
250 250
Emotional correlations between color and ...
Happy-Sad Strong-Weak music r = .97 r = .96
Happy-Sad Strong-Weak faces r = .97 r = .91
Happy-Sad Angry-NotAngry gestures r = .95 r = .94
D = Yellow/Blue D = Red/Green D = Light/Dark D = Saturation
Minor
Major
High
HighLow
Low
Minor
MajorHigh
High
Low
Low
Slow-Sparse Slow-DenseFast-Sparse
Fast-Dense-150
-100
-50
0
50
100
150
200
250
CM
A (Y
B)
Happy-Sad Strong-Weak
Note Rate
Slow-Sparse Slow-Dense
Fast-SparseFast-Dense
-150
-100
-50
0
50
100
150
200
250
CM
A (R
G)
Note Rate
Slow-Sparse Slow-DenseFast-Sparse
Fast-Dense-150
-100
-50
0
50
100
150
200
250
CM
A (L
D)
Note Rate
Slow-Sparse Slow-DenseFast-Sparse
Fast-Dense-150
-100
-50
0
50
100
150
200
250
CM
A (S
at)
Note Rate
Minor
Major
Slow Medium Fast
-200
0
200
Saturation
Tempo
Col
or-M
usic
Ass
ocia
tion Major
Minor
Slow Medium Fast
-200
0
200
Light/Dark
TempoSlow Medium Fast
-200
0
200
Red/Green
TempoSlow Medium Fast
-200
0
200
Yellow/Blue
Tempo
Satu
rate
dD
esat
ura
ted
Lig
htn
ess
Dar
knes
s
Red
nes
sG
reen
nes
s
Yello
wn
ess
Blu
enes
s
Minor
Major
High
High
Low
Low
Minor
Major
These data provide strong support for the emotional mediation hypothesis
Color-Emotion Associations
High
Low
Minor
Major
Minor
Major
-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1-0.5
-0.4
-0.3
-0.2
-0.1
0
0.1
0.2
0.3
0.4
0.5
Dimension 1
Dim
ensi
on 2
Major Medium Low
Major Fast High
Major Fast Low
Major Slow High
Major Slow Low
Major Medium Low
Minor Medium High
Minor Medium Low
Minor Fast High
Minor Fast Low
Minor Slow High
Minor Slow Low
Minor Medium High
Minor Medium Low
Happy
Sad
Emotional MDS of Melodies
Slow
Fast
Minor Major
Strong
Weak
Major Medium HighMajor Medium High
Happy
Sad
Strong
Weak
Dimension 1
Dim
ensi
on 2
-1.5
-1
-0.5
0
0.5
1
1.5
-1.5 -1 -0.5 0 0.5 1D
esat
urat
edSa
tura
ted
Dark Light
Emotional MDS of ColorsMDS of Emotional Spaces for Melodies and Colors
Melodies: A 16x16 proximity matrix was derived from the correlations of the average emotional ratings for each of the 16 melody types (major/minor, fast/slow, high/low, dense/sparse).
This matrix was input to ALSCALE, a multidimensional scaling (MDS) program, and the best-�tting 2-D solution was found. The primary emotional axes appear to be happy-sad and strong-weak.
Colors: A 37x37 proximity matrix was derived from the correlations of the average emotional ratings for each of the 37 colors (8 hues at each of 4 lightness-saturation levels, plus 5 grays).
This matrix was input to ALSCALE, a multidimensional scaling (MDS) program, and the best-�tting 2-D solution was found. The primary emotional axes also appear to be happy-sad and strong-weak.
Each of 64 melodies were rated on four bipolar emotional scales, one at a time: Agitated-Calm Happy-Sad Strong-Weak Angry-Not Angry
E = Happy/Sad E = Angry/Not Angry E = Agitated/CalmE = Strong/Weak