Lecture 3: Theories of Emotionpeople.ict.usc.edu/~gratch/CSCI534/Lecture2015-03.pdf · Lecture 3:...
-
Upload
duongquynh -
Category
Documents
-
view
229 -
download
1
Transcript of Lecture 3: Theories of Emotionpeople.ict.usc.edu/~gratch/CSCI534/Lecture2015-03.pdf · Lecture 3:...
CSCI 534(Affective Computing) – Lecture by Jonathan Gratch
Affective Computing in the news
Dacher Keltner
Berkeley
CSCI 534(Affective Computing) – Lecture by Jonathan Gratch
Outline
Review why emotion theory useful– Give some positive and negative examples
Introduce some features that distinguish different theories– Emotions as discrete or continuous
– Emotions as “atoms”, “molecules”, or “mxtures”
– Emotions as a consequence or antecedent of emotion
Review some specific influential theories
In-class “experiment” (3-unit students welcome to depart)
CSCI 534(Affective Computing) – Lecture by Jonathan Gratch
What is a Theory
Theory explains how some aspect of human behavior
or performance is organized. It thus enables us to make
predictions about that behavior.– Provides a set of interrelated concepts, definitions, and propositions
that explains or predicts aspects of human behavior by specifying
relations among variables.
– Allows us to explain what we see and to figure out how to bring about
change.
– Is a tool that enables us to identify a problem and to plan a means for
altering the situation.
– Create a basis for future research. Researchers use theories to form
hypotheses that can then be tested.
– Creates a basis for building software: suggests what variables are
important to measure and how they relate to each other
CSCI 534(Affective Computing) – Lecture by Jonathan Gratch
Does a learned model count as a theory?
We’ll learn about machine learning approaches– Collect bunch of data
– Look at lots of features and try to predict some outcome
Allow us to make predictions?– Yes
Give insight into underlying mechanism?– Not typically (black box). But can answer what features are relevant
Indicate how to bring about change?– Not typically
Input
Features(events in a
video game)
Predicted
output
CSCI 534(Affective Computing) – Lecture by Jonathan Gratch
And can be misleading: Famous example
In ‘80s, the Pentagon wanted to harness computer technology to make
their tanks harder to attack.
The research team went out and took 100 photographs of tanks hiding
behind trees, and then took 100 photographs of trees - with no tanks.
They trained a neural network. It reached near-perfect accuracy
Independent testing showed all “no tank” photos taken on sunny day
and all “tank” photos taken on cloudy day
Because neural network was “black box”, this not easy to discover
CSCI 534(Affective Computing) – Lecture by Jonathan Gratch
Affective Computing Example
Last week, showed you system that tIn
tries to recognize nonverbal signs of depression and PTSD
We collected data from two populations
– Craigslist (and online job-recruiting service)
– US Vets: organization that provides mental-health service for former soldiers
Tried machine learning approach
Discovered vocal pitch a strong predictor of depression
– Lower pitch predicted depression severity
– Not predicted by existing theory
Turns out there was big imbalance in our data
– US Vets had highest rates of depression
– US Vets also had highest rates of Males (most soldiers are male)
– We actually “discovered” that men speak with a lower pitch
CSCI 534(Affective Computing) – Lecture by Jonathan Gratch
Advantages of building on theory
Theory makes explicit the mechanisms that (are
claimed to) underlie some behavior– Allows us to explain what we see and to figure out how to bring about
change
Theories (typical) have good empirical support– The theories we will discuss are supported by dozens of empirical
studies
– They may still be incorrect of insufficient but are unlikely to suffer the
sort of mistakes we just discussed
CSCI 534(Affective Computing) – Lecture by Jonathan Gratch
10
World Events Mental State(beliefs, goals)
Example: Appraisal Theory
Argues for importance of three
interrelated concepts
• World events
• Mental state (e.g. goals)
• Emotional Response
If we know two of these
variables, we can make
predictions about the third
Response= f(Env., Mind)
Body
Expression
Action Tendency
Physiological Response
CSCI 534(Affective Computing) – Lecture by Jonathan Gratch
11
EnvironmentBeliefs,
Goals
E.G.: Generating Emotional Response R=f(E,M)
COMPUTER PREDICITONS:
• Computer could predict what
emotion a person might hold
• Computer could generate a
believable emotion to user
Emotional Response
Expression
Action Tendency
Physiological Response
CSCI 534(Affective Computing) – Lecture by Jonathan Gratch
12
Emotional Response
Expression
Action Tendency
Physiological Response
EnvironmentBeliefs,
Goals
E.G.: inferring emotional antecedents M=f-1(E,R)Reverse Appraisal
COMPUTER PREDICITONS:
• Computer could predict what
goal person has (i.e., what
team are they rooting for
CSCI 534(Affective Computing) – Lecture by Jonathan Gratch
“Angry” “Happy”
“Sad”“Apa-
thetic”
Influential theory: Galen’s 4-process model of emotion
“Valence”
“Arousal”
Posits 4 “prototypical emotions”.
Emotions organized in 2-dimensional space (valence, arousal)
Argues emotions tend to transition along arrows.
CSCI 534(Affective Computing) – Lecture by Jonathan Gratch
“Angry” “Happy”
“Sad”“Apa-
thetic”
Galen’s 4-process model of emotion
“Valence”
“Arousal”
YB
BB
Bl
Ph
Prototypical emotions associated with 4 specific physiological systems
CSCI 534(Affective Computing) – Lecture by Jonathan Gratch
Popular 2-dimensional model of emotion
“Valence”
“Arousal”
YB
BB
Bl
Ph
Happy
Apathetic
Angry
Sad
Each prototypical state associated with a characteristic expression
C S
M P
CSCI 534(Affective Computing) – Lecture by Jonathan Gratch
Again, this theory affords implementation and prediction
Dimension 2
Dim
en
sio
n 1
YB
BB
Bl
Ph
Happy
Apathetic
Angry
Sad
C S
M P
Recognition “language”– 4 “prototypical” emotion labels but
– 2 dimensions
Predictions– If we recognize Anger expect YB is active
– If recognized Anger, don’t expect transition to
Apathy
– If BB active, expect sad expressions and self-
report of Sadness
Control– Can create Apathy by activating Ph system or
suppressing other systems
– Can’t control Ph by activating Apathy
CSCI 534(Affective Computing) – Lecture by Jonathan Gratch
Choleric Sanguine
Melan-
choly
Phleg-
matic
Theory of the Four Humoursby Galen of Pergamun (c. 180AD)
Wetness
Tem
pera
ture
Yellow
Bile
Black
Bile
Blood
Phlegm
Wet
(Water)
Dry
(Earth)
Cold
(Air)
Hot
(Fire)
A Hippocratic physician would prescribe treatment to void the
body of imbalanced humor. if it was a fever -- a hot, dry disease --
the culprit was yellow bile or blood. So, the doctor could reduce
this by, e.g., bleeding the patient, or increase its opposite,
phlegm, by prescribing cold baths.
CSCI 534(Affective Computing) – Lecture by Jonathan Gratch
Proof of the Theory of the Four Humoursby Galen of Pergamun (c. 180AD)
1793 an epidemic of yellow fever struck Philadelphia
Benjamin Rush (signer of Declaration of Independence)
treated by vigorous bloodletting
Each patient that recovered and survived served to prove the
theory
Each patient that died confirmed that the patient was too ill for
the treatment to work
Any issue with this?– Example of confirmation bias: a common decision-making bias
– Another example: “proof that aliens have landed on earth”
CSCI 534(Affective Computing) – Lecture by Jonathan Gratch
Falsifiability
A good theory is falsifiable– Falsifiability or refutability of a theory is an inherent possibility to
prove it to be false.
– Theories that are so vague they can explain anything (ex. Psychic
readings) are not falsifiable
– The more specific a theory it is, the more likely it is falsifiable
Galen’s theory is falsifiable (and has be falsified)– Even if Benjamin Rush failed to test it properly
Karl Popper: on falsifiability, testability
‘What characterises the empirical method is its manner of
exposing to falsification, in every conceivable way, the
system to be tested. Its aim is not to save the lives of
untenable systems but, on the contrary, to select the one
which is by comparison the fittest, by exposing them all to
the fiercest struggle for survival’
CSCI 534(Affective Computing) – Lecture by Jonathan Gratch
Why should we care about emotion theory
Provides a definition of “emotion” and other related concepts
that influence, or are influenced by emotion, and thus a
starting point for affective computing
Unfortunately, psychology hasn’t sorted it all out yet
– Different theories suggest different concepts and relationships between them
E.g., Say we want to recognize emotion
– Give labeled data to machine learning algorithm
– But what are the labels?
Joy vs. Hope vs. Fear?
Positive vs. Negative?
Affective computing researchers must make educated guess
about which theory to use
– But their success or failure can inform research in the social sciences
CSCI 534(Affective Computing) – Lecture by Jonathan Gratch
21
Human
Behavior
Theory
• Psychology
• Linguistics
• Neuroscience
• Economics
Data
Integrated
“Test bed”
Theory
• e.g., Rapport (positive,
contingent, nonverbal feedback)
facilitates conflict
resolution
Affective computing is interdisciplinary science
MRE SASO-ST Gunslinger DCAPSRapport
RapportEmbed capability
within interactive
virtual human
testbed
CSCI 534(Affective Computing) – Lecture by Jonathan Gratch
22
Human
Behavior
Integrated
“Test bed”
Human
Studies
Affective computing is interdisciplinary science
Verify Implementation
• Consistent with prior
findings?
• Treated “as if” real
Test theoretical predictions
MRE SASO-ST Gunslinger DCAPS
Inform theoretical
debate in social
science
Rapport
CSCI 534(Affective Computing) – Lecture by Jonathan Gratch
For us,A theory should answer “What is emotion?”
Emotion is a feeling
Emotion is a state (of physiological arousal)
A brain process that computes the value of an experience --- Le Doux
A word we assign to certain configuration of bodily states, thoughts, and
situational factors – Feldman Barrett.
God’s punishment for disobedience -- St Augustine
CSCI 534(Affective Computing) – Lecture by Jonathan Gratch
But also, what is emotion NOT?
From Scherer (optional reading)
CSCI 534(Affective Computing) – Lecture by Jonathan Gratch
So what is the accepted theory of emotion?
Unfortunately, none exists
CSCI 534(Affective Computing) – Lecture by Jonathan Gratch
What is an emotion?
Components of emotion
Emphasizes that emotion potentially impacts several aspects
– Cognitive: influences or influenced by thinking
– Physiological: related to hormones, heart-rate, sweating…
– Expressive: relates to facial expressions, posture, vocal features
– Motivation: relates to goals and drives
– Feeling: relates to conscious awareness being in an emotional state
CSCI 534(Affective Computing) – Lecture by Jonathan Gratch
What is an emotion?
Phases of emotion: Emphasizes that emotions have “stages”
– Low-level: automatic cognitive processes (e.g., reflexes)
– Hi-level: deliberate, conscious cognitive processes
– Goals/need setting
– Examining action alternative: decision-making/action-selection
– Behavior preparation
– Behavior execution
– Communication with other
CSCI 534(Affective Computing) – Lecture by Jonathan Gratch
What is an emotion?
Different theories emphasize different aspects:
– Appraisal theories emphasize cognitive antecedents of emotion
– Discrete emotion theories emphasize physiological and expressive
consequences of emotion
Affective computing researchers tend to draw on different
theories depending on the aspects they focus on
– E.g : emotion recognition techniques often draw upon discrete
emotion theory and avoid appraisal models
CSCI 534(Affective Computing) – Lecture by Jonathan Gratch
What is an emotion: theoretical disagreements
Different theories can be distinguished by how they
chose to define emotion with respect to the
previously-mentioned components and phases
– Is emotion discrete or continuous?
– Is emotion an “atom” or “molecule”? (Barrett)
– Is emotion an antecedent or consequent of cognition?
CSCI 534(Affective Computing) – Lecture by Jonathan Gratch
Emotions as discrete categories,
biologically fixed, universal to all humans
and many animals
Basic Emotions: Anger, disgust, fear,
happiness, sadness, surprise
Rene Decartes, Silvan Tomkins, Paul
Ekman
Emotions are a combination of several
psychological dimensions
Wilhelm Wundt, James Russell, Lisa
Feldman Barrett
CSCI 534(Affective Computing) – Lecture by Jonathan Gratch
Emotions as discrete categories,
biologically fixed, universal to all humans
and many animals
Basic Emotions: Anger, disgust, fear,
happiness, sadness, surprise
Rene Decartes, Silvan Tomkins, Paul
Ekman
Emotions are a combination of several
psychological dimensions
Wilhelm Wundt, James Russell, Lisa
Feldman Barrett
CSCI 534(Affective Computing) – Lecture by Jonathan Gratch
Some discrete emotion theories
Tomkins
– Excitement, joy, surprise, distress,
anger, fear, shame,
dissmell (reaction to bad smell),
disgust (reaction to bad taste)
Ekman
– Sadness, happiness, anger, fear,
disgust, and surprise,
sometimes includes contempt
E.g. Le Doux fear
circuit
CSCI 534(Affective Computing) – Lecture by Jonathan Gratch
Some Dimensional models
Russell & Mehrabian’s ‘77 PAD model (pleasure,
arousal, dominance) Russell’s ‘80 circumplex model
High self-control ↔“letting go”
Mania ↔ depression
CSCI 534(Affective Computing) – Lecture by Jonathan Gratch
Implications for classification / measurement
Continuous
CSCI 534(Affective Computing) – Lecture by Jonathan Gratch
Implications for classification / measurement
Discrete
Disgust Fear Surprise
Continuous
CSCI 534(Affective Computing) – Lecture by Jonathan Gratch
Emotion components are tightly-coupled
and can be treated as a circuit linking
stimuli and response
Jaak Panksepp, Joseph LeDoux, Paul
Ekman
Emotions are defined by loose
configuration of different components
Phoebe Ellsworth, Klaus Scherer, Lisa
Feldman Barrett
Atom Molecule or Mixture
CSCI 534(Affective Computing) – Lecture by Jonathan Gratch
Emotion components are tightly-coupled
and can be treated as a circuit linking
stimuli and response
Jaak Panksepp, Joseph LeDoux, Paul
Ekman
Emotions are defined by loose
configuration of different components
Phoebe Ellsworth, Klaus Scherer, Lisa
Feldman Barrett
Atom Molecule or Mixture
CSCI 534(Affective Computing) – Lecture by Jonathan Gratch
Implications for classification / measurement
If emotion is atomic circuit, all components should be aligned
– i.e., Facial expressions, physiological response and felt emotion
should be consistently-aligned with each other
– “Emotion” can refer to the overall circuit but can be measured by any
of the components
– Measured expressions should predict physiology and felt emotion
– Multi-modal recognition should perform the best
CSCI 534(Affective Computing) – Lecture by Jonathan Gratch
Implications for classification / measurement
If emotion is atomic circuit, all components should be aligned
– i.e., Facial expressions, physiological response and felt emotion
should be consistently-aligned with each other
– “Emotion” can refer to the overall circuit but can be measured by any
of the components
– Measured expressions should predict physiology and felt emotion
– Multi-modal recognition should perform the best
If emotion a molecule or mixture, components not aligned
– Allow that components influence each other but may be out of sync
– Expressions need not accurately reflect physiology and felt emotion
– Constructivist Theories (Feldman Barrett): Emotion is a label we
assign to our sensed physiological state
– Appraisal theories (Scherer & Ellsworth): Emotion is a label a scientist
might apply when different components align in a prototypical way
CSCI 534(Affective Computing) – Lecture by Jonathan Gratch
Atom or Mixture
Discrete: redundancy across channels– Multimodal should be strictly better than unimodal
Predicted
output
CSCI 534(Affective Computing) – Lecture by Jonathan Gratch
Atom or Mixture
Discrete: redundancy across channels– Multimodal should be strictly better than unimodal
– Late fusion should be great
Mixture: not so fast…– Or at least, association between modalities and predicted emotion is complex
Predicted
output
CSCI 534(Affective Computing) – Lecture by Jonathan Gratch
Thought precedes emotion. Emotion
precedes and motivates behavior
Walter Cannon, Phoebe Ellsworth, Klaus
Scherer
Behavioral response precedes our
labelling the situation as emotional
William James, Stanley Schachter, Lisa
Feldman Barrett
Top down(e.g. Appraisal Theory)
Behav. Drives emotion(e.g., constructivist theories)
CSCI 534(Affective Computing) – Lecture by Jonathan Gratch
Thought precedes emotion. Emotion
precedes and motivates behavior
Walter Cannon, Phoebe Ellsworth, Klaus
Scherer
Behavior and body response precedes
and motivates emotion and cognition
William James, Stanley Schachter, Lisa
Feldman Barrett
Top down(e.g. Appraisal Theory)
Bottom up (e.g., constructivist theories)
CSCI 534(Affective Computing) – Lecture by Jonathan Gratch
Appraisal Theory
“Bottom up” theories argue “seeing the bear” produces fear-like
reactions automatically
What if we knew the bear was friendly?
What if we knew the bear was chained up?Magda
Arnold
CSCI 534(Affective Computing) – Lecture by Jonathan Gratch
Appraisal Theory
Appraisal models emphasize the prior beliefs and goals determine
shape emotional responses
Explain this by arguing that cognitive processes ESSENTIAL in
initiating emotional responses
World events are “appraised” along a number of dimensions:
– Is the event good or bad with respect to my goals
– Did I expect the event
– Can I control the event
– Who do I blame for the event
Different patterns of appraisal will lead to different emotions
– I blame someone else for something bad Anger
CSCI 534(Affective Computing) – Lecture by Jonathan Gratch
Some Appraisal Theories
Ortony, Clore and Collins (OCC) Appraisal Variables
• desirability
•appealingness
•praiseworthyness
•certainty
CSCI 534(Affective Computing) – Lecture by Jonathan Gratch
Some Appraisal Models
Scherer sequential checking theory
Appraisal Variables
• Relevance
• Implication
• Coping potential
• Normative significance
CSCI 534(Affective Computing) – Lecture by Jonathan Gratch
Appraisal theory takeaway
Emotions arise from appraisal of goals and beliefs– Emphasizes centrality of beliefs, desires and intentions to emotion elicitation
Event has no meaning in of itself
Emotion arises from how event impacts goals and beliefs
Same event will have different meaning to different people
CSCI 534(Affective Computing) – Lecture by Jonathan Gratch
But not always so simple
Belief: I’m standing in a room
CSCI 534(Affective Computing) – Lecture by Jonathan Gratch
But not always so simple
Belief: I’m standing in a room
Does this contradict appraisal theory?
CSCI 534(Affective Computing) – Lecture by Jonathan Gratch
Example (Constructivist Theory)
Argues first step in the experience of emotion is
physiological arousal– Seeing the bear triggers low-level automatic reactions such as arousal
and running away
We next try to find a label to explain our feelings,
usually by looking at what we are doing (behavior)
and what else is happening at the time of arousal
(environment)
Thus, we don’t just feel angry, happy, etc. We
experience general feeling and then decide what
they mean (a specific emotion)
CSCI 534(Affective Computing) – Lecture by Jonathan Gratch
Basic emotion
Theory
Constructivist
Theory
Appraisal
Theory
CSCI 534(Affective Computing) – Lecture by Jonathan Gratch
Lesson: Definitions matter
Geocentricity
– Placing earth at center of universe makes it difficult to predict motion
of the planets
Alchemy
– All substance can be decomposed into earth, water, air and fire
making it difficult to predict consequences of chemical reactions
Point:
– Theory important: allows us make specific predictions and explain
variance
– Important steps on way to deeper understanding
– Recognize that technological choices depend (implicitly or explicitly)
on (folk or scientific) theoretical assumptions and failure of the
technology may reflect problems with theory, not software
CSCI 534(Affective Computing) – Lecture by Jonathan Gratch
In-class exercise
Split class into 3 groups
• Need group of four volunteers to watch a video
• Most of class will stay put and watch this group
• Need one more group of four to watch the class
I’ll give out some handouts
• First group will mark down how they feel watching the video
• Class will guess what the first group is feeling based on their reactions
• Last group will guess what the first group is feeling based on class’s
reactions
NO TALKING
CSCI 534(Affective Computing) – Lecture by Jonathan Gratch
Discussion
Classification– What featured did you use to identify the felt emotion
Dimensions vs. Basic emotions– Which framework best captured the “meaning” of the interaction
Observers– Why were (or weren’t) the 3rd group able to infer what is going on
Mirroring
How do you think a computer would do?