Emotional Design - California State University Channel...
Transcript of Emotional Design - California State University Channel...
Emotional Design
• D. Norman (Emotional Design, 2004)
• Model with three levels
– Visceral (lowest level)
– Behavioral (middle level)
– Reflective (top level)
Emotional Intelligence (EI)
• IQ is not the only indicator of intelligence (Emotional Intelligence book by Daniel Goleman, 1995)
• EI: Awareness and ability to manage one’s emotions in a healthy manner.
• EI: Ability to sense, perceive, understand, and assess own and other people’s emotions
Is Mr. Spock intelligent?
• Spock is only rational
• Descarte’s Error (Damasio, 1994)
• Artificial intelligence searches unlimited
search space to make a rational decision
• Missing ‘somatic markers’ that associate
feelings with decisions
Damasio’s Somatic Marker Hypothesis
• Originated from the observation of individuals who had sustained damage to the ventromedial prefrontal cortex.
• Normal intellectual function
• Normal Neuropsychological function
• Normal on tests sensitive to frontal lobe function
• However, severe impairment in personal and social decision making and conduct. – Difficulty with planning in the immediate, and future.
– No longer able to make personally advantageous decisions
– Often sustain social, personal, economic losses
• The only deficit that could be detected was one in which these individuals failed to display emotion in situations in which emotion would be normatively expected.
• This led Damasio to posit that these individuals manifest a deficit in reasoning that is secondary to deficits in emotional processing.
What is Affect?
• The type and degree of emotion a person
displays
• The experienced, subjective, and
conscious aspect of feeling or emotion
– Positive
– Negative
– Neutral
Affect Theory
• Developed by Silvan S. Tomkins in 1962
• Tomkins book Affect Imagery (3 vols.)
• Believed that the affect system is the
motivating force in human life.
• Organized affect into 3 main categories:
– Positive, negative, and neutral
– Each has a low/high intensity label
Tomkins nine affects
• Positive:
• Enjoyment/Joy - smiling, lips wide and out
• Interest/Excitement - eyebrows down, eyes tracking, eyes looking, closer listening
• Neutral:
• Surprise/Startle - eyebrows up, eyes blinking
• Negative:
• Anger/Rage - frowning, a clenched jaw, a red face
• Disgust - the lower lip raised and protruded, head forward and down
• Dissmell (reaction to bad smell) - upper lip raised, head pulled back
• Distress/Anguish - crying, rhythmic sobbing, arched eyebrows, mouth lowered
• Fear/Terror - a frozen stare, a pale face, coldness, sweat, erect hair
• Shame/Humiliation - eyes lowered, the head down and averted, blushing
Affective Computing
• Computing that relates to, arises from, or
deliberately influences emotions
• Coined by Rosalind Picard
– Founder and director of the Affective Computing
Research Group at the MIT Media Lab.
– Her book, Affective Computing (1997) lays the
groundwork for giving machines the skills of
emotional intelligence.
Affective computing is related to other computing disciplines:
Artificial Intelligence (AI),
Virtual Reality (VR) and
Human Computer interaction (HCI).
Questions that need to be answered:
What is an affective state (typically feelings, moods, sentiments etc.)?
Which human communicative signals convey information about affective state?
How various kinds of affective information can be combined to optimize inferences
about affective states?
How to apply affective information to designing systems?
Affective Computing
• Recognize emotions
• Express emotions
• ‘Have’ emotions
Recognize Emotions
• Bio-signals (wearable sensors)
• Brain Signals, skin temperature, blood
pressure, heart rate, respiration rate
• Facial Expressions
• Speech/Vocal expressions
• Gestures
• Limbic movements
• Text
Recognition
• we need an emotion model that allows us to
differentiate between emotional states
• we need a classification scheme that uses
specific features from an input signal to recognize
the user’s emotions
Text
Sentiment Analyzing Discussion Board
http://socialxyz.com/SAD/
The Natural Language Toolkit, or more commonly NLTK, is a
suite of libraries and programs for symbolic and statistical
natural language processing (NLP) for the Python
programming language.
Speech
Paralinguistic Features of Speech – how is it said?
Prosodic features (e.g., pitch-related feature, energy-related features,
and speech rate)
Spectral features (e.g., MFCC - Mel-frequency cepstral coefficient
and cepstral features)
Spectral tilt, LFPC (Log Frequency Power Coefficients)
F0 (fundamental frequency of speech), Long-term spectrum
Studies show that pitch and energy contribute the most to affect
recognition
Speech disfluencies (e.g., filler and silence pauses)
Context information (e.g., subject, gender, and turn-level features
representing local and global aspects of the dialogue)
Nonlinguistic vocalizations (e.g., laughs and cries, decode other
affective signals such as stress, depression, boredom, and excitement)
Feature Extraction Pre-processing
Speech Signal
Classification
Classified Result Audio recordings collected in call centers
and, meetings, Wizard of Oz scenarios
interviews and other dialogue systems
• Accuracy rates from speech are somewhat lower
(35%) than facial expressions for the basic emotions .
• Sadness, anger, and fear are the emotions
that are best recognized through voice, while
disgust is the worst.
]M. Pantic, N. Sebe, J. F. Cohn, and T. Huang. Affective multimodal human-computer interaction. In ACM International Conference on Multimedia (MM), 2005. Rafael A. Calvo, Sidney D'Mello, "Affect Detection: An Interdisciplinary Review of Models, Methods, and Their Applications
Lucey, P.; Cohn, J.F.; Kanade, T.; Saragih, J.; Ambadar, Z.; Matthews, I.; , "The Extended Cohn-Kanade Dataset (CK+): A complete dataset for action unit and emotion-specified expression," Computer Vision and Pattern Recognition Workshops (CVPRW), 2010 IEEE Computer Society Conference on , vol., no., pp.94-101, 13-18 June 2010
Facial Expressions
Example: Active Appearance Model (AAM)
(AAM) based system which uses AAMs to track the face
and extract visual features. Support vector machines are used
(SVMs) to classify the facial expressions and emotions.
Bio-signals 22
• Physiological signals derived from Autonomic
Nervous System (ANS) of human body.
– Fear for example increases heartbeat and
respiration rates, causes palm sweating, etc.
• Psychological Metrics used are:
– GSR - Galvanic Skin Resistance
– RESSP - Respiration
– BVP - Blood Pressure
– Skin Temperature
• Electroencephalogram (EEG), Electrocardiogram
(ECG), Electrodermal activity (EDA),
Electromyogram (EMG)
• Skin conductivity sensors, blood volume sensors, and
respiration sensors may be integrated with shoes,
earrings or watches, and T-shirts
Huaming Li and Jindong Tan. 2007. Heartbeat driven medium access control for body sensor networks. In Proceedings of the 1st ACM SIGMOBILE international workshop on Systems and networking support for healthcare and assisted living environments (HealthNet '07). ACM, New York, NY, USA, 25-30.
gesture and body motion information is an important modality for human affect recognition; combination of face and gesture is 35% more accurate than facial expression alone.
Two categories of Body-Motion-based affect recognition
Stylized
The entirety of the movement encodes a particular emotion.
Non-stylized
More natural - knocking door, lifting hand, walking etc.
Gestures
Fusion
• Neural Networks (NN) • Hidden Markov Models (HMM)
• K-Nearest Neighbors (KNN)
• Linear Discriminant Analysis (LDA)
• Support Vector Machines (SVM)
• Gaussian Mixture Models (GMM)
• Discriminant Function Analysis (DFA)
• Sequential Forward Floating Search (SFFS)
Frequently used Detection and
Estimation Techniques
Express emotions
• Kismet (Breazeal and Scassellati, 2002)
• Emotional expression for
communication and social co-ordination
• Emotion for organisation of behaviour
(action selection, attention and learning)
• Arbib and Fellous (2004)
• More effective expression than humans:
• Human expression identified 50% of the
time. Computer expression identified
70% of the time (Elliott, 1997).
https://www.youtube.com/watch?v=PtCIbGjJV4c
1
3
2
4
1
3
2
4
1. Listening
2. Understand
3. Confused
4. Waving goodbye
EXPERIMENTS AND RESULTS
35 35
Navigation Agent
Safety Agent
Driving Aid Agent
Affective Multimedia Agent
Audio Linguistic / Non-linguistic
Facial Expression
Seat Pressure
Actions •Steering Movement
•Interaction with Gas / Break Paddle
Bio-signals
Stress Level Basic Emotions
Feature Detector
Feature Detector
Feature Detector
…………...
Feature Estimator
Complex Emotions
……
Route Selection
Inter agent
communication
to aid decision
making
Notify
in case of
Emergency
Speed, ABS,
Traction Control
Music, Climate
Control
Alert the Driver
Have emotions
• Can machines feel?
• How would we know?
Criteria for having emotions
• System has behavior that appears to arise from emotions
• System has fast ‘primary’ emotional responses to certain inputs
• System can cognitively generate emotions
• System can have emotional experience
• System’s emotions interact with other processes (e.g. memory)