Post on 12-Jan-2016
Outline
1. Physiological sensors and EEG
2. Classification of affective states
2.1 Background & experiment design
2.2 Features, Classification and Resumee
The sensorsElectroencephalogram (EEG)
Electroocculogram (EOG)
Electromyogram (EMG)
Electrocardiogram (ECG)
Galvanic skin response (GSR)
Blood volume pulse (BVP)
Respiration
Temperature
central nervoussystem
peripheral nervoussystem
Blood volume pulse
infrared photoelectric sensor
detects changes in tissue blood volume
60 – 80 pulses a minute
> heart rate and inter-beat-interval can be derived
faster when exercised or aroused
Galvanic skin response
impedance of the skin is measured
eccrine glands at hands and food soles
increases linear with arousal
Respiration
circumference of chest is measured via strain gauge
normal breath rate: 12 - 16 breath per minute (~500ml)
faster and more shallow with higher arousal
Electroencephalogramnormal signal
theta band
alpha band
beta band
gamma band
0 – 1000 Hz
4 – 8 Hz
13 – 30 Hz
8 – 12 Hz
30 – 100 Hz
Time- vs frequency domain
normal signal
theta band
alpha band
beta band
gamma band
0 – 1000 Hz
4 – 8 Hz
13 – 30 Hz
8 – 12 Hz
30 – 100 Hz
Event-related potentials (ERPs)
The raw signal with stimuli presentations
The averaging of the raw signal epochsleading to the ERP
Time-frequency analysis
evoked frequency analysis
induced frequency analysis
Tallon-Baudry & Bertrand, 1999
Recapitulation
EEG measures the electrical potentials from synchroneously active pyramidal cells in the cortex
high temporal resolution vs low spatial resolution
only weak signals from deeper structures & closed fields “blindness”
ERP analysis in time domain shows mainly low frequency components
analysis in frequency domain (evoked vs induced) also pics up high frequency components
Motivation
Affective Computing aimes at the enrichment of human-computer interaction
Physiological and neurophysiological signals give access to (non-observable) affective and cognitive states
(Sander et al. 2005)
Emotion Models
2 dimensional model of emotion
valence
arou
sal
posneg
low
high
boredom
joy
fear
relaxation
An emotion, or an affective state, is a reaction to an internal (e.g. thought)or external event (e.g. visual or auditory stimuli), and a behavioural disposition.
frustration
Regulation
Affective State
Appraisal
Stimulus presentation
+/-
+/-
Hippocampus (HC)
dorsal Anterior Cingulate Gyrus (dACG)
dorsal prefrontal cortex (PFC)
Amygdala
Insula
ventral Anterior Cingulate Gyrus (vACG)
ventral prefrontal cortex (PFC)
Inside
Outside
HC
Insula
Amygdala
dACG
vACG
PFC
Neurobiology of Emotional PerceptionPhillips et al. 2003
Overview Issues aBCI
Elicitation of emotions
Ground truth construction
Sensor modalities
Modality fusion
Feature selection and reduction
Classification
Elicitation of emotions
not easy to establish a natural context of emotion elicitation:
subject- vs event-elicited
lab setting vs. real world
expression vs feeling
open vs hidden recording
emotion vs other purpose (for subject)
Ground truth construction
normed stimulus sets (e.g. IAPS, IADS)
versus
self-assessment after each trial
Sensor modalitiesphysiological modalities:
slow response to stimulus (seconds)
long inter-trial intervals (seconds)
few (but long presented) stimuli per subject
• neurophysiological modality:– fast response to stimulus (miliseconds)– short inter-trial intervals (miliseconds)– many (but short presented) stimuli per subject
Experiment: Design
Resting Stimulus SAM rating
3 – 4 s 6 s not limited
1. emotion induction with visual affective stimuli
2. self-assessment of (induced) affective state after each stimulus with Self-Assessment Maneken (SAM)
Feature selection and reduction
Selection:manual selection (e.g. literature research)
automatical selection / “wrapper” method
Reduction:• mapping from high into low dimensional space
(e.g. PCA,ICA) > “filter” method
Feature extractionEEG: 6 features (from 6 frequency bands over 6 brain regions)
physiological Sensors: m
ean, variance, and
(minimum & maximum)
from GSR, blood pressure,
heart rate, respiration, and
temperature
Classification
• many possible classification approaches:decision trees, linear discriminant analysis, support vector machine, neural networks…
• optimal method depends on structure of data
• !keep training trials appart from test data, to avoid the contamination of the classifier.. !
Classification• physiological studies:
– 4 class: up to 95%– 8 class: 81% (!)– generalization of classifiers over subjects and time
• neurophysiological studies:– 3 class: up to 67%– 2 class: up to 79% – ? generalization of classifiers over subjects and
time ?
Recapitulation
• natural context of emotion elicitation
• characteristics of sensors are important
• many different approaches for data fusion, feature selection/reduction and classification – no optimal method per se
• still a long way toward an affect classification in an natural and multimodal real-world setting.. but we are getting there!