Classifying Different Emotional States by Means of EEG-Based Functional Connectivity Patterns
-
Upload
elvis-saravia -
Category
Technology
-
view
189 -
download
0
Transcript of Classifying Different Emotional States by Means of EEG-Based Functional Connectivity Patterns
![Page 1: Classifying Different Emotional States by Means of EEG-Based Functional Connectivity Patterns](https://reader031.fdocuments.us/reader031/viewer/2022030311/58ef46681a28ab41508b45c9/html5/thumbnails/1.jpg)
Classifying Different Emotional States by Means of EEG-Based Functional Connectivity Patterns
You-un Lee, Shulan HsiehPLOS ONE, 2014
![Page 2: Classifying Different Emotional States by Means of EEG-Based Functional Connectivity Patterns](https://reader031.fdocuments.us/reader031/viewer/2022030311/58ef46681a28ab41508b45c9/html5/thumbnails/2.jpg)
Emotion Classification
![Page 3: Classifying Different Emotional States by Means of EEG-Based Functional Connectivity Patterns](https://reader031.fdocuments.us/reader031/viewer/2022030311/58ef46681a28ab41508b45c9/html5/thumbnails/3.jpg)
What is emotion classification?
![Page 4: Classifying Different Emotional States by Means of EEG-Based Functional Connectivity Patterns](https://reader031.fdocuments.us/reader031/viewer/2022030311/58ef46681a28ab41508b45c9/html5/thumbnails/4.jpg)
Motivation
Business aspect: Assist in the decision-making process
Research opportunity: Help us to understand other behaviours such as intentions
![Page 5: Classifying Different Emotional States by Means of EEG-Based Functional Connectivity Patterns](https://reader031.fdocuments.us/reader031/viewer/2022030311/58ef46681a28ab41508b45c9/html5/thumbnails/5.jpg)
Emotion Classification - Textual information
I am very excited today :)
I am very excited today
I am very excited today... but also feeling very tired
![Page 6: Classifying Different Emotional States by Means of EEG-Based Functional Connectivity Patterns](https://reader031.fdocuments.us/reader031/viewer/2022030311/58ef46681a28ab41508b45c9/html5/thumbnails/6.jpg)
Multilingual Emotion Classification
Informality is a major issue
Imbalanced data
![Page 7: Classifying Different Emotional States by Means of EEG-Based Functional Connectivity Patterns](https://reader031.fdocuments.us/reader031/viewer/2022030311/58ef46681a28ab41508b45c9/html5/thumbnails/7.jpg)
Alternatives
Brainwaves(Universal)
(Sophisticated)
Audio(tones)
![Page 8: Classifying Different Emotional States by Means of EEG-Based Functional Connectivity Patterns](https://reader031.fdocuments.us/reader031/viewer/2022030311/58ef46681a28ab41508b45c9/html5/thumbnails/8.jpg)
Brainwaves as data
Pros:
Universal - reliable data source (language not an issue)
Sophisticated - opportunity to mine gems of knowledge (lots of data)
Cons:
Universal - difficult to interpret
Sophisticated - data preprocessing (e.g., noise filtering)
![Page 9: Classifying Different Emotional States by Means of EEG-Based Functional Connectivity Patterns](https://reader031.fdocuments.us/reader031/viewer/2022030311/58ef46681a28ab41508b45c9/html5/thumbnails/9.jpg)
First thing is first...
How to collect the neuroimaging data?
● Functional magnetic resonance imaging (fMRI)
● Positron emission tomography (PET)
● Electroencephalography (EEG)
● Magnetoencephalography (MEG)
![Page 10: Classifying Different Emotional States by Means of EEG-Based Functional Connectivity Patterns](https://reader031.fdocuments.us/reader031/viewer/2022030311/58ef46681a28ab41508b45c9/html5/thumbnails/10.jpg)
EEG(Electroencephalography)
![Page 11: Classifying Different Emotional States by Means of EEG-Based Functional Connectivity Patterns](https://reader031.fdocuments.us/reader031/viewer/2022030311/58ef46681a28ab41508b45c9/html5/thumbnails/11.jpg)
EEG - Fundamental Concepts
Monitors electrical activity in the brain through electrodes placed along the scalp
![Page 12: Classifying Different Emotional States by Means of EEG-Based Functional Connectivity Patterns](https://reader031.fdocuments.us/reader031/viewer/2022030311/58ef46681a28ab41508b45c9/html5/thumbnails/12.jpg)
EEG - Fundamental Concepts
EEG Bands - are defined by the frequency of brainwaves.
5 different types of brainwaves:
❖ Gamma
❖ Beta
❖ Alpha
❖ Theta
❖ Delta
Source: http://psychedelic-information-theory.com/eeg-bands
![Page 13: Classifying Different Emotional States by Means of EEG-Based Functional Connectivity Patterns](https://reader031.fdocuments.us/reader031/viewer/2022030311/58ef46681a28ab41508b45c9/html5/thumbnails/13.jpg)
EEG - Fundamental Concepts
Each band can be associated with different emotional and mental states
Examples:❖ Rapid eye movement (REM) sleep (slower frequencies involved)❖ ADHD (too much theta, not enough alpha and beta)
![Page 14: Classifying Different Emotional States by Means of EEG-Based Functional Connectivity Patterns](https://reader031.fdocuments.us/reader031/viewer/2022030311/58ef46681a28ab41508b45c9/html5/thumbnails/14.jpg)
EEG - Fundamental Concepts
Artifacts - electrical activities usually not originating from the brain.
![Page 15: Classifying Different Emotional States by Means of EEG-Based Functional Connectivity Patterns](https://reader031.fdocuments.us/reader031/viewer/2022030311/58ef46681a28ab41508b45c9/html5/thumbnails/15.jpg)
EEG - Fundamental Concepts
Brain Maps - illustrates the electrical powerat each frequency
Green region - normal electrical activity
Red region - abnormal electrical activity
![Page 16: Classifying Different Emotional States by Means of EEG-Based Functional Connectivity Patterns](https://reader031.fdocuments.us/reader031/viewer/2022030311/58ef46681a28ab41508b45c9/html5/thumbnails/16.jpg)
![Page 17: Classifying Different Emotional States by Means of EEG-Based Functional Connectivity Patterns](https://reader031.fdocuments.us/reader031/viewer/2022030311/58ef46681a28ab41508b45c9/html5/thumbnails/17.jpg)
ObjectiveCapture the relationship between brain activity and emotional states.
![Page 18: Classifying Different Emotional States by Means of EEG-Based Functional Connectivity Patterns](https://reader031.fdocuments.us/reader031/viewer/2022030311/58ef46681a28ab41508b45c9/html5/thumbnails/18.jpg)
Important considerations
Single-electrode level vs. Functional connectivity
Emotion is a complex behaviour Electrical activity is usually dispersed
![Page 19: Classifying Different Emotional States by Means of EEG-Based Functional Connectivity Patterns](https://reader031.fdocuments.us/reader031/viewer/2022030311/58ef46681a28ab41508b45c9/html5/thumbnails/19.jpg)
How to estimate EEG functional connectivity?
Using three popular connectivity indices:
Correlation - (independent of amplitude)
Coherence - (amplitude and phase important)
Phase synchronization - (phase important)
Combination is important since each connectivity index is sensitive to different characteristics of EEG signals (phase, polarity, and amplitude).
http://predictablynoisy.com/correlation-and-coherence-whats-the-difference/
![Page 20: Classifying Different Emotional States by Means of EEG-Based Functional Connectivity Patterns](https://reader031.fdocuments.us/reader031/viewer/2022030311/58ef46681a28ab41508b45c9/html5/thumbnails/20.jpg)
Basic intuition
A particular connectivity index might be better at recognizing a particular emotion
No such thing as a perfect measure
![Page 21: Classifying Different Emotional States by Means of EEG-Based Functional Connectivity Patterns](https://reader031.fdocuments.us/reader031/viewer/2022030311/58ef46681a28ab41508b45c9/html5/thumbnails/21.jpg)
Materialand
Method
![Page 22: Classifying Different Emotional States by Means of EEG-Based Functional Connectivity Patterns](https://reader031.fdocuments.us/reader031/viewer/2022030311/58ef46681a28ab41508b45c9/html5/thumbnails/22.jpg)
Participants
40 healthy studentsNo psychiatric illness
24 hour away from caffeine or tobaccoNT $1000
For 6 hours
![Page 23: Classifying Different Emotional States by Means of EEG-Based Functional Connectivity Patterns](https://reader031.fdocuments.us/reader031/viewer/2022030311/58ef46681a28ab41508b45c9/html5/thumbnails/23.jpg)
Clips use for Emotional stimuli
Standard Chinese Emotional Film Clips Database (not for free; need to pay)
![Page 24: Classifying Different Emotional States by Means of EEG-Based Functional Connectivity Patterns](https://reader031.fdocuments.us/reader031/viewer/2022030311/58ef46681a28ab41508b45c9/html5/thumbnails/24.jpg)
Clips consideration
Overpowering of a particular emotion was counterbalanced using Latin Square Design
Sad Joy Anticipation
Anticipation Sad Joy
Joy Anticipation Sad
![Page 25: Classifying Different Emotional States by Means of EEG-Based Functional Connectivity Patterns](https://reader031.fdocuments.us/reader031/viewer/2022030311/58ef46681a28ab41508b45c9/html5/thumbnails/25.jpg)
EEG Measurement
Electrooculography (EOG) was measured to capture ocular artifacts. The eye component was later removed.
EOG and EEG amplified (500 Hz per channel)
NeuroScan 4.3.1
![Page 26: Classifying Different Emotional States by Means of EEG-Based Functional Connectivity Patterns](https://reader031.fdocuments.us/reader031/viewer/2022030311/58ef46681a28ab41508b45c9/html5/thumbnails/26.jpg)
Feature selection
Functional connectivity in four bands for all pairs of 19 electrodes: Theta band (4-7 Hz)Alpha band (8-12 Hz)Beta band (13-30 Hz)Gamma band (31-50 Hz)
Transformation of raw EEG signals: Fast Fourier Transformation (FFT)
The connectivity indices for all pairs of electrodes at each frequency band were selected
as features. Features where ANOVAs results was significant (p<= 0.05) were kept.
Capture relevant interactions within the brain
![Page 27: Classifying Different Emotional States by Means of EEG-Based Functional Connectivity Patterns](https://reader031.fdocuments.us/reader031/viewer/2022030311/58ef46681a28ab41508b45c9/html5/thumbnails/27.jpg)
Pattern Classification
Quadratic Discriminant Analysis (QDA)
❖ Reason: performs extremely faster evaluations compared to other algorithms
Two 2-fold cross validation
❖ Reason: each data point used for training and testing on each fold
Accuracy as an evaluation metric
❖ Reason: Imbalanced dataset problem.
![Page 28: Classifying Different Emotional States by Means of EEG-Based Functional Connectivity Patterns](https://reader031.fdocuments.us/reader031/viewer/2022030311/58ef46681a28ab41508b45c9/html5/thumbnails/28.jpg)
Experiments
![Page 29: Classifying Different Emotional States by Means of EEG-Based Functional Connectivity Patterns](https://reader031.fdocuments.us/reader031/viewer/2022030311/58ef46681a28ab41508b45c9/html5/thumbnails/29.jpg)
Experimental procedure
1. 60-s go/nogo task to keep participants in a neutral state.
2. Two 90-s baseline resting EEGs (eyes open, then closed)
3. The film was then shown to participant
4. Spacebar when emotion changes or is triggered
5. 60-s resting period
6. SAM self-assessment
16s (8192 data-points) signal
![Page 30: Classifying Different Emotional States by Means of EEG-Based Functional Connectivity Patterns](https://reader031.fdocuments.us/reader031/viewer/2022030311/58ef46681a28ab41508b45c9/html5/thumbnails/30.jpg)
Experimental setup
Scale in terms of valence:
Negative (surprising / amusing clips)
Positive - (disgust / fear clips)
Neutral - (no emotion clips)
Data cleaning: remove data of users that did not felt the correct emotion when viewing
the clips (29 out of 40 got it right!)
1 2 3 4 5 6 7 8 n
Valence scores
Negative Neutral Positive
Better dataset or more participants?
![Page 31: Classifying Different Emotional States by Means of EEG-Based Functional Connectivity Patterns](https://reader031.fdocuments.us/reader031/viewer/2022030311/58ef46681a28ab41508b45c9/html5/thumbnails/31.jpg)
Evaluation metric
“Balanced accuracy” across 50 trials
½(TP/P + TN/N) Where P = TP+FN, N = TN+FP
Confusion Matrix
Actual Prediction
Malignant Benign
Benign Benign
Benign Benign
Benign Benign
Malignant Benign
Malignant Benign
Benign Benign
Benign Benign
Benign Benign
Problem with imbalanced data
![Page 32: Classifying Different Emotional States by Means of EEG-Based Functional Connectivity Patterns](https://reader031.fdocuments.us/reader031/viewer/2022030311/58ef46681a28ab41508b45c9/html5/thumbnails/32.jpg)
Connectivity Indices - Correlation
Significant only in:
Theta
Neg-N → T,O
N-P → T,P,O
Alpha
Neg-N → F7-P7
Neg-P → P,O
N-P → RT
![Page 33: Classifying Different Emotional States by Means of EEG-Based Functional Connectivity Patterns](https://reader031.fdocuments.us/reader031/viewer/2022030311/58ef46681a28ab41508b45c9/html5/thumbnails/33.jpg)
Result (Correlation)
![Page 34: Classifying Different Emotional States by Means of EEG-Based Functional Connectivity Patterns](https://reader031.fdocuments.us/reader031/viewer/2022030311/58ef46681a28ab41508b45c9/html5/thumbnails/34.jpg)
Connectivity Indices - Coherence
Significant results in Theta, Alpha, Beta
Any other patterns?
![Page 35: Classifying Different Emotional States by Means of EEG-Based Functional Connectivity Patterns](https://reader031.fdocuments.us/reader031/viewer/2022030311/58ef46681a28ab41508b45c9/html5/thumbnails/35.jpg)
Result (Coherence)
![Page 36: Classifying Different Emotional States by Means of EEG-Based Functional Connectivity Patterns](https://reader031.fdocuments.us/reader031/viewer/2022030311/58ef46681a28ab41508b45c9/html5/thumbnails/36.jpg)
Connectivity Indices - Phase synchronization (PSI)
Significant results in all bands
Any other patterns?
![Page 37: Classifying Different Emotional States by Means of EEG-Based Functional Connectivity Patterns](https://reader031.fdocuments.us/reader031/viewer/2022030311/58ef46681a28ab41508b45c9/html5/thumbnails/37.jpg)
Result (PSI)
![Page 38: Classifying Different Emotional States by Means of EEG-Based Functional Connectivity Patterns](https://reader031.fdocuments.us/reader031/viewer/2022030311/58ef46681a28ab41508b45c9/html5/thumbnails/38.jpg)
Result - Multiple bands
All frequency bands combined as features
![Page 39: Classifying Different Emotional States by Means of EEG-Based Functional Connectivity Patterns](https://reader031.fdocuments.us/reader031/viewer/2022030311/58ef46681a28ab41508b45c9/html5/thumbnails/39.jpg)
Conclusion
● Did other stimuli affect the results? SAM helps to remove this concern.
● Better results with feature selection
● All bands can help towards emotion analysis and not just one in particular.
● PSI performs better in most cases
● Gender was not an underlying factor in the study
![Page 40: Classifying Different Emotional States by Means of EEG-Based Functional Connectivity Patterns](https://reader031.fdocuments.us/reader031/viewer/2022030311/58ef46681a28ab41508b45c9/html5/thumbnails/40.jpg)
Research Opportunities
Utilize other emotion-eliciting stimuli such as music and picture viewing.
Analysis of other emotions (e.g., anticipation)
Deep Learning algorithms for automatic feature learning
![Page 41: Classifying Different Emotional States by Means of EEG-Based Functional Connectivity Patterns](https://reader031.fdocuments.us/reader031/viewer/2022030311/58ef46681a28ab41508b45c9/html5/thumbnails/41.jpg)
References
http://neurosky.com/2015/04/reading-your-brainwaves-understanding-the-basics-of-eeg
/
https://addyssey.wordpress.com/2013/09/27/qeeg-as-a-diagnostic-tool-in-the-assessme
nt-of-addadhd/
http://mentalhealthdaily.com/2014/04/15/5-types-of-brain-waves-frequencies-gamma-b
eta-alpha-theta-delta/