CogniMeter: EEG-based Emotion, Mental Workload and Stress Visual Monitoring

Xiyuan Hou, Yisi Liu, Olga Sourina and Wolfgang Mueller-Wittig Fraunhofer IDM@NTU

Nanyang Technological University Singapore

houxy, liuys, eosourina, askwmwittig@ntu.edu.sg

Abstract—Real-time EEG (Electroencephalogram)-based user’s emotion, mental workload and stress monitoring is a new direction in research and development of human-machine interfaces. It has attracted recently more attention from the research community and industry as wireless portable EEG devices became easily available on the market. EEG-based technology has been applied in anesthesiology, psychology, serious games or even in marketing. In this work, we describe available real-time algorithms of emotion recognition, mental workload, and stress recognition from EEG and propose a novel interface CogniMeter for the user’s mental state visual monitoring. The system can be used in real time to assess human current emotions, levels of mental workload and stress. Currently, it is applied to monitor the user’s emotional state, mental workload and stress in simulation scenarios or used as a tool to assess the subject’s mental state in human factor study experiments.

Keywords—visual interface; EEG; stress; mental workload; emotions

I. INTRODUCTION As more and more wearable devices (Apple Watch,

Fitbit, and Jowbone) are available in commercial market, it becomes popular to use wearable devices and mobile phones to monitor our daily physiological states based on number of walking steps, heart rate, calories consumption, and sleep pattern. All signals of these wearable devices come from the muscle activities or movement of the user. Thus, these devices have some limitations in monitoring people’s mental states. A brain computer interface is more suitable for such analysis since Electroencephalogram (EEG) signals are directly recorded from the brain. A number of EEG-based methods and the corresponding applications are designed and implemented in order to recognize the user’s mental states [1-3]. However, most of the current works neither process EEG data in real time nor provide intuitive visualization tools for experiment analysis and decision making.

The EEG-based visualization of mental states has a potential to help researchers in their experiments. For offline data analyses, it can enable researchers to evaluate and improve human factor models based on the experiment results. For real-time data analyses, it can enable researchers to directly monitor the relationship between task variables and human mental states. In this paper, we propose a novel EEG-based CogniMeter system that integrates recognition algorithms to monitor emotion, mental workload and stress in real-time simultaneously.

This paper is structured as follows: Section II introduces related works such as EEG visualizations tools and EEG-

based emotion, mental workload and stress recognition algorithms. Section III introduces the methodology used in the implemented algorithms and structure of the proposed visualization system. Section IV presents the details of the visualization tools to monitor the emotions and the meters to monitor the mental workload and stress from EEG. Section V gives the conclusion.

II. RELATED WORK

A. Visualization Tools There is a number of tools for analysis and visualization

of EEG data such as EEGLab [4], Brainstorm [5], and ELAN [6]. The systems provide interactive graphic user interfaces which enable the users to interactively process high-density EEG data. For intuitive observation of activities in the brain, the amplitudes of EEG signals can be mapped to a 2-D or 3-D model of the scalp according to the EEG channel positions. These visualization tools are able to process EEG data epochs using spectral analysis or independent component analysis [4]. In [7-8], a blobby model is implemented for the EEG signals visualization on a 3D head model.

Because EEG signals are non-stationary and high-dimensional, machine learning methods can be used for real-time analysis of brain activities with specific tasks [9]. In this way, the visualization tools can provide further interpretation of mental states from EEG data instead of showing only the amplitude of the signals. In [10], an EEG-based workload gauge is implemented and the workload level is monitored when the subjects are doing cognitive and operational Air Traffic Controller’s (ATC) tasks. It is critical to reliably measure the mental states and performance of the controllers/pilots when the mode of automation is changed or new tasks appears. During the simulation of ATC task, the workload gauge is updated with the recognized level of workload (three workload levels). Other gauges depict dynamic changes in EEG signals and show spectrum powers in the form of brain maps simultaneously.

In [9], an EEG-based mental text entry system Hex-o-Spell is proposed. By imagining left/right hand movement, the user can generate different brain patterns. These patterns are used to control visual text input. The visualization tools include a text dial, an arrow, and a bar. The orientation of the arrow and the length of the visual bar are controlled by the states of the human brain.

B. Emotion Emotions can be defined from a dimensional perspective

where arousal, valence, and dominance dimensions are considered [11]. In the dimensional model, the arousal dimension ranges from not aroused to excite, the valence dimension ranges from negative to positive, and the dominance dimension ranges from being controlled to being in control. The dimensional model is preferable for emotion recognition because it allows locate discrete emotions in the dimensional space. Even the feeling that cannot be labeled with a certain word can be located in the dimensional model [12, 13]. Emotions can be induced by different kinds of stimuli such as audio, visual and combined ones [14]. Different algorithms have been proposed for EEG-based emotion recognition. [3] extracted power features from EEG data and used Support Vector Machine (SVM) as a classifier. 82.37% accuracy for distinguishing of four emotions was achieved by using 32 channels. Short Time Fourier Transform (STFT) as a feature extraction method and SVM as a classifier were applied in [15] and a mean accuracy of 62.07% was obtained with 16 channels. However, all these researches are off-line emotion recognition. Additionally, as the EEG signal is nonlinear and chaotic, traditional features may not be able to capture the nonlinear property of EEG. The fractal dimension (FD) can reflect the changes of the EEG signal during different mental tasks in real-time [16-18]. In [19], FD is used in real-time EEG-based emotion recognition.

C. Mental Workload Mental workload is described as a noticeable relationship

between the human cognitive capacity and an effort required to process a particular function [20]. There are mainly three broad categories for workload definition including physiological workload, subjective workload, and cognitive workload [21]. In this research, we are interested in cognitive workload which indicates the capability of a person to complete a task with some amount of the mental effort. The performance in the task ascertains the cognitive workload [22]. In [9], mental workload is evaluated in online EEG monitoring during the security surveillance task. Comparing the mental workload index with the error rate for the subjects, the correlation coefficient is approximately 0.7, which indicates that when the workload increases people tend to make more errors. The significant correlation between workload and theta band power has been proved in [23-25]. In experiment [23], it is shown that the EEG theta band power increases when the workload is induced by mental arithmetic task. In [24], the driver’s mental workload is significantly correlates with both theta and alpha power. In different driving tasks, the frontal theta activity shows significant increases when working memory load increases. In another experiment studying the workload and fatigue in aircraft pilots [25], increased EEG theta band power and decreased alpha band power are observed in high mental workload comparing with low mental workload. Additionally, in [25] it is shown that when the pilots have high mental workload and mental fatigue, their EEG theta band power as well as the delta and alpha bands power increases.

D. Stress Stress is a human state which can be caused by a number

of reasons, including high mental workload, emotions, or environmental influences [26]. The stress can be measured and assessed from physiological variables including EEG [1, 2, 27-29], blood pressure [30], heart rate variability [28, 31], skin conductance level [32], and electromyography [33]. EEG can be used to detect human stress levels. In work [30], the experiment shows that the stress is positively correlated with beta EEG power at the anterior temporal lobe. In [2], higher order spectra features are used for stress recognition. The SVM with RBF kernel is chosen as a classifier and the accuracy calculated with 5-fold cross validation for recognition of two stress states is 79.2%. In [27], the features such as Gaussian mixtures of EEG spectrogram, fractal dimension and magnitude square coherence estimation are used in stress recognition algorithm. The classification of two levels of mental stress is done by k-Nearest Neighbor (k-NN) and SVM classifiers, and the best accuracy is 90%. However, neither [2] nor [27] uses standard stressor to induce stress in the experiments. In [1], a Stroop colour-word test is used to induce stress. The discrete cosine transform is applied to reduce the data size and extract features from the frequency domain. Classification is implemented with artificial neural network, linear discriminant analysis and k-NN. The best classification result for two stress states is 72% with k-NN. In [29], the band power of theta, alpha, and beta are used as features in logistic regression and are fed into the k-NN classifier. The results show a median accuracy of 73.96% for the recognition of relaxed and stressed states.

III. METHODOLOGY In our real-time EEG-based mental states monitoring

system, the emotion is recognized using machine learning method, the workload is identified by theta power, and stress is inferred from a combination of the recognized emotion and workload.

A. EEG Device The Emotiv Headset [34] is used to capture the users’

EEG signals wirelessly with the USB receiver. It has 14 channels locating at AF3, F7, F3, FC5, T7, P7, O1, O2, P8, T8, FC6, F4, F8, and AF4 as shown in Fig. 1.

B. Feature Extraction The power spectrum feature is used in the real-time

workload level recognition while the FD and statistical features are used in real-time EEG-based emotional recognition [19]. All features are extracted using a 4 seconds sliding window with 3 seconds overlapping.

1) Power Feature

Power spectrum is one of the most commonly used features for EEG-based mental states recognition. The power spectrum over a time interval is obtained by the fast fourier transform. The EEG power spectrum is subdivided into bandwidths known as delta (0.5-4 Hz), theta (4-8 Hz), alpha (8-12 Hz), beta (12-30 Hz), and gamma (above 30 Hz). As

introduced in Section II C, the theta band power has been proved to be positively correlated with mental workload in [23-25]. In our study, the average of power spectrum density of theta band is used in the mental workload recognition. The composed feature vector is theta thetaFV P= .

Fig. 1. Location of 14 electrodes of Emotiv EEG device.

2) Fractal Dimension Feature

FD measures the complexity and irregularity of time series [35]. It can be used as an index for characterizing the complexities of EEG signals. For a regular signal, the fractal dimension value is low. If the signal becomes irregular, the fractal dimension value increases accordingly. Wang et al. [36] proposed to use Higuchi fractal dimension to recognize different arithmetic mental tasks from EEG. It is also used in EEG-based serious games to identify attention level [37]. In this paper, the Higuchi algorithm [35] is used to calculate FD feature from EEG data in emotion recognition algorithm proposed in [19]. The idea of Higuchi algorithm is as follows.

Let ( ) ( ) ( )1 , 2 , ,X X X N… be a finite set of time series samples. Then, the newly constructed time series is

( ) ( ): , , ,m

t

N mX X m X m t X m t

t−

+ + ⋅⎛ ⎞⎡ ⎤⎜ ⎟⎢ ⎥⎣ ⎦⎝ ⎠

… . (1)

where 1,2,...,m t= is the initial time and t is the interval time [35].

t sets of ( )mL t are calculated by

( ) ( ) ( )( )1

11 /m

N mt

i

L tNX m it X m i t t

N m tt

−⎡ ⎤⎢ ⎥⎣ ⎦

=

=

⎧ ⎫⎛ ⎞⎪ ⎪−⎪ ⎪⎜ ⎟+ − + − ⋅⎨ ⎬⎜ ⎟ −⎡ ⎤⎜ ⎟⎪ ⎪⋅⎝ ⎠ ⎢ ⎥⎪ ⎪⎣ ⎦⎩ ⎭

∑ . (2)

( )L t denotes the average value of ( )mL t , and one relationship exists

( ) dimHL t t −∝ . (3)

Then the Fractal Dimension dimH can be obtained by logarithmic plotting between different t (ranging from 1 to maxt ) and its associated ( )L t [35].

ln ( )dim

lnH

L tt

=−

. (4)

Thus, the FD feature vector composed is

(Higuchi) [dim ]HFDFV = . (5)

3) Statistical Feature

Statistical features are simple and widely used in EEG-based brain state recognition. For example, they are used in EEG-based emotion recognition algorithms [38, 39]. Six statistical features including mean

Xμ , standard

deviationX

σ , mean of absolute values of the first differences

Xδ , mean of absolute values of the first differences of

normalized signals X

δ , mean of absolute values of the second differences

Xγ , and mean of the second differences of the normalized signals

Xγ are extracted from EEG for

emotion recognition. The composed feature vector of statistical features is [ , , , , , ]

X X X X X XstatisticalFV μ σ δ δ γ γ= .

C. Data Processing 1) Emotion Recognition

As mentioned above, the emotion recognition algorithm proposed in work [19] is used in this paper. By combining the FD and statistical as the feature and using SVM as the classifier, it can recognize up to 8 emotions with the accuracy of 52.67%. If only positive and negative emotions are needed to be recognized, the best accuracy can be 91.07% [39]. As the proposed emotion recognition algorithm is a subject-dependent one, before real-time emotion recognition, a classifier model has to be trained. Different types of stimuli can be used to evoke certain emotions such as sound clips from International Affective Digital Sounds database [40] or pictures from International Affective Picture System database [41]. A pipeline of data processing in the offline emotion training session is shown in Fig. 2. The raw EEG data recorded from four electrodes (AF3, F4, FC5 and F7) are labeled with different emotions by the self-assessment questionnaires. Then, the EEG data stream is filtered by a 2-42 Hz bandpass filter. The statistical and FD features are extracted from the filtered EEG data using a sliding window with the size of 512 (4 seconds) with 384 samples (3 seconds) overlapping. The feature vector FV for emotion classification is defined as follows:

FV = [FV1, FV2, FV3, FV4] ’. (6)

where 1 denotes FC5 channel, 2 denotes F4 channel, 3 denotes F7 channel, 4 denotes AF3 channel, and FVi is the feature vector per channel. Here, the FVi is composed by the combinations of FD and statistical features and denoted as

FVcombination_1. Normalization is applied to the FD and statistical features across the four channels.

_1 [ , , , , , , ]conbination X X X X X X HFV dimμ σ δ δ γ γ= ’. (7)

Then, the extracted features are fed into the SVM classifier to train a classifier model for a particular subject. The polynomial kernel is used with penalty parameter 10C = , degree 3d = , gamma 1g = , coefficient 1r = . The trained model is saved and used in the real-time emotion recognition phase.

Fig. 2. The pipeline of offline classifier training.

2) Mental Workload Recognition

As reviewed in Section II C, in work [23-25], mental workload is proved to be significantly correlated with theta and alpha band power. Especially, theta band power increases in high memory load and high mental workload. Thus, we use theta band power calculated from channel P8 for real-time mental workload monitoring. The theta band power is calculated from a sliding window with the size of 512 (4 seconds) with 384 samples (3 seconds) overlapping. For each subject, we set two thresholds to discriminate three levels of workload. The thetaFV is compared with the pre-defined thresholds and then the current level of mental workload is recognized.

3) Stress Recognition

In order to recognize stress level, we carried out an experiment to study the correlation between workload, emotion and stress. 7 subjects were recruited and their age ranges from 21 to 28. All subjects reported that they had no history of mental diseases. The experiment consisted of 5 segments and the Stroop Color Word Test [42] is adapted to elicit different levels of stress. In segments 1 and 2, the subjects just relaxed and looked at the screen. In segment 3, the Stroop test was given, and the subjects were asked to identify the ink color of the words on the computer screen, given the words that match with its font color. In segment 4 and 5, the font color of the word and the word differs. The subjects were told to identify the ink color of the word. However, time limitation was added in segment 5 which is supposed to elicit a higher level of stress than in segment 4. Upon completion of each segment, a questionnaire was given to the subjects to rate arousal, valence, dominance, workload and stress. Each rating has a 1-9 scale for the subjects to select. For arousal ratings, 1 indicates the calmest state and 9

indicates the most excited state. For valence ratings, 1 indicates the most negative state and 9 indicates the most positive state. For dominance ratings, 1 indicates the feeling of fully being controlled and 9 indicates the feeling of full control. For mental workload ratings, 1 indicates the lowest workload and 9 indicates the highest workload. For stress level ratings, 1 indicates the lowest stress level (relaxed) and 9 indicates the highest stress level. The correlation coefficient is calculated among all ratings. As it is shown in Table I, the most significant correlation is found between stress and mental workload (positive correlation), and between stress and valence (negative correlation), which means when subjects have higher mental workload and more negative emotion, they tend to experience more stress. Thus in this work, we use the combinations of valence and mental workload to recognize stress.

TABLE I. CORRELATION COEFFICIENT BETWEEN STRESS AND AROUSAL, VALENCE, DOMINANCE, AND MENTAL WORKLOAD.

Stress

r p Arousal 0.2735 0.1119 Valence -0.5666 0.0004

Dominance -0.3835 0.0230 Mental Workload 0.9335 <0.0001

TABLE II. MAPPING BETWEEN COMBINATIONS OF EMOTION AND WORKLOAD AND STRESS LEVELS.

Combination Stress Level Emotion Workload

Positive Low Relaxed

Negative Low Low

Positive Medium Lower-Medium

Negative Medium Medium

Positive High Higher-Medium

Negative High High

Based on this experiment result, we designed a novel stress recognition method where stress is identified from a combination of the recognized emotions and workload. The mental stress is classified into six levels (relaxed, low, lower-medium, medium, higher-medium, and high stress). Each level of stress corresponds to one of the combinations of the recognized valence and workload. Table II shows the mapping of the emotional state (positive and negative emotions) and workload level (low, medium, and high workload) to stress levels. As listed in the table, stress level is “Relaxed” when positive emotion and low workload is recognized. The low stress level corresponds to the combination of negative emotion and low workload. Low-medium stress level corresponds to positive emotion and medium workload. The medium stress level corresponds to negative emotion and medium workload. The high-medium stress level corresponds to positive emotion and high workload. The high stress level is experienced when negative emotion and high workload appear simultaneously.

D. Real-time Mental States Visual Monitoring Based on the above-mentioned emotion, workload and

stress recognition algorithms, we proposed a real-time mental states visual monitoring system named CogniMeter. The overall pipeline is illustrated in Fig. 3. In the proposed EEG-based real-time mental states monitoring system, first, the raw EEG data are obtained from the EEG device. Then, the EEG data are passed through the bandpass filter. After that, the features are extracted from the filtered EEG data for emotion and workload recognition. In the thread of emotion recognition, the statistical feature and FD feature are used. These extracted features are fed into the pre-trained SVM classifier model and then emotions are recognized in real-time. In the thread of workload recognition, the theta power is extracted from EEG. Then, the current workload level is recognized by comparing the calculated theta band power values and the pre-defined thresholds. Finally, the current stress level is determined by combining the current state of emotion (positive and negative) and the current level of workload (low, medium, and high). CogniMeter is implemented to visualize the recognized emotion, workload, and stress levels.

Fig. 3. The overall pipeline of real-time EEG-based emotion, workload

and stress monitoring system.

IV. VISUALIZATION A real-time EEG-based emotion, workload and stress

visualization system called CogniMeter is implemented with Visual Studio 2010 in C++, as shown in Fig. 4.

Fig. 4. Screenshot of the CogniMeters of real-time mental stats

monitoring system. Top-left meter shows the current level of workload. Top-right shows recognized emotion. Botom-left meter shows current level

of stress. Bottom-right shows real-time EEG signals from 14 channels.

A. Emotion Recogniton Visualization As the emotion recognition algorithm is subject

dependent, it requires a training/calibration to be done before real-time emotion recognition. The screenshot of the calibration module interface is shown in Fig. 5. When the "Start Recording" button is clicked, the subject listens to sound clips from IADS database [40] to induce different emotions. At the same time, the EEG data are recoded. After each sound clip, the subject needs to fill a prompted questionnaire to evaluate and describe his/her current emotion. After all recordings for the training are completed, the "Train SVM" is clicked and a SVM model is trained based on the recorded EEG data and evaluated emotional labels. The trained emotion model is saved and can be loaded again for future use. When "Go to Emotion Recognition" is clicked, the software starts real-time emotion recognition based on the trained SVM model.

Fig. 5. Screenshot of the emotion training interface.

Fig. 6. Real-time emotion monitering interface with Haptek avatar.

Fig. 7. Haptek avatar with facial expressions for different emotions. (a) Fear, (b) Frustrated, (c)Sad, (d) Happy, e) Pleasant, (f) Satisfied [43].

The visualization of real-time emotion recognition is shown in Fig. 6. The word label of the recognized current emotion is shown on the left of the interface and the corresponding arousal, valance and dominance levels are presented as the horizontal bars. Besides, the recognized emotion is illustrated by the facial expressions of a 3D avatar using Haptek system [44]. An example of six facial expressions such as fear, frustrated, sad, happy, pleasant and satisfied to represent the emotions recognized from EEG is

given in Fig. 7. More emotions can be visualized by changing the parameters in the Haptek system.

B. Mental Workload and Stress Monitoring The workload and stress levels are visualized by the

CogniMeter system as shown in Fig. 8. The meter shows the current level of workload or stress with a pointer changing its position and color (green, yellow, red). The green pointer means the low level; the yellow pointer means the medium level, and the red pointer means high level. The word describing the current level of workload or stress is updated at the bottom of the meter and corresponds to the changes of the pointer. When the real-time EEG-based mental state monitoring is completed, a report is generated to summarize the changes of emotion state, workload and stress during the monitoring time.

Fig. 8. Screenshots of CogniMeter for workload and stress levels. (a) The workload meter shows that the current worklaod level is high. (b) The stress meter shows that the current stress level is lower-medium.

V. CONCLUSION In this paper, a novel CogniMerter system is proposed

and implemented that allows monitor the user’s real-time emotion, mental workload and stress using EEG device mounted on the user head. Visual meters and bars provide the possibility to directly observe the relationship between the user’s emotions, mental workload, stress, and his/her task performance. It can help researchers to propose new hypotheses and refine the experimental procedures. We extract the fractal dimension and statistical features and use the SVM classifier for real-time emotion recognition. The workload is measured from theta band power of the EEG using predefined thresholds. The stress is identified based on the combinations of the recognized positive or negative emotion and the levels of mental workload. To monitor emotions, the interface presents the recognized emotions using facial expressions on the 3-D avatar implemented in Haptek system [44] and using the implemented arousal, valence and dominance bars. To monitor workload and stress, the dynamic meters are proposed and implemented. In the next step, the mental workload algorithm will be tested on the benchmark database. We also plan to integrate in the CogniMeter a stress recognition algorithm described in [45] and compare its performance with the presented one.

ACKNOWLEDGMENT The work is supported by Fraunhofer IDM@NTU, which

is funded by the National Research Foundation (NRF) and managed through the multi-agency Interactive & Digital Media Programme Office (IDMPO).

REFERENCE [1] C.-K. A. Lim and W. C. Chia, "Analysis of Single-Electrode EEG

Rhythms Using MATLAB to Elicit Correlation with Cognitive Stress," International Journal ofComputer Theory and Engineering, vol. 7, pp. 149-155, 2015.

[2] S. A. Hosseini, M. A. Khalilzadeh, M. B. Naghibi-Sistani, and V. Niazmand, "Higher Order Spectra Analysis of EEG Signals in Emotional Stress States," in Information Technology and Computer Science (ITCS), 2010 Second International Conference on, 2010, pp. 60-63.

[3] L. Yuan-Pin, W. Chi-Hong, W. Tien-Lin, J. Shyh-Kang, and C. Jyh-Horng, "EEG-based emotion recognition in music listening: A comparison of schemes for multiclass support vector machine," in . IEEE International Conference on Acoustics, Speech and Signal Processing, pp. 489-492, 2009.

[4] A. Delorme and S. Makeig, "EEGLAB: an open source toolbox for analysis of single-trial EEG dynamics including independent component analysis," Journal of Neuroscience Methods, vol. 134, pp. 9-21, 2004.

[5] F. Tadel, S. Baillet, J. C. Mosher, D. Pantazis, and R. M. Leahy, "Brainstorm: A User-Friendly Application for MEG/EEG Analysis," Computational Intelligence and Neuroscience, vol. 2011, p. 13, 2011.

[6] P.-E. Aguera, K. Jerbi, A. Caclin, and O. Bertrand, "ELAN: A Software Package for Analysis and Visualization of MEG, EEG, and LFP Signals," Computational Intelligence and Neuroscience, vol. 2011, 2011.

[7] V. Kulish, A. Sourin, O. Sourina, "Human electroencephalograms seen as fractal time series: mathematical analysis and visualization," Computers in Biology and Medicine, Elsevier-Pergamon, 36(3):291-302, 2006.

[8] V. Kulish, A. Sourin, O. Sourina, "Analysis and visualization of human electroencephalograms seen as fractal time series," Journal of Mechanics in Medicine & Biology, World Scientific, 26(2):175-188, 2006.

[9] K.-R. Muller, M. Tangermann, G. Dornhege, M. Krauledat, G. Curio, and B. Blankertz, "Machine learning for real-time single-trial EEG-analysis: From brain–computer interfacing to mental state monitoring," Journal of Neuroscience Methods, vol. 167, pp. 82-90, 2008.

[10] M. Weiland, D. Roberts, M. Fine, and M. Caywood, "Real Time Research Methods Monitoring Air Traffic Controller Workload During Simulation Studies Using Electroencephalography (EEG)." In Proceedings of the Human Factors and Ergonomics Society Annual Meeting, vol. 57, no. 1, pp. 1615-1619. SAGE Publications, 2013.

[11] A. Mehrabian, "Pleasure-Arousal-Dominance: A general framework for describing and measuring individual differences in temperament," Current Psychology, vol. 14, pp. 261- 292, 1996.

[12] G. Chanel, "Emotion assessment for affective-computing based on brain and peripheral signals," Ph.D. thesis, University of Geneva, 2009.

[13] I. B. Mauss and M. D. Robinson, "Measures of emotion: A review," Cognition & emotion, vol. 23, pp. 209-237, 2009.

[14] K. Takahashi, "Remarks on emotion recognition from multi-modal bio-potential signals," presented at the IEEE International Conference on Industrial Technology (IEEE ICIT '04), vol. 3, pp. 1138-1143, 2004.

[15] K. Schaaff, "EEG-based emotion recognition," Ph.D. thesis, Universitat Karlsruhe (TH), 2008.

[16] A. Accardo, M. Affinito, M. Carrozzi, and F. Bouquet, "Use of the fractal dimension for the analysis of electroencephalographic time series," Biological Cybernetics, vol. 77, pp. 339-350, 1997.

[17] N. Pradhan and D. N. Dutt, "Use of running fractal dimension for the analysis of changing patterns in electroencephalograms," Computers in Biology and Medicine, vol. 23, pp. 381-388, 1993.

[18] W. Lutzenberger, T. Elbert, N. Birbaumer, W. J. Ray, and H. Schupp, "The scalp distribution of the fractal dimension of the EEG and its variation with mental tasks," Brain Topography, vol. 5, pp. 27-34, 1992.

[19] Y. Liu and O. Sourina, "Real-Time Subject-Dependent EEG-Based Emotion Recognition Algorithm," in Transactions on Computational Science XXIII, ed: Springer, 2014, pp. 199-223.

[20] S. Hart and L. Staveland, "Development of a multi-dimensional workload rating scale: Results of empirical and theoretical research. In, PA Hancock and N. Meshkati," Human Mental Workload, 1988.

[21] D. De Waard and V. Studiecentrum, "The measurement of drivers' mental workload," Groningen University, Traffic Research Center, 1996.

[22] J. Sweller, "Cognitive load during problem solving: Effects on learning," Cognitive science, vol. 12, pp. 257-285, 1988.

[23] G. Sammer, C. Blecker, H. Gebhardt, M. Bischoff, R. Stark, K. Morgen, et al., "Relationship between regional hemodynamic activity and simultaneously recorded EEG-theta associated with mental arithmetic-induced workload," Human Brain Mapping, vol. 28, pp. 793-803, 2007.

[24] S. Lei and M. Roetting, "Influence of Task Combination on EEG Spectrum Modulation for Driver Workload Estimation," Human Factors: The Journal of the Human Factors and Ergonomics Society, vol. 53, pp. 168-179, 2011.

[25] G. Borghini, L. Astolfi, G. Vecchiato, D. Mattia, and F. Babiloni, "Measuring neurophysiological signals in aircraft pilots and car drivers for the assessment of mental workload, fatigue and drowsiness," Neuroscience & Biobehavioral Reviews, vol. 44, pp. 58-75, 2014.

[26] M. Hagmüller, E. Rank, and G. Kubin, "Evaluation of the human voice for indications of workload-induced stress in the aviation environment," EEC Note, 18/06 (2006).

[27] R. Khosrowabadi, Q. Chai, A. Kai Keng, T. Sau Wai, and M. Heijnen, "A Brain-Computer Interface for classifying EEG correlates of chronic mental stress," in Neural Networks (IJCNN), The 2011 International Joint Conference on, 2011, pp. 757-762.

[28] S.-H. Seo and J.-T. Lee, "Stress and EEG." Convergence and Hybrid Information Technologies, Marius Crisan (Ed.): InTech, pp. 413-426, 2010.

[29] T. K. Calibo, J. A. Blanco, and S. L. Firebaugh, "Cognitive stress recognition," in Instrumentation and Measurement Technology Conference (I2MTC), 2013 IEEE International, 2013, pp. 1471-1475.

[30] S. S. Hassellund, A. Flaa, L. Sandvik, S. E. Kjeldsen, and M. Rostrup, "Long-term stability of cardiovascular and catecholamine responses to stress tests: An 18-year follow-up study," Hypertension, vol. 55, pp. 131-136, 2010.

[31] S. Ssanghee, G. Yeongjun, and L. Jungtae, "The Relation between Affective Style of Stressor on EEG Asymmetry and Stress Scale during Multimodal Task," in Third International Conference on Convergence and Hybrid Information Technology, 2008. ICCIT '08, pp. 461-466, 2008.

[32] S. C. Jacobs, R. Friedman, J. D. Parker, G. H. Tofler, A. H. Jimenez, J. E. Muller, et al., "Use of skin conductance changes during mental stress testing as an index of autonomic arousal in cardiovascular research," American Heart Journal, vol. 128, pp. 1170-1177, 1994.

[33] R. Luijcks, H. J. Hermens, L. Bodar, C. J. Vossen, J. v. Os, and R. Lousberg, "Experimentally Induced Stress Validated by EMG Activity," PLoS ONE, vol. 9, p. e95215, 2014.

[34] The Emotiv EEG device. Available: http://www.emotiv.com/eeg/ features.php

[35] T. Higuchi, "Approach to an irregular time series on the basis of the fractal theory," Physica D: Nonlinear Phenomena, vol. 31, pp. 277-283, 1988.

[36] W. Qiang and O. Sourina, "Real-Time Mental Arithmetic Task Recognition From EEG Signals," Neural Systems and Rehabilitation Engineering, IEEE Transactions on, vol. 21, pp. 225-232, 2013.

[37] Q. Wang, O. Sourina, and M. Nguyen, "Fractal dimension based neurofeedback in serious games," The Visual Computer, vol. 27, pp. 299-309, 2011.

[38] Y. Liu and O. Sourina, "EEG Databases for Emotion Recognition," in 2013 International Conference on Cyberworlds (CW), pp. 302-309, 2013.

[39] Z. Lan, O. Sourina, L. Wang, and Y. Liu, "Stability of Features in Real-Time EEG-based Emotion Recognition Algorithm," in 2014 International Conference on Cyberworlds (CW), pp. 137-144, 2014.

[40] M. M. Bradley and P. J. Lang, "The International Affective Digitized Sounds ( IADS-2): Affective ratings of sounds and instruction manual," University of Florida, Gainesville, FL, Tech. Rep. B-3, 2007.

[41] P. Lang and M. M. Bradley, "The International Affective Picture System (IAPS) in the study of emotion and attention," Handbook of emotion elicitation and assessment, vol. 29, 2007.

[42] J. Tulen, P. Moleman, H. Van Steenis, and F. Boomsma, "Characterization of stress reactions to the Stroop Color Word Test," Pharmacology Biochemistry and Behavior, vol. 32, pp. 9-15, 1989.

[43] Y. Liu, O. Sourina, and M. K. Nguyen, "Real-time EEG-based human emotion recognition and visualization," in 2010 International Conference on Cyberworlds (CW), pp. 262-269, 2010.

[44] Haptek. Available: http://www.haptek.com/ [45] X. Hou, Y. Liu, O. Sourina, E. Tan, L. Wang, W. Mueller-Wittig,

"EEG based Stress Monitoring", in IEEE International Conference on in Systems, Man and Cybernetics (SMC), in press, 2015.