Author's copy - produced at Lamar University
SPECTRAL ESTIMATION OF ELECTROENCEPHALOGRAM SIGNAL USING
ARMAX MODEL AND PARTICLE SWARM OPTIMIZATION
A Thesis
Presented to
The Faculty of College of Graduate Studies
Lamar University
In Partial Fulfillment
of the Requirements of the Degree
Master of Engineering Science
by
Bijaya Gautam
Date
November 2010
Author's copy - produced at Lamar University
SPECTRAL ESTIMATION OF ELECTROENCEPHALOGRAM SIGNAL USING
ARMAX MODEL AND PARTICLE SWARM OPTIMIZATION
BIJAYA GAUTAM
ABSTRACT
The spectrum estimate of an electroencephalogram (EEG) signal is crucial to
various medical applications. Since the non parametric approach of spectrum estimate is
not reliable for short length data, the parametric approach of spectrum estimate is widely
suggested. In the presented work, a parametric ARMAX model was implemented to find
the spectrum estimate of the EEG signal. Performance of the ARMAX model (based on
the spectrum estimate of the EEG signal) was contrasted with parametric methods (e.g.,
AR model and ARMA) and non parametric method (e.g., periodogram). All the
programming and visualization were performed in a MATLAB environment.
Coefficients of ARMAX and ARMA model were estimated using the particle
swarm optimization (PSO) technique based on swarm intelligence. The PSO algorithm
adapted with the system identification technique and statistics yielded highly satisfactory
results in finding the coefficients of the ARMAX and ARMA models. AR model
parameters were found using the Modified covariance method.
The EEG spectral estimate obtained by the proposed method was able to depict
distinct spectral components not resolved by other methods, even compared to models of
higher orders. ARMAX models of the order three were generally found suitable for EEG
Author's copy - produced at Lamar University
analysis, while the ARMA-based method generally required seventh order to produce
decent spectral estimates. AR-based analyzers were unable to produce spectral estimates
of sufficiently high resolution. The periodogram-based estimator, while showing decent
spectral resolution for high frequency content, failed to resolve low-frequency EEG
components.
Based on these observations, we recommend the ARMAX model for spectral
analysis of EEG.
Author's copy - produced at Lamar University
ii
TABLE OF CONTENTS
Contents Page
List of Figure………………………………………………………………………………v
Chapter
1. Introduction………………………………………………………………………..1
1.1 Background ............................................................................................... 1
1.2 Research Motivation .................................................................................. 2
1.3 Objective ................................................................................................... 2
2. Methods in EEG Signal Analysis………………………………………................4
2.1 Spectral Analysis ....................................................................................... 4
2.1.1 Nonparametric methods ......................................................... 5
2.1.2 Parametric methods ................................................................ 6
3. ARMAX Model…………………………………………………………………..9
3.1 AR Model ................................................................................................ 10
3.2 ARX Model ............................................................................................. 11
3.3 ARMA Model .......................................................................................... 12
3.4 ARMAX .................................................................................................. 13
4. Parameters Estimation…………………………………………………………...15
4.1.1 Models.................................................................................. 15
4.1.2 Estimation ............................................................................ 16
4.1.3 Algorithm ............................................................................. 16
Author's copy - produced at Lamar University
iii
4.1.4 Validation ............................................................................. 16
4.1.5 Model Fit .............................................................................. 17
5. Particle Swarm Optimization…………………………………………….............19
5.1 Introduction of Particle Swarm Optimization ......................................... 19
5.2 PSO Algorithm Flow Diagram ................................................................ 22
6. Experiments and Results…………………………………………………………24
6.1 EEG Signal .............................................................................................. 24
6.2 White Noise ............................................................................................. 25
6.3 Exogenous Input ...................................................................................... 25
6.4 PSO Based ARMAX Parameter Estimation Process .............................. 27
6.4.1 PSO initialization ................................................................. 28
6.4.2 Loss Function ....................................................................... 30
6.4.3 Global and Self best position ............................................... 32
6.4.4 Optimization ........................................................................ 34
7. Spectral Analysis………………………………………………………………...38
7.1 White Noise ............................................................................................. 38
7.2 Exogenous Input ...................................................................................... 39
7.3 Frequency Response of Filters A, B and C ............................................. 41
7.4 ARMAX Model Output ........................................................................... 43
7.5 Performance Comparison for different model orders .............................. 45
7.6 Comparison with an ARMA model ......................................................... 46
7.7 Comparison with an AR model and with the Periodogram method. ....... 49
Author's copy - produced at Lamar University
iv
8. Conclusion……………………………………………………………………….54
8.1 Conclusion ............................................................................................... 54
8.2 Future Work ............................................................................................ 54
9. Reference………………………………………………………………………...56
10. Appendix...……………………………………………………………….………63
10.1 Modified Covariance Method ........................................................ 63
10.2 Model validation data……………………………………………..65
Author's copy - produced at Lamar University
v
LIST OF FIGURES
Figure Number Figure Name Page
Figure 3-1 General Linear Model ................................................................................. 9
Figure 3-2 AR Model .................................................................................................. 10
Figure 3-3 ARX Model ............................................................................................... 11
Figure 3-4 ARMA Model ........................................................................................... 12
Figure 3-5 ARMAX model ......................................................................................... 13
Figure 5-1 PSO Algorithm Flow Diagram (Schutte 2005) ........................................ 22
Figure 6-1 EEG Signal ............................................................................................... 24
Figure 6-2 White Noise realization ............................................................................ 25
Figure 6-3 Exogenous Signal ..................................................................................... 26
Figure 6-4 ARMAX Parameters as Swarm of PSO ................................................... 29
Figure 6-5 AFPE of Each Swarm .............................................................................. 31
Figure 6-6 Global FPE at Each Iteration.................................................................... 33
Figure 6-7 Model Validation ..................................................................................... 35
Figure 6-8 MSE During Model Validation ................................................................ 36
Figure 7-1 Noise Spectral Estimate ........................................................................... 39
Figure 7-2 PSD Estimate of Exogenous Input ........................................................... 40
Figure 7-3 Frequency Response of filters C and B. ................................................... 42
Figure 7-4 Frequency Response of filter A................................................................ 43
Figure 7-5 Spectral Estimate of EEG......................................................................... 44
Figure 7-6 EEG spectral estimates for different model orders .................................. 45
Author's copy - produced at Lamar University
vi
Figure 7-7 EEG spectral estimates by ARMA and ARMAX methods .................... 47
Figure 7-8 Different Order ARMA Model. .............................................................. 48
Figure 7-9 PSD Comparison for Order 3 .................................................................. 50
Figure 7-10 PSD Comparison for Order 5 .................................................................. 51
Figure 7-11 PSD Comparison for Order 7 .................................................................. 52
Author's copy - produced at Lamar University
Bijaya 1
CHAPTER 1
Introduction
1.1 Background
The word Electroencephalography (EEG) was derived from the Greek words
“electro” (related to electron or electrical), ‘enkephalos’ (marrow in the head)
(thefreedictionary 2010), and ‘graphy’ (“writing” or a ‘field of study) (graphy-Wikipedia
2010). Therefore, the literal translation of EEG would be the writing, or study of the
electrical signals in the brain. Electroencephalograph would record the electrical activity
taken from the human scalp (using sensors of a special type) over, usually, a period of
time (Paul 2006). The sensors would be placed in multiple locations of the subject’s
scalp. Recordings of the electrical signals are simultaneously performed for all channels.
From the signal processing viewpoint, EEG is a spatial and non-stationary time-series
process (EEG- Wiki 2010). An analysis of the EEG signals is one of the key areas of
biomedical data processing due to the information contained in these signals. EEG could
be extremely beneficial for studying the conditions and status of the human brain (BSA-
EE Herald 2010).
Similar to other naturally-generated signals, EEG also contains information that
could be extracted by using signal processing techniques. Various types of such
techniques have been developed to analyze EEG signals. An accurate analysis could
provide valuable clinical, psychological, and physical information in reference to the
brain. In particular, EEG waveforms would disclose information about certain changes
Author's copy - produced at Lamar University
Bijaya 2
due to drugs, emotions, thoughts, and diseases. Measurements of these changes available
in EEG waveforms in real-time may be used to determine the status and conditions of the
subject (Niederhauser, et al. 2003).
1.2 Research Motivation
In the study of EEG, the usual task is to estimate the spectrum of the signal. Short
Time Fourier Transform Method (Spectrogram Method), based on Fourier
transformation, is a traditional approach to this study, but, “it has a serious drawback,
which is the implicit assumption of stationary within each segment and unsatisfactory
time/frequency resolution” (Maghsoudi and White 1993).
Parametric spectral analysis methods based on autoregressive (AR) and auto
regressive moving average (ARMA) models have been suggested as a better approach for
the spectral estimates of the EEG signal (Tseng, Chong and Kuo 1995). This research
will focus on applying the "auto regressive moving average with the exogenous input"
(ARMAX) method to estimate the spectrum of the EEG. There has not been much
scholarly work done with ARMAX in the spectral estimation of EEG, and ARMAX may
allow for a more accurate and flexible representation of the EEG time series.
1.3 Objective
The basic objective of this research was to test the suitability of an ARMAX
model for a spectral analyss of the EEG signal. In order to perform this analysis,
parameters of the ARMAX model needed to be estimated. For this purpose we propose
Author's copy - produced at Lamar University
Bijaya 3
employing an innovative approach, the Particle Swarm Optimization (PSO) technique.
The PSO algorithm was implemented and its performance was evaluated using
MATLAB.
Author's copy - produced at Lamar University
Bijaya 4
CHAPTER 2
Methods in EEG Signal Analysis
2.1 Spectral Analysis
The time series signal could also be represented in the frequency domain
(Estimation theory 2010). Power-frequency distribution of the signal sometimes becomes
more important than its time domain representation. Spectral estimation would portray
the distribution of the power contained in a signal over the frequency within a finite set of
data (Murugesan and Sukanesh 2009). Knowledge of the power spectral density (PSD) in
the signal is useful in various situations e.g. detection of a signal masked in wideband
noise (S D-Wiki 2010).
The Wiener–Khinchin theorem defines the concept of a spectrum for a wide sense
stationary (WSS) process. The theorem states, “The PSD S(f) of a wide-sense stationary
random process x(τ) is the Fourier transform of its autocorrelation function R(τ)”, as
shown in Equation 2-2 (S D-Wiki 2010).
��τ� = � x�t + τ�x�t�dt =��� � x�t�x�t − τ�dt��� Equation 2-1
Thus,
���� = � ���������������� = Ϝ{����} Equation 2-2
If R(τ) is known for all τ, evaluating the power spectrum would be
straightforward. The two basic problems for spectral estimation of the EEG data are:
Author's copy - produced at Lamar University
Bijaya 5
a. The amount of data is always limited.
b. Data is frequently corrupted by noise or contaminated with an interfering
signal (Hayes 1996).
The two most abundant methods of spectrum estimation are as follows.
(Bingham, Godfrey and Tukey 1967).
a) Nonparametric methods relied on the direct use of the available data.
b) Parametric methods relied on the model assumed for the signal generation.
Therefore, the choice between parametric methods and nonparametric methods to
obtain a PSD estimate is a choice between straightforward (less accurate) nonparametric
estimators and a computationally complex (yet more accurate) parametric PSD estimator.
2.1.1 Nonparametric methods
Nonparametric methods would use signals to determine the power spectrum
density. If x[n] is a finite-length signal, the simplest method of a PSD estimate is the
periodogram P(ejw
) (Schuster 1898) that is based on Fourier transform as shown in
Equation 2-3.
������ = �� ����� ∗����� = �� | #���$|� Equation 2-3
Here, X(e^jw) is DTFT of x[n] and N denotes the total number of samples. Welch
modified the now commonly used periodogram in 1967. His modification is known as
Welch's method (Welch 1967).
Author's copy - produced at Lamar University
Bijaya 6
Akin has used the periodogram method of spectral estimation for clinical
applications (Akin and Kiymik 2000). Bullock and his colleagues used an additive
periodogram to determine the stochastic nature of EEG signal and to monitor the
temporal fine structure of the EEG data (Bullock, Enright and Chong 1998). Sign
periodogram was successfully used to identify seizure precursors from the depth of the
EEG (Niederhauser, et al. 2003). Recently, Qihou Zhou and colleagues reported a time-
frequency analysis method by using an adaptive length periodogram technique for EEG
data processing. This technique was reported to have optimal resolution in both the
frequency and time domain. (Qihou, Matthew and Jade 2008). The nonparametric
methods - like periodogram, Bartlett, Welch, and Blackman Tukey’s - are generally
limited by the length of data. A few other general limitations known about nonparametric
methods are (Marple 1989):
a. Correlation is assumed to be zero beyond N. N is thethe number of available
data samples.
b. If two frequencies are separated by ∆f, then we need N ≥ �∆' data samples to
resolve them.
c. Resolution limit imposed by N also causes bias.
2.1.2 Parametric methods
In the Parametric methods, the frequency domain output of a model (usually
driven by white noise) is viewed as the power spectral density of the signal. Some
examples would be the Yule-Walker autoregressive (AR) structure and the Burg method
Author's copy - produced at Lamar University
Bijaya 7
(Bingham, Godfrey and Tukey 1967). Parametric methods would produce better
resolution than nonparametric methods when applied to a short length signal (Brown and
Hwang 1983 and Tseng, Chong and Kuo 1995).
Subspace methods, also known as high-resolution methods of spectral estimation, would
use an Eigen-analysis or Eigen-decomposition of the correlation matrix (Spectral
Analysis S E M 2010). Multiple signal classification (MUSIC) method and the
eigenvector (EV) method are examples of subspace algorithms. Subspace techniques are
found useful for the line spectra and can be used when sinusoids are corrupted by noise
with low signal-to-noise ratios (Stoica and Eriksson 1995).With an accuracy of 95.6%,
Melancholia disease has been diagnosed by Enping using the Eigen vector method to
EEG signals (En'ping, Sheng and Shini 2009).
Reports are suggesting to use the autoregressive moving average (ARMA) models
for the EEG analysis (Maghsoudi and White 1993). In the method proposed by
Maghsoudi, coefficients of the ARMA model are identified and used to depict the
waveform. Extended Least Squares and Recursive Extended Least Squares methods were
used to estimate the parameters of the first order ARMA model. The identified model
was then used to describe the time and frequency-domain properties of the EEG
waveforms. Parametric methods suffer from the rise of computation time. However, the
benefit of the ARMA method (over the nonparametric estimation) is its ability to track
the time-varying process (Maghsoudi and White 1993).
The evaluation of the performance of parametric methods for spectral estimation
of EEG was conducted by Tseng (Tseng, Chong and Kuo 1995). The Akaike information
criterion (AIC) (Akaike 1974) was used for determining the orders of AR and ARMA
Author's copy - produced at Lamar University
Bijaya 8
models. The tests had suggested that the AR model would require a higher model order
(8.67 on average) than the ARMA model order (6.17 on average). It was also found that
about 96% of the total EEG segments each 1.024 seconds long were efficiently
represented by the AR model, and only about 78% could be represented by the ARMA
model.
Author's copy - produced at Lamar University
Bijaya 9
CHAPTER 3
ARMAX Model
The system shown in Figure 3-1 is called a general linear model and could be
described using the Equation 3-1 (Candy 2006).
Figure 3-1 General Linear Model
)�*�+�,� = ��,�-�*� + .�,�/�* − 0� Equation 3-1
When A(q), B(q) and C(q) are the polynomials of corresponding filters’ coefficients.
k is delay,
e(n) indicates white noise and u(n) represents the deterministic signal.
��,�-�*� resembles the stochastic part of the system and
.�,�/�* − 0� resembles the deterministic part of the system.
y(n) indicates the output of the system.
Author's copy - produced at Lamar University
Bijaya 10
General linear model structure is shown in Figure 3-1. A simple change in
parameters could yield a completely new model. Interestingly, a nonlinear optimization
method that requires serious calculation without any global convergence could be used to
estimate the parameter of the general linear model (Instruments 2010).
By setting the filter polynomials A (q), B (q) or C (q) in Equation 3-1, equal to
one or zero, we could derive somewhat simpler models such as the ARX, AR and ARMA
structures. Selection of the appropriate model would depend entirely on the
characteristics of the problem since these models having different characteristics and
applications.
3.1 AR Model
A simple AR model is shown in figure 3-2 and is defined by Equation 3-2. AR
model uses only A(q) filter. e(n) and y(n) indicates white noise input and system output
respectively.
Figure 3-2 AR Model
)�*�+�,� = ��,� Equation 3-2
Author's copy - produced at Lamar University
Bijaya 11
The AR model is used when current output is dependent only on the previous
outputs. There are different types of parameter estimation techniques for the AR model.
One of the techniques is the modified covariance method. This method is described in
appendix I (Hayes 1996).
3.2 ARX Model
Addition of exogenous signal .�,� as an input to a simple AR model give rise to
ARX model as shown in Figure 3-3 and described by equation 3-3. ARX model uses
A(q) filter and B(q) filter with k delay samples.
Figure 3-3 ARX Model
)�*�+�,� = ��,� + .�,�/�* − 0� Equation 3-3
This model could give the best result among the polynomial estimations as the
model is obtained as a solution of a linear equation in analytic form. “The solution is
unique and always satisfies the global minimum loss function.” (Instruments 2010).
Author's copy - produced at Lamar University
Bijaya 12
3.3 ARMA Model
Combining Autoregressive (AR) and moving average (MA) processes, a highly
flexible class of uni-variate processes called the ARMA process was introduced by Box
and Jenkins (Jenkins and Box 1970). A block diagram of an ARMA model is shown in
Figure 3-4.
.
Figure 3-4 ARMA Model
)�*�+�,� = ��,�-�*� Equation 3-4
ARMA model incorporates A(q) filter (Autoregressive component) and C(q) filter
(Moving Average component). The ARMA model does not have exogenous input and is
a special case of ARMAX models.
Author's copy - produced at Lamar University
Bijaya 13
3.4 ARMAX
The autoregressive moving average model with the exogenous inputs (ARMAX
model) incorporates the disturbance dynamics known as exogenous input. This model is
widely used in the presence of dominating disturbances that appear early in the process,
such as the input. The ARMAX model could be significantly more efficient at
management of disturbance modeling than the ARX model (Instruments 2010).
Figure 3-5 ARMAX model
The diagram in Error! Reference source not found.5 can be represented by
quation 3-5 (System Identification Toolbox 2010).
+�,� + 1�+�, − 1�+. . . +145+�, − ,5� = 6�.#, − ,7� + ⋯ + 645.�, − ,7 −,9 + 1� + :���, − 1� + ⋯ + :4;��, − ,;� + ��,�� Equation 3-5
Where, y(n) Output at sample n.
na Number of poles of filter A.
nb Number of zeroes of filter B plus 1.
nc Number of poles of filter C.
nk is the dead time in the system. For discrete systems with no dead time nk =1.
Author's copy - produced at Lamar University
Bijaya 14
u (n) is exogeneous input.
e(n) is whit noise.
The parameters na, nb, and nc are the orders of the ARMAX model; therefore,
filter polynomials A, B and C can be written as follows:
)�*� = 1 + 1�*�� + ⋯ + 14<*�4<
/�*� = 6� + 6�*�� + ⋯ + 64=*�4=>�
-�*� = 1 + :�*�� + ⋯ + :4?*�4? Equation 3-6
Author's copy - produced at Lamar University
Bijaya 15
CHAPTER 4
Parameters Estimation
Depending upon repeatability and extraneous variations, there are two types of
signals; the deterministic signal and random signal. Deterministic signals are repeatable
and do not vary extraneously. Random signals are not repeatable and vary extraneously
(Candy 2006). The real world’s experimental input-output data were used to estimate
parameters of mathematical models. The process of estimating parameters of the model
with the real world’s experimental input-output data is based on the statistical theory
(Ljung 2010). The theory is centered on the following concepts. (Ljung 2010)
4.1.1 Models
The model represents a relation between system input and the output (Candy
2006). Mathematical expression or a graph or table may be used to define this
relationship. We denote the model by ‘m’ (Ljung 2010).
Depending upon the nature of the problem, the model could be deterministic or
probabilistic. Usually, models contain information such as the process of signal
generation, characteristics of noise, and the deterministic and probabilistic structure, etc.
(Candy 2006).
Author's copy - produced at Lamar University
Bijaya 16
4.1.2 Estimation
Information about the object to be modeled, the nature of the problem and the
observed data can be used to select an appropriate model. This process of selection is
called ‘model estimation’. Estimation data, usually known as training data obtained from
experiments, are used in model selection. We denote the estimation of data by zen (Ljung
2010). Where, ‘n’ is the size of the data.
4.1.3 Algorithm
Various factors must be considered while determining a correct estimation
algorithm e.g computation time, error, data length etc. According to Candy, “a
preconceived knowledge of the similar problem structure of the estimator greatly
influences the resulting algorithm” (Candy 2006). Performance of the algorithm can be
assessed by evaluating the estimation error. To yield a good estimation algorithm, the
estimation error must be minimized (Ljung 2010).
4.1.4 Validation
Excluding the evaluation set of data, a model should be useful to the other set of
data that was never used for training purposes. Data sets used for this purpose are called
validation data. We denote validation data by zCD (Ljung 2010).
Author's copy - produced at Lamar University
Bijaya 17
4.1.5 Model Fit
The model fit is a scalar value representing how well a model is able to fit the
data set Z. We denote the model fit by F(m,Z) (Ljung 2010).
With a model parameters optimization algorithm, it is not difficult to find a model
that would represent the estimation data set. The actual performance of a model is given
by the model fit value with the validation data E�F, HI4�. The average fit of the validation
data is known as ‘expected fit’ E�F, HI4� ,and is generally smaller than the fit of the
estimation data known as ‘actual fit’ E�F, HJ4� (Ljung 2010).
The data set HJ4 is represented by a model FK , estimated from an estimation data
set HJ4 ,in a model set M using model estimation techniques as described above. The
model fit of FK with the estimation data HJ4 is given by E�FK, HJ4� and model fit of FK to
validation data HI4 and given by E�FK, HI4�.
E�FK, HI4� = E�FK, HJ4� + ��-�L�, M� Equation 4-1
The mean square error between the expected and actual fit is known as the fitness
of the model (Ljung 2010). C is the complexity of the model that would decrease with the
increase of N (Ljung 1999). Since FK is a random variable of the set M, Equation 4-1 is
probabilistic. The expectation of this function Nℱ PPP�FK, HI4� is given by various forms as
shown in Equation 4-2.
Author's copy - produced at Lamar University
Bijaya 18
Nℱ PPP�FK, HI4� ≈ �>RS��RS ℱ�FK, HJ4� ……..(a)
≈ 1 + �T� ℱ�FK, HJ4� ……..(b)
≈ ����RS�U ℱ�FK, HJ4� ..……(c) Equation 4-2
Equation 4-2(a) is known as Akaike’s Final Prediction Error (FPE); this function
is used to determine the model order and acts as a criterion function. Equation 4-2(b) is
the Akaike’s Information Criterion (AIC) (Akaike 1974), and Equation 4-2 (c) is the
generalized cross-validation (GCV) criterion, (Wahba and Craven 1979). ‘d’ is the
complexity measure of the model set.
Author's copy - produced at Lamar University
Bijaya 19
CHAPTER 5
Particle Swarm Optimization
5.1 Introduction of Particle Swarm Optimization
The Particle Swarm Optimization is one of the stochastic optimization techniques.
PSO is based on the social and personal behavior of a swarm. It was first introduced by
Eberhart and Kennedy in 1995 (Eberhart and Kennedy 1995). The researchers described
how PSO can be applied to a nonlinear optimization problem through the simulation of a
social system characterized by swarms (e.g. bees). PSO would continuously search for a
new position for each swarm based on their social best position and self best position. Shi
and Eberhart introduced a third factor, inertia weight, responsible for keeping a swarm in
its original direction (Eberhart and Shi 1998). In the same year, Angeline developed a
“hybrid PSO with the addition of a standard selection process from evolutionary
computations in order to produce a high-quality solution within a short computation
time” (Angeline 1998). PSO is being used for various applications around the world such
as solving a computationally complex -“travelling salesman problem” (Wang, et al.
2003).
PSO and evolutionary computation techniques, like a Genetic Algorithm, are
especially similar. Both algorithms would work by searching solutions enhanced with
updating generations. However, there is nothing to the equivalent of ‘evolution operators’
in PSO, which is common in Genetic Algorithm. Most importantly, PSO does not have a
global gradient (Jenigiri 1996). Research on PSO is evolving around the world and
different hybrid modules of PSO are also being proposed. Jiao Wei and colleagues have
Author's copy - produced at Lamar University
Bijaya 20
successfully incorporated a mutation in PSO (Jiao, Liu and Xian 2008). Gudsiz has
compared PSO with Artificial Neural Network (ANN) and efficiently cited ANN (Gudsiz
and Venayagamoorthy 2003). PSO is also being used in system identification problems.
Huang and colleagues have demonstrated that PSO can be used for identifying an
ARMAX Model for Short- term Load Forecasting (Huang, Huang and Wang 2005).
The Basic PSO algorithm is described by Equation 5-1 and Equation 5-2
(Eberhart and Shi 1998). The position of an individual particle is updated using Equation
5-2 with the velocity calculated in Equation 5-1.
V7>�W = XV7W + Y� .� × [�6�\�7W − �7W ] + Y�.� × [^6�\�W − �7W ] Equation 5-1
�7>�W = �7W + V7>�W Equation 5-2
Where, �7W - Position of the ith
particle during kth
iteration.
V7W - Velocity of the ith
particle during kth
iteration.
�6�\�7W - Individual best fitness achieved by a ith
particle till kth
iteration.
^6�\�W- Global best fitness of ith
particle till kth
iteration.
X - inertial coefficient.
Y�and Y�- Cognitive and social parameters.
.�and .� - Random numbers between 0 and 1.
Equation 5-1 is the summation of three different velocity update parameters. Each
term has its own important role in optimizing the PSO algorithm.
a) Inertial component: The first term XV7W is known as an inertia component
that is responsible for preserving the original direction of particle. (Blodin
Author's copy - produced at Lamar University
Bijaya 21
2009). Generally, the inertial coefficient w may be chosen between 0.8 and 1.2
(Eberhart, Kennedy and Shi 2001).
b) Cognitive component: The second term Y� .�[�6�\�7W − �7W ], is called the
cognitive component. Blodin defines this component as “the memory of the
particle, forcing it to move toward the search space that has achieved the best
individual fitness (pbesti) ever” (Blodin 2009). The cognitive coefficient Y� is
chosen close to 2. This coefficient is responsible for the step size, measuring
what a particle would take to reach its ‘individual best solution’ (Eberhart,
Kennedy and Shi 2001).
c) Social component: The third term Y�.�[^6�\�W − �7W ] is known as the social
component. This component is also known as the memory of the particle. This
term would force the particle to move toward the search space that has
achieved the best global fitness (gbesti) (Blodin 2009). The social coefficient
Y� is also chosen close to 2. Unlike the cognitive component, the social
component would drive a particle toward the best global candidate solution
with a step size of Y� (Eberhart, Kennedy and Shi 2001).
.� and .� in the cognitive and social component are random values responsible
for stochastic behavior of the velocity update. Presence of these stochastic parameters
causes each particle to move in a semi-random manner toward the direction of the pbest
of the particle and gbest of the swarm (Blodin 2009).
Author's copy - produced at Lamar University
Bijaya 22
5.2 PSO Algorithm Flow Diagram
The PSO algorithm flow diagram is shown in Figure 5-1 (Kennedy and Eberhart
1997) and (Angeline 1998).
Figure 5-1 PSO Algorithm Flow Diagram (Schutte 2005)
Author's copy - produced at Lamar University
Bijaya 23
The basic process is outlined as followed:
1. Initialization
a. Specify number of iteration (Kmax), Y�, Y� and w.
b. Randomly initialize all particles’ positions Ri .
c. Randomly initialize all particle’s velocity vi such that v
i < v
max.
d. Set k=1
2. Optimization
a. Evaluate loss function value `ab using design space coordinates cab .
b. If `ab < `efghb then iefghab = cab .
c. If `ab < `efghj then kefghab = cab
d. Update all the particle velocities vi.
e. Update all the particle position Ri.
f. Check stopping condition
a. Yes: display result and quit.
b. No: increase i.
g. Check condition if i is less than total number of particle
a. Yes: Set i=1 and increase k then go to 2(a).
b. No: Go to 2(a)
Author's copy - produced at Lamar University
Bijaya 24
CHAPTER 6
Experiments and Results
6.1 EEG Signal
The EEG data used in this research was obtained from Henri Begleiter from the
Neurodynamics Laboratory at the State University of New York Health Center in
Brooklyn. “These data were recorded to examine EEG correlates of genetic
predisposition to alcoholism. It contains measurements from 64 electrodes placed on
subject's scalps which were sampled at 256 Hz (3.9-msec epoch)” (Begleiter 1999). For
our experiment, we used 30 samples i.e. 0.1172 Sec length of signal from randomly
selected data set. An example of the experimental data is shown in Figure 6-1.
Figure 6-1EEG Signal
Author's copy - produced at Lamar University
Bijaya 25
6.2 White Noise
The discrete-time random process represented by a random vector w is a white
noise vector with the mean vector l� and autocorrelation function ���that would satisfy
the Equation 6-1 and Equation 6-2 (white noise 2010 ).
l� = N{X} = 0 Equation 6-1
��� = N{XXn} = o�p Equation 6-2
Where, I is identity matrix. Figure 6-2 shows the white noise used for the
experiment.
Figure 6-2 White Noise realization
6.3 Exogenous Input
Unlike ARMA, an ARMAX model requires an exogenous input. This input
should be a deterministic signal. In this study, the exogenous input .��� was formed by
Author's copy - produced at Lamar University
Bijaya 26
the summation of three sinusoids of f1, f2 and f3 as represented by Equation 6-3 and
shown Figure 6-3. While the EEG signal has been sampled with sampling frequency fs =
256 Hz i.e. sampling time Ts= 0.0039 Sec, the exogenous signal was generated for 30
epochs (0.1172 Seconds). In this research, the sum of pure sinusoidal waves of the
frequencies 5 Hz, 10 Hz and 18 Hz is taken as an exogenous Input.
.��� = sin�2t���� + sin�2t���� + sin�2t���� Equation 6-3
Figure 6-3 Exogenous Signal
Author's copy - produced at Lamar University
Bijaya 27
6.4 PSO Based ARMAX Parameter Estimation Process
First, we needed to verify that our algorithm would work properly. For this
purpose, the ARMAX model with the following arbitrarily selected parameters was
created of the order of 3, 5 and 7.
A= [1 -0.1400 -0.4244 0.3963 0.3921 0.3764 0.0796 0.0321]
/ = [-0.2347 0.1348 -0.2084 -0.0903 0.3789 -0.0220 -0.1044 0.0021]
- = [1 0.0137 0.1123 0.1459 -0.0266 -0.1099 -0.1540 0.0419] Equation 6-4
Equation 6-4 represents the ARMAX model from order 7. Models of Orders 5 and
3 may be obtained by excluding appropriate higher order coefficients from the same
filters A, B and C. The ARMAX model of a particular order was simulated with
exogenous input u(n) and white noise e(n). The output of the model y(n) was recorded
along with u(n) in a data set Ze, as shown in Equation 6-5. So obtained output of the
model y(n) was used as the ‘target data’ to train an ARMAX model.
Ze ={ [+�(n) u1(n)], [+�(n) u2(n)], [+u(n) u3(n)]…..[ +W (n),ui(n)]} Equation 6-5
One hundred of data sets were generated so that each set contained a pair of
exogenous input u(n) and training data y(n). With these training data sets and noise e(n),
ARMAX model parameters were estimated using the PSO technique. The objective of
finding the ARMAX model parameters is to validate the PSO algorithm being used.
Author's copy - produced at Lamar University
Bijaya 28
Parameters estimated using PSO are referred to as )K , /P and -K . The estimation process
was performed for the orders 3, 5, and 7.
A new ARMAX model was created with [)v /P -K ] and simulated with exogenous
input and white noise. The simulated output of the ARMAX model with parameters [)v /P
-K ] was compared with the output of the ARMAX model with parameters [A B C]. If
both of the outputs were similar, we concluded that the PSO technique could produce the
ARMAX parameters imitating a nonlinear system (like an EEG sequence). Once the PSO
technique was validated, we used real EEG signals to estimate the ARMAX model
parameters. Analyzing the model enables us to produce a spectrum of that model that can
be used as a spectral estimate of a real EEG signal.
The PSO technique as described in chapter 5 was used to estimate the ARMAX
model parameters. The following estimation steps were followed with each data set.
6.4.1 PSO initialization
The initial trial parameter vectors Ri were randomly generated, where Ri =
uniform(a,b)d, i=1,2,……k, d = n+m+r are uniformly distributed in [a,b] in the d-
dimensional space. Here, n,m and r were the orders of filters A, B and C respectively.
For example, if all filters’ orders n, m and r equal to three, then d=3+3+3=9. The
dimension of random matrix would be a random matrix of 9 X Ri. Each element of Ri is
uniform distribution between integers ‘a’ and ‘b’. In this matrix, each Ri column is
potential ARMAX parameters. With these possible parameters, an ARMAX model is
simulated with white noise (as discussed in chapter 6-2) and with an exogenous input (as
discussed in chapter 6-3).
Author's copy - produced at Lamar University
Bijaya 29
Figure 6-4 ARMAX Parameters as Swarm of PSO
The third order ARMAX model represented by Equation 3-1 can also be
described as:
+�,� = 1 − 1�+�, − 1�−1�+�, − 2�−1u+�, − 3� + 6�.�, − 1� + 6�.�, − 2�+ 6u.�, − 3� + :���, − 1� + :���, − 1� + :���, − 1� + ��,�
Equation 6-6
In this step, a number of tentative ARMAX models Ri with the same model
structure of n, m and r but with different combinations of parameter vectors were
Author's copy - produced at Lamar University
Bijaya 30
established. In the initialization stage ‘i’ the number of the ARMAX model was created
as shown in Error! Reference source not found.. Each Ri is called a ‘current position’
f swarm according to PSO terminology.
6.4.2 Loss Function
‘ith
’ model output was obtained by simulating each ARMAX model represented
by Ri with exogenous input ui . New data sets Hx i containing ui and +yi were created as
specified by Equation 6-7.
Hx i=( [+y1 (n) u1(n)], [+y2 (n) u2(n)], [+y3 (n) u2(n)]…..[ +yi (n),ui(n)] Equation 6-7
Here, +y and u are the output and the input of ARMAX model Ri.
The difference between the output +z generated with the training data and a real
output y is known as a residual or prediction error (Equation 6-8). A model producing
small prediction error is considered a good model (Rahiman, Taib and Salleh 2007).
ε�n� = y�n� − yy�n� Equation 6-8
y(n) is the target data and yy�n� is the predicted output of the ARMAX model Ri. The loss
function of ith
model Ri is calculated using a prediction error }�,� and a transpose of }�,�
(Ljung 2001) as follows:
Author's copy - produced at Lamar University
Bijaya 31
~���� = det [�4 �}�,� × }�,���] Equation 6-9
The Akaike Final Prediction Error (AFPE) is obtained using Equation 6-10.
)E�N = ~���, �� × �> ���R���� ������ Equation 6-10
The loss function shows the accuracy of the fit for the model. The AFPE of each
Ri is the accuracy for the fit termed ‘Swarm fitness’ of the swarm Ri. .Among all the
Swarm fitness of Ri, at Kth
iteration, the swarm fitness that is the absolute minimum (i.e.
nearest to zero) is called the ‘global best swarm fitness’ (Gbestk). Figure 6-5 illustrates
fitness (AFPE) of each Ri.
Figure 6-5 AFPE of Each Swarm
Author's copy - produced at Lamar University
Bijaya 32
As shown in Figure 6-5, for each Ri an AFPE is evaluated. The minimum AFPE is
chosen as the global best fitness ‘^6�\�’ at that step. Analysis indicates that the AFPE is
at its minimum at the 5th
position of swarm for the case shown in Figure 6-5. Therefore,
the 5th
position swarm (R5) has the minimum AFPE among all Ri. If Ri keeps updating
with iterations, Gbest changes accordingly.
6.4.3 Global and Self best position
Each particle Ri remembers its own loss function and designates the minimum
function. This is called the self best position �6�\�W. The particle with the best loss
function among all Ri is referred to as the global best position ^6�\�. In the first iteration,
each particle’s Ri is set directly to �6�\�Wand the particle with the best FPE value among
�6�\� is set to ^6�\�. An example of how �6�\�W and ^6�\� change in each iteration has
been shown in Table 1.
Table 1. An Example of global and local fitness
Gbest
������� �/�6�\��
������� �/�6�\��
��u���� �/�6�\�u
………..
��W���� �/�6�\�W
Iteration
(Step k)
0.2
0.256 -0.421 0.20
………..
0.436
1 0.256 -0.421 0.20 0.436
0.17
0.17 -0.514 0.21
………..
0.201
2 0.17 -0.421 0.20 0.201
0.073
0.104 -0.073 0.243
……
0.154
3 0.104 -0.073 0.20 0.154
Author's copy - produced at Lamar University
Bijaya 33
The first iteration �6�\�W of each Ri is set directly equal to the AFPE of Ri. R3 is
the minimum AFPE at the first step, so gbest is set to R3AFPE. During the second iteration,
a new gbest is obtained, but pbest is not changed for all the swarms. R2AFPE and R3AFPE
are higher than the first iteration, so pbest of these two swarms are not changed during
the second iteration. During the third iteration, pbest of all the swarms are changed except
from R3. Similarly, pbest and gbest are updated until the predefined number of iterations
is completed. Figure 6-6 illustrates the change in gbest during iteration.
Figure 6-6 Global FPE at Each Iteration
Author's copy - produced at Lamar University
Bijaya 34
Figure 6-6 shows that at each step of the swarm (at each iteration), ^6�\� is
decreasing. This indicates that the difference between the target signal and the model
output are decreasing at each step, and all particles are converging toward the solution in
search space.
6.4.4 Optimization
The optimization of the PSO algorithm is based on the velocity and the position
update step. Velocity would refer to the difference between the swarm positions at kth
iteration and at (k+1)th
iteration. The Velocity of the Swarm is calculated according to
the global and local best positions using Equation 5-1.
When the new velocity and the previous position are known, an offspring
vector is calculated as the sum of the velocity and the previous position as shown in
Equation 5-2. Offspring vector is the new position of the swarm.
The new position is set as the current position of a swarm at kth
iteration. Now, a
new set of Ri is ready to simulate the ARMAX model. Steps 6.4.2 to 6.4.4 are repeated
until the number of iteration assigned is completed. Because of this, Ri has the best
solution, and Gbest in the population has been obtained. Vectors of Ribest containing
coefficients [)v /P -v ] are regarded as a solution, or as an optimized model. The optimized
model should more precisely pass diagnostic checks. Otherwise, the model identification
process described above should be repeated until an appropriate model is obtained.
The best solution obtained using PSO algorithm has to pass the validation test. To
validate the model, the validation data V(n) is used. This data should never be used to
train the model. The model is simulated, and this time the output of model is compared
Author's copy - produced at Lamar University
Bijaya 35
with V(n) instead of target data y(n). The Mean Square Error (MSE) between validation
data and the model output is obtained by using Equation 6-11.
MSE�,� = �� ���,� − +y�,��� Equation 6-11
When V(n) is the validation data, yy�n� is the predicted output of the ARMAX
model with [)v. /P -v ], and N is the length of data.
Figure 6-7 demonstrates the validation data from the initial coefficients [A B C]
and simulated output of the PSO based coefficients [)v. /P -v ].
Figure 6-7 Model Validation
Author's copy - produced at Lamar University
Bijaya 36
It can be observed that the model output follows the validation data very closely.
This observation indicates that the ARMAX model created with [)v. /P -v ] has succeeded
to depict the target signal.
Figure 6-8 shows the mean square error (MSE) defined by Equation 6-11. It is
seen that the MSE between ��,�and +y�,� is very small. This indicates that the
performance of the estimated model is satisfactory with a good model fit.
Figure 6-8 MSE During Model Validation
The process of the ARMAX parameter estimation and validation was repeated for
100 independent data sets. Therefore, we needed to record the MSE as a single valued
Author's copy - produced at Lamar University
Bijaya 37
score so that we could compare the MSE with two different results of simulation. The
combination of the Akaike final prediction error (Akaike 1974) and Cramer’s rule of
determinant (Weisstein 2010) yields Equation 6-12 giving us a single valued score of the
MSE.
L�N����� = abs[������ L�N × L�N��] Equation 6-12
MSE(det) is calculated from the score taken from the best ARMAX parameters.
This score signifies how well the model output may resemble the validation data. When
the MSE(det) is smaller, the model performance is better. The simulation of the results for
one hundred data sets, and their respective MSE(det) is presented in Appendix II. The
average for the MSE(det) was found to be as small as 2.42 E-06. This result indicates the
functionality and convergence of the PSO algorithm and ARMAX model.
Author's copy - produced at Lamar University
Bijaya 38
CHAPTER 7
Spectral Analysis
7.1 White Noise
Theoretically, the spectral density of a white noise is expected to be a constant
(Brown and Hwang 1983). Since power is equally distributed in all frequencies, the total
power of the signal should be infinite, making it impossible to generate in practice.
“Since, an infinite-bandwidth white noise signal is only theoretically possible” (white
noise 2010 ), in this experiment, we dealt with white noise by using a flat spectrum over a
defined wide band.
Equation 6-1 and Equation 6-2 do imply that a “white noise random process has
zero mean and infinite power at zero time shifts. Autocorrelation function of random
process is the Dirac delta function,” (white noise 2010 ) and the Fourier transform of the
autocorrelation function is also the power spectral density (PSD) (Papoulis 1991).
����X� = ��� Equation 7-1
A sample pseudo-random noise process has been generated, and its PSD was
estimated by the periodogram method. The estimate of noise PSD is shown in Figure 7-1
Author's copy - produced at Lamar University
Bijaya 39
Figure 7-1 Noise Spectral Estimate
Since we were using a finite sample of the noise signal, the power spectral density
was estimated by using a periodogram as shown in Figure 7-1. The graph is only an
estimate of the noise spectrum, not a true noise spectrum.
7.2 Exogenous Input
The frequency domain representation of .��� described by Equation 6-3 could be
obtained using the Fourier transform. The Fourier transform of .���is demonstrated in
Equation 7-2.
Ϝ{.���} = .#���$ = �2 [��� + ��� − ��� − ��� + ��� + ��� − ��� − ���
+��� + �u� − ��� − �u�] Equation 7-2
Author's copy - produced at Lamar University
Bijaya 40
In this project, the sum of pure sinusoidal waves of frequencies 5 Hz, 10 Hz and
18 Hz was used as an exogenous input. PSD of the exogenous signal was estimated by
using the periodogram method and is shown in Figure 7-2.
Figure 7-2 PSD Estimate of Exogenous Input
In the Figure 7-2, we observe three distinct peaks that are due to the three
frequency components f1, f2, and f3 described in the Equation 6-3 and Equation 7-2. It
should be noticed that the exact mirror image of the spectrum shown in the positive
Author's copy - produced at Lamar University
Bijaya 41
frequency range also exists in the negative frequency range. In the entire spectrum, the
total six main peaks exist as described in Equation 7-2.
7.3 Frequency Response of Filters A, B and C
The ARMAX parameters would resemble three filters: A, B and C. Each filter
has coefficients computed with the PSO algorithm as explained in CHAPTER 5. The
frequency responses of these filters are computed using the discrete Fourier transform
method. For a discrete set of complex or real values x[n], where n is integer, the DTFT of
x[n] is
��� = ∑ ����D[,]�4��� Equation 7-3
The frequency response of each filter was obtained with 512 point DFT. The
frequency responses of filters B and C are shown in Figure 7-3. The product of filter B,
the exogenous input and product of noise, and filter C (an ARMA part of the ARMAX
model) are also shown in the same figure.
Author's copy - produced at Lamar University
Bijaya 42
Figure 7-3 Frequency Response of filters C and B.
It is seen in Figure 7-3 that the filter C is a low pass filter. Since the EEG may be
viewed as a signal having most information within the lower frequency band, it is
reasonably predictable that filter C is low pass filter. The sum of an ARMA part and an
exogenous part is shown in Figure 7-3. This result is filtered by the filter A. The
frequency response of filter A is also shown in Figure 7-4.
Author's copy - produced at Lamar University
Bijaya 43
Figure 7-4 Frequency Response of filter A
It could be seen in Figure 7-4 that filter 1/A is a band stop filter. Filter responses
of these filters are not distinct; they vary with the length of data.
7.4 ARMAX Model Output
When the sum of the exogenous part and the ARMA part is filtered with the filter
A, the final output of the ARMAX model is obtained. This output is the spectral
representation of y(n) i.e. Y(ejw
), thus the spectral estimate of the EEG analyzed. The
output of the third order ARMAX model for 0.2 seconds (51 samples) of the EEG data is
shown in Figure 7-5.
Author's copy - produced at Lamar University
Bijaya 44
Figure 7-5 Spectral Estimate of EEG
The spectral estimate indicates that the EEG signal has higher energy at lower
frequency, and the power is decreasing as the frequency increases. Jorge Baztarrica
Ochoa in his thesis on classification of EEG signals for brain computer interface
application depicts that EEG signal upto 4Hz known as ‘Delta type’ is normally present
with high amplitude in all adults and babies during sleeping (Ochoa 2002). Magnitude of
amplitude at frequency range 4-7 Hz, known as ‘Theta type’ suggests the drowsiness and
arousal in older children and Adults. Similarly frequency range of 8-12 Hz known as
‘Alpha type’ is associated with closing of eyes and state of relaxation. Frequency range
Author's copy - produced at Lamar University
Bijaya 45
of 12-30 Hz known as ‘Beta type’, corresponds to the activeness, alertness of mind
(Kirmizi-Alsan, et al. 2006). Since our EEG data is of alcoholic patient it is expected to
have theta type EEG signal along with delta (sleeping), alpha (relaxed, closing eyes) and
beta (alert) types. Figure 7-5 shows that magnitude of delta, theta and alpha type EEG
signal are higher than beta type, which clearly suggests the patient being less active.
7.5 Performance Comparison for different model orders
Similar experiments were performed for the model orders 3, 5 and 7. The results
of all three orders are compared in Figure 7-6 for the same 0.2 Seconds (51 samples)
fragment of EEG.
Figure 7-6 EEG spectral estimates for different model orders
Author's copy - produced at Lamar University
Bijaya 46
As seen in Figure 7-6, an increase in the model order results in spectral estimates
of a seemingly higher resolution. In order to recommend the ARMAX model over other
parametric or DFT-based spectral estimation methods, the results above were compared
with the Modified Covariance AR estimator, the ARMA model, and the periodogram
method. The procedure of the parameter estimation of Modified Covariance method is
described in Appendix II.
7.6 Comparison with an ARMA model
ARMA models have been used by many researchers to estimate the EEG
spectrum for various applications. The ARMA is a special case of an ARMAX model. By
keeping an exogenous input equal to zero, the same procedure used for the parameter
estimation of ARMAX was implemented to estimate the parameters of an ARMA model.
Once the parameters were found, the spectral estimation was performed similarly to the
ARMAX spectral estimator. The final result obtained for the third order ARMA model is
compared with the result of the third order of the ARMAX model.
Author's copy - produced at Lamar University
Bijaya 47
Figure 7-7 EEG spectral estimates by ARMA and ARMAX methods
Figure 7-7 shows that an estimate based on an ARMA model may seem less
informative than one of an ARMAX model because the ARMA model-based estimator
has depicted only the general outline of the power distribution. Whereas, the ARMAX
model-based estimator was able to distinguish between different components of the
spectrum, and higher power is observed at lower frequencies. EEG contains most of its
information in lower frequencies; therefore, a better resolution at lower frequencies is
desirable for an EEG analysis. The spectral estimates obtained by the third order ARMA
model are insufficient to provide enough information to resolve these components in the
spectrum. Spectral estimations using ARMA models with different orders were
conducted next, and the results are presented in Figure 7-8.
Author's copy - produced at Lamar University
Bijaya 48
Figure 7-8 Different Order ARMA Model.
As seen in Figure7-8, when the model order increases, spectral resolution
becomes better. However, we may conclude that the resolution of these spectral estimates
is still insufficient. Tseng suggested that the higher order ARMA models are needed to
represent an EEG signal (Tseng, Chong and Kuo 1995). Our results agree with Tseng’s
conclusion. Therefore, the ARMAX model is deemed more appropriate for an EEG
analysis than the ARMA model.
Author's copy - produced at Lamar University
Bijaya 49
7.7 Comparison with an AR model and with the Periodogram method.
Next, the comparison of an ARMAX-based spectral estimator was performed
with an AR model and the periodogram method. The modified covariance technique was
used to estimate the coefficients of the AR model. Comparisons were conducted for
different orders of AR and ARMAX models using different fragments of EEG sequence.
Author's copy - produced at Lamar University
Bijaya 50
a) Fragment 1.
Order of AR model =3
Order of ARMAX model=3
Start time= 23 Seconds.
Data length = 0.13 Seconds
Number of samples= 34.
Figure 7-9 shows a comparison of power spectral densities obtained using
periodogram, AR and ARMAX model of fragment 1.
Figure 7-9 PSD Comparison for Order 3
Author's copy - produced at Lamar University
Bijaya 51
b) Fragment 2.
Order of AR model =5
Order of ARMAX model=5
Start time= 33 Seconds.
Data length = 0.1 Seconds
Number of samples= 26
Figure 7-10 shows a comparison of power spectral densities obtained using
periodogram, AR and ARMAX model of fragment 2.
Figure 7-10 PSD Comparison for Order 5
Author's copy - produced at Lamar University
Bijaya 52
c) Fragment 3.
Order of AR model =7
Order of ARMAX model=7
Start time= 60 Seconds.
Data length = 0.9 Seconds.
Number of samples= 24.
Figure 7-11 shows a comparison of power spectral densities obtained using
periodogram, AR and ARMAX model of fragment 3.
Figure 7-11 PSD Comparison for Order 7
Author's copy - produced at Lamar University
Bijaya 53
Spectral estimates obtained using different estimators are not equal to the true
spectrum of a signal. However, they resemble the true spectrum. Since, the research is
concerned in obtaining clear information of power distribution along frequency domain
an estimator that gives a high spectral resolution is considered better than the estimator
with a low spectral resolution. Figures 7-9, 7-10, and 7-11 indicates that the spectral
estimates obtained by an ARMAX model are somewhat similar to the spectral estimates
obtained using other methods. On the other hand, the ARMAX-based estimates show a
higher resolution. The AR method has only depicted general trends in power distribution.
Similar to ARMA, the low-order AR models are barely suitable for an EEG spectral
analysis. A DFT-based periodogram method shows a higher-resolution estimate of power
distribution. However, a close inspection reveals that the periodogram at lower
frequencies is unable to resolve different spectral components. The lack of distinct
components at lower frequencies would make the periodogram not ideal for EEG
analysis. On the other hand, spectral estimates obtained with the ARMAX model show a
higher spectral resolution even at lower frequencies. At higher frequencies, the ARMAX-
based estimates were very similar to the periodogram-based estimates.
Author's copy - produced at Lamar University
Bijaya 54
CHAPTER 8
Conclusions and Future Work
8.1 Conclusions
In this thesis, an ARMAX model based on a parametric method of spectral
estimation was successfully developed and applied to the analysis of EEG signals.
Coefficients of the ARMAX model were estimated by using a novel optimization
technique, known as particle swarm optimization. The coefficients were designed,
implemented, and verified with different types of data sets. This optimization technique
was found very useful while estimating the coefficients of ARMA and ARMAX models.
The performance of the ARMAX spectral estimator was compared with other parametric
and nonparametric techniques such as: the periodogram method, ARMA-based analyzer,
and an AR-based spectral estimator. The estimates obtained using the ARMAX model
were found superior to other methods, especially in a low frequency diapason. We have
observed that an ARMAX model-based estimator may show higher spectral resolution at
lower frequencies. Finally, it was illustrated that the ARMAX model of a significantly
lower order - compared to ARMA or AR models - may be sufficient for an accurate
spectral estimation.
8.2 Future Work
In the future, the selection of the model order of a proposed method needs to be
optimized. Such an optimization would allow avoiding over-ordering the estimator while
Author's copy - produced at Lamar University
Bijaya 55
maintaining sufficient spectral resolution. Perhaps, traditional order selection criteria
(Akaike or MDL, for instance) may be modified for the ARMAX.
The study of the effects of short data fragments (while using a model parameters
estimation process) is another important area of future research. Reliable spectral
estimation for short data sequences is of great interest in many fields. Therefore, knowing
practical limits of the proposed method (i.e., the minimum duration of data sequence still
yielding acceptable estimates) would be important for its applications.
This research may result in development of a common tool capable for estimating the
spectrum of all the channels of EEG simultaneously. The latter may be used to identify
the characteristics of interest in the EEG.
Finally, the proposed spectral estimation algorithm may also be applied for the
analysis of other naturally-generated signals, such as speech or seismological data.
Appropriate tests would be necessary before recommending the ARMAX model for those
applications.
Author's copy - produced at Lamar University
Bijaya 56
CHAPTER 9
Reference
Akaike H. "A new look at the statistical model identification." IEEE Transactions on
Automatic Control (IEEE Transactions on Automatic Control) 19, no. 6 (1974):
716-723.
Akin A, and Kiymik K. "Application of Periodogram and AR Spectral Analysis to EEG
Signals." Journal of Medical Systems (Springer Netherlands) 24, no. 4 (2000):
247-256.
Angeline P. J. "Using selection to improve particle swarm optimization." International
Conference on Evolutionery Computation. Anchorage, AK: IEEE, 1998. 84-89 .
Aydın S. "Comparison of Power Spectrum Predictors in Computing Coherence Functions
for Intracortical EEG Signals." Annals of Biomedical Engineering, 2008: 192-
200.
Begleiter H. EEG correlates of genetic predisposition to alcoholism. EEG Data, at
Brooklyn: Neurodynamics Laboratory at the State University of New York Health
Center , 1999.
Bingham M. D, Godfrey, and Tukey J. W. "Modern techinques of power spectrum
estimation." IEEE transaction on Audio and Electoacoustics, 1967: 55-66.
Blodin J. "Particle swarm optimization." A tutorial, 2009.
Brown R.G., and Hwang P. Y. C. Introduction to Random Signal Analysis and Kalman
Filtering. John Willey & Sons Inc, 1983.
Author's copy - produced at Lamar University
Bijaya 57
BSA-EE Herald. 2010. http://www.eeherald.com/section/design-guide/dg100008.html
(accessed 05 06, 2010).
Bullock T.H., Enright J.T, and Chong K.M. "Forays with the additive periodogram
applied to the EEG." 5th Joint Symp on Neural Computation. Sa Diego:
University of california, 1998. 25-28.
Candy J. V. Model-Based Signal Processing. New Jersey: John Wiley & Sons, Inc, 2006.
Eberhart R., Kennedy J, and Shi Y. Swarm intelligence. morgan kaufmann, 2001.
Eberhart R., and Kennedy J. "Particle swarm optimisation." IEEE International
Conference on Neural Networks (IEEE Int Conf. Neural Netw) 4 (1995): 1942-
1948.
Eberhart R., and Shi Y "A modified particle swarm optimizer." IEEE. Anchorage, AK,:
IEEE International. Conference on Evolutionary Computation, 1998. 69-73.
EEG- Wiki. 2010. http://bmet.wikia.com/wiki/EEG (accessed 06 02, 2010).
En'ping L., Sheng Z., and Shini Q. "Application of eigenvector estimation and SVM for
EEG Signals Classification." 9th International Conference on Electronic
Measurement & Instruments ICEMI '09. Beijing: IEEE, 2009. 4-182 - 4-185.
Estimation theory. Wikimedia Foundation, Inc. 6 5, 2010.
http://en.wikipedia.org/wiki/Parametric_estimating (accessed 12 4, 2009).
graphy-Wikipedia. 07 3, 2010. http://en.wikipedia.org/wiki/-graphy (accessed 07 07,
2010).
Gudsiz V.G., and Venayagamoorthy G.K. "Comparison of particle swarm optimization
and backpropagation as training algorithms for neural networks." IEEE Swarm
Author's copy - produced at Lamar University
Bijaya 58
Intelligence Symposium 2003 (SIS 2003). Indianapolis, Indiana, USA: IEEE,
2003. 110-117.
Hayes M. H. Statistical Digital Signal Processing and Modeling. John Wiley & Sons Inc,
1996.
Huang C.M., Huang C.J., and Wang M.L. "A Particle Swarm Optimization to Identifying
the ARMAX Model for Short-Term Load Forecasting." IEEE Transactions on
Power Systems (IEEE TRANSACTIONS ON POWER SYSTEMS) 20, no. 2 pp
1126-1133 (2005): 1126-1133.
Instruments, National. Selecting a Model Structure in the System Identification Process.
National Instruments Corporation. 2010.
http://zone.ni.com/devzone/cda/tut/p/id/4028 (accessed 01 05, 2010).
Jenigiri S. "Comparative study of efficiency of genetic algorithms and particle swarm
optimization technique to solve permutation problems." internal Rapport,
Computer Society of India,. Mysore: University of Mysore, 1996. 490-507 .
Jenkins M., and Box G. E. P. "Time series analysis: forecasting and control." Holden-
day. San Francisco, CA: University of Wisconsut. Madison. WI and University of
Lancaster, England, 1970.
Jiao W., Liu G., and Xian L.D. "Elite Particle Swarm Optimization with mutation." 7th
International Conference on -System Simulation and Scientific Computing, 2008.
ICSC 2008. Asia Simulation Conference. Beijing: IEEE, 2008. 800 - 803.
Kennedy, J, and Eberhart R.. "A discrete binary version of the particle swarm algorithm."
International Conference on Systems, Man, and Cybernetics. Piscataway, NJ:
IEEE , 1997. 4104 - 4108.
Author's copy - produced at Lamar University
Bijaya 59
Ljung L, System Identification, Theory for the User. Upper Saddle River, N.J: Prentice-
Hall, 1999.
Ljung L and Glad T. "On global identifiability of arbitrary." Automatica, 1994: 265–276.
Ljung L. "Estimating linear time invariant models of non-linear time-varying systems."
European Journal of control, 2001: 203–219.
Ljung L. "Perspectives on System Identification" Annual Reviews in Control, 2010: 1–12.
Maghsoudi and White C.D. "Real-time identification of parameters of the ARMA model
of the human EEG waveforms." Biomed Sci Instrum. San Antonio: US National
Library of Health, 1993.
Mandic D. "Lecture 2: Nonparametric Spectrum." www3.imperial.ac.uk. 2009.
http://www.commsp.ee.ic.ac.uk/~mandic/SE_ASP_LN/SE_ASP_Lecture_2_Non
parametric_SE.pdf (accessed 06 2010, 25).
Murugesan M and Sukanesh R. "Towards Detection of Brain Tumor in
Electroencephalogram Signals Using Support Vector Machines." International
Journal of Computer Theory and Engineering. IACSIT, 2009. 1793-8201.
Niederhauser J.J, Esteller R, Echauz J, Vachtsevanos G, and Litt B.. "Detection of seizure
precursors from depth-EEG using a sign periodogram transform." IEEE
Transactions on Biomedical Engineering (IEEE transaction on Biomedical
Engineering) 50, no. 4 (2003): 449-458.
Papoulis, A. Probability, Random Variables and Stochastic Processes. McGraw-Hill
Companies, 1991.
Author's copy - produced at Lamar University
Bijaya 60
Paul S. R. Electroencephalography. 03 1, 2006.
http://www.scholarpedia.org/article/Electroencephalogram (accessed 10 01,
2010).
Qihou Z., Matthew B., and Jade M. "Analysis of EEG Data Using an Adaptive
Periodogram Technique." International Conference on BioMedical Engineering
and Informatics (International Conference on BioMedical Engineering and
Informatics) 2 (2008): 353-357.
Rahiman M. H. F., Taib M. N., and Salleh Y. M. "Model order criterions and model order
selection based on data from the essential oil extraction system." 3rd
International Colloquium on Signal Processing and its Applications. Melaka,
Malaysia: CSPA, 2007.
S D-Wiki. Wikimedia Foundation, Inc. 06 05, 2010.
http://en.wikipedia.org/wiki/Spectral_density (accessed 07 4, 2010).
Schuster A. "On the investigation of hidden periodicities with application to a supposed
26 day period of meteorological phenomena." Journal of Geophysical Research
(Terrestrial Magnetism and Atmospheric Electricity, ) 3, 13-41 (1898): 13-41.
Schutte J, F. "The Particle Swarm Optimization Algorithm." Lecture note on Structural
Optimization Fall 2005, 2005.
Spectral Analysis S E M. 2010.
http://www.spectralanalysis.net/spectral_estimation_method.asp (accessed 08 22,
2010).
Author's copy - produced at Lamar University
Bijaya 61
Stoica P. and Eriksson A. "MUSIC estimation of real-valued sine-wave frequencies."
Elsevier (Elsevier North-Holland, Inc) 42, no. 2 Pages: 139 - 146 (1995): 139-
146.
System Identification Toolbox. 2010.
http://www.mathworks.com/access/helpdesk/help/toolbox/ident/ref/armax.html
(accessed 04 12, 2010).
thefreedictionary. 2010. http://www.thefreedictionary.com/encephalo- (accessed 08 26,
2010).
Tseng C. R.C., Chong F.C, and Kuo T.S. "Evaluation of parametric methods in EEG
signal analysis." Medical Engineering and Physics (Medical Engineering &
Physics) 17, no. 1 (1995): 71-78.
Wahba G, and Craven. "Smoothing noisy data with spline functions: estimating the
correct degreeof smoothing by the method of generalized cross-validation."
Numer math, 1979: 377-403.
Wang K.P., Huang L., Zhou C.G., and Pang W. "Particle swarm optimization for
traveling salesman problem." IEEE. Piscataway,NJ,USA: International
Conference on Machine Learning and Cybernetics, 2003. 1583 - 1585.
Weisstein, E. W. Cramer's Rule. 2010. http://mathworld.wolfram.com/CramersRule.html
(accessed 07 22, 2010).
Welch, P.D. "The Use of Fast Fourier Transform for the Estimation of Power Spectra: A
Method Based on Time Averaging Over Short, Modified Periodograms." IEEE
Trans. Audio Electroacoustics (IEEE Trans. Audio Electroacoust) AU-15 Pgs.70-
73 (1967): 70-73.
Author's copy - produced at Lamar University
Bijaya 62
white noise. Wikimedia Foundation, Inc. May 25, 2010 .
http://en.wikipedia.org/wiki/White_noise (accessed July 02, 2010).
Author's copy - produced at Lamar University
Bijaya 63
CHAPTER 10
Appendix
10.1 Modified Covariance Method
“An autoregressive (AR) process xn is represented as the output of an all-pole
filter that is driven by unit variance white noise” (Hayes 1996):
�¡� = 9��>∑ 5¢£¤¢¥¢¦§ Equation 10 -1
The power spectrum of pth
order AR process is
�����¨� = |9�||�>∑ 5¢£¤©¢ª¥¢¦§ | Equation 10-2
If 17 is found, the estimate of the power spectrum may be formed using Equation 10-3.
�«����¨� = |9«�||�>∑ 5y¢£¤©¢ª¥¢¦§ | Equation 10-3
Covariance method of coefficient estimation:
Coefficient 17can be found by solving a set of linear equations presented below
Author's copy - produced at Lamar University
Bijaya 64
¬���1,1� ���2,1� … ���®, 1����1,2� ���2,1� … ���®, 2�⋮ ⋮ ⋱ ⋮���1, ®� ���2, ®� … ���®, ®�± ²1�1�⋮1³´ = ²���0,1����0,2�⋮���0, ®�´ Equation 10-4
Where, ���0, ~� is autocorrelation of x(n) at l shift lag defined by Equation 10-4.
���0, ~� = ∑ xD�µxD�µ∗���4�³ Equation 10 -5
AR parameters in the Modified Covariance method are found by using slightly
different method of computing ���0, ~�.
Here ���~, ®� is computed using Equation 10-6.
���0, ~� = ¶ xD�µxD�µ∗ + xD�·xD�·>¸∗���4�³ Equation 10 -6
Author's copy - produced at Lamar University
Bijaya 65
10.2 Model validation data
Original parameter
A B C
1 -0.2559 -0.2165 0.3316 0.1541 0.1142 1 0.2896 0.1763
Calculated parameters
)v /P -v MSE
Error
1 -0.4232 0.0021 -0.0677 0.4672 0.3228 1 -0.014 0.3944 7.90E-07
1 -0.2669 -0.1365 0.3124 0.1319 0.0856 1 0.1934 0.1167 4.54E-06
1 -0.3627 -0.1492 0.0926 0.072 0.0477 1 0.1396 0.2345 4.23E-06
1 -0.2613 -0.1572 0.0836 0.2721 0.4231 1 0.3801 0.2032 9.80E-07
1 -0.2252 -0.1513 0.287 0.0218 0.2677 1 0.22 0.2973 1.38E-06
1 -0.2685 -0.1831 0.3254 0.252 0.2005 1 0.056 0.157 1.23E-06
1 0.0548 -0.2496 0.2936 0.3207 -0.0032 1 0.3937 0.3029 2.55E-06
1 -0.1512 -0.3913 0.4621 0.311 -0.0737 1 -0.0018 0.362 1.27E-06
1 -0.2407 -0.1211 0.2976 0.1112 0.121 1 0.3043 0.1525 8.69E-06
1 -0.2777 -0.1546 0.3843 0.3218 0.3255 1 0.329 0.1776 2.02E-06
1 -0.2827 -0.1294 0.4505 0.193 0.1635 1 0.2363 0.1396 6.20E-07
1 -0.3184 -0.2581 0.1593 0.1114 -0.1462 1 0.3267 0.1345 3.69E-06
1 -0.2663 -0.1963 0.4013 0.3752 0.3055 1 0.1826 0.1259 2.49E-06
1 -0.1833 -0.1389 0.1681 0.2507 0.2729 1 0.2221 0.293 2.10E-06
1 -0.2476 -0.3464 0.1909 0.1249 0.1927 1 0.1599 0.1855 1.69E-06
1 -0.0647 -0.3346 0.2274 0.1858 0.3278 1 0.2315 -0.1291 2.00E-06
1 -0.2796 -0.0118 0.4399 0.2201 0.2484 1 0.3352 0.2401 3.37E-06
1 -0.0415 -0.213 0.1674 0.3487 0.2082 1 0.2418 0.1424 1.74E-06
1 -0.3133 -0.1276 0.1816 0.2345 0.2598 1 0.1321 0.2209 1.23E-06
1 -0.1737 0.0012 0.246 0.1019 0.0621 1 0.1794 0.3332 2.16E-06
1 -0.334 -0.172 0.4526 0.0419 0.2739 1 0.3057 0.1538 6.60E-07
1 -0.2335 -0.005 0.335 0.2534 0.4041 1 0.238 0.2148 4.47E-06
1 -0.0506 -0.2706 0.2276 0.2849 -0.1544 1 0.3379 0.4486 8.50E-07
1 -0.2285 -0.0547 0.3577 0.1972 0.2091 1 0.305 0.3094 8.30E-07
1 -0.2052 -0.23 0.2951 0.2919 0.1445 1 0.074 0.2237 3.08E-06
1 -0.2057 -0.0989 -0.0628 0.1464 0.0808 1 0.1583 0.2503 1.82E-05
1 -0.2431 -0.097 0.4395 0.3221 0.2123 1 0.3798 0.1942 3.70E-07
1 -0.2014 -0.1597 0.3559 -0.0578 0.3458 1 0.3129 -0.0435 6.58E-06
1 -0.2212 -0.088 0.3364 0.1196 0.1615 1 0.4584 0.2636 1.27E-06
1 -0.4243 -0.0981 0.3182 0.2438 0.1603 1 0.3084 0.266 8.10E-07
1 -0.1033 -0.2001 0.072 0.2345 0.2595 1 0.3956 0.0936 6.67E-06
1 -0.2227 -0.1064 0.185 0.0826 0.1773 1 0.3483 0.2909 4.21E-06
1 -0.2716 -0.1718 0.2614 0.1871 0.1763 1 0.0824 0.0902 8.50E-07
1 -0.3643 -0.2561 0.4286 0.102 -0.3282 1 0.2401 0.3866 5.70E-07
1 -0.1215 -0.1663 0.2238 0.1756 0.0445 1 0.3695 0.1907 1.85E-06
Author's copy - produced at Lamar University
Bijaya 66
1 -0.3266 -0.266 0.4491 0.2763 0.0758 1 -0.0869 0.0644 2.93E-06
1 -0.2083 -0.1694 0.5363 0.3864 0.2578 1 0.1027 0.1173 1.14E-06
1 -0.1177 -0.1737 0.2394 0.4211 0.1353 1 0.1169 0.2549 5.02E-06
1 -0.2573 -0.2142 0.2721 0.2921 0.4155 1 -0.022 0.1186 3.40E-07
1 -0.1705 -0.1543 0.3064 0.1803 0.3609 1 0.2226 0.0372 1.29E-06
1 -0.2138 0.0494 0.0983 0.1232 0.1052 1 0.0509 0.1392 4.06E-06
1 -0.066 -0.1571 0.2643 0.2888 0.1807 1 0.1152 0.4421 3.80E-06
1 -0.2761 -0.1889 0.3722 -0.1797 0.2972 1 0.3768 -0.1622 1.90E-06
1 -0.2581 -0.2215 0.3345 0.2453 0.1651 1 0.0764 0.1981 8.30E-07
1 -0.3479 -0.1813 0.3109 0.4092 -0.0042 1 -0.0695 0.2035 1.38E-06
1 -0.2142 -0.245 0.226 0.1881 0.1887 1 0.2264 0.2622 1.16E-06
1 -0.2062 -0.0494 0.3374 0.0693 -0.0011 1 0.1763 0.4417 3.02E-06
1 -0.2885 -0.136 0.3395 0.2799 0.0388 1 0.0331 0.1445 3.73E-06
1 -0.1945 -0.0584 0.3527 0.2887 0.098 1 0.2658 0.251 2.73E-06
1 -0.4927 -0.1012 0.3129 0.1961 0.1906 1 0.0132 0.2162 9.70E-07
1 -0.1101 -0.086 0.2824 0.3244 0.1515 1 0.4651 0.2309 1.14E-06
1 -0.3601 -0.0696 0.3847 0.1065 0.1966 1 0.0104 0.2774 3.88E-06
1 -0.2934 -0.0266 0.1766 0.0664 0.2651 1 0.2779 0.459 3.50E-06
1 -0.1421 -0.3131 0.4044 0.3607 0.1331 1 0.2163 0.2176 2.74E-06
1 -0.2191 -0.2019 0.3368 0.1402 0.1725 1 0.0849 0.3294 5.50E-07
1 -0.2708 -0.1214 0.25 -0.0838 0.3578 1 0.321 0.2095 1.09E-06
1 -0.2763 -0.2336 0.4102 0.3595 -0.0169 1 0.1605 0.0497 2.51E-06
1 -0.2503 -0.1272 0.2344 0.1777 0.0245 1 0.4227 0.3672 4.14E-06
1 -0.274 -0.1675 0.2896 0.1102 -0.1795 1 -0.0744 0.2651 4.85E-06
1 -0.3395 -0.2213 0.2855 0.2107 -0.0688 1 0.2748 0.0668 1.49E-06
1 -0.0647 -0.3479 0.1748 0.2749 0.1581 1 0.2982 0.1778 1.14E-06
1 -0.2089 -0.309 0.1488 0.3011 0.2507 1 0.3536 -0.1034 8.50E-07
1 -0.2117 -0.2827 -0.017 0.4557 0.3089 1 0.2645 0.2419 3.10E-06
1 -0.1804 -0.0574 0.3159 0.2402 0.2505 1 0.2152 0.1979 1.63E-06
1 -0.1697 -0.1303 0.3348 0.1275 0.2396 1 0.2182 0.2762 2.44E-06
1 -0.3934 -0.168 0.1679 0.3404 0.2194 1 -0.0384 0.0981 4.05E-06
1 -0.0427 -0.3704 0.0425 0.4029 0.0262 1 0.2851 0.2577 1.50E-06
1 -0.3161 -0.0768 0.3532 0.0938 -0.026 1 0.2752 0.1823 3.53E-06
1 -0.2918 0.182 0.0869 0.1178 0.5084 1 0.3329 0.164 4.83E-06
1 0.1246 -0.1851 -0.0912 0.4266 0.1289 1 0.2201 0.2705 2.04E-06
1 -0.1481 -0.1029 0.0963 0.3644 0.4425 1 0.1548 0.1746 1.48E-06
1 -0.1203 -0.1254 0.291 0.1962 0.1567 1 0.3526 0.3883 8.00E-07
1 -0.2658 -0.2443 0.4261 0.1387 0.2924 1 0.4114 0.1364 7.80E-07
1 -0.394 0.0153 0.1129 0.2352 0.2837 1 0.0368 0.0296 3.06E-06
1 -0.2643 -0.1864 0.3664 0.1292 -0.0715 1 -0.0868 0.0036 1.87E-06
1 -0.2464 -0.1596 0.2285 0.358 0.0878 1 0.1229 0.0433 2.07E-06
1 -0.2447 -0.2584 0.2548 0.3947 0.0711 1 0.1746 0.2808 1.37E-06
1 -0.2594 -0.0782 0.3213 0.2401 0.2977 1 0.0224 0.0116 2.70E-06
1 -0.1584 -0.1805 -0.0188 0.3855 0.1988 1 0.298 0.2911 9.80E-07
1 -0.3779 -0.1716 0.3979 -0.0466 0.2143 1 0.2017 0.3523 1.00E-06
1 -0.14 -0.3255 0.1902 -0.0317 0.3965 1 0.3076 -0.2083 2.78E-06
Author's copy - produced at Lamar University
Bijaya 67
1 -0.2942 -0.1173 0.1582 -0.1898 0.4259 1 0.3108 0.3565 2.18E-06
1 -0.3483 -0.2246 0.2737 0.0385 -0.087 1 0.27 0.4246 4.11E-06
1 -0.1706 -0.1707 0.2167 0.3927 0.0089 1 0.0691 0.2426 1.44E-06
1 -0.1851 -0.184 0.2991 0.2795 0.0585 1 0.3033 0.317 1.37E-06
1 -0.1351 -0.1902 0.1976 0.3616 -0.0986 1 0.2892 0.3352 9.10E-07
1 -0.2416 -0.1982 0.3262 0.0242 0.1713 1 0.4756 0.1756 4.51E-06
1 -0.1546 -0.0644 0.2075 0.415 0.4179 1 0.3003 0.1008 1.06E-06
1 -0.1281 -0.2326 0.294 0.253 0.0909 1 0.2113 -0.0758 3.15E-06
1 -0.2153 -0.1466 0.0114 0.075 0.1419 1 0.2396 0.3055 8.30E-07
1 -0.2185 -0.3155 0.3708 0.1964 0.0308 1 0.482 -0.1052 2.03E-06
1 -0.168 -0.2442 0.0391 0.2377 0.3429 1 0.0213 0.0549 1.21E-06
1 -0.3965 -0.0474 0.4189 0.0802 0.1569 1 0.1597 0.2761 2.42E-06
1 -0.3378 -0.2568 0.3865 0.3608 0.2716 1 0.2252 -0.2687 1.74E-06
1 -0.2987 -0.0679 0.2769 -0.0133 -0.0366 1 0.2053 0.4378 3.53E-06
1 -0.1052 -0.2972 0.1159 0.0787 0.2906 1 0.3528 0.2463 1.36E-06
1 -0.3129 -0.0095 0.4474 0.3099 0.205 1 0.0726 0.0867 2.34E-06
1 -0.2311 -0.002 0.1099 0.3496 0.2476 1 0.2746 0.4717 9.00E-07
1 -0.0679 -0.247 0.2314 0.0495 0.2301 1 0.2389 0.0629 1.93E-06
1 -0.2124 -0.0836 0.3697 0.0622 0.2616 1 0.3602 0.3699 3.00E-07
Average MSE 2.42E-06
Top Related