EEMD-LSSVR-based Decomposition-and-Ensemble Methodology with Application to Nuclear Energy Consumption Forecasting

Ling Tang, Shuai Wang Institute of Policy and Management, Chinese Academy

of Sciences Beijing 100190, China

Graduate University of Chinese Academy of Sciences, Beijing 100049, China

e-mail:lingziyu3@163.com

Lean Yu Institute of Systems Science, Academy of Mathematics

and Systems Science, Chinese Academy of Sciences Beijing 100190, China

e-mail: yulean@amss.ac.cn

Abstract—Based on the principle of “decomposition and ensemble” and strategy of “the divide and conquer” [1,2], a hybrid Methodology integrating ensemble empirical mode decomposition (EEMD) and least squares support vector regression (LSSVR) is proposed for nuclear energy consumption forecasting. In the proposed EEMD-LSSVR-based Decomposition-and-Ensemble Methodology, the EEMD is first applied to decompose the original data of nuclear energy consumption into a number of independent intrinsic mode functions (IMFs). Then the LSSVR is implemented to predict all the extracted IMFs independently. Finally, the predicted IMFs are aggregated into an ensemble result as the final prediction using another LSSVR. The empirical results demonstrate that the novel methodology can strikingly outperform some other popular forecasting models both in level forecasting accuracy and in direction prediction accuracy.

Keywords- Nuclear energy consumption; forecasting; Ensemble empirical mode decomposition; Least squares support vector regression

I. INTRODUCTION Facing with the dilemma between energy shortage and

environmental protection, nuclear energy with its unique merits, particularly in environmental protection, becomes one of the most promising energy options for Chinese government [3]. What’s more, Chinese government is launching an ambitious nuclear energy development program. It is known as the medium- and long- term nuclear energy development plan (2005-2020), in order to increase nuclear energy generation capacity to 40 million kilowatts and the nuclear energy generation is expected to arrive at 260-280 billion kilowatt hours per year by 2020 [4]. Therefore, an accurate forecasting of nuclear energy consumption is urgently needed under such a background.

There are quite few studies on nuclear energy consumption forecasting. Even in recent related research, prediction results of nuclear energy are usually obtained using scenario analysis method without numeric models, e.g., in the study of Comsan (2010) [5] and Besmann (2010) [6]. The difficulty in modeling nuclear energy consumption mainly results from its intrinsic complexity and irregularity, in terms of the involving factors, such as various electricity

market factors, investment supports, technology progress, infrastructure improvement and so on [7-9].

Fortunately, these complicated forecasting tasks can be partially solved by using the “decomposition and ensemble” principle (or “divide and conquer” strategy), introduced by Wang et al. (2005) [1] and Yu et al. (2008) [2]. Intrigued by this principle, this study attempts to propose a hybrid methodology integrating ensemble empirical mode decomposition (EEMD) and least squares support vector regression (LSSVR) for Chinese nuclear energy consumption forecasting. In the EEMD-LSSVR-based Decomposition-and-Ensemble methodology, the EEMD paradigm, as a efficient decomposition method, is first applied to divide the original data of nuclear energy consumption into a number of independent intrinsic mode functions (IMFs) of original data. Then the LSSVR, as a powerful forecasting tool both in prediction accuracy and time saving, is applied to predict different IMFs independently. Finally, all the predicted IMFs obtained in the previous phases are fused into an ensemble result as the final prediction via another LSSVR.

This study aims to propose the EEMD-LSSVR-based Decomposition-and-Ensemble methodology, and apply it to forecast the nuclear energy consumption in China. The rest of the paper is organized as follows: section II formulates the methodology; the empirical study is section III; section IV concludes.

II. METHODOLOGY FORMULATION In this section, the EEMD-LSSVR based Decomposition-

and-Ensemble methodology is proposed. First, the EEMD technique and LSSVR algorithm are briefly introduced. Then the EEMD-LSSVR-based Decomposition-and-Ensemble methodology is formulated, and accordingly its overall steps of the proposed novel hybrid Decomposition-and-Ensemble methodology are finally summarized.

A. Ensemble Empirical Mode Decomposition (EEMD) In the novel Decomposition-and-Ensemble methodology,

ensemble empirical mode decomposition (EEMD) paradigm is employed as a competitive decomposition method. THE EEMD paradigm which proposed by Wu and Huang (2004) [10] is an improved version of empirical model decomposition (EMD) first proposed by Huang et al (1998) [11]. It is proposed to overcome the intrinsic drawbacks of mode mixing in the EMD method.

2011 Fourth International Joint Conference on Computational Sciences and Optimization

978-0-7695-4335-2/11 $26.00 © 2011 IEEE

DOI 10.1109/CSO.2011.304

589

Unlike other traditional decomposition methodologies, EEMD methods are empirical, intuitive, direct and self-adaptive data processing methods which are designed especially for nonlinear and non-stationary data. EEMD can decompose the original data into a series of independent and nearly periodic intrinsic modes based on local characteristic scale, by which the concrete implication of each mode can be identified. Finally, the data series can be decomposed into several empirical mode functions (IMF) and one residue:

, ,1

nt j t n tj

x c r=

= +� (1)

where rn,t indicates the final residue, which presents the main trend of original data xt, and cj,t (j = 1, 2, …, n) is the jth IMF. There are n IMFs in total.

Since the seminal work of EMD and EEMD is published, EMD and EEMD have been widely used for some complex objects [2,12,13].

B. Least Squares Support Vector Regression (LSSVR) In contrast to other forecasting approaches, support

vector machines (SVM), which was firstly proposed by Vapnik in 1995 based on the principle of structural risk minimization, has been proved to possess its own excellent capabilities in classification or prediction even for small sample by minimizing an upper bound of the generalization error [14]. Nevertheless, SVM training is a time consuming process when analyzing huge data. For this purpose, least squares support vector machine (LSSVM) is proposed to overcome these shortcomings [15]. Generally speaking, LSSVM can be categorized into LSSVR and LSSVC respectively for regression and classification purposes. Here the LSSVR is used for a powerful prediction tool.

In LSSVR, the basic idea is first to map the original data x into a high-dimensional feature space via a nonlinear mapping function �(•) and then to make linear regression in this high-dimension feature space. Usually, the LSSVR regression function can be formulated as follows,

( )Tx w x bϕ= +� (2) where �(x) is called the nonlinear function mapping from

input space x into a high-dimensional feature space, and x� is the estimated value. Coefficients wT and b are obtained by minimizing the upper bound of generalization error, which can be transformed into the following optimization problem:

2

1

1 1min2 2

. . ( ) , ( 1,2,..., )

TT

tt

Tt t t

w w

s t x w x b t T

γ ξ

ϕ ξ=

+

= + + =

��

(3)

where tξ are the slack variables and γ the penalty parameter. By introducing Lagrangian function and the KKT conditions for optimality, the final solution of the primal problem can be represented in the following form:

1

( , )T

t tt

x w K x x b=

= +�� (4)

In Eq. (5), K(•) is the so-called kernel function which can simplify the use of a mapping. Usually any symmetric kernel function K(•) satisfying Mercer’s condition corresponds to a

dot product in some feature spaces. Gaussian RBF, ( )2( , ) exp 2t tK x x x x σ= − with γ width of �, may be one

of the most popular used kernel functions.

C. EEMD-LSSVR-based Decomposition-and-Ensemble Methodology Given that there is a time series xt, (t = 1, 2, …, T) we

would like to make m-step ahead prediction, i.e., xt+l. Depending on the previous techniques, a three-step EEMD-LSSVR-based Decomposition-and-Ensemble methodology can be formulated, as illustrated in Fig. 1.

Figure 1. Overall framework of the EEMD-LSSVR-based

Decomposition-and-Ensemble methodology.

As shown in Fig. 1, the proposed EEMD-LSSVR-based Decomposition-and-Ensemble methodology can be generally composed of the following three main steps:

(1) The original time series xt, is decomposed into n intrinsic mode function (IMF) components, cj,t, (j = 1,2,…,n), and one residual component rn,t using the EEMD algorithm.

(2) For each extracted IMF components and the residue component, the LSSVR is used as a single forecasting tool to model each decomposed component.

(3) The prediction results of all IMF components and the residue component are combined to generate an aggregated output, which can be seen as the final prediction tx� for the original time series xt, using another LSSVR model here as an ensemble tool.

To summarize, the proposed EEMD-LSSVR-based Decomposition-and-Ensemble methodology can be abbreviated as an “EEMD(decomposition)-LSSVR(single prediction)-LSSVR(ensemble prediction)” Decomposition-and-Ensemble methodology, based on the principle of “decomposition and ensemble”.

III. EXPERIMENTAL STUDY For illustration and comparison purpose, the nuclear

energy consumption in China is used as testing target, and some other popular single forecasting approaches are

590

employed as benchmark models to compare with the EEMD-LSSVR-based Decomposition-and-Ensemble methodology.

A. Data Descriptions and Experiment Design The sample data are monthly data of nuclear energy

consumption in China, covering the period from March 1993 to January 2010 with a total of 203 observations which are obtained from Wind Database (http://www.wind.com.cn/) and originally collected by National Bureau of Statistics, P. R. China, , as shown in Fig.2.

Figure 2. The original monthly nuclear power consumption series in China (Period: 03/1993-01/2010).

The sample data are divided into two subsets: training subset and testing subset. We treat the data from March 1993 to December 2006 as the training set, including 166 observations, which is used for model training purpose. Similarly, the data from January 2007 to January 2010 with a total of 37 observations is treated as the testing set, which is used to evaluate the performance of prediction. It is worth noting that not only one-step-ahead prediction is performed to assess the short-term prediction performance, but also three-step-ahead prediction and six-step-ahead prediction are performed to test the medium- and long-term forecasting capability of the proposed hybrid Decomposition-and-Ensemble methodology.

For comparison purpose, some other popular single forecasting approaches such as Auto-Regressive Integrated Moving Average (ARIMA) model [16], single SVR, single Artificial Neural Networks (ANN) [17], and some variants of Decomposition-and-Ensemble approaches with other decomposition methods, e.g. EMD, and other ensemble method, e.g. simple addition (ADD), are employed as benchmark models to compare with the proposed EEMD-LSSVR-based Decomposition-and-Ensemble methodology.

To measure the forecasting performance, three main criteria are used for prediction accuracy evaluation. Firstly, the root mean squared error (RMSE) and the mean absolute percent error (MAPE) are selected to appraise the accuracy of level prediction. Typically, the RMSE and MAPE can be represented as

( )2

1

1 Mt tt

RMSE x xM =

= −� � , 1

1 M t tt

t

x xMAPEM x=

−= �� (5)

where M is the number of observations in testing dataset. Inaddition, the predict movement direction accuracy can

be measured by a directional statistic (Dstat) [2], which can be expressed as

1

1 100%M

stat tt

D aM =

= � (6)

where ta =1 if +1 1( )( ) 0t t t tx x x x+− − ≥� , and ta =0 otherwise.

B. Experimental Results In terms of the experiment design and the methodology

formulation, the proposed EEMD-LSSVR-based Decomposition-and-Ensemble methodology is first used to perform numerical experiments.

In the proposed EEMD-LSSVR-based paradigm, the first step is to apply EEMD algorithm to decompose the original nuclear energy consumption data series into several independent IMF components and one residue. In our study, the ensemble member is set to 100 and the added white noise in each ensemble member has a standard deviation of 0.2. The threshold and tolerance levels of the stop criterion are specified to [threshold, threshold2, tolerance] = [0.05, 0.5, 0.05]. Fig. 3 illustrates the decomposition results of Chinese nuclear energy consumption using EEMD in the order from the highest frequency to the lowest frequency.

Figure 3. The IMFs and Residue for Chinese nuclear energy consumption data through EEMD

The second and third steps of the EEMD-LSSVR based paradigm are the individual forecasting and ensemble forecasting in terms of LSSVR. In each LSSVR model, the most popular kernel function, Gaussian RBF, is selected. According to Francies and Cao (2001), 5-fold cross validation grid search method and the trail-and-error approach are utilized to determine the optimal parameters values of γ and �2 which produce the smallest error in the training set.

For comparison purpose, other benchmarks are also used. In single ANN prediction models, a FNN-based ANN (I-H-O) is built in this study using seven hidden nodes, one output neuron and I input neurons where I is referred to the lag orders of the predicted data series [17]. The ANN models are iteratively run 10000 times to train the model using the training subset. In the single SVR and LSSVR models, the kernel function, Gaussian RBF, is also selected, and the values of parameters, i.e., γ , �2 are the same in the determination method as the LSSVR of the proposed EEMD-LSSVR-based Decomposition-and-Ensemble methodology. In addition, the specified value of ε in SVR is

591

set to 0.1. In the ARIMA (p-d-q) model, the best ARIMA model for each training sample is determined, based on Schwarz Criterion (SC) minimization, but the maximum p and q are only restricted to lag orders 5.

Figs 4-6 show the performance comparisons between different models in all prediction horizons, respectively in terms of the MAPE, RMSE and Dstat criteria.

Figure 4. Performance comparisons for different methods in terms of

MAPE criteria

Figure 5. Performance comparisons for different methods in terms of RMSE criteria

Figure 6. Performance comparisons for different methods in terms of Dstat criteria

From Figs 4-6, it can be consistently concluded that the forecasting performances of the proposed EEMD-LSSVR-based Decomposition-and-Ensemble methodology (i.e., EEMD-LSSVR-LSSVR) significantly outperforms all other forecasting methodologies employed in Chinese nuclear energy consumption forecasting. In all models listed in this

study, the proposed EEMD-LSSVR based learning paradigm do not only achieve the highest accuracy in the level measurement, which is measured by the RMSE and MAPE criteria, but also get the highest hit rate in direction decision, which is measured by Dstat criterion, indicating that our proposed EEMD-LSSVR based methodology can be proved to perform the best in the cases of nuclear energy consumption forecasting in both short-, medium- and long- term predictions.

In the case of level prediction accuracy, the results of RMSE and MAPE criteria show that the EEMD-LSSVR based ensemble paradigm performs the best in all cases, followed by the EEMD-LSSVR-ADD model, EMD-LSSVR-LSSVR model, EMD-LSSVR-ADD model, and then the single AI models. The poorest model is the single ARIMA model. It is worth noticing that the RMSE and MAPE results of the four hybrid Decomposition-and-Ensemble models (i.e., EEMD-LSSVR-LSSVR, EEMD-LSSVR-ADD, EMD-LSSVR-LSSVR and EMD-LSSVR-ADD) are all significantly better than those of the four single models (i.e., LSSVR, SVR, ANN and ARIMA) without exception in all of the three different prediction horizons (i.e., one-, three-, and six-step-ahead forecasting). The main reason could be that the decomposition strategy does effectively improve the prediction performance.

Focusing on the four Decomposition-and-Ensemble methods, we further discuss the level prediction performances of different decomposition and ensemble methods, respectively. When comparing the decomposition methods, the proposed EEMD-LSSVR-LSSVR algorithm can gain better prediction accuracy than the EMD-LSSVR-LSSVR and EMD-LSSVR-ADD methods in all cases. This demonstrates that the EEMD algorithm is much more efficient in decomposition than the EMD algorithm. In the comparison of ensemble strategy, the performances of the EEMD-LSSVR-LSSVR and EMD-LSSVR-LSSVR methods are mostly better than those of the EEMD-LSSVR-ADD and EMD-LSSVR-ADD in terms of both RMSE and MAPE criterion, indicating that the LSSVR is a more powerful ensemble method.

In the case of single prediction models, the ARIMA model mostly ranks the last in the all models, while the AI methodologies (i.e., LSSVR, SVR and ANN methods) can gain far better results in terms of level prediction measurement. The possible reason is that ARIMA is a typical linear model, which is not suitable for capturing the nonlinear patterns hiding in nuclear energy consumption with high volatility and irregularity. It can be also found that the level prediction accuracy of LSSVR is mostly the highest among the three AI models, followed by ANN and SVR method. However, such superiority is not as significant as that when comparing AI approaches with ARIMA model.

However, a high hit rate in forecasting direction is also a crucial measure to estimate the prediction performance of models, where the Dstat measurement is implemented. From the estimation results illustrated also in the Figs 6, some similar conclusions can be easily found. Firstly, the proposed EEMD-LSSVR-based Decomposition-and-Ensemble approach perform significantly better than all other models in

592

all cases, followed by the other three Decomposition-and-Ensemble models, then the single AI models, and lastly the ARIMA model in terms of Dstat criterion. Secondly, the four Decomposition-and-Ensemble methods mostly outperform the single prediction models. Thirdly, the Decomposition-and-Ensemble methods with EEMD decomposition method or with LSSVR ensemble method are better than their counterpart methods with EMD method or ADD method. Finally, though there is relatively little difference between the four signal methods, we can also find that the Dstat rank of the ARIMA model is the lowest without exception.

Interestingly, there exist a few exceptions. For example, in the one-step-ahead and three-step-ahead predictions, the MAPE results of the EMD-LSSVR-LSSVR model are slightly worse than those of EMD-LSSVR-ADD; in the three-step-ahead prediction, the single ARIMA model performs a little better than AI methods in terms of RMSE criterion; in the three-step-ahead prediction, the Dstat value of SVR model is somehow higher than some Decomposition-and-Ensemble models. The reasons leading to these results are still unknown, which is worth further exploring in the future research.

Generally speaking, from the analysis of the prediction experiments presented in this study, we can draw the following four main conclusions, as follows.

(1) The empirical prediction results show that the proposed EEMD-LSSVR-based Decomposition-and-Ensemble methodology is significantly superior to the other methods list in the study in terms of both level measurement and direction measurement.

(2) The Decomposition-and-Ensemble methods perform strikingly better than the single models, indicating that the strategy of “decomposition and ensemble” can effectively improve the prediction performance in the case of nuclear energy consumption.

(3) Due to the nonlinear and non-stationary appearance of nuclear energy consumption, the nonlinear models are more suitable for prediction than linear methodologies. Thus, this leads to the fourth conclusion.

(4) The proposed EEMD-LSSVR-based Decomposition-and-Ensemble methodology, which is of effective decomposition and powerful prediction, can be used as a promising solution for the nuclear energy consumption forecasting.

IV. CONCLUSIONS Due to the intrinsic complexity of nuclear energy

consumption in terms of its interactive involving factors, a hybrid Decomposition-and-Ensemble methodology integrating EEMD and LSSVR based on the principle of “decomposition and ensemble” is proposed for the nuclear energy consumption forecasting. In the terms of empirical study, we find that the proposed EEMD-LSSVR-based Decomposition-and-Ensemble methodology can improve the

prediction performance significantly and can statistically outperform other popular used methods listed in this study in terms of both the level accuracy and direction accuracy measurement. This indicates that the proposed EEMD-LSSVR-based Decomposition-and-Ensemble methodology with effective decomposition, nonlinear prediction and ensemble can be used as a very promising methodology for complex forecasting problems, such as nuclear energy consumption forecasting with high volatility and irregularity.

REFERENCES [1] S.Y.Wang, L.Yu, and K. K.Lai, Crude oil price forecasting with

TEI@I methodology. Journal of Systems Sciences and Complexity, vol.18 (2), 2005, pp.145-166.

[2] L.Yu, S.Y.Wang, and K. K. Lai, Forecasting crude oil price with an EMD-based neural network ensemble learning paradigm. Energy Economics, vol. 30 (5), 2008, pp.2623-2635.

[3] M. M.Abu-Khader, Recent advances in nuclear power: A review. Progress in Nuclear Energy, vol. 51 (2), 2009, pp. 225-235.

[4] National Bureau of Statistics, P. R. China, China Energy Statistical Yearbook 2008. China Statistics Press, Beijing (in Chinese), 2008.

[5] M. N. H. Comsan, Nuclear electricity for sustainable development: Egypt a case study. Energy Conversion and Management, vol. 51 (9), 2010, pp. 1813-1817.

[6] T. M. Besmann, Projections of US GHG reductions from nuclear power new capacity based on historic levels of investment. Energy Policy, vol. 38 (5), 2010, pp. 2431-2437.

[7] A. H. Ghorashi, Prospects of nuclear power plants for sustainable energy development in Islamic Republic of Iran. Energy Policy, vol. 35 (3), 2007, pp. 1643-1647.

[8] Y. C. Xu, Nuclear energy in China: Contested regimes. Energy, vol. 33 (8), 2008, pp. 1197-1205.

[9] Q. Wang, China needing a cautious approach to nuclear power strategy. Energy Policy, vol. 37 (7), 2009, pp. 2487-2491.

[10] Z.Wu, and N. E. Huang, Ensemble empirical mode decomposition: a noise-assisted data analysis method. Centre for Ocean-Land-Atmosphere Studies, Technical Report No.193. 51, 2004.

[11] N. E. Huang, Z.Shen, and S. R Long, The empirical mode decomposition and the Hilbert spectrum for nonlinear and nonstationary time series analysis. Proceedings of the Royal Society , vol 454A, 1998, pp. 903-99.

[12] X. Zhang, K. K. Lai, S. Y. Wang, A new approach for crude oil price analysis based on Empirical Mode Decomposition. Energy Economics, vol. 30 (3), 2008, pp. 905-918.

[13] Q. W. Xie, X. Bo, S. L. Peng, J. P. Li , W. X.Xu, and H. Han, Bandwidth Empirical Mode Decomposition and its Application. International Journal of Wavelets, Multiresolution and Information Processing, vol. 6(6), 2008, pp.777-798.

[14] V. Vapnik, The nature of statistical learning theory. Springer, Verlag, New York, 1995.

[15] J. A. K. Suykens and J.Vandewalle, Least squares support vector machine classifiers. Neural Processing Letters, vol. 9 (3), 1999, pp. 293–300.

[16] G. E. P. Box, and G. Jenkins, Time series analysis: Forecasting and control, first ed. Holden Day Press, San Francisco, 1970.

[17] H. White, Connectionist nonparametric regression: multilayer feedforward networks can learn arbitrary mappings. Neural Networks, vol. 3, 1990, pp. 535-549.

593