Practical implementation of a hybrid fuzzy neural network for one-day-a head load forecasting

D. Srinivasan S. S. Ta n C.S.Chang E. K. Cha n

Indexing terms: Fuzzy neurul networks, Loud forecusting

Abstract: The paper presents the development and practical implementation of a hybrid short- term electrical load forecasting model for a power system control centre. This hybrid architecture incorporates a Kohonen self-organising feature map with unsupervised learning for classification of daily load patterns, a supervised backpropagation neural network for mapping the temperature/load relationship, and a fuzzy expert system for postprocessing of neural network outputs. This load forecaster requires minimum operator intervention and can be trained adaptively on-line. The developed model has been tested extensively in the actual operating environment and has been shown to outperform the existing regression-based model.

1 Introduction

Accurate prediction of hourly load demand with lead times ranging from 1 h to several days is important for various economic operations and planning activities in modern power systems. The amount of savings in the operating costs and the reliability of the power supply depend on the degree of accuracy of load predictions.

Several methods have been investigated for short- term load forecasting and the enhancement of its over- all accuracy over a wide range of operating conditions. These methods include statistical methods, regression- causal methods, autoregressive methods and pattern recognition-based methods [l, 21. Utilities around the world have been using powerful models employing one or more of these techniques for short-term load fore- casting. However, accurate modelling is difficult due to the nonlinear and complex relationship between load and the factors it depends on. Sophisticated statistical techniques are frequently needed to effectively capture the load deviations due to sudden weather changes and special events.

~ ~

0 IEE, 1998 IEE Proceedings online no. 19982363 Paper first received 26th February 1997 and in revised form 30th April 1998 D. Srinivasan, S.S. Tan and C.S. Chang are with the Department of Electrical Engineering, National University of Singapore, Singapore 1 19260 E.K. Chan is with Poer Automation Re. Ltd., 89 Science Park Drive #04-13, Singapore 118261

IEE Proc.-Gener. Transm. Distrib., Vol. 145, No. 6, November 1998

Artificial neural networks with their capability to form complex nonlinear mappings between uncorre- lated inputs and outputs have shown great potential for application in electric load forecasting [3-81. A rela- tively new area of research involves the hybrid fuzzy- neural approaches to load forecasting. Several collabo- rative combinations of fuzzy logic with neural networks have been explored, including models with fuzzy pre- processing of neural inputs, fuzzy postprocessing of neural network outputs, and a parallel fuzzy-neural forecaster [9-141.

Although the superiority of neural network based models for load forecasting has already been empha- sised by many authors, their implementation in actual operating environments for consistently reliable per- formance is still a subject of widespread interest and the focus of this paper.

The model described here is a modified version of the approach presented by the authors in [IO], where a fuzzy expert system was used for preprocessing of input data fed to a multilayer neural network. Although the approach showed very impressive results, modification was deemed necessary due to practical problems in on- line acquisition of weather data related to humidity and rain indication. Keeping the basic idea the same, vari- ous changes in the model architecture were thus made to suit the requirements for on-line implementation, and to automate the data and knowledge acquisition as much as possible.

In this hybrid model, Kohonen’s networks are used as pattern matchers for classification of load, based on the type of day (Monday, weekday, Sunday, etc.). Mul- tilayer perceptrons are then employed to further proc- ess the output obtained from the Kohonen networks for temperature-related corrections. The forecast of the power demand from these neural networks is then manipulated by a fuzzy postprocessor to determine the final forecast.

Implementation and field experience with this new hybrid fuzzy neural forecasting model are discussed in the following Sections. In comparison to the existing load forecaster, the fuzzy neural model shows superior performance in terms of forecasting accuracy and robustness not only on normal days, but also during public holidays and special events. The results obtained have been very promising, with an average error of 0.97% on weekdays, 0.86% on weekends and 1.58% on public holidays.

687

2 Problem description

The problem of electrical load forecasting is a challeng- ing task due to the variation of load demand in response to different operating conditions encountered in electrical utilities, and the difficulty in identifying rel- evant variables for the forecasting model.

2. I Features Load in Singapore is affected by various factors, such as day type, meteorological condition and economical condition. The overall load demand of the system has been continuously increasing due to sustained eco- nomic growth of between 8 and 100/0. Investigations have shown that the most important meteorological factors for one-day-ahead forecasting are the minimum and maximum temperature and humidity. Seasonal effects are usually small and can be easily incorporated in a trend factor [lo]. However, special days and occa- sions strongly influence the load behaviour. For accu- rate forecasting, it is important to identify these variables with strong correlation with electrical load.

2.2 Load pattern The shape of the load curve follows daily and weekly cycles, with small random variations due to changes in industrial activities and climatic conditions.

Fig. 1 shows the shapes of the electrical load curve during a week, from Monday to Sunday. The load pat- tern on normal week days, when work is already in full swing, remains constant. The load values for normal days are functions of the short-term historical data and forecast values of weather parameters (high and low temperatures, and percentage humidity). However, the load on Mondays and Fridays is different from that on other weekdays due to pick-up loads on Monday mornings when all businesses and industries just start work, and evening loads on Fridays because of its proximity to the weekend. The load pattern on Satur- days is different from rest of the weekdays.

15001 I 0 12 24 36 48 60 72 84 96 108 120 132 144156168 Mon Tue Wed Thur Fri Sat Sun

time, h Fig. 1 Typicul loud shapes

The peak load also takes a dip on Saturday, which is a rest day for some and a half-day for others. The shape of the load curve on Sundays is similar to that on holidays. The peak load decreases considerably before and after major public holidays. Any abnormal- ities in the forecast values can be explained by the occurrence of special events.

3

It has been shown that both Kohonen’s neural net- works and fully feed-forward backpropagation neural

Overview of hybrid fuzzy-neural model

688

networks are individually capable of modelling the input-output relationship for a load forecasting prob- lem with good accuracy [3-71. However, the examina- tion of their internal structure indicates that their unique strengths in pattern classification and generali- sation can be combined to yield a better overall model. Furthermore, the task of training such individual net- works from a finite input-output data set can be trou- blesome. This is because these networks may learn an input-output mapping which has a different underlying structure from that of the system being modelled. This can typically lead to the trained networks having poor generalisation abilities.

In this paper, these two types of neural networks are used together as modules of the hybrid load forecasting system. This modular structure has produced an effi- cient high performance model which combines the strengths of unsupervised and supervised neural net- works for function approximation and prediction. Kohonen’s feature maps with unsupervised learning are used to identify days with similar daily load patterns. Feed-forward 3-layer neural networks with supervised learning are designed to predict the hourly loads for the following day. The inputs to these neural networks include the relevant weather related information that has significant impact (effect) on the future load. The fuzzy postprocessor contains knowledge explicitly encoded as fuzzy if-then rules which can be executed efficiently.

3. I Classification of load Kohonen’s network is selected in view of its ability to perform pattern classification and identification, and hence can be used for storing the load patterns. The input layer of each of the 2-layer Kohonen network has the same number of nodes as the number of dimensions of the input data. During training, input data are pre- sented one by one to the network. Each input is com- pared with all the nodes in the Kohonen layer using the Euclidean distance metric. The weights of the winning node and its neighbours are altered to more closely resemble the input vector. The network is trained such that the nodes which are physically close in the Kohonen layer will respond to inputs having a similar pattern. Once trained, previously unseen inputs can be presented to the network and classified in various cate- gories.

Fig. 2 shows the Kohonen networks developed for day type identification. The daily load patterns were separated into eight different classes, namely, Monday, Tuesday, Wednesday, Thursday, Friday, Saturday, Sunday and Public Holiday. Kohonen’s networks were trained to classify the input patterns in these eight dif- ferent classes. The load patterns pertaining to a partic- ular day type were sent to the corresponding Kohonen network. As a result, a total of eight Kohonen net- works were employed for training and testing of these day types. The representative load pattern for each class was obtained by averaging the loads of several similar days in the past. The result of this analysis is classification of characteristic load profiles for the days of a year into several groups (Fig. 2).

Each Kohonen network thus caters for a distinct day type. During actual on-line operation, the program would separate the load data according to their day types and direct these data into their respective Kohonen network for day type identification.

IEE Pioc -Genet Trumm D i r t r i b , Vol 145 N o 6 November 199X

input data-daily load curves

deviations load actual day

(min. and ma.)

day type 8 Kohonen

holiday temperatures ’ networks

% /

self-organising Kohonen networks

with n outputs

MLP for temperature correction

/“;“I . . .

indication

1 1 1 1 characteristic daily load curves for each class

\ J

~

Fig. 2 Classification of daily loud curves

final load

3.2 Temperature-load relationship Once the characteristic curve for the day is acquired from the Kohonen classifier, multilayer perceptrons (MLPs) with supervised learning are used to correct the temperature-related load deviations.

Feed-forward networks with an input layer, two hid- den layers and an output layer were developed. These neural networks were trained to capture the weather- load correlations from the historical temperature and load data.

During the learning process, the neural networks learn the nonlinear mapping between the desired out- put vector and the n-dimensional input vector. A gener- alised delta rule was used to adjust weights during the learning process. Training of this model was carried out based on load data for a period of one and a half years.

3.3 Fuzzy postprocessing Fuzzy postprocessing of the output load predictions obtained from these neural networks is employed to rectify the load deviations due to extreme weather con- ditions, public holidays, and to account for unusual

min. temp.

post-processor ma. temp fuzzy 4

4

load behaviour on days before and after holidays. The factors that have a direct effect on load are fuzzified and the results obtained from the application of fuzzy rules are used to make the appropriate corrections to the load curve.

The inputs to the fuzzy postprocessor include maxi- mum and minimum temperature for the forecast day, day-type, preholiday and postholiday indications, and any other a priori linguistic information related to the occurrence of special events. The inputs are defined as linguistic variables represented as triangular and trape- zoidal membership functions, each with their own range of possible values. The day types are divided into three groups [weekdays (W), Saturdays (S) and Sun- days (SS)]. Holidays form two subsets [normal (N) and special holidays (SH)]. The corresponding output varia- bles are the 24-hourly load corrective factors in terms of fractional values. The membership values for these corrective factors were derived from tests and analysis of the hourly load prediction errors from the neural networks.

The fuzzy knowledge base contains expert rules of if- then type [lo, 141, acquired by the human operators through years of experience. The knowledge base con- tains additional rules to reflect the change in load due to special events, religious festivals, etc.

The results from this fuzzy postprocessor are load correction factors for each hour of the day, which are added to the hourly load values obtained from the MLP to give the final load forecast.

pre-and post- holiday indication

4 Implementation

holiday indication

The fuzzy-neural load forecasting model described in Section 3 underwent several modifications and refine- ment before arriving at its final stage. The final model (Fig. 3) was built using the software ECANSE (envi- ronment for computer aided neural software engineer- ing, from Siemens Ltd.) on a Sun Sparc 20 workstation.

Eight Kohonen networks are employed in the ANN load forecasting model, each for memorising the load curves of a different day type. The input data comprise vectors with a 24-hourly load, and so the number of

24h desired load (actual day) .fP] 24h Kohonen

24h load I 3

24h actual day load deviation output

24h desired load

Fig. 3 Hybrid loud forecasting model

IEE Proc -Gener. Trurism. Distrib., Vol. 145, No. 6, November I998 689

input neurons is defined as 24. The map length and height are set to IO, giving a total of 100 neurons in each network. The learning rate was initialised to 0.3.

The MLP is used to categorise unknown inputs, i.e. temperatures, and produce the correct outputs, i.e. load deviations. A single MLP is used for carrying out the task of temperature-related load correction on all days. After several trials, it is deduced that an MLP with two hidden layers of 36 and 30 neurons, respectively, an input layer of two neurons and an output layer of 24 neurons yielded the best results. Pattern learning is adopted for training the network, and the learning rate is assigned a constant value of 0.001.

The inputs to the fuzzy processor consist of 24- hourly load values predicted by the cascaded neural networks and other input variables mentioned in Sec- tion 3. Based on these inputs, the output variables pro- duce appropriate fractional corrections for the load output from the neural networks. There were several fuzzy rules for load corrections. On normal days, only the rules for correction due to very high maximum temperature and unusually low minimum temperature were used. The fuzzy knowledge base also included rules to account for the thermal build-up effect due to very high temperature for a few days in a row. Addi- tional fuzzy rules were developed for holiday eves, days before and after public holidays, and for the Chinese New Year period. Continuous refinement, updating when new input data were used for training, and test- ing helped to enhance the accuracy of the fuzzy knowl- edge base.

5 Validation and results

The training of the FNN load forecasting model was performed on one and a half years of load and temper- ature data in Singapore (1 January 1994 to 26 July 1995) obtained from Singapore Power Pte Ltd. The input data for the model included day type, date, daily maximum and minimum temperatures, holiday indica- tions, preholiday and postholiday indications and his- torical hourly load data.

Training time required for executing 50 000 iterations was approximately 1 h. This was found to be the opti- mal number of steps required to train the neural net- works without resulting in a loss of generality. The model has facilities for on-line training when new data are available.

During the training process, 10% of the training sam- ples were kept aside for testing at regular intervals. The corresponding error on this separate set of training data was closely monitored in order to ensure that the neural network was trained without losing generality. The training was stopped when it was found that this error decreased to an almost constant minimum of 0.8%. The average error during the training process was 0.8% with a standard deviation of 2%.

The neural networks were trained on a monthly basis by adopting a monthly moving data set, i.e. for each new month the networks would retrain using a new set of data which has moved forward by one month to the month just before the month to be predicted.

Testing was carried out on data from 27 July 1995 to 31 December 1995. Public holidays such as the National Day period (8 August 1995 to 10 August 1995), Deepavali, and Christmas and New Year week (24 December 1995 to 31 December 1995) have also been tested. Since the testing was performed over a

690

-0.2 -

-0.4

period of 5 months, the neural networks underwent a total of five training sessions, each of which contained 1.5 years of moving data.

The error of the forecast results is calculated based on the mean absolute percentage error (MAPE), given by

] (1) N 2=1 N [ actual, M A P E = -

actual, - forecast,

where N is the total number of hours, actuali is the actual load at hour i andforecast, is the forecast value of the load at that hour.

- 1 I

I 0 24 48 72 96 120 144 168

time, h Fig. 4 Plot of hourly 96 error’ for I-day-clhead forecasting

0.4 t

-0.6 1 I 0 24 48 72 96 120 144 168

time, h Fig. 5 Plot of hourly standard deviation error

Fig. 4 shows the plot of hourly percentage error for a typical week. The hourly standard deviation error for a typical week is shown in Fig. 5 , where the error is cal- culated using the following equation:

le,l - M A E J M A E ( 1 - M A E )

where MAE is the mean absolute error for 168h and le,l is the absolute relative error for hour i.

The average forecasting errors for 5 months of previ- ously unseen test data are shown in Table 1. The results obtained with this model are compared with those predicted using the currently used method that employs multiple linear regression (MLR). Evidently, the forecasting errors with the proposed model are much lower than with the MLR model for all types of days, including public holidays.

The forecasting errors with the proposed hybrid fuzzy neural (FNN) model were also considerably smaller compared to the MLR model for days before and after holidays, and during extreme weather condi- tions. Two examples are shown in Figs. 6 and 7.

(2) 2, =

IEE Pioc -Gener Transm Distrib Vol I45 N o 6 November 1998

Table 1: Average forecasting errors

3100

2700

2300

MAPE with MAPE with

model, % % Day type fuzzy neural MLR model,

Weekdays (including days before 0.97 1.96 and after public holidays

Saturdays 0.81 2.44

Sundays 0.91 2.51

Public holidays 1.58 6.42

- .. _ . - - - -

-

-

3500 r

t ,I ,' -\

1500 1 3 5 7 9 11 13 15 17 19 21 23

Fig. 6 - actual load; - - - - MLR model; ~ ~ - - FNN model

Forecasting results for a Monday

, .. ___----- . r

1 3 5 7 9 11 13 15 17 19 21 23 Forecasting results for a Monday, just before public holiday Fig. 7

__ actual load; - - - - MLR model; - - ~ - FNN model

Fig. 6 shows the comparison of forecast load on a Monday with unusually low minimum temperature. The average error (MAPE) from the MLR model was 4.8%, while that from the FNN model was 1.4%. In addition to accounting for the load reduction in response to lower temperature, the FNN model takes into consideration the expected load drop on Monday mornings. Fig. 7 shows the results obtained for a Mon- day when a public holiday fell on the following day. The average error MAPE with FNN model was only 1.8%, while that with the MLR model was 7.8%.

Significant reductions in forecasting errors were pos- sible during holiday periods. Examples are shown in Figs. 8-10 for the Christmas period. Fig. 8 shows the results obtained for Christmas Eve, which fell on a cold Sunday. Fig. 9 shows the results for Christmas day itself, and Fig. 10 shows the results for the day after. The MAPE from the MLR model for these days was 7.56, 6.42 and 7.480/0, respectively, while that from the FNN model was 1.44, 1.71 and 1.82%. It should be noted that a fairly accurate forecast was possible from the FNN model because of the application of rules for load reduction during special holiday periods, and load drop due to persistent low temperatures.

IEE Proc-Gener. Transm. Distrib., Vol. 145. No. 6, November 1998

2200

2000

1800

1 3 5 7 9 11 13 15 17 19 21 23 Fig. 8 Forecasting results for Christmas eve (Sunday) __ actual load; - - - - MLR model; ~ - - - FNN model

1 3 5 7 9 11 13 15 17 19 21 23 Fig. 9

~ actual load; - - - - MLR model; ~ - - - FNN model Forecasting results for Christmas Day (Monday)

'-.F--/ - _ _ e

15001 i i i i i i ' I ' I

1 3 5 7 9 11 13 15 17 19 21 23 Fig. 10 Forecasting results for Boxing day (Tuesday) ___ actual load; - - - - MLR model; ~ ~ ~ - FNN model

These results have proven that the hybrid fuzzy neu- ral load forecaster is superior to the conventional model in terms of accuracy of load prediction for all types of days. Although the results are very promising for weekends and special days, the forecasting error for normal weekdays is still higher than expected. Further improvements to the model are needed to improve the prediction on these days. The main reason attributing to this high error is the omission of humidity or rain indication as an input to the model, which has a defi- nite correlation with the forecast load [lo]. However, this information was not readily available in the con- trol centre, and could not be included in the inputs for this forecasting model.

The FNN model has been implemented with the load characteristic in Singapore in mind. It can be further improved with the availability of more data such as humidity, rain factor and a report of special events (other than public holidays) that have occurred and have a strong influence on future load. Modifications

691

and additions to the fuzzy knowledge base can be eas- ily performed to enhance the load forecast or adapt it to suit the power demand of another country.

The capability of the hybrid network for on-line pre- diction, robustness, learning ability and interpolation ability are confirmed by simulation results.

6 Summary and conclusions

The paper describes the development and implementa- tion of a one-day-ahead load forecaster based on a hybrid fuzzy-neural approach. Neural networks were successfully employed for forming nonlinear relation- ships between uncorrelated inputs and outputs, and for generalising between training patterns. A fuzzy post- processor was used to further adjust the outputs from the neural network. This postprocessing was found to be particularly useful for predicting load on days with unusual weather conditions, and on special days, for which sufficient historical data were not available for neural network learning.

Inputs to the fuzzy inference engine include day type, daily minimum and maximum temperatures, holiday indications and pre- or postholiday indications. These variables played an essential role in determining the hourly output factors of the fuzzy rules. The forecast hourly outputs from the neural networks were multi- plied by these factors to ‘stretch’ or ‘compress’ the load patterns into the most accurate forecast. It was observed that the greatest improvement by the fuzzy rules came from those days characterised by extreme weather conditions, i.e. very hot or very cold days, or for those occasions where the public holidays had dif- ferent behaviour from the usual public holidays. Another group of special days which benefited from this scheme were the days surrounding a public holi- day.

The results indicate that accurate forecasting of load curves can be achieved by the proposed fuzzy-neural network in an efficient manner. The fuzzy postproces- sor increases the robustness of the forecast model in tackling uncertainties and ambiguity in the inputs. In addition, it avoids the use of large chunks of historical data, and frequent retraining in response to changing input conditions. The results obtained have been prom- ising, with an average error of 0.97% on weekdays, 0.86% on weekends and 1.58% on public holidays.

7 Acknowledgments

The authors would like to thank the people at the Con- trol Center of Power Grid Pte. Ltd., Singapore, for providing the valuable system data and operating expe- rience. The authors also wish to thank Siemens, Aus- tria for the use of ECANSE software.

8

1

2

3

4

5

6

7

8

9

References

VEMURI, S., HUANG, W.L., and NELSON, D.J.: ‘On line algorithms for forecasting hourly loads of an electric utility’, IEEE Truns., 1981, PAS-100, pp, 3775-3784 DEHDASHTI, A.S.: ‘Forecasting of hourly load by pattern rec- ognition - a deterministic approach’, IEEE Trans., 1982, PAS- 101, pp. 3290-3294 LEE, K.Y., CHA, Y.T., and PARK, J.H.: ‘Short-term load fore- casting using an artificial neural network’, IEEE Trans. Power Syst., 1992, 7, ( l ) , pp. 124-132 HSU, Y.Y., and YANG. C.C.: ‘Design of artificial neural net- works for short-term load forecasting. Part 1 : Self-organizing fea- ture maps for day type identification’, IEE Proc. C, 1991, 138, ( 9 , pp. 407418 PENG, T.M., HUBELE, N.F., and KARADY, G.G.: ‘Advance- ment in the application of neural networks for short-term load forecasting’, ZEEE Trans. Power Syst., 1992, 7, (l), pp. 250-257 LU, C.N., WU, H.T., and VEMURI, S.: ‘Neural network based short term load forecasting’, IEEE Truns. Power Syst., 1993, 8, (I), pp. 336-342 PAPALEXOPOULOS, A.D., HAO, S., and PENG, T.M.: ‘An implementation of a neural network based load forecasting model for the EMS’, IEEE Truns. Power Syst., 1994, 9, (4), pp. 1956- 1962 RANAWEERA, D.K., HUBELE, N.F., and PAPALEXOPOU- LOS, A.D.: ‘Application of radial basis function neural network model for short-term load forecasting’, IEE Proc. C, 1995, 142, (l), pp. 45-50 DASH, P.K., LIEW, A.C., and RAHMAN, S.: ‘Peak load fore- casting using a fuzzv neural network’. Electr. Power Svst. Res.. 1995,32, ( I C pp. 19123

10 SRINIVASAN. D.. LIEW. A.C.. and CHANG. C.S.: ‘Forecast- ing daily load curves using a hybrid approach’, IEE Proc.- Gener. Transm. Distrib., 1994, 141, (6), pp. 561-567

11 SRINIVASAN, D., and LEE, M.A.: ‘Survey of hybrid fuzzy neu- ral approaches to electric load forecasting’. Proceedings of the IEEE conference on Systeni, man and cybernetics, Vancouver, Canada, October 1995, Vol. 5, pp. 4004-4008

12 PIRAS, A., GERMOND, A., BUCHENEL, B., IMHOF, K., and JACCARD, Y.: ‘Heterogeneous artificial neural network for short term electrical load forecasting’, IEEE Trans. Power Syst., 1996, 11, (l), pp. 397402

13 KOHONEN, T.: ‘Self-organization and associative memory’ (Springer-Verlag 1989, 3rd edn.)

14 YAGER, R., and ZADEH, L.A.: ‘An introduction to fuzzy logic applications in intelligent systems’ (Kluwer Academic, Boston, 1992)

692 IEE Proc -Grner. Transm. Disfrib.. Vnl. 145. No. 6. November 1998