IBM WindEnergy White
-
Upload
avner-abrami -
Category
Documents
-
view
150 -
download
1
Transcript of IBM WindEnergy White
© 2009 IBM Corporation
Forecasting aggregated energy demand in the presence of missing data and outliers with double (or more) seasonality
Avner Abrami – Columbia University / Mentor : Dr. Younghun Kim
August 31st 2015
© 2009 IBM Corporation2
Agenda
Motivation of the problem
Accurate demand forecasts are essential to energy companies
Traditional approaches have limitations
Proposed approach
Formulating exponential smoothing as an optimization problem
Formulation handles missing data
Formulation resists outliers
Future work and extensions
IBM Smarter Energy – Industries and Solutions
© 2009 IBM Corporation3
MOTIVATION OF THE PROBLEM
IBM Smarter Energy – Industries and Solutions
© 2009 IBM Corporation4
Motivation of the problemAccurate demand forecasting is essential for energy companies
• Optimal maintenance and upgrade planning of existing infrastructure
• Competitive advantage in energy trading
• Optimal scheduling generation operations• E.g. Electricity case: 5% reduction in load uncertainty translates into the reduction of
5% spinning reserve requirements, which is 2,046 million kWh (Savings of $204M).• E.g. Gas case: Avoidance of low pressure situation ; efficient compressor station
matintenance and operation.
• Assessing the feasibility of renewable integration (which increases load uncertainty)
IBM Smarter Energy – Industries and Solutions
Smart meters are the promising big data resource for such quantitative analysis, despite their generating challenging data volume and complexity.
© 2009 IBM Corporation5
Motivation of the problemReal data
IBM Smarter Energy – Industries and Solutions
The real energy dataset is primary characterized by a weekly & yearly seasonality
Data from smart meters may have missing data and outliers– Very common to have communication errors from smart meters (Missing data)– Malfunction of smart meters (Outliers)
© 2009 IBM Corporation6
Motivation of the problemTraditional approaches have limitations
• Seasonal autoregressive integrated moving average models (SARIMA)– Becomes obsolete when dealing with more than one seasonality-> No available R packages to handle multi-seasonality ARIMA– Very difficult to handle missing data in the ARMA framework, especially when
incorporating seasonality.– Not robust to outliers (Autoregressive models)
• Gaussian process regression– Very sensitive to choice of kernel and means functions (which could vary depending
on the kind of seasonal data we are dealing with).– Periodic kernel/ locally periodic kernel allows to model functions which repeat
themselves.– Very efficient on simulated data.– Possibility to handle double seasonality by combining periodic kernels.
IBM Smarter Energy – Industries and Solutions
)',()',()',()',()',( 21 xxkxxkxxkxxkxxk NoisePeriodicPeriodicRBF
)',()',()',()',( 21 xxkxxkxxkxxk NoisePeriodicPeriodic
© 2009 IBM Corporation7
PROPOSED APPROACH
IBM Smarter Energy – Industries and Solutions
© 2009 IBM Corporation8
Proposed approachAn enhanced additive exponential smoothing formulation
• Exponential smoothing weights past observations with exponentially decreasing weights to forecast future values
• Allow structured modeling of the time series’ evolution:
– Mean
– Trend
– Seasonality
• Formulated as a single source of error (SSOE) statistical model
IBM Smarter Energy – Industries and Solutions
)1)(1()(1
1
1ty blsl t tptt
bllb tttt 11 )1()(
sbls ptttt
1
1111t1
1 )1()(y
© 2009 IBM Corporation9
IBM Smarter Energy – Industries and Solutions
Rsbl ttt
3),,(
tptt
ttt
tttt
tpttt
ssbbbllsbl
1
1
1
11
11
1
111tyRecursive formulation State-space formulation
Step 1: From the recursive to the state-space univariate formulation
Proposed approachAn enhanced additive exponential smoothing formulation
sblsbllbblsl
sbl
ptttt
tttt
tptt
tpttt
t
1
1111t1
111
1
1
1t
1
111t
)1()()1()(
)1)(1()(
y
yy
)1,0(~ Nt
R3),,(
Mean, trend, seasonality functions at time t
Noise
yt Observation at time t Smoothing parameters
© 2009 IBM Corporation10
IBM Smarter Energy – Industries and Solutions
...
...
2
2
2
2
1
1
1
11
11
2
2
1
111ty
tptt
tptt
ttt
tttt
tptpttt
ssssbbbllssbl
One seasonality Double seasonality(or more..)
Step 2: Introducing double (or more) seasonality
tptt
ttt
tttt
tpttt
ssbbbllsbl
1
1
1
11
11
1
111ty
Proposed approachAn enhanced additive exponential smoothing formulation
© 2009 IBM Corporation11
Proposed approachAn enhanced additive exponential smoothing formulation
IBM Smarter Energy – Industries and Solutions
Recursive form Recursive matrix form
Step 3: Formulate the state-space model in matrix form
(...)
...)(
2
2
2
2
1
1
1
11
11
2
2
1
111ty
tptt
tptt
ttt
tttt
tptpttt
ssssbbbllssbl
ttt
ttt
gA
w
xxxy
1
1
0
'
)',,,( 2
2
1
1 ssblx ptptttt
© 2009 IBM Corporation12
Proposed approachAn enhanced additive exponential smoothing formulation
IBM Smarter Energy – Industries and Solutions
Recursive matrix form Least-square form
Step 4: Exponential smoothing as least-square
x
xx
y
y
y
wIA
wIA
w
MxY
n
2
1
n
2
1
'
'
'
0
0
0
ttt
ttt
gA
w
xxxy
1
1
0
'
)',,,( 2
2
1
1 ssblx ptptttt
© 2009 IBM Corporation13
Proposed approachAn enhanced additive exponential smoothing formulation
IBM Smarter Energy – Industries and Solutions
s
wIA
wIA
w
sMxY
x
xx
y
y
y
n
2
1
n
2
1
'
'
'
0
0
0
'
'1
~~
~~
~
gg
ggV
gg
),0(~ VNs
Formulating a maximum likelihood problem is complicated by the singularity of V (Rank(V)=1 hence non invertible).
In this case, the least square problem becomes:
sVMxYts
sgsx
2/1
2,,
:.
||||21min
Where Bayesian optimization was used to fit smoothing parameters g
Which constitutes the smoothing problem
© 2009 IBM Corporation14
Proposed approachFilling missing data and forecast future time series values
IBM Smarter Energy – Industries and Solutions
Incorporating missing data is easy in this formulation - we just include a diagonal matrix D, with diagonal entries corresponding to missing values set to 0.
sDVDMxDYts
sgsx
2/1
2,,
:.
||||21min
Where Bayesian optimization was used to fit smoothing parameters g
Which constitutes the smoothing problem with missing data
The forecasting problem is equivalent. We just have to consider future values as missing values!
© 2009 IBM Corporation15
Proposed approachA large scale prediction can be efficiently solved
IBM Smarter Energy – Industries and Solutions
sDVDMxDYts
sgsx
2/1
22,,
:.
||||min
sDVDMxDYts
sgsx
2/1
1,,
:.
||||min
sDVDMxDY
tsts
ts
sgsx
2/1
,,
:.
min
Linear Programming RelaxationLinear Programming Relaxation
Norm changeL2 norm to L1 norm Equivalent
The structure of the problem is sparse in most cases, the problem can be solved exploiting the sparsity of the matrices.
Numerically equivalent
© 2009 IBM Corporation16
Proposed approachSimulated demand
IBM Smarter Energy – Industries and Solutions
We simulate energy consumption to test different forecasting algorithm as follows:
For example, a weekend high demand and weekend low demand can be effectively captured using the square function
)()2(*2)2sin(*1)(21
ttHAtAoffsetty ff
H: Sawtooth functionH: Sawtooth functionH: Square functionH: Square function
© 2009 IBM Corporation17
Proposed approachSimulated demand
IBM Smarter Energy – Industries and Solutions
The algorithm can extract the double seasonality, and all deterministic behaviors of the time series effectively.
© 2009 IBM Corporation
Proposed approachSome results on simulated data
Objective NoiseStd
MAPE (%)Sawtooth
MAPE (%)Square
M1 L2 0.75 6.35 5.01
L2 2.25 11.25 12.9
M2 L1 0.75 6.75 5.86
L1 2.25 10.95 11.7
GPs 0.75 4.98 4.74
22.25 9.55 8.49
3 month forecasting 6 month forecasting
12 month forecasting
Objective NoiseStd
MAPE (%)Sawtooth
MAPE (%)Square
M1 L2 0.75 5.9 7.68
L2 2.25 11.18 14.66
M2 L1 0.75 6.2 7.34
L1 2.25 11.05 13.57
GPs 0.75 6.95 10.95
2.25 10.02 11.05
Objective NoiseStd
MAPE (%)Sawtooth
MAPE (%)Square
M1 L2 0.75 6.4 7.39
L2 2.25 11.52 13.24
M2 L1 0.75 6.78 7.10
L1 2.25 10.85 12..98
GPs 0.75 6.5 7.08
2.25 14.12 11.89
© 2009 IBM Corporation
Proposed approachSome results on simulated data
Objective NoiseStd
MAPE (%)Sawtooth
MAPE (%)Square
M1 L2 0.75 6.35 5.01
L2 2.25 11.25 12.9
M2 L1 0.75 6.75 5.86
L1 2.25 10.95 11.7
GPs 0.75 4.98 4.74
22.25 9.55 8.49
3 month forecasting 6 month forecasting
12 month forecasting
Objective NoiseStd
MAPE (%)Sawtooth
MAPE (%)Square
M1 L2 0.75 5.9 7.68
L2 2.25 11.18 14.66
M2 L1 0.75 6.2 7.34
L1 2.25 11.05 13.57
GPs 0.75 6.95 10.95
2.25 10.02 11.05
Objective NoiseStd
MAPE (%)Sawtooth
MAPE (%)Square
M1 L2 0.75 6.4 7.39
L2 2.25 11.52 13.24
M2 L1 0.75 6.78 7.10
L1 2.25 10.85 12..98
GPs 0.75 6.5 7.08
2.25 14.12 11.89
~ Same performance for 10%
~ Same performance for 10%
uniformly missing observations
uniformly missing observations
© 2009 IBM Corporation
Proposed approachRobustness
IBM Smarter Energy – Industries and Solutions
Formulation for SSOE models are inherently robust against outliers providing the training set is large enough (4 years of data at least)
Robust simulation procedure :
- Double seasonal (sinusoid + sawtooth/square) signal with noise = N(0,(15%(max-min)) **2)
- Add 2.5% of outliers : Random(sign)*(mean + Uniform(3*(max-mean),5*(max-min))
- 4 years of training, 6 month of forecasting (but could be more)
)(:.
||||21min
2/1
22,,
sDVDMxDYts
sgsx
Where Bayesian optimization was used to fit smoothing parameters g
Which constitutes the robust problem with missing data
© 2009 IBM Corporation21
Proposed approachResults: Robustness
IBM Smarter Energy – Industries and Solutions
• Robust for the 4 year smoothing!
• Robust for the 6 month forecasting!
© 2009 IBM Corporation
An enhanced additive exponential smoothing formulationWhy is the procedure robust?
IBM Smarter Energy – Industries and Solutions
There is a competition between:
- The objective function that aims at minimizing the residual s
- The constraints which enforces the recursive equation to be fulfilled.
When the algorithm detects an outlier, the objective wants to fit it perfectly. Yet if it did, the constraints would impose outliers for the next value of the time series which would result in a much higher ||s||2 than the robust version.
To be robust, we hence need a large enough dataset.
)(:.
||||21min
2/1
2,,
sDVDMxDYts
sgsx
...
...
2
2
2
21
1
1
11
11
2
2
1
111ty
tptt
tptt
ttt
tttt
tptpttt
ssssbbbllssbl
Objective
© 2009 IBM Corporation23
IBM Smarter Energy – Industries and Solutions
ss AABCD '
An enhanced additive exponential smoothing formulationWhy is the procedure robust?
So the second scheme is preferred by the algorithm!
© 2009 IBM Corporation
24
Simulation OutcomesReal demand
IBM Smarter Energy – Industries and Solutions
Smoothing problem 1 year forecasting
)',()',()',()',()',( 21 xxkxxkxxkxxkxxk NoisePeriodicPeriodicRBF
GP regression
Enhanced ES
© 2009 IBM Corporation
Proposed approachSome results on real data
Objective MAPE (%)
GPs 69.87
M1 L1 15.42
M2 L2 14.28
6 month forecasting
12 month forecasting
3 month forecasting
Objective MAPE (%)
GPs 71.54
M1 L1 18..58
M2 L2 17.85
Objective MAPE (%)
GPs 75.26
M1 L1 19.25
M2 L2 19.84
© 2009 IBM Corporation26
FUTURE WORK AND EXTENSION
IBM Smarter Energy – Industries and Solutions
© 2009 IBM Corporation27
Forecasting aggregated energy demand in the presence of missing data and outliers with double (or more) seasonality
A tailored optimization to improve speed and process even more data and time series patterns.
Add daily periodicity of energy consumption (computationally intensive except for a tailored solver).
Increase the model’s resistance to jitter in the periodicity.
Exploit correlation for collection of over two time series.
IBM Smarter Energy – Industries and Solutions
© 2009 IBM Corporation28
THANK YOU TO
Younghun Kim
&
Sasha Aravkin
IBM Smarter Energy – Industries and Solutions
© 2009 IBM Corporation29
Q&A
IBM Smarter Energy – Industries and Solutions