Forecasting JY Le Boudec 1. Contents 1.What is forecasting ? 2.Linear Regression 3.Avoiding...
-
date post
19-Dec-2015 -
Category
Documents
-
view
219 -
download
0
Transcript of Forecasting JY Le Boudec 1. Contents 1.What is forecasting ? 2.Linear Regression 3.Avoiding...
![Page 1: Forecasting JY Le Boudec 1. Contents 1.What is forecasting ? 2.Linear Regression 3.Avoiding Overfitting 4.Differencing 5.ARMA models 6.Sparse ARMA models.](https://reader037.fdocuments.us/reader037/viewer/2022103123/56649d2d5503460f94a03672/html5/thumbnails/1.jpg)
Forecasting
JY Le Boudec
1
![Page 2: Forecasting JY Le Boudec 1. Contents 1.What is forecasting ? 2.Linear Regression 3.Avoiding Overfitting 4.Differencing 5.ARMA models 6.Sparse ARMA models.](https://reader037.fdocuments.us/reader037/viewer/2022103123/56649d2d5503460f94a03672/html5/thumbnails/2.jpg)
Contents
1. What is forecasting ?2. Linear Regression
3. Avoiding Overfitting4. Differencing
5. ARMA models6. Sparse ARMA models
7. Case Studies
2
![Page 3: Forecasting JY Le Boudec 1. Contents 1.What is forecasting ? 2.Linear Regression 3.Avoiding Overfitting 4.Differencing 5.ARMA models 6.Sparse ARMA models.](https://reader037.fdocuments.us/reader037/viewer/2022103123/56649d2d5503460f94a03672/html5/thumbnails/3.jpg)
1. What is forecasting ?
Assume you have been able to define the nature of the loadIt remains to have an idea about its intensity
It is impossible to forecast without error
The good engineer shouldForecast what can be forecast
Give uncertainty intervals
The rest is outside our control
3
![Page 4: Forecasting JY Le Boudec 1. Contents 1.What is forecasting ? 2.Linear Regression 3.Avoiding Overfitting 4.Differencing 5.ARMA models 6.Sparse ARMA models.](https://reader037.fdocuments.us/reader037/viewer/2022103123/56649d2d5503460f94a03672/html5/thumbnails/4.jpg)
4
![Page 5: Forecasting JY Le Boudec 1. Contents 1.What is forecasting ? 2.Linear Regression 3.Avoiding Overfitting 4.Differencing 5.ARMA models 6.Sparse ARMA models.](https://reader037.fdocuments.us/reader037/viewer/2022103123/56649d2d5503460f94a03672/html5/thumbnails/5.jpg)
2. Linear Regression
Simple, for simple casesBased on extrapolating the explanatory variables
5
![Page 6: Forecasting JY Le Boudec 1. Contents 1.What is forecasting ? 2.Linear Regression 3.Avoiding Overfitting 4.Differencing 5.ARMA models 6.Sparse ARMA models.](https://reader037.fdocuments.us/reader037/viewer/2022103123/56649d2d5503460f94a03672/html5/thumbnails/6.jpg)
6
![Page 7: Forecasting JY Le Boudec 1. Contents 1.What is forecasting ? 2.Linear Regression 3.Avoiding Overfitting 4.Differencing 5.ARMA models 6.Sparse ARMA models.](https://reader037.fdocuments.us/reader037/viewer/2022103123/56649d2d5503460f94a03672/html5/thumbnails/7.jpg)
7
![Page 8: Forecasting JY Le Boudec 1. Contents 1.What is forecasting ? 2.Linear Regression 3.Avoiding Overfitting 4.Differencing 5.ARMA models 6.Sparse ARMA models.](https://reader037.fdocuments.us/reader037/viewer/2022103123/56649d2d5503460f94a03672/html5/thumbnails/8.jpg)
8
![Page 9: Forecasting JY Le Boudec 1. Contents 1.What is forecasting ? 2.Linear Regression 3.Avoiding Overfitting 4.Differencing 5.ARMA models 6.Sparse ARMA models.](https://reader037.fdocuments.us/reader037/viewer/2022103123/56649d2d5503460f94a03672/html5/thumbnails/9.jpg)
Estimation and Forecasting
In practice we estimate from y, …, yt
When computing the forecast, we pretend is known, and thus make an estimation errorIt is hoped that the estimation error is much less than the confidence interval for forecast
In the case of linear regression, the theorem gives the global error exactlyIn general, we won’t have this luxury
9
![Page 10: Forecasting JY Le Boudec 1. Contents 1.What is forecasting ? 2.Linear Regression 3.Avoiding Overfitting 4.Differencing 5.ARMA models 6.Sparse ARMA models.](https://reader037.fdocuments.us/reader037/viewer/2022103123/56649d2d5503460f94a03672/html5/thumbnails/10.jpg)
10
![Page 11: Forecasting JY Le Boudec 1. Contents 1.What is forecasting ? 2.Linear Regression 3.Avoiding Overfitting 4.Differencing 5.ARMA models 6.Sparse ARMA models.](https://reader037.fdocuments.us/reader037/viewer/2022103123/56649d2d5503460f94a03672/html5/thumbnails/11.jpg)
We saw this alreadyA case where estimation error versus prediction uncertainty can be quantified
Prediction interval if model is knownPrediction interval accounting for estimation (t = 100 observed points)
11
![Page 12: Forecasting JY Le Boudec 1. Contents 1.What is forecasting ? 2.Linear Regression 3.Avoiding Overfitting 4.Differencing 5.ARMA models 6.Sparse ARMA models.](https://reader037.fdocuments.us/reader037/viewer/2022103123/56649d2d5503460f94a03672/html5/thumbnails/12.jpg)
3. The Overfitting Problem
The best model is not necessarily the one that fits best
12
![Page 13: Forecasting JY Le Boudec 1. Contents 1.What is forecasting ? 2.Linear Regression 3.Avoiding Overfitting 4.Differencing 5.ARMA models 6.Sparse ARMA models.](https://reader037.fdocuments.us/reader037/viewer/2022103123/56649d2d5503460f94a03672/html5/thumbnails/13.jpg)
Prediction for the better model
13
This is the overfitting problem
![Page 14: Forecasting JY Le Boudec 1. Contents 1.What is forecasting ? 2.Linear Regression 3.Avoiding Overfitting 4.Differencing 5.ARMA models 6.Sparse ARMA models.](https://reader037.fdocuments.us/reader037/viewer/2022103123/56649d2d5503460f94a03672/html5/thumbnails/14.jpg)
How to avoid overfittingMethod 1: use of test dataMethod 2: information criterion
14
![Page 15: Forecasting JY Le Boudec 1. Contents 1.What is forecasting ? 2.Linear Regression 3.Avoiding Overfitting 4.Differencing 5.ARMA models 6.Sparse ARMA models.](https://reader037.fdocuments.us/reader037/viewer/2022103123/56649d2d5503460f94a03672/html5/thumbnails/15.jpg)
15
![Page 16: Forecasting JY Le Boudec 1. Contents 1.What is forecasting ? 2.Linear Regression 3.Avoiding Overfitting 4.Differencing 5.ARMA models 6.Sparse ARMA models.](https://reader037.fdocuments.us/reader037/viewer/2022103123/56649d2d5503460f94a03672/html5/thumbnails/16.jpg)
16
![Page 17: Forecasting JY Le Boudec 1. Contents 1.What is forecasting ? 2.Linear Regression 3.Avoiding Overfitting 4.Differencing 5.ARMA models 6.Sparse ARMA models.](https://reader037.fdocuments.us/reader037/viewer/2022103123/56649d2d5503460f94a03672/html5/thumbnails/17.jpg)
Best Model for Internet Data, polynomial of degree up to 2
17
![Page 18: Forecasting JY Le Boudec 1. Contents 1.What is forecasting ? 2.Linear Regression 3.Avoiding Overfitting 4.Differencing 5.ARMA models 6.Sparse ARMA models.](https://reader037.fdocuments.us/reader037/viewer/2022103123/56649d2d5503460f94a03672/html5/thumbnails/18.jpg)
18
d = 1
![Page 19: Forecasting JY Le Boudec 1. Contents 1.What is forecasting ? 2.Linear Regression 3.Avoiding Overfitting 4.Differencing 5.ARMA models 6.Sparse ARMA models.](https://reader037.fdocuments.us/reader037/viewer/2022103123/56649d2d5503460f94a03672/html5/thumbnails/19.jpg)
Best Model for Internet Data, polynomial of degree up to 10
19
![Page 20: Forecasting JY Le Boudec 1. Contents 1.What is forecasting ? 2.Linear Regression 3.Avoiding Overfitting 4.Differencing 5.ARMA models 6.Sparse ARMA models.](https://reader037.fdocuments.us/reader037/viewer/2022103123/56649d2d5503460f94a03672/html5/thumbnails/20.jpg)
4. Differencing the Data
20
![Page 21: Forecasting JY Le Boudec 1. Contents 1.What is forecasting ? 2.Linear Regression 3.Avoiding Overfitting 4.Differencing 5.ARMA models 6.Sparse ARMA models.](https://reader037.fdocuments.us/reader037/viewer/2022103123/56649d2d5503460f94a03672/html5/thumbnails/21.jpg)
21
![Page 22: Forecasting JY Le Boudec 1. Contents 1.What is forecasting ? 2.Linear Regression 3.Avoiding Overfitting 4.Differencing 5.ARMA models 6.Sparse ARMA models.](https://reader037.fdocuments.us/reader037/viewer/2022103123/56649d2d5503460f94a03672/html5/thumbnails/22.jpg)
22
![Page 23: Forecasting JY Le Boudec 1. Contents 1.What is forecasting ? 2.Linear Regression 3.Avoiding Overfitting 4.Differencing 5.ARMA models 6.Sparse ARMA models.](https://reader037.fdocuments.us/reader037/viewer/2022103123/56649d2d5503460f94a03672/html5/thumbnails/23.jpg)
Point Predictions from Differenced Data
23
![Page 24: Forecasting JY Le Boudec 1. Contents 1.What is forecasting ? 2.Linear Regression 3.Avoiding Overfitting 4.Differencing 5.ARMA models 6.Sparse ARMA models.](https://reader037.fdocuments.us/reader037/viewer/2022103123/56649d2d5503460f94a03672/html5/thumbnails/24.jpg)
Background On Filters (Appendix B)
24
We need to understand how to use discrete filters.
Example: write the Matlab command for
![Page 25: Forecasting JY Le Boudec 1. Contents 1.What is forecasting ? 2.Linear Regression 3.Avoiding Overfitting 4.Differencing 5.ARMA models 6.Sparse ARMA models.](https://reader037.fdocuments.us/reader037/viewer/2022103123/56649d2d5503460f94a03672/html5/thumbnails/25.jpg)
25
![Page 26: Forecasting JY Le Boudec 1. Contents 1.What is forecasting ? 2.Linear Regression 3.Avoiding Overfitting 4.Differencing 5.ARMA models 6.Sparse ARMA models.](https://reader037.fdocuments.us/reader037/viewer/2022103123/56649d2d5503460f94a03672/html5/thumbnails/26.jpg)
26
A simple filter
Q: compute X back from Y
![Page 27: Forecasting JY Le Boudec 1. Contents 1.What is forecasting ? 2.Linear Regression 3.Avoiding Overfitting 4.Differencing 5.ARMA models 6.Sparse ARMA models.](https://reader037.fdocuments.us/reader037/viewer/2022103123/56649d2d5503460f94a03672/html5/thumbnails/27.jpg)
27
![Page 28: Forecasting JY Le Boudec 1. Contents 1.What is forecasting ? 2.Linear Regression 3.Avoiding Overfitting 4.Differencing 5.ARMA models 6.Sparse ARMA models.](https://reader037.fdocuments.us/reader037/viewer/2022103123/56649d2d5503460f94a03672/html5/thumbnails/28.jpg)
28
![Page 29: Forecasting JY Le Boudec 1. Contents 1.What is forecasting ? 2.Linear Regression 3.Avoiding Overfitting 4.Differencing 5.ARMA models 6.Sparse ARMA models.](https://reader037.fdocuments.us/reader037/viewer/2022103123/56649d2d5503460f94a03672/html5/thumbnails/29.jpg)
29
![Page 30: Forecasting JY Le Boudec 1. Contents 1.What is forecasting ? 2.Linear Regression 3.Avoiding Overfitting 4.Differencing 5.ARMA models 6.Sparse ARMA models.](https://reader037.fdocuments.us/reader037/viewer/2022103123/56649d2d5503460f94a03672/html5/thumbnails/30.jpg)
30
Impulse Response
![Page 31: Forecasting JY Le Boudec 1. Contents 1.What is forecasting ? 2.Linear Regression 3.Avoiding Overfitting 4.Differencing 5.ARMA models 6.Sparse ARMA models.](https://reader037.fdocuments.us/reader037/viewer/2022103123/56649d2d5503460f94a03672/html5/thumbnails/31.jpg)
31
![Page 32: Forecasting JY Le Boudec 1. Contents 1.What is forecasting ? 2.Linear Regression 3.Avoiding Overfitting 4.Differencing 5.ARMA models 6.Sparse ARMA models.](https://reader037.fdocuments.us/reader037/viewer/2022103123/56649d2d5503460f94a03672/html5/thumbnails/32.jpg)
32
![Page 33: Forecasting JY Le Boudec 1. Contents 1.What is forecasting ? 2.Linear Regression 3.Avoiding Overfitting 4.Differencing 5.ARMA models 6.Sparse ARMA models.](https://reader037.fdocuments.us/reader037/viewer/2022103123/56649d2d5503460f94a03672/html5/thumbnails/33.jpg)
33
A filter with stable inverse
![Page 34: Forecasting JY Le Boudec 1. Contents 1.What is forecasting ? 2.Linear Regression 3.Avoiding Overfitting 4.Differencing 5.ARMA models 6.Sparse ARMA models.](https://reader037.fdocuments.us/reader037/viewer/2022103123/56649d2d5503460f94a03672/html5/thumbnails/34.jpg)
How is this prediction done ?
This is all very intuitive
34
![Page 35: Forecasting JY Le Boudec 1. Contents 1.What is forecasting ? 2.Linear Regression 3.Avoiding Overfitting 4.Differencing 5.ARMA models 6.Sparse ARMA models.](https://reader037.fdocuments.us/reader037/viewer/2022103123/56649d2d5503460f94a03672/html5/thumbnails/35.jpg)
35
![Page 36: Forecasting JY Le Boudec 1. Contents 1.What is forecasting ? 2.Linear Regression 3.Avoiding Overfitting 4.Differencing 5.ARMA models 6.Sparse ARMA models.](https://reader037.fdocuments.us/reader037/viewer/2022103123/56649d2d5503460f94a03672/html5/thumbnails/36.jpg)
Prediction assuming differenced data is iid
36
![Page 37: Forecasting JY Le Boudec 1. Contents 1.What is forecasting ? 2.Linear Regression 3.Avoiding Overfitting 4.Differencing 5.ARMA models 6.Sparse ARMA models.](https://reader037.fdocuments.us/reader037/viewer/2022103123/56649d2d5503460f94a03672/html5/thumbnails/37.jpg)
Prediction Intervals
A prediction without prediction intervals is only a small part of the storyThe financial crisis might have been avoided if investors had been aware of prediction intervals
37
![Page 38: Forecasting JY Le Boudec 1. Contents 1.What is forecasting ? 2.Linear Regression 3.Avoiding Overfitting 4.Differencing 5.ARMA models 6.Sparse ARMA models.](https://reader037.fdocuments.us/reader037/viewer/2022103123/56649d2d5503460f94a03672/html5/thumbnails/38.jpg)
38
![Page 39: Forecasting JY Le Boudec 1. Contents 1.What is forecasting ? 2.Linear Regression 3.Avoiding Overfitting 4.Differencing 5.ARMA models 6.Sparse ARMA models.](https://reader037.fdocuments.us/reader037/viewer/2022103123/56649d2d5503460f94a03672/html5/thumbnails/39.jpg)
39
![Page 40: Forecasting JY Le Boudec 1. Contents 1.What is forecasting ? 2.Linear Regression 3.Avoiding Overfitting 4.Differencing 5.ARMA models 6.Sparse ARMA models.](https://reader037.fdocuments.us/reader037/viewer/2022103123/56649d2d5503460f94a03672/html5/thumbnails/40.jpg)
Compare the Two
40
Linear Regression with 3 parameters + variance Assuming differenced data is iid
![Page 41: Forecasting JY Le Boudec 1. Contents 1.What is forecasting ? 2.Linear Regression 3.Avoiding Overfitting 4.Differencing 5.ARMA models 6.Sparse ARMA models.](https://reader037.fdocuments.us/reader037/viewer/2022103123/56649d2d5503460f94a03672/html5/thumbnails/41.jpg)
41
![Page 42: Forecasting JY Le Boudec 1. Contents 1.What is forecasting ? 2.Linear Regression 3.Avoiding Overfitting 4.Differencing 5.ARMA models 6.Sparse ARMA models.](https://reader037.fdocuments.us/reader037/viewer/2022103123/56649d2d5503460f94a03672/html5/thumbnails/42.jpg)
5. Using ARMA Models
When the differenced data appears stationary but not iid
42
![Page 43: Forecasting JY Le Boudec 1. Contents 1.What is forecasting ? 2.Linear Regression 3.Avoiding Overfitting 4.Differencing 5.ARMA models 6.Sparse ARMA models.](https://reader037.fdocuments.us/reader037/viewer/2022103123/56649d2d5503460f94a03672/html5/thumbnails/43.jpg)
Test of iid-ness
43
![Page 44: Forecasting JY Le Boudec 1. Contents 1.What is forecasting ? 2.Linear Regression 3.Avoiding Overfitting 4.Differencing 5.ARMA models 6.Sparse ARMA models.](https://reader037.fdocuments.us/reader037/viewer/2022103123/56649d2d5503460f94a03672/html5/thumbnails/44.jpg)
44
![Page 45: Forecasting JY Le Boudec 1. Contents 1.What is forecasting ? 2.Linear Regression 3.Avoiding Overfitting 4.Differencing 5.ARMA models 6.Sparse ARMA models.](https://reader037.fdocuments.us/reader037/viewer/2022103123/56649d2d5503460f94a03672/html5/thumbnails/45.jpg)
ARMA Process
45
![Page 46: Forecasting JY Le Boudec 1. Contents 1.What is forecasting ? 2.Linear Regression 3.Avoiding Overfitting 4.Differencing 5.ARMA models 6.Sparse ARMA models.](https://reader037.fdocuments.us/reader037/viewer/2022103123/56649d2d5503460f94a03672/html5/thumbnails/46.jpg)
46
![Page 47: Forecasting JY Le Boudec 1. Contents 1.What is forecasting ? 2.Linear Regression 3.Avoiding Overfitting 4.Differencing 5.ARMA models 6.Sparse ARMA models.](https://reader037.fdocuments.us/reader037/viewer/2022103123/56649d2d5503460f94a03672/html5/thumbnails/47.jpg)
ARMA Processes are Gaussian (non iid)
47
![Page 48: Forecasting JY Le Boudec 1. Contents 1.What is forecasting ? 2.Linear Regression 3.Avoiding Overfitting 4.Differencing 5.ARMA models 6.Sparse ARMA models.](https://reader037.fdocuments.us/reader037/viewer/2022103123/56649d2d5503460f94a03672/html5/thumbnails/48.jpg)
48
![Page 49: Forecasting JY Le Boudec 1. Contents 1.What is forecasting ? 2.Linear Regression 3.Avoiding Overfitting 4.Differencing 5.ARMA models 6.Sparse ARMA models.](https://reader037.fdocuments.us/reader037/viewer/2022103123/56649d2d5503460f94a03672/html5/thumbnails/49.jpg)
49
![Page 50: Forecasting JY Le Boudec 1. Contents 1.What is forecasting ? 2.Linear Regression 3.Avoiding Overfitting 4.Differencing 5.ARMA models 6.Sparse ARMA models.](https://reader037.fdocuments.us/reader037/viewer/2022103123/56649d2d5503460f94a03672/html5/thumbnails/50.jpg)
ARIMA Process
50
![Page 51: Forecasting JY Le Boudec 1. Contents 1.What is forecasting ? 2.Linear Regression 3.Avoiding Overfitting 4.Differencing 5.ARMA models 6.Sparse ARMA models.](https://reader037.fdocuments.us/reader037/viewer/2022103123/56649d2d5503460f94a03672/html5/thumbnails/51.jpg)
Fitting an ARMA Process
Called the Box-Jenkins methodDifference the data until stationaryExamine ACF to get a feeling of order (p,q)Fit an ARMA model using maximum likelihood
51
![Page 52: Forecasting JY Le Boudec 1. Contents 1.What is forecasting ? 2.Linear Regression 3.Avoiding Overfitting 4.Differencing 5.ARMA models 6.Sparse ARMA models.](https://reader037.fdocuments.us/reader037/viewer/2022103123/56649d2d5503460f94a03672/html5/thumbnails/52.jpg)
Fitting an ARIMA ModelApply Scientific Method
1. make stationary and normal (how ?)2. bound orders p,q3. fit an ARMA model to Yt -
i.e. Yt - » ARMA 4. compute residuals and verify white noise and normal
Fitting an ARMA modelPb is :
given orders p,q given (x1, …xn) (transformed data) compute the parameters of an ARMA (p,q) model that maximizes
the likelihood
52
A: the mean , the polynomial coefficients k and k , the noise variance 2
Q:What are the parameters ?
![Page 53: Forecasting JY Le Boudec 1. Contents 1.What is forecasting ? 2.Linear Regression 3.Avoiding Overfitting 4.Differencing 5.ARMA models 6.Sparse ARMA models.](https://reader037.fdocuments.us/reader037/viewer/2022103123/56649d2d5503460f94a03672/html5/thumbnails/53.jpg)
This is a non-linear optimization problem
Maximizing the likelihood is a non-linear optimization problems
Usually solved by iterative, heuristic algorithms, may converge to a local maximummay not converge
Some simple, non MLE, heuristics exist for AR or MA modelsEx: fit the AR model that has the same theoretical ACF as the sample ACF
Common practice is to bootstrap the optimization procedure by starting with a “best guess”
AR or MA fit, using heuristic above
53
![Page 54: Forecasting JY Le Boudec 1. Contents 1.What is forecasting ? 2.Linear Regression 3.Avoiding Overfitting 4.Differencing 5.ARMA models 6.Sparse ARMA models.](https://reader037.fdocuments.us/reader037/viewer/2022103123/56649d2d5503460f94a03672/html5/thumbnails/54.jpg)
Fitting ARMA Model is Same as Minimizing One-Step ahead prediction error
54
![Page 55: Forecasting JY Le Boudec 1. Contents 1.What is forecasting ? 2.Linear Regression 3.Avoiding Overfitting 4.Differencing 5.ARMA models 6.Sparse ARMA models.](https://reader037.fdocuments.us/reader037/viewer/2022103123/56649d2d5503460f94a03672/html5/thumbnails/55.jpg)
Best Model Order
55
![Page 56: Forecasting JY Le Boudec 1. Contents 1.What is forecasting ? 2.Linear Regression 3.Avoiding Overfitting 4.Differencing 5.ARMA models 6.Sparse ARMA models.](https://reader037.fdocuments.us/reader037/viewer/2022103123/56649d2d5503460f94a03672/html5/thumbnails/56.jpg)
Check the Residuals
56
![Page 57: Forecasting JY Le Boudec 1. Contents 1.What is forecasting ? 2.Linear Regression 3.Avoiding Overfitting 4.Differencing 5.ARMA models 6.Sparse ARMA models.](https://reader037.fdocuments.us/reader037/viewer/2022103123/56649d2d5503460f94a03672/html5/thumbnails/57.jpg)
Example
57
![Page 58: Forecasting JY Le Boudec 1. Contents 1.What is forecasting ? 2.Linear Regression 3.Avoiding Overfitting 4.Differencing 5.ARMA models 6.Sparse ARMA models.](https://reader037.fdocuments.us/reader037/viewer/2022103123/56649d2d5503460f94a03672/html5/thumbnails/58.jpg)
58
![Page 59: Forecasting JY Le Boudec 1. Contents 1.What is forecasting ? 2.Linear Regression 3.Avoiding Overfitting 4.Differencing 5.ARMA models 6.Sparse ARMA models.](https://reader037.fdocuments.us/reader037/viewer/2022103123/56649d2d5503460f94a03672/html5/thumbnails/59.jpg)
59
Forecasting with ARMA
Assume Yt is fitted to an ARMA process
The prediction problem is: given Y1=y1,…,Yt=yt find the conditional distribution of Yt+h
We know it is normal, with a mean that depends on (y1,…,yt) and a variance that depends only on the fitted parameters of the ARMA process
There are many ways to compute this; it is readily done by Matlab
![Page 60: Forecasting JY Le Boudec 1. Contents 1.What is forecasting ? 2.Linear Regression 3.Avoiding Overfitting 4.Differencing 5.ARMA models 6.Sparse ARMA models.](https://reader037.fdocuments.us/reader037/viewer/2022103123/56649d2d5503460f94a03672/html5/thumbnails/60.jpg)
60
![Page 61: Forecasting JY Le Boudec 1. Contents 1.What is forecasting ? 2.Linear Regression 3.Avoiding Overfitting 4.Differencing 5.ARMA models 6.Sparse ARMA models.](https://reader037.fdocuments.us/reader037/viewer/2022103123/56649d2d5503460f94a03672/html5/thumbnails/61.jpg)
61
![Page 62: Forecasting JY Le Boudec 1. Contents 1.What is forecasting ? 2.Linear Regression 3.Avoiding Overfitting 4.Differencing 5.ARMA models 6.Sparse ARMA models.](https://reader037.fdocuments.us/reader037/viewer/2022103123/56649d2d5503460f94a03672/html5/thumbnails/62.jpg)
Forecasting Formulae for ARIMA
Y = original dataX = differenced data, fitted to an ARMA model1. Obtain point prediction for X using what we just saw2. Apply Proposition 6.4.1 to obtain point prediction for Y3. Apply formula for prediction interval
There are several other methods, but they may have numerical problems. See comments in the lecture notes after prop 6.5.2
62
![Page 63: Forecasting JY Le Boudec 1. Contents 1.What is forecasting ? 2.Linear Regression 3.Avoiding Overfitting 4.Differencing 5.ARMA models 6.Sparse ARMA models.](https://reader037.fdocuments.us/reader037/viewer/2022103123/56649d2d5503460f94a03672/html5/thumbnails/63.jpg)
63
![Page 64: Forecasting JY Le Boudec 1. Contents 1.What is forecasting ? 2.Linear Regression 3.Avoiding Overfitting 4.Differencing 5.ARMA models 6.Sparse ARMA models.](https://reader037.fdocuments.us/reader037/viewer/2022103123/56649d2d5503460f94a03672/html5/thumbnails/64.jpg)
Improve Confidence Interval If Residuals are not Gaussian (but appear to be iid)
Assume residuals are not gaussian but are iidHow can we get confidence intervals ?
Bootstrap by sampling from residuals
64
![Page 65: Forecasting JY Le Boudec 1. Contents 1.What is forecasting ? 2.Linear Regression 3.Avoiding Overfitting 4.Differencing 5.ARMA models 6.Sparse ARMA models.](https://reader037.fdocuments.us/reader037/viewer/2022103123/56649d2d5503460f94a03672/html5/thumbnails/65.jpg)
65
![Page 66: Forecasting JY Le Boudec 1. Contents 1.What is forecasting ? 2.Linear Regression 3.Avoiding Overfitting 4.Differencing 5.ARMA models 6.Sparse ARMA models.](https://reader037.fdocuments.us/reader037/viewer/2022103123/56649d2d5503460f94a03672/html5/thumbnails/66.jpg)
With bootstrap from residuals
With gaussian assumption
66
![Page 67: Forecasting JY Le Boudec 1. Contents 1.What is forecasting ? 2.Linear Regression 3.Avoiding Overfitting 4.Differencing 5.ARMA models 6.Sparse ARMA models.](https://reader037.fdocuments.us/reader037/viewer/2022103123/56649d2d5503460f94a03672/html5/thumbnails/67.jpg)
6. Sparse ARMA Models
Problem: avoid many parameters when the degree of the A and C polynomials is high, as in the previous example
Based on heuristicsMultiplicative ARIMA, constrained ARIMAHolt Winters
67
![Page 68: Forecasting JY Le Boudec 1. Contents 1.What is forecasting ? 2.Linear Regression 3.Avoiding Overfitting 4.Differencing 5.ARMA models 6.Sparse ARMA models.](https://reader037.fdocuments.us/reader037/viewer/2022103123/56649d2d5503460f94a03672/html5/thumbnails/68.jpg)
68
![Page 69: Forecasting JY Le Boudec 1. Contents 1.What is forecasting ? 2.Linear Regression 3.Avoiding Overfitting 4.Differencing 5.ARMA models 6.Sparse ARMA models.](https://reader037.fdocuments.us/reader037/viewer/2022103123/56649d2d5503460f94a03672/html5/thumbnails/69.jpg)
Holt Winters Model 1: EWMA
69
![Page 70: Forecasting JY Le Boudec 1. Contents 1.What is forecasting ? 2.Linear Regression 3.Avoiding Overfitting 4.Differencing 5.ARMA models 6.Sparse ARMA models.](https://reader037.fdocuments.us/reader037/viewer/2022103123/56649d2d5503460f94a03672/html5/thumbnails/70.jpg)
70
![Page 71: Forecasting JY Le Boudec 1. Contents 1.What is forecasting ? 2.Linear Regression 3.Avoiding Overfitting 4.Differencing 5.ARMA models 6.Sparse ARMA models.](https://reader037.fdocuments.us/reader037/viewer/2022103123/56649d2d5503460f94a03672/html5/thumbnails/71.jpg)
EWMA is OK when there is no trend and no periodicity
71
![Page 72: Forecasting JY Le Boudec 1. Contents 1.What is forecasting ? 2.Linear Regression 3.Avoiding Overfitting 4.Differencing 5.ARMA models 6.Sparse ARMA models.](https://reader037.fdocuments.us/reader037/viewer/2022103123/56649d2d5503460f94a03672/html5/thumbnails/72.jpg)
72
![Page 73: Forecasting JY Le Boudec 1. Contents 1.What is forecasting ? 2.Linear Regression 3.Avoiding Overfitting 4.Differencing 5.ARMA models 6.Sparse ARMA models.](https://reader037.fdocuments.us/reader037/viewer/2022103123/56649d2d5503460f94a03672/html5/thumbnails/73.jpg)
73
![Page 74: Forecasting JY Le Boudec 1. Contents 1.What is forecasting ? 2.Linear Regression 3.Avoiding Overfitting 4.Differencing 5.ARMA models 6.Sparse ARMA models.](https://reader037.fdocuments.us/reader037/viewer/2022103123/56649d2d5503460f94a03672/html5/thumbnails/74.jpg)
74
![Page 75: Forecasting JY Le Boudec 1. Contents 1.What is forecasting ? 2.Linear Regression 3.Avoiding Overfitting 4.Differencing 5.ARMA models 6.Sparse ARMA models.](https://reader037.fdocuments.us/reader037/viewer/2022103123/56649d2d5503460f94a03672/html5/thumbnails/75.jpg)
75
![Page 76: Forecasting JY Le Boudec 1. Contents 1.What is forecasting ? 2.Linear Regression 3.Avoiding Overfitting 4.Differencing 5.ARMA models 6.Sparse ARMA models.](https://reader037.fdocuments.us/reader037/viewer/2022103123/56649d2d5503460f94a03672/html5/thumbnails/76.jpg)
76
![Page 77: Forecasting JY Le Boudec 1. Contents 1.What is forecasting ? 2.Linear Regression 3.Avoiding Overfitting 4.Differencing 5.ARMA models 6.Sparse ARMA models.](https://reader037.fdocuments.us/reader037/viewer/2022103123/56649d2d5503460f94a03672/html5/thumbnails/77.jpg)
77
![Page 78: Forecasting JY Le Boudec 1. Contents 1.What is forecasting ? 2.Linear Regression 3.Avoiding Overfitting 4.Differencing 5.ARMA models 6.Sparse ARMA models.](https://reader037.fdocuments.us/reader037/viewer/2022103123/56649d2d5503460f94a03672/html5/thumbnails/78.jpg)
Sparse models give less accurate predictions but have much fewer parameters and are simple to fit.
78
Constrained ARIMA
(corrected or not)
![Page 79: Forecasting JY Le Boudec 1. Contents 1.What is forecasting ? 2.Linear Regression 3.Avoiding Overfitting 4.Differencing 5.ARMA models 6.Sparse ARMA models.](https://reader037.fdocuments.us/reader037/viewer/2022103123/56649d2d5503460f94a03672/html5/thumbnails/79.jpg)
79
7. Case Studies
![Page 80: Forecasting JY Le Boudec 1. Contents 1.What is forecasting ? 2.Linear Regression 3.Avoiding Overfitting 4.Differencing 5.ARMA models 6.Sparse ARMA models.](https://reader037.fdocuments.us/reader037/viewer/2022103123/56649d2d5503460f94a03672/html5/thumbnails/80.jpg)
80
![Page 81: Forecasting JY Le Boudec 1. Contents 1.What is forecasting ? 2.Linear Regression 3.Avoiding Overfitting 4.Differencing 5.ARMA models 6.Sparse ARMA models.](https://reader037.fdocuments.us/reader037/viewer/2022103123/56649d2d5503460f94a03672/html5/thumbnails/81.jpg)
81
![Page 82: Forecasting JY Le Boudec 1. Contents 1.What is forecasting ? 2.Linear Regression 3.Avoiding Overfitting 4.Differencing 5.ARMA models 6.Sparse ARMA models.](https://reader037.fdocuments.us/reader037/viewer/2022103123/56649d2d5503460f94a03672/html5/thumbnails/82.jpg)
82
![Page 83: Forecasting JY Le Boudec 1. Contents 1.What is forecasting ? 2.Linear Regression 3.Avoiding Overfitting 4.Differencing 5.ARMA models 6.Sparse ARMA models.](https://reader037.fdocuments.us/reader037/viewer/2022103123/56649d2d5503460f94a03672/html5/thumbnails/83.jpg)
83
![Page 84: Forecasting JY Le Boudec 1. Contents 1.What is forecasting ? 2.Linear Regression 3.Avoiding Overfitting 4.Differencing 5.ARMA models 6.Sparse ARMA models.](https://reader037.fdocuments.us/reader037/viewer/2022103123/56649d2d5503460f94a03672/html5/thumbnails/84.jpg)
84
![Page 85: Forecasting JY Le Boudec 1. Contents 1.What is forecasting ? 2.Linear Regression 3.Avoiding Overfitting 4.Differencing 5.ARMA models 6.Sparse ARMA models.](https://reader037.fdocuments.us/reader037/viewer/2022103123/56649d2d5503460f94a03672/html5/thumbnails/85.jpg)
85
![Page 86: Forecasting JY Le Boudec 1. Contents 1.What is forecasting ? 2.Linear Regression 3.Avoiding Overfitting 4.Differencing 5.ARMA models 6.Sparse ARMA models.](https://reader037.fdocuments.us/reader037/viewer/2022103123/56649d2d5503460f94a03672/html5/thumbnails/86.jpg)
86
![Page 87: Forecasting JY Le Boudec 1. Contents 1.What is forecasting ? 2.Linear Regression 3.Avoiding Overfitting 4.Differencing 5.ARMA models 6.Sparse ARMA models.](https://reader037.fdocuments.us/reader037/viewer/2022103123/56649d2d5503460f94a03672/html5/thumbnails/87.jpg)
87
![Page 88: Forecasting JY Le Boudec 1. Contents 1.What is forecasting ? 2.Linear Regression 3.Avoiding Overfitting 4.Differencing 5.ARMA models 6.Sparse ARMA models.](https://reader037.fdocuments.us/reader037/viewer/2022103123/56649d2d5503460f94a03672/html5/thumbnails/88.jpg)
88
h = 1h = 1
![Page 89: Forecasting JY Le Boudec 1. Contents 1.What is forecasting ? 2.Linear Regression 3.Avoiding Overfitting 4.Differencing 5.ARMA models 6.Sparse ARMA models.](https://reader037.fdocuments.us/reader037/viewer/2022103123/56649d2d5503460f94a03672/html5/thumbnails/89.jpg)
89
![Page 90: Forecasting JY Le Boudec 1. Contents 1.What is forecasting ? 2.Linear Regression 3.Avoiding Overfitting 4.Differencing 5.ARMA models 6.Sparse ARMA models.](https://reader037.fdocuments.us/reader037/viewer/2022103123/56649d2d5503460f94a03672/html5/thumbnails/90.jpg)
90
h = 2h = 2
![Page 91: Forecasting JY Le Boudec 1. Contents 1.What is forecasting ? 2.Linear Regression 3.Avoiding Overfitting 4.Differencing 5.ARMA models 6.Sparse ARMA models.](https://reader037.fdocuments.us/reader037/viewer/2022103123/56649d2d5503460f94a03672/html5/thumbnails/91.jpg)
91
![Page 92: Forecasting JY Le Boudec 1. Contents 1.What is forecasting ? 2.Linear Regression 3.Avoiding Overfitting 4.Differencing 5.ARMA models 6.Sparse ARMA models.](https://reader037.fdocuments.us/reader037/viewer/2022103123/56649d2d5503460f94a03672/html5/thumbnails/92.jpg)
92
logh = 1log
h = 1
![Page 93: Forecasting JY Le Boudec 1. Contents 1.What is forecasting ? 2.Linear Regression 3.Avoiding Overfitting 4.Differencing 5.ARMA models 6.Sparse ARMA models.](https://reader037.fdocuments.us/reader037/viewer/2022103123/56649d2d5503460f94a03672/html5/thumbnails/93.jpg)
Conclusion
Forecasting is useful when savings matter; for exampleSave money on server space rentalSave energy
Capturing determinism is perhaps most important and easiest
Prediction intervals are useful to avoid gross mistakes
Re-scaling the data may help
… à vous de jouer.
93
![Page 94: Forecasting JY Le Boudec 1. Contents 1.What is forecasting ? 2.Linear Regression 3.Avoiding Overfitting 4.Differencing 5.ARMA models 6.Sparse ARMA models.](https://reader037.fdocuments.us/reader037/viewer/2022103123/56649d2d5503460f94a03672/html5/thumbnails/94.jpg)
94
Joyeuses Pâques !