7 Transformations
-
Upload
justinlondon -
Category
Documents
-
view
223 -
download
1
description
Transcript of 7 Transformations
Forecasting using
7. Transformations and adjustments
OTexts.com/fpp/2/4/
Forecasting using R 1
Rob J Hyndman
Outline
1 Exponential smoothing
2 Transformations
3 Adjustments
Forecasting using R Exponential smoothing 2
Exponential smoothing
ets() function
Automatically chooses a model by default usingthe AIC, AICc or BIC.
Can handle any combination of trend,seasonality and damping
Produces prediction intervals for every model
Ensures the parameters are admissible(equivalent to invertible)
Produces an object of class ets.
Forecasting using R Exponential smoothing 3
Exponential smoothing
ets() function
Automatically chooses a model by default usingthe AIC, AICc or BIC.
Can handle any combination of trend,seasonality and damping
Produces prediction intervals for every model
Ensures the parameters are admissible(equivalent to invertible)
Produces an object of class ets.
Forecasting using R Exponential smoothing 3
Exponential smoothing
ets() function
Automatically chooses a model by default usingthe AIC, AICc or BIC.
Can handle any combination of trend,seasonality and damping
Produces prediction intervals for every model
Ensures the parameters are admissible(equivalent to invertible)
Produces an object of class ets.
Forecasting using R Exponential smoothing 3
Exponential smoothing
ets() function
Automatically chooses a model by default usingthe AIC, AICc or BIC.
Can handle any combination of trend,seasonality and damping
Produces prediction intervals for every model
Ensures the parameters are admissible(equivalent to invertible)
Produces an object of class ets.
Forecasting using R Exponential smoothing 3
Exponential smoothing
ets() function
Automatically chooses a model by default usingthe AIC, AICc or BIC.
Can handle any combination of trend,seasonality and damping
Produces prediction intervals for every model
Ensures the parameters are admissible(equivalent to invertible)
Produces an object of class ets.
Forecasting using R Exponential smoothing 3
Exponential smoothing
ets objects
Methods: coef(), plot(), summary(),
residuals(), fitted(), simulate()
and forecast()
plot() function shows time plots of the
original time series along with the
extracted components (level, growth
and seasonal).
Forecasting using R Exponential smoothing 4
Exponential smoothing
ets objects
Methods: coef(), plot(), summary(),
residuals(), fitted(), simulate()
and forecast()
plot() function shows time plots of the
original time series along with the
extracted components (level, growth
and seasonal).
Forecasting using R Exponential smoothing 4
Exponential smoothing
Forecasting using R Exponential smoothing 5
200
400
600
obse
rved
250
350
450
leve
l
0.99
01.
005
slop
e
0.9
1.1
1960 1970 1980 1990 2000 2010
seas
on
Time
Decomposition by ETS(M,Md,M) methodplot(fit)
Goodness-of-fit
> accuracy(fit)ME RMSE MAE MPE MAPE MASE
0.17847 15.48781 11.77800 0.07204 2.81921 0.20705
> accuracy(fit2)ME RMSE MAE MPE MAPE MASE
-0.11711 15.90526 12.18930 -0.03765 2.91255 0.21428
Forecasting using R Exponential smoothing 6
Forecast intervals
Forecasting using R Exponential smoothing 7
Forecasts from ETS(M,Md,M)
1995 2000 2005 2010
300
350
400
450
500
550
600
> plot(forecast(fit,level=c(50,80,95)))
Forecast intervals
Forecasting using R Exponential smoothing 7
Forecasts from ETS(M,Md,M)
1995 2000 2005 2010
300
350
400
450
500
550
600
> plot(forecast(fit,fan=TRUE))
Exponential smoothing
ets() function also allows refitting model to newdata set.
> usfit <- ets(usnetelec[1:45])> test <- ets(usnetelec[46:55], model = usfit)
> accuracy(test)ME RMSE MAE MPE MAPE MASE
-3.35419 58.02763 43.85545 -0.07624 1.18483 0.52452
> accuracy(forecast(usfit,10), usnetelec[46:55])ME RMSE MAE MPE MAPE MASE
40.7034 61.2075 46.3246 1.0980 1.2620 0.6776
Forecasting using R Exponential smoothing 8
Unstable models
ETS(M,M,A)
ETS(M,Md,A)
ETS(A,N,M)
ETS(A,A,M)
ETS(A,Ad,M)
ETS(A,M,N)
ETS(A,M,A)
ETS(A,M,M)
ETS(A,Md,N)
ETS(A,Md,A)
ETS(A,Md,M)Forecasting using R Exponential smoothing 9
Unstable models
ETS(M,M,A)
ETS(M,Md,A)
ETS(A,N,M)
ETS(A,A,M)
ETS(A,Ad,M)
ETS(A,M,N)
ETS(A,M,A)
ETS(A,M,M)
ETS(A,Md,N)
ETS(A,Md,A)
ETS(A,Md,M)Forecasting using R Exponential smoothing 9
In practice, the modelswork fine for short- tomedium-term forecastsprovided the data arestrictly positive.
Forecastability conditions
ets(y, model="ZZZ", damped=NULL, alpha=NULL,beta=NULL, gamma=NULL, phi=NULL,additive.only=FALSE,lower=c(rep(0.0001,3),0.80),upper=c(rep(0.9999,3),0.98),opt.crit=c("lik","amse","mse","sigma"),nmse=3,bounds=c("both","usual","admissible"),ic=c("aic","aicc","bic"), restrict=TRUE)
Forecasting using R Exponential smoothing 10
The magic forecast() function
forecast returns forecasts when applied to anets object (or the output from many other timeseries models).
If you use forecast directly on data, it willselect an ETS model automatically and thenreturn forecasts.
Forecasting using R Exponential smoothing 11
The magic forecast() function
forecast returns forecasts when applied to anets object (or the output from many other timeseries models).
If you use forecast directly on data, it willselect an ETS model automatically and thenreturn forecasts.
Forecasting using R Exponential smoothing 11
Outline
1 Exponential smoothing
2 Transformations
3 Adjustments
Forecasting using R Transformations 12
Transformations to stabilize the variance
If the data show different variation at different levels of theseries, then a transformation can be useful.Denote original observations as y1, . . . , yn and transformedobservations as w1, . . . ,wn.
Mathematical transformations for stabilizing variation
Square root wt =√yt ↓
Cube root wt = 3√yt Increasing
Logarithm wt = log(yt) strength
Logarithms, in particular, are useful because they are more
interpretable: changes in a log value are relative (percent)
changes on the original scale.
Forecasting using R Transformations 13
Transformations to stabilize the variance
If the data show different variation at different levels of theseries, then a transformation can be useful.Denote original observations as y1, . . . , yn and transformedobservations as w1, . . . ,wn.
Mathematical transformations for stabilizing variation
Square root wt =√yt ↓
Cube root wt = 3√yt Increasing
Logarithm wt = log(yt) strength
Logarithms, in particular, are useful because they are more
interpretable: changes in a log value are relative (percent)
changes on the original scale.
Forecasting using R Transformations 13
Transformations to stabilize the variance
If the data show different variation at different levels of theseries, then a transformation can be useful.Denote original observations as y1, . . . , yn and transformedobservations as w1, . . . ,wn.
Mathematical transformations for stabilizing variation
Square root wt =√yt ↓
Cube root wt = 3√yt Increasing
Logarithm wt = log(yt) strength
Logarithms, in particular, are useful because they are more
interpretable: changes in a log value are relative (percent)
changes on the original scale.
Forecasting using R Transformations 13
Transformations to stabilize the variance
If the data show different variation at different levels of theseries, then a transformation can be useful.Denote original observations as y1, . . . , yn and transformedobservations as w1, . . . ,wn.
Mathematical transformations for stabilizing variation
Square root wt =√yt ↓
Cube root wt = 3√yt Increasing
Logarithm wt = log(yt) strength
Logarithms, in particular, are useful because they are more
interpretable: changes in a log value are relative (percent)
changes on the original scale.
Forecasting using R Transformations 13
Transformations
Forecasting using R Transformations 14
Square root electricity production
Year
1960 1970 1980 1990
4060
8010
012
0
Transformations
Forecasting using R Transformations 14
Cube root electricity production
Year
1960 1970 1980 1990
1214
1618
2022
24
Transformations
Forecasting using R Transformations 14
Log electricity production
Year
1960 1970 1980 1990
7.5
8.0
8.5
9.0
9.5
Transformations
Forecasting using R Transformations 14
Inverse electricity production
Year
1960 1970 1980 1990
−8e
−04
−6e
−04
−4e
−04
−2e
−04
Box-Cox transformations
Each of these transformations is close to a memberof the family of Box-Cox transformations:
wt =
{log(yt), λ = 0;(yλt − 1)/λ, λ 6= 0.
λ = 1: (No substantive transformation)
λ = 12: (Square root plus linear transformation)
λ = 0: (Natural logarithm)
λ = −1: (Inverse plus 1)
Forecasting using R Transformations 15
Box-Cox transformations
Each of these transformations is close to a memberof the family of Box-Cox transformations:
wt =
{log(yt), λ = 0;(yλt − 1)/λ, λ 6= 0.
λ = 1: (No substantive transformation)
λ = 12: (Square root plus linear transformation)
λ = 0: (Natural logarithm)
λ = −1: (Inverse plus 1)
Forecasting using R Transformations 15
Box-Cox transformations
Each of these transformations is close to a memberof the family of Box-Cox transformations:
wt =
{log(yt), λ = 0;(yλt − 1)/λ, λ 6= 0.
λ = 1: (No substantive transformation)
λ = 12: (Square root plus linear transformation)
λ = 0: (Natural logarithm)
λ = −1: (Inverse plus 1)
Forecasting using R Transformations 15
Box-Cox transformations
Each of these transformations is close to a memberof the family of Box-Cox transformations:
wt =
{log(yt), λ = 0;(yλt − 1)/λ, λ 6= 0.
λ = 1: (No substantive transformation)
λ = 12: (Square root plus linear transformation)
λ = 0: (Natural logarithm)
λ = −1: (Inverse plus 1)
Forecasting using R Transformations 15
Box-Cox transformations
Each of these transformations is close to a memberof the family of Box-Cox transformations:
wt =
{log(yt), λ = 0;(yλt − 1)/λ, λ 6= 0.
λ = 1: (No substantive transformation)
λ = 12: (Square root plus linear transformation)
λ = 0: (Natural logarithm)
λ = −1: (Inverse plus 1)
Forecasting using R Transformations 15
Box-Cox transformations
Forecasting using R Transformations 16
Box-Cox transformations
yλt for λ close to zero behaves like logs.If some yt = 0, then must have λ > 0if some yt < 0, no power transformation ispossible unless all yt adjusted by adding aconstant to all values.Choose a simple value of λ. It makesexplanation easier.Results are relatively insensitive to value of λOften no transformation (λ = 1) needed.Transformation often makes little difference toforecasts but has large effect on PI.Choosing λ = 0 is a simple way to forceforecasts to be positive
Forecasting using R Transformations 17
Box-Cox transformations
yλt for λ close to zero behaves like logs.If some yt = 0, then must have λ > 0if some yt < 0, no power transformation ispossible unless all yt adjusted by adding aconstant to all values.Choose a simple value of λ. It makesexplanation easier.Results are relatively insensitive to value of λOften no transformation (λ = 1) needed.Transformation often makes little difference toforecasts but has large effect on PI.Choosing λ = 0 is a simple way to forceforecasts to be positive
Forecasting using R Transformations 17
Box-Cox transformations
yλt for λ close to zero behaves like logs.If some yt = 0, then must have λ > 0if some yt < 0, no power transformation ispossible unless all yt adjusted by adding aconstant to all values.Choose a simple value of λ. It makesexplanation easier.Results are relatively insensitive to value of λOften no transformation (λ = 1) needed.Transformation often makes little difference toforecasts but has large effect on PI.Choosing λ = 0 is a simple way to forceforecasts to be positive
Forecasting using R Transformations 17
Box-Cox transformations
yλt for λ close to zero behaves like logs.If some yt = 0, then must have λ > 0if some yt < 0, no power transformation ispossible unless all yt adjusted by adding aconstant to all values.Choose a simple value of λ. It makesexplanation easier.Results are relatively insensitive to value of λOften no transformation (λ = 1) needed.Transformation often makes little difference toforecasts but has large effect on PI.Choosing λ = 0 is a simple way to forceforecasts to be positive
Forecasting using R Transformations 17
Box-Cox transformations
yλt for λ close to zero behaves like logs.If some yt = 0, then must have λ > 0if some yt < 0, no power transformation ispossible unless all yt adjusted by adding aconstant to all values.Choose a simple value of λ. It makesexplanation easier.Results are relatively insensitive to value of λOften no transformation (λ = 1) needed.Transformation often makes little difference toforecasts but has large effect on PI.Choosing λ = 0 is a simple way to forceforecasts to be positive
Forecasting using R Transformations 17
Box-Cox transformations
yλt for λ close to zero behaves like logs.If some yt = 0, then must have λ > 0if some yt < 0, no power transformation ispossible unless all yt adjusted by adding aconstant to all values.Choose a simple value of λ. It makesexplanation easier.Results are relatively insensitive to value of λOften no transformation (λ = 1) needed.Transformation often makes little difference toforecasts but has large effect on PI.Choosing λ = 0 is a simple way to forceforecasts to be positive
Forecasting using R Transformations 17
Box-Cox transformations
yλt for λ close to zero behaves like logs.If some yt = 0, then must have λ > 0if some yt < 0, no power transformation ispossible unless all yt adjusted by adding aconstant to all values.Choose a simple value of λ. It makesexplanation easier.Results are relatively insensitive to value of λOften no transformation (λ = 1) needed.Transformation often makes little difference toforecasts but has large effect on PI.Choosing λ = 0 is a simple way to forceforecasts to be positive
Forecasting using R Transformations 17
Box-Cox transformations
yλt for λ close to zero behaves like logs.If some yt = 0, then must have λ > 0if some yt < 0, no power transformation ispossible unless all yt adjusted by adding aconstant to all values.Choose a simple value of λ. It makesexplanation easier.Results are relatively insensitive to value of λOften no transformation (λ = 1) needed.Transformation often makes little difference toforecasts but has large effect on PI.Choosing λ = 0 is a simple way to forceforecasts to be positive
Forecasting using R Transformations 17
Back-transformation
We must reverse the transformation (orback-transform) to obtain forecasts on the originalscale. The reverse Box-Cox transformations aregiven by
yt =
{exp(wt), λ = 0;(λWt + 1)1/λ, λ 6= 0.
plot(BoxCox(elec,lambda=1/3))fit <- snaive(elec, lambda=1/3)plot(fit)plot(fit, include=120)
Forecasting using R Transformations 18
Back-transformation
We must reverse the transformation (orback-transform) to obtain forecasts on the originalscale. The reverse Box-Cox transformations aregiven by
yt =
{exp(wt), λ = 0;(λWt + 1)1/λ, λ 6= 0.
plot(BoxCox(elec,lambda=1/3))fit <- snaive(elec, lambda=1/3)plot(fit)plot(fit, include=120)
Forecasting using R Transformations 18
Automated Box-Cox transformations
BoxCox.lambda(elec)
This attempts to balance the seasonalfluctuations and random variation across theseries.
Always check the results.
A low value of λ can give extremely largeprediction intervals.
Forecasting using R Transformations 19
Automated Box-Cox transformations
BoxCox.lambda(elec)
This attempts to balance the seasonalfluctuations and random variation across theseries.
Always check the results.
A low value of λ can give extremely largeprediction intervals.
Forecasting using R Transformations 19
Automated Box-Cox transformations
BoxCox.lambda(elec)
This attempts to balance the seasonalfluctuations and random variation across theseries.
Always check the results.
A low value of λ can give extremely largeprediction intervals.
Forecasting using R Transformations 19
Automated Box-Cox transformations
BoxCox.lambda(elec)
This attempts to balance the seasonalfluctuations and random variation across theseries.
Always check the results.
A low value of λ can give extremely largeprediction intervals.
Forecasting using R Transformations 19
ETS and transformations
A Box-Cox transformation followed by anadditive ETS model is often better than an ETSmodel without transformation.
A Box-Cox transformation followed by STL +ETS is often better than an ETS model withouttransformation.
It makes no sense to use a Box-Coxtransformation and a non-additive ETS model.
Forecasting using R Transformations 20
ETS and transformations
A Box-Cox transformation followed by anadditive ETS model is often better than an ETSmodel without transformation.
A Box-Cox transformation followed by STL +ETS is often better than an ETS model withouttransformation.
It makes no sense to use a Box-Coxtransformation and a non-additive ETS model.
Forecasting using R Transformations 20
ETS and transformations
A Box-Cox transformation followed by anadditive ETS model is often better than an ETSmodel without transformation.
A Box-Cox transformation followed by STL +ETS is often better than an ETS model withouttransformation.
It makes no sense to use a Box-Coxtransformation and a non-additive ETS model.
Forecasting using R Transformations 20
Outline
1 Exponential smoothing
2 Transformations
3 Adjustments
Forecasting using R Adjustments 21
Calendar adjustments
Some of the variation in a time series may be due tothe variation in the number of trading days eachmonth. It is a good idea to adjust for this knownsource of variation to allow study of otherinteresting features.
Month length
Trading day
Forecasting using R Adjustments 22
Calendar adjustments
Some of the variation in a time series may be due tothe variation in the number of trading days eachmonth. It is a good idea to adjust for this knownsource of variation to allow study of otherinteresting features.
Month length
Trading day
Forecasting using R Adjustments 22
Month length adjustment
If this is not removed, it shows up as a seasonaleffect, which may not cause problems though itdoes make any seasonal pattern hard to interpret. Itis easily adjusted for:
y∗t = yt ×no. of days in an average month
no. of days in month t
= yt ×365.25/12
no. of days in month t
where yt has already been transformed if necessary.
monthdays gives the number of days in each monthor quarter.
Forecasting using R Adjustments 23
Month length adjustment
If this is not removed, it shows up as a seasonaleffect, which may not cause problems though itdoes make any seasonal pattern hard to interpret. Itis easily adjusted for:
y∗t = yt ×no. of days in an average month
no. of days in month t
= yt ×365.25/12
no. of days in month t
where yt has already been transformed if necessary.
monthdays gives the number of days in each monthor quarter.
Forecasting using R Adjustments 23
Trading day adjustmentoccurs in monthly data when there is also aweekly cycle, since proportions of various daysin given month vary from year to year.number of trading days is predictable, buteffects of various days are unknown.Simplest case: All trading days assumed tohave same effect.
y∗t = yt ×no. of trading days in average month
no. of trading days in month t.
where yt has already been adjusted for monthlength and transformed if necessary.If weekly cycle more complex, these effectsmust be estimated from data.
Forecasting using R Adjustments 24
Trading day adjustmentoccurs in monthly data when there is also aweekly cycle, since proportions of various daysin given month vary from year to year.number of trading days is predictable, buteffects of various days are unknown.Simplest case: All trading days assumed tohave same effect.
y∗t = yt ×no. of trading days in average month
no. of trading days in month t.
where yt has already been adjusted for monthlength and transformed if necessary.If weekly cycle more complex, these effectsmust be estimated from data.
Forecasting using R Adjustments 24
Trading day adjustmentoccurs in monthly data when there is also aweekly cycle, since proportions of various daysin given month vary from year to year.number of trading days is predictable, buteffects of various days are unknown.Simplest case: All trading days assumed tohave same effect.
y∗t = yt ×no. of trading days in average month
no. of trading days in month t.
where yt has already been adjusted for monthlength and transformed if necessary.If weekly cycle more complex, these effectsmust be estimated from data.
Forecasting using R Adjustments 24
Trading day adjustmentoccurs in monthly data when there is also aweekly cycle, since proportions of various daysin given month vary from year to year.number of trading days is predictable, buteffects of various days are unknown.Simplest case: All trading days assumed tohave same effect.
y∗t = yt ×no. of trading days in average month
no. of trading days in month t.
where yt has already been adjusted for monthlength and transformed if necessary.If weekly cycle more complex, these effectsmust be estimated from data.
Forecasting using R Adjustments 24
Explainable variation
Examples:
Calendar variation
Increasing population
Inflation
Strikes
Changes in government
Changes in law
Try and understand all possible sources of variationbefore modelling the time series.
Forecasting using R Adjustments 25
Explainable variation
Examples:
Calendar variation
Increasing population
Inflation
Strikes
Changes in government
Changes in law
Try and understand all possible sources of variationbefore modelling the time series.
Forecasting using R Adjustments 25
Explainable variation
Examples:
Calendar variation
Increasing population
Inflation
Strikes
Changes in government
Changes in law
Try and understand all possible sources of variationbefore modelling the time series.
Forecasting using R Adjustments 25
Explainable variation
Examples:
Calendar variation
Increasing population
Inflation
Strikes
Changes in government
Changes in law
Try and understand all possible sources of variationbefore modelling the time series.
Forecasting using R Adjustments 25
Explainable variation
Examples:
Calendar variation
Increasing population
Inflation
Strikes
Changes in government
Changes in law
Try and understand all possible sources of variationbefore modelling the time series.
Forecasting using R Adjustments 25
Explainable variation
Examples:
Calendar variation
Increasing population
Inflation
Strikes
Changes in government
Changes in law
Try and understand all possible sources of variationbefore modelling the time series.
Forecasting using R Adjustments 25
Explainable variation
Examples:
Calendar variation
Increasing population
Inflation
Strikes
Changes in government
Changes in law
Try and understand all possible sources of variationbefore modelling the time series.
Forecasting using R Adjustments 25
Explainable variation
Examples:
Calendar variation
Increasing population
Inflation
Strikes
Changes in government
Changes in law
Try and understand all possible sources of variationbefore modelling the time series.
Forecasting using R Adjustments 25