Post on 31-Aug-2019
Lesson 9: Autoregressive-MovingAverage (ARMA) models
Umberto Triacca
Dipartimento di Ingegneria e Scienze dell’Informazione e MatematicaUniversita dell’Aquila,
umberto.triacca@ec.univaq.it
Umberto Triacca Lesson 9: Autoregressive-Moving Average (ARMA) models
Introduction
We have seen that in the class of stationary, zero mean,Gaussian processes the probabilistic structure of a stochasticprocess is completly characterized by the autocovariancefunction.
Umberto Triacca Lesson 9: Autoregressive-Moving Average (ARMA) models
Autocovariance function
Stationary, zero mean, Gaussian process
&%'$
DGP
?
γx(k)
�����������7
��
��x1, ..., xT
Umberto Triacca Lesson 9: Autoregressive-Moving Average (ARMA) models
Introduction
However, in general, to know the autocovariance functionmeans to know a sequence composed by an infinite number ofelements.
We have to estimate a infinite number of parameters
γx(0), γx(1), γx(2), ...,
from observed data.
This mission is impossible
Umberto Triacca Lesson 9: Autoregressive-Moving Average (ARMA) models
Introduction
We introduce a very important class of stochastic processes,which autocovariance functions depend on a finite number ofunknown parameters:
the class of the AutoregRessive Moving Average (ARMA)processes.
Umberto Triacca Lesson 9: Autoregressive-Moving Average (ARMA) models
Autoregressive-Moving Average (ARMA) models
Definition. The process {xt ; t ∈ Z} is an autoregressivemoving average process of order (p, q), denoted with
xt ∼ ARMA(p, q),
if
xt −φ1xt−1− ...−φpxt−p = ut + θ1ut−1 + ...+ θqut−q ∀t ∈ Z,
where ut ∼ WN(0, σ2u), and φ1, ..., φp, θ1, ..., θq are p + q
constants and the polynomials
φ(z) = 1− φ1z − ...− φpzp
andθ(z) = 1 + θ1z ... + θqz
q
have no common factors.Umberto Triacca Lesson 9: Autoregressive-Moving Average (ARMA) models
Autoregressive-Moving Average (ARMA) models
For q = 0 the process reduces to an autoregressive process oforder p, denoted with xt ∼ AR(p),
xt − φ1xt−1 − ...− φpxt−p = ut ∀t ∈ Z,
For p = 0 to a moving average process of order q, denotedwith xt ∼ MA(q)
xt = ut + θ1ut−1 + ... + θqut−q ∀t ∈ Z,
Umberto Triacca Lesson 9: Autoregressive-Moving Average (ARMA) models
An example of Autoregressive-Moving Average
(ARMA) process
The process {xt ; t ∈ Z} defined by
xt = 0.3xt−1 + ut + 0.7ut−1 ∀t ∈ Z,
where ut ∼ WN(0, σ2u), is an ARMA(1,1) process.
Hereφ(z) = 1− 0.3z
andθ(z) = 1 + 0.7z .
Umberto Triacca Lesson 9: Autoregressive-Moving Average (ARMA) models
An example of Autoregressive-Moving Average
(ARMA) process
A realizzation of the ARMA(1,1) processxt = 0.3xt−1 + ut + 0.7ut−1 is presented in the following figure.
Umberto Triacca Lesson 9: Autoregressive-Moving Average (ARMA) models
An example of Autoregressive (AR) process
The process {xt ; t ∈ Z} defined by
xt = 0.7xt−1 − 0.5xt−1 + ut ∀t ∈ Z,
where ut ∼ WN(0, σ2u), is an AR(2) process.
Hereφ(z) = 1− 0.7z + 0.5z2
Umberto Triacca Lesson 9: Autoregressive-Moving Average (ARMA) models
An example of Autoregressive (AR) process
A realizzation of the AR(2) processxt = 0.7xt−1− 0.5xt−2 + ut is presented in the following figure.
Umberto Triacca Lesson 9: Autoregressive-Moving Average (ARMA) models
An example of Moving Average (MA) process
The process {xt ; t ∈ Z} defined by
xt = ut + 0.7ut−1 ∀t ∈ Z,
where ut ∼ WN(0, σ2u), is an MA(1) process.
Hereθ(z) = 1 + 0.7z
Umberto Triacca Lesson 9: Autoregressive-Moving Average (ARMA) models
An example of Moving Average (MA) process
A realizzation of the MA(1) process xt = ut + 0.7ut−1 ispresented in the following figure.
Umberto Triacca Lesson 9: Autoregressive-Moving Average (ARMA) models
An example of over-parameterization
Consider the process {xt ; t ∈ Z} defined by
xt = xt−1 − 0.21xt−2 + ut − 0.7ut−1 ∀t ∈ Z,
where ut ∼ WN(0, σ2u).
This process looks like an ARMA(2,1) process but it is not anARMA(2,1) process.
Umberto Triacca Lesson 9: Autoregressive-Moving Average (ARMA) models
An example of over-parameterization
Here
φ(z) = 1− z + 0.21z2 = (1− 0.7z)(1− 0.3z)
andθ(z) = 1− 0.7z
We note that both polynomials have a common factor, namely1− 0.7z . Discarding the common factor in each leaves
φ∗(z) = 1− 0.3z
andθ∗(z) = 1.
Thus the process is an AR(1) process, defined byxt = 0.3xt−1 + ut
Umberto Triacca Lesson 9: Autoregressive-Moving Average (ARMA) models
Causal Autoregressive-Moving Average (ARMA)
models
Definition. An ARMA(p, q) process {xt ; t ∈ Z} is causal(strictly, a causal function of {ut ; t ∈ Z}) if there existsconstants ψ0, ψ1, ... such that
∞∑j=0
|ψj | <∞
and
xt =∞∑j=0
ψjut−j ∀t.
Umberto Triacca Lesson 9: Autoregressive-Moving Average (ARMA) models
Autoregressive-Moving Average (ARMA) models
Here, it is important to clarify the meaning of equality
xt =∞∑j=0
ψjut−j ∀t
It means that
limn→∞
E
(xt − n∑j=0
ψjut−j
)2 = 0.
The equality is defined in terms of a limit in the quadraticmean.
Umberto Triacca Lesson 9: Autoregressive-Moving Average (ARMA) models
Autoregressive-Moving Average (ARMA) models
The following two theorems provide, respectively, acharacterization of the of causality and stationarity of anARMA(p, q) process.
Umberto Triacca Lesson 9: Autoregressive-Moving Average (ARMA) models
Autoregressive-Moving Average (ARMA) models
Theorem. An ARMA(p, q) process {xt ; t ∈ Z} is causal ifand only if
φ(z) = 1− φ1z − ...− φpzp 6= 0 for all |z | ≤ 1.
Theorem. An ARMA(p, q) process {xt ; t ∈ Z} is stationaryif and only if
φ(z) = 1− φ1z − ...− φpzp 6= 0 for all |z | = 1.
Umberto Triacca Lesson 9: Autoregressive-Moving Average (ARMA) models
Autoregressive-Moving Average (ARMA) models
The causality and the stationarity of an ARMA process dependentirely on the autoregressive parameters and not on themoving-average ones.
Umberto Triacca Lesson 9: Autoregressive-Moving Average (ARMA) models
Autoregressive-Moving Average (ARMA) models
Further, we note that if an ARMA(p, q) process is causal, thenis stationary, but stationarity does not imply causality.
Umberto Triacca Lesson 9: Autoregressive-Moving Average (ARMA) models
Autoregressive-Moving Average (ARMA) models
Consider, for example, the following AR(1) process:
xt = 3xt−1 + ut
where ut WN(0, σ2u). We have that
φ(z) = 1− 3z 6= 0 for all |z | = 1.
and hence the process is stationary, but non causal since
φ(z) = 1− 3z = 0 for z = 1/3.
Umberto Triacca Lesson 9: Autoregressive-Moving Average (ARMA) models
Autoregressive-Moving Average (ARMA) models
An important result: There is a one-to-one correspondencebetween the parameters of a causal ARMA(p,q) process andthe autocovariance function.
Umberto Triacca Lesson 9: Autoregressive-Moving Average (ARMA) models
Autoregressive-Moving Average (ARMA) models
It is important to underline that if we consider the set ofautocorrelation functions there is not a one-to-onecorrespondence between the parameters of a causalARMA(p,q) process and the autocorrelation function.
Umberto Triacca Lesson 9: Autoregressive-Moving Average (ARMA) models
Autoregressive-Moving Average (ARMA) models
Consider the following two MA(1) processes.
xt = ut + θut−1
where ut ∼ WN(0, σ2u), with |θ| < 1 and
yt = ut +1
θut−1
where ut ∼ WN(0, σ2u).
Sinceθ
1 + θ2=
1/θ
1 + (1/θ)2 ,
we have that both processes share the same autocorrelationfunction. Thus it cannot be used to distinguish between thetwo parametrizations.
Umberto Triacca Lesson 9: Autoregressive-Moving Average (ARMA) models
Autoregressive-Moving Average (ARMA) models
This example shows that an MA(1)-process is not uniquelydetermined by its autocorrelation function. There is anidentification problem with the MA(1) models.
In general, (if all roots of θ(z) = 0 are real) there can be 2q
different MA(q) processes with the same autocorrelationfunction.
Umberto Triacca Lesson 9: Autoregressive-Moving Average (ARMA) models
Autoregressive-Moving Average (ARMA) models
Definition. An ARMA(p, q) process {xt ; t ∈ Z} is invertible(strictly, an invertible function of {ut ; t ∈ Z}) if there existsconstants π0, π1, ... such that
∞∑j=0
|πj | <∞
and
ut =∞∑j=0
πjxt−j ∀t.
Umberto Triacca Lesson 9: Autoregressive-Moving Average (ARMA) models
Autoregressive-Moving Average (ARMA) models
The following theorem provides a necessary and sufficientcondition for the invertibility.
Theorem. An ARMA(p, q) process {xt ; t ∈ Z} is invertibleif and only if
θ(z) = 1 + θ1z + ... + θqzq 6= 0 for all |z | ≤ 1.
Umberto Triacca Lesson 9: Autoregressive-Moving Average (ARMA) models
Autoregressive-Moving Average (ARMA) models
We note that an AR(p) process is always invertible, even if itis non-stationary, while an MA(q) process is always stationary,even if it is non-invertible.
Umberto Triacca Lesson 9: Autoregressive-Moving Average (ARMA) models
Autoregressive-Moving Average (ARMA) models
The invertibility can be used in order to ensure theidentifiability of MA processes.
Umberto Triacca Lesson 9: Autoregressive-Moving Average (ARMA) models
Autoregressive-Moving Average (ARMA) models
In general, (if all roots of θ(z) = 0 are real) there can be 2q
different MA(q) processes with the same autocorrelationfunction, but only one of these is invertible.
Umberto Triacca Lesson 9: Autoregressive-Moving Average (ARMA) models
Conclusion
In the class of the mean-zero causal and invertible GaussianARMA processes there is a one-to-one correspondencebetween the family of the finite dimensional distributions ofthe process and the finite parametric representation of process.
Umberto Triacca Lesson 9: Autoregressive-Moving Average (ARMA) models
Conclusion
In the class of the mean-zero causal and invertibleGaussian ARMA processes the probabilistic properties of theprocess are completely characterized by the finite set ofparameters {
φ1, φ2, ..., φp, θ1, θ2, ..., θq, σ2u
}Now, we have to estimate a finite number (p + q + 1) ofparameters from observed data.
This mission is possible.
Umberto Triacca Lesson 9: Autoregressive-Moving Average (ARMA) models
Conclusion
Zero-mean causal invertible Gaussian ARMA process
&%'$
DGP
?
{φ1, φ2, ..., φp, θ1, θ2, ..., θq, σ2u}
�����������7
��
��x1, ..., xT
Umberto Triacca Lesson 9: Autoregressive-Moving Average (ARMA) models