Time series

Introduction to Time Series Analysis. Lecture 6.Peter Bartlett

www.stat.berkeley.edu/bartlett/courses/153-fall2010

Last lecture:

1. Causality

2. Invertibility

3. AR(p) models4. ARMA(p,q) models

1



1. ARMA(p,q) models2. Stationarity, causality and invertibility

3. The linear process representation of ARMA processes: .

4. Autocovariance of an ARMA process.

5. Homogeneous linear difference equations.

2

Review: Causality

A linear process {Xt} is causal (strictly, a causal functionof {Wt}) if there is a

(B) = 0 + 1B + 2B2 +

withj=0

|j |

Review: Invertibility

A linear process {Xt} is invertible (strictly, an invertiblefunction of {Wt}) if there is a

pi(B) = pi0 + pi1B + pi2B2 +

withj=0

|pij |

Review: AR(p), Autoregressive models of order p

An AR(p) process {Xt} is a stationary process that satisfies

Xt 1Xt1 pXtp =Wt,

where {Wt} WN(0, 2).

Equivalently, (B)Xt =Wt,

where (B) = 1 1B pBp.

5

Review: AR(p), Autoregressive models of order p

Theorem: A (unique) stationary solution to (B)Xt =Wtexists iff the roots of (z) avoid the unit circle:

|z| = 1 (z) = 1 1z pzp 6= 0.

This AR(p) process is causal iff the roots of (z) are outsidethe unit circle:

|z| 1 (z) = 1 1z pzp 6= 0.

6

Reminder: Polynomials of a complex variable

Every degree p polynomial a(z) can be factorized as

a(z) = a0 + a1z + + apzp = ap(z z1)(z z2) (z zp),

where z1, . . . , zp C are the roots of a(z). If the coefficients a0, a1, . . . , apare all real, then the roots are all either real or come in complex conjugatepairs, zi = zj .

Example: z + z3 = z(1 + z2) = (z 0)(z i)(z + i),that is, z1 = 0, z2 = i, z3 = i. So z1 R; z2, z3 6 R; z2 = z3.Recall notation: A complex number z = a+ ib has Re(z) = a, Im(z) = b,z = a ib, |z| = a2 + b2, arg(z) = tan1(b/a) (pi, pi].

7

Review: Calculating for an AR(p): general case

(B)Xt =Wt, Xt = (B)Wtso 1 = (B)(B)

1 = (0 + 1B + )(1 1B pBp) 1 = 0, 0 = j (j < 0),

0 = (B)j (j > 0).

We can solve these linear difference equations in several ways: numerically, or by guessing the form of a solution and using an inductive proof, or by using the theory of linear difference equations.

8

Introduction to Time Series Analysis. Lecture 6.1. Review: Causality, invertibility, AR(p) models2. ARMA(p,q) models3. Stationarity, causality and invertibility




9

ARMA(p,q): Autoregressive moving average models

An ARMA(p,q) process {Xt} is a stationary process thatsatisfies

Xt1Xt1 pXtp =Wt+1Wt1+ +qWtq,

where {Wt} WN(0, 2). AR(p) = ARMA(p,0): (B) = 1. MA(q) = ARMA(0,q): (B) = 1.

10

ARMA(p,q): Autoregressive moving average models

An ARMA(p,q) process {Xt} is a stationary process thatsatisfies

Xt1Xt1 pXtp =Wt+1Wt1+ +qWtq,

where {Wt} WN(0, 2).Usually, we insist that p, q 6= 0 and that the polynomials

(z) = 1 1z pzp, (z) = 1 + 1z + + qzq

have no common factors. This implies it is not a lower order ARMA model.

11

ARMA(p,q): An example of parameter redundancy

Consider a white noise process Wt. We can write

Xt =Wt

Xt Xt1 + 0.25Xt2 =Wt Wt1 + 0.25Wt2(1B + 0.25B2)Xt = (1B + 0.25B2)Wt

This is in the form of an ARMA(2,2) process, with

(B) = 1B + 0.25B2, (B) = 1B + 0.25B2.

But it is white noise.

12

ARMA(p,q): An example of parameter redundancy

ARMA model: (B)Xt = (B)Wt,

with (B) = 1B + 0.25B2,(B) = 1B + 0.25B2

Xt = (B)Wt

(B) = (B)(B)

= 1.

i.e., Xt =Wt.

13





14

Recall: Causality and Invertibility

A linear process {Xt} is causal if there is a

(B) = 0 + 1B + 2B2 +

withj=0

|j |

ARMA(p,q): Stationarity, causality, and invertibility

Theorem: If and have no common factors, a (unique) sta-tionary solution to (B)Xt = (B)Wt exists iff the roots of(z) avoid the unit circle:

|z| = 1 (z) = 1 1z pzp 6= 0.

This ARMA(p,q) process is causal iff the roots of (z) are out-side the unit circle:

|z| 1 (z) = 1 1z pzp 6= 0.

It is invertible iff the roots of (z) are outside the unit circle:

|z| 1 (z) = 1 + 1z + + qzq 6= 0.

16


Example: (1 1.5B)Xt = (1 + 0.2B)Wt.

(z) = 1 1.5z = 32

(z 2

3

),

(z) = 1 + 0.2z =1

5(z + 5) .

1. and have no common factors, and s root is at 2/3, which is not onthe unit circle, so {Xt} is an ARMA(1,1) process.2. s root (at 2/3) is inside the unit circle, so {Xt} is not causal.3. s root is at 5, which is outside the unit circle, so {Xt} is invertible.

17


Example: (1 + 0.25B2)Xt = (1 + 2B)Wt.

(z) = 1 + 0.25z2 =1

4

(z2 + 4

)=

1

4(z + 2i)(z 2i),

(z) = 1 + 2z = 2

(z +

1

2

).

1. and have no common factors, and s roots are at 2i, which is noton the unit circle, so {Xt} is an ARMA(2,1) process.2. s roots (at 2i) are outside the unit circle, so {Xt} is causal.3. s root (at 1/2) is inside the unit circle, so {Xt} is not invertible.

18

Causality and Invertibility

Theorem: Let {Xt} be an ARMA process defined by(B)Xt = (B)Wt. If all |z| = 1 have (z) 6= 0, then thereare polynomials and and a white noise sequence Wt suchthat {Xt} satisfies (B)Xt = (B)Wt, and this is a causal,invertible ARMA process.

So well stick to causal, invertible ARMA processes.

19





20

Calculating for an ARMA(p,q): matching coefficients

Example: Xt = (B)Wt (1 + 0.25B2)Xt = (1 + 0.2B)Wt,so 1 + 0.2B = (1 + 0.25B2)(B)

1 + 0.2B = (1 + 0.25B2)(0 + 1B + 2B2 + ) 1 = 0,

0.2 = 1,

0 = 2 + 0.250,

0 = 3 + 0.251,

.

.

.

21

Calculating for an ARMA(p,q): example

1 = 0, 0.2 = 1,0 = j + 0.25j2 (j 2).

We can think of this as j = (B)j , with 0 = 1, j = 0 for j < 0, j > q.

This is a first order difference equation in the js.We can use the js to give the initial conditions and solve it using the theoryof homogeneous difference equations.

j =(1, 15 , 14 , 120 , 116 , 180 , 164 , 1320 , . . .

).

22

Calculating for an ARMA(p,q): general case

(B)Xt = (B)Wt, Xt = (B)Wtso (B) = (B)(B)

1 + 1B + + qBq = (0 + 1B + )(1 1B pBp) 1 = 0,

1 = 1 10,2 = 2 11 20,

.

.

.

This is equivalent to j = (B)j , with 0 = 1, j = 0 for j < 0, j > q.

23





24

Autocovariance functions of linear processes

Consider a (mean 0) linear process {Xt} defined by Xt = (B)Wt.

(h) = E (XtXt+h)

= E (0Wt + 1Wt1 + 2Wt2 + ) (0Wt+h + 1Wt+h1 + 2Wt+h2 + )

= 2w (0h + 1h+1 + 2h+2 + ) .

25

Autocovariance functions of MA processes

Consider an MA(q) process {Xt} defined by Xt = (B)Wt.

(h) =

2wqh

j=0 jj+h if h q,0 if h > q.

26

Autocovariance functions of ARMA processes

ARMA process: (B)Xt = (B)Wt.

To compute , we can compute , and then use

(h) = 2w (0h + 1h+1 + 2h+2 + ) .

27

Autocovariance functions of ARMA processes

An alternative approach:

Xt 1Xt1 pXtp=Wt + 1Wt1 + + qWtq,

so E ((Xt 1Xt1 pXtp)Xth)= E ((Wt + 1Wt1 + + qWtq)Xth) ,

that is, (h) 1(h 1) p(h p)= E (hWthXth + + qWtqXth)

= 2w

qhj=0

h+jj . (Write 0 = 1).

This is a linear difference equation.

28

Autocovariance functions of ARMA processes: Example

(1 + 0.25B2)Xt = (1 + 0.2B)Wt, Xt = (B)Wt,

j =

(1,

1

5,1

4, 1

20,1

16,1

80, 1

64, 1

320, . . .

).

(h) 1(h 1) 2(h 2) = 2wqhj=0

h+jj

(h) + 0.25(h 2) =

2w (0 + 0.21) if h = 0,0.22w0 if h = 1,

0 otherwise.

29

Autocovariance functions of ARMA processes: Example

We have the homogeneous linear difference equation

(h) + 0.25(h 2) = 0

for h 2, with initial conditions

(0) + 0.25(2) = 2w (1 + 1/25)(1) + 0.25(1) = 2w/5.

We can solve these linear equations to determine .Or we can use the theory of linear difference equations...

30



1. ARMA(p,q) models2. Stationarity, causality and invertibility




31

Difference equations

Examples:

xt 3xt1 = 0 (first order, linear)xt xt1xt2 = 0 (2nd order, nonlinear)

xt + 2xt1 x2t3 = 0 (3rd order, nonlinear)

32

Homogeneous linear diff eqns with constant coefficients

a0xt + a1xt1 + + akxtk = 0 (a0 + a1B + + akBk)xt = 0

a(B)xt = 0auxiliary equation: a0 + a1z + + akzk = 0

(z z1)(z z2) (z zk) = 0

where z1, z2, . . . , zk C are the roots of this characteristic polynomial.Thus,

a(B)xt = 0 (B z1)(B z2) (B zk)xt = 0.

33


a(B)xt = 0 (B z1)(B z2) (B zk)xt = 0.

So any {xt} satisfying (B zi)xt = 0 for some i also satisfies a(B)xt = 0.Three cases:

1. The zi are real and distinct.

2. The zi are complex and distinct.

3. Some zi are repeated.

34


1. The zi are real and distinct.

a(B)xt = 0

(B z1)(B z2) (B zk)xt = 0 xt is a linear combination of solutions to

(B z1)xt = 0, (B z2)xt = 0, . . . , (B zk)xt = 0 xt = c1zt1 + c2zt2 + + ckztk ,

for some constants c1, . . . , ck.

35


1. The zi are real and distinct. e.g., z1 = 1.2, z2 = 1.3

0 2 4 6 8 10 12 14 16 18 200.8

0.6

0.4

0.2

0

0.2

0.4

0.6

0.8

1c1 z1

t + c2 z2

t

c1=1, c2=0c1=0, c2=1c1=0.8, c2=0.2

36

Reminder: Complex exponentials

a+ ib = rei = r(cos + i sin ),

where r = |a+ ib| =a2 + b2

= tan1(b

a

) (pi, pi].

Thus, r1ei1r2ei2 = (r1r2)ei(1+2),

zz = |z|2.

37


2. The zi are complex and distinct.

As before, a(B)xt = 0

xt = c1zt1 + c2zt2 + + ckztk .If z1 6 R, since a1, . . . , ak are real, we must have the complex conjugateroot, zj = z1. And for xt to be real, we must have cj = c1. For example:

xt = c zt1 + c z1

t

= r ei|z1|teit + r ei|z1|teit

= r|z1|t(ei(t) + ei(t)

)

= 2r|z1|t cos(t )where z1 = |z1|ei and c = rei.

38


2. The zi are complex and distinct. e.g., z1 = 1.2 + i, z2 = 1.2 i

0 2 4 6 8 10 12 14 16 18 201

0.8

0.6

0.4

0.2

0

0.2

0.4

0.6

0.8

1c1 z1

t + c2 z2

t

c=1.0+0.0ic=0.0+1.0ic=0.80.2i

39


2. The zi are complex and distinct. e.g., z1 = 1 + 0.1i, z2 = 1 0.1i

0 10 20 30 40 50 60 702

1.5

1

0.5

0

0.5

1

1.5

2c1 z1

t + c2 z2

t

c=1.0+0.0ic=0.0+1.0ic=0.80.2i

40

Time series

Documents

Transcript of Time series