Lecture 10 Time Series Model

Introduction to ANN & Fuzzy Systems

Lecture 10Time Series Model

Introduction to ANN & Fuzzy Systems(C) 2001-2013 by Yu Hen Hu 2

Outline

• Time series models• Linear Time Series Models

– Moving Average Model – Auto-regressive Model – ARMA model

• Nonlinear Time Series Estimation

• Applications


Time Series • What is a time series?

– A scalar or vector-valued function of time indices

• Examples:– Stock prices– Temperature readings– Measured signals of all

kinds

• What is the use of a time series?– Prediction of future time

series values based on past observations

Modeling of a time seriesValues of a time series at

successive time indices are often correlated. Otherwise, prediction is impossible.

Most time series can be modeled mathematically as a wide-sense stationary (WSS) random process. The statistical properties do not change with respect to time.

Some time series exhibits chaotic nature. A chaotic time series can be described by a deterministic model but behaves as if it is random, and highly un-predictable.


Time Series Models

• Most time series are sampled from continuous time physical quantities at regular sampling intervals. One may label each such interval with an integer index. E.g.

{y(t); t = 0, 1, 2, …}. • A time series may have a

starting time, say t = 0. If so, it will have an initial value.

• In other applications, a time series may have been run for a while, and its past value can be traced back to t = .

• Notations– y(t): time series value at

present time index t. – y(t-1): time series value one

unit sample interval before t.

– y(t+1): the next value in the future.

• Basic assumption– y(t) can be predicted with

certainly degree of accuracy by its past values {y(tk); k > 0} and/or the present and past values of other time series such as {x(tm); m 0}


Time Series Prediction

• Problem StatementGiven {y(i); i = t1, …} estimate y(t+to), to 0 such that

is minimized. when to = 1, it is called a 1-step prediction.Sometimes, additional time series {u(i); i = t, t1, …} may be available to aid the prediction of y(i)

• The estimate of y(t + t0)that minimizes C is the conditional expectation given past value and other relevant time series.

• This conditional expectation can be modeled by a linear function (linear time series model) or a nonlinear function.

2

2

)(ˆ)(

)(

oo

o

ttyttyE

tteEC

1

ˆ( )

( ) |{ ( )} ,{ ( )}

o

t to i i

y t t

E y t t y i u i


A Dynamic Time Series Model

• State {x(t)}– Past values of a time series

can be summarized by a finite-dimensional state vector.

• Input {u(t)}– Time series that is not

dependent on {y(t)}

• The mapping is a dynamic system as y(t) depends on both present time inputs as well as past values.

))(),(()( tutxfty

Time series model

memory

y(t)

x(t)

Inputu(t)

x(t+1)

x(t) = [x(t) x(t1) … x(t p)] consists past values of {y(t)}u(t) = [u(t) u(t1) … u(t q)]


Linear Time Series Models

• y(t) is a linear combination of x(t) and/or u(t).

• White noise random process model of input {u(t)}:– E(u(t)) = 0

– E{u(t)u(s)} = 0 if t s; = s2 if t = s.

• Three popular linear time series models: 1. Moving Average (MA) Model:

2. Auto-Regressive (AR) Model:

3. Moving Average, Auto-regressive (ARMA) Model:

M

m

mtumbty0

)()()(

)()()()(1

tuntynatyN

n

M

m

N

n

mtumbntynaty01

)()()()()(


Moving Average Model

• Cross correlation function

• Auto-correlation function:

• An MA model is recognized by the finite number of non-zero auto-correlation lags.

• {b(m)} can be solved from {Ry(k)} using optimization procedure.

• Example: If {u(t)} is the stock price, then

is a moving average model – An average that moves with respect to time!

M

m

mtumbty0

)()()(

| |

2

0

( ) ( ) ( )

( ) ( ) | | ;

0 | |

y

M k

m

R k E y t y t k

b m b m k k M

k M

0

2

( ) ( ) ( )

( ) ( ) ( )

( ) 0 ,

0

if

otherwise.

yu

M

m

R i E y t u t i

b m E u t m u t i

b i i M

4

)3()2()1()()(

tututututy


Finding MA Coefficients• Problem: Given a MA time series

{y(t)}, how to find {b(m)} withtout knowing {u(t)}, except the knowledge of 2?

• One way to find the MA model coefficients {b(m)} is spectral factorization.

• Consider an example:

• Given {y(t); t = 1, …, T}, estimate auto-correlation lag

• For this MA(1) model, R(k)0 for k > 1.

• Power spectrum

• Spectral factorization:– Compute S(z) from {R(k)} and

factorize its zeros and poles to construct B(z)

• Or comparing the coefficients of polynomial and solve a set of nonlinear equations.

)1()1()()0()( tubtubty

kT

t

ktytykT

kR1

)()(1

)(ˆ

1

)1()0()1()0(

)()/1()()(

*

1*

zbb

z

bzb

zkRzBzBzSk

YY

B(z) B(1/z*)

zeros b(1)/b(0) (b(1)/b(0))*

poles 0


Auto-Regressive Model

• Ry(m) is replaced with R(m) to simplify notations.

• R is a Töeplitz matrix and is positive definite. Fast Cholesky factorization algorithms such as the Levinson algorithm can be devised to solve the Y-W equation effectively.

:formmatrix in or

001

:equationWalker -Yule the toleads This

)()()(

)()()()()(

)()()(

Hence,

)()(,0)(

)()()()(

22

1

2

1

2

1

eRa

T

N

ny

N

n

y

N

n

nkRnak

ktyntyEnaktytuE

ktytyEkR

tutyEtyE

tuntynaty

e2

=

aR

0001

1

01

10110 2σ

–a(N)

)–a(

)R()R(N–R(N)

)R(–N+)R()R(R(–N))R(–)R(


Auto-Regressive, Moving Average (ARMA) Model

• A combination of AR and MA model.

• DenoteThen

Thus,

M

m

N

n

mtumbntynaty01

)()()()()(

Ms

mtustyEmb

tvstyEM

m

for 0

)()()(

)()(

0

M

m

mtumbtv0

)()()(

.)()(

)()()()()(

)()()()()(

)()()(

1

01

01

MknkRna

mtuktyEmbnkRna

ktymtumbntynaE

ktytyEkR

N

ny

M

m

N

ny

M

m

N

n

y

if

)(

)1()(

)(

)1(

21

1121

LkR

kRkR

Na

a

N)LR(k)LR(k)LR(k

)R(k–N+)R(kR(k)R(k–N))R(k)R(k

=

Higher order Y-W equation


Nonlinear Time Series Model

• f(x(t), u(t)) is a nonlinear function or mapping:– MLP– RBF

• Time Lagged Neural Net (TLNN)– The input of a MLP network is

formed by a time-delayed segment of a time series.

• A neuronal filter

MLP

D D Du(tM)u(t1)

e(t)

)(ˆ ty

+

u(t)

y(t)

D

u(tM)

u(t1)

+

u(t)

D

D

f()


System Identification Problem

Consider an unknown system (Plant) with output y(t) which depends on current and past input u(t).

• System Identification Problem – Given: input u(t) and output y(t), 0 t tmax,

Find T[•] such that

u Unknown Plant

y

)()]([ˆ tytuT(t)y


Control Problem

Given: desired output y*(t), t1 t t2

Find: input u(t), t0 t t2 (t0 t1)

such that y(t) —> y*(t) for t1 t t2

• Path-Following Control Problem – Entire tragectory of the desired output sequence is specified (t1 ~ t0)

• Reinforcement Learning Problem – Only the destination is given. The intermediate path is not specified (t1 >> t0).


System Identification

• With the same input {u(t)}, find a mathematical model which will best approximate the output sequence.

• Essentially, a function approximation problem. Due to the particular dynamics of the plant, recurrent ANN are often considered.

+Unknown Plant

Model

e(t)y(t)

y(t)

u(t)

^

Lecture 10 Time Series Model

Documents

Transcript of Lecture 10 Time Series Model