Estimating Models Based on Markov JumpProcesses Given Fragmented Observation Series

Markus Hahn

Johann Radon Institute for Computational and Applied Mathematics (RICAM), LinzAustrian Academy of Sciences

Joint work with S. Fruhwirth-Schnatter (JKU Linz) and J. Sass (TU Kaiserslautern)

Linz, December 2, 2008

Work funded by FWF project P17947

Introduction

Problem

I Estimation from set of observed seriesI Independent series

I Data with breaks

I Each single series is (based) Markov processI Same generator for all series

I How to estimate common generator?

I How to cope with short observation series?

2 / 37

Introduction

Outline

Introduction

Markov Jump Processes

Merged Markov Jump Processes

Inference for Merged Markov Jump Processes

Generalization: Markov Switching Models

Conclusion

3 / 37

Markov Jump Processes

Finite state Markov jump process (MJP)

I Y = (Yt)t∈[0,T ) is a continuous time Markov process

I Finite state space {1, . . . , d}

I Y is time homogeneous

I Jumps of Y are governed by rate matrix Q ∈ Rd×d

I Exponential rate of leaving state k:

λk = −Qkk =∑l 6=k

Qkl < ∞

i.e. average waiting time for leaving k is 1/λk

I Conditional transition probability:

P(Yt = l |Yt− = k,Yt 6= Yt−) = Qkl/λk

4 / 37

Markov Jump Processes

Inference about rate matrix

I Ok occupation time of state k

I Nkl number of jumps from k to l

I Maximum likelihood estimation:

Qkl = Nkl/Ok

I Observing a path (Yt)t∈[0,T ), Ok and Nkl are sufficient forestimating Qkl

I Unbiased?

E(Qkl |Q) = ∞E(Qkl |Q,Ok > 0) = ∞

I Q is consistent, i.e.

limT→∞

P(|Qkl − Qkl | > ε) = 0

5 / 37

Markov Jump Processes

Some Bayesian inference

I Using uninformative prior Qkl ∼ Ga(1, 0):

Qkl |Y ∼ Ga(Nkl + 1,Ok)

I Mean: (Nkl + 1)/Ok

I Variance: (Nkl + 1)/O2k , hence decreasing with 1/T

I Mode: Nkl/Ok

6 / 37

Markov Jump Processes

Remarks on inference

I Q is well-known and looks nice

I In fact, estimation of Q is not so easy

I We need some observations for each transition k to l

I Basically, T needs to be as large as possible

7 / 37

Merged Markov Jump Processes

Introduction

Markov Jump Processes

Merged Markov Jump Processes

Inference for Merged Markov Jump Processes

Generalization: Markov Switching Models

Conclusion

8 / 37

Merged Markov Jump Processes

Observing a number of series of MJPs

I M series of MJPs are observed

I All Y (m) characterized by same rate matrix Q

I Series may be independent or come from data with breaks

0 0.051

2

3

0 0.051

2

3

0 0.051

2

3

0 0.051

2

3

0 0.051

2

3

Figure: Processes Y (1), . . . , Y (M)

9 / 37

Merged Markov Jump Processes

Observing merged MJPs

I Merged process of M single MJPs is observed

I Single MJPs characterized by same rate matrix Q

0 0.05 0.1 0.15 0.2 0.251

2

3

Figure: Merged process Y

10 / 37

Merged Markov Jump Processes

Merged MJPs

I Given: Observation process Y = (Yt)t∈[0,T )

I Y is concatenation of MJPs Y (1), . . . ,Y (M)

I In detail, Y (m) = (Y(m)t )t∈[0,T ), T = M T , and

Yt = Y(1)t if 0 ≤ t < T ,

Yt = Y(2)t−T if T ≤ t < 2T ,

...

Yt = Y(m)t−mT if (m − 1)T ≤ t < mT

I All Y (m) characterized by same rate matrix Q

I NB: Y itself is not Markov!

I Assumption of equal length is for notational convenience only

11 / 37

Merged Markov Jump Processes

Merged MJPs – Example

M = 5, d = 3, T = 0.05, T = 0.25, Q =

(−50 30 2020 −40 2030 40 −70

)

0 0.05 0.1 0.15 0.2 0.251

2

3

0 0.051

2

3

0 0.051

2

3

0 0.051

2

3

0 0.051

2

3

0 0.051

2

3

Figure: Merged process Y and single processes Y (1), . . . , Y (M)

12 / 37

Inference for Merged Markov Jump Processes

Introduction

Markov Jump Processes

Merged Markov Jump Processes

Inference for Merged Markov Jump Processes

Generalization: Markov Switching Models

Conclusion

13 / 37

Inference for Merged Markov Jump Processes

Splitting merged process

I Split Y into Y (1), . . . ,Y (M)

0 0.051

2

3

0 0.051

2

3

0 0.051

2

3

0 0.051

2

3

0 0.051

2

3

I Pooling:

Qkl =1

M

M∑m=1

Q(m)kl

I Problem: MLE does not exist if some O(m)k = 0;

short occupation times lead to unstable results

14 / 37

Inference for Merged Markov Jump Processes

Estimating directly from merged process

I Consider Y

0 0.05 0.1 0.15 0.2 0.251

2

3

I Number of jumps and occupation time:

Nkl =M∑

m=1

N(m)kl + N+

kl , where N+kl =

M−1∑m=1

I{Y

(m)T−=k, Y

(m+1)0 =l

},

Ok =M∑

m=1

O(m)k

15 / 37

Inference for Merged Markov Jump Processes

Estimating directly from merged process – first try

I First attempt: Qkl = Nkl/Ok

I Problem: Bias caused from “artificial” jumps N+kl /Ok

I If N+kl is observed explicitly, things are easy

I We assume N+kl cannot be observed

I Location of splitting points unknown

I Process not directly observed

16 / 37

Inference for Merged Markov Jump Processes

Estimating directly from merged process – bias

I Assume Y(m)0 ∼ π, where π stationary distribution

I Extra jumps N+ do not affect distribution of occupation times andstationary distribution

I Hence, π = π and π = π(Q) is “unbiased” estimate for π

I Joining two independent stationary processes generates a jump fromk to l with probability πkπl :

P(N+

kl |π)

= Bin(M − 1, πkπl)

17 / 37

Inference for Merged Markov Jump Processes

Estimating directly from merged process – bias (2)

I P(N+

kl |π)

= Bin(M − 1, πkπl) is justified if

I Y (m) are independent series

I Data with breaks:Length of break τ such that ρτ (Y ) is close to zero, where

ρt(Y ) =

Pdk=1 πk(Xkk(t) − πk)Pd

k=1 πk(1 − πk)

and

Xkl(t) = P(Yt = l |Y0 = k) = exp(Q t)kl

I.e. Xkk(τ) should be close to πk

18 / 37

Inference for Merged Markov Jump Processes

Estimating directly from merged process – bias (3)

I Note: If π is given, N+kl and Ok are independent

I Recall: P(N+

kl |π)

= Bin(M − 1, πkπl)

I As

T

Ok

P−−−−→T→∞

1

πk,

we have

N+kl

Ok

≈ (M − 1)πkπl

πkT= πl

M − 1

T

I As π is unbiased estimate for π, these quantities can be estimatedknowing Q

19 / 37

Inference for Merged Markov Jump Processes

Estimating directly from merged process – correction

I 2-step construction for corrected estimate:

1) Qkl = Nkl/Ok

2) Qkl = Qkl − (M − 1) πl/T

I “Merging”

20 / 37

Inference for Merged Markov Jump Processes

Comparison: Splitting vs. Merging

I Variance for Splitting for Ga(1, 0) prior:

Var

(M−1

M∑m=1

Q(m)kl

∣∣∣Y (1), . . . ,Y (M)

)=

1

M2

M∑m=1

N(m)kl + 1(O

(m)k

)2I Variance for Merging for Ga(1, 0) prior:

Var(Qkl | Y

)=

∑Mm=1 N

(m)kl + 1(∑M

m=1 O(m)k

)2I For rather short single observation times T we expect Merging to

give more reliable results

21 / 37

Inference for Merged Markov Jump Processes

Numerical example

I M = 100, d = 3, T = 0.25, T = 25, Q =

(−100 60 4040 −70 3040 60 −100

)

I On average, about 22 jumps in each single process Y (m)

I Simulate merged data and apply Splitting and Merging

I Repeat 100 000 times and consider sampling distributions of Qkl

and Qkl

I π = (0.29 0.46 0.25) and Q = Q −

(−2.8 1.8 1.01.1 −2.1 1.01.1 1.8 −2.9

)

22 / 37

Inference for Merged Markov Jump Processes

Numerical example – results

−140 −120 −100 −800

0.05

0.1

50 70 900

0.05

0.1

0.15

30 40 50 600

0.1

0.2

30 40 500

0.1

0.2

−90 −80 −70 −600

0.1

0.2

20 30 400

0.1

0.2

30 40 50 600

0.1

0.2

50 70 900

0.05

0.1

0.15

−140 −120 −100 −800

0.05

0.1

Green circles: true Qkl , blue: Merging Qkl , red: Splitting Qkl

23 / 37

Inference for Merged Markov Jump Processes

Numerical example – remarks

I Qkl outperforms Qkl wrt. both location and dispersion

I Qkl is skewed towards over-estimation:If some O

(m)k is very small, Q

(m)kl heavily over-estimates Qkl and

also Qkl is too high

I This is the more severe, the smaller T is

24 / 37

Inference for Merged Markov Jump Processes

Numerical example (2)

I M = 62, d = 3, T = 1/M, T = 1, Q =

(−800 450 350100 −200 100300 400 −700

)

I On average, about 6 jumps in each single process Y (m)

I Probability that one state is visited never or only for a very shorttime is high!

I Simulate merged data and apply Splitting and Merging

I Repeat 100 000 times and consider sampling distributions of Qkl

and Qkl

I π = (0.15 0.68 0.17) and Q = Q −

(−51.8 41.4 10.49.1 −19.6 10.59.1 41.4 −50.5

)

25 / 37

Inference for Merged Markov Jump Processes

Numerical example (2) – results

−3,000 −2,000 −1,000 00

0.003

0.006

0 500 1000 15000

0.004

0.008

0 500 1000 15000

0.005

0.01

0 100 2000

0.02

0.04

−400 −200 00

0.01

0.02

0.03

0 100 2000

0.02

0.04

0 500 1,000 1,5000

0.005

0.01

0 500 1000 15000

0.005

0.01

−3,000 −2,000 −1,000 00

0.004

0.008

Green circles: true Qkl , blue: Merging Qkl , red: Splitting Qkl

26 / 37

Generalization: Markov Switching Models

Introduction

Markov Jump Processes

Merged Markov Jump Processes

Inference for Merged Markov Jump Processes

Generalization: Markov Switching Models

Conclusion

27 / 37

Generalization: Markov Switching Models

A continuous-time Markov switching model

I Observation process R = (Rt)t∈[0,T ] (e.g. stock returns) withdynamics

dRt = µt dt + σt dWt ,

Rt =

∫ t

0

µs ds +

∫ t

0

σs dWs

I W standard Brownian motion

I Drift and volatility jump between d levels:µt = µ(Yt), σt = σ(Yt)

I State process Y is a MJP with state space {1, . . . , d}, Y ⊥ W

28 / 37

Generalization: Markov Switching Models

Example

0 0.2 0.4 0.6 0.8 11

2

3State process

0 0.2 0.4 0.6 0.8 1−2

02

Drift process

0 0.2 0.4 0.6 0.8 10.1

0.15

0.2

Volatility process

0 0.2 0.4 0.6 0.8 1−0.03

0

0.03 Daily stock returns

0 0.2 0.4 0.6 0.8 10.8

1

1.2Price process

∆t = 1250 , µ = (3, 0,−2), σ = (0.20, 0.12, 0.15), Q =

(−70 40 3020 −40 2030 50 −80

)29 / 37

Generalization: Markov Switching Models

MSMs in Finance

I Short rate models: Elliott/Hunter/Jamieson (2001)

I Investment problems: Zhang (2001), Guo (2005)

I Risk measures for derivatives: Elliott/Siu/Chan (2008)

I Portfolio optimization: Honda (2003), Zhou/Yin (2003),Sass/Haussmann (2004), Bauerle/Rieder (2005), . . .

I Option pricing: Guo (2001), Buffington/Elliott (2002),Chan/Elliott/Siu (2005), Liu/Zhang/Yin (2006),Yao/Zhang/Zhou (2006), . . .

I . . .

30 / 37

Generalization: Markov Switching Models

Estimation from discretely observed data

I Return process R is observable at times t = i ∆t,

Vi = ∆Ri =

∫ i ∆t

(i−1) ∆t

µs ds +

∫ i ∆t

(i−1) ∆t

σs dWs , i = 1, . . . ,H

I Vi daily stock returns

I State process Y is independent of W and not observable (hidden)

I Wanted: µ(k), σ(k), Q

I Problems:I ∆t given and fixedI Noise high compared to signalI High-frequency switching of states, i.e. λk ∆t highI Number of observations low (say, less than 5000)

31 / 37

Generalization: Markov Switching Models

Data with breaks / merged processes

I Often, we encounter data with breaks

I Weekends for daily data, nights for intra-day data, . . .

I Observable discrete time process:V = (Vi )i=1,...,H is concatenation of V (1), . . . ,V (M)

V (m) = (V(m)i )i=1,...,H and H = M H

I Hidden continuous-time state process:Y is concatenation of Y (1), . . . ,Y (M)

Y (m) = (Y(m)t )t∈[0,T ) and T = M T

32 / 37

Generalization: Markov Switching Models

Estimation from data with breaks

I Proceed similarly as for MJPs

I Use merged data V for estimation

I Employ arbitrary method to obtain estimates µ, σ, and Q

I µ and σ are not affected by merging, hence µ = µ, σ = σ

I Correction for Q: As described for MJPsI Point estimates: Correct Q to obtain Q

I Simulation based: Correct each sample Qj to obtain samples Qj

33 / 37

Generalization: Markov Switching Models

Numerical example

I T = 10, H = 10 000, ∆t = 1/1000

I d = 2, Q =(−60 60

40 −40

), µ = (2 − 1), σ = (0.10 0.05),

i.e. µ∆t = (0.002 − 0.001), σ√

∆t = (0.0032 0.0016)

I M, T , H varying s.t. T = M T and H = M H

I M = 1 corresponds to one coherent seriesM = 400 corresponds to 400 series with H = 25 observations each

I Simulate merged data

I Perform method of moments-type estimation for merged data

I Repeat 1 000 times

34 / 37

Generalization: Markov Switching Models

Numerical example – results

µ(1) µ(2) σ(1) σ(2) λ1 λ2 λ1 λ2

true 2.00 -1.00 0.100 0.050 60.0 40.0 60.0 40.0

M = 1, 2.02 -0.97 0.100 0.052 58.6 37.5 58.6 37.5τ = 10 0.11 0.05 0.002 0.002 9.8 6.5 9.8 6.5

M = 50, 2.02 -0.97 0.100 0.052 61.5 39.5 58.6 37.7τ = 0.2 0.12 0.05 0.001 0.003 10.7 6.3 10.7 6.7

M = 100, 2.03 -0.97 0.100 0.052 64.7 41.2 58.6 37.3τ = 0.1 0.11 0.05 0.001 0.003 11.6 6.4 10.5 6.8

M = 200, 2.03 -0.96 0.100 0.052 71.2 45.1 59.0 37.4τ = 0.05 0.12 0.06 0.001 0.003 15.6 8.2 10.7 6.7

M = 400, 2.06 -0.96 0.100 0.053 85.3 52.9 60.7 37.6τ = 0.025 0.13 0.06 0.002 0.003 28.4 15.1 12.6 8.0

Table: Results for MSM (T = 10, H = 10 000): mean (top), RMSE (bottom)

35 / 37

Generalization: Markov Switching Models

Numerical example – remarks

I Estimates of µ(k), σ(k) are not affected by merging

I Quality of corrected estimates Qkl is (nearly) independent of M

I Ignoring breaks can lead to considerable bias in estimate for Q

I Method of moments-type estimation requires a lot of observations,Splitting for M > 2 not applicable;ML or Bayesian methods could be applied for M ≤ 10, but arecomputationally much more costly

36 / 37

Conclusion

Conclusion

I Estimation for set of short (independent) observation series ordata containing (long) breaks

I Applicable to processes based on MJPs

I First, estimate parameters for merged seriesSecond, correct rates for bias afterwards

I Post-processing correction

I Works with arbitrary estimation approach for coherent series

I Single series need not be of same length /splitting times need not be known –only number of breaking points required

I Works similarly for discrete time processes

37 / 37