Density Approximations for Multivariate Di usions via an...

44
Density Approximations for Multivariate Diffusions via an Itˆ o-Taylor Expansion Approach Nian Yang * Nan Chen Xiangwei Wan October 14, 2017 Abstract In this paper we develop a new Itˆ o-Taylor expansion approach to deriv- ing analytical approximations to the transition densities of multivariate dif- fusions. The obtained approximations can thereby be used to carry out the maximum likelihood estimation for the diffusions with discretely sampled data. Different from the existing density expansion methodologies that are usually model-specific, this new approach is universally applicable for a wide spectrum of models, particularly time-inhomogeneous non-affine irreducible multivariate diffusions. We manage to show that the expansion converges to the true proba- bility density under some regularity conditions. The method enjoys a significant computational efficiency over the other alternatives in the literature because on- ly differentiation operations are required. Extensive numerical experiments on a variety of models demonstrate the accuracy and efficiency of the estimators based on the approximate densities yielded from our approach. * Department of Finance and Insurance, School of Business, Nanjing University, Nanjing, China. Email: [email protected] Department of Systems Engineering and Engineering Management, The Chinese University of Hong Kong, Shatin, N. T., Hong Kong. Email: [email protected] Antai College of Economics and Management, Shanghai Jiao Tong University, Shanghai, China. Email: [email protected] 1

Transcript of Density Approximations for Multivariate Di usions via an...

Page 1: Density Approximations for Multivariate Di usions via an ...nchenweb/working_paper/MLE_JE_20171014.pdf · Density Approximations for Multivariate Di usions ... approach in the current

Density Approximations for Multivariate Diffusions via an

Ito-Taylor Expansion Approach

Nian Yang∗ Nan Chen† Xiangwei Wan‡

October 14, 2017

Abstract

In this paper we develop a new Ito-Taylor expansion approach to deriv-

ing analytical approximations to the transition densities of multivariate dif-

fusions. The obtained approximations can thereby be used to carry out the

maximum likelihood estimation for the diffusions with discretely sampled data.

Different from the existing density expansion methodologies that are usually

model-specific, this new approach is universally applicable for a wide spectrum

of models, particularly time-inhomogeneous non-affine irreducible multivariate

diffusions. We manage to show that the expansion converges to the true proba-

bility density under some regularity conditions. The method enjoys a significant

computational efficiency over the other alternatives in the literature because on-

ly differentiation operations are required. Extensive numerical experiments on

a variety of models demonstrate the accuracy and efficiency of the estimators

based on the approximate densities yielded from our approach.

∗Department of Finance and Insurance, School of Business, Nanjing University, Nanjing, China.

Email: [email protected]†Department of Systems Engineering and Engineering Management, The Chinese University of

Hong Kong, Shatin, N. T., Hong Kong. Email: [email protected]‡Antai College of Economics and Management, Shanghai Jiao Tong University, Shanghai, China.

Email: [email protected]

1

Page 2: Density Approximations for Multivariate Di usions via an ...nchenweb/working_paper/MLE_JE_20171014.pdf · Density Approximations for Multivariate Di usions ... approach in the current

Keyword: Density approximations, maximum likelihood estimation, mul-

tivariate time-inhomogeneous diffusions, closed-form expansion, Ito-Taylor ex-

pansion, convergence

JEL Classification: C13, C32, C63

1 Introduction

The transition probability densities constitute the essential inputs when we intend to

apply the maximum likelihood estimation (MLE) method to diffusion models for the

purpose of parameter estimation on the basis of discretely observed data (see, e.g.,

Lo, 1988). However, except a handful of simple examples, closed-form expressions of

the transition density of a general multivariate diffusion process is not available. To

overcome this technical difficulty, an active research line, initiated by the seminal con-

tribution of Aıt-Sahalia (2002; 2008), aims to constructing tractable approximations

to the transition density.

In the same spirit of this literature, we propose a new Ito-Taylor expansion based

approach in the current paper to deriving density approximations for multivariate

time-inhomogeneous diffusion processes, from which a new type of approximate max-

imum likelihood estimators can be obtained. The Ito-Taylor expansion, a stochastic

analogy to the Taylor expansion in the sense of classical calculus, originates from

iterated applications of the Ito-Dynkin formula on a “smooth function” of diffusion

processes (see, e.g., p.133 of Dynkin, 1965; Kallenberg, 2002). Note that we can rep-

resent the process’ transition density as a conditional expectation of the Dirac delta

function of the diffusion. Approximating the delta function by a sequence of smooth

functions enables us to apply the Ito-Taylor expansion to the expectation to yield

density approximations.

The proposed new approach contributes to the literature in three aspects. First,

it provides a unified treatment on various types of diffusions, nesting univariate and

multivariate, reducible and irreducible, time-homogeneous and time-inhomogeneous,

2

Page 3: Density Approximations for Multivariate Di usions via an ...nchenweb/working_paper/MLE_JE_20171014.pdf · Density Approximations for Multivariate Di usions ... approach in the current

affine and non-affine diffusions. This is in a sharp contrast to the existing literature

that typically develop model-specific methods to cope with different types of diffusion-

s. Second, we manage to show that the obtained approximations will converge to the

true probability densities of the underlying diffusion processes under some regularity

conditions as the observational time interval shrinks. The numerical experiments in

the paper further demonstrate the accuracy of the maximum likelihood estimators

based on our density approximations for a wide range of diffusion models. Third, the

implementation of our approach is computationally cheap because only differentiation

operations are involved. The procedure is so mechanical that we can easily utilize

some symbolic computation programs to facilitate the calculation. In this way, we

avoid solving the Kolmogorov partial differential equations (PDE) for approximating

transition densities as required by the conventional approaches (see, e.g., Aıt-Sahalia,

2008 and Choi, 2013).

As noted previously, diffusion density approximation and the related MLE have

been well investigated in the literature. A fruitful breakthrough was made by Aıt-

Sahalia (2002), who managed to use a sequence of mutually orthogonal Hermite

polynomials to obtain accurate approximations to the transition density of a univari-

ate diffusion process. This work triggers a substantial body of discussion on how

to search for diffusion density approximations for the purpose of implementing the

MLE method. Just to name a few, Schaumburg (2001) incorporated a jump compo-

nent in the process dynamic; Egorov et al. (2003) extended the result to univariate

time-inhomogeneous diffusions; Bakshi and Ju (2005) provided some refinements to

Aıt-Sahalia (2002); Lee et al. (2014) obtained a general expression of the coefficients

of the Hermite expansion of Aıt-Sahalia (2002) and rearranged it to get a new ap-

proximation.

One crucial step in the method proposed in Aıt-Sahalia (2002) is to apply the

Lamperti transform to unitize the process volatility. However, not every multivariate

process is amenable to such a transformation. This obstacle prevents us from extend-

3

Page 4: Density Approximations for Multivariate Di usions via an ...nchenweb/working_paper/MLE_JE_20171014.pdf · Density Approximations for Multivariate Di usions ... approach in the current

ing the method to irreducible diffusion processes. To tackle it, Aıt-Sahalia (2008)

suggested a new expansion of the probability densities of multivariate diffusions on

the basis of a double series in both the time and state variables, where the coeffi-

cients are determined recursively through solving the Kolmogorov PDEs. Yu (2007)

and Choi (2013; 2015) investigated the extensions of Aıt-Sahalia’s method in the cas-

es of multivariate jump diffusions and multivariate time-inhomogeneous diffusions,

respectively.

Beyond the aforementioned research line initiated by the seminal work of Aıt-

Sahalia (2002; 2008), alternative attempts have also been made to obtain closed-form

approximations of transition densities in multivariate models, including the saddle-

point approximation proposed by Aıt-Sahalia and Yu (2006), the orthonormal poly-

nomials expansion of Filipovic et al. (2013) for affine jump-diffusion models, and

the Malliavin calculus theoretic approach in Li (2013) and Li and Chen (2016) for

time-homogeneous models.

From this brief literature review, we can see that many of the existing approximate

MLEs for multivariate diffusions are model-specific, exploiting the special structures

of one certain class of processes. Taking an innovative path relative to these litera-

tures, our approach aims to establish a versatile expansion that approximate accu-

rately to the transition probability densities for a general family of diffusion models.

Our research is also related to a rich body of literature about the Ito-Taylor ex-

pansion; see Milstein (1975), Kessler (1997), Stanton (1997), Fan and Zhang (2003),

Aıt-Sahalia and Mykland (2003), Kristensen and Mele (2011), Uchida and Yoshida

(2012), Xiu (2014), and Li and Li (2015) for a variety of applications of this expansion

in moment computing and option pricing. It is worthwhile to mention that all of the

above papers just directly apply the Ito-Taylor expansion to expand the conditional

expectation of smooth functions. But what we encounter in this paper is challenging:

the irregularity of the Dirac delta function may cause divergence of our approxima-

tions. As a major theoretical contribution, we are the first ones to propose the idea

4

Page 5: Density Approximations for Multivariate Di usions via an ...nchenweb/working_paper/MLE_JE_20171014.pdf · Density Approximations for Multivariate Di usions ... approach in the current

of approximating the true transition density by a special Ito-Taylor expansion and

manage to prove its convergence.

The rest of the paper is organized as follows. In Section 2, we develop our Ito-

Taylor expansion to approximate the transition density of a general multivariate d-

iffusion process, and the convergence of the expansion to the true transition density

is proved. Section 3 presents some convergence results about the resulting approxi-

mate maximum likelihood estimators. Section 4 contains numerical evidence of the

performance of the approximate transition densities and the approximate maximum

likelihood estimators under various diffusions. Technical assumptions, lemmas, and

proofs are collected in the Appendix.

For the convenience of reference, we would like to define here some notations that

will be used throughout the paper. Let D ⊂ Rm be the domain of state variables

and denote Dc as a compact subset of D. Let ‖ · ‖ be the Euclidean norm. Denote

> to be the transpose operation on matrices or vectors. Let h = (h1, h2, · · · , hm) be

an index vector with nonnegative integer components and |h| :=∑m

i=1 hi. Let ei be

a special index vector, in which the i-th component is 1, and the others are 0. Define

∂t := ∂/∂t to be the partial derivative with respect to the time variable, and ∂h :=

∂|h|/(∂xh11 · · · ∂xhmm ) to be the partial derivatives with respect to the state variable

x := (x1, x2, · · · , xm)> ∈ D. For example, ∂ei = ∂/∂xi and ∂ei+ej = ∂2/(∂xi∂xj).

2 The Ito-Taylor Expansion of the Transition Den-

sity of Multivariate Diffusions

2.1 The Model

Consider a multivariate diffusion process

dX(t) = µ(t,X(t); θ)dt+ σ(t,X(t); θ)dW (t), (1)

5

Page 6: Density Approximations for Multivariate Di usions via an ...nchenweb/working_paper/MLE_JE_20171014.pdf · Density Approximations for Multivariate Di usions ... approach in the current

where X(t) is an m×1 vector of state variables in the domain D ⊂ Rm, W (t), t ≥ 0

is a d-dimensional standard Brownian motion, µ(t,X(t); θ) and σ(t,X(t); θ) are an

m × 1 drift vector and an m × d volatility (or dispersion) matrix, respectively. The

explicit forms of both are known. We emphasize that each component of the drift

vector and volatility matrix is a function dependent on the time variable t. Therefore

the models considered in this paper are allowed to be time-inhomogeneous. The

unknown parameter θ belongs to a compact set Θ ⊂ RK . Furthermore, we assume

that all the four assumptions in Appendix A.1 hold in the remaining part of the

paper.

We intend to develop a maximum likelihood estimator for θ in this paper. Intro-

duce more notations to make the subsequent discussion more precise. Fix two time

points t′ > t. Let p(t′, x′|t, x; θ) be the conditional transition density function of the

process driven by the stochastic differential equation (SDE) in (1); that is,

P[X(t′) ∈ dx′|X(t) = x] = p(t′, x′|t, x; θ)dx′.

Assume that we have a sequence of observations on the state variables at a discrete

time grid t = ti : i = 0, 1, . . . , n. By the Markovian setting of (1), the log-likelihood

function1 is given by

`n(θ) :=n∑i=1

ln p(ti, X(ti)|ti−1, X(ti−1); θ). (2)

Then, the maximizer

θn := arg maxθ∈Θ

`n(θ)

is the MLE of parameter θ.

However, it is well known that a major technical difficulty with the MLE method

for diffusions resides in the fact that closed-form expressions of p(t′, x′|t, x; θ) is u-

navailable for most cases. As noted in the introduction, we shall first proposes an

1As in Aıt-Sahalia (2002; 2008), we ignore the unconditional density of the first observation

(t0, X(t0)).

6

Page 7: Density Approximations for Multivariate Di usions via an ...nchenweb/working_paper/MLE_JE_20171014.pdf · Density Approximations for Multivariate Di usions ... approach in the current

approximation to the function p via an Ito-Taylor expansion approach and then apply

it to derive the MLE of θ.

2.2 Heuristic Idea of the Ito-Taylor Expansion

To facilitate the exposition, we would like to present the heuristic idea behind our

Ito-Taylor expansion in this subsection. Consider any sufficiently smooth function

G(s, y). By the Ito formula, we have

G(s,X(s)) =G(t,X(t)) +

∫ s

t

∂uG(u,X(u))du+m∑i=1

∫ s

t

µi(u,X(u); θ)∂eiG(u,X(u))du

+1

2

m∑i,j=1

∫ s

t

νij(u,X(u); θ)∂ei+ejG(u,X(u))du

+m∑

i,j=1

∫ s

t

∂eiG(u,X(u))σij(u,X(u); θ)dWj(u).

Here νij is the (i, j)-element of the variance-covariance (or diffusion) matrix of the

diffusion X defined below:

ν(t, x; θ) := σ(t, x; θ)σ(t, x; θ)>.

Let Et,x[·] denote expectation conditional on X(t) = x. Taking expectations on

both sides of the above equality, we have

Et,x[G(s,X(s))] = G(t, x) +

∫ s

t

Et,x[(∂u + L)G(u,X(u))]du, (3)

where L is the infinitesimal generator of process (1) such that

(LG)(u, y) =m∑i=1

µi(u, y; θ)∂eiG(u, y) +1

2

m∑i,j=1

νij(u, y; θ)∂ei+ejG(u, y), (4)

for any u ∈ (t, s) and y ∈ Rm. We may continue to apply the above idea to expand

Et,x[(∂u+L)G(u,X(u))], treating (∂u+L)G(u,X(u)) as a new function on the process

X. This will lead to

Et,x[(∂u + L)G(u,X(u))] = (∂t + L)G(t, x) +

∫ u

t

Et,x[(∂u2 + L)2G(u2, X(u2))]du2.

(5)

7

Page 8: Density Approximations for Multivariate Di usions via an ...nchenweb/working_paper/MLE_JE_20171014.pdf · Density Approximations for Multivariate Di usions ... approach in the current

Substituting (5) back into (3), we have

Et,x[G(s,X(s))] =G(t, x) + (∂t + L)G(t, x) · (s− t)

+ Et,x[∫ s

t

du1

∫ u1

t

(∂u2 + L)2G(u2, X(u2))du2

].

In this way, repeatedly applying the expansion for J times yields

Et,x[G(s,X(s))] =J∑

N=0

(s− t)N

N !(∂t + L)NG(t, x) +RJ , (6)

where the remainder term RJ is given by

RJ = Et,x[∫ s

t

du1

∫ u1

t

du2 · · ·∫ uJ

t

(∂uJ+1+ L)J+1G(uJ+1, X(uJ+1))duJ+1

].

Now, we turn to apply the Ito-Taylor expansion (6) to approximate the transition

density p. Intuitively, the density function admits the following expression:

p(t′, x′|t, x; θ) = Et,x[δx′(X(t′))], (7)

where δx′(·) is the Dirac delta function centered at x′. The function δx′(·) does not

have any derivatives in the classical sense. To circumvent this obstacle to applying

the above expansion, we need to “mollify” the irregularity of δx′(·) by introducing a

sequence of smooth functions to approximate it. More precisely, given µ0 ∈ Rm and

ν0 ∈ Rm×m, define a function G such that, for t ≤ s < t′ and y ∈ Rm,

Gt′,x′(s, y) :=1

(2π(t′ − s))m2 det(ν0)12

exp

(−(x′ − y − (t′ − s)µ0)>ν−1

0 (x′ − y − (t′ − s)µ0)

2(t′ − s)

).

(8)

In other words, if we fix t′, s and x′, Gt′,x′(s, ·) is the probability density function of

a multivariate normal distribution with mean x′ − (t′ − s)µ0 and covariance matrix

(t′ − s)ν0.

It is apparent to see that, as s → t′, the function Gt′,x′ converges to the Dirac

delta function δx′ in the following sense:

lims↑t′

Et,x[Gt′,x′(s,X(s))] = Et,x[δx′(X(t′))] = p(t′, x′|t, x; θ). (9)

8

Page 9: Density Approximations for Multivariate Di usions via an ...nchenweb/working_paper/MLE_JE_20171014.pdf · Density Approximations for Multivariate Di usions ... approach in the current

Since the function Gt′,x′ is infinitely differentiable, we now can invoke the Ito-Taylor

expansion (6) to expand the conditional expectation Et,x[Gt′,x′(s,X(s))] on the left-

hand side of (9). That is,

Et,x[Gt′,x′(s,X(s))] ≈J∑

N=0

(s− t)N

N !(∂t + L)NGt′,x′(t, x),

if we skip the residual term RJ . From (9), letting s tend to t′ in the above expression

results in our density approximation

p(t′, x′|t, x; θ) ≈J∑

N=0

(t′ − t)N

N !(∂t + L)NGt′,x′(t, x) =: p(J)(t′, x′|t, x; θ) (10)

for any positive integer J . From now on, we will refer to p(J) as the J-th order

Ito-Taylor expansion of the density p.

2.3 The Ito-Taylor Expansion of the Transition Density

Noting that the derivation in Section 2.2 is heuristic, in this section we shall estab-

lish the convergence property of our Ito-Taylor density approximation rigorously. It

turns out that not all the choices of µ0 and ν0 will warrant a convergent expansion.

Therefore we just focus on one of such choices that µ0 = 0 and ν0 = ν(t, x; θ) (the

diffusion matrix of SDE (1) at time t when Xt = x).

From (10), it is easy to see that we construct the approximation p(J) via repeat-

edly applying the differential operator L on a known function Gt′,x′ . The analytical

form of p(J) is thus achievable. Define some more notations to facilitate the result

presentation. Let φ(y; Σ) be the density of the m-dimensional normal distribution

with mean 0 and covariance matrix Σ, i.e.,

φ(y; Σ) =1

(2π)m/2 det(Σ)1/2exp

(−y>Σ−1y

2

). (11)

For an m-dimensional nonnegative integer vector h = (h1, · · · , hm), define Hh(y; ν0)

to be the corresponding multivariate Hermite polynomial associated with this normal

9

Page 10: Density Approximations for Multivariate Di usions via an ...nchenweb/working_paper/MLE_JE_20171014.pdf · Density Approximations for Multivariate Di usions ... approach in the current

density; that is,

Hh(y; Σ) := (−1)|h|φ−1(y; Σ)∂hφ(y; Σ). (12)

Willink (2005) provides a recursive approach to computing these multivariate Hermite

polynomials. Namely, for any vector h with hj ≥ 0, j = 1, · · · ,m, we have

Hh+ek(y; Σ) =

(m∑j=1

Σ(kj)yj

)Hh(y; Σ)−

m∑j=1

Σ(kj)hjHh−ej(y; Σ), (13)

where Σ(kj) is the (k, j)-element of the matrix Σ−1. In addition, H0(y; Σ) = 1 and

Hh(y; Σ) = 0 for any h with minh1, · · · , hm < 0. With the help of the Hermite

polynomials, we can show

Theorem 1. Set µ0 = 0 and ν0 = ν(t, x; θ). For J ≥ 1, there exists a sequence of

functions wN,h such that

p(J)(t′, x′|t, x; θ) = q(t′, x′|t, x)

1 +J∑

N=1

∑1≤|h|≤3N/2

wN,h(t, x)Hh(z; ν0)

N !∆N− |h|

2

,

(14)

where Hh(z; ν0) are the Hermite polynomials defined through (13) with

∆ = t′ − t, z =x′ − x√

∆,

and

q(t′, x′|t, x) :=1

(2π∆)m/2 det(ν0)1/2exp

(−(x′ − x)>ν−1

0 (x′ − x)

2∆

). (15)

Theorem 1 characterizes the structure of p(J) clearly: it can be expressed in terms

of a linear combination of Hermite polynomials. We can analytically determine the

coefficient functions w in a recursive fashion. More specifically, fixed (t, x), introduce

a function sequence wN,h(s, y) := wN,h(s, y; t, x, θ) : s ≥ 0, y ∈ D indexed by a

positive integer N and an m-dimensional vector h = (h1, · · · , hm) with hj ≥ −2 for

10

Page 11: Density Approximations for Multivariate Di usions via an ...nchenweb/working_paper/MLE_JE_20171014.pdf · Density Approximations for Multivariate Di usions ... approach in the current

all j = 1, · · · ,m. For any N ≥ 1, define wN,h(s, y) ≡ 0 if either minh1, · · · , hm < 0,

either h = 0, or |h| > 2N . When N = 1,w1,ei(s, y) = µi(s, y; θ), i = 1, · · · ,m;

w1,2ei(s, y) = 12(νii(s, y; θ)− νii(t, x; θ)), i = 1, · · · ,m;

w1,ei+ej(s, y) = νij(s, y; θ)− νij(t, x; θ), i 6= j, i, j = 1, · · · ,m,

(16)

where µi(·, ·; θ) is the i-component of the drift vector µ and νij(·, ·; θ) is the (i, j)-

element of the diffusion matrix ν. When N > 1 and all the components in h are

nonnegative, 0 < |h| ≤ 2N , define recursively

wN,h(s, y) =(∂s + L)wN−1,h(s, y)

+m∑

i,j=1

νij(s, y; θ)∂ejwN−1,h−ei(s, y) +m∑i=1

µi(s, y; θ)wN−1,h−ei(s, y)

+1

2

m∑i,j=1

(νij(s, y; θ)− νij(t, x; θ))wN−1,h−ei−ej(s, y), (17)

where the infinitesimal generator L is given in (4) acting on the state variable y.

Note that only differentiation operations are involved in the computation of coef-

ficient functions w in (17). Unlike the other existing density expansion methods, it

avoids the complicated integrations in the step of solving Kolmogorov PDEs. There-

fore, the implementation of our expansion is computationally convenient. We can even

simply use some symbolic computation programs such as Mathematica to accomplish

the derivation.

The next theorem establishes one of the main outcomes in this paper. Under some

regularity conditions (cf. Appendix A.1), we manage to show that the expansion p(J)

uniformly converges to the true transition density p.

Theorem 2. Suppose that Assumptions A.1-A.4 hold. Define p(J)(t′, x′|t, x; θ) through

(14). Then, given any positive integer J > 2m− 1 and compact subset Dc ⊂ D,

limt′→t

sup(t,x,x′,θ)∈[0,T ]×Dc×D×Θ

∣∣p(J)(t′, x′|t, x; θ)− p(t′, x′|t, x; θ)∣∣ = 0. (18)

11

Page 12: Density Approximations for Multivariate Di usions via an ...nchenweb/working_paper/MLE_JE_20171014.pdf · Density Approximations for Multivariate Di usions ... approach in the current

This theorem provides a theoretical guarantee of the accuracy of our Ito-Taylor ex-

pansion based approximation. It shows the convergence to the true transition density

p holds as the observational time interval t′ − t shrinks to 0, which ensures that the

resulted approximate maximum likelihood estimator should also converge to the un-

known true maximum likelihood estimator. By the technical assumptions A.1-A.4,

we can see that the structural features of a diffusion model, whether it is univariate

or multivariate, reducible or irreducible, affine or non-affine, time-homogeneous or

not, does not play a major role in the proof of convergences. In a sharp contrast, the

previous literatures, including Aıt-Sahalia (2002; 2008), Egorov et al. (2003), Choi

(2013; 2015), Li (2013), and Filipovic et al. (2013), suggest different methods to obtain

approximations to the densities of processes of different types.

The proof of Theorem 2 is deferred to Appendix A.3. The key observation is that

the coefficients wN,h(t, x) are all zeros for |h| > 3N/2, by which we build up a tight

upper bound estimation on the approximation error and ensure the convergence of

our density approximation. This observation is not true for any other choice of ν0

than ν0 = ν(t, x; θ). One may refer to Lemmas 1 and 2, Example 1, and the proofs

in Appendix A.3 for more details.

3 Approximate Maximum Likelihood Estimator

In this section, we shall use the Ito-Taylor expansion p(J) in (14) as an approximate to

the true but unknown transition density p to compute an approximate MLE. Given

a set of discretely observed data X(t0), X(t1), . . . , X(tn) with ti − ti−1 = ∆, i =

12

Page 13: Density Approximations for Multivariate Di usions via an ...nchenweb/working_paper/MLE_JE_20171014.pdf · Density Approximations for Multivariate Di usions ... approach in the current

1, . . . , n, replace p in (2) with its approximation p(J)2

`(J)n (θ) :=

n∑i=1

ln p(J)(ti, X(ti)|ti−1, X(ti−1); θ)

and solve for θ(J)n := arg maxθ∈Θ `

(J)n (θ).

Assume that the drift vector µ(t, x; θ) and the diffusion matrix ν(t, x; θ) are in-

finitely continuous differentiable with respect to θ ∈ Θ. Suppose the log-likelihood

function `n(θ) has a unique maximizer θn ∈ Θ, which is the true MLE of parameter

θ. With the help of the result of the uniform convergence established in Theorem 2,

we can show that θ(J)n converges in probability to θn, as ∆→ 0.

Theorem 3. Fix the sample size n and J > 2m − 1. Denote the true value of the

parameter vector to be θ0. Under Assumptions A.1-A.4, we have

θ(J)n − θn → 0

in Pθ0-probability as ∆→ 0.

Theorem 3 constitutes a very useful step towards establishing the asymptotic

consistency of our approximate MLE θ(J)n . Specifically, we can make the speed, at

which θ(J)n − θn converges to zero, arbitrarily small for any given sample size n by

taking ∆ → 0 sufficiently fast. In general, if we happen to know that the true MLE

θn converges to θ0 as n→∞, we can choose a sequence ∆n → 0 such that the same

asymptotic property also holds for θ(J)n (the approximate MLE corresponding to the

observation time intervals ∆n).

Aıt-Sahalia (2002) and Chang and Chen (2011) investigated the asymptotic prop-

erties of the true and approximate MLEs for univariate diffusions. To our best knowl-

edge, the corresponding results for multivariate diffusions are challenging and still

2p(J) may be negative in our expansion. To make the computation of logarithm feasible, we

truncate the approximate density at a sufficiently small positive number when implementing the

expansion in the numerical examples. More precisely, we take ln(·) on maxp(J), ε/J for some fixed

small ε > 0. Similar procedures are also used in Aıt-Sahalia (2002) and Egorov et al. (2003).

13

Page 14: Density Approximations for Multivariate Di usions via an ...nchenweb/working_paper/MLE_JE_20171014.pdf · Density Approximations for Multivariate Di usions ... approach in the current

open in the literature. We have provided some supportive evidences in numerical

experiments. A thorough theoretic investigation is beyond the scope of the current

paper. We leave it for future research.

4 Numerical Experiments

Below we undertake some numerical experiments to examine the performance of our

Ito-Taylor expansion based approximate densities and the associated approximate M-

LE. Six different types of models are considered: the univariate Ornstein-Uhlenbeck

(OU) model, the Cox-Ingersoll-Ross (CIR) model, the bivariate Ornstein-Uhlenbeck

(BOU) model, the Heston model, the time-inhomogeneous bivariate Ornstein-Uhlenbeck

model (BOUI), and the non-affine GARCH model. The purpose of taking so many

processes is to check out how the approach will perform under a variety of modeling

choices. The first two are univariate diffusions. The third and fourth models are

multivariate reducible and irreducible diffusions, respectively. All the first four mod-

els are time-homogenous with time-independent affine coefficients. In contrast, the

BOUI model is not time-homogenous and the last one is non affine.

The assessments on the efficiency of our proposed approach mainly consist of two

parts: one is about density approximation and the other is about MLE approxima-

tion. The subsequent contents in this section are thus organized as follows. In Section

4.1, we provide more detailed modeling information about these six processes. In Sec-

tion 4.2, we illustrate through numerical experiments that the approximate densities

stemmed from the Ito-Taylor expansion converges to the true density in a very fast

manner. In Section 4.3, we provide Monte Carlo evidence to show the accuracy and

efficiency of our approximate MLE.

14

Page 15: Density Approximations for Multivariate Di usions via an ...nchenweb/working_paper/MLE_JE_20171014.pdf · Density Approximations for Multivariate Di usions ... approach in the current

4.1 Models

Model 1. Ornstein-Uhlenbeck (OU) Model.

dX(t) = κ(α−X(t))dt+ σdW (t).

The OU process was first used by Vasicek (1977) to model the short term interest

rate. Its true transition density p(t′, x′|t, x) is normally distributed with mean

α + (x− α)e−κ(t′−t)

and varianceσ2

2κ(1− e−2κ(t′−t)).

Model 2. Cox-Ingersoll-Ross (CIR) Model.

dX(t) = κ(α−X(t))dt+ σ√X(t)dW (t).

It can be shown that X(t) remains nonnegative almost surely for all t ≥ 0. In

addition, the process has a tendency of reverting to its long-run mean α. For these

two important features, the CIR model is widely used in the literature to describe

the movements of the short term interest rates (Cox et al., 1985) or equity volatilities

(Heston, 1993). The true transition density of this model is given by

p(t′, x′|t, x) =eκ(t′−t)

2c(t′ − t)

(x′eκ(t′−t)

x

) d−24

exp

(−x+ x′eκ(t′−t)

2c(t′ − t)

)Id/2−1

(√xx′e−κ(t′−t)

c(t′ − t)

),

where

c(t) =σ2

4κ(eκt − 1), d =

4κθ

σ2,

and

Iγ(x) =+∞∑k=0

(x/2)2k+γ

k!Γ(k + γ + 1)

is the modified Bessel function of the first kind.

15

Page 16: Density Approximations for Multivariate Di usions via an ...nchenweb/working_paper/MLE_JE_20171014.pdf · Density Approximations for Multivariate Di usions ... approach in the current

Model 3. Bivariate Ornstein-Uhlenbeck (BOU) Model.

dX(t) = κ(α−X(t))dt+ dW (t),

where X(t) = (X1(t), X2(t))>, α = (α1, α2)>, and W (t) = (W1(t),W2(t))>, t ≥ 0 is a

2-dimensional standard Brownian motion. κ is a 2× 2 matrix:

κ =

κ11 0

κ21 κ22

.

The BOU diffusion is one of the few multivariate processes with explicitly known

transition densities. Assume that κ has full rank. The future state X(t′), conditional

on the current state X(t) = x, is bivariate-normally distributed (Aıt-Sahalia, 2008).

The mean of the distribution is

α + e−κ(t′−t)(x− α)

and the covariance matrix is λ− e−κ(t′−t′)λe−κ>(t′−t), where

λ =1

2tr(κ)det(κ)

(det(κ)Id2 + (κ− tr(κ)Id2)(κ− tr(κ)Id2)>

)with Id2 being a 2× 2 identity matrix.

Model 4. Bivariate Time-Inhomogeneous Ornstein-Uhlenbeck (BOUI) Model.

dX(t) = κ(α + βt−X(t))dt+ dW (t),

is obtained if we add a deterministic term βt on the drift coefficient of Model 3, where

β = (β1, β2)>.

Similar to the BOU model, the BOUI model’s transition density is explicitly known.

Under it, the distribution of a future state X(t′), conditional on the current state

X(t) = x, is also a normal with the same covariance matrix as the BOU model. But

its mean is given by

α + βt+ e−κ(t′−t)(x− α− βt) + e−κ(t′−t)∫ t′−t

0

eκuκβudu.

16

Page 17: Density Approximations for Multivariate Di usions via an ...nchenweb/working_paper/MLE_JE_20171014.pdf · Density Approximations for Multivariate Di usions ... approach in the current

Model 5. Consider the following model

d

S(t)

Y (t)

=

µS(t)

κ(α− Y (t))

dt+

√(1− ρ2)Y (t)S(t) ρ

√Y (t)S(t)

0 σY β(t)

dW (t),

where W (t) = (W1(t),W2(t))> is a 2-dimensional standard Brownian motion, µ, κ,

α, ρ, and β are all constants, β ≥ 1/2. Express the dynamic of S(t) in terms of

X(t) = ln(S(t)). We have

d

X(t)

Y (t)

=

µ− Y (t)/2

κ(α− Y (t))

dt+

√(1− ρ2)Y (t) ρ

√Y (t)

0 σY β(t)

dW (t). (19)

This class of models nests several important stochastic volatility processes that are

widely used in describing asset price dynamics. When we take β = 1/2 in (19), we

have the model proposed by Heston (1993); when β = 1, it will be identical as the

continuous-time GARCH model (cf. Nelson, 1990; Duan, 1995). The Heston model is

affine and irreducible, while the GARCH model is an example of non-affine processes.

4.2 Density Approximation

This subsection examines the accuracy of the expansion p(J) as an approximation to

the true transition probability density of the underlying process. For the convenience

of comparison, we consider the first four models out of the above six, i.e., OU, CIR,

BOU, and BOUI, because the densities of all of them are explicitly known. We use the

maximum absolute error between the J-th order Ito-Taylor expansion and the true

density as a measure of approximation error. As noted in Section 2.3, only differen-

tiation operations are involved in the expansion formula (14). We use Mathematica

in the numerical experiments to compute the expansion coefficients wN,h.

We take the value of κ in both the BOU and BOUI models to be the same as

what were used in Aıt-Sahalia (2008) and Choi (2013), respectively. Such a choice

guarantees that the eigenvalues of the matrix κ are real positive numbers, which is a

necessary restriction to make sure that these parameters are identifiable in these two

17

Page 18: Density Approximations for Multivariate Di usions via an ...nchenweb/working_paper/MLE_JE_20171014.pdf · Density Approximations for Multivariate Di usions ... approach in the current

continuous models with discretely observed data as discussed in Aıt-Sahalia (2008).

One may refer to, for example, Pedersen (1995), Hansen and Sargent (1983), and

Kessler and Rahbek (2004) for more discussions on the identification problem of

model specification.

Figure 1 displays the approximation errors of the Ito-Taylor expansion under these

four models. Two general patterns arise in the experiment outcomes. First, for a fixed

number of terms J , the error of our density approximation decreases as the observa-

tional time interval t′−t shrinks. When we change ∆ := t′−t from 1/12 to 1/252, i.e.,

the observation frequency from monthly to daily, the maximum absolute error of the

Ito-Taylor expansions reduced very significantly. Second, when we fix the observation

frequency ∆, the expansion with a larger J will lead to a smaller approximation error.

Both patterns corroborate the theoretical statements in Theorem 2.

[Figure 1 about here.]

Through Figure 2, we intend to compare the maximum absolute error of the Ito-

Taylor expansion proposed in this paper with those ones given by Aıt-Sahalia (2008)

and Choi (2013). As the order of the Ito-Taylor expansion increases, the performance

of our Ito-Taylor expansion exceeds that of the 3rd-order density expansion of Aıt-

Sahalia (2008) or the 2nd-order density expansion of Choi (2013) in terms of the

maximum absolute error.3

[Figure 2 about here.]

4.3 Monte Carlo Evidences for the Approximate MLE

In this subsection, we shall provide Monte Carlo evidences for the BOU, BOUI,

Heston, and GARCH models to investigate the performance of the approximate MLE

3Prof. Yacine Aıt-Sahalia provides a 3rd-order density approximate formula for the OU, CIR

and BOU model on his website https://www.princeton.edu/~yacine/. We thank Professor Se-

ungmoon Choi for sharing with us his 2nd-order density approximate formulas for the BOUI model.

18

Page 19: Density Approximations for Multivariate Di usions via an ...nchenweb/working_paper/MLE_JE_20171014.pdf · Density Approximations for Multivariate Di usions ... approach in the current

resulted from the Ito-Taylor expansion. Due to the space limit and the similarity in

the result pattern, we focus only on these four multivariate diffusions here.

Recall that n is the number of observations in each path. In light of the following

decomposition

θ(J)n − θ0 = (θ(J)

n − θn) + (θn − θ0),

we identify two sources of errors contributing to the estimation error of our Ito-Taylor

expansion based MLE. The first one is θn−θ0, the discrepancy between the true MLE

and the true parameter value. It measures the error caused intrinsically by the maxi-

mum likelihood method, having nothing to do with the proposed approximation. The

other one is θ(J)n − θn, which is affected by the accuracy of our density approximations.

Since the true densities are explicitly known under the BOU and BOUI models, we

can compute the true MLEs in the experiments so as to discern quantitatively the

impacts of these two types of errors. We tabulate the means and standard deviations

of these errors in Tables 1 and 2.

Table 1 contains the estimation results for the BOU model. It shows that θ(J)n − θn,

the difference between our approximate MLE and the true MLE, decreases rapidly as

J increases. In addition, this difference is dominated (at least one order of magnitude)

by θn−θ0, the difference between the true MLE and the true parameter values. Hence,

for the purpose of estimating θ0, our Ito-Taylor expansion based estimator θ(J)n with

a relatively small J (say, J = 4) can be used as a meaningful substitute for the

(generally incomputable) MLE θn. Note that the BOU process is an example of

the reducible multivariate diffusions, amenable to the Lamperti transform. We also

report the estimation results from the 3rd-order expansion of Aıt-Sahalia (2008) in

Table 1. The comparison shows that the accuracy of our estimator when taking J = 7

outperforms his estimators for most of the parameters.

[Table 1 about here.]

Table 2 compares the performance of a variety of MLEs under a time-inhomogeneous

19

Page 20: Density Approximations for Multivariate Di usions via an ...nchenweb/working_paper/MLE_JE_20171014.pdf · Density Approximations for Multivariate Di usions ... approach in the current

model BOUI, including the MLE derived from the true process density, the approx-

imate MLE developed in Choi (2013), and the approximate MLE based on the Ito-

Taylor expansion. As we can see, the accuracy of the MLE yielded by the 4th-order

Ito-Taylor expansion exceeds that of the approximate MLE using the 2nd-order Choi’s

expansion. Like what Table 1 reveals, the approximation error caused by the Ito-

Taylor expansion (cf. the last column in Table 2) is dominated by the sampling error

of the maximum likelihood method (cf. the second column in Table 2).

[Table 2 about here.]

We consider the Heston and GARCH models in Table 3, demonstrating the per-

formance of our method for irreducible and nonaffine diffusion processes. Neither of

these two models admits closed-form expressions for their transition densities. Hence,

we compute the difference between the true parameter values and the Ito-Taylor M-

LE to assess the method’s accuracy. Aıt-Sahalia (2008) expanded the log-likelihood

functions of such processes from their accompanying Kolmogorov equations to devel-

op approximate MLEs. We also include the estimation results from his method4 in

Table 3 for the purpose of comparison. With J = 4, the biases of our estimators

under both the Heston and GARCH models are very small relative to the true pa-

rameter values. The accuracy of our results are comparable to that of the 2nd order

approximation of Aıt-Sahalia (2008).

[Table 3 about here.]

Finally, we examine the standard deviation of our estimators with finite samples.

In theory, when a process is known to be stationary, its true MLE should have a local

asymptotic normal structure, that is,

√n(θn − θ0)→ N (0, i(θ0)−1),

4The 2nd order density approximation formulas for the Heston and GARCH models are also

available on Yacine Aıt-Sahalia’s website.

20

Page 21: Density Approximations for Multivariate Di usions via an ...nchenweb/working_paper/MLE_JE_20171014.pdf · Density Approximations for Multivariate Di usions ... approach in the current

as n→∞ with ∆ fixed, where i(θ0) is the Fisher’s information matrix defined as

i(θ) = E

[(∂

∂θln p(X(t1)|X(t0); θ)

)(∂

∂θln p(X(t1)|X(t0); θ)

)>].

In Table 4, we take certain parameters under which the BOU model is stationary.

Using its explicit density function and our approximations, we compute the asymp-

totic standard deviation (ASD) of the true MLE and the finite-sample standard de-

viation (FSSD) of the approximate MLE respectively. The table demonstrates that

the finite-sample standard deviations of our estimators are very close to the efficient

asymptotic standard deviations, for different number of observations (n = 500, 2000,

5000). Moreover, the rate they decrease is the same as the order of√n, consistent

with what predicted by the local asymptotic normal structure.

[Table 4 about here.]

We carry out the same experiments for some other non-stationary diffusions. The

results in Table 5 show that the standard deviations of our estimators for the BOUI,

Heston and GARCH models all decrease at a rate of√n, similar as that in the

stationary case.

[Table 5 about here.]

5 Conclusion

This paper constructs a closed-form approximation of the transition densities for

multivariate diffusions using the Ito-Taylor expansion, in which only differentiation

is involved. We manage to prove that it will converge to the true density under a set

of very mild technical conditions. In this way, the method unifies the treatment for

a wide range of models, especially time-inhomogeneous irreducible non-affine multi-

variate diffusions. Numerical experiments illustrate the computational efficiency and

accuracy of the approximate MLE resulted from our Ito-Taylor density expansion.

21

Page 22: Density Approximations for Multivariate Di usions via an ...nchenweb/working_paper/MLE_JE_20171014.pdf · Density Approximations for Multivariate Di usions ... approach in the current

Acknowledgement

The authors are grateful to Prof. Dacheng Xiu of the University of Chicago, Prof.

George J. Jiang of Washington State University, for their valuable suggestions. Part

of the research of the second author is supported by Hong Kong Research Grant

Council under GRF Grant No. CUHK14201114. The third author thanks the Na-

tional Natural Science Foundation of China (NSFC) for the financial support under

Grants No. 71401103 and 71421002.

A The Assumptions and Proofs

A.1 The Assumptions

We need the following technical conditions in our convergence analysis.

Assumption A.1. Let D = Πmi=1(xi, xi) be the domain of diffusion X defined in

(1). It is possible that xi = −∞ and/or xi = +∞. Moreover, the boundary of D is

unattainable for the process X.

The literature has developed a systematic approach to testing whether or not a bound-

ary is attainable for a diffusion process through its drift and volatility coefficients. For

instance, Karatzas and Shreve (1991) and Aıt-Sahalia (2002) consider the issue of u-

nattainability for univariate diffusions; Friedman (1976) discusses about multivariate

diffusions in Chapter 11.

The Ito-Taylor expansion involves repeated differentiation of the drift µ(t, x; θ)

and the diffusion matrix ν(t, x; θ), so we need,

Assumption A.2. All the components of µ(t, x; θ) and ν(t, x; θ) are infinitely differ-

entiable in (t, x), at any (t, x, θ) ∈ [0,+∞)×D ×Θ.

22

Page 23: Density Approximations for Multivariate Di usions via an ...nchenweb/working_paper/MLE_JE_20171014.pdf · Density Approximations for Multivariate Di usions ... approach in the current

The differentiability of (µ, ν), or consequentially (µ, σ), implies they are locally Lip-

schitz continuous. This assumption thus ensures that the solution to SDE (1) is

strongly unique in the sense of Definition 5.2.3 in Karatzas and Shreve (1991). How-

ever, it is not sufficient to guarantee the existence of a transition density for X in

(1). Therefore, to ensure the existence of a transition density, we impose two classical

conditions below (see, e.g., Friedman, 1964, Chapter 1, Theorem 10; Friedman, 1975,

Chapter 6, Theorem 4.5)

Assumption A.3. The diffusion matrix ν(t, x; θ) is uniformly positive definite; that

is, there exists a positive constant c0 such that ξ>ν(t, x; θ)ξ ≥ c0ξ>ξ for any nonzero

vector ξ ∈ Rm and (t, x, θ) ∈ [0,+∞)×D ×Θ.

Assumption A.4. µ(t, x; θ) and ν(t, x; θ) are bounded and their derivatives exhibit

at most polynomial growth in x for (t, x, θ) ∈ [0,+∞)×D ×Θ.

The above two assumptions are sufficient conditions for the existence. It is in general

difficult (if not impossible) to relax the two conditions. A theoretical relaxation should

be considered case-by-case.

A.2 Technical Lemmas

Lemma 1. Fix (t′, x′) and (t, x).5 For N ≥ 1,

(∂s + L)Nq(t′, x′|s, y) =2N∑|h|=1

wN,h(s, y)∂hq(t′, x′|s, y), (20)

where the operator L is defined by (4) acting on the state variable y, the multivariate

normal density q(t′, x′|s, y) is defined as follows

q(t′, x′|s, y) :=1

(2π(t′ − s))m/2 det(ν0)1/2exp

(−(x′ − y)>ν−1

0 (x′ − y)

2(t′ − s)

),

and the coefficient function wN,h(s, y) is defined by (16) and (17).

5Thus, ν0 = ν(t, x; θ) is also fixed.

23

Page 24: Density Approximations for Multivariate Di usions via an ...nchenweb/working_paper/MLE_JE_20171014.pdf · Density Approximations for Multivariate Di usions ... approach in the current

Proof of Lemma 1. We use mathematical induction to prove this lemma. For sim-

plicity, we omit the arguments in the functions q(t′, x′|s, y), µi(s, y; θ), νij(s, y; θ), and

wN,h(s, y) without confusion hereafter.

For N = 1, by the definition of q(t′, x′|s, y), we have

∂sq = −L0q, (21)

where L0 is a differential operator acting on y defined by

L0 =1

2

m∑i,j=1

ν0ij∂ei+ej .

Then we have

(∂s + L)q = (−L0 + L)q =m∑i=1

µi∂eiq +1

2

m∑i,j=1

(νij − ν0ij)∂ei+ejq.

Recalling the definition of wN,h in (16), (20) holds for N = 1.

Next, assume that (20) holds for N . Then, for N + 1, we have

(∂s + L)N+1q = (∂s + L)( 2N∑|h|=1

wN,h∂hq)

=2N∑|h|=1

(∂s(wN,h∂hq) + L(wN,h∂hq)

). (22)

Applying (21) to the first term gives that

∂s(wN,h∂hq) = (∂swN,h)∂hq + wN,h(∂s∂hq) = (∂swN,h)∂hq + wN,h(∂h∂sq)

= (∂swN,h)∂hq − wN,h(∂hL0q) = (∂swN,h)∂hq − wN,hL0(∂hq).

The second term on the right hand side of (22) follows

L(wN,h∂hq) =m∑i=1

µi∂ei(wN,h∂hq) +1

2

m∑i,j=1

νij∂ei+ej(wN,h∂hq)

= (LwN,h)∂hq + wN,hL(∂hq) +m∑

i,j=1

νij(∂ejwN,h)∂h+eiq.

24

Page 25: Density Approximations for Multivariate Di usions via an ...nchenweb/working_paper/MLE_JE_20171014.pdf · Density Approximations for Multivariate Di usions ... approach in the current

Putting them together, we have

(∂s + L)N+1q =2N∑|h|=1

((∂s + L)wN,h)∂hq

+2N∑|h|=1

m∑i,j=1

νij(∂ejwN,h)∂h+eiq +2N∑|h|=1

m∑i=1

wN,h · µi∂h+eiq

+2N∑|h|=1

1

2

m∑i,j=1

wN,h · (νij − ν0ij)∂h+ei+ejq.

Rewriting the index in the summation gives that

(∂s + L)N+1q =2N∑|h|=1

((∂s + L)wN,h

)∂hq

+2N+1∑|h|=2

m∑i,j=1

νij(∂ejwN,h−ei)∂hq +2N+1∑|h|=2

m∑i=1

µiwN,h−ei∂hq

+2N+2∑|h|=3

1

2

m∑i,j=1

(νij − ν0ij)wN,h−ei−ej∂hq

=2N+2∑|h|=1

wN+1,h∂hq.

The last equality holds by the definition of wN+1,h in (17) and the assignment that

wN,h(s, y) ≡ 0 if minh1, · · · , hm < 0, or h = 0, or |h| > 2N . Hence, (20) holds for

N + 1.

Lemma 2. Fix t and x. Recalling the functions wN,h(s, y) ≡ wN,h(s, y; t, x, θ) :

s ≥ 0, y ∈ D are defined by (16) and (17), the following statement holds: for each

N ≥ 1,

wN,h(s, y)|s=t,y=x = 0, if 3N/2 < |h| ≤ 2N. (23)

Proof of Lemma 2. The statement (23) obviously holds for N = 1. Indeed, for N = 1,

3N/2 < |h| ≤ 2N implies that |h| = 2. Thus by (16), we havew1,2ei(t, x) = 12

(νii(s, y; θ)|s=t,y=x − νii(t, x; θ)

)= 0, i = 1, · · · ,m;

w1,ei+ej(t, x) = νij(s, y; θ)|s=t,y=x − νij(t, x; θ) = 0, i 6= j, i, j = 1, · · · ,m.(24)

25

Page 26: Density Approximations for Multivariate Di usions via an ...nchenweb/working_paper/MLE_JE_20171014.pdf · Density Approximations for Multivariate Di usions ... approach in the current

Inspired by (24), by Taylor’s Theorem and Assumption A.2, we can expand the

function wN,h(s, y) at (t, x) up to the K-th order as follows:

wN,h(s, y) =∑

0≤a0+|a|≤K

ξN,ha0,a· (s− t)a0(y − x)a +

∑a0+|a|=K

ΩN,ha0,a

(s, y) · (s− t)a0(y − x)a,

where a0 is a nonnegative integer number, a = (a1, · · · , am) is a vector index with

nonnegative integer components, and xa = xa11 · · ·xamm . For each a0, a, the coefficient

ξN,ha0,a:= ∂a0s ∂awN,h(s, y)|s=t,y=x is a constant, and the function ΩN,h

a0,a(s, y) satisfies

lims→t,y→x ΩN,ha0,a

(s, y) = 0. Thus, we can define the Order of the function wN,h(s, y)

at (t, x) as follows

Or(wN,h) := mink∣∣ ξN,ha0,a

6= 0, a0 + |a| = k.

Obviously, wN,h(s, y)|s=t,y=x = 0 if Or(wN,h) ≥ 1. Then, we can prove the statement

(23) by showing a stronger statement below: for each N ≥ 1

Or(wN,h) ≥ 2|h| − 3N, if 3N/2 < |h| ≤ 2N. (25)

It is equivalent to say, for each N ≥ 1 and 3N/2 < |h| ≤ 2N ,

wN,h(s, y) =∑

a0+|a|=2|h|−3N

(ξN,ha0,a

+ ΩN,ha0,a

(s, y))· (s− t)a0(y − x)a. (26)

We use mathematical induction to verify that (25) holds. For N = 1, (25) holds

by (24). Assume that (25) holds for N . Note that (i) the Order of a summation of

functions is at least the minimum Order of each function; (ii) a first order differential

operator acting on a function will decrease its Order at most by 1; (iii) the Order

of a multiplication of functions is the summation of the Order of each function; (iv)

Or(νij(s, y; θ)− νij(t, x; θ)) ≥ 1. Therefore for N + 1, by equation (17), we have

Or(wN+1,h) ≥ minOr(wN,h)− 2, Or(wN,h−ei), Or(wN,h−ei)− 1, 1 +Or(wN,h−ei−ej)

≥ min(2|h| − 3N)− 2, (2(|h| − 1)− 3N)− 1, 1 + (2(|h| − 2)− 3N)

≥ 2|h| − 3(N + 1).

Hence we have verified that (25) holds for N + 1. This completes the proof.

26

Page 27: Density Approximations for Multivariate Di usions via an ...nchenweb/working_paper/MLE_JE_20171014.pdf · Density Approximations for Multivariate Di usions ... approach in the current

Example 1. Regarding the CIR model (cf. Model 2), we verify the result (23) in

Lemma 2 for N = 1, 2, 3, 4. By a direct computation (cf. Equations (16) and (17)),

the coefficients wN,h(s, y), 3N/2 < h ≤ 2N are given by

w1,2(s, y) =1

2σ2(y − x);

w2,4(s, y) =1

4σ4(y − x)2;

w3,5(s, y) =3

4σ6y(y − x) +

3

4κ(α− y)σ4(y − x)2;

w3,6(s, y) =1

8σ6(y − x)3;

w4,7(s, y) =3

4σ8y(y − x)2 +

1

2κ(α− y)σ6(y − x)3;

w4,8(s, y) =1

16σ8(y − x)4.

It is obviously that all above are zeros when y = x.

A.3 The Proofs

Proof of Theorem 1. Fix (t, x) and (t′, x′). For any s ∈ [t, t′) and y ∈ D, by the

definition in (10), we have

p(J)(t′, x′|s, y; θ) =J∑

N=0

(t′ − s)N

N !(∂s + L)Nq(t′, x′|s, y), (27)

where the operator L is defined by (4) acting on state variable y. By (20) in Lemma

1, we have

p(J)(t′, x′|s, y; θ) = q(t′, x′|s, y) +J∑

N=1

2N∑|h|=1

(t′ − s)N

N !wN,h(s, y)∂hq(t

′, x′|s, y). (28)

Note that (cf. (12))

∂hq(t′, x′|s, y) = (t′ − s)−

|h|2 Hh(z; ν0)q(t′, x′|s, y) (29)

with z = (x′ − y)/√t′ − s. Plugging it into (28), we have

p(J)(t′, x′|s, y; θ) = q(t′, x′|s, y)(

1 +J∑

N=1

2N∑|h|=1

(t′ − s)N−|h|2

N !wN,h(s, y)Hh(z; ν0)

). (30)

27

Page 28: Density Approximations for Multivariate Di usions via an ...nchenweb/working_paper/MLE_JE_20171014.pdf · Density Approximations for Multivariate Di usions ... approach in the current

Taking s = t and y = x, by (23) in Lemma 2, we see that equation (14) holds. This

completes the proof.

Proof of Theorem 2. Firstly, fix (t, x), then µ = 0 and ν0 = ν(t, x; θ) are also fixed.

For s ∈ [t, t′) and y ∈ D, consider p(t′, x′|s, y; θ) and p(J)(t′, x′|s, y; θ). Note that,

p(t′, x′|s, y; θ) satisfies the backward Kolmogorov PDE associated with SDE (1) (see,

e.g., Section 5.1 in Karatzas and Shreve, 1991)

(∂s + L)p(t′, x′|s, y; θ) = 0, limt′−s→0

p(t′, x′|s, y; θ) = δ(x′ − y), (31)

where L is defined in (4). Applying (∂s + L) to p(J)(t′, x′|s, y; θ) (cf. (27)), we have

(∂s + L)p(J)(t′, x′|s, y; θ) =J∑

N=0

(∂s + L)

((t′ − s)N

N !(∂s + L)Nq(t′, x′|s, y)

)=

(t′ − s)J

J !(∂s + L)J+1q(t′, x′|s, y) := ψJ(s, y; t, x, t′, x′). (32)

Note that

limt′−s→0

q(t′, x′|s, y) = δ(x′ − y). (33)

To establish a similar initial condition for p(J), consider any test function ϕ(·), which

is continuous with compact support on D. By (30),∫D

(p(J)(t′, x′|s, y; θ)− q(t′, x′|s, y)

)ϕ(x′)dx′

=J∑

N=1

2N∑|h|=1

(t′ − s)N−|h|/2

N !wN,h(s, y)

∫D

q(t′, x′|s, y)Hh(z; ν0)ϕ(x′)dx′, (34)

where z = (x′ − y)/√t′ − s. Note that (cf. (11))

q(t′, x′|s, y) = (t′ − s)−m/2φ(z; ν0).

Changing the variable from x′ to z, and denoting DZ as the domain of z, we have∫D

q(t′, x′|s, y)Hh(z; ν0)ϕ(x′)dx′ =

∫DZ

φ(z; ν0)Hh(z; ν0)ϕ(y + z√t′ − s)dz.

28

Page 29: Density Approximations for Multivariate Di usions via an ...nchenweb/working_paper/MLE_JE_20171014.pdf · Density Approximations for Multivariate Di usions ... approach in the current

Then, taking t′ − s→ 0 on both sides of (34), we have

limt′−s→0

∫D

(p(J)(t′, x′|s, y; θ)− q(t′, x′|s, y)

)ϕ(x′)dx′

=ϕ(y)J∑

N=1

∑|h|=2N

1

N !wN,h(s, y)

∫DZ

φ(z; ν0)Hh(z; ν0)dz, (35)

which is zero when s = t and y = x, because wN,h(t, x) = 0 for all |h| = 2N (cf. (23)).

For any J , such that J > 2m − 1, define the error between the partial sum p(J)

and the true density p as

r(J)(t′, x′|s, y; θ) = p(J)(t′, x′|s, y; θ)− p(t′, x′|s, y; θ).

Then, by (31) and (32), it satisfies the following Kolmogorov PDE:

(∂s + L)r(J)(t′, x′|s, y; θ) = ψJ(s, y; t, x, t′, x′). (36)

Besides, by (31), (33) and (35), when s = t and y = x, the initial condition becomes

limt′−t→0

r(J)(t′, x′|t, x; θ) = 0. (37)

Thus,

r(J)(t′, x′|t, x; θ) =

∫ t′

t

∫D

ψJ(s, y; t, x, t′, x′) · p(s, y|t, x; θ)dyds. (38)

Recalling (32) and (20), we decompose ψJ into three terms:

ψJ :=ψ(1)J + ψ

(2)J + ψ

(3)J , (39)

where for i = 1, 2, 3,

ψ(i)J (s, y; t, x, t′, x′) =

∑h∈Ii

(t′ − s)J

J !wJ+1,h(s, y)∂hq(t

′, x′|s, y),

and I1 =h∣∣1 ≤ |h| ≤ 3(J + 1)/2

, I2 =

h∣∣3(J + 1)/2 < |h| < 2(J + 1)

,

I3 =h∣∣|h| = 2(J + 1)

. Then, by (38) and (39), we can rewrite the error term into

a summation r(J) = r(J)1 + r

(J)2 + r

(J)3 , where

r(J)i (t′, x′|t, x; θ) =

∫ t′

t

∫D

ψ(i)J (s, y; t, x, t′, x′) · p(s, y|t, x; θ)dyds, i = 1, 2, 3.

29

Page 30: Density Approximations for Multivariate Di usions via an ...nchenweb/working_paper/MLE_JE_20171014.pdf · Density Approximations for Multivariate Di usions ... approach in the current

Recalling z = (x′ − y)/√t′ − s and (29), we further have

r(J)i =

∫ t′

t

∫D

∑h∈Ii

(t′ − s)J−|h|2

J !Hh(z; ν0)q(t′, x|s, y)wJ+1,h(s, y)p(s, y|t, x; θ)dyds.

(40)

Next, consider the bounds for r(J)1 , r

(J)2 , and r

(J)3 , respectively. To bound each

remainder term in (40), the basic idea is to decompose the integrand in (40) into two

parts: Hh(z; ν0)q(t′, x|s, y) and wJ+1,h(s, y)p(s, y|t, x; θ), and bound each of them.

Consider the first term rJ1 with h ∈ I1 (i.e., 1 ≤ |h| ≤ 3(J + 1)/2). Recall that

∂hq(t′, x′|s, y) = (t′− s)−|h|/2Hh(z; ν0)q(t′, x′|s, y) in (29), together with Theorem 1 in

Chapter 9 of Friedman (1964), we have

|Hh(z; ν0)q(t′, x|s, y)| ≤ C(t′ − s)−m2 e−λ0‖x

′−y‖2

2(t′−s) , (41)

where λ0, C are positive constants depending only on ν(t, x; θ), t, x, J . Moreover, by

Assumptions A.3 and A.4 and (6.12) in Chapter 1 of Friedman (1964), there exists

λ1 > 0 such that

|p(s, y|t, x; θ)| ≤ C(s− t)−m2 e−

λ1‖y−x‖2

2(s−t) .

Since wJ+1,h is a polynomial and x belongs to a compact set Dc, there exists λ2 ∈

(0, λ1) such that

|wJ+1,h(s, y)p(s, y|t, x; θ)| ≤ C(s− t)−m2 e−

λ2‖y−x‖2

2(s−t) . (42)

Combining (41) and (42), we have∫D

|Hh(z; ν0)q(t′, x|s, y)wJ+1,h(s, y)p(s, y|t, x; θ)|dy

≤C∫D

(t′ − s)−m2 e−λ‖x

′−y‖22(t′−s) (s− t)−

m2 e−

λ‖y−x‖22(s−t) dy ≤ C(t′ − t)−

m2 e−λ‖x

′−x‖22(t′−t) ,

where λ = minλ0, λ2 is a positive constant depending only on t, x, and J . Recalling

30

Page 31: Density Approximations for Multivariate Di usions via an ...nchenweb/working_paper/MLE_JE_20171014.pdf · Density Approximations for Multivariate Di usions ... approach in the current

(40) and h ∈ I1 (i.e., 1 ≤ |h| ≤ 3(J + 1)/2), thus we have

|r(J)1 | ≤

∑h∈I1

C(t′ − t)−m2 e−λ‖x

′−x‖22(t′−s)

∫ t′

t

(t′ − s)J−|h|2

J !ds

≤C(t′ − t)J+14−m

2 e−λ‖x

′−x‖22(t′−t)

=O((t′ − t)J+14−m

2 ), for small (t′ − t). (43)

For the term r(J)2 with h ∈ I2 (i.e., 3(J + 1)/2 < |h| < 2(J + 1)), the inequality in

(41) still holds. Using the expansion of wJ+1,h in (26), we have an alternative bound

as follows

|wJ+1,h(s, y)p(s, y|t, x; θ)| ≤ C(s− t)a0+|a|2−m

2 e−λ2‖y−x‖

2

2(s−t)

= C(s− t)a02

+2|h|−3(J+1)

2−m

2 e−λ2‖y−x‖

2

2(s−t) , (44)

where a0 is a nonnegative integer satisfying a0 + |a| = 2|h| − 3(J + 1), and the first

inequality holds due to the following fact

∣∣∣∣( y − x√s− t

)ap(s, y|t, x; θ)

∣∣∣∣ ≤ C

∣∣∣∣( y − x√s− t

)a∣∣∣∣ e−λ1‖y−x‖

2

2(s−t)

(s− t)m2≤ C(s− t)−

m2 e−

λ2‖y−x‖2

2(s−t) .

Then, combining (40), (41) with (44) and h ∈ I2 (i.e., 3(J + 1)/2 < |h| < 2(J + 1)),

we have

|r(J)2 | ≤C

∑h∈I2

(t′ − t)−m2 e−λ‖x

′−x‖22(t′−t) ·

∫ t′

t

(t′ − s)J−|h|2

J !(s− t)

a02

+2|h|−3(J+1)

2 ds (45)

=C∑h∈I2

(t′ − t)a02

+|h|−(J+1)

2−m

2 e−λ‖x

′−x‖22(t′−t) ·B(1+J− |h|

2,1+

a02

+2|h|−3(J+1)

2 )

=O((t′ − t)J+14−m

2 ), for small (t′ − t), (46)

where the beta function B(x, y) is finite for x > 0 and y > 0.

For the term r(J)3 with h ∈ I3 (i.e., |h| = 2(J + 1)), the integrand (t′ − s)J−|h|/2 =

(t′ − s)−1 in (45) is not integrable around s = t′. To overcome the problem, we use

integration by parts to reduce the order of differentiation by one. By ∂hq(t′, x′|s, y) =

31

Page 32: Density Approximations for Multivariate Di usions via an ...nchenweb/working_paper/MLE_JE_20171014.pdf · Density Approximations for Multivariate Di usions ... approach in the current

∂ei∂h−eiq(t′, x′|s, y) and integration by parts,6 then we have,∫

D

∂ei∂h−eiq(t′, x′|s, y) · wJ+1,h(s, y)p(s, y|t, x; θ)dy

=

∫D

∂h−eiq(t′, x′|s, y) ·

(∂eiwJ+1,h(s, y) · p(s, y|t, x; θ) + wJ+1,h(s, y) · ∂eip(s, y|t, x; θ)

)dy.

Recalling (29), similar to (41), we have

|∂h−eiq(t′, x′|s, y)| = |(t′ − s)−|h|−1

2 Hh−ei(z; ν0)q(t′, x′|s, y)| ≤ C(t′ − s)−|h|−1+m

2 e−λ0‖x

′−y‖2

2(t′−s) .

(47)

Similar to (44), by (26), we have

|∂eiwJ+1,h(s, y) · p(s, y|t, x; θ)| ≤ C(s− t)a02

+2|h|−3(J+1)−1

2−m

2 e−λ2||y−x||

2

2(s−t) . (48)

Similar to (42), by (26) and (6.13) in Chapter 1 of Friedman (1964), we have

|wJ+1,h(s, y) · ∂eip(s, y|t, x; θ)| ≤ C(s− t)a02

+2|h|−3(J+1)

2−m+1

2 e−λ2||y−x||

2

2(s−t) . (49)

Recall r(J)3 defined in (38) and (39) with h ∈ I3 (i.e., |h| = 2(J + 1)). Then, similar

to the argument for r(J)2 , combining (47-49), we have

|r(J)3 | ≤C

∑h∈I3

(t′ − t)−m2 e−λ||x

′−x||22(t′−t) ·

∫ t′

t

(t′ − s)J−(|h|−1)/2

J !(s− t)

a02

+2|h|−3(J+1)

2− 1

2ds

=C∑h∈I3

(t′ − t)a02

+|h|−(J+1)

2−m

2 e−λ||x

′−x||22(t′−t) ·B(1+J− |h|−1

2,1+

a02

+2|h|−3(J+1)

2− 1

2)

=O((t′ − t)J+12−m

2 ), for small (t′ − t). (50)

Finally, putting (43), (46), and (50) into together, the error |r(J)(t′, x′|t, x; θ)| =

O((t′ − t)J+14−m

2 ) as t′ − t→ 0, uniformly for (t, x, x′, θ) ∈ [0, T ]×Dc ×D ×Θ.

Proof of Theorem 3. For fixed x ∈ Dc, T > 0 and 0 ≤ t < t′ ≤ T , let

R(J)(t′, x′|t, x; Θ) := supθ∈Θ|p(t′, x′|t, x; θ)− p(J)(t′, x′|t, x; θ)|.

6Since the values at the boundaries are of order exp(−c0||x||2/(s − t)) (if it is not zero), which

decays faster than any polynomials, we do not consider the values at the boundaries when using

integration by parts.

32

Page 33: Density Approximations for Multivariate Di usions via an ...nchenweb/working_paper/MLE_JE_20171014.pdf · Density Approximations for Multivariate Di usions ... approach in the current

According to Theorem 2, for fixed x and t, p(J)(t′, x′|t, x) converges to p(t′, x′|t, x; θ)

uniformly in x′ ∈ D and θ ∈ Θ as ∆→ 0. That is, for any ε > 0, there exists a positive

∆ε independent of x′ and θ, such that for all ∆ ≤ ∆ε, we have R(J)(t′, x′|t, x; Θ) < ε,

and

Eθ0 [R(J)(t′, X(t′)|t,X(t); Θ)|X(t) = x] =

∫D

R(J)(t′, x′|t, x; Θ)p(t′, x′|t, x; θ0)dx′

≤ ε

∫D

p(t′, x′|t, x; θ0)dx′ = ε.

Thus, by Chebyshev’s inequality, the sequence R(J)(t′, Xt′ |t,Xt; Θ) converges to zero

in probability, given Xt = x, that is, given any ζ > 0,

lim∆→0

Prob[|R(J)(t′, Xt′|t,Xt; Θ)| > ζ|Xt = x; θ0] = 0.

Then,

Prob[|R(J)(t′, Xt′ |t,Xt; Θ)| > ζ; θ0]

=

∫D

Prob[|R(J)(t′, Xt′ |t,Xt; Θ)| > ζ|Xt = x; θ0]πt(x; θ0)dx,

where πt(x; θ0) is the marginal density of Xt at the true parameter value θ0. Since the

probability is bounded and the density is integrable, by the dominated convergence

theorem, we have

lim∆→0

Prob[|R(J)(t′, Xt′ |t,Xt; Θ)| > ζ; θ0] = 0,

that is, R(J)(t′, Xt′ |t,Xt; Θ) converges to zero in probability, as ∆ → 0. And the

convergence of supθ∈Θ |`(J)n (θ)−`n(θ)| to zero in probability follows from the continuity

of the logarithm functions (e.g. Lemma 4.3 in Kallenberg, 2002). The existence of

the approximate MLE θ(J)n and its convergence in probability to the true MLE θn are

guaranteed by the continuous differentiability in θ of the log-likelihood function `n(θ)

and the approximate log-likelihood functions `(J)n (θ) for all J .

33

Page 34: Density Approximations for Multivariate Di usions via an ...nchenweb/working_paper/MLE_JE_20171014.pdf · Density Approximations for Multivariate Di usions ... approach in the current

References

Aıt-Sahalia, Y. (2002): “Maximum Likelihood Estimation of Discretely Sampled

Diffusions: A Closed-Form Approximation Approach,” Econometrica, 70, 223–262.

Aıt-Sahalia, Y. (2008): “Closed-Form Likelihood Expansions for Multivariate D-

iffusions,” The Annals of Statistics, 36, 906–937.

Aıt-Sahalia, Y., and P. A. Mykland (2003): “The Effects of Random and

Discrete Sampling when Estimating Continuous-Time Diffusions,” Econometrica,

71, 483–549.

Aıt-Sahalia, Y., and J. Yu (2006): “Saddlepoint Approximations for Continuous-

Time Markov Processes,” Journal of Econometrics, 134, 507–551.

Bakshi, G., and N. Ju (2005): “A Refinement to Aıt-Sahalia’s (2002) Maximum

Likelihood Estimation of Discretely Sampled Diffusions: A Closed-Form Approxi-

mation Approach,” Journal of Business, 78, 2037–2052.

Chang, J., and S. X. Chen(2011): “On the Approximate Maximum Likelihood

Estimation for Diffusions Processes,” The Annals of Statistics, 39, 2820-2851.

Choi, S. (2013): “Closed-Form Likelihood Expansions for Multivariate Time-

Inhomogeneous Diffusions,” Journal of Econometrics, 174, 45–65.

Choi, S. (2015): “Explicit Form of Approximate Transition Probability Density

Functions of Diffusion Processes,” Journal of Econometrics, 187, 57–73.

Cox, J. C., J. E. Ingersoll, and S. A. Ross (1985): “A Theory of the Term

Structure of Interest Rates,” Econometrica, 53, 385–407.

Duan, J.C. (1995): “The GARCH Option Pricing Model,” Mathematical Finance,

5, 13-32.

34

Page 35: Density Approximations for Multivariate Di usions via an ...nchenweb/working_paper/MLE_JE_20171014.pdf · Density Approximations for Multivariate Di usions ... approach in the current

Dynkin, E.B. (1965): Markov Processes: Volume 1. Academic Press, New York.

Egorov, A.V., H. Li, and Y. Xu (2003): “Maximum Likelihood Estimation of

Time-Inhomogeneous Diffusions,” Journal of Econometrics, 114, 107–139.

Fan, J., and C. Zhang(2003): “A Reexamination of Diffusion Estimators with

Applications to Financial Model Validation,” Journal of the American Statistical

Association, 98, 118–134.

Filipovic, D., E. Mayerhofer, and P. Schneider (2013): “Density Approxi-

mations for Multivariate Affine Jump-Diffusion Processes,” Journal of Economet-

rics, 176, 93–111.

Friedman, A. (1964): Partial Differential Equations Of Parabolic Type. Dover Pub-

lications, New York.

Friedman, A. (1975): Stochastic Differential Equations and Applications: Volume

1. Academic Press, New York.

Friedman, A. (1976): Stochastic Differential Equations and Applications: Volume

2. Academic Press, New York.

Hansen, L. P., and T. J. Sargent (1983): “The Dimensionality of the Aliasing

Problem in Models with Rational Spectral Densities,” Econometrica, 51, 377–387.

Heston, S. L. (1993): “A Closed-Form Solution for Options with Stochastic Volatil-

ity with Applications to Bond and Currency Options,” Review of Financial Studies,

6, 327–343.

Kallenberg, O. (2002): Foundations of Modern Probability. Springer, New York.

Karatzas, I., and S. E. Shreve (1991): Brownian Motion and Stochastic Calcu-

lus. Springer, New York.

35

Page 36: Density Approximations for Multivariate Di usions via an ...nchenweb/working_paper/MLE_JE_20171014.pdf · Density Approximations for Multivariate Di usions ... approach in the current

Kessler, M. (1997): “Estimation of An Ergodic Diffusion from Discrete Observa-

tions,” Scandinavian Journal of Statistics, 24, 211–229.

Kessler, M., and A. Rahbek (2004): “Identification and Inference for Multi-

variate Cointegrated and Ergodic Gaussian Diffusions,” Statistical Inference for

Stochastic Processes, 7, 137–151.

Kristensen, D., and A. Mele (2011): “Adding and Subtracting Black-Scholes: A

New Approach to Approximating Derivative Prices in Continuous-Time Models,”

Journal of Financial Economics, 102, 390–415.

Lee, Y. D., S. Song, and E. K. Lee (2014): “The Delta Expansion for the

Transition Density of Diffusion Models,” Journal of Econometrics, 178, 694–705.

Li, C. (2013): “Maximum-Likelihood Estimation for Diffusion Processes via Closed-

Form Density Expansions,” The Annals of Statistics, 41, 1350–1380.

Li, C., and D. Chen (2016): “Estimating Jump-diffusions Using Closed-form Like-

lihood Expansions,” Journal of Econometrics, 195, 51–70.

Li, C., and X. Li (2015): “A Closed-Form Expansion Approach for Pricing Dis-

cretely Monitored Variance Swaps,” Operations Research Letters, 43, 450–455.

Lo, A. W. (1988): “ Maximum Likelihood Estimation of Generalized Ito Processes

with Discretely Sampled Data,” Econometric Theory, 4, 231–247.

Nelson, D. B. (1990): “ARCH Models as Diffusion Approximations,” Journal of

Econometrics, 45, 7–38.

Milstein, G. N. (1975): “Approximate Integration of Stochastic Differential Equa-

tions,” Theory of Probability and Its Applications. 19, 557–562.

Pedersen, A. R. (1995): “A New Approach to Maximum Likelihood Estimation for

Stochastic Differential Equations Based on Discrete Observations,” Scandinavian

Journal of Statistics, 22, 55–71.

36

Page 37: Density Approximations for Multivariate Di usions via an ...nchenweb/working_paper/MLE_JE_20171014.pdf · Density Approximations for Multivariate Di usions ... approach in the current

Schaumburg, E. (2001): Maximum Likelihood Estimation of Jump Processes with

Applications to Finance. Ph.D. thesis, Princeton University.

Stanton, R. (1997): “A Nonparametric Model of Term Structure Dynamics and

the Market Price of Interest Rate Risk,” Journal of Finance, 52, 1973–2002.

Uchida, M., and N. Yoshida (2012): “Adaptive Estimation of An Ergodic Diffu-

sion Process Based on Sampled Data,” Stochastic Processes and Their Applications,

122, 2885–2924.

Vasicek, O. (1977): “An Equilibrium Characterisation of the Term Structure,”

Journal of Financial Economics, 5, 177–188.

Willink, R. (2005): “Normal Moments and Hermite Polynomials,” Statistics and

Probability Letters, 73, 271–275.

Xiu, D. C. (2014): “Hermite Polynomial Based Expansion of European Option

Prices,” Journal of Econometrics, 179, 158–177.

Yu, J. (2007): “Closed-Form Likelihood Approximation and Estimation of Jump-

Diffusions with An Application to the Realignment Risk of the Chinese Yuan,”

Journal of Econometrics, 141, 1245–1280.

37

Page 38: Density Approximations for Multivariate Di usions via an ...nchenweb/working_paper/MLE_JE_20171014.pdf · Density Approximations for Multivariate Di usions ... approach in the current

Figure 1: Maximum Absolute Error between True and Approximate Densities

Notes: The maximum absolute error between the true densities p and the Ito-Taylor expansion p(J)

(14) is defined as maxx′∈D |p(t′, x′|t, x; θ)− p(J)(t′, x′|t, x; θ)|, where x, t, and t′ are fixed, the regionD of forward state x′ is large enough to include several standard deviations from the mean. We usethe following values for the parameters in these four models: OU: (κ, α, σ)=(0.5, 0.06, 0.03); CIR:(κ, α, σ)=(0.5, 0.06, 0.15); BOU: (α1, α2, κ11, κ21, κ22)=(0, 0, 5, 1, 10); BOUI: (α1, α2, κ11, κ21,κ22, β1, β2)=(0, 0, 5, 1, 10, 0.1, 0.1).

38

Page 39: Density Approximations for Multivariate Di usions via an ...nchenweb/working_paper/MLE_JE_20171014.pdf · Density Approximations for Multivariate Di usions ... approach in the current

Figure 2: Comparison of Maximum Absolute Error between Different Expansions

Notes: This picture compares the maximum absolute error of our Ito-Taylor expansion and thatof Aıt-Sahalia (2008) and Choi(2013). The definition of maximum absolute error is similar to thatdefined in Figure 1. The values of parameters in these four models are given in the following: OU:(κ, α, σ, t′ − t)=(0.5, 0.06, 0.03, 1/52); CIR: (κ, α, σ, t′ − t)=(0.5, 0.06, 0.15, 1/52); BOU: (α1, α2,κ11, κ21, κ22, t′ − t)=(0, 0, 5, 1, 10,1/252); BOUI: (α1, α2, κ11, κ21, κ22, β1, β2, t′ − t, t)=(0, 0, 5,1, 10, 0.1, 0.1, 1/252, 0).

39

Page 40: Density Approximations for Multivariate Di usions via an ...nchenweb/working_paper/MLE_JE_20171014.pdf · Density Approximations for Multivariate Di usions ... approach in the current

Table 1: Monte Carlo Evidence for the BOU Model with Different Order of Expansion

θ0 θn − θ0 θ(AS)n − θn θ

(4)n − θn θ

(5)n − θn θ

(6)n − θn θ

(7)n − θn

κ11 = 5 0.41888 0.00186 0.00774 -0.00368 0.00222 0.00080(1.11974) (0.04472) (0.14139) (0.10338) (0.06851) (0.06916)

κ21 = 1 0.03538 0.00105 0.01423 -0.00017 0.00276 -0.00123(1.19813) (0.02933) (0.15667) (0.11368) (0.08693) (0.05589)

κ22 = 10 0.61443 0.00324 0.04357 -0.000490 0.00239 -0.00097(1.54834) (0.02971) (0.27662) (0.22413) (0.10551) (0.10502)

α1 = 0 0.00486 0.00091 0.00021 0.00105 0.00132 0.00052(0.06284) (0.01477) (0.02000) (0.01959) (0.01962) (0.01983)

α2 = 0 -0.00033 0.00009 0.00003 -0.00008 0.00003 -0.00003(0.03351) (0.00443) (0.00675) (0.00653) (0.00637) (0.00563)

Notes: We use θ0 to generate 1000 sample paths. Each of them contains 500 weekly observations(i.e., we take t′ − t = 1/52). The first column reports true parameter values θ0. The secondcolumn reports the bias and the standard derivation (values in parentheses) of the true maximum

likelihood estimator θn. The third column shows the difference between true maximum likelihoodestimator θn and the 3rd-order approximate estimator θ

(AS)n developed by Aıt-Sahalia (2008). The

remaining columns report the differences between true maximum likelihood estimator θn and Ito-

Taylor expansion estimator θ(J)n developed in this paper, with the standard derivation in parentheses.

The order of our Ito-Taylor expansion takes values from J = 4 to J = 7.

40

Page 41: Density Approximations for Multivariate Di usions via an ...nchenweb/working_paper/MLE_JE_20171014.pdf · Density Approximations for Multivariate Di usions ... approach in the current

Table 2: Monte Carlo Evidence for the BOUI Model

θ0 θn − θ0 θ(Choi)n − θn θ

(4)n − θn

κ11 = 5 0.59446 0.08046 -0.01367(1.17004) (0.26004) (0.33544)

κ21 = 1 0.10497 -1.11154 0.00273(1.24774) (1.94661) (0.34327)

κ22 = 10 0.62750 -0.37080 -0.11460(1.57026) (0.38605) (0.52059)

α1 = 0 -0.00163 0.00546 0.00263(0.08942) (0.07228) (0.07931)

α2 = 0 0.00205 0.00137 -0.00024(0.06047) (0.03442) (0.03417)

β1 = 0.1 0.00112 -0.00057 -0.00047(0.01839) (0.01124) (0.01241)

β2 = 0.1 -0.00085 0.00046 0.00007(0.01122) (0.00540) (0.00560)

Notes: We use θ0 to generate 1000 sample paths. Each of them contains 500 weekly observations(i.e., we take t′ − t = 1/52). The first column reports true parameter values θ0. The second

column contains the bias of the true maximum likelihood estimator θn. We display the differences

between the true maximum likelihood estimator θn and the 2nd order approximate estimator θ(Choi)n

developed by Choi (2013) in the third column. The fourth column shows the differences between θnand the 4th-order approximate estimator θ

(4)n using our Ito-Taylor expansion in (14). All standard

deviations are reported in the parentheses.

41

Page 42: Density Approximations for Multivariate Di usions via an ...nchenweb/working_paper/MLE_JE_20171014.pdf · Density Approximations for Multivariate Di usions ... approach in the current

Table 3: Monte Carlo Evidence for Heston and GARCH Models

Heston Model (β = 1/2) GARCH Model (β = 1)

θ0 θ(AS)n − θ0 θ

(4)n − θ0 θ

(AS)n − θ0 θ

(4)n − θ0

σ = 0.25 0.00064 0.00060 0.00060 0.00060(0.00644) (0.00643) (0.00644) (0.00644)

ρ = −0.8 -0.00075 -0.00059 -0.00069 -0.00068(0.01339) (0.01335) (0.01338) (0.01339)

α = 0.1 0.00024 0.00026 -0.00003 0.00002(0.00735) (0.00747) (0.00225) (0.00228)

µ = 0.03 -0.00284 -0.00342 0.00219 0.00013(0.08155) (0.08164) (0.07899) (0.07998)

κ = 3 0.14815 0.14498 0.15436 0.15563(0.53684) (0.54298) (0.53960) (0.54548)

Notes: We use θ0 to generate 1000 sample paths. Each of them contains 500 weekly observations(i.e., we take t′ − t = 1/52). The two subtables correspond to the results from the Heston andGARCH models, respectively. The first column of each subtable displays the bias of the 2nd order

approximate estimator θ(AS)n developed by Aıt-Sahalia (2008); the second column illustrates the bias

of the 4th-order approximate estimator θ(4)n using our Ito-Taylor expansion in (14). All standard

deviations are reported in the parentheses.

42

Page 43: Density Approximations for Multivariate Di usions via an ...nchenweb/working_paper/MLE_JE_20171014.pdf · Density Approximations for Multivariate Di usions ... approach in the current

Table 4: Comparison between Asymptotic and Finite-Sample Standard Deviation forthe BOU Model

500 observations 2000 observations 5000 observationsθ0 ASD FSSD ASD FSSD ASD FSSDκ11 = 5 1.02 1.12 0.51 0.51 0.32 0.33κ21 = 1 1.08 1.18 0.54 0.54 0.34 0.34κ22 = 10 1.45 1.53 0.73 0.72 0.46 0.43α1 = 0 0.065 0.063 0.032 0.030 0.020 0.019α2 = 0 0.033 0.034 0.016 0.017 0.010 0.011

Notes: This table presents the asymptotic standard deviation (ASD) and the finite-sample standarddeviation (FSSD) as the number of observations are 500, 2000, and 5000, respectively. The numberof simulation trials is 1000 for all three cases. The length of the time interval is fixed at t′−t = 1/52.

All the results for FSSD are based on the 4th-order approximate estimator θ(4)n using our Ito-Taylor

expansion in (14).

43

Page 44: Density Approximations for Multivariate Di usions via an ...nchenweb/working_paper/MLE_JE_20171014.pdf · Density Approximations for Multivariate Di usions ... approach in the current

Table 5: Finite-Sample Standard Deviation for the BOUI, Heston, GARCH Modelswith Different Number of Observations

θ0 500 observations 2000 observations 5000 observationsBOUI

κ11 = 5 1.18 0.50 0.34κ21 = 1 1.20 0.54 0.33κ22 = 10 1.52 0.71 0.48α1 = 0 0.094 0.037 0.021α2 = 0 0.062 0.030 0.015β1 = 0 0.019 0.0020 0.00050β2 = 0 0.012 0.0014 0.00038

Hestonσ = 0.25 0.0064 0.0031 0.0021ρ = −0.8 0.013 0.0070 0.0042α = 0.1 0.0075 0.0036 0.0021µ = 0.03 0.082 0.037 0.022κ = 3 0.54 0.25 0.17

GARCHσ = 0.25 0.0064 0.0030 0.0020ρ = −0.8 0.013 0.0065 0.0042α = 0.1 0.0023 0.0013 0.0008µ = 0.03 0.080 0.046 0.030κ = 3 0.55 0.25 0.16

Notes: This table presents the finite-sample standard deviation for the BOUI, Heston, GARCHModels. The number of observations are 500, 2000, and 5000, respectively. The number of simulationtrials is 1000 for all three cases. The length of the time interval is fixed at t′ − t = 1/52. All the

results are based on the 4th-order approximate estimator θ(4)n using our Ito-Taylor expansion in

(14).

44