Theory of Stochastic Processes 13. Reviesei/lec/ohp_SP13.pdf · Theory of Stochastic Processes 13....
Transcript of Theory of Stochastic Processes 13. Reviesei/lec/ohp_SP13.pdf · Theory of Stochastic Processes 13....
.
.
. ..
.
.
Theory of Stochastic Processes13. Review
Tomonari [email protected]
Department of Mathematical Informatics, University of Tokyo
July 13, 2017
http://www.stat.t.u-tokyo.ac.jp/~sei/lec.html
1 / 13
Handouts & Announcements
Handouts:
Slides (this one)
Solved problems
Announcements:
About the final exam (reminder)
The final exam is held on July 20 (Thu), 10:25–, 90 min.Contact me if you cannot attend it due to some unavoidable reasons.The exam is open-book and open-note. Write your answer in English.The exam will cover the whole material up to July 6. At least onequestion will be from the topics before the midterm exam.
2 / 13
Outline today
.
. .
1 Review of last week’s material
.
. .
2 Solved problems
3 / 13
Review: Ito calculus
Let Wt be the standard Brownian motion.An Ito process Xt is defined by
dXt = µtdt + σtdWt .
The conditional distribution given Ft = {Ws}s≤t is approximately
Xt+∆t |Ft ∼ N(µt∆t, σ2t ∆t).
Xt is a martingale if µt = 0.
Ito’s formula:
d{f (t, Xt)} =∂f
∂tdt +
∂f
∂xdXt +
1
2
∂2f
∂x2(dXt)
2.
4 / 13
Diffusion processes
A diffusion process is a solution to the stochastic differential equation
dXt = µ(t, Xt)dt + σ(t, Xt)dWt .
Xt is Markov.
The future behavior depends only on the present value.
5 / 13
Markov processes
We encountered many examples of Markov processes.
discrete time continuous time
discrete state
5 10 15 20 25
12
34
56
timest
ate
0.0 0.1 0.2 0.3 0.4
02
46
8
time
stat
e
Markov chain Poisson process
continuous state
5 10 15 20 25
−2
−1
01
time
stat
e
0 200 400 600 800 1000
−3.
0−
2.0
−1.
00.
0
timest
ate
AR(1) process diffusion process
6 / 13
The forward equation
Denote the transition density from Xs = x to Xt = y by p(t, y |s, x).The forward equation (Fokker-Planck equation) is
∂p
∂t= − ∂
∂y(µp) +
1
2
∂2
∂y2(σ2p).
.
Key exercise
.
.
.
. ..
.
.
If µ and σ are constant, then Xt ∼ N(µt, σ2t). Write down the transitiondensity p(t, y) = p(t, y |0, 0) and check that the forward equation holds.
Proof sketch: Ito’s formula implies
dE [f (Xt)] = E [f ′(Xt)µ + (1/2)f ′′(Xt)σ2]dt
for any function f . Since E [f (Xt)] =∫
f (y)p(t, y |s, x)dy , we have∫f (y)
∂p
∂tdy =
∫{f ′(y)µ + (1/2)f ′′(y)}pdy .
The result follows from the integral-by-parts formula.7 / 13
Langevin Monte Carlo
The diffusion process
dXt =1
2(log π∗(Xt))
′dXt + dWt
has the stationary distribution π(x) = π∗(x)/Z .
Indeed, p(t, y) = π(y) satisfies the forward equation:
−(
1
2(log π∗)
′π
)′+
1
2π′′ = −
(1
2π′
)′+
1
2π′′ = 0.
MCMC using the diffusion process is called Langevin Monte Carlo.
8 / 13
Metropolis-Adjusted Langevin algorithm (MALA)
In practice, discretize the equation as
Xt+∆t = Xt + µ(Xt)∆t + ∆Wt , ∆Wt ∼ N(0,√
∆t)
(Euler-Maruyama scheme), where µ(x) = (1/2)(log π∗)′(x).
The conditional density of Xt+∆t given Xt = x is
q(y |x) =1√
2π∆texp
(−(y − x − µ(x)∆t)2
2∆t
).
It is used as a proposal density for the Metropolis-Hastings algorithm.
Specifically, the algorithm is
.
.
.
1 Set an initial value x .
.
.
.
2 Generate a random number y according to q(y |x).
.
.
.
3 Compute the acceptance probability a = min(1, π∗(y)q(x|y)π∗(x)q(y |x) ).
.
.
.
4 Genrate U ∼ U(0, 1). If U ≤ a, then x ← y . Otherwise, x ← x .
.
.
.
5 Go to step 2.
9 / 13
Example 1
Let π∗(x) =1
{1 + (x − 2)2}3{1 + (x + 2)2}3.
Sample paths
∆t = 0.01 without MH ∆t = 1 with MH
t ∈ [0, 100]0 20 40 60 80 100
−4
−2
02
4
time
X
0 20 40 60 80 100
−4
−2
02
4
time
Xt ∈ [0, 10000]
0 2000 4000 6000 8000 10000
−4
−2
02
4
time
X
0 2000 4000 6000 8000 10000−
4−
20
24
time
X
10 / 13
Example 2
Let π∗(x) = φ(x)Φ(x), φ(x) = e−x2/2/√
2π and Φ(x) =∫ x−∞ φ(z)dz .
Comparison of Langevin MC (left) and random walk MC (right),where the random walk MC uses Xt+∆t = Xt + ∆Wt as the proposal.
0 200 600 1000
−1
01
23
Langevin MC
Index
X
0 200 600 1000
−1
01
23
random walk MC
Index
X
0 5 10 15 20 25 30
0.0
0.4
0.8
Lag
AC
F
0 5 10 15 20 25 30
0.0
0.4
0.8
Lag
AC
F
11 / 13
The underlined problems are solved. See the handout.
June 8 (Lec 8) Markov chain Monte Carlo
§6.5: Problem 1, 8*, 9.§6.14: Problem 1.§6.15: Problem 2.
June 15 (Lec 9) Stationary processes
Problem 1, 2, 3, 4, 5, 6.
June 22 (Lec 10) Martingales
§12.1: Problem 1, 2, 3, 4, 5, 6, 7*, 8, 9*.§12.2: Problem 1, 2.
June 29 (Lec 11) Queues
§8.4: Problem 1, 2, 3, 4*, 5*.§11.2: Problem 3, 7*.§11.3: Problem 1* .
July 6 (Lec 12) Diffusion processes
Problem 1, 2, 3, 4, 5, 6, 7, 8, 9.
12 / 13
Final remark
Before After
We have lost fresh flowers but obtained many leaves of knowledge!
Thanks
13 / 13