Stationary Probability Vector of a Higher-order Markov Chain By Zhang Shixiao Supervisors: Prof....
-
Upload
zachary-maxfield -
Category
Documents
-
view
216 -
download
0
Transcript of Stationary Probability Vector of a Higher-order Markov Chain By Zhang Shixiao Supervisors: Prof....
![Page 1: Stationary Probability Vector of a Higher-order Markov Chain By Zhang Shixiao Supervisors: Prof. Chi-Kwong Li and Dr. Jor-Ting Chan.](https://reader036.fdocuments.us/reader036/viewer/2022062619/5516e29855034603568b45ae/html5/thumbnails/1.jpg)
Stationary Probability Vectorof a Higher-order Markov Chain
By Zhang Shixiao
Supervisors: Prof. Chi-Kwong Li and Dr. Jor-Ting Chan
![Page 2: Stationary Probability Vector of a Higher-order Markov Chain By Zhang Shixiao Supervisors: Prof. Chi-Kwong Li and Dr. Jor-Ting Chan.](https://reader036.fdocuments.us/reader036/viewer/2022062619/5516e29855034603568b45ae/html5/thumbnails/2.jpg)
Content
• 1. Introduction: Background
• 2. Higher-order Markov Chain
• 3. Conclusion
![Page 3: Stationary Probability Vector of a Higher-order Markov Chain By Zhang Shixiao Supervisors: Prof. Chi-Kwong Li and Dr. Jor-Ting Chan.](https://reader036.fdocuments.us/reader036/viewer/2022062619/5516e29855034603568b45ae/html5/thumbnails/3.jpg)
1. Introduction: Background
• Matrices are widely used in both science and engineering.
• In statisticsStochastic process: flow direction of a particular system or process.Stationary distribution: limiting behavior of a stochastic process.
![Page 4: Stationary Probability Vector of a Higher-order Markov Chain By Zhang Shixiao Supervisors: Prof. Chi-Kwong Li and Dr. Jor-Ting Chan.](https://reader036.fdocuments.us/reader036/viewer/2022062619/5516e29855034603568b45ae/html5/thumbnails/4.jpg)
Discrete Time-HomogeneousMarkov Chains
• A stochastic process with a discrete finite state space S
Pr (𝑋 𝑡+1= 𝑗∨𝑋 𝑡=𝑖 , 𝑋𝑡 −1=𝑖𝑡−1 , 𝑋 𝑡−2=𝑖𝑡− 2 ,… , ,𝑋 1=𝑖1 , 𝑋 0=𝑖0 )¿ Pr (𝑋 𝑡+1= 𝑗∨𝑋 𝑡=𝑖 )=𝑝𝑖𝑗
𝑃= (𝑝𝑖𝑗 , 𝑖 , 𝑗∈𝑆 )
• A unit sum vector X is said to be a stationary probability distribution of a finite Markov Chain if PX=X where
![Page 5: Stationary Probability Vector of a Higher-order Markov Chain By Zhang Shixiao Supervisors: Prof. Chi-Kwong Li and Dr. Jor-Ting Chan.](https://reader036.fdocuments.us/reader036/viewer/2022062619/5516e29855034603568b45ae/html5/thumbnails/5.jpg)
Discrete Time-HomogeneousMarkov Chains
• In other words
a coutinuous function f: which preserves at least one fixed point.
![Page 6: Stationary Probability Vector of a Higher-order Markov Chain By Zhang Shixiao Supervisors: Prof. Chi-Kwong Li and Dr. Jor-Ting Chan.](https://reader036.fdocuments.us/reader036/viewer/2022062619/5516e29855034603568b45ae/html5/thumbnails/6.jpg)
2. Higher-order Markov Chain
• a stochastic process with a sequence of random variables, , which takes on a finite set called the state set of the process
• Definition 2.1 Suppose the probability independent of time satisfying
Pr (𝑋 𝑡+1=𝑖∨𝑋 𝑡=𝑖1 ,𝑋 𝑡− 1=𝑖2 , 𝑋𝑡 −2=𝑖3 ,…, ,𝑋 1=𝑖𝑡 , 𝑋 0=𝑖𝑡+1 )¿ Pr (𝑋 𝑡+1=𝑖∨𝑋 𝑡=𝑖1 ,𝑋 𝑡− 1=𝑖2 , 𝑋𝑡 −2=𝑖3 , …,𝑋 𝑡 −𝑚+1=𝑖𝑚)
¿𝑝𝑖 ,𝑖1 , 𝑖2 ,⋯ , 𝑖𝑚
![Page 7: Stationary Probability Vector of a Higher-order Markov Chain By Zhang Shixiao Supervisors: Prof. Chi-Kwong Li and Dr. Jor-Ting Chan.](https://reader036.fdocuments.us/reader036/viewer/2022062619/5516e29855034603568b45ae/html5/thumbnails/7.jpg)
2. Higher-order Markov Chain
• Definition 2.2 Write to be a three-order n-dimensional tensor
where and define an n-dimensional column vector
![Page 8: Stationary Probability Vector of a Higher-order Markov Chain By Zhang Shixiao Supervisors: Prof. Chi-Kwong Li and Dr. Jor-Ting Chan.](https://reader036.fdocuments.us/reader036/viewer/2022062619/5516e29855034603568b45ae/html5/thumbnails/8.jpg)
2. Higher-order Markov Chain
• Example: is a three-order 2-dimensional tensor where and
![Page 9: Stationary Probability Vector of a Higher-order Markov Chain By Zhang Shixiao Supervisors: Prof. Chi-Kwong Li and Dr. Jor-Ting Chan.](https://reader036.fdocuments.us/reader036/viewer/2022062619/5516e29855034603568b45ae/html5/thumbnails/9.jpg)
Conditions forInfinitely Many Solutions over the Simplex
• Theorem 2.1 Now we are consideringwhere all
𝑥 ( 𝑎1 𝑏1
1−𝑎1 1−𝑏1)( 𝑥
1−𝑥)+ (1−𝑥 )( 𝑎2 𝑏2
1 −𝑎2 1 −𝑏2)( 𝑥
1−𝑥 )=( 𝑥1−𝑥)
![Page 10: Stationary Probability Vector of a Higher-order Markov Chain By Zhang Shixiao Supervisors: Prof. Chi-Kwong Li and Dr. Jor-Ting Chan.](https://reader036.fdocuments.us/reader036/viewer/2022062619/5516e29855034603568b45ae/html5/thumbnails/10.jpg)
Conditions forInfinitely Many Solutions over the Simplex
• Then one of the following holds
If , then we must have two solutions or to the above equation.If , then we must have infinitely many solutions, namely, every with is a solution to the above equation.
Otherwise, we must have a unique solution.
![Page 11: Stationary Probability Vector of a Higher-order Markov Chain By Zhang Shixiao Supervisors: Prof. Chi-Kwong Li and Dr. Jor-Ting Chan.](https://reader036.fdocuments.us/reader036/viewer/2022062619/5516e29855034603568b45ae/html5/thumbnails/11.jpg)
Conditions forInfinitely Many Solutions over the Simplex
• Then we want to extend the condition for infinitely many solutions for case
![Page 12: Stationary Probability Vector of a Higher-order Markov Chain By Zhang Shixiao Supervisors: Prof. Chi-Kwong Li and Dr. Jor-Ting Chan.](https://reader036.fdocuments.us/reader036/viewer/2022062619/5516e29855034603568b45ae/html5/thumbnails/12.jpg)
Main Theorem 2.2
would have infinitely many solutions over the whole set
if and only if
![Page 13: Stationary Probability Vector of a Higher-order Markov Chain By Zhang Shixiao Supervisors: Prof. Chi-Kwong Li and Dr. Jor-Ting Chan.](https://reader036.fdocuments.us/reader036/viewer/2022062619/5516e29855034603568b45ae/html5/thumbnails/13.jpg)
Main Theorem 2.2
12 1
121
1
1
1
1
n
n
a a
aA
a
12
12 23 2
2 23
2
1
1
1
1
n
n
a
a a a
A a
a
![Page 14: Stationary Probability Vector of a Higher-order Markov Chain By Zhang Shixiao Supervisors: Prof. Chi-Kwong Li and Dr. Jor-Ting Chan.](https://reader036.fdocuments.us/reader036/viewer/2022062619/5516e29855034603568b45ae/html5/thumbnails/14.jpg)
Main Theorem 2.2
1
2
1,
,1 , 12 1,
, 1
,
1
1
1
1
1
1
i
i
i i
ii ni i ii i i
i i
i n
a
a
aA
aa aa a
a
a
![Page 15: Stationary Probability Vector of a Higher-order Markov Chain By Zhang Shixiao Supervisors: Prof. Chi-Kwong Li and Dr. Jor-Ting Chan.](https://reader036.fdocuments.us/reader036/viewer/2022062619/5516e29855034603568b45ae/html5/thumbnails/15.jpg)
Main Theorem 2.2
1
2
1,
1 2 1,
1
1
1
1
n
n
n
n n
n n n n
a
a
A
a
a a a
![Page 16: Stationary Probability Vector of a Higher-order Markov Chain By Zhang Shixiao Supervisors: Prof. Chi-Kwong Li and Dr. Jor-Ting Chan.](https://reader036.fdocuments.us/reader036/viewer/2022062619/5516e29855034603568b45ae/html5/thumbnails/16.jpg)
Main Theorem 2.2
Proof:Sufficiency:For , infinitely many solutions
![Page 17: Stationary Probability Vector of a Higher-order Markov Chain By Zhang Shixiao Supervisors: Prof. Chi-Kwong Li and Dr. Jor-Ting Chan.](https://reader036.fdocuments.us/reader036/viewer/2022062619/5516e29855034603568b45ae/html5/thumbnails/17.jpg)
Main Theorem 2.2
![Page 18: Stationary Probability Vector of a Higher-order Markov Chain By Zhang Shixiao Supervisors: Prof. Chi-Kwong Li and Dr. Jor-Ting Chan.](https://reader036.fdocuments.us/reader036/viewer/2022062619/5516e29855034603568b45ae/html5/thumbnails/18.jpg)
Main Theorem 2.2
12 1, 1
121
1, 1
1
1
1
n
n
a a
aA
a
12
12 23 2, 1
232
2, 1
1
1
1
1
n
n
a
a a a
aA
a
1
2
1,
, 11 , 12 1,
, 1
, 1
1
1
1
1
1
1
i
i
i i
ii ni i ii i i
i i
i n
a
a
aA
aa aa a
a
a
1, 1
2, 1
1
2, 1
1, 1 2, 1 2, 1
1
1
1
1
n
n
n
n n
n n n n
a
a
A
a
a a a
![Page 19: Stationary Probability Vector of a Higher-order Markov Chain By Zhang Shixiao Supervisors: Prof. Chi-Kwong Li and Dr. Jor-Ting Chan.](https://reader036.fdocuments.us/reader036/viewer/2022062619/5516e29855034603568b45ae/html5/thumbnails/19.jpg)
Main Theorem 2.2
12 13 1
12 22 2
13 22
1,
1,1 2
1
1
1
1
n
n
n n
n nn n
a a a
a a a
a aM
a
aa a
![Page 20: Stationary Probability Vector of a Higher-order Markov Chain By Zhang Shixiao Supervisors: Prof. Chi-Kwong Li and Dr. Jor-Ting Chan.](https://reader036.fdocuments.us/reader036/viewer/2022062619/5516e29855034603568b45ae/html5/thumbnails/20.jpg)
Other
• Given any two solutions lying on the interior of1-dimensional face of the boundary of the simplex, then the whole 1-dimensional face must be a set of collection of solutions to the above equation.
• Conjecture: given any k+1 solutions lying in the interior of the k-dimensional face of the simplex, then any point lying in the whole k-dimensional face, including the vertexes and boundaries, will be a solution to the equation.
![Page 21: Stationary Probability Vector of a Higher-order Markov Chain By Zhang Shixiao Supervisors: Prof. Chi-Kwong Li and Dr. Jor-Ting Chan.](https://reader036.fdocuments.us/reader036/viewer/2022062619/5516e29855034603568b45ae/html5/thumbnails/21.jpg)
3. Conclusion
![Page 22: Stationary Probability Vector of a Higher-order Markov Chain By Zhang Shixiao Supervisors: Prof. Chi-Kwong Li and Dr. Jor-Ting Chan.](https://reader036.fdocuments.us/reader036/viewer/2022062619/5516e29855034603568b45ae/html5/thumbnails/22.jpg)
Thank you!