Jiann-Ming Wu, Ya-Ting Zhou, Chun-Chang Wu National Dong Hwa University Department of Applied...
-
Upload
norma-sullivan -
Category
Documents
-
view
223 -
download
1
Transcript of Jiann-Ming Wu, Ya-Ting Zhou, Chun-Chang Wu National Dong Hwa University Department of Applied...
![Page 1: Jiann-Ming Wu, Ya-Ting Zhou, Chun-Chang Wu National Dong Hwa University Department of Applied Mathematics Hualien, Taiwan Learning Markov-chain embedded.](https://reader035.fdocuments.us/reader035/viewer/2022062515/56649f555503460f94c78fe2/html5/thumbnails/1.jpg)
Jiann-Ming Wu, Ya-Ting Zhou, Chun-Chang Wu
National Dong Hwa University
Department of Applied Mathematics
Hualien, Taiwan
Learning Markov-chain embedded recurrence relations for chaotic time series analysis
![Page 2: Jiann-Ming Wu, Ya-Ting Zhou, Chun-Chang Wu National Dong Hwa University Department of Applied Mathematics Hualien, Taiwan Learning Markov-chain embedded.](https://reader035.fdocuments.us/reader035/viewer/2022062515/56649f555503460f94c78fe2/html5/thumbnails/2.jpg)
Outline
Introduction High-order Markov processes for stochastic
modeling Nonlinear recurrence relations for deterministic
modelingRecurrence relation approximation by
supervised learning of radial or projective basis functions
Markov-chain embedded recurrence relationsNumerical Simulations Conclusions
![Page 3: Jiann-Ming Wu, Ya-Ting Zhou, Chun-Chang Wu National Dong Hwa University Department of Applied Mathematics Hualien, Taiwan Learning Markov-chain embedded.](https://reader035.fdocuments.us/reader035/viewer/2022062515/56649f555503460f94c78fe2/html5/thumbnails/3.jpg)
High-order Markov assumptionLet Z[t] denote time series, where t is positive
integersHigh-order Markov assumption-
Given chaotic time series are oriented from a generative source well characterized by a high-order Markov process.
An order- Markov process obeys memory-less property
Current event only depends on instances of most recently events instead of all historic events
])[Z],...,1[Z|][ZPr(])1[Z],...,[Z],...,1[Z|][ZPr( tttttt
![Page 4: Jiann-Ming Wu, Ya-Ting Zhou, Chun-Chang Wu National Dong Hwa University Department of Applied Mathematics Hualien, Taiwan Learning Markov-chain embedded.](https://reader035.fdocuments.us/reader035/viewer/2022062515/56649f555503460f94c78fe2/html5/thumbnails/4.jpg)
Recurrence relationConditional expectation of an upcoming
event to most recently events is expressed by a recurrence relation
eventcurrent denotes
][][
events previous denotes
])[],...,1[(][
]),[(G][
])[],...,1[(G
] -z[t,1],-z[t|Z[t]][
tzty
tztzt
where
tty
lyequivalent
tztz
ty
T
x
x
![Page 5: Jiann-Ming Wu, Ya-Ting Zhou, Chun-Chang Wu National Dong Hwa University Department of Applied Mathematics Hualien, Taiwan Learning Markov-chain embedded.](https://reader035.fdocuments.us/reader035/viewer/2022062515/56649f555503460f94c78fe2/html5/thumbnails/5.jpg)
1000,,
]),[]2[]1[sin(
)][],...,1[G(][
5
521
t
tzatzatza
tztztz
Ttztztzt ][,],2[],1[][ x
])[sin(
])[(G][T t
tty
xa
x
][][ tytz
T21 ),,,( aaa a
Recurrence relation for time series modeling
predictor
target
![Page 6: Jiann-Ming Wu, Ya-Ting Zhou, Chun-Chang Wu National Dong Hwa University Department of Applied Mathematics Hualien, Taiwan Learning Markov-chain embedded.](https://reader035.fdocuments.us/reader035/viewer/2022062515/56649f555503460f94c78fe2/html5/thumbnails/6.jpg)
0 500 1000 15000.2
0.4
0.6
0.8
1
1.2
1.4
1.6
Mackey-Glass 30 chaotic time series data
Chaotic time series
Laser data 10000 from the SFI competition
![Page 7: Jiann-Ming Wu, Ya-Ting Zhou, Chun-Chang Wu National Dong Hwa University Department of Applied Mathematics Hualien, Taiwan Learning Markov-chain embedded.](https://reader035.fdocuments.us/reader035/viewer/2022062515/56649f555503460f94c78fe2/html5/thumbnails/7.jpg)
RECURRENCE RELATION APPROXIMATION
• Learning neural networks for approximating underlying recurrence relation
• F denotes a mapping realized by radial or projective basis functions
• denotes adaptive network parameters
)][(])[(G ][ tFtty xx
![Page 8: Jiann-Ming Wu, Ya-Ting Zhou, Chun-Chang Wu National Dong Hwa University Department of Applied Mathematics Hualien, Taiwan Learning Markov-chain embedded.](https://reader035.fdocuments.us/reader035/viewer/2022062515/56649f555503460f94c78fe2/html5/thumbnails/8.jpg)
Recurrence relation approximation
Form paired predictor and target by assigning
Define the mean square error of approximating
Apply Levenberg-Marquardt learning to resolve unconstrained optimization
Apply the proposed pair-data generative model to formulate F
)(min Eopt
])[],[( tytx
ttztytztzt T allfor ][][ and ])[],...,1[(][ x
)][(by ][ tFty x
2)|][(][(1
)( tFtyN
Et x
![Page 9: Jiann-Ming Wu, Ya-Ting Zhou, Chun-Chang Wu National Dong Hwa University Department of Applied Mathematics Hualien, Taiwan Learning Markov-chain embedded.](https://reader035.fdocuments.us/reader035/viewer/2022062515/56649f555503460f94c78fe2/html5/thumbnails/9.jpg)
Pair-data generative model (PGM)
K sub-models
![Page 10: Jiann-Ming Wu, Ya-Ting Zhou, Chun-Chang Wu National Dong Hwa University Department of Applied Mathematics Hualien, Taiwan Learning Markov-chain embedded.](https://reader035.fdocuments.us/reader035/viewer/2022062515/56649f555503460f94c78fe2/html5/thumbnails/10.jpg)
Mixtures of paired Gaussians
A stochastic model for formation emulation of given paired data
Each time one of joined pairs is selected according to a set of prior probabilities
Apply the selected paired Gaussians to generate paired data
])[],[( tytx
![Page 11: Jiann-Ming Wu, Ya-Ting Zhou, Chun-Chang Wu National Dong Hwa University Department of Applied Mathematics Hualien, Taiwan Learning Markov-chain embedded.](https://reader035.fdocuments.us/reader035/viewer/2022062515/56649f555503460f94c78fe2/html5/thumbnails/11.jpg)
Each pair is exactly generated by a sub-modelLet denote the exclusive
membership of where denotes a unitary vector with the ith bit
active
By exclusive membership
The conditional expectation of y to given x is defined by
r denotes local means of the target variable
Exclusive Memberships
,,,][ M21 eeeδ t
model-subkth by the generated is ][],[ if ][ tytt k xeδ
0,,1,,0 i e
][],[ tytx
ie
rδx Ty
![Page 12: Jiann-Ming Wu, Ya-Ting Zhou, Chun-Chang Wu National Dong Hwa University Department of Applied Mathematics Hualien, Taiwan Learning Markov-chain embedded.](https://reader035.fdocuments.us/reader035/viewer/2022062515/56649f555503460f94c78fe2/html5/thumbnails/12.jpg)
Overlapping membershipsA Potts random variable is applied to
encode overlapping membershipThe probability of being the kth state is set
to
where modulates the overlapping degree and
denotes local mean of the predictor
δ
)][exp(Pr2
kk t μxeδ
kμ
![Page 13: Jiann-Ming Wu, Ya-Ting Zhou, Chun-Chang Wu National Dong Hwa University Department of Applied Mathematics Hualien, Taiwan Learning Markov-chain embedded.](https://reader035.fdocuments.us/reader035/viewer/2022062515/56649f555503460f94c78fe2/html5/thumbnails/13.jpg)
Normalized radial basis functions ( NRBF )
The conditional expectation exactly sketches a mapping realized by normalized radial basis functions
][exp
][exp
,,Let
][exp
][expPr
followsit
1Pr and )][exp(Pr Since
2
2
1
1
2
2
1
2
k
hh
kkT
TM
M
hh
kkk
M
kkkk
t
try
vv
t
tv
t
μx
μxrvx
v
μx
μxeδ
eδμxeδ
![Page 14: Jiann-Ming Wu, Ya-Ting Zhou, Chun-Chang Wu National Dong Hwa University Department of Applied Mathematics Hualien, Taiwan Learning Markov-chain embedded.](https://reader035.fdocuments.us/reader035/viewer/2022062515/56649f555503460f94c78fe2/html5/thumbnails/14.jpg)
Figure 4
![Page 15: Jiann-Ming Wu, Ya-Ting Zhou, Chun-Chang Wu National Dong Hwa University Department of Applied Mathematics Hualien, Taiwan Learning Markov-chain embedded.](https://reader035.fdocuments.us/reader035/viewer/2022062515/56649f555503460f94c78fe2/html5/thumbnails/15.jpg)
0 500 1000 15000
0.5
1
1.5source
0 50 100 150 200 250 300 350 400 450 5000
0.5
1
1.5approximation
0 50 100 150 200 250 300 350 400 450 500-4
-2
0
2
4x 10
-3 approximating error
Figure 9
Mackey-Glass 17 chaotic time series data
![Page 16: Jiann-Ming Wu, Ya-Ting Zhou, Chun-Chang Wu National Dong Hwa University Department of Applied Mathematics Hualien, Taiwan Learning Markov-chain embedded.](https://reader035.fdocuments.us/reader035/viewer/2022062515/56649f555503460f94c78fe2/html5/thumbnails/16.jpg)
Multiple recurrence relationsMultiple recurrence relations for modeling
more complex chaotic time series
Chaotic time series
Laser data 10000 from the SFI competition
![Page 17: Jiann-Ming Wu, Ya-Ting Zhou, Chun-Chang Wu National Dong Hwa University Department of Applied Mathematics Hualien, Taiwan Learning Markov-chain embedded.](https://reader035.fdocuments.us/reader035/viewer/2022062515/56649f555503460f94c78fe2/html5/thumbnails/17.jpg)
Markov-chain embedded recurrence relations
A Markov chain of PGMs (pair-data generative models)
Transition matrix
denotes the probability of transition from model i to model j
, jiT
jiT ,
![Page 18: Jiann-Ming Wu, Ya-Ting Zhou, Chun-Chang Wu National Dong Hwa University Department of Applied Mathematics Hualien, Taiwan Learning Markov-chain embedded.](https://reader035.fdocuments.us/reader035/viewer/2022062515/56649f555503460f94c78fe2/html5/thumbnails/18.jpg)
Data generation
Emulate data generation by a stochastic Markov chain of PGMs
![Page 19: Jiann-Ming Wu, Ya-Ting Zhou, Chun-Chang Wu National Dong Hwa University Department of Applied Mathematics Hualien, Taiwan Learning Markov-chain embedded.](https://reader035.fdocuments.us/reader035/viewer/2022062515/56649f555503460f94c78fe2/html5/thumbnails/19.jpg)
Inverse problem of Markov chain embedded PGMs
ies.probabilitn transitioEstimate 2.
on.segmentatifor points switching Find 1.
as stated becan tion reconstruc modelfor problem inverse The
PGMs embeddedchain -Markovby generated sequence orderedan Given
![Page 20: Jiann-Ming Wu, Ya-Ting Zhou, Chun-Chang Wu National Dong Hwa University Department of Applied Mathematics Hualien, Taiwan Learning Markov-chain embedded.](https://reader035.fdocuments.us/reader035/viewer/2022062515/56649f555503460f94c78fe2/html5/thumbnails/20.jpg)
Segmentation for phase changeA time tag is regarded as a switching point
if its moving average error greater than a threshold value
2))];[(][(][ θxt
ti
tFtyterror
![Page 21: Jiann-Ming Wu, Ya-Ting Zhou, Chun-Chang Wu National Dong Hwa University Department of Applied Mathematics Hualien, Taiwan Learning Markov-chain embedded.](https://reader035.fdocuments.us/reader035/viewer/2022062515/56649f555503460f94c78fe2/html5/thumbnails/21.jpg)
A simple rule for merging two PGMsThe goodness of fitting the ith PGM to
paired data in Sj is defined by
Two PGMs are merged. Si and Sj are regarded from the same PGM if (Ei,j+Ej,i)/2 is less than a threshold value
2
][, ))];[(][(
1E j
Stiji
j
tFtyS
θxx
state.hidden same theform oriented regarded are and
2
1 If ,,
ji
ijji
SS
EE
![Page 22: Jiann-Ming Wu, Ya-Ting Zhou, Chun-Chang Wu National Dong Hwa University Department of Applied Mathematics Hualien, Taiwan Learning Markov-chain embedded.](https://reader035.fdocuments.us/reader035/viewer/2022062515/56649f555503460f94c78fe2/html5/thumbnails/22.jpg)
NUMERICAL SIMULATIONS – Synthetic data
on.segmentatilength -fixedfor sampling regressive-auto of size window the:
thresholdpositive determined-pre a :
scale. short time :
reduction after stateshidden required ofnumber the:
0N
K
] 200 0.1, 5, 3, [),,,(
exp,cos,sin : functionsnonlinear -post
0
321
NK
xxx TTT
aaa
matrix nTranslatio
![Page 23: Jiann-Ming Wu, Ya-Ting Zhou, Chun-Chang Wu National Dong Hwa University Department of Applied Mathematics Hualien, Taiwan Learning Markov-chain embedded.](https://reader035.fdocuments.us/reader035/viewer/2022062515/56649f555503460f94c78fe2/html5/thumbnails/23.jpg)
Temporal sequence generated by MC-embedded PGMs
![Page 24: Jiann-Ming Wu, Ya-Ting Zhou, Chun-Chang Wu National Dong Hwa University Department of Applied Mathematics Hualien, Taiwan Learning Markov-chain embedded.](https://reader035.fdocuments.us/reader035/viewer/2022062515/56649f555503460f94c78fe2/html5/thumbnails/24.jpg)
Numerical results – original and reconstructed MC-embedded PGMs
matrix nTranslatio
![Page 25: Jiann-Ming Wu, Ya-Ting Zhou, Chun-Chang Wu National Dong Hwa University Department of Applied Mathematics Hualien, Taiwan Learning Markov-chain embedded.](https://reader035.fdocuments.us/reader035/viewer/2022062515/56649f555503460f94c78fe2/html5/thumbnails/25.jpg)
Chaotic time series
Markov chain embedded recurrence relations
Generated chaotic time series
Laser data 10000 from the SFI competition
M=60,[K, , , N0 ] = [ 10, 10, 0.001, 500 ]
Learning
![Page 26: Jiann-Ming Wu, Ya-Ting Zhou, Chun-Chang Wu National Dong Hwa University Department of Applied Mathematics Hualien, Taiwan Learning Markov-chain embedded.](https://reader035.fdocuments.us/reader035/viewer/2022062515/56649f555503460f94c78fe2/html5/thumbnails/26.jpg)
Conclusions This work has presented learning Markov-chain
embedded recurrence relations for complex time series analysis.
Levenberg-Marquardt supervised learning of neural networks has been shown potential for extracting essential recurrence relation underlying given time series
Markov-chain embedded recurrence relations are shown applicable for characterizing complex chaotic time series
The proposed systematic approach integrates pattern segmentation, hidden state absorption and transition probability estimation based on supervised learning of neural networks