Time-Varying Mixed Graphical Modelsjmbh.github.io/figs/About/TVG_CCS2016.pdfTime-Varying Mixed...
Transcript of Time-Varying Mixed Graphical Modelsjmbh.github.io/figs/About/TVG_CCS2016.pdfTime-Varying Mixed...
![Page 1: Time-Varying Mixed Graphical Modelsjmbh.github.io/figs/About/TVG_CCS2016.pdfTime-Varying Mixed Graphical Models Jonas Haslbeck and Lourens Waldorp Psychological Methods University](https://reader035.fdocuments.us/reader035/viewer/2022070906/5f76a010686bf9687a3c5dbf/html5/thumbnails/1.jpg)
Time-Varying Mixed GraphicalModels
Jonas Haslbeck and Lourens Waldorp
Psychological MethodsUniversity of Amsterdam, the Netherlands
Conference on Complex Systems 2016
![Page 2: Time-Varying Mixed Graphical Modelsjmbh.github.io/figs/About/TVG_CCS2016.pdfTime-Varying Mixed Graphical Models Jonas Haslbeck and Lourens Waldorp Psychological Methods University](https://reader035.fdocuments.us/reader035/viewer/2022070906/5f76a010686bf9687a3c5dbf/html5/thumbnails/2.jpg)
Goal: Approximate Complex Personalized System
lalala
![Page 3: Time-Varying Mixed Graphical Modelsjmbh.github.io/figs/About/TVG_CCS2016.pdfTime-Varying Mixed Graphical Models Jonas Haslbeck and Lourens Waldorp Psychological Methods University](https://reader035.fdocuments.us/reader035/viewer/2022070906/5f76a010686bf9687a3c5dbf/html5/thumbnails/3.jpg)
Goal: Approximate Complex Personalized System
lalala
![Page 4: Time-Varying Mixed Graphical Modelsjmbh.github.io/figs/About/TVG_CCS2016.pdfTime-Varying Mixed Graphical Models Jonas Haslbeck and Lourens Waldorp Psychological Methods University](https://reader035.fdocuments.us/reader035/viewer/2022070906/5f76a010686bf9687a3c5dbf/html5/thumbnails/4.jpg)
Goal: Approximate Complex Personalized System
... using time-varying Mixed Graphical Models
![Page 5: Time-Varying Mixed Graphical Modelsjmbh.github.io/figs/About/TVG_CCS2016.pdfTime-Varying Mixed Graphical Models Jonas Haslbeck and Lourens Waldorp Psychological Methods University](https://reader035.fdocuments.us/reader035/viewer/2022070906/5f76a010686bf9687a3c5dbf/html5/thumbnails/5.jpg)
What are Graphical Models?
XA ⊥⊥ XB |XC
XA 6⊥⊥ XC |XB ⇐⇒
XC 6⊥⊥ XB |XAA
B
C
![Page 6: Time-Varying Mixed Graphical Modelsjmbh.github.io/figs/About/TVG_CCS2016.pdfTime-Varying Mixed Graphical Models Jonas Haslbeck and Lourens Waldorp Psychological Methods University](https://reader035.fdocuments.us/reader035/viewer/2022070906/5f76a010686bf9687a3c5dbf/html5/thumbnails/6.jpg)
Example: Gaussian Graphical Model
Σ−1 =
X1 X2 X3 X4
X1 3.45 0 0 3.18X2 0 2.14 0 0.82X3 0 0 3.21 1.05X4 3.18 0.82 1.05 8.77
⇐⇒1
2
3
4
P(X1, . . . ,Xp) =1√
(2π)p|Σ|exp
{−1
2(x − µ)>Σ−1(x − µ)
}
![Page 7: Time-Varying Mixed Graphical Modelsjmbh.github.io/figs/About/TVG_CCS2016.pdfTime-Varying Mixed Graphical Models Jonas Haslbeck and Lourens Waldorp Psychological Methods University](https://reader035.fdocuments.us/reader035/viewer/2022070906/5f76a010686bf9687a3c5dbf/html5/thumbnails/7.jpg)
Time-invariant Graphical Model
![Page 8: Time-Varying Mixed Graphical Modelsjmbh.github.io/figs/About/TVG_CCS2016.pdfTime-Varying Mixed Graphical Models Jonas Haslbeck and Lourens Waldorp Psychological Methods University](https://reader035.fdocuments.us/reader035/viewer/2022070906/5f76a010686bf9687a3c5dbf/html5/thumbnails/8.jpg)
Parameters may change over time!
Sleep
Energy
Time
Par
amet
er V
alue
Time VaryingStationary
![Page 9: Time-Varying Mixed Graphical Modelsjmbh.github.io/figs/About/TVG_CCS2016.pdfTime-Varying Mixed Graphical Models Jonas Haslbeck and Lourens Waldorp Psychological Methods University](https://reader035.fdocuments.us/reader035/viewer/2022070906/5f76a010686bf9687a3c5dbf/html5/thumbnails/9.jpg)
Parameters may change over time!
Sleep
Energy
Time
Par
amet
er V
alue
Time VaryingStationary
Stress
![Page 10: Time-Varying Mixed Graphical Modelsjmbh.github.io/figs/About/TVG_CCS2016.pdfTime-Varying Mixed Graphical Models Jonas Haslbeck and Lourens Waldorp Psychological Methods University](https://reader035.fdocuments.us/reader035/viewer/2022070906/5f76a010686bf9687a3c5dbf/html5/thumbnails/10.jpg)
Parameters may change over time!
Sleep
Energy
Time
Par
amet
er V
alue
Time VaryingStationary
StressAdverse Life Event
![Page 11: Time-Varying Mixed Graphical Modelsjmbh.github.io/figs/About/TVG_CCS2016.pdfTime-Varying Mixed Graphical Models Jonas Haslbeck and Lourens Waldorp Psychological Methods University](https://reader035.fdocuments.us/reader035/viewer/2022070906/5f76a010686bf9687a3c5dbf/html5/thumbnails/11.jpg)
Introducing time-varying Mixed Graphical Models
1) Stationary Mixed Graphical Models
2) Extension to the time-varying case
![Page 12: Time-Varying Mixed Graphical Modelsjmbh.github.io/figs/About/TVG_CCS2016.pdfTime-Varying Mixed Graphical Models Jonas Haslbeck and Lourens Waldorp Psychological Methods University](https://reader035.fdocuments.us/reader035/viewer/2022070906/5f76a010686bf9687a3c5dbf/html5/thumbnails/12.jpg)
Mixed Exponential Graphical Model
![Page 13: Time-Varying Mixed Graphical Modelsjmbh.github.io/figs/About/TVG_CCS2016.pdfTime-Varying Mixed Graphical Models Jonas Haslbeck and Lourens Waldorp Psychological Methods University](https://reader035.fdocuments.us/reader035/viewer/2022070906/5f76a010686bf9687a3c5dbf/html5/thumbnails/13.jpg)
Construction of Mixed Exponential Graphical Model
Conditional univariate members of the exponential family
P(Xs |X\s) = exp{Es(X\s)φs(Xs) + Cs(Xs)− Φ(X\s)
},
factorize to a global multivariate distribution which factorsaccording the graph defined by the node-neighborhoods if and onlyif Es(X\s) has the form:
θs +∑
t∈N(s)
θstφt(Xt) + ...+∑
t2,...,tk∈N(s)
θt2,...,tk
k∏j=2
φtj (Xtj ),
(Yang et al., 2014)
![Page 14: Time-Varying Mixed Graphical Modelsjmbh.github.io/figs/About/TVG_CCS2016.pdfTime-Varying Mixed Graphical Models Jonas Haslbeck and Lourens Waldorp Psychological Methods University](https://reader035.fdocuments.us/reader035/viewer/2022070906/5f76a010686bf9687a3c5dbf/html5/thumbnails/14.jpg)
Neighborhood Regression Method
1
2
3
1
2
3
1
2
3
(Meinshausen & Buehlmann, 2006)
![Page 15: Time-Varying Mixed Graphical Modelsjmbh.github.io/figs/About/TVG_CCS2016.pdfTime-Varying Mixed Graphical Models Jonas Haslbeck and Lourens Waldorp Psychological Methods University](https://reader035.fdocuments.us/reader035/viewer/2022070906/5f76a010686bf9687a3c5dbf/html5/thumbnails/15.jpg)
Estimation Algorithm
For each node s :
1. Regress X\s on Xs
I min(θ0,θ)∈Rp
[1N
∑Ni=1(yi − θ0 − XT
\s;iθ)2+ λn||θ||1]
I Select λn using EBIC
2. Threshold Parameter Estimates
I τn �√d ||θ||2
√log pn
Combine Estimates from both regressions
I AND-rule: Edge present if both parameters are nonzero
I OR-rule: Edge present if at least one parameter is nonzero
(Loh & Wainwright, 2013)
![Page 16: Time-Varying Mixed Graphical Modelsjmbh.github.io/figs/About/TVG_CCS2016.pdfTime-Varying Mixed Graphical Models Jonas Haslbeck and Lourens Waldorp Psychological Methods University](https://reader035.fdocuments.us/reader035/viewer/2022070906/5f76a010686bf9687a3c5dbf/html5/thumbnails/16.jpg)
Recab: Time-invariant mixed Graphical Model
![Page 17: Time-Varying Mixed Graphical Modelsjmbh.github.io/figs/About/TVG_CCS2016.pdfTime-Varying Mixed Graphical Models Jonas Haslbeck and Lourens Waldorp Psychological Methods University](https://reader035.fdocuments.us/reader035/viewer/2022070906/5f76a010686bf9687a3c5dbf/html5/thumbnails/17.jpg)
Time-varying mixed Graphical Model
But: we have the scaling τn �√d ||θ||2
√log pn
![Page 18: Time-Varying Mixed Graphical Modelsjmbh.github.io/figs/About/TVG_CCS2016.pdfTime-Varying Mixed Graphical Models Jonas Haslbeck and Lourens Waldorp Psychological Methods University](https://reader035.fdocuments.us/reader035/viewer/2022070906/5f76a010686bf9687a3c5dbf/html5/thumbnails/18.jpg)
Time-varying mixed Graphical Model
But: we have the scaling τn �√d ||θ||2
√log pn
![Page 19: Time-Varying Mixed Graphical Modelsjmbh.github.io/figs/About/TVG_CCS2016.pdfTime-Varying Mixed Graphical Models Jonas Haslbeck and Lourens Waldorp Psychological Methods University](https://reader035.fdocuments.us/reader035/viewer/2022070906/5f76a010686bf9687a3c5dbf/html5/thumbnails/19.jpg)
Local Stationarity
Sleep
Energy
Time = 1Time
Par
amet
er V
alue
Assumption: Edge parameters are a smooth function of time
![Page 20: Time-Varying Mixed Graphical Modelsjmbh.github.io/figs/About/TVG_CCS2016.pdfTime-Varying Mixed Graphical Models Jonas Haslbeck and Lourens Waldorp Psychological Methods University](https://reader035.fdocuments.us/reader035/viewer/2022070906/5f76a010686bf9687a3c5dbf/html5/thumbnails/20.jpg)
Local Stationarity
Assumption: Edge parameters are a smooth function of time
![Page 21: Time-Varying Mixed Graphical Modelsjmbh.github.io/figs/About/TVG_CCS2016.pdfTime-Varying Mixed Graphical Models Jonas Haslbeck and Lourens Waldorp Psychological Methods University](https://reader035.fdocuments.us/reader035/viewer/2022070906/5f76a010686bf9687a3c5dbf/html5/thumbnails/21.jpg)
Local Stationarity Violated!
Sleep
Energy
Time = 1Time
Par
amet
er V
alue
Edge parameters are not a smooth function of time!
![Page 22: Time-Varying Mixed Graphical Modelsjmbh.github.io/figs/About/TVG_CCS2016.pdfTime-Varying Mixed Graphical Models Jonas Haslbeck and Lourens Waldorp Psychological Methods University](https://reader035.fdocuments.us/reader035/viewer/2022070906/5f76a010686bf9687a3c5dbf/html5/thumbnails/22.jpg)
Local Stationarity Violated!
Edge parameters are not a smooth function of time!
![Page 23: Time-Varying Mixed Graphical Modelsjmbh.github.io/figs/About/TVG_CCS2016.pdfTime-Varying Mixed Graphical Models Jonas Haslbeck and Lourens Waldorp Psychological Methods University](https://reader035.fdocuments.us/reader035/viewer/2022070906/5f76a010686bf9687a3c5dbf/html5/thumbnails/23.jpg)
Again: Local Stationarity
t
Time
Par
amet
er V
alue
![Page 24: Time-Varying Mixed Graphical Modelsjmbh.github.io/figs/About/TVG_CCS2016.pdfTime-Varying Mixed Graphical Models Jonas Haslbeck and Lourens Waldorp Psychological Methods University](https://reader035.fdocuments.us/reader035/viewer/2022070906/5f76a010686bf9687a3c5dbf/html5/thumbnails/24.jpg)
Time-varying Graphs via Weighted Regression
Time Steps (Observations)
Wei
ght(
t)
t
Weighted cost function:
min(θ0,θ)∈Rp
[1N
∑Ni=1 wi ;t(yi ;t − θ0;t − XT
\s;iθt)2 + λn||θt ||1
]
![Page 25: Time-Varying Mixed Graphical Modelsjmbh.github.io/figs/About/TVG_CCS2016.pdfTime-Varying Mixed Graphical Models Jonas Haslbeck and Lourens Waldorp Psychological Methods University](https://reader035.fdocuments.us/reader035/viewer/2022070906/5f76a010686bf9687a3c5dbf/html5/thumbnails/25.jpg)
What is the right bandwidth?
High Bandwidth
Time
Par
amet
er V
alue
More Information for estimation
Low Bandwidth
Time
Par
amet
er V
alue
Higher sensitivity to change
Scaling: τn �√d ||θ||2
√log pn
![Page 26: Time-Varying Mixed Graphical Modelsjmbh.github.io/figs/About/TVG_CCS2016.pdfTime-Varying Mixed Graphical Models Jonas Haslbeck and Lourens Waldorp Psychological Methods University](https://reader035.fdocuments.us/reader035/viewer/2022070906/5f76a010686bf9687a3c5dbf/html5/thumbnails/26.jpg)
Small Simulation: Typical ESM Data
00.8
0.8
0.8
1
2
3
4
5
6
7
8
9
10
Time = 1
−0.3
0.0
0.3
0.6
0.9
−0.3
0.0
0.3
0.6
0.9
−0.3
0.0
0.3
0.6
0.9
−0.3
0.0
0.3
0.6
0.9
−0.3
0.0
0.3
0.6
0.9
50 100 150 210
10 measurements/day × 3 weeks = 210
![Page 27: Time-Varying Mixed Graphical Modelsjmbh.github.io/figs/About/TVG_CCS2016.pdfTime-Varying Mixed Graphical Models Jonas Haslbeck and Lourens Waldorp Psychological Methods University](https://reader035.fdocuments.us/reader035/viewer/2022070906/5f76a010686bf9687a3c5dbf/html5/thumbnails/27.jpg)
Small Simulation: Typical ESM Data
10 measurements/day × 3 weeks = 210
![Page 28: Time-Varying Mixed Graphical Modelsjmbh.github.io/figs/About/TVG_CCS2016.pdfTime-Varying Mixed Graphical Models Jonas Haslbeck and Lourens Waldorp Psychological Methods University](https://reader035.fdocuments.us/reader035/viewer/2022070906/5f76a010686bf9687a3c5dbf/html5/thumbnails/28.jpg)
Typical ESM Data: Select Bandwidth
Bandwidth = 0.3
10 50 100 150 210
0.00
0.25
0.50
0.75
1.00
Weight
Bandwidth = 0.15
10 50 100 150 210
0.00
0.25
0.50
0.75
1.00
WeightBandwidth = 0.05
10 50 100 150 210
0.00
0.25
0.50
0.75
1.00
Bandwidth = 0.01
10 50 100 150 210
0.00
0.25
0.50
0.75
1.00
![Page 29: Time-Varying Mixed Graphical Modelsjmbh.github.io/figs/About/TVG_CCS2016.pdfTime-Varying Mixed Graphical Models Jonas Haslbeck and Lourens Waldorp Psychological Methods University](https://reader035.fdocuments.us/reader035/viewer/2022070906/5f76a010686bf9687a3c5dbf/html5/thumbnails/29.jpg)
Small Simulation: Results
●●
●●●
●● ●
●
●
●
●●
●
●
●●
●
●●●
●
●
10 50 100 150 210
0.00
0.25
0.50
0.75
1.00
Edg
e w
eigh
t●
●
●
●
●●
●
● ●
●
●●
●
●●
●
●
●
●●●●●
●
●
●●●●●●●●●●●●●● ●●●●
●
●
●
●
●●●●
●
●
●
●
●
●
●
10 50 100 150 210
0.00
0.25
0.50
0.75
1.00
Edg
e w
eigh
t
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●●●●●●
●
●●●
●
●
●
●
●
●
●
●
●●
●
●●●●●
●●●● ●
●●●●
●
●●●●●
●
●
●
●
●●
●●
10 50 100 150 210
0.00
0.25
0.50
0.75
1.00
Edg
e w
eigh
t
●
●●
●
●●
●
●
●●●
●●
●
●
●
●
●●
●
●●
●●
●●●●●● ●●●
●●●
●●
●
●●
●
●
●
●
●
●
●●●
●
●●
10 50 100 150 210
0.00
0.25
0.50
0.75
1.00
Edg
e w
eigh
t
●●
●●●●
●
●●●●●●●●●●●●●●
●
●●●
●
●●
●
●
●●
●
●●
●
●
●
●●●
●
●
●
●
●
●●●
●
●
10 50 100 150 210
0.00
0.25
0.50
0.75
1.00
Edg
e w
eigh
t
![Page 30: Time-Varying Mixed Graphical Modelsjmbh.github.io/figs/About/TVG_CCS2016.pdfTime-Varying Mixed Graphical Models Jonas Haslbeck and Lourens Waldorp Psychological Methods University](https://reader035.fdocuments.us/reader035/viewer/2022070906/5f76a010686bf9687a3c5dbf/html5/thumbnails/30.jpg)
Small Simulation: Results
●●
●●●
●● ●
●
●
●
●●
●
●
●●
●
●●●
●
●
10 50 100 150 210
0.00
0.25
0.50
0.75
1.00
Edg
e w
eigh
t●
●
●
●
●●
●
● ●
●
●●
●
●●
●
●
●
●●●●●
●
●
●●●●●●●●●●●●●● ●●●●
●
●
●
●
●●●●
●
●
●
●
●
●
●
10 50 100 150 210
0.00
0.25
0.50
0.75
1.00
Edg
e w
eigh
t
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●●●●●●
●
●●●
●
●
●
●
●
●
●
●
●●
●
●●●●●
●●●● ●
●●●●
●
●●●●●
●
●
●
●
●●
●●
10 50 100 150 210
0.00
0.25
0.50
0.75
1.00
Edg
e w
eigh
t
●
●●
●
●●
●
●
●●●
●●
●
●
●
●
●●
●
●●
●●
●●●●●● ●●●
●●●
●●
●
●●
●
●
●
●
●
●
●●●
●
●●
10 50 100 150 210
0.00
0.25
0.50
0.75
1.00
Edg
e w
eigh
t
●●
●●●●
●
●●●●●●●●●●●●●●
●
●●●
●
●●
●
●
●●
●
●●
●
●
●
●●●
●
●
●
●
●
●●●
●
●
10 50 100 150 210
0.00
0.25
0.50
0.75
1.00
Edg
e w
eigh
t
10 50 100 150 210
0.00
0.25
0.50
0.75
1.00S
peci
ficity
![Page 31: Time-Varying Mixed Graphical Modelsjmbh.github.io/figs/About/TVG_CCS2016.pdfTime-Varying Mixed Graphical Models Jonas Haslbeck and Lourens Waldorp Psychological Methods University](https://reader035.fdocuments.us/reader035/viewer/2022070906/5f76a010686bf9687a3c5dbf/html5/thumbnails/31.jpg)
Application to Event Sampling Data
I Time series of 1 person
I 43 variables (continuous &categorical)
I Up to 10 measurements a day
I For 36 weeks
![Page 32: Time-Varying Mixed Graphical Modelsjmbh.github.io/figs/About/TVG_CCS2016.pdfTime-Varying Mixed Graphical Models Jonas Haslbeck and Lourens Waldorp Psychological Methods University](https://reader035.fdocuments.us/reader035/viewer/2022070906/5f76a010686bf9687a3c5dbf/html5/thumbnails/32.jpg)
Time-varying Graph of Psychopathology
![Page 33: Time-Varying Mixed Graphical Modelsjmbh.github.io/figs/About/TVG_CCS2016.pdfTime-Varying Mixed Graphical Models Jonas Haslbeck and Lourens Waldorp Psychological Methods University](https://reader035.fdocuments.us/reader035/viewer/2022070906/5f76a010686bf9687a3c5dbf/html5/thumbnails/33.jpg)
Time-varying Mixed Graphical Models
Summary
I Time varying model under assumption of local stationarity
I Allows for mixed variables (e.g. categorical and continuous)
I Scales well for large p and allows for p > n
I Works in realistic situations
I Also a VAR version implemented
I Available in ’mgm’ R-package on CRAN
Contact
I Email: [email protected]
I Website: http://jmbh.github.io
![Page 34: Time-Varying Mixed Graphical Modelsjmbh.github.io/figs/About/TVG_CCS2016.pdfTime-Varying Mixed Graphical Models Jonas Haslbeck and Lourens Waldorp Psychological Methods University](https://reader035.fdocuments.us/reader035/viewer/2022070906/5f76a010686bf9687a3c5dbf/html5/thumbnails/34.jpg)
![Page 35: Time-Varying Mixed Graphical Modelsjmbh.github.io/figs/About/TVG_CCS2016.pdfTime-Varying Mixed Graphical Models Jonas Haslbeck and Lourens Waldorp Psychological Methods University](https://reader035.fdocuments.us/reader035/viewer/2022070906/5f76a010686bf9687a3c5dbf/html5/thumbnails/35.jpg)
Small Simulation: Same as above N=210
●●
●●●
●● ●
●
●
●
●●
●
●
●●
●
●●●
●
●
10 50 100 150 210
0.00
0.25
0.50
0.75
1.00
Edg
e w
eigh
t●
●
●
●
●●
●
● ●
●
●●
●
●●
●
●
●
●●●●●
●
●
●●●●●●●●●●●●●● ●●●●
●
●
●
●
●●●●
●
●
●
●
●
●
●
10 50 100 150 210
0.00
0.25
0.50
0.75
1.00
Edg
e w
eigh
t
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●●●●●●
●
●●●
●
●
●
●
●
●
●
●
●●
●
●●●●●
●●●● ●
●●●●
●
●●●●●
●
●
●
●
●●
●●
10 50 100 150 210
0.00
0.25
0.50
0.75
1.00
Edg
e w
eigh
t
●
●●
●
●●
●
●
●●●
●●
●
●
●
●
●●
●
●●
●●
●●●●●● ●●●
●●●
●●
●
●●
●
●
●
●
●
●
●●●
●
●●
10 50 100 150 210
0.00
0.25
0.50
0.75
1.00
Edg
e w
eigh
t
●●
●●●●
●
●●●●●●●●●●●●●●
●
●●●
●
●●
●
●
●●
●
●●
●
●
●
●●●
●
●
●
●
●
●●●
●
●
10 50 100 150 210
0.00
0.25
0.50
0.75
1.00
Edg
e w
eigh
t
10 50 100 150 210
0.00
0.25
0.50
0.75
1.00S
peci
ficity
![Page 36: Time-Varying Mixed Graphical Modelsjmbh.github.io/figs/About/TVG_CCS2016.pdfTime-Varying Mixed Graphical Models Jonas Haslbeck and Lourens Waldorp Psychological Methods University](https://reader035.fdocuments.us/reader035/viewer/2022070906/5f76a010686bf9687a3c5dbf/html5/thumbnails/36.jpg)
Small Simulation: Same as above but now N=100
●●●●●●●●●●●●●● ●●●●●●●●● ●●● ●●● ● ●● ●
●
● ●●
●●
●
●
●
●
●● ●● ●●● ●●●●●● ●●●●●●●● ●●●●●●●●●●●●●●●
10 50 100 150 210
0.00
0.25
0.50
0.75
1.00
Edg
e w
eigh
t
● ●
●
●
●
●●
●
●
●
●●
●●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●●●●●●●●● ●●●●●● ●●● ● ● ●● ●● ●●●●●
10 50 100 150 210
0.00
0.25
0.50
0.75
1.00
Edg
e w
eigh
t
●●●●●●●●● ●●●●●● ●●●●●● ●●● ●●● ●●●● ●●●● ●●●●●●● ●●●●●●●●●●●●●●
● ●
●●
●
●
●
●
●●
●
●
●
●
●●
●●
●
●
●
●
●
●●●●
●
●
●
●
●
●
●
●●
●
●
10 50 100 150 210
0.00
0.25
0.50
0.75
1.00
Edg
e w
eigh
t
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●●●
●
●
●
●
●
●
●●
●
●
●
●
●●●●●●●●●●●●●●●●●● ●●●●●●●●●●●●●
●
●●●●●●●●● ●●●●●●●●● ●●●●●●●●●●●●●
●
●●●●●●●●●●●●●● ●●●●●●●●●●●●●●●●●●
10 50 100 150 210
0.00
0.25
0.50
0.75
1.00
Edg
e w
eigh
t
●●●●●●●●●●● ●●●● ●●● ●●●● ●●●● ●●●● ●●●●● ●●●●●●●●●●●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
10 50 100 150 210
0.00
0.25
0.50
0.75
1.00
Edg
e w
eigh
t
10 50 100 150 210
0.00
0.25
0.50
0.75
1.00S
peci
ficity
![Page 37: Time-Varying Mixed Graphical Modelsjmbh.github.io/figs/About/TVG_CCS2016.pdfTime-Varying Mixed Graphical Models Jonas Haslbeck and Lourens Waldorp Psychological Methods University](https://reader035.fdocuments.us/reader035/viewer/2022070906/5f76a010686bf9687a3c5dbf/html5/thumbnails/37.jpg)
Small Simulation: Same as above but now N=50
●●●●●●●●●●●●●●●●●●●●●●●
10 50 100 150 210
0.00
0.25
0.50
0.75
1.00
Edg
e w
eigh
t
●
● ●
●
●
●
●
●
●●●
●●
●
●
●
●
● ●
●
●
●
●●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
10 50 100 150 210
0.00
0.25
0.50
0.75
1.00
Edg
e w
eigh
t
●
●●
●●
●
●
●●
●
●
●●
●
●●
●●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
10 50 100 150 210
0.00
0.25
0.50
0.75
1.00
Edg
e w
eigh
t
●●●
●
●
●
●
●
●●
●●●
●●
●
●
●
●
●
●●
●●
●
●●
●
●
●
●●
●
●
●●
●
●
●
●
10 50 100 150 210
0.00
0.25
0.50
0.75
1.00
Edg
e w
eigh
t
●●●●●●●●●●●●●●●●●●●●●●● ●●●●●●●●●●●●●●●●●● ●●●●●●●●●●●●●●●●●●●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●●
●●
●●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
10 50 100 150 210
0.00
0.25
0.50
0.75
1.00
Edg
e w
eigh
t
10 50 100 150 210
0.00
0.25
0.50
0.75
1.00S
peci
ficity
![Page 38: Time-Varying Mixed Graphical Modelsjmbh.github.io/figs/About/TVG_CCS2016.pdfTime-Varying Mixed Graphical Models Jonas Haslbeck and Lourens Waldorp Psychological Methods University](https://reader035.fdocuments.us/reader035/viewer/2022070906/5f76a010686bf9687a3c5dbf/html5/thumbnails/38.jpg)
Larger Simulation: Setup
I Binary-Gaussian Graphical Model
I 20 Nodes
I Always 19 edges present
I Of these are always 6 changing
I Type of change: smooth vs.sudden
![Page 39: Time-Varying Mixed Graphical Modelsjmbh.github.io/figs/About/TVG_CCS2016.pdfTime-Varying Mixed Graphical Models Jonas Haslbeck and Lourens Waldorp Psychological Methods University](https://reader035.fdocuments.us/reader035/viewer/2022070906/5f76a010686bf9687a3c5dbf/html5/thumbnails/39.jpg)
Simulation: Results
Continuous change Sudden change
Specificity Sensitivity Specificity Sensitivity
N = 600Nt = 205.9Nt/p = 10.3
●
●
●
GaussianBinaryBinary−Gaussian
0
0.25
0.5
0.75
1
1 200 400 600
0
0.25
0.5
0.75
1
1 200 400 600
Time steps Time steps
bandwidth = 0.8/N1/3 ≈ 0.095
![Page 40: Time-Varying Mixed Graphical Modelsjmbh.github.io/figs/About/TVG_CCS2016.pdfTime-Varying Mixed Graphical Models Jonas Haslbeck and Lourens Waldorp Psychological Methods University](https://reader035.fdocuments.us/reader035/viewer/2022070906/5f76a010686bf9687a3c5dbf/html5/thumbnails/40.jpg)
Mixed Graphical Model: Conditional Distribution
Conditional univariate members of the exponential family
P(Xs |X\s) = exp{Es(X\s)φs(Xs) + Cs(Xs)− Φ(X\s)
},
factorize to a global multivariate distribution which factorsaccording the graph defined by the node-neighborhoods if and onlyif Es(X\s) has the form:
θs +∑
t∈N(s)
θstφt(Xt) + ...+∑
t2,...,tk∈N(s)
θt2,...,tk
k∏j=2
φtj (Xtj ),
![Page 41: Time-Varying Mixed Graphical Modelsjmbh.github.io/figs/About/TVG_CCS2016.pdfTime-Varying Mixed Graphical Models Jonas Haslbeck and Lourens Waldorp Psychological Methods University](https://reader035.fdocuments.us/reader035/viewer/2022070906/5f76a010686bf9687a3c5dbf/html5/thumbnails/41.jpg)
Mixed Graphical Model: Joint Distribution
The joint distribution has the form
P(X ; θ) = exp{∑s∈V
θsφs(Xs) +∑s∈V
∑t∈N(s)
θstφs(Xs)φt(Xt)+
· · ·+∑
t1,...,tk∈Cθt1,...,tk
k∏j=1
φtj (Xtj ) +∑s∈V
Cs(Xs)− Φ(θ)}
![Page 42: Time-Varying Mixed Graphical Modelsjmbh.github.io/figs/About/TVG_CCS2016.pdfTime-Varying Mixed Graphical Models Jonas Haslbeck and Lourens Waldorp Psychological Methods University](https://reader035.fdocuments.us/reader035/viewer/2022070906/5f76a010686bf9687a3c5dbf/html5/thumbnails/42.jpg)
Example Mixed Graphical Model: Ising-Gaussian
P(Y ,Z ) ∝ exp{ ∑
s∈VY
θysσs
Ys +∑r∈VZ
θzr Zr +∑
(s,t)∈EY
θyystσsσt
YsYt+
∑(r ,q)∈EZ
θzzrqZrZq +∑
(s,r)∈EYZ
θyzsrσs
YsZr −∑s∈VY
Y 2s
2σ2s
}If Xs Bernoulli, the node-conditional has the form:
P(Xs |X\s) ∝ exp{θzr Zr +
∑q∈N(r)Z
θzzrqZrZq +∑
t∈N(r)Y
θyzrtσt
ZrYt
}If Xs Gaussian, the node-conditional has the form:
P(Xs |X\s) ∝ exp{θysσs
Ys +∑
t∈N(s)Y
θyystσsσt
YsYt +∑
r∈N(s)Z
θyzsrσs
YsZr −Y 2s
2σ2s
}