Inferring structure to make substantive conclusions: How does it work?
description
Transcript of Inferring structure to make substantive conclusions: How does it work?
1
Inferring structure to make substantive conclusions:How does it work?
Hypothesis testing approaches:
Tests on deviances, possibly penalised (AIC/BIC, etc.), MDL, cross-validation...
Problem is how to search model space when dimension is large
2
Inferring structure to make substantive conclusions:How does it work? Bayesian approaches:
Typically place prior on all graphs, and conjugate prior on parameters (hyper-Markov laws, Dawid & Lauritzen), then use MCMC to update both graphs and parameters to simulate posterior distribution
3
Graph moves
Giudici & Green (Biometrika, 1999) develop a full Bayesian methodology for model selection in Gaussian models, assuming
decomposability
(= graph triangulated
= no chordless
-cycles)
7 6 5
2 3 414
4
• How many graphs are decomposable?
• Models using decomposable graphs are ‘dense’
Number ofvertices
Proportion of graphsthat aredecomposable
3 all 4 61/64 – all but: 6 ~80% 16 ~45%
Is decomposability a serious constraint?
22n
out of
5
Is decomposability any use?
• Maximum likelihood estimates can be computed exactly in decomposable models
• Decomposability is a key to the ‘message passing’ algorithms for probabilistic expert systems (and peeling genetic pedigrees)
lj
jkllijijkl n
nnNE
)(ˆ1 2
4 3
6
Graph moves
We can traverse graph space by adding and deleting single edges
Some are OK,but others makegraphnon-decomposable
7 6 5
2 3 41
7
Graph moves
Frydenberg & Lauritzen (1989) showed that all decomposable graphs are connected by single-edge moves
Can we test formaintaining decomposabilitybefore committing tomaking the change?
7 6 5
2 3 41
8
Cliques
A clique is a maximal complete subgraph: here the cliques are {1,2},{2,6,7}, {2,3,6}, and {3,4,5,6}
7 6 5
2 3 41
9
Deleting edges?
Deleting an edge maintains decomposability if and only if it is contained in exactly one clique of the current graph (Frydenberg & Lauritzen)
7 6 5
2 3 41
10
7 6 5
2 3 41
12
267 236 345626 36
2
a cliqueanother cliquea separator
The running intersection property:For any 2 cliques C and D, CD is a subset of every node between them in the junction tree
A graph is decomposableif and only if it can be represented by ajunction tree (which isnot unique)
11
7 6 5
2 3 41
12
267 236 345626 36
2
a cliqueanother cliquea separator
A graph is decomposableif and only if it can be represented by ajunction tree (which isnot unique)
The running intersection property:For any 2 cliques C and D, CD is a subset of every node between them in the junction tree
12
Non-uniquenessof junction tree
7 6 5
2 3 41
12
267 236 345626 36
2
13
7 6 5
2 3 41
12
267 236 345626 36
2
12
2
Non-uniquenessof junction tree
14
Adding edges? (Giudici & Green)
Adding an edge (a,b) maintains decomposability if and only if either:
7 6 5
2 3 41
• there exist sets R and T such that aR and bT are cliques and RT is a separator on the path in the junction tree between them
• a and b are in different connected components, or
15
You can add edge (1,7) since 1R and 7T are cliques (with R={2} and T={2,6}) and RT={2} is a separator on path between them
7 6 5
2 3 41
12
267 236 345626 36
2
16
You cannot add edge (1,4) since the only cliques containing 1 and 4 resp. are {1,2} and {3,4,5,6}, and {2}{3,5,6} is not a separator on path between them
7 6 5
2 3 41
12
267 236 345626 36
2
17
Adding edges? (Giudici & Green)
Adding an edge (a,b) maintains decomposability if and only if either:
7 6 5
2 3 41
• there exist sets R and T such that aR and bT are cliques and RT is a separator on the path in the junction tree between them
• a and b are in different connected components, or
18
Proof (in connected case)
First suppose that there are no such sets R and T. We have to show that adding edge (a,b) makes graph non-deomposable.Let aR and bT be the cliques containing a and b that have shortest connecting path in the junction tree: by assumption, RT is not a separator (it may be empty): so all separators on the path are proper supersets of RT. So there is a shortest path in the original graph: arv1...vktb with k0, rR\T, tT\R and all v’s RT. Joining (a,b) will make a chordless (k+4)-cycle, making the graph non-decomposable.
19
You cannot add edge (1,4) since the only cliques containing 1 and 4 resp. are {1,2} and {3,4,5,6}, and {2}{3,5,6} is not a separator on path between them
7 6 5
2 3 41
12
267 236 345626 36
2
20
Proof (in connected case)
S bSaS
abS
Conversely, suppose such sets R and T do exist.We can suppose aR and bT are adjacent in the junction tree (otherwise it is quite easy to show that the junction tree can be manipulated until this is true).Let S=RT, P=R\T and Q=T\R. There are 4 cases according to whether P and Q are empty or not.
Both P and Q empty:
(it is easy to see that you still have a tree & that running intersection property is maintained)
21
S bSaSP
abS bS bSQaSaSP
S bSQaS
S bSQaSP
aS abSaSP bS bSQabS
Neither P nor Q empty:
Only Q empty: Only P empty:
22
Once the test is complete, actually committing to adding or deleting the edge is little work
7 6 5
2 3 41
12
267 236 345626 36
2
23
7 6 5
2 3 41
127
267 236 345626 36
27
12
2
It makes onlya (relatively)local change to the junction tree
Once the test is complete, actually committing to adding or deleting the edge is little work
24
7 6 5
2 3 41
127
267 236 345626 36
27
It makes onlya (relatively)local change to the junction tree
6
Once the test is complete, actually committing to adding or deleting the edge is little work
26
Once the test is complete, actually committing to adding or deleting the edge is little work
7 6 5
2 3 4
It makes onlya (relatively)local change to the junction tree
127
267 236 35626 36
27
345
35
1
The End