Class4 - Donald Bren School of Information and Computer ... · Algorithms for Reasoning with...
Transcript of Class4 - Donald Bren School of Information and Computer ... · Algorithms for Reasoning with...
![Page 1: Class4 - Donald Bren School of Information and Computer ... · Algorithms for Reasoning with graphical models Class4 Rina Dechter class2 828X 2019 1](https://reader033.fdocuments.us/reader033/viewer/2022052005/60188487e0320b7b965dd8be/html5/thumbnails/1.jpg)
Algorithms for Reasoning with graphical models
Class4Rina Dechter
class2 828X 2019 1
![Page 2: Class4 - Donald Bren School of Information and Computer ... · Algorithms for Reasoning with graphical models Class4 Rina Dechter class2 828X 2019 1](https://reader033.fdocuments.us/reader033/viewer/2022052005/60188487e0320b7b965dd8be/html5/thumbnails/2.jpg)
The Induced-Width
Width along d, w(d): max # of previous parents
Induced width w*(d): The width in the ordered
induced graph Induced-width w*:
Smallest induced-width over all orderings
Finding w* NP-complete (Arnborg,
1985) but greedy heuristics (min-fill).
class2 828X 2019
E
D
A
C
B
2
![Page 3: Class4 - Donald Bren School of Information and Computer ... · Algorithms for Reasoning with graphical models Class4 Rina Dechter class2 828X 2019 1](https://reader033.fdocuments.us/reader033/viewer/2022052005/60188487e0320b7b965dd8be/html5/thumbnails/3.jpg)
Road Map Graphical models Constraint networks Model Inference
Variable elimination for Constraints Variable elimination for CNFs Greedy search for induced-width orderings Variable elimination for Linear Inequalities
Constraint propagation Search Probabilistic Networks
class2 828X 2019
![Page 4: Class4 - Donald Bren School of Information and Computer ... · Algorithms for Reasoning with graphical models Class4 Rina Dechter class2 828X 2019 1](https://reader033.fdocuments.us/reader033/viewer/2022052005/60188487e0320b7b965dd8be/html5/thumbnails/4.jpg)
Finding a Small Induced-Width NP-complete A tree has induced-width of ? Greedy algorithms:
Min width Min induced-width Max-cardinality and chordal graphs Fill-in (thought as the best)
Anytime algorithms Search-based [Gogate & Dechter 2003] Stochastic (CVO) [Kask, Gelfand & Dechter 2010]
4class2 828X 2019
![Page 5: Class4 - Donald Bren School of Information and Computer ... · Algorithms for Reasoning with graphical models Class4 Rina Dechter class2 828X 2019 1](https://reader033.fdocuments.us/reader033/viewer/2022052005/60188487e0320b7b965dd8be/html5/thumbnails/5.jpg)
Min-width Ordering
Proposition: algorithm min-width finds a min-width ordering of a graphWhat is the Complexity of MW? O(e) class2 828X 2019
![Page 6: Class4 - Donald Bren School of Information and Computer ... · Algorithms for Reasoning with graphical models Class4 Rina Dechter class2 828X 2019 1](https://reader033.fdocuments.us/reader033/viewer/2022052005/60188487e0320b7b965dd8be/html5/thumbnails/6.jpg)
Greedy Orderings Heuristics Min-induced-width
From last to first, pick a node with smallest width, then connect parent and remove
Min-Fill From last to first, pick a node with smallest
fill-edges
Complexity? O(𝑛 )
6class2 828X 2019
![Page 7: Class4 - Donald Bren School of Information and Computer ... · Algorithms for Reasoning with graphical models Class4 Rina Dechter class2 828X 2019 1](https://reader033.fdocuments.us/reader033/viewer/2022052005/60188487e0320b7b965dd8be/html5/thumbnails/7.jpg)
Min-Fill Heuristic Select the variable that creates the fewest “fill-in”
edges
A
E D
C
F
A
D
CB
F
A
E D
CB
F
Eliminate B next?Connect neighbors“Fill-in” = 3: (A,D), (C,E), (D,E)
Eliminate E next?Neighbors already connected“Fill-in” = 0
7class2 828X 2019
![Page 8: Class4 - Donald Bren School of Information and Computer ... · Algorithms for Reasoning with graphical models Class4 Rina Dechter class2 828X 2019 1](https://reader033.fdocuments.us/reader033/viewer/2022052005/60188487e0320b7b965dd8be/html5/thumbnails/8.jpg)
class2 828X 2019
Example
![Page 9: Class4 - Donald Bren School of Information and Computer ... · Algorithms for Reasoning with graphical models Class4 Rina Dechter class2 828X 2019 1](https://reader033.fdocuments.us/reader033/viewer/2022052005/60188487e0320b7b965dd8be/html5/thumbnails/9.jpg)
Different Induced-Graphs
A Min-fill orderingA Miw ordering
9class2 828X 2019
![Page 10: Class4 - Donald Bren School of Information and Computer ... · Algorithms for Reasoning with graphical models Class4 Rina Dechter class2 828X 2019 1](https://reader033.fdocuments.us/reader033/viewer/2022052005/60188487e0320b7b965dd8be/html5/thumbnails/10.jpg)
A graph is chordal if every cycle of length at least 4 has a chord
Deciding chordality by max-cardinality ordering: from 1 to n, always assigning a next node connected to a largest set of
previously selected nodes.
A graph along max-cardinality order has no fill-in edges iff it is chordal.
The maximal cliques of chordal graphs form a tree
Chordal Graphs
[Tarjan & Yanakakis 1980] 10class2 828X 2019
![Page 11: Class4 - Donald Bren School of Information and Computer ... · Algorithms for Reasoning with graphical models Class4 Rina Dechter class2 828X 2019 1](https://reader033.fdocuments.us/reader033/viewer/2022052005/60188487e0320b7b965dd8be/html5/thumbnails/11.jpg)
Greedy Orderings Heuristics Min-Induced-width
From last to first, pick a node with smallest width
Min-Fill From last to first, pick a node with smallest fill-
edges
• Max-Cardinality search From first to last, pick a node with largest neighbors
already ordered.
Complexity? O(𝑛 )
Complexity? O(𝑛 𝑚)
11class2 828X 2019
[Tarjan & Yanakakis 1980]
![Page 12: Class4 - Donald Bren School of Information and Computer ... · Algorithms for Reasoning with graphical models Class4 Rina Dechter class2 828X 2019 1](https://reader033.fdocuments.us/reader033/viewer/2022052005/60188487e0320b7b965dd8be/html5/thumbnails/12.jpg)
Max-cardinality ordering
![Page 13: Class4 - Donald Bren School of Information and Computer ... · Algorithms for Reasoning with graphical models Class4 Rina Dechter class2 828X 2019 1](https://reader033.fdocuments.us/reader033/viewer/2022052005/60188487e0320b7b965dd8be/html5/thumbnails/13.jpg)
class2 828X 2019
ExampleWe see again that G in the Figure (a) is not chordalsince the parents of A are not connected in the max-cardinality ordering in Figure (d). If we connect B and C, the resulting induced graph is chordal.
![Page 14: Class4 - Donald Bren School of Information and Computer ... · Algorithms for Reasoning with graphical models Class4 Rina Dechter class2 828X 2019 1](https://reader033.fdocuments.us/reader033/viewer/2022052005/60188487e0320b7b965dd8be/html5/thumbnails/14.jpg)
Which Greedy Algorithm is Best?
Min-Fill, prefers a node who add the least number of fill-in arcs.
Empirically, fill-in is the best among the greedy algorithms (MW,MIW,MF,MC)
Complexity of greedy orderings? MW is O(e), MIW: O( ) MF O( ) MC is
O(e+n)
class2 828X 2019
![Page 15: Class4 - Donald Bren School of Information and Computer ... · Algorithms for Reasoning with graphical models Class4 Rina Dechter class2 828X 2019 1](https://reader033.fdocuments.us/reader033/viewer/2022052005/60188487e0320b7b965dd8be/html5/thumbnails/15.jpg)
K-trees
class2 828X 2019 15
![Page 16: Class4 - Donald Bren School of Information and Computer ... · Algorithms for Reasoning with graphical models Class4 Rina Dechter class2 828X 2019 1](https://reader033.fdocuments.us/reader033/viewer/2022052005/60188487e0320b7b965dd8be/html5/thumbnails/16.jpg)
Finding a Small Induced-Width NP-complete A tree has induced-width of ? Greedy algorithms:
Min width (MW) Min induced-width (MIW) Max-cardinality and chordal graphs (MC) Min-Fill (thought as the best) (MIN-FILL)
Anytime algorithms Search-based [Gogate & Dechter 2003] Stochastic (CVO) [Kask, Gelfand & Dechter 2010]
16class2 828X 2019
![Page 17: Class4 - Donald Bren School of Information and Computer ... · Algorithms for Reasoning with graphical models Class4 Rina Dechter class2 828X 2019 1](https://reader033.fdocuments.us/reader033/viewer/2022052005/60188487e0320b7b965dd8be/html5/thumbnails/17.jpg)
Summary Of Inference Scheme Bucket elimination is time and memory
exponential in the induced-width.
Finding the w* is hard, but greedy schemes work quit well to approximate. Most popular is fill-edges
W(d) is the induced-width along an ordering d. Smallest induced-width is also called tree-width.
17class2 828X 2019
![Page 18: Class4 - Donald Bren School of Information and Computer ... · Algorithms for Reasoning with graphical models Class4 Rina Dechter class2 828X 2019 1](https://reader033.fdocuments.us/reader033/viewer/2022052005/60188487e0320b7b965dd8be/html5/thumbnails/18.jpg)
Recent work in my group Vibhav Gogate and Rina Dechter. "A Complete Anytime
Algorithm for Treewidth". In UAI 2004. Andrew E. Gelfand, Kalev Kask, and Rina Dechter.
"Stopping Rules for Randomized Greedy Triangulation Schemes" in Proceedings of AAAI 2011.
Kask, Gelfand and Dechter, BEEM: Bucket Elimination with External memory, AAAI 2011 or UAI 2011
Potential project
class2 828X 2019
![Page 19: Class4 - Donald Bren School of Information and Computer ... · Algorithms for Reasoning with graphical models Class4 Rina Dechter class2 828X 2019 1](https://reader033.fdocuments.us/reader033/viewer/2022052005/60188487e0320b7b965dd8be/html5/thumbnails/19.jpg)
class2 828X 2019
Greedy Algorithms for Induced-Width
Min-width ordering Min-induced-width ordering Max-cardinality ordering Min-fill ordering Chordal graphs Hypergraph partitionings
(Project: present papers on induced-width, run algorithms for induced-width on new benchmarks…)
![Page 20: Class4 - Donald Bren School of Information and Computer ... · Algorithms for Reasoning with graphical models Class4 Rina Dechter class2 828X 2019 1](https://reader033.fdocuments.us/reader033/viewer/2022052005/60188487e0320b7b965dd8be/html5/thumbnails/20.jpg)
Min-width Ordering
Proposition: algorithm min-width finds a min-width ordering of a graphComplexity:? O(e) class2 828X 2019
![Page 21: Class4 - Donald Bren School of Information and Computer ... · Algorithms for Reasoning with graphical models Class4 Rina Dechter class2 828X 2019 1](https://reader033.fdocuments.us/reader033/viewer/2022052005/60188487e0320b7b965dd8be/html5/thumbnails/21.jpg)
Greedy Orderings Heuristics
min-induced-width (miw)input: a graph G = (V;E), V = {v1; :::; vn}output: A miw ordering of the nodes d = (v1; :::; vn).1. for j = n to 1 by -1 do2. r a node in V with smallest degree.3. put r in position j.4. connect r's neighbors: E E union {(vi; vj)| (vi; r) in E; (vj ; r) 2 in E},5. remove r from the resulting graph: V V - {r}.
min-fill (min-fill)input: a graph G = (V;E), V = {v1; :::; vn}output: An ordering of the nodes d = (v1; :::; vn).1. for j = n to 1 by -1 do2. r a node in V with smallest fill edges for his parents.3. put r in position j.4. connect r's neighbors: E E union {(vi; vj)| (vi; r) 2 E; (vj ; r) in E},5. remove r from the resulting graph: V V –{r}.
Theorem: A graph is a tree iff it has both width and induced-width of 1.
class2 828X 2019 21
![Page 22: Class4 - Donald Bren School of Information and Computer ... · Algorithms for Reasoning with graphical models Class4 Rina Dechter class2 828X 2019 1](https://reader033.fdocuments.us/reader033/viewer/2022052005/60188487e0320b7b965dd8be/html5/thumbnails/22.jpg)
class2 828X 2019
Chordal Graphs;Max-Cardinality Ordering
A graph is chordal if every cycle of length at least 4 has a chord
Finding w* over chordal graph is easy using the max-cardinality ordering.
The induced graph is chordal K-trees are special chordal graphs. Finding the max-clique in chordal graphs is
easy (just enumerate all cliques in a max-cardinality ordering
![Page 23: Class4 - Donald Bren School of Information and Computer ... · Algorithms for Reasoning with graphical models Class4 Rina Dechter class2 828X 2019 1](https://reader033.fdocuments.us/reader033/viewer/2022052005/60188487e0320b7b965dd8be/html5/thumbnails/23.jpg)
Road Map Graphical models Constraint networks Model Inference
Variable elimination for Constraints Variable elimination for CNFs Greedy search for induced-width orderings Variable elimination for Linear Inequalities
Constraint propagation Search Probabilistic Networks
class2 828X 2019
![Page 24: Class4 - Donald Bren School of Information and Computer ... · Algorithms for Reasoning with graphical models Class4 Rina Dechter class2 828X 2019 1](https://reader033.fdocuments.us/reader033/viewer/2022052005/60188487e0320b7b965dd8be/html5/thumbnails/24.jpg)
Linear Inequalities
class2 828X 2019
Variables domains are the real numbers
![Page 25: Class4 - Donald Bren School of Information and Computer ... · Algorithms for Reasoning with graphical models Class4 Rina Dechter class2 828X 2019 1](https://reader033.fdocuments.us/reader033/viewer/2022052005/60188487e0320b7b965dd8be/html5/thumbnails/25.jpg)
class2 828X 2019
Linear Inequalities: Fourier Elimination
25
![Page 26: Class4 - Donald Bren School of Information and Computer ... · Algorithms for Reasoning with graphical models Class4 Rina Dechter class2 828X 2019 1](https://reader033.fdocuments.us/reader033/viewer/2022052005/60188487e0320b7b965dd8be/html5/thumbnails/26.jpg)
Directional linear elimination, DLE :generates a backtrack-free representation
class2 828X 2019 26
![Page 27: Class4 - Donald Bren School of Information and Computer ... · Algorithms for Reasoning with graphical models Class4 Rina Dechter class2 828X 2019 1](https://reader033.fdocuments.us/reader033/viewer/2022052005/60188487e0320b7b965dd8be/html5/thumbnails/27.jpg)
class2 828X 2019
Example
27
![Page 28: Class4 - Donald Bren School of Information and Computer ... · Algorithms for Reasoning with graphical models Class4 Rina Dechter class2 828X 2019 1](https://reader033.fdocuments.us/reader033/viewer/2022052005/60188487e0320b7b965dd8be/html5/thumbnails/28.jpg)
class2 828X 2019
Example
28
![Page 29: Class4 - Donald Bren School of Information and Computer ... · Algorithms for Reasoning with graphical models Class4 Rina Dechter class2 828X 2019 1](https://reader033.fdocuments.us/reader033/viewer/2022052005/60188487e0320b7b965dd8be/html5/thumbnails/29.jpg)
Algorithms for Reasoning with graphical models
Class5Rina Dechter
class2 828X 2019 29
![Page 30: Class4 - Donald Bren School of Information and Computer ... · Algorithms for Reasoning with graphical models Class4 Rina Dechter class2 828X 2019 1](https://reader033.fdocuments.us/reader033/viewer/2022052005/60188487e0320b7b965dd8be/html5/thumbnails/30.jpg)
Road Map Graphical models Constraint networks Model Inference
Variable elimination for Constraints
Variable elimination for CNFs Variable elimination for Linear Inequalities Constraint propagation (chapter 3 Dechter2)
Search Probabilistic Networks
class2 828X 2019
![Page 31: Class4 - Donald Bren School of Information and Computer ... · Algorithms for Reasoning with graphical models Class4 Rina Dechter class2 828X 2019 1](https://reader033.fdocuments.us/reader033/viewer/2022052005/60188487e0320b7b965dd8be/html5/thumbnails/31.jpg)
Outline Arc-consistency algorithms Path-consistency and i-consistency Arc-consistency, Generalized arc-
consistency, relational arc-consistency Global and bound consistency Distributed (generalized) arc-
consistency Consistency operators: join, resolution,
Gausian eliminationclass2 828X 2019
![Page 32: Class4 - Donald Bren School of Information and Computer ... · Algorithms for Reasoning with graphical models Class4 Rina Dechter class2 828X 2019 1](https://reader033.fdocuments.us/reader033/viewer/2022052005/60188487e0320b7b965dd8be/html5/thumbnails/32.jpg)
Sudoku –Approximation: Constraint Propagation
Each row, column and major block must be alldifferent
“Well posed” if it has unique solution: 27 constraints
2 34 62
•Variables: empty slots
•Domains = {1,2,3,4,5,6,7,8,9}
•Constraints: •27 all-different
•Constraint •Propagation
•Inference
class2 828X 2019 32
![Page 33: Class4 - Donald Bren School of Information and Computer ... · Algorithms for Reasoning with graphical models Class4 Rina Dechter class2 828X 2019 1](https://reader033.fdocuments.us/reader033/viewer/2022052005/60188487e0320b7b965dd8be/html5/thumbnails/33.jpg)
Approximating Inference:Local Constraint Propagation
Problem: Adaptive-consistency/Bucket-elimination algorithms are intractable when induced-width is large
Approximation: bound the size of recorded dependencies, i.e. perform local constraint propagation (local inference)
class2 828X 2019
![Page 34: Class4 - Donald Bren School of Information and Computer ... · Algorithms for Reasoning with graphical models Class4 Rina Dechter class2 828X 2019 1](https://reader033.fdocuments.us/reader033/viewer/2022052005/60188487e0320b7b965dd8be/html5/thumbnails/34.jpg)
From Global to Local Consistency
class2 828X 2019 34
![Page 35: Class4 - Donald Bren School of Information and Computer ... · Algorithms for Reasoning with graphical models Class4 Rina Dechter class2 828X 2019 1](https://reader033.fdocuments.us/reader033/viewer/2022052005/60188487e0320b7b965dd8be/html5/thumbnails/35.jpg)
Arc-ConsistencyA binary constraint R(X,Y) is arc-consistent w.r.t. X is every valueIn X’s domain has a match in Y’s domain.
YXRR YX constraint },3,2,1{ },3,2,1{
Only domains are reduced: X YXYX DRR
class2 828X 2019
![Page 36: Class4 - Donald Bren School of Information and Computer ... · Algorithms for Reasoning with graphical models Class4 Rina Dechter class2 828X 2019 1](https://reader033.fdocuments.us/reader033/viewer/2022052005/60188487e0320b7b965dd8be/html5/thumbnails/36.jpg)
Arc-Consistency
Definition: Given a constraint graph G, A variable Xi is arc-consistent relative to Xj iff for every value
aDXi there exists a value bDXj | (a, b)RXi,Xj.
The constraint RXi,Xj is arc-consistent iff Xi is arc-consistent relative to Xj and Xj is arc-consistent relative to Xi.
A binary CSP is arc-consistent iff every constraint (or sub-graph of size 2) is arc-consistent.
1
2 2
3
1
2
3
1
2
3
Xi XjXjXi
class2 828X 2019
![Page 37: Class4 - Donald Bren School of Information and Computer ... · Algorithms for Reasoning with graphical models Class4 Rina Dechter class2 828X 2019 1](https://reader033.fdocuments.us/reader033/viewer/2022052005/60188487e0320b7b965dd8be/html5/thumbnails/37.jpg)
32,1,
32,1, 32,1,
1 X, Y, Z, T 3X YY = ZT ZX T
X Y
T Z
32,1,
=
Arc-consistency
class2 828X 2019 37
Question: What will be the domain of Y once the network is arc-consistent?Or, how many values will it have?
![Page 38: Class4 - Donald Bren School of Information and Computer ... · Algorithms for Reasoning with graphical models Class4 Rina Dechter class2 828X 2019 1](https://reader033.fdocuments.us/reader033/viewer/2022052005/60188487e0320b7b965dd8be/html5/thumbnails/38.jpg)
1 X, Y, Z, T 3X YY = ZT ZX T
X Y
T Z
=
1 3
2 3
X YXYX DRR
Arc-consistency
class2 828X 2019 38
![Page 39: Class4 - Donald Bren School of Information and Computer ... · Algorithms for Reasoning with graphical models Class4 Rina Dechter class2 828X 2019 1](https://reader033.fdocuments.us/reader033/viewer/2022052005/60188487e0320b7b965dd8be/html5/thumbnails/39.jpg)
class2 828X 2019
Revise for Arc-Consistency
)( jijiii DRDD
⋈=⨂
39
![Page 40: Class4 - Donald Bren School of Information and Computer ... · Algorithms for Reasoning with graphical models Class4 Rina Dechter class2 828X 2019 1](https://reader033.fdocuments.us/reader033/viewer/2022052005/60188487e0320b7b965dd8be/html5/thumbnails/40.jpg)
Revise for Arc-Consistency
)( jijiii DRDD ⋈
class2 828X 2019 40
Complexity?
O(𝑘
![Page 41: Class4 - Donald Bren School of Information and Computer ... · Algorithms for Reasoning with graphical models Class4 Rina Dechter class2 828X 2019 1](https://reader033.fdocuments.us/reader033/viewer/2022052005/60188487e0320b7b965dd8be/html5/thumbnails/41.jpg)
class2 828X 2019
A matching diagram describing a network of constraints that is not arc-consistent (b) An arc-consistent equivalent network.
![Page 42: Class4 - Donald Bren School of Information and Computer ... · Algorithms for Reasoning with graphical models Class4 Rina Dechter class2 828X 2019 1](https://reader033.fdocuments.us/reader033/viewer/2022052005/60188487e0320b7b965dd8be/html5/thumbnails/42.jpg)
class2 828X 2019
A matching diagram describing a network of constraints that is not arc-consistent (b) An arc-consistent equivalent network.
![Page 43: Class4 - Donald Bren School of Information and Computer ... · Algorithms for Reasoning with graphical models Class4 Rina Dechter class2 828X 2019 1](https://reader033.fdocuments.us/reader033/viewer/2022052005/60188487e0320b7b965dd8be/html5/thumbnails/43.jpg)
AC-1
)( 3enkO
class2 828X 2019 43
Proof: Convergence? Completeness?
![Page 44: Class4 - Donald Bren School of Information and Computer ... · Algorithms for Reasoning with graphical models Class4 Rina Dechter class2 828X 2019 1](https://reader033.fdocuments.us/reader033/viewer/2022052005/60188487e0320b7b965dd8be/html5/thumbnails/44.jpg)
AC-1
)( 3enkO
Complexity (Mackworth and Freuder, 1986): e = number of arcs, n variables, k values (𝑒𝑘 each loop, nk number of loops), best-case = ek Arc-consistency is: Complexity of AC-1: O(en𝑘 )
)( 2ek
class2 828X 2019 44
![Page 45: Class4 - Donald Bren School of Information and Computer ... · Algorithms for Reasoning with graphical models Class4 Rina Dechter class2 828X 2019 1](https://reader033.fdocuments.us/reader033/viewer/2022052005/60188487e0320b7b965dd8be/html5/thumbnails/45.jpg)
AC-3
)( 3ekO Complexity: Best case O(ek), since each arc may be processed in O(2k)
class2 828X 2019 45
![Page 46: Class4 - Donald Bren School of Information and Computer ... · Algorithms for Reasoning with graphical models Class4 Rina Dechter class2 828X 2019 1](https://reader033.fdocuments.us/reader033/viewer/2022052005/60188487e0320b7b965dd8be/html5/thumbnails/46.jpg)
class2 828X 2019
Exercise: Apply Arc-Consistency in Class
Draw the network’s primal and dual constraint graph
Network = Domains {1,2,3,4} Constraints: y < x, z < y, t < z, f<t,
x<=t+1, Y<f+2 Apply AC-3?
![Page 47: Class4 - Donald Bren School of Information and Computer ... · Algorithms for Reasoning with graphical models Class4 Rina Dechter class2 828X 2019 1](https://reader033.fdocuments.us/reader033/viewer/2022052005/60188487e0320b7b965dd8be/html5/thumbnails/47.jpg)
Spring 2014 47
AC-4 (just FYI)
)( 2ekO Complexity: (Counter is the number of supports to 𝑎 in 𝑥 from 𝑥 . 𝑆 , is the
set of pairs that (𝑥 , 𝑎 ) supports)
![Page 48: Class4 - Donald Bren School of Information and Computer ... · Algorithms for Reasoning with graphical models Class4 Rina Dechter class2 828X 2019 1](https://reader033.fdocuments.us/reader033/viewer/2022052005/60188487e0320b7b965dd8be/html5/thumbnails/48.jpg)
Spring 2014 48
Example applying AC-4
![Page 49: Class4 - Donald Bren School of Information and Computer ... · Algorithms for Reasoning with graphical models Class4 Rina Dechter class2 828X 2019 1](https://reader033.fdocuments.us/reader033/viewer/2022052005/60188487e0320b7b965dd8be/html5/thumbnails/49.jpg)
Arc-Consistency Algorithms
)( 3ekO
AC-1: brute-force, distributed
AC-3, queue-based
AC-4, context-based, optimal
AC-5,6,7,…. Good in special cases
Important: applied at every node of search
n=number of variables, e=#constraints, k=domain sizeMackworth and Freuder (1977,1983), Mohr and Anderson, (1985)…
)( 3nekO
)( 2ekO
class2 828X 2019
![Page 50: Class4 - Donald Bren School of Information and Computer ... · Algorithms for Reasoning with graphical models Class4 Rina Dechter class2 828X 2019 1](https://reader033.fdocuments.us/reader033/viewer/2022052005/60188487e0320b7b965dd8be/html5/thumbnails/50.jpg)
From Arc-Consistency to Relational Arc-Consistency
Sound
Incomplete
Always converges(polynomial)
A B
CD
321A
321B
321D
321C
<
<< =
A < B1 2
2 3
A < D1 2
2 3
D < C1 2
2 3
B = C1 1
2 2
3 3
class2 828X 2019
![Page 51: Class4 - Donald Bren School of Information and Computer ... · Algorithms for Reasoning with graphical models Class4 Rina Dechter class2 828X 2019 1](https://reader033.fdocuments.us/reader033/viewer/2022052005/60188487e0320b7b965dd8be/html5/thumbnails/51.jpg)
Relational Distributed Arc-Consistency Primal Dual
AB
AD
AB1 2
2 3
AD1 2
2 3
AAB AD1 2 1 2
2 3 2 3
A B
CD
321A
321B
321D
321C
A < B1 2
2 3A < D1 2
2 3
D < C1 2
2 3
B = C1 1
2 2
3 3
DC
BC
D
C
B
AB BC1 2 2 2
2 3 3 3
BC1 1
2 2
3 3
DC1 2
2 3
BC DC2 2 1 2
3 3 2 3
AD DC1 2 2 3
class2 828X 2019 51
![Page 52: Class4 - Donald Bren School of Information and Computer ... · Algorithms for Reasoning with graphical models Class4 Rina Dechter class2 828X 2019 1](https://reader033.fdocuments.us/reader033/viewer/2022052005/60188487e0320b7b965dd8be/html5/thumbnails/52.jpg)
class2 828X 2019 52
All Arc-consistent algorithms converge to an equivalent and loosest arc-consistent network!!!
![Page 53: Class4 - Donald Bren School of Information and Computer ... · Algorithms for Reasoning with graphical models Class4 Rina Dechter class2 828X 2019 1](https://reader033.fdocuments.us/reader033/viewer/2022052005/60188487e0320b7b965dd8be/html5/thumbnails/53.jpg)
class2 828X 2019
Constraint Checking Arc-consistency
[ 5.... 18]
[ 4.... 15]
[ 1.... 10 ]B < C
A < B
B
A
2 < C - A < 5
C
2- A: [ 2 .. 10 ]
C: [ 6 .. 14 ]
3- B: [ 5 .. 13 ]
C: [ 6 .. 15 ]
1- B: [ 5 .. 14 ]1413
6
2
14
![Page 54: Class4 - Donald Bren School of Information and Computer ... · Algorithms for Reasoning with graphical models Class4 Rina Dechter class2 828X 2019 1](https://reader033.fdocuments.us/reader033/viewer/2022052005/60188487e0320b7b965dd8be/html5/thumbnails/54.jpg)
class2 828X 2019
Is Arc-Consistency Enough?
Example: a triangle graph-coloring with 2 values. Is it arc-consistent? Is it consistent?
It is not path, or 3-consistent.
![Page 55: Class4 - Donald Bren School of Information and Computer ... · Algorithms for Reasoning with graphical models Class4 Rina Dechter class2 828X 2019 1](https://reader033.fdocuments.us/reader033/viewer/2022052005/60188487e0320b7b965dd8be/html5/thumbnails/55.jpg)
Outline Arc-consistency algorithms Path-consistency and i-consistency Arc-consistency, Generalized arc-
consistency, relation arc-consistency Global and bound consistency Distributed (generalized) arc-
consistency Consistency operators: join, resolution,
Gausian eliminationclass2 828X 2019
![Page 56: Class4 - Donald Bren School of Information and Computer ... · Algorithms for Reasoning with graphical models Class4 Rina Dechter class2 828X 2019 1](https://reader033.fdocuments.us/reader033/viewer/2022052005/60188487e0320b7b965dd8be/html5/thumbnails/56.jpg)
Path-Consistency A pair (x, y) is path-consistent relative to Z, if every
consistent assignment (x, y) has a consistent extension to z.
class2 828X 2019
![Page 57: Class4 - Donald Bren School of Information and Computer ... · Algorithms for Reasoning with graphical models Class4 Rina Dechter class2 828X 2019 1](https://reader033.fdocuments.us/reader033/viewer/2022052005/60188487e0320b7b965dd8be/html5/thumbnails/57.jpg)
Example: Path-Consistency
class2 828X 2019 57
![Page 58: Class4 - Donald Bren School of Information and Computer ... · Algorithms for Reasoning with graphical models Class4 Rina Dechter class2 828X 2019 1](https://reader033.fdocuments.us/reader033/viewer/2022052005/60188487e0320b7b965dd8be/html5/thumbnails/58.jpg)
Spring 2014 58
Revise-3
)( kjkikijijij RDRRR
Complexity: O(𝑘 Best-case: O(t) Worst-case O(tk)
![Page 59: Class4 - Donald Bren School of Information and Computer ... · Algorithms for Reasoning with graphical models Class4 Rina Dechter class2 828X 2019 1](https://reader033.fdocuments.us/reader033/viewer/2022052005/60188487e0320b7b965dd8be/html5/thumbnails/59.jpg)
Spring 2014 59
PC-1
Complexity: O(𝑛 ) triplets, each take O(𝑘 ) steps O(𝑛 𝑘 ) Max number of loops: O(𝑛 𝑘 ) .
)( 55knO
![Page 60: Class4 - Donald Bren School of Information and Computer ... · Algorithms for Reasoning with graphical models Class4 Rina Dechter class2 828X 2019 1](https://reader033.fdocuments.us/reader033/viewer/2022052005/60188487e0320b7b965dd8be/html5/thumbnails/60.jpg)
Spring 2014 60
PC-2
)( 53knO Complexity: Optimal PC-4: (each pair deleted may add: 2n-1 triplets, number of pairs: O(𝑛 𝑘 ) size
of Q is O(𝑛 𝑘 ), processing is O(𝑘 ))
)( 33knO
![Page 61: Class4 - Donald Bren School of Information and Computer ... · Algorithms for Reasoning with graphical models Class4 Rina Dechter class2 828X 2019 1](https://reader033.fdocuments.us/reader033/viewer/2022052005/60188487e0320b7b965dd8be/html5/thumbnails/61.jpg)
Path-consistency Algorithms
)( kjkikijijij RDRRR
Apply Revise-3 (O( )) until no change
Path-consistency (3-consistency) adds binary constraints.
PC-1: PC-2: PC-4 optimal:
)( 55knO)( 53knO
)( 33knO
class2 828X 2019
⋈ ⋈
![Page 62: Class4 - Donald Bren School of Information and Computer ... · Algorithms for Reasoning with graphical models Class4 Rina Dechter class2 828X 2019 1](https://reader033.fdocuments.us/reader033/viewer/2022052005/60188487e0320b7b965dd8be/html5/thumbnails/62.jpg)
Local i-Consistency
i-consistency: Any consistent assignment to any i-1 variables isconsistent with at least one value of any i-th variable
class2 828X 2019 62
![Page 63: Class4 - Donald Bren School of Information and Computer ... · Algorithms for Reasoning with graphical models Class4 Rina Dechter class2 828X 2019 1](https://reader033.fdocuments.us/reader033/viewer/2022052005/60188487e0320b7b965dd8be/html5/thumbnails/63.jpg)
Directional i-Consistency
DCBR
A
ECD
B
D
C B
E
DC B
E
DC B
E
:AB A:BBC :C
AD C,D :DBE C,E D,E :E
Adaptive d-arcd-path
DBDC RR ,CBR
DRCRDR
class2 828X 2019 63
![Page 64: Class4 - Donald Bren School of Information and Computer ... · Algorithms for Reasoning with graphical models Class4 Rina Dechter class2 828X 2019 1](https://reader033.fdocuments.us/reader033/viewer/2022052005/60188487e0320b7b965dd8be/html5/thumbnails/64.jpg)
class2 828X 2019
Boolean Constraint Propagation
(A V B) and (B) B is arc-consistent relative to A but not vice-versa
Arc-consistency by resolution:res((A V B),B) = A
Given also (B V C), path-consistency:res((A V B),(B V C) = (A V C)Relational arc-consistency rule = unit-resolution
BAGGBA ,,
![Page 65: Class4 - Donald Bren School of Information and Computer ... · Algorithms for Reasoning with graphical models Class4 Rina Dechter class2 828X 2019 1](https://reader033.fdocuments.us/reader033/viewer/2022052005/60188487e0320b7b965dd8be/html5/thumbnails/65.jpg)
Gausian and Boolean Propagation, Resolution
Linear inequalities
Boolean constraint propagation, unit resolution
2,2 yx
)( CA
65
13,15 zzyx
)(),( BCBA
MIT 2018
![Page 66: Class4 - Donald Bren School of Information and Computer ... · Algorithms for Reasoning with graphical models Class4 Rina Dechter class2 828X 2019 1](https://reader033.fdocuments.us/reader033/viewer/2022052005/60188487e0320b7b965dd8be/html5/thumbnails/66.jpg)
class2 828X 2019
Unit Propagation
66
![Page 67: Class4 - Donald Bren School of Information and Computer ... · Algorithms for Reasoning with graphical models Class4 Rina Dechter class2 828X 2019 1](https://reader033.fdocuments.us/reader033/viewer/2022052005/60188487e0320b7b965dd8be/html5/thumbnails/67.jpg)
class2 828X 2019
Variable Elimination
Eliminate variablesone by one:“constraintpropagation”
Solution generation after elimination is backtrack-free
3
67
![Page 68: Class4 - Donald Bren School of Information and Computer ... · Algorithms for Reasoning with graphical models Class4 Rina Dechter class2 828X 2019 1](https://reader033.fdocuments.us/reader033/viewer/2022052005/60188487e0320b7b965dd8be/html5/thumbnails/68.jpg)
Road Map Graphical models Constraint networks Model Inference
Variable elimination:
Tree-clustering Constraint propagation
Search Probabilistic Networks
class2 828X 2019