15-780: Graduate AI Lecture 15. Uncertainty,...
Transcript of 15-780: Graduate AI Lecture 15. Uncertainty,...
![Page 1: 15-780: Graduate AI Lecture 15. Uncertainty, Inferenceggordon/780-spring09/slides/15-uncertainty.pdf · (quadtree, parti-game) 3. RRTs Build tree by randomly picking a point, extending](https://reader033.fdocuments.us/reader033/viewer/2022060510/5f27da6bdb41485c1d185919/html5/thumbnails/1.jpg)
15-780: Graduate AILecture 15. Uncertainty, Inference
Geoff Gordon (this lecture)Tuomas Sandholm
TAs Sam Ganzfried, Byron Boots
1
![Page 2: 15-780: Graduate AI Lecture 15. Uncertainty, Inferenceggordon/780-spring09/slides/15-uncertainty.pdf · (quadtree, parti-game) 3. RRTs Build tree by randomly picking a point, extending](https://reader033.fdocuments.us/reader033/viewer/2022060510/5f27da6bdb41485c1d185919/html5/thumbnails/2.jpg)
Review
2
![Page 3: 15-780: Graduate AI Lecture 15. Uncertainty, Inferenceggordon/780-spring09/slides/15-uncertainty.pdf · (quadtree, parti-game) 3. RRTs Build tree by randomly picking a point, extending](https://reader033.fdocuments.us/reader033/viewer/2022060510/5f27da6bdb41485c1d185919/html5/thumbnails/3.jpg)
Spatial planning
C-spaceWays of splitting up C-space
Visibility graphVoronoiCell decompositionVariable resolution or adaptive cells (quadtree, parti-game)
3
![Page 4: 15-780: Graduate AI Lecture 15. Uncertainty, Inferenceggordon/780-spring09/slides/15-uncertainty.pdf · (quadtree, parti-game) 3. RRTs Build tree by randomly picking a point, extending](https://reader033.fdocuments.us/reader033/viewer/2022060510/5f27da6bdb41485c1d185919/html5/thumbnails/4.jpg)
RRTs
Build tree by randomly picking a point, extending tree toward itOptionally:
build forward and backward at oncecross-link within tree
Plan within tree to get close enough to goal
4
![Page 5: 15-780: Graduate AI Lecture 15. Uncertainty, Inferenceggordon/780-spring09/slides/15-uncertainty.pdf · (quadtree, parti-game) 3. RRTs Build tree by randomly picking a point, extending](https://reader033.fdocuments.us/reader033/viewer/2022060510/5f27da6bdb41485c1d185919/html5/thumbnails/5.jpg)
RRTs
Tend to break up large Voronoi regions
So, RRT search is coarse to fine
First path found usually suboptimal; if we continue to grow tree and cross-link, get optimality in limit
5
![Page 6: 15-780: Graduate AI Lecture 15. Uncertainty, Inferenceggordon/780-spring09/slides/15-uncertainty.pdf · (quadtree, parti-game) 3. RRTs Build tree by randomly picking a point, extending](https://reader033.fdocuments.us/reader033/viewer/2022060510/5f27da6bdb41485c1d185919/html5/thumbnails/6.jpg)
Probability
Random variablesEvents (atomic, composite, AND/OR/NOT)Distributions: joint, marginalLaw of total probability
P(X) = P(X, Y=y1) + P(X, Y=y2) + …
6
![Page 7: 15-780: Graduate AI Lecture 15. Uncertainty, Inferenceggordon/780-spring09/slides/15-uncertainty.pdf · (quadtree, parti-game) 3. RRTs Build tree by randomly picking a point, extending](https://reader033.fdocuments.us/reader033/viewer/2022060510/5f27da6bdb41485c1d185919/html5/thumbnails/7.jpg)
Probability
7
![Page 8: 15-780: Graduate AI Lecture 15. Uncertainty, Inferenceggordon/780-spring09/slides/15-uncertainty.pdf · (quadtree, parti-game) 3. RRTs Build tree by randomly picking a point, extending](https://reader033.fdocuments.us/reader033/viewer/2022060510/5f27da6bdb41485c1d185919/html5/thumbnails/8.jpg)
Conditional: incorporating observations
A .41
A- .35
B+ .24
15-7
80
A A- B+
A .21 .17 .07
A- .17 .15 .06
B+ .07 .06 .04
HW
4
15-780
P(15-780=A | HW4=B+) = .04 / (.07+.06+.04)8
![Page 9: 15-780: Graduate AI Lecture 15. Uncertainty, Inferenceggordon/780-spring09/slides/15-uncertainty.pdf · (quadtree, parti-game) 3. RRTs Build tree by randomly picking a point, extending](https://reader033.fdocuments.us/reader033/viewer/2022060510/5f27da6bdb41485c1d185919/html5/thumbnails/9.jpg)
Conditioning
In general, divide a row or column by sumP(X | Y=y) = P(X, Y=y) / P(Y=y)
P(Y=y) is a marginal probabilityP(X | Y=y) is a row or column of tableThought experiment: what happens if we condition on an event of zero probability?
9
![Page 10: 15-780: Graduate AI Lecture 15. Uncertainty, Inferenceggordon/780-spring09/slides/15-uncertainty.pdf · (quadtree, parti-game) 3. RRTs Build tree by randomly picking a point, extending](https://reader033.fdocuments.us/reader033/viewer/2022060510/5f27da6bdb41485c1d185919/html5/thumbnails/10.jpg)
Notation
P(X | Y) is a function: x, y → P(X=x | Y=y)
As is standard, expressions are evaluated separately for each realization:
P(X | Y) P(Y) means the function x, y → P(X=x | Y=y) P(Y=y)
10
![Page 11: 15-780: Graduate AI Lecture 15. Uncertainty, Inferenceggordon/780-spring09/slides/15-uncertainty.pdf · (quadtree, parti-game) 3. RRTs Build tree by randomly picking a point, extending](https://reader033.fdocuments.us/reader033/viewer/2022060510/5f27da6bdb41485c1d185919/html5/thumbnails/11.jpg)
Independence
X and Y are independent if, for all possible values of y, P(X) = P(X | Y=y)
equivalently, for all possible values of x, P(Y) = P(Y | X=x)equivalently, P(X, Y) = P(X) P(Y)
Knowing X or Y gives us no information about the other
11
![Page 12: 15-780: Graduate AI Lecture 15. Uncertainty, Inferenceggordon/780-spring09/slides/15-uncertainty.pdf · (quadtree, parti-game) 3. RRTs Build tree by randomly picking a point, extending](https://reader033.fdocuments.us/reader033/viewer/2022060510/5f27da6bdb41485c1d185919/html5/thumbnails/12.jpg)
Independence: probability = product of marginals
Wea
ther
AAPL price
up same down
sun .09 .15 .06
rain .21 .35 .14
0.3
0.7
0.3 0.5 0.4
12
![Page 13: 15-780: Graduate AI Lecture 15. Uncertainty, Inferenceggordon/780-spring09/slides/15-uncertainty.pdf · (quadtree, parti-game) 3. RRTs Build tree by randomly picking a point, extending](https://reader033.fdocuments.us/reader033/viewer/2022060510/5f27da6bdb41485c1d185919/html5/thumbnails/13.jpg)
Bayes Rule
For any X, Y, CP(X | Y, C) P(Y | C) = P(Y | X, C) P(X | C)
Simple version (without context)P(X | Y) P(Y) = P(Y | X) P(X)
Proof: both sides are just P(X, Y)P(X | Y) = P(X, Y) / P(Y) (by def’n of conditioning)
13
![Page 14: 15-780: Graduate AI Lecture 15. Uncertainty, Inferenceggordon/780-spring09/slides/15-uncertainty.pdf · (quadtree, parti-game) 3. RRTs Build tree by randomly picking a point, extending](https://reader033.fdocuments.us/reader033/viewer/2022060510/5f27da6bdb41485c1d185919/html5/thumbnails/14.jpg)
Continuous distributions
What if X is real-valued?Atomic event: (X ≤ x), (X ≥ x)
any* other subset via AND, OR, NOTX=x has zero* probability
P(X=x, Y=y) means lim P(x ≤ X ≤ x+ε, Y = y) / ε ε → 0+
14
![Page 15: 15-780: Graduate AI Lecture 15. Uncertainty, Inferenceggordon/780-spring09/slides/15-uncertainty.pdf · (quadtree, parti-game) 3. RRTs Build tree by randomly picking a point, extending](https://reader033.fdocuments.us/reader033/viewer/2022060510/5f27da6bdb41485c1d185919/html5/thumbnails/15.jpg)
Factor Graphs
15
![Page 16: 15-780: Graduate AI Lecture 15. Uncertainty, Inferenceggordon/780-spring09/slides/15-uncertainty.pdf · (quadtree, parti-game) 3. RRTs Build tree by randomly picking a point, extending](https://reader033.fdocuments.us/reader033/viewer/2022060510/5f27da6bdb41485c1d185919/html5/thumbnails/16.jpg)
Factor graphs
Strict generalization of SAT to include uncertaintyFactor graph = (variables, constraints)
variables: set of discrete r.v.sconstraints: set of (possibly) soft or probabilistic constraints called factors or potentials
16
![Page 17: 15-780: Graduate AI Lecture 15. Uncertainty, Inferenceggordon/780-spring09/slides/15-uncertainty.pdf · (quadtree, parti-game) 3. RRTs Build tree by randomly picking a point, extending](https://reader033.fdocuments.us/reader033/viewer/2022060510/5f27da6bdb41485c1d185919/html5/thumbnails/17.jpg)
Factors
Hard constraint: X + Y ≥ 3Soft constraint: X + Y ≥ 3 is more probable than X + Y < 3, all else equalDomain of factor: set of relevant variables
{X, Y} in this case
17
![Page 18: 15-780: Graduate AI Lecture 15. Uncertainty, Inferenceggordon/780-spring09/slides/15-uncertainty.pdf · (quadtree, parti-game) 3. RRTs Build tree by randomly picking a point, extending](https://reader033.fdocuments.us/reader033/viewer/2022060510/5f27da6bdb41485c1d185919/html5/thumbnails/18.jpg)
Factors
0 1 20 1 1 1
1 1 1 3
2 1 3 3
X
Y
0 1 20 0 0 0
1 0 0 1
2 0 1 1
X
Y
Hard Soft
18
![Page 19: 15-780: Graduate AI Lecture 15. Uncertainty, Inferenceggordon/780-spring09/slides/15-uncertainty.pdf · (quadtree, parti-game) 3. RRTs Build tree by randomly picking a point, extending](https://reader033.fdocuments.us/reader033/viewer/2022060510/5f27da6bdb41485c1d185919/html5/thumbnails/19.jpg)
Factor graphs
Variables and factors connected according to domains
19
![Page 20: 15-780: Graduate AI Lecture 15. Uncertainty, Inferenceggordon/780-spring09/slides/15-uncertainty.pdf · (quadtree, parti-game) 3. RRTs Build tree by randomly picking a point, extending](https://reader033.fdocuments.us/reader033/viewer/2022060510/5f27da6bdb41485c1d185919/html5/thumbnails/20.jpg)
Factor graphs
Omit single-argument factors from graph to reduce clutter
20
![Page 21: 15-780: Graduate AI Lecture 15. Uncertainty, Inferenceggordon/780-spring09/slides/15-uncertainty.pdf · (quadtree, parti-game) 3. RRTs Build tree by randomly picking a point, extending](https://reader033.fdocuments.us/reader033/viewer/2022060510/5f27da6bdb41485c1d185919/html5/thumbnails/21.jpg)
Factor graphs: probability model
21
![Page 22: 15-780: Graduate AI Lecture 15. Uncertainty, Inferenceggordon/780-spring09/slides/15-uncertainty.pdf · (quadtree, parti-game) 3. RRTs Build tree by randomly picking a point, extending](https://reader033.fdocuments.us/reader033/viewer/2022060510/5f27da6bdb41485c1d185919/html5/thumbnails/22.jpg)
Normalizing constant
Also called partition function
22
![Page 23: 15-780: Graduate AI Lecture 15. Uncertainty, Inferenceggordon/780-spring09/slides/15-uncertainty.pdf · (quadtree, parti-game) 3. RRTs Build tree by randomly picking a point, extending](https://reader033.fdocuments.us/reader033/viewer/2022060510/5f27da6bdb41485c1d185919/html5/thumbnails/23.jpg)
Factors
Ra O W Φ1
T T T 3T T F 1T F T 3T F F 3F T T 3F T F 3F F T 3F F F 3
W M Ru Φ2
T T T 3T T F 1T F T 3T F F 3F T T 3F T F 3F F T 3F F F 3
23
![Page 24: 15-780: Graduate AI Lecture 15. Uncertainty, Inferenceggordon/780-spring09/slides/15-uncertainty.pdf · (quadtree, parti-game) 3. RRTs Build tree by randomly picking a point, extending](https://reader033.fdocuments.us/reader033/viewer/2022060510/5f27da6bdb41485c1d185919/html5/thumbnails/24.jpg)
Unary factors
Ra Φ3
T 1
F 2
W Φ4
T 1
F 1
O Φ5
T 5
F 2
M Φ6
T 10
F 1
Ru Φ7
T 1
F 324
![Page 25: 15-780: Graduate AI Lecture 15. Uncertainty, Inferenceggordon/780-spring09/slides/15-uncertainty.pdf · (quadtree, parti-game) 3. RRTs Build tree by randomly picking a point, extending](https://reader033.fdocuments.us/reader033/viewer/2022060510/5f27da6bdb41485c1d185919/html5/thumbnails/25.jpg)
Inference Qs
Is Z > 0?What is P(E)?What is P(E1 | E2)?Sample a random configuration according to P(.) or P(. | E)Hard part: taking sums of Φ (such as Z)
25
![Page 26: 15-780: Graduate AI Lecture 15. Uncertainty, Inferenceggordon/780-spring09/slides/15-uncertainty.pdf · (quadtree, parti-game) 3. RRTs Build tree by randomly picking a point, extending](https://reader033.fdocuments.us/reader033/viewer/2022060510/5f27da6bdb41485c1d185919/html5/thumbnails/26.jpg)
Example
What is P(T, T, T, T, T)?
26
![Page 27: 15-780: Graduate AI Lecture 15. Uncertainty, Inferenceggordon/780-spring09/slides/15-uncertainty.pdf · (quadtree, parti-game) 3. RRTs Build tree by randomly picking a point, extending](https://reader033.fdocuments.us/reader033/viewer/2022060510/5f27da6bdb41485c1d185919/html5/thumbnails/27.jpg)
Example
What is P(Rusty=T | Rains=T)?This is P(Rusty=T, Rains=T) / P(Rains=T)P(Rains) is a marginal of P(…)So is P(Rusty, Rains)Note: Z cancels, but still have to sum lots of entries to get each marginal
27
![Page 28: 15-780: Graduate AI Lecture 15. Uncertainty, Inferenceggordon/780-spring09/slides/15-uncertainty.pdf · (quadtree, parti-game) 3. RRTs Build tree by randomly picking a point, extending](https://reader033.fdocuments.us/reader033/viewer/2022060510/5f27da6bdb41485c1d185919/html5/thumbnails/28.jpg)
Relationship to SAT, ILP
Easy to write a clause or a linear constraint as a factor: 1 if satisfied, 0 o/wFeasibility problem: is Z > 0?
more generally, count satisfying assignments (determine Z)NP or #P complete (respectively)
Sampling problem: return a satisfying assignment uniformly at random
28
![Page 29: 15-780: Graduate AI Lecture 15. Uncertainty, Inferenceggordon/780-spring09/slides/15-uncertainty.pdf · (quadtree, parti-game) 3. RRTs Build tree by randomly picking a point, extending](https://reader033.fdocuments.us/reader033/viewer/2022060510/5f27da6bdb41485c1d185919/html5/thumbnails/29.jpg)
Inference
29
![Page 30: 15-780: Graduate AI Lecture 15. Uncertainty, Inferenceggordon/780-spring09/slides/15-uncertainty.pdf · (quadtree, parti-game) 3. RRTs Build tree by randomly picking a point, extending](https://reader033.fdocuments.us/reader033/viewer/2022060510/5f27da6bdb41485c1d185919/html5/thumbnails/30.jpg)
Notation
Boldface = vector of r.v.s or valuesX = (X1, X2, X3, X4, X5)x = (x1, x2, x3, x4, x5)
Set index = subvectorIf D = {1, 3, 4} then XD = (X1, X3, X4)
In particular, XD(Φ) = input to factor Φ
30
![Page 31: 15-780: Graduate AI Lecture 15. Uncertainty, Inferenceggordon/780-spring09/slides/15-uncertainty.pdf · (quadtree, parti-game) 3. RRTs Build tree by randomly picking a point, extending](https://reader033.fdocuments.us/reader033/viewer/2022060510/5f27da6bdb41485c1d185919/html5/thumbnails/31.jpg)
Finding x w/ P(X=x)>0
{x | P(X=x)>0} = support of P(X)
Replace each factor Φ with Φ’ = I(Φ>0)Now we have a CSP (or SAT); do DPLL
31
![Page 32: 15-780: Graduate AI Lecture 15. Uncertainty, Inferenceggordon/780-spring09/slides/15-uncertainty.pdf · (quadtree, parti-game) 3. RRTs Build tree by randomly picking a point, extending](https://reader033.fdocuments.us/reader033/viewer/2022060510/5f27da6bdb41485c1d185919/html5/thumbnails/32.jpg)
Counting support
DPLL also works to count {x | P(X=x)>0}Or, we can get smarter: dynamic programmingExample: (A ∨ B) ∧ (B ∨ C) ∧ (C ∨ D)
32
![Page 33: 15-780: Graduate AI Lecture 15. Uncertainty, Inferenceggordon/780-spring09/slides/15-uncertainty.pdf · (quadtree, parti-game) 3. RRTs Build tree by randomly picking a point, extending](https://reader033.fdocuments.us/reader033/viewer/2022060510/5f27da6bdb41485c1d185919/html5/thumbnails/33.jpg)
DP: (A ∨ B) ∧ (B ∨ C) ∧ (C ∨ D)
Consider B=T and B=F separatelyB=F: 1; B=T: 2*2 = 4Subtotal: |supp(ABC)| = 5; note C=F in 2
A B Φ1T T 1T F 1F T 1F F 0
B C Φ2T T 1T F 1F T 1F F 0
C D Φ3T T 1T F 1F T 1F F 0
33
![Page 34: 15-780: Graduate AI Lecture 15. Uncertainty, Inferenceggordon/780-spring09/slides/15-uncertainty.pdf · (quadtree, parti-game) 3. RRTs Build tree by randomly picking a point, extending](https://reader033.fdocuments.us/reader033/viewer/2022060510/5f27da6bdb41485c1d185919/html5/thumbnails/34.jpg)
DP: (A ∨ B) ∧ (B ∨ C) ∧ (C ∨ D)
Consider C=T and C=F separatelyC=F: 2*1=2; C=T: 3*2 = 6Total: |support| = 8
A B Φ1T T 1T F 1F T 1F F 0
B C Φ2T T 1T F 1F T 1F F 0
C D Φ3T T 1T F 1F T 1F F 0
34
![Page 35: 15-780: Graduate AI Lecture 15. Uncertainty, Inferenceggordon/780-spring09/slides/15-uncertainty.pdf · (quadtree, parti-game) 3. RRTs Build tree by randomly picking a point, extending](https://reader033.fdocuments.us/reader033/viewer/2022060510/5f27da6bdb41485c1d185919/html5/thumbnails/35.jpg)
Quiz
What is size of support for(A ∨ B) ∧ (B ∨ C) ∧ (C ∨ D) ∧ (D ∨ E)
35
![Page 36: 15-780: Graduate AI Lecture 15. Uncertainty, Inferenceggordon/780-spring09/slides/15-uncertainty.pdf · (quadtree, parti-game) 3. RRTs Build tree by randomly picking a point, extending](https://reader033.fdocuments.us/reader033/viewer/2022060510/5f27da6bdb41485c1d185919/html5/thumbnails/36.jpg)
DP for finding Z
Same trick works for finding Znot surprising: Z is a sum over support
36
![Page 37: 15-780: Graduate AI Lecture 15. Uncertainty, Inferenceggordon/780-spring09/slides/15-uncertainty.pdf · (quadtree, parti-game) 3. RRTs Build tree by randomly picking a point, extending](https://reader033.fdocuments.us/reader033/viewer/2022060510/5f27da6bdb41485c1d185919/html5/thumbnails/37.jpg)
Factors
Ra O W Φ1
T T T 3T T F 1T F T 3T F F 3F T T 3F T F 3F F T 3F F F 3
Ra Φ3
T 1
F 2
W Φ4
T 1
F 1
O Φ5
T 5
F 2
37
![Page 38: 15-780: Graduate AI Lecture 15. Uncertainty, Inferenceggordon/780-spring09/slides/15-uncertainty.pdf · (quadtree, parti-game) 3. RRTs Build tree by randomly picking a point, extending](https://reader033.fdocuments.us/reader033/viewer/2022060510/5f27da6bdb41485c1d185919/html5/thumbnails/38.jpg)
Factors
Ra O W Φ1Φ3 Φ4Φ5 *T T T 3 1 1 5 15T T F 1 1 1 5 5T F T 3 1 1 2 6T F F 3 1 1 2 6F T T 3 2 1 5 30F T F 3 2 1 5 30F F T 3 2 1 2 12F F F 3 2 1 2 12
Ra Φ3
T 1
F 2
W Φ4
T 1
F 1
O Φ5
T 5
F 2
38
![Page 39: 15-780: Graduate AI Lecture 15. Uncertainty, Inferenceggordon/780-spring09/slides/15-uncertainty.pdf · (quadtree, parti-game) 3. RRTs Build tree by randomly picking a point, extending](https://reader033.fdocuments.us/reader033/viewer/2022060510/5f27da6bdb41485c1d185919/html5/thumbnails/39.jpg)
Factors
Ra O W *T T T 15T T F 5T F T 6T F F 6F T T 30F T F 30F F T 12F F F 12
W λ
T 63
F 53
39
![Page 40: 15-780: Graduate AI Lecture 15. Uncertainty, Inferenceggordon/780-spring09/slides/15-uncertainty.pdf · (quadtree, parti-game) 3. RRTs Build tree by randomly picking a point, extending](https://reader033.fdocuments.us/reader033/viewer/2022060510/5f27da6bdb41485c1d185919/html5/thumbnails/40.jpg)
Factors
W M RuΦ2Φ6Φ7 λ *T T T 3 10 1 63 1890T T F 1 10 3 63 1890T F T 3 1 1 63 189T F F 3 1 3 63 567F T T 3 10 1 53 1590F T F 3 10 3 53 4770F F T 3 1 1 53 159F F F 3 1 3 53 477
M Φ6
T 10
F 1
Ru Φ7
T 1
F 3
W λ
T 63
F 5340
![Page 41: 15-780: Graduate AI Lecture 15. Uncertainty, Inferenceggordon/780-spring09/slides/15-uncertainty.pdf · (quadtree, parti-game) 3. RRTs Build tree by randomly picking a point, extending](https://reader033.fdocuments.us/reader033/viewer/2022060510/5f27da6bdb41485c1d185919/html5/thumbnails/41.jpg)
Factors
W M Ru *T T T 1890T T F 1890T F T 189T F F 567F T T 1590F T F 4770F F T 159F F F 477
Z = 11532
41
![Page 42: 15-780: Graduate AI Lecture 15. Uncertainty, Inferenceggordon/780-spring09/slides/15-uncertainty.pdf · (quadtree, parti-game) 3. RRTs Build tree by randomly picking a point, extending](https://reader033.fdocuments.us/reader033/viewer/2022060510/5f27da6bdb41485c1d185919/html5/thumbnails/42.jpg)
Variable elimination
Basic step: move a sum inward as far as possible, then do sum for each possible way of setting neighbors
42
![Page 43: 15-780: Graduate AI Lecture 15. Uncertainty, Inferenceggordon/780-spring09/slides/15-uncertainty.pdf · (quadtree, parti-game) 3. RRTs Build tree by randomly picking a point, extending](https://reader033.fdocuments.us/reader033/viewer/2022060510/5f27da6bdb41485c1d185919/html5/thumbnails/43.jpg)
Eliminating partway
For Z, want to sum over all XFor marginal P(X45), eliminate X123678
Sum out 123, then 876, to get Z P(X45)Sum out 45 to get Z
43
![Page 44: 15-780: Graduate AI Lecture 15. Uncertainty, Inferenceggordon/780-spring09/slides/15-uncertainty.pdf · (quadtree, parti-game) 3. RRTs Build tree by randomly picking a point, extending](https://reader033.fdocuments.us/reader033/viewer/2022060510/5f27da6bdb41485c1d185919/html5/thumbnails/44.jpg)
Eliminating partway
Conditional P(X45 | X6) = P(X456) / P(X6)Sum out 123, then 87, to get Z P(X456)Sum out 45 to get Z P(X6)Divide
44
![Page 45: 15-780: Graduate AI Lecture 15. Uncertainty, Inferenceggordon/780-spring09/slides/15-uncertainty.pdf · (quadtree, parti-game) 3. RRTs Build tree by randomly picking a point, extending](https://reader033.fdocuments.us/reader033/viewer/2022060510/5f27da6bdb41485c1d185919/html5/thumbnails/45.jpg)
A more difficult example
45
![Page 46: 15-780: Graduate AI Lecture 15. Uncertainty, Inferenceggordon/780-spring09/slides/15-uncertainty.pdf · (quadtree, parti-game) 3. RRTs Build tree by randomly picking a point, extending](https://reader033.fdocuments.us/reader033/viewer/2022060510/5f27da6bdb41485c1d185919/html5/thumbnails/46.jpg)
Treewidth
Elimination order E: sum x1, then x5, …treewidth(E) =
(size of largest factor formed) – 1treewidth = minE treewidth(E)Variable elimination uses space, time exponential in treewidthWorse: even computing treewidth is NP-complete
46
![Page 47: 15-780: Graduate AI Lecture 15. Uncertainty, Inferenceggordon/780-spring09/slides/15-uncertainty.pdf · (quadtree, parti-game) 3. RRTs Build tree by randomly picking a point, extending](https://reader033.fdocuments.us/reader033/viewer/2022060510/5f27da6bdb41485c1d185919/html5/thumbnails/47.jpg)
Belief propagation
Suppose we want all 1-variable marginalsCould do N runs of variable eliminationOr: the BP algorithm simulates N runs for the price of 2For details: Kschischang et al. reading
47
![Page 48: 15-780: Graduate AI Lecture 15. Uncertainty, Inferenceggordon/780-spring09/slides/15-uncertainty.pdf · (quadtree, parti-game) 3. RRTs Build tree by randomly picking a point, extending](https://reader033.fdocuments.us/reader033/viewer/2022060510/5f27da6bdb41485c1d185919/html5/thumbnails/48.jpg)
Approximate Inference
48
![Page 49: 15-780: Graduate AI Lecture 15. Uncertainty, Inferenceggordon/780-spring09/slides/15-uncertainty.pdf · (quadtree, parti-game) 3. RRTs Build tree by randomly picking a point, extending](https://reader033.fdocuments.us/reader033/viewer/2022060510/5f27da6bdb41485c1d185919/html5/thumbnails/49.jpg)
Most of the time…
Treewidth is bigVariables are high-arity or continuousCan’t afford exact inference
Partition function = numerical integration (and/or summation)We’ll look at randomized algorithms
49
![Page 50: 15-780: Graduate AI Lecture 15. Uncertainty, Inferenceggordon/780-spring09/slides/15-uncertainty.pdf · (quadtree, parti-game) 3. RRTs Build tree by randomly picking a point, extending](https://reader033.fdocuments.us/reader033/viewer/2022060510/5f27da6bdb41485c1d185919/html5/thumbnails/50.jpg)
Numerical integration
−1
−0.5
0
0.5
1−1 −0.8 −0.6 −0.4 −0.2 0 0.2 0.4 0.6 0.8 1
0
1
2
3
4
5
YX
f(X,Y)
50
![Page 51: 15-780: Graduate AI Lecture 15. Uncertainty, Inferenceggordon/780-spring09/slides/15-uncertainty.pdf · (quadtree, parti-game) 3. RRTs Build tree by randomly picking a point, extending](https://reader033.fdocuments.us/reader033/viewer/2022060510/5f27da6bdb41485c1d185919/html5/thumbnails/51.jpg)
Integration in 1000s of dims
Elia
zar a
nd P
arr,
IJC
AI-0
3
51
![Page 52: 15-780: Graduate AI Lecture 15. Uncertainty, Inferenceggordon/780-spring09/slides/15-uncertainty.pdf · (quadtree, parti-game) 3. RRTs Build tree by randomly picking a point, extending](https://reader033.fdocuments.us/reader033/viewer/2022060510/5f27da6bdb41485c1d185919/html5/thumbnails/52.jpg)
Simple 1D problem
−1 −0.5 0 0.5 10
10
20
30
40
50
60
70
52
![Page 53: 15-780: Graduate AI Lecture 15. Uncertainty, Inferenceggordon/780-spring09/slides/15-uncertainty.pdf · (quadtree, parti-game) 3. RRTs Build tree by randomly picking a point, extending](https://reader033.fdocuments.us/reader033/viewer/2022060510/5f27da6bdb41485c1d185919/html5/thumbnails/53.jpg)
Simplest randomized algorithm
−1 −0.5 0 0.5 10
10
20
30
40
50
60
70
Uniform sampling: sum(f(xi))/N53
![Page 54: 15-780: Graduate AI Lecture 15. Uncertainty, Inferenceggordon/780-spring09/slides/15-uncertainty.pdf · (quadtree, parti-game) 3. RRTs Build tree by randomly picking a point, extending](https://reader033.fdocuments.us/reader033/viewer/2022060510/5f27da6bdb41485c1d185919/html5/thumbnails/54.jpg)
Uniform sampling
So, V E(f(X)) is desired integralBut standard deviation can be bigCan reduce it by averaging many samplesBut only at rate 1/sqrt(N)
E(f(X)) =!
P (x)f(x)dx
=1V
!f(x)dx
54
![Page 55: 15-780: Graduate AI Lecture 15. Uncertainty, Inferenceggordon/780-spring09/slides/15-uncertainty.pdf · (quadtree, parti-game) 3. RRTs Build tree by randomly picking a point, extending](https://reader033.fdocuments.us/reader033/viewer/2022060510/5f27da6bdb41485c1d185919/html5/thumbnails/55.jpg)
Nonuniform (importance) sampling
−1 −0.5 0 0.5 10
10
20
30
40
50
60
70
55
![Page 56: 15-780: Graduate AI Lecture 15. Uncertainty, Inferenceggordon/780-spring09/slides/15-uncertainty.pdf · (quadtree, parti-game) 3. RRTs Build tree by randomly picking a point, extending](https://reader033.fdocuments.us/reader033/viewer/2022060510/5f27da6bdb41485c1d185919/html5/thumbnails/56.jpg)
Importance sampling
Instead of x ~ uniform, use x ~ Q(x)Q = importance distributionShould have Q(x) large where f(x) is largeProblem:
EQ(f(X)) =!
Q(x)f(x)dx
56
![Page 57: 15-780: Graduate AI Lecture 15. Uncertainty, Inferenceggordon/780-spring09/slides/15-uncertainty.pdf · (quadtree, parti-game) 3. RRTs Build tree by randomly picking a point, extending](https://reader033.fdocuments.us/reader033/viewer/2022060510/5f27da6bdb41485c1d185919/html5/thumbnails/57.jpg)
Importance sampling
h(x) ! f(x)/Q(x)
EQ(h(X)) =!
Q(x)h(x)dx
=!
Q(x)f(x)/Q(x)dx
=!
f(x)dx
57
![Page 58: 15-780: Graduate AI Lecture 15. Uncertainty, Inferenceggordon/780-spring09/slides/15-uncertainty.pdf · (quadtree, parti-game) 3. RRTs Build tree by randomly picking a point, extending](https://reader033.fdocuments.us/reader033/viewer/2022060510/5f27da6bdb41485c1d185919/html5/thumbnails/58.jpg)
Importance sampling
So, take samples of h(X) instead of f(X)wi = 1/Q(xi) is importance weightQ = uniform yields uniform sampling
58
![Page 59: 15-780: Graduate AI Lecture 15. Uncertainty, Inferenceggordon/780-spring09/slides/15-uncertainty.pdf · (quadtree, parti-game) 3. RRTs Build tree by randomly picking a point, extending](https://reader033.fdocuments.us/reader033/viewer/2022060510/5f27da6bdb41485c1d185919/html5/thumbnails/59.jpg)
Variance
How does this help us control variance?Suppose f big ==> Q bigAnd Q small ==> f smallThen h = f/Q never gets too bigVariance of each sample is lower ==> need fewer samplesA good Q makes a good IS
59
![Page 60: 15-780: Graduate AI Lecture 15. Uncertainty, Inferenceggordon/780-spring09/slides/15-uncertainty.pdf · (quadtree, parti-game) 3. RRTs Build tree by randomly picking a point, extending](https://reader033.fdocuments.us/reader033/viewer/2022060510/5f27da6bdb41485c1d185919/html5/thumbnails/60.jpg)
Importance sampling, part II
Suppose we want
Pick N samples xi from proposal Q(X)Average wi g(xi), where wi = P(xi)/Q(xi) is importance weight
!f(x)dx =
!P (x)g(x)dx = EP (g(X))
EQ(Wg(X)) =!
Q(x)[P (x)/Q(x)]g(x)dx =!
P (x)g(x)dx
60
![Page 61: 15-780: Graduate AI Lecture 15. Uncertainty, Inferenceggordon/780-spring09/slides/15-uncertainty.pdf · (quadtree, parti-game) 3. RRTs Build tree by randomly picking a point, extending](https://reader033.fdocuments.us/reader033/viewer/2022060510/5f27da6bdb41485c1d185919/html5/thumbnails/61.jpg)
Parallel importance sampling
Suppose we want
But P(x) is unnormalized (e.g., represented by a factor graph)—know only Z P(x)
!f(x)dx =
!P (x)g(x)dx = EP (g(X))
61
![Page 62: 15-780: Graduate AI Lecture 15. Uncertainty, Inferenceggordon/780-spring09/slides/15-uncertainty.pdf · (quadtree, parti-game) 3. RRTs Build tree by randomly picking a point, extending](https://reader033.fdocuments.us/reader033/viewer/2022060510/5f27da6bdb41485c1d185919/html5/thumbnails/62.jpg)
Parallel IS
Pick N samples xi from proposal Q(X)If we knew wi = P(xi)/Q(xi), could do ISInstead, set wi = ZP (xi)/Q(xi)
62
![Page 63: 15-780: Graduate AI Lecture 15. Uncertainty, Inferenceggordon/780-spring09/slides/15-uncertainty.pdf · (quadtree, parti-game) 3. RRTs Build tree by randomly picking a point, extending](https://reader033.fdocuments.us/reader033/viewer/2022060510/5f27da6bdb41485c1d185919/html5/thumbnails/63.jpg)
Parallel IS
So, is an unbiased estimate of Zw =1N
!
i
wi
E(W ) =!
Q(x)(ZP (x)/Q(x))dx
= Z
!P (x)dx
= Z
63
![Page 64: 15-780: Graduate AI Lecture 15. Uncertainty, Inferenceggordon/780-spring09/slides/15-uncertainty.pdf · (quadtree, parti-game) 3. RRTs Build tree by randomly picking a point, extending](https://reader033.fdocuments.us/reader033/viewer/2022060510/5f27da6bdb41485c1d185919/html5/thumbnails/64.jpg)
Parallel IS
So, is an estimate of wi, computed without knowing ZFinal estimate:
wi/w
!f(x)dx ! 1
n
"i
wiw g(xi)
64
![Page 65: 15-780: Graduate AI Lecture 15. Uncertainty, Inferenceggordon/780-spring09/slides/15-uncertainty.pdf · (quadtree, parti-game) 3. RRTs Build tree by randomly picking a point, extending](https://reader033.fdocuments.us/reader033/viewer/2022060510/5f27da6bdb41485c1d185919/html5/thumbnails/65.jpg)
Parallel IS is biased
0 1 2 30
0.5
1
1.5
2
2.5
3
mean(weights)
1 / m
ean(
weig
hts)
E(mean(weights))
E(W ) = Z, but E(1/W ) != 1/Z in general65
![Page 66: 15-780: Graduate AI Lecture 15. Uncertainty, Inferenceggordon/780-spring09/slides/15-uncertainty.pdf · (quadtree, parti-game) 3. RRTs Build tree by randomly picking a point, extending](https://reader033.fdocuments.us/reader033/viewer/2022060510/5f27da6bdb41485c1d185919/html5/thumbnails/66.jpg)
−2 −1 0 1 2−2
−1
0
1
2
Q : (X, Y ) ! N(1, 1) ! ! U("",")f(x, y, !) = Q(x, y, !)P (o = 0.8 | x, y, !)/Z
66
![Page 67: 15-780: Graduate AI Lecture 15. Uncertainty, Inferenceggordon/780-spring09/slides/15-uncertainty.pdf · (quadtree, parti-game) 3. RRTs Build tree by randomly picking a point, extending](https://reader033.fdocuments.us/reader033/viewer/2022060510/5f27da6bdb41485c1d185919/html5/thumbnails/67.jpg)
−2 −1 0 1 2−2
−1
0
1
2
Posterior E(X, Y, !) = (0.496, 0.350, 0.084)
67
![Page 68: 15-780: Graduate AI Lecture 15. Uncertainty, Inferenceggordon/780-spring09/slides/15-uncertainty.pdf · (quadtree, parti-game) 3. RRTs Build tree by randomly picking a point, extending](https://reader033.fdocuments.us/reader033/viewer/2022060510/5f27da6bdb41485c1d185919/html5/thumbnails/68.jpg)
Back to high dimensions
Picking a good importance distribution is hard in high-DMajor contributions to integral can be hidden in small areas
recall, want (f big ==> Q big)Would like to search for areas of high f(x)But searching could bias our estimates
68
![Page 69: 15-780: Graduate AI Lecture 15. Uncertainty, Inferenceggordon/780-spring09/slides/15-uncertainty.pdf · (quadtree, parti-game) 3. RRTs Build tree by randomly picking a point, extending](https://reader033.fdocuments.us/reader033/viewer/2022060510/5f27da6bdb41485c1d185919/html5/thumbnails/69.jpg)
MCMC
69
![Page 70: 15-780: Graduate AI Lecture 15. Uncertainty, Inferenceggordon/780-spring09/slides/15-uncertainty.pdf · (quadtree, parti-game) 3. RRTs Build tree by randomly picking a point, extending](https://reader033.fdocuments.us/reader033/viewer/2022060510/5f27da6bdb41485c1d185919/html5/thumbnails/70.jpg)
Markov-Chain Monte Carlo
Design a randomized search procedure M which tends to increase f(x) if it is smallRun M for a while, take resulting x as a sampleImportance weight Q(x)?
70
![Page 71: 15-780: Graduate AI Lecture 15. Uncertainty, Inferenceggordon/780-spring09/slides/15-uncertainty.pdf · (quadtree, parti-game) 3. RRTs Build tree by randomly picking a point, extending](https://reader033.fdocuments.us/reader033/viewer/2022060510/5f27da6bdb41485c1d185919/html5/thumbnails/71.jpg)
Markov-Chain Monte Carlo
Design a randomized search procedure M which tends to increase f(x) if it is smallRun M for a while, take resulting x as a sampleImportance weight Q(x)? Stationary distribution of M
71
![Page 72: 15-780: Graduate AI Lecture 15. Uncertainty, Inferenceggordon/780-spring09/slides/15-uncertainty.pdf · (quadtree, parti-game) 3. RRTs Build tree by randomly picking a point, extending](https://reader033.fdocuments.us/reader033/viewer/2022060510/5f27da6bdb41485c1d185919/html5/thumbnails/72.jpg)
Designing a search chain
Would like Q(x) = P(x)Turns out we can get this exactly, using Metropolis-Hastings
!f(x)dx =
!P (x)g(x)dx = EP (g(x))
72
![Page 73: 15-780: Graduate AI Lecture 15. Uncertainty, Inferenceggordon/780-spring09/slides/15-uncertainty.pdf · (quadtree, parti-game) 3. RRTs Build tree by randomly picking a point, extending](https://reader033.fdocuments.us/reader033/viewer/2022060510/5f27da6bdb41485c1d185919/html5/thumbnails/73.jpg)
Metropolis-Hastings
Way of designing chain w/ Q(x) = P(x)Basic strategy: start from arbitrary xRepeatedly tweak x to get x’If P(x’) ≥ P(x), move to x’If P(x’) << P(x), stay at xIn intermediate cases, randomize
73
![Page 74: 15-780: Graduate AI Lecture 15. Uncertainty, Inferenceggordon/780-spring09/slides/15-uncertainty.pdf · (quadtree, parti-game) 3. RRTs Build tree by randomly picking a point, extending](https://reader033.fdocuments.us/reader033/viewer/2022060510/5f27da6bdb41485c1d185919/html5/thumbnails/74.jpg)
Proposal distribution
Left open: what does “tweak” mean?Parameter of MH: Q(x’ | x)
one-step proposal distributionGood proposals explore quickly, but remain in regions of high P(x)Optimal proposal?
74
![Page 75: 15-780: Graduate AI Lecture 15. Uncertainty, Inferenceggordon/780-spring09/slides/15-uncertainty.pdf · (quadtree, parti-game) 3. RRTs Build tree by randomly picking a point, extending](https://reader033.fdocuments.us/reader033/viewer/2022060510/5f27da6bdb41485c1d185919/html5/thumbnails/75.jpg)
MH algorithm
Sample x’ ~ Q(x’ | x)
Compute p =
With probability p, set x := x’Repeat for T steps (usually < T distinct samples)
P (x!)P (x)
Q(x | x!)Q(x! | x)
75
![Page 76: 15-780: Graduate AI Lecture 15. Uncertainty, Inferenceggordon/780-spring09/slides/15-uncertainty.pdf · (quadtree, parti-game) 3. RRTs Build tree by randomly picking a point, extending](https://reader033.fdocuments.us/reader033/viewer/2022060510/5f27da6bdb41485c1d185919/html5/thumbnails/76.jpg)
MH notes
Only need P(x) up to a constant factorEfficiency determined by:
how fast Q(x’ | x) moves us aroundhow high acceptance probability p is
Tension between fast Q and high p
76