Graphical Models - Inference - Wolfram Burgard, Luc De Raedt, Kristian Kersting, Bernhard Nebel...
-
date post
18-Dec-2015 -
Category
Documents
-
view
248 -
download
0
Transcript of Graphical Models - Inference - Wolfram Burgard, Luc De Raedt, Kristian Kersting, Bernhard Nebel...
![Page 1: Graphical Models - Inference - Wolfram Burgard, Luc De Raedt, Kristian Kersting, Bernhard Nebel Albert-Ludwigs University Freiburg, Germany PCWP CO HRBP.](https://reader030.fdocuments.us/reader030/viewer/2022032800/56649d245503460f949fb285/html5/thumbnails/1.jpg)
Graphical Models- Inference -
Graphical Models- Inference -
Wolfram Burgard, Luc De Raedt, Kristian Kersting, Bernhard Nebel
Albert-Ludwigs University Freiburg, Germany
PCWP CO
HRBP
HREKG HRSAT
ERRCAUTERHRHISTORY
CATECHOL
SAO2 EXPCO2
ARTCO2
VENTALV
VENTLUNG VENITUBE
DISCONNECT
MINVOLSET
VENTMACHKINKEDTUBEINTUBATIONPULMEMBOLUS
PAP SHUNT
ANAPHYLAXIS
MINOVL
PVSAT
FIO2PRESS
INSUFFANESTHTPR
LVFAILURE
ERRBLOWOUTPUTSTROEVOLUMELVEDVOLUME
HYPOVOLEMIA
CVP
BP
Mainly based on F. V. Jensen, „Bayesian Networks and Decision Graphs“, Springer-Verlag New York, 2001.
AdvancedI WS 06/07
Variable Elimination
![Page 2: Graphical Models - Inference - Wolfram Burgard, Luc De Raedt, Kristian Kersting, Bernhard Nebel Albert-Ludwigs University Freiburg, Germany PCWP CO HRBP.](https://reader030.fdocuments.us/reader030/viewer/2022032800/56649d245503460f949fb285/html5/thumbnails/2.jpg)
Bayesian Networks
Bayesian Networks
AdvancedI WS 06/07 Outline
• Introduction • Reminder: Probability theory• Basics of Bayesian Networks• Modeling Bayesian networks• Inference• Excourse: Markov Networks• Learning Bayesian networks• Relational Models
![Page 3: Graphical Models - Inference - Wolfram Burgard, Luc De Raedt, Kristian Kersting, Bernhard Nebel Albert-Ludwigs University Freiburg, Germany PCWP CO HRBP.](https://reader030.fdocuments.us/reader030/viewer/2022032800/56649d245503460f949fb285/html5/thumbnails/3.jpg)
Bayesian Networks
Bayesian Networks
AdvancedI WS 06/07 Elimination in Chains
• Forward pass
• Using definition of probability, we have
A B C ED
- Inference (Variable Elimination)
- Inference (Variable Elimination)
![Page 4: Graphical Models - Inference - Wolfram Burgard, Luc De Raedt, Kristian Kersting, Bernhard Nebel Albert-Ludwigs University Freiburg, Germany PCWP CO HRBP.](https://reader030.fdocuments.us/reader030/viewer/2022032800/56649d245503460f949fb285/html5/thumbnails/4.jpg)
Bayesian Networks
Bayesian Networks
AdvancedI WS 06/07 Elimination in Chains
• By chain decomposition, we get
A B C ED
- Inference (Variable Elimination)
- Inference (Variable Elimination)
![Page 5: Graphical Models - Inference - Wolfram Burgard, Luc De Raedt, Kristian Kersting, Bernhard Nebel Albert-Ludwigs University Freiburg, Germany PCWP CO HRBP.](https://reader030.fdocuments.us/reader030/viewer/2022032800/56649d245503460f949fb285/html5/thumbnails/5.jpg)
Bayesian Networks
Bayesian Networks
AdvancedI WS 06/07 Elimination in Chains
• Rearranging terms ...
A B C ED
- Inference (Variable Elimination)
- Inference (Variable Elimination)
![Page 6: Graphical Models - Inference - Wolfram Burgard, Luc De Raedt, Kristian Kersting, Bernhard Nebel Albert-Ludwigs University Freiburg, Germany PCWP CO HRBP.](https://reader030.fdocuments.us/reader030/viewer/2022032800/56649d245503460f949fb285/html5/thumbnails/6.jpg)
Bayesian Networks
Bayesian Networks
AdvancedI WS 06/07
• Now we can perform innermost summation
Elimination in Chains
A B C ED
- Inference (Variable Elimination)
- Inference (Variable Elimination)
XA has been eliminated
![Page 7: Graphical Models - Inference - Wolfram Burgard, Luc De Raedt, Kristian Kersting, Bernhard Nebel Albert-Ludwigs University Freiburg, Germany PCWP CO HRBP.](https://reader030.fdocuments.us/reader030/viewer/2022032800/56649d245503460f949fb285/html5/thumbnails/7.jpg)
Bayesian Networks
Bayesian Networks
AdvancedI WS 06/07
• Rearranging and then summing again, we get
Elimination in Chains
A B C ED
- Inference (Variable Elimination)
- Inference (Variable Elimination)
X XB has been eliminated
…
![Page 8: Graphical Models - Inference - Wolfram Burgard, Luc De Raedt, Kristian Kersting, Bernhard Nebel Albert-Ludwigs University Freiburg, Germany PCWP CO HRBP.](https://reader030.fdocuments.us/reader030/viewer/2022032800/56649d245503460f949fb285/html5/thumbnails/8.jpg)
Bayesian Networks
Bayesian Networks
AdvancedI WS 06/07
Elimination in Chains with Evidence
• Similar due to Bayes’ Rule
• We write the query in explicit form
A B C ED
- Inference (Variable Elimination)
- Inference (Variable Elimination)
![Page 9: Graphical Models - Inference - Wolfram Burgard, Luc De Raedt, Kristian Kersting, Bernhard Nebel Albert-Ludwigs University Freiburg, Germany PCWP CO HRBP.](https://reader030.fdocuments.us/reader030/viewer/2022032800/56649d245503460f949fb285/html5/thumbnails/9.jpg)
Bayesian Networks
Bayesian Networks
AdvancedI WS 06/07
• Write query in the form
• Iteratively– Move all irrelevant terms outside of
innermost sum– Perform innermost sum, getting a new term– Insert the new term into the product
Variable Elimination
- Inference (Variable Elimination)
- Inference (Variable Elimination)
Semantic Bayesian Network
Eliminate irrelevant variables
![Page 10: Graphical Models - Inference - Wolfram Burgard, Luc De Raedt, Kristian Kersting, Bernhard Nebel Albert-Ludwigs University Freiburg, Germany PCWP CO HRBP.](https://reader030.fdocuments.us/reader030/viewer/2022032800/56649d245503460f949fb285/html5/thumbnails/10.jpg)
Bayesian Networks
Bayesian Networks
AdvancedI WS 06/07 Asia - A More Complex Example
Visit to Asia
Smoking
Lung CancerTuberculosis
Abnormalityin Chest
Bronchitis
X-Ray Dyspnoea
- Inference (Variable Elimination)
- Inference (Variable Elimination)
![Page 11: Graphical Models - Inference - Wolfram Burgard, Luc De Raedt, Kristian Kersting, Bernhard Nebel Albert-Ludwigs University Freiburg, Germany PCWP CO HRBP.](https://reader030.fdocuments.us/reader030/viewer/2022032800/56649d245503460f949fb285/html5/thumbnails/11.jpg)
Bayesian Networks
Bayesian Networks
AdvancedI WS 06/07
Initial factors
We want to compute P(d)Need to eliminate: v,s,x,t,l,a,b
V S
LT
A B
X D
- Inference (Variable Elimination)
- Inference (Variable Elimination)
![Page 12: Graphical Models - Inference - Wolfram Burgard, Luc De Raedt, Kristian Kersting, Bernhard Nebel Albert-Ludwigs University Freiburg, Germany PCWP CO HRBP.](https://reader030.fdocuments.us/reader030/viewer/2022032800/56649d245503460f949fb285/html5/thumbnails/12.jpg)
Bayesian Networks
Bayesian Networks
AdvancedI WS 06/07
Initial factors
We want to compute P(d)Need to eliminate: v,s,x,t,l,a,b
V S
LT
A B
X D
- Inference (Variable Elimination)
- Inference (Variable Elimination)
Eliminate: v
Compute:
but in general the result of eliminating a variable is not necessarily a probability term !
![Page 13: Graphical Models - Inference - Wolfram Burgard, Luc De Raedt, Kristian Kersting, Bernhard Nebel Albert-Ludwigs University Freiburg, Germany PCWP CO HRBP.](https://reader030.fdocuments.us/reader030/viewer/2022032800/56649d245503460f949fb285/html5/thumbnails/13.jpg)
Bayesian Networks
Bayesian Networks
AdvancedI WS 06/07
Initial factors
We want to compute P(d)Need to eliminate: s,x,t,l,a,b
V S
LT
A B
X D
- Inference (Variable Elimination)
- Inference (Variable Elimination)
Eliminate: s
Compute:
Summing on s results in a factor with two arguments fs(b,l). In general, result of elimination may be a function of several variables
![Page 14: Graphical Models - Inference - Wolfram Burgard, Luc De Raedt, Kristian Kersting, Bernhard Nebel Albert-Ludwigs University Freiburg, Germany PCWP CO HRBP.](https://reader030.fdocuments.us/reader030/viewer/2022032800/56649d245503460f949fb285/html5/thumbnails/14.jpg)
Bayesian Networks
Bayesian Networks
AdvancedI WS 06/07
Initial factors
We want to compute P(d)Need to eliminate: x,t,l,a,b
V S
LT
A B
X D
- Inference (Variable Elimination)
- Inference (Variable Elimination)
Eliminate: x
Compute:
Note that fx(a)=1 for all values of a
![Page 15: Graphical Models - Inference - Wolfram Burgard, Luc De Raedt, Kristian Kersting, Bernhard Nebel Albert-Ludwigs University Freiburg, Germany PCWP CO HRBP.](https://reader030.fdocuments.us/reader030/viewer/2022032800/56649d245503460f949fb285/html5/thumbnails/15.jpg)
Bayesian Networks
Bayesian Networks
AdvancedI WS 06/07
Initial factors
We want to compute P(d)Need to eliminate: t,l,a,b
V S
LT
A B
X D
- Inference (Variable Elimination)
- Inference (Variable Elimination)
Eliminate: t
Compute:
![Page 16: Graphical Models - Inference - Wolfram Burgard, Luc De Raedt, Kristian Kersting, Bernhard Nebel Albert-Ludwigs University Freiburg, Germany PCWP CO HRBP.](https://reader030.fdocuments.us/reader030/viewer/2022032800/56649d245503460f949fb285/html5/thumbnails/16.jpg)
Bayesian Networks
Bayesian Networks
AdvancedI WS 06/07
Initial factors
We want to compute P(d)Need to eliminate: l,a,b
V S
LT
A B
X D
- Inference (Variable Elimination)
- Inference (Variable Elimination)
Eliminate: l
Compute:
![Page 17: Graphical Models - Inference - Wolfram Burgard, Luc De Raedt, Kristian Kersting, Bernhard Nebel Albert-Ludwigs University Freiburg, Germany PCWP CO HRBP.](https://reader030.fdocuments.us/reader030/viewer/2022032800/56649d245503460f949fb285/html5/thumbnails/17.jpg)
Bayesian Networks
Bayesian Networks
AdvancedI WS 06/07
Initial factors
We want to compute P(d)Need to eliminate: a,b
V S
LT
A B
X D
- Inference (Variable Elimination)
- Inference (Variable Elimination)
Eliminate: a,bCompute:
![Page 18: Graphical Models - Inference - Wolfram Burgard, Luc De Raedt, Kristian Kersting, Bernhard Nebel Albert-Ludwigs University Freiburg, Germany PCWP CO HRBP.](https://reader030.fdocuments.us/reader030/viewer/2022032800/56649d245503460f949fb285/html5/thumbnails/18.jpg)
Bayesian Networks
Bayesian Networks
AdvancedI WS 06/07 Variable Elimination (VE)
• We now understand VE as a sequence of rewriting operations
• Actual computation is done in elimination step
• Exactly the same procedure applies to Markov networks
• Computation depends on order of elimination
- Inference (Variable Elimination)
- Inference (Variable Elimination)
![Page 19: Graphical Models - Inference - Wolfram Burgard, Luc De Raedt, Kristian Kersting, Bernhard Nebel Albert-Ludwigs University Freiburg, Germany PCWP CO HRBP.](https://reader030.fdocuments.us/reader030/viewer/2022032800/56649d245503460f949fb285/html5/thumbnails/19.jpg)
Bayesian Networks
Bayesian Networks
AdvancedI WS 06/07
• How do we deal with evidence?
• Suppose get evidence V = t, S = f, D = t• We want to compute P(L, V = t, S = f, D
= t)
Dealing with EvidenceV S
LT
A B
X D
- Inference (Variable Elimination)
- Inference (Variable Elimination)
![Page 20: Graphical Models - Inference - Wolfram Burgard, Luc De Raedt, Kristian Kersting, Bernhard Nebel Albert-Ludwigs University Freiburg, Germany PCWP CO HRBP.](https://reader030.fdocuments.us/reader030/viewer/2022032800/56649d245503460f949fb285/html5/thumbnails/20.jpg)
Bayesian Networks
Bayesian Networks
AdvancedI WS 06/07
• We start by writing the factors:
Dealing with EvidenceV S
LT
A B
X D
- Inference (Variable Elimination)
- Inference (Variable Elimination)
![Page 21: Graphical Models - Inference - Wolfram Burgard, Luc De Raedt, Kristian Kersting, Bernhard Nebel Albert-Ludwigs University Freiburg, Germany PCWP CO HRBP.](https://reader030.fdocuments.us/reader030/viewer/2022032800/56649d245503460f949fb285/html5/thumbnails/21.jpg)
Bayesian Networks
Bayesian Networks
AdvancedI WS 06/07
• We start by writing the factors:
• Since we know that V = t, we don’t need to eliminate V
• Instead, we can replace the factors P(V) and P(T|V) with
Dealing with EvidenceV S
LT
A B
X D
- Inference (Variable Elimination)
- Inference (Variable Elimination)
![Page 22: Graphical Models - Inference - Wolfram Burgard, Luc De Raedt, Kristian Kersting, Bernhard Nebel Albert-Ludwigs University Freiburg, Germany PCWP CO HRBP.](https://reader030.fdocuments.us/reader030/viewer/2022032800/56649d245503460f949fb285/html5/thumbnails/22.jpg)
Bayesian Networks
Bayesian Networks
AdvancedI WS 06/07
• We start by writing the factors:
• Since we know that V = t, we don’t need to eliminate V
• Instead, we can replace the factors P(V) and P(T|V) with
• This “selects” the appropriate parts of the original factors given the evidence
• Note that fP(v) is a constant, and thus does not appear in elimination of other variables
Dealing with EvidenceV S
LT
A B
X D
- Inference (Variable Elimination)
- Inference (Variable Elimination)
![Page 23: Graphical Models - Inference - Wolfram Burgard, Luc De Raedt, Kristian Kersting, Bernhard Nebel Albert-Ludwigs University Freiburg, Germany PCWP CO HRBP.](https://reader030.fdocuments.us/reader030/viewer/2022032800/56649d245503460f949fb285/html5/thumbnails/23.jpg)
Bayesian Networks
Bayesian Networks
AdvancedI WS 06/07
• Initial factors, after setting evidence
Dealing with EvidenceV S
LT
A B
X D
- Inference (Variable Elimination)
- Inference (Variable Elimination)
Given evidence V = t, S = f, D = t, compute P(L, V = t, S = f, D = t )
![Page 24: Graphical Models - Inference - Wolfram Burgard, Luc De Raedt, Kristian Kersting, Bernhard Nebel Albert-Ludwigs University Freiburg, Germany PCWP CO HRBP.](https://reader030.fdocuments.us/reader030/viewer/2022032800/56649d245503460f949fb285/html5/thumbnails/24.jpg)
Bayesian Networks
Bayesian Networks
AdvancedI WS 06/07
• Initial factors, after setting evidence
• Eliminating x, we get
Dealing with EvidenceV S
LT
A B
X D
- Inference (Variable Elimination)
- Inference (Variable Elimination)
Given evidence V = t, S = f, D = t, compute P(L, V = t, S = f, D = t )
![Page 25: Graphical Models - Inference - Wolfram Burgard, Luc De Raedt, Kristian Kersting, Bernhard Nebel Albert-Ludwigs University Freiburg, Germany PCWP CO HRBP.](https://reader030.fdocuments.us/reader030/viewer/2022032800/56649d245503460f949fb285/html5/thumbnails/25.jpg)
Bayesian Networks
Bayesian Networks
AdvancedI WS 06/07
• Initial factors, after setting evidence
• Eliminating x, we get
• Eliminating t, we get
Dealing with EvidenceV S
LT
A B
X D
- Inference (Variable Elimination)
- Inference (Variable Elimination)
Given evidence V = t, S = f, D = t, compute P(L, V = t, S = f, D = t )
![Page 26: Graphical Models - Inference - Wolfram Burgard, Luc De Raedt, Kristian Kersting, Bernhard Nebel Albert-Ludwigs University Freiburg, Germany PCWP CO HRBP.](https://reader030.fdocuments.us/reader030/viewer/2022032800/56649d245503460f949fb285/html5/thumbnails/26.jpg)
Bayesian Networks
Bayesian Networks
AdvancedI WS 06/07
• Initial factors, after setting evidence
• Eliminating x, we get
• Eliminating t, we get
• Eliminating a, we get
Dealing with EvidenceV S
LT
A B
X D
- Inference (Variable Elimination)
- Inference (Variable Elimination)
Given evidence V = t, S = f, D = t, compute P(L, V = t, S = f, D = t )
![Page 27: Graphical Models - Inference - Wolfram Burgard, Luc De Raedt, Kristian Kersting, Bernhard Nebel Albert-Ludwigs University Freiburg, Germany PCWP CO HRBP.](https://reader030.fdocuments.us/reader030/viewer/2022032800/56649d245503460f949fb285/html5/thumbnails/27.jpg)
Bayesian Networks
Bayesian Networks
AdvancedI WS 06/07
• Initial factors, after setting evidence
• Eliminating x, t, a, we get
• Finally, when eliminating b, we get
Dealing with EvidenceV S
LT
A B
X D
- Inference (Variable Elimination)
- Inference (Variable Elimination)
Given evidence V = t, S = f, D = t, compute P(L, V = t, S = f, D = t )
![Page 28: Graphical Models - Inference - Wolfram Burgard, Luc De Raedt, Kristian Kersting, Bernhard Nebel Albert-Ludwigs University Freiburg, Germany PCWP CO HRBP.](https://reader030.fdocuments.us/reader030/viewer/2022032800/56649d245503460f949fb285/html5/thumbnails/28.jpg)
Bayesian Networks
Bayesian Networks
AdvancedI WS 06/07
• Initial factors, after setting evidence
• Eliminating x, t, a, we get
• Finally, when eliminating b, we get
Dealing with EvidenceV S
LT
A B
X D
- Inference (Variable Elimination)
- Inference (Variable Elimination)
Given evidence V = t, S = f, D = t, compute P(L, V = t, S = f, D = t )
![Page 29: Graphical Models - Inference - Wolfram Burgard, Luc De Raedt, Kristian Kersting, Bernhard Nebel Albert-Ludwigs University Freiburg, Germany PCWP CO HRBP.](https://reader030.fdocuments.us/reader030/viewer/2022032800/56649d245503460f949fb285/html5/thumbnails/29.jpg)
Bayesian Networks
Bayesian Networks
AdvancedI WS 06/07 Outline
• Introduction • Reminder: Probability theory• Basics of Bayesian Networks• Modeling Bayesian networks• Inference (VE, Junction Tree)• Excourse: Markov Networks• Learning Bayesian networks• Relational Models