Understanding Independence: Common ancestorsUnderstanding Independence: Common ancestors alarm smoke...

Understanding Independence: Common ancestors

smokealarm

alarm and smoke are

dependent

alarm and smoke are

independent

given fire

Intuitively, fire canexplain alarm and smoke;learning one can affectthe other by changingyour belief in fire.

c©D. Poole and A. Mackworth 2019 Artificial Intelligence, Lecture 8.3 1 / 20

smokealarm

alarm and smoke aredependent

alarm and smoke are

independent

given fire

smokealarm

alarm and smoke are

independent

given fire

smokealarm

alarm and smoke areindependent given fire

smokealarm

alarm and smoke areindependent given fire

Understanding Independence: Chain

report

leaving

alarm and report are

dependent

independent

givenleaving

Intuitively, the only waythat the alarm affectsreport is by affectingleaving .

report

leaving

alarm and report aredependent

independent

givenleaving

report

leaving

independent

givenleaving

report

leaving

alarm and report areindependent givenleaving

report

leaving

alarm and report areindependent givenleaving

Understanding Independence: Common descendants

tampering

fire tampering and fire are

independent

tampering and fire are

dependent

given alarm

Intuitively, tamperingcan explain away fire

tampering

fire tampering and fire areindependent

dependent

given alarm

tampering

dependent

given alarm

tampering

tampering and fire aredependent given alarm

tampering

tampering and fire aredependent given alarm

Understanding independence: example

B CA D E F

I J K L

Understanding independence: questions

1. On which given probabilities does P(N) depend?

2. If you were to observe a value for B, which variables’probabilities will change?

3. If you were to observe a value for N, which variables’probabilities will change?

4. Suppose you had observed a value for M; if you were to thenobserve a value for N, which variables’ probabilities willchange?

5. Suppose you had observed B and Q; which variables’probabilities will change when you observe N?

What variables are affected by observing?

If you observe variable(s) Y , the variables whose posteriorprobability is different from their prior are:I The ancestors of Y andI their descendants.

Intuitively (if you have a causal belief network):I You do abduction to possible causes andI prediction from the causes.

d-separation

A connection is a meeting of arcs in a belief network. Aconnection is open is defined as follows:I If there are arcs A→ B and B → C such that B 6∈ Z , then the

connection at B between A and C is open.I If there are arcs B → A and B → C such that B 6∈ Z , then the

connection at B between A and C is open.I If there are arcs A→ B and C → B such that B (or a

descendent of B) is in Z , then the connection at B between Aand C is open.

X is d-connected from Y given Z if there is a path from X toY , along open connections.

X is d-separated from Y given Z if it is not d-connected.

X is independent Y given Z for all conditional probabilities iffX is d-separated from Y given Z

d-separation

Markov Random Field

A Markov random field is composed of

of a set of random variables: X = {X1,X2, . . . ,Xn} and

a set of factors {f1, . . . , fm}, where a factor is a non-negativefunction of a subset of the variables.

and defines a joint probability distribution:

P(X = x) ∝∏k

fk(Xk = xk) .

P(X = x) =1

fk(Xk = xk) .

Z =∑

fk(Xk = xk)

where fk(Xk) is a factor on Xk ⊆ X, andxk is x projected onto Xk .Z is a normalization constant known as the partition function.

Markov Random Field

P(X = x) ∝∏k

fk(Xk = xk) .

P(X = x) =1

fk(Xk = xk) .

Z =∑

fk(Xk = xk)

Markov Random Field

P(X = x) ∝∏k

fk(Xk = xk) .

P(X = x) =1

fk(Xk = xk) .

Z =∑

fk(Xk = xk)

Markov Networks and Factor graphs

A Markov network is a graphical representation of a Markovrandom field where the nodes are the random variables andthere is an arc between any two variables that are in a factortogether.

A factor graph is a bipartite graph, which contains a variablenode for each random variable and a factor node for eachfactor. There is an edge between a variable node and a factornode if the variable appears in the factor.

A belief network is a type of Markov random field where thefactors represent conditional probabilities, there is a factor foreach variable, and directed graph is acyclic.

A belief network is a

type of Markov random field where thefactors represent conditional probabilities, there is a factor foreach variable, and directed graph is acyclic.

Independence in a Markov Network

The Markov blanket of a variable X is the set of variables thatare in factors with X .

A variable is independent of the other variables given itsMarkov blanket.

X is connected to Y given Z if there is a path from X to Y inthe Markov network, which does not contain an element of Z .

X is separated from Y given Z if it is not connected.

A positive distribution is one that does not contain zeroprobabilities.

X is independent Y given Z for all positive distributions iff Xis separated from Y given Z

Canonical Representations

The parameters of a graphical model are the numbers thatdefine the model.

A belief network is a canonical representation: given thestructure and the distribution, the parameters are uniquelydetermined.

A Markov random field is not a canonical representation.Many different parameterizations result in the samedistribution.

Representations of Conditional Probabilities

There are many representations of conditional probabilities andfactors:

Tables

Decision Trees

Weighted Logical Formulae

Logistic Function

Neural network

Representations of Conditional Probabilities

There are many representations of conditional probabilities andfactors:

Tables

Decision Trees

Logistic Function

Neural network

Tabular Representation

P(D | A,B,C ) :

A B C D Probtrue true true true 0.9true true true false 0.1true true false true 0.9true true false false 0.1true false true true 0.2true false true false 0.8true false false true 0.2true false false false 0.8false true true true 0.3false true true false 0.7false true false true 0.4false true false false 0.6false false true true 0.3false false true false 0.7false false false true 0.4false false false false 0.6

Decision Tree Representation

P(d | A,B,C )

falsetrue

0.9 0.2 0.4

Rule Representation

0.9 : d ← a ∧ b

0.2 : d ← a ∧ ¬b

0.3 : d ← ¬a ∧ c

0.4 : d ← ¬a ∧ ¬c

d ↔ ((a ∧ b ∧ n0)

∨ (a ∧ ¬b ∧ n1)

∨ (¬a ∧ c ∧ n2)

∨ (¬a ∧ ¬c ∧ n3))

ni are independent:

P(n0) = 0.9

P(n1) = 0.2

P(n2) = 0.3

P(n3) = 0.4

Logistic Functions

P(h | e) =P(h ∧ e)

=P(h ∧ e)

P(h ∧ e) + P(¬h ∧ e)

1 + P(¬h ∧ e)/P(h ∧ e)

1 + e− logP(h∧e)/P(¬h∧e)

= sigmoid(log odds(h | e))

sigmoid(x) =1

1 + e−x

odds(h | e) =P(h ∧ e)

P(¬h ∧ e)

Logistic Functions

P(h | e) =P(h ∧ e)

=P(h ∧ e)

P(h ∧ e) + P(¬h ∧ e)

1 + P(¬h ∧ e)/P(h ∧ e)

sigmoid(x) =1

1 + e−x

P(¬h ∧ e)

Logistic Functions

P(h | e) =P(h ∧ e)

=P(h ∧ e)

P(h ∧ e) + P(¬h ∧ e)

1 + P(¬h ∧ e)/P(h ∧ e)

sigmoid(x) =1

1 + e−x

P(¬h ∧ e)

Logistic Functions

P(h | e) =P(h ∧ e)

=P(h ∧ e)

P(h ∧ e) + P(¬h ∧ e)

1 + P(¬h ∧ e)/P(h ∧ e)

sigmoid(x) =1

1 + e−x

P(¬h ∧ e)

Logistic Functions

P(h | e) =P(h ∧ e)

=P(h ∧ e)

P(h ∧ e) + P(¬h ∧ e)

1 + P(¬h ∧ e)/P(h ∧ e)

sigmoid(x) =1

1 + e−x

P(¬h ∧ e)

Logistic Functions

A conditional probability is the sigmoid of the log-odds.

00.10.20.30.40.50.60.70.80.9

-10 -5 0 5 10

1 + e- x

sigmoid(x) =1

1 + e−x

A logistic function is the sigmoid of a linear function.

Logistic Representation of Conditional Probability

P(d | A,B,C ) = sigmoid(0.9† ∗ A ∗ B

+ 0.2† ∗ A ∗ (1− B)

+ 0.3† ∗ (1− A) ∗ C

+ 0.4† ∗ (1− A) ∗ (1− C ))

where 0.9† is sigmoid−1(0.9).

P(d | A,B,C ) = sigmoid(0.4†

+ (0.2† − 0.4†) ∗ A

+ (0.9† − 0.2†) ∗ A ∗ B

Logistic Representation of Conditional Probability

P(d | A,B,C ) = sigmoid(0.9† ∗ A ∗ B

+ 0.2† ∗ A ∗ (1− B)

+ 0.3† ∗ (1− A) ∗ C

+ 0.4† ∗ (1− A) ∗ (1− C ))

where 0.9† is sigmoid−1(0.9).

P(d | A,B,C ) = sigmoid(0.4†

+ (0.2† − 0.4†) ∗ A

+ (0.9† − 0.2†) ∗ A ∗ B

Neural Network

Build a neural network to predict D from A, B, C

A neural network tries to predict expected values

The expected value of a variable with domain {0, 1} is itsprobability.Typically use a sigmoid activation at the root, and optimizewith log likelihood.— such a neural network is a logistic regression model withlearned features

For other discrete variables, the expected value is not theprobability.We create a Boolean ({0, 1}) variable for each value —indicator variable ≡ having an output for each value

For other domains, a Bayesian neural network can representthe distribution over the outputs (not just a point prediction).

Neural Network

The expected value of a variable with domain {0, 1} is itsprobability.

Typically use a sigmoid activation at the root, and optimizewith log likelihood.— such a neural network is a logistic regression model withlearned features

Neural Network

The expected value of a variable with domain {0, 1} is itsprobability.Typically use a sigmoid activation at the root, and optimizewith log likelihood.

— such a neural network is a logistic regression model withlearned features

Neural Network

For other discrete variables, the expected value is not theprobability.

We create a Boolean ({0, 1}) variable for each value —indicator variable ≡ having an output for each value

Neural Network

Understanding Independence: Common ancestorsUnderstanding Independence: Common ancestors alarm smoke...

Documents

Transcript of Understanding Independence: Common ancestorsUnderstanding Independence: Common ancestors alarm smoke...

USFA - US Residential Smoke Alarm Requirements

smart smoke alarm - cdn.webshopapp.com

Radio Wireless Interconnect SMOKE ALARM

Smoke Alarm Product Guide

Cavius Smoke Alarms - The World's Smallest Smoke Alarm - CAVIUS… · 2019-09-27 · Mains powered smoke alarm device Model number: 2203-002 Developed by CAVIUS™ This smoke alarm

Smoke alarm installation guide print

MARYLAND SMOKE ALARM TECHNOLOGY TASK FORCE: …

Smoke Alarm User’s Guide - Cable Organizer€¦ · Smoke Alarm User’s Guide Smoke Alarm Model Number (located on back of alarm): Date Code (located on back of alarm): The National

Smoke Alarm using Photodiode

EN Smoke alarm - cdn.webshopapp.com

Smoke Alarm Information Sheet - Kansas

Smoke Alarm User’s Guide - Carrier

SMOKE ALARM SUMMIT Residential Smoke Alarms: A Discussion ...

Smoke Alarm Installation Guide

Smoke Alarm Fraud Exposed Report - Tenants

Smoke Alarm Research at NIST - Vision 20/20 · Smoke Alarm Research at NIST Thomas G. Cleary Fire Research Division National Institute of Standards and Technology Smoke Alarm Summit

SMOKE ALARM - Safelincs

USER’S MANUAL INTRODUCTION Smoke & Carbon Monoxide Alarmbrkelectronics.com/pdfs/2008/10/13/491b55ae.pdf · Smoke & Carbon Monoxide Alarm AC Powered Smoke & Carbon Monoxide Alarm

USER’S MANUAL BEFORE YOU INSTALL THIS SMOKE ALARM … · • If the alarm sounds when no smoke is visible, try cleaning or relocating the smoke alarm. The cover may be dirty. •

Hidden Pictures - SparkyHidden Pictures ANSWER KEY smoke alarm birthday present smoke alarm smoke alarm birthday hat battery comb escape plan dog bone golf club pair of dice pencil