Discrete Random Events

Thurs. March 11, 1999– Review with Examples– Discrete Outcome Sample Spaces– Influence Diagrams & Probability Trees– Moments (e.g., expectation, variance, kth central) – Probability Functions (Marginal, Joint, Conditional)

Next week: Influence Diagrams (Tues.) and Dr. Anil Varma, Guest Speaker from GE’s AI/Expert Systems Group (Thurs.)

Example: Venn Diagram and Independence

We know from the Venn Diagram– Pr(A|B) = 1– Pr(A)– Pr(B)

Are A and B independent? Is Pr(AB)=Pr(A)Pr(B)?

B

A

A B

Disk Drive Failure Example

c (C, C’) x (X, X’)

Brand ‡ X X' TotalCrash C 0.6 0.1 0.7C' 0.2 0.1 0.3Total 0.8 0.2 1.0

Pr(C) =Pr(CI)

= Pr(CX) + Pr(CX’)

= 0.6 + 0.1 = 0.7

Pr(X) = Pr (XI)

=Pr(XC) + Pr(XC’)

0.6 + 0.2 = 0.8

Pr(C|X) =Pr(CX)/Pr(X)

= 0.6/0.8 = 0.75

Outcome Sample Spaces

It is useful in performing probability manipulations to define the random events "Ei" as unions of elementary mutually exclusive and collectively exhaustive events.

Outcome sample spaces are analogous to Venn diagrams where the universe I, is made up of these elementary mutually exclusive and collectively exhaustive events.

The probability of any event Ei is then the sum of the probabilities (due to the mutual exclusivity) of the elementary events which comprise Ei.

Talus Bone Example

Assume that we are to roll two "fair" talus bones; one is the heel bone of a sheep, the other is a heelbone of a deer. Each one has four sides, and if it is "fair" the likelihood of any one side coming up on a roll is equally likely.

Let us further assume that each side of the talus bone has dots on it one dot, two dots, three dots and four dots (similar to a conventional die but with four sides instead of six).

Talus Bone Events

Assume both the sheep and deer talus bones are thrown randomly on the ground. Define the events:– Si = the number of dots on the up side of the sheep talus

bone.– Dj = the number of dots on the up side of the deer talus bone.

Assume the throw of the sheep bone is independent of the throw of the deer bone, – Pr(SiDj ) = Pr(Si )Pr(Dj )

Sample Space for Joint Events

The joint events DiSj (i,j = 1, 2, 3, 4) represent the 16 possible outcomes independent rolls for each talus bone.

The probability of rolling any side is equal to 1/4 for each bone.

Thus Pr(DiSj ) = Pr(Di ) Pr(Sj ) = (1/4)(1/4) = 1/16

1

2

3

4

Deer Talus

Sheep Talus

1 2 3 4

D S

1 1

D S

2 1

D S

3 1

D S

4 1

D S

1 2

D S

2 2

D S

3 2

D S

4 2

D S

1 3

D S

2 3

D S

3 3

D S

4 3

D S

1 4

D S

2 4

D S

3 4

D S

4 4

Deer Talus

She

ep T

alus

Example: Find probability of E4 = the event that the sum of the two numbers is four.

1

2

3

4

Deer Talus

1 2 3 4

2 3 4 5

3 4 5 6

4 5 6 7

5 6 7 8

Sheep Talus

Because the events are mutually exclusive, the probability of event E4 is 1/16 times the number of ways a sum of four can occur.

There are three events that sum to four and thus:

Pr(E4) = 3(1/16) = 3/16

Deer Talus

She

ep T

alus

Example: F<10 = the event that the product of the two numbers is less than ten

Because the events are mutually exclusive, the probability of event F<10 is 1/16 times the number of ways a product less than ten can occur.

There are thirteen events with a product less than ten and thus:

Pr( F<10 ) = 13(1/16) = 13/16

Deer Talus

She

ep T

alus

What if the Talus Bones are Biased?

Pr(D )

Pr(S )

i

i

Suppose the probability of getting four dots is higher than getting three and so on as defined by the probability distribution:

2i-1

Pr(Si) = Pr(Di) = 16

Probability Mass Functionon Si and Di

Biased Talus Bones - Probability Mass Function?

Pr(D )

Pr(S )

i

i

Because these events are mutually exclusive and collectively exhaustive, the sum of the probabilities of all events Si or Di equal one. Consider the sheep talus:

∑i=1,4 Pr(Si) = 1 =

1 + 3 + 5 + 7

16Probability Mass Function

on Si and Di

Biased Talus Bones - Joint Probabilities

Pr(D )

Pr(S )

i

i

Probability Mass Functionon Si and Di

Deer Talus

She

ep T

alus

Joint Probabilities, Pr(Si, Di) x 256

Conditional Probability Expansions of Talus Bone Example

How would one represent events E4 and F<10 as a conditional probability expansion? There are many possible and consistent representations.

If you find it natural to think of the sum of the outcomes of the tosses to be influenced by the outcome of each individual toss, state E would be expressed mathematically by the following expansion: Pr(E ) = Pr(Sum(Di and Sj) ) =Pr( Sum(Di and Sj) | Di ,Sj ) Pr( Di | Sj) Pr(Sj )

Because Di and Sj are assumed to be independent, this reduces to: Pr(E) =Pr(Sum(Di and Sj)) =Pr( Sum(Di and Sj) | Di ,Sj ) Pr( Di) Pr(Sj )

Influence Diagrams & Talus Bone Example: E4

Because Di and Sj are assumed to be independent, this reduces to:

Pr(E) =Pr(Sum(Di and Sj)) =Pr( Sum(Di and Sj) | Di ,Sj ) Pr( Di) Pr(Sj )

Sum

D

S

E

Influence Diagrams & F<10 Example:

Pr(F) = Pr(Product(Di and Sj) )

=Pr( Product(Di and Sj) | Di ,Sj) Pr( Di | Sj) Pr(Sij)

Because Di and Sj are assumed to be independent, this reduces to:

Pr( Product(Di and Sj) | Di ,Sj) Pr( Di ) Pr(Sj )

D

S

F

Product

How do we represent both events E and F in the same

influence diagram?

Both diagrams show Di and Sj as being independent, which would remain unchanged for the combined diagram. But should there be an arc between E and F?

The question to be answered is: given we know the outcome of Di and Sj, would knowing the product give us any new information about the sum and vice versa. The answer is NO, and thus there is no arc between E and F. E and F are said to be conditionally independent of each other given Di and Sj.

E and F are said to be conditionally independent of each other given Di and Sj

Sum

D

S

FE

Product

Given we know the outcome of Di and Sj, knowing the product give us any new information about the sum and vice versa.

The lack of an arc reveals this conditional independence graphically.

Are E and F independent if we do not conditional information on D and S?

Given we know the outcome of Di and Sj, knowing the product give us any new information about the sum and vice versa.

The lack of an arc reveals this conditional independence graphically.

ESum

FProduct

Probability Tree Representation

1

Sheep Talus Outcome

Pr(S | H)

Deer Talus Outcome

Pr(D | S, H)

1

2

3

4

2

3

4

1

2

3

4

1

2

3

4

1

2

3

4

Sample Point Probability

Pr(S D | H} = Pr(S | H) • Pr(D | H)1 1

S D1 1 1 1

Pr(S D | H} = Pr(S | H) • Pr(D | H)1 2

S D1 2 1 2

Pr(S D | H} = Pr(S | H) • Pr(D | H)1 3

S D1 3 1 3

Pr(S D | H} = Pr(S | H) • Pr(D | H)1 4

S D1 4 1 4

Pr(S D | H} = Pr(S | H) • Pr(D | H)2 1

S D2 1 2 1

Pr(S D | H} = Pr(S | H) • Pr(D | H)2 2

S D2 2 2 2

Pr(S D | H} = Pr(S | H) • Pr(D | H)2 3

S D2 3 2 3

Pr(S D | H} = Pr(S | H) • Pr(D | H)2 4

S D2 4 2 4

Pr(S D | H} = Pr(S | H) • Pr(D | H)3 1

S D3 1 3 1

Pr(S D | H} = Pr(S | H) • Pr(D | H)3 2

S D3 2 3 2

Pr(S D | H} = Pr(S | H) • Pr(D | H)3 3

S D3 3 3 3

Pr(S D | H} = Pr(S | H) • Pr(D | H)3 4

S D3 4 3 4

Pr(S D | H} = Pr(S | H) • Pr(D | H)4 1

S D4 1 4 1

Pr(S D | H} = Pr(S | H) • Pr(D | H)4 2

S D4 2 4 2

Pr(S D | H} = Pr(S | H) • Pr(D | H)4 3

S D4 3 4 3

Pr(S D | H} = Pr(S | H) • Pr(D | H)4 4

S D4 4 4 4

Expectation of a Random Variable

E(x|H) = ∑ xi Pr(xi)

xi

or E(x|H) = <x|H>

Expectation of a Function of a Random Variable

Let g(x) be any single-valued function of the random variable x.

The expected value of g(x) is designated as E[g(x)] and defined as:

E[g(x)] = ∑ g(xi)Pr(x=xi) = <g(x)>

Is E[g(x)] = g[E(x)]?

Probability Mass Function, Pr(x)

The probability that the random variable x takes on the discrete value xo is defined by means of the probability mass function Pr(x) with the notation below:

Pr(x) = Pr(x=xi) for all xi

i

1/16

3/16

5/16

7/16

Pr(S | H)i

S

1

21 3 4

Biased Talus Bone Example

Cumulative Probability Distribution, Pr(x≤y|y)

Pr(x≤y|y)

= ∑ x≤y Pr(x)

= Pr(x≤y|y=yi) for all yi1/16

4/16

iPr(S ≤ z | z, H)

9/16

16/16

z

1

21 3 4

Biased Talus Bone Example

Cumulative Probability is Summation (Integral) of the Mass Function

1/16

4/16

iPr(S ≤ z | z, H)

9/16

16/16

z

1

21 3 4

i

1/16

3/16

5/16

7/16

Pr(S | H)i

S

1

21 3 4

Probability Mass Function Cumulative Probability Distribution

Complementary Probability Distribution, Pr(x>y|y)

Pr(x>y|y)

= 1 - Pr(x≤y|y)

iPr(S > z | z, H)

z

115/16

12/16

7/16

21 3 4

Biased Talus Bone Example

Joint Discrete Mass Functions

1

2

3

4

Di

Pr(D S | H)

x256

i i

49

35

35

25

21

21

15

15

9

7

7

5

5

3

3

1

S

i

21 3 4

1 3 5 7

3 9 15 21

5 25 3515

7 35 4921

D

i

S

i

21 3 4

2

1

3

4

Two Representations for Joint Probability Mass Function for Two Biased Talus Tosses

Bayesian Pair of Bolts

Suppose that a "pick and place" robot with two manipulators has been designed to pick two bolts of the same size from a bin and place each pair in separate containers on a belt for assembly. Suppose also that there are three different kinds of pairs of bolts that are used in the assembly of this product:

Pair 1 two bolts both made of alloy A

Pair 2 two bolts both made of alloy B

Pair 3 one of alloy A and the other of alloy B

Bayesian Pair of Bolts: Bayes Theorem

The alloys look identical and the only way to determine whether one is A or B requires performing a test.

Suppose that one of the pairs of bolts falls on the ground and the operator picks it up. The probability of getting any specific pair is 1/3.

Now suppose she randomly picks one of the bolts from the pair, performs a test on it and determines that it is made of alloy A.

What is the probability that the other bolt is of alloy A?

Bayesian Pair of Bolts: Define Events

A common erroneous answer to this question is 1/2. But this is incorrect! Define the following events:– P1: The event that pair 1 was picked.– P2: The event that pair 2 was picked.– P3: The event that pair 3 was picked.– A1: The event that the first member of the pair is of alloy A.– A2: The event that the second member of the pair is of alloy A.– B1: The event that the first member of the pair is of alloy B.– B2: The event that the second member of the pair is of alloy B.

What we want to know is Pr(A2|A1,S). We know that Pr(P1|S) = Pr(P2|S) = Pr(P3|S) = 1/3. Using Bayes' theorem and conditional probability find Pr(A2|A1,S).