CHAPTER 3 Probability Theory (Abridged Version) B asic Definitions and Properties C onditional...

CHAPTER 3

Probability Theory (Abridged Version)

Basic Definitions and Properties Conditional Probability and Independence Bayes’ Formula

2

POPULATION

Random variable X

SAMPLE of size n

x1

x2

x3

x4

x5

x6

…etc….xn

Data xi

Relative Frequenciesf (xi ) = fi /n

x1 f (x1)

x2 f (x2)

x3 f (x3)

⋮ ⋮

xk f (xk)

1

Frequency Table

Density Histogram

X

Total Area = 1

)()(

)(

xfxxs

xfxx

nn 22

1

???Probability Table Probability Histogram

… at least if X is discrete.

(Chapter 4)

3

20%

20%

20%

20%

20%

Outcome

Red

Orange

Yellow

Green

Blue

• Sample Space The set of all possible outcomes of an experiment.

Definitions

Perform repeated trials of the following experiment: Randomly select an individual from the population, and record its color.

POPULATION (Pie Chart)

• An outcome is the result of an experiment on a population.

S = {Red, Orange, Yellow, Green, Blue}

Venn Diagram

• Event Any subset of S (including the empty set , and S itself).

E = “Primary Color” = {Red, Yellow, Blue}

(using basic Set Theory)

Red

Yellow

Green

Orange

Blue

#(S) = 5

#(E) = 3 ways

E

4

20%

20%

20%

20%

20%• Sample Space The set of all possible outcomes of an experiment.

Definitions




S = {Red, Orange, Yellow, Green, Blue}.

Venn Diagram



F = “Hot Color” = {Red, Orange, Yellow}


Red

Yellow

Green

Orange

Blue

#(S) = 5

#(E) = 3 ways

#(F) = 3 ways

F

Outcome

Red

Orange

Yellow

Green

Blue

5

20%

20%

20%

20%


Definitions





Venn Diagram





“Cold Color” = {Green, Blue}“Not F” =Complement F C =

Red

Yellow

Green

Orange

Blue

#(S) = 5

#(E) = 3 ways

#(F) = 3 ways

F

Outcome

Red

Orange

Yellow

Green

Blue

#(FC) = 2 ways

6

20%

20%

20%

20%


Definitions





Venn Diagram






Red

Yellow

Green

Orange

Blue

#(S) = 5

#(E) = 3 ways

#(F) = 3 ways

F

Outcome

Red

Orange

Yellow

Green

Blue

#(FC) = 2 ways

7

20%

20%

20%

20%


Definitions





Venn Diagram






Red

Yellow

Green

Orange

Blue

#(S) = 5

#(E) = 3 ways

#(F) = 3 ways

E

F

Intersection E ⋂ F =

{Red, Yellow}“E and F” =

Outcome

Red

Orange

Yellow

Green

Blue

#(FC) = 2 ways

8

20%

20%

20%

20%


Definitions





Venn Diagram






Red

Yellow

Green

Orange

Blue

#(S) = 5

#(E) = 3 ways

#(F) = 3 ways

E

F


{Red, Yellow}“E and F” = #(E ⋂ F) = 2

Outcome

Red

Orange

Yellow

Green

Blue

#(FC) = 2 ways

9

20%

20%

20%

20%


Definitions





Venn Diagram






Red

Yellow

Green

Orange

Blue

#(S) = 5

#(E) = 3 ways

#(F) = 3 ways

A B



Outcome

Red

Orange

Yellow

Green

Blue Note: A = {Red, Green} ⋂ B = {Orange, Blue} = A and B are disjoint, or mutually exclusive events

#(FC) = 2 ways

10

20%

20%

20%

20%


Definitions





Venn Diagram






Red

Yellow

Green

Orange

Blue

#(S) = 5

#(E) = 3 ways

#(F) = 3 ways



Outcome

Red

Orange

Yellow

Green


#(FC) = 2 ways

“E or F” =

E

F

11

20%

20%

20%

20%


Definitions





Venn Diagram






Red

Yellow

Green

Orange

Blue

#(S) = 5

#(E) = 3 ways

#(F) = 3 ways



Outcome

Red

Orange

Yellow

Green


#(FC) = 2 ways

Union E ⋃ F =

{Red, Orange, Yellow, Blue}“E or F” =

E

F

#(E ⋃ F) = 4

12

A

B

A ⋂ BA ⋂ Bc Ac ⋂ B

“A only” “B only”

Ac ⋂ Bc

“Neither A nor B ”

“A and B”

In general, for two events A and B…

13

20%

20%

20%

20%


Definitions





Venn Diagram






Red

Yellow

Green

Orange

Blue

#(S) = 5

#(E) = 3 ways

#(F) = 3 ways



Outcome

Red

Orange

Yellow

Green


#(FC) = 2 ways

Union E ⋃ F =

{Red, Orange, Yellow, Blue}“E or F” =

E

F

#(E ⋃ F) = 4

What about probability?


14

20%

20%

20%

20%


DefinitionsPOPULATION (Pie Chart)



Venn Diagram






#(S) = 5

#(E) = 3 ways

#(F) = 3 ways


{Red, Yellow}“E and F“ = #(E ⋂ F) = 2Note: A = {Red, Green} ⋂ B = {Orange, Blue} =

A and B are disjoint, or mutually exclusive events

#(FC) = 2 ways

Union E ⋃ F =

{Red, Orange, Yellow, Blue}“E or F” =#(E ⋃ F) =

4

Outcome Probability

Red 0.20

Orange 0.20

Yellow 0.20

Green 0.20

Blue 0.20

1.00

Red

Yellow

Green

Orange

Blue

E

F

“The probability of Red is equal to 0.20”

P(Red) = 0.20

# trials

…

# Red# trials

…… But what does it mean??

What happens to this “long run” relative frequency as # trials → ∞?

All probs are > 0, and sum = 1.

15

20%

20%

20%

20%


Definitions





Venn Diagram






#(S) = 5

#(E) = 3 ways

#(F) = 3 ways




#(FC) = 2 ways

Union E ⋃ F =

{Red, Orange, Yellow, Blue}“E or F“ =#(E ⋃ F) =

4

Outcome Probability

Red 0.20

Orange 0.20

Yellow 0.20

Green 0.20

Blue 0.20

1.00

Red

Yellow

Green

Orange

Blue

E

F

What about probability of events?

For any event E,

P(E) = P(Outcomes in E).

BUT…

General Fact:


16

20%

20%

20%

20%


Definitions





Venn Diagram






#(S) = 5

#(E) = 3 ways

#(F) = 3 ways




#(FC) = 2 ways

Union E ⋃ F =


4

Outcome Probability

Red 0.20

Orange 0.20

Yellow 0.20

Green 0.20

Blue 0.20

1.00

Red

Yellow

Green

Orange

Blue

E

F

These outcomes are said to be “equally likely.”

When this is the case, P(E) = #(E) / #(S), for any event E in the sample space S.


P() = 0


17

20%

20%

20%

20%


Definitions





Venn Diagram






#(S) = 5

#(E) = 3 ways

#(F) = 3 ways


{Red, Yellow}“E and F” = #(E ⋂ F) = 2Note: A = {Red, Green} ⋂ B = {Orange, Blue} =


#(FC) = 2 ways

Union E ⋃ F =


4

Outcome Probability

Red 0.20

Orange 0.20

Yellow 0.20

Green 0.20

Blue 0.20

1.00

Red

Yellow

Green

Orange

Blue

E

F

P(E) = 3/5 = 0.6

P(F) = 3/5 = 0.6

P(FC) = 2/5 = 0.4

P(E ⋂ F) = 0.4

P(E ⋃ F) = 4/5 = 0.8


P() = 0


18

20%

20%

20%

20%


Definitions





Venn Diagram






#(S) = 5



Note: A = {Red, Green} ⋂ B = {Orange, Blue} = A and B are disjoint, or mutually exclusive events

Union E ⋃ F =

{Red, Orange, Yellow, Blue}“E or F“ =

Outcome Probability

Red 0.20

Orange 0.20

Yellow 0.20

Green 0.20

Blue 0.20

1.00

Red

Yellow

Green

Orange

Blue

E

F

P(E) = 3/5 = 0.6

P(F) = 3/5 = 0.6

P(FC) = 2/5 = 0.4

P(E ⋂ F) = 0.4

P(E ⋃ F) = 4/5 = 0.8

15%

20%

25%

30%

10%

Outcome Probability

Red 0.10

Orange 0.15

Yellow 0.20

Green 0.25

Blue 0.30

1.00

These outcomes are NOT “equally likely.”


Outcome Probability

Red 0.10

Orange 0.15

Yellow 0.20

Green 0.25

Blue 0.30

1.00

P() = 0

19

20%

20%

20%

20%


Definitions





Venn Diagram









Union E ⋃ F =


Red

Yellow

Green

Orange

Blue

E

F

15%

20%

25%

30%

10%

P(E) = 0.60


Outcome Probability

Red 0.10

Orange 0.15

Yellow 0.20

Green 0.25

Blue 0.30

1.00

P() = 0

20

20%

20%

20%

20%


Definitions





Venn Diagram









Union E ⋃ F =


Red

Yellow

Green

Orange

Blue

E

F

15%

20%

25%

30%

10%

P(E) = 0.60

P(F) = 0.45


Outcome Probability

Red 0.10

Orange 0.15

Yellow 0.20

Green 0.25

Blue 0.30

1.00

P() = 0

21

20%

20%

20%

20%


Definitions





Venn Diagram









Union E ⋃ F =


Red

Yellow

Green

Orange

Blue

E

F

15%

20%

25%

30%

10%

P(E) = 0.60

P(F) = 0.45

P(FC) = 1 – P(F) = 0.55


Outcome Probability

Red 0.10

Orange 0.15

Yellow 0.20

Green 0.25

Blue 0.30

1.00

P() = 0

22

20%

20%

20%

20%


Definitions





Venn Diagram









Union E ⋃ F =


Red

Yellow

Green

Orange

Blue

E

F

15%

20%

25%

30%

10%

P(E) = 0.60

P(F) = 0.45

P(E ⋂ F) = 0.3

P(FC) = 1 – P(F) = 0.55


Outcome Probability

Red 0.10

Orange 0.15

Yellow 0.20

Green 0.25

Blue 0.30

1.00

P() = 0

23

20%

20%

20%

20%


Definitions





Venn Diagram









Union E ⋃ F =


Red

Yellow

Green

Orange

Blue

E

F

15%

20%

25%

30%

10%

P(E) = 0.60

P(F) = 0.45

P(E ⋂ F) = 0.3

P(E ⋃ F) = 0.75

P(FC) = 1 – P(F) = 0.55


Outcome Probability

Red 0.10

Orange 0.15

Yellow 0.20

Green 0.25

Blue 0.30

1.00

P() = 0

24

20%

20%

20%

20%


Definitions





Venn Diagram









Union E ⋃ F =


Red

Yellow

Green

Orange

Blue

E

F

15%

20%

25%

30%

10%

P(E) = 0.60

P(F) = 0.45

P(E ⋂ F) = 0.3

P(E ⋃ F) = 0.75

P(E ⋃ F) =

P(FC) = 1 – P(F) = 0.55


P(E ⋃ F) =

Outcome Probability

Red 0.10

Orange 0.15

Yellow 0.20

Green 0.25

Blue 0.30

1.00

P() = 0

25

20%

20%

20%

20%


Definitions





Venn Diagram









Union E ⋃ F =


Red

Yellow

Green

Orange

Blue

E

F

15%

20%

25%

30%

10%

P(E) = 0.60

P(F) = 0.45

P(E ⋂ F) = 0.3

P(E ⋃ F) = 0.75

P(E ⋃ F) = P(E)

P(FC) = 1 – P(F) = 0.55


P(E ⋃ F) = P(E)

Outcome Probability

Red 0.10

Orange 0.15

Yellow 0.20

Green 0.25

Blue 0.30

1.00

P() = 0

26

20%

20%

20%

20%


Definitions





Venn Diagram









Union E ⋃ F =


15%

20%

25%

30%

10%

P(E) = 0.60

P(F) = 0.45

P(E ⋂ F) = 0.3

P(E ⋃ F) = 0.75

P(FC) = 1 – P(F) = 0.55

Red

Yellow

Green

Orange

Blue

E

F

P(E ⋃ F) = P(E) + P(F)


P(E ⋃ F) = P(E) + P(F)

Outcome Probability

Red 0.10

Orange 0.15

Yellow 0.20

Green 0.25

Blue 0.30

1.00

P(E ⋃ F) = 0.75

P() = 0

20%

20%

20%

20%


Definitions





Venn Diagram









Union E ⋃ F =

“E or F“ =

15%

20%

25%

30%

10%

P(E) = 0.60

P(F) = 0.45

P(E ⋂ F) = 0.3

P(FC) = 1 – P(F) = 0.55

Red

Yellow

Green

Orange

Blue

E

F

P(E ⋃ F) = P(E) + P(F) – P(E ⋂ F)

{Red, Orange, Yellow, Blue}All probs are > 0, and sum = 1.

P(E ⋃ F) = P(E) + P(F) – P(E ⋂ F)

Outcome Probability

Red 0.10

Orange 0.15

Yellow 0.20

Green 0.25

Blue 0.30

1.00

P(E ⋃ F) = 0.75

P() = 0

20%

20%

20%

20%


Definitions





Venn Diagram









Union E ⋃ F =

“E or F“ =

15%

20%

25%

30%

10%

P(E) = 0.60

P(F) = 0.45

P(E ⋂ F) = 0.3

P(FC) = 1 – P(F) = 0.55

Red

Yellow

Green

Orange

Blue

E

F

P(E ⋃ F) = P(E) + P(F) – P(E ⋂ F) = 0.60 + 0.45 – 0.30


Outcome Probability

Red 0.10

Orange 0.15

Yellow 0.20

Green 0.25

Blue 0.30

1.00

P(E ⋃ F) = 0.75

P() = 0

20%

20%

20%

20%


Definitions





Venn Diagram









Union E ⋃ F =

“E or F“ =

15%

20%

25%

30%

10%

P(E) = 0.60

P(F) = 0.45

P(E ⋂ F) = 0.3

P(FC) = 1 – P(F) = 0.55

Red

Yellow

Green

Orange

Blue

E

F

P(E ⋃ F) = P(E) + P(F) – P(E ⋂ F) = 0.60 + 0.45 – 0.30


E EC

F P(F)FC P(FC)

P(E) P(EC) 1.0

Outcome Probability

Red 0.10

Orange 0.15

Yellow 0.20

Green 0.25

Blue 0.30

1.00

20%

20%

20%

20%


Definitions





Venn Diagram





15%

20%

25%

30%

10%

P(E) = 0.60

P(F) = 0.45

Red

Yellow

Green

Orange

Blue

E

F

E EC

F P(E ⋂ F) P(EC ⋂ F) P(F)FC P(E ⋂ FC) P(EC ⋂ FC) P(FC)

P(E) P(EC) 1.0

Probability Table


Outcome Probability

Red 0.10

Orange 0.15

Yellow 0.20

Green 0.25

Blue 0.30

1.00

20%

20%

20%

20%


Definitions





Venn Diagram





15%

20%

25%

30%

10%

P(E) = 0.60

P(F) = 0.45

Red

Yellow

Green

Orange

Blue

E

F

E EC

F 0.30 0.15 0.45

FC 0.30 0.25 0.55

0.60 0.40 1.0

Probability Table


Outcome Probability

Red 0.10

Orange 0.15

Yellow 0.20

Green 0.25

Blue 0.30

1.00

20%

20%

20%

20%


Definitions









15%

20%

25%

30%

10%

P(E) = 0.60

P(F) = 0.45

0.15

0.30

0.250.30

E

F

E EC

F 0.30 0.15 0.45

FC 0.30 0.25 0.55

0.60 0.40 1.0

Probability Table

Venn Diagram


~ Summary of Basic Properties of Probability ~Population Hypothesis Experiment Sample space 𝓢 of possible outcomes

Event E ⊆ 𝓢 Probability P(E) = ?

• Def: P(E) = “limiting value” of as experiment is repeated indefinitely.

• P(E) = P(outcomes) = always a number between 0 and 1. (That is, 0 ≤ P(E) ≤ 1.)

• If AND ONLY IF all outcomes in are 𝓢 equally likely, then P(E) =

• If E and F are any two events, then so are the following:

33

trials

occursEtimes

#

#

.)(#

)(# Sinoutcomes

Einoutcomes

Event Description Notation Terminology Probab

Not E “E does not occur.”complement

of E1 – P(E)

E and F“Both E and F occur

simultaneously.” E ⋂ F intersection of E and F -

E or F“Either E occurs, or F occurs (or both).” E ⋃ F union

of E and FP(E) + P(F) – P(E ⋂ F)

,,EEC

��

EC

E

E F

E F

FE

“If E occurs, then F occurs.” E ⊆ F E is a subset

of FP(E ⋂

F)

What percentage receives T1 only?

Example: Two treatments exist for a certain disease, which can either be taken separately or in combination. Suppose:

70% of patient population receives T1

50% of patient population receives T2

30% of patient population receives both T1 and T2

34

T1 T2

T1c ⋂ T2 T1 ⋂ T2

c

T1 ⋂ T2 (w/ or w/o T2)

(w/ or w/o T1)

(w/o T2)

P(T2) = 0.5 P(T1 ⋂ T2) = 0.3

P(T1 ⋂ T2c) = 0.7 – 0.3 = 0.4…. i.e., 40%

What percentage receives T2 only? (w/o T1)

P(T1c ⋂ T2) = 0.5 – 0.3 = 0.2…. i.e., 20%

What percentage receives neither T1 nor T2?

P(T1c ⋂ T2

c) = 1 – (0.4 + 0.3 + 0.2) = 0.1…. i.e., 10%

T1c ⋂ T2

c

0.3

0.4

0.2

0.1

P(T1) = 0.7

T1 T1c

T2

T2c

1.0

0.7 0.30.5

0.4

0.2

0.1

0.3

0.5

Column marginal sums

Row marginal sums

35

A

B

C

A ⋂ B ⋂ C

A ⋂ B ⋂ Cc

A ⋂ Bc⋂ C Ac ⋂ B ⋂ C

A ⋂ Bc ⋂ Cc Ac ⋂ B ⋂ Cc

Ac ⋂ Bc ⋂ C

“A only”

“C only”

“B only”

Ac ⋂ Bc ⋂ Cc

“Neither A nor B nor C”

In general, for three events A, B, and C…

CHAPTER 3

Probability Theory (Abridged Version)


Probability of “Primary Color,” given “Hot Color” = ?

E EC

F 0.30 0.15 0.45

FC 0.30 0.25 0.55

0.60 0.40 1.0

Outcome Probability

Red 0.10

Orange 0.15

Yellow 0.20

Green 0.25

Blue 0.30

1.00

POPULATIONE = “Primary Color” = {Red, Yellow, Blue}


15%

20%

25%

30%

10%

P(E) = 0.60

P(F) = 0.45

Probability Table

Venn Diagram

0.15

0.30

0.250.30

E

F

Blue

Green

Orange

RedYellow

0.15

0.30

0.250.30

E

F

Blue

Green

Orange

RedYellow


E EC

F 0.30 0.15 0.45

FC 0.30 0.25 0.55

0.60 0.40 1.0



P(E) = 0.60

P(F) = 0.45

Probability Table

Venn Diagram

0.15

0.30

0.250.30

E

F

Blue

Green

Orange

RedYellow

Outcome Probability

Red 0.10

Orange 0.15

Yellow 0.20

Green 0.25

Blue 0.30

1.00

POPULATION

15%

20%

25%

30%

10%

P(E | F)( )

( )

P E F

P F

0.30

0.45 0.667

Conditional Probability

=

P(F | E) ( )

( )

P F E

P E

E EC

F 0.30 0.15 0.45

FC 0.30 0.25 0.55

0.60 0.40 1.0



P(E) = 0.60

P(F) = 0.45

Probability Table

Venn Diagram

0.15

0.30

0.250.30

E

F

Blue

Green

Orange

RedYellow


P(E | F)( )

( )

P E F

P F

0.30

0.45 0.667

Outcome Probability

Red 0.10

Orange 0.15

Yellow 0.20

Green 0.25

Blue 0.30

1.00

POPULATION

15%

20%

25%

30%

10%


P(F | E) 0.30

0.60 0.5( )

( )

P F E

P E

=

E EC

F 0.30 0.15 0.45

FC 0.30 0.25 0.55

0.60 0.40 1.0



P(E) = 0.60

P(F) = 0.45

Probability Table

Venn Diagram

0.15

0.30

0.250.30

E

F

Blue

Green

Orange

RedYellow


P(E | F) P(EC | F)( )

( )

P E F

P F

0.30

0.45 0.667 = 1 – 0.667 = 0.333

Outcome Probability

Red 0.10

Orange 0.15

Yellow 0.20

Green 0.25

Blue 0.30

1.00

POPULATION

15%

20%

25%

30%

10%


P(F | E) 0.30

0.60 0.5( )

( )

P F E

P E

E EC

F 0.30 0.15 0.45

FC 0.30 0.25 0.55

0.60 0.40 1.0



P(E) = 0.60

P(F) = 0.45

Probability Table

Venn Diagram

0.15

0.30

0.250.30

E

F

Blue

Green

Orange

RedYellow


P(E | F) P(EC | F)

P(E | FC)

( )

( )

P E F

P F

0.30

0.45 0.667 = 1 – 0.667 = 0.333

Outcome Probability

Red 0.10

Orange 0.15

Yellow 0.20

Green 0.25

Blue 0.30

1.00

POPULATION

15%

20%

25%

30%

10%


P(F | E) 0.30

0.60 0.5( )

( )

P F E

P E

0.30

0.55 0.545

RedYellow

42

Def: The conditional probability of event A, given event B, is denoted by P(A|B), and calculated via the formula

.

)(

)()|(

BP

BAPBAP

Thus, for any two events A and B, it follows that P(A ⋂ B) = P(A | B) × P(B).

B occurs with prob P(B) Given that B occurs, A occurs with prob P(A | B) Both A and B occur, with

prob P(A ⋂ B) Example: P(Live to 75) × P(Live to 80 | Live to 75) = P(Live to 80)

Tree Diagrams

P(B)

P(Bc)

P(A | B)

P(Ac | B)

P(A | Bc)

P(Ac | Bc)

P(A ⋂ B)

P(Ac ⋂ B)

P(A ⋂ Bc)

P(Ac ⋂ Bc)

Event A Ac

B P(A ⋂ B) P(Ac ⋂ B)

Bc P(A ⋂ Bc) P(Ac ⋂ Bc)

A B

A ⋂ BA ⋂ Bc Ac ⋂ B

Ac ⋂ Bc

Multiply together “branch probabilities” to obtain “intersection probabilities”

A

B

43

Example:

Bob must take two trains to his home in Manhattan after work: the A and the B, in either order. At 5:00 PM…• The A train arrives first with probability 0.65, and takes 30 mins to reach its last stop at Times Square. • The B train arrives first with probability 0.35, and takes 30 mins to reach its last stop at Grand Central Station.• At Times Square, Bob exits, and catches the second train. The A arrives first with probability 0.4, then travels to Brooklyn. The B train arrives first with probability 0.6, and takes 30 minutes to reach a station near his home.• At Grand Central Station, the A train arrives first with probability 0.8, and takes 30 minutes to reach a station near his home. The B train arrives first with probability 0.2, then travels to Queens.

With what probability will Bob be exiting the subway at 6:00 PM?

44

Example:

1( )=P A

1( )=P B

1A

5:00 5:30 6:00

1B

2 1( | )=P A A

2 1( | )=P B A

2 1( | )=P A B

2 1( | )=P B B

1 2( )=P A A

1 2( )=P A B

1 2( )=P B A

1 2( )=P B B

0.65

0.35

0.4

0.6

0.8

0.2

MULTIPLY:

0.26

0.39

0.28

0.07

ADD:

0.67

Bob must take two trains to his home in Manhattan after work: the A and the B, in either order. At 5:00 PM…• The A train arrives first with probability 0.65, and takes 30 mins to reach its last stop at Times Square. • The B train arrives first with probability 0.35, and takes 30 mins to reach its last stop at Grand Central Station.• At Times Square, Bob exits, and catches the second train. The A arrives first with probability 0.4, then travels to Brooklyn. The B train arrives first with probability 0.6, and takes 30 minutes to reach a station near his home.• At Grand Central Station, the A train arrives first with probability 0.8, and takes 30 minutes to reach a station near his home. The B train arrives first with probability 0.2, then travels to Queens.

With what probability will Bob be exiting the subway at 6:00 PM?

45

15%

20%

25%

30%

10%

POPULATION

Outcome Probability

Red 0.10

Orange 0.18

Yellow 0.17

Green 0.22

Blue 0.33

1.00

POPULATION

18%

17%

22%

33%

10%

E EC

F 0.27 0.18 0.45

FC 0.33 0.22 0.55

0.60 0.40 1.0

Probability Table

Venn Diagram

0.18

0.27

0.220.33

E

F

Blue

Green

Orange

RedYellow

0.27

0.45

E EC

F 0.27 0.18 0.45

FC 0.33 0.22 0.55

0.60 0.40 1.0



P(E) = 0.60

Probability Table

Venn Diagram

0.18

0.27

0.220.33

E

F

Blue

Green

Orange

RedYellow

Outcome Probability

Red 0.10

Orange 0.18

Yellow 0.17

Green 0.22

Blue 0.33

1.00

POPULATION


P(E | F) ( )

( )

P E F

P F

P(F | E) ( )

( )

P F E

P E

18%

17%

22%

33%

10%

0.60 = P(E)

0.27

0.600.45 = P(F)

P(F) = 0.45

E EC

F 0.27 0.18 0.45

FC 0.33 0.22 0.55

0.60 0.40 1.0



P(E) = 0.60

Probability Table

Venn Diagram

0.18

0.27

0.220.33

E

F

Blue

Green

Orange

RedYellow

Outcome Probability

Red 0.10

Orange 0.18

Yellow 0.17

Green 0.22

Blue 0.33

1.00

POPULATION


P(E | F)

P(F | E)

18%

17%

22%

33%

10%

= P(E)

= P(F)

P(F) = 0.45

Events E and F are “statistically independent”

49

Example: According to the American Red Cross, US pop is distributed as shown.

Rh Factor

Blood Type + – Row marginals:

O .384 .077 .461

A .323 .065 .388

B .094 .017 .111

AB .032 .007 .039

Column marginals:

.833 .166 .999

Def: Two events A and B are said to be statistically independent if

P(A | B) = P(A),

Example: Are events A = “Ace” and B = “Black” statistically independent?P(A) = 4/52 = 1/13, P(B) = 26/52 = 1/2, P(A ⋂ B) = 2/52 = 1/26 YES!

Neither event provides any information about the other.

Are “Type O” and “Rh+” statistically independent?

= P(O)

= P(Rh+)

Is .384 = .461 × .833?

P(O ⋂ Rh+) = .384

YES!

which is equivalent to P(A ⋂ B) = P(A | B) × P(B).

If either of these two conditions fails, then A and B are statistically dependent.

P(A)

A and B are statistically independent if:

P(A | B) = P(A)

IMPORTANT FORMULAS

P(Ac) = 1 – P(A)

P(A ⋃ B) = P(A) + P(B) – P(A ⋂ B)

50

= 0 if A and B are disjoint

P(A ⋂ B) = P(A | B) P(B) .)(

)()|(

BP

BAPBAP

P(A ⋂ B) = P(A) P(B)

DeMorgan’s Laws

(A ⋃ B)c = Ac ⋂ Bc

“Not (A or B)” = “Not A” and “Not B” = “Neither A nor B”

(A ⋂ B)c = Ac ⋃ Bc

“Not (A and B)” = “Not A” or “Not B”

A B

A B

A B

Example: In a population of individuals:

60% of adults are male

P(B | A) = 0.6

40% of males are adults

P(A | B) = 0.4

30% are men

P(A ⋂ B) = 0.3

What percentage are adults?

51

A = Adult B = Male

What percentage are males?

Are “adult” and “male” statistically independent in this population?

0.3Men Boy

sWome

n

Girls

Example: In a population of individuals:

60% of adults are male

P(B | A) = 0.6

40% of males are adults

P(A | B) = 0.4

30% are men

P(A ⋂ B) = 0.3

⟹ P(B A) = 0.6 ⋂ P(A)0.3

P(A) = 0.3 / 0.6

What percentage are adults?

52

A = Adult B = Male

What percentage are males?

Are “adult” and “male” statistically independent in this population?

0.3

⟹ P(A ⋂ B) = 0.4 P(B)0.3

P(B) = 0.3 / 0.4

0.2 0.45

Adult Child

Male 0.30 0.45 0.75

Female 0.20 0.05 0.25

0.50 0.50 1.00

0.05

P(A | B) = P(A)? OR P(B | A) = P(B)? OR P(A ⋂ B) = P(A) P(B)?

NO

0.4 ≠ 0.5 0.6 ≠ 0.75

P(A) = 0.3 / 0.6 = 0.5, or 50%

0.5 – 0.3 = …

P(B) = 0.3 / 0.4 = 0.75, or 75%

0.75 – 0.3 = …

0.3 ≠ (0.5)(0.75)

Men Boys

Women

Girls

CHAPTER 3

Probability Theory


P(B) P(B )

Bayes’ Formula

Exactly how does one event A affect the probability of another event B?

54

AP(B)

prior probability

posterior probability

P(B A)P(A)

But what if the numerator and denominator are not explicitly given?

Example: Vitamin B-complex deficiency among general population

B1

Thiamine

B2

Riboflavin

B3

Niacin

B4

No B deficiency

Assume B1, B2, B3, B4 “partition” the population, i.e., they are disjoint and exhaustive.

“10% of pop is B1-deficient (only), 20% is B2-deficient (only), and 30% is B3-deficient (only). The remaining 40% is not B-deficient.”

Given:


B1

Thiamine

B2

Riboflavin

B3

Niacin

B4

No B deficiency


P(B2) = .20 P(B3) = .30P(B1) = .10 P(B4) = .40

A = Alcoholic

Ac = Not Alcoholic

Given:

P(A ∩ B1)

To find these intersection probabilities, we need more information!

Prior probs 1.00

P(A ∩ B2) P(A ∩ B3) P(A ∩ B4)

P(Ac ∩ B1) P(Ac ∩ B2) P(Ac ∩ B3) P(Ac ∩ B4)


B1

Thiamine

B2

Riboflavin

B3

Niacin

B4

No B deficiency


P(B2) = .20 P(B3) = .30P(B1) = .10 P(B4) = .40

Alsogiven…

“Alcoholics comprise 35%, 30%, 25%, and 20% of the B1, B2, B3, B4 groups, respectively.”

Given:

A = Alcoholic

Ac = Not Alcoholic

Prior probs 1.00

P(A ∩ B1) P(A ∩ B2) P(A ∩ B3) P(A ∩ B4)



B1

Thiamine

B2

Riboflavin

B3

Niacin

B4

No B deficiency


P(B2) = .20 P(B3) = .30P(B1) = .10 P(B4) = .40

Alsogiven… P(A | B1) = .35 P(A | B2) = .30 P(A | B3) = .25 P(A | B4) = .20

Prior probs 1.00

Given:

A = Alcoholic

Ac = Not Alcoholic

P(A ∩ B1) P(A ∩ B2) P(A ∩ B3) P(A ∩ B4)



B1

Thiamine

B2

Riboflavin

B3

Niacin

B4

No B deficiency


Alsogiven…

Prior probs 1.00

Given:

A = Alcoholic

Ac = Not Alcoholic

P(A ∩ B1) P(A ∩ B2) P(A ∩ B3) P(A ∩ B4)


P(A | B1) = .35 P(A | B2) = .30 P(A | B3) = .25 P(A | B4) = .20

P(B2) = .20 P(B3) = .30P(B1) = .10 P(B4) = .40

P(A ∩ B) = P(A | B) P(B) Recall:


B1

Thiamine

B2

Riboflavin

B3

Niacin

B4

No B deficiency


Alsogiven…

Prior probs 1.00

Given:

A = Alcoholic

Ac = Not Alcoholic P(Ac ∩ B1) P(Ac ∩ B2) P(Ac ∩ B3) P(Ac ∩ B4)

P(A ∩ B) = P(A | B) P(B) Recall:

P(A | B1) = .35 P(A | B2) = .30 P(A | B3) = .25 P(A | B4) = .20

.10 .35 .20 .30 .30 .25 .40 .20 .035 .060 .075 .080

P(B2) = .20 P(B3) = .30P(B1) = .10 P(B4) = .40


B1

Thiamine

B2

Riboflavin

B3

Niacin

B4

No B deficiency


Prior probs 1.00

Given:

A = Alcoholic

Ac = Not Alcoholic

.10 .35 .20 .30 .30 .25 .40 .20 .035 .060 .075 .080

.065 .140 .225 .320

P(B2) = .20 P(B3) = .30P(B1) = .10 P(B4) = .40

P(B1 | A) = ? P(B2 | A) = ? P(B3 | A) = ? P(B4 | A) = ?

Posterior probabilities

B1

Thiamine

B2

Riboflavin

B3

Niacin

B4

No B deficiency



P(B2) = .20 P(B3) = .30P(B1) = .10 P(B4) = .40

P(B1 | A) = ? P(B2 | A) = ? P(B3 | A) = ? P(B4 | A) = ?

.035 .060 .075 .080

.065 .140 .225 .320

P(A) = .25

P(Ac) = .75

.035

1.00

P(B1 ∩ A)

P(A)

Prior probsGiven:

A = Alcoholic

Ac = Not Alcoholic


B1

Thiamine

B2

Riboflavin

B3

Niacin

B4

No B deficiency



P(B1 | A) = P(B2 | A) = P(B3 | A) = P(B4 | A) =

.035 .060 .075 .080

.065 .140 .225 .320

P(A) = .25

P(Ac) = .75

.035

.035

.25

.060

.060

.25

.075

.075

.25

.080

.080

.25

Prior probsGiven:

A = Alcoholic

Ac = Not Alcoholic


1.00P(B2) = .20 P(B3) = .30P(B1) = .10 P(B4) = .40

B1

Thiamine

B2

Riboflavin

B3

Niacin

B4

No B deficiency



P(B1 | A) = .14 P(B2 | A) = .24 P(B3 | A) = .30 P(B4 | A) = .32

.035 .060 .075 .080

.065 .140 .225 .320

P(A) = .25

P(Ac) = .75

1.00

INCREASE INCREASE DECREASENO CHANGE;A and B3 are independent!

Prior probsGiven:

A = Alcoholic

Ac = Not Alcoholic


P(B2) = .20 P(B3) = .30P(B1) = .10 P(B4) = .40

B1

Thiamine

B2

Riboflavin

B3

Niacin

B4

No B deficiency



P(B1 | Ac) = ?? P(B2 | Ac) = ?? P(B3 | Ac) = ?? P(B4 | Ac) = ??

.035 .060 .075 .080

.065 .140 .225 .320

P(A) = .25

P(Ac) = .75

1.00

Exercise:

Prior probsGiven:

A = Alcoholic

Ac = Not Alcoholic


P(B2) = .20 P(B3) = .30P(B1) = .10 P(B4) = .40



A (Yes)

Ac (No)

Alc

oh

olic

etc.

Non-deficient

Thiamine-deficient

Riboflavin-deficient

Niacin-deficient

C1

C2C5 C6

C4 C3

C7C8

C1 C2 C3 C4 C5 C6 C7 C8

Prior probabilities:

BAYES’ FORMULAAssume B1, B2, …, Bn “partition” the population, i.e., they are disjoint and exhaustive.

A

Ac

B1 B2 B3 ……etc……. Bn

Given…

P(B1) P(B2) P(B3) ……etc……. P(Bn) 1

Then…

Posterior probabilities: P(B1|A) P(B2|A) P(B3|A) ……etc……. P(Bn|A) are computed via

P(Bi | A) = P(Bi ∩ A)

P(A)

P(A | Bi) P(Bi)

P(A | B1) P(B1) + P(A | B2) P(B2) + …+ P(A | Bn) P(Bn)=

for i = 1, 2, 3,…, n

P(A ∩ B1) P(A ∩ B2) P(A ∩ B3) P(A ∩ Bn)……etc…….

P(A)

P(Ac ∩ B1) P(Ac ∩B2) P(Ac ∩ B3) ……etc…….

P(Ac ∩ Bn) P(Ac)

……etc……? ? ? ? INTERPRET!

CHAPTER 3 Probability Theory (Abridged Version) B asic Definitions and Properties C onditional...

Documents

Transcript of CHAPTER 3 Probability Theory (Abridged Version) B asic Definitions and Properties C onditional...