1 Copyright M.R.K. Krishna Rao 2003 Chapter 5. Discrete Probability Everything you have learned...

1Copyright M.R.K. Krishna Rao 2003

Chapter 5. Discrete ProbabilityChapter 5. Discrete Probability

Everything you have learned about counting Everything you have learned about counting constitutes the basis for computing the constitutes the basis for computing the probabilityprobability of events to happen. of events to happen.

In the following, we will use the notion In the following, we will use the notion experimentexperiment for a procedure that yields one of a for a procedure that yields one of a given set of possible outcomes.given set of possible outcomes.

This set of possible outcomes is called the This set of possible outcomes is called the sample spacesample space of the experiment. of the experiment.

An An eventevent is a subset of the sample space. is a subset of the sample space.


Discrete ProbabilityDiscrete Probability

If all outcomes in the sample space are equally If all outcomes in the sample space are equally likely, the following definition of probability likely, the following definition of probability applies:applies:

The probability of an event E, which is a subset of The probability of an event E, which is a subset of a finite sample space S of equally likely outcomes, a finite sample space S of equally likely outcomes, is given by p(E) = |E|/|S|.is given by p(E) = |E|/|S|.

Probability values range from 0 (for an event that will never happen) to 1 (for an event that will always happen whenever the experiment is carried out).



Example:Example:

An urn contains four blue balls and five red balls. An urn contains four blue balls and five red balls. What is the probability that a ball chosen from the What is the probability that a ball chosen from the urn is blue?urn is blue?

Solution:Solution:

There are nine possible outcomes, and the event There are nine possible outcomes, and the event “blue ball is chosen” comprises four of these “blue ball is chosen” comprises four of these outcomes. Therefore, the probability of this event outcomes. Therefore, the probability of this event is 4/9 or approximately 44.44%.is 4/9 or approximately 44.44%.

4

Discrete ProbabilityDiscrete Probabilityand Sets/Cardinalitiesand Sets/Cardinalities

EG: WhatEG: What’’s the probability of tossing a coin 3 s the probability of tossing a coin 3 times and getting all heads or all tails?times and getting all heads or all tails?

Can consider set of ways of tossing coin 3 times Can consider set of ways of tossing coin 3 times ((sample spacesample space ): ):SS = {HHH,HHT,HTH,HTT,THH,THT,TTH,TTT} = {HHH,HHT,HTH,HTT,THH,THT,TTH,TTT}

Next, consider set of ways of tossing all heads Next, consider set of ways of tossing all heads or all tails (or all tails (eventevent ): ): EE = {HHH,TTT} = {HHH,TTT}

Assuming all outcomes equally likelyAssuming all outcomes equally likely……DEF: The probability of the event DEF: The probability of the event E E is the ratio is the ratio

p p ((E E ) = |) = |E E | / || / |S S ||EG: Our case: EG: Our case: p p ((E E ) = 2/8 = 0.25) = 2/8 = 0.25



Example:Example:

What is the probability of winning the lottery 6/49, i.e., What is the probability of winning the lottery 6/49, i.e., picking the correct set of six numbers out of 49?picking the correct set of six numbers out of 49?

Solution:Solution:

There are C(49, 6) possible outcomes. Only one of There are C(49, 6) possible outcomes. Only one of these outcomes will actually make us win the lottery.these outcomes will actually make us win the lottery.

p(E) = 1/C(49, 6) = 1/13,983,816 p(E) = 1/C(49, 6) = 1/13,983,816

Example 2/P.261: Example 2/P.261: What is the probability that when What is the probability that when 2 dice are rolled, the sum of the numbers is 7 ?2 dice are rolled, the sum of the numbers is 7 ?


Complimentary EventsComplimentary Events

Let E be an event in a sample space S. The Let E be an event in a sample space S. The probability of an event –E, the probability of an event –E, the complimentary complimentary eventevent of E, is given by p(-E) = 1 – p(E). of E, is given by p(-E) = 1 – p(E).

This can easily be shown:This can easily be shown:

p(-E) = (|S| - |E|)/|S| = 1 - |E|/|S| = 1 – p(E).

This rule is useful if it is easier to determine the probability of the complimentary event than the probability of the event itself.



Example:Example:

A sequence of 10 bits is randomly generated. What A sequence of 10 bits is randomly generated. What is the probability that at least one of these bits is is the probability that at least one of these bits is zero?zero?

Solution:Solution: There are 2 There are 21010 = 1024 possible outcomes = 1024 possible outcomes of generating such a sequence. The event –E, of generating such a sequence. The event –E, “none of the bits is zero”“none of the bits is zero”, includes only one of , includes only one of these outcomes, namely the sequence 1111111111.these outcomes, namely the sequence 1111111111.

Therefore, p(-E) = 1/1024.Therefore, p(-E) = 1/1024.

Now p(E) can easily be computed as Now p(E) can easily be computed as p(E) = 1 – p(-E) = 1 – 1/1024 = 1023/1024.p(E) = 1 – p(-E) = 1 – 1/1024 = 1023/1024.



Example:Example:

What is the probability that at least two out of 36 What is the probability that at least two out of 36 people have the same birthday?people have the same birthday?

Solution:Solution: The sample space S encompasses all The sample space S encompasses all possibilities for the birthdays of the 36 people,possibilities for the birthdays of the 36 people,so |S| = 365so |S| = 3653636..

Let us consider the event –E (“no two people out of Let us consider the event –E (“no two people out of 36 have the same birthday”). –E includes P(365, 36 have the same birthday”). –E includes P(365, 36) outcomes (365 possibilities for the first 36) outcomes (365 possibilities for the first person’s birthday, 364 for the second, and so on). person’s birthday, 364 for the second, and so on).

Then p(-E) = P(365, 36)/365Then p(-E) = P(365, 36)/3653636 = 0.168, = 0.168,so p(E) = 0.832 or 83.2%so p(E) = 0.832 or 83.2%



Let ELet E11 and E and E22 be events in the sample space S. be events in the sample space S.Then we have:Then we have:

p(Ep(E11 E E22) = p(E) = p(E11) + p(E) + p(E22) ) - p(E- p(E11 E E22) )

Does this remind you of something?Does this remind you of something?

Of course, the principle of Of course, the principle of inclusion-inclusion-exclusionexclusion..



Example:Example: What is the probability of a positive What is the probability of a positive integer selected at random from the set of integer selected at random from the set of positive integers not exceeding 100 to be positive integers not exceeding 100 to be divisible by 2 or 5? divisible by 2 or 5?

Solution:Solution:

EE22: “integer is divisible by 2”: “integer is divisible by 2”EE55: “integer is divisible by 5”: “integer is divisible by 5”

EE22 = {2, 4, 6, …, 100} = {2, 4, 6, …, 100}

|E|E22| = 50| = 50

p(Ep(E22) = 0.5 ) = 0.5



EE55 = {5, 10, 15, …, 100} = {5, 10, 15, …, 100}

|E|E55| = 20| = 20

p(Ep(E55) = 0.2) = 0.2

EE22 E E55 = {10, 20, 30, …, 100} = {10, 20, 30, …, 100}

|E|E22 E E55| = 10| = 10

p(Ep(E22 E E55) = 0.1) = 0.1

p(Ep(E22 E E55) = p(E) = p(E22) + p(E) + p(E55)) – p(E– p(E22 E E5 5 ))

p(Ep(E22 E E55) = 0.5 + 0.2 – 0.1 = 0.6) = 0.5 + 0.2 – 0.1 = 0.6



What happens if the outcomes of an experiment What happens if the outcomes of an experiment are are notnot equally likely? equally likely?

In that case, we assign a probability p(s) to each In that case, we assign a probability p(s) to each outcome soutcome sS, where S is the sample space.S, where S is the sample space.

Two conditions have to be met:Two conditions have to be met:(1): 0 (1): 0 p(s) p(s) 1 for each s 1 for each sS, andS, and(2): (2): ssSS p(s) = 1 p(s) = 1

This means, as we already know, that (1) each This means, as we already know, that (1) each probability must be a value between 0 and 1, and probability must be a value between 0 and 1, and (2) the probabilities must add up to 1, because (2) the probabilities must add up to 1, because one of the outcomes is one of the outcomes is guaranteedguaranteed to occur. to occur.



Example I:Example I: A die is biased so that the number 3 A die is biased so that the number 3 appears twice as often as each other number.appears twice as often as each other number.

What are the probabilities of all possible outcomes?What are the probabilities of all possible outcomes?

Solution:Solution: There are 6 possible outcomes s There are 6 possible outcomes s11, …, s, …, s66. .

p(sp(s11) = p(s) = p(s22) = p(s) = p(s44) = p(s) = p(s55) = p(s) = p(s66))

p(sp(s33) = 2p(s) = 2p(s11))

Since the probabilities must add up to 1, we have:Since the probabilities must add up to 1, we have:

5p(s5p(s11) + 2p(s) + 2p(s11) = 1) = 1

7p(s7p(s11) = 1) = 1

p(sp(s11) = p(s) = p(s22) = p(s) = p(s44) = p(s) = p(s55) = p(s) = p(s66) = 1/7, p(s) = 1/7, p(s33) = ) = 2/72/7



Example II:Example II: For the biased die from Example I, For the biased die from Example I, what is the probability that an odd number what is the probability that an odd number appears when we roll the die?appears when we roll the die?

Solution:Solution:

EEoddodd = {s = {s11, s, s33, s, s55}}

Remember the formula Remember the formula p(E) = p(E) = ssEE p(s). p(s).

p(Ep(Eoddodd) = ) = ssEEoddodd p(s) = p(s p(s) = p(s11) + p(s) + p(s33) + p(s) + p(s55))

p(Ep(Eoddodd) = 1/7 + 2/7 + 1/7 = 4/7 = 57.14%) = 1/7 + 2/7 + 1/7 = 4/7 = 57.14%


Conditional ProbabilityConditional Probability

If we toss a coin three times, what is the If we toss a coin three times, what is the probability that an odd number of tails appears probability that an odd number of tails appears (event E)(event E), if the first toss is a tail , if the first toss is a tail (event F)(event F) ? ?

If the first toss is a tail, the possible sequences If the first toss is a tail, the possible sequences are TTT, TTH, THT, and THH. are TTT, TTH, THT, and THH.

In two out of these four cases, there is an odd In two out of these four cases, there is an odd number of tails. number of tails.

Therefore, the probability of E, under the Therefore, the probability of E, under the condition that F occurs, is 0.5.condition that F occurs, is 0.5.

We call this We call this conditional probabilityconditional probability. .



If we want to compute the conditional probability If we want to compute the conditional probability of E given F, we use F as the sample space.of E given F, we use F as the sample space.

For any outcome of E to occur under the condition For any outcome of E to occur under the condition that F also occurs, this outcome must also be inthat F also occurs, this outcome must also be inE E F. F.

Definition:Definition: Let E and F be events with p(F) > 0. Let E and F be events with p(F) > 0.The conditional probability of E given F, denoted The conditional probability of E given F, denoted by p(E | F), is defined asby p(E | F), is defined as

p(E | F) = p(E p(E | F) = p(E F)/p(F) F)/p(F)



Example:Example: What is the probability of a random bit What is the probability of a random bit string of length four contains at least two string of length four contains at least two consecutive 0s, given that its first bit is a 0 ?consecutive 0s, given that its first bit is a 0 ?

Solution:Solution:

E: “bit string contains at least two consecutive 0s”E: “bit string contains at least two consecutive 0s”

F: “first bit of the string is a 0”F: “first bit of the string is a 0”

We know the formula We know the formula p(E | F) = p(E p(E | F) = p(E F)/p(F) F)/p(F)..

E E F = {0000, 0001, 0010, 0011, 0100} F = {0000, 0001, 0010, 0011, 0100}

p(E p(E F) = 5/16 F) = 5/16

p(F) = 8/16 = 1/2p(F) = 8/16 = 1/2

p(E | F) = (5/16)/(1/2) = 10/16 = 5/8 = 0.625p(E | F) = (5/16)/(1/2) = 10/16 = 5/8 = 0.625



Probability of rain with no prior knowledge: Probability of rain with no prior knowledge: p p ((E E ) = |) = |E E |/||/|S S | = 67/366| = 67/366

Probability of rain, if day was overcast:Probability of rain, if day was overcast:

p p ((E E ||F F )) = |= |E E |/||/|F F | = 67/147| = 67/147

““The probability of The probability of E E given given F”F”

F: 147 overcastdays

S: 366 days in 2000

E: 67 rain days



EG: What is the probability that a length 4 bit EG: What is the probability that a length 4 bit string contains 00 given that it starts with 1?string contains 00 given that it starts with 1?

E E = {contain 00} = {contain 00} F F = {starts with 1}= {starts with 1}

EEF F = {1000,1001,1100}= {1000,1001,1100}

p p ((E\F E\F ) = |) = |EEF F | / || / |F F | = 3/2| = 3/233


IndependenceIndependence

Let us return to the example of tossing a coin Let us return to the example of tossing a coin three times.three times.

Does the probability of event E (odd number of Does the probability of event E (odd number of tails) tails) dependdepend on the occurrence of event F (first on the occurrence of event F (first toss is a tail) ?toss is a tail) ?

In other words, is it the case thatIn other words, is it the case thatp(E | F) p(E | F) p(E) ? p(E) ?

We actually find that p(E | F) = 0.5 and p(E) = 0.5,We actually find that p(E | F) = 0.5 and p(E) = 0.5,so we say that E and F are so we say that E and F are independent eventsindependent events..



Because we have p(E | F) = p(E Because we have p(E | F) = p(E F)/p(F), F)/p(F),p(E | F) = p(E) if and only if p(E p(E | F) = p(E) if and only if p(E F) = p(E)p(F). F) = p(E)p(F).

Definition:Definition: The events E and F are said to be The events E and F are said to be independent if and only if p(E independent if and only if p(E F) = p(E)p(F). F) = p(E)p(F).

Obviously, this definition is Obviously, this definition is symmetricalsymmetrical for E and for E and F. If we have p(E F. If we have p(E F) = p(E)p(F), then it is also F) = p(E)p(F), then it is also true that p(F | E) = p(F).true that p(F | E) = p(F).



Example:Example: Suppose E is the event that a Suppose E is the event that a randomly generated bit string of length four randomly generated bit string of length four begins with a 1, and F is the event that a begins with a 1, and F is the event that a randomly generated bit string contains an even randomly generated bit string contains an even number of 0s. Are E and F independent?number of 0s. Are E and F independent?

Solution:Solution: Obviously, p(E) = p(F) = 0.5. Obviously, p(E) = p(F) = 0.5.

E E F = {1111, 1001, 1010, 1100} F = {1111, 1001, 1010, 1100}p(E p(E F) = 0.25 F) = 0.25p(E p(E F) = p(E)p(F) F) = p(E)p(F)

Conclusion: E And F are Conclusion: E And F are independentindependent..


Bernoulli TrialsBernoulli Trials

Suppose an experiment with Suppose an experiment with two possible two possible outcomesoutcomes, such as tossing a coin. , such as tossing a coin.

Each performance of such an experiment is called Each performance of such an experiment is called a a Bernoulli trialBernoulli trial..

We will call the two possible outcomes a We will call the two possible outcomes a successsuccess or a or a failurefailure, respectively., respectively.

If p is the probability of a success and q is the If p is the probability of a success and q is the probability of a failure, it is obvious thatprobability of a failure, it is obvious thatp + q = 1.p + q = 1.



Often we are interested in the probability of Often we are interested in the probability of exactly k successesexactly k successes when an experiment when an experiment consists of consists of n independent Bernoulli trialsn independent Bernoulli trials..

Example:Example: A coin is biased so that the probability of head is A coin is biased so that the probability of head is 2/3. What is the probability of exactly four heads 2/3. What is the probability of exactly four heads to come up when the coin is tossed seven times?to come up when the coin is tossed seven times?



Solution:Solution:

There are 2There are 277 = 128 possible outcomes. = 128 possible outcomes.

The number of possibilities for four heads among The number of possibilities for four heads among the seven trials is C(7, 4).the seven trials is C(7, 4).

The seven trials are independent, so the The seven trials are independent, so the probability of each of these outcomes isprobability of each of these outcomes is(2/3)(2/3)44(1/3)(1/3)33..

Consequently, the probability of exactly four Consequently, the probability of exactly four heads to appear isheads to appear is

C(7, 4)(2/3)C(7, 4)(2/3)44(1/3)(1/3)33 = 560/2187 = 25.61% = 560/2187 = 25.61%



Bernoulli Formula: Consider an experiment Bernoulli Formula: Consider an experiment which repeats a Bernoulli trial n times. which repeats a Bernoulli trial n times. Suppose each Bernoulli trial has possible Suppose each Bernoulli trial has possible outcomes outcomes A, B A, B with respective probabilities with respective probabilities of success of success p p and probability of failure and probability of failure qq = 1- = 1-pp. The probability that . The probability that AA occurs exactly occurs exactly k k times in times in n n trials istrials is

b(k; n, p) = p b(k; n, p) = p kk · (· (qq))n-kn-k ··C C ((n,k n,k ))

Q: Suppose Bernoulli trial consists of flipping Q: Suppose Bernoulli trial consists of flipping a fair coin. What are a fair coin. What are AA, , BB, , pp and 1- and 1-pp..



A: A: A A = coin comes up “heads”= coin comes up “heads”

B B == coin comes up “tails”coin comes up “tails”

pp = 1- = 1-pp = = ½½

Q: What is the probability of getting exactly Q: What is the probability of getting exactly 10 heads if you flip a coin 20 times?10 heads if you flip a coin 20 times?

Recall:Recall: P P ((A A occurs occurs k k times out of times out of nn))

= = p p kk · (1-· (1-pp))n-kn-k ··C C ((n,k n,k ))



A: (1/2)A: (1/2)1010 · (1/2)· (1/2)1010 ··C C (20(20,,10)10)

== 184756 / 2184756 / 22020

== 184756 / 1048576 184756 / 1048576

= = 0.1762…0.1762…

Blackboard ExercisesBlackboard Exercises: 10, 20: 10, 20

1 Copyright M.R.K. Krishna Rao 2003 Chapter 5. Discrete Probability Everything you have learned...

Documents

Transcript of 1 Copyright M.R.K. Krishna Rao 2003 Chapter 5. Discrete Probability Everything you have learned...