Chapter 5: Probability

Randomness● A random phenomenon is a situation in which we know

what outcomes could happen, but we don't know which particular outcome did or will happen

● However, we can calculate the probability with which each outcome will happen

● People are not good at identifying truly random samples or random experiments, so we need to rely on outside mechanisms such as coin flips or random number tables

Simulating Randomness● Flip a coin to generate random 0's and 1's● Pick a card to generate random numbers● Pick a number out of a hat● Use a Random Number table...

Simulating Randomness● Use a Random Number table

Ex. Simulate rolling a die 10 times

● Pick any line to start, say line 30 to start● Go through the numbers, picking numbers 1-6 and ignoring

0,7,8,9.● The“random”numbers are: 5, 4, 5, 3, 4, 6, 2, 5, 3, 1

16% of cars fail pollution tests in CaliforniaWe can represent this in many ways:

● In a bag with 100 chips, 16 are red and represent cars that fail pollution tests, the remaining 84 are black and represent cars that pass the tests.

● In a bag with 50 marbles, 8 are orange and represent cars that fail pollution tests, the remaining 42 are blue and represent cars that pass the tests.

● On a random number table numbers 00-15 represent cars that fail pollution tests and 16-99 represent cars that pass the tests.

Estimating Probabilities via SimulationWe would like to estimate the probability that an entire fleet of seven cars would pass using a random number table. We assume each car is independent.

● Start at row 1 of the random number table and read two digits at a time.

● If the random number is between 00-15, the car will fail the pollution test. If the number is between 16-99, the car will pass the test.

● A fleet of cars is comprised of seven cars, i.e. seven 2-digit random numbers. If all seven cars pass the test, record a 1, if not record a 0.

● Repeat many times and calculate the proportion 1s among the total number of runs, i.e. number of fleets where all cars passed the pollution test.

Run 1

Run 2

Run 3

Did all the cars in run 3 pass?

a) Yes?b) No?

Run 4

Estimation of ProbabilityBased on the simulation results, estimate the probability that an entire fleet of seven cars would pass the pollution test.

Notes● 4 runs is usually not considered sufficient, for homework use

at least 5 runs.● You can start at any row you like on the random number

table, but you should make sure to note it when you’re writing up your simulation scheme.

● You should not arbitrarily pick numbers from the random number table. Just pick a row and follow across. Otherwise the numbers you’re using won’t be random (they will instead be your choices).

Estimation of ProbabilityBased on the simulation results, estimate the average number of cars that fail the pollution test in a fleet of seven cars.

a) 0 b) 1 c) 2 d) 3 e) 4

Randomness● A random phenomenon

is a situation in which we know what outcomes could happen, but we don't know which particular outcome did or will happen

● However, we can calculate the probability with which each outcome will happen

Randomness to Probability

Let's say there are 1000 songs in your iTunes library. Only 5 of these songs are by Justin Bieber. What is the probability that next time you hit shuffle you get a Justin Bieber song?

Prob(JB song) = 5/1000

= 0.005

Definitions of Probability● A Frequentist: The probability of an event is its long-run

relative frequency.● Theoretical: We may not be able to predict which song we play

each time we hit shuffle, however we know that in the long run 5 out of 1000 songs will be Justin Bieber songs.

● Empirical: We listen to 100 songs on shuffle and 4 of them are Justin Bieber songs. The empirical probability is 4/100=0.04

● A Bayesian: ● Prior probability of an event is the best guess by the observer of

an event's probability. ● Posterior probability of an event is the probability of an event

after collecting some empirical data. It is obtained by updating information from the prior probability with additional data related to the event in question. The prior probability may be subjective, but with enough data, has little impact on the posterior probability.

Definitions of Probability

Note: In this course we will use the frequentist definition of probability, though we will make some use of Bayesian tools.

More Definitions● For any random phenomenon, each attempt is called a

trial and each trial generates an outcome● Each time you hit shuffle is a trial.● The song that plays as a result of hitting shuffle is an outcome.

● Sample space is the collection of all possible outcomes of a trial● Sample space is the entire iTunes library of 1000 songs.

● A combination of outcomes is called an event● For example playing two Justin Bieber songs in a row

in shuffle

Sample SpaceA couple has two kids, what is the sample space for the gender of these kids?

Calculating ProbabilitiesIf a couple has two kids what is the probability that both are boys?

If a couple has two kids, the list of possible scenarios are:

BB, GG, BG, GB

Since each scenario is equally likely:

Prob(BB) = ¼ = 0.25

IndependenceWhen thinking about what happens with combinations of outcomes, things are simplified if the individual trials are independent.

● Roughly speaking, this means that the outcome of one trial doesn't influence or change the outcome of another.

● If the iTunes shuffle is truly random then the songs played are independent of each other.

● In other words, the iTunes shuffle is memoryless, it does not say to itself“Wait, I just played an Justin Bieber song, I shouldn't play one again”

● Similarly, if the genders of kids a couple has are independent, then the probability of having a boy for a second child doesn't change based on whether or not the first child of the couple was a girl.

Definitions – Coin toss example● Trial: each coin toss● Outcome: Heads or Tails● Probability: P(H) = 0.5 and

P(T) = 0.5Note: Each time we toss a coin we can't tell which side will come up, however in the long run tails will come up 50% of the time and heads will come up 50% of the time

● Sample Space: Tossed once: S = {H,T} Tossed twice: S = {HT,TH,HH,TT}

● Independence: The outcome of one coin toss does not affect the outcome of the next

Dice● Trial: each die roll● Outcome: A die has six sides● Probability: With a fair die the

probability of getting each side is the same and is 1/6

● Sample Space: Rolled once: S = {1, 2, 3, 4, 5, 6}

● Independence: The outcome of one die roll does not affect the outcome of the next

Deck of Cards● A deck is made up of 52 cards● There are 4 suits: clubs, spades,

diamonds, hearts● Each suit has 13 cards● Within each suit there is one ace,

numbers 2-10, a jack, a queen, and a king

● Clubs and spades are black cards, diamonds and hearts are red cards

Law of Large Numbers● The Law of Large Numbers (LLN) says that the long-run

relative frequency of repeated independent events gets closer and closer to the true relative frequency as the number of trials increases● If a coin is tossed many times, the overall percentage of

heads should settle down to about 50% as the number of tosses increases

● This is why we define probability as the long run relative-frequency of an event

● The common (mis)understanding of the LLN is that random phenomena are supposed to compensate for whatever happened in the past; this is just not true and if also called gambler's fallacy (or law of averages)

Law of Large NumbersWhen tossing a fair coin, if heads comes up on each of the first 10 toss, what do you think the chance is that another head will come up on the next toss?

H H H H H H H H H H H H H H ?

● The probability is still 0.5, or there is still a 50% chance that another head will come up on the next toss

● Prob(H on 11th toss) = Prob(T on 11th toss) = 0.5● The coin is not due for a tails

Equally Likely Outcomes● Equally likely outcomes have the same probability of

happening● It's equally likely to get any one of six outcomes from

the roll of a fair die● It's equally likely to get heads or tails from the toss

of a fair coin● Not all outcomes are equally likely

● Its not equally likely to draw an ace as a red card● A skilled basketball player has a better than 50-50

chance of making a free throw

Personal Probability● WIn everyday speech, when we express a degree of

uncertainty without basing it on long-run relative frequencies, we are stating subjective or personal probabilities

● Personal probabilities don't display the kind of consistency that we will need probabilities to have, so we will stick with formally defined probabilities

Impossibility to a CertaintyProbabilities must be between 0 and 1, inclusive.

● A probability of 0 indicates impossibility● A probability of 1 indicates certainty

● Note: If you calculate a probability and get a negative number or a number greater than 1 you are making a mistake, go back and check your work.

Something has to happen ruleThe probability of the seat of all possible outcomes of a trial must be 1

Prob(sample space) = 1

● Coin toss: Prob(H) + Prob(T) = 0.5 + 0.5 =1● Cards: Prob(face card) + Prob(non – face card)

= 12/52 + 40/52 = 52/52 = 1

In a class 30% of the students are freshmen, 20% are sophomores, 10% are juniors and the rest are seniors. If we randomly pick a student what is the probability that he or she is a senior?

a) 0.3

b) 0.4

c) 0.2

c) 0.1

Complement RuleThe set of outcomes that are not in the event A is called the complement of A, denoted . The probability of an event occurring is 1 minus the probability that it does not occur.

In a survey, 52% of respondents said they are Democrats. What is the probability that a randomly selected respondent from this sample is a Republican?

a) greater than 48%

b) less than 48%

c) equal to 48%

d) cannot be computed from the information

Disjoint EventsEvents that have no outcomes in common (and, thus, cannot occur together) are called disjoint (or mutually exclusive).

● The outcome of a coin toss cannot be a head and a tail● A student cannot fail and pass a class● A card drawn from a deck cannot be an ace and a queen

Addition RuleFor two disjoint events A and B, the probability that one OR the other occurs is the sum of the probabilities of the two events.

P(A or B) = P(A) + P(B)

Do the sum of two disjoint events always add up to one?

– Yes?– No?

Multiplication RuleFor two independent events A and B, the probability that both A AND B occur is the product of the probabilities of the two events.

P(A or B) = P(A) x P(B)

In a multiple choice exam there are 5 questions and 4 choices for each question (a, b, c, d). Nancy has not studied for the exam at all, and decided to randomly guess the answers. What is the probability that she gets the first question right?

a) 1 c) 0.2

b) 0.25 d) 0.75

What is the probability that she gets the second question right?

a) 1 c) 0.25

b) 0.2 d) 0.4

Example – Guessing at RandomWhat is the probability that the first question she gets right is the 5th question?

Prob(Wrong Wrong Wrong Wrong Right)

= 0.75 x 0.75 x 0.75 x 0.75 x 0.25

= 0.0791

What is the probability that she gets all the questions right?

a) 1.25 c) 0.0039

b) 0.0010 d) 0

Example – Guessing at RandomWhat is the probability that the first question she gets at least one question right?

If there are 5 questions, the possible number of questions she gets right are:

S = {0,1,2,3,4,5}

We are interested in instances where she gets at least 1 question right:

S = {0,1,2,3,4,5}

So we can divide up the sample space into two categories:

S = {0, at least 1}

Example – Guessing at Random

Independent vs. DisjointCan two events be independent and disjoint?

Remember: Independence means that the outcome of one trial doesn't influence the outcome of another and disjoint means that two events cannot occur together.

If we know that the outcome of a coin toss is heads, then we know that it is not tails. So whether or not the outcome is tails depends on whether or not the outcome is heads. Therefore these two events are disjoint and dependent.

Independent vs. Disjoint● Disjoint events cannot be independent

● Since we know that disjoint events have no outcomes in common, knowing that one occurred means the other didn't happen.

● Thus, the probability of the one event occurring changed based on our knowledge that the other occurred.

● Therefore the two events are not independent.● Non-disjoint events may or may not be independent

● Just because two events can occur at the same time does not mean they need to also be dependent.

● It is possible that two randomly selected students get an A in a class, but how they do in the class may have nothing to do with each other.

● However if you and a close friend of yours that you study with both get an A in a class, how well the two of you do in that class are possibly dependent on each other.

Properties of Probability - SummaryWhen thinking about what happens with combinations of outcomes, things are simplified if the individual trials are independent.

● Roughly speaking, this means that the outcome of one trial doesn't influence or change the outcome of another.

● If the iTunes shuffle is truly random then the songs played are independent of each other.

● IIn other words, the iTunes shuffle is memoryless, it does not say to itself“Wait, I just played an Justin Bieber song, I shouldn't play one again”

Example – Absence from SchoolA middle school estimates that 20% of its students miss one day of school per semester due to sickness, 13% miss two days, and 5% miss three or more days. What is the probability that a student chosen at random doesn't miss any days of school due to sickness?

Prob(no misses) = 1 - (0.20 + 0.13 + 0.05) = 0.62

Example – Absence from SchoolA middle school estimates that 20% of its students miss one day of school per semester due to sickness, 13% miss two days, and 5% miss three or more days. What is the probability that a student chosen at random doesn't miss any days of school due to sickness?

a) 0.38 c) 0.18b) 0.33 d) 0.62

Example – Absence from SchoolA middle school estimates that 20% of its students miss one day of school per semester due to sickness, 13% miss two days, and 5% miss three or more days. What is the probability that a student chosen at random misses no more than one day?

Prob(at most 1 miss) =

Prob(no misses) + Prob(1 miss) =

0.62 + 0.20 = 0.82

Example – Absence from SchoolA middle school estimates that 20% of its students miss one day of school per semester due to sickness, 13% miss two days, and 5% miss three or more days. If a parent has two kids at this middle school, what is the probability that neither will miss any school?

Prob(neither miss any) =

Prob(no miss) x Prob(no miss)

0.62 x 0.62 = 0.3844

Example – Absence from SchoolWe used the multiplication rule to calculate the probability in the previous slide.

a) What must be true about these kids to make that approach valid?

b) Do you think this assumption is reasonable?

Example – Absence from SchoolA middle school estimates that 20% of its students miss one day of school per semester due to sickness, 13% miss two days, and 5% miss three or more days. If a parent has two kids at this middle school, assuming that the kids are independent with respect to missing school, what is the probability that both will miss some school, i.e. at least one day? Choose the closest answer.

a) 0.14 c) 0.38

b) 0.24 d) 0.76

Example – Absence from SchoolA middle school estimates that 20% of its students miss one day of school per semester due to sickness, 13% miss two days, and 5% miss three or more days. If a parent has two kids at this middle school, assuming that the kids are independent with respect to missing school, what is the probability that one kid misses some school and the other doesn't miss any?

General Addition Rule

Example – Second LanguageIn a class where everyone's native language is English, 70% of students speak Spanish, 45% speak French as a second language and 20% speak both. Assume that there are no students who speak another second language. What is the probability that a randomly selected students speaks Spanish or French?

Prob(Spanish or French) =

Prob(Spanish) + Prob(French) - Prob(Spanish and French) =

0.70 + 0.45 - 0.20 = 0.95

Picturing ProbabilitiesThe most common kind of picture we will make to picture probabilities is called Venn Diagram.

Example – Second LanguageIn a class where everyone's native language is English, 70% of students speak Spanish, 45% speak French as a second language and 20% speak both. Assume that there are no students who speak another second language.

Conditional Probability

Example – Second LanguageIn a class where everyone's native language is English, 70% of students speak Spanish, 45% speak French as a second language and 20% speak both. Assume that there are no students who speak another second language. What is the probability that a randomly selected student speaks French given that they speak Spanish?

Example – Second LanguageIn a class where everyone's native language is English, 70% of students speak Spanish, 45% speak French as a second language and 20% speak both. Assume that there are no students who speak another second language. What is the probability that a randomly selected student speaks Spanish given that they speak French?

a) 0.45 c) 2.25

b) 0.44 d) 0.05

Example – Burgers in LAA survey conducted in 2010 by Survey USA for KABC-TV Los Angeles asked 500 Los Angeles residents“What is the best hamburger place in Southern California? Five Guys Burgers? In-N-Out Burger? Fat Burger? Tommys Hamburgers? Umami Burger? Or somewhere else?” Consider the results of this survey, shown in the table below, in each of the following questions.

Example – Burgers in LARefer to the table on the previous slide:

● What is the probability that a randomly chosen male likes In-N-Out the best?

● What is the probability that a randomly chosen female likes In-N-Out the best?

● What is the probability that a man AND a woman who are dating both like In-N-Out the best?

Example – Burgers in LA● What is the probability that a randomly chosen person likes

Umami best or that the person is a female?

a) 0.006 c) 0.514

b) 0.504 d) 0.516

General Multiplication Rule

Checking for independence using conditional probabilities

Example – Second LanguageIn a class where everyone's native language is English, 70% of students speak Spanish, 45% speak French as a second language and 20% speak both. Assume that there are no students who speak another second language. Are the variables speaking French and speaking Spanish independent?

Independent vs disjoint - recap

● If A and B are disjoint:● Prob(A and B) = 0● A and B are not independent

● If A and B are not disjoint:● Prob(A and B) > 0● A and B may or may not be independent

ExampleAn online survey shows that 25% of bloggers own a DSLR camera, 90% own a point-and-shoot camera, and 20% own both types of cameras. Are owning a DSLR and point-and-shoot camera disjoint?

ExampleAn online survey shows that 25% of bloggers own a DSLR camera, 90% own a point-and-shoot camera, and 20% own both types of cameras. Are the variables owning a DSLR and owning a point-and-shoot camera independent?