Post on 05-Apr-2018
7/31/2019 02 Probability, Bayes Theorem and the Monty Hall Problem
1/34
Probability, Bayes Theorem
and the Monty Hall Problem
7/31/2019 02 Probability, Bayes Theorem and the Monty Hall Problem
2/34
PSYC 6130, PROF. J. ELDER 2
Probability Distributions
A random variable is a variable whose value is uncertain.
For example, the height of a randomly selected person in this classis a random variableI wont know its value until the person isselected.
Note that we are not completely uncertain about most randomvariables.
For example, we know that height will probably be in the 5-6 range.
In addition, 56 is more likely than 50 or 60 (for women).
The function that describes the probability of each possible value ofthe random variable is called a probability distribution.
7/31/2019 02 Probability, Bayes Theorem and the Monty Hall Problem
3/34
PSYC 6130, PROF. J. ELDER 3
Probability Distributions
Probability distributions are closely related to frequencydistributions.
7/31/2019 02 Probability, Bayes Theorem and the Monty Hall Problem
4/34
PSYC 6130, PROF. J. ELDER 4
Probability Distributions
Dividing each frequency by the total number of scoresand multiplying by 100 yields a percentage distribution.
7/31/2019 02 Probability, Bayes Theorem and the Monty Hall Problem
5/34
PSYC 6130, PROF. J. ELDER 5
Probability Distributions
Dividing each frequency by the total number of scoresyields a probability distribution.
7/31/2019 02 Probability, Bayes Theorem and the Monty Hall Problem
6/34
PSYC 6130, PROF. J. ELDER 6
Probability Distributions
For a discrete distribution, the probabilities over allpossible values of the random variable must sum to 1.
7/31/2019 02 Probability, Bayes Theorem and the Monty Hall Problem
7/34
PSYC 6130, PROF. J. ELDER 7
Probability Distributions
For a discrete distribution, we can talk about the probability of a particular
score occurring, e.g., p(Province = Ontario) = 0.36. We can also talk about the probability of any one of a subset of scores
occurring, e.g., p(Province = Ontario or Quebec) = 0.50.
In general, we refer to these occurrences as events.
7/31/2019 02 Probability, Bayes Theorem and the Monty Hall Problem
8/34
PSYC 6130, PROF. J. ELDER 8
Probability Distributions
For a continuous distribution, the probabilities over all possiblevalues of the random variable must integrate to 1 (i.e., the areaunder the curve must be 1).
Note that the height of a continuous distribution can exceed 1!
Shadedarea=0.683 Shadedarea=0.954 Shadedarea=0.997
7/31/2019 02 Probability, Bayes Theorem and the Monty Hall Problem
9/34
PSYC 6130, PROF. J. ELDER 9
Continuous Distributions
For continuous distributions, it does not make sense to talk about theprobability of an exact score.
e.g., what is the probability that your height is exactly 65.485948467 inches?
55 60 65 70 750
0.02
0.04
0.060.08
0.1
0.12
0.14
0.16
Height (in)
Probabilityp
Normal Approximation to probability distribution for height of Canadian females(parameters from General Social Survey, 1991)
5'3.8"
2.6"s
?
7/31/2019 02 Probability, Bayes Theorem and the Monty Hall Problem
10/34
PSYC 6130, PROF. J. ELDER 10
Continuous Distributions
It does make sense to talk about the probability of observing a score that falls within a certainrange
e.g., what is the probability that you are between 53 and 57?
e.g., what is the probability that you are less than 510?
55 60 65 70 750
0.02
0.04
0.060.08
0.1
0.12
0.14
0.16
Height (in)
Probabilityp
Normal Approximation to probability distribution for height of Canadian females(parameters from General Social Survey, 1991)
5'3.8"
2.6"s
Valid events
7/31/2019 02 Probability, Bayes Theorem and the Monty Hall Problem
11/34
7/31/2019 02 Probability, Bayes Theorem and the Monty Hall Problem
12/34
PSYC 6130, PROF. J. ELDER 12
Probability of Combined Events
More generally, if and are mutually exclusive,( or ) ( ) ( ) ( an
notd )
A Bp A B p A p B p A B
Canadian Community Health Survey, SleepingExam Haple: bits
Let event that respondent sleeps less than 6 hours per night.A
Let event that respondent reports trouble sleeping most or all of the timeB
( ) 0.139p A
( ) 0.152p B
( and ) 0.061p A B
Thus
( or ) 0.139 0.152 0.061 0.230p A B
7/31/2019 02 Probability, Bayes Theorem and the Monty Hall Problem
13/34
PSYC 6130, PROF. J. ELDER 13
Exhaustive Events
Two or more events are said to be exhaustive if at leastone of them must occur.
For example, if A is the event that the respondent sleeps
less than 6 hours per night and B is the event that therespondent sleeps at least 6 hours per night, then A andBare exhaustive.
(Although A is probably the more exhausted!!)
7/31/2019 02 Probability, Bayes Theorem and the Monty Hall Problem
14/34
PSYC 6130, PROF. J. ELDER 14
Independence
Two events are if the occurence of one
in no way affects the probability of
ind
the
ependent
other.
If events and are independent, then
( and ) ( ) ( )
A B
p A B p A p B
If events and are not independent, then
( and ) ( ) ( | )
A B
p A B p A p B A
: pick a card, anyExam cple ard.
7/31/2019 02 Probability, Bayes Theorem and the Monty Hall Problem
15/34
PSYC 6130, PROF. J. ELDER 15
An Example: The Monty Hall Problem
http://www.nytimes.com/2008/04/08/science/08monty.html7/31/2019 02 Probability, Bayes Theorem and the Monty Hall Problem
16/34
PSYC 6130, PROF. J. ELDER 16
Problem History
When problem first appeared in Parade, approximately10,000 readers, including 1,000 PhDs, wrote claimingthe solution was wrong.
In a study of 228 subjects, only 13% chose to switch.
7/31/2019 02 Probability, Bayes Theorem and the Monty Hall Problem
17/34
PSYC 6130, PROF. J. ELDER 17
Intuition
Before Monty opens any doors, there is a 1/3 probabilitythat the car lies behind the door you selected (Door 1),and a 2/3 probability it lies behind one of the other twodoors.
Thus with 2/3 probability, Monty will be forced to open aspecific door (e.g., the car lies behind Door 2, so Montymust open Door 3).
This concentrates all of the 2/3 probability in theremaining door (e.g., Door 2).
7/31/2019 02 Probability, Bayes Theorem and the Monty Hall Problem
18/34
PSYC 6130, PROF. J. ELDER 18
http://www.youtube.com/watch?v=hcFkic2I8zU7/31/2019 02 Probability, Bayes Theorem and the Monty Hall Problem
19/34
PSYC 6130, PROF. J. ELDER 19
Analysis
Switchingloses withprobability 1/6
Switching wins with probability 2/3Switching loses with
probability 1/3
Switching wins with probability1/3
Switching wins with probability1/3
Switchingloses withprobability 1/6
Host must open Door 2Host must open Door 3Host opens either Door 2 or 3
Player initially picks Door 1
Car hidden behind Door 3Car hidden behind Door 2Car hidden behind Door 1
7/31/2019 02 Probability, Bayes Theorem and the Monty Hall Problem
20/34
PSYC 6130, PROF. J. ELDER 20
Notes
It is important that
Monty must open a door that reveals a goat
Monty cannot open the door you selected
These rules mean that your choice may constrain whatMonty does.
If you initially selected a door concealing a goat, then there isonly one door Monty can open.
One can rigorously account for the Monty Hall problemusing a Bayesian analysis
7/31/2019 02 Probability, Bayes Theorem and the Monty Hall Problem
21/34
End of Lecture 2
Sept 17, 2008
7/31/2019 02 Probability, Bayes Theorem and the Monty Hall Problem
22/34
PSYC 6130, PROF. J. ELDER 22
Conditional Probability
To understand Bayesian inference, we first need to understand the
concept of conditional probability.
What is the probability I will roll a 12 with a pair of (fair) dice?
What if I first roll one die and get a 6? What now is the probabilitythat when I roll the second die they will sum to 12?
Let be the state of die 1
Let B be the state of die 2
Let be the sum of die 1 and 2
A
C
( 6) __?p A
( 6) __?p B
( 12) __?p C
( 12 | 6) __?p C A
Probability of C givenA
7/31/2019 02 Probability, Bayes Theorem and the Monty Hall Problem
23/34
PSYC 6130, PROF. J. ELDER 23
Conditional Probability
The conditional probabilityof A given B is thejointprobabilityof A and B, divided by the marginalprobabilityof B.
Thus if A and B are statistically independent,
However, if A and B are statistically dependent, then
( , )( | )
( )
p A Bp A B
p B
( , ) ( ) ( )( | ) ( ).
( ) ( )
p A B p A p B p A B p A
p B p B
( | ) ( ).p A B p A
7/31/2019 02 Probability, Bayes Theorem and the Monty Hall Problem
24/34
PSYC 6130, PROF. J. ELDER 24
Bayes Theorem
Bayes Theorem is simply a consequence of the
definition of conditional probabilities:
( , )
( | ) ( , ) ( | ) ( )( )
p A Bp A B p A B p A B p B
p B
( , )
( | ) ( , ) ( | ) ( )( )
p A Bp B A p A B p B A p A
p A
Thus ( | ) ( ) ( | ) ( )p A B p B p B A p A
( | ) ( )
( | )( )
p B A p Ap A B
p B
Bayes Equation
7/31/2019 02 Probability, Bayes Theorem and the Monty Hall Problem
25/34
PSYC 6130, PROF. J. ELDER 25
Bayes Theorem
Bayes theorem is most commonly used to estimate the
state of a hidden, causal variable Hbased on themeasured state of an observable variable D:
( | ) ( )
( | )( )
p D H p H p H D
p D
Evidence
PriorLikelihood
Posterior
7/31/2019 02 Probability, Bayes Theorem and the Monty Hall Problem
26/34
PSYC 6130, PROF. J. ELDER 26
Bayesian Inference
Whereas the posterior p(H|D) is often difficult to estimatedirectly, reasonable models of the likelihood p(D|H) canoften be formed. This is typically because H is causal onD.
Thus Bayes theorem provides a means for estimating
the posterior probability of the causal variable Hbasedon observations D.
7/31/2019 02 Probability, Bayes Theorem and the Monty Hall Problem
27/34
PSYC 6130, PROF. J. ELDER 27
Marginalizing
To calculate the evidence p(D)in Bayes equation, wetypically have to marginalizeover all possible states ofthe causal variable H.
( | ) ( )( | )( )
p D H p H p H Dp D
1 2
1 1 2 2
( ) ( , ) ( , ) ( , )
( | ) ( ) ( | ) ( ) ( | ) ( )n
n n
p D p D H p D H p D H
p D H p H p D H p H p D H p H
7/31/2019 02 Probability, Bayes Theorem and the Monty Hall Problem
28/34
PSYC 6130, PROF. J. ELDER 28
The Full Monty
Lets get back to The Monty Hall Problem.
Lets assume you initially select Door 1.
Suppose that Monty then opens Door 2 to reveal a goat.
We want to calculate the posteriorprobability that a carlies behind Door 1 after Monty has provided these newdata.
7/31/2019 02 Probability, Bayes Theorem and the Monty Hall Problem
29/34
PSYC 6130, PROF. J. ELDER 29
The Full Monty
Let represent the state that the car lies behind Door , [1,2,3].iC i i
Let represent the event that Monty opens door , [1,2,3],
revealing a goat.
iM i i
2 1 11 2
2
( | ) ( )We seek ( | )( )
p M C p C p C Mp M
7/31/2019 02 Probability, Bayes Theorem and the Monty Hall Problem
30/34
PSYC 6130, PROF. J. ELDER 30
The Full Monty
2 2 3 2 1 2Since ( | ) 0, we can obtain ( | ) by subtracting ( | ) from 1
(Remember that the probabilities of exhaustive events add to 1!)
p C M p C M p C M
3 2However, we can also calculate ( | ) directly:p C M
2 3 33 2
2
( | ) ( )( | )
( )
p M C p C p C M
p M
7/31/2019 02 Probability, Bayes Theorem and the Monty Hall Problem
31/34
PSYC 6130, PROF. J. ELDER 31
But were not on Lets Make a Deal!
Why is the Monty Hall Problem Interesting?
It reveals limitations in human cognitive processing ofuncertainty
It provides a good illustration of many concepts of probability
It get us to think more carefully about how we deal with andexpress uncertainty as scientists.
What else is Bayes theorem good for?
7/31/2019 02 Probability, Bayes Theorem and the Monty Hall Problem
32/34
PSYC 6130, PROF. J. ELDER 32
Clinical Example
Christiansen et al (2000) studied the mammogram results of 2,227women at health centers of Harvard Pilgrim Health Care, a largeHMO in the Boston metropolitan area.
The women received a total of 9,747 mammograms over 10 years.Their ages ranged from 40 to 80. Ninety-three different radiologists
read the mammograms, and overall they diagnosed 634mammograms as suspicious that turned out to be false positives.
This is a false positive rate of 6.5%.
The false negative rate has been estimated at 10%.
7/31/2019 02 Probability, Bayes Theorem and the Monty Hall Problem
33/34
PSYC 6130, PROF. J. ELDER 33
Clinical Example
There are about 58,500,000 women between the ages of40 and 80 in the US
The incidence of breast cancer in the US is about
184,200 per year, i.e., roughly 1 in 318.
7/31/2019 02 Probability, Bayes Theorem and the Monty Hall Problem
34/34
PSYC 6130 PROF J ELDER 34
Clinical Example
0Let represent the absence of cancer.C
0Let represent a negative mammogram result.M
1Let represent a positive mammogram result.M
1Let represent the presence of cancer.C
1 1 1 1Remember: ( | ) ( | )!p C M p M C
Suppose your friend receives a positive mammogram result.
What quantity do you want to compute?