Reasoning with Uncertainty

Reasoning with UncertaintyOften, we want to reason from observable information to unobservable information

We want to calculate how our prior beliefs change given new available evidence

Bayes rule tells us how to optimally reason with uncertainty. Do people reason like Bayes rule?

Bayes Rule Prior probabilityEvidencePosterior ProbabilityBayes rule tells us how the available evidence should alter our belief in something being true

Difficulties in Reasoning with UncertaintyProblems reasoning with probabilitiesfrequencies are easier to understand

Problems understanding Conditional probabilitydoctors need to calculate the probability of disease given the observed symptoms: P( disease | symptoms )

Sometimes P( symptoms | disease ) is used incorrectly when reasoning about the likelihood of a disease

Why is this wrong?

The base rate is importantTo get P( disease | symptom ), you need to know about P( symptom | disease ) and also the base rate -- prevalence of the disease before you have seen patient

More intuitive example:what is the probability of being tall given you are player in the NBA? what is the probability of being a player in the NBA given that you are tall?

P( NBA player | tall ) P( tall | NBA player )

Reasoning with base ratesSuppose there is a disease that affects 1 out of 100 people

There is a diagnostic test with the following properties:If the person has the disease, the test will be positive 98% of the timeif the person does not have the disease, the test will be positive 1% of the time

A person tests positive, what is the probability that this person has the disease?Frequent answer = .98Correct answer .50

Are we really that bad in judging probabilities?According to some researchers (e.g., Gigerenzer), it matters how the information is presented and processed. Processing frequencies is more intuitive than probabilities (even it leads to the same outcome).

A counting heuristic (in tree form)10,000 people100 have disease9,900 do not98 test positive2 test negative99 test positive9801 test negativeP( disease | test positive ) = 98 / ( 98 + 99 ) .50

The same thing in words ...Lets take 10,000 people.

On average, 100 out of 10,000 actually have the disease and 98 of those will test positive (98% true positive rate)

Among the 9,900 who do not have the disease, the test will falsely identify 1% as having it. 1% of 9,900 = 99

On average, out of 10,000 people: 98 test positive and they have the disease 99 test positive and they do not have the disease.

Therefore, a positive test outcome implies a 98/(98+99)50% chance of having the disease

Change the exampleWhat now if the disease affects only 1 out of 10,000 people?

Assume same diagnosticity of test (98% true positive rate, 1% false positive rate)

A person tests positive, what now is the probability that this person has the disease?

A counting heuristic (in tree form)1,000,000 people100 have disease999,900 do not have the disease98 test positive2 test negative9999 test positive989901 test negativeP( disease | test positive ) = 98 / ( 98 + 9999 ) = .0097 (smaller than 1%)

Bayes RuleThe previous example essentially is a simple way to apply Bayes rule:

Normative ModelBayes rule tells you how you should reason with probabilities it is a prescriptive (i.e., normative) model

But do people reason like Bayes? In certain circumstances, the base rates are neglected base rate neglect

The Taxi Problem: version 1A witness sees a crime involving a taxi in Carborough. The witness says that the taxi is blue. It is known from previous research that witnesses are correct 80% of the time when making such statements.

What is the probability that a blue taxi was involved in the crime?

The Taxi Problem: version 2A witness sees a crime involving a taxi in Carborough. The witness says that the taxi is blue. It is known from previous research that witnesses are correct 80% of the time when making such statements. The police also know that 15% of the taxis in Carborough are blue, the other 85% being green. What is the probability that a blue taxi was involved in the crime?

Base Rate Neglect: The Taxi ProblemFailure to take prior probabilities (i.e., base rates) into account

In the taxi story, the addition of:The police also know that 15% of the taxis in Carborough are blue, the other 85% being green.

has little influence on rated probability

Base Rate Neglect (2)Kahneman & Tversky (1973). group A: 70 engineers and 30 lawyers group B: 30 engineers and 70 lawyers

What is probability of picking an engineer in group A and B? Subjects can do this

Provide some evidence Jack is a 45 year-old man. He is married and has four children. He is generally conservative, careful, and ambitious. He shows no interest in political and social issues and spends most of his free time on his many hobbies, which include home carpentry, sailing, and mathematical puzzles

What now is probability Jack is an engineer?

Estimates for both group A and group B was P = .9

Heuristics and BiasesTversky & Kahneman propose that people often do not follow rules of probability

Instead, decision making may be based on heuristics

Lower cognitive load but may lead to systematic errors and biases

Example heuristicsrepresentativenessavailability

All the families having exactly six children in a particular city were surveyed. In 72 of the families, the exact order of the births of boys and girls was: G B G B B G

What is your estimate of the number of families surveyed in which the exact order of births was: B G B B B B

Answer: a) < 72 b) 72 c) >72

Representativeness HeuristicThe sequence G B G B B G is seen as

A) more representative of all possible birth sequences. B) better reflecting the random process of B/G

A coin is flipped. What is a more likely sequence?A) H T H T T HB) H H H H H H

A) #H = 3 and #T = 3 (in some order)B) #H = 6

Gamblers fallacy: wins are perceived to be more likely after a string of losses

Does the hot hand phenomenon exist?Most basketball coaches/players/fans refer to players having a Hot hand or being in a Hot zone and show Streaky shooting

However, making a shot after just making three shots is pretty much as likely as after just missing three shots

not much statistical evidence that basketball players switch between a state of hot hand and cold hand

(Gilovich, Vallone, & Tversky, 1985)

Availability HeuristicAre there more words in the English language that begin with the letter V or that have V as their third letter?

What about the letter R, K, L, and N? (Tversky & Kahneman, 1973)

Linda is 31 years old, single, outspoken, and very bright. She majored in philosophy. As a student, she was deeply concerned with issues of discrimination and social justice, and also participated in anti-nuclear demonstrations.

Rate the likelihood that the following statements about Linda are true:

a) Linda is active in the feminist movement b) Linda is a bank teller c) Linda is a bank teller and is active in the feminist movementRating C as more likely than B and A is a Conjunction Fallacy

What to make of these results?One interpretation of Tversky & Kahnemans findings: people do not use proper probabilistic reasoningpeople use arbitrary mechanisms/ heuristics with no apparent rationale

However, Gigerenzer and Todd show in their Fast and Frugal Heuristics research program that heuristics can often be very effective

Which city has a larger population?A) San DiegoB) San Antonio

66% accuracy with University of Chicago undergraduates. However, 100% accuracy with German students.

San Diego was recognized as American cities by 78% of German students. San Antonio: 4%

With lack of information, use recognition heuristic (Goldstein & Gigerenzer, 2002)

How to pick a stockProblem: what stocks to invest in?

Solution 1optimizing:Gather lots of info about many companiesProcess with sophisticated tools and choose

Solution 2the recognition heuristic:Purchase stocks from recognized companies(slide from Peter Todd)

Paying for the name.(slide from Peter Todd)

Picking Stocks with Recognition HeuristicBorges et al. (1999) can ignorance beat the stock market?180 German lay-people recognition of German stocks6 month return on DAX 30: Dec 1996 Jun 1997

Note: this result has not replicated in other studies (e.g., Boyd, 2001; Rakow, 2002) -- dont rush to use this heuristic on your own money!

Market IndexRecognition Rate > 90%Recognition Rate < 10%+34%+47%+13%

In the probabilistic world in which we operate there are many pieces of potentially useful information we can use one of these is the ability to recognize objects. As we know advertisers capitalise on this by bombarding us with brand names in the hope that brand recognition will influence our behavior some companies such as Benetton take this to extremes with adverts that do not even feature or mention the product being advertised. So how does recognition affect our decision-making? Borges, B., Goldstein, D. G., Ortmann, A., & Gigerenzer, G. (1999). Can ignorance beat the stock market? In Gigerenzer, G., Todd, P. M. & the ABC Group, Simple Heuristics That Make Us Smart. New York: Oxford University Press.