Bayesian Inference in Psychology
Jeffrey N. Rouder
April, 2013
Jeffrey N. Rouder
Bayesian Inference in Psychology
Collaborators
I Richard Morey, Groningen
I Mike Pratte, Vanderbilt
I Jory Province, Mizzou
I Paul Speckman, Mizzou, Stats
I Dongchu Sun, Mizzou, Stats
Jeffrey N. Rouder
Bayesian Inference in Psychology
Crisis of Confidence
I Belief That Evidence For Effects Has Been Overstated
I Publication of Fantastic Extra-Sensory Perception Claims inMainstream Journals
I Several Cases of Outright Fraud
I Crisis in How We Produce, Understand, and EvaluateEvidence.
Jeffrey N. Rouder
Bayesian Inference in Psychology
Crisis of Confidence
I Belief That Evidence For Effects Has Been Overstated
I Publication of Fantastic Extra-Sensory Perception Claims inMainstream Journals
I Several Cases of Outright Fraud
I Crisis in How We Produce, Understand, and EvaluateEvidence.
Jeffrey N. Rouder
Bayesian Inference in Psychology
Crisis of Confidence
I Belief That Evidence For Effects Has Been Overstated
I Publication of Fantastic Extra-Sensory Perception Claims inMainstream Journals
I Several Cases of Outright Fraud
I Crisis in How We Produce, Understand, and EvaluateEvidence.
Jeffrey N. Rouder
Bayesian Inference in Psychology
Crisis of Confidence
I Belief That Evidence For Effects Has Been Overstated
I Publication of Fantastic Extra-Sensory Perception Claims inMainstream Journals
I Several Cases of Outright Fraud
I Crisis in How We Produce, Understand, and EvaluateEvidence.
Jeffrey N. Rouder
Bayesian Inference in Psychology
Crisis of Confidence
I Belief That Evidence For Effects Has Been Overstated
I Publication of Fantastic Extra-Sensory Perception Claims inMainstream Journals
I Several Cases of Outright Fraud
I Crisis in How We Produce, Understand, and EvaluateEvidence.
Jeffrey N. Rouder
Bayesian Inference in Psychology
Crisis of Confidence
The p < .05 Rule
I People Know It is Not Perfect
I Performs Admirably as a Threshold to Keep Spurious FindingsOut of The Literature.
I Provides A Mutual-Assurance Standard
I Easy To Teach
I Feels About Right / Natural
Jeffrey N. Rouder
Bayesian Inference in Psychology
Crisis of Confidence
The p < .05 Rule
I People Know It is Not Perfect
I Performs Admirably as a Threshold to Keep Spurious FindingsOut of The Literature.
I Provides A Mutual-Assurance Standard
I Easy To Teach
I Feels About Right / Natural
Jeffrey N. Rouder
Bayesian Inference in Psychology
Crisis of Confidence
The p < .05 Rule
I People Know It is Not Perfect
I Performs Admirably as a Threshold to Keep Spurious FindingsOut of The Literature.
I Provides A Mutual-Assurance Standard
I Easy To Teach
I Feels About Right / Natural
Jeffrey N. Rouder
Bayesian Inference in Psychology
Crisis of Confidence
The p < .05 Rule
I People Know It is Not Perfect
I Performs Admirably as a Threshold to Keep Spurious FindingsOut of The Literature.
I Provides A Mutual-Assurance Standard
I Easy To Teach
I Feels About Right / Natural
Jeffrey N. Rouder
Bayesian Inference in Psychology
Crisis of Confidence
The p < .05 Rule
I People Know It is Not Perfect
I Performs Admirably as a Threshold to Keep Spurious FindingsOut of The Literature.
I Provides A Mutual-Assurance Standard
I Easy To Teach
I Feels About Right / Natural
Jeffrey N. Rouder
Bayesian Inference in Psychology
Crisis of Confidence
The p < .05 Rule
I People Know It is Not Perfect
I Performs Admirably as a Threshold to Keep Spurious FindingsOut of The Literature.
I Provides A Mutual-Assurance Standard
I Easy To Teach
I Feels About Right / Natural
Jeffrey N. Rouder
Bayesian Inference in Psychology
Crisis of Confidence
I Why the Crisis? What is to be done?
I We are prone to bad practices that violate the basicassumptions of the p < .05 Rule and consequently inflate theprobability of getting an effect.
I Bad practices include censoring data, adding more subjects,ignoring pilots, etc, and are now called p-hacking.
I Solution: Be good.
I Increased focus on recording all data and intent.
I Increased focus on replication
I Prudent to examine the p < .05 Rule.
Jeffrey N. Rouder
Bayesian Inference in Psychology
Crisis of Confidence
I Why the Crisis? What is to be done?
I We are prone to bad practices that violate the basicassumptions of the p < .05 Rule and consequently inflate theprobability of getting an effect.
I Bad practices include censoring data, adding more subjects,ignoring pilots, etc, and are now called p-hacking.
I Solution: Be good.
I Increased focus on recording all data and intent.
I Increased focus on replication
I Prudent to examine the p < .05 Rule.
Jeffrey N. Rouder
Bayesian Inference in Psychology
Crisis of Confidence
I Why the Crisis? What is to be done?
I We are prone to bad practices that violate the basicassumptions of the p < .05 Rule and consequently inflate theprobability of getting an effect.
I Bad practices include censoring data, adding more subjects,ignoring pilots, etc, and are now called p-hacking.
I Solution: Be good.
I Increased focus on recording all data and intent.
I Increased focus on replication
I Prudent to examine the p < .05 Rule.
Jeffrey N. Rouder
Bayesian Inference in Psychology
Crisis of Confidence
I Why the Crisis? What is to be done?
I We are prone to bad practices that violate the basicassumptions of the p < .05 Rule and consequently inflate theprobability of getting an effect.
I Bad practices include censoring data, adding more subjects,ignoring pilots, etc, and are now called p-hacking.
I Solution: Be good.
I Increased focus on recording all data and intent.
I Increased focus on replication
I Prudent to examine the p < .05 Rule.
Jeffrey N. Rouder
Bayesian Inference in Psychology
Crisis of Confidence
I Why the Crisis? What is to be done?
I We are prone to bad practices that violate the basicassumptions of the p < .05 Rule and consequently inflate theprobability of getting an effect.
I Bad practices include censoring data, adding more subjects,ignoring pilots, etc, and are now called p-hacking.
I Solution: Be good.
I Increased focus on recording all data and intent.
I Increased focus on replication
I Prudent to examine the p < .05 Rule.
Jeffrey N. Rouder
Bayesian Inference in Psychology
Crisis of Confidence
I Why the Crisis? What is to be done?
I We are prone to bad practices that violate the basicassumptions of the p < .05 Rule and consequently inflate theprobability of getting an effect.
I Bad practices include censoring data, adding more subjects,ignoring pilots, etc, and are now called p-hacking.
I Solution: Be good.
I Increased focus on recording all data and intent.
I Increased focus on replication
I Prudent to examine the p < .05 Rule.
Jeffrey N. Rouder
Bayesian Inference in Psychology
Crisis of Confidence
I Why the Crisis? What is to be done?
I We are prone to bad practices that violate the basicassumptions of the p < .05 Rule and consequently inflate theprobability of getting an effect.
I Bad practices include censoring data, adding more subjects,ignoring pilots, etc, and are now called p-hacking.
I Solution: Be good.
I Increased focus on recording all data and intent.
I Increased focus on replication
I Prudent to examine the p < .05 Rule.
Jeffrey N. Rouder
Bayesian Inference in Psychology
The Free Lunch
I The p < .05 Rule a “A Free Lunch” Property.
I Inference is based on the null hypothesis alone and the analystneed not make assumptions about the alternative.
I Lack of assumptions about the alternative is the “Free-Lunch”part.
I Contrast with power analysis. Power requires assumptionsabout the alternative, “Paid Lunch.”
I Theme: You Must Pay For Lunch If Inference Is To BePrincipled.
I The free lunch is rotten, and eating it contributes to thestench in the field.
Jeffrey N. Rouder
Bayesian Inference in Psychology
The Free Lunch
I The p < .05 Rule a “A Free Lunch” Property.
I Inference is based on the null hypothesis alone and the analystneed not make assumptions about the alternative.
I Lack of assumptions about the alternative is the “Free-Lunch”part.
I Contrast with power analysis. Power requires assumptionsabout the alternative, “Paid Lunch.”
I Theme: You Must Pay For Lunch If Inference Is To BePrincipled.
I The free lunch is rotten, and eating it contributes to thestench in the field.
Jeffrey N. Rouder
Bayesian Inference in Psychology
The Free Lunch
I The p < .05 Rule a “A Free Lunch” Property.
I Inference is based on the null hypothesis alone and the analystneed not make assumptions about the alternative.
I Lack of assumptions about the alternative is the “Free-Lunch”part.
I Contrast with power analysis. Power requires assumptionsabout the alternative, “Paid Lunch.”
I Theme: You Must Pay For Lunch If Inference Is To BePrincipled.
I The free lunch is rotten, and eating it contributes to thestench in the field.
Jeffrey N. Rouder
Bayesian Inference in Psychology
The Free Lunch
I The p < .05 Rule a “A Free Lunch” Property.
I Inference is based on the null hypothesis alone and the analystneed not make assumptions about the alternative.
I Lack of assumptions about the alternative is the “Free-Lunch”part.
I Contrast with power analysis. Power requires assumptionsabout the alternative, “Paid Lunch.”
I Theme: You Must Pay For Lunch If Inference Is To BePrincipled.
I The free lunch is rotten, and eating it contributes to thestench in the field.
Jeffrey N. Rouder
Bayesian Inference in Psychology
The Free Lunch
I The p < .05 Rule a “A Free Lunch” Property.
I Inference is based on the null hypothesis alone and the analystneed not make assumptions about the alternative.
I Lack of assumptions about the alternative is the “Free-Lunch”part.
I Contrast with power analysis. Power requires assumptionsabout the alternative, “Paid Lunch.”
I Theme: You Must Pay For Lunch If Inference Is To BePrincipled.
I The free lunch is rotten, and eating it contributes to thestench in the field.
Jeffrey N. Rouder
Bayesian Inference in Psychology
The Free Lunch
I The p < .05 Rule a “A Free Lunch” Property.
I Inference is based on the null hypothesis alone and the analystneed not make assumptions about the alternative.
I Lack of assumptions about the alternative is the “Free-Lunch”part.
I Contrast with power analysis. Power requires assumptionsabout the alternative, “Paid Lunch.”
I Theme: You Must Pay For Lunch If Inference Is To BePrincipled.
I The free lunch is rotten, and eating it contributes to thestench in the field.
Jeffrey N. Rouder
Bayesian Inference in Psychology
Outline
I I. Frequentist And Bayesian Probability
I II. What Do We Want To Know: Invariances and Effects
I III. Pay For Lunch: The Frequentist Perspective
I IV. Bayes Factor Inference
Jeffrey N. Rouder
Bayesian Inference in Psychology
Outline
I I. Frequentist And Bayesian Probability
I II. What Do We Want To Know: Invariances and Effects
I III. Pay For Lunch: The Frequentist Perspective
I IV. Bayes Factor Inference
Jeffrey N. Rouder
Bayesian Inference in Psychology
Outline
I I. Frequentist And Bayesian Probability
I II. What Do We Want To Know: Invariances and Effects
I III. Pay For Lunch: The Frequentist Perspective
I IV. Bayes Factor Inference
Jeffrey N. Rouder
Bayesian Inference in Psychology
Outline
I I. Frequentist And Bayesian Probability
I II. What Do We Want To Know: Invariances and Effects
I III. Pay For Lunch: The Frequentist Perspective
I IV. Bayes Factor Inference
Jeffrey N. Rouder
Bayesian Inference in Psychology
Outline
I I. Frequentist And Bayesian Probability
I II. What Do We Want To Know: Invariances and Effects
I III. Pay For Lunch: The Frequentist Perspective
I IV. Bayes Factor Inference
Jeffrey N. Rouder
Bayesian Inference in Psychology
Probability
I Kolmogorov Axioms: Probability as a relative weight.
I Not Enough: The relative weight of the body parts of aperson obey the Kolomogov axioms.
I Needed: One more property to link relative weight touncertainty.
I Bayesian and Frequentists differ on this additional property.
Jeffrey N. Rouder
Bayesian Inference in Psychology
Probability
I Kolmogorov Axioms: Probability as a relative weight.
I Not Enough: The relative weight of the body parts of aperson obey the Kolomogov axioms.
I Needed: One more property to link relative weight touncertainty.
I Bayesian and Frequentists differ on this additional property.
Jeffrey N. Rouder
Bayesian Inference in Psychology
Probability
I Kolmogorov Axioms: Probability as a relative weight.
I Not Enough: The relative weight of the body parts of aperson obey the Kolomogov axioms.
I Needed: One more property to link relative weight touncertainty.
I Bayesian and Frequentists differ on this additional property.
Jeffrey N. Rouder
Bayesian Inference in Psychology
Probability
I Kolmogorov Axioms: Probability as a relative weight.
I Not Enough: The relative weight of the body parts of aperson obey the Kolomogov axioms.
I Needed: One more property to link relative weight touncertainty.
I Bayesian and Frequentists differ on this additional property.
Jeffrey N. Rouder
Bayesian Inference in Psychology
Frequentist Interpretation of Probability
Coin Flip:
I Probability is a long-run property
I# heads
# of flips, limit of lots of flips
I Probability is a property of the coin, much like weight andarea.
I Frequentists Obligation: Get It Right In The Long Run.
Jeffrey N. Rouder
Bayesian Inference in Psychology
Frequentist Interpretation of Probability
Coin Flip:
I Probability is a long-run property
I# heads
# of flips, limit of lots of flips
I Probability is a property of the coin, much like weight andarea.
I Frequentists Obligation: Get It Right In The Long Run.
Jeffrey N. Rouder
Bayesian Inference in Psychology
Frequentist Interpretation of Probability
Coin Flip:
I Probability is a long-run property
I# heads
# of flips, limit of lots of flips
I Probability is a property of the coin, much like weight andarea.
I Frequentists Obligation: Get It Right In The Long Run.
Jeffrey N. Rouder
Bayesian Inference in Psychology
Frequentist Interpretation of Probability
Coin Flip:
I Probability is a long-run property
I# heads
# of flips, limit of lots of flips
I Probability is a property of the coin, much like weight andarea.
I Frequentists Obligation: Get It Right In The Long Run.
Jeffrey N. Rouder
Bayesian Inference in Psychology
Frequentist Interpretation of Probability
Coin Flip:
I Probability is a long-run property
I# heads
# of flips, limit of lots of flips
I Probability is a property of the coin, much like weight andarea.
I Frequentists Obligation: Get It Right In The Long Run.
Jeffrey N. Rouder
Bayesian Inference in Psychology
A Constructive Challenge
I Dylan Byers, Politico Blogger
I Blog Entry Entitled: Nate Silver: One-term Celebrity?
I Context: Who will win the 2012 Presidential Election as seenin late October
Jeffrey N. Rouder
Bayesian Inference in Psychology
A Constructive Challenge
I Dylan Byers, Politico Blogger
I Blog Entry Entitled: Nate Silver: One-term Celebrity?
I Context: Who will win the 2012 Presidential Election as seenin late October
Jeffrey N. Rouder
Bayesian Inference in Psychology
A Constructive Challenge
I Dylan Byers, Politico Blogger
I Blog Entry Entitled: Nate Silver: One-term Celebrity?
I Context: Who will win the 2012 Presidential Election as seenin late October
Jeffrey N. Rouder
Bayesian Inference in Psychology
A Constructive Challenge
I Dylan Byers, Politico Blogger
I Blog Entry Entitled: Nate Silver: One-term Celebrity?
I Context: Who will win the 2012 Presidential Election as seenin late October
Jeffrey N. Rouder
Bayesian Inference in Psychology
Byers’ Column
Silver is an ideologue, wants Obama to win.
I Joe Scarborough, “Nate Silver says this is a 73.6 percentchance that the president is going to win? Nobody in thatcampaign thinks they have a 73 percent chance. They thinkthey have a 50.1 percent chance of winning. And you talk tothe Romney people, it’s the same thing. Both sidesunderstand that it is close, and it could go either way. Andanybody that thinks that this race is anything but a tossupright now is such an ideologue, they should be kept away fromtypewriters, computers, laptops and microphones for the next10 days, because they’re jokes.”
I “And even then [if Obama wins], you won’t know if heactually had a 50.1 percent chance or a 74.6 percent chanceof getting there.”
Jeffrey N. Rouder
Bayesian Inference in Psychology
Byers’ Column
Silver is an ideologue, wants Obama to win.
I Joe Scarborough, “Nate Silver says this is a 73.6 percentchance that the president is going to win? Nobody in thatcampaign thinks they have a 73 percent chance. They thinkthey have a 50.1 percent chance of winning. And you talk tothe Romney people, it’s the same thing. Both sidesunderstand that it is close, and it could go either way. Andanybody that thinks that this race is anything but a tossupright now is such an ideologue, they should be kept away fromtypewriters, computers, laptops and microphones for the next10 days, because they’re jokes.”
I “And even then [if Obama wins], you won’t know if heactually had a 50.1 percent chance or a 74.6 percent chanceof getting there.”
Jeffrey N. Rouder
Bayesian Inference in Psychology
Byers’ Column
Silver is an ideologue, wants Obama to win.
I Joe Scarborough, “Nate Silver says this is a 73.6 percentchance that the president is going to win? Nobody in thatcampaign thinks they have a 73 percent chance. They thinkthey have a 50.1 percent chance of winning. And you talk tothe Romney people, it’s the same thing. Both sidesunderstand that it is close, and it could go either way. Andanybody that thinks that this race is anything but a tossupright now is such an ideologue, they should be kept away fromtypewriters, computers, laptops and microphones for the next10 days, because they’re jokes.”
I “And even then [if Obama wins], you won’t know if heactually had a 50.1 percent chance or a 74.6 percent chanceof getting there.”
Jeffrey N. Rouder
Bayesian Inference in Psychology
Byers’ Column
“And even then [if Obama wins], you won’t know if he actually hada 50.1 percent chance or a 74.6 percent chance of getting there.”
I By symmetry: a 99% chance or a 1% chance
I In what sense is frequentist probability a useful concept here?No large sample limit.
Jeffrey N. Rouder
Bayesian Inference in Psychology
Bayesian Interpretation of Probabilty
I Probability is a statement of belief
I Bayesian focus is on how beliefs should change in light of data.
I Long-run properties are consequences rather than primitives.
I Probability is used to describe the observer or analyst, not thecoin (or system more generally).
Jeffrey N. Rouder
Bayesian Inference in Psychology
Bayesian Interpretation of Probabilty
I Probability is a statement of belief
I Bayesian focus is on how beliefs should change in light of data.
I Long-run properties are consequences rather than primitives.
I Probability is used to describe the observer or analyst, not thecoin (or system more generally).
Jeffrey N. Rouder
Bayesian Inference in Psychology
Bayesian Interpretation of Probabilty
I Probability is a statement of belief
I Bayesian focus is on how beliefs should change in light of data.
I Long-run properties are consequences rather than primitives.
I Probability is used to describe the observer or analyst, not thecoin (or system more generally).
Jeffrey N. Rouder
Bayesian Inference in Psychology
Bayesian Interpretation of Probabilty
I Probability is a statement of belief
I Bayesian focus is on how beliefs should change in light of data.
I Long-run properties are consequences rather than primitives.
I Probability is used to describe the observer or analyst, not thecoin (or system more generally).
Jeffrey N. Rouder
Bayesian Inference in Psychology
Bayesian Probability
Probability is a statement of belief.
I Silver’s p = .75 means 3:1 odds.
I Silver should bet Byers $2.5 to win $1 if Obama winsI Silver should not bet $3.5 to win $1 if Obama wins.
I Probabilities may be placed on anything
I Unreplicated events.I Models, Theories, ConclusionsI How much would you wager for your conclusions to be correct?
What wager should a journal editor accept?
Jeffrey N. Rouder
Bayesian Inference in Psychology
Bayesian Probability
Probability is a statement of belief.I Silver’s p = .75 means 3:1 odds.
I Silver should bet Byers $2.5 to win $1 if Obama winsI Silver should not bet $3.5 to win $1 if Obama wins.
I Probabilities may be placed on anything
I Unreplicated events.I Models, Theories, ConclusionsI How much would you wager for your conclusions to be correct?
What wager should a journal editor accept?
Jeffrey N. Rouder
Bayesian Inference in Psychology
Bayesian Probability
Probability is a statement of belief.I Silver’s p = .75 means 3:1 odds.
I Silver should bet Byers $2.5 to win $1 if Obama winsI Silver should not bet $3.5 to win $1 if Obama wins.
I Probabilities may be placed on anything
I Unreplicated events.I Models, Theories, ConclusionsI How much would you wager for your conclusions to be correct?
What wager should a journal editor accept?
Jeffrey N. Rouder
Bayesian Inference in Psychology
Bayesian Probability
Probability is a statement of belief.I Silver’s p = .75 means 3:1 odds.
I Silver should bet Byers $2.5 to win $1 if Obama winsI Silver should not bet $3.5 to win $1 if Obama wins.
I Probabilities may be placed on anything
I Unreplicated events.I Models, Theories, ConclusionsI How much would you wager for your conclusions to be correct?
What wager should a journal editor accept?
Jeffrey N. Rouder
Bayesian Inference in Psychology
Bayesian Probability
Probability is a statement of belief.I Silver’s p = .75 means 3:1 odds.
I Silver should bet Byers $2.5 to win $1 if Obama winsI Silver should not bet $3.5 to win $1 if Obama wins.
I Probabilities may be placed on anythingI Unreplicated events.I Models, Theories, ConclusionsI How much would you wager for your conclusions to be correct?
What wager should a journal editor accept?
Jeffrey N. Rouder
Bayesian Inference in Psychology
Bayesian Probability
I Bayesian focus is on how beliefs should change in light of data.
I Bayes Rule: How to update beliefs in light of data
I Bayesian Obligation: Use Bayes’ Rule Always.
Jeffrey N. Rouder
Bayesian Inference in Psychology
Bayesian Probability
I Bayesian focus is on how beliefs should change in light of data.
I Bayes Rule: How to update beliefs in light of data
I Bayesian Obligation: Use Bayes’ Rule Always.
Jeffrey N. Rouder
Bayesian Inference in Psychology
Bayesian Probability
I Bayesian focus is on how beliefs should change in light of data.
I Bayes Rule: How to update beliefs in light of data
I Bayesian Obligation: Use Bayes’ Rule Always.
Jeffrey N. Rouder
Bayesian Inference in Psychology
Prior Beliefs: How Well Can I Shoot Free Throws?
0.0 0.2 0.4 0.6 0.8 1.0
0.0
0.5
1.0
1.5
2.0
Probability of Success
Den
sity
Jeffrey N. Rouder
Bayesian Inference in Psychology
Prior Beliefs: How Well Can I Shoot Free Throws?
0.0 0.2 0.4 0.6 0.8 1.0
0.0
0.5
1.0
1.5
2.0
Probability of Success
Den
sity
Uncommited
Jeffrey N. Rouder
Bayesian Inference in Psychology
Prior Beliefs: How Well Can I Shoot Free Throws?
0.0 0.2 0.4 0.6 0.8 1.0
0.0
0.5
1.0
1.5
2.0
Probability of Success
Den
sity
Looks Good
Jeffrey N. Rouder
Bayesian Inference in Psychology
Prior Beliefs: How Well Can I Shoot Free Throws?
0.0 0.2 0.4 0.6 0.8 1.0
0.0
0.5
1.0
1.5
2.0
Probability of Success
Den
sity
Looks Bad
Jeffrey N. Rouder
Bayesian Inference in Psychology
Posterior Beliefs: 8 Makes in 12 Tries
0.0 0.2 0.4 0.6 0.8 1.0
0.0
0.5
1.0
1.5
2.0
Probability of Success
Den
sity
0.0 0.2 0.4 0.6 0.8 1.00
12
34
Probability of Success
Den
sity
Jeffrey N. Rouder
Bayesian Inference in Psychology
Posterior Beliefs: 8 Makes in 12 Tries
0.0 0.2 0.4 0.6 0.8 1.0
0.0
0.5
1.0
1.5
2.0
Probability of Success
Den
sity
0.0 0.2 0.4 0.6 0.8 1.00
12
34
Probability of Success
Den
sity
Jeffrey N. Rouder
Bayesian Inference in Psychology
Posterior Beliefs: 8 Makes in 12 Tries
0.0 0.2 0.4 0.6 0.8 1.0
0.0
0.5
1.0
1.5
2.0
Probability of Success
Den
sity
0.0 0.2 0.4 0.6 0.8 1.00
12
34
Probability of Success
Den
sity
Jeffrey N. Rouder
Bayesian Inference in Psychology
Posterior Beliefs: 8 Makes in 12 Tries
0.0 0.2 0.4 0.6 0.8 1.0
0.0
0.5
1.0
1.5
2.0
2.5
3.0
Probability of Success
Den
sity
0.0 0.2 0.4 0.6 0.8 1.00
24
68
10
Probability of Success
Den
sity
Jeffrey N. Rouder
Bayesian Inference in Psychology
Simplified Example
Do women and men differ in working-memory capacity?
I If significant difference, then go publish.
I If no significant difference, then better luck next time.
I Focus is on effects, not on invariances.
Jeffrey N. Rouder
Bayesian Inference in Psychology
Simplified Example
Do women and men differ in working-memory capacity?
I If significant difference, then go publish.
I If no significant difference, then better luck next time.
I Focus is on effects, not on invariances.
Jeffrey N. Rouder
Bayesian Inference in Psychology
Simplified Example
Do women and men differ in working-memory capacity?
I If significant difference, then go publish.
I If no significant difference, then better luck next time.
I Focus is on effects, not on invariances.
Jeffrey N. Rouder
Bayesian Inference in Psychology
Simplified Example
Do women and men differ in working-memory capacity?
I If significant difference, then go publish.
I If no significant difference, then better luck next time.
I Focus is on effects, not on invariances.
Jeffrey N. Rouder
Bayesian Inference in Psychology
Invariances are at the heart of the physical sciences
Jeffrey N. Rouder
Bayesian Inference in Psychology
Johannes Kepler (1571-1630)
I Planets varied greatly inthe speed & direction oftheir paths through thesky.
I Kepler extracted theinvariants of celestialmotion (e.g., ellipticalorbits, equal areacircumscribed in equaltime).
Jeffrey N. Rouder
Bayesian Inference in Psychology
Johannes Kepler (1571-1630)
I Planets varied greatly inthe speed & direction oftheir paths through thesky.
I Kepler extracted theinvariants of celestialmotion (e.g., ellipticalorbits, equal areacircumscribed in equaltime).
Jeffrey N. Rouder
Bayesian Inference in Psychology
Johannes Kepler (1571-1630)
I Planets varied greatly inthe speed & direction oftheir paths through thesky.
I Kepler extracted theinvariants of celestialmotion (e.g., ellipticalorbits, equal areacircumscribed in equaltime).
Jeffrey N. Rouder
Bayesian Inference in Psychology
Invariances At The Heart of Science
I Conservation Laws: e.g., F = MA implies F1M1
= F2M2
in acommon gravitational field.
I In genetics, adenine binds to thymine, guanine binds tocytosine, across all DNA in all species.
I In chemistry, mechanisms of covalent bonding are the sameacross all atoms.
Jeffrey N. Rouder
Bayesian Inference in Psychology
Invariances At The Heart of Science
I Conservation Laws: e.g., F = MA implies F1M1
= F2M2
in acommon gravitational field.
I In genetics, adenine binds to thymine, guanine binds tocytosine, across all DNA in all species.
I In chemistry, mechanisms of covalent bonding are the sameacross all atoms.
Jeffrey N. Rouder
Bayesian Inference in Psychology
Invariances At The Heart of Science
I Conservation Laws: e.g., F = MA implies F1M1
= F2M2
in acommon gravitational field.
I In genetics, adenine binds to thymine, guanine binds tocytosine, across all DNA in all species.
I In chemistry, mechanisms of covalent bonding are the sameacross all atoms.
Jeffrey N. Rouder
Bayesian Inference in Psychology
Invariances At The Heart of Science
I Conservation Laws: e.g., F = MA implies F1M1
= F2M2
in acommon gravitational field.
I In genetics, adenine binds to thymine, guanine binds tocytosine, across all DNA in all species.
I In chemistry, mechanisms of covalent bonding are the sameacross all atoms.
Jeffrey N. Rouder
Bayesian Inference in Psychology
Hot-Hand Phenomena
I Gilovich, Vallone, and Tversky (1985)
I Hot Hand: The outcome of a shot attempt in basketball iscorrelated with the outcome of previous attempts.
I Conclusion: No such correlation. Probability of success isinvariant to previous history.
Jeffrey N. Rouder
Bayesian Inference in Psychology
Commentary
I Maybe there are no invariances (Cohen)
I Example: Planets don’t follow ellipses to arbitrary precision
I Perhaps there are exact invariances. For example, imaginingyourself winning the lottery may not improve to any degreewhatsoever your chances of winning the lottery.
I Models are not true or false, they just vary in usefulness
I Constructive Challenge: The goal is to find theoretically-usefulplatonic invariances.
Jeffrey N. Rouder
Bayesian Inference in Psychology
Commentary
I Maybe there are no invariances (Cohen)
I Example: Planets don’t follow ellipses to arbitrary precision
I Perhaps there are exact invariances. For example, imaginingyourself winning the lottery may not improve to any degreewhatsoever your chances of winning the lottery.
I Models are not true or false, they just vary in usefulness
I Constructive Challenge: The goal is to find theoretically-usefulplatonic invariances.
Jeffrey N. Rouder
Bayesian Inference in Psychology
Commentary
I Maybe there are no invariances (Cohen)
I Example: Planets don’t follow ellipses to arbitrary precision
I Perhaps there are exact invariances. For example, imaginingyourself winning the lottery may not improve to any degreewhatsoever your chances of winning the lottery.
I Models are not true or false, they just vary in usefulness
I Constructive Challenge: The goal is to find theoretically-usefulplatonic invariances.
Jeffrey N. Rouder
Bayesian Inference in Psychology
Commentary
I Maybe there are no invariances (Cohen)
I Example: Planets don’t follow ellipses to arbitrary precision
I Perhaps there are exact invariances. For example, imaginingyourself winning the lottery may not improve to any degreewhatsoever your chances of winning the lottery.
I Models are not true or false, they just vary in usefulness
I Constructive Challenge: The goal is to find theoretically-usefulplatonic invariances.
Jeffrey N. Rouder
Bayesian Inference in Psychology
Commentary
I Maybe there are no invariances (Cohen)
I Example: Planets don’t follow ellipses to arbitrary precision
I Perhaps there are exact invariances. For example, imaginingyourself winning the lottery may not improve to any degreewhatsoever your chances of winning the lottery.
I Models are not true or false, they just vary in usefulness
I Constructive Challenge: The goal is to find theoretically-usefulplatonic invariances.
Jeffrey N. Rouder
Bayesian Inference in Psychology
I Lawfulness, regularity, invariance, and constraint correspondto null hypotheses.
Jeffrey N. Rouder
Bayesian Inference in Psychology
The p < .05 Rule Is Unhelpful for Stating Evidence for Constraints
I Wrong side of the null
I Can’t state evidence for a null.
I State a lack of evidence for an effect.
Jeffrey N. Rouder
Bayesian Inference in Psychology
The p < .05 Rule Is Unhelpful for Stating Evidence for Constraints
I Wrong side of the null
I Can’t state evidence for a null.
I State a lack of evidence for an effect.
Jeffrey N. Rouder
Bayesian Inference in Psychology
The p < .05 Rule Is Unhelpful for Stating Evidence for Constraints
I Wrong side of the null
I Can’t state evidence for a null.
I State a lack of evidence for an effect.
Jeffrey N. Rouder
Bayesian Inference in Psychology
The p < .05 Rule Is Unhelpful for Stating Evidence for Constraints
I Wrong side of the null
I Can’t state evidence for a null.
I State a lack of evidence for an effect.
Jeffrey N. Rouder
Bayesian Inference in Psychology
Researchers Often Accept The Null
HOW?
I Inspection: “Figure x shows the lack of effect”
I The p > .05 Rule
I Extreme Case: Oberauer & Lewandowsky (2008): p < .05,but the effect is inconsequentially small (“oops, we ran toomany subjects”)
I Unprinicpled
Jeffrey N. Rouder
Bayesian Inference in Psychology
Researchers Often Accept The Null
HOW?
I Inspection: “Figure x shows the lack of effect”
I The p > .05 Rule
I Extreme Case: Oberauer & Lewandowsky (2008): p < .05,but the effect is inconsequentially small (“oops, we ran toomany subjects”)
I Unprinicpled
Jeffrey N. Rouder
Bayesian Inference in Psychology
Researchers Often Accept The Null
HOW?
I Inspection: “Figure x shows the lack of effect”
I The p > .05 Rule
I Extreme Case: Oberauer & Lewandowsky (2008): p < .05,but the effect is inconsequentially small (“oops, we ran toomany subjects”)
I Unprinicpled
Jeffrey N. Rouder
Bayesian Inference in Psychology
Researchers Often Accept The Null
HOW?
I Inspection: “Figure x shows the lack of effect”
I The p > .05 Rule
I Extreme Case: Oberauer & Lewandowsky (2008): p < .05,but the effect is inconsequentially small (“oops, we ran toomany subjects”)
I Unprinicpled
Jeffrey N. Rouder
Bayesian Inference in Psychology
Researchers Often Accept The Null
HOW?
I Inspection: “Figure x shows the lack of effect”
I The p > .05 Rule
I Extreme Case: Oberauer & Lewandowsky (2008): p < .05,but the effect is inconsequentially small (“oops, we ran toomany subjects”)
I Unprinicpled
Jeffrey N. Rouder
Bayesian Inference in Psychology
p < .05 Rule is Useless
I Suppose it is theoretically meaningful to show effects.
I Shouldn’t we use the p < .05 Rule then? No
Jeffrey N. Rouder
Bayesian Inference in Psychology
p < .05 Rule is Useless
I Suppose it is theoretically meaningful to show effects.
I Shouldn’t we use the p < .05 Rule then? No
Jeffrey N. Rouder
Bayesian Inference in Psychology
Consistency
I A consistent method is one that leads to the right decisionalways in the large-sample limit.
I It is a minimal obligation for the interpretation of frequentistprobability.
I Significance Testing is not consistent:
I If there is a true effect, then in the large-sample limit,Pr(reject) = 1
I BUT, if there is no effect, then Pr(reject) = α = .05, even inthe large-sample limit.
I Bias toward rejecting the null.
Jeffrey N. Rouder
Bayesian Inference in Psychology
Consistency
I A consistent method is one that leads to the right decisionalways in the large-sample limit.
I It is a minimal obligation for the interpretation of frequentistprobability.
I Significance Testing is not consistent:
I If there is a true effect, then in the large-sample limit,Pr(reject) = 1
I BUT, if there is no effect, then Pr(reject) = α = .05, even inthe large-sample limit.
I Bias toward rejecting the null.
Jeffrey N. Rouder
Bayesian Inference in Psychology
Consistency
I A consistent method is one that leads to the right decisionalways in the large-sample limit.
I It is a minimal obligation for the interpretation of frequentistprobability.
I Significance Testing is not consistent:
I If there is a true effect, then in the large-sample limit,Pr(reject) = 1
I BUT, if there is no effect, then Pr(reject) = α = .05, even inthe large-sample limit.
I Bias toward rejecting the null.
Jeffrey N. Rouder
Bayesian Inference in Psychology
Consistency
I A consistent method is one that leads to the right decisionalways in the large-sample limit.
I It is a minimal obligation for the interpretation of frequentistprobability.
I Significance Testing is not consistent:
I If there is a true effect, then in the large-sample limit,Pr(reject) = 1
I BUT, if there is no effect, then Pr(reject) = α = .05, even inthe large-sample limit.
I Bias toward rejecting the null.
Jeffrey N. Rouder
Bayesian Inference in Psychology
Consistency
I A consistent method is one that leads to the right decisionalways in the large-sample limit.
I It is a minimal obligation for the interpretation of frequentistprobability.
I Significance Testing is not consistent:I If there is a true effect, then in the large-sample limit,
Pr(reject) = 1
I BUT, if there is no effect, then Pr(reject) = α = .05, even inthe large-sample limit.
I Bias toward rejecting the null.
Jeffrey N. Rouder
Bayesian Inference in Psychology
Consistency
I A consistent method is one that leads to the right decisionalways in the large-sample limit.
I It is a minimal obligation for the interpretation of frequentistprobability.
I Significance Testing is not consistent:I If there is a true effect, then in the large-sample limit,
Pr(reject) = 1I BUT, if there is no effect, then Pr(reject) = α = .05, even in
the large-sample limit.
I Bias toward rejecting the null.
Jeffrey N. Rouder
Bayesian Inference in Psychology
Consistency
I A consistent method is one that leads to the right decisionalways in the large-sample limit.
I It is a minimal obligation for the interpretation of frequentistprobability.
I Significance Testing is not consistent:I If there is a true effect, then in the large-sample limit,
Pr(reject) = 1I BUT, if there is no effect, then Pr(reject) = α = .05, even in
the large-sample limit.
I Bias toward rejecting the null.
Jeffrey N. Rouder
Bayesian Inference in Psychology
Consistent Frequentist Inference
I Usual: α(N) = .05, N is sample size
I To Be Consistent: α(N) must go to 0 as N increases
I Idea: α(N) = min(.05, β(N)), where β(N) is Type II errorrate, or 1-Power
I Consistent because as N increases β(N) goes to zero andα(N) goes to zero too.
I To compute β(N), need to specify an alternative (Pay ForLunch).
I δ, effect size, is .4
Jeffrey N. Rouder
Bayesian Inference in Psychology
Consistent Frequentist Inference
I Usual: α(N) = .05, N is sample size
I To Be Consistent: α(N) must go to 0 as N increases
I Idea: α(N) = min(.05, β(N)), where β(N) is Type II errorrate, or 1-Power
I Consistent because as N increases β(N) goes to zero andα(N) goes to zero too.
I To compute β(N), need to specify an alternative (Pay ForLunch).
I δ, effect size, is .4
Jeffrey N. Rouder
Bayesian Inference in Psychology
Consistent Frequentist Inference
I Usual: α(N) = .05, N is sample size
I To Be Consistent: α(N) must go to 0 as N increases
I Idea: α(N) = min(.05, β(N)), where β(N) is Type II errorrate, or 1-Power
I Consistent because as N increases β(N) goes to zero andα(N) goes to zero too.
I To compute β(N), need to specify an alternative (Pay ForLunch).
I δ, effect size, is .4
Jeffrey N. Rouder
Bayesian Inference in Psychology
Consistent Frequentist Inference
I Usual: α(N) = .05, N is sample size
I To Be Consistent: α(N) must go to 0 as N increases
I Idea: α(N) = min(.05, β(N)), where β(N) is Type II errorrate, or 1-Power
I Consistent because as N increases β(N) goes to zero andα(N) goes to zero too.
I To compute β(N), need to specify an alternative (Pay ForLunch).
I δ, effect size, is .4
Jeffrey N. Rouder
Bayesian Inference in Psychology
Consistent Frequentist Inference
I Usual: α(N) = .05, N is sample size
I To Be Consistent: α(N) must go to 0 as N increases
I Idea: α(N) = min(.05, β(N)), where β(N) is Type II errorrate, or 1-Power
I Consistent because as N increases β(N) goes to zero andα(N) goes to zero too.
I To compute β(N), need to specify an alternative (Pay ForLunch).
I δ, effect size, is .4
Jeffrey N. Rouder
Bayesian Inference in Psychology
Consistent Frequentist Inference
I Usual: α(N) = .05, N is sample size
I To Be Consistent: α(N) must go to 0 as N increases
I Idea: α(N) = min(.05, β(N)), where β(N) is Type II errorrate, or 1-Power
I Consistent because as N increases β(N) goes to zero andα(N) goes to zero too.
I To compute β(N), need to specify an alternative (Pay ForLunch).
I δ, effect size, is .4
Jeffrey N. Rouder
Bayesian Inference in Psychology
A Small Change To Make Significance Testing Consistent
Sample Size
Obs
erve
d E
ffect
Siz
e N
eede
d To
Rej
ect N
ull
0.0
0.2
0.4
0.6
0.8
1.0
5 10 20 50 100 200 500 1000
Jeffrey N. Rouder
Bayesian Inference in Psychology
A Small Change To Make Significance Testing Consistent
Sample Size
Obs
erve
d E
ffect
Siz
e N
eede
d To
Rej
ect N
ull
0.0
0.2
0.4
0.6
0.8
1.0
5 10 20 50 100 200 500 1000
Jeffrey N. Rouder
Bayesian Inference in Psychology
A Small Change To Make Significance Testing Consistent
Sample Size
Obs
erve
d E
ffect
Siz
e N
eede
d To
Rej
ect N
ull
0.0
0.2
0.4
0.6
0.8
1.0
5 10 20 50 100 200 500 1000
α = 0.05α = min (0.05, β)α = β 5
Jeffrey N. Rouder
Bayesian Inference in Psychology
A Small Change To Make Significance Testing Consistent
Sample Size
Obs
erve
d E
ffect
Siz
e N
eede
d To
Rej
ect N
ull
0.0
0.2
0.4
0.6
0.8
1.0
5 10 20 50 100 200 500 1000
α = 0.05α = min (0.05, β)α = β 5
Jeffrey N. Rouder
Bayesian Inference in Psychology
Tail Whispering
I Does whispering “tails” to coins increase the odds of tails?
I 527 out of 1000 flips, p < .05
I Implication: Alternative is substantially more likely than thenull.
I Let q be true probability of a tailH0: q = .5H1: q > .5.
Jeffrey N. Rouder
Bayesian Inference in Psychology
Tail Whispering
I Does whispering “tails” to coins increase the odds of tails?
I 527 out of 1000 flips, p < .05
I Implication: Alternative is substantially more likely than thenull.
I Let q be true probability of a tailH0: q = .5H1: q > .5.
Jeffrey N. Rouder
Bayesian Inference in Psychology
Tail Whispering
I Does whispering “tails” to coins increase the odds of tails?
I 527 out of 1000 flips, p < .05
I Implication: Alternative is substantially more likely than thenull.
I Let q be true probability of a tailH0: q = .5H1: q > .5.
Jeffrey N. Rouder
Bayesian Inference in Psychology
Tail Whispering
I Does whispering “tails” to coins increase the odds of tails?
I 527 out of 1000 flips, p < .05
I Implication: Alternative is substantially more likely than thenull.
I Let q be true probability of a tailH0: q = .5H1: q > .5.
Jeffrey N. Rouder
Bayesian Inference in Psychology
Tail Whispering
I Does whispering “tails” to coins increase the odds of tails?
I 527 out of 1000 flips, p < .05
I Implication: Alternative is substantially more likely than thenull.
I Let q be true probability of a tailH0: q = .5H1: q > .5.
Jeffrey N. Rouder
Bayesian Inference in Psychology
If q = .5
●●●●●●●●●●●
●●
●●
●●
●●
●●
●●
●●
●●
●●
●●
●●
●●
●●●●●●●●●●●●
●●
●●
●●
●●
●●
●●
●●
●●
●●
●●
●●
●●
●●●●●●●●●●
460 480 500 520 540
0.00
50.
015
0.02
5
Outcome
Pro
babi
lity
of O
utco
me
●●
●●
●●●●●●●●●●
Jeffrey N. Rouder
Bayesian Inference in Psychology
Tail Whispering
I Does whispering “tails” to coins increase the odds of tails?
I 527 out of 1000 flips p ≈ .05
I If we reject the null, then for what alternative?
I Let’s look at q = .527, most favorable alternative
Jeffrey N. Rouder
Bayesian Inference in Psychology
Tail Whispering
I Does whispering “tails” to coins increase the odds of tails?
I 527 out of 1000 flips p ≈ .05
I If we reject the null, then for what alternative?
I Let’s look at q = .527, most favorable alternative
Jeffrey N. Rouder
Bayesian Inference in Psychology
q = .5 vs. q = .527
●●●●●●●●●●●
●●
●●
●●
●●
●●
●●
●●
●●
●●
●●
●●
●●
●●●●●●●●●●●●
●●
●●
●●
●●
●●
●●
●●
●●
●●
●●
●●
●●
●●●●●●●●●●
460 480 500 520 540
0.00
50.
015
0.02
5
Outcome
Pro
babi
lity
of O
utco
me
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●
●●
●●
●●
●●
●●
●●
●●
●●
●●
●●
●●
●●●●●●●●●●●●
●●
●●
●●
●
●
●
Jeffrey N. Rouder
Bayesian Inference in Psychology
Overstating Evidence Against The Null
I Does whispering “tails” to coins increase the odds of tails?I 527 out of 1000 flips p ≈ .05
1. q = .5 vs. q = .527, Evidence: 4.3 to 1 for alternativeA researcher should bet $4 to win $1, but not $5 to win $1.
2. q = .5 vs. q = .55, Evidence: 1.5 to 1 for alternative3. q = .5 vs .5 < q < .6, Evidence: 6.2 to 1 for null
I No alternative is much better than the null
I Many reasonable ones are less concordant with the data thanthe null.
Jeffrey N. Rouder
Bayesian Inference in Psychology
Overstating Evidence Against The Null
I Does whispering “tails” to coins increase the odds of tails?I 527 out of 1000 flips p ≈ .05
1. q = .5 vs. q = .527, Evidence: 4.3 to 1 for alternativeA researcher should bet $4 to win $1, but not $5 to win $1.
2. q = .5 vs. q = .55, Evidence: 1.5 to 1 for alternative
3. q = .5 vs .5 < q < .6, Evidence: 6.2 to 1 for null
I No alternative is much better than the null
I Many reasonable ones are less concordant with the data thanthe null.
Jeffrey N. Rouder
Bayesian Inference in Psychology
Overstating Evidence Against The Null
I Does whispering “tails” to coins increase the odds of tails?I 527 out of 1000 flips p ≈ .05
1. q = .5 vs. q = .527, Evidence: 4.3 to 1 for alternativeA researcher should bet $4 to win $1, but not $5 to win $1.
2. q = .5 vs. q = .55, Evidence: 1.5 to 1 for alternative3. q = .5 vs .5 < q < .6, Evidence: 6.2 to 1 for null
I No alternative is much better than the null
I Many reasonable ones are less concordant with the data thanthe null.
Jeffrey N. Rouder
Bayesian Inference in Psychology
Overstating Evidence Against The Null
I Does whispering “tails” to coins increase the odds of tails?I 527 out of 1000 flips p ≈ .05
1. q = .5 vs. q = .527, Evidence: 4.3 to 1 for alternativeA researcher should bet $4 to win $1, but not $5 to win $1.
2. q = .5 vs. q = .55, Evidence: 1.5 to 1 for alternative3. q = .5 vs .5 < q < .6, Evidence: 6.2 to 1 for null
I No alternative is much better than the null
I Many reasonable ones are less concordant with the data thanthe null.
Jeffrey N. Rouder
Bayesian Inference in Psychology
Overstating Evidence Against The Null
I Does whispering “tails” to coins increase the odds of tails?I 527 out of 1000 flips p ≈ .05
1. q = .5 vs. q = .527, Evidence: 4.3 to 1 for alternativeA researcher should bet $4 to win $1, but not $5 to win $1.
2. q = .5 vs. q = .55, Evidence: 1.5 to 1 for alternative3. q = .5 vs .5 < q < .6, Evidence: 6.2 to 1 for null
I No alternative is much better than the null
I Many reasonable ones are less concordant with the data thanthe null.
Jeffrey N. Rouder
Bayesian Inference in Psychology
Overstating-Evidence Argument
0.0 0.2 0.4 0.6 0.8 1.0
05
1015
p−Value
Den
sity
Eff.Size=.2, N=50Eff.Size=.2, N=500Null
Jeffrey N. Rouder
Bayesian Inference in Psychology
Overstating-Evidence Argument
0.00 0.02 0.04 0.06 0.08
01
23
45
p−Value
Den
sity
Eff.Size=.2, N=50Null
Jeffrey N. Rouder
Bayesian Inference in Psychology
Overstating-Evidence Argument
0.00 0.02 0.04 0.06 0.08
01
23
45
p−Value
Den
sity
Eff.Size=.2, N=500Null
Jeffrey N. Rouder
Bayesian Inference in Psychology
Interim Summary
I The p < .05 Rule is unprincipled. Overstates evidence againstnull.
I Principled frequentist inference requires paying for lunch.
Jeffrey N. Rouder
Bayesian Inference in Psychology
Interim Summary
I The p < .05 Rule is unprincipled. Overstates evidence againstnull.
I Principled frequentist inference requires paying for lunch.
Jeffrey N. Rouder
Bayesian Inference in Psychology
Bayesians place beliefs on models
I Odds
I Prior Beliefs:P(M0)
P(M1)
I Posterior Beliefs:P(M0|Data)
P(M1|Data)
I Use Bayes Theorem to Update Beliefs:
P(M0|Data)
P(M1|Data)=
P(Data|M0)
P(Data|M1)× P(M0)
P(M1)
Jeffrey N. Rouder
Bayesian Inference in Psychology
Bayesians place beliefs on models
I Odds
I Prior Beliefs:P(M0)
P(M1)
I Posterior Beliefs:P(M0|Data)
P(M1|Data)
I Use Bayes Theorem to Update Beliefs:
P(M0|Data)
P(M1|Data)=
P(Data|M0)
P(Data|M1)× P(M0)
P(M1)
Jeffrey N. Rouder
Bayesian Inference in Psychology
Bayesians place beliefs on models
I Odds
I Prior Beliefs:P(M0)
P(M1)
I Posterior Beliefs:P(M0|Data)
P(M1|Data)
I Use Bayes Theorem to Update Beliefs:
P(M0|Data)
P(M1|Data)=
P(Data|M0)
P(Data|M1)× P(M0)
P(M1)
Jeffrey N. Rouder
Bayesian Inference in Psychology
Bayesians place beliefs on models
I Odds
I Prior Beliefs:P(M0)
P(M1)
I Posterior Beliefs:P(M0|Data)
P(M1|Data)
I Use Bayes Theorem to Update Beliefs:
P(M0|Data)
P(M1|Data)=
P(Data|M0)
P(Data|M1)× P(M0)
P(M1)
Jeffrey N. Rouder
Bayesian Inference in Psychology
Bayesians place beliefs on models
I Odds
I Prior Beliefs:P(M0)
P(M1)
I Posterior Beliefs:P(M0|Data)
P(M1|Data)
I Use Bayes Theorem to Update Beliefs:
P(M0|Data)
P(M1|Data)=
P(Data|M0)
P(Data|M1)× P(M0)
P(M1)
Jeffrey N. Rouder
Bayesian Inference in Psychology
Bayes Factor: A Fully Bayesian Approach
I Use Bayes Theorem to Update Beliefs:
P(M0|Data)
P(M1|Data)=
P(Data|M0)
P(Data|M1)× P(M0)
P(M1)
I Bayes factors are the updating factor
B01 =P(Data|M0)
P(Data|M1).
Jeffrey N. Rouder
Bayesian Inference in Psychology
Bayes Factor: A Fully Bayesian Approach
I Use Bayes Theorem to Update Beliefs:
P(M0|Data)
P(M1|Data)=
P(Data|M0)
P(Data|M1)× P(M0)
P(M1)
I Bayes factors are the updating factor
B01 =P(Data|M0)
P(Data|M1).
Jeffrey N. Rouder
Bayesian Inference in Psychology
Bayes Factors Change In Beliefs
P(M0|Data)
P(M1|Data)= B01 ×
P(M0)
P(M1)
I Bayes factor is the change in belief due to the data
I Bayes factor is independent of prior odds
I Ratios can be interpreted in terms of wagers.
I Prior odds are a good place to add value.
I B10 = 1/B01
Jeffrey N. Rouder
Bayesian Inference in Psychology
Bayes Factors Change In Beliefs
P(M0|Data)
P(M1|Data)= B01 ×
P(M0)
P(M1)
I Bayes factor is the change in belief due to the data
I Bayes factor is independent of prior odds
I Ratios can be interpreted in terms of wagers.
I Prior odds are a good place to add value.
I B10 = 1/B01
Jeffrey N. Rouder
Bayesian Inference in Psychology
Bayes Factors Change In Beliefs
P(M0|Data)
P(M1|Data)= B01 ×
P(M0)
P(M1)
I Bayes factor is the change in belief due to the data
I Bayes factor is independent of prior odds
I Ratios can be interpreted in terms of wagers.
I Prior odds are a good place to add value.
I B10 = 1/B01
Jeffrey N. Rouder
Bayesian Inference in Psychology
Bayes Factors Change In Beliefs
P(M0|Data)
P(M1|Data)= B01 ×
P(M0)
P(M1)
I Bayes factor is the change in belief due to the data
I Bayes factor is independent of prior odds
I Ratios can be interpreted in terms of wagers.
I Prior odds are a good place to add value.
I B10 = 1/B01
Jeffrey N. Rouder
Bayesian Inference in Psychology
Bayes Factors Change In Beliefs
P(M0|Data)
P(M1|Data)= B01 ×
P(M0)
P(M1)
I Bayes factor is the change in belief due to the data
I Bayes factor is independent of prior odds
I Ratios can be interpreted in terms of wagers.
I Prior odds are a good place to add value.
I B10 = 1/B01
Jeffrey N. Rouder
Bayesian Inference in Psychology
Bayes Factors Change In Beliefs
P(M0|Data)
P(M1|Data)= B01 ×
P(M0)
P(M1)
I Bayes factor is the change in belief due to the data
I Bayes factor is independent of prior odds
I Ratios can be interpreted in terms of wagers.
I Prior odds are a good place to add value.
I B10 = 1/B01
Jeffrey N. Rouder
Bayesian Inference in Psychology
Bayes Factor & Priors on Parameters
What is P(Data|M)?
I Models have parameters: θ
I P(Data|θ) is easy. Likelihood, L(θ).
I If we define the null and alternative as specific parametervalues θ0 and θ1, then
B01 =P(Data|M0)
P(Data|M1)=
L(θ0)
L(θ1)
Jeffrey N. Rouder
Bayesian Inference in Psychology
Bayes Factor & Priors on Parameters
What is P(Data|M)?
I Models have parameters: θ
I P(Data|θ) is easy. Likelihood, L(θ).
I If we define the null and alternative as specific parametervalues θ0 and θ1, then
B01 =P(Data|M0)
P(Data|M1)=
L(θ0)
L(θ1)
Jeffrey N. Rouder
Bayesian Inference in Psychology
Bayes Factor & Priors on Parameters
What is P(Data|M)?
I Models have parameters: θ
I P(Data|θ) is easy. Likelihood, L(θ).
I If we define the null and alternative as specific parametervalues θ0 and θ1, then
B01 =P(Data|M0)
P(Data|M1)=
L(θ0)
L(θ1)
Jeffrey N. Rouder
Bayesian Inference in Psychology
Bayes Factor & Priors on Parameters
What is P(Data|M)?
I Models have parameters: θ
I P(Data|θ) is easy. Likelihood, L(θ).
I If we define the null and alternative as specific parametervalues θ0 and θ1, then
B01 =P(Data|M0)
P(Data|M1)=
L(θ0)
L(θ1)
Jeffrey N. Rouder
Bayesian Inference in Psychology
Bayes Factor & Priors on Parameters
Parameters can take on more than a point:
B01 =P(Data|M0)
P(Data|M1)=
∫θ L0(θ0)f0(θ0)dθ0∫θ L1(θ1)f1(θ1)dθ1
I Marginal or averaged likelihood of a model with respect to theprior f0(θ0) or f1(θ1).
I If priors are too broad, then the average will include manyparameter values with low likelihood.
I Specification of priors is paying for lunch.
Jeffrey N. Rouder
Bayesian Inference in Psychology
Bayes Factor & Priors on Parameters
Parameters can take on more than a point:
B01 =P(Data|M0)
P(Data|M1)=
∫θ L0(θ0)f0(θ0)dθ0∫θ L1(θ1)f1(θ1)dθ1
I Marginal or averaged likelihood of a model with respect to theprior f0(θ0) or f1(θ1).
I If priors are too broad, then the average will include manyparameter values with low likelihood.
I Specification of priors is paying for lunch.
Jeffrey N. Rouder
Bayesian Inference in Psychology
Bayes Factor & Priors on Parameters
Parameters can take on more than a point:
B01 =P(Data|M0)
P(Data|M1)=
∫θ L0(θ0)f0(θ0)dθ0∫θ L1(θ1)f1(θ1)dθ1
I Marginal or averaged likelihood of a model with respect to theprior f0(θ0) or f1(θ1).
I If priors are too broad, then the average will include manyparameter values with low likelihood.
I Specification of priors is paying for lunch.
Jeffrey N. Rouder
Bayesian Inference in Psychology
Bayes Factor & Priors on Parameters
Parameters can take on more than a point:
B01 =P(Data|M0)
P(Data|M1)=
∫θ L0(θ0)f0(θ0)dθ0∫θ L1(θ1)f1(θ1)dθ1
I Marginal or averaged likelihood of a model with respect to theprior f0(θ0) or f1(θ1).
I If priors are too broad, then the average will include manyparameter values with low likelihood.
I Specification of priors is paying for lunch.
Jeffrey N. Rouder
Bayesian Inference in Psychology
Bayes Factor & Priors on Parameters
Toy Problem:
I yi ∼ Normal(µ, 1)
I Null, M0 : µ = 0.
I Alternative, M1 : µ ∼ Normal(0, σ20).
I Specification of σ20 is paying for lunch.
Jeffrey N. Rouder
Bayesian Inference in Psychology
Bayes Factor & Priors on Parameters
Prior Standard Deviation sigma_0
Bay
es F
acto
r B
10
0.01 0.1 1 10 100 1000 1e+05
1e−
050.
011
100
1e+
05
y = 0
y = 0.15
y = 0.22
Jeffrey N. Rouder
Bayesian Inference in Psychology
Consequence #1: Sample Size Considerations
Sample Size
Crit
ical
t−va
lue
5 10 20 50 100 500 2000 5000
23
45
6
B10 = 10B10 = 3B10 = 1p−value = .05
Jeffrey N. Rouder
Bayesian Inference in Psychology
Consequence #2: Calibration in Real Data
p−Value
Bay
es F
acto
r
0.5
110
40
0.001 0.01 0.05 0.1
Jeffrey N. Rouder
Bayesian Inference in Psychology
Consequence #3: Respects Resolution of Data
●●
●
●●
●●
●● ● ●
●
●
●
Bay
es F
acto
r
5 10 20 50 100
200
500
1000
2000
5000
1000
0
2000
0
5000
0
1e+
05
0.01
0.1
1
10
Sample Size
Jeffrey N. Rouder
Bayesian Inference in Psychology
Bayes Factor
A Most Excellent Means of Model Comparison:
I Principle: Update Beliefs Rationally in Light of Data
I State evidence, avoid decisions
I Evidence is in odds, natural language in terms of wagers.
I State evidence for or against invariances, as dictated by data
I Context through prior odds
I Natural penalty for model complexity.
I Respects resolution of data; best account of data.
Jeffrey N. Rouder
Bayesian Inference in Psychology
Bayes Factor
A Most Excellent Means of Model Comparison:
I Principle: Update Beliefs Rationally in Light of Data
I State evidence, avoid decisions
I Evidence is in odds, natural language in terms of wagers.
I State evidence for or against invariances, as dictated by data
I Context through prior odds
I Natural penalty for model complexity.
I Respects resolution of data; best account of data.
Jeffrey N. Rouder
Bayesian Inference in Psychology
Bayes Factor
A Most Excellent Means of Model Comparison:
I Principle: Update Beliefs Rationally in Light of Data
I State evidence, avoid decisions
I Evidence is in odds, natural language in terms of wagers.
I State evidence for or against invariances, as dictated by data
I Context through prior odds
I Natural penalty for model complexity.
I Respects resolution of data; best account of data.
Jeffrey N. Rouder
Bayesian Inference in Psychology
Bayes Factor
A Most Excellent Means of Model Comparison:
I Principle: Update Beliefs Rationally in Light of Data
I State evidence, avoid decisions
I Evidence is in odds, natural language in terms of wagers.
I State evidence for or against invariances, as dictated by data
I Context through prior odds
I Natural penalty for model complexity.
I Respects resolution of data; best account of data.
Jeffrey N. Rouder
Bayesian Inference in Psychology
Bayes Factor
A Most Excellent Means of Model Comparison:
I Principle: Update Beliefs Rationally in Light of Data
I State evidence, avoid decisions
I Evidence is in odds, natural language in terms of wagers.
I State evidence for or against invariances, as dictated by data
I Context through prior odds
I Natural penalty for model complexity.
I Respects resolution of data; best account of data.
Jeffrey N. Rouder
Bayesian Inference in Psychology
Bayes Factor
A Most Excellent Means of Model Comparison:
I Principle: Update Beliefs Rationally in Light of Data
I State evidence, avoid decisions
I Evidence is in odds, natural language in terms of wagers.
I State evidence for or against invariances, as dictated by data
I Context through prior odds
I Natural penalty for model complexity.
I Respects resolution of data; best account of data.
Jeffrey N. Rouder
Bayesian Inference in Psychology
Bayes Factor
A Most Excellent Means of Model Comparison:
I Principle: Update Beliefs Rationally in Light of Data
I State evidence, avoid decisions
I Evidence is in odds, natural language in terms of wagers.
I State evidence for or against invariances, as dictated by data
I Context through prior odds
I Natural penalty for model complexity.
I Respects resolution of data; best account of data.
Jeffrey N. Rouder
Bayesian Inference in Psychology
Bayes Factor
A Most Excellent Means of Model Comparison:
I Principle: Update Beliefs Rationally in Light of Data
I State evidence, avoid decisions
I Evidence is in odds, natural language in terms of wagers.
I State evidence for or against invariances, as dictated by data
I Context through prior odds
I Natural penalty for model complexity.
I Respects resolution of data; best account of data.
Jeffrey N. Rouder
Bayesian Inference in Psychology
Bayes Factor Computations
I Bayes factor computations are hard because one mustintegrate out all of the parameters
I My group knows how to do it for:
I t-tests (paired and grouped)Web applet at pcl.missouri.edu
I Multiple regressionWeb applet at pcl.missouri.edu
I ANOVA (within, between, mixed)BayesFactor package for R
Jeffrey N. Rouder
Bayesian Inference in Psychology
Bayes Factor Computations
I Bayes factor computations are hard because one mustintegrate out all of the parameters
I My group knows how to do it for:
I t-tests (paired and grouped)Web applet at pcl.missouri.edu
I Multiple regressionWeb applet at pcl.missouri.edu
I ANOVA (within, between, mixed)BayesFactor package for R
Jeffrey N. Rouder
Bayesian Inference in Psychology
Bayes Factor Computations
I Bayes factor computations are hard because one mustintegrate out all of the parameters
I My group knows how to do it for:
I t-tests (paired and grouped)Web applet at pcl.missouri.edu
I Multiple regressionWeb applet at pcl.missouri.edu
I ANOVA (within, between, mixed)BayesFactor package for R
Jeffrey N. Rouder
Bayesian Inference in Psychology
Bayes Factor Computations
I Bayes factor computations are hard because one mustintegrate out all of the parameters
I My group knows how to do it for:I t-tests (paired and grouped)
Web applet at pcl.missouri.edu
I Multiple regressionWeb applet at pcl.missouri.edu
I ANOVA (within, between, mixed)BayesFactor package for R
Jeffrey N. Rouder
Bayesian Inference in Psychology
Bayes Factor Computations
I Bayes factor computations are hard because one mustintegrate out all of the parameters
I My group knows how to do it for:I t-tests (paired and grouped)
Web applet at pcl.missouri.eduI Multiple regression
Web applet at pcl.missouri.edu
I ANOVA (within, between, mixed)BayesFactor package for R
Jeffrey N. Rouder
Bayesian Inference in Psychology
Bayes Factor Computations
I Bayes factor computations are hard because one mustintegrate out all of the parameters
I My group knows how to do it for:I t-tests (paired and grouped)
Web applet at pcl.missouri.eduI Multiple regression
Web applet at pcl.missouri.eduI ANOVA (within, between, mixed)
BayesFactor package for R
Jeffrey N. Rouder
Bayesian Inference in Psychology
Other “Bayesian” Competitors
I Some analysts recommend using Bayesian estimation butNOT Bayes factors.
I Focus on Kruschke as his approach is now the recommendedone for The Psychonomic Society
Jeffrey N. Rouder
Bayesian Inference in Psychology
Parameter µ
Pos
terio
r D
ensi
ty
−2 0 2 4 6
95% of Area
Jeffrey N. Rouder
Bayesian Inference in Psychology
Parameter µ
Pos
terio
r D
ensi
ty
−2 0 2 4 6
95% of Area
Jeffrey N. Rouder
Bayesian Inference in Psychology
Other “Bayesian” Competitors
I Credible Interval Logic: Make a decision unfavorable towardthe null if it is not in the 95% posterior credible interval.Proxy for the null having low plausibility.
I Q. How plausible does one believe the null is before collectingdata?
I A. Because the priors are flat, the probability that µ is in afinite interval is 0. So, a priori one is sure there is an effect.
I Q. How are these beliefs updated in light of data.
I A. The posterior probability that µ is in a finite interval isfinite. So a posteriori, one is no longer sure there is an effect.Thus, beliefs were updated by an infinite factor incontradiction of Bayes Rule.
Jeffrey N. Rouder
Bayesian Inference in Psychology
Other “Bayesian” Competitors
I Credible Interval Logic: Make a decision unfavorable towardthe null if it is not in the 95% posterior credible interval.Proxy for the null having low plausibility.
I Q. How plausible does one believe the null is before collectingdata?
I A. Because the priors are flat, the probability that µ is in afinite interval is 0. So, a priori one is sure there is an effect.
I Q. How are these beliefs updated in light of data.
I A. The posterior probability that µ is in a finite interval isfinite. So a posteriori, one is no longer sure there is an effect.Thus, beliefs were updated by an infinite factor incontradiction of Bayes Rule.
Jeffrey N. Rouder
Bayesian Inference in Psychology
Other “Bayesian” Competitors
I Credible Interval Logic: Make a decision unfavorable towardthe null if it is not in the 95% posterior credible interval.Proxy for the null having low plausibility.
I Q. How plausible does one believe the null is before collectingdata?
I A. Because the priors are flat, the probability that µ is in afinite interval is 0. So, a priori one is sure there is an effect.
I Q. How are these beliefs updated in light of data.
I A. The posterior probability that µ is in a finite interval isfinite. So a posteriori, one is no longer sure there is an effect.Thus, beliefs were updated by an infinite factor incontradiction of Bayes Rule.
Jeffrey N. Rouder
Bayesian Inference in Psychology
Other “Bayesian” Competitors
I Credible Interval Logic: Make a decision unfavorable towardthe null if it is not in the 95% posterior credible interval.Proxy for the null having low plausibility.
I Q. How plausible does one believe the null is before collectingdata?
I A. Because the priors are flat, the probability that µ is in afinite interval is 0. So, a priori one is sure there is an effect.
I Q. How are these beliefs updated in light of data.
I A. The posterior probability that µ is in a finite interval isfinite. So a posteriori, one is no longer sure there is an effect.Thus, beliefs were updated by an infinite factor incontradiction of Bayes Rule.
Jeffrey N. Rouder
Bayesian Inference in Psychology
Other “Bayesian” Competitors
I Credible Interval Logic: Make a decision unfavorable towardthe null if it is not in the 95% posterior credible interval.Proxy for the null having low plausibility.
I Q. How plausible does one believe the null is before collectingdata?
I A. Because the priors are flat, the probability that µ is in afinite interval is 0. So, a priori one is sure there is an effect.
I Q. How are these beliefs updated in light of data.
I A. The posterior probability that µ is in a finite interval isfinite. So a posteriori, one is no longer sure there is an effect.Thus, beliefs were updated by an infinite factor incontradiction of Bayes Rule.
Jeffrey N. Rouder
Bayesian Inference in Psychology
Other “Bayesian” Competitors
I Credible Interval Logic: Make a decision unfavorable towardthe null if it is not in the 95% posterior credible interval.Proxy for the null having low plausibility.
I Q. How plausible does one believe the null is before collectingdata?
I A. Because the priors are flat, the probability that µ is in afinite interval is 0. So, a priori one is sure there is an effect.
I Q. How are these beliefs updated in light of data.
I A. The posterior probability that µ is in a finite interval isfinite. So a posteriori, one is no longer sure there is an effect.Thus, beliefs were updated by an infinite factor incontradiction of Bayes Rule.
Jeffrey N. Rouder
Bayesian Inference in Psychology
Pay For Lunch...
I Frequentist:
I Consistent Inference is Possible When the Null and AlternativeAre Well Specified.
I Principled Frequentist Analysts Pay For Lunch.
I Bayesian:
I Proper Updating Is Possible When the Null and AlternativeAre Well Specified.
I Principled Bayesian Analysts Pay For Lunch.
I Or Don’t Eat.
Jeffrey N. Rouder
Bayesian Inference in Psychology
Pay For Lunch...
I Frequentist:I Consistent Inference is Possible When the Null and Alternative
Are Well Specified.I Principled Frequentist Analysts Pay For Lunch.
I Bayesian:
I Proper Updating Is Possible When the Null and AlternativeAre Well Specified.
I Principled Bayesian Analysts Pay For Lunch.
I Or Don’t Eat.
Jeffrey N. Rouder
Bayesian Inference in Psychology
Pay For Lunch...
I Frequentist:I Consistent Inference is Possible When the Null and Alternative
Are Well Specified.I Principled Frequentist Analysts Pay For Lunch.
I Bayesian:I Proper Updating Is Possible When the Null and Alternative
Are Well Specified.I Principled Bayesian Analysts Pay For Lunch.
I Or Don’t Eat.
Jeffrey N. Rouder
Bayesian Inference in Psychology
Pay For Lunch...
I Frequentist:I Consistent Inference is Possible When the Null and Alternative
Are Well Specified.I Principled Frequentist Analysts Pay For Lunch.
I Bayesian:I Proper Updating Is Possible When the Null and Alternative
Are Well Specified.I Principled Bayesian Analysts Pay For Lunch.
I Or Don’t Eat.
Jeffrey N. Rouder
Bayesian Inference in Psychology
Does the Free Lunch Make Us Ill?
I The free-lunch is mindless. Usage promotes a shallow if notcavalier attitude toward analysis.
I With the free lunch, there is a mental distance between whatwe know and what we can show.
I It is in this mental distance between what we know and whatwe can show that we may compromise our methodology.
Jeffrey N. Rouder
Bayesian Inference in Psychology
Does the Free Lunch Make Us Ill?
I The free-lunch is mindless. Usage promotes a shallow if notcavalier attitude toward analysis.
I With the free lunch, there is a mental distance between whatwe know and what we can show.
I It is in this mental distance between what we know and whatwe can show that we may compromise our methodology.
Jeffrey N. Rouder
Bayesian Inference in Psychology
Does the Free Lunch Make Us Ill?
I The free-lunch is mindless. Usage promotes a shallow if notcavalier attitude toward analysis.
I With the free lunch, there is a mental distance between whatwe know and what we can show.
I It is in this mental distance between what we know and whatwe can show that we may compromise our methodology.
Jeffrey N. Rouder
Bayesian Inference in Psychology
Does the Free Lunch Make Us Ill?
I The free-lunch is mindless. Usage promotes a shallow if notcavalier attitude toward analysis.
I With the free lunch, there is a mental distance between whatwe know and what we can show.
I It is in this mental distance between what we know and whatwe can show that we may compromise our methodology.
Jeffrey N. Rouder
Bayesian Inference in Psychology
Current Psychonomic Society Guidelines
I The Psychomonic Society is a great organization whereexceptional people give freely and generously of their time andskills. The guidelines committee worked hard and I appreciatetheir efforts.
I Many of the current guidelines are reasonable when takenindividually
I Unaddressed is the free-lunch problem, which is a matter ofprinciple.
I The guidelines may be summarized as, “You can eat the freelunch, just clean up your crumbs and leave the lunchroomlooking nice.”
Jeffrey N. Rouder
Bayesian Inference in Psychology
Current Psychonomic Society Guidelines
I The Psychomonic Society is a great organization whereexceptional people give freely and generously of their time andskills. The guidelines committee worked hard and I appreciatetheir efforts.
I Many of the current guidelines are reasonable when takenindividually
I Unaddressed is the free-lunch problem, which is a matter ofprinciple.
I The guidelines may be summarized as, “You can eat the freelunch, just clean up your crumbs and leave the lunchroomlooking nice.”
Jeffrey N. Rouder
Bayesian Inference in Psychology
Current Psychonomic Society Guidelines
I The Psychomonic Society is a great organization whereexceptional people give freely and generously of their time andskills. The guidelines committee worked hard and I appreciatetheir efforts.
I Many of the current guidelines are reasonable when takenindividually
I Unaddressed is the free-lunch problem, which is a matter ofprinciple.
I The guidelines may be summarized as, “You can eat the freelunch, just clean up your crumbs and leave the lunchroomlooking nice.”
Jeffrey N. Rouder
Bayesian Inference in Psychology
Current Psychonomic Society Guidelines
I The Psychomonic Society is a great organization whereexceptional people give freely and generously of their time andskills. The guidelines committee worked hard and I appreciatetheir efforts.
I Many of the current guidelines are reasonable when takenindividually
I Unaddressed is the free-lunch problem, which is a matter ofprinciple.
I The guidelines may be summarized as, “You can eat the freelunch, just clean up your crumbs and leave the lunchroomlooking nice.”
Jeffrey N. Rouder
Bayesian Inference in Psychology
My Guidelines
I Good scientists do not believe that science is objective; goodanalysts do not believe that inference is objective.
I Good analysts add value by considering well-specified modelsthat embed meaningful constraint. Good analysts statesupport for or against meaningful constraint by consideringjudiciously chosen alternatives.
I Good analysts are transparent in and responsible for theirchoices.
I Good readers have the responsibility for critically assessing theanalyst’s choice of models as well as for forming a personalopinion about the interpretation of evidence.
Jeffrey N. Rouder
Bayesian Inference in Psychology
My Guidelines
I Good scientists do not believe that science is objective; goodanalysts do not believe that inference is objective.
I Good analysts add value by considering well-specified modelsthat embed meaningful constraint. Good analysts statesupport for or against meaningful constraint by consideringjudiciously chosen alternatives.
I Good analysts are transparent in and responsible for theirchoices.
I Good readers have the responsibility for critically assessing theanalyst’s choice of models as well as for forming a personalopinion about the interpretation of evidence.
Jeffrey N. Rouder
Bayesian Inference in Psychology
My Guidelines
I Good scientists do not believe that science is objective; goodanalysts do not believe that inference is objective.
I Good analysts add value by considering well-specified modelsthat embed meaningful constraint. Good analysts statesupport for or against meaningful constraint by consideringjudiciously chosen alternatives.
I Good analysts are transparent in and responsible for theirchoices.
I Good readers have the responsibility for critically assessing theanalyst’s choice of models as well as for forming a personalopinion about the interpretation of evidence.
Jeffrey N. Rouder
Bayesian Inference in Psychology
My Guidelines
I Good scientists do not believe that science is objective; goodanalysts do not believe that inference is objective.
I Good analysts add value by considering well-specified modelsthat embed meaningful constraint. Good analysts statesupport for or against meaningful constraint by consideringjudiciously chosen alternatives.
I Good analysts are transparent in and responsible for theirchoices.
I Good readers have the responsibility for critically assessing theanalyst’s choice of models as well as for forming a personalopinion about the interpretation of evidence.
Jeffrey N. Rouder
Bayesian Inference in Psychology
My Guidelines
I Good scientists do not believe that science is objective; goodanalysts do not believe that inference is objective.
I Good analysts add value by considering well-specified modelsthat embed meaningful constraint. Good analysts statesupport for or against meaningful constraint by consideringjudiciously chosen alternatives.
I Good analysts are transparent in and responsible for theirchoices.
I Good readers have the responsibility for critically assessing theanalyst’s choice of models as well as for forming a personalopinion about the interpretation of evidence.
Jeffrey N. Rouder
Bayesian Inference in Psychology
Top Related