Science and Psychology

Psych 395 - DeShon

Card Selection Task (Wason,1966)

4 Cards (letter/number) Rule:

– If a card has a vowel on one side, then it has an even number on the other side

Which card or cards do you have to flip to determine whether the rule is violated?

E K 4 7

Answer

Pick These Cards:

E 7

Let’s get a little symbolic

If a card has a vowel on one side, then it has an even number on the other side

If P then Q Becomes a problem in deductive logic…

Answers

Flip E card: If P then Q means…if P then Q. (Modus Ponens)

Flip 7 card: If P then Q means…if not Q then not P. (Modus Tollens)

Flip K card: Error – Irrelevant to Argument Flip 4 card: Error – Rule doesn’t say that Q

cannot occur in the absence of P. Only About 10% of participants opt for both

E and 7.

Implication

Humans are not naturally good at all aspects of formal logic. (Scientists included)

Scientific methods exist to prevent scientists from deceiving themselves (e.g., Levine & Parkinson, 1994).

The nature of Science is the focus of today’s lecture

But first an aside on the subjectivity of science…

Popper (1968, p. 51)

“There is no more rational procedure than the method of trial and error – of conjecture and refutation: of boldly proposing theories; of trying our best to show that these are erroneous; and of accepting them tentatively if our critical efforts are unsuccessful.”

Deductive Reasoning and Scientific Reasoning

Deductively Valid Arguments

Definition: Arguments where the truth of the premises guarantees the truth of the conclusion.

Consider Two Forms: – Modus Ponens (Mode that Affirms)– Modus Tollens (Mode that Denies)

Modus Ponens

Formally:– Premise 1: If P then Q.– Premise 2: P– Conclusion: Therefore, Q

In words:– If an animal is pig then the animal oinks.– Wilber is a pig.– Therefore, Wilber oinks.

Modus Tollens

Formally:– Premise 1: If P then Q.– Premise 2: Not Q– Conclusion: Therefore, not P

In words:– If an animal is pig then the animal oinks.– Wilber does not oink.– Therefore, Wilber is not a pig.

Is This How It Works in Science?

Theory: All pigs oink. Make some observations about a newly

discovered species. Observe Q - These animals oink. Conclude that P is True. These new animals

are pigs.

Fallacy of Affirming the Consequent

In reality, other animals besides pigs may also oink!

So this method of reasoning is not a path to scientific progress.

Basic Idea: We CANNOT prove that scientific theories are TRUE. (Sir Karl Popper)

Scientific Reasoning is Not About Confirmation. Science is about Refutation.

Scientific Reasoning - Refutation

We can show that the consequence of a theory are not empirically supported. Modus Tollens.

According to Popper, falsification is the line of demarcation between science and non-science.

If we observe “Not Q” then we can conclude that P is False.

If the new animals don’t oink then they are not pigs.

Sounds Easy Enough, But…

No theory is tested in Isolation (Duhem-Quine Thesis) There are lots of auxiliary theories involved in

conducting a scientific study.– For example, we assume that our instruments are

good indicators of the constructs under investigation.– This is why we spend so much time on

measurement in 395. We don’t know what we have precisely falsified when

we observe “NOT Q” – the Big P theory or one of the auxiliary theories.

Fundamental Inferences

IV Construct DV Construct(Real Relation)

Indicator of IV Indicator of DV(Observed Relation)

Example

Strain Theory in Sociology (Merton, 1938) – Social Strain Produces Crime.

Possible Prediction: Blocked Opportunities should Correlate with Crime.

Observation: Perceptions of blocked Opportunities are not correlated with arrest records.

What do we conclude?

Fundamental Inferences

Strain Crime(Positive Relation)

Self-Reports ofBlocked Opps

Official Arrests(Observed)

Revision of Theory

Error!

Findings

Prediction from Theory

Deduction

Induction

Feedback Loop Modified from Figure 2A in Box (1976, p. 791).

Separate Justification from Criticism (e.g., Bartley, 1984)

Searching for complete justification is not going to get us very far.

Instead, let’s acknowledge that there is a continuum from speculation to well-established knowledge.

Argue about the reasonableness and degree of empirical support for particular propositions.

For Psychological Researchers…

Specify models, test models, revise models… Researchers must be transparent so that their work

can be subject to maximum criticism. Full disclose of what was done and what was found.

Accuracy and clarity are paramount virtues in the reporting of psychological research.– Hence, good writing is important.

Further Thoughts on Science and Models

Just a Few Reasons Why Psychology is Hard (e.g., Lykken, 1991; Meehl, 1978)

Everybody is an armchair psychologist. This provides resistance to simplification.

Experimental control is difficult. Psychology seeks to understand one of the

most complicated “machines” in the known universe – human brains which give rise to thoughts, feelings, and behaviors.

Pragmatic Viewpoint

“Every scientist in the back of [her] mind takes it for granted that even the best theory is likely to be an approximation to the true state of affairs”

– Paul Meehl (1990, p. 113)

“Models, of course, are never true, but fortunately it is only necessary that they be useful. For this it is usually needful only that they not be grossly wrong.”

– George E. P. Box (1979, p. 2)

More Thoughts on Models…

“The great advantage of the model-based approach over the ad hoc approach, it seems to me, is that at any given time we know what we are doing.” (Box, 1979, p.2)

Since all models are wrong the scientist must be alert to what is importantly wrong. It is inappropriate to be concerned with mice when there are tigers [around]. (Box, 1976, p. 792).

What to Do…

Appreciate how science really operates. Favor theories that make risky predictions

and survive rigorous test of those theories (e.g., Meehl, 1978, 1990).

Think Beyond One Study to Multiple Studies -- Focus on Replication (Lykken, 1968)

Importance of Replication

Scientists have known for centuries that a single study will not resolve a major issue. Indeed, a small sample study will not even resolve a minor issue. Thus, the foundation of science is the culmination of knowledge from the results of many studies.

Source: Hunter, Schmidt, & Jackson (1982, p. 10)

Causality

Science vs. Societal Values

The history of science reflects a clash between what we currently believe to be true and the new “truths” we discover about the world.

Virtually everything we once “knew to be true” has been shown to be false.

– Magic– Geocentric models of the universe– Flat earth– Medicine (blood letting, arsenic, etc.)– Newtonian Physics?

Example: Rind et al (1998)

Societal Belief: Child Sexual Abuse is wrong and causes substantial harm and results in negative outcomes in adults.

Rind et al (1998) performed a meta-analysis (a statistical summary) of the existing literature and did not find support for the claim that CSA results in substantial negative long-term outcomes in adults

< 1% of variance in outcomes

Huge public and political response

Lilienfeld (2002) documents the response Nearly caused the downfall of the APA Dr. Laura condemned the research Religious groups united against the research Politicians with no scientific training began

criticizing the study APA caved to political and public pressure

and renounced the study

Aftermath

Public policy review initiated at APA Congress voted 355 to 0 to condemn and denounce

the findings Senate passed the resolution the same month Scientific review panels found no serious fault with

the study Highlights a huge gap between popular press and

academic science

Some Terminology

Variables: Any attribute that changes values across things that are being studied.

Hypothesis: A testable statement describing the relation between two or more constructs.

Constructs: Abstract qualities that we attempt to measure.

Operational Definition: Statement of the process we use to measure constructs.

– Constructs Actual Measures in a given study

Operational Definitions

Going from the Unobserved to the Observed…

X YUnobservableConstructs

x yMeasuredVariables

Scientific Approach to Causation

“Development of Western science is based on two great achievements: the invention of the formal logical system (in Euclidean geometry) by the Greek philosophers, and the discovery of the possibility to find out causal relationships by systematic experiment (during the Renaissance).”

– Albert Einstein (1953)

Causal Relations

John Stuart Mill (1806-1873) Three Conditions for Cause and Effect

Relations.– Temporal Ordering (Cause Preceded the Effect)– Covariation (Cause and Effect are Associated)– Rule Out Third Variables (No Plausible

Explanation for the Effect Other than the Cause)

Newton’s 4 Rules of Scientific Reasoning (Principia, 1680)

1. Admit no more causes of things than are both true and sufficient to explain their appearance.

– Occam’s razor– Rule of Parsimony– Sherlock Holmes– The simplest theory that fits the facts of a problem is the

one that should be selected.

Example: Geocentric view of solar systemExample: Crop circles

Newton’s 4 rules

2. To the same natural effects we must, as far as possible, assign the same causes.

– Generality– Example: Gravity and falling objects

3. The qualities of bodies which are found to belong to all bodies within the reach of experimentation should be assumed to be universal qualities of all bodies

– Generalization– Example: Gravity on other planets

Newton’s 4 rules

4. Theories obtained from experiments should be considered true until new experimental evidence shows the old to be incorrect.

– Values should not lead to changes in theories

– The history of science represents a continuous struggle between existing values and empirical data (new and old beliefs)

Experimental Approach to Studying Causal Relations

Intervention, Observation, and Control Random assignment of participants to two or

more conditions of an experiment. The Independent variable is the variable that

is manipulated or the experimental conditions (e.g., treatment or no-treatment).

The Dependent variable is the outcome of interest in the particular study.

Where do research ideas come from?

Observe your world!– Why do some people work so hard?– Do social systems function like ant colonies?– Why is karaoke so popular?– Why do people sacrifice long term gain for short term

pleasure (delay of gratification)?– What is boredom?– What do individuals get out of belonging to a counter-

culture?– Why do people collect stuff?

Ask questions about your own behavior!– Why did I do that?

Bathroom Behavior

Personal Space Invasion in the Lavatory: Suggestive Evidence for Arousal - Middlemist et al. (1976)

Invasions of Privacy are Bad

Do Space Violations Cause Arousal?Theory: Space Violations Arousal

Responses to Personal Space ViolationWhere can we study personal space

violations?

The Men’s Room….

Let’s Put this in Scientific Terms…

“Urinals are open and placed side by side so that, under crowded conditions, men stand shoulder to shoulder, coactively engaging in private elimination” (Middlemist et al., 1976, p. 542)

Conducted research on micturation – “a process sensitive to arousal”

“Fright and embarrassment inhibited relaxation of the external sphincter of the urethra” (Middlemist et al., 1976, p. 542)

Hypothesis (Middlemist, et al., p. 542)

“If personal space invasions produce arousal, then subjects standing closest to others … would show increases in the delay of onset of micturation and decreases in the persistence of micturation”

Translation:

Design

60 Men randomly assigned to one of three conditions:

– Close Distance (16 to 18 inches)– Moderate Distance (52 to 54 inches)– Control (Peeing Solo)

An observer hides in toilet stall and makes a “direct visual sighting of the stream of urine”

“Subjects were not informed that they had participated in an experiment”

Close Distance Condition

Moderate Distance

Results – Delay of Micturation

0

2

4

6

8

10

Delay

Control

Moderate

Close

Results – Persistence of Micturation

0

5

10

15

20

25

Persistence

Control

Moderate

Close

Rephrase these Results…

Conclusions (Middlemist et al., p. 545)

Micturation delay and persistence were sensitive to situational differences.

The pattern was consistent with arousal theory Space Violations Arousal Q: How was arousal measured? “The present study implies that [delay and

persistence] have some construct validity as indicators of arousal” (p. 545)

Let’s Answer Some Key Questions

What was the hypothesis? What was the Independent Variable? What were the Dependent Variables? What were the Operational Definitions? What were the Results? What was their Interpretation of the Results?

A Measurement Issue with Experiments

It seems straightforward enough – manipulate an IV and measure the effects on a DV.

But what exactly did we manipulate? What exactly causes the observed responses?

We know what we manipulated at the operational level. How do we know what psychological construct was actually manipulated?

Scientific Research Process

Develop a Research Idea - Identify an interesting question - Review the relevant literature

Convert idea into a hypothesis - Specific, testable proposition

Define & Measure Variables - Determine measurement process - Evaluate reliability of measures - Evaluate validity of measures

Identify Participants - How many? - What population? - Sampling design - Ethical treatment issues

Select a Research Strategy - Internal vs. external validity - Experimental vs. Quasi-experimental vs. Observational

Select a Research Design - Between Subjects vs. within-subjects - Factorial - Covariates

Conduct the study - Collect the data! - Input and organize data

Evaluate the Data - Statistical Analysis

Report the Results - APA Format - Accurate picture

Refine Idea - modify or expand idea

1

2

3

4 5

6

7

8 9

10