Variations on Scientific Variations on Scientific MethodologyMethodology

1. Observe some regularity2. Hypothesize an explanation

(cause?)3. Test that hypothesis via

experiment & observation4. Refine hypothesis & start over

with (1)

Hypotheses?Hypotheses?

Observation as tool for establishing regularities & delineating phenomena

What is Correlation?Simpsons

How do we establishing a correlation:Intervention (Experimentation)Non-Intervention (Observation)

Delineating Phenomena Delineating Phenomena w/in ‘Memory’w/in ‘Memory’

Raise your hand if you Raise your hand if you had:had:

• BAG• DOG• FAN• GAS• HAT• KID• LOG• PAD

• SOD• VEX• WIN• ZIP

Characteristics of Characteristics of Observational ResearchObservational Research

• Make some sort of record and analyze data obtained from it

• Does NOT manipulate or ‘intervene’ in the scenario.

Naturalistic ObservationNaturalistic Observation

• Observations made in the ‘natural’ setting of the organism.

• Researchers must immerse themselves in the setting.

• Task: to describe the setting, events, individuals observed w/out influencing the situation

• Often Qualitative, not Quantitative• Often NOT a matter of testing a hypothesis,

but rather gathering data to develop a testable hypothesis.

DataData

• Field notes, journal entries, interviews, recording ‘artifacts’

Famous Naturalistic Famous Naturalistic ObservationsObservations

• Jane Goodall• Charles Darwin• Survivor?

GoodallGoodall

• ‘Termite fishing’ and tool-use• Click Here

DarwinDarwin

• Sexual dimorphism is caused by three possible mechanisms:

1. mechanisms of sexual selection,2. fecundity selection3. ecological causation, e.g., resource-

partitioning

Darwin confirmedDarwin confirmed

Naturalistic Study 1Naturalistic Study 1

• Marmots• Cows • Horses• Hare • Squirrel• Deer

Problems for Systematic Problems for Systematic ObservationObservation

• Equipment (Nielson ratings)• Reactivity of subjects• Reliability (Mate selection in Blue

Tit)• Sampling• Confirmation Bias

Complexity of ObservationComplexity of Observation

• Expectations & Perception– Anomalous playing cards– Multi-modal feedback (Data from lyric

study)– Underdetermined Perception

• Influence of early hypotheses– Ratman study data

• Extending Perception w/ Instruments– Galileo / Golgi (carl) / Hale-Bopp

Anomalous Playing CardsAnomalous Playing Cards

• Link

Lyrics StudyLyrics Study

• Wash U• UCSD

Ramones CorrectRamones Correct

Ramones IncorrectRamones Incorrect

Systematic ObservationSystematic Observation

• Careful, often quantitative, observations of one or more specific behaviors.

• Observations made in ‘quasi-natural’ setting

• Researchers often do NOT immerse themselves in the setting

• Quantitative, not qualitative (using coding systems)

• Often a matter of testing a hypothesis

Coding SystemsCoding Systems

• Marmots 2

SamplingSampling

• Continuous• Time Sampling• Event Sampling

CorrelationCorrelationFinal Score & Post Test v Reading Report

0

5

10

15

20

25

30

35

0 50 100 150 200 250 300

Number of modules read

Fin

al S

core

Post Test Score

Final Score

Final Grade v. QuestionsFinal Grade v. QuestionsFinal Score & Post Test v Question Report

0

5

10

15

20

25

30

35

0 200 400 600 800 1000 1200 1400

Number of Question Attempts

Fin

al S

core

Post Test Score

Final Score

Final Grade v. Post TestFinal Grade v. Post TestPostTest v. Final

0

5

10

15

20

25

30

35

0 2 4 6 8 10 12 14 16 18

Post Test

Fin

al

Sleep Study @ UCSDSleep Study @ UCSD

Establishing CorrelationEstablishing Correlation

1. Intervention (experimentation)2. Non-Intervention (observation)

Vitamin E!

Types of Variables Types of Variables (review)(review)

• Independent (manipulated)

• Dependent

a variable of interest that is suspected to have a causal impact on the dependent variable, and is manipulated in an experiment to put this to the test.

a variable of interest that is suspected to be affected by the independent variable, and is measured after (and possibly before) the independent variable is manipulated to put this to the test.

Types of Variables (2)Types of Variables (2)

• Controlled

– Procedural

– Subject

Confound variables that have been dealt with: extraneous variables in the same system that may be causally related to the IV or DV and therefore must be controlled.

most often controlled by constancy

most often controlled by randomization

Setting up a studySetting up a study

• Consider the following:Research at the University of Pennsylvania and the Children’s

Hospital of Philadelphia indicates that children who sleep in a dimly lighted room until age two may be up to five times more likely to develop myopia (nearsightedness) when they grow up.

The researchers asked the parents of children who had been patients at the researcher’s eye clinic to recall the lighting conditions in the children’s bedroom from birth to age two.

Of a total of 172 children who slept in darkness, 10 percent were nearsighted. Of a total of 232 children who slept with a night light, 34 percent where nearsighted. Of a total of 75 who slept with a lamp on, 55 percent were nearsighted.

The lead ophthalmologist, D. Graham E. Quinn, said that, “just as the body needs to rest, this suggests that the eyes need a period of darkness”

Operationalize the Operationalize the VariablesVariables

Operationalization Part 2Operationalization Part 2

Assign possible values to the variables:

Explanation 1:Explanation 1:A Direct Causal LinkA Direct Causal Link

Internal Validity and Internal Validity and Confounding Variables.Confounding Variables.

• Internal Validity is the extent to which the study’s design ensures that its results correspond to reality.– In other words, the experiment is internally

valid if there are no confounding variables that may explain the correlation of the IV and DV.

A Confounding Variable is one that covaries with the IV or DV.

Experimental design controls Experimental design controls confounding variables.confounding variables.

• The goal of an experiment is to show that the only possible cause of the values of the DV is the IV:

New studies reported in the Journal of the American Medical Association indicate that vasectomy is safe. A group headed by Frank Massey of UCLA paired 10,500 vasectomized me with a like number of men who had not had the operation. The average follow-up time was 7.9 years, and 2,300 pairs were followed for more than a decade. The researchers reported that, aside from inflammation of the testes, the incidence of diseases for vasectomized men was similar to that in their paired controls.

A second study done under federal sponsorship at the Battelle Human Affairs Research Centers in Seattle compared heard disease in 1,400 vasectomized me and 3,600 men who had not had the operation. Over an average follow-up time of fifteen years, the incidence of heart diseases was the same among men in both groups.

A Confounding Variable?A Confounding Variable?

Controls: Setting up the Controls: Setting up the groupsgroups

• First, we must create a control group: – there must be some kind of comparison

condition that will enable us to say that the DV depends solely on the IV.

Poor Design as a Result of Poor Design as a Result of Poor Control GroupPoor Control Group

• Fallacy 1: No Control Group• Fallacy 2: Nonequivalent Control Groups.• Pre-Test Post-Test Pitfalls:

HistoryMaturationTestingInstrument DecayStatistical Regression

Independent Groups Independent Groups DesignDesign

• Simple Random Assignment

• Matched Pairs Assignment

Just what it sounds like: randomly assign each subject to one of the groups (a coin flip is sufficient for two groups).

First match subjects into pairs based on some characteristic related to the IV. Then randomly assign the members of each pair to one of the two experimental groups.

Repeated Measures Repeated Measures DesignDesign

• The same individuals participate in both experimental conditions, after which the dependent variable is measured.– Advantages: Fewer individuals are required.– If training (or acclimatizing) is required, this

design can save valuable resources.– As the individuals in the experimental

conditions are identical, more confounds are controlled.

Problems with Repeated Problems with Repeated Measures DesignMeasures Design

• Order effects

– Practice effect– Fatigue effect– Contrast effect

The order in which the measures are administered affects the DV.

Problems for Many Problems for Many Sciences.Sciences.

• How do we observe / experiment on the internal workings of something (I.e. cognition)?

Sternberg’s ExperimentSternberg’s Experiment

Sternberg’s ResultsSternberg’s Results

Response Time = 398+38(S)Gravitational Force =

(A constant called G) x (mass of first object) x (mass of second object)(the square of the distance between them)

MechanismMechanism

MechanismMechanism

MechanismMechanism

MechanismMechanism

MechanismMechanism

Models & Mechanisms:Models & Mechanisms:

• Mechanism: entities and activities organized to produce a phenomenon (teleological?)

• Entities and activities organized in such a way as to realize a functional role.

‘‘Model’?Model’?• A Model is a description of some

phenomena / onA model is verdical insofar as corresponds to the actual phenomena it seeks to model.

A model, just like a ‘law’ or a ‘theory’ explains phenomena / on and can be used to make predictions about novel / unobserved aspects of the phenomena it seeks to model.

Therefore, it is plays the same roll as ‘law’ or ‘theory’ in the H-D method or D-N model of explanation.

ModelsModels

Categorization of different Categorization of different Models / Systems:Models / Systems:

Scientific Reasoning Scientific Reasoning ConclusionConclusion

If I’m right that the main structure of explanation in scientific inquiry is the investigation of underlying mechanisms, then…

1. Correlational / observational studies are primarily used for establishing the parameters of the mechanism’s behavior.

2. Modeling is a fundamental, essential part of scientific activity.

3. Models serve the same roll in scientific inquiry as Popper’s ‘laws’ – they entail falsifiable predictions.

4. The line between science & pseudoscience is more clear:

Psychology v. AstrologyPsychology v. Astrology

Phenomenon explained / predicted: human behavior and personality.Mechanism: beliefs and desires interact to determine human behavior, which beliefs and desires get precedence in any one choice is influenced by the hodge-podge of previous experiences and genetic dispositions we call ‘personality’.

Phenomenon explained / predicted: human behavior and personality.Mechanism: the forces of the planets at time of birth.

Biology v. Creation Biology v. Creation ScienceScience

Phenomenon to be explained: Variation of species over time and space.Mechanism: Natural Selection (random mutations are replicated if they help the creature reproduce by (a) increasing survival in the environment (b) changing the number of offspring the creature has or (c) increasing the chances that that will creature mate.)

Phenomenon to be explained: Variation of species over time and space.Mechanism: ?

Evaluating Competing Evaluating Competing Mechanism:Mechanism:

Evaluating Competing Evaluating Competing MechanismsMechanisms

Ptolemaic Astronomy

Phenomenon:

Copernican Astronomy

Phenomenon:

Parameters:Parameters:

Fit the location of the planets & stars in the sky(They’re equal on this one)

“Other” Values:The Copernican system is far simpler

and more elegant.

Venus

VenusVenus

Galileo deduced that:If the Ptolemaic system is correct, then

Venus should not show phases. AndIf the Copernican system is correct, the

Venus should show phases.Venus shows phases.Therefore, the Ptolemaic system is not

correct.

Scientific RevolutionsScientific Revolutions

The Ptolemaic system dominated western and eastern science from 388BC until the 16th century. So why the change?Chemistry: LavoisierBiology: DarwinPhysics: Newton -> Einstein -> Quantum

(Bohr / Heisenberg) -> String Theory

REVOLUTION!REVOLUTION!

‘‘Real’ Revolutions as Real’ Revolutions as metaphor.metaphor.

• Scientific Revolutions are those ‘non-cumulative developmental episodes in which an older paradigm is replaced in whole or in part by an incompatible one’ Thomas Kuhn The Structure of Scientific Revolutions

Analogical points:Analogical points:

1. Revolutions are inaugurated by a ‘growing sense, often restricted to a segment of the political community, that existing institutions have ceased to adequately meet the problems posed by an environment that they have in part created’

2. Revolutions often seem revolutionary only to those whose paradigms are affected to them.

3. Success of a revolution necessitates, in part, the ‘relinquishment of one set of institutions in favor of another, an in the interim, society is not governed by institutions at all.’

Conclusion:Conclusion:

• Well, that’s the point:– During revolutions, society is divided

into competing camps or parties – one seeking to defend the old, others seeking to replace it with new.

– (There may be competing new camps as well)

– Once that kind of polarization occurs, political recourse fails.

• The parties are fighting over the legitimacy of institutions by which political decisions can be made – for that very reason, there is no political mechanism for adjudicating between the parties.

• So, the parties must ‘take to the streets’ – appeal to something other than political will (such as God, history, etc) or resort to force.

• The success of the winner is determined not by political institutions, but by extrapolitical institutions – by the very fact that they replace those institutions by which they legitimize themselves.

Therefore, by analogy…Therefore, by analogy…

• Scientific revolutions gain legitimacy not by factors internal to science, but by extra-scientific methods, such as social factors. And this is precisely because the issue at stake is the legitimacy of factors internal to science.

Modeling

Formulae relating observables

Investigation of underlying structure

‘Mathematical Models’ in Psych Discovered Models Invented Models

‘Experimental Systems’

MathematicalSymbolic

Neural Network

V = d/t

F=ma

11stst use: positing unobservables use: positing unobservablesPerformed by Jameson and Hurvich in 1957. A test light is shown to a subject. If the light appears greenish, a red-appearing light is added until the test light no longer appears at all greenish.

Jameson and Hurvich Jameson and Hurvich ResultsResults

Cone Sensitivity CurvesCone Sensitivity Curves

Mathematical Transformation Mathematical Transformation of Cone Sensitivity Functionsof Cone Sensitivity Functions

• We decorrelate the responses of the L, M and S cones by weighting each signal with a constant, and combining those results:

C1() = 1.0L() + 0.0M() + 0.0S()

C2() = -0.59L() + 0.80M() + -0.12S()

C3() = -0.34L() + -0.11M() + 0.93S()

Opponent Processing Opponent Processing ModelModel

Modeling

Formulae relating observables

Investigation of underlying structure

‘Mathematical Models’ in Psych Discovered Models Invented Models

‘Experimental Systems’

MathematicalSymbolic

Neural Network

V = d/t

F=ma

22ndnd use: relating observables use: relating observables

• The most simple use of a mathematical model is to fit a mathematical function to some data collected in an experiment. That function can then be used to make predictions about novel or unobserved behavior.

• Sternberg’s Memory Scanning Model – Response Time = 398 + 38(Memory Set Size)

• De Castro and Brewer – Intake of food = s(Number of People Present)0.22

Sternberg’s ResultsSternberg’s Results

Response Time = 398+38(S)Gravitational Force =

(A constant called G) x (mass of first object) x (mass of second object)(the square of the distance between them)

The importance of The importance of Mathematical Models:Mathematical Models:

Quick: what is the most famous mathematical model in the US right now?

The BCS FormulaThe BCS Formula

• ‘Fit’?• Data: team record, opponent’s

record (‘strength of schedule’), poll rankings over the season, team losses & ‘quality wins’.

Example: Oklahoma 2000?Example: Oklahoma 2000?

• AP & Coaches poll end of season rank = 1.

• Average rank over the course of the season= 1.86.

• Average of AP & Coaches poll + average over season = 2.86.

• (Thanks to Richard Billingsley at ESPN for the explanation).

Strength of scheduleStrength of schedule

• Add the opponent’s records together = 73 Wins, 62 losses.

• Drop wins against teams that were not 1-A, and you have 70W.

• Drop losses from opponent’s schedule that were against OK, and you get 50 losses.

• Total: 70 Wins, 50 losses.

Opponent’s winning %.Opponent’s winning %.

• The winning percentage is 70/120 = 58.3% or 0.583.

• 0.583 * 2/3 = 0.3889• Do the same ‘opponent’ calculation for

each of the opponent’s opponents and weight it by 1/3 = 0.1749

• Add these 2 together and you get 0.5638

Now…Now…

• Rank all the teams according to this ‘strength of schedule’. OK is 11th

• Finally, take that rank / 25 = 0.44.

• Add ‘Team losses’ (0 for OK) and ‘Quality wins’ (0 for OK).

• Add that to ‘Poll average’ and you get 3.30.

‘‘Mathematical’?Mathematical’?

– Obvious: algebra / calculus– Recursive functions– Game Theory

• Other kinds of models– Physical (geology)– Virtual

• Neural Network• Symbolic

– Animal• In Vitro• In vivo