1 Experimental Designs HOW DO HOW DO WE FIND WE FIND THE ANSWERS ? THE ANSWERS ?

1

Experimental DesignsExperimental DesignsHOW DO WE FIND

THE ANSWERS

?

2

Characteristics of Experimental DesignsCharacteristics of Experimental Designs

Manipulation of one or more factors (Independent Variables)

Measurement of the effects of manipulation (Dependent Variables)

Validity Are we in control?

Reliability Can the results be replicated?

Sensitivity Are we measuring what we want to measure?

3

Validity (Internal)Validity (Internal)

amount of control over experimental conditions allows conclusion that the IV causes an effect on

the DV allows exclusion of other variables causing an

effect on the DV

4

Internal ValidityInternal Validity

Challenges to Internal Validity Using intact groups

(such as classes of students) Not balancing extraneous variables

(individual differences) * hypnosis volunteers early or late in term

Subject Loss mechanical subject loss (equipment failure) selective subject loss (related to paradigm?)

5

Validity (External)Validity (External)

Can findings be generalized to other species to other individuals to other settings or situations to other conditions

6


For some lab experiments we do not establish external validity

Is external validity needed? Mook (1983) argues that external validity is

irrelevant if we are testing a specific hypothesis in a laboratory setting

Lab experiments typically try to test a specific hypothesis instead of imitating a typical situation

7


External validity needed when results are to generalized to a population

External validity requires a representative sample

Partial replication (repeating some but not all of the experimental conditions) can provide evidence for external validity

8

SensitivitySensitivity

Is our measure appropriate for the effect we are looking for?

Are we measuring enough of the effect? Are we measuring too much of the effect

(even if we get an effect, is it meaningful?)

9

Issues of ControlIssues of Control

Methods of Control Manipulation Holding conditions constant Balancing

10

ControlControl

Manipulation Systematic varying of an Independent

Variable

11

ControlControl

Hold Conditions Constant Make the IV the only variable the

differentiates between the groups Example: Use only males to hold the gender

effect constant

12

ControlControl

Balancing Technique used to control for individual

differences of participants Used in independent groups designs Insures that all groups are equivalent in

areas such as age, motivation, sex, intelligence, etc.

13

Independent Groups DesignIndependent Groups Design

Each group represents a different condition Conditions are defined by the level of the

IV Groups are formed by participants being

assigned to conditions Nature of group formation makes balancing

a major consideration of control

14

Random Groups DesignRandom Groups Design

Groups are formed prior to introducing the IV

Subjects are sampled in such a way that the selection of one subject in no way influences the selection of another subject

All subjects have an equal chance of being in any given group

May be accomplished by random selection or random assignment

15

Random SelectionRandom Selection

Requires a well defined population Requires randomization processes for

selection of subjects Subjects are randomly selected for each

group

16

Random AssignmentRandom Assignment

Used when random selection is not possible Most samples are accidental and not from

well defined populations (Intro Psyc students)

Random assignment is then used to randomize subjects into different groups instead of random selection

17

Random AssignmentRandom Assignment

Block Randomization Most often used for random assignment Have number of blocks = number of

subjects in each condition Randomize conditions in each block Assign subjects to each condition in each

block until all blocks are filled

18

Matched Groups DesignMatched Groups Design

Used when comparable groups is required Instead of random assignment

the researcher makes the groups equivalent by matching the subjects in each group

Most useful if a good matching task is used

19

Matched Group DesignMatched Group Design

Example 1) pretest for dependent variable (BP) 2) match subjects by BP level and group by the

number of conditions 3) randomly assign to conditions 4) compare BP of subjects by condition at

posttest

20

Natural Groups DesignNatural Groups Design

Subjects are selected based on levels of IV Used when impossible to manipulate IV

age, gender, personality traits, etc. Used when not ethical to manipulate IV

married, divorced, widowed, etc.

21

Repeated Measures DesignRepeated Measures Design

One group of subjects Subjects receive all levels of the IV Eliminates problem of Individual Differences Reduces the number of subjects required

Counterbalancing necessary for control

22

CounterbalancingCounterbalancing

Counterbalancing necessary to control practice effect. ABBA design optimal (IVs A & B)

ABBA (complete counterbalance)

23


Problems with ABBA design Each subject has to complete all presentations of IV

ABBA As IV levels increase design becomes unmanageable

IVs A, B, & C ABCACBBACBCACABCBA

(Complete Counterbalance)

24


Alternate to complete ABBA design Use Incomplete design

½ of group receives conditions AB ½ of group receives conditions BA

IVs ABC (complete design) ABCACBBACBCACABCBA

(Complete Counterbalance – each subject receives 18 conditions) ABC ACB BAC BCA CAB CBA

(Incomplete Counterbalance – each subject receives 3 conditions)

25

Design Problems and SolutionsDesign Problems and Solutions

Independent Groups Design Individual Differences

(Differences between subjects in each group) Use Repeated Measures Design to eliminate individual

differences (using same subjects)

Repeated Measures Design Differential Transfer

(Carryover effects) Use Independent Groups Design to eliminate differential

transfer

26

Complex (Factorial) DesignsComplex (Factorial) Designs

Main Effects Effects of the Main IVs

Two possible Main Effects in your experiment Difference in RT between Caffeine and No Caffeine

(IV # 1 or “A”) Difference in RT of PH and NPH

(IV # 2 or “B”)

27

Complex (Factorial) DesignsComplex (Factorial) Designs

Interaction Effects How one IV (A) may impact another IV (B)

Will Caffeine influence RT in one hand but not the other hand?

Four possible Interaction Effects in your experiment

This is a 2 X 2 Factorial Design 2 IVs (Caffeine & Handedness) Each has 2 levels (Caffeine or No Caffeine & PH or NPH)

Caffeine No Caffeine

PH RT (Caff&PH) RT (NoCaff&PH)

NPH RT (Caff&NPH) RT (NoCaff&NPH)

28

Analysis of Factorial DesignsAnalysis of Factorial Designs

Analyze with a Factorial ANOVA (F test) F test analyses reflects

Systematic variance due to manipulation Error variance due to confounds

Including Individual differences of subjects F = variation between groups variation within groups

F = error variation + systematic variation error variation

29


F test analyses reflects

F = variation between groups variation within groups

F = error variation + systematic variation error variation

F test may indicate significant differences in Main Effects and Interaction Effects

Requires a Post Hoc text to determine differences

30


Post Hoc test for Main Effects One-Way or Repeated Measures ANOVAS if needed

Post Hoc test for Interaction Effects Graph data

Parallel lines indicate no interaction

Converging or Intersecting lines indicate interactionsCaffeine No Caffeine

0

200

400

600

PH

NPH

Caffeine No Caffeine0

100200300400500

PH

NPH

Caffein

e

No Caff

eine

0

200

400

PH

NPH

Caffein

e

No Caff

eine

0

200

400

PH

NPH

31

Experimental DesignsExperimental Designs

EXPERIMENTAL DESIGNS THAT

ARE CORRECTLY

EXECUTED

RESULT IN SUCESSFUL OUTCOMES

32

Experimental DesignsExperimental Designs

QUESTIONS?

1 Experimental Designs HOW DO HOW DO WE FIND WE FIND THE ANSWERS ? THE ANSWERS ?

Documents

Transcript of 1 Experimental Designs HOW DO HOW DO WE FIND WE FIND THE ANSWERS ? THE ANSWERS ?