Guidelines for IDing Experimental Design

Levels/participant?

One More than one

IndependentGroups design

RepeatedMeasures design

Once: incomplete

More than once:complete

Times each level/Participant?

Yes: all possible orders

No: selected orders

Four or fewer conditions?

Anticipation problems?

Yes: block randomization

No: ABBA counterbalancing

Complex Designs

When a “two-way” is more than spaghetti and chili!

What are they?

• Simple versus complex designs– “One-way” versus “Two-way”, etc.

• Types of factorial designs– Completely randomized– Completely within– Mixed

• Variable and level shorthand

2 x 2 2 x 3 2 x 2 x 3

What good are they?

• Advantages of complex designs

• Economy

• Understanding

• External validity—interactions

• Example: social facilitation versus evaluation apprehension

Some Data

Identifying Main Effects and Interactions

• Interpretation– Main effects: the overall effect of one

independent variable on the dependent variable.

– Interactive effects: the effect of the first independent variable on the dependent variable that is contingent on a particular level of the second independent variable.

• ordinal interactions• disordinal (crossover) interactions

Analysis of Complex Designs

• Simplest case: the 2 x 2 design– Three classes of scientific hypotheses

• Main effect of treatment method: e.g., behavioral method will be more effective than the cognitive method

• Main effect of presenting problem: e.g., treatment will be more effective for habit problems than learning problems

• Interaction effect of treatment method x presenting problem: e.g., cognitive therapy will be more effective for learning problems, but behavioral therapy will be more effective for habit problems

Interactions: Looking at the picture…

Analysis: When Interaction is Present…

• Evaluate evidence descriptively– Graphs (non-parallel lines)– Tables (subtraction method)

• Confirm by inferential statistics: complex ANOVA

• Qualifies our interpretation of the main effect

Analysis: When Interaction is NOT present…

• Focus is on interpreting the main effect(s)

• Analytical comparisons of the marginal means and confidence intervals with planned comparisons

The “many” faces of 2 x 2s:Possible Results #1

What do we have here?

The “many” faces of 2 x 2s:Possible Results #2

What do we have here?

The “many” faces of 2 x 2s:Possible Results #3

What do we have here?

The “many” faces of 2 x 2s:Possible Results #4

What do we have here?

The “many” faces of 2 x 2s:Possible Results #5

What do we have here?

The “many” faces of 2 x 2s:Possible Results #6

What do we have here?

The “many” faces of 2 x 2s:Possible Results #7

What do we have here?

The “many” faces of 2 x 2s:Possible Results #8

What do we have here?

Interpreting Interactions

• Theory testing

• External validity

• Ceiling and floor effects

• Natural groups design