Simple Group Comparisons Limits you to simple explanatory variables simple potential relationships.

Simple Group Comparisons

Limits you to

simple explanatory variables

simple potential relationships

Complex Group Difference Designs

Types

Multilevel Designs – single IV, 3 or more levels

Single source of possible systematic variability, but more than a simple difference possible

Multifactor Designs (Factorial) – multiple IVs

Multiple possible sources of systematic variability

Not all effects have single variable causes

Multilevel Designs

Situations requiring a multilevel approach

Characteristics of the IV

appropriate representation of IV variabilityLevel of Sense of Humor and Perceived Personality

No two levels of SoH are likely to represent important differences

Characteristics of the IV DV relationship

a nonlinear relationship anticipated Level of Test Anxiety and Test Performance

Would not expect a linear relationship

Evaluating the Results –

searching for the systematic variability in the DV

no single place (difference) to assess,

variability could appear

in multiple places - examples

or

in multiple forms - examples

Multilevel Designs

Two common approaches

Overall test for ‘systematic variability’

- with follow-up (post hoc) tests to see where differences ‘might’ exist(Means vary systematically, as opposed to no more than unsystematically)

Test for ‘fit’ with predicted pattern across groups

Using planned comparisons or ‘contrasts’ to specify pattern(Means vary systematically in a predicted form or pattern)

Multilevel Designs

Overall test for ‘systematic variability’

Ho: M1 = M2 = M3…………

variability among sample means will be no

greater than variability expected due to

unsystematic variability

Possibly two steps in the analysis

1. Test to see if Ho can be rejected

If yes, there is evidence of systematic variability

2. Examine differences between means to see where systematic variability occurs

Multilevel Designs

Test for ‘fit’ to predicted pattern across groups

Ho: M1 = M2 = M3…………

Specify pattern expected (contrast) and

test to see if systematic variability does

‘fit’ that pattern- allowed as many ‘orthogonal’ contrasts as there are df overall

- if doing nonorthogonal contrasts, may need to adjust Type 1

L = cM, where c is the ‘weight’ assigned to each mean

to represent the pattern predicted (and add up to 0)

Multilevel Designs

Test for ‘fit’ to predicted pattern across groups

Ho: M1 = M2 = M3…………

Specify pattern expected (contrast) and

test to see if systematic variability does

‘fit’ that patternContrast to test linear relationship between Sense of humor and Liking

as SOH goes up, Liking goes up

Contrast to test curvilinear relationship between Test Anxiety and outcome

low and high anxiety lower the outcome, relative to moderate

Multilevel Designs

Situations requiring a Multifactor approach

multiple possible Causes (each sufficient, not necessary)

more efficient to assess in a single designlaugh at your joke (or not)

similar attitudes (or not)

interactions among Causes needed to produce effect

(necessary, not sufficient causal influence)high choice in behaving counter to one’s attitude (vs low choice)

high consequences for behavior (vs low consequences)

need both to be High to get effect

assess possible role of an EV in affecting DVpsychological vs physical disorder (IV)

gender or worker with disorder (EV)

Multifactor Designs

Main Effects vs. Interaction Effects

Main effects – relationship of each IV with the DV

-one possible main effect for each IV in the design

Interaction effects – combinations of levels of different IVs

-effect of changes in one IV (on DV) depends upon

level of another IV present

Multifactor Designs

Understanding Multifactor Design notation

How many main effects and interactions?

2 x 2 design - Chant (2) x Program (2) Kumbaya vs Chant

2 x 4 design – Program (2) x Chant (4)None, Kumbaya, Stats 1, Stats 2

2 x 2 x 3 design – Program (2) x Gender (2) x Chant (3) Kumbaya, Stats 1, Stats 2

Multifactor Designs

What is being compared?

Numbers that represent main effects and interactions

main effects in ‘margins”

interactions ‘inside’look at some examples with Means

2 x 2 design

2 x 4 design

2 x 2 x 3 design

Multifactor Designs

75 85

75 75What if the groups have unequal n’s?

Following up on significant main effects and interactions

Multilevel main effects

Simple main effects in interactions

Multifactor Designs

Interpreting results when there are significant

main effects and interaction effects

Interactions take precedence in interpretations

Multifactor Designs

Independent Levels vs.Related Levels Independent Variables in Complex Group Designs

Independent Levels Multilevel

Related Levels Multilevel

Independent Levels Multifactor

Related Levels Multifactor

Mixed Multifactor

Complex Group Difference Designs

Now – for the analyses

Recall

Variance = (x-M)2

df

Allows you to combine all the different deviations from the Mean to get a Sum (of squares), and then to find the ‘typical’ deviation (squared) by dividing by df

Need to be able to assess multiple possible differences that might reflect “systematic” variability along a single dimension - Multilevel Variables/Designs

and/or could be the result of multiple independent sources of ‘systematic’ variability –

Multifactor Designs

Evaluating Results in Complex Group Difference Designs – Interval/Ratio Data

Building from the t-test

single deviation (M – M)

typical deviation (se)

Numerator could be treated as two deviations

Group 1 mean – Population mean (estimated)

Group 2 mean – Population mean (estimated)

Sum the deviations (after squaring), and find the ‘average’ (what do you have?)

t =

Building from the t-test

single deviation (treat as 2 deviations from Mean)

typical deviation (square to convert to variance)

Now you could compare that ‘variance’ (numerator)

(may be due to a situation where Ho is not true)

to the typical variance (when Ho true) by squaring the denominator

Building from the t-test (based on deviations)

single deviation (systematic and unsystematic)

typical deviation unsystematic

F statistic – from Analysis of Variance

Variance (systematic and unsystematic)

Variance unsystematic

t =

F =

When you have only two groups, F = t2,

but now, with F, you can include any number of means in the numerator of the F-ratio.

What you have is:

estimate of population variance

estimate of population variance

One estimate is sensitive to systematic variability

One estimate is unaffected by systematic variability

F =

Want to ‘estimate’ variance in the population

population

Sample 2

IV level 2

Sample 3

IV level 3

Sample variance 1 Sample variance 2 Sample variance 3

Most logical strategy is to use the variances from your samples to estimate the variance in the population.

Sample 1

IV level 1

Want to ‘estimate’ variance in the population

population

Sample 1

IV level 1

Sample 2

IV level 2

Sample 3

IV level 3

Sample mean 1 Sample mean 2 Sample mean 3

Can also estimate the population variance by using means from samples, but this estimate will only be accurate when the Ho is true

estimate of population variance using means

estimate of population variance using variances

Means are sensitive to systematic variability

Variances are unaffected by systematic variability

systematic + unsystematic variability

unsystematic variability

F =

F =

Analysis of Variance involves ‘partitioning’ the variability of the DV into the variability relevant to the parts of the F ratio.

Recall that: Sum of Squares

df

So the ‘sum of squares’ reflects the variability of DV

(before it is ‘averaged’)

Some of the variability (sum of squares) reflects only unsystematic variability

Some of the variability (sum of squares) reflects systematic + unsystematic variability

Variance =

Partitioning the Sum of Squares

(in a balanced design)

Two groups (for simplicity)

Non chanters (25)

Chanters (25)

x

‘Sample Mean’

‘Group Mean’‘Group Mean’

SSTotal – sum of the squared deviations of individual scores from the mean of the entire sample – ignoring group membership

SSWithin – sum of the squared deviations of individual scores from the mean of the individual’s group, based on variability within each group

‘Sample Mean’

‘Group Mean’‘Group Mean’

SSBetween – sum of the squared deviations of typical score from group from the mean of the entire sample – treats individuals as if they are ‘typical’ for their group

All 50 students as a sample

Students divided by

Level of IV

Students as ‘typical’ members of their group

When equal n, SST = SSW + SSB


SST = (x – MG)2 as if one group of participants

SSW = (xg1 – Mg1)2 variability within each

+ (xg2 – Mg2)2 group – relative to

+ (xg3 – Mg3)2 each group’s mean

SSB = n(Mg1 – MG)2 variability among group

+ n(Mg2 – MG)2 means, relative to the

+ n(Mg3 – MG)2 mean of the entire sample


Analysis of Variance

Assumptions

1. Interval/ratio data

2. Independent observations

3. Normal sampling distribution of the means (seldom a problem)

4. Homogeneity of variances (same guidelines as for t-test)

Example – multilevel design

IV is Sense of Humor of Target Person(3 levels: Below Ave - Average - Above Ave)

DV is rated liking, from low (1) to (7) high

4 participants per group (power = .09)

Levels of Sense of Humor of Target

Below Ave Average Above Ave

Means 2 4 6

Variances .67 .67 .67

low

(1)

to (

7)

hig

h lik

ea

ble

Ove

rall S

am

ple

Me

an

De

via

tio

n

Sq

ua

red

De

via

tio

n

Gro

up

Me

an

De

via

tio

n in

Gro

up

Sq

ua

red

De

via

tio

n

Gro

up

Me

an

Ove

rall S

am

ple

Me

an

M sa

mple -

- M

group

Sq

ua

red

De

via

tio

n

BA 1 4 -3 9 2 -1 1 2 4 -2 4

BA 2 4 -2 4 2 0 0 2 4 -2 4

BA 2 4 -2 4 2 0 0 2 4 -2 4

BA 3 4 -1 1 2 1 1 2 4 -2 4

2

AVE 3 4 -1 1 4 -1 1 4 4 0 0

AVE 4 4 0 0 4 0 0 4 4 0 0

AVE 4 4 0 0 4 0 0 4 4 0 0

AVE 5 4 1 1 4 1 1 4 4 0 0

2

AA 5 4 1 1 6 -1 1 6 4 2 4

AA 6 4 2 4 6 0 0 6 4 2 4

AA 6 4 2 4 6 0 0 6 4 2 4

AA 7 4 3 9 6 1 1 6 4 2 4

2

SS Total 38 SS Within 6 SS Between 32

Deviations from Overall Sample MeanDeviations within each group from group mean

Deviations of Group Means from Sample Mean

7

1

2

4

5

3

6

Rating

Subjects

1 2 3 4 5 6 7 8 9 10 11 12

Below Average Average Above Average

Overall sample mean

Group mean

Mean Square Between = Sum of Squares Between/dfbetween

(estimate of population variance based upon Means)

Mean Square Within = Sum of Squares Within/dfwithin

(estimate of population variance based upon group Variances)

MSB -- biased estimateMSW -- unbiased estimate

??

Does the variability among the means (relative to the estimated population mean) appear to be greater than the variability expected when the Null Hypothesis is true (error variability only)?

F (dfb, dfw) =

F (dfb, dfw) =

Descriptive Statistics

Dependent Variable: Liking of person on 1 to 7 scale

2.0000 .81650 4

4.0000 .81650 4

6.0000 .81650 4

4.0000 1.85864 12

Sense of Humor Levelbelow ave

average

above average

Total

Mean Std. Deviation N

Levene's Test of Equality of Error Variancesa


.000 2 9 1.000F df1 df2 Sig.

Tests the null hypothesis that the error variance ofthe dependent variable is equal across groups.

Design: Intercept+ivsoha.

Tests of Between-Subjects Effects


32.000a 2 16.000 24.000 .000 .842

192.000 1 192.000 288.000 .000 .970

32.000 2 16.000 24.000 .000 .842

6.000 9 .667

230.000 12

38.000 11

SourceCorrected Model

Intercept

ivsoh

Error

Total

Corrected Total

Type III Sumof Squares df Mean Square F Sig.

Partial EtaSquared

R Squared = .842 (Adjusted R Squared = .807)a.

The variance in each group is .81652 or .667 – the same as the MSW in the Table below

Since this is the best estimate of the population variance

If we assume each person’s rating in a group was equal to the mean (typical) rating for that group, the estimated population variance is the MSB (ivsoh)

Why are these the df’s

NOT Partial

eta2 = SSB/SST

SSBSSW

SST

Power to detect a moderate effect (eta2 of 6%) = .09

Pairwise Comparisons


-2.000* .577 .007 -3.306 -.694

-4.000* .577 .000 -5.306 -2.694

2.000* .577 .007 .694 3.306

-2.000* .577 .007 -3.306 -.694

4.000* .577 .000 2.694 5.306

2.000* .577 .007 .694 3.306

(J) Sense of Humor Levelaverage

above average

below ave

above average

below ave

average

(I) Sense of Humor Levelbelow ave

average

above average

MeanDifference

(I-J) Std. Error Sig.a

Lower Bound Upper Bound

95% Confidence Interval forDifference

a

Based on estimated marginal means

The mean difference is significant at the .05 level.*.

Adjustment for multiple comparisons: Least Significant Difference (equivalent to no adjustments).a.

Multiple Comparisons


Tukey HSD

-2.0000* .57735 .018 -3.6120 -.3880

-4.0000* .57735 .000 -5.6120 -2.3880

2.0000* .57735 .018 .3880 3.6120

-2.0000* .57735 .018 -3.6120 -.3880

4.0000* .57735 .000 2.3880 5.6120

2.0000* .57735 .018 .3880 3.6120

(J) Sense of Humor Levelaverage

above average

below ave

above average

below ave

average

(I) Sense of Humor Levelbelow ave

average

above average

MeanDifference

(I-J) Std. Error Sig. Lower Bound Upper Bound

95% Confidence Interval

Based on observed means.

The mean difference is significant at the .05 level.*.

Liking of person on 1 to 7 scale

Tukey HSDa,b

4 2.0000

4 4.0000

4 6.0000

1.000 1.000 1.000

Sense of Humor Levelbelow ave

average

above average

Sig.

N 1 2 3

Subset

Means for groups in homogeneous subsets are displayed.Based on Type III Sum of SquaresThe error term is Mean Square(Error) = .667.

Uses Harmonic Mean Sample Size = 4.000.a.

Alpha = .05.b.

Go through the steps in SPSS

AnalyzeGeneral Linear Model

Univariate

Choose DVChoose IV (Fixed Factor)

OptionsEstimated Marginal Means only needed in Multifactor Designs

Must use for Main Effects if unequal n in cellsCan use Pairwise Comparisons to get CI’s for Main Effects

Descriptive StatisticsEstimates of Effect SizeHomogeneity Tests

Post hocChoose as desiredCan reset the alpha level in Options

Examples in Handouts

Multilevel ANOVA

“Multilevel” Chi square

Situations requiring a Multilevel approach

- single IV in the design – 3 or more levels

SST = SSW + SSBiv

MSBMSW

Multilevel Designs

F (dfb, dfw) =

Situations requiring a Multifactor approach

- including multiple IVs in the design

so for the 2 x 2 multifactor design

SST = SSW + SSBiv becomes

SST = SSW + SSBiv1 + SSBiv2 + SSBinteraction

To answer 3 questions

Multifactor Designs

Independent Variable 1 – Music Level 1 = Rock Level 2 = ClassicalIndependent Variable 2 = Volume Level 1 = Low Level 2 = HighDependent Variable = Number of words recalled from a list of 20.

• Independent Variable 1• MUSIC• Rock Classical_ _ _ • 13 8• 10 5 sum = 90• Level 1 9 M = 10 11 M = 8 • Low 7 8 M = 9• 11 8• Independent Variable 2 ___________________________ ____ • VOLUME 19 3 • 16 6 sum = 110• Level 2 18 M = 16 9 M = 6 • High 13 6 M = 11• 14 6• _________________ _______ _______ • • sum = 130 sum = 70 sum = 200• • M = 13 M = 7 Ms = 10•

Overall Sample Mean

Power = .18

For eta2 = .06 (moderate effect)

(1) (2) SSW (4) SST (6) SSB || (1) (2) SSW (4) SST (6) SSBXi (Xi-Mg) (Xi-Mg)2 (Xi-MS) (Xi-MS)2 (Mg-MS) (Mg-MS)2 Xi (Xi-Mg) (Xi-Mg)2 (Xi-MS) (Xi-MS)2 (Mg-MS) (Mg-MS)2 13 13-10 9 13-10 9 10-10 0 || 8 8-8 0 8-10 4 8-10

410 10-10 0 10-10 0 10-10 0 || 5 5-8 9 5-10 25 8-10

4 9 9-10 1 9-10 1 10-10 0 || 11 11-8 9 11-10 1 8-10

4 7 7-10 9 7-10 9 10-10 0 || 8 8-8 0 8-10 4 8-10

411 11-10 1 11-10 1 10-10 0 || 8 8-8 0 8-10 4 8-10

450 0 20 0 20 0 0 || 40 0 18 -10 38 -10

20 |G1 G2|__________________________________________________

---------------------------------------------------|G3 G4|---------------------------------------------------19 19-16 9 19-10 81 16-10 36 || 3 3-6 9 3-10 49 6-10 1616 16-16 0 16-10 36 16-10 36 || 6 6-6 0 6-10 16 6-10 1618 18-16 4 18-10 64 16-10 36 || 9 9-6 9 9-10 1 6-10 1613 13-16 9 13-10 9 16-10 36 || 6 6-6 0 6-10 16 6-10 1614 14-16 4 14-10 16 16-10 36 || 6 6-6 0 6-10 16 6-10 1680 0 26 30 206 30 180 || 30 0 18 -20 98 -20 80

SST = (Xi – MS)2 SST = 20 + 38 + 206 + 98 = 362

SSW =(Xig1 – Mg1)2 + (Xig2 – Mg2)2+ (Xig3 – Mg3)2 + (Xig4 – Mg4)2

SSW = 20 + 18 + 26 + 18 = 82

SSBIV1 = n(Mgl+g3 – MS)2 + n(Mg2+g4- MS)2

SSBIV1 = 10(13-10)2 + 10(7-10)2 = 180 SSBIV2 = n(Mg1+g2 – MS)2 + n(Mg3+g4 – MS)2

SSBIV2 = 10(9-10)2 + 10(11-10)2 = 20

SSBX = SSB – SSBIV1 – SSBIV2SSBX = 280 – 180 – 20 = 80

• ANALYSIS OF VARIANCE TABLE

•

• Source SS df MS F partial eta2

• Total 362 19

• Between

• Music 180 1 180.00 35.12 .687

• Volume 20 1 20.00 3.90 .196

• Interaction 80 1 80.00 15.61 .494

• Within 82 16 5.125

• Critical F(1,16) = 4.49, p< .05, two-tailed test.

• Partial eta2 SSB/(SSB+SSW) eta2 SSB/SST

• Music 180/(180 + 82) = .687 180/362 = .497

• Volume 20/(20 + 82) = .196 20/362 = .055

• Interaction 80/(80+82) = .494 80/362 = .221

• Total 1.377 .773

Follow up analyses that might be needed with Multifactor Analyses

If Multilevel IV main effect is significant

need Multiple Comparison TestsNote – if unbalanced design, use tests available in Options to insure unweighted means are used

If Interaction is significant

need Simple Main effects tests

which could include Multiple Comparisons

Go to handouts for examples

Assumptionsinterval/ratio dataindependent observations (‘relatedness removed’)normality of sampling distributionhomogeneity of variances of difference scores (sphericity)

violations of homogeneity can be serious, and inflate the F valuesin post hoc tests, may want to use conservative test

Repeated Measures Multilevel Design

Research – testing 3 separate recipes for Chocolate Chip Cookies

IV Cookie Recipe – 3 levels

DV Rated taste 1 2 3 4 5 6 7 8 9 Awful Delicious

5 participants, each tastes and rates each cookie (design issues?)cookie-a cookie-b cookie-c Means for Ps

p1 5 7 9 7p2 3 5 7 5p3 1 3 8 3p4 3 7 8 6p5 5 7 9 7

Mcookie 3.4 5.8 8.2 5.8

Main effect for Participants – reflects differences across the participants

Main effect for Cookie

Examples of SPSS in Handouts

Show how to define RM variables in SPSS

Combines aspects of both types (independent levels and related levels)

but the error (MSW) used depends on whether the ‘effect’ is:

truly independent levels, orrelated levels that have been

‘adjusted’Assumptions

interval ratio datanormality of the sampling distributionindependent observationsfor independent levels effects – homogeneity of variancesfor related levels effects – homogeneity of variances of

difference scores (sphericity)

Mixed Designs

Mixed exampleCookie Recipe (2) x Temperature (2) [ x Participants (n per group)]DV Rated taste

Awful 1 2 3 4 5 6 7 8 9 Delicious

Two cookies tasted by each person - within subjects effectsome get warm cookies (n = 5), - between subjects

effectsome get room temperature (cold) cookies (n = 5),

Warm ColdCookie 1 Cookie 2 M Cookie 1 Cookie 2 M

1 5 7 6 3 5 42 6 8 7 4 7 5.53 8 7 7.5 3 5 44 6 8 7 3 5 45 6 8 7 4 6 5

6.2 7.6 6.9 3.4 5.6 4.5

SSB – uncontaminated by relatednessSSB (Temperature – 2 levels)SSW (Participants within groups –

variability among means of participants)

SSW – systematic, but contaminated by relatednessSSB (cookies – 2 repeated measures levels)SSB interaction (2 x 2)SSW variability among participants

2 x 2 Mixed Multifactor (treated as if 2 x 2 x 5) – but ignore Participants effects Partition SS into

Simple Group Comparisons Limits you to simple explanatory variables simple potential relationships.

Documents

Transcript of Simple Group Comparisons Limits you to simple explanatory variables simple potential relationships.