MEASUREMENT MODELS

BASIC EQUATION

• x = + e

• x = observed score = true (latent) score: represents

the score that would be obtained over many independent administrations of the same item or test

• e = error: difference between y and

ASSUMPTIONS and e are independent

(uncorrelated)

• The equation can hold for an individual or a group at one occasion or across occasions:

• xijk = ijk + eijk (individual)

• x*** = *** + e*** (group)

• combinations (individual across time)

x x

e

RELIABILITY

• Reliability is a proportion of variance measure (squared variable)

• Defined as the proportion of observed score (x) variance due to true score ( ) variance:

2x = xx’

• = 2 / 2

x

Var()

Var(x)

Var(e)

reliability

Reliability: parallel forms

• x1 = + e1 , x2 = + e2

(x1 ,x2 ) = reliability

• = xx’

• = correlation between parallel forms

x1 x

e

x2

e

x

xx’ = x * x

ASSUMPTIONS and e are independent

(uncorrelated)

• The equation can hold for an individual or a group at one occasion or across occasions:

• xijk = ijk + eijk (individual)

• x*** = *** + e*** (group)

• combinations (individual across time)

Reliability: Spearman-Brown

• Can show the reliability of the composite is

kk’ = [k xx’]/[1 + (k-1) xx’ ]

• k = # times test is lengthened

• example: test score has rel=.7

• doubling length produces rel = 2(.7)/[1+.7] = .824

Reliability: parallel forms

• For 3 or more items xi, same general form holds

• reliability of any pair is the correlation between them

• Reliability of the composite (sum of items) is based on the average inter-item correlation: stepped-up reliability, Spearman-Brown formula

RELIABILITY

Generalizability d - coefficients ANOVA

g - coefficients

Cronbach’s alpha

test-retest internal consistency

inter-rater

parallel form Hoyt

dichotomous split halfscoring

KR-20 SpearmanKR-21 Brown

averageinter-itemcorrelation

COMPOSITES AND FACTOR STRUCTURE

• 3 MANIFEST VARIABLES REQUIRED FOR A UNIQUE IDENTIFICATION OF A SINGLE FACTOR

• PARALLEL FORMS REQUIRES:– EQUAL FACTOR LOADINGS– EQUAL ERROR VARIANCES– INDEPENDENCE OF ERRORS

x1

x

e

x2

e

x

xx’ = xi * xj

x3

e

x

RELIABILITY FROM SEM• TRUE SCORE VARIANCE OF THE

COMPOSITE IS OBTAINABLE FROM THE LOADINGS:

K = 2

i i=1

K = # items or subtests

• = K2x

Hancock’s Formula

• Hj = 1/ [ 1 + {1 / (Σ[l2ij/(1- l2

ij )] ) }

• Ex. l1 = .7, l2= .8, l3 = .6

• H = 1 / [ 1 +1/( .49/.51 + .64/.36 + .36/.64 )]

= 1 / [ 1 + 1/ ( .98 +1.67 + .56 ) ]

= 1/ (1 + 1/3.21)

= .76

Hancock’s Formula Explained

Hj = 1/ [ 1 + {1 / (Σ[l2ij/(1- l2

ij )] ) }

now assume strict parallelism: then l2ij= 2

xt

thus Hj = 1/ [ 1 + {1 / (Σ[2xt /(1- 2

xt)] ) }

= k 2xt / [1 + (k-1) 2

xt ]

= Spearman-Brown formula

RELIABILITY FROM SEM

• RELIABILITY OF THE COMPOSITE IS OBTAINABLE FROM THE LOADINGS:

= K/(K-1)[1 - 1/ ]

• example 2x = .8 , K=11

= 11/(10)[1 - 1/8.8 ] = .975

SEM MODELING OF PARALLEL FORMS

• PROC CALIS COV CORR MOD;

• LINEQS

• X1 = L1 F1 + E1,

• X2 = L1 F1 + E1,

• …

• X10 = L1 F1 + E1;

• STD E1=THE1, F1= 1.0;

TAU EQUIVALENCE

• ITEM TRUE SCORES DIFFER BY A CONSTANT:

i = j + k

• ERROR STRUCTURE UNCHANGED AS TO EQUAL VARIANCES, INDEPENDENCE

TESTING TAU EQUIVALENCE

• ANOVA: TREAT AS A REPEATED MEASURES SUBJECT X ITEM DESIGN:

• PROC VARCOMP;CLASS ID ITEM;

• MODEL SCORE = ID ITEM;

• LOW VARIANCE ESTIMATE CAN BE TAKEN AS EVIDENCE FOR PARALLELISM (UNLIKELY TO BE EXACTLY ZERO

CONGENERIC MODEL

• LESS RESTRICTIVE THAN PARALLEL FORMS OR TAU EQUIVALENCE:– LOADINGS MAY DIFFER– ERROR VARIANCES MAY DIFFER

• MOST COMPLEX COMPOSITES ARE CONGENERIC:– WAIS, WISC-III, K-ABC, MMPI, etc.

x1

x1

e1

x2

e2

x2

(x1 , x2 )= x1 * x2

x3

e3

x3

COEFFICIENT ALPHA

xx’ = 1 - 2E /2

X

• = 1 - [2i (1 - ii )]/2

X ,

• since errors are uncorrelated = K/(K-1)[1 - (s2

i )/ s2X ]

• where X = xi (composite score)

s2i = variance of subtest xi

sX = variance of composite

• Does not assume knowledge of subtest ii

COEFFICIENT ALPHA- NUNNALLY’S COEFFICIENT

• IF WE KNOW RELIABILITIES OF EACH SUBTEST, i

N = K/(K-1)[s2i (1- rii )/ s2

X ]

• where rii = coefficient alpha of each subtest

• Willson (1996) showed N

SEM MODELING OF CONGENERIC FORMS

MPLUS EXAMPLE: this is an example of a CFA

DATA: FILE IS ex5.1.dat;

VARIABLE: NAMES ARE y1-y6;

MODEL: f1 BY y1-y3;

f2 BY y4-y6;

OUTPUT: SAMPSTAT MOD STAND;

x1

x1

e1

x2

e2

x2

XiXi = 2xi + s2

i

x3

e3

x3

s1

NUNNALLY’S RELIABILITY CASE

s2

s3

x1

x1

e1

x2

e2

x2

Specificities can be

misinterpreted as a correlated

error model if they are

correlated or a second factor

x3

e3

x3

s

CORRELATED ERROR PROBLEMS

s3

x1

x1

e1

x2

e2

x2

Specificieties can be

misinterpreted as a

correlated error model

if specificities are

correlated or are a

second factor

x3

e3

x3

CORRELATED ERROR PROBLEMS

s3

SEM MODELING OF CONGENERIC FORMS- CORRELATED ERRORS

MPLUS EXAMPLE: this is an example of a CFA

DATA: FILE IS ex5.1.dat;

VARIABLE: NAMES ARE y1-y6;

MODEL: f1 BY y1-y3;

f2 BY y4-y6;

y4 with y5;

OUTPUT: SAMPSTAT MOD STAND;

specifies residuals of previous model, correlates them

MULTIFACTOR STRUCTURE

• Measurement Model: Does it hold for each factor?– PARALLEL VS. TAU-EQUIVALENT VS.

CONGENERIC

• How are factors related?

• What does reliability mean in the context of multifactor structure?

SIMPLE STRUCTURE

• PSYCHOLOGICAL CONCEPT:

• Maximize loading of a manifest variable on one factor ( IDEAL = 1.0 )

• Minimize loadings of the manifest variables on all other factors ( IDEAL = 0 )

SIMPLE STRUCTURE

Example

SUBTEST FACTOR1 FACTOR2 FACTOR3

A 1 0 0

B 1 0 0

C 0 1 0

D 0 1 0

E 0 0 1

F 0 0 1

MULTIFACTOR ANALYSIS

• Exploratory: determine number, composition of factors from empirical sampled data– # factors # eigenvalues > 1.0 (using squared

multiple correlation of each item/subtest i with

the rest as a variance estimate for 2xi

– empirical loadings determine structure

MULTIFACTOR ANALYSISTITLE:this is an example of an exploratory

factor analysis with continuous factor

indicators

DATA: FILE IS ex4.1.dat;

VARIABLE:NAMES ARE y1-y12;

ANALYSIS:TYPE = EFA 1 4;

MULTIFACTOR MODEL WITH THEORETICAL

PARAMETERS

MPLUS EXAMPLE: this is an example of a CFA

DATA: FILE IS ex5.1.dat;

VARIABLE: NAMES ARE y1-y6;

MODEL: f1 BY y1@.7 y2@.8 y3@.6;

f2 BY y4@.6 y5@.7 y6@.8;

f1 with f2@.7;

OUTPUT: SAMPSTAT MOD STAND;

1

x1

x11

e1

x2

e2

x22

x3

e3

x31

MINIMAL CORRELATED FACTOR STRUCTURE

2

x4e4

x42

12

FACTOR RELIABILITY• Reliability for Factor 1:

= 2(x11 * x31 ) / (1 + x11 * x31 )(Spearman-Brown for Factor 1 reliability

based on the average inter-item correlation

• Reliability for Factor 2:

= 2(x22 * x42 ) / (1 + x22 * x42 )

FACTOR RELIABILITY• Generalizes to any factors- reliability is

simply the measurement model reliability for the scores for that factor

• This has not been well-discussed in the literature– problem has been exploratory analyses produce

successively smaller eigenvalues for factors due to the extraction process

– second factor will in general be less reliable using loadings to estimate interitem r’s

FACTOR RELIABILITY• Theoretically, each factor’s reliability should be

independent of any other’s, regardless of the covariance between factors

• Thus, the order of factor extraction should be independent of factor structure and reliability, since it produces maximum sample eigenvalues (and sample loadings) in an extraction process.

• Composite is a misnomer in testing if the factors are treated as independent constructs rather than subtests for a more global composite score (separate scores rather than one score created by summing subscale scores)

CONSTRAINED FACTOR MODELS

• If reliabilities for scales are known independent of the current data (estimated from items comprising scales, for example), error variance can be constrained:

• s2ei = s[1 - i ]

x1

x1

e1

x2

e2

x2

x3

e3

x3

CONSTRAINED SEM- KNOWN RELIABILITY

sx3 [1- 3 ]1/2

sx1 [1- 1 ]1/2 sx2 [1- 2 ]1/2

CONSTRAINED SEM-KNOWN RELIABILITY

MPLUS EXAMPLE: this is an example of a CFA with known error unreliabilities

DATA: FILE IS ex5.1.dat;

VARIABLE: NAMES ARE y1-y6;

MODEL: f1 BY y1-y3;

f2 BY y4-y6;

y1@.4;

y2@.3;

OUTPUT: SAMPSTAT MOD STAND;

similar statement for each item

SEM Measurement Procedures

• 1. Evaluate the theoretical measurement model for ALL factors (not single indicator variables included)

• Demonstrate discriminant validity by showing the factors are separate constructs

• Revise factors as needed to demonstrate- drop some manifest variables if necessary and not theoretically damaging

• Ref: Anderson & Gerbing (1988)