D1

o

Structural equation modeling

Rex B Kline Concordia University

Montréal

ISTQL Set D CFA models

D2

Resources o Bollen, K. A., & Hoyle, R. H. (2012). Latent variable

models in structural equation modeling. In R. H. Hoyle (Ed.), Handbook of structural equation

modeling (pp. 56–67). New York: Guilford. o Fabrigar, L. R., & Wegener, D. T. (2012). Exploratory

factor analysis. New York: Oxford University Press. o Kline, R. B. (2013b). Exploratory and confirmatory

factor analysis. In Y. Petscher & C. Schatsschneider (Eds.), Applied quantitative analysis in the social

sciences (pp. 171–207). New York: Routledge.

D3

EFA

o Phases:

1. Specification

2. Extraction

3. Retention

4. Rotation

D4

Extraction methods

1. Principle components analysis (PCA)

2. Principle axis factoring (PAF)

3. Alpha factoring

4. ML factoring

D5

PCA

B A

X4 X5 X6 X3 X2 X1

D6

PAF

A B

X6

E6

X5

E5

X4

E4

X1

E1

X2

E2

X3

E3

D7

Indicator variance

1 − rXX

Error

Unique

Common Specific

Systematic

D8

EFA

o Retention:

No need to specify

But best by theory

D9

EFA

o Retention:

Parallel analysis

Scree plots

D10

4

2

0

3

1

1 2 3 4 5 6 7 8

Factor

Eig

en

va

lue

D11

EFA

o Rotation:

1. Orthogonal

2. Oblique

D12

EFA

o Orthogonal:

1. Varimax

2. Quartimax

3. Equamax

D13

EFA

o Oblique:

1. Promax

2. Oblimin

D14

EFA

o Rotation:

Infinite

Not identified

D15

a) EFA (unrestricted; rotation)

A B

X6

E6

X5

E5

X4

E4

X1

E1

X2

E2

X3

E3

1

1

X1

E1

1

X2

E2

1

X3

E3

A

1

1

X4

E4

1

X5

E5

1

X6

E6

B

b) CFA (restricted; no rotation)

D16

CFA after EFA

o Does not “confirm” EFA:

Restricted vs. unrestricted

Items are “noisy”

Follow EFA with EFA

D17

CFA after EFA o Osborne, J. W., & Fitzpatrick, D. C. (2012). Replication

analysis in exploratory factor analysis: What it is and why it makes your analysis better. Practical

Assessment, Research & Evaluation, 17. Retrieved from http://pareonline.net/pdf/ v17n15.pdf

o van Prooijen, J.-W., & van der Kloot, W. A. (2001). Confirmatory analysis of exploratively obtained factor structures. Educational and Psychological

Measurement, 61, 777–792.

D18

D19

Background

1

EE

Ethnic Age

1

EA

Gender

1

EG

1

D20

A

1

E3

X3 X2

1

E2

X1

1

E1

1 +

−

D21

CFA specification

o Standard model:

Continuous indicators (X)

A → X ← E

D22

Reflective measurement

X T E= +

2 2 2

X T Eσ = σ + σ

2

2

σ=

σT

XX

X

r

D23

Reflective measurement

1−XXr

but rXX estimates a single source

D24

CFA specification

o Standard model:

Independent E

A B

D25

CFA specification

o Unidimensional:

Simple indicator (A → X only)

No Ei Ej

D26

CFA specification

o Unidimensional:

Precise test

Convergent validity

Discriminant validity

D27

CFA specification

o Multidimensional:

Complex indicator

Ei Ej

D28

CFA specification

o Ei Ej:

Indicators share something

Repeated measures

D29

CFA specification

o Multidimensional caution:

Increases complexity

“Cheap” way to improve fit

D30

CFA specification

o Special variations:

Hierarchical CFA

MTMM models

D31

g

1

1

X4

E4

1

X5

E5

1

X6

E6

Visual-

Spatial

1

DVS

1

1

X1

E1

1

X2

E2

1

X3

E3

1

DVe

1

1

X7

E7

1

X8

E8

1

X9

E9

Memory

1

DMe

1

Verbal

D32

X4 X5 X6 X1 X2 X3 X7 X8 X9

1 1 1

Trait 1 Trait 2 Trait 3

1 1 1

Method 1 Method 2 Method 3

D33

Trait 2 Trait 1 Trait 3

1 1 1

X1

1

E1

X2

1

E2

X3

1

E3

X4

1

E4

X5

1

E5

X6

1

E6

X7

1

E7

X8

1

E8

X9

1

E9

D34

CFA specification

o Eid, M., Nussbeck, F. W., Geiser, C., Cole,

D. A., Gollwitzer, M., & Lischetzke, T. (2008). Structural equation modeling of multitrait-multimethod data: Different models for different types of methods. Psychological Methods, 13, 230–253.

D35

CFA identification

o Necessary:

dfM ≥ 0

Scale each latent

D36

Scale E

ULI constraint:

D37

Scale factor 1. Reference (marker) variable

ULI = 1, unstandardized

2. Standardize factors UVI = 1

3. Effects coding AVE = 1, all same metric

D38

1

1

X1

E1

1

X2

E2

1

X3

E3

A

1

1

X4

E4

1

X5

E5

1

X6

E6

B

D39

1

1

X4

E4

1

X5

E5

1

X6

E6

B 1

1

X1

E1

1

X2

E2

1

X3

E3

A

D40

1 2 3 13

λ + λ + λ=

λ1

1

X1

E1

1

X2

E2

1

X3

E3

A

λ3 λ2

D41

1 2 3 13

λ + λ + λ=

1 2 33λ = − λ − λ

2 1 33λ = − λ − λ

3 1 23λ = − λ − λ

D42

CFA identification

o Counting parameters:

1. Exog: Vars. + Covs.

2. Endog: Direct effects

D43

CFA identification

o Standard models:

1 factor, ≥ 3 indicators

≥ 2 factors, ≥ 2 indicators

But…

D44

CFA identification

o Nonstandard models:

No single heuristic

Undecidable

Ambiguous status

D45

TABLE 6.1. Identification Rule 6.6 for Nonstandard Confirmatory Factor Analysis Models with

Measurement Error Correlations

For a nonstandard CFA model with measurement error correlations (Rule 6.6)

to be identified, all three of the conditions listed next must hold:

For each factor, at least one of the following must hold: (Rule 6.6a)

1. There are at least three indicators whose errors are uncorrelated with each other.

2. There are at least two indicators whose errors are uncorrelated and either

a. the errors of both indicators are not correlated with the error term of a third

indicator for a different factor, or

b. an equality constraint is imposed on the loadings of the two indicators.

For every pair of factors, there are at least two indicators, one from (Rule 6.6b)

each factor, whose error terms are uncorrelated.

For every indicator, there is at least one other indicator (not necessarily (Rule 6.6c)

of the same factor) with which its error term is not correlated.

D46

For each factor, at least one of the following must hold: (Rule 6.6a)

1. There are at least three indicators whose errors are uncorrelated with each other.

2. There are at least two indicators whose errors are uncorrelated and either

a. the errors of both indicators are not correlated with the error term of a third

indicator for a different factor, or

b. an equality constraint is imposed on the loadings of the two indicators.

(c)

X1

EX1

1

X2

EX2

1

A

1

X3

EX3

1

X4

EX4

1

B

1

X1

EX1

1

X2

EX2

1

A

1

X3

EX3

1

X4

EX4

1

B

1

(d)

� �

D47

TABLE 6.2. Identification Rule 6.7 for Multiple Loadings of Complex Indicators in Nonstandard

Confirmatory Factor Analysis Models and Rule 6.8 for Error Correlations of Complex Indicators

Factor loadings

For every complex indicator in a nonstandard CFA model: (Rule 6.7)

In order for the multiple factor loadings to be identified, both

of the following must hold:

1. Each factor on which the complex indicator loads must satisfy

Rule 6.6a for a minimum number of indicators.

2. Every pair of those factors must satisfy Rule 6.6b that each

factor has an indicator that does not have an error correlation

with a corresponding indicator on the other factor of that pair.

Error correlations

In order for error correlations that involve complex indicators (Rule 6.8)

to be identified, both of the following must hold:

1. Rule 6.7 is satisfied.

2. For each factor on which a complex indicator loads, there must be

at least one indicator with a single loading that does not have an

error correlation with the complex indicator.

D48

CFA estimates

o Unstandardized:

1. Indicators loadings (B)

2. Factor, error variances

3. Factor, error covariances

D49

CFA estimates

o Standardized:

1. Indicators loadings (r, b)

2. Proportion unexplained

3. Factor, error correlations

D50

CFA estimates

o Failure to converge:

1. Data matrix (NPD)

2. Poor start values

3. Small N, 2 ind./factor

D51

CFA estimates

o Heywood cases (inadmissible):

1. Error variance < 0

2. | r or R2 | > 1.0

3. NPD parameter matrix

D52

1 1

Family of Origin

Marital

Adjustment

Father

1

EFa

Mother

1

EMo

Father- Mother

1

EFM

1

EIn

Intimacy

1

EPr

Problems

D53

Group 2: Wives

THETA-DELTA

problems intimacy father mother fa_mo

-------- -------- -------- -------- --------

problems 520.305

(130.844)

3.977

intimacy - - -27.093

(104.927)

-0.258

father - - - - 32.147

(29.214)

1.100

mother - - - - 9.967 63.416

(26.870) (28.138)

0.371 2.254

fa_mo - - - - - - - - 97.049

(25.232)

3.846

Squared Multiple Correlations for X - Variables

problems intimacy father mother fa_mo

-------- -------- -------- -------- --------

0.520 1.052 0.821 0.661 0.531

D54

CFA estimates

o Heywood causes:

Identification

Poor start values

Small N, 2 inds./factor

D55

CFA analysis

o Testing strategy:

1. Fit 1-factor model

2. Nested under higher-order

3. Compare with 2

Dχ

D56

1

ETr

Triangles

1

Spatial Memory

ESM

1

Matrix Analogies

EMA

1

Gestalt Closure

EGC

1

Photo Series

EPS

1

EHM

Hand Movements

1

Number Recall

ENR

1

Word Order

EWO

1

General

1 1 1 1 1 1 1 1

1 1

Sequential Processing

EHM

Hand Movements

Number Recall

ENR

Word Order

EWO

Simultaneous Processing

ETr

Triangles Spatial

Memory

ESM

Matrix Analogies

EMA

Gestalt Closure

EGC

Photo Series

EPS

D57

CFA analysis

o Example: 4-factor model:

4 vs. 3

4 vs. 2

4 vs. 1

D58

CFA respecify

o Options:

1. Number of factors

2. Indicator-factor match

3. Error correlations

D59

CFA respecify

o Residual patterns:

Result Correlation residuals Respecification

Indicator has low standardized loading on original factor

High correlation residuals with indicators of another factor

Switch loading of indicator to other factor

Indicator has reasonably high standardized loading on original factor

High correlation residuals with indicators of another factor

Allow indicator to also load on the other factor Allow measurement errors to covary

D60

CFA respecify

o Wrong number of factors:

Discriminant validity

Convergent validity

D61

CFA respecify

o MIs in latent variable models:

Approach with caution

Nonsensical respecification

May not be identified

D62

Observations = v (v + 1)/2 = 36

Parameters = 17

dfM = 19

1 1 1 1 1 1 1 1

1 1

Sequential Processing

EHM

Hand Movements

Number Recall

ENR

Word Order

EWO

Simultaneous Processing

ETr

Triangles Spatial

Memory

ESM

Matrix Analogies

EMA

Gestalt Closure

EGC

Photo Series

EPS

D63

Exogenous variables

Direct effects on endogenous variables Variances Covariances Total

Sequential → NR Sequential → WO Seq, Sim Seq Sim 17

Simultaneous → Tr Simultaneous → SM E terms (8)

Simultaneous → MA Simultaneous → PS

D64

Example

o Amos

o EQS o lavaan

o LISREL

o Mplus

o Stata

D65

title: principles and practice of sem (4th ed.), rex kline

two-factor model of the kabc-i, figure 9.7, table 13.1

data: file is "kabc-mplus.dat";

type is stdeviations correlation;

nobservations = 200;

variable: names are handmov numbrec wordord gesclos triangle spatmem

matanalg photser;

analysis: type is general;

model:

Sequent by handmov numbrec wordord;

Simul by gesclos triangle spatmem matanalg photser

! first indicator in each list is automatically

! specified as the reference variable

output: sampstat modindices(all, 0) residual standardized tech4;

! requests sample data matrix, residual diagnostics,

! modification indexes > 0, all standardized

! solutions (STDYX is reported), and estimated

! correlation matrix for all variables

D66

3.40 2.40 2.90 2.70 2.70 4.20 2.80 3.00

1.00

.39 1.00

.35 .67 1.00

.21 .11 .16 1.00

.32 .27 .29 .38 1.00

.40 .29 .28 .30 .47 1.00

.39 .32 .30 .31 .42 .41 1.00

.39 .29 .37 .42 .58 .51 .42 1.00

D67

CFA indicators

o Indicators:

Scale: Default ML

Likert: Other method

D68

CFA indicators

o Item distributions:

1. Binary (e.g., T / F)

2. Likert (3-6)

3. Likert (≥ 7)

D69

CFA indicators

o Estimation options:

1. Corrected ML:

a. Robust SEs

b. Santorra-Bentler

D70

CFA indicators

o Estimation options:

2. Robust WLS:

a. Item thresholds

b. Latent response variable

D71

CFA indicators

o Threshold:

Location on latent dimension

Differentiates categories

Estimated as z

D72

Example: 1 = disagree

2 = not sure

3 = agree

−1.62 1.15

D73

A

1X*

1 1X

E *

2X*

1 2X

E *

3X*

1 3X

E *

X1 X2 X3

D74

CFA indicators

o Latent response variables:

Sample polychoric

Predicted polychoric

Correlation residuals

D75

CFA indicators

o Estimation options:

3. ML + numerical integration

a. ↑ computation b. Markov chain Monte

Carlo

D76

D77

CFA indicators

o Estimation options:

4. IRT, ICC

a. Difficulty, discrimination

b. Logit, probit link

D78

3.0 2.0 1.0 0 −1.0 −2.0 −3.0

Latent Ability (θ)

.9

.8

.7

.6

.5

.4

.3

.2

.1

0

1.0

Pro

ba

bili

ty o

f C

orr

ec

t

Re

spo

nse

tangent line

difficulty

ICC

D79

CFA indicators

o Estimation options:

5. Bootstrapping:

a. Very biased small N

b. Not as developed

D80

CFA indicators

o Estimation options:

6. Create parcels:

a. Homogenous item set

b. Total score

D81

● ● ●

1

A

It 1 It 2 It 33 ● ● ●

1

B

It 66 It 34 It 35 ● ● ●

1

It 99 It 67 It 68

C

A

1

Pr 1 (It 1–It 11)

Pr 2 (It 12–It 22)

Pr 3 (It 23–It 33)

B

1

Pr 4 (It 34–It 44)

Pr 5 (It 45–It 55)

Pr 6 (It 26–It 66)

C

1

Pr 8 (It 78–It 88)

Pr 9 (It 89–It 99)

Pr 7 (It 67–It 77)

D82

Cautions about parcels

1. Assumes unidimensional

2. Ways to parcel

3. Mask multidimensionality

D83

CFA indicators

o Edwards, M. C., Wirth, R. J., Houts, C. R., & Xi, N.

(2012). Categorical data in the structural equation modeling framework. In R. Hoyle (Ed.), Handbook of structural equation

modeling (pp. 195–208). New York: Guilford Press.

o Wirth, R. J., & Edwards, M. C. (2007). Item factor analysis: Current approaches and future directions. Psychological Methods, 12, 58–79.

D84

CFA indicators

o Bernstein, I. H., & Teng, G. (1989). Factoring items and

factoring scales are different: Spurious evidence for multidimensionality due to item categorization. Psychological Bulletin, 105, 467–477.

o Bandalos, D. L., & Finney, S. J. (2001). Item parceling issues in structural equation modeling. In G. A. Marcoulides and R. E. Schumaker (Eds.), New

developments and techniques in structural

equation modeling (pp. 269–296). Mahwah, NJ: Erlbaum.

D85

Exploratory SEM

o CFA-EFA-SR hybrid

o Restricted + unrestricted

o EFA part is rotated

D86

Y6 Y5 Y4 Y3 Y1 Y2

1 1

DF 1

DC

EY1

1 EY2

1 EY3

1 EY4

1 EY5

1 EY6

1

C F

X1 EX1 1

EX2 1

X2

X3 EX3 1

X4 EX4 1

X5 EX5 1

X6 EX6 1

1

A

1

B

D87

Exploratory SEM

o Marsh, H. W., Morin, A. J. S., Parker, P. D., &

Kaur, G. (2014). Exploratory structural equation modeling: Integration of the best features of exploratory and confirmatory factor analysis. Annual

Review of Clinical Psychology, 10, 85–110.