Post on 05-Jan-2016
Multigroup Models
Beaujean Chapter 4Brown Chapter 7
Terms
• Invariant – Equivalent – Means that the structures, items, etc. are equal– Even better definition:• Under different conditions, do the measurements yield
the same measurement of attributes?
Terms
• Means and covariances model– For these type of models, you must estimate both
the means and covariances– This change lets you test many different pieces of
the model, rather than just the correlations between items
Terms
• Intercepts – think about regular regression:– The Y-intercept is the mean of Y – that is your best
guess for what someone is going to score on that item without any other X information.
– Same idea here, it’s the starting value for a manifest variable.
Terms
• Latent mean– The score of each item * the path for each item all
averaged together. – Think about this value as a weighted mean.
Terms
• Endogenous questions will get intercepts, while exogenous variables will get means.
Identification
• When you add means and intercepts, you are obviously estimating more things
• Ways to make sure it’s identified:– Standardize the latent (sets the latent mean to
zero, estimates all the intercepts)– Scale on a manifest variable (fixes one variable to
one and that intercept to zero)– Effects coding (eww)
Questions
• Do items act the same across groups?• Is the factor structure the same across
groups?• Are the paths equal across groups?• Are the latent means equal across groups?• Does this replicate?
Quick Note
• Most people only consider two groups at a time.– You can do more than two groups but it gets very
complex
So how do we test this?
• We start by working with the least restrictive model– This model is the base CFA you are working with.– You start by putting everyone together regardless
of group (because if the regular CFA is bad, multigroup testing is not appropriate).
So how do we test this?
• We start by working with the least restrictive model– Then you put them all together in one big model,
separated by group but:– You let the estimates be different across groups
So how do we test this?
• Then we move to more restrictive models– Force estimates to be equal across groups– Pick one estimate at a time, so you can see where
the model breaks (or doesn’t)
Steps
• First, test the model as a regular CFA with everyone in the dataset – Do not group them– You must establish that the CFA is good first
Steps
• Second, test each group separately– Subsetting!
Steps
• Second, test each group separately– Pick the one group, get the fit indices– Switch to the other group, get the fit indices
• Fit?– If the fit for one group is extremely different (or
bad), you would stop here. You need them to be roughly equal.
Steps
• Nesting!– The next steps will be to nest the two models
together.– Nesting is like stacking the models together (like
pancakes).
Steps
• For the nested model steps, most people use Brown’s terminology and procedure.
• The Beaujean terms are a mix of different ones (there’s nothing wrong with them, but I’ve mostly seen Brown’s published).
Steps
• All the possible paths:– The whole model (the picture)– Loadings (regression weights)– Intercepts (y-intercept for each item)– Error variances (variance)– Factor variances (variances for the latents)– Factor covariances (correlation)– Factor means (latent means)
Steps
• Y = a + bx + e– A = intercept– B = Loading– E = residual
Steps
• Equal form / configural invariance– In this model, you put the two groups together
into the same model.– You do not force any of the paths to be the same,
but you are forcing the model picture to be the same.
– You are testing if both groups show the same factor structure (configuration).
Steps
• Metric Invariance– In this model, you are forcing all the factor
loadings (regression weights) to be exactly the same
– This step will tell you if the groups have the same weights for each question – or if some questions have different signs or strengths
Steps
• Scalar Invariance– In this model, you are forcing the intercepts of the
items to be the same.– This step will tell you if items have the same
starting point – remember that the y-intercept is the mean of the item.
– If a MG model is going to indicate non-invariance – this step is usually the one that breaks.
Steps
• Strict Factorial Invariance– In this model, you are forcing the error variances
for each item to be the same. – This step will tell you if the variance (the spread)
of the item is the same for each item. If you get differences, that indicates one group has a larger range of answers than another. (means they are more heterogeneous).
Steps
• Population Heterogeneity – Equal factor variances • Testing if latents have the same set of variance – means
that the overall score has the same spread
– Equal factor covariances• Testing if the correlations between factors is the same
for each group
– Equal latent means• Testing if the overall latent means are equal for each
group
How to tell?
• How can I tell if steps are invariant?– You will expect fit to get worse as you go because
you are being more and more restrictive.– You can use a change in chi-square test (not
suggested too much because of chi-square issues we’ve discussed before).
– Most people use the change in CFI test.
What next?
• What do I do if steps are NOT invariant?• Partial invariance – when strict invariance
cannot be met, you can test for partial invariance
Partial Invariance
• Partial invariance occurs when most of the items are invariant but a couple.– You have to meet the invariance criteria, so you
trying to bring your bad step “up” to the invariant level
– You want to do as few of items as possible (see table in handout).
Partial Invariance
• So, how do I test partial invariance?– You will change ONE item at a time.
Lavaan
• We’ve talked before about setting paths equal to each other by calling them the same parameter name– Cheese =~ a*feta + a*swiss– So feta and swiss will be estimated at the same
value.
Lavaan
• A way to set all things equal– group.equal = “keyword”– Put in the sem() or cfa() function
Lavaan
• Options for group.equal– loadings (for manifest)– intercepts (for manifest)– means (for latents)– residuals (for manifest)– residual.covariances (for manifest)– lv.variances (for latents)– lv.covariances (for latents)– regressions (for manifest)
Example
• RS14 Scale!– Men = 1– Women = 2– Black = 1– White = 2
Overall Model
• First, let’s program the overall model to make sure it fits ok.
• NEW: meanstructure = TRUE– Gives you the intercepts and means in the model.
Overall Model
• Fit for the model– CFI = .927– RMSEA = .086– SRMR = .041– X2(77) = 368.984
Separate Group Models
• Test men and then women separately before nesting them together.
Separate Group ModelsMen Women
CFI .920 .895
RMSEA .102 .094
SRMR .041 .052
X2(77) 271.457 258.610
The women model fit is a little bit worse, but we won’t know if it’s significant (or where) until we combine them.
Configural Invariance
• Now, combine the models together and determine if they have the same factor structure. – Use the group function in the CFA code.– group = “variable name”
Configural InvarianceConfigural Metric Scalar Strict
CFI .909
RMSEA .098
SRMR .047
X2 (154) = 530.067
Metric Invariance
• For metric invariance, we are going to set loadings across groups equal.– Use the code group.equal=c(“loadings”) in the
sem() or cfa() function. • Check out what that does in the output
Metric InvarianceConfigural Metric Scalar Strict
CFI .909 .909
RMSEA .098 .094
SRMR .047 .057
X2 (154) = 530.067 (167) = 545.518
DIFFERENT? n/a NOPECFI = 0.000
Scalar Invariance
• Intercepts are set to equal by adding “intercepts” to the group.equal part.– We are slowly making the model more restrictive,
so you want to add constraints, not just change them.
Scalar Invariance
Scalar InvarianceConfigural Metric Scalar Strict
CFI .909 .909 .902
RMSEA .098 .094 .094
SRMR .047 .057 .060
X2 (154) = 530.067 (167) = 545.518 (180) = 585.327
DIFFERENT? n/a NOPECFI = 0.000
NOPECFI = .007
Strict Invariance
• Residuals for the manifest variables are set to equal by adding in residuals to the group.equal part.
Strict InvarianceConfigural Metric Scalar Strict
CFI .909 .909 .902 .888
RMSEA .098 .094 .094 .097
SRMR .047 .057 .060 .062
X2 (154) = 530.067 (167) = 545.518 (180) = 585.327 (194) = 660.398
DIFFERENT? n/a NOPECFI = 0.000
NOPECFI = .007
YESCFI = .014
So what now?
• Partial invariance– We have to figure out where the problem is for
our residuals that says they aren’t equal.– We’ll use the modindices function to find the
biggest issue but only for the items in the step you were having problems with.
Partial Invariance
• Figure out which level you need:– ##metric =~ for loadings– ##scalar ~1 for intercepts– ##strict ~~ for variances
• Subset out that information:– variances = partialmod[partialmod$op == "~~", ]– variances[order(variances$mi,
decreasing=TRUE), ]
Partial Invariance
• Remember that not everything is useful!– So, in this case, adding correlated variances is not
helpful.– We want to look for the times that variances are
not equal to variances– RS9~~RS9
Partial Invariance
• Since that constraint is the problem, we are going to let it go free!– group.partial = c("RS9 ~~ RS9")– You can add more here if you want, but do one of
them at a time to determine when to stop.
Partial Invariance
• Women
• Men
Think about partial invariance as mini t-tests.
Strict InvarianceScalar Strict RS9 Free
CFI .902 .888 .891
RMSEA .094 .097 .096
SRMR .060 .062 .062
X2 (180) = 585.327
(194) = 660.398
(193) = 646.559
DIFFERENT? NOPECFI = .007
YESCFI = .014
YESCFI = .011
NOTE: compare to scalar not strict.
Partial Invariance
• Women
• Men
Think about partial invariance as mini t-tests.
Strict InvarianceScalar Strict RS9 Free RS13 Free
CFI .902 .888 .891 .894
RMSEA .094 .097 .096 .095
SRMR .060 .062 .062 .062
X2 (180) = 585.327
(194) = 660.398
(193) = 646.559
(192) = 633.111
DIFFERENT? NOPECFI = .007
YESCFI = .014
YESCFI = .011
NOCFI = .008
NOTE: compare to scalar not strict.
Latent Means
• You can if latent means are different across groups using the same procedures with group.equal = c(“means”)– However, let’s say you want to use the weighted
means as a predictor in a later analysis – Or we can calculate a t.test to determine if they
are different and get effect sizes.
Latent Means
• Think about this for a second:– We normally use EFA/CFA to show that each
question has a nice loading and the questions “go together”
– And then we totally ignore the fact that the loadings are different and just create total scores or average scores.
– Why lose that information?
Latent Means
• You will calculate the latent score for each person by multiplying their individual item score times the loading– Then you can average them or total them
depending on how the scale is traditionally scored
Latent Means
• You will take the estimates for the loadings for each factor– So, if you have two factors you will need the
weights for each one but separately, aka don’t just average them all together.
– Make sure you are multiplying the right things!
Latent Means
• First, save the loadings:– loadings = parameterestimates(partial.fit)– Now view the loadings – it’s very similar to saving
the modinces, which means we can subset them.
Latent Means
• Subset out the loadings by group and type.• We want to use =~ for regression weights. • The select part gets you only the estimates.– menload = subset(loadings, group == "1" & op ==
"=~", select = "est")– womenload = subset(loadings, group == "2" & op
== "=~", select = "est")
Latent Means
• Now, we have the loadings and want to multiply them by the original scores.
• Be sure you’ve separated the data into groups!
Latent Means
• Apply function:– Arguments (dataset, 1 for rows / 2 for columns,
function)– Make sure the columns in the same order!– menmultiply = apply(men[, 4:ncol(men)], 1,
function(x) { x * menload })– womenmultiply = apply(women[, 4:ncol(women)],
1, function(x) { x * womenload })
Latent Means
• Apply is great! But it saves as a list. Lame.• So change that over to a data frame.– menmultiply = as.data.frame(menmultiply)– menmultiply = t(menmultiply) – ##flip it so it's people per row rather than
question by row
Latent Means
• To get the mean for each person:– menlatent = rowMeans(menmultiply)– womenlatent = rowMeans(womenmultiply)
• If you want to get the sum:– menlatent = rowSums(menmultiply)– womenlatent = rowSums(womenmultiply)
Latent Means
• Let’s do a t.test to see if they are different– t.test(group 1, group 2, …)• The rest of the arguments are not required but wanted
to show them to you if you wanted to change them to one tailed.
– t.test(menlatent, womenlatent, alternative = "two.sided", paired=FALSE, na.action=TRUE)
Latent Means
• Effect sizes!• Install the effsize package– library(effsize)– cohen.d(menlatent, womenlatent)– cohen.d(group 1, group 2, paired=FALSE)