Research Methodology: Tools - schwarz & partners 02_EN_2014 Item... · Case Processing Summary 7.3%...

Research Methodology: Tools

Applied Data Analysis (with SPSS)

Lecture 02: Item Analysis / Scale Analysis / Factor Analysis

February 2014

Prof. Dr. Jürg Schwarz

Lic. phil. Heidi Bruderer Enzler

MSc Business Administration

Slide 2

Contents

Aims of the Lecture ______________________________________________________________________________________ 3

Typical Syntax ___________________________________________________________________________________________ 4

Introduction _____________________________________________________________________________________________ 5

An Example of a Construct ..................................................................................................................................................................................... 5

What are Constructs? ............................................................................................................................................................................................. 7

Evaluating the Quality of an Instrument .................................................................................................................................................................. 8

Part I: Item Analysis and Scale Analysis _____________________________________________________________________ 9

Descriptive Analysis ............................................................................................................................................................................................... 9

Item Difficulty (Also Called p-Value) ..................................................................................................................................................................... 11

Item Discrimination ............................................................................................................................................................................................... 13

Reliability Analysis ................................................................................................................................................................................................ 15

Part I: Item Analysis and Scale Analysis with SPSS ___________________________________________________________ 16

Part II: Factor Analysis ___________________________________________________________________________________ 19

Main Steps for Performing a Factor Analysis ........................................................................................................................................................ 19

Problematic Aspects ............................................................................................................................................................................................. 20

Part II: Factor Analysis with SPSS _________________________________________________________________________ 21

First Step: Select the Variables ............................................................................................................................................................................ 21

Third Step: Determining the Number of Factors .................................................................................................................................................... 32

Fifth Step (Optional): Calculate Sum Scales or Factor Scores .............................................................................................................................. 41

References ........................................................................................................................................................................................................... 42

Appendix ______________________________________________________________________________________________ 43

Aims of the Lecture

You will understand the term "construct".

You will understand item difficulty and how this can be calculated with SPSS.

You will understand item discrimination and how this can be calculated with SPSS.

You will understand Cronbach's Alpha as a measure for reliability, and how it can be calculated with SPSS.

You will understand the concept of factor analysis and know how to perform it with SPSS (Principal Component Analysis).

Specifically, you will understand how C

◦ a correlation matrix is interpreted.

◦ determine the "correct" number of factors (Scree Plot, Kaiser criterion).

◦ Varimax rotation is used to be better able to interpret a factor solution.

◦ to interpret factors in relationship to their meaning.

◦ how factor scores and sum scales are calculated.

Slide 4

Items (list incomplete)

Extractions method

Rotation

Saving factor scores

Items (list incomplete)

Cronbachs Alpha

Typical Syntax

Reliability analysis RELIABILITY /VARIABLES=ghqconc ghqsleep ghquse ghqdecis … /SCALE('ALL VARIABLES') ALL /MODEL=ALPHA /STATISTICS=DESCRIPTIVE /SUMMARY=TOTAL.

Factor analysis FACTOR /VARIABLES ghqconc ghqconfi ghqdecis ghqenjoy … /MISSING LISTWISE /ANALYSIS ghqconc ghqconfi ghqdecis ghqenjoy … /PRINT INITIAL CORRELATION SIG KMO INV EXTRACTION ROTATION /PLOT EIGEN ROTATION /CRITERIA MINEIGEN(1) ITERATE(25) /EXTRACTION PC /CRITERIA ITERATE(25) /ROTATION VARIMAX /SAVE REG(ALL) /METHOD=CORRELATION.

Introduction

An Example of a Construct

Measuring psychosocial well-being with the "General Health Questionnaire"?

In the 2003 Health Survey for England the "General Health Questionnaire" (GHQ-12) was used

to measure "psychosocial well-being". The GHQ-12 is a battery of 12 items.

An extract from the questionnaire:

Question I:

Are these 12 items collectively suitable

for measuring a construct?

=> Perform an item analysis and

a scale analysis

Slide 6

Underlying dimensions of the General Health Questionnaire?

There could be groups of items on the GHQ-12 that measure different aspects (dimension, fac-

tor) of "psychosocial well-being"

Question II: Does the GHQ-12 have a structure?

Are there dimensions that must be considered as separate?

=> Perform a factor analysis

Positive health

Psychological distress

ghqdecis Felt capable of making decisions

ghquse Felt playing useful part in things

ghqsleep Lost sleep over worry

ghqover Felt couldn’t overcome difficulties

ghqworth Been thinking of self as worthless

ghqface Been able to face problems

ghqenjoy Able to enjoy day-to-day activities

ghqconc Able to concentrate

ghqunhap Been feeling unhappy and depressed

ghqhappy Been feeling reasonably happy

ghqconfi Been losing confidence in self

ghqstrai Felt constantly under strain

Dimensions (factors) of "psychosocial well-being"

Construct "psychosocial

well-being"

What are Constructs?

◦ A construct is an abstract idea that cannot be directly observed and measured.

Examples: Psychosocial well-being, motivation, anxiety, employee satisfaction

◦ Complex constructs often contain various aspects (dimensions, factors).

Examples: Psychosocial well-being = positive health + psychological distress

Stress reactivity = Play-dead reflex + heart phobia + negativism + disturbance

Intelligence = Verbal intelligence + mathematical-logical intelligence + C

◦ In order to measure constructs, indicators that are themselves measurable must be found.

Example: Social status = Income + years of education + job category

◦ In order to measure constructs, multiple items with rating scales are used often. It is assumed that these items are an indicator for the construct. Technically, a construct is therefore often regarded as an item battery. In SPSS, articles and this script, an item battery is also designated as a scale.

Example: General Health Questionnaire (GHQ-12)

◦ There are different possibilities for compressing these items into a score.

As a sum scale: The item values of an item battery are summed up.

An example is the Apgar Score (determining the vital signs of a newborn).

Factor ↔ technical term

Dimension ↔ theoretical term

Item ↔ in the questionnaire

Indicator ↔ theoretical term

Slide 8

Evaluating the Quality of an Instrument

How can the quality of an instrument be assessed?

Common sense: Content considerations

Item analysis und scale analysis

◦ Descriptive analysis

◦ Item difficulty

◦ Item discrimination

◦ Reliability analysis

Examination of dimensionality

◦ Factor analysis

When do we want to assess the quality of an instrument for which ends?

◦ Pretests: Finding out where and how a questionnaire can be improved.

◦ Existing datasets: Choosing appropriate items for further analysis.

"Part I" of the lecture

"Part II" of the lecture

Part I: Item Analysis and Scale Analysis

Descriptive Analysis

Example GHQ-12 in the Health Survey for England 2003 (n = 8,833)

◦ Missing values: Around 6% missing values appears unproblematic. In addition, all items have a similar proportion of missing values. ghquse ("felt playing useful part in things") may be somewhat vague, and so less frequently answered.

◦ Mean / Median / Skewness / Kurtosis: Difference between mean and median → distribu-tions are not symmetric. All distributions are more or less right skewed (skewness > 0) and more peaked than a normal distribution (kurtosis > 0).

◦ Standard deviation: Variability is assessed with the help of the histograms (see below).

◦ Minimum / Maximum: Values between 1 and 4 → The entire range of the scaled was used.

Slide 10

Example GHQ-12: Histograms

ghqconc ghqsleep ghquse ghqdecis ghqstrai ghqover

ghqenjoy ghqface ghqunhap ghqconfi ghqworth ghqhappy

Variables ghqconc, ghquse, ghqdecis, ghqenjoy, ghqface and ghqhappy:

Items display little variability, could be problematic.

The most frequently chosen category 2 represents "same as always".

Variables qhqsleep, ghqstrai, ghqover, qhqunhap, ghqconfi and ghqworth:

These items exhibit more variance, but are strongly skewed to the right.

Floor effects, especially with qhqunhap, ghqconfi and ghqworth.

The most commonly chosen value of 1 represents an especially good condition of health.

Item Difficulty (Also Called p-Value)

The difficulty of an item indicates the proportion of respondents who respond to the item in a

manner that indicates the characteristic in question is present to a greater extent.

Computing item difficulty

◦ Continuous items (Rating scales): Item difficulty = arithmetic mean of the item

◦ Binary items (for example, Yes/No): Item difficulty is the proportion of respondents who an-swered a question affirmatively or correctly, when the coding is: affirmative or correct = 1, negative or false = 0

= =Item i

Item i

Item i

Number of correct responsesItem difficulty p

Number of valid responses

Interpretation of item difficulty

◦ High value → Item is "easy"

◦ Low value → Item is "difficult"

As a rule, a good mix in an item battery is desirable.

Items with extreme values should be revised.

With binary items the p-value lies between 0 and 1 (.5 = middle difficulty): Items with p < .1 (very

difficult) or p > .9 (very easy) should be revised.

Slide 12

Example item difficulty with binary items: A fictitious questionnaire about taxes

For example item 5: "In your opinion, should taxes be lowered? Yes or No?"

Item 1 is too "easy", no distinction is visible.

Item 2 appears to be too "difficult" or is not understood.

→ Both items should be eliminated.

Item 3 has an average difficulty. Item 4 is relatively difficult, item 5 relatively simple.

An example of items with a rating scale follows below.

1 2 3 4 5

1 1 n 1 1 1

2 1 n 0 0 1

3 1 0 1 0 1

4 1 0 0 0 1

5 1 0 1 0 0

6 1 0 0 0 n

7 1 0 1 1 n

ny 7 0 4 2 4

na 7 5 7 7 5

nt 7 7 7 7 7

p 1.00 0.00 0.57 0.29 0.80

Item

Respondent

Legend

0 = no, 1 = yes, n = no answer

ny = number of those who answered "Yes"

na = number of those who answered

nt = total number of those asked

p = item difficulty = ny/na

Item Discrimination

Item discrimination is the correlation between an item and the item battery without this item.

Item discrimination is computed for every item.

◦ The discrimination value of an item indicates how well this single item predicts the value of the item battery.

◦ Values range from -1 to 1.

◦ The higher the value, the better the item measures what the item battery measures. Positive values near 1 are desirable.

◦ If the value is negative, this may be due to the rotation of the item (direction/polarity). If the rotation is correct, however, the item should be discarded or revised.

Rule of thumb: Items with a discrimination value under .30 are discarded or revised.

Changing item rotation in SPSS

Example: item v01 with for possible values 1, 2, 3, and 4.

RECODE v01 (1 = 4)(2 = 3)(3 = 2)(4 = 1) INTO v01_r.

FREQUENCIES v01 v01_r.

Slide 14

Example GHQ-12

Analyze�Scale�Reliability AnalysisC �Statistics: � "Scale if item deleted"

Column "Corrected Item-Total Correlation":

Item discrimination ranges from .487 to .738.

Thus all values are above the threshold of .30.

This means that each of the items reflects sufficiently well what the scale as a whole measures.

Reliability Analysis

◦ Reliability analysis is used to quantify how well all items in a battery collectively measure the same theoretical construct, for example, psychosocial well-being.

◦ A high average correlation between the items indicates that they all measure the same con-struct.

◦ Usually, reliability is measured through the Cronbach's alpha coefficient. Cronbach's alpha is a measurement of the "internal consistency" of a scale. Cronbach's alpha is a positive function of the average correlation among items in the battery, and is positively correlated with the number of items and the size of the sample.

Preconditions for calculating Cronbach's alpha

◦ All items measure the same theoretical construct

◦ Metric variables

◦ All items are scaled in the same direction (for example, low values for the items represent high psychosocial well-being)

◦ All items have the same distribution (ideally: normal distribution)

Evaluation of Cronbach's alpha

Alpha should be > .80. In practice, however, values of .60 or .70 are acceptable.

Slide 16

Part I: Item Analysis and Scale Analysis with SPSS General Health Questionnaire (GHQ-12) in the Health Survey for England 2003 (n = 8,833)

Analyze�Scale�Reliability AnalysisC

Results

Case Processing Summary

7.3% of the cases are excluded because of missing values. → OK

Reliability Statistics

The reliability of the scale is high (Cronbach's alpha = .891).

→ OK

Item Statistics

Since the item difficulty (p-value) is the arithmetic mean, it is

found in the "Mean" column.

Scale from 1 to 4 → average difficulty of 2.5. However, in this

case the wording suggests that 2 is a type of middle category

("same as usual").

The item difficulties vary between 1.40 and 2.14. That means

that the mixture is not very high; some items are very "difficult"

(low p-values).

Slide 18

Item-Total Statistics

Since item discrimination is the correlation be-

tween a single item and the total score without this

item, it appears in the column "Corrected Item-

Total Correlation".

Item discrimination varies between .487 and .738

and therefore lies above .30 → OK

In the column "Cronbach's Alpha if Item Deleted", we see for each item how high Cronbach's

alpha would be if this item were omitted. (Including all items: Alpha = .891)

In the example, Cronbach's alpha would not be higher if any of items were omitted.

This indicates that all items should remain. → OK

Summary Question I

Each of the items represents the entire battery.

However, the total battery displays little variation.

From the standpoint of reliability, the GHQ-12 items can be used as a scale.

Part II: Factor Analysis

Main Steps for Performing a Factor Analysis

1. Select the variables

◦ Include only theoretically relevant variables

◦ Sufficient number of variables (4 or more per factor)

◦ Not too small a sample

◦ Perform descriptive analysis (as in Part I: item analysis and reliability analysis)

◦ Consider the correlation matrix

2. Test suitability and extract the factors

◦ Determination of suitability: Evaluate inverse of correlation matrix, Bartlett's test and KMO

◦ Select extraction method (principal component analysis, principal axis analysis)

3. Specify the number of factors

◦ Criteria: Eigenvalue, scree plot, rules of thumb

4. Interpret the factors

◦ Rotation of the factor matrix, mapping variables to factors, interpretation

5. Optional: Compute sum scales or factor scores

◦ Computing sum scales or factor scores

Slide 20

Problematic Aspects

Many decisions regarding extraction and interpretation of factors are subjective.

The same dataset can produce different results, depending on the "decision path".

Although variables must be at least interval-scaled, in practice, variables with lower scale levels

are often included, which can lead to false conclusions.

Sample size

There is no scientifically exact rule how large the sample should be. One of the possible rules of

thumb suggests that there should be at least 10 subjects per item ("Rule of 10").

Problematic missing values

In item batteries, there are often many missing values.

Results are different, depending on "missing treatment" in relation to

◦ Number of factors

◦ Interpretation of factors

There is no single solution for handling missing data.

Depending on the available data and context,

another approach may be needed.

Part II: Factor Analysis with SPSS Item battery "General Health Questionnaire" (GHQ-12)

in the Health Survey for England 2003

First Step: Select the Variables

◦ Theoretically relevant variables Scale GHQ-12: All plausible items → OK

◦ Sufficient number of variables Assumption: 2 factors with 6 variables each ◦ Psychological distress ◦ Positive health

→ OK

◦ Not too small a sample Very large sample (n = 8,833) → OK

◦ Descriptive analysis See histograms (see Part I). In an ideal situation, normally distributed variables.

◦ Correlation matrix See below

Slide 22

How can dimensions (factors) be discovered?

◦ A factor analysis deals with how different items relate to each other, and how they can be grouped into factors.

◦ The goal is to group items together as a factor and to replace them with general terms. The general term reflects the underlying content.

◦ Each factor represents several items. It is more efficient to represent something through fewer factors than through many single items.

◦ Attention: Theoretical and empirical facts are incorporated into a factor analysis.

Basic idea of factor analysis

Assumption: Some variables tend to be related.

Example GHQ-12:

ghqstrai ("felt constantly under strain")

ghqconfi ("been losing confidence in self")

Three possible causes for the ghqstrai – ghqconfi correlation:

◦ Variable ghqstrai influences variable ghqconfi.

◦ Variable ghqconfi influences variable ghqstrai.

◦ Both variables are influenced by a factor.

From slide 6:

Correlations matrix of the variables

Analyze�Correlate�Bivariate C Correlations

b

ghqsleep ghqstrai ghqover ghqworth ghqunhap ghqconfi ghqhappy ghqface ghqconc ghquse ghqdecis ghqenjoy

ghqsleep Pearson correlation coefficient 1 .545

** .483

** .387

** .547

** .459

** .343

** .312

** .330

** .228

** .226

** .297

**

Significance (2-sided) .000 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000

ghqstrai Pearson correlation coefficient .545

** 1 .598

** .418

** .592

** .492

** .376

** .365

** .352

** .217

** .268

** .351

**


ghqover Pearson correlation coefficient .483

** .598

** 1 .517

** .593

** .582

** .397

** .443

** .369

** .344

** .345

** .379

**


ghqworth Pearson correlation coefficient .387

** .418

** .517

** 1 .574

** .689

** .449

** .395

** .319

** .362

** .330

** .323

**


ghqunhap Pearson correlation coefficient .547

** .592

** .593

** .574

** 1 .671

** .502

** .428

** .362

** .321

** .308

** .377

**


ghqconfi Pearson correlation coefficient .459

** .492

** .582

** .689

** .671

** 1 .460

** .427

** .379

** .368

** .355

** .363

**


ghqhappy Pearson correlation coefficient .343

** .376

** .397

** .449

** .502

** .460

** 1 .487

** .370

** .368

** .380

** .416

**


ghqface Pearson correlation coefficient .312

** .365

** .443

** .395

** .428

** .427

** .487

** 1 .417

** .389

** .485

** .469

**


ghqconc Pearson correlation coefficient .330

** .352

** .369

** .319

** .362

** .379

** .370

** .417** 1 .389

** .438

** .440

**


ghquse Pearson correlation coefficient .228

** .217

** .344

** .362

** .321

** .368

** .368

** .389

** .389** 1 .488

** .439

**


ghqdecis Pearson correlation coefficient .226

** .268

** .345

** .330

** .308

** .355

** .380

** .485

** .438** .488

** 1 .367

**


ghqenjoy Pearson correlation coefficient .297

** .351

** .379

** .323

** .377

** .363

** .416

** .469

** .440** .439

** .367

** 1


**. The correlation is significant at the 0.01 level (2-sided). b. Listwise N=8188

There could be two factors (blue, green), where ghqhappy and ghqface cannot be clearly as-

signed. But: We cannot decide on the number of factors on the basis of a correlation matrix.

=> Perform a factor analysis!

Attention: For teaching purposes,

the variables are grouped together

so as to better observe the struc-

ture. This is not normally the case!

Slide 24

What is important about evaluating the correlation matrix?

Significance levels of the correlations

◦ The significance level shows whether the correlation coefficient is merely different from zero by chance, or whether there is a high probability that it is truly different from zero.

◦ The significance level should be chosen at the beginning of the study (1% or 5%). This depends on the sample size and the goals of the analysis. On rare occasions, especially with large sample sizes, the significance level 0.1% is chosen.

Values of the correlation coefficients

◦ A factor analysis is problematic when there are many low, yet no high correlation coefficients present. In this case, the data structure is too heterogeneous.

◦ Best would be if there are clusters of highly correlated variables that are separated. These clusters are indication of an underlying factor structure.

Second Step: Test of Suitability and Extraction of Factors

Analyze��Dimension Reduction��Factor F

Optional

Slide 26

Inverse of correlation matrix: Evaluating suitability of data I

The correlation structure is suitable for a factor analysis if its inverse approaches a diagonal ma-

trix. Essentially, it acts as a visual aid.

Evaluation:

◦ There are no universal rules!

◦ A matrix is diagonal if the non-diagonal values are as close as possible to zero.

That means that the non-diagonal values should be much smaller than those on the diagonal.

Example GHQ-12

The non-diagonal values are significantly smaller than the values on the diagonal.

→ The correlation structure is well-suited for a factor analysis.

Diagonal

Bartlett's test: Evaluating suitability of data II

Null hypothesis H0:

The sample is drawn from a population

in which all variables are completely uncorrelated.

Assumption: The data are normally distributed.

Example GHQ-12

In the case of the GHQ-data, Bartlett's test is significant (Sig. = .000) and correspondingly the

null hypothesis can be rejected. The variables are not completely uncorrelated.

So the factor analysis can be continued.

Attention: The statement "The variables are correlated." is false.

The alternative hypothesis cannot be postulated.

Slide 28

Kaiser-Meyer-Olkin (KMO): Evaluating suitability of data III

Kaiser, Meyer and Olkin developed a "Measure of Sampling Adequacy" (MSA),

which is the standard test for assessing whether the data are suitable for factor analysis.

MSA values refer to single variables. The KMO index is a generalization for the entire dataset.

The KMO index shows the extent to which the variables belong together and so helps to deter-

mine whether or not a factor analysis is appropriate. It tests whether the partial correlations be-

tween the variables are small. If these are small, then the KMO is high.

Rule of thumb: The KMO should be .60 or higher in order to continue with the factor analysis.

Kaiser (1970) suggests a lower limit of .50, though a value of .80 or higher would be desirable.

Example GHQ-12

KMO value

.00 to .49 unacceptable

.50 bis .59 miserable

.60 bis .69 mediocre

.70 bis .79 middling

.80 bis .89 meritorious

.90 bis 1.00 marvellous

Example GHQ-12: Comparing the extraction results

Factor loadings

The factor loading of a variable is the correlation between the variable and the factor.

Typical statement: "The variable ghqconc loads with .619 on factor 1."

Values between -1 and +1 are theoretically possible. The magnitude of the factor loading shows

how closely a variable correlates with a factor: Magnitudes close to 0 indicate that a relationship

barely exists. The higher the magnitude, the closer the relationship.

Factor loading

Slide 30

Graphical interpretation of the factor loadings

Each variable can be described as a vector in a coordinate system.

The vector is formed by the factor loadings of the variable.

The factor loadings can be interpreted as coordinates.

-1.0

-0.8

-0.6

-0.4

-0.2

0.0

0.2

0.4

0.6

0.8

1.0

-1.0 -0.8 -0.6 -0.4 -0.2 0.0 0.2 0.4 0.6 0.8 1.0

Factor 2

Factor 1

ghqdecis

ghquseghqenjoy

ghqconc ghqface

ghqhappy

ghqworthghqconfi

ghqoverghqunhap

ghqstraighqsleep

.679

.118

Communality

The variables cannot usually be completely explained by the factors.

Communality is the total amount of a variable's variance that can be explained by all factors.

Communality indicates the extent to which a variable is explained through the factors.

Example GHQ-12

Example variable ghqconq:

Communality after extraction .481

→ 48.1% of the variance of ghqconq is

explained through factors 1 and 2.

Association with Component Matrix

0.481 = .6192 + .314

2

48.1% = 38.3% + 9.8%

Slide 32

Third Step: Determining the Number of Factors

There is no unambiguous method for determining the number of factors.

Common sense: Limit the number of factors to those which are understood.

The number of factors is usually significantly less than the number of variables.

◦ Kaiser Criterion The Kaiser criterion removes all components whose eigenvalue is under 1.0. It is the default in SPSS and most other statistical programs. The Kaiser criterion, however, is not recom-mended as the only basis for a decision.

◦ Scree Plot The scree plot shows the components along the x-axis and the corresponding eigenvalues along the y-axis. All components past the elbow are omitted.

◦ Explained variance as the criterion Some researchers apply the rule that there be a sufficient number of factors to describe 90% (sometimes 80%) of the variance.

Kaiser criterion (Eigenvalues > 1.0)

◦ An eigenvalue shows how much of the total variance is explained by the factor.

◦ The Kaiser criterion requires that all components with an eigenvalue < 1.0 be rejected.

◦ The Kaiser criterion is not recommended as the only basis for a decision, since it commonly leads to choosing too many factors.

Example GHQ-12

2 factors at most ↔ Kaiser criterion (Eigenvalue > 1.0)

The eigenvalue corresponds to the proportion of the total vari-ance that is explained by the factor. The variables are z-transformed for this purpose (standard de-viation 1 and mean 0). Thus the total variance of the GHQ-12 to be explained (12 variables) = 12. The first factor explains 5.588 of this amount, and so 5.588/12 (46.568%) of the variance. The second factor explains 1.330/12 (11.082%) of the variance.

Slide 34

Scree Plot: Example GHQ-12

Scree plot: Example with random data

htt

p://e

n.w

ikip

edia

.org

/wik

i/S

cre

e

No elbow

Scree

Elbow criterion:

◦ If the factors arise randomly, then the slope is flat.

◦ Only factors above the elbow are counted.

elbow flat slope

The elbow occurs at 3 → 2 factors

Typical statement:

I have decided on two factors, because this is theoretically

plausible and consistent with the Kaiser criterion (eigen-

value > 1) as well as the scree plot.

Fourth Step: Interpretation of the Factors

Rotation

Rotation leads to a better readability of the results.

Varimax rotation preserves the independence of the factors.

The factors are rotated, so that the variance of the squared loadings

per factor is maximized. Mid-level loadings tend to become "smaller"

or "larger", so that the factor structure is easier to interpret.

Symbolic representation:

Before rotation: After rotation:

Factor 2

Factor 1 Factor 1 Factor 2

Slide 36

Example GHQ-12: Rotation and loading plots in SPSS

Loading plots essentially constitute a visualization.

Before rotation: After rotation:

Evaluating the loadings and assigning variables to factors

In principle, each variable is assigned to the factor on which it loads highest.

How high should a factor loading minimally be in order to be interpreted?

There are different possible rules of thumb:

◦ Factor loadings lower than .20 should not be considered. If an item does not load any higher on any of the factors: Discard the item and redo the analysis.

◦ Factor loadings of ±.30 to ±.40 are minimally acceptable but higher values are desirable (particularly if the sample is small and only few variables are considered)

◦ Irrespective of sample size: A factor can be interpreted ifC

◦ at least 4 variables show loadings of ≥ .60.

◦ at least 10 variables have a loading of ≥ .40.

◦ If n < 300, factors with low loadings only should not be interpreted.

Example GHQ-12

The factors show 6 and 5 variables with loadings ≥ .6, respectively.

Slide 38

Cross-loadings

A cross-loading occurs if a variable has high loadings on more

than one factor.

(two or more factor loadings >.3 or >.4)

The item is related with more than one factor.

It correlates with other items that load on the affected factors.

If the goal is to find sharply-defined factors, and from them, for

example, form sum scores, the difference between loadings is

considered.

◦ Large difference (> .2): The variable can be assigned to the factor with the highest loading.

◦ Small difference (< .2): The variable cannot be assigned to a factor. → Exclude and perform the analysis again. (Unless theoretical considerations contradict this.)

If the goal is to demonstrate commonality of concepts, cross-loadings are of theoretical interest

and are retained. They must be theoretically plausible.

Content interpretation of the factors I

The items are sorted within the factor according to

the decreasing magnitude of the loading.

SPSS can facilitate doing so.

Not sorted: Sorted by size:

Slide 40

Content interpretation of the factors II

The sign of the loading (or the loading itself) is noted.

What is the "theme" of the factors? What do the factors stand for?

◦ Inspect item text. The highest-loading variables are especially helpful (so-called "marker variables").

◦ The context of the study can also provide information.

Example GHQ-12

Factor 1 Factor 2 Possible general term:

ghqunhap Been feeling unhappy and depressed .810 .261

"psychological distress"

ghqstrai Felt constantly under strain .769 .164

ghqconfi Been losing confidence in self .747 .328

ghqover Felt couldn’t overcome difficulties .730 .304

ghqsleep Lost sleep over worry .728 .124

ghqworth Been thinking of self as worthless .671 .320

ghqdecis Felt capable of making decisions .128 .764

"positive health"

ghquse Felt playing useful part in things .133 .743

ghqface Been able to face problems .331 .665

ghqenjoy Able to enjoy day-to-day activities .259 .663

ghqconc Able to concentrate .267 .640

ghqhappy Been feeling reasonably happy .440 .530

Fifth Step (Optional): Calculate Sum Scales or Factor Scores

To use factors in further analyses, we need to generate one variable per factor to represent it.

Usually, sum scales are calculated on the basis of the factor solution.

Alternatively, factor scores can be calculated.

Calculating sum scales in SPSS

Transform�Compute variablesC

Test each factor for reliability beforehand! (Cronbach's alpha)

Calculating factor scores in SPSS

It is implemented as part of SPSS factor analysis (Option "Scores").

The "regression method" is most often used.

This results in one variable per factor, e.g. FAC1_1, FAC1_2.

Slide 42

References

Hankins, M. (2008)

The factor structure of the twelve item General Health Questionnaire (GHQ-12): the result of

negative phrasing? Clinical Practice and Epidemiology in Mental Health, 4:10,

www.cpementalhealth.com/content/4/1/10 (Date of access: February 2014)

Jackson, C. (2007)

The General Health Questionnaire. Occupational Medicine, 57:79,

http://occmed.oxfordjournals.org/content/57/1/79.full (Date of access: February 2014)

Kaiser, H. (1970).

A second generation little jiffy. Psychometrika, 35(4), 401-415.

www.ats.ucla.edu/stat/spss/faq/alpha.html (Date of access: February 2014)

Appendix Types of Factorizations: Principal Components Analysis vs. Principal Axis Factoring

Principal Component Analysis (PCA, default in SPSS)

◦ Goal: Reproduce the data structure

◦ No causal relationship between factors and variables

◦ Factors are "general terms" and are often called components.

◦ Process: The first factor is chosen so that it describes the greatest possible proportion of var-iance in the variables. Each additional factor describes a maximum amount of the remaining variance. Factors are extracted until the total variance among the variables is explained.

◦ If variables are added, the factor loadings change.

Principal Axis Factoring (PAF)

◦ Goal: Determine the cause of the correlation structure

◦ Causal interpretation: Factors cause the correlations among variables.

◦ Process: The first factor is chosen so that it explains the greatest possible portion of the common variance of the variables. Each additional factor explains a maximum portion of the remaining common variance of the variables. Factors are extracted until the factors can ex-plain all the common variance in a set of variables.

◦ In principle it is possible to add variables without affecting the factor loadings.

Slide 44

Notes:

Research Methodology: Tools - schwarz & partners 02_EN_2014 Item... · Case Processing Summary 7.3%...

Documents

Transcript of Research Methodology: Tools - schwarz & partners 02_EN_2014 Item... · Case Processing Summary 7.3%...