Factor Analysis.ppt
Transcript of Factor Analysis.ppt
-
7/24/2019 Factor Analysis.ppt
1/28
2007 Prentice Hall 19-1
Factor Analysis
2007 Prentice Hall
Advanced Statistical Modeling
Adnan ButtAssistant Professor!ra "niveristy# $arac%i
-
7/24/2019 Factor Analysis.ppt
2/28
2007 Prentice Hall 19-2
Factor Analysis Factor analysisis a general na&e denoting a class of
'rocedures 'ri&arily used for data reduction andsu&&ari(ation)
Factor analysis is an interdependence techniquein t%at anentire set of interde'endent relations%i's is e*a&ined +it%out&a,ing t%e distinction et+een de'endent and inde'endent
variales) Factor analysis is used in t%e follo+ing circu&stances.
/o identify underlying di&ensions# or factors# t%at e*'laint%e correlations a&ong a set of variales)
/o identify a ne+# s&aller# set of uncorrelated variales tore'lace t%e original set of correlated variales in suse!uent&ultivariate analysis regression or discri&inant analysis)
/o identify a s&aller set of salient variales fro& a larger setfor use in suse!uent &ultivariate analysis)
-
7/24/2019 Factor Analysis.ppt
3/28
2007 Prentice Hall 19-
Factors UnderlyingSelected Psychographics
and Lifestyles
Factor 2
Footall Baseall
3vening at %o&e
Factor 1
Ho&e is est 'lace4o to a 'arty
Plays Movies
-
7/24/2019 Factor Analysis.ppt
4/28
2007 Prentice Hall 19-5
Statistics Associated with FactorAnalysis
Bartlett's test of sphericity)Bartlett6s test of s'%ericityis a test statistic used to e*a&ine t%e %y'ot%esis t%at t%evariales are uncorrelated in t%e 'o'ulation) n ot%er
+ords# t%e 'o'ulation correlation &atri* is an identity
&atri* eac% variale correlates 'erfectly +it% itself r8 1ut %as no correlation +it% t%e ot%er variales r8 0)
Correlation matrix)A correlation &atri* is a lo+ertriangle &atri* s%o+ing t%e si&'le correlations#r#
et+een all 'ossile 'airs of variales included in t%eanalysis) /%e diagonal ele&ents# +%ic% are all 1# are
usually o&itted)
-
7/24/2019 Factor Analysis.ppt
5/28
2007 Prentice Hall 19-
Communality):o&&unality is t%e a&ount of variance
a variale s%ares +it% all t%e ot%er variales eingconsidered) /%is is also t%e 'ro'ortion of variancee*'lained y t%e co&&on factors)
igen!alue)/%e eigenvalue re'resents t%e total
variance e*'lained y eac% factor)
Factor loadings)Factor loadings are si&'lecorrelations et+een t%e variales and t%e factors)
Factor loading plot)A factor loading 'lot is a 'lot of
t%e original variales using t%e factor loadings ascoordinates)
Factor matrix) A factor &atri* contains t%e factorloadings of all t%e variales on all t%e factors e*tracted)
Statistics Associated with FactorAnalysis
-
7/24/2019 Factor Analysis.ppt
6/28
2007 Prentice Hall 19-;
Factor scores) Factor scores are co&'osite scoresesti&ated for eac% res'ondent on t%e derived factors)
"aiser#$eyer#%l&in "$%( measure of samplingadequacy)/%e $aiser-Meyer-erences et+een t%e oserved
correlations# as given in t%e in'ut correlation &atri*# and t%ere'roduced correlations# as esti&ated fro& t%e factor &atri*)
Scree plot)A scree 'lot is a 'lot of t%e 3igenvalues againstt%e nu&er of factors in order of e*traction)
Statistics Associated with FactorAnalysis
-
7/24/2019 Factor Analysis.ppt
7/28
2007 Prentice Hall 19-7
Conducting Factor Analysis)SP%*+*,
*U$B) -. -/ -0 -1 -2 -3
. 4566 0566 3566 1566 /566 1566
/ .566 0566 /566 1566 2566 1566
0 3566 /566 4566 1566 .566 0566
1 1566 2566 1566 3566 /566 2566
2 .566 /566 /566 0566 3566 /566
3 3566 0566 3566 1566 /566 1566
4 2566 0566 3566 0566 1566 0566
7 3566 1566 4566 1566 .566 1566
8 0566 1566 /566 0566 3566 0566
.6 /566 3566 /566 3566 4566 3566
.. 3566 1566 4566 0566 /566 0566
./ /566 0566 .566 1566 2566 1566
.0 4566 /566 3566 1566 .566 0566
.1 1566 3566 1566 2566 0566 3566
.2 .566 0566 /566 /566 3566 1566
.3 3566 1566 3566 0566 0566 1566
.4 2566 0566 3566 0566 0566 1566
.7 4566 0566 4566 1566 .566 1566
.8 /566 1566 0566 0566 3566 0566
/6 0566 2566 0566 3566 1566 3566
/. .566 0566 /566 0566 2566 0566
// 2566 1566 2566 1566 /566 1566
/0 /566 /566 .566 2566 1566 1566
/1 1566 3566 1566 3566 1566 4566
/2 3566 2566 1566 /566 .566 1566
/3 0566 2566 1566 3566 1566 4566
/4 1566 1566 4566 /566 /566 2566
/7 0566 4566 /566 3566 1566 0566
/8 1566 3566 0566 4566 /566 4566
06 /566 0566 /566 1566 4566 /566
-
7/24/2019 Factor Analysis.ppt
8/28
2007 Prentice Hall 19-?
Conducting Factor Analysis
:onstruction of t%e :orrelation Matri*
Met%od of Factor Analysis
@eter&ination of u&er of Factors
@eter&ination of Model Fit
Prole& for&ulation
:alculation ofFactor Scores
nter'retation of Factors
otation of Factors
Selection ofSurrogate =ariales
-
7/24/2019 Factor Analysis.ppt
9/28
2007 Prentice Hall 19-9
Conducting FactorAnalysisFormulate the Pro9lem /%e oCectives of factor analysis s%ould e identiDed)
/%e variales to e included in t%e factor analysis
s%ould e s'eciDed ased on 'ast researc%# t%eory#and Cudg&ent of t%e researc%er) t is i&'ortant t%att%e variales e a''ro'riately &easured on aninterval or ratio scale)
An a''ro'riate sa&'le si(e s%ould e used) As aroug% guideline# t%ere s%ould e at least four or Dveti&es as &any oservations sa&'le si(e as t%ereare variales)
-
7/24/2019 Factor Analysis.ppt
10/28
2007 Prentice Hall 19-10
Correlation$atrix
/ale 19)2
-
7/24/2019 Factor Analysis.ppt
11/28
2007 Prentice Hall 19-11
/%e analytical 'rocess is ased on a &atri* ofcorrelations et+een t%e variales)
Bartlett6s test of s'%ericity can e used to test t%e null%y'ot%esis t%at t%e variales are uncorrelated in t%e
'o'ulation. in ot%er +ords# t%e 'o'ulation correlation&atri* is an identity &atri*) f t%is %y'ot%esis cannot ereCected# t%en t%e a''ro'riateness of factor analysiss%ould e !uestioned)
Anot%er useful statistic is t%e $aiser-Meyer-
-
7/24/2019 Factor Analysis.ppt
12/28
2007 Prentice Hall 19-12
n principal components analysis#t%e total variance int%e data is considered) /%e diagonal of t%e correlation&atri* consists of unities# and full variance is roug%t intot%e factor &atri*) Princi'al co&'onents analysis isreco&&ended +%en t%e 'ri&ary concern is to deter&ine t%e&ini&u& nu&er of factors t%at +ill account for &a*i&u&
variance in t%e data for use in suse!uent &ultivariateanalysis) /%e factors are calledprincipal components)
ncommon factor analysis#t%e factors are esti&ated
ased only on t%e co&&on variance) :o&&unalities areinserted in t%e diagonal of t%e correlation &atri*) /%is&et%od is a''ro'riate +%en t%e 'ri&ary concern is toidentify t%e underlying di&ensions and t%e co&&onvariance is of interest) /%is &et%od is also ,no+n as
principal axis factoring)
Conducting Factor Analysis+etermine the $ethod of Factor
Analysis
-
7/24/2019 Factor Analysis.ppt
13/28
2007 Prentice Hall 19-1
)esults of PrincipalComponents Analysis
/ale 19)
-
7/24/2019 Factor Analysis.ppt
14/28
2007 Prentice Hall 19-15
)esults of PrincipalComponents Analysis
/ale 19)#cont)
-
7/24/2019 Factor Analysis.ppt
15/28
2007 Prentice Hall 19-1
)esults of PrincipalComponents Analysis
/ale 19)#cont)
-
7/24/2019 Factor Analysis.ppt
16/28
2007 Prentice Hall 19-1;
/%e lo+er-left triangle contains t%e re'roducedcorrelation &atri* t%e diagonal# t%e co&&unalitiest%e u''er-rig%t triangle# t%e residuals et+een t%e
oserved correlations and t%e re'roducedcorrelations)
)esults of PrincipalComponents Analysis
/ale 19)#cont)
-
7/24/2019 Factor Analysis.ppt
17/28
2007 Prentice Hall 19-17
A Priori +etermination5 So&eti&es# ecause of 'rior,no+ledge# t%e researc%er ,no+s %o+ &any factors toe*'ect and t%us can s'ecify t%e nu&er of factors to ee*tracted efore%and)
+etermination Based on igen!alues5 n t%isa''roac%# only factors +it% 3igenvalues greater t%an 1)0are retained) An 3igenvalue re'resents t%e a&ount ofvariance associated +it% t%e factor) Hence# only factors
+it% a variance greater t%an 1)0 are included) Factors +it%variance less t%an 1)0 are no etter t%an a single variale#since# due to standardi(ation# eac% variale %as a varianceof 1)0) f t%e nu&er of variales is less t%an 20# t%isa''roac% +ill result in a conservative nu&er of factors)
Conducting Factor Analysis+etermine the *um9er of
Factors
-
7/24/2019 Factor Analysis.ppt
18/28
2007 Prentice Hall 19-1?
+etermination Based on Scree Plot5 A scree 'lot isa 'lot of t%e 3igenvalues against t%e nu&er of factorsin order of e*traction) 3*'eri&ental evidence indicatest%at t%e 'oint at +%ic% t%e scree egins denotes t%etrue nu&er of factors) 4enerally# t%e nu&er offactors deter&ined y a scree 'lot +ill e one or a fe+&ore t%an t%at deter&ined y t%e 3igenvalue criterion)
+etermination Based on Percentage of -ariance5n t%is a''roac% t%e nu&er of factors e*tracted isdeter&ined so t%at t%e cu&ulative 'ercentage ofvariance e*tracted y t%e factors reac%es a satisfactorylevel) t is reco&&ended t%at t%e factors e*tracteds%ould account for at least ;0 of t%e variance)
Conducting Factor Analysis+etermine the *um9er of
Factors
-
7/24/2019 Factor Analysis.ppt
19/28
2007 Prentice Hall 19-19
Scree Plot
0)
2 5
;
:o&'onent u&er
0)0
2)0
)0
3igenva
lue
1)0
1)
2)
1
Fig) 19)
-
7/24/2019 Factor Analysis.ppt
20/28
2007 Prentice Hall 19-20
+etermination Based on Split#;alf )elia9ility5/%e sa&'le is s'lit in %alf and factor analysis is
'erfor&ed on eac% %alf)
-
7/24/2019 Factor Analysis.ppt
21/28
2007 Prentice Hall 19-21
Alt%oug% t%e initial or unrotated factor &atri* indicatest%e relations%i' et+een t%e factors and individualvariales# it seldo& results in factors t%at can einter'reted# ecause t%e factors are correlated +it%
&any variales) /%erefore# t%roug% rotation t%e factor&atri* is transfor&ed into a si&'ler one t%at is easier tointer'ret)
n rotating t%e factors# +e +ould li,e eac% factor to %avenon(ero# or signiDcant# loadings or coeGcients for only
so&e of t%e variales) i,e+ise# +e +ould li,e eac%variale to %ave non(ero or signiDcant loadings +it%only a fe+ factors# if 'ossile +it% only one)
/%e rotation is called orthogonal rotationif t%e a*esare &aintained at rig%t angles)
Conducting FactorAnalysis
)otate Factors
-
7/24/2019 Factor Analysis.ppt
22/28
2007 Prentice Hall 19-22
/%e &ost co&&only used &et%od for rotation is t%e!arimax procedure) /%is is an ort%ogonal &et%odof rotation t%at &ini&i(es t%e nu&er of variales
+it% %ig% loadings on a factor# t%erey en%ancing t%einter'retaility of t%e factors)
-
7/24/2019 Factor Analysis.ppt
23/28
2007 Prentice Hall 19-2
Factor $atrix Before and After)otation
Factors
a
Hig% oadingsBefore otation
Fig) 19)5
Hig% oadingsAfter otation
Factors
=ariales
1
2
5
;
1
I
II
I
I
2
I
I
I
I
1
I
I
I
2
I
I
I
=ariales
1
2
5
;
-
7/24/2019 Factor Analysis.ppt
24/28
2007 Prentice Hall 19-25
A factor can t%en e inter'reted in ter&s oft%e variales t%at load %ig% on it)
Anot%er useful aid in inter'retation is to 'lott%e variales# using t%e factor loadings as
coordinates) =ariales at t%e end of an a*isare t%ose t%at %ave %ig% loadings on onlyt%at factor# and %ence descrie t%e factor)
Conducting FactorAnalysis=nterpret Factors
-
7/24/2019 Factor Analysis.ppt
25/28
2007 Prentice Hall 19-2
Factor LoadingPlot
Fig) 19)
.56
652
656
#652
#.56
Component.
Component Plotin)otated Space
.56 652 656 #652 #.56
-.
-0
-3
-/
-2
-1
Component-aria9le . /
-. 6583/ #/533#
6/
-/ #254/#6/ 65717
-0 65801 #65.13
-1 #8570#6/ 65721
-2 #65800 #7516#6/
-3 75004#6/ 65772
)otated Component $atrix
Component /
-
7/24/2019 Factor Analysis.ppt
26/28
2007 Prentice Hall 19-2;
/%e factor scoresfor t%e it% factor &aye esti&ated
as follo+s.
Fi= Wi1X1+ Wi2X2+ Wi3X3+ . . . + WikXk
Conducting FactorAnalysisCalculate Factor Scores
-
7/24/2019 Factor Analysis.ppt
27/28
2007 Prentice Hall 19-27
By e*a&ining t%e factor &atri*# one could selectfor eac% factor t%e variale +it% t%e %ig%est
loading on t%at factor) /%at variale could t%en eused as a surrogate variale for t%e associatedfactor)
Ho+ever# t%e c%oice is not as easy if t+o or &ore
variales %ave si&ilarly %ig% loadings) n suc% acase# t%e c%oice et+een t%ese variales s%oulde ased on t%eoretical and &easure&entconsiderations)
Conducting Factor
AnalysisSelect Surrogate-aria9les
-
7/24/2019 Factor Analysis.ppt
28/28
2007 Prentice Hall 19-2?
/%e correlations et+een t%e variales can e
deduced or re'roduced fro& t%e esti&atedcorrelations et+een t%e variales and t%efactors)
/%e di>erences et+een t%e oserved correlationsas given in t%e in'ut correlation &atri* and t%ere'roduced correlations as esti&ated fro& t%efactor &atri* can e e*a&ined to deter&ine&odel Dt) /%ese di>erences are called residuals)
Conducting FactorAnalysis+etermine the $odel Fit