Preisliste Exzenterschneckenpumpen Price list progressing ...
Gucciardi Et Al 2012 Progressing MT Measurement
description
Transcript of Gucciardi Et Al 2012 Progressing MT Measurement
Progressing Measurement in Mental Toughness:A Case Example of the Mental Toughness Questionnaire 48
Daniel F. GucciardiThe University of Queensland, St. Lucia,
Queensland, Australia
Sheldon HantonCardiff Metropolitan University, Cardiff, Wales
Clifford J. MallettThe University of Queensland, St. Lucia, Queensland, Australia
Mental toughness has received increasing attention in the field of performance psy-chology, yet issues remain about its measurement by self-report. In this article, we havesummarized mental toughness measurement issues and, as an example, provided apsychometric examination of the most frequently used measure. In an effort to opera-tionalize mental toughness, Clough, Earle, and Sewell (2002) developed the MentalToughness Questionnaire 48 (MTQ 48) and provided initial evidence for its reliabilityand validity. Subsequent research has partially supported the internal reliability andvalidity of the MTQ 48. However, no research has rigorously tested the factorialstructure of the hypothesized model underlying this scale. Using two independentsamples of performers from various sports (n � 686) and the workplace (n � 639), wesought to examine the factorial validity of the MTQ 48 using confirmatory factoranalysis (CFA) and exploratory structural equation modeling (ESEM). Both CFA andESEM indicated that the hypothesized correlated four factor model did not fit the datawell in the athlete and workplace samples. Our overview of measurement issues andempirical case study of the MTQ 48 underscore the importance of having a clearlyarticulated conceptual underpinning combined with rigorous statistical procedures inattempting to develop a mental toughness inventory.
Keywords: exploratory structural equation modeling, factorial validity, mentally tough, personalresources, scale development
Regardless of the achievement context (e.g.,sport, workplace, education), individuals mustsuccessfully negotiate a variety of differentstressors, challenges, and adversities (e.g., in-jury, performance expectations and targets,work�life balance) if they are to perform totheir potential and reach their goals. What is it
that enables some performers to thrive underpressure situations, to overcome setbacksquickly, and to maintain a high level of func-tioning in the face of continuous challenges?Many suggest the answer lies in their psycho-logical makeup, which is commonly concep-tualized under the umbrella term mentaltoughness. It is not surprising then that mentaltoughness has attracted increasing empiricalattention in recent years, with much of thiswork devoted to its conceptualization and defi-nition (for a review, see Gucciardi & Gordon,2011). Coinciding with this increased attentionhas been the development of psychometric toolsdesigned to operationalize these different con-ceptualizations of mental toughness.
In a recent review of the mental toughnessmeasurement literature, Gucciardi, Mallett,Hanrahan, and Gordon (2011) concluded that,at present, no comprehensively sound measureexists. Gucciardi et al. considered several tradi-
This article was published Online First February 13, 2012.Daniel F. Gucciardi and Clifford J. Mallett, School of
Human Movement Studies, The University of Queensland,St. Lucia, Queensland, Australia; Sheldon Hanton, CardiffSchool of Sport, Cardiff Metropolitan University, Cardiff,Wales.
Gucciardi is supported by a University of QueenslandPostdoctoral Research Fellowship. Appreciation is extendedto Denise Hill, Rich Neil, Steve Mellalieu, Ross Wadey, andChris Wagstaff for their assistance with data collection.
Correspondence concerning this article should be addressedto Daniel F. Gucciardi, School of Human Movement Studies,The University of Queensland, St. Lucia, Queensland, Austra-lia, 4072. E-mail: [email protected]
Sport, Exercise, and Performance Psychology © 2012 American Psychological Association2012, Vol. 1, No. 3, 194–214 2157-3905/12/$12.00 DOI: 10.1037/a0027190
194
This
doc
umen
t is c
opyr
ight
ed b
y th
e A
mer
ican
Psy
chol
ogic
al A
ssoc
iatio
n or
one
of i
ts a
llied
pub
lishe
rs.
This
arti
cle
is in
tend
ed so
lely
for t
he p
erso
nal u
se o
f the
indi
vidu
al u
ser a
nd is
not
to b
e di
ssem
inat
ed b
road
ly.
tional indicators of reliability (e.g., internal con-sistency, test�retest) and validity (e.g., content,factorial, predictive), as well as three fundamen-tal issues related to conceptual underpinning,statistical procedures, and practical utility inassessing the utility of the available measures.Although strengths of each instrument wereidentified, several limitations existed with eachquestionnaire according to the guiding criteria.These strengths and weaknesses are briefly re-viewed in the following section; interested read-ers are encouraged to consult Gucciardi et al.for a detailed discussion of these issues.
Gucciardi et al. (2011) reviewed both sport-general (i.e., developed with and for use withmultiple sports and athletes) and sport-specific(i.e., developed with and for use in a singlesport) measures of mental toughness. With re-gard to sport-general measures, the Psycholog-ical Performance Inventory (Loehr, 1986) wasdeemed inadequate according statistical (i.e.,lack of psychometric support) and conceptualcriteria (i.e., lack of a detailed conceptual un-derpinning), although the intuitive appeal of thismeasure in capturing some of the primary com-ponents of mental toughness was noted as a keystrength. Similar concerns about the conceptualunderpinning were made of its amended ver-sion, namely, the Psychological PerformanceInventory�A (Golby, Sheard, & van Wersch,2007), although strengths existed with regard topractical utility (i.e., item brevity) and the pre-liminary evidence supported its factorial valid-ity. The Sport Mental Toughness Questionnaire(Sheard, Golby, & van Wersch, 2009) was con-sidered strong in terms of the statistical proce-dures employed to support its psychometricproperties and practical utility (i.e., item brev-ity), yet it lacked an explicit conceptual modelunderpinning its factor structure. Finally, theMental Toughness Questionnaire 48 (MTQ 48;Clough et al., 2002), which has also been usedto examine mental toughness in nonsport con-texts (e.g., workplace, rehabilitation), wasdeemed inadequate according to statistical (i.e.,psychometric support for the hypothesizedmodel is currently unavailable) and conceptualcriteria (i.e., little information about the ratio-nale for adopting hardiness theory), althoughhaving a conceptual model for its developmentwas considered a key strength.
Gucciardi et al. (2011) also reviewed twosport-specific measures. The conceptual under-
pinning of the Australian Football MentalToughness Inventory (Gucciardi, Gordon, &Dimmock, 2009) was considered a key strength,despite the identification of weaknesses withregard to both statistical (i.e., cross-validationof the hypothesized measurement model wasnot supported) and practical utility (i.e., limitedusefulness beyond Australian football) criteria.The adoption of rigorous, hypothesis-testingstatistical procedures was considered a keystrength of the Cricket Mental Toughness In-ventory (Gucciardi & Gordon, 2009), yet con-cerns about the generalizability of the modelremained (i.e., practical utility).
From this brief review of the available mentaltoughness measures, it becomes apparent thatresearchers have tended to place greater impor-tance on either a strong conceptual underpin-ning or rigorous statistical procedures to de-velop and validate tools. However, both of thesecriteria are important for scale developmentsuch that the marginalization of one criterioncan have significant consequences for the integrityof an instrument (for a review, see MacKenzie,Podsakoff, & Podsakoff, 2011). In this article,we examined the extent to which a preferencefor a conceptual underpinning over rigorousstatistical analyses might compromise the psy-chometric integrity of the most commonly em-ployed mental toughness inventory.
The MTQ 48 (Clough et al., 2002), whichevolved from a noteworthy body of researchthat examined the stress�illness relationship, isthe most widely employed tool for assessingmental toughness both in sport and nonsportcontexts (for an overview, see Table 1). Emerg-ing from research on stress reactions in thehealth psychology literature is the hardinessconstruct, which is conceptualized as a combi-nation of three attitudes—commitment, control,and challenge (3Cs)—that provide an individ-ual with existential courage and motivation toappraise stressful situations as opportunities forgrowth (i.e., choose to approach the unknownrealms of the future, rather than repeating past,familiar experiences) (Maddi, 2004). ForClough et al. (2002), the three hardiness atti-tudes closely resembled but did not fully encap-sulate mental toughness. Accordingly, theyadded a fourth dimension, confidence, to ac-count for the ecologically valid views of keystakeholders (i.e., athletes, coaches, sport psy-chologists). Within the context of their 4Cs
195MEASURING MENTAL TOUGHNESS
This
doc
umen
t is c
opyr
ight
ed b
y th
e A
mer
ican
Psy
chol
ogic
al A
ssoc
iatio
n or
one
of i
ts a
llied
pub
lishe
rs.
This
arti
cle
is in
tend
ed so
lely
for t
he p
erso
nal u
se o
f the
indi
vidu
al u
ser a
nd is
not
to b
e di
ssem
inat
ed b
road
ly.
Tab
le1
Ove
rvie
wof
Stud
ies
Em
ploy
ing
the
MT
Q48
asa
Mea
sure
ofM
enta
lT
ough
ness
Stud
yPa
rtic
ipan
tsM
etho
dsPr
imar
yfin
ding
s
Clo
ugh,
Ear
le,
&Se
wel
l(2
002)
23pa
rtic
ipan
ts(d
emog
raph
icin
form
atio
nun
avai
labl
e,e.
g.,
age,
sex,
skill
leve
l)Se
lf-r
atin
gs(p
hysi
cal
dem
ands
,m
enta
lde
man
ds,
effo
rt,
MT
Q48
),V
O2m
ax,
and
cycl
ing
Inte
rnal
relia
bilit
yes
timat
esw
ere
not
prov
ided
.M
edia
nsp
litba
sed
onm
enta
lto
ughn
ess;
diff
eren
ces
inpe
rcei
ved
phys
ical
dem
ands
exis
ted
only
whe
nw
orkl
oad
was
high
(70%
VO
2m
ax),
but
not
whe
nlo
w(3
0%V
O2m
ax)
orm
oder
ate
(50%
VO
2m
ax).
Clo
ugh
etal
.(2
002)
79pa
rtic
ipan
ts(d
emog
raph
icin
form
atio
nun
avai
labl
e,e.
g.,
age,
sex,
skill
leve
l)R
ecei
ved
posi
tive
orne
gativ
efe
edba
ckaf
ter
com
plet
ing
anu
mbe
rof
mot
orta
sks,
then
com
plet
eda
plan
ning
task
(i.e
.,co
gniti
veex
erci
se)
Inte
rnal
relia
bilit
yes
timat
esw
ere
not
prov
ided
.M
enta
llyto
ughe
rpa
rtic
ipan
tspe
rfor
med
bette
ron
the
plan
ning
task
;in
tera
ctio
nef
fect
betw
een
men
tal
toug
hnes
san
dfe
edba
ck,
such
that
type
offe
edba
ckw
asir
rele
vant
for
men
tally
toug
her
part
icip
ants
,bu
tle
ssm
enta
llyto
ughe
rpa
rtic
ipan
tspe
rfor
med
wor
seaf
ter
nega
tive
feed
back
.C
rust
(200
9)55
mal
e(M
age
�22
.58
year
s)an
d57
fem
ale
(Mage
�21
.11
year
s)at
hlet
esfr
oma
vari
ety
ofsp
orts
Cro
ss-s
ectio
nal
surv
eyco
ntai
ning
the
Aff
ect
Inte
nsity
Mea
sure
(Lar
sen,
1984
)an
dth
eM
TQ
48
Inte
rnal
relia
bilit
yes
timat
epr
ovid
edfo
rto
tal
men
tal
toug
hnes
son
ly(�
�.8
6).
No
sign
ifica
ntdi
ffer
ence
sin
men
tal
toug
hnes
sbe
twee
nm
ale
and
fem
ale
athl
etes
,an
dre
crea
tiona
lan
dcl
ubor
high
erpa
rtic
ipan
tle
vel.
No
rela
tions
hips
obse
rved
betw
een
men
tal
toug
hnes
san
daf
fect
inte
nsity
.C
rust
&A
zadi
(200
9)66
mal
e(M
age
�30
.1ye
ars)
and
37fe
mal
e(M
age
�28
.6ye
ars)
athl
etes
from
club
orun
iver
sity
(n�
36)
and
coun
ty(n
�67
)st
anda
rd
Cro
ss-s
ectio
nal
surv
eyco
ntai
ning
the
Lea
ders
hip
Scal
efo
rSp
orts
(Che
lladu
rai
&Sa
leh,
1978
)an
dth
eM
TQ
48
Inte
rnal
relia
bilit
yes
timat
esw
ere
not
prov
ided
.C
orre
latio
nsev
iden
ced
betw
een
trai
ning
and
inst
ruct
ion
and
tota
lm
enta
lto
ughn
ess
(r�
.40)
,and
all
othe
rfa
cets
ofm
enta
lto
ughn
ess
(rra
nge:
.22�
.36)
exce
ptfo
rin
terp
erso
nal
confi
denc
e;de
moc
ratic
beha
vior
san
dco
nfide
nce
inab
ilitie
s(r
��
.27)
;so
cial
supp
ort
and
confi
denc
ein
abili
ties
(r�
�.2
0).
Reg
ress
ion
anal
yses
reve
aled
rela
tions
hips
betw
een
com
mitm
ent
(��
.26)
and
chal
leng
e(�
�.2
4)w
ithtr
aini
ngan
din
stru
ctio
n(1
8%va
rian
ceex
plai
ned)
;em
otio
nal
cont
rol
(��
�.2
1),c
onfid
ence
inab
ilitie
s(�
��
.29)
,and
life
cont
rol
(��
.28)
with
dem
ocra
ticbe
havi
ors
(14%
);ch
alle
nge
(��
.25)
and
com
mitm
ent
(��
�.2
2)w
ithau
tocr
atic
beha
vior
s(7
%);
chal
leng
e(�
�.1
9),
emot
iona
lco
ntro
l(�
��
.19)
,and
confi
denc
ein
abili
ties
(��
�.2
1)w
ithso
cial
supp
ort
196 GUCCIARDI, HANTON, AND MALLETT
This
doc
umen
t is c
opyr
ight
ed b
y th
e A
mer
ican
Psy
chol
ogic
al A
ssoc
iatio
n or
one
of i
ts a
llied
pub
lishe
rs.
This
arti
cle
is in
tend
ed so
lely
for t
he p
erso
nal u
se o
f the
indi
vidu
al u
ser a
nd is
not
to b
e di
ssem
inat
ed b
road
ly.
Tab
le1
(con
tinu
ed)
Stud
yPa
rtic
ipan
tsM
etho
dsPr
imar
yfin
ding
s
(8%
);an
dem
otio
nal(
��
�.2
1)an
dlif
eco
ntro
l(�
�.2
7)w
ithpo
sitiv
efe
edba
ck(6
%).
Cru
st&
Aza
di(2
010)
67m
ale
(Mage
�22
.6ye
ars)
and
40fe
mal
e(M
age
�21
.1ye
ars)
athl
etes
from
club
orun
iver
sity
(n�
36)
and
coun
ty(n
�71
)st
anda
rd
Cro
ss-s
ectio
nal
surv
eyco
ntai
ning
the
Tes
tof
Perf
orm
ance
Stra
tegi
es(T
hom
as,
Mur
phy,
&H
ardy
,19
99)
and
the
MT
Q48
Inte
rnal
relia
bilit
yes
timat
esw
ere
notp
rovi
ded.
Com
mitm
ente
vide
nced
the
grea
test
num
ber
ofst
atis
tical
lysi
gnifi
cant
corr
elat
ions
with
psyc
holo
gica
lski
llus
age
inpr
actic
e(n
�7,
rra
nge:
.19–
.31)
and
com
petit
ion
(n�
6,r
rang
e�
�.3
2to
.40)
,fol
low
edby
tota
lmen
tal
toug
hnes
s(n
pra
ctic
e�
4,r
rang
e:.2
4�.3
5;n c
om
p�
6,r
rang
e:�
.47
to.2
4)an
dch
alle
nge
(npra
ctic
e�
3,r
rang
e:.1
9�.2
2;n c
om
p�
3,r
rang
e:�
.37
to.2
4).R
egre
ssio
nan
alys
esre
veal
edth
esu
perio
rity
ofco
mm
itmen
tin
term
sof
the
grea
test
num
ber
ofst
atis
tical
lysi
gnifi
cant
rela
tions
hips
with
psyc
holo
gica
lski
llsus
age
(n�
9,�
rang
e:.1
9�.4
6),f
ollo
wed
byem
otio
nalc
ontro
l(n
�9,
�ra
nge:
.21�
.42)
and
confi
denc
ein
abili
ties
(n�
4,�
rang
e:�
.24
to.4
2).
Cru
st&
Kee
gan
(201
0)69
mal
e(M
age
�22
.2ye
ars)
and
36fe
mal
e(M
age
�24
.6ye
ars)
stud
ent
athl
etes
from
recr
eatio
nal
(n�
32)
club
orun
iver
sity
(n�
55)
and
coun
ty(n
�18
)st
anda
rd
Cro
ss-s
ectio
nal
surv
eyco
ntai
ning
the
Atti
tude
sT
owar
dsR
isks
Que
stio
nnai
re(F
rank
en,
Gib
son,
&R
owla
nd,
1992
)an
dth
eM
TQ
48
Tot
alm
enta
ltou
ghne
ss,c
onfid
ence
inab
ilitie
s,an
din
terp
erso
nalc
onfid
ence
had
acce
ptab
lele
vels
ofin
tern
alre
liabi
lity
(i.e.
,��
.70)
,whe
reas
chal
leng
e(�
�.6
8),c
omm
itmen
t(�
�.6
2),
emot
iona
lcon
trol(
��
.60)
,and
life
cont
rol
(��
.56)
did
not.
With
rega
rdto
phys
ical
risks
,to
talm
enta
ltou
ghne
ss(r
�.3
0),c
halle
nge
(r�
.43)
,com
mitm
ent(
r�
.20)
,and
confi
denc
ein
abili
ties
(r�
.21)
had
stat
istic
ally
sign
ifica
ntre
latio
nshi
ps.O
nly
inte
rper
sona
lcon
fiden
ce(r
�.2
4)w
assi
gnifi
cant
lyre
late
dto
psyc
holo
gica
lris
ks.M
endi
spla
yed
high
erle
vels
ofto
talm
enta
lto
ughn
ess
(p�
.04)
and
confi
denc
ein
abili
ties
(p�
.04)
than
wom
en.
Cru
st,
Nes
ti,&
Litt
lew
ood
(201
0a)
112
mal
efo
otba
llers
aged
12to
18ye
ars
Cro
ss-s
ectio
nal
surv
eyco
ntai
ning
the
MT
Q18
Inte
rnal
relia
bilit
yes
timat
epr
ovid
edfo
rto
talm
enta
lto
ughn
ess
(��
.69)
.No
sign
ifica
ntdi
ffer
ence
sin
men
talt
ough
ness
wer
ere
veal
edbe
twee
npl
ayer
sw
how
ere
reta
ined
and
rele
ased
atth
een
dof
the
seas
on,o
rag
egr
oups
(i.e.
,Und
er13
/Und
er14
/Und
er16
/Und
er19
).
197MEASURING MENTAL TOUGHNESS
This
doc
umen
t is c
opyr
ight
ed b
y th
e A
mer
ican
Psy
chol
ogic
al A
ssoc
iatio
n or
one
of i
ts a
llied
pub
lishe
rs.
This
arti
cle
is in
tend
ed so
lely
for t
he p
erso
nal u
se o
f the
indi
vidu
al u
ser a
nd is
not
to b
e di
ssem
inat
ed b
road
ly.
Tab
le1
(con
tinu
ed)
Stud
yPa
rtic
ipan
tsM
etho
dsPr
imar
yfin
ding
s
Cru
st,
Nes
ti,&
Litt
lew
ood
(201
0b)
21m
ale
foot
ball
play
ers
aged
16to
18ye
ars;
2co
ache
s(d
emog
raph
ics
unav
aila
ble)
Pros
pect
ive
surv
eyof
the
MT
Q18
attw
otim
epo
ints
disp
erse
dby
thre
em
onth
s
Inte
rnal
relia
bilit
yes
timat
esw
ere
not
prov
ided
.Pl
ayer
sre
port
edsi
gnifi
cant
lyhi
gher
leve
lsof
men
tal
toug
hnes
sth
anon
eof
the
coac
hes
(p�
.05)
but
not
the
othe
rco
ach.
Cor
rela
tions
betw
een
the
thre
era
ting
sour
ces
wer
eno
tsi
gnifi
cant
.M
enta
lto
ughn
ess
ratin
gsfo
rea
chra
ting
sour
cew
ere
stab
leov
erth
e3-
mon
thpe
riod
(rra
nge:
.94�
.99)
.L
evel
ofag
reem
ent
betw
een
ratin
gso
urce
sw
aslo
w(i
ntra
clas
sr
rang
e:.0
8�.2
9).
Cru
st&
Swan
n(2
011)
110
mal
est
uden
tat
hlet
es(M
age
�20
.8ye
ars)
from
ava
riet
yof
spor
tsC
ross
-sec
tiona
lsu
rvey
cont
aini
ngth
eSp
orts
Men
tal
Tou
ghne
ssQ
uest
ionn
aire
(She
ard,
Gol
by,
&va
nW
ersc
h,20
09)
and
the
MT
Q48
All
subs
cale
sof
the
MT
Q48
,ex
cept
for
emot
iona
lco
ntro
l(�
�.4
5)an
dlif
eco
ntro
l(�
�.5
0),
show
edad
equa
tele
vels
ofin
tern
alre
liabi
lity
(i.e
.,�
�.7
0).
Sign
ifica
ntco
rrel
atio
nsw
ere
evid
ence
dbe
twee
nth
ehi
gher
-ord
er(i
.e.,
glob
alor
tota
lsc
ores
)fa
ctor
s(r
�.7
5).
Subs
cale
sw
ithov
erla
ppin
gco
ncep
tual
desc
ript
ions
(i.e
.,co
nfide
nce,
cont
rol,
com
mitm
ent/c
onst
ancy
)w
ere
mod
erat
ely
corr
elat
ed(r
rang
e:.4
9�.6
1).
Kai
sele
r,Po
lman
,&
Nic
holls
(200
9)48
2at
hlet
es(3
05m
ales
)ag
ed16
to45
year
s(M
age
�20
.44
year
s)fr
omin
tern
atio
nal
tocl
ubor
univ
ersi
tyle
vels
Cro
ss-s
ectio
nal
surv
eyco
ntai
ning
ast
ress
orty
pean
dap
prai
sal,
copi
ngan
dco
ping
effe
ctiv
enes
s(C
rock
er&
Gra
ham
,19
95),
and
the
MT
Q48
Inte
rnal
relia
bilit
yes
timat
esfo
rto
tal
men
tal
toug
hnes
s(�
�.9
2)an
dfiv
efa
cets
wer
ead
equa
te(�
�.6
9),
exce
ptfo
rem
otio
nal
cont
rol
(��
.55)
.M
enta
lto
ughn
ess
exhi
bite
dlo
w-t
o-m
oder
ate
corr
elat
ions
(rra
nge:
�.6
7to
.30)
with
copi
ngan
dco
ping
effe
ctiv
enes
s.T
otal
men
tal
toug
hnes
san
dits
six
face
tsac
coun
ted
for
am
inim
alam
ount
ofth
eva
rian
cein
stre
ssin
tens
ity(3
%an
d7%
,re
spec
tivel
y)an
dpe
rcei
ved
cont
rol
(4%
and
5%,
resp
ectiv
ely)
.T
otal
men
tal
toug
hnes
san
dits
six
face
tsac
coun
ted
for
alo
w-t
o-m
oder
ate
amou
ntof
the
vari
ance
inex
tent
ofus
e(0
–24%
and
3–48
%,
resp
ectiv
ely)
and
perc
eive
def
fect
iven
ess
(0–7
%an
d1–
10%
,re
spec
tivel
y)of
copi
ngst
rate
gies
.
198 GUCCIARDI, HANTON, AND MALLETT
This
doc
umen
t is c
opyr
ight
ed b
y th
e A
mer
ican
Psy
chol
ogic
al A
ssoc
iatio
n or
one
of i
ts a
llied
pub
lishe
rs.
This
arti
cle
is in
tend
ed so
lely
for t
he p
erso
nal u
se o
f the
indi
vidu
al u
ser a
nd is
not
to b
e di
ssem
inat
ed b
road
ly.
Tab
le1
(con
tinu
ed)
Stud
yPa
rtic
ipan
tsM
etho
dsPr
imar
yfin
ding
s
Lev
y,Po
lman
,C
loug
h,M
arch
ant,
&E
arle
(200
6)70
reha
bilit
atio
npa
tient
s(4
4m
ale;
Mage
�32
.5ye
ars)
Cro
ss-s
ectio
nal
surv
eyco
ntai
ning
am
easu
reof
adhe
renc
e,Sp
ort
Inju
ryR
ehab
ilita
tion
Bel
iefs
Surv
ey(D
aly,
Bre
wer
,&
Van
Raa
lte,
1995
),Sp
ort
Inve
ntor
yfo
rPa
in(M
eyer
s,B
ourg
eois
,St
ewar
t,&
LeU
nes,
1992
)an
dth
eM
TQ
18
Inte
rnal
relia
bilit
yes
timat
efo
rm
enta
lto
ughn
ess
was
less
than
adeq
uate
(��
65).
Thr
eem
enta
lto
ughn
ess
grou
ps(l
ow,
med
ium
,hi
gh)
wer
ecr
eate
d(i
nfor
mat
ion
onho
wth
ese
grou
psw
ere
form
edw
asno
tpr
ovid
ed).
The
high
men
tal
toug
hnes
sgr
oup
repo
rted
sign
ifica
ntly
low
erle
vels
ofpe
rcei
ved
susc
eptib
ility
than
both
the
med
ium
(p�
.05)
and
the
low
(p�
.01)
grou
ps;
the
med
ium
men
tal
toug
hnes
sgr
oup
repo
rted
sign
ifica
ntly
high
erle
vels
ofpe
rcei
ved
seve
rity
than
the
high
men
tal
toug
hnes
sgr
oup
(p�
.01)
;th
ehi
ghm
enta
lto
ughn
ess
grou
pre
port
edsi
gnifi
cant
lyhi
gher
leve
lsof
copi
ngw
ithpa
inth
anbo
thth
em
ediu
m(p
�.0
1)an
dth
elo
w(p
�.0
01)
grou
ps;
the
low
men
tal
toug
hnes
sgr
oup
repo
rted
high
erle
vels
ofpa
inca
tast
roph
eth
anth
ehi
ghm
enta
lto
ughn
ess
grou
p(p
�.0
1);
the
high
men
tal
toug
hnes
sgr
oup
repo
rted
sign
ifica
ntly
low
erle
vels
ofcl
inic
reha
bilit
atio
nad
here
nce
than
both
the
med
ium
and
the
low
(p�
.01)
grou
ps,
and
high
erat
tend
ance
atre
habi
litat
ion
than
the
low
men
tal
toug
hnes
sgr
oup
(p�
.05)
.
199MEASURING MENTAL TOUGHNESS
This
doc
umen
t is c
opyr
ight
ed b
y th
e A
mer
ican
Psy
chol
ogic
al A
ssoc
iatio
n or
one
of i
ts a
llied
pub
lishe
rs.
This
arti
cle
is in
tend
ed so
lely
for t
he p
erso
nal u
se o
f the
indi
vidu
al u
ser a
nd is
not
to b
e di
ssem
inat
ed b
road
ly.
Tab
le1
(con
tinu
ed)
Stud
yPa
rtic
ipan
tsM
etho
dsPr
imar
yfin
ding
s
Mar
chan
tet
al.
(200
8)52
2m
ange
rs(2
10m
en)
from
orga
niza
tions
base
din
the
Uni
ted
Kin
gdom
Cro
ss-s
ectio
nal
surv
eyco
ntai
ning
the
MT
Q48
Inte
rnal
relia
bilit
yes
timat
efo
rto
tal
men
tal
toug
hnes
s(�
�.8
9)w
asad
equa
te,
with
the
subs
cale
sre
port
edas
bein
gab
ove
.70
(spe
cific
valu
esno
tre
port
ed).
Seni
orm
anag
ers
scor
edsi
gnifi
cant
lyhi
gher
than
mid
dle
and
juni
orm
anag
ers
onal
lM
TQ
48su
bsca
les
(p�
.01)
;m
iddl
em
anag
ers
scor
edhi
gher
leve
lsof
tota
lm
enta
lto
ughn
ess
(p�
.05)
,lif
eco
ntro
l,an
din
terp
erso
nal
confi
denc
e(p
�.0
1)th
anju
nior
man
ager
san
dcl
eric
alst
aff;
mid
dle
man
ager
ssc
ored
high
erle
vels
ofch
alle
nge
and
com
mitm
ent
(p�
.01)
than
cler
ical
staf
f,an
dco
nfide
nce
inab
ilitie
s(p
�.0
1)th
anju
nior
man
agem
ent.
Ana
lyse
sre
veal
edm
ain
effe
cts
ofag
efo
rto
tal
men
tal
toug
hnes
s(p
�.0
1),
com
mitm
ent
(p�
.01)
,em
otio
nal
cont
rol
(p�
.05)
,an
dlif
eco
ntro
l(p
�.0
01).
Tre
nds
(i.e
.,no
sign
ifica
nce
leve
lsre
port
ed)
inpo
stho
cco
mpa
riso
nsof
age
sugg
este
dol
der
part
icip
ants
wer
ege
nera
llyhi
gher
inm
enta
lto
ughn
ess
than
youn
ger
part
icip
ants
.N
icho
lls,
Lev
y,Po
lman
,&
Cru
st(2
011)
206
athl
etes
(182
mal
es;
Mag
e�
17.7
5ye
ars)
from
inte
rnat
iona
lto
club
orun
iver
sity
leve
ls.
Com
plet
edth
eM
TQ
48an
dC
opin
gSe
lf-E
ffica
cySc
ale
(Che
sney
,N
eila
nds,
Cha
mbe
rs,
Tay
lor,
&Fo
lkm
an,
2006
)on
the
day
ofa
com
petit
ion,
and
am
easu
reof
copi
ngef
fect
iven
ess
(Got
tlieb
&R
oone
y,20
04)
30m
inaf
ter
the
com
petit
ion
Inte
rnal
relia
bilit
yes
timat
esfo
rfiv
efa
cets
wer
ead
equa
te(�
�.6
9),
exce
ptfo
rch
alle
nge
(��
.54)
.A
step
wis
ere
gres
sion
reve
aled
that
tota
lm
enta
lto
ughn
ess
(��
.12)
expl
aine
dan
addi
tiona
l3%
vari
ance
inco
ping
effe
ctiv
enes
sth
anco
ping
self
-ef
ficac
y(6
%,
��
.18)
.A
seco
ndst
epw
ise
regr
essi
onw
ithth
esi
xm
enta
lto
ughn
ess
subs
cale
sre
veal
edco
mm
itmen
t(�
�.2
0)as
the
only
sign
ifica
ntfa
cet,
whi
chex
plai
ned
anad
ditio
nal
5%va
rian
cein
copi
ngef
fect
iven
ess
than
copi
ngse
lf-e
ffica
cy(6
%,
��
.18)
.
200 GUCCIARDI, HANTON, AND MALLETT
This
doc
umen
t is c
opyr
ight
ed b
y th
e A
mer
ican
Psy
chol
ogic
al A
ssoc
iatio
n or
one
of i
ts a
llied
pub
lishe
rs.
This
arti
cle
is in
tend
ed so
lely
for t
he p
erso
nal u
se o
f the
indi
vidu
al u
ser a
nd is
not
to b
e di
ssem
inat
ed b
road
ly.
Tab
le1
(con
tinu
ed)
Stud
yPa
rtic
ipan
tsM
etho
dsPr
imar
yfin
ding
s
Nic
holls
,Po
lman
,L
evy,
&B
ackh
ouse
(200
8)67
7at
hlet
es(4
54m
ales
)ag
ed15
to58
year
s(M
age
�22
.66)
from
inte
rnat
iona
lto
begi
nner
leve
ls
Cro
ss-s
ectio
nal
surv
eyco
ntai
ning
the
Cop
ing
Inve
ntor
yfo
rC
ompe
titiv
eSp
ort
(Gau
drea
u&
Blo
ndin
,20
02),
Lif
eO
rien
tatio
nT
est
(Sch
eier
&C
arve
r,19
85),
and
the
MT
Q48
Inte
rnal
relia
bilit
yes
timat
efo
rto
tal
men
tal
toug
hnes
sw
asad
equa
te(�
�.8
7),
with
subs
cale
relia
bilit
ies
rang
ing
from
.58
to.7
1.M
enta
lto
ughn
ess
was
posi
tivel
yre
late
dw
ithpr
oble
mor
appr
oach
copi
ngst
rate
gies
(rra
nge:
�.1
5to
.32)
and
nega
tivel
yre
late
dw
ithav
oida
nce
copi
ngst
rate
gies
(rra
nge:
�.2
8to
.03)
,al
thou
ghex
cept
ions
inth
ere
latio
nshi
pdi
rect
ion
did
exis
t.M
enta
lto
ughn
ess
was
posi
tivel
yre
late
dto
optim
ism
(rra
nge:
.08�
.56)
and
nega
tivel
yre
late
dto
pess
imis
m(r
rang
e:�
.16
to�
.49)
.N
icho
lls,
Polm
an,
Lev
y,&
Bac
khou
se(2
009)
677
athl
etes
(454
mal
es)
aged
15to
58ye
ars
(Mage
�22
.66)
from
inte
rnat
iona
lto
begi
nner
leve
ls
Cro
ss-s
ectio
nal
surv
eyco
ntai
ning
the
MT
Q48
Inte
rnal
relia
bilit
yes
timat
efo
rto
tal
men
tal
toug
hnes
sw
asad
equa
te(�
�.8
7),
with
subs
cale
relia
bilit
ies
rang
ing
from
.58
to.7
1.M
ensc
ored
sign
ifica
ntly
high
er(p
�.0
5)th
anw
omen
onch
alle
nge,
emot
iona
lco
ntro
l,lif
eco
ntro
l,an
dco
nfide
nce
inab
ilitie
s.N
osi
gnifi
cant
diff
eren
ces
inm
enta
lto
ughn
ess
acro
ssac
hiev
emen
tle
vels
and
spor
tty
pe(t
eam
/indi
vidu
al,
cont
act/n
onco
ntac
t)w
ere
foun
d.A
gean
dye
ars
ofpl
ayin
gex
peri
ence
cont
ribu
ted
toth
eam
ount
ofva
rian
ceex
plai
ned
into
tal
men
tal
toug
hnes
s(3
%,
��
.18;
3%,
��
.17)
,ch
alle
nge
(2%
,�
�.1
4;3%
,�
�.1
8),
com
mitm
ent
(4%
,�
�.2
1;3%
,�
�.1
7),
and
life
cont
rol
(2%
,�
�.1
6;3%
,�
�.1
8).
Not
e.M
TQ
48�
Men
tal
Tou
ghne
ssQ
uest
ionn
aire
48;
VO
2m
ax�
max
imal
oxyg
enco
nsum
ptio
n;M
TQ
18�
Men
tal
Tou
ghne
ssQ
uest
ionn
aire
18.
201MEASURING MENTAL TOUGHNESS
This
doc
umen
t is c
opyr
ight
ed b
y th
e A
mer
ican
Psy
chol
ogic
al A
ssoc
iatio
n or
one
of i
ts a
llied
pub
lishe
rs.
This
arti
cle
is in
tend
ed so
lely
for t
he p
erso
nal u
se o
f the
indi
vidu
al u
ser a
nd is
not
to b
e di
ssem
inat
ed b
road
ly.
model of mental toughness, mentally tough in-dividuals (a) view negative experiences (e.g.,stress and anxiety) as a challenge that they canovercome but also a natural and essential cata-lyst for growth and development; (b) believethat they are influential in dealing with andcontrolling negative life experiences; (c) aredeeply involved in what they are doing andcommitted to achieving their goals; and (d) areconfident in their ability to deal with and over-come negative life experiences. The MTQ 48 isdesigned to measure the 4Cs’ conceptualizationof mental toughness (Clough et al., 2002).
Although having a conceptual foundation forits development represents a key strength of theMTQ 48, psychometric examinations of the ro-bustness of its factor structure are required tostatistically substantiate the hypothesizedmodel. Construct validation is an ongoing pro-cess (e.g., Marsh, 1997), and central to thisprogress is the adoption of methodologies thatdemonstrate rigor, reliability, and validity. Fac-torial validity, in particular, has implicationsboth for practice (e.g., how an instrument isscored, defining subscales based on item con-tent) and theory (e.g., dimensionality, hierarchi-cal representation), and it is important to ascer-tain this type of validity before other forms,such as predictive and concurrent validity (Gig-nac, 2009; Marsh, Martin, & Jackson, 2010). Asthe measure of choice for most mental tough-ness researchers and practitioners, it is impor-tant that we have confidence in the psychomet-ric integrity of this inventory.
It appears that the MTQ 48 has been uncrit-ically adopted as a preferred tool for mentaltoughness measurement before a thorough ex-amination of its dimensionality has been under-taken. Indeed, several of the available studiesthat have employed the MTQ 48 as a measure ofmental toughness have included samples sizesin excess of 400 (e.g., Kaiseler, Polman, &Nicholls, 2009; Marchant et al., 2009; Nicholls,Polman, Levy, & Backhouse, 2008, 2009), yetwith one exception (Horsburgh, Schermer, Ve-selka, & Vernon, 2009) its factor structure has notbeen rigorously examined. As detailed in Table 1,internal reliability estimates are sometimes ig-nored, and when reported reveal inadequacieswith several MTQ 48 subscales, according to rec-ommended minimum levels for exploratory re-search (i.e., Cronbach’s alpha � .70; Nunnally &Bernstein, 1994). The lack of information about
the psychometric procedures employed to de-velop the MTQ 48 becomes even more prob-lematic when one considers the differing factorstructures reported in recent research, namely,four (e.g., Clough et al., 2002; Veselka,Schermer, Petrides, & Vernon, 2009), six (e.g.,Crust & Azadi, 2009, 2010; Nicholls, Levy,Polman, & Crust, 2011), and nine factor models(e.g., Horsburgh et al., 2009). Regardless of thetype of measurement model adopted, only onestudy to date has reported an examination of thefactorial validity of the MTQ 48. Unfortunately,however, Horsburgh et al. did not report anyempirical data (i.e., fit indices, parameter esti-mates) to support their conclusion about thesuperiority of the correlated, four factor modelwhen compared with a unidimensional modelwith a sample of the general population. Theinclusion of such empirical data and a descrip-tion of the criteria on which the adequacy of themodel�data fit is evaluated are importantpieces of information to support the veracity ofone’s conclusions.
Reexaminations of the factor structure ofmeasurement instruments are an important con-sideration for the robustness of theoretical mod-els, especially when examining multidimen-sional constructs across different populations tothose employed in the initial validation of aquestionnaire. Numerous examples exist bothwithin the mental toughness field (e.g., Guc-ciardi, 2009) and beyond (e.g., Lane, Harwood,Terry, & Karageorghis, 2004; Martens & Web-ber, 2002) in which measurement instrumentsthat were developed within one context or sam-ple failed to generalize to others. Despite itsongoing influence on research and practice asthe most frequently adopted measure for mostresearchers and practitioners, the MTQ 48 hasyet to be subjected to a rigorous psychometricexamination. According to Marsh, Martin, etal., “To move too quickly to potentially super-ficial between-construct research is to risk with-in-construct problems that characterize manypsychological measures” (2010, p. 464). Re-searchers (e.g., Connaughton & Hanton, 2009;Gucciardi et al., 2011) have questioned the use-fulness of the MTQ 48 as a measure of mentaltoughness according to both empirical (i.e., de-tailed information on the scale construction pro-cess and factorial validity is unavailable; inad-equate internal reliability) and conceptual con-siderations (i.e., 75% of the underlying model is
202 GUCCIARDI, HANTON, AND MALLETT
This
doc
umen
t is c
opyr
ight
ed b
y th
e A
mer
ican
Psy
chol
ogic
al A
ssoc
iatio
n or
one
of i
ts a
llied
pub
lishe
rs.
This
arti
cle
is in
tend
ed so
lely
for t
he p
erso
nal u
se o
f the
indi
vidu
al u
ser a
nd is
not
to b
e di
ssem
inat
ed b
road
ly.
hardiness theory, little information on the ratio-nale for the underlying theoretical model), al-though researchers have reported evidence tosupport its concurrent validity (see Table 1).
In response to these untested concerns, theprimary aim of this study was to examine thefactorial validity of the MTQ 48 in two broadachievement contexts. Although four (e.g.,Clough et al., 2002; Veselka et al., 2009), six(e.g., Crust & Azadi, 2009, 2010), and ninefactor models (e.g., Horsburgh et al., 2009) ofthe MTQ 48 have been employed in previousresearch, our primary focus was on the fourfactor model, which is consistent with the orig-inal conceptualization of mental toughness for-warded by Clough and his colleagues (i.e., 4Csmodel). An athlete sample was chosen becausethe majority of published research has used theMTQ 48 as a measure of mental toughness insport contexts (see Table 1). It was also deemedimportant to test the veracity of the hypothe-sized model in workplace contexts because har-diness theory emerged primarily from a 12-yearlongitudinal study of stress reactions amongmanagers at a telephone company (cf. Maddi &Kobasz, 1984). Because this study is the first toexamine to factorial validity of the MTQ 48 ina sample of athletes or workplace performers, anull hypothesis was adopted; that is, it washypothesized that the original, four-factormodel would evidence an adequate level of fitwith the data.
Because there is a hypothesized conceptualmodel underlying the MTQ 48 (Clough et al.,2002), it may be argued that state-of-the-artanalytical techniques such as confirmatory fac-tor analysis (CFA; Hagger & Chatzisarantis,2009, p. 513), which test an a priori structureagainst the data, should be employed to exam-ine the robustness of its measurement model(i.e., factorial validity). Reflecting the hypothe-sis that a specific number of factors are influ-enced by certain indicators, each item is al-lowed to load on one factor only (i.e., no crossloadings) and all nontarget loadings are con-strained to be zero in this highly restrictive dataanalytical approach (Thompson, 2004). To takea strictly confirmatory approach to analysis,therefore, it is important that psychometric in-struments are developed from a clearly articu-lated theoretical model and have simple mea-surement structure (Asparouhov & Muthen,2009). Owing to the limited published informa-
tion on the rationale for the conceptual modeland empirical evidence on its psychometricproperties (e.g., model fit, parameter estimates),CFA may not be suitably justifiable as an ana-lytical approach for the assessment of theMTQ 48.
When a strictly confirmatory approach toanalysis is not well suited, exploratory struc-tural equation modeling (ESEM) offers an al-ternative method for evaluating the psychomet-ric integrity of measurement instruments whenmodel�data fit is of primary interest (Marsh etal., 2009). Recently introduced to the academiccommunity, ESEM is a novel methodologicalextension of traditional factor analyses in whichthe strengths of both CFA and exploratory fac-tor analysis (EFA) are integrated within a struc-tural equation modeling framework (Asp-arouhov & Muthen, 2009). Specifically, ESEMavoids the strict requirements of CFA (i.e., onlycertain items load onto certain factors, nontargetloadings are constrained to be zero) by allowingall item indicators to be directly influenced byall common factors as in EFA, while at the sametime providing access to robust indicators ofmodel adequacy (e.g., parameter estimates,goodness-of-fit statistics, standard errors) thatare typically associated with CFA. When com-pared with CFA, ESEM is less likely to distort(i.e., inflate and bias) factors and structural re-lations and thereby improve the likelihood ofadequate model�data fit because it does notinappropriately impose nontarget loadings to beconstrained to zero (Asparouhov & Muthen,2009; Marsh et al., 2009). An emerging body ofresearch has supported the superiority of ESEMfor the examination of psychological constructssuch as the “Big Five” (e.g., Marsh, Ludtke, etal., 2010), student teacher evaluations (Marsh etal., 2009), motivation and engagement (Marsh,Liem, Martin, Morin, & Nagengast, 2011), bul-lying and victimization (Marsh, Nagengast, etal., 2011), physical self (Morin & Maıano,2011), and coaching efficacy (Myers, Chase,Pierce, & Martin, 2011). As a secondary objec-tive, therefore, we also sought to examine theutility of ESEM in offering a viable alternativeto assessing model�data fit of the MTQ 48(Clough et al., 2002) than the restrictive CFAapproach. We hypothesized that ESEM wouldresult in more favorable indices of model fit andmeasurement properties (e.g., factor correla-tions) than CFA.
203MEASURING MENTAL TOUGHNESS
This
doc
umen
t is c
opyr
ight
ed b
y th
e A
mer
ican
Psy
chol
ogic
al A
ssoc
iatio
n or
one
of i
ts a
llied
pub
lishe
rs.
This
arti
cle
is in
tend
ed so
lely
for t
he p
erso
nal u
se o
f the
indi
vidu
al u
ser a
nd is
not
to b
e di
ssem
inat
ed b
road
ly.
Methods
Participants
Participants from two achievement contextswere recruited to participate in the currentstudy.1 A total of 686 athletes (men � 354,women � 328, missing � 4) aged 17�47 years(M � 19.79, SD � 3.27) participated. Thesports represented included a variety of team(team � 466; e.g., basketball, hockey, netball,rugby) and individual sports (individual � 209;e.g., tennis, athletics, triathlon); 11 participantsdid not report their sport. At the time of com-pleting the questionnaire package, these athleteshad been competing in their sport for betweenone and 35 years (M � 9.11, SD � 4.41).Athletes’ highest level of participation includedinternational (9%), national (26%), state orcounty (25%), or district or local (39%) com-petition; a small portion (1%) did not reporttheir playing level.
A total of 639 full-time employees (men �369, women � 269, missing � 1) aged 20�65years (M � 30.02, SD � 8.69) participated.They were employed primarily in the informa-tion technology and communications (30%), ed-ucation (16%), business or finance (16%),health care (11%), academia or research (10%),and consumer or retail (10%) industries. At thetime of completing the survey, participants hadbeen engaged in their current role (M � 4.77,SD � 5.34) and industry (M � 6.31,SD � 6.26) for between 0 and 36 years.Participants’ highest level of education in-cluded an associate degree (n � 36), bache-lor’s degree (n � 426), master’s degree (n �152), and doctorate (n � 25).
Measure
Mental Toughness Questionnaire 48.The MTQ 48 (Clough et al., 2002) is a 48-itemscale that was developed to assess the 4Csmodel of mental toughness, comprising fourkey dimensions or subscales as defined by thedevelopers of the MTQ 48, namely, Control(e.g., “I generally feel in control”), Commit-ment (e.g., “I generally try to give 100%”),Challenge (e.g., “I usually enjoy a challenge”),and Confidence (e.g., “I am generally confidentin my own abilities”). Participants rated them-selves on a scale from 1 (strongly disagree) to 5
(strongly agree). There is some evidence for itsfactorial validity in nonsports contexts (e.g.,Horsburgh et al., 2009), and satisfactory inter-nal reliability and construct validity within sportcontexts (see Table 1).
Procedures
Participants completed an online survey con-taining the aforementioned measure at a timeand place most convenient to them. Studentathletes enrolled in undergraduate courses inpsychology and sports-related subjects at uni-versities in Australia and the United Kingdomwere invited to participate; Australian studentswere offered course credit for their participa-tion. For Australian students, study invitationswere distributed using an established researchparticipation scheme or during the first week ofclasses during a lecture. Email invitations fromcourse coordinators were distributed to studentsin the United Kingdom. The workplace samplewas recruited using an online survey panel (So-cialSci) that links researchers with participantswho have agreed to complete surveys for aca-demic research. Prior to completing the pack-age, all participants were assured of confidenti-ality and anonymity in responses, and informedof their right to withdraw participation at anytime before obtaining their consent. Ethics ap-proval was obtained from both coordinating in-stitutions prior to the commencement of datacollection.
Data Analyses
Our data analyses involved two stages. In thefirst stage, we screened the data for missingresponses in SPSS, Version 18.0 (SPSS Inc.). Inthe second stage, we examined the degree ofmodel�data fit of the MTQ 48 (Clough et al.,2002) using both CFA and ESEM.2 Both factorvalidity analyses were performed withMplus 6.12 (Muthen & Muthen, 2010). Weemployed the robust maximum likelihood esti-mator (MLR), which produces standard errors
1 Following reviewer recommendations, we collected ad-ditional data in an attempt to alleviate limitations associatedwith inadequate statistical power (i.e., increase our athletesample size) and the robustness of the findings (i.e., acrosstwo achievement contexts).
2 We thank an anonymous reviewer for the recommen-dation of ESEM.
204 GUCCIARDI, HANTON, AND MALLETT
This
doc
umen
t is c
opyr
ight
ed b
y th
e A
mer
ican
Psy
chol
ogic
al A
ssoc
iatio
n or
one
of i
ts a
llied
pub
lishe
rs.
This
arti
cle
is in
tend
ed so
lely
for t
he p
erso
nal u
se o
f the
indi
vidu
al u
ser a
nd is
not
to b
e di
ssem
inat
ed b
road
ly.
and tests of fit that are robust in relation tononnormality of observations and the use ofcategorical variables when there are at least fouror more response categories (e.g., Beauducel &Herzberg, 2006; Dolan, 1994; Muthen & Ka-plan, 1985). With regard to CFA, it was hypoth-esized that responses to the MTQ 48 would beexplained by four correlated factors in whicheach item would load on one factor only (i.e.,nonzero loading on its intended factor, withzero loadings on all other factors) and errorterms would be uncorrelated. With regard toESEM, it was hypothesized that responses tothe MTQ 48 would be explained by four corre-lated factors in which each item would show astatistically significant loading on its intendedfactor as well as a small, nonsignificant loadingon the other factors. As recommended (e.g.,Marsh et al., 2009; Marsh, Ludtke, et al., 2010),we used an oblique geomin rotation3 (the de-fault in Mplus) with an epsilon value of 0.5 forESEM. Default constraints that are built into theMplus estimation process to achieve modelidentification were employed (for further de-tails, see Asparouhov & Muthen, 2009; Marsh,Liem, et al., 2011).
In addition to the chi-square goodness-of-fitstatistic, several other traditional criteria (com-parative fit index [CFI] and Tucker�Lewis in-dex [TLI] � .90; root mean square error ofapproximation [RMSEA] scores and standard-ized root-mean-square residual [SRMR] � .08;Browne & Cudeck, 1992) were adopted as in-dicators of adequate model�data fit with Huand Bentler’s (1999) criteria (CFI and TLI�.95, and RMSEA and SRMR scores � .06) asevidence of good fit. Collectively, these indicesprovide a more conservative and comprehen-sive evaluation of model fit than any singleindex alone. However, caution has been urgedin the strict adherence to such recommendationsin psychometric evaluations of measures com-prising 50 or more items loading onto five ormore factors (e.g., Marsh, Hau, & Grayson,2005), and their relevance to ESEM is not en-tirely clear (Marsh et al., 2009; Marsh, Ludtke,et al., 2010). Thus, we also examined standard-ized solutions to evaluate the significance andstrength of parameter estimates. Standardizedfactor loadings were interpreted using Comreyand Lee’s (1992) recommendations (i.e., �.71 � excellent, � .63 � very good, � .55 �good, � .45 � fair, �. 32 � poor). Finally, a
composite reliability coefficient (Raykov, 1997)was calculated to estimate the level of internalreliability for each factor, because there arelimitations associated with Cronbach’s equation(Bentler, 2009; Sijtsma, 2009). With regard tothe ESEM findings, items that evidenced a sta-tistically significant loading on a latent factor atthe p � .01 level were included in the assess-ment of composite reliability.
Results
Preliminary Analyses
The amount of missing data was negligible(athletes � .02%, workplace � .01%) andtherefore were replaced using the expectation-maximization method prior to the factorial va-lidity analyses (Graham, 2009).
Factorial Validity Analyses
Athlete sample. The CFA revealed thatthe hypothesized correlated four factor modelof the MTQ 48 was unsatisfactory, according tothe multiple indices of model fit, �2(1074) �5511.88, p � .001, CFI � .487, TLI � .462,SRMR � .104, RMSEA � .078, 90% confi-dence interval [CI] [.076, .080]. In addition tothe poor model fit, the solution was improper, asindicated by a factor correlation between theControl and Confidence dimensions that ex-ceeded 1.0. The factor loadings, and factor cor-relations and composite reliabilities are detailedin Tables 2 and 3, respectively. All factors dem-onstrated an adequate level of internal reliabil-ity (i.e., composite reliability � .70), with theexception of Control. Collectively, CFA modelfit indices and parameters estimates did not sup-port the hypothesized correlated four factormodel of the MTQ 48 with the athlete sample.
The ESEM revealed that the hypothesized cor-
3 Although a geomin rotation with an epsilon value of 0.5is consistent with previous applications of ESEM (e.g.,Marsh et al., 2009; Marsh, Nagengast et al., 2011), anequally strong case could have been made for a targetrotation in which the analyst freely estimates a priori factorloadings and specifies cross-loadings with a target value ofzero (for further details, see Asparouhov & Muthen, 2009;Browne, 2001). As recommended (Morin & Maıano, 2011),therefore, we also explored solutions based on this alterna-tive rotation procedure. The results of these analyses can beobtained from coauthor Gucciardi.
205MEASURING MENTAL TOUGHNESS
This
doc
umen
t is c
opyr
ight
ed b
y th
e A
mer
ican
Psy
chol
ogic
al A
ssoc
iatio
n or
one
of i
ts a
llied
pub
lishe
rs.
This
arti
cle
is in
tend
ed so
lely
for t
he p
erso
nal u
se o
f the
indi
vidu
al u
ser a
nd is
not
to b
e di
ssem
inat
ed b
road
ly.
Table 2Standardized Parameter Estimates for the CFA and ESEM of the MTQ 48 With the Athlete Sample(n � 686)
Factor 1(Challengesubscale)
Factor 2(Commitment
subscale)Factor 3
(Control subscale)
Factor 4(Confidence
subscale)
ESEM (R2) CFA (R2)ESEM CFA ESEM CFA ESEM CFA ESEM CFA
Mt4 .23� .55�� �.04 .18 .33� .29�� .31��
Mt6 .07 .12� .31�� �.23� .24�� .24�� .02Mt14 .13� .12� .31�� �.23�� .17� .21�� .01Mt23 .47�� .69�� �.03 .19 .22 .43�� .47��
Mt30 .37�� .60�� �.18 .24 .14 .35�� .37��
Mt40 .29� .45�� �.03 .14 .06 .15�� .20��
Mt44 .20 .59�� �.06 .17 .43�� .34�� .35��
Mt48 .44�� .67�� �.22�� .12 .30 .45�� .45��
Mt1 .30�� �.03 .38�� .00 .20 .16� .14��
Mt7 .22� �.19�� .51�� .15� .37�� .30�� .26��
Mt11 �.17�� .46�� .32�� .05 .34� .40�� .10�
Mt19 .19� �.28� .47�� .21� .33�� .32�� .22��
Mt22 �.13� .14 .39�� �.07 .51�� .30�� .15��
Mt25 .12 �.12 .58�� .12 .47�� .30�� .34��
Mt29 .11 .13 .50�� �.07 .46�� .29�� .25��
Mt35 �.29�� .24 .26�� .02 .44�� .31�� .07�
Mt39 .07 �.17 .57�� .28�� .48�� .39�� .33��
Mt42 .03 .29 .39�� �.05 .50�� .41�� .24��
Mt47 .04 .25� .36�� .06 .27�� .19�� .13��
Mt2 .58�� .04 .08 .58�� .05 .41�� .34��
Mt5 .11 �.13 .50�� .49�� .13 .38�� .24��
Mt9 �.20�� .31�� �.12� �.07 .23 .22�� .01Mt12 .60�� .14�� .17�� .66�� .00 .49�� .44��
Mt15 �.02 .45�� �.07 .15�� .25 .33�� .02Mt33 .10 .33�� �.17 .04 �.01 .14�� .00Mt41 .21�� .41�� .11 .46�� .23 .38�� .21��
Mt21 .14 .40�� .00 .16�� �.07 .17� .03Mt26 .00 .07 �.58�� �.30�� .17�� .35�� .09��
Mt27 .10 .53�� �.09 .13�� �.03 .29�� .23��
Mt31 .35� �.07 .12 .48�� .21 .26�� .02Mt34 .08 �.33�� �.35�� �.16�� .17 .19 .02Mt37 �.20 .26� .00 .10 .41�� .28�� .01Mt45 .39�� .13 .23� .55�� .06 .31�� .31��
Mt3 .59�� .09 .20� .00 .67�� .49�� .45��
Mt8 .44�� .10� .36�� .07 .70�� .49�� .48��
Mt10 �.07 .50�� �.07 .07 .05 .28�� .00Mt13 .68�� .03 �.02 .00 .55�� .45�� .31��
Mt16 .55�� .09 .07 .10 .59�� .40�� .35��
Mt18 .20�� .49�� .00 .19� .39�� .39�� .15��
Mt24 .17� .15 .16� �.19 .20�� .09 .04�
Mt32 .15 .51�� �.01 .10 .29�� .33�� .08��
Mt36 .09 .54�� .04 .06 .25�� .32�� .06�
Mt17 �.03 .08 .67�� .01 .45�� .43�� .20��
Mt20 .20�� �.14 .45�� .17� .54�� .40�� .29��
Mt28 �.02 .38�� .20�� .04 .23�� .18�� .05Mt38 .08 .10 .54�� .05 .51�� .35�� .26��
Mt43 �.08 .03 .61�� .02 .38�� .35�� .14��
Mt46 �.09 .35�� .32�� .12 .28�� .24�� .08��
Note. CFA � confirmatory factor analysis; ESEM � exploratory structural equation modeling; MTQ 48 � MentalToughness Questionnaire 48; Mt � item number of the MTQ 48.� Statistically significant parameter estimates at p � .05. �� statistically significant parameter estimates at p � .01.
206 GUCCIARDI, HANTON, AND MALLETT
This
doc
umen
t is c
opyr
ight
ed b
y th
e A
mer
ican
Psy
chol
ogic
al A
ssoc
iatio
n or
one
of i
ts a
llied
pub
lishe
rs.
This
arti
cle
is in
tend
ed so
lely
for t
he p
erso
nal u
se o
f the
indi
vidu
al u
ser a
nd is
not
to b
e di
ssem
inat
ed b
road
ly.
related four factor model of the MTQ 48 wasunsatisfactory, according to the multiple indices ofmodel fit, �2(942) � 2970.25, p � .001, CFI �.766, TLI � .719, SRMR � .045, RMSEA �.056, 90% CI [.054, .058]. The factor loadings,and factor correlations and composite reliabili-ties are detailed in Tables 2 and 3, respectively.All factors demonstrated an adequate level ofinternal reliability (i.e., composite reliability �.70), with the exception of Factor 3. Inspectionof the factor loadings revealed a large degree ofinconsistency between the hypothesized struc-ture, according to the correlated four factormodel proposed by Clough et al. (2002), and thecurrent data. Collectively, ESEM model fit in-dices and parameters estimates did not supportthe hypothesized correlated four factor model ofthe MTQ 48 with the athlete sample.
Workplace sample. The CFA revealedthat the hypothesized correlated four factormodel of the MTQ 48 was unsatisfactory, ac-cording to the multiple indices of model fit,�2(1074) � 4928.95, p � .001, CFI � .521,TLI � .497, SRMR � .093, RMSEA � .075,90% CI [.073, .077]. In addition to the poormodel fit, the solution was improper (i.e., notpositive definite), as indicated by a factor cor-relation between the Control and Confidencedimensions that approached 1.0. The factor cor-relations and composite reliabilities, and factorloadings are detailed in Tables 3 and 4, respec-
tively. Collectively, CFA model fit indices andparameters estimates did not support for hy-pothesized correlated four factor model of theMTQ 48 with the workplace sample.
The ESEM revealed that the hypothesized cor-related four factor model of the MTQ 48 wasunsatisfactory, according to the multiple indices ofmodel fit, �2(942) � 2744.20, p � .001, CFI �.776, TLI � .732, SRMR � .045, RMSEA �.055, 90% CI [.052, .057]. The factor correlationsand composite reliabilities, and factor loadingsare detailed in Tables 3 and 4, respectively. Allfactors demonstrated an adequate level of inter-nal reliability (i.e., composite reliability � .70),with the exception of Factor 1. Collectively,ESEM model fit indices and parameters esti-mates did not support the hypothesized corre-lated four factor model of the MTQ 48 with theworkplace sample.4
4 We also tested a variety of other MTQ 48 models thathave appeared in the literature, such as the six (e.g., Crust &Azadi, 2009, 2010) and nine factor models (e.g., Horsburghet al., 2009). Both CFA and ESEM revealed that thesemodels were unsatisfactory, according to the multiple cri-teria of model fit. The results of these analyses can beobtained from coauthor Gucciardi.
Table 3Latent Factor Correlations and Composite Reliability Estimates for the CFA and ESEM of the MTQ 48
ESEM CFA
Factor 1 Factor 2 Factor 3 Factor 4
Factor 1(Challengesubscale)
Factor 2(Commitment
subscale)
Factor 3(Controlsubscale)
Factor 4(Confidence
subscale)
Athlete sample (n � 686)
Factor 1 (.73) (.75)Factor 2 .02 (.72) .68�� (.74)Factor 3 .38�� �.06 (.50) .85�� .67�� (.50)Factor 4 .28�� .25�� .20�� (.76) .81�� .69�� 1.01�� (.77)
Workplace sample (n � 639)
Factor 1 (.39) (.71)Factor 2 .23�� (.79) .67�� (.78)Factor 3 .17�� .27�� (.74) .73�� .90�� (.67)Factor 4 .38�� .13�� .10� (.71) .78�� .76�� .96�� (.79)
Note. Composite reliability estimates are enclosed in parentheses. CFA � confirmatory factor analysis; ESEM �exploratory structural equation modeling; MTQ 48 � Mental Toughness Questionnaire 48.� Statistically significant parameter estimates at p � .05. �� statistically significant parameter estimates at p � .01.
207MEASURING MENTAL TOUGHNESS
This
doc
umen
t is c
opyr
ight
ed b
y th
e A
mer
ican
Psy
chol
ogic
al A
ssoc
iatio
n or
one
of i
ts a
llied
pub
lishe
rs.
This
arti
cle
is in
tend
ed so
lely
for t
he p
erso
nal u
se o
f the
indi
vidu
al u
ser a
nd is
not
to b
e di
ssem
inat
ed b
road
ly.
Table 4Standardized Parameter Estimates for the CFA and ESEM of the MTQ 48 With the Workplace Sample(n � 639)
Factor 1(Challengesubscale)
Factor 2(Commitment
subscale)Factor 3
(Control subscale)
Factor 4(Confidence
subscale)
ESEM (R2) CFA (R2)ESEM CFA ESEM CFA ESEM CFA ESEM CFA
Mt4 .47� .55�� �.05 .06 .20 .34�� .30��
Mt6 .06 .37�� .35�� .12� .08 .20�� .14��
Mt14 �.07 .26�� .18 .39�� .01 .21�� .07��
Mt23 .38 .57�� .26�� �.04 .15 .33�� .32��
Mt30 .28 .54�� .01 .10 .29 .25�� .39��
Mt40 .18 .34�� .12 �.20�� .21 .14�� .11��
Mt44 .40 .65�� �.07 .08 .37 .42�� .42��
Mt48 .47 .61�� .06 .01 .19 .35�� .38��
Mt1 .54 �.07 .42�� .10 .06 .33 .18��
Mt7 .36 �.08 .40�� .19�� .16 .24�� .16��
Mt11 .01 .37�� .53�� .34�� .06 .34�� .28��
Mt19 .39� �.16� .48�� .32�� .14 .33�� .23��
Mt22 .20 .33�� .54�� .25�� �.08 .29�� .29��
Mt25 .49 �.02 .43�� .07 .06 .27 .18��
Mt29 .18 .01 .61�� .63�� .01 .47�� .38��
Mt35 .10 .36�� .46�� .27�� �.15 .28�� .21��
Mt39 .34�� .10 .46�� .07 .19 .26�� .21��
Mt42 .15 .12 .63�� .62�� �.05 .50�� .39��
Mt47 .01 .05 .48�� .53�� .10 .32�� .23��
Mt2 .36 .17 �.01 .49�� .22 .30�� .24��
Mt5 .24�� �.06 .05 .37�� .38 .27�� .14��
Mt9 .10 .05 .61�� .47�� �.05 .41�� .22��
Mt12 .30 .21�� �.04 .44�� .21 .26�� .20��
Mt15 �.04 .26�� .46�� .42�� �.02 .33�� .17��
Mt33 �.08 .06 .53�� .33�� .02 .29�� .11��
Mt41 .16 .08 .57�� .54�� �.01 .42�� .29��
Mt21 �.04 .47�� .16�� .36�� �.04 .28�� .13��
Mt26 .30 .08 .10 .01 �.55 .29 .00Mt27 �.07 .61�� .01 .35�� �.02 .36�� .12��
Mt31 .43 .33�� �.08 .45�� .03 .35�� .20��
Mt34 .28 .02 �.32�� �.18�� �.30 .20� .03�
Mt37 .12 .39�� .21�� .40�� �.13 .27�� .16��
Mt45 .19 .36�� �.11� .45�� .31 .35�� .21��
Mt3 .36 .03 .24�� .21 .58�� .34�� .33��
Mt8 .39�� .10 .07 .30�� .58�� .39�� .34��
Mt10 �.06 .50�� .02 .02 .34�� .24�� .11��
Mt13 .33 .22� �.06 .13 .46�� .24�� .21��
Mt16 .45 .28�� �.12� .08 .51�� .36�� .26��
Mt18 .19 .44�� .16�� .02 .54�� .35�� .29��
Mt24 .06 .52�� �.38�� .11 .26�� .34�� .07��
Mt32 �.07 .33�� .34�� �.07 .33�� .27�� .11��
Mt36 �.06 .62�� .13 .08 .48�� .45�� .23��
Mt17 �.04 .04 �.03 .67�� .40�� .44 .16��
Mt20 .26 �.14�� .11�� .46�� .44�� .39�� .19��
Mt28 �.03 .31�� .27�� .23� .50�� .29�� .24��
Mt38 .12 .17�� �.07 .52�� .48�� .37�� .23��
Mt43 �.06 �.04 �.05 .62�� .28�� .36�� .08��
Mt46 �.15 .10 .57�� .38�� .50�� .51�� .25��
Note. CFA � confirmatory factor analysis; ESEM � exploratory structural equation modeling; MTQ 48 � MentalToughness Questionnaire 48; Mt � item number of the MTQ 48.� Statistically significant parameter estimates at p � .05. �� statistically significant parameter estimates at p � .01.
208 GUCCIARDI, HANTON, AND MALLETT
This
doc
umen
t is c
opyr
ight
ed b
y th
e A
mer
ican
Psy
chol
ogic
al A
ssoc
iatio
n or
one
of i
ts a
llied
pub
lishe
rs.
This
arti
cle
is in
tend
ed so
lely
for t
he p
erso
nal u
se o
f the
indi
vidu
al u
ser a
nd is
not
to b
e di
ssem
inat
ed b
road
ly.
Discussion
The MTQ 48 (Clough et al., 2002) has beenthe most popular measure for many researchersinterested in examining mental toughness overthe past decade. Despite the increasing popular-ity of the MTQ 48 as a measure of mentaltoughness, its factor structure has not yet beensubjected to a rigorous psychometric analysis inan athletic or a workplace sample. Both CFAand ESEM failed to support the hypothesizedcorrelated four factor model (i.e., Control,Commitment, Challenge, and Confidence) ofmental toughness in two independent samples,thereby revealing incongruence between theMTQ 48 and its measurement of the underlyingtheoretical model.
Although evidence of concurrent validity ofthe MTQ 48 is amassing (see Table 1), evidenceof nomological validity relies on a sound inter-nal structure (i.e., factorial validity, internal re-liability; Gignac, 2009). Some researchers havegone so far to say that:
. . . the representational effectiveness [correspondencebetween indicators and hypothesized constructs] of atest is in fact far more important to a scale’s scientificvalue [than predictive validity]. . . If the meaning of thescores on a scale is unclear, the accuracy of any infer-ences about constructs made on the basis of that scaleis in doubt. (McGrath, 2005, p. 113)
Additionally, correlational analyses involv-ing observed variables have been the primarymeans by which researchers have sought toestablish the construct validity of the MTQ 48.However, limitations associated with these tra-ditional analyses mean they are suboptimal forexaminations of theoretical models includingmultiple latent constructs. For example, rela-tionships between multiple antecedent (e.g.,stress), intervening (e.g., mental toughness),and outcome (e.g., behavior, performance) vari-ables cannot be simultaneously estimated, andparameter estimates do not take measurementerror into consideration (i.e., observed variablesare assumed to be measured without error).Structural equation modeling is ideally suitedfor such substantive inquiries, yet they requirepsychometrically sound instruments (Byrne,2010).
The utility of an instrument for research (e.g.,validity of conclusions, scoring, defining sub-scales), theory (e.g., dimensionality, hierarchi-cal representation), and practice (e.g., appropri-
ateness for different populations) is under-pinned by the degree to which that measurevalidly captures the construct in its intendedmanner (Gignac, 2009; Marsh, Martin, et al.,2010; McGrath, 2005). Preliminary researchwith individuals from the general populationhas supported the adequacy of hypothesizedfour factor model of the MTQ 48 (Horsburgh etal., 2009). Despite these initial findings fromnonsport contexts, researchers (e.g., Con-naughton & Hanton, 2009; Gucciardi et al.,2011) have expressed both empirical (i.e., de-tailed information on the scale constructionprocess and factorial validity is unavailable;inadequate internal reliability) and conceptualconcerns (i.e., 75% of the underlying model ishardiness theory, little information on the ratio-nale for the underlying theoretical model) withthe MTQ 48 as a measure of mental toughness.Aligned with these concerns, but contrary toprevious factor analytical research (Horsburghet al., 2009), empirical evidence detailed hereraises questions about the viability of the cor-related four factor model hypothesized to un-derpin the MTQ 48. Although there is someevidence to support the equivalency of the fac-tor structure or latent mean structures of psy-chometric tools collected via online or tradi-tional paper and pencil methods (e.g., Lonsdale,Hodge, & Rose, 2006), the difference in datacollection methods between the studies (i.e.,online here vs. hardcopy with Horsburgh et al.,2009) may provide an explanation for the dis-crepancy in findings.
Even when a psychological instrument has awell-defined EFA structure, its psychometricintegrity sometimes fails to replicate within ahighly restrictive CFA framework (Marsh et al.,2009). The CFA findings of the current studyare consistent with this general observation, be-cause EFA has been employed as the focalanalysis in the original development of theMTQ 48 (Clough et al., 2002) and its onlysubsequent psychometric evaluation (Hors-burgh et al., 2009). ESEM has recently beenproposed as a flexible alternative when there isinsufficient theory to guide a strictly confirma-tory approach (e.g., Asparouhov & Muthen,2009; Marsh et al., 2009; Myers, Chase et al.,2011). Consistent with an emerging body ofresearch from both nonsport (e.g., Marsh et al.,2009; Marsh, Nagengast et al., 2011) and sportcontexts (e.g., Morin & Maıano, 2011; Myers,
209MEASURING MENTAL TOUGHNESS
This
doc
umen
t is c
opyr
ight
ed b
y th
e A
mer
ican
Psy
chol
ogic
al A
ssoc
iatio
n or
one
of i
ts a
llied
pub
lishe
rs.
This
arti
cle
is in
tend
ed so
lely
for t
he p
erso
nal u
se o
f the
indi
vidu
al u
ser a
nd is
not
to b
e di
ssem
inat
ed b
road
ly.
Chase et al., 2011), the current study supportedthe superiority of ESEM when compared withCFA in terms of model fit indices and parame-ters estimates. Nevertheless, ESEM failed tosupport the hypothesized correlated four factormodel of mental toughness proposed to encap-sulate the MTQ 48 (Clough et al., 2002) in boththe athlete and the workplace samples, as evi-denced by a significant degree of model misfitand inconsistent parameter estimates (i.e., non-significant loadings for hypothesized indicator-�factor relationships, significant item cross-loadings on unintended factors, statistically sig-nificant indicator�factor associations in anonhypothesized direction) in both the athleteand the workplace samples.
Conceptual and empirical limitations with thecontent of several items on the MTQ 48 mayhave contributed to the empirical problems ob-served in the current study. With regard to CFA,empirical weaknesses were observed at the in-dividual item level. For example, only 14 (29%)of the 48 items evidenced very good to excellentloadings on their hypothesized factor (i.e., �.55) in both the athlete and the workplace sam-ples. With regard to the athlete sample, perhapsmost troubling is that 17 (35%) of the 48 itemswere very poor, in that they did not load morethan .32 on their hypothesized factor. ESEMalso identified a number of problematic items inboth samples (e.g., nonsignificant loadings onintended factors according to hypothesizedmodel, statistically significant associations in anonhypothesized direction). Problems at theitem level might indicate issues with partici-pants’ comprehension of the meaning of an itemin the context of its hypothesized subscale.From a conceptual standpoint, the Confidencesubscale, for example, contains several itemswhose content is not entirely consistent with a“high sense of self-belief” (Clough et al., 2002,p. 38). Items such as “I generally feel that I ama worthwhile person” and “At times I feel com-pletely useless” appear to be capturing aspectsof one’s self-esteem (i.e., self-evaluation or ap-praisal of one’s own worth; Harter, 1999),whereas other items such as “However badthings are, I usually feel they will work outpositively in the end” and “I generally look onthe bright side of life” are capturing one’s op-timistic outlook on life (i.e., tendency to per-ceive, react, and adapt to challenges in one’s lifein a positive manner; Scheier & Carver, 1985).
Collectively, our empirical data and conceptualconcerns detailed here and elsewhere (e.g.,Connaughton & Hanton, 2009; Gucciardi et al.,2011) support the idea that the entire frameworkof the MTQ 48 needs to be reconsidered andthat the magnitude of the problem is beyond forwhat post hoc modifications are intended. Asone of the most neglected stages of scale devel-opment, it is important that a clear articulationof the construct domain—both what is intendedto be captured as well as how it can be distin-guished from related constructs—is providedfrom the outset to reduce the likelihood of po-tential problems at later stages in the validationprocess (MacKenzie et al., 2011).
The acquisition of knowledge in a particulartopic area is in part dependent on psychometri-cally sound measures or inventories. Instru-ments that are theoretically derived and psycho-metrically sound according to multiple criteria(for reviews, see Gignac, 2009; Gucciardi et al.,2011) underpin substantive research questionsand theory development. Thus, it is important toascertain this type of validity before otherforms, such as predictive and concurrent valid-ity. According to Gignac, “. . .factorial validityis crucially important to the validity enterprise,as it helps determine what composite scoresderived from an inventory measure from a di-mensional perspective, or more specifically,how many dimensions are measured by thescores of an inventory?” (2009, p. 25). Both theCFA and ESEM findings of the current studycast doubt on the notion that the MTQ 48 ade-quately captures the 4Cs of mental toughness inathlete and workplace samples (Clough et al.,2002; Horsburgh et al., 2009). In particular, thelack of support in the workplace sample is aconcern because much of the original hardinessresearch came from a 12-year longitudinal studyof managers at a telephone company (cf. Maddi& Kobasa, 1984), and one would have expectedthose data to align well with the hypothesizedmodel. Although traditional psychometric eval-uations (e.g., internal consistency, predictivevalidity) are important, they are not sufficient indetermining the factorial validity (i.e., dimen-sionality, strength and direction of parameters)of the indicators. In this study, we have alsodemonstrated the potential problems of usinginternal reliability estimates (e.g., composite re-liability) as the sole indicator of an instrument’sappropriateness in a given sample. Thus, al-
210 GUCCIARDI, HANTON, AND MALLETT
This
doc
umen
t is c
opyr
ight
ed b
y th
e A
mer
ican
Psy
chol
ogic
al A
ssoc
iatio
n or
one
of i
ts a
llied
pub
lishe
rs.
This
arti
cle
is in
tend
ed so
lely
for t
he p
erso
nal u
se o
f the
indi
vidu
al u
ser a
nd is
not
to b
e di
ssem
inat
ed b
road
ly.
though preliminary and requiring replication inindependent samples, the current findings ques-tion some of the conclusions of previous re-search using the MTQ 48 as its central measureof mental toughness (cf. Hogan & Nicholson,1998).
Study Limitations
The key strength of this study is the use ofrigorous statistical analyses that accounted formeasurement error and the inclusion of moder-ately sized samples of performers from twoachievement contexts (i.e., sports and the work-place). Nevertheless, the results of the currentstudy should also be interpreted within thecontext of study limitations, which included areliance on self-reported mental toughness, across-sectional design, and online survey meth-odology. Future research that integrates multi-ple methods of assessment (e.g., self and coachor supervisor ratings), experimental manipula-tions, applies prospective longitudinal designs,and/or examines the impact of survey adminis-tration (i.e., online vs. paper-and-pencil, com-mon method bias) and nonindependence ofobservations (e.g., inclusion of team sports ath-letes) on factorial validity would prove fruitfulin addressing these concerns. It is particularlyimportant that the results of the current studyare verified on a larger sample of performers.Myers, Ahn, and Ying (2011) have recentlydemonstrated the usefulness of Monte Carlomethods for making an a priori decision aboutthe required sample size or for estimating thepower of a given sample size post hoc (see alsoMyers, Chase et al., 2011). Owing to the largedegree of model misfit, it was deemed inappro-priate to pursue a post hoc assessment of thepower with our sample using Monte Carlomethods (cf. Hancock, 2006), thereby suggest-ing the need for future research with even largersamples than the current study.
Conclusion
The current findings provide preliminary ev-idence that the psychometric properties of theMTQ 48 may not be adequate, particularly withrespect to its hypothesized underlying concep-tual model. In light of the current findings, it isimportant that researchers report empirical datapertaining to the factorial validity of the
MTQ 48 when this tool has been employed ameasure of mental toughness, as well as com-plete validation studies to address the within-network measurement issues (i.e., factorial va-lidity, internal reliability) revealed in both ourathlete and workplace samples. Until evidenceto support the factorial validity of the MTQ 48has been shown, researchers and practitionersshould proceed with caution when using theMTQ 48 as a measure of mental toughness.More broadly for the future of mental toughnessmeasurement research, the current findingshighlight the importance of having a clearlyarticulated definition and conceptual model thatunderpins item development, as well as apply-ing rigorous statistical procedures to purify andrefine the item pool before the underlying modelis cross-validated and assessed for its validity(cf. MacKenzie et al., 2011). Nevertheless, it isimportant to acknowledge that a number of con-ceptual and rhetoric debates still exist as to whatmental toughness is and of what it is made up(Connaughton & Hanton, 2009; for recent re-views, see Gucciardi & Gordon, 2011). Thus,because mental toughness is a construct with asparse and diverse theoretical and empirical no-mological network, it might be more prudent forresearchers to pursue a common understandingto formulate a consensual definition and theprimary facets that do and do not belong to it,rather than taking a statistical approach to val-idate a specific measurement model.
References
Asparouhov, T., & Muthen, B. (2009). Exploratory struc-tural equation modeling. Structural Equation Model-ing, 16, 397–438. doi:10.1080/10705510903008204
Beauducel, A., & Herzberg, P. Y. (2006). On theperformance of maximum likelihood versus meansand variance adjusted weighted least squares esti-mation in CFA. Structural Equation Modeling, 13,186–203. doi:10.1207/s15328007sem1302_2
Bentler, P. M. (2009). Alpha, dimension-free, and model-based internal consistency reliability. Psy-chometrika, 74, 137–143. doi:10.1007/s11336-008-9100-1
Browne, M. W. (2001). An overview of analyticrotation in exploratory factor analysis. Multivari-ate Behavioral Research, 36, 111–150. doi:10.1207/S15327906MBR3601_05
Browne, M. W., & Cudeck, R. (1992). Alternativeways of assessing model fit. Sociological Methods
211MEASURING MENTAL TOUGHNESS
This
doc
umen
t is c
opyr
ight
ed b
y th
e A
mer
ican
Psy
chol
ogic
al A
ssoc
iatio
n or
one
of i
ts a
llied
pub
lishe
rs.
This
arti
cle
is in
tend
ed so
lely
for t
he p
erso
nal u
se o
f the
indi
vidu
al u
ser a
nd is
not
to b
e di
ssem
inat
ed b
road
ly.
& Research, 21, 230 –258. doi:10.1177/0049124192021002005
Byrne, B. M. (2010). Structural equation modelingwith AMOS: Basic concepts, applications, andprogramming (2nd ed.). Mahwah, NJ: LawrenceErlbaum Associates.
Chelladurai, P., & Saleh, S. (1978). Preferred lead-ership in sports. Canadian Journal of AppliedSport Sciences, 3, 85–92.
Chesney, M. A., Neilands, T. B., Chambers, D. B.,Taylor, J. M., & Folkman, S. (2006). A validityand reliability study of coping self-efficacy scale.British Journal of Health Psychology, 11, 421–437. doi:10.1348/135910705X53155
Clough, P., Earle, K., & Sewell, D. (2002). Mentaltoughness: The concept and its measurement. In I.Cockerill (Ed.), Solutions in sport psychology (pp.32–45). London, England: Thomson.
Comrey, A. L., & Lee, H. B. (1992). A first course infactor analysis. Hillsdale, NJ: Erlbaum.
Connaughton, D., & Hanton, S. (2009). Mentaltoughness in sport: Conceptual and practical is-sues. In S. D. Mellalieu & S. Hanton (Eds.), Ad-vances in applied sport psychology: A review (pp.317–346). London, England: Routledge.
Crocker, P. R. E., & Graham, T. R. (1995). Copingby competitive athletes with performance stress:Gender differences and relationships with affect.The Sport Psychologist, 9, 325–338.
Crust, L. (2009). The relationship between mentaltoughness and affect intensity. Personality andIndividual Differences, 47, 959–963. doi:10.1016/j.paid.2009.07.023
Crust, L., & Azadi, K. (2009). Leadership prefer-ences of mentally tough athletes. Personality andIndividual Differences, 47, 326–330. doi:10.1016/j.paid.2009.03.022
Crust, L., & Azadi, K. (2010). Mental toughnessathletes’ use of psychological strategies. EuropeanJournal of Sport Science, 10, 43–51. doi:10.1080/17461390903049972
Crust, L., & Keegan, R. (2010). Mental toughnessand attitudes to risk-taking. Personality and Indi-vidual Differences, 49, 164–168. doi:10.1016/j.paid.2010.03.026
Crust, L., Nesti, M., & Littlewood, M. (2010a). A cross-sectional analysis of mental toughness in a professionalfootball academy. Athletic Insight: The Online Journalof Sport Psychology, 2, 165–174.
Crust, L., Nesti, M., & Littlewood, M. (2010b). Player andcoach ratings of mental toughness in an elite associationfootball academy. Athletic Insight: The Online Journalof Sport Psychology, 2, 239–250.
Crust, L., & Swann, C. (2011). Comparing two mea-sures of mental toughness. Personality and Indi-vidual Differences, 50, 217–221. doi:10.1016/j.paid.2010.09.032
Daly, J. M., Brewer, B. W., & Van Raalte, J. L.(1995). Cognitive appraisal, emotional adjustment,and adherence to rehabilitation following knee sur-gery. Journal of Sport Rehabilitation, 4, 23–30.
Dolan, C. V. (1994). Factor analysis of variableswith 2, 3, 5 and 7 response categories: A compar-ison of categorical variable estimators using sim-ulated data. British Journal of Mathematical andStatistical Psychology, 47, 309–326. doi:10.1111/j.2044-8317.1994.tb01039.x
Franken, R., Gibson, K., & Rowland, G. (1992). Sensationseeking and the tendency to view the world as threat-ening. Personality and Individual Differences, 13, 31–38. doi:10.1016/0191-8869(92)90214-A
Gaudreau, P., & Blondin, J. P. (2002). Development of aquestionnaire for the assessment of coping strategiesemployed by athletes in competitive sport settings. Psy-chology of Sport and Exercise, 3, 1–34. doi:10.1016/S1469-0292(01)00017-6
Gignac, G. E. (2009). Psychometrics and the mea-surement of emotional intelligence. In C. Stough,D. H. Saklofske, & J. D. A. Parker (Eds.), Assess-ing emotional intelligence: Theory, research, andapplications (pp. 9–40). New York, NY: Springer.doi:10.1007/978-0-387-88370-0_2
Golby, J., Sheard, M., & van Wersch, A. (2007). Evalu-ating the factor structure of the psychological perfor-mance inventory. Perceptual and Motor Skills, 105,309–325. doi:10.2466/PMS.105.5.309-325
Gottlieb, B. H., & Rooney, J. A. (2004). Copingeffectiveness: Determinants and relevance to themental health and affect of family caregivers ofpersons with dementia. Aging & Mental Health, 8,364–373. doi:10.1080/13607860410001709719
Graham, J. W. (2009). Missing data analysis: Makingit work in the real world. Annual Review of Psy-chology, 60, 549 –576. doi:10.1146/annurev.psych.58.110405.085530
Gucciardi, D. F. (2009). Do developmental differ-ences in mental toughness exist between special-ized and invested Australian footballers? Person-ality and Individual Differences, 47, 985–989. doi:10.1016/j.paid.2009.08.001
Gucciardi, D. F., & Gordon, S. (2009). Development andpreliminary validation of the Cricket Mental ToughnessInventory. Journal of Sports Sciences, 27, 1293–1310.doi:10.1080/02640410903242306
Gucciardi, D. F., & Gordon, S. (Eds.). (2011). Mentaltoughness in sport: Developments in research andtheory. London, England: Routledge.
Gucciardi, D. F., Gordon, S., & Dimmock, J. A.(2009). Development and preliminary validationof a mental toughness inventory for Australianfootball. Psychology of Sport and Exercise, 10,201–209. doi:10.1016/j.psychsport.2008.07.011
Gucciardi, D. F., Mallett, C. J., Hanrahan, S. J., &Gordon, S. (2011). Measuring mental toughness insport: Current status and future directions. In D. F.
212 GUCCIARDI, HANTON, AND MALLETT
This
doc
umen
t is c
opyr
ight
ed b
y th
e A
mer
ican
Psy
chol
ogic
al A
ssoc
iatio
n or
one
of i
ts a
llied
pub
lishe
rs.
This
arti
cle
is in
tend
ed so
lely
for t
he p
erso
nal u
se o
f the
indi
vidu
al u
ser a
nd is
not
to b
e di
ssem
inat
ed b
road
ly.
Gucciardi & S. Gordon (Eds.), Mental toughnessin sport: Developments in research and theory (pp.108–132). London, England: Routledge.
Hagger, M. S., & Chatzisarantis, N. L. D. (2009).Assumptions in research in sport and exercise psy-chology. Psychology of Sport and Exercise, 10,511–519. doi:10.1016/j.psychsport.2009.01.004
Hancock, G. R. (2006). Power analysis in covariancestructure modeling. In G. R. Hancock, & R. O.Mueller (Eds.), Structural equation modeling: Asecond course (pp. 69–115). Greenwich, CT: In-formation Age Publishing.
Harter, S. (1999). The construction of the self. NewYork, NY: Guilford Press.
Hogan, R., & Nicholson, R. A. (1988). The meaning ofpersonality test scores. American Psychologist, 43, 621–626. doi:10.1037/0003-066X.43.8.621
Horsburgh, V. A., Schermer, J. A., Veselka, L., &Vernon, P. A. (2009). A behavioral genetic studyof mental toughness and personality. Personalityand Individual Differences, 46, 100–105. doi:10.1016/j.paid.2008.09.009
Hu, L. T., & Bentler, P. M. (1999). Cutoff criteria forfit indexes in covariance structure analysis: Con-ventional criteria versus new alternatives. Struc-tural Equation Modeling, 6, 1–55. doi:10.1080/10705519909540118
Kaiseler, M., Polman, R., & Nicholls, A. (2009).Mental toughness, stress, stress appraisal, copingand coping effectiveness in sport. Personality andIndividual Differences, 47, 728–733. doi:10.1016/j.paid.2009.06.012
Lane, A. M., Harwood, C., Terry, P. C., & Karageorghis,C. I. (2004). Confirmatory factor analysis of the Test ofPerformance Strategies (TOPS) among adolescent ath-letes. Journal of Sports Sciences, 22, 803–812. doi:10.1080/02640410410001716689
Larsen, R. J. (1984). Theory and measurement ofaffect intensity as an individual difference charac-teristic. Dissertation Abstracts International: Sec-tion B. Sciences and Engineering, 45(7), 2297.
Levy, A. R., Polman, R. C. J., Clough, P. J., March-ant, D. C., & Earle, K. (2006). Mental toughness asa determinant of beliefs, pain, and adherence insport injury rehabilitation. Journal of Sport Reha-bilitation, 15, 246–254.
Loehr, J. E. (1986). Mental toughness training forsports: Achieving athletic excellence. Lexington,MA: Stephen Greene Press.
Lonsdale, C., Hodge, K., & Rose, E. A. (2006).Pixels vs. paper: Comparing online and traditionalsurvey methods in sport psychology. Journal ofSport & Exercise Psychology, 28, 100–108.
MacKenzie, S. B., Podsakoff, P. M., & Podsakoff,N. P. (2011). Construct measurement and valida-tion procedures in MIS and behavioral research:Integrating new and existing techniques. MISQuarterly, 35, 293–334.
Maddi, S. R. (2004). Hardiness: An operationaliza-tion of existential courage. Journal of HumanisticPsychology, 44, 279 –298. doi:10.1177/0022167804266101
Maddi, S. R., & Kobasa, S. C. (1984). The hardyexecutive: Health under stress. Homewood, IL:Dow-Jones Irwin.
Marchant, D. C., Polman, R. C. J., Clough, P. J.,Jackson, J. G., Levy, A. R., & Nicholls, A. R.(2009). Mental toughness: Managerial and age dif-ferences. Journal of Managerial Psychology, 24,428–437. doi:10.1108/02683940910959753
Marsh, H. W. (1997). The measurement of physical self-concept: A construct validation approach. In K. Fox(Ed.), The physical self: From motivation to well-being(pp. 27–58). Champaign, IL: Human Kinetics.
Marsh, H. W., Hau, K.-T., & Grayson, D. (2005).Goodness of fit evaluation in structural equationmodeling. In A. Maydeu-Olivares & J. McArdle(Eds.), Contemporary psychometrics: A festschriftfor Roderick P. McDonald (pp. 275–340). Mah-wah, NJ: Erlbaum.
Marsh, H. W., Liem, G. A. D., Martin, A. J.,Morin, A. J. S., & Nagengast, B. (2011). Meth-odological measurement fruitfulness of explor-atory structural equation modeling (ESEM):New approaches to key substantive issues inmotivation and engagement. Journal of Psy-choeducational Assessment, 29, 322–346. doi:10.1177/0734282911406657
Marsh, H. W., Ludtke, O., Muthen, B., Asparouhov,T., Morin, A. J. S., Trautwein, U., & Nagengast, B.(2010). A new look at the big-five factor structurethrough exploratory structural equation modeling.Psychological Assessment, 22, 471– 491. doi:10.1037/a0019227
Marsh, H. W., Martin, A. J., & Jackson, S. A. (2010).Introducing a short version of the Physical SelfDescription Questionnaire: New strategies, short-form evaluation evaluative criteria, and applica-tions of factor analyses. Journal of Sport & Exer-cise Psychology, 32, 438–482.
Marsh, H. W., Muthen, B., Asparouhov, T., Ludtke,O., Robitzsch, A., Morin, A. J. S., & Trautwein, U.(2009). Exploratory structural equation modeling,integrating CFA and EFA: Application to students’evaluations of university teaching. StructuralEquation Modeling, 16, 439–476. doi:10.1080/10705510903008220
Marsh, H. W., Nagengast, B., Morin, A. S. J., Parada,R. H., Craven, R. G., & Hamilton, L. R. (2011).Construct validity of the multidimensional struc-ture of bullying and victimization: An applicationof exploratory structural equation modeling. Jour-nal of Educational Psychology, 103, 701–732. doi:10.1037/a0024122
Martens, M. P., & Webber, S. N. (2002). Psychometricproperties of the sport motivation scale: An evaluation
213MEASURING MENTAL TOUGHNESS
This
doc
umen
t is c
opyr
ight
ed b
y th
e A
mer
ican
Psy
chol
ogic
al A
ssoc
iatio
n or
one
of i
ts a
llied
pub
lishe
rs.
This
arti
cle
is in
tend
ed so
lely
for t
he p
erso
nal u
se o
f the
indi
vidu
al u
ser a
nd is
not
to b
e di
ssem
inat
ed b
road
ly.
with college varsity athletes from the US. Journal ofSport & Exercise Psychology, 24, 254–270.
McGrath, R. E. (2005). Conceptual complexity and con-struct validity. Journal of Personality Assessment, 85,112–124. doi:10.1207/s15327752jpa8502_02
Meyers, M. C., Bourgeois, A. E., Stewart, S., &LeUnes, A. (1992). Predicting pain response inathletes: Development and assessment of theSports Inventory for Pain. Journal of Sport &Exercise Psychology, 14(3), 249–261.
Morin, A. J. S., & Maıano, C. (2011). Cross-validation ofthe short form of the physical self-inventory (PSI-S)using exploratory structural equation modeling(ESEM). Psychology of Sport and Exercise, 12, 540–554. doi:10.1016/j.psychsport.2011.04.003
Muthen, B. O., & Kaplan, D. (1985). A comparisonof some methodologies for the factor analysis ofnonnormal Likert variables. British Journal ofMathematical and Statistical Psychology, 38, 171–189. doi:10.1111/j.2044-8317.1985.tb00832.x
Muthen, L. K., & Muthen, B. O. (2010). Mplus user’sguide (6th ed.). Los Angeles, CA: Authors.
Myers, N. D., Ahn, S., & Ying, J. (2011). Samplesize and power estimates for a confirmatory factoranalytic model in exercise and sport: A MonteCarlo approach. Research Quarterly for Exerciseand Sport, 82, 412–423.
Myers, N. D., Chase, M. A., Pierce, S. W., & Martin,E. (2011). Coaching efficacy and exploratorystructural equation modeling: A substantive-methodological synergy. Journal of Sport & Exer-cise Psychology.
Nicholls, A. R., Levy, A. R., Polman, R. C. J., &Crust, L. (2011). Mental toughness, coping self-efficacy, and coping effectiveness among athletes.International Journal of Sport Psychology, 42,513�524.
Nicholls, A. R., Polman, R. C. J., Levy, A. R., &Backhouse, S. H. (2008). Mental toughness, opti-mism, pessimism, and coping among athletes. Per-sonality and Individual Differences, 44, 1182–1192. doi:10.1016/j.paid.2007.11.011
Nicholls, A. R., Polman, R. C. J., Levy, A. R., &Backhouse, S. H. (2009). Mental toughness in
sport: Achievement level, gender, age, experience,and sport type differences. Personality and Indi-vidual Differences, 47, 73–75. doi:10.1016/j.paid.2009.02.006
Nunnally, J., & Bernstein, I. (1994). Psychometrictheory. New York, NY: McGraw-Hill.
Raykov, T. (1997). Estimation of composite reliabil-ity for congeneric measures. Applied Psychologi-cal Measurement, 21, 173–184. doi:10.1177/01466216970212006
Scheier, M. F., & Carver, C. S. (1985). Optimism,coping, and health: Assessment and implicationsof generalized outcome expectancies. Health Psy-chology, 4, 219 –247. doi:10.1037/0278-6133.4.3.219
Sheard, M., Golby, J., & van Wersch, A. (2009).Progress toward construct validation of the SportsMental Toughness Questionnaire (SMTQ). Euro-pean Journal of Psychological Assessment, 25,186–193. doi:10.1027/1015-5759.25.3.186
Sijtsma, K. (2009). On the use, the misuse, and thevery limited usefulness of Cronbach’s alpha. Psy-chometrika, 74, 107–120. doi:10.1007/s11336-008-9101-0
Thomas, P. R., Murphy, S. M., & Hardy, L. (1999).Test of performance strategies: Development andpreliminary validation of a comprehensive mea-sure of athletes’ psychological skills. Journal ofSports Sciences, 17, 697–711. doi:10.1080/026404199365560
Thompson, B. (2004). Exploratory and confirmatoryfactor analysis: Understanding concepts and ap-plications. Washington, DC: American Psycholog-ical Association. doi:10.1037/10694-000
Veselka, L., Schermer, J. A., Petrides, K. V., &Vernon, P. A. (2009). Evidence for a heritablegeneral factor of personality in two studies. TwinResearch and Human Genetics, 12, 254–260. doi:10.1375/twin.12.3.254
Received August 23, 2011Revision received December 22, 2011
Accepted December 27, 2011 �
E-Mail Notification of Your Latest Issue Online!
Would you like to know when the next issue of your favorite APA journal will beavailable online? This service is now available to you. Sign up at http://notify.apa.org/ andyou will be notified by e-mail when issues of interest to you become available!
214 GUCCIARDI, HANTON, AND MALLETT
This
doc
umen
t is c
opyr
ight
ed b
y th
e A
mer
ican
Psy
chol
ogic
al A
ssoc
iatio
n or
one
of i
ts a
llied
pub
lishe
rs.
This
arti
cle
is in
tend
ed so
lely
for t
he p
erso
nal u
se o
f the
indi
vidu
al u
ser a
nd is
not
to b
e di
ssem
inat
ed b
road
ly.