Development and Validation of a Scale to Measure Optimal ... · Development and Validation of a...

JOURNAL OF SPORT & EXERCISE PSYCHOLOGY, 1996, 18, 17-35 O 1996 Human Kinetics Publishers, Inc.

Development and Validation of a Scale to Measure Optimal Experience: The Flow State Scale

Susan A. Jackson Herbert W. Marsh University of Queensland University of Western Sydney, Macarthur

The Flow State Scale (FSS) is a new measure of flow in sport and physical activity settings. The nine FSS scales of the 36-item instrument represent the dimensions of flow discussed by Csikszentmihalyi (1990, 1993), and each scale is measured by four items. Development of items was based on (a) past research with flow state both within and outside of sport settings, (b) qualitative analysis of interviews with elite athletes, and (c) quantitative analyses conducted in the present investigation. Internal consistency estimates for the nine FSS scales were reasonable (alpha M = 3 3 ) for administration of the scale to 394 athletes. Confirmatory factor analyses supported the nine scales. Consistent with the theoretical basis of the FSS, there was also support for a hierarchical model in which one global (higher order) flow factor explained correlations among the nine first-order FSS factors. Suggestions for use of the scale and for further research are discussed.

Key words: flow research, scale development, confirmatory factor analysis, construct validity

The flow state, a positive experiential state, occurs when the performer is totally connected to the performance, in a situation where personal skills equal required challenges. It is a state aspired to by elite athletes (Jackson, 1992, in press), but also one that can be enjoyed by any level of sport participant (Csikszentmihalyi, 1992; Stein, Kimiecik, Daniels, & Jackson, 1995). Research of flow has lagged behind experiential awareness of the state due to the inherent difficulties of applying empirical methods to phenomenological experiences. Due to the importance of flow state to concepts such as motivation, peak performance, peak experience, and enjoyment, attempts to develop ways of assessing flow in sport and activity settings are warranted. This investigation was designed to

Susan A. Jackson is with the Departments of Human Movement Studies and Psychol- ogy at the University of Queensland, Brisbane, QLD, 4072, Australia. Herbert W. Marsh is with the University of Western Sydney, Macarthur, P.O. Box 555, Campbelltown, NSW 2560, Australia.

18 / Jackson and Marsh

develop a psychometrically valid scale to assess flow state in sport and physical activity settings. The primary aims of the project were to develop a scale that assesses flow state in sport and physical activity and to conduct psychometric assessments to establish the validity, reliability, and factor structure of the scale.

The Construct of Flow

Flow is an optimal psychological state that has been described at length by Csikszentmihalyi (1975, 1990, 1993) and substantiated by others in a variety of settings, including work, school, leisure, and sports (see Csikszentmihalyi & Csikszentmihalyi, 1988; Jackson, 1992, in press). When in flow, aperson becomes totally involved in an activity and experiences a number of positive experiential characteristics, including freedom from self-consciousness and great enjoyment of the process. Flow is an intrinsically enjoyable state and is accompanied by an order in consciousness whereby the person experiences clarity of goals and knowledge of performance, complete concentration, feelings of control, and feelings of being totally in tune with the performance. Jackson and Roberts (1992) hypothesized that flow is the psychological process underlying peak performance, and through examining athletes' descriptions of optimal performances and scores on measures of flow, found correlational support for this idea. In addition, qualitative analyses showed athletes' best performances were associated with process-focused descriptions and flow state characteristics.

Understanding how flow is experienced by athletes is an interesting question whose answer can help develop the theoretical understanding of the flow construct. Studies by Jackson (1992, in press) focused on understanding elite athletes' descriptions of being in flow, as well as those factors these athletes perceived as influencing their ability to experience flow. Support was found for the relevance of Csikszentmihalyi's characterization of flow into nine dimensions (Jackson, in press). These dimensions are defined below and are illustrated with data from Jackson's (1994) qualitative content analysis of elite athletes' flow descriptions.

Challenge-Skill Balance

In flow, the person perceives a balance between the challenges of a situation and one's skills, with both operating at a personally high level. Csikszentmihalyi and Csikszentmihalyi (1988) explain this dimension as occumng when a person's skill is at just the right level to cope with the situational demands, which are above average for the person. "Was challenging, but also seemed automatic," is how a track and field athlete described this flow dimension.

Action-Awareness Merging

Involvement in the flow activity is so deep that it becomes spontaneous or automatic. There is no awareness of self as separate from the actions one is performing. Statements such as "in the groove" and "things happen automatically" were used by several athletes to describe action-awareness merging.

Flow State Scale / 19

Clear Goals

Goals in the activity are clearly defined (either set in advance or developed out of involvement in the activity), giving the person in flow a strong sense of what he or she is going to do. "Really knowing what you were going to do," is an example of this dimension from a rower's perspective.

Unambiguous Feedback

Immediate and clear feedback is received, usually from the activity itself, allowing the person to know he or she is succeeding in the set goal. For example, for one rower, "receiving feedback from my movements that I was at the right pace" illustrates the ongoing feedback that sport activities give the performer. Csikszentmihalyi and Csikszentmihalyi (1988) state that the kind of feedback can be very diverse, but the result is the same: information that one is succeeding in one's goal.

Concentration on Task at Hand

Total concentration on the task at hand occurs when in flow. "Feel really focused," describes this dimension for a marathon runner. Total concentration is one of the most frequently mentioned flow dimensions (Csikszentmihalyi, 1990).

Sense of Control

A sense of exercising control is experienced, without the person actively trying to exert control. "Feel like can do anything in that state," and "You can't imagine anything going wrong," illustrate how a runner and a rugby player, respectively, experienced the sense of control when in flow. The labeling of this dimension by Csikszentmihalyi has changed from being "in control" (1975, p. 44), to the "paradox of control" (1990, p. 59), to "sense of control" (1993, p. 181). What seems critical to this dimension is that it is the potential for control, especially the sense of exercising control in difficult situations, that is central to the flow experience.

Loss of Self-Consciousness

Concern for the self disappears during flow as the person becomes one with the activity. When freed from self-consciousness, the athlete often becomes a more natural performer, where "doing things instinctively and confidently" becomes evident in the athlete's actions. The absence of preoccupation with self does not mean the person is unaware of what is happening in mind or body, but rather is not focusing on the information normally used to represent to oneself who one is.

Transformation of Time

Time alters perceptibly, either slowing down, as illustrated by a track runner saying she had "time to think," or speeding up, giving the perception that the


event was "over so fast" for a cyclist. Alternatively, time may simply become irrelevant and out of one's awareness. Although listed as one of the dimensions of flow, Csikszentmihalyi (1990) acknowledges that there are situations where awareness of time is nedessary to successful execution of the activity, implying that this time dimension may not be as universal as the other dimensions. Cer- tainly, in some sports, knowledge of time is part of performing well (e.g., knowing one's splits in swimming), and in qualitative research by Jackson (1992, l994), this dimension has received more equivocal support from athlete populations.

Autotelic Experience

An autotelic experience is an intrinsically rewarding experience. This dimension is described by Csikszentmihalyi as the end result of being in flow. It is illustrated by statements from athletes such as "really enjoy the experience" and "leaves you on a high." Csikszentmihalyi (1990) describes the evolution of the term autotelic as being derived from the Greek words auto (self) and telos (goal) (p. 67). An activity is autotelic if it is done for its own sake, with no expectation of some future reward or benefit.

This study examined the flow construct from the nine dimensions described above. Items were developed to reflect each of the dimensions, and psychometric tests were conducted to determine the most appropriate items and empirically test the proposed multidimensional structure of flow.

Measurement of Flow

Flow, as a concept, is regarded as a critical psychological state that epito- mizes optimal experience during sport participation (Jackson, 1992; Jackson & Roberts, 1992; Kimiecik & Stein, 1992). However, little sport and exercise psychology research has been conducted with flow as a variable, due to the difficulty in measuring the concept. One approach, the experience-sampling method (Csikszentmihalyi & Larson, 1992; Kimiecik & Stein, 1992), requires individuals to respond to a short questionnaire (or experience-sampling form) whenever they receive random signals from an electronic beeper that is worn for a predetermined period of time (typically a week). Further research is required, however, to establish the reliability and validity of responses in sport and exercise settings, and to address practical problems inherent in using this technique in these settings (see Jackson, 1992; Kimiecik & Stein, 1992).

More generally, a multimethod approach is needed to understand flow, incorporating both qualitative and quantitative research. Interview-based research (Jackson, 1992, 1995, in press) has provided richness of description and insights into athletes' experiences of flow. Greater understanding of flow and how it relates to other psychological constructs will be possible when assessment of the flow state can take place close to when it occurs, and when the measurement procedures allow for comparison among psychological constructs. The development of a psychometrically valid scale will open up possibilities for quantitatively based investigations of flow, which can involve state assessments and compari- sons with other psychological states.

The richness and complexity of a construct such as flow necessitates mea- surements that are inclusive rather than exclusive. Several dimensions of the


flow experience have been theoretically discussed and supported by research (Csikszentmihalyi & Csikszentmihalyi, 1988; Jackson, 1992, in press). It is important to establish (through construct validation approaches) the dimensional nature of flow and to develop instruments designed to measure the dimensions. Sport and exercise psychology research has recognized the need for multidimensional and sport-specific measurement instruments (Gill, Dzewaltowski, & Deeter, 1998; Vealey, 1986), and there are examples of systematic research programs that have advanced the measurement of several psychological variables important to the field through the development of such instruments (Gill et al., 1988; Marsh, in press; Martens, Burton, Vealey, Bump, & Smith, 1990; Smith, Smoll, & Schutz, 1990). This investigation aims to provide a measurement instrument that assesses flow as a multidimensional construct, and that can be used easily in sport and physical activity settings.

A Construct Validity Approach

Flow is a hypothetical construct, and therefore, its usefulness must be established by investigations of construct validity. Marsh (1990) discusses construct validity as incorporating two approaches: within- and between-network studies. Within-network studies explore the internal structure of flow. For example, a within-network study might examine the dimensionality of flow in order to establish whether the flow construct has consistent, distinct multidimensional components. Within-network studies often use a factor analytic approach. Be- tween-network studies attempt to establish a logical, theoretically consistent pattern of relations between measures of flow and other constructs. It is important to address at least some of the within-construct issues before moving to between- construct research.

Consistent with this construct validity approach, Gill et al. (1988) argued for the construction of multidimensional sport and exercise instruments based on theory, followed by item and reliability analysis, exploratory and confirmatory factor analysis, tests of convergent and divergent validity, and application in research and practice. The usefulness of confirmatory factor analysis (CFA) for these purposes is widely endorsed but is just beginning to be widely used in the sports sciences. Schutz and Gessaroli (1993), for example, claim that this is the statistical tool for the 1990s in many disciplines but lament its nonuse in sport psychology. Hence, the purpose of the present investigation is to use CFA to evaluate within-network issues about the internal structure of FSS responses and to compare the ability of alternative models to explain FSS responses.

Methods

Initial Item Development I

Items were developed from the definitions of each of the nine proposed dimensions of flow (Csikszentmihalyi, 1990, 1993; Jackson, in press). In forming an initial pool of items, earlier self-report scales designed to measure flow independently or in combination with other variables (Begly, 1979; Csikszentmi- halyi & Csikszentmihalyi, 1988; Privette, 1984; Privette & Bundrick, 1991)


were examined and used as a reference base from which items were developed. Jackson's (1992, 1995, in press) qualitative studies of flow state in elite athletes were particularly important in developing the wording of items in that it provided actual descriptions of flow states in athlete's own words.

An initial pool of 54 items, 6 items per scale, was then evaluated independently by seven researchers familiar with the flow concept and who had used it in sport research. These evaluators rated each item in terms of perceived relevancy to its proposed dimension and provided feedback in terms of item wording. Items rated as less relevant were replaced, and the wording of the items was improved based on feedback from these evaluators. This process resulted in the 54 items used on the initial version of the instrument. A pilot study was conducted with this instrument using a sample of 252 respondents, all of whom were actively participating in a sport or physical activity. Levels of involvement varied from primarily recreational (29%), to club level (39%), to regional, state, or national representative (42%).

The current version of the FSS was based, in part, on changes made in responses to this earlier pilot study. In particular, several negatively or ambigu- ously worded items were found to be less effective in item analyses of this pilot data. These weak items were replaced with more clearly stated, positively worded items. Of particular relevance to the present investigation, reliability estimates for the current version of the FSS are substantially higher than those based on the earlier version used in this pilot study.

Participants and Procedures

Participants in the present investigation were 394 athletes (67% male, 33% female) from the United States (n = 244) and Australia (n = 150). The sample represented a total of 38 different nationalities, although the majority were from the United States (49%) or Australia (35%). A total of 41 different sports and physical activities were represented, with the most frequently mentioned being basketball (20%), track and field (1 I%), and field hockey (7%). Physical activities such as aerobics, hiking, weight training, and jogging comprised approximately 5% of the total sample. Participants varied in age from 14 to 50, with a mean age of 22 (SD = 5.4). Levels of participation varied from primarily recreational (39%), to club or league (17%) or university varsity teams (19%), to state (15%) or national (10%) representatives. As a group, the study participants had participated in their chosen sport or activity from 1 to 37 years (M = 9.7 years), and 7 1 % had participated for over 5 years. A wide variety of settings was included in the sample, with participants coming from physical activity classes, recreational leagues, and state or national sport teams. Participants were recruited by con- tacting coaches and physical educational instructors who provided initial permis- sion to administer the questionnaire. The purpose of the study and standardized instructions were given to all subjects in written or verbal form, along with a guarantee of anonymity.

When answering the FSS, participants were asked to recall an optimal experience during their sport participation, defined as "one where you were totally absorbed in what your were doing, and which was very enjoyable." This definition focused on two major components of the flow experience-absorption


and enjoyment-without actually using the term flow or possibly related constructs such as peak performance or peak experience. Initially, participants were asked to think of one specific experience that occurred while they were participating in their sport or physical activity that constituted an optimal experience. Athletes were asked to name the experience, when it occurred, and the degree to which challenges and skills were in balance during that experience. They then responded to the flow items using a 5-point Likert-type response format (1 = strongly disagree to 5 = strongly agree) and to provide background demographic information including their age, gender, education level, and sporting involvement.

In order to ascertain that participants in this study did experience flow, or optimal experiences, a question was asked about the frequency with which an optimal experience was encountered over a year, and any participants who responded zero were excluded from analysis. Further, any respondents who failed to identify a specific optimal experience, as set out in the instructions for answering the scale, were also excluded. As a further check of the validity of the optimal experience identified, participants were asked to rate the challenges and skills of the situation, in line with Csikszentmihalyi's (e.g., Csikszentmihalyi & Csiks- zentmihalyi, 1988) operational definition of flow as occurring when the perceived situation is challenging but skills are perceived to be high enough to meet the challenge. A mean score of 8.3 for challenges and 7.3 for skills, both on 10- point scales, supported at a group level the assumption that the experiences participants identified and responded to when answering the scale were flow experiences.

Participants were asked to think back to a time when they were in flow and to answer the scale in relation to that particular experience to enable a sufficiently large sample of flow experiences to be included in the analyses. The scale instructions were subsequently modified to enable it to be used immediately after an event or activity (see Appendix), as this was considered to be a more useful format for future research with the scale.

Statistical Analyses

Confirmatory Factor Analysis. CFAs, performed with the mainframe version of LISREL 7 (Joreskog & Sorbom, 1989), were used to test the a priori factor structure underlying the FSS responses. In CFA, the researcher posits an a priori structure and tests the ability of a solution based on this structure to fit the data by demonstrating that (a) the solution is well defined, (b) parameter estimates are consistent with theory and a priori predictions, and (c) the chi- square likelihood ratio and subjective indices of fit are reasonable (Marsh, Balla, & McDonald, 1988; McDonald & Marsh, 1990). For present purposes the Relative Noncentrality Index (RNI) and the Nonnormed Fit Index (NNFI, also called the Tucker-Lewis Index) recommended by McDonald and Marsh (1990) are considered, as well as the chi-square test statistic and the root mean square error of approximation (RMSEA).

The NNFI and RNI vary along a 0-to-l continuum in which values greater than .9 are typically taken to reflect an acceptable fit, whereas the optimal RMSEA is 0 and values less than .05 are typically taken to reflect an acceptable fit. The RNI contains no penalty for a lack of parsimony so that the addition of new


parameters automatically leads to an improved fit that may reflect capitalization on chance, whereas the NNFI and RMSEA contain a penalty for a lack of parsimony.

In the present study, a series of CFA models were tested, and results of these analyses were used to develop a shorter version of the FSS. Three models were fit to responses to the 54 items hypothesizing: (a) one first-order factor (i.e., all 54 items loaded on a single factor); (b) nine first-order factors in which each item was allowed to load on only one factor, six items were used to define each scale, and correlations among the nine first-order factors were freely estimated; and (c) one higher order factor in which correlations among the nine first-order factors were hypothesized to reflect a single higher order factor. Support for the one first-order factor model may support a global measure of flow but would be inconsistent with the multidimensional perspective underlying the design of the FSS. Support for the nine first-order factor model would suppoh the multidimensionality of the flow construct, whereas the juxtaposition of this first-order model and the corresponding higher order model provides a test of a global flow construct.

An important aim of the study was to develop a shorter version of the FSS that retained most of the psychometric strength of the longer instrument. Our goal was to select an optimal set of four items per scale such that responses to each scale maintained high levels of reliability (coefficient alphas of at least .a), discriminated well among the nine hypothesized factors, and were well described by the a priori nine-factor solution. Criteria used to select items included (a) the size of factor loadings in the nine first-order factor solution, (b) corrected item- total correlations from a traditional item analysis (which are highly related to the CFA factor loadings), (c) goodness-of-fit measures for the nine first-order factor model, and (d) LISREL's modification indices (Joreskog & Sorbom, 1989), which provide an index of how much or how highly each item would load on factors other than it was intended to define if allowed to do so (thus allowing us to eliminate complex items that were substantially related to more than one factor.

Absolute cut-off values for these various criteria were not appropriate, because the goal was to select the best four items from each scale so long as adequate reliability and overall goodness-of-fit could be maintained, and because we used a subjective combination of several different criteria in selecting the best items. More items would have been retained if we could not achieve adequate reliability with only four items, but this was not necessary. Although this process of item selection involved a degree of subjectivity, the process is supported in that we were able to achieve the desired criteria of psychometric support for FSS responses. The same three models (above) that were tested with responses to the original 54 items were subsequently tested with responses to the subset of 36 items selected on the basis of these criteria.

Results and Discussion

Reliability

Coefficient alpha estimates of reliability are presented (Table 1) for responses to the current 54-item and 36-item versions of the FSS used in the present


Table 1 Coefficient Alpha Estimates of Reliability From Different Versions of the Flow State Scale

Scale

54-item Current Current pilot 54-item 36-item

version version version (n = 252) (n = 394) (n = 394)

Challenge-skill Action-awareness Clear goals Unambiguous feedback Concentration Sense of control Loss of self-consciousness Transformation of time Autoletic experience

Mean

Note. Reliability estimates under the column "54-item pilot version" are based on results from a previously unpulished study using an earlier version of the FSS. The re- maining results are based on responses collected as part of the present investigation using the entire 54 items and a subset of 36 of these 54 items.

investigation, and for responses to the earlier, 54-item pilot version of the FSS. These preliminary results show that responses to the current 54-item and the 36- item versions considered here are both substantially more reliable than responses to the previous 54-item version used in the earlier pilot study. Whereas the average reliability of scales from the 36-item version ( M = .83) are marginally lower than the 54-item version ( M = .84), the differences are very small given the one-third reduction in length. Furthermore, all nine FSS scales in the 36- item version have reliabilities of at least .8, thus satisfying this criterion.

Factor Structure Underlying Responses

Goodness of fit was evaluated for alternative models based on the current 54-item (6 items per scale) and 36-item (4 items per scale) versions of the FSS (Table 2). Both sets of analyses are consistent in demonstrating that a model positing only one first-order factor provides a poor fit to the data, whereas a model positing nine first-order (freely correlated) factors fits the data slightly better than a model with nine first-order factors and one higher order factor. Both the first-order and higher order models provide a reasonable fit of responses to the 36-item version of the scale (e.g., RNIs > .9), whereas the fit of these models to responses to the 54-item version is marginal. These results, even more strongly than the corresponding reliability estimates (see Table l), provide support

26 / Jackson and Mar-sh

Table 2 Goodness of Fit: Alternative Models Based on 54 and 36 Items

Model

Six items per factor- 1 First-order 5,83 1.47 1,377 .582 .566 .092 9 First-order 3,273.39 1,341 .819 .807 ,061 1 Higher-order 3,449.62 1,368 ,805 ,796 .063

Four- items per- jLhctor 1 First-order 3,447.37 594 .573 ,547 .I 12 9 First-order 1,124.95 585 .915 ,904 .05 1 1 Higher-order 1,254.21 585 .900 ,892 .055

Note. RNI = Relative Noncentrality Index. NNFI = Nonnormed Fit Index. RMSEA = root mean square error of approximation. Alternative models posited one first-order factor, nine first-order factors (the a priori model), and one-higher order factor (based on the correlations among the nine first-order factors).

for the shorter version of the FSS. For this reason, we only present parameter estimates from the 36-item version.

Parameter estimates based on responses to the 36-item version of the FSS (Table 3) provide good support for the a priori nine-factor structure with freely estimated factor correlations and, thus, the construct validity of FSS responses. The factor loadings are substantial in that all are greater than .5, and most are greater than .7 (median factor loading = .74). It is also important to note that although all 36 correlations among the nine a priori factors are all positive, none is greater than .73 or approaches 1.0. The size of the correlations, varying from .I77 to .724 (median r = SO), provides good support for the separation of the FSS factors.

The corresponding higher order factor structure needs to be evaluated carefully (for further discussion of the evaluation of higher order factor models see Marsh, 1987; Marsh & Hocevar, 1985). The higher order model is nested under the first-order model in that it attempts to explain the correlations among the nine first-order factors in terms of a single higher order factor. Because the two models are nested, the chi-square for the higher order model must be as large or larger than the chi-square for the corresponding first-order model and the difference in chi-squares for the two models (129.26) relative to the difference in degree of freedom (27) can be used to test whether the difference between the two models is statistically significant. Although the difference is statistically significant, the goodness-of-fit indices demonstrate that the differences are not large (e.g., NNFIs of .904 vs. .892).

Because the factor loadings (and, thus the uniquenesses) for the two models are very similar, only these estimates for the first-order model (Table 3) are presented. There are, however, some substantial differences in the sizes of the factor correlations inferred from the two models. Although the freely estimated factor correlations already summarized (Table 3) are relatively modest, the correlations among factors based on the higher order model (Table 4) are somewhat


Table 3 Nine-Factor Confirmatory Factor Analysis Solution

Factor Factor loadings

item Chal. Act Goal Fdbk. Conc. Cont. Loss Tran. Enjoy Uniqueness

Challenge-skill balance

Q 1 .608 .O .O .O .O .O .O .O .O Q 1 0 ,770 .O .O .O .O .O .O .O .O Q 1 9 ,725 .O .O .O .O .O .O .O .O Q28 .732 .O .O .O .O .O .O .O .O Action-a~jareness merging Q2 .O ,662 .O .O .O .O .O .O .O Q l l .O .746 .O .O .O .O .O .O .O 420 .O .783 .O .O .O .O .O .O .O Q29 .O .790 .O .O .O .O .O .O .O Clear goals 43 .O .O . 7 1 3 .O .O .O .O .O .O Q12 .O .O .772 .O .O .O .O .O .O 421 .O .O .787 .O .O .O .O .O .O Q30 .O .O .731 .O .O .O .O .O .O Unambiguous feedback Q4 .O .O .O .636 .O .O .O .O .O Q13 .O .O .O 232 .O .O .O .O .O 422 .O .O .O 304 .O .O .O .O .O 4 3 1 .O .O .O ,798 .O .O .O .O .O Concentration on task at hand Q5 .O .O .O .O .683 .O .O .O .O Q14 .O .O .O .O .607 .O .O .O .O 423 .O .O .O .O 359 .O .O .O .O 432 .O .O .O .O 209 .O .O .O .O Paradox of control 46 .O .O .O .O .o .749 .o .o .o Q15 .O .O .O .O .O .746 .O .O .O Q24 .O .O .O .O .O .765 .O .O .O 433 .O .O .O .O .O 369 .o .o .o Loss of self-consciousness 4 7 .O .O .O .O .O .O .729 .O .O 416 .O .O .O .O .O .O .561 .O .O 425 .O .O .O .O .O .O .708 .O .O Q34 .O .O .O .O .O .O .882 .O .O Transformation of time 4 8 .O .O .O .O .O .O .O .739 .O 417 .O .O .O .O .O .O .O .748 .O 426 .O .O .O .O .O .O .O .717 .O Q35 .O .O .O .O .O .O .O . 7 1 5 .O

(continued)


Table 3 (continued)

Factor Factor loadings

item Chal. Act Goal Fdbk. Conc. Cont. Loss Tran. Enjoy Uniqueness

Autotelic experience Q9 .O .O .O .O .O .O .O .O .659 .435 Q18 .O .O .O .O .O .O .O .O .736 .541 427 .O .O .O .O .O .O .O .O .689 .475 436 .O .O .O .O .O .O .O .O .771 .594

Factor correlations

Factor Chal. Act Goal Fdbk. Conc. Cont. Loss Tran. Enjoy

Chal. Act. Goal Fdbk. Conc. Cont. Loss Tran. Enjoy

Note. Chal. = challenge-skill balance: Act = action-awareness merging; Goal = clear goals; Fdbk. = unambiguous feedback; Conc. = total concentration; Cont. = sense of control; Loss = loss of self-consciousness; Tran. = transformation of time; Enjoy = autotelic (enjoyable) experience. All parameter estimates are presented in completely standardized form and are statistically significant at the .O1 level. See Appendix for the wording of the items.

larger (rs vary from .224 to .795, median r = .53). This is, of course, consistent with the requirement that all correlations among the factors fit with a single higher order factor.

It is also relevant to compare the sizes of factor loadings of each first- order on the higher order (global flow) factor or, equivalently, the correlation between each first-order factor and the higher order factor. The largest factor loading is for the Sense of Control factor, followed closely by the Challenge-Skill Balance, Clear Goals, and Concentration factors. Although all first-order factors load significantly on the higher order factor, it is interesting to note that the factor loadings for the Transformation of Time and, to a lesser extent, Loss of Self-Consciousness, are substantially lower. This may call into question the importance of at least the Transformation of Time component of flow, although the evaluation of this suggestion requires further research.

In evaluating the solution based on the higher order model, it is also


Table 4 Correlations Among First-Order and Higher Order Factors

Factor Chal. Act Goal Fdbk. Conc. Cont. Loss Tran. Enjoy

First-order Chal. 1.0 Act .624 1.0 Goal .726 .588 1.0 Fdbk. .658 .533 .620 1.0 Conc. .725 .588 .683 .620 1.0 Cont. .795 .644 .749 .679 .748 1.0 Loss .509 .412 .479 .435 .479 .525 1.0 Tran. .339 .275 .320 .290 .319 .350 .224 1.0 Enjoy .607 .492 .572 .519 .572 .626 .401 .267 1.0 Higher order Global .878 .711 .827 .750 326 .905 .580 .387 ,692 Residual (unexplained) variance in first-order factors

.289 .591 .444 .543 .342 .242 .731 378 .549

Note. Chal. = challenge-skill balance; Act = action-awareness merging; Goal = clear goals; Fdbk. = unambiguous feedback; Conc. = total concentration; Cont. = sense of control; Loss = loss of self-consciousness; Tran. = transformation of time; Enjoy = Autotelic (enjoyable) experience. Correlations among first-order factors are based on the CFA model positing one higher order factor and tend to be higher than those based on corresponding nine-factor solution (Table 3) with no higher order factor. The correlation between each first-order factor and the higher order factor is equal to the factor loading of each first-order factor on the higher order factor. Residual variances are the proportion of "true score variance" (i.e., nonerror variance) in each first-order factor that is unxplained by the higher order factor. All parameters were statistically significant ( p < .01).

informative to evaluate the residual variance estimates for each first-order factor- the proportion of variance in that factor that cannot be accounted for by the higher order factor (see Marsh, 1987). These estimates vary from .242 to 378 (i.e., between 24% and 88% of the variance in the first-order factors is unexplained by the higher order factor), indicating that much of the variance cannot be explained by the higher order factor. This observation is also consistent with the very poor fit of the one first-order factor model in which covariation among responses to all 36 items is explained by a single factor. Hence, even though the results support the higher order representation of the flow construct as assessed by the FSS, the FSS responses cannot be explained very well by a single score or factor.

Summary and Implications

This investigation is part of an ongoing attempt to'develop a psychometrically valid and usable scale for assessing flow in sport and physical activity

30 I Jackson and Marsh

settings. The results showing good reliability and support for the hypothesized factor structure are promising. The a priori model hypothesizing nine FSS factors was supported, and there was also support for a hierarchical model. Of particular importance these two models fit the data substantially better than the model hypothesizing only one factor.

The comparison of the first-order model with nine correlated factors with the higher order factor model has important practical implications about how to best represent FSS responses. Support for the higher order model with a single global flow factor does not mean that FSS responses can be explained in terms of a single score. Support for this contention would require that the first-order model positing one factor could fit the data, and this is clearly not the case. In fact, the large residual variances for at least some of the nine first-order factors demonstrates that there is considerable variance in at least some of the first-order factors that cannot be explained in terms of a higher order factor. Hence, to explain adequately the variance in FSS responses requires nine scores representing

I the nine FSS factors. The measurement of distinct components of flow provides a better basis

for evaluating the theoretical underpinning of the FSS than reliance on a global score. For example, the relatively lower factor loadings for the Transformation of Time and Loss of Self-Consciousness factors found in the CFA may mean these dimensions of the flow experience are less universally important than other of the flow dimensions. Previous research has found both of these dimensions to be less supported than the other dimensions in athletes' experiences (Jackson, 1992, 1994; Jackson & Roberts, 1992). It may be that the nature of sport performances demands awareness of time and of how the self is being presented, making these two factors less significant or relevant to the athlete's flow experience. On the other hand, the moderate second-order factor loading relating Autotelic Experience to global flow was unexpected. As described by Csikszentmihalyi (1990), this dimension is crucial to the flow experience. Indeed, Csikszentmihalyi often substitutes the terms autotelic experience or enjoyment for flow, implying that this factor may be seen as having a more global nature than the other flow dimensions.

The fact that other dimensions, such as concentration, control, and challenge-skill balance had higher factor loadings than autotelic experience may indicate that the enjoyment dimension is less central than other aspects of flow to athletes. Perhaps enjoyment is taken for granted to some extent in a free choice activity like sport. Another possibility may be that because of the goal-directed nature of competitive sport, enjoyment is seen as somewhat antithetical to the serious nature of the endeavor. The Clear Goals factor was substantially related to the higher order flow construct, providing some support to this suggestion. Studies focusing on the relationship among the various first-order factors and the higher order factor, as well as their relationship with specified person and situation factors, are needed.

The relative usefulness of a single global FSS score compared to the set of nine FSS scale scores is a question open to further consideration. In evaluating this issue, it is informative to reflect on the analogous, more widely considered issue of the relative usefulness of a global measure of intelligence (e.g., IQ) compared to the specific components of intelligence that make up intelligence. Although a single global measure of intelligence may suffice in some situations,

Flow State Scale / 3 1

there are other situations in which the more detailed information about specific abilities is required. More than 100 years of research into this debate in intelligence testing has not resolved the issue. Similarly, there may be instances where a single global flow will suffice, but other situations where the more detailed information from the specific components of flow are more useful. Hence, there is no absolute answer to the question about whether it is appropriate to limit consideration to a single global flow score.

In pondering this dilemma, however, several considerations are relevant. A global measure of flow implicitly implies that the weighting assigned to each specific component is the same for all situations and all individuals, but a more flexible definition of flow that allows the weighting of specific components to vary depending on the application may be appropriate. Also, it is a mathematical necessity that an optimally weighted average of the specific components will be able to explain as much or more variance in any criterion measure as a single global score derived from the same responses. Furthermore, it is possible that very distinct profiles of specific flow components having substantively important implications could result in the same global score component.

Because there is some variance in most of the first-order factors and substantial amounts in a few that cannot be explained by the higher order factor, it is possible (and perhaps likely) that some external criterion variables of interest can be explained better by first-order factors than they can by the second-order factor. Hence, the relative usefulness of the global and specific components of FSS responses cannot be fully evaluated until the FSS instrument has been used much more extensively and related to a much wider set of validity criteria. Therefore, we prefer to consider both until there is a sufficient research basis for providing a better evaluation of these alternative operationalizations of the flow construct.

Two limitations of the present investigation are the retrospective approach to data collection and the problems inherent in attempting to quantify experiential states. In relation to the retrospective approach taken to data collection, it was decided that asking participants to respond to a previous flow experience that stood out for them would be more useful than having respondents complete the scale after a performance that may or may not have been a flow experience. However, the scale as it is presented here (see Appendix) is designed to be used immediately, or soon after, performance, as an assessment of flow state characteristics experienced during the performance. Hence, further research is needed to ascertain that results presented are general- izable. Also, a trait version of the FSS instrument is currently being developed to help establish whether there are individual differences that may be related to ability to experience flow.

The second limitation of this study is one encountered in all research that attempts to quantify experiential states. Qualitative research may be better able to capture the richness of a complex phenomenological state such as flow. However, there are limitations to qualitative approaches, including the problems of retrospective recall and the time-consuming nature of the research process. The richness and depth of data acquired through qualitative methods can be combined with the more easily collected and comparable state assessments of a flow scale to provide a more complete picture of the flow construct. The fact that a diverse sample was used in the present study adds support to the potential


generalizability of the scale. However, it is also possible that the factor structure based on such a heterogeneous sample may not generalize to specific subgroups within this larger group and to more specific samples in future research. Hence, it is important to replicate these results with other samples, both diverse and specific.

Any attempts to investigate flow are fraught with difficulties and limitations. Part of the attraction of the flow state lies in its mystique. Flow cannot be fully captured by a score on a questionnaire, experience sampling methods, or in- depth interviews. Csikszentmihalyi (1992) cautions against putting too much weight on any empirical measures of flow, lest the experience of flow be lost in the process: "The moment we say that 'flow is the balance of challenges and skills,' or that 'flow is a score of "x" on the flow questionnaire,' we have lost it. We have mistaken the reflection for the reality" (p. 183).

In accord with this statement, the FSS instrument is presented as one apparently useful indicator of the flow construct. The development of this instrument was motivated by a desire to bring the flow construct to a level of research potential that will hopefully lead to elucidation of its nature and of the factors and situations conducive to its experience. Better understanding of flow and the factors related to its occurrence (see Jackson, 1995) is the path to making flow a more accessible experience to all athletes and all physical activity participants.

We do not claim that the FSS instrument is the only, or even the best, way to study the flow construct. Indeed, a useful direction for further research is to more systematically compare the results of various methods of inferring flow when used with the same group of respondents. This type of multimethod study is a logical next step in the construct validity approach that is the basis of the present investigation. Between-network studies that examine the pattern of relationships between flow state, as measured by the FSS, and other psychological constructs will also help to further understanding of the flow construct, its antecedents, and its consequences.

We endorse a broad approach to construct validation (e.g., Marsh, in press). From this perspective, useful directions of future research include relating FSS responses to a wide variety of external validity criteria (including alternative measures of flow), comparing FSS scores from a variety of different groups predicted a priori to differ in terms of flow, relating changes in flow to changes in other constructs in multiwave studies, and using the FSS as an outcome variable in intervention studies. Consistent with this perspective, instrument design and evaluation, theory, research, and practice are inexorably intertwined so that all will suffer if any one is neglected.

References

Begly, G. (1979). A self-report measure to assess flow in physical activities. Unpublished masters thesis, Pennsylvania State University.

Csikszentmihalyi, M. (1975). Beyond boredom and anxiety. San Francisco: Jossey-Bass. Csikszentmihalyi, M. (1990). Flow: The psychology of optimal experience. New York:

Harper & Row. Csikszentmihalyi, M. (1992). A response to the Kimiecik & Stein and Jackson papers.

Journal of Applied Sport Psychology, 4 , 18 1 - 183.


Csikszentmihalyi, M. (1993). The evolving self. New York: Harper & Row. Csikszentmihalyi, M., & Csikszentmihalyi, I. (1988). Optimal experience: Psychological

studies offlow in consciousness. New York: Cambridge University Press. Csikszentmihalyi, M., & Larson, R. (1992). Validity and reliability of the experience

sampling method. In M. de Vries (Ed.), The experience of psychopathology. New York: Cambridge University Press.

Gill, D.L., Dzewaltowski, D.A., & Deeter, T.E. (1988). The relationship of competitiveness and achievement orientation to participation in sport and nonsport activities. Journal of Sport & Exercise Psychology, 10, 139- 150.

Jackson, S.A. (1992). Athletes in flow: A qualitative investigation of flow states in elite figure skaters. Journal of Applied Sport Psychology, 4, 161-180.

Jackson, S.A. (1995). Factors influencing the occurrence of flow state in elite athletes. Journal of Applied Sport Psychology, 7 , 138- 166.

Jackson, S.A. (in press). Athletes in flow: Toward a conceptual understanding of flow state in elite athletes. Research Quarterly for Exercise and Sport.

Jackson, S.A., & Roberts, G.C. (1992). Positive performance states of athletes: Toward a conceptual understanding of peak performance. The Sport Psychologist, 6, 156- 171.

Joreskog, K.G., & Sorbom, D. (1989). Lisrel 7: A guide to program applications (2nd ed.). Chicago: SPSS.

Kimiecik, J.C., & Stein, G.L. (1992). Examining flow experiences in sport contexts: Conceptual issues and methodological concerns. Journal of Applied Sport Psychol- ogy, 4, 144-160.

Marsh, H.W. (1987). The hierarchical structure of self-concept: An application of hierarchical confirmatory factor analysis. Journal of Educational Measurement, 24, 17-39.

Marsh, H.W. (1990). A multidimensional, hierarchical self-concept: Theoretical and empirical justification. Educational Psychology Review, 2, 77-171.

Marsh, H.W. (in press). The measurement of physical self-concept: A construct validation approach. In K.R. Fox, (Ed.) The physical self: From motivation to well-being. Champaign, IL: Human Kinetics.

Marsh, H.W., Balla, J.R., & McDonald, R.P. (1988). Goodness-of-fit indices in confirmatory factor analysis: The effect of sample size. Psychological Bulletin, 102, 391- 410.

Marsh, H.W., & Hocevar, D. (1985). The application of confirmatory factor analysis to the study of self-concept: First and higher order factor structures and their invariance across age groups. Psychological Bulletin, 97, 562-582.

Martens, R., Burton, D., Vealey, R., Bump, L.A., & Smith, D.E. (1990). Development and validation of the Competitive State Anxiety Inventory-2. In R. Martens, R.S. Vealey, & D. Burton, Competitive state anxiety in sport (pp. 117-190). Champaign, IL: Human Kinetics.

McDonald, R.P., & Marsh, H.W. (1990). Choosing a multivariate model: Noncentrality and goodness-of-fit. Psychological Bulletin, 107, 247-255.

Privette, G. (1984). Experience Questionnaire. Pensacola: University of West Florida. Privette, G., & Bundrick, C.M. (1991). Peak experience, peak performance, and flow:

Personal descriptions and theoretical constructs. Journal of Social Behavior and Personality, 6, 169-188.

Schutz, R.W., & Gessaroli, M.E. (1993). Use, misuse, and disuse of psychometrics in sport psychology research. In R.N. Singer, M. Murphy, & L.K. Tennant (Eds.), Handbook of research on sport psychology (pp. 901-9 17). New York: Macmillan.

Smith, R.E., Smoll, F.L., & Schutz, R.W. (1990). Measurement and correlates of sport- specific cognitive and somatic trait anxiety: The Sport Anxiety Scale. Anxiety Research, 2, 263-280.


Stein, G.1., Kimiecik, J.C., Daniels, J., & Jackson, S.A. (1995). Psychological antecedents of flow in recreational sport. Personality and Social Psycholo,py Bulletin, 21, 125- 135.

Vealey, R.S. (1986). Conceptualization of sport-confidence and competitive orientation: Preliminary investigation and instrument development. Journal oflport Psycholo,qy, 8, 221-246.

Appendix

Flow State Scale

Please answer the following questions in relation to your experience in the event you have just completed. These questions relate to the thoughts and feelings you may have experienced during the event. There are no right or wrong answers. Think about how you felt during the event and answer the questions using the rating scale below. Circle the number that best matches your experience from the options to the right of each question.

Rating Scale:

Strongly Neither agree Strongly disagree Disagree nor disagree Agree agree

1 2 3 4 5

1. I was challenged, but I believed my skills would allow me to meet the challenge.

2. I made the correct movements without thinking about trying to do so.

3. I knew clearly what I wanted to do. 4. It was really clear to me that I was doing well. 5. My attention was focused entirely on what I

was doing. 6. I felt in total control of what I was doing. 7. I was not concerned with what others may

have been thinking of me. 8. Time seemed to alter (either slowed down or

speeded up). 9. I really enjoyed the experience.

10. My abilities matched the high challenge of the situation.

1 1. Things just seemed to be happening automatically.

12. I had a strong sense of what I wanted to do. 13. I was aware of how well I was performing. 14. It was no effort to keep my mind on what was

happening. 15. I felt like I could control what I was doing. 16. I was not worried about my performance dur-

ing the event.

Strongly disagree

Strongly agree


17. The way time passed seemed to be different 1 2 3 4 5 from normal.

18. I loved the feeling of that performance and 1 2 3 4 5 want to capture it again.

19. I felt I was competent enough to meet the high demands of the situation.

20. I performed automatically. 21. I knew what I wanted to achieve. 22. I had a good idea while I was performing

about how well I was doing. 23. I had total concentration. 24. I had a feeling of total control. 25. I was not concerned with how I was presenting

myself. 26. It felt like time stopped while I was per-

forming. 27. The experience left me feeling great. 28. The challenge and my skills were at an equally

high level. 29. I did things spontaneously and automatically

without having to think. 30. My goals were clearly defined. 1 2 3 4 5 31. I could tell by the way I was performing how 1 2 3 4 5

well I was doing. 32. 1 was completely focused on the task at hand. 1 2 3 4 5 33. I felt in total control of my body. 1 2 3 4 5 34. I was not worried about what others may have 1 2 3 4 5

been thinking of me. 35. At times, it almost seemed like things were 1 2 3 4 5

happening in slow motion. 36. I found the experience extremely rewarding. 1 2 3 4 5

O S.A. Jackson, University of Queensland, I995

Acknowledgments

We would like to thank the following for their assistance: Wally Karnilowicz, for assistance in early phases of data analyses; Daryl Marchant, Robert Eklund, Jeff Martin, Nicole Djmarjin, Nicholas Francis, and Jeremy Dover, for their assistance with data collection; and the seven researchers who evaluated the initial pool of items. We would also like to thank the Victorian Institute of Sport and the many coaches who granted access to their athletes as participants in this study. Part of this study was funded by a grant from Victoria University of Technology when the first author was working there. The Flow State Scale (Appendix) may be used for research purposes without any prior written consent as long as appropriate recognition is given, but the first author would appreciate being sent copies of resulting publications.

Manuscript submitted: April 4, 1995 Revision accepted: August 13, 1995

Development and Validation of a Scale to Measure Optimal ... · Development and Validation of a...

Documents

Transcript of Development and Validation of a Scale to Measure Optimal ... · Development and Validation of a...