s3-eu-west-1.amazonaws.com · Web viewConcepts of Achievement-Related Emotions Across Cultures:...

Concepts of Achievement-Related Emotions Across Cultures: Supplemental Material 1

Supplemental Online Material for

Are Concepts of Achievement-Related Emotions Universal Across Cultures? A Semantic Profiling Approach

This file includes:

1. Theoretical Background: Additional Information2. Method: Additional Information3. Results: Additional Information for Findings Reported in the Main Text4. Results: Additional Analyses of Cross-Cultural Similarity of Concepts of Achievement- Related Emotions Using Pearson Correlation Coefficients5. Supplemental Tables S1-S96. Supplemental Figures S1-S47. References Cited in the Supplemental Material


1. Theoretical Background: Additional Information

Do Concepts of Achievement-Related Emotions also Vary across Sociocultural Contexts?

As noted, the assumption that constructs are equivalent across cultural groups under study is typically tested using multigroup confirmatory factor analysis (Byrne & van de Vijver, 2010). However, measurement invariance cannot guarantee that items ‘mean the same thing’ and trigger congruent information in semantic memory across individuals (Berger & Karabenick, 2016; Karabenick et al., 2007; Messick, 1995). Evidence suggests that while psychometric properties like factorial invariance may be adequate for a given scale, its items and target construct may not be construed in the same way by different groups of respondents (Koskey, Karabenick, Woolley, Bonney, & Dever, 2010).

2. Method: Additional Information

The GRID ParadigmSemantic profiling using Fontaine et al.’s (2013) GRID paradigm provides a

theoretically-grounded approach for systematically examining cross-cultural similarity and differences in decontextualized emotion concepts across languages and cultures (see also Scherer & Fontaine, 2019). In the GRID, native speakers of a given language rate to which degree a set of affective, cognitive, motivational, physiological, and expressive features apply to the concept denoted by a specific emotion word. The paradigm is grounded in a lexical approach proposing that language reflects prototypical meaning evolutionarily derived from human experience (Cattell, 1945; De Raad & Mlačić, 2017).

The features included in the GRID instrument were derived from an extensive review of a broad range of prominent theories of emotion and emotional responding, including appraisal theories of emotion (Scherer & Moors, 2019), the psychophysiological emotion literature (e.g., Stemmler, 2003), Frijda, Kuipers, and ter Schure‘s(1989) action tendency theory, dimensional models of human affect (e.g., Osgood, 1957; Russell, 1980, 2003; Yik, Russell, & Barrett, 1999), and Ekman and Friesen’s (1969) repertoire of nonverbal expression of emotion (see Fontaine et al., 2013, chapter 5, for further details).

Development of the Achievement Emotions CoreGRID: Additional InformationThe AECG encompasses 84 features (see Table S1 for full list). New features added

to the CoreGRID, their theoretical background, and the internal structures of the five emotion components are described below. In general, in developing the AECG, we surveyed prominent theories on achievement emotions, including control-value theory (Pekrun & Perry, 2014), expectancy-value models of shame and related emotions (e.g., Turner & Schallert, 2001), attribution theory (e.g., Graham & Taylor, 2014; Weiner, 1985), and models addressing the effects of emotions on learning and performance (Fredrickson, 2001; Zeidner, 1998), to identify characteristics of achievement emotions to be incorporated into the extant list of CoreGRID features.

Affective component. The affective component contains the 10 original CoreGRID features tapping into valence (good, bad), tense (calm, restless) and energetic arousal (tired, awake), and power (weak, strong), the latter of which is closely related to perceived control in the CVT (Pekrun & Perry, 2014). Intensity and duration of emotional experience are also assessed (Table S1).

Cognitive component. This component was realized through the 21 CoreGRID and three additional features contained in the original GRID relevant to achievement contexts (i.e., the event was caused intentionally, the consequences of the event were positive,


desirable for the person, and the person was treated unjustly).To increase the salience of the achievement contextualization, references to ‘the event’ to be appraised in the original features were replaced with ‘success or failure’ (S/F; e.g., the success was caused intentionally). In line with the CVT (Pekrun & Perry, 2014), features pertain to appraisals of control (e.g., power – the person was powerless in the achievement situation involving S/F; agency– the S/F was caused by the person’s own behavior) and value (e.g., pleasantness – the S/F was pleasant for the person; personal relevance – the S/F was important and relevant to the person’s goals or needs). We expanded the list by two items representing commonly contrasted causal attributions referring to different internal causes of performance outcomes (the S/F was a result of the effort or lack of effort invested by the person, respectively; the S/F was a result of the person’s ability or lack thereof, respectively; Graham & Taylor, 2014; Weiner, 1985). We added two further items capturing positive (the activity in the achievement situation was pleasant) versus negative value (the activity in the achievement situation was unpleasant) of achievement-related activities as relevant precursors of achievement emotions (see Pekrun & Perry, 2014) and expectancy-value accounts of achievement emotions (e.g., Turner & Schallert, 2001). Theoretically proposed linkages between different achievement emotions and appraisals of control and value are detailed in Pekrun and Perry (2014).

Motivational component. Fourteen features make up the motivational component of the CoreGRID. Largely grounded in Frijda’s work (Frijda, Kuipers, & ter Schure, 1989), the features assess several motivational and behavioural mechanisms of emotions addressed by CVT (Pekrun & Perry, 2014). For example, both the general presence versus absence of motivation (wanted to do nothing, lacked the motivation to pay attention) as well as qualitative differences in motivation are considered (e.g., approach – wanted to tackle the situation; avoidance – wanted to run away). Furthermore, items such as wanted someone else to take the initiative tap into postulated differences in internal versus external regulatory foci associated with different achievement emotions in the literature (e.g., Pekrun, Goetz, Titz, & Perry, 2002). Three new items were developed based on previous findings addressing relations between achievement emotions and cognitive resource allocation (was able to concentrate) as well as cognitive styles of processing (wanted to approach the task systematically, wanted to try new or creative ways of approaching the task at hand; see also Fredrickson, 2001; Zeidner, 1998), raising the number of action tendency items to 18.

Physiological component. This component is comprised of 11 CoreGRID featuresand one additional feature taken from the original GRID instrument (blushed) depicting changes in peripheral autonomic arousal, including body temperature and slowed versus increased heartrate. The items were felt to adequately capture bodily reactions associated with different achievement emotions. We chose to incorporate blushing in the AECG because it is proposed as a typical characteristic of achievement-related shame across different theories (e.g., Graham & Taylor, 2014; Turner & Schallert, 2001).

Expressive component. In the CoreGRID, this component is defined by 12 features covering facial (e.g., frowned, closed their eyes) and vocal expression (e.g., spoke in a trembling voice, spoke more rapidly). We added four items on postural expression (held their head up high, hung their head low, slumped, and tensed the upper body)We added four features on postural expression (i.e., held their head up high, hung their head low, slumped, and tensed the upper body), as posture has been shown to be a useful indicator for diagnosing emotional states in achievement contexts (D'Mello & Graesser, 2009; Reisenzein, Junge, Studtmann, & Huber, 2014).


Analysis of Emotion Concept Similarity: Additional InformationAs noted, the present study compared mean culture-level emotion concepts by means

of computing ‘profile’ correlations in the form of double-entry intraclass correlations across culture-level mean feature scores. As such, emotion concept similarity was estimated at the between-variable rather than between-person level: Profile correlations index similarity across multiple variables for a given pair of persons, groups, or other entities to be compared, whereas between-person correlations quantify similarity for a given pair of variables across many persons. Empirically-grounded recommendations specify that profiles contain at least eight items to ensure sufficient reliability (Kenny, Kashy, & Cook, 2006; Rogers, Wood, & Furr, 2017; Wood & Furr, 2016).

Preliminary Analyses: Additional InformationWe first examined absolute within-sample agreement on the feature profiles of each

emotion by computing intraclass correlations (ICC) based on two-way random effects models of the type ‘average measure of k raters’ (McGraw & Wong, 1996). These ICCs reflect classic ‘inter-observer agreement’ used in observational research, with participants representing different ‘observers’, and the 84 features representing separate ‘observations’ made for each emotion. The resulting ICCs indicated excellent within-sample agreement. This convergence also extended to component-level analyses of within-sample agreement: Of the 320 (16 emotions x 5 components x 4 samples) ICCs in Table S5, the majority (94.4%) were ≥.90, and nearly all (98.8%) were >.75. Thus, as required for meaningful analysis, our subsequent cross-cultural comparisons are based on highly consistent and reliable within-sample emotion concepts.

Analysis of Cross-Cultural Emotion Profile Similarity: Additional InformationICC-DEs (Furr, 2010; McCrae, 2008) have been used to examine cross-cultural

similarity of culture-level aggregate personality traits (Hřebíčková et al., 2018; McCrae & Terracciano, 2005). They are estimated by computing Pearson product-moment correlations between two doubly-entered profiles created by appending each profile to the end of the other: For instance, to estimate similarity between Canadian and German concepts of achievement-related anxiety, we created two columns containing 168 data points (i.e., 2 x 84 features): Column 1 contained the Canadian feature ratings as cases 1-84 and the appended German feature ratings as cases 85-168, column 2 contained the German feature ratings as cases 1-84 and the Canadian feature ratings as cases 85-168. The ICC-DE quantifying similarity of Canadian and German anxiety concepts was then computed using the following equation (see Furr, 2010, for its derivation):

ICC−DEanx=2σ CA σ ¿rCA ,≥¿−1

2(M¿¿CA−M ¿)

2

σ CA2 +σ ¿

2+12( M¿¿CA−M ¿)

2¿¿¿ (1)

In this equation, MCA and MGE constitute the means (i.e., elevation) of the Canadian and German feature profiles for anxiety; σ CA, σ CA

2 , σ ¿, and σ ¿2 are the standard deviations and

variances (i.e., scatter) of these profiles; and rCA,GE is the Pearson correlation (i.e., shape similarity) between the profiles.

As noted in the main text, we took an effect size approach to evaluating ICC-DEs in terms of their magnitude. In reviewing literature on correlational indexes of profile similarity, Hřebíčková et al. (2018) contend that correlations of .40 indicate high agreement (i.e., ratings


converge 70% of the time). In the present study, we opted for a more conservative approach and relied on Cohen’s (1988) guidelines.

3. Results: Additional Information for Findings Reported in the Main Text

Follow-Up Analyses: Cultural Differences in Concepts of Physiological Components of Boredom

The low correspondence in perceived characteristics of boredom, particularly with regard to its physiological components, may be due to the specific Chinese translation chosen in this study: Boredom was translated as 厌烦 (yàn fán), one of four Chinese expressions denoting ‘a state of being bored’ (Cambridge Dictionary, 2019). This translation is implemented in a suggested Chinese adaption of the AEQ (Zhao & Cai, 2012) and implies a state of annoyance or irritation resulting from boredom, suggesting unpleasant arousal. In contrast, the variant 厌倦 (yàn juàn) implies deactivation involving inaction and fatigue, which may correspond more closely to typical Canadian, German, and Colombian concepts of boredom.

Notably, Ng et al.’s (2015) translation of the Multidimensional State Boredom Scale employed in their study uses both 厌烦 (yàn fán) and 无聊 (yàn juàn) to refer to the target emotion. Moreover, it is worth noting that their dictionary definitions recall the different valence-arousal profiles of achievement-related boredom described by Goetz et al. (2014), suggesting that more inquiry into the semantics of this particular emotion is needed. In essence, these findings reiterate the importance of considering translation versus conceptual equivalence in emotion measurement (Ogarkova, 2016).

4. Results: Additional Analyses of Cross-Cultural Similarity of Concepts of Achievement-Related Emotions Using Pearson Correlation Coefficients

Following ongoing debates about the strengths and limitations of different indexes of profile similarity (e.g., Hřebíčková et al., 2018), we computed simple Pearson correlations in addition to ICC-DEs for quantifying cross-cultural similarity of achievement emotion concepts. Overall, Pearson correlations were typically slightly larger than corresponding ICC-DEs, which can be traced back to the statistical properties of these indexes: As mentioned in the main text (see section on analysis of cross-cultural emotion profile similarity), ICC-DEs constitute a more conservative index of profile similarity that takes into account not only the shape, but also elevation and scatter of emotion profiles. Importantly however, as detailed below, the supplemental Pearson correlation analyses yield the same insights and conclusions about cross-cultural similarity of achievement emotion concepts as the main analyses.Cross-Cultural Similarity of Feature Profiles of Emotions

Corroborating the patterns of results obtained from the main analyses, the Pearson correlations reported in Table S5 indicate that cross-cultural similarity of the full 84-feature profiles of achievement-related emotions was generally high, with 96.9% of the correlations exceeding .50, and 86.5% exceeding .75. The resulting grand mean correlation indexing conceptual similarity across all 16 emotions and cultural pairings was .83 (SD = .13, 95% CI [.80, .86]) and thus slightly higher than the corresponding mean ICC-DE of .79 (see main text, results section).

Furthermore, in line with the main analyses, average cross-cultural Pearson correlations across all 16 emotions were of comparable magnitude for the CA-GE (M = .89, SD = .04, 95% CI [.87, .91]), CA-CO (M = .84, SD = .09, 95% CI [.80, .88]), and GE-CO (M


= .86, SD = .07, 95% CI = [.83, .89]), as indicated by the overlapping confidence intervals. Similarities for comparisons involving the Chinese sample fell below rs of .80 (see Table S5). The non-overlapping confidence intervals of the CA-GE with the GE-CH (95% CI [.69, .85]) indicate that the similarity of emotion concepts across the Canadian and German samples was, on average, higher than for the GE-CH comparisons. As such, the patterns largely correspond to the findings produced by the main analyses and corroborate our exploratory hypothesis that conceptual similarity should be highest among Western cultures (i.e., Canada, Germany, Colombia).

In a similar vein, average cross-cultural Pearson correlations of achievement-related emotion concepts reached values of .74 or higher, with boredom again constituting the only exception (M = .53, SD = .26). However, the mean r of .53 indexing relatively high similarity with regard to profile shape as compared with the ICC-DE value combining profile shape, elevation, and scatter (MICC-DE = .37, SD = .36; see main text). Again, boredom concepts were highly similar for the CA-GE (.89), CA-CO (.84), and GE-CO (.86) comparisons, but fell at r ≤ .49 for comparisons involving the Chinese sample, indicating similar discrepancies between Chinese and Canadian, German, as well as Colombian students’ concepts of boredom, as found in the main analyses.Cross-Cultural Similarity of Feature Profiles of Emotion Components

Component-level similarities (Pearson rs) of achievement-related emotion concepts across cultures are documented in Table S6. Of the 480 Pearson correlations (16 emotions x 6 pairwise cultural comparisons x 5 components), 93.7% exceeded .50, indicating that component-level shape similarity of achievement emotion profiles was high as well. Paralleling the main ICC-DE analyses, correlations < .50 were most common for cultural comparisons involving the Chinese sample, as well as for the physiological and expressive emotion components. As documented in Table S6, average component-level ICC-DEs across all 16 achievement emotions and pairwise cultural comparisons were high, and of comparable magnitude, for the affective (95% CI [.89, .93]), cognitive (95% CI [.79, .86]), and motivational components (95% CI [.85, .89]; see Table S6 for means and standard deviations). Corroborating the main analyses, these values indicate substantially higher degrees of shape similarity for these components as compared with the physiological components of emotions (95% CI [.6, .78]), and average similarity of the affective and motivational component profiles was significantly higher than average similarity of the expressive component profiles (95% CI [.74, .84]).

Furthermore, Pearson rs < .50 were most common for boredom and frustration (see Table S6). As such, these supplemental analyses focusing on shape similarity of achievement emotion concepts across cultures largely parallel the main analyses based on ICC-DEs.


5. Tables S1-S9

Table S1Features Included in the Achievement Emotions CoreGRID (English Translation)

Feature SourceAffective Component (10 features)

1. felt the emotion very intensely Original CoreGRID2. felt the emotion for a long time Original CoreGRID3. felt good Original CoreGRID4. felt bad Original CoreGRID5. felt calm Original CoreGRID6. felt restless Original CoreGRID7. felt tired Original CoreGRID8. felt awake Original CoreGRID9. felt weak Original CoreGRID10. felt strong Original CoreGRID

Cognitive Component (28 features)11. the success or failure occurred suddenly Original CoreGRID; adapted12. the success or failure was unpredictable Original CoreGRID; adapted13. the success or failure confirmed the expectations of the person Original CoreGRID; adapted14. the success or failure was pleasant for the person Original CoreGRID; adapted15. the success or failure was important for and relevant to the person’s

goals or needsOriginal CoreGRID; adapted

16. the success or failure was important for and relevant to the goals or needs of somebody else

Original CoreGRID; adapted

17. the success or failure happened by chance Original CoreGRID; adapted18. the success or failure was caused by the person’s own behavior Original CoreGRID; adapted19. the success or failure was caused by somebody else’s behavior Original CoreGRID; adapted20. the success or failure had consequences that were predictable Original CoreGRID; adapted21. the success or failure had negative, undesirable consequences for the

personOriginal CoreGRID; adapted

22. the success or failure required an immediate response Original CoreGRID; adapted23. the person had power over the consequences of the success or failure Original CoreGRID; adapted24. the person had control over the consequences of the success or failure Original CoreGRID; adapted25. the person could live with the consequences of the success or failure Original CoreGRID; adapted26. the success or failure was inconsistent with the person’s own standards

and idealsOriginal CoreGRID; adapted

27. the success or failure involved the violation of socially accepted norms Original CoreGRID; adapted28. the person was powerless in this achievement situation involving

success or failureOriginal CoreGRID; adapted

29. the person had a dominant role in the achievement situation involving success or failure

Original CoreGRID; adapted

30. there was no urgency in the situation involving success or failure Original CoreGRID; adapted31. the success or failure was uncontrollable Original CoreGRID; adapted32. the success or failure was caused intentionally Original GRID; adapted33. the consequences of the success or failure were positive, desirable for

the personOriginal GRID; adapted

34. the person was treated unjustly Original GRID35. the activity in the achievement situation was pleasant New item 36. the activity in the achievement situation was unpleasant New item 37. the success or failure was a result of the effort or lack of effort invested

by the person, respectivelyNew item

38. the success or failure resulted from the person's ability or a lack of personal ability, respectively

New item

Motivational Component (18 features)39. wanted the ongoing situation to last or be repeated Original CoreGRID40. wanted to stop what he/she was doing Original CoreGRID41. wanted to undo what was happening Original CoreGRID42. wanted to comply with someone else's wishes Original CoreGRID


Feature Source43. wanted someone else to take the initiative Original CoreGRID; adapted44. wanted to do nothing Original CoreGRID45. lacked the motivation to pay attention to what was happening Original CoreGRID; adapted46. wanted to disappear or hide from others Original CoreGRID47. wanted to do damage, hit, or say something that hurts Original CoreGRID48. wanted to oppose someone or something Original CoreGRID49. wanted to tackle the situation Original CoreGRID50. wanted to overcome an obstacle Original CoreGRID51. wanted to run away in any direction Original CoreGRID52. wanted to sing and dance Original CoreGRID53. wanted to try harder New item54. was able to concentrate on what was going on New item55. wanted to approach the task systematically New item56. wanted to try new or creative ways of approaching the task at hand New item

Physiological Component (12 features)57. feeling weak in the limbs Original CoreGRID; adapted58. becoming pale Original CoreGRID59. stomach disturbance Original CoreGRID60. slowed heartrate Original CoreGRID61. rapid heartrate Original CoreGRID62. tense muscles Original CoreGRID; adapted63. slowed breathing Original CoreGRID64. rapid breathing Original CoreGRID65. feeling warm Original CoreGRID66. sweating Original CoreGRID67. feeling cold Original CoreGRID68. blushed GRID

Expressive Component (16 features)69. smiled70. dropped their jaw Original CoreGRID; adapted71. raised their eyebrows Original CoreGRID72. frowned Original CoreGRID73. closed their eyes Original CoreGRID74. had tears in their eyes Original CoreGRID75. spoke more loudly Original CoreGRID76. spoke in a trembling voice Original CoreGRID77. spoke in a firm voice Original CoreGRID78. had speech disturbances Original CoreGRID79. spoke more rapidly Original CoreGRID80. spoke more slowly Original CoreGRID81. tensed their upper body New item82. slumped New item83. held their head up high New item84. hung their head low New item

Note. Items marked as part of the original GRID or CoreGRID (i.e., a short version of the original GRID) were developed by Fontaine et al. (2013) and included in the AECG without further adaptation. Descriptions of emotion components and component-specific feature rating instructions provided to participants are provided in Table S3.


Table S2Emotion Terms Included in the English, German, Spanish, and Chinese Versions of the AECG Emotion list English German Spanish Chinese b

2 joy Freude alegría 喜悦 (xǐ yuè)1 hope Hoffnung esperanza 希望 (xī wàng)2 pride Stolz orgullo 自豪 (zì háo)1 relief Erleichterung alivio 放松 (fàng song)2 contentment Zufriedenheit satisfacción 满意 (mǎn yì)2 anxiety Angst ansiedad 焦虑 (jiāo lǜ)1 anger Ärger ira 愤怒 (fèn nù)2 frustration Frustration frustración 挫败 (cuò bài)2 shame Scham vergüenza 羞愧 (xiū kuì)1 guilt Schuld culpa 内疚 (nèi jiù)1 disappointment Enttäuschung decepción 失望 (shī wàng)1 sadness Traurigkeit tristeza 悲伤 (bēi shāng)2 hopelessness Hoffnungslosigkeit desesperanza 无望 (wú wàng)1 despair Verzweiflung desesperación 绝望 (jué wàng)2 boredom Langeweile aburrimiento 厌烦 (yàn fán)1 surprise Überraschung sorpresa 惊讶 (jīng yà)

Note. The English AECG was translated into German, Colombian Spanish, and Mandarin Chinese by bilingual translator teams using the translation-backtranslation procedure outlined in Fontaine et al. (2013, chapter 6).a The AECG was administered in two sessions with a minimum of a one hour break in between. Each session covered one of two emotion lists comprising eight emotions each, as indicated above.b The AECG was presented using Simplified Chinese characters. Corresponding pinyin transcriptions are provided in parentheses.


Table S3

Descriptions of Emotion Components and Component-Specific Feature Rating Instructions Provided to Participants (English AECG)

Emotion Component Description Feature Rating InstructionsAffective This section lists features describing the feeling referred to by the

emotion words related to success or failure.If a person uses the following emotion words (in the left-hand column) to describe an emotional experience related to success or failure, how likely is it that this person felt…

Cognitive This section lists features describing the evaluation or appraisal (conscious or not) of the success or failure that led to the emotion.

If a person uses the following emotion words (in the left-hand column) to describe an emotional experience related to success or failure, how likely is it that…

Motivational This section lists features describing tendencies to behave in certain ways that may occur during an emotional experience related to success or failure.

If a person uses the following emotion words (in the left-hand column) to describe an emotional experience related to success or failure, how likely is it that this person…

Physiological This section lists features describing the bodily reactions that can occur during an emotional experience related to success or failure.

If a person uses the following emotion words (in the left-hand column) to describe an emotional experience related to success or failure, how likely is it that this person had the following bodily reactions…

Expressive This section lists features describing the facial, vocal and postural expressions that occur during emotional experiences related to success or failure.

If a person uses the following emotion words (in the left-hand column) to describe an emotional experience related to success or failure, how likely is it that this person…

Note. Feature rating instructions were followed by the specific feature item to be rated. Features are listed in Table S1.


Table S4Intraclass Correlations (Including 95% Confidence Intervals) and Average Participant-Rest Correlations Indicating Within-Sample Agreement on Full Feature Profiles of Emotions

CA (N = 25) GE (N = 29) CO (N = 42) CH (N = 30)Emotion ICC r̅i(t-i) ICC r̅i(t-i) ICC r̅i(t-i) ICC r̅i(t-i)

joy.97

[.96, .98].75 .98

[.97, .99].79 .99

[.98, .99].82 .97

[.96, .98].75

hope.94

[.92, .96].63 .96

[.95, .97].68 .98

[.97, .98].72 .95

[.94, .97].66

pride.96

[.95, .97].71 .98

[.97, .98].75 .98

[.97, .98].72 .98

[.97, .98].77

relief.92

[.89, .94].56 .96

[.94, .97].66 .98

[.98, .99].76 .95

[.94, .97].67

contentment.94

[.92, .96].65 .98

[.97, .99].79 .99

[.98, .99].80 .97

[.96, .98].73

anxiety.96

[.95, .97].70 .97

[.96, .98].74 .97

[.96, .98].66 .95

[.93, .96].62

anger.95

[.94, .97].68 .97

[.95, .98].70 .98

[.97, .99].75 .96

[.94, .97].65

frustration.95

[.93, .96].67 .95

[.94, .97].65 .97

[.97, .98].72 .95

[.93, .96].62

shame.95

[.94, .97].68 .97

[.95, .98].71 .98

[.98, .99].76 .95

[.93, .96].61

guilt.95

[.93, .96].66 .97

[.95, .98].70 .97

[.97, .98].69 .93

[.91, .95].56

disappointment.94

[.93, .96].65 .96

[.95, .97].70 .98

[.97, .98].71 .94

[.92, .96].60

sadness.96

[.94, .97].72 .97

[.97, .98].77 .98

[.98, .99].78 .95

[.93, .96].60

hopelessness.96

[.94, .97].70 .97

[.96, .98].71 .98

[.97, .99].73 .95

[.94, .97].63

despair.95

[.94, .97].68 .97

[.96, .98].71 .98

[.97, .98].72 .96

[.95, .97].66

boredom.92

[.89, .94].59 .94

[.92, .95].59 .98

[.98, .99].75 .95

[.93, .96].61

surprise.94

[.92, .96].65 .96

[.95, .97].70 .96

[.94, .97].60 .89

[.85, .92].47

Note. Intraclass correlations (ICC) reported here were computed by means two-way random effects models of the type ‘average measure [mean] of k raters’ and quantify absolute agreement. They correspond to ICC(2, k) in the Shrout-Fleiss convention (Shrout & Fleiss, 1979) and ICC(A, k) in the McGraw-Wong convention (McGraw & Wong, 1996), where k denotes the number of raters per sample. Emotion features were treated as ‘subjects’ (i.e., the rated targets) and participants as variables (i.e., raters). Average participant-rest correlations (r̅i(t-i)) indicate average similarity of an individual’s emotion feature ratings with the mean ratings of that sample.


Table S5Intraclass Correlations (Including 95% Confidence Intervals; Brackets) and Average Participant-Rest Correlations (Parentheses) Indicating Within-Sample Agreement on Feature Profiles of Emotion Components

CA (N = 25) GE (N = 29) CO (N = 42) CH (N = 30)Emotion aff cog mot phys exp aff cog mot phys exp aff cog mot phys exp aff cog mot phys exp

joy

.98 [.95, .99]

(.83)

.96 [.93, .98]

(.70)

.97 [.95, .99]

(.82)

.95 [.91, .98]

(.70)

.97 [.95, .99]

(.79)

.98 [.97, .99]

(.82)

.97 [.96, .99]

(.77)

.98 [.97, .99]

(.82)

.98 [.96, .99]

(.80)

.98 [.97, .99]

(.82)

.99 [.97, 1.00] (.89)

.98 [.97, 1.00] (.73)

.99 [.99, 1.00] (.89)

.98 [.96, .99]

(.78)

.99 [.98, 1.00] (.88)

.99 [.97, 1.00] (.84)

.97 [.95, .98]

(.72)

.98 [.96, .99]

(.78)

.97 [.93, .99]

(.74)

.97 [.95, .99]

(.75)

hope

.96 [.90, .99]

(.70)

.89 [.83, .94]

(.51)

.97 [.94, .97]

(.75)

.87 [.74, .96]

(.52)

.94 [.89, .98]

(.68)

.98 [.95, .99]

(.77)

.91 [.85, .95]

(.52)

.98 [.96, .99]

(.78)

.96 [.91, .99]

(.67)

.97 [.94, .99]

(.74)

.98 [.96, .99]

(.70)

.98 [.93, .98]

(.58)

.99 [.98, .99]

(.84)

.93 (.85, .97]

(.47)

.98 [.96, .99]

(.74)

.97 [.93, .99]

(.73)

.92 [.88, .96]

(.55)

.98 [.96, .99]

(.80)

.93 [.87, .98]

(.93)

.94 [.90, .98]

(.62)

pride

.98 [.95, .99]

(.82)

.95 [.92, .97]

(.66)

.96 [.93, .98]

(.74)

.93 [.85, .97]

(.65)

.97 [.94, .99]

(.79)

.99 [.97, 1.00] (.84)

.97 [.95, .98]

(.70)

.97 [.95, .99]

(.73)

.96 [.93, .99]

(.71)

.99 [.97, .99]

(.86)

.99 [.97, 1.00] (.80)

.96 [.94, .98]

(.62)

.98 [.987, .99] (.74)

.94 [.89, .98]

(.56)

.99 [.98, .99]

(.81)

.99 [.97, 1.00] (.85)

.97 [.95, .99]

(.73)

.97 [.95, .99]

(.77)

.96 [.92, .99]

(.71)

.98 [.96, .99]

(.81)

relief

.96 [.91, .99]

(.71)

.90 [.83, .94]

(.51)

.90 [.82, .96]

(.54)

.90 [.80, .97]

(.51)

.92 [.85, .97]

(.60)

.98 [.96, .99]

(.81)

.94 [.91, .97]

(.61)

.96 [.94, .98]

(.71)

.95 [.89, .98]

(.63)

.96 [.92, .98]

(.67)

.99 [.98, 1.00] (.85)

.97 [.95, .98]

(.66)

.99 [.97, .99]

(.82)

.97 [.95, .99]

(.70)

.98 [.97, .99]

(.76)

.97 [.93, .99]

(.74)

.94 [.90, .97]

(.61)

.95 [.91, .98]

(.66)

.92 [.84, .97]

(.53)

.95 [.92, .98]

(.69)

contentment

.97 [.93, .99]

(.73)

.93 [.88, .96]

(.63)

.95 [.90, .98]

(.66)

.93 [.85, .97]

(.63)

.93, [.87, .97]

(.63)

.99 [.98, 1.00] (.88)

.97 [.96, .99]

(.77)

.97 [.95, .99]

(.76)

.98 [.95, .99]

(.79)

.98 [.97, .99]

(.81)

(.99 [.98,

1.00] .85)

.98 [.97, .99]

(.74)

.99 [.98, 1.00] (.87)

.95 [.91, .98]

(.60)

.99 [.98, 1.00] (.87)

.98 [.96, .99]

(.82)

.97 [.95, .98]

(.72)

.97 [.95, .99]

(.76)

.93 [.85, .98]

(.55)

.97 [.95, .99]

(.75)

anxiety

.98 [.95, .99]

(.81)

.93 [.89, .96]

(.60)

.96 [.93, .98]

(.70)

.97 [.94, .99]

(.77)

.96 [.93, .98]

(.73)

.99 [.97, 1.00] (.85)

.96 [.93, .98]

(.67)

.97 [.94, .98]

(.72)

.98 [.97, 1.00] (.86)

.97 [.94, .99]

(.72)

.99 [.97, 1.00] (.78)

.95 [.91, .97]

(.54)

.96 [.93, .98]

(.63)

.99 [.97 1.00] (.81)

.96 [.92, .98]

(.61)

.98 [.95, .99]

(.78)

.93 [.89, .96]

(.58)

.96 [.92, .98]

(.96)

.92 [.84, .97]

(.54)

.95 [.91, .98]

(.63)

anger

.96 [.91, .99]

(.68)

.93 [.89, .96]

(.63)

.95 [.90, .98]

(.67)

.98 [.95, .99]

(.82)

.96 [.92, .98]

(.70)

.98 [.95, .99]

(.78)

.96 [.94, .98]

(.70)

.95 [.91, .98]

(.70)

.98 [.96, .99]

(.79)

.96 [.93, .98]

(.68)

.99 [.97, 1.00] (.78)

.97 [.94, .98]. (.64)

.98 [.97, .99]

(.77)

.99 [.98, 1.00] (.84)

.99 [.97, .99]

(.80)

.96 [.91, .99]

(.66)

.93 [.89, .96]

(.55)

.95 [.92, .98]

(.65)

.98 [.96, .99]

(.81)

.97 [.94, .99]

(.71)

frustration

.97 [.94, .99]

(.76)

.93 [.89, .97]

(.63)

.96 [.92, .98]

(.72)

.96 [.91, .98]

(.72)

.95 [.91, .98]

(.69)

.98 [.96, .99]

(.81)

.96 [.94, .98]

(.69)

.95 [.91, .98]

(.65)

.90 [.79, .96]

(.50)

.94 [.88, .97]

(.59)

.99 [.98, 1.00] (.88)

.96 [.94,.98] (.62)

.98 [.97, .99]

(.80)

.75 [.51, .91]

(.25)

.99 [.97, .99]

(.80)

.97 [.93, .99]

(.69)

.91 [.86, .95]

(.51)

.96 [.93, .98]

(.66)

.85 [.70, .95]

(.41)

.97 [.95, .99]

(.76)shame .97

[.93, .99]

.92 [.87, .96]

.97 [.94, .97]

.83 [.66, .94]

.98 [.95, .99]

.98 [.95, .99]

.94 [.91, .97]

.97 [.94, .99]

.94 [.90, .98]

.98 [.97, .99]

.99 [.98, 1.00]

.96 [.94, .98]

.99 [.98, .99]

.97 [.94, .99]

.99 [.98, 1.00]

.95 [.89, .98]

.91 [.86, .95]

.95 [.91, .98]

.93 [.85, .97]

.97 [.95, .99]


CA (N = 25) GE (N = 29) CO (N = 42) CH (N = 30)Emotion aff cog mot phys exp aff cog mot phys exp aff cog mot phys exp aff cog mot phys exp

(.76) (.56) (.78) (.40) (.81) (.80) (.61) (.72) (.62) (.83) (.83) (.61) (.82) (.67) (.87) (.61) (.50) (.65) (.57) (.74)

guilt

.95 [.89, .98]

(.66)

.94 [.91, .97]

(.63)

.95 [.92, ,98]

(.73)

.90 [.79, .96]

(.54)

.97 [.95, .99]

(.78)

.98 [.95, .99]

(.79)

.96 [.93, .98]

(.65)

.96 [.94, ,98]

(.71)

.94 [.89, .98]

(.63)

.98 [.96, .99]

(.78)

.99 [.98, 1.00] (.86)

.96 [.94, .98]

(.62)

.98 [.97, .99]

(.72)

.84 [.68, .94] .3

5)

.98 [.97, .99]

(.80)

.94 [.86, .98]

(.60)

.93 [.89, .96]

(.55)

.93 [.87, .97]

(.56)

.67 [.35, .88]

(.25)

.97 [.94, .99]

(.71)

disappointment

.96 [.92, .99]

(.71)

.94 [.90, .97]

(.63)

.93 [.87, .97]

(.67)

.87 [.73, .95]

(.47)

.97 [.95, .99]

(.80)

.98 [.95, .99]

(.80)

.96 [.93, .98]

(.67)

.95 [.91, .98]

(.65)

.95 [.90, .98]

(.64)

.98 [.97, .99]

(.82)

.99 [.98, 1.00] (.86)

.96 [.93, .98]

(.60)

.98 [.97, .99]

(.76)

.88 [.76, .96]

(.38)

.99 [.98, .99]

(.84)

.96 [.91, .99]

(.69)

.93 [.89, .96]

(.55)

.94 [.88, .97]

(.58)

.91 [.81, .97]

(.51)

.97 [.94, .99]

(.75)

sadness

.97 [.94, .99]

(.78)

.94 [.90, .97]

(.63)

.95 [.91, .98]

(.71)

.94 [.89, .98]

(.70)

.98 [.96, .99]

(.86)

.98 [.96, .99]

(.80)

.96 [.94, .98)

(.71)

.95 [.91, .98]

(.63)

.98 [.96, .99]

(.80)

.99 [.98, 1.00] (.90)

.99 [.98, 1.00] (.87)

.96 [.94, .98] (.62)

.99 [.98, .99]

(.82)

.96 [.93, .99]

(.65)

.99 [.99, 1.00] (.92)

.96 [.91, .99]

(.63)

.93 [.88, .96]

(.54)

.94 [.89, .97]

(.58)

.91 [.82, .97]

(.49)

.98 [.95, .99]

(.77)

hopelessness

.98 [.95, .99]

(.81)

.91 [.86, .95]

(.54)

.97 [.94, .99]

(.76)

.88 [.75, .96]

(.53)

.98 [.96, .99]

(.84)

.98 [.96, 1.00] (.82)

.95 [.93, .98]

(.65)

.96 [.93, .98]

(.67)

.96 [.93, .99]

(.70)

.98 [.97, .99]

(.84)

.99 [.98, 1.00] (.85)

.95 [.93, .98]

(.58)

.98 [.97, .99]

(.77)

.96 [.92, .99]

(.61)

.99 [.99, 1.00] (.89)

.96 [.92, .99]

(.68)

.93 [.89, .96]

(.55)

.97 [.94, .98]

(.70)

.95 [.90, .98]

(.62)

.97 [.94, .99]

(.72)

despair

.96 [.92, .99]

(.73)

.93 [.89, .96]

(.61)

.97 [.94, .98]

(.77)

.91 [.81, .97]

(.54)

.98 [.96, .99]

(.81)

.98 [.96, 1.00] (.83)

.96 [.94, .98]

(.71)

.96 [.93, .98]

(.70)

.95 [.90, .98]

(.66)

.98 [.96, .99]

(.77)

.99 [.98, 1.00] (.83)

.96 [.94, .98]

(.64)

.98 [.97, .99]

(.77)

.98 [.96, .99]

(.78)

.98 [.96, .99]

(.72)

.97 [.94, .99]

(.71)

.94 [.91, .97]

(.60)

.98 [.93, .98]

(.66))

.95 [.91, .98]

(.65)

.98 [.96, .99]

(.78)

boredom

.89 [.77, .97]

(.49)

.85 [.75, .92]

(.50)

.93 [.87, .97]

(.61)

.96 [.91, .99]

(.76)

.94 [.89, .98]

(.65)

.95 [.89, .98]

(.63)

.81 [.69, .90]

(.39)

.92 [.86, .97]

(.53)

.98 [.95, .99]

(.78)

.96 [.93, .99]

(.69)

.99 [.98, 1.00] (.84)

.95 [.91, .97]

(.56)

.98 [.97, .99]

(.79)

.98 [.96, .99]

(.77)

.98 [.99, 1.00] (.88)

.97 [.95, .99]

(.77)

.93 [.88, .96]

(.55)

.96 [.94, .98]

(.70)

.81 [.63, .94]

(.34)

.95 [.90, .98]

(.62)

surprise

.94 [.86, .98]

(.62)

.93 [.89, .96]

(.62)

.59 [.32, .81]

(.28)

.97 [.94, .99]

(.79)

.97 [.95, .99]

(.80)

.98 [.97, 1.00] (.82)

.95 [.92, .97]

(.62)

.91 [.84, .96]

(.49)

.98 [.96, .99]

(.82)

.97 [.95, .99]

(.76)

.96 [.90, .99]

(.59)

.91 [.85, .95]

(.42)

.80 (.64, .91]

(.27)

.97 [.94, .99]

(.69)

.98 [.97, .99]

(.77)

.91 [.80, .97]

(.50)

.73 [.60, .86]

(.29)

.52 [.17, .78]

(.22)

.92 [.84, .97]

(.59)

.95 [.91, .98]

(.63)

Note. Intraclass correlations (ICC) reported here were computed by means two-way random effects models of the type ‘average measure [mean] of k raters’ and quantify absolute agreement. They correspond to ICC(2, k) in the Shrout-Fleiss convention (Shrout & Fleiss, 1979) and ICC(A, k) in the McGraw-Wong convention (McGraw & Wong, 1996), where k denotes the number of raters per sample. For each component, emotion features were treated as ‘subjects’ (i.e., the rated targets) and participants as variables (i.e., raters). Average participant-rest correlations ( r̅i(t-i)) presented in parentheses indicate average similarity of an individual’s emotion component feature ratings with the mean ratings of that sample.


Table S6Cross-Cultural Similarity (Pearson correlations) of Feature Profiles of Emotions Across all 84 Features

Emotion CA-GE CA-CO CA-CH GE-CO GE-CH CO-CH Memotion SDjoy .95 .94 .90 .94 .87 .92 .92 .03hope .92 .87 .91 .81 .90 .84 .87 .04pride .93 .93 .91 .94 .89 .90 .92 .02relief .83 .77 .71 .78 .65 .88 .77 .08contentment .91 .82 .83 .86 .81 .93 .86 .05anxiety .88 .88 .84 .85 .73 .87 .84 .05anger .93 .92 .89 .90 .83 .90 .89 .03frustration .78 .63 .52 .85 .77 .89 .74 .13shame .88 .85 .85 .90 .81 .85 .86 .03guilt .88 .85 .85 .90 .81 .85 .86 .03disappointment .90 .89 .87 .88 .83 .87 .87 .02sadness .92 .91 .92 .91 .88 .90 .91 .01hopelessness .92 .91 .89 .91 .87 .91 .91 .02despair .87 .67 .90 .79 .78 .56 .76 .12boredom .89 .73 .25 .67 .18 .49 .53 .26surprise .89 .88 .68 .85 .72 .78 .80 .08

Mcultural pairing .89 .84 .79 .86 .77 .83 .83 --SD .04 .09 .18 .07 .17 .12 .12 --

Note. CA = Canadian sample, GE = German sample, CO = Colombian sample, and CH = Chinese sample.Given k = 84 features, all rs ≥ .22 are significant at p < .05. Positive correlations indicate higher levelsof similarity, negative correlations indicate lower levels of similarity. Shaded cells highlight the grand mean profile correlation and corresponding standard deviation.


Table S7

Cross-Cultural Similarity (ICC-DE) of Feature Profiles of Emotion Components Affective component Cognitive component Motivational component Physiological component Expressive component

Em.

CA-

GE

CA-

CO

CA-

CH

GE-

CO

GE-

CH

CO-

CH

CA-

GE

CA-

CO

CA-

CH

GE-

CO

GE-

CH

CO-

CH

CA-

GE

CA-

CO

CA-

CH

GE-

CO

GE-

CH

CO-

CH

CA-

GE

CA-

CO

CA-

CH

GE-

CO

GE-

CH

CO-

CH

CA-

GE

CA-

CO

CA-

CH

GE-

CO

GE-

CH

CO-

CHjo .99 .96 .96 .92 .92 .98 .95 .96 .92 .95 .86 .96 .96 .96 .89 .94 .88 .91 .92 .91 .95 .92 .97 .96 .92 .93 .85 .96 .77 .83

ho .96 .93 .93 .92 .94 .96 .91 .85 .86 .84 .86 .93 .95 .98 .96 .96 .94 .98 .88 .20 .83 .00 .87 -.08 .90 .93 .89 .80 .87 .69

pr .98 .97 .98 .96 .97 .96 .91 .97 .93 .95 .91 .96 .92 .96 .85 .91 .86 .83 .92 .86 .95 .91 .98 .96 .89 .88 .88 .96 .80 .87

re .90 .93 .76 .86 .74 .88 .90 .78 .66 .77 .61 .89 .96 .93 .90 .91 .90 .95 .82 .73 .79 .49 .50 .93 .53 .50 .64 .71 .45 .71

co .99 .95 .91 .94 .91 .98 .95 .93 .91 .95 .88 .96 .90 .83 .88 .91 .87 .92 .85 .38 .56 .31 .48 .87 .85 .50 .76 .83 .66 .86

ax .91 .98 .90 .85 .74 .97 .82 .85 .87 .90 .86 .89 .96 .88 .82 .84 .78 .93 .94 .88 .89 .80 .84 .96 .74 .87 .79 .85 .43 .68

ag .94 .98 .83 .91 .69 .86 .89 .91 .88 .90 .85 .87 .97 .89 .81 .88 .81 .96 .98 .96 .96 .89 .91 .98 .92 .89 .92 .97 .90 .85

fr .93 .89 .93 .93 .95 .95 .89 .86 .88 .94 .83 .90 .94 .70 .70 .81 .81 .93 .61 .69 -.30 .55 .26 .08 .48 .16 .08 .87 .69 .92

sh .93 .91 .87 .90 .87 .94 .81 .70 .79 .85 .81 .81 .97 .93 .92 .91 .93 .89 .79 .67 .50 .73 .60 .89 .86 .89 .92 .99 .76 .83

gu .98 .97 .89 .96 .90 .92 .92 .80 .71 .76 .69 .82 .95 .97 .86 .94 .84 .85 .89 .79 .56 .62 .75 .34 .84 .86 .87 .99 .87 .90

di .97 .97 .95 .94 .95 .94 .85 .85 .85 .87 .81 .88 .93 .89 .85 .82 .78 .92 .93 .86 .88 .88 .88 .84 .89 .92 .90 .97 .85 .87

sa .98 .96 .98 .93 .97 .98 .85 .86 .94 .85 .87 .87 .93 .91 .90 .87 .84 .90 .98 .97 .91 .97 .92 .92 .89 .93 .94 .99 .91 .94

hl .98 .95 .96 .96 .95 .95 .89 .85 .91 .88 .84 .89 .93 .96 .94 .90 .89 .93 .97 .73 .78 .82 .87 .94 .92 .94 .92 .98 .90 .94

de .90 .88 .96 .95 .89 .87 .88 .83 .85 .87 .84 .83 .85 .89 .95 .93 .76 .86 .76 -.23 .85 .34 .38 -.57 .94 .57 .93 .71 .87 .44

bo .86 .72 .58 .67 .50 .92 .85 .45 .22 .26 -.02 .84 .93 .83 .55 .80 .51 .77 .90 .85 -.85 .94 -.79 -.79 .90 .81 .15 .62 .13 .00

su .94 .92 .90 .83 .91 .92 .91 .70 .51 .95 .42 .79 .71 .75 .50 .83 .56 .73 .91 .94 .89 .90 .97 .86 .92 .97 .62 .94 .72 .68

Mcu. .95 .93 .89 .90 .86 .94 .87 .82 .79 .84 .74 .88 .92 .89 .83 .88 .81 .89 .88 .70 .63 .69 .65 .57 .84 .78 .75 .88 .72 .75SD .04 .06 .10 .07 .13 .04 .04 .12 .19 .16 .23 .05 .06 .08 .13 .05 .12 .07 .09 .31 .48 .27 .43 .57 .13 .22 .26 .12 .21 .23Mco. .91 .83 .87 .69 .79

SD .08 .16 .10 .41 .21

Note. Em = Emotion word. CA = Canadian sample, GE = German sample, CO = Colombian sample, and CH = Chinese sample. Emotion terms: jo = joy, ho = hope, pr = pride, re = relief, co = contentment, ax = anxiety, ag = anger, sh = shame, gu = guilt, fr = frustration, de = despair, di = disappointment, sa = sadness, hl = hopelessness, bo = boredom, su = surprise. Mcu = mean for respective cultural pairing, Mco = mean for respective emotion component. Note that the corresponding mean correlations across components for each emotion can be obtained from Table S5. For the affective component, rs ≥ .55 are significant at p < .05 (based on k = 10 features). For the cognitive component, rs ≥ .32 are significant at p < .05 (based on k = 28 features). For the motivational component, rs ≥ .41 are significant at p < .05 (based on k = 18 features). For the physiological component, rs ≥ .50 are significant at p < .05 (based on k = 12 features). For the expressive component, rs ≥ .43 are significant at p < .05 (based on k = 16 features).Positive rs indicate higher levels of similarity, negative rs indicate lower levels of similarity.


Table S8

95% Confidence Intervals for Canadian, German, and Chinese Ratings of Affective Valence of Achievement-Related Suprise

Feature ‘felt good’ Feature ‘felt bad’UL LL UL LL

Canada 5.44 6.40 4.27 5.33Germany 5.71 6.57 3.14 4.18China 5.30 6.30 3.93 5.01


Table S9Feature-Level Comparisons of Aggregated Canadian, German, and Colombian Feature Ratings with Chinese Ratings for the Physiological Component of Boredom

Feature MCA,GE,CO SD MCH SD t(124) pcorr a d

rapid breathing 2.48 1.14 5.87 1.33 -13.8 < .000 -2.88rapid heartrate 2.66 1.27 5.43 1.50 -9.99 < .000 -2.09slowed breathing 6.71 1.34 4.17 1.44 8.92 < .000 1.87slowed heartrate 6.59 1.54 3.97 1.43 8.30 < .000 1.74blushed 2.51 1.32 4.90 1.79 -7.89 < .000 -1.65sweating 2.70 1.41 5.07 1.71 -7.63 < .000 -1.60tense muscles 3.01 1.60 5.40 1.38 -7.35 < .000 -1.54feeling warm 3.63 1.39 5.63 1.54 -6.71 < .000 -1.40stomach disturbance 3.78 1.83 5.73 1.46 -5.33 < .000 -1.11feeling weak in the limbs 5.66 2.13 4.50 1.15 2.85 .015 0.60feeling cold 5.38 1.84 4.40 1.54 2.63 .019 0.55becoming pale 4.25 1.93 4.97 1.35 -1.89 .060 -0.40

Note. CA = Canadian sample, GE = German sample, CO = Colombian sample, and CH = Chinese sample. a Bonferroni-Holm-corrected p-value (based on α = .05).


6. Figures S1-S4

Figure S1. Instructions for rating the likelihood of occurrence (9-point scale) of different features of emotional experience (here: sample feature for the motivational component) for different achievement emotions in the AECG.

CA-GE CA-CO CA-CH GE-CO GE-CH CO-CH0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Figure S2. Pairwise mean cross-cultural emotion profile similarity (ICC-DE) across emotion words and emotion features. CA = Canadian sample, GE = German sample, CO = Colombian sample, and CH = Chinese sample. Error bars indicate standard errors based on n = 16 emotions.


jo ho pr re co ax ag fr sh gu di sa hl de bo su0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Figure 3. Mean cross-cultural emotion profile similarity (ICC-DEs) across all emotion features and cultural comparisons. Emotion words: jo = joy, ho = hope, pr = pride, re = relief, co = contentment, ax = anxiety, ag = anger, sh = shame, gu = guilt, fr = frustration, de = despair, di = disappointment, sa = sadness, hl = hopelessness, bo = boredom, su = surprise. Error bars indicate standard errors based on n = 6 cultural comparisons per emotion word.

Affective Cognitive Motivational Physiological Expressive 0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

Figure 4. Mean cross-cultural emotion profile agreement (ICC-DEs) for respective emotion component across cultural comparisons and emotion words. Error bars indicate standard errors based on n = 96 ICC-DEs (16 emotion words x 6 cultural comparisons).


7. References Cited in the Supplemental Material

Berger, J.-L., & Karabenick, S. A. (2016). Construct validity of self-reported metacognitive learning strategies. Educational Assessment, 21, 19–33. https://doi.org/10.1080/10627197.2015.1127751

Boredom [Def.]. (n.d.). In Cambridge Dictionary Online, Retrieved August 8, 2019, from https://dictionary.cambridge.org/dictionary/english-chinese-simplified/boredom.

Byrne, B. M., & van de Vijver, F. J. R. (2010). Testing for measurement and structural equivalence in large-scale cross-cultural studies: Addressing the issue of nonequivalence. International Journal of Testing, 10, 107–132. https://doi.org/10.1080/15305051003637306

Cattell, R. B. (1945). The description of personality: Principles and findings in a factor analysis. American Journal of Psychology, 58, 69–90. https://doi.org/10.2307/1417576

D'Mello, S., & Graesser, A. (2009). Automatic detection of learner's affect from gross body language. Applied Artificial Intelligence, 23, 123–150. https://doi.org/10.1080/08839510802631745

De Raad, B., & Mlačić, B. (2017). The lexical foundation of the big five factor model. In T. A. Widiger (Ed.), The Oxford handbook of the five factor model (pp. 191–216). Oxford: Oxford University Press.

Ekman, P., & Friesen, W. V. (1969). The repertoire of nonverbal behavior: Categories, origins, usage, and coding. Semiotica, 1, 49–98. https://doi.org/10.1515/semi.1969.1.1.49

Fontaine, J. J. R., Scherer, K. R., & Soriano, C. (Eds.). (2013). Components of emotional meaning: A sourcebook. Oxford: Oxford University Press.

Fredrickson, B. L. (2001). The role of positive emotions in positive psychology: The broaden-and-build theory of positive emotions. American Psychologist, 56, 218–226. https://doi.org/10.1037/0003-066X.56.3.218.

Frijda, N. H., Kuipers, P., & ter Schure, E. (1989). Relations among emotion, appraisal, and emotional action readiness. Journal of Personality and Social Psychology, 57, 212–228. https://doi.org/10.1037/0022-3514.57.2.212

Furr, R. M. (2010). The double-entry intraclass correlation as an index of profile similarity: Meaning, limitations, and alternatives. Journal of Personality Assessment, 92, 1–15. https://doi.org/10.1080/00223890903379134

Goetz, T., Frenzel, A. C., Hall, N. C., Nett, U. E., Pekrun, R., & Lipnevich, A. A. (2014). Types of boredom: An experience sampling approach. Motivation and Emotion, 38, 401–419. https://doi.org/10.1007/s11031-013-9385-y

Graham, S., & Taylor, A. Z. (2014). An attributional approach to emotional life in the classroom. In R. Pekrun & L. Linnenbrink-Garcia (Eds.), International handbook of emotions in education (pp. 96–119). New York, NY: Routledge.

Hřebíčková, M., Mõttus, R., Graf, S., Jelínek, M., Realo, A., & Jackson, J. (2018). How accurate are national stereotypes? A test of different methodological approaches. European Journal of Personality, 32, 87–99. https://doi.org/10.1002/per.2146


Karabenick, S. A., Woolley, M. E., Friedel, J. M., v. Ammon, B., Blazevski, J., Rhee Bonney, C., . . . Kelly, K. L. (2007). Cognitive processing of self-report items in educational research: Do they think what we mean? Educational Psychologist, 42, 139–151. https://doi.org/10.1080/00461520701416231

Kenny, D. A., Kashy, D. A., & Cook, W. L. (2006). Dyadic data analysis. New York, NY: Guilford Press.

Koskey, K. L.K., Karabenick, S. A., Woolley, M. E., Bonney, C. R., & Dever, B. V. (2010). Cognitive validity of students’ self-reports of classroom mastery goal structure: What students are thinking and why it matters. Contemporary Educational Psychology, 35, 254–263. https://doi.org/10.1016/j.cedpsych.2010.05.004

McCrae, R. R. (2008). A note on some measures of profile agreement. Journal of Personality Assessment, 90, 105–109. https://doi.org/10.1080/00223890701845104

McCrae, R. R., & Terracciano, A. (2005). Personality profiles of cultures: Aggregate personality traits. Journal of Personality and Social Psychology, 89, 407–425. https://doi.org/10.1037/0022-3514.89.3.407

McGraw, K. O., & Wong, S. P. (1996). Forming inferences about some intraclass correlation coefficients. Psychological Methods, 1, 30–46. https://doi.org/10.1037/1082-989X.1.1.30

Messick, S. (1995). Validity of psychological assessment: Validation of inferences from persons' responses and performances as scientific inquiry into score meaning. American Psychologist, 50, 741-749. https://doi.org/10.1037/0003-066X.50.9.741

Osgood, C. (1957). The nature and measurement of meaning. Urbana: University of Illinois Press.

Pekrun, R., Goetz, T., Titz, W., & Perry, R. P. (2002). Academic emotions in students’ self-regulated learning and achievement: A program of qualitative and quantitative research. Educational Psychologist, 37, 91–105. https://doi.org/10.1207/S15326985EP3702_4

Pekrun, R., & Perry, R. P. (2014). Control-Value Theory of Achievement Emotions. In R. Pekrun & L. Linnenbrink-Garcia (Eds.), Educational Psychology Handbook. International Handbook of Emotions in Education (pp. 120–141). New York: Taylor and Francis.

Reisenzein, R., Junge, M., Studtmann, M., & Huber, O. (2014). Observational Approaches to the Measurement of emotions. In R. Pekrun & L. Linnenbrink-Garcia (Eds.), International Handbook of Emotions in Education (pp. 580–606). New York: Taylor and Francis.

Rogers, K. H., Wood, D., & Furr, R. M. (2017). Assessment of similarity and self-other agreement in dyadic relationships. Journal of Social and Personal Relationships, 35, 112–134. https://doi.org/10.1177/0265407517712615

Russell, J. A. (1980). A circumplex model of affect. Journal of Personality and Social Psychology, 39, 1161–1178. https://doi.org/10.1037/h0077714

Russell, J. A. (2003). Core affect and the psychological construction of emotion. Psychological Review, 110, 145–172. https://doi.org/10.1037/0033-295X.110.1.145

Scherer, K. R., & Fontaine, J. R. J. (2019). The semantic structure of emotion words across languages is consistent with componential appraisal models of emotion. Cognition and Emotion, 33, 673–682. https://doi.org/10.1080/02699931.2018.1481369


Scherer, K. R. S., & Moors, A. (2019). The emotion process: Event appraisal and component differentiation. Annual Review of Psychology, 70, 719–745. https://doi.org/10.1146/annurev-psych-122216-011854

Shrout, P. E., & Fleiss, J. L. (1979). Intraclass correlations: Uses in assessing rater reliability. Psychological Bulletin, 86, 420–428.

Stemmler, G. (2003). Methodological considerations in the psychophysiological study of emotion. In R. J. Davidson, K. R. Scherer, & H. H. Goldsmith (Eds.), Handbook of affective sciences (pp. 225–255). Oxford: Oxford University Press.

Turner, J. E., & Schallert, D. L. (2001). Expectancy-value relationships of shame reactions and shame resiliency. Journal of Educational Psychology, 93, 320–329. https://doi.org/10.1037/0022-0663.93.2.320

Weiner, B. (1985). An attributional theory of achievement motivation and emotion. Psychological review, 92, 548–573. https://doi.org/10.1037/0033-295X.92.4.548

Wood, D., & Furr, R. M. (2016). The correlates of similarity estimates are often misleadingly positive: The nature and scope of the problem, and some solutions. Personality and Social Psychology Review, 20, 79–99. https://doi.org/10.1177/1088868315581119

Yik, M. S. M., Russell, J. A., & Barrett, L. F. (1999). Structure of self-reported current affect: Integration and beyond. Journal of Personality and Social Psychology, 77, 600–619. https://doi.org/10.1037/0022-3514.77.3.600

Zeidner, M. (1998). Test anxiety. The state of the art. New York: Plenum.Zhao, S., & Cai, T. (2012). 大学生学业情绪量表_AEQ_中文版的修订 [Reliability and

validity of AEQ in Chinese]. Chinese Journal of Clinical Psychology, 20, 448–450.

s3-eu-west-1.amazonaws.com · Web viewConcepts of Achievement-Related Emotions Across Cultures:...

Documents

Transcript of s3-eu-west-1.amazonaws.com · Web viewConcepts of Achievement-Related Emotions Across Cultures:...