Satisfaction measurements in a cross-cultural environment: How valid are subjective judgements?...

University of Cologne

Center for Evaluation Faculty of Management, Economics and Social Sciences

Dr. Christian Bosau

Satisfaction measurements in a cross-cultural environment:

How valid are subjective judgements?

Bosau – satisfaction measurements in a cross-cultural environment 19. – 22.09.2011 – Statistische Woche – Leipzig Seite 2

Background of the study

Are we really measuring „true“ satisfation?

Job satisfaction (JS) in organizations: §  keyword: international employee satisfaction survey §  the absolute level of JS often has direct consequences for leaders §  intercultural measurement problems often are NOT considered

Main question: How can we compare the results of JS across national and cultural borders?


Cross-cultural JS-results

8,4 8,48,2

8,18,0 7,9 7,9 7,8 7,8 7,7 7,7 7,7 7,7 7,7 7,6 7,6 7,6 7,6 7,6 7,6 7,5 7,5 7,5 7,5

7,3 7,3 7,27,1 7,1 7,1 7,1 7,1 7,0 7,0 7,0 6,9 6,9 6,9

6,7 6,7 6,6 6,6 6,6 6,5

6,26,1

5,9

5,5

5

6

7

8

9

Switz

erla

nd

Mal

ta

D

enm

ark

Nor

way

Can

ada

I

cela

nd

I

rela

nd

A

ustr

ia

U

SA

Swed

en

B

elgi

um

N

orth

Irel

and

Mex

ico

Japa

n

A

rgen

tina

Chi

le

F

inla

nd

Net

herl

ands

Bra

zil

Lux

embo

urg

Pola

nd

N

iger

ia

Po

rtug

al

U

nite

d K

ingd

om

Italy

G

erm

any

- Wes

t

Sl

oven

ia

C

zech

Rep

ublic

H

unga

ry

Sout

h A

fric

a - W

hite

G

erm

any

- Eas

t

Spai

n

Indi

a

Chi

na

Sl

ovak

ia

Fran

ce

Li

thua

nia

Gre

ece

C

roat

ia

E

ston

ia

R

uman

ia

Latv

ia

B

ulga

ria

Kor

ea

Rus

sia

Turk

ey

U

krai

ne

B

elar

us

•  results from World-Values-Survey:

•  very often: deskriptive intercultural results are published (Slocum & Topichak, 1972; Lincoln et al, 1981; Griffeth & Hom, 1987; Near & Rechner, 1993; Chiu & Kosinski, 1999; van de Vliert & Janssen, 2002; Llorente & Macias, 2005)

Unclear: Can we really just interpret those descriptive differences of JS directly?


Problems of cross-cultural research

response styles and culture: §  different cultures show different response styles, e.g. acquiescence (ARS) (Hui & Triandis, 1989; Johnson et al., 1997; Chen et al.; 1995; Takahashi et al., 2002; Harzing, 2006)

§  different response style should be understood as different communicational behaviour, not just methodological bias (Smith, 2004; Smith & Fischer R., 2006)

standardising of measures (see Fischer, R.; 2004): with-in-subject, group centering, grand mean centering

problems: §  measures are not independent from each other anymore §  absolute level of measures is lost §  interpretation is possible only in relation to standardising value (group or grand mean, etc.)

What can we do about it?

Note: Especially problematic if we want to compare absolute measurements, like JS-levels of countries/subsidiaries/etc.!


Examples: - Does well-being increase with higher GDP? - The more interesting a job is, the higher the JS is? - etc.

Methodological classificiation

level oriented structure oriented vs.

Examples: - How high is JS in different countries? - In which countries are people more satisfied with their lives? - Are Americans more extroverted than Germans? -  Is the image of my product better in Germany compared to Spain? -  etc.

construct bias method bias vs. item bias vs.

different understanding of construct across cultural

boundaries

bad item translation or culturally inapropriate wording

different response styles


The 3 important constructs

What is the relationship looking like between these constructs?

response style

culture

job satisfaction level

? ?

? -

individualism

acquiescent response style

Known: Negative relationship between individualism and acquiescent response style (ARS)


Method - secondary analysis

§  Data from World-Values-Survey & Eurobarometer

§  operationalisation of acquiescence response style (see Harzing, 2006) : acquiescence-index 5-point-scale: ALL items, having received 5 or 4 MINUS ALL items, having received 2 or 1

§  aggregation to national level

§  correlation of national means (controlled for soziodemographic differences of nations, i.e. age, gender)

Important question: How will the JS-measurement be influenced by response styles?


Nation-level-results - 1

results: §  job satisfaction and ARS are indeed correlated: BUT negatively. §  i.e., high measures of job satisfaction only occur in countries/cultures with small or no ARS response tendency norms Keeping in mind: in all countries respondents are on average satisfied with their jobs §  Note: if ARS exist, respondents do not report dissatisfaction but only lower satisfaction §  Fits with the self-construal results of Markus & Kitayama (1991): interdependent/collectivist people – having higher ARS – are restrained in telling their personal feelings.

-.13 (14)

-.68** (11)

-.59*** (16)

-.71*** (16)

-.68*** (16)

Job Satisfaction (nation-level-mean from

Eurobarometer)1

-.29 (25) ARS-index from Hofstede (2001)

-.47** (23) ARS-index from Harzing (2006)

-.38*** (47) ARS-index 3 from WVS (4-point-scale ‚agree/disagree‘-label)

-.47*** (42) ARS-index 2 from WVS (5-point-scale ‚important/unimportant‘-label)


Job Satisfaction (nation-level-mean from World-Values-Survey)1

1 pearson-correlation coefficient; significance: *** p < .01, ** p < .05, * p < .10; in parentheses: number of countries


Nation-level-results - 2

-.13 (14)

-.68** (11)

-.59*** (16)

-.71*** (16)

-.68*** (16)

Job Satisfaction (nation-level-mean from

Eurobarometer)1

-.29 (25) ARS-index from Hofstede (2001)

-.47** (23) ARS-index from Harzing (2006)


-.47*** (42) ARS-index 2 from WVS (5-point-scale ‚important/unimportant‘-label)


Job Satisfaction (nation-level-mean from World-Values-Survey)1

1 pearson-correlation coefficient; significance: *** p < .01, ** p < .05, * p < .10; in parentheses: number of countries

§  the correlation is higher in european nations (countries of the EU)!

§  Note: european nations have – on average – better working conditions, since economic wealth is higher (compared wordwidely) §  we know: job satisfaction measurements are certainly influenced by working conditions as well – not only communicational norms


Re-test of WVS-data

§  Operationalisation working conditions : GDP (gross domestic product)

§  Median split into good (rich countries) and bad (poor countries) working conditions

Rich countries Poor countries

Job satisfaction (nation-level-mean from World-Values-Survey)

Job satisfaction (nation-level-means from

Word-Values-Survey)

ARS-index 1 from WVS (5-point-scale ‚agree/disagree‘-label) -.51* (22) -.02 (24)

ARS-index 2 from WVS (5-point-scale ‚important/unimportant‘-label) -.51*** (22) -.34 (20)

ARS-index 3 from WVS (4-point-scale ‚agree/disagree‘-label) -.37* (23) -.04 (24)

ARS-index from Harzing (2006) -.61** (13) .08 (10)

ARS-index from Hofstede (2001) -.34 (19) sample to small

§  result: correlation remains important and significant only in rich countries with good working conditions, almost no correlation within poor countries with bad working conditions §  possible interpretation: following communicational norms becomes important only if a sufficient working standard is established

1 pearson-correlation coefficient; significance: *** p < .01, ** p < .05, * p < .10; in parentheses (number of countries)


Re-test of the WVS-data

Very robust effect:

correlation remains the same by including economic indicators: -  HDI -  GDP -  Quality-of-Life-Index

Acquiescent response style

Wor

ld-V

alue

s-Su

rvey

– jo

b sa

tisfa

ctio

n le

vel Group of countries

rich countries poor countries rich countries poor countries


- acquiescent response style

individualism

Negative relationship between acquiescent response style and job satisfaction level (in rich countries)

?

? job satisfaction level

Relationship of culture and job satisfaction: JS-Level is higher in individualistic countries (see Bosau, 2008, as well as Hofstede, Judge, etc.)

- +

New finding



+ -

acquiescent response style

individualism

?

job satisfaction level

-

Hypothesis: Relationship of individualism and JS is only a spurious correlation

Instead: culture à response style à JS



§  statistical analysis: mediation analysis (Baron & Kenney, 1986)

Test of mediation model

analysis by rich countries of WVS (N=21)

§  interpretation: relationship of culture and JS can be (almost completely) mediated by ARS; thus: culture à response tendency à JS-level

total effect Individualism (Hofstede) job satisfaction β = .39* (p=.08)

mediation model

job satisfaction β = .20 (p=.39)

β = -.49** (p=.03) β = -.39 (p=.11)

Individualism (Hofstede)

ARS

§  same analysis with poor countries showed no total and no mediation effect

§  using several indicators for Individualism, ARS and job satisfaction in 8 different mediation models we always get the same pattern of results


§  statistical analysis: stepwise multilevel analysis with HLM 6 (Bryk & Raudenbush)

Multi-Level analysis

analysis by rich countries of WVS (N=21)

ARS individual individual level

national level ARS norm

JS

.072 -1.43* Step 2: ARS norm 5.31** (1)

Step 3: ARS individual .07 .078 4.31** (1)

Step 4: ARS individual – random slope 70.60** (1)

§  result: negative relationship with national ARS norm remains significant; on individual level no clear relationship of ARS and JS

§  same analysis with poor countries showed no effects at all

Step 1: Basic Model, controlled for age & gender

unstand. beta SE increase in model fit Chi-Square (df)

Step 5: ARS norm x ARS individual -.04 .889 0.002 (1)


JS and ARS on the individual level

7.10

7.56

8.03

8.50

8.96

Arbeitszufriedenheit

-1.25 -0.75 -0.25 0.25 0.75

ZustimmungstendenzARS individual

JS in

divi

dual


Conclusion

Results: §  indeed: JS-measurements are influenced by response styles (ARS) §  consequence: measurements from different cultures cannot be compared directly §  national level: negative relationship, i.e. the higher the norm to agree, the more moderate the JS-measurement §  individual level: no clear relationship, i.e. in some countries negative and in some countries positive §  individualism-JS-relationship can be understood as a spurious correlation §  instead: cultural values and their socialization leeds to a specific communication style that in turn influences the JS-measurements

In conclusion: Cross-culturally, we are not measuring „true“ satisfaction. To a great extent we are getting results that are an expression of culturally socialized communication norms!


Life satisfaction

All countries together: N-ARS 5-point-scale

‚agree/disagree‘-label WVS 1

Subjective Well-Being (Diener et al., 1995) -.52*** (40)

Life Satisfaction (Diener et al., 2000) -.36** (30)

Life Satisfaction (Suh & Oishi, 2002) -.50*** (29)

Happiness (Inglehart & Klingemann, 2000) -.42*** (51)

Life Satisfaction (Inglehart & Klingemann, 2000) -.54*** (51)

Ideal Life Satisfaction (Suh & Oishi, 2002) -.29 (29)

N-ARS 5-point-scale


- Poor countries -

N-ARS 5-point-scale


- Rich countries -

Subjective Well-Being (Diener et al., 1995) -.08 (11) -.58*** (21)

Life Satisfaction (Diener et al., 2000) -.02 (11) -.20 (11)

Life Satisfaction (Suh & Oishi, 2002) -.05 (11) -.53* (12)

Happiness (Inglehart & Klingemann, 2000) -.02 (22) -.57*** (21)

Life Satisfaction (Inglehart & Klingemann, 2000) -.19 (22) -.61*** (21)

Ideal Life Satisfaction (Suh & Oishi, 2002) -.03 (11) .02 (12)

Split into poor and rich countries:

1 pearson-correlation coefficient; signifikance: *** p < .01, ** p < .05, *p < .10; in parentheses: number of countries

1 pearson-correlation coefficient; signifikance: *** p < .01, ** p < .05, *p < .10; in parentheses: number of countries


contact details: Dr. Christian Bosau Dipl.-Psych. & Master of HRM & IR

Center for Evaluation Faculty of Management, Economics and Social Sciences University of Cologne Herbert-Lewin-Str.2 50931 Cologne/Germany Tel. +49 (0)221 470-4120 [email protected]

Thanks for your attention

Satisfaction measurements in a cross-cultural environment: How valid are subjective judgements?...

Business

Transcript of Satisfaction measurements in a cross-cultural environment: How valid are subjective judgements?...