Race and the beauty premium: Mechanical Turk workers’

evaluations of Twitter accountsAnne Groggel, Shirin Nilizadeh, Yong-Yeol Ahn, Apu Kapadia & Fabio RojasJournal of Information, Communication & Society, vol. 22, no. 5, 709-716

The Ugly Duchess (c. 1513), Quentin Matsys,National Gallery, London

Venus de Milo (c. 130BC), Alexandros(?),Louvre Museum, Paris

Question

• In offline, face to face interactions, it is widely known that race, gender, and attractiveness strongly affect perceptions of competence, trustworthiness, and personality traits.

• Does this hold in online?

https://www.mturk.com

• Use Gardenhorse Twitter stream from February 2015 (10% sample of entire public Twitter timeline)

• 816 Twitter profiles, stratified sampling based on # of followers

• Each profile evaluated by three random workers; each worker evaluated three profiles

• Workers are US residents who are 18+ year old.

Trust and desire to follow twitter account

We found that while trustworthiness was positively associated with whether the workerindicated they would follow the account, this relationship was not statistically significant.

Trust

We find that attractiveness was positively associated with evaluations of trust (p < 0.001).But while a beauty premium appears to lead to positive evaluations of trust for all types ofTwitter accounts, the effect of attractiveness carries greater importance for some Twitterusers than others. Based on our estimates, the premium of good looks was considerablysmaller for White males than they are for White females or Black males.

Very attractive White females had a higher predicted probability of being viewed astrustworthy than very unattractive White females, holding other variables at theirmeans (p < 0.05). Similarly, very attractive Black male and Black female Twitter accountshad a higher predicted probability of being perceived as trustworthy than very unattractiveBlack accounts (p < 0.001). Very attractive Black female accounts had a higher probabilitycompared to their unattractive counterparts (p < 0.001).

Our results also suggest that workers were more likely to evaluate White Twitteraccounts as trustworthy. Black male and female Twitter accounts were viewed as less

Table 1. Summary of variables.Description Variable categories Notes

Trust How trustworthy is theman/woman?

0-untrustworthy and veryuntrustworthy, 1-above average,trustworthy, and very trustworthy

Desire toFollow

How likely are you to followthis account?

0- unsure, unlikely, and veryunlikely, 1- very likely and likely

Gender Workers estimate genderbased on the profileimage, description,username, and name.

0-Male, 1-Female, 2-Ambiguous If the combined characteristics had50% or more agreement then theaccount was classified into binarygender categories. Accounts thatdid not match this threshold wereclassified as ambiguous.

Race Please indicate whether theadult pictured in theprofile appears to be: (listof racial categories)

0-White, 1-Black, 2-Other racialcategory

Other racial category consists ofAmerican Indian, Alaska Native,Asian, or Native Hawaiian PacificIslander.

Attractiveness How physically attractive isthe man/woman?

1-Very Unattractive, 2-Unattractive,3-about average, 4-Attractive, 5-Very attractive

We created variables of female,male, and overall attractiveness.

WhiteHomophily

N/A 0-White worker evaluated Whiteaccount, 1- White workerevaluated Black accounts, 2-White worker evaluated accountclassified as American Indian,Alaska Native, Asian, or NativeHawaiian Pacific Islander

Racial homophily limited to theevaluations of White workersbecause our worker sample waspredominantly White.

Person inPublic

How easy or difficult wouldit be to use this profileimage to identify thisperson in public?

0-difficult or very difficult, 1- veryeasy, easy, or neither

Real Identity How easy or difficult wouldit be to use this account toestablish a person’s realidentity?

0-difficult or very difficult, 1- veryeasy, easy, or neither

INFORMATION, COMMUNICATION & SOCIETY 711

Trust and desire to follow twitter account

We found that while trustworthiness was positively associated with whether the workerindicated they would follow the account, this relationship was not statistically significant.

Trust

We find that attractiveness was positively associated with evaluations of trust (p < 0.001).But while a beauty premium appears to lead to positive evaluations of trust for all types ofTwitter accounts, the effect of attractiveness carries greater importance for some Twitterusers than others. Based on our estimates, the premium of good looks was considerablysmaller for White males than they are for White females or Black males.

Very attractive White females had a higher predicted probability of being viewed astrustworthy than very unattractive White females, holding other variables at theirmeans (p < 0.05). Similarly, very attractive Black male and Black female Twitter accountshad a higher predicted probability of being perceived as trustworthy than very unattractiveBlack accounts (p < 0.001). Very attractive Black female accounts had a higher probabilitycompared to their unattractive counterparts (p < 0.001).

Our results also suggest that workers were more likely to evaluate White Twitteraccounts as trustworthy. Black male and female Twitter accounts were viewed as less

Table 1. Summary of variables.Description Variable categories Notes

Trust How trustworthy is theman/woman?

0-untrustworthy and veryuntrustworthy, 1-above average,trustworthy, and very trustworthy

Desire toFollow

How likely are you to followthis account?

0- unsure, unlikely, and veryunlikely, 1- very likely and likely

Gender Workers estimate genderbased on the profileimage, description,username, and name.

0-Male, 1-Female, 2-Ambiguous If the combined characteristics had50% or more agreement then theaccount was classified into binarygender categories. Accounts thatdid not match this threshold wereclassified as ambiguous.

Race Please indicate whether theadult pictured in theprofile appears to be: (listof racial categories)

0-White, 1-Black, 2-Other racialcategory

Other racial category consists ofAmerican Indian, Alaska Native,Asian, or Native Hawaiian PacificIslander.

Attractiveness How physically attractive isthe man/woman?

1-Very Unattractive, 2-Unattractive,3-about average, 4-Attractive, 5-Very attractive

We created variables of female,male, and overall attractiveness.

WhiteHomophily

N/A 0-White worker evaluated Whiteaccount, 1- White workerevaluated Black accounts, 2-White worker evaluated accountclassified as American Indian,Alaska Native, Asian, or NativeHawaiian Pacific Islander

Racial homophily limited to theevaluations of White workersbecause our worker sample waspredominantly White.

Person inPublic

How easy or difficult wouldit be to use this profileimage to identify thisperson in public?

0-difficult or very difficult, 1- veryeasy, easy, or neither

Real Identity How easy or difficult wouldit be to use this account toestablish a person’s realidentity?

0-difficult or very difficult, 1- veryeasy, easy, or neither

INFORMATION, COMMUNICATION & SOCIETY 711

trustworthy compared to their White counterparts. White male accounts had a higherprobability of being viewed as trustworthy compared to accounts believed to belong toBlack male accounts (p < 0.01). Likewise, White female accounts had a higher probabilityof being evaluated as trustworthy compared to Black female accounts, holding other vari-ables at their means (p < 0.001).

Desire to follow

Overall, we find a beauty premium, with workers more likely to follow Twitter accountswith images of very attractive individuals compared to accounts evaluated as very

Table 2. Descriptive statistics.Mean Std. Dev. Minimum Maximum

Male 0.25 – 0 1Female 0.25 – 0 1Gender Ambiguous 0.50 – 0 1White 0.35 – 0 1Black 0.54 – 0 1Other Races 0.10 – 0 1Male Attractiveness 3.14 0.75 1 5Very Unattractive 0.02 – – –Unattractive 0.12 – – –About Average 0.58 – – –Attractive 0.25 – – –Very Attractive 0.03 – – –Female Attractiveness 3.66 0.85 1 5Very Unattractive 0.01 – – –Unattractive 0.06 – – –About Average 0.33 – – –Attractive 0.44 – – –Very Attractive 0.15 – – –Male Trustworthiness 3.09 0.73 1 5Very Untrustworthy 0.02 – – –Untrustworthy 0.13 – – –About Average 0.60 – – –Trustworthy 0.22 – – –Very Trustworthy 0.02 – – –Female Trustworthiness 3.24 0.69 1 5Very Untrustworthy 0.01 – – –Untrustworthy 0.07 – – –About Average 0.62 – – –Trustworthy 0.26 – – –Very Trustworthy 0.04 – – –Desire to Follow 1.80 0.98 1 5Very Unlikely 0.49 – – –Unlikely 0.30 – – –Unsure 0.13 – – –Likely 0.06 – – –Very Likely 0.01 – – –Friends 733.31 1648.11 1 24,868Account Age (years) 3.61 1.83 0 8Status Updates 9793.12 17,879.36 1 189,485Verified Account 0.01 – 0 1Has Description 0.96 – 0 1Has URL 0.29 – 0 1Has Location 0.91 – 0 1Has Image 0.99 – 0 1Person in Public 3.32 0.98 1 5Real Identity 3.49 0.81 1 5N = 2,448

712 A. GROGGEL ET AL.

unattractive. However, while attractive Twitter accounts fare better, the associationbetween physical attractiveness and whether the worker would follow the account isonly significant for White male Twitter accounts and Black female Twitter accounts.Very attractive White males have a higher predicted probability that the worker would fol-low the account compared to very unattractive White males (p < 0.05). Similarly, very

Table 3. Demographic of MTurk workers.Variables Mean

SexMale 0.54Female 0.45Prefer not to answer 0.01Relationship StatusMarried 0.28Divorced 0.07Single 0.48Living with partner 0.13Separated 0.03Widowed 0.01Age18–29 years old 0.3930–44 years old 0.4556–59 years old 0.12over 60 years old 0.04RaceHispanic 0.08Black 0.08White 0.85Asian 0.08American Indian or Alaska Native 0.03Hawaiian or Pacific Islander 0.01Level of EducationLess than High School 0.13Some College 0.38Bachelor’s degree 0.40Graduate School 0.08IncomeLess than 15,000 0.1215,000–39,000 0.2240,000–59,000 0.2260,000–79,000 0.1280,000 or more 0.15N = 956

Table 4. Summary of findings.Overall

Workers are more likely to follow trustworthy Twitter accounts. N/AEvaluations of TrustWorkers are more likely to evaluate attractive accounts as trustworthy. ✓Workers are more likely to evaluate White accounts as trustworthy compared to other racial categories ✓Potential VisibilityWorkers are more likely to follow attractive Twitter accounts ✓Workers are more likely to follow White Twitter accounts compared to Black Twitter accounts N/AWhite workers are more likely to follow White accounts compared to other racial categories ✓

Notes: Results based upon mixed-effect logistic regression analyses ✓ Significant Finding; N/A not a statistically significantrelationship.

Controls: Status Updates, Account age (years), Friends, Verified Account, Has description, Has a URL, 20 topical controls.

INFORMATION, COMMUNICATION & SOCIETY 713

attractive Black females have a higher probability of the worker following their accountthan very unattractive Black female accounts, holding other variables at the means (p <0.001). In terms of perceived race, while Black accounts were negatively associated withthe likelihood that workers would follow the account this relationship was not statisticallysignificant.

White homophily

We found evidence of a significant association between workers’ evaluations and racialhomophily. White workers were more likely to evaluate White accounts as trustworthy.In this case, racial homophily, defined as White workers evaluating profile images ofWhite individuals, takes on greater significance for unattractive accounts. Black maleand female Twitter accounts evaluated as unattractive were less likely to be viewed as trust-worthy than their White counterparts by White workers (p < 0.01).

Also, White workers were more likely to follow Twitter accounts perceived as Whitecompared to other racial categories. We find that workers’ race is significantly relatedto the hypothesized action of following both male and female Twitter accounts (p <0.05). While our analyses cannot fully test for in-group racial preference, these findingsreveal the importance of exploring the relationship between racial homophily and onlinevisibility.

Limitations

When exploring differences in trustworthiness and potential visibility across race, attrac-tiveness, and gender we lost statistical power because we were examining smaller sub-groups. Given this restriction we focused on racial homophily and include worker’sself-reported gender as a control variable. Future analysis will explore how social mediausers’ perceived social characteristics fully relates to the demographics of their online audi-ence. While further experimental work is needed to confirm a causal effect of Twitterusers’ race, gender, or attractiveness, our comparisons illustrate the importance of consid-ering the intersectionality of social media users’ self-presentation when studying onlineinteractions. Trust may be influenced by other variables such as Tweet content orsentiment.

Another limitation is that workers’ evaluations do not capture a live Twitter audi-ence. In our study, an active link was embedded that populated a new window forworkers to view Twitter profiles and they could read through previous tweets. Toaddress this shortcoming, we tracked the amount of time workers spent evaluatingthree profiles. Workers spent roughly three and a half minutes looking at each profileand answering all of the corresponding questions. However, MTurk workers are notthe same as social media users. In this study, we explore individual preferences ofworkers as a source of homophily, but cannot examine biases defined by the chancesthat people interact with individuals or test the effects of homogeneity of Twitter as awhole. Understanding how trustworthiness relates to social categorization carries impli-cations for interventions to reduce biases. For instance, light-skinned participantsassigned to enact a dark-skinned body in virtual reality had reduced implicit racialbias (Peck, Seinfeld, Aglioti, & Slater, 2013).

714 A. GROGGEL ET AL.

attractive Black females have a higher probability of the worker following their accountthan very unattractive Black female accounts, holding other variables at the means (p <0.001). In terms of perceived race, while Black accounts were negatively associated withthe likelihood that workers would follow the account this relationship was not statisticallysignificant.

White homophily

We found evidence of a significant association between workers’ evaluations and racialhomophily. White workers were more likely to evaluate White accounts as trustworthy.In this case, racial homophily, defined as White workers evaluating profile images ofWhite individuals, takes on greater significance for unattractive accounts. Black maleand female Twitter accounts evaluated as unattractive were less likely to be viewed as trust-worthy than their White counterparts by White workers (p < 0.01).

Also, White workers were more likely to follow Twitter accounts perceived as Whitecompared to other racial categories. We find that workers’ race is significantly relatedto the hypothesized action of following both male and female Twitter accounts (p <0.05). While our analyses cannot fully test for in-group racial preference, these findingsreveal the importance of exploring the relationship between racial homophily and onlinevisibility.

Limitations

When exploring differences in trustworthiness and potential visibility across race, attrac-tiveness, and gender we lost statistical power because we were examining smaller sub-groups. Given this restriction we focused on racial homophily and include worker’sself-reported gender as a control variable. Future analysis will explore how social mediausers’ perceived social characteristics fully relates to the demographics of their online audi-ence. While further experimental work is needed to confirm a causal effect of Twitterusers’ race, gender, or attractiveness, our comparisons illustrate the importance of consid-ering the intersectionality of social media users’ self-presentation when studying onlineinteractions. Trust may be influenced by other variables such as Tweet content orsentiment.

Another limitation is that workers’ evaluations do not capture a live Twitter audi-ence. In our study, an active link was embedded that populated a new window forworkers to view Twitter profiles and they could read through previous tweets. Toaddress this shortcoming, we tracked the amount of time workers spent evaluatingthree profiles. Workers spent roughly three and a half minutes looking at each profileand answering all of the corresponding questions. However, MTurk workers are notthe same as social media users. In this study, we explore individual preferences ofworkers as a source of homophily, but cannot examine biases defined by the chancesthat people interact with individuals or test the effects of homogeneity of Twitter as awhole. Understanding how trustworthiness relates to social categorization carries impli-cations for interventions to reduce biases. For instance, light-skinned participantsassigned to enact a dark-skinned body in virtual reality had reduced implicit racialbias (Peck, Seinfeld, Aglioti, & Slater, 2013).

714 A. GROGGEL ET AL.

unattractive. However, while attractive Twitter accounts fare better, the associationbetween physical attractiveness and whether the worker would follow the account isonly significant for White male Twitter accounts and Black female Twitter accounts.Very attractive White males have a higher predicted probability that the worker would fol-low the account compared to very unattractive White males (p < 0.05). Similarly, very

Table 3. Demographic of MTurk workers.Variables Mean

SexMale 0.54Female 0.45Prefer not to answer 0.01Relationship StatusMarried 0.28Divorced 0.07Single 0.48Living with partner 0.13Separated 0.03Widowed 0.01Age18–29 years old 0.3930–44 years old 0.4556–59 years old 0.12over 60 years old 0.04RaceHispanic 0.08Black 0.08White 0.85Asian 0.08American Indian or Alaska Native 0.03Hawaiian or Pacific Islander 0.01Level of EducationLess than High School 0.13Some College 0.38Bachelor’s degree 0.40Graduate School 0.08IncomeLess than 15,000 0.1215,000–39,000 0.2240,000–59,000 0.2260,000–79,000 0.1280,000 or more 0.15N = 956

Table 4. Summary of findings.Overall

Workers are more likely to follow trustworthy Twitter accounts. N/AEvaluations of TrustWorkers are more likely to evaluate attractive accounts as trustworthy. ✓Workers are more likely to evaluate White accounts as trustworthy compared to other racial categories ✓Potential VisibilityWorkers are more likely to follow attractive Twitter accounts ✓Workers are more likely to follow White Twitter accounts compared to Black Twitter accounts N/AWhite workers are more likely to follow White accounts compared to other racial categories ✓

Notes: Results based upon mixed-effect logistic regression analyses ✓ Significant Finding; N/A not a statistically significantrelationship.

Controls: Status Updates, Account age (years), Friends, Verified Account, Has description, Has a URL, 20 topical controls.

INFORMATION, COMMUNICATION & SOCIETY 713

What is beautiful is good - Dion, Berscheid, Walster, 1972 Journal of Personality and Social Psychology, vol. 24, no. 3, 285-290

https://faculty.uncfsu.edu/tvancantfort/Syllabi/Gresearch/Readings/17Dion.pdf

• “Several processes may operate to make the soothsayers' prophecies more logical and accurate than would appear at first glance.

• First, it is possible that a correlation between inward character and appearance exists because certain personality traits influence one's appearance. For example, a calm, relaxed person may develop fewer lines and wrinkles than a tense, irritable person.

• Second, cultural stereotypes about the kinds of personalities appropriate for beautiful or ugly people may mold the personalities of these individuals (…) Many have noted that one's self-concept develops from observing what others think about oneself. Thus, if the physically attractive person is consistently treated as a virtuous person, he may become one.”

Edward P. Lemay Jr., Margaret S. Clark, and Aaron Greenberg Personality and Social Psychology Bulletin, vol. 36 no. 3, 339–353, 2010

“See, it’s biology, we are programmed to be biased”

• We may be, but,

• Ethics, and civilisations, are human constructs: cannot make excuses based on biology alone.