Political Fact Checking on Twitter: When Do Corrections

Have an Effect?Aniko Hannak, Drew Margolin, Ingmar Weber

6/12/2015 IC2S2

Proposition

• People are more likely to pay attention to facts when group members tell them to than when non-group members tell them to

• Factual challenges on Twitter:• One Tweeter “snopes” another

• Do they follow each other?

Operationalization

6/12/2015 IC2S2

Fact-checking websites

• Snopes.com• Politifact.com• Factcheck.org

6/12/2015 IC2S2

Fact-checking websites

Advantages• Clear true /

false judgment

• Same ultimate sources for all subjects

6/12/2015 IC2S2

Method

• Look for “snoping” tweets

1. Begin with @mention (reply)

2. Contain Fact-check url

Presumption: “Snoper” is replying to “Snopee” to trying to correct them

6/12/2015 IC2S2

The Real Snopes of Social Media

😄

👮

6/12/2015 IC2S2

@Snopee

@Snopee

@Snoper

Method – Dataset #1

• 1,369 snoping events • from “garden hose” sample of Twitter (April 2012-March 2013)1

• where both snoper and snopee history available1

• 303 where snope url is explicitly political (mentions Romney or Obama or is categorized as such)

1See further details in ICWSM 2014 paper:

http://www.brianckeegan.com/papers/ICWSM_2014.pdf

6/12/2015 IC2S2

What are Snopers doing?

• Crowdflower coders code as “correction”

6/12/2015 IC2S2

Corrections by Snope Topic

6/12/2015 IC2S2

Not-Political(sample)

Political(all)

Non-Correction 83 (31%) 64 (21%)

Correction 182 (69%) 236 (78%)

Not codable 0 3 (1%)

Total 265 303

Chi-Sq. = 7.29, p < .01

Political snopes are more likely to be “corrections”

Independent Variables

6/12/2015 IC2S2

Incidence – Who Snopes Whom on Politics?

Relationship PoliticalCases

% of Cases Correcting % Correcting

Stranger 152 50% 117 78%

Followee 14 5% 11 85%

Follower 49 16% 41 84%

Friend (mutual following) 88 29% 67 76%

Total 303 100% 236

Chi-Sq. = 1.38, p = .71

6/12/2015 IC2S2

Friends/strangers are equivalent in propensity to use snopes to correct

Are Political Corrections Acknowledged?

“Snopee” Ignores

“Snopee” Responds

Snoped by Stranger

Snoped by Friend

N = 97

N = 57

95% 5%

86% 14%

p =.05

When snope is a correction about politics, people are 2.8 times more likely to respond when it comes from a friend

6/12/2015 IC2S2

Replies to Political vs. Non-Political Corrections

• When snope is a correction not about politics• People are 3-5 times more likely to respond

• But difference between stranger vs. friend is less• only 1.5 times more likely to respond to snope from a friend vs. a stranger

• No difference in response rates for non-correcting snopes between friends and strangers (political or non-political)

6/12/2015 IC2S2

Method – Dataset #2

• 4,733 snoping events • Explicitly “political” snopes according to the categorization of the rumor

• from “fire hose”

• 2,037 where both snoper and snopee history and friend / follower information available

6/12/2015 IC2S2

Consistent w/prior results

“Snopee” Ignores

“Snopee” Responds

Snoped by Stranger

Snoped by Friend

N = 1532

N = 505

Tweeters are significantly more likely to reply (at any time) to snoping tweet when it comes from a friend (someone they follow)

78% 22%

55% 45%

p < .001 6/12/2015IC2S2

What Type of Replies?Author (Margolin) and research assistant hand coded 445 snopes where the snopeereplied• 351 (79%) were “correcting snopes”, of

these…• 173 (49%) the snopee “accepted

the snope fact as true”

6/12/2015IC2S2

👮

😄

😄

@Snopee

@Snopee

@Snopee

@Snoper

@Snoper

Among the replies?

“Snopee” Does Not Accept

“Snopee” Accepts

Snoped by Stranger

Snoped by Friend

N = 219

N = 132

Friends more likely to “accept” snope fact than strangers(remains significant when controlling for friends/followers in common)

58% 42%

39% 61%

p < .001 6/12/2015IC2S2

Other kinds of reactions?Coded for “defends original position”• 94 (27%) of “correcting snopes”

Early results suggest snopees are more likely to defend in this way when:

1. Snoper is stranger (p < .05)2. Snoper is unpopular (has fewer

followers) (p < .05)[“double-down against weaker opponents”?]

Number of snopee followers did not matter

6/12/2015IC2S2

👮

😄

😄

@Snopee

@Snopee

@Snoper@Snopee

@Snoper

Other kinds of reactions?Coded for “defends original position”• 94 (27%) of “correcting snopes”

Coded for “defends original position with new basis”• 67 (19%) of “correcting snopes”

6/12/2015IC2S2

👮

😄

😄

Acknowledgement of snope fact truth

Asserts new basis

@Snopee

@Snopee

@Snoper@Snopee

@Snoper

Other kinds of reactions?Coded for “defends original position”• 94 (27%) of “correcting snopes”

Coded for “defends original position with new basis”• 67 (19%) of “correcting snopes”

Early results suggest snopees are more likely to defend in this way when:

1. Snoper is popular (more followers) (p < .01) [“shift against authority”?]

Friend/following between snoper-snopeedid not matter

6/12/2015IC2S2

👮

😄

😄

@Snopee

@Snopee

@Snoper@Snopee

@Snoper

Ongoing …

• Political leaning

• What happens before

• Change of relationship due to snoping

• Closeness of relationship

6/12/2015 IC2S2