Why Robots CommunicateEran Banin
1
Background
User-generated content is critical to the success of any online platforms (CNN, Facebook, StackOverflow).
These sites engage their users by allowing them to contribute and discuss content, strengthening their sense of ownership and loyalty.
While most users tend to be civil, others may engage in antisocial behavior, negatively affecting other users and harming the community.
2
Examples-
3
Mechanisms for preventing
Moderator - person who is responsible for a newsgroup or mailing list on the Internet and for checking the messages before sending them to the group.
Up & Down voting.
Blocking users.
Yet, antisocial behavior is a significant problem that can result in offline harassment and threats of violence.
4
Past research
Despite its severity and prevalence, surprisingly little is known about online antisocial behavior.
Most research reports qualitative studies that focus on characterizing antisocial studying the behavior of a small number users in specific communities.
In order to create new methods for identifying undesirable users, large-scale and longitudinal analysis (of the phenomenon) is required.
5
Present:
Past:
Rely on a community and its moderators to decide who they consider to be disruptive and harmful
Antisocial behavior has been widely discussed in past literature in order to formally define the behavior.
6
Present:
Past:
Large-scale data-driven analysis with the goal of obtaining insights and developing tools for the early detection of trolls. Moreover, the research provide prior study of the effects of community feedback on user behavior.
Research has tended to be largely qualitative, generally involving deep case study analyses of a small number of manually-identified trolls.
7
Present:
Past:
Predict whether individual users will be subsequently banned from a community based on their overall activity. Also show how our models generalize across multiple communities.
Focus on detecting vandalism with respect for their language and reputation. Other works tried to identified undesirable comments based on their relevance to the discussed article and the presence of insults.
8
Questions
1. Are there users that only become antisocial later in their community life, or is deviant behavior innate?
2. Does a community’s reaction to users’ antisocial behavior help them improve, or does it instead cause them to become more antisocial?
3. Can antisocial users be effectively identified early on?
9
Contents
Source research
18 months, 1.7 million users contributed nearly 40 million posts and more than 100 million votes.
Members that repeatedly violate community norms are eventually banned permanently. Such individuals are clear instances of antisocial users, and constitute “ground truth” in our analyses.
11
List of posts in the same article
In addition, complete time-stamped trace of user activity from March 2012 to August 2013, as well as a list of users that were banned from posting in these communities.
12
Measuring undesired behavior
On a discussion forum, undesirable behavior may be signaled in several ways:
down-vote
comment
ban a user from ever posting again in the forum.
However, down-voting may signal disagreement rather than undesirability. Further, one would need to define arbitrary thresholds needed to label a user as antisocial.
In contrast, we find that post deletions are a highly precise indicator of undesirable behavior, as only community moderators can delete posts.
13
Measuring undesired behavior
Bans are similarly strong indicators of antisocial behavior, as only community moderators can ban users.
Thus, we focus on users who moderators have subsequently banned from a community .
While such an approach does not necessarily identify every antisocial user, this results in a more precise set of users who were explicitly labeled as undesirable.
Filter by-
Apart from insults and profanity,
these include repeated attempts to bait users (“Ouch, ya
got me. What’s Google?”), provoke arguments (“Liberalism
truly breeds violence...this is evidence of that FACT”), or derail
discussions (“All I want to know is...was there a broom
involved in any shape or form?”).
14
Matching FBUs and NBUs
We note that FBUs tend to post more frequently than average users.
For example, on CNN, a typical FBU makes 264 posts, but an average user makes only 22 posts.
To control for this large disparity, we use matching - statistical technique used to support causality claims in observational studies, to control for the number of posts a user made and the number of posts made per day.
15
Unbiased measure of the quality or appropriateness of a post
Dictionary-based approaches may miss non-dictionary words
classifier trained on the text of deleted posts may confound post content and community bias
Thus, we instead obtained human judgments of the appropriateness of a specific post.
Collect post of random FBU & NBU and asked workers for evaluate how appropriate is a post (on a scale of 1 to 5). Each post was labeled by three independent workers, and their ratings averaged.
131 workers completed these tasks, and they rated deleted posts significantly lower than non-deleted posts (2.4 vs. 3.0)
16
Measuring text quality
Using these labeled posts, we then trained a logistic regression model on text bigrams to predict the text quality of a post. Posts with a rating higher than 3 were labeled as appropriate, and the other labeled as inappropriate.
Under ten-fold cross validation, the AUC attained by this classifier was 0.70.
This suggests that while the classifier is able to partially capture this human decision making process and allow us to observe overall trends in the data, other factors may play a significant role in determining whether a post is appropriate.
17
Understanding Antisocial Behavior
To understand antisocial behavior in the context of a discussion community, we first characterize how users who are banned differ from those who are not in the terms of how they write and how they act in a community.
Then, we analyze changes in behavior over the lifetimes of these users to understand the effects of post quality, community bias, and excessive censorship.
18
Differences in how FBUs & NBUs write
The similarity of a post to previous posts in a same thread may reveal how users are contributing to a community.
Here we compare the average text similarity of a user’s post with the previous three posts in the same thread
FBUs make less of an effort to integrate or stay on-topic
Post deletion is weakly negatively correlated with text similarity, suggests that off-topic posts are more likely to be deleted
Differences in how FBUs & NBUs write
Next, we measure each post with respect to several readability tests, including the Automated Readability Index (ARI), which are designed to gauge how understandable a piece of text is.
FBUs appear to be less readable than those written by NBUs
20
Differences in how FBUs & NBUs write
Prior research also suggests that trolls tend to make inflammatory posts.
FBUs are less likely to use positive words
Use less conciliatory language
More likely to swear
How do FBUs generate activity around themselves
Do FBUs purposefully try to create discussions, or opportunistically respond to an on-going discussion?
While FBUs may either create or contribute to (already existing) discussions depending on the community, they generally get more replies from other users, and concentrate on fewer threads.
22
Evolution over time
how does FBU’s behavior and the community’s perception of them change over time?
The rate of post deletion increases over time for FBUs, but is effectively constant for NBUs.
23
Evolution over time
The increase in the post deletion rate could have two causes-
1) A decrease in posting quality
2) An increase in community bias
Text quality is decreasing over time, suggesting that 1) may be supported
24
Evolution over time
Random Select of 400 FBUs & NBUs and from each user sampled a random post from the first 10% or last 10% of their entire posting history.
The posts presented to examiners who rated the appropriateness off each post
FBUs start out writing worse than NBUs and worsen more than NBUs over the course of their life.
25
Community tolerance over time
For testing it, random posts with similar predicted text quality were matched to pairs. Each pair contain one post from the first 10% of a user’s life, and one from the final 10%.
Wilcoxon Signed-rank performed for check which post is more likely to be deleted.
Results - among FBUs, found a significant effect of post time in all communities, mean that a post was made in the last 10% of an FBU’s life is more likely to be deleted in contrast to NBUs.
26
Does draconian post deletion policy could exacerbate undesirable behavior?
For testing it, users with at least 10 posts were taken and divided into 2 groups : 4/5 post s deleted among their 5 first posts and the others.
Pairs were matched based on mean text quality of their 5 first posts and the compared by the last 5.
Here, a Wilcoxon Signed-rank test shows a significant effect of the deletion rate – mean that “unfairly” deletion cause worsen writing.
27
Types of Antisocial Users
The following figure suggest that there are 2 kinds of FBUs-
Hi-FBU - proportion of deleted posts above 0.5
Lo-FBU - proportion of deleted posts below 0.5
Across all communities, the number of users in each population is split fairly equally between the two groups.
28
use language that is less accommodating
receive more replies
Their post deletion rate starts high and remains high
In contrast, Lo-FBUs tend to have a constant low rate, until the second half of their life.
29
Increasing reasons
The text similarity of these users’ posts with other posts in the same thread is significantly lower across their last five posts.
Additionally, they start to post more frequently, and in fewer threads later in their life.
Thus, a combination of a large number of less relevant posts in a short period of time potentially makes them more visible to other members of the community.
30
Antisocial Behavior in Two Phases
Attempt to characterize this change over a user’s lifetime by splitting it into two halves, with the goal of understanding how users may change across them.
Fit two linear regression lines to a user’s post deletion rate over time, one for each half of the user’s life obtained by bucketing posts into tenths .
Computing (m1,m2) – the slope in first and second half respectively.
31
Antisocial Behavior in Two Phases
By plotting the points we can identify quadrants corresponding to how the deletion rate of a user’s posts changes over time:
fraction of users who are getting worse is higher for FBUs
fraction of users who are improving is higher for NBUs
32
When looking only on users with high initial deletion rates:
high proportion of NBUs -
many users who should be banned are in fact not
fraction of users who are improving is higher for NBUs
33
Main goal
In the communities we studied, FBUs tend to live for a long time before actually getting banned . communities response slowly to toxic users.
For example , on CNN, FBU write an average of 264 posts (over 42 days), with 124 posts deleted before they are finally banned.
Therefore, our currently main goal, is to build tools for automatic early identification of potently FBUs.
34
Factors for identify FBUs
Motivated by our observations and insights, we could begin by designing features for identification –
Post: Number of words, Readability metrics (ARI), LIWC - content analysis and text-mining software.
Activity: number of posts per day/thread ,fraction of replied posts, votes up & down.
Community: votes received per post, fraction of up-votes received, fraction of posts reported, number of replies per post.
Moderator: fraction of posts deleted, slope and intercept of linear regression lines (i.e. m1,m2).
35
36
Rationale – the combination of learning models increase the classification accuracy.
Main idea – To average noisy and unbiased models to create a model with low variance .
Random forest tree works as a large collection of decorrelated decision trees.
40
Random forest classifier
Algorithm- assume we have set of features (F1,…,Fn) and dataset for training (X1,…,Xm)-
Create the following matrix :
Chose subset of rows from the matrix and create decision tree.
For testing X’ , train on all the trees and average the results.
F1(x1) F2(x1) … Fn(x1) X1
K-fold cross validation
The original sample is randomly partitioned into k equal sized subsamples.
One of the k subsamples is retained as the validation data for testing the model, and the remaining k − 1 subsamples are used as training data.
 The k results from the folds can then be averaged.
42
Area under the ROC curve
In order to measure the classifier performance AUC technique been used:
43
Performance seem to peak near to 10 post.
In case user make to mach posts, it’s difficult to predict.
45
Prediction performance change over “distance” from getting banned
It becomes increasingly difficult to predict whether a user will get banned the further in time the examined posts are from when the user gets banned
46
47
Conclusions
The paper presents a data-driven study of antisocial behavior in online discussion communities by analyzing users that are eventually banned from a community.
Leads to characterization of antisocial users and to an investigation of the evolution of their behavior and of community response
Also, the paper proposes a typology of antisocial users based on post deletion rates.
Finally, introduces a system for identifying undesired users early on in their community life.
48
Potentially improvements
A more fine-grained labeling of users may reveal a greater range of behavior and lead to better performance.
A better analysis of the content of posts, and of the relation between the posts in a thread.
Expend the analysis to temporary banned users and different communities.
Research for understanding how antisocial users may steer individual discussions
49
While we present effective mechanisms for identifying and weeding antisocial users, taking extreme action against small infractions can exacerbate antisocial behavior.
Though average classifier precision is relatively high (0.80), one in five users identified as antisocial are nonetheless misclassified.
Possible solutions -
Trading off overall performance for lower rate of such mistake.
Employ a human moderator for approve any bans.
Giving antisocial users a chance to redeem themselves.
50