The rise of online social platforms in the last two decades brought people from around the world closer together in an unprecedented manner. The forming of online communities created opportunities for spreading knowledge, initiating fruitful conversations, and openly expressing opinions among people from different geographical and social origins. With these online communities becoming larger, they face the inherent challenge of controlling undesirable social behavior.

Justin Cheng et al. address this challenge in their research: Antisocial Behavior in Online Discussion Communities by conducting a data-driven analysis of antisocial behavior on three different social platforms, then using insights gained from their analysis to develop a prediction model that can help the early detection of antisocial behavior. This work counts as a substantial contribution towards automating the process of identifying an undesirable online behavior.

Below are some facts that we could infer from this work:

1. A simple way of defining an undesirable behavior could be the actions that cause someone to be expelled from a community. Justin Cheng et al. consider users who are banned from posting or commenting on a social platform to be those who post undesirable or antisocial content. Of course, this could be subjective to a community, but it is still one of the most indicative acts of undesirability. 
2. Regarding selecting features for the prediction task, those features selected from the actions of community members and moderators have a more significant effect than those selected from the textual content. This could lead to two conclusions. It could strengthen the argument that undesirability is subjective to a community, but it could also be that selecting textual or content features to describe undesirable or antisocial behavior is a more challenging problem. But given that the prediction model performs relatively well across different platforms, we can conclude that antisocial acts could be reliably defined by the reaction of other community members.
3. The automation of detecting antisocial behavior in online communities could help moderators to better control the content, but it could not yet completely replace them. The involvement of a human to approve the prediction is necessary.

If I could have a chance to build over this work, I could focus on the following points:

1. Although the features selected from the reaction of the community are more descriptive of undesirable behavior, it has a drawback of being less effective for posts that are further in time than the time a user was banned, making it harder to make an earlier detection of an undesirable or antisocial behavior. Hence, improving the prediction through features selected from the content of the post could help to address this limitation.
2. As mentioned above, the automated detection of antisocial behavior could not yet completely replace the human decision. We can make use of this fact to even enhance the prediction by utilizing the decisions of human moderators to correct the prediction error and build a continuously improving model.

Finally, It is highly important and also challenging to control the spread of undesirable content in online social platforms. An undesirable content could range from simply being off-topic, to using swear words, to discrimination and bigotry, to spreading rumors and misinformation.

 

Read More

Cheng, J., Danescu-Niculescu-Mizil, C., & Leskovec, J. (2015). Antisocial Behavior in Online Discussion Communities. Proceedings of the Ninth International AAAI Conference on Web and Social Media Antisocial, 61–70. Retrieved from http://www.aaai.org/ocs/index.php/ICWSM/ICWSM15/paper/view/10469

The focus of the paper is categorizing anti-social behavior in online discussion communities. The inferential statistical approach taken by the paper by corroborating all claims with statistics is one that I appreciate. The approach in itself needs to be picked apart with a fine tooth comb to both understand the method followed and point a few discrepancies.

The paper claims to have adopted a “retrospective longitudinal analyses“. The long-term observational study in a subjects naturalistic environment is close to home as my current research hopes to study the “Evolution of trust”. A few key takeaways here are:

• The pool of study is limited to online discussion forums and not extended to general social media platforms. Since the author has not claimed the same or provided any evidence for the possibility of the same it is safe to say that this model is not completely generalizable. In platforms like Twitter where the site structure may be similar, the model adopted here might fail. A possible reason could be the option of retweeting on Twitter.
• The use of the propensity scores to determine causal effects by matching, according to my understanding, is a representational and reductional technique. It is representational because it considers a section of the data to represent all of it. Reductional because it discards a section of the data not used for the mapping. I wonder if this data loss has an impact on the outcome.
• The use of Mechanical Turk is always a good way to complete work that is not possible for Artificial Intelligence. In the above Human Intelligence Task the paper mentions the use of 131 workers with each post being averaged for three workers. The question that seemed important is whether this is required if a model is being built for another platform not covered by the one mentioned in the paper. As human hours can be expensive an alternative could be explored by compromising on the quality of the label classification and building a better model which will also make it more robust.
• The main question in the paper that I was hoping that was clearly answered but felt was not was “Can antisocial users be effectively identified early on?”. This can be a huge boon to have for any social media platform developer and/or designer. The promise of having very less or no trolls is like giving the customers a Charlies Chocolate Factory.

I wonder if this can be achieved by the introduction of an “Actor-Critic Reinforcement Learning algorithm“[1]. The use of a reinforcement learning algorithm lets the AI agent venture into the dark maze to find an exit. By rewarding the classification or flagging of a user in the right category we will be pushing it to train itself into becoming a good classifier of Anit-social behavior. The advantage of this model will be that the critic will ensure that the actor i.e. the agent performing the classification will not learn very quickly and will learn only the right things. It takes care of any anomalies that could occur. If the possibility exists then I feel this can be an area definitely worth pursuing through a course project.

REFERENCES:

[1] Konda, V. R., & Tsitsiklis, J. N. (2003). Actor-Critic Algorithms. Control Optim, 42(4), 1143–1166. https://doi.org/10.1137/S0363012901385691

Read More

1. Justin Cheng, Cristian Danescu-Niculescu-Mizil, Jure Leskovec. “Antisocial Behavior in Online Discussion Communities.”

In this day and age when “information overload” is widespread, the commodity everyone is eager to capture is attention. Users having the ability to do the same are sought after by companies trying to tout their next revolutionary product. But, there is one group of users which has particular ability to capture attention but the way they achieve it, make them, thankfully, undesirable to these establishments. These users through their vile and provocative mechanisms can send even the most civil of the netizens off their rails. But who and how these rogue actors function, can their behaviour be profiled at scale and then be used for nipping such bad actors from forums early from the bud? These are the questions [1] is trying to answer.

To start with, the paper distinguishes users from three websites namely CNN, IGN and Breitbart over a period of 18 months into two categories, Future Banned Users (FBUs) and Never Banned Users (NBUs). The FBUs are observed to have two subgroups, ones who concentrate their efforts to few threads or groups and ones who distribute their efforts to multiple forums. The author then measures the readability of the posts by these different categories of users to observe that the FBUs tend to have higher automated readability indices (ARIs) and displayed more negative emotions than NBUs. The author also measures the trend of user’s behaviour overtime to note any shift in their category label. The author later uses four different feature set namely post, activity, community and moderator to build a model to predict if a user will be banned or not.

To start with, the dataset is annotated by 131 workers in AMT. But due to the nature of selection of the workers nothing is known about the race, educational background or even political alignments which can definitely change the definition of “anti-social.” The diversity of opinion on what constitutes as “anti-social” is extremely important which the author hasn’t given much credence to.

Given the use of metric of using user deletions, effectiveness of such a model in forums where such user feedback mechanism is not present or in forums while such behaviour is norm and rampant, I believe, would be extremely low. What could be the metrics that could be adopted in forums like the ones mentioned? This could be an interesting avenue to explore.

Also, could these anti-social elements have a coordinated attack in-order take control over the platform? The individuals can bench members with more reports and and use its members who have less of these reports. The individuals can even create new accounts helping them steer a conversation to their cause. These are interesting strategies these individuals could adopt, the methods described in the paper would fail to detect. Can profiling these users’ content in order to ascertain their true identities create a slightly more robust model? This is something one can definitely try to work on in the future.

Another, interesting work that could be done is to identify the different methods through which these trolls try to elicit inflammatory behavior from their target. Also one could try to see how these mechanisms evolve, if they do, over time, as old ones tend to lose their ability to elicit such behaviour.

Can identifying users’ susceptibility in different forums or networks, be used to take preventive steps against anti-social behaviour? If one were to do that what are the different features that could be used to predict such susceptibility? Couple of features without much could be the number of replies the user gives to these trolls, the timespan the user has been active in the network and length of replies along with the sentiment. This if done could also be used to identifies trolls who have more sway over people.

Although, the intent and the motivation of the paper was excellent the content the paper left much to be desired.

Read More

Paper-

The paper questions, quite early on, about whether negative community interaction to antisocial behavior encourages or discourages continuation of that behavior, and finds that it actually exacerbates the problem. There are clear parallels between this finding, and certain studies on the effectiveness of the prison system, where “correctional” facilities do nothing to actually steer criminals away from their previous life of crime once they are released from prison.

The paper tries to compare the behavioral patterns of FBUs against NBUs, but through a process called “matching”, they only select NBUs who have the same posting frequency as FBUs. It is worth noting that this frequency is 10 times the posting frequency of regular users, so NBUs themselves may have anomalous usage patterns, or might be a special subset of users. Despite the paper’s claims that this selection choice gives better results, it might be useful to balance this out by collecting the same statistics about a third additional set of random users.

Moreover, the paper claims that FBUs, despite not contributing anything positive to the discussion, receive many more replies on their comments. The parallel to this is news shows with sensationalized, inflammatory news, or deliberately incendiary news panel guests, where the panel discussion does not enlighten the viewers to the issue, but the ensuing argument attracts a large viewership.

The predictive model that the authors create, could be augmented with other features, like post time data, login/logout times, and data about frequency and duration of personal messages between antisocial users and other community users. I suspect that anti-social users would have a number of short, high volume personal message exchanges with other users (maybe an argument with users who were angry enough to personally message the antisocial individual), but not many sustained long-term exchanges with other users. The predictive model, as the paper mentions, could be something more powerful/expressive than a simple piecewise linear model, like a neural network or an SVM.

Lastly, the predictive model, if implemented in real world systems to ban potentially antisocial users, has some problems. Firstly, as the paper briefly mentions, it raises an interesting question about whether we should give these “antisocial” users the benefit of the doubt, and whether it is okay for an algorithm to pre-emptively ban someone, before society (in this case the moderators or the community) has decided that time has come for them to be banned (as is the case in today’s online and real world systems).

 

Read More

The paper for today’s discussion:

Cheng, Justin et al. (2015) – “Antisocial Behavior in Online Discussion Communities”- Proceedings of the Ninth International AAAI Conference on Web and Social Media (61-70).

Summary

The paper was mainly focused on analyzing the antisocial behaviors in large online community namely, CNN.com, Breitbart.com and IGN.com. The authors describe undesirable behavior such as trolling, flaming, bullying, harassment and other unwanted online interactions as anti-social behaviors. They have categorized users who display such unwanted attitudes into two broad groups, namely, the Future-Banned Users (FBUs) and the Never-Banned User (NBUs). The authors conducted statistical modelling to predict individual users who will eventually be banned from the community. They collected data from the above-mentioned sites via Disqus for a period of about 13 months. They based their measure of undesirable behaviors on the posts that were deleted by the moderators. The main characteristic of an FBU post are

• They post more than an average user would and contribute more towards posts per thread.
• They generally post off topic conversation and which generally are negative emotions.
• Posting quality decreases over time and this may be as a result of censorship.

It was found that at times the community tolerance changes as well and become less tolerant of an users’ post over time.

The authors further classified the FBU’s into Hi-FBU and Lo-FBU with the name signifying the amount of post deletion that occurs. It was found that Hi-FBUs exhibited strong anti-social characteristics and their post deletion rate were always high. Whereas, for the Lo-FBUs the post deletion rates were low until the second half of their lives where it rose. Lo-FBU start to attract attention (the negative kind) in their later life.  Few features were established in the paper for the identification of the antisocial users namely the post features, activity features, community features and the moderator features. Thus, the authors were able to establish a system that would identify undesirable users early on.

Reflection.

This paper was an interesting read on how the authors conducted the data-driven study of anti-social behavior in online communities. The paper on Identity and Deception by Judith had introduced us to online “trolls” and how their posts are not welcomed by the community and might even lead to the system administrators banning them from such posts. This paper delved further into the topic with analyzing the types of anti-social users.

One issue which comes to my mind is how are the moderators going to block users when the platform is anonymous? The paper on 4chen’s popular board /b/, which was also assigned as a reading, focused on the anonymity of users posting on threads and that the majority of the site attracted anti-social behavior. Is it possible to segregate users and ultimately block them from creating profanity in anonymous platforms?

One platform where I have witnessed such online unwanted comments is YouTube. The famous platform by google has a comment section where anyone having a google account can post their views. I recently read an article “Text Analysis of YouTube Comments” [1]. The article focused on videos from few categories like comedy, science, TV and news & politics. It was observed that news and political related channels attracted the majority of the negative comments whereas, the TV category are mostly positive. This leads me to think that the subject of discussion is sometimes important as well. What kind of topics do generate the most amount of anti-social characteristic in the discussion communities?

The social media in general has now become a platform for cyberbullying and unwanted comments. If these users and their patterns are detected and if such comments are automatically filtered out as “anti-social”, it would be a huge step in the right direction.

[1] https://www.curiousgnu.com/youtube-comments-text-analysis

 

Read More

Cheng, Justin, Cristian Danescu-Niculescu-Mizil, and Jure Leskovec. “Antisocial Behavior in Online Discussion Communities.”

Summary
In this paper, Cheng et al. exhaustively study antisocial behaviour in online communities. They classify their dataset into Future Banned Users (FBU) and Never Banned Users (NBU) for the purpose of comparing the difference in their activities in the following factors – post content, user activity, community response and actions of the community moderators. The paper suggest that the content of the posts by FBUs tend to be difficult to understand and full of profanity, they tend to attract more attention to themselves and engage/instigate pointless arguments. With such users even tolerant communities over time begin to penalise FBUs more harshly than they did in the beginning. This maybe because the quality of the FBUs posts have degraded or simply because the community no longer wanted to put up with the user. The paper points out, after extensive quantitative analyses, it is possible for FBU users to be identified as early as 10 posts into their contribution to discussion forums.

Reflection
As I read this paper, there are a few questions that I wondered about:
1. What was the basis of the selection of their dataset? While trolling is prevalent in many communities, I wonder if Facebook or Instagram may have been a better place because trolling is at its most vitriolic when the perpetrator has some access to your personal data.
2. One of the bases for the classification was the quality of the text. There are several groups of people who have reasons other than trolling for the quality of text viz. non-native speakers of English, teens who have taken to unsavoury variations of words like lyk(like), wid (with), etc.
3. Another characteristic of anti-social users online was people who led other users of the community into pointless and meaningless discussions. I have been part of a group that was frequently led into pointless discussions by legitimate well-meaning members of the community. In this community ‘Adopt a Pet’, users are frequently outraged by the enthusiasm that people show in adopting pedigrees versus local mutts. Every time there is a post about pedigree adoptions, there are always a number of users who will be outraged. Are these users considered anti-social?
4. The paper mentions that some NBUs have started out being deviant but had improved over time. If as this paper proposes, platforms begin banning members based on a couple of posts soon after they join, wouldn’t we be losing on these users? And as suggested by the paper, users that believe they have been wrongly singled out (in deleted posts whereas other posts with similar content were not deleted) tend to become more deviant. When people feel like they’ve been wrongly characterised, based on a few posts, wouldn’t they come back with a vengeance to create more trouble on the site?
5. Looking back at the discussion in our previous class, how would this anti-social behaviour be managed in largely anonymous websites like 4chan? It isn’t really possible to ‘ban’ any member of that community. However, maybe because of the ephemerality of the website, if the community ignores trolls, the post may disappear on its own.
6. If we look at communities where deviant behaviour is welcome. If visitors who visit say r/watchpeopledie reports a post to the mod as would the moderator have to delete the post given that it is the norm on that discussion board?

Read More

[1] Cheng, Justin, Cristian Danescu-Niculescu-Mizil, and Jure Leskovec. “Antisocial Behavior in Online Discussion Communities.” Icwsm. 2015.

Summary

In this paper, Cheng et al. study three online discussion driven communities (CNN, IGN, and Breitbart) to understand the dynamics behind antisocial behavior in Online Communities. Their results are greatly derived from a quantitative study, where they inspect data to find trends in antisocial behavior and use the same to support their conclusions. This study also tracks how the activities of the users transform over time. The main deciding factor for an individual to be classified as an antisocial entity is the fact that they have been banned by moderators. The paper then goes on to explain a model to predict antisocial behavior using various features that they had discussed earlier.

 

Reflection

The paper answers many questions while leaving many unanswered. Although many conclusions seem intuitive to understand, it is amazing how simply going through the data answers the same questions. Like the one where the authors discuss “if excessive censorship causes users to write worse”. Intuitively, if one is punished for doing the right thing, the chances of repeating the same nice thing again reduces considerably.

What exactly is antisocial behavior? From one online community to another, this definition will change. There can not be a single defining line. For instance, 4chan users will tolerate more antisocial elements than users on Quora. Also, as we move from one geographical area to other, speaking habits will change. What is offensive and inappropriate in some culture might not be inappropriate in some other culture. So, what content is acceptable and to what extent?

Antisocial posts in this paper are labeled by the moderators. These are human moderators and their views are subjective. How can we validate that these posts are actually antisocial and not a positive criticism or some form of sarcasm? Secondly, on huge social networking websites which produce millions of posts every day how can moderation be translated at such a large scale? The paper provides four features and amongst them, the “moderator” feature has more weight in the classifier than the others. But with such large-scale networks, how can one rely on community and moderator features? The model also has a decent accuracy but when extrapolated to a large user base, it could result in banning of millions of innocent user accounts.

Coming to the technical side, the model shows relatively high accuracy during cross-platform testing using simple random forest classifiers and basic NLP techniques. While “Bag of words” model with random forest classifiers is a strong combination, they are insufficient to build the “post features”, in this case. Users have many different writing styles and much depends on the context in which words appear, so something more advanced than “bag of words” is needed. Word vectors would be a very good choice as they help capture context using the relative distance between two words. They can be easily tailored to the common writing style of the platform.

By taking, posts from the same user, we can build a sentiment index for each user. Sentiment index will help predict what the user, in general, feels about a particular topic and prevent incorrect banning. It is comparable to a browser keeping your search history to understand your usage patterns. One can also look at all posts from a general perspective and create an “antisocial index” for each post and only if the index is above a certain threshold, should the user be banned or be penalized. Penalties could include disabling users posting privileges for certain hours, so as to ensure that even if there is a false positive, an NBU is not banned.

In conclusion, the paper provides an informative and intriguing baseline to track antisocial behavior. Many techniques can be used to enhance the proposed model and create an autonomous content filtering mechanism.

Read More

Antisocial Behavior in Online Communities

Summary:

In this paper authors perform a quantitative, large-scale longitudinal study of antisocial behavior on three online discussion communities namely CNN, IGN and Breitbart by analyzing users who were banned from these platforms. They find that such users use obscene language that contains less positive words and is harder to understand. Their posts are limited to a few threads and are likely to amass more responses from other users. The longitudinal analysis reveals that the quality of their posts degrade with passage of time and community becomes less and less tolerant to their posts. The authors also discover that excessive censorship in the initial stages might aggravate the antisocial behavior in the future. They identify features that can be used to predict whether a user is likely to be banned or not namely content of the post, community’s response & reaction to the post, user activities ranging from posts per day to votes given to other users and actions of moderators. Finally, they build a classifier that can make the aforementioned prediction after observing just 5-10 posts with 80% AOC.

Reflections:

Antisocial behavior can manifest in several forms like spamming, bullying, trolling, flaming and harassment. Leslie Jones, actress starring in movie “Ghostbusters” became a target of online abuse. She started receiving misogynistic and racist comments on her twitter feed from several people including a polemicist Milo Yiannoloulos and his supporters. He was then permanently banned from twitter. Sarahah was removed from app stores after Katrina Collin’s started an online petition accusing the app of breeding haters after her 13 year old daughter received hateful messages, one even saying “i hope your daughter kills herself”. According to an article, 1.4 million people interacted with Russian spam accounts on twitter during the 2016 US elections. Detecting such content has become increasingly important. 

Authors say several techniques exist in the online social communities to discourage antisocial behavior ranging from down voting, reporting posts, mute feature, blocking a user, comments to manual human moderation. It would be interesting to find how these features fit in the design of the community. How are these features being used? Are all these “signals” true indicators of anti-social behavior? e.g. the authors suggest in the paper that downvoting is sometimes used to express disagreement rather than antisocial behavior which is quite true in case of quora and youtube. Both these websites have an option to downvote as well as to report the post. Will undesirable content always have larger number of downvotes? Do posts of users exhibiting antisocial behavior receive more downvotes, do their posts get muted by most of their online friends?

All of the author’s inferences make sense. The FBUs use more profanity and less positive words and get more replies which is expected since they use provocative arguments and attempt to bait users. We saw examples of the similar behavior in the last paper we read ”Identity and deception in the virtual community”. I also decided to visit the 4chan website to see if concept of moderators exist there. Surprisingly it does. But as answered in one of the FAQs one hardly gets to see moderated posts since there are no public records of deletion and since the content is deleted it gets removed from the page. I wonder if it’s possible to study the moderated content using the archives and if the archives keep temporal snapshots of the website’s content. Secondly, the website is famous for it’s hateful and pornographic content. How do you pick less hateful stuff from the hateful. I wondered if hate and sexual content are even considered criteria there. On checking their wiki I found the answer to “how to get banned in 4chan” {https://encyclopediadramatica.rs/4chan_bans=>quite an interesting read}. This makes one thing clear, criteria to moderate content is not universal. It depends a lot on the values and culture of the online community.

Having an automated approach to detect users will definitely lessen the burden from the shoulders of human moderators. But I wonder about the false positive cases. How will it affect the community if a series of harmless posts gets some users banned? Also, some users might redeem themselves later. In Fig 6 c) and corresponding explanation in “Characterizing users who were not banned”, we find that even though some FBUs were improving they still get banned. Punishing someone for improving will make sure that person will never improve in life. And the community might loose faith in the moderators. Considering these factors, is it wise to ban a user after observing initial few posts? Even exposing such users to moderators will make the later biased against the former. How long one should wait to form a judgment?

Overall I think it was a good paper, thorough in every aspect: from data collection, annotation, analysis to specifying the future work. I’ll end by mentioning a special app “ReThink[1]” that I saw in an episode of Sharktank (a show where millionaires invest their money in unique ideas). This app detects when a user writes an offensive message and gives him a chance to reconsider sending that message by showing an alert. Aimed for adolescents, the app’s page mentions that 93% of people do change their mind when alerted. Use of such apps by young people might make them responsible adults and might help in reducing the anti social behavior that we see online.

[1] http://www.rethinkwords.com/whatisrethink

Read More