Automated Hate Speech Detection and the Problem of Offensive Language

Quick Summary

Why is this study important? Classifying hate speech vs. offensive language. Hate speech targets and potentially harms disadvantaged social groups (promotes violence or social disorder). Without differentiating the two, we erroneously consider many to be hate speakers.

How did they do it? The research team got a lexicon of “hate words” from hatebase.org to find over 30k twitter users who used those hate words. They extracted each user’s timeline and took a random sample of 25k tweets containing terms from the lexicon. They used CrowdFlower crowdsourcing to manually label each tweet as hate speech, offensive, or neither. They then considered many features from these tweets and used them to train a classifier. They used a logistic regression model with L2 regularization, making a separate classifier for each class using scikit-learn. Each tweet was classified by the most confident classifier.

What were the results? They found that racist and homophobic tweets are more likely to be classified as hate speech but that sexist tweets are generally classified as offensive. Human coders appear to consider sexist/derogatory words towards women are only offensive. While the classifier was great at predicting non-offensive or merely offensive tweets, its struggled to distinguish true hate speech from offensive language.

Reflection

Ways to Improve the Study:

• Only a small percentage of the tweets flagged by the hatebase lexicon were considered hate speech by human coders. This means that the dictionary used in identifying “hateful” words was extremely broad. Using a more precise lexicon would increase the accuracy of the classifier.
• I think studying the syntax of hate speech could be particularly interesting. It would be interesting to try to train a classifier without using particular keywords.

What I liked About the Study:

• The use of CrowdFlower was a very interesting approach to labeling the tweets. Although there were clearly some user-errors in the classifications, the idea of crowd-sourcing to get a human perspective it intriguing, and I plan on looking into this for my future project.
• They used a TON of features for classifications and tested many models. I think a big reason why the classifiers were so accurate was because of the detail the team took in creating their classifier.

Early Public Responses to the Zika-Virus on YouTube: Prevalence of and Differences Between Conspiracy Theory and Informational Videos

Quick Summary

Why is this Study Important? When alarming news breaks, many internet users consider it a chance to spread conspiracy theories to garner attention. It is important that we learn to distinguish between the truth and these fake news/conspiracy theories.

How did they do it? The team collected the user reactions (comments, shares, likes, dislikes, and the content/sentiment of user responses) to the 35 most popular videos posted on YouTube when the Zika virus began its outbreak in 2016,

What were the Results? The results were not surprising. 12 of the 35 videos in the data set focus on conspiracy theories, but there were no statistical differences between the two types of videos. Both true/informational videos and conspiracy theory videos shared similar numbers of responses, unique users per view, and additional responding per unique user. Both types of videos has similarly negative responses, but informational videos’ comments are concerned with the outcome of the virus, while conspiracy theory videos’ comments were concerned with where the virus came from.

Reflection:

What I would do to Improve the Study:

• Study user interactions.responses more closely. User demographics might tell a much bigger story about the reactions to these types of videos in comparison to each other. For example, older people might be less susceptible to conspiracy theories and respond less than younger people.
• Study different aspects of the videos all together. Clearly, user responses/interaction with informational videos and conspiracy theory videos are similar. However, looking at differences in content, titles, and publisher credibility of the video would make a lot more sense in distinguishing the two.

What I liked About the Study:

• The semantic map of user comments was highly interesting and I wish I had seen more studies using a similar form of expressing data. The informational videos actually used more offensive words and were more clustered than the conspiracy theory videos. A lot of the information in this graphic seemed obvious (conspiracy theory comments were more concerned with foreign entities), but much of the data we could pull from it was useful. I will definitely be looking into making cluster graphs like this a part of my project.

Read More

Early Public Responses to the Zika-Virus on YouTube: Prevalence of and Differences Between Conspiracy Theory and Informational Videos

This paper focuses its research on the initial user reaction to informational and conspiracy videos on YouTube about the Zika-Virus. Their data set was the 35 most popular videos on the issue. Through their analysis, they were able to find that most (23) of the videos were purely information while the remaining (12) offered different conspiracy theories. They found that user activity between the videos seemed to be the same, whether or not the content was informational or conspiracy, with low user engagement. The difference between the two was within the contents and sentiments of the reactions. Typically, informational responses focused on the disease itself and how it affects babies and women in Brazil while conspiracy responses seemed to to target certain people or demographics for starting the virus.

One thing that surprised me was how oddly specific the subject matter was. The authors chose to research YouTube videos from a virus that had appeared two years earlier. I thought that such a targeted subject would be very limited in its analysis but it does make some sense as most, if not all, of the content for the Zika-Virus has been uploaded and the data set is non-continuous. I would have liked to see them tackle multiple different health issues on YouTube in order to avoid different biases that may appear for specific subjects. That being said, the scope for the project was smaller as a result and might be a great example for choosing a feasible semester project.

One thing that didn’t make sense to me was why the authors chose only the “first phase” of the Zika-Virus in 2016. It seems they want to test the initial response of the population but don’t seem to explain why. How do the initial phases differ from the entire ordeal altogether? I think that the results and types of user interactions would have the same if not similar to the found results, but the data-set would be much larger and reliable than the 35 videos they were able to find.

Some additional research questions that could come out of this:

• Just given the comments of individual YouTube videos, can we use them to predict whether or a video is a conspiracy or informational video?
• Along the point about the “initial phase” could we compare the different phases of the Zika-Virus and the user interaction / dissemination?

Automated Hate Speech Detection and the Problem of Offensive Language

This paper researches the differences between hate speech and offensive language. Their data set consisted of 25,000 tweets that contained speech lexicon found on Hatebase.org. Through their analysis, they found that it was quite easy to classify offensive language, with a 91% prediction rate but had a little more trouble identifying hate speech with a 61% prediction rate. They found that those that were correctly identified as hate speech often were extreme and contained multiple racial and homophobic slurs.

What I would have liked to see are more explanations on the features that they used. What are the TF-IDF, POS, and reading score methods they decided to use and why are they helpful to the research? This is something I found useful in some of the other papers we have read so far.

Some additional research questions that could come out of this:

• Using the terms generally associated with hate speech. Can we find a percentage of tweets that are hateful / offensive that contain them?
• As noted in the conclusions, another topic would be to look at the users who have been known to use hate speech by looking at their social structures and motivations. But also I think it would be interesting to see data on their demographics and how they might change over time as well considering the responses to their hate speech.

Read More

Automated Hate Speech Detection and the Problem of Offensive Language

Summary:

This paper used crowd-sourcing to label a group of tweets into three different categories: containing hate speech, only offensive language, and those with neither. The researchers of this paper accomplished this by training a multi-class classifier to differentiate between these three categories. The main obstacle addressed was being able to identify hate speech versus offensive speech because there are similar overlapping offensive remarks used in both types of speech.

Reflection:

Something that I’m interested to see in the future is how major social media platforms will respond to the criticism they receive about regulating hate speech. Because of the increasing legal implications of individuals who post hate speech, Twitter and Facebook must be careful when identifying hate speech. If social media platforms autonomously removing posts, how accurate would their algorithms be? As we can see by numerous other studies done on identifying hate speech versus offensive speech, algorithms are still being improved. However, if they manually remove posts, would their removal rate be too slow compared to the rate of post being created that have hate speech? Whatever happens in the future, social media platforms must address the growing problem of hate speech in a careful manner.

Another thing that caught my attention in this paper was that they properly defined hate speech and the process the researchers used for labeling the data. By giving three or more coders their specific definition of hate speech for labeling each tweet, I believe their process makes a lot of sense and does a good job in making sure that they accurately label tweets for their classifier.

Lastly, I appreciate the fact that they used a variety of models to find which model(s) perform(s) the best, instead of simply choosing one or two models. However, one thing I am curious about what features that were used in the final model that was a logistic regression with L2 regularization.

Further Work:

I believe that some future work to improve the model from this paper are to check if the quotes from a song are being used for appreciation or hate speech and checking for cultural context for some offensive language. Furthermore, I am curious if the current definition for hate speech can be even more specific in order to improve the labeling of tweets, therefore, improve the classifier. Lastly, the best way of truly addressing hate speech is by understanding what the root cause is. Maybe by researching different media sources that incite hate we could try to better identify users that use hate speech instead of posts of hate speech.

Early Public Responses to the Zika-Virus on YouTube: Prevalence of and Differences Between Conspiracy Theory and Informational Videos

Summary:

This paper sought to research how much informational and conspiracy theory videos differ in terms of user activity such as number of comments, shares, and likes and dislikes. Furthermore, the also analyzed the sentiment and content of the user responses. They collected data for this study by finding YouTube videos with at least 40,000 views on July 11, 2016. Their search for YouTube videos resulted in a data set containing 35 videos. Their results were that 12 out of the 35 videos were focused on conspiracy theories. However, no statistical differences were found in the number of user activity and sentiment between informational and conspiracy theory videos.

Reflection:

In the present day, YouTube is one of the largest platforms where countless number of people are accessing and posting new video. It can be said that communication have been substantially influenced by platforms like YouTube since it is very easy for people around the world to post videos. With growth of YouTube comes a lot of challenges such as the Ebola outbreak in 2016. I appreciate the effort this study made in trying to differentiate information and conspiracy theory videos. The researchers of this paper provided detailed definitions on the two types of videos and clearly explained how their data collection process. Personally, I am surprised that the sentiment in both types of videos were similar–I had thought there would be a significant difference. However, this study had a small dataset and didn’t have strong arguments.

Future Work:

A sample of size of 35 seems too small when doing any sort of significance test. In the present day, YouTube is one the largest platforms for videos where numerous videos are being posted every hour and the researchers of this study found only 35 videos. My suggestion to these researchers are to increase their sample size by finding more videos. In addition, research any other features that can help when differentiating informational and conspiracy theory videos.

Read More

Summary:

Automated Hate Speech Detection: This study is about differentiating hate speech from other forms of offensive language. This study uses crowd-sourcing to create a dataset of tweets classified into three categories: hate speech, offensive language, and neither. Multiple models were tested: logistic regression, naïve Bayes, decision trees, random forests, and linear SVMs. The results show that 40% of hate speech was misclassified as offensive or neither.

Early Public Responses: This paper studies the presence of conspiracy theories on YouTube, specifically in Zika virus-related videos. The study looks for differences between user activity, sentiment, and content of two classifications of videos: informational and conspiracy. It is found that user activity and sentiment are similar between the two classifications, but content is different.

Reflection:

Accuracy of CrowdFlower: I wonder how similar is the labeling of different CrowdFlower workers. This could be tested as follows: Let all the workers classify the same set of 100 tweets. For each tweet, calculate the variance in the classifications made by the workers. High variances mean that the workers are classifying the same tweets differently.

Flagged posts: I think it would be interesting to investigate patterns in posts that get flagged for review or reported as inappropriate. Do people tend to flag posts excessively, or not enough? I think these results might affect how social media sites, such as Facebook and Twitter, develop policies on hate speech.

Hate speech in retweets and replies: The paper didn’t mention if the dataset they studied contained only original tweets, or also retweets and replies. I think it would be interesting to study how hate speech differs between these types of tweets. Where is hate speech most prevalent on Twitter?

I think the conclusion of the “Automated Hate Speech Detection” study can be improved. The significance of the findings and future work should both be a lot clearer and concrete.

In the “Early Public Responses,” I think the data could be presented better. Bar graphs would probably be easier to understand than tables.

Small sample size: The sample size is very small (n=23 for information and n=12 for conspiracy). I think the paper should have talked more about how this might affect their results.

Conspiracy theories on other types of social media platforms: I think the same type of study can be done on social media platforms other than YouTube. For example, we can study the prevalence of conspiracy theories in Twitter. Number of views would be replaced by number of retweets, and replies and likes would stay the same.

Read More

Early Public Responses to the Zika-Virus on YouTube: Prevalence of and Differences Between Conspiracy Theory and Information Video

Summary:

The study analyzes 35 videos on Youtube that were posted during the first phase of the Zika Virus in 2016. The videos were posted between December 2015 and July 2016, and they all have at least 40,000 views. Of the 35 videos, 12 are focused on conspiracy theories, and the rest are informational videos. User responses to the videos are analyzed to see if there are different sentiments, and the implications for future online health promotion campaigns are discussed.

Reflection:

The study provided the following conclusions:

• Users respond similarly to both informational and conspiracy type videos.
• Results contradict Vousoughi, Roy and Aral who found that false news triggered more negative sentiments than real news.

With the prevalence of the internet, news travels much more quickly than it did during the pre-internet days. While this is generally a good thing, it unfortunately means that fake news also travels more quickly. It is concerning that user responses on conspiracy videos are similar to user responses on legitimate, informational videos. This brings up the questions of:

How can we best distinguish whether or not an article/video is a conspiracy theory or not? Comments on articles and video can influence viewers/readers. For some, it can be the main factor in determining if certain news is legitimate, or fake. Since this study found that there is no significant difference, how should we go about informing the public on spotting conspiracy theories?A

Why do these results contradict Vousoughi, Roy and Aral?
Since these results contradict Vousoughi, Roy and Aral, there may be another variable to consider besides video type (information or conspiracy).

It is important to keep this study in mind when publishing information on health. It has been found that at least for the Zika virus, it can be hard to tell the difference between conspiracy videos and informational videos.

Automated Hate Speech Detection and the Problem of Offensive Language

Summary:

It is hard for amateur coders to automate the process of detecting hate speech. If hate speech is associated with offensive words, then we inaccurately consider many people to be hate speakers, when in fact, they only use profane language. Hate speech is difficult to determine simply from looking for key words.

Reflection:

The following conclusions were found in this study:

• Hate speech is a difficult phenomenom, and it can not easily be categorized.
• What we consider hate speech tends to reflect our own subjective biases.

It turns out it is not as easy to point out hate speech as we may think it is. People perform well at identifying certain types of hate speech, but seem to miscategorize other types. For instance, people see racist and homophobic slurs as hate speech, but tend to see sexist language as offensive, but not hate speech. The article brings up the following question:

What really differentiates hate speech from offensive language?

As humans, we can subconsciously sort of tell when something is hate speech vs merely offensive. The question, however, is what exactly is it that makes us realize that hate speech is hate speech? Is there a way to quantify how hateful a particular statement is? Hate speech is monolithic, and comes in many forms, but is there a way to quantify what it is that makes hate speech hate speech, and not simply offensive?

It is important to be able to recognize hate speech online. Because of the ambiguity that exists in determining hate speech vs offensive speech, it can be hard to tell if groups are being attacked or praised, especially when you factor in that sarcasm can be used to both praise and attack various groups of people.

Read More

Summary:

This paper was primarily about the differences between informational and conspiracy videos referring to the Zika virus during the most recent outbreak. Through analyzing the 35 most popular informational and conspiracy videos concerning the virus, comparisons were made between these types of videos based on a set of features. Some of the key conclusions drawn are listed below.

• A majority of the videos were classified as being informational videos (23 out of 35).

• There were no differences found in user activity or the sentiment of user responses between the two types of videos.

Reflection:

I have listed below a line that interested me in the paper.

“YouTube considers the number of views as the fundamental parameter of video popularity. Hence, collecting videos with the highest number of views, for our given search string, allows us to capture those videos that would be listed first by search engines”

I’m very hesitant to believe that views are the fundamental parameter of video popularity. Although views are the most obvious factor in portraying video popularity, they can easily be skewed. It’s impossible to determine if views are paid or organic, and if a user wants to get a point across with a video that likely won’t garnish much attention (i.e. a conspiracy theory), they may resort to this method. I think another potential factor to consider in choosing videos would be audience retention, as there are users who see a title and click on a video, but immediately stop watching after a few seconds.

Other Thoughts:

Overall, I was surprised by the similar sentiment found in both types of videos, as I had imagined there to be a drastic difference. One statistic that did not surprise me was the lower vaccination rates in states where misinformation was prevalent.

Automated Hate Speech Detection and the Problem of Offensive Language

Summary:

This paper was primarily about hate-speech detection, and differentiating between hate speech, offensive language, and neither. Through analyzing tweets from each of these categories, a crowd-sourced classifier was trained to classify between these groups. Some of the key conclusions drawn are listed below.

• Tweets with the highest probability of being hate speech usually contain racial or homophobic slurs.

• Lexical methods are effective ways to identify offensive language, but are inaccurate in determining hate speech.

Reflection:

I have listed below a line that interested me in the paper.

“While these tweets contain terms that can be considered racist and sexist it is apparent than many Twitter users use this type of language in their everyday communications.”

Classifying hate speech appears to be an arduous task, arguably more difficult than detecting fake news articles. This is especially due to the fact that many Twitter users use hate speech trigger words when they are not tweeting out hate speech (a common misclassification is with song lyrics). In order to better detect and classify hate speech, I believe that there needs to be a more consistent definition of what the term encompasses. More research also needs to be done on being able to factor in false words such as “love” that consistently fool hate speech detectors.

Other Thoughts:

Overall, I found myself pretty engaged in this topic. I think that the topic of targeting hateful content is interesting, and I’m heavily considering working on it for the semester project.

Read More

Title: Early Public Responses to the Zika-Virus on YouTube: Prevalence of and Differences Between Conspiracy Theory and Informational Videos

Summary:

The purpose of the study was to determine the difference in user response (views, replies, and likes) between informative and conspiracy-based YouTube videos.  The research was conducted over a set of data collected on the most popular videos on YouTube during the zika-virus outbreak in 2016.  Their result showed that 12 out of the 35 videos in the data set focused on conspiracy theories, but no statistical differences were found. The result of research can be used to improve future online health promotion campaigns and combat the spread of false information.  Based on multiple findings, the study provided the following conclusions:

• Results on user activity showed no statistically significant differences across the video types
• YouTube users respond in similar ways, in terms of views, shares and likes, to videos containing informational and conspiracy theory content. 
• Understanding the various types of contestation present in YouTube video user responses on the Zika-virus is important for future online health promotion campaigns 

Reflection

YouTube has changed the way we acquire and spread information in our society. Everyone has now easy access to start a podcast or channel to spread information. This also brings a lot of challenges and one of them is the spread of Conspiracy theories. We don’t have to look no more than the Ebola outbreak in 2016 to see the threat it posed in our society (see the New York’s time article titled “Ebola Conspiracy Theories” for more information). I believe that this study provided a step forward in tackling this problem. Even though, I am really impressed by their findings and conclusion, the study lacked concreate arguments and a broader data set to back up their findings. Moreover, a lot of assumptions were taken which affects the creditability of their study. Nevertheless, their research gives a great insight for future studies. Considering their findings and summarization, we can reflect on different aspects and their implications. 

Part 1: From their findings that amazes me the most is that results on user activity showed no statistically significant differences across the video types. 

 Questions and further research 

1. One question we can ask is there any difference in terms of video types. I believe that conducting a research on users across different videos give insights about why conspiracy videos spread easily.  
2. Can we determine the standard a video based on users’ activity or account information? This will help us to perfectly identify. 

Part 2: that stack out of me after reading the study is that do conspiracy videos differ in terms of their approach? Do they change their approach time to time or stay consistent? I believe that doing more research on multiple Conspiracy videos on YouTube will help us to solve this problem. 

Title: Automated Hate Speech Detection and the Problem of Offensive Language

Summery: 

The purpose of the study was to improve the detection method for hate speech from other instances of offensive language. They used a crowd-sourced hate speech lexicon to collect tweets containing hate speech keywords and trained a multi-class classifier to distinguish between these different categories. Based on multiple findings, the study provided the following conclusions:

• racist and homophobic tweets are more likely to be classified as hate speech but that sexist tweets are generally classified as offensive 
• Tweets with the highest predicted probabilities of being hate speech tend to contain multiple racial or homophobic slurs 
• Tweets without explicit hate keywords are also more difficult to classify 

Reflection: 

With Twitter and Facebook becoming the most powerful medium to reach the public, it is essential that we combat the spread of heat speech through those platforms. The research did a great job in terms reliably separating hate speech from other offensive language. But I believe still more work has to be done to improve the classifier. I am not convinced that the human classification is the perfect way to classify tweets. Maybe using smart algorithms can improve the results. 

 Questions and further research 

• Does a difference in culture affect or influence hate speech? Conducting research on different group of people will provide some meaningful findings  
• What kind of content does hateful users consume? We can easily identify the root cause of hate speech by studding users who consume or spread heat speech. 
• Is there any significant difference in word usage between hate and offensive speech? We might be able to determine what type of a speech is based on usage stop-words and nouns, proper nouns and verb phrases. 

Read More

Early Public Responses to the Zika-Virus on YouTube: Prevalence of and Differences Between Conspiracy Theory and Information Video

Summary

This paper focuses on comparing viewer responses on informative and conspiracy videos about the early-phased epidemics (Zika-virus). Authors collected 35 top-YouTube-ranked videos and studied the user activities and the comments and replies. The results show that there is no statistically significant differences across informative and conspiracy video types, in terms of user activities, and there is no differences in the sentiment of the user responses to the two video types. The only significant result is that neither of the two types of video content promotes additional responding per unique user.

Reflection

This paper is a very well-written paper. It has very detailed definition on conspiracy theories and the problem’s background. The authors carefully explained how they obtained the data set, and suitable scenarios of different statistical tests used during the research. 

Different from previous papers that we read for this class, this paper does not draw any significant insights from the all the tests. No significant differences on the selected features between informational and conspiracy theory videos was the final conclusion.

This conclusion is definitely a surprise to me. The 12 collected conspiracy videos are very much like fake news. They are mostly negative and mention a lot of proper nouns in their contents, so these conspiracy videos getting the “same” reactions as the informative ones didn’t reflect the comparison between real and fake news.

I thought the size of the data set (only 35 videos) was a limitation of this work. However, this research focuses solely on the first phase of the Zika-virus, so it might be difficult to collect a larger set of sampling videos. I also realized that people are very unlikely to go further than the top 35 videos on YouTube with any search query, not to mention interacting with the video (like, dislike, comment, etc.) 

One solution to this limitation could be surveying videos from different phases of Zika-virus. This future work enables the following questions

• Is there a shift on topic weights on comments of both types of videos?
• Does user activity change over different phases on informative and conspiracy videos?

In the first phase of any new epidemic outbreak, only little facts are known to scientists. However, as more studies are done, maybe scientists will have a better understanding of the epidemic, and informative videos will shift topic from consequences to causes. This topic shift in informative videos might also be reflected in the comments. We can already see that the weight of informative videos’ comments is heavier on “affected stakeholders” and “consequences of Zika” instead of “causes of Zika”, while conspiracy videos’ comments are more focused on the causes.

There also might be changes in user activities on the two types of videos. Conspiracy videos’ contents might be consistent throughout different phases, since the conspiracies are all “explanations for important events that involve secret plots by powerful and malevolent groups.” Informative videos’ contents will for sure change with the increase of known facts and evidences, so people might be more willing to share and view informative videos.

We could also further this study by surveying the first phase of other epidemics; however, YouTube might not be the best social platform if we want to survey a wide range of epidemics.

Automated Hate Speech Detection and the Problem of Offensive Language

Summary

This paper addresses automating hate speech detection using a classifier to identify hate speech and offensive language tweets. Authors collected 33,458 Twitter users and selected 25k random tweets from total 85.4 million tweets. These tweets then get manually labeled into “hate speech”, “offensive language”, and “neither” for supervising learning algorithms and models. Then, the paper went on to talk about the output from the trained classifier.

Reflection

The topic of hate speech vs. offensive language is something that I have never thought about, and I’m surprised that there have been researches that systematically study the difference between the two categories of language. I like how this paper demonstrates the difficulty of this topic by listing out errors in previous studies. I don’t like how the paper is very brief on the Model section. It wasn’t detailed on why exactly they decided to use these certain algorithms and the features used in these models.

Because of the need of large amounts of crowd sourcing and outside resources (hatebase.org), there are a lot of inaccuracies in both the dataset and the results. One obvious solution for the imprecision of the Hatebase lexicon is to find a better outside resource that better identify hate speech lexicon. However, it might be difficult to find a “perfect” outside resource since there is no formal definition for “hate speech”.

Figure 1 in the paper shows that the classifier is more accurate on the “Offensive” and “Neither” categories than on the “Hate” category. I’m wondering whether this is because of the strict definition of “hate speech”.

Future work of this topic may include running considering other methods on the dataset. For example, 

• Can natural language processing algorithms, such as Named Entity Disambiguation/Recognition, help the classifier to determine what the event trigger is and then make a decision?
• Can non-textual features, such as location and registered race, help identifying the context?

Read More

Early Public Responses to the Zika-Virus on YouTube: Prevalence of and Differences Between Conspiracy Theory and Informational Videos

Summary:

As YouTube emerged as a popular social media platform, it also became the favorite playground of fake news and conspiracy theory videos. Surprisingly, most of the videos on this website are health-related. This creates a concern that fake news and conspiracy theory content can mislead and install fear among the audience thus affecting the spread of epidemics. The researchers collected the content of the 35 most widespread YouTube videos during the first phase of the Zika-virus outbreak – from December 30, 2015, to March 30, 2016 – and user reaction to those videos. This paper aims to find the dissimilarities in terms of user activity (number of comments, shares, likes, and dislikes), and the attitude and content of user responses between informational and conspiracy videos.

Unexpectedly, there is no significant difference between these two types of videos in most cases. Both informational and conspiracy videos share not only the same amount of responses and unique users per view but also a low rate of additional responding per unique user. Furthermore, the user comments on these two types of videos are slightly negative, which challenging Vousoughi, Roy and Aral’s conclusion that false news provokes more negative sentiments than true news. However, there is a dissimilarity in the content of user response between informational and conspiracy videos. Informational videos user responses focus on the consequences of the virus. Whereas, the conspiracy theory videos user responses are more interested in finding out who is responsible for the outbreak.

Reflection:

    Even though the studying of misleading health-related content on YouTube might be worth researching, the authors’ choice to limit their research to the topic Zika-virus is, sadly, not a wise decision. First of all, despite the fact that YouTube has over one billion highly active users watching around six billion hours of videos per month, it is also true that distribution of user age and interest are extremely skewed. Also, a large percentage of YouTube users are teenagers and young adults who have little or no interest in Zika-virus. In fact, there is a huge gap in user activity and engagement between popular topics (gaming and beauty) and other videos. Much of the popular health-related content (anorexia, tanning, vaccines, etc.) is also linking to beauty trends. In other words, YouTube is not an ideal social media forum for studying user response to different type of videos of Zika-virus or most health issues. The perfect proof for this is the extremely small dataset the authors were able to acquire on the topic. So, if the same question was asked about videos related to vaccines, the result might have been more interesting.

    Another concern about the dataset is that there is a lack of explanation and inspection of the data collected. What are the user statistics: age range, gender, prior behavior, etc.? Do the dataset includes videos that were reported as many conspiracy theory videos can be flagged as inappropriate? Why did the authors choose 40,000 as the cut off for their dataset? How popular are Zika-virus related videos compared to other videos posted on the same day? 

    As mentioned above, the most active group of YouTube users are not attracted to the topic of Zika-virus. Older users are less likely to show reactions on social media forums, especially on controversial topics. Therefore, the user inactivity and passiveness can make the difference in user activity between informational and conspiracy theory videos seem insignificant. Also, the low rate of additional responding can also be the result of the user behavior on social media forums rather. There needs to be more analysis of the user behavior before concluding that the user activity is similar across the two types of videos.

    The most interesting analysis of this research is the semantic maps of user comments between informational and conspiracy theory videos. The clusters of the informational videos are bigger and more concentrated. Surprisingly, offensive words (shit, ass, etc.) are used more frequently in informational videos. Moreover, the comments in conspiracy theory videos are more concerned with foreign forces such as Brazil, Africa, etc. Meanwhile, the audience of informational videos focuses more on the nation’s eternal conflicts between parties and religions. What might be the cause of the difference in interest and usage of offensive language? The fact that Bill Gates, the only businessman, is not only mentioned but also has his name misspelled frequently is interesting. Why did he appear as much as presidents of the United States in the comment? Does the common misspelling of his name indicate the education level of the audience?    

Automated Hate Speech Detection and the Problem of Offensive Language

Summary:

    The paper address one of the biggest challenges of automatic hate-speech detection: distinguish hate speech from offensive language. The authors define hate speech “as a language that is used to expresses hatred towards a targeted group or is intended to be derogatory, to humiliate, or to insult the members of the group.” Even with a stricter definition of hate speech than previous studies, the authors still find the distinction between these two types of expression too ambiguous and case specific. The raw data contains tweets with terms from the hate-speech lexicon, which is compiled by the website Hatebase.org. Then, the tweets were reviewed and grouped into three categories: hate-speech, offensive language, and none of the above. After trials and errors, the authors decided to build a logistic regression model. Even though the algorithm achieves a relatively high overall precision of 91%, it does not accurately differentiate hate-speech from offensive languages with a misclassification rate of 40%.

    Despite the improvement in precision and accuracy compared to previous models, the research acknowledges that there are plenty of issues with the current model. The algorithm often flags tweets as less hateful or offensive than human coders. The heavy reliance on the presence of particular terms and neglect of the tone and context of the tweets result in a high misclassification. Moreover, it is noticeable that sexist tweets are much less likely to be considered as hateful than racist or homophobic tweets. The authors suggest future research should consider social context and conversations in which hate-speech arises. Also, studying the behavior and motivation of hate speakers can also provide more insight into the characteristics of these expressions. On the other hand, another pitfall of the current model is that it performs poorly on less common types of hate-speech such as those targeting Chinese immigrants.

Reflection:

    It is not an exaggeration to say that even humans are struggling to differentiate hateful speeches from those that are merely offensive, let alone an algorithm. This research shows that the presence and frequency of keywords are by far sufficient in distinguishing hate speech. Indeed, in most cases, the decisive factors determining whether a tweet is hateful or offensive are the context of the conversation, the sentiment of the tweet, and the user speaking pattern. However, analyzing these factors is much easier said than done.

    The authors suggest user attributes such as gender and ethnicity can be helpful in the classification of hate-speech. But acquiring these data might be impossible. Even in the few cases in which users agree to provide their demographic information, the trustworthiness is unguaranteed. However, these attributes can still be derived from analyzing the user behavior in the forum. For example, a user who often likes and retweets cooking recipes and cat pictures have a higher chance to be a woman. Therefore, future studies can consider looking into a user prior behavior to predict their tendency of expressing hate-speech and classify their tweets’ nature.

    One of the main causes of the high misclassification of hate-speech is that the word choices and writing styles vary greatly among users. Even though most hateful expressions are aggressive and contain many curse or offensive terms, there are plentiful exceptions. An expression can still “intended to be derogatory, to humiliate, or to insult” disadvantaged groups without using a single offensive term. In fact, this is the most dangerous form of hate-speech as it often disguises itself as a statement of truth or solid argument. On the other side, there are statements using flagged terms yet convey positive messages. Therefore, rather than focusing on the vocabulary of the tweets, it might be better to analyze the messages the users want to convey.  

Read More

Automated Hate Speech Detection and the Problem of Offensive Language

Summary:

In this article the authors attempted to use crowd sourcing to label a sample of tweets into three categories. These categories included hate speech, only offensive language, and those with neither. They did this by training a multi-class classifier to distinguish between these different categories. One of the key challenges is separating hate speech from other instances of offensive language.

Reflection:

Something that was very interesting to me was that both Facebook and twitter have gotten a lot of criticism for not doing enough to prevent hate speech on their sites. They responded to this by instituting policies to prohibit attacking people based on characteristics like a race, ethnicity, gender, and sexual orientation. Now I know a major way of getting these tweets or Facebook posts taken down is based off of user input, like reporting a post. I wonder if either of these platforms will take an autonomous route and remove posts automatically if they’ve reached the arbitrary quota for hate speech or offensive language. Something that would concern me about implementing this feature is if it takes it too far. For example a game that I play, Rainbow six siege, has very strict text limitations for in game chat. You can get banned for saying words that are not offensive but register as hate speech or offensive language to the machine.

Overall this is one of the articles that was very insightful, looking at it now I realize that distinguishing hate speech from offensive language is difficult but it’s not something that I would’ve thought about before reading the article.

Future Work:

There are two major things that I think would be contributing factors to distinguishing between hate speech, offensive language, and those containing neither. One would be seeing if the language was quoted from lyrics and if it was offensive or if it was just someone appreciating a song. The second would be if the text was just cultural difference that triggers some of the offensive language or hate speech words.

Early Public Responses to the Zika-Virus on YouTube: Prevalence of and Differences Between Conspiracy Theory and Informational Videos

Summary:

In this paper the authors wanted to examine the extent to which informational and conspiracy theories deferred in terms of user activity. They also wanted to check the sentiment and content of the user responses. They collected their data from the most popular videos posted on YouTube in the first phase of the Zika-virus outbreak in 2016. The results show that 12 out of the 35 videos in the data set focus on conspiracy theories however there were no statistical differences between the two types of videos.

Reflection:

Upon first reading, this paper seems fairly insignificant. Many times the authors said that there were no statistically significant findings. Something interesting the author said was YouTube videos often have misinformation on health related issues. I’d be curious to see what percentage of you youtubers that released videos on the Zika-virus were actually informed enough to make a video for the general public. I know the author wanted to see if this could be generalized towards other youtube video categories as well and I believe that there’ll be a lot of similar issues at least within the health field. Speaking from my sample set of videos I watch, there’s so much misinformation when it comes to videos related to powerlifting or bodybuilding. Often times there aren’t enough facts to back a statement, it just comes down to personal preference and what works best for you as an individual. When it comes to information that can be varied from user to user, I wonder how they would label what “misleading” exactly is.

Future Work:

Something that can be done to further research is to have a greater sample size than 35 videos. Also seeing if there are any more features that I can help to distinguish between informational videos and misleading videos.

Read More