Analyzing Right-wing YouTube Channels: Hate, Violence and Discrimination

            Ottoni, R. et al (2018) investigated the differences between user interactions on far-right wing YouTube channels, compared to a baseline sample of channels.  The study specifically searched for differences in lexical and topical between the two types of channels, as well as the instances and degree of implicant bias in the texts.  The two main research questions were as follows:

• Is hateful vocabulary, violent content and discriminatory biases more, less, or equally prevalent in right-wing channels?
• Are video commenters more, less or equally aggravated than video hosts, and do they express more, less, or and equal amount of hate and discrimination?

Ottoni, R. et al (2018) hoped that the methods detailed in the study could be extended to any type of text.  The following results were found:

• Right-wing channels tended to contain more “negative” semantics fields, such as “hate”, “kill”, “anger”, “sadness”, and “swearing”.  This is opposed to “positive” semantics such as “joy”, “love”, “optimism”, and “politeness”. 
• Typically approached topics related to war and terrorism.
• Demonstrated more discriminatory bias against Muslims and towards LGBT people.

I personally found this study to be fascinating, as well as some of the results.  I disagree that results of a study have to be surprising to be worthwhile.  We are very good at misleading ourselves (i.e. superstitious behaviors, etc.), thus I find merit in finding unsurprising results if they answer a research question that has not been pursued before.  Having said that:

• I did not find the results of Ottoni, R. et al (2018) to be particularly surprising.  While I am sure in group members with right-wing leanings likely have different opinions on the matter, I believe there has already been a lot of research on the emotions and language choices displayed my left- and right-wing leaning people.  I personally conducted a badly designed study on the differences in emotion displayed by left- and right- wing leaning people when viewing internet memes (a study I would like to revisit eventually).  Many of the results I found corresponded to the results found in this study.
• The study implemented a variety of analysis methods beyond lexical analysis.  In the previous reflection, I made the point that perhaps lexical analysis on its own does not provide as much precision within the model as we would hope for.  The inclusion of topical analysis and implicate bias analysis appears to cover a few additional facets to perceiving intent behind the text. 

Work Cited:

Ottoni, R., Cunha, E., Magno, G., Bernardina, P., Jr., W. M., & Almeida, V. (2018). Analyzing Right-wing YouTube Channels. Proceedings of the 10th ACM Conference on Web Science – WebSci 18. doi:10.1145/3201064.3201081

Read More

Automated Hate Speech Detection and the Problem of Offensive Language

            Davidson, T. et al (2017) studied the detection of hate speech and offensive language on social media platforms.  Previous lexical detection methods failed to separate offensive speech from hate speech.  The study defined hate speech as: language that is used to expresses hatred towards a targeted group or is intended to be derogatory, to humiliate, or to insult the members of the group.  This definition allows for the classification of offensive language that uses words other models would normally classify as hate speech.  The model was formed by a collection of CrowdFlower workers classifying a sample of tweets as hate speech, offensive speech, and normal speech.  It was found that by creating additional classifications for different types of speech (for example offensive speech) leads to more reliable models.  While this study found methods to increase reliability in some cases, it also addresses way that the model could be improved with further research. 

            While I found the paper to be fascinating, and most definitely an area we need further research in, I found a few issues with the methods used in gathering and interpreting the data fed into the model, as well as the model itself. 

• Judging from the results, especially with classifying sexism as offensive language versus hate speech, there is likely to be a human created bias in the classification of the tweets.  This may be due to a non-random sample provided by CrowdFlower.  Another possible, albeit unfortunate, explanation is that sexism is commonplace enough that it is not regarded as hate speech.  Either way, the bias of the people analyzing and feeding data into models is something that should be explored further (see Weapons of Math Destruction for further insight into this issue). 
• Another issue is that the coders appear to be prone to errors as well, which can affect the reliability of the model.  Davidson, T. et al (2017) found that there was a small number of cases where the coders misclassified speech.  This concern is somewhat related to the previous point I made.  Using convenience sampling (CrowdFlower, M-Turk, etc. are arguably forms of convenience sampling), introduces threats to internal validity.  Thus, with convenience sampling, there is the possibility of introducing bias into the study, as well as reducing the generalizability of the results.
• The lexical method appears to look more at word choice, than at context of word choice, which lead to a large majority of the misclassifications in the model.  While the method can be used as part of a more wholistic approach, it seems like a whole new method should be explored.  Perhaps a potential approach is classifying the likelihood of an individual user posting hate speech, thus creating more of a predictive model.  Although the ethical implications of such a model would need to be explored first.

Overall, I found Davidson, T. et al (2017) to be thought provoking and providing of additional research directions.

Early Public Responses to the Zika-Virus on YouTube: Prevalence of and Differences Between Conspiracy Theory and Informational Videos

            Nerghes, A. et al (2018) explored the user activity differences between informational and conspiracy videos on YouTube, specifically related to the 2016 Zika-virus outbreak.  The study sought to answer the following questions:

• What type of Zika-related videos (informational vs. conspiracy) were most often viewed on YouTube?
• How did the number of comments, replies, likes and shares differ across the two video types?
• How did the sentiment of the user responses differ between the two video types?
• How did the content of the user responses differ between the video types?

The study inspected 35 of the most popular Zika-virus related videos to answer these questions.  It was found that there are no statistical differences found between informational and conspiracy videos, as well as no statistical differences found in the number of comments, replies, like, and shares between the two video classifications.  It was also found that the users respond differently to sub-topics.

            One of the largest questions the Nerghes, A. et al (2018) study raised for me was:

• Are these results generalizable to other topics?  While the Zika-virus outbreak was significant, I have to ask how many people were truly invested in pursuing new information.  I find it likely that given other issues, the results could be different. 

Assuming that the results are generalizable to additional topics, the study continues to raise additional questions:

• Disregarding the fact that this study, and the associated results, were directed towards the health field, what can be done to increase user engagement?
• The next question we must ask is whether or not we should attempt to direct traffic away from conspiracy videos.  This question further depends on whether or not discussions on conspiracy videos yield positive results.  It would be interesting to explore how non-toxic engagement on items we would label as fake news or conspiracy theories results, and what some of the best approaches to starting and maintaining those discussions would be. 

Overall, I find that there are many new directions that could be pursued related to this subject.

Works Cited:

Davidson, T., Warmsley, D., Macy, M., & Weber, I. (2017). Automated Hate Speech Detection and the Problem of Offensive Language. Retrieved February 3, 2019, from https://aaai.org/ocs/index.php/ICWSM/ICWSM17/paper/view/15665

Nerghes, A., Kerkhof, P., & Hellsten, I. (2018). Early Public Responses to the Zika-Virus on YouTube. Proceedings of the 10th ACM Conference on Web Science – WebSci 18. doi:10.1145/3201064.3201086

Read More

This Just In: Fake News Packs a Lot in Title, Uses Simpler, Repetitive Content in Text Body, More Similar to Satire than Real News

Horne and Sibel (2017) sought to explore the differences between real and fake news.  Unlike prior studies, satirical news was considered an additional category. The primary question approached in the study was: “Is there any systematic stylistic and other content differences between fake and real news?”  To answer this question, Horne and Sibel (2017) gathered three different data sets:

1. Buzzfeed’s analysis of real and fake news items from the 2016 US Elections.
2. Self-collected news articles on US politics from real, fake, and satire news sources.
3. A previous study’s data set containing real and satire articles.

Contrary to the assumption that fake news is written to look like real news and fool the reader, Horne and Sibel (2017) found that the assumption is not true.  Rather, the articles employ heuristics such as title structure and noun choice.  It was also concluded that fake news specifically targets those who do not read beyond the title. 

While I found many of the finding to be fascinating, I was once again unsurprised by many of the findings.  The conclusion that fake news is most easily discernible, and most effective, via the title is something I have observed through the shared posts and associated comment sections on Facebook and Twitter for years.  However, beyond this initial concern, the study raised a number of concerns and questions:

• Foremost of these concerns is their strategy for dividing sources into real, fake, and satirical sources.  While these categories will work in most cases, increasingly often reputable (real) sources will have vast differences in the news they report.  Depending on the event, both sources cannot be correct, and perhaps neither source is correct.  Thus, bias also plays a large part in the real versus fake news cycle.  It would be erroneous to determine that all real sources post only real news, and fake news sources only post fake news. 
• Additionally, many, if not all, of the news articles concerned US politics.  This raises the question as to whether or not these findings can be generalized to other issues. 
• While Horne and Sibel (2017) raised some of the issues with reversing or combating fake news, they later failed to offer suggestions as to how to utilize their data.  I feel as researchers and information scientists, it is also our duty to take the next step beyond the study, even if that next step is just providing possible uses for the data or suggesting finding derived approaches to the issue at hand.  We are responsible for the information we find. 

Horne, & Sibel. (2017, March 28). This Just In: Fake News Packs a Lot in Title, Uses Simpler, Repetitive Content in Text Body, More Similar to Satire than Real News. Retrieved January 30, 2019, from https://arxiv.org/abs/1703.09398

Read More

Journalists and Twitter: A Multidimensional Quantitative Description of Usage Patterns

Bagdouri, M. et al (2016) conducted one of the largest to date studies on the usage patterns of journalists, news organizations, and news consumers on Twitter. The study also explored the differences in the aforementioned user groups between English and Arab users. At the offset, Bagdouri, M. et al (2016) proposed the following research questions:

• Do journalists engage personally with their audience compared to news organizations?
• Do observations about English journalists—who are typically studied in previous work—apply to journalists from different regional, cultural, and lingual backgrounds (e.g., Arab journalists)?
• Do journalists use Twitter in a manner dissimilar from news consumers, and do these (dis)similarities hold across different regions?
• Are journalists a homogeneous group, or do they differ as a function of the type of the news outlet they work for?
• To which extent do journalists who speak the same language, but belong to different countries share similar characteristics?

Through the Welch and Kolmogorov-Smirnov statistical tests, Bagdouri, M. et al (2016) compared fourteen features found in 13 million tweets of 5,358 Twitter accounts of journalists and news organizations, as well as two billion posts from over one million of their connections. The following results were observed:

• Organizations broadcast a number of tweets to a large audience, while journalists will target their tweets towards individual users.
• Arab journalists tend to broadcast to a larger follower base, while English journalists tend to have a more individual directed approach to a smaller follower base.
• Arab journalists are more distinguishable than their English counterparts.
• Print and radio journalists have dissimilar behaviors; however, television journalists share commonalities with both print and radio journalists.
• British and Irish journalists are largely similar.

     Although Bagdouri, M. et al (2016) succeeded in answering their original questions, I found the lack of explanation beyond the statistical analysis to raise a number of questions. Although quantifying certain phenomena is an important first step, establishing correlation would help us better utilize the data. Questions such as the following quickly rose to mind:

• Does the difference in broadcasting and targeting behaviors lead to a significant different in the reception of the news?
• What social (or other factors) lead to differences between Arab and English journalists, what can we learn from these differences, and how can we utilize that data moving forward?
• Does commonality between same language journalists extend beyond the British Isles?

While Bagdouri, M. et al (2016) occasionally attempted to answer a few of these questions, many of the explanations were closer to conjecture than to empirical evidence.

Cite:

Bagdouri, M. (2016). Journalists and Twitter: A Multidimensional Quantitative Description of Usage Patterns. Proceedings of the Tenth International AAAI Conference on Web and Social Media. Retrieved January 29, 2019.

Read More