Mitra, Tanushree, Graham P. Wright, and Eric Gilbert. “A parsimonious language model of social media credibility across disparate events.” Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing. ACM, 2017.
Summary:
The authors of this paper examined how people perceived the credibility of events that were reported by users. The goal was to build a model that would be able to identify, with a good degree of accuracy, how credible a person will perceive an event based on the text of the tweets (and replies to the tweets) related to the event. To do this, they used a dataset of collected twitter streams with the tweets classified by events and the time when they were collected. These tweets were also rated for their credibility on a 5 point scale. The authors also put forward 15 language features that could be used to influence the credibility perception. With all this in hand, the authors were able to analyze and identify the words and phrases in the different language feature categories that corresponded to high and low perceived credibility scores for the various events. Though the authors advise against using the model on its own, it can be used to complement other systems such as fact checking systems.
Reflection:
What I found most interesting was the phenomenon of how the perception of credibility seems to flip for positive to negative and vice versa between the original tweets and replies. I first thought there might be a parallel here between tweets-replies and news article-comments but of course that doesn’t work because there are cases where the replies are perceived more credible than the original so that parallel doesn’t always work because the original tweets are not always credible. (Then again, there are cases where a news article is not necessarily credible so maybe there is a parallel here after all? I’m sorry, I’m grasping at straws here.)
“Everything you read on the internet is true. – Mahatma Gandhi.” This is a joke that you’ll sometimes see on Reddit but also serves as a warning against believing everything you read because you perceive it to be credible. The authors of this paper mentioned how the model can be used to identify content with low credibility and boot it from the platform before it can spread. But could it also be used by trolls or people with malicious intent to augment their own tweets (or other output) in order to increase the perceived credibility of their tweets? This could certainly cause some damage as we are talking about false information being more widely believed because it was improved thanks to algorithmic help where otherwise it may have had a low perceived credibility.
Another thing to consider is longer form content. This analysis is necessarily limited by their dataset which only consists of tweets. But I have often found that I am more likely to believe something if it is longer and articulate. This is especially apparent to me when browsing Reddit where I am more likely to believe a well written, long paragraph or a multi paragraph comment. I try to be skeptical but I still catch myself believing something because it happens to be longer (and also believe it more when it is gilded, but that’s for a different reflection). So the question that arises is: What effect does length have on perceived credibility? And how do the 15 language factors the authors identified affect perceived credibility in such longer form text?
