Paper: Data ex Machina: Introduction to Big Data

The main theme of the paper is to critically examine the potential of big data in the field of sociology. The authors start by defining big data and its types (digital life, digital traces, digitalized life). They have reviewed several big data projects and research papers to highlight the opportunities and the challenges of using big data in the field of computational social science. They conclude by discussing six future trends that will affect the use of big data in future.

The paper for this week is a perfect way to conclude the course. It summarizes almost everything that we have learnt so far in the semester. The need for the social computing field is described perfectly in the paper in the line “Big data thus requires a computational social science—a mash-up of computer science for inventing new tools to deal with complex data, and social science, because the substantive substrate of the data is the collective behavior of humans (Lazer et al. 2009).” [Lazer et al]

The authors talk about biases in the self-reported behavior. Qualitative analysis is an important part of this research field that sometimes heavily relies on surveys and interviews. Thus, understanding and reducing biases from surveys and interviews is very important.

Now casting was a new term for me. It was interesting to see the impact of projects like “Billion Prices Project” [how Argentina stopped publishing inflation numbers and used this project to infer inflation]

The authors also reviewed projects where researchers have studied underrepresented population eg. people suffering from depression and having suicidal ideation. But not every population is represented well in all kinds of the online datasets- internet access is still limited in developing countries.

The authors talk about the core issues of big data, prominent issue being that scale of data can lead to the illusion that it contains all the relevant information about all kinds of people. So it’s important to understand what your data is. But how much data is needed to make “general claims” is a question no one has answer to.

A line in the paper “Twitter has become to social media scholars what the fruit fly is to biologists—a model organism.” indicates overuse of Twitter data for research due to its easy availability. The author argues that relying on a single platform can produce issues for generalizability.

The authors in the end discuss future trends, how data is only going to increase in the future coz of several digitization initiatives (almost everything is moving online, paper records are diminishing). Popularity of text based platforms is decreasing and platforms like snapchat and Instagram are rising in popularity. It seems in future, the bulk of data will consist of images and videos. It will be interesting to see different fields (computer vision + data analytics + sociology) coming together to analyse this data.

 

 

Read More

Paper 1: A 61-million-person experiment in social influence and political mobilization

Summary:

The authors study the effects of social influence for effecting behavior change. The results of their randomized controlled trial of political mobilization messages shows that these messages influence users’ political self-expression and real world voting behavior. The results also indicate the importance of social ties in spreading real and online behavior in humans.

Reflection:

The users were unevenly divided into the three groups with number of people in ‘social message’ being much larger than the ones in ‘informational’ and control group. But otherwise the experimental design and the way real life behavior was observed (by checking the public voting records) is amazing!! Also the scale of their experiment is huge!

The authors don’t mention if they were aware of the political leaning of the users or whether the dataset was balanced with equal no. of democrats and republican supporters. I read online that FB received backlash for this research since the dataset is assumed to be skewed towards democrats and it was claimed that the study influenced the voting results since more democrats came to vote.

The authors “arbitrarily” define close friends as the ones lying in the 8^th percentile or higher of frequency of interaction among all friendships. I’m not clear about this assumption. Was it just done so that in their dataset 98% of people end up having atleast one close friend (the avg being 10).

One of the premise in the paper is that social ties are crucial in spreading real and online behavior in humans. If that’s the case, can it be exploited to make people come out of filter bubbles. If we are able to identify “close friends” with opposing political ideologies and rank their political posts higher in the feed, will it make people click on the shared content?

Both papers are great examples of how social media can influence human behavior and thus the importance of social computing research.

 

Paper 2: Experimental evidence of massive-scale emotional contagion through social networks

Summary:

In this paper, the facebook researchers study emotional contagion outside of in-person interactions between individuals by modifying the amount of emotional content in News Feed of about 0.68 million people. The results of the study showed that displaying fewer positive updates in people’s feeds causes them to post fewer positive and more negative messages of their own. People who were exposed to fewer emotional posts in their News Feed were less expressive overall on the following days.

Reflection:

Design of Experiment: Not much information is given about the 0.68 million people selected for the experiment. Gender, age group and demographics can control the results of such an experiment. So knowing these details would have thrown more light on the results.

The researchers determined whether a post is negative or positive depending upon if it contained atleast one positive or negative word. I’m not convinced about this strategy of classifying facebook posts.

People who posted at least one status update during the experimental period were selected for the study. The threshold of one seems too less to claim the emotional state of a user.

Evaluation of results: Have the author’s considered scenarios where some event could have lead people to post more positive/negative posts [e.g home country winning a sport event etc], rather than the effect of emotional contagion.

The way people express their emotions on FB is complex [use of sarcasm, double negatives, images along with text etc.] So I’m not convinced that counting total number of positive and negative words accurately depicts the emotional state of a user. It seems very naïve. [Paper “Does counting emotion words on online social networks provide a window into people’s subjective experience of emotion? A case study on Facebook.” challenges the same belief]

Ethical consideration: The paper says ‘it was consistent with Facebook’s Data Use Policy, to which all users agree prior to creating an account on Facebook, constituting informed consent for this research.’ I wonder if the participants whose feed were affected were personally informed and if they explicitly agreed to be a part of it. If a study is expected to negatively affect someone’s mood, then I believe explicit consent is required.

Overall I’m still skeptical about the main claim of the paper “results suggest that the emotions expressed by friends, via online social networks, influence our own moods”. I believe other’s emotions might affect what we express on social media, but in no way it is a representative of the user’s actual emotional state.

Read More

Examining the Alternative Media Ecosystem through the Production of Alternative Narratives of Mass Shooting Events on Twitter

Reflection:

In this paper, the author studies the “fake news” phenomenon by generating and studying domain network graphs and qualitatively analysing tweets collected over a ten-month period. She explains the political leanings of the alternate news sources and describes how these websites propagate and shape alternate narratives.

Examples

“Global warming is a hoax created by the UN and politically funded scientists/environmentalists, aided by Al Gore and many others to put cap and trade in place for monetary gain”

” Osama bin Laden is not dead / has been dead for long / is an invention of the American government.”

” That ample evidence of alien life and civilization exists in our solar system, but is covered up by NASA.”

These are some of the popular conspiracy theories. Several others can be found on website that is dedicated to listing and categorizing these theories.

What is the need to study conspiracy theories?

Research has shown that if you start believing in one conspiracy theory, there is a high chance that you might start believing in another. “Individuals drawn to these sites out of a concern with the safety of vaccines, for example, may come out with a belief in a Clinton-backed paedophilia ring”, stated Kate Starbird in one of her published articles.

Why do people believe in conspiracy theories and false narratives? How to change their mind?

The author of an article replicated a part an existing study. He conducted a twitter poll asking users if the sequence 0 0 1 1 0 0 1 0 0 1 0 0 1 1 has any pattern. 56% agreed that there is indeed a pattern even though sequence was generated by just randomly flipping a coin.

The author cites the paper “Connecting the dots: how personal need for structure produces false consumer pattern perceptions” [I couldn’t find the pdf version, I just read the abstract] and states that

 One of the reasons why conspiracy theories spring up with such regularity is due to our desire to impose structure on the world and incredible ability to recognise patterns” and

“facts and rational arguments really aren’t very good at altering people’s beliefs”

The same article discusses several approaches and cites several studies on how to convey authentic information and make people change their mind: –

• Use stories: People engage with narratives much more strongly than with argumentative or descriptive dialogues.
• Don’t mention the myths while making your point since it has been seen that myths are better remembered than facts.
• While debunking the fake theories, offer explanations that resonate with people’s pre-existing beliefs. For example, conservative climate-change deniers are much more likely to shift their views if they are also presented with the pro-environment business opportunities.

Use of Bots

One of the things I noticed in the paper is the use of bots to spread the conspiracy theories and misinformation. It seems that during major events, the bot activity increases manifold. I found two studies analysing bot activity (not just limited to spread of conspiracy theories) “NEWS BOTS Automating news and information dissemination on Twitter”[1] and “Broker Bots: Analysing automated activity during High Impact Events on Twitter”[2].

More diverse events

The data in the study was limited to shooting events. The analysis could extend to other high impact events like natural disasters, elections, policy changes and find out the similarities if any in the sources spreading the misinformation and the misinformation itself.

Influence of multiple online communities

Certain subreddits (the_donald) and 4chan (/pol/) communities have been accused of generating and disseminating alternative narratives and conspiracy theories. What is the role of these communities in spreading the rumours? Who participates in these discussions? And how are users influenced by these communities?

Identify conspiracy theories and fake news

How do rumors originate? How do they propagate in an ecosystem? What is the lifespan of these rumors?

I feel an important step to identify conspiracy theories is to study how the language and structure of the articles coming from alternate sources differ from those of the mainstream ones. Not just the articles but also the carriers i.e the posts/tweets sharing them. How is the story in these articles weaved and supported? We saw an example in the paper where Sandy Hook shootings (2012) were referenced in Orlando shooting (2016) as an evidence to support the alternate narrative. What other sources are used to substantiate their claims? Authors in paper “How People Weave Online Information Into Pseudoknowledge” find out that people draw from a wealth of information sources to substantiate and enrich their false narrative, including mainstream media, scholarly work, popular fiction, and other false narratives.

Artcles/videos related to conspiracy theories among top search results

A simple search for “vaccination” in YouTube gives atleast 3-4 anti-vaccination videos in the top 30 results where people share theories/reasons about why vaccines are not safe (link1, link2). More conspiratorial content in the top results will expose more people to fake stories which might end up influencing them.

Read More

Video: Partisanship and the search for engaging news

In the video, Natalie Stroud discusses two studies. In the first she examines how different buttons affect people’s response to comments in a comment section. She introduces buttons with three different labels: like, respect and recommend. Button “like” I believe forces people to have extreme views especially when it comes to hard content like politics. You like it means you strongly agree with it. So it makes sense to have labels that help people in supporting an opposing viewpoint, basically have a tool to signal that although I don’t agree with the content, I think the argument presented is actually a strong one. These tools will ensure that a person doesn’t out rightly disregards a comment just by looking at the number of “likes”. And what I really like is that these buttons serve two purposes: business (it had significant number of clicks) and democracy (since people started interacting with information that was counter to their beliefs). The business aspect is very important and is the one that is hardly ever considered. Almost all online platforms are actually businesses. So until and unless the suggested research increases user engagement or benefits the platform in some way, why would a platform implement it in the first place?

I can think of another label for a button which might have similar affects like respect does. News platforms can promote “share” button if they don’t already. If a person sees an article of opposing view being shared multiple number of times, curiosity might make him click that article and check what’s so special about it.

So conclusion from this study is that a design component should not make people to have extreme views (agree/like or dislike/disagree). Rather it should make people want to listen to others especially the ones who are singing a different tune. There is one aspect of this research that we should further extend. While people can be more receptive to opposing comments when it comes to topics like “gay rights” or “favorite music genre and pop artists” are they equally receptive to political content or do they end up “respecting” the politically like-minded comments. Study should include more topics to quantify the effects of changing button labels on different topics. Another line of research could be to study the after effects of “respecting” an opposing viewpoint. After reading and reacting to a strong opposing comment/post, will the user click/google search more on opposing viewpoints? Or will he return to reading like-minded news.

In the second study, Natalie studies effects of punishment (flagging a comment) and incentives (recommendations and top news picks) on partisanship. Swearing/Profanity increases the chances of a comment to get rejected and flagged while decreases the chances of getting selected as NYT Pick. But on the other hand, partisanship and incivility also increases recommendations. Natalie suspects that this makes the news room moderators to treat partisan incivility differently. While a comment containing profane language and swear words might get out-rightly rejected, content containing partisan incivility might get accepted. Does that mean one should extensively start moderating uncivil comments? Won’t that make the comment section bland and highly uninteresting and will probably decrease user engagement. It would be interesting to see how user behavior varies with varying strictness in moderation.

Can constructive comments always promote user engagement? Will a platform where everyone is nice and right attract diverse opinions in the first place? I believe a little incivility can add the essential flavor to the discussions. Probably we can measure the extent of incivility in comments that would promote healthy discussions & debates and develop automated tools that can detect and predict when uncivil comments will deteriorate the quality of discussion and promote heated exchanges among readers. To detect uncivil comments that do not contain swear words is further challenging.

Read More

Paer 1: Social Translucence: An Approach to Designing Systems that Support Social Processes

Paper 2: The Chat Circles Series: Explorations in designing abstract graphical communication interfaces

In the first paper, authors introduce a design theory called “social translucence” and describe it’s three characteristics: visibility, awareness and accountability. They embody these principles in a system called “Babble”. In the second paper, authors design series of graphical chat programs, starting from a simple interface to ones with advanced features and study implications of these features on social communication.

Although it’s been two decades since the Ist paper got published, the principles are still considered the core elements of the design of online social systems.

Making User’s identity and activity Visible through social signals: Users expose their identities using display picture and profiles while activity is visible using the feed, posts shared, likes and comments. The question is how much information is too much? Not all features will increase user engagement. I remember facebook removed its “ticker” feature that used to summarize the activities of our friends on right side of the news feed.

Visibility is again associated with design constraints (word limit of a tweet, ephemerality of snapchat video) as well as privacy concerns. Algorithmic bias might also affect visibility of activities of certain friends.

In the study done in 1^st paper, authors considered small number of people where almost everyone is visible to everyone. That doesn’t hold true today. It’s impossible to have millions of people together in a single interface. Thus, today we have the concept of friend circle, followers, friend network etc.

So then comes the question, are people outside our network important? Do we want social cues from these people? Will I be as interested in seeing a post of “friend of a friend” as I will be of my immediate connection? And more importantly do we have a true estimate of our invisible audience? An interesting study was done in paper “Quantifying the Invisible Audience in Social Networks”. The authors found that “social media users consistently underestimate their audience size for their posts, guessing that their audience is just 27% of its true size”.

Does the above finding still holds true? Today although the online platforms are providing privacy settings to control every aspect of your data. But are users fully aware of these settings? Are they using these settings to their full potential? Can we do a study where people are shown the actual audience of their post, and study the after effects of such disclosure.

Authors describe the other two features in the best way. Awareness:  “what do I know?”. Accountability: “I know that you know that I know”. Visibility gives rise to awareness. Blue double ticks on my whatsapp indicate that my friend has read the message. Awareness also brings social rules into picture. We are aware that trolling someone is bad. If I troll and abuse, I will be held accountable for my actions: I can be banned, my post could be removed, my answer could be downvoted. So awareness leads to accountability.

The papers introduce two systems: babble and chat series. Both study the effect of several graphical features on communication. Today, communication is no longer limited to text, we have images, videos, GIFs, emojis. Technological and infrastructure barriers in using realist (video calls) and mimetic(3D) means of displaying information no longer exist. Some of the findings in Chat circle paper still hold today, some not. Like in chat circle 2, a background image is added to give chatspace a theme. Today, the chat spaces are not bounded by themes and almost everything is discussed everywhere. But display pictures can have themes, like a rainbow effect on your dp in Facebook shows that you support gay rights. Having friends in circles failed for google plus. Chatscapes feature to follow or avoid someone very much exists today. In chatscape one can modify other’s appearances by labelling them as “funny” etc. Similar to this, we have reactions (anger, laugh, like) but these are given to posts and not people.

 

Read More

Auditing Algorithms: Research Methods for Detecting Discrimination on Internet Platforms by Sandvig et al

Measuring Personalization of Web Search by Hannak et al

Summary:

In the first paper authors believe that every algorithm deserves scrutiny as it might be manipulated and discriminatory. They introduce the social scientific audit study used to test for discrimination in housing etc. and propose a similar idea of “algorithm audits” to find out algorithmic bias. They then outline five algorithmic audit designs namely code audit, non-invasive user audit, scraping audit, sock puppet audit and collaborative or crowdsourced audit and discuss the advantages and disadvantages of each. They find the ‘crowdsourced audit technique’ to be most promising.

In the second paper, authors study personalization in algorithms using the “crowdsourced audit” technique described in the first paper. They propose a methodology to measure personalized web results and apply it on 200 mechanical turks and observe that 11.7% of search results show differences due to personalization. Only two factors- being logged in to google account and geographic location leads to measurable personalization. Queries associated with ‘politics’ and ‘companies’ were associated with highest personalization.

Reflection:

Companies have always been altering algorithms to their benefit. Google takes advantage of its market share to promote its services like ‘google maps’. One will almost never find search results containing URLs to MapQuest, Here WeGo and yahoo news. Market monopoly can be a dangerous thing. It kills competition. Who knows if google starts charging for its free services in the future when we all get used to its products.

A case of gender discrimination was found in Linked where in response to search for a female contact name you’ll be prompted male versions of that name. Example: “A search for “Stephanie Williams brings up a prompt asking if the searcher meant to type “Stephen Williams” instead.”[1]. While google example shows an intentional bias which although is not harming the users directly but is killing the market competition, the linkedin incident seems to be an unintentional bias that cropped up in their algorithm since it depended on relative frequencies of words appearing in the queries. So probably ‘Stephan’ was searched more than ‘Stephanie’.  Citing their spokesperson “The search algorithm is guided by relative frequencies of words appearing in past queries and member profiles, it is not anything to do [with] gender.” So authors are right when they say that no algorithm should be considered unbiased.

Some companies are building Tools to detect bias in their AI algorithms like Facebook (Fairness Flow)[2], Microsoft [3] and Accenture[4]. But the problem is that just like their algorithms these tools will be a black box for us. And we will never know if these companies found bias in their algorithms.

Privacy vs personalization/convenience:  Shouldn’t users have the control over their data. Of what they want to share with the companies? Google was reading our mails for almost a decade for personalised advertisements before it stopped that in 2017 [5]. It still reads them though. It knows about our flight schedules, restaurant reservations. My phone number get distributed to so many retailers, I wonder who is selling them this data

In the second paper the authors mention that once the user logs in to one of the google services they are automatically logged-in to all. So does that mean my YouTube search affects my Google search?

According to an article [6] google autocomplete feature is leading to spread of misinformation. The first suggestion that comes up when you type “climate change is” comes out to be “climate change is a hoax”. How is misinformation and conspiracy theories ranking up on these platforms?

Determining bias seems like a very complex problem with online algorithms changing everyday. And there could be multiple dimensions to bias: gender, age, economic status, language, geographical location etc. The collaborative auditing seems to be a good way of collecting data provided it is done systematically and testers are chosen properly. But then again, how many turkers one should hire? Can a few 100 represent the billion population that is using the internet?

[1] https://www.seattletimes.com/business/microsoft/how-linkedins-search-engine-may-reflect-a-bias/

[2] https://qz.com/1268520/facebook-says-it-has-a-tool-to-detect-bias-in-its-artificial-intelligence/

[3] https://www.technologyreview.com/s/611138/microsoft-is-creating-an-oracle-for-catching-biased-ai-algorithms/

[4] https://techcrunch.com/2018/06/09/accenture-wants-to-beat-unfair-ai-with-a-professional-toolkit/

[5] https://variety.com/2017/digital/news/google-gmail-ads-emails-1202477321/

[6] https://www.theguardian.com/technology/2016/dec/16/google-autocomplete-rightwing-bias-algorithm-political-propaganda

 

Read More

Paper 1: Exposure to ideologically diverse news and opinion on Facebook

Summary:

In this paper the authors claim that our tendency to associate with like-minded people traps us into echo chambers. Basically the central premise is that “like attracts like”. The authors conduct a study on data set that includes 10.1 million active U.S. users who have reported their ideological affiliation and 7 million distinct URLs shared by them. They discover that the likelihood of individual clicks on a cross-cutting content relative to a consistent content is 17% for conservatives and 6% for liberals. After ranking the news, there is less amount of cross-cutting news since the ranking algorithm considers the way user interacts with friends as well as previous clicks.

Reflection:

Out of the 7 million URLs, only 7% were found to be hard content (politics, news etc.). This shows that facebook is meant more for sharing personal stuff. Since we don’t know the affiliation of all of the user’s friends it’s difficult to say if facebook friendships are based on shared political ideologies. Similar study should be conducted on platforms where people share more of the hard stuff….probably twitter….or google search history. The combined results will give better insights on whether people associate themselves with people having similar political ideologies on online platforms or not.

We can conduct a study to find out how adaptive and intelligent facebook’s news feed algorithm is by having a group of people who have declared their political ideology to start liking, clicking and sharing {both in support as well as disapproval} articles of opposing ideologies. We should then compare the before and after news feed to see if the ranking of the news articles change. Does the algorithm figure out whether the content was shared to show support or to denounce the news piece and modify the feed accordingly?

I wonder if users are actually interested in getting access to cross cutting content. A longitudinal study can be conducted where users are shown balanced news (half supporting their ideology and half opposing) and see if after a few months their click pattern changes: whether they click more cross cutting stuff or in the extreme case, do they change their political ideology. This kind of a study will show if people really care about getting trapped in an echo chamber or not. If not then we certainly can’t blame facebook’s algorithms.

This study is not generalizable. It was conducted on young population, specifically those who chose to reveal their political ideologies. Similar studies should be performed in different countries with users from different demographics. Also the paper doesn’t talk much about those who are neutral. How are political articles ranked for their news feed?

This kind of study will probably not hold for the soft content. People usually don’t hold extreme views about about soft content like music, sports etc.

Paper 2: “I always assumed that I wasn’t really that close to [her]”: Reasoning about invisible algorithms in the news feed &

Summary:

In this paper the authors want to study whether users should be made aware of the presence of news feed curation algorithm and how will this insight affect their future experience. They conduct a study where they made 40 FB users use FeedVis, a system that conveys the difference between the algorithmically altered and unadulterated news feed. More than half were not aware of the algorithm’s presence and got angry initially. They were upset that they couldn’t see close friends and family members in their feed and attributed this to their friend’s decision to either deactivate the account or to exclude them. Following up with the participants after a few months revealed that after the awareness about the presence of algorithm made them actively engage with facebook.

Reflection:

In paper “Experimental evidence of massive-scale emotional contagion through social networks”, authors did a scientific study on “emotion contagion”. The results of the study showed that displaying fewer positive updates in people’s feeds causes them to post fewer positive and more negative messages of their own. That’s how powerful Facebook’s algorithms can be!

In this paper authors try to answer two important questions: should users be made aware of the presence of algorithms in their daily digital lives and how will this insight affect their future experience with the online platform. We find out how ignorance about these algorithms can be dangerous. It can lead people to develop misconceptions about their personal relationships. How to educate users about the presence of these algorithms is still a challenge. Who will take up this role? Online platforms? Or do we need third party tools like FeedVis.

I found ‘Manipulating the Manipulation’ section extremely interesting. It’s amazing to see the ways adopted by people to manipulate the algorithm. The author’s could have included a section describing how far were these users successful in this manipulation. Which technique worked the best. Were changes in the news feed quite evident?

Two favourite lines from the paper “Because I know now that not everything I post everyone else will see, I feel less snubbed when I make posts that get minimal or no response. It feels less personal”

“whenever a software developer in Menlo Park adjusts a parameter, someone somewhere wrongly starts to believe themselves to be unloved “

It’s probably the best qualitative paper I’ve read so far.

Read More

An Army of Me: Sockpuppets in Online Discussion Communities

Summary:

In this paper the authors study sockpuppetry across nine different communities in Disqus, a commenting social media platform. They study how sockpuppets are different from the ordinary users in terms of the language they use, online posting activity and social network. They discover that sockpuppets write shorter sentences, use more singular pronouns like ‘I’, swear more, participate in controversial topics & existing discussions and are likely to interact with other sockpuppets. Keeping these findings in mind, the authors build predictive models to differentiate pair of sockpuppets from pair of ordinary users.

Reflection:

“Twitter Bots And Sockpuppets Used Trump’s Tweets To Mess With Virginia’s Governor’s Race” [1]. Facebook detected and removed 32 fake accounts and pages from both Facebook and Instagram after identifying coordinated inauthentic behaviour before midterm elections [2] Fake accounts have not only affected elections but has also taken lives in some cases [e.g. death of Megan Meier[3]].

Sockpuppetry has inflicted almost every social media platform. The intention behind creating sockpuppets might vary though. While primary purpose of creating fake accounts on online discussion platforms might be to sway public opinion by creating census, on Quora it might be to gain upvotes and On Facebook to create a false second identity[double life hypothesis]. I’ve come accross multiple instances where people create multiple profiles with incorrect age, gender, marital status and sexual orientation. I came across a website that sells Quora downvotes “https://www.upyourviews.com/buy-quora-downvotes/”. I wonder if the operating model of such companies involve creation of several bots or sockpuppets accounts!!! Also, I can’t think of any motivation behind existence of sockpuppet accounts on websites like 4chan where users are anonymous and content is ephemeral. 

I wonder if the online communities are willing to share user data along with IP traces like Disqus did. Like the authors mentioned, availability of ground truth data is also a big issue in this research. It is very difficult to build classifiers that give high accuracy without such data. Also the data used by authors will miss users that use different physical machines to access different accounts. I believe when sockpuppetry happens at a larger and professional scale, the miscreants will have the infrastructure to deploy multiple computers for multiple accounts. How to detect sockpuppets then?

Also detection of sockpuppets is one problem. How do we stop creation of such accounts? Tying accounts with some unique social identity like Aadhar or SSN? E.g Aadhar verification is mandatory on multiple matrimonial sites in india [4]

One of the author’s observation is that sockpuppets have a high page rank than ordinary users. They didnt justify or elaborate on this. Can this be contributed to their account’s high posting activity and larger network?

The authors say “Non-pretenders on the other hand have similar display names and this may implicitly signal to the community that they are controlled by the same individual, and thus may be less likely to be malicious.” First part of the statement will only be true if both sockpuppet accounts share the same network.

Authors divide the sockpuppets into three groups: supporters, non-supporters and dissenters with majority being non-supporters. While role of supporters and dissenters is clear since they are two extremes. I am not sure how non-supporters behave? A few examples from this category could have made it clearer.

The authors restricted their study to sockpuppet size 2 since majority of groups contained that no. of sockpuppets. Studying groups with >=3 sockpuppets might lead to interesting discoveries.

[1] https://www.huffingtonpost.com/entry/twitter-bots-sockpuppets-trump-virginia_us_5a01039de4b0368a4e869817

[2] https://www.cnbc.com/2018/07/31/facebook-has-detected-attempts-to-interfere-in-mid-terms.html

[3] https://en.wikipedia.org/wiki/Sockpuppet_(Internet)

[4] http://www.forbesindia.com/article/special/aadhaar-latest-tool-against-fake-matrimonial-profiles/50215/1

Read More

A Parsimonious Language Model of Social Media Credibility Across Disparate Events

Summary:

In this paper, the authors uncover that language used in tweets can indicate whether an event will be perceived as highly credible or less credible. After examining millions of tweets corresponding to thousands of twitter events they identify 15 linguistic features that can act as predictors of credibility and present a parsimonious model that maps linguistic constructs to different levels of credibility (“Low” < ”Medium” < ”High” < ”Perfect”). The events were annotated based on the proportion of annotations rating the reportage as ‘Certainly Accurate’. Authors use Penalised Logistic Regression for modeling and find that subjectivity is the most explanatory feature followed by positive & negative emotion. Overall, the results show that certain words and phrases are strong predictors of credibility.

Reflection:

As defined by Hiemstra et al in their paper “Parsimonious Language Models for Information Retrieval”, a parsimonious model optimizes its ability to predict language use but minimizes the total number of parameters needed to model the data.

All the papers we read for the last two reflections stressed the fact how linguistic constructs define the identity of a group/individual on online social communities. While the he use of racist, homophobic and sexist language was part of the identity of /b/ board in 4chan, users of a group in Usenet used “Geek Code” to proclaim their geek identity. We also learned how banned users on CNN used more negative emotion words and less conciliatory language.

I liked how authors validated their method of annotating the events with the HAC based clustering approach to group the events. They use Rand similarity coefficient to find the similarity between the two clustering techniques. The high R value indicates agreement between the two. I agree with author’s selection of annotation technique since it’s more generalizable.

Each mechanical turk needs to be aware of the event before annotating it. Otherwise they need to search for it online. How can can we ensure that the online news is not making the turker biased. Are turkers reading all the tweets in the pop up window before selecting a category or do they just base their decision by reading the first few tweets. I believe how an event is reported can greatly vary. So making a judgment by reading the first few tweets might not give a clear picture. Also, was the order of tweets in pop up window same for all the turkers? I believe I’ll find the answers to these questions after reading the Credbank paper.

The unit of analysis in this paper is an event rather than a tweet. And an event is perceived highly credible if large number of annotators rate the reportage as ‘certainly accurate’. But is the news perceived as credible actually credible? It will be interesting to see whether events perceived as credible are actually credible or not. A lot of work is going on in fake news detection and rumor propagation on social media platforms. Also, can people/organizations make use of this research to structure rumors in such a way that they are perceived credible? Will this reverse approach work too?

I believe a better way of calculating the value of “Questions” feature would be to calculate the proportion of tweets carrying question mark rather than counting the total number of question marks present in the tweets corresponding to an event.

One of the other features to determine credibility could be presence of URL in tweets. Specially URLs of trusted news agencies like CNN.

In the end I’ll reiterate the author and say that linguistic features combined with other domain specific features could act as foundation for an automated system to detect fake news.

Read More

Antisocial Behavior in Online Communities

Summary:

In this paper authors perform a quantitative, large-scale longitudinal study of antisocial behavior on three online discussion communities namely CNN, IGN and Breitbart by analyzing users who were banned from these platforms. They find that such users use obscene language that contains less positive words and is harder to understand. Their posts are limited to a few threads and are likely to amass more responses from other users. The longitudinal analysis reveals that the quality of their posts degrade with passage of time and community becomes less and less tolerant to their posts. The authors also discover that excessive censorship in the initial stages might aggravate the antisocial behavior in the future. They identify features that can be used to predict whether a user is likely to be banned or not namely content of the post, community’s response & reaction to the post, user activities ranging from posts per day to votes given to other users and actions of moderators. Finally, they build a classifier that can make the aforementioned prediction after observing just 5-10 posts with 80% AOC.

Reflections:

Antisocial behavior can manifest in several forms like spamming, bullying, trolling, flaming and harassment. Leslie Jones, actress starring in movie “Ghostbusters” became a target of online abuse. She started receiving misogynistic and racist comments on her twitter feed from several people including a polemicist Milo Yiannoloulos and his supporters. He was then permanently banned from twitter. Sarahah was removed from app stores after Katrina Collin’s started an online petition accusing the app of breeding haters after her 13 year old daughter received hateful messages, one even saying “i hope your daughter kills herself”. According to an article, 1.4 million people interacted with Russian spam accounts on twitter during the 2016 US elections. Detecting such content has become increasingly important. 

Authors say several techniques exist in the online social communities to discourage antisocial behavior ranging from down voting, reporting posts, mute feature, blocking a user, comments to manual human moderation. It would be interesting to find how these features fit in the design of the community. How are these features being used? Are all these “signals” true indicators of anti-social behavior? e.g. the authors suggest in the paper that downvoting is sometimes used to express disagreement rather than antisocial behavior which is quite true in case of quora and youtube. Both these websites have an option to downvote as well as to report the post. Will undesirable content always have larger number of downvotes? Do posts of users exhibiting antisocial behavior receive more downvotes, do their posts get muted by most of their online friends?

All of the author’s inferences make sense. The FBUs use more profanity and less positive words and get more replies which is expected since they use provocative arguments and attempt to bait users. We saw examples of the similar behavior in the last paper we read ”Identity and deception in the virtual community”. I also decided to visit the 4chan website to see if concept of moderators exist there. Surprisingly it does. But as answered in one of the FAQs one hardly gets to see moderated posts since there are no public records of deletion and since the content is deleted it gets removed from the page. I wonder if it’s possible to study the moderated content using the archives and if the archives keep temporal snapshots of the website’s content. Secondly, the website is famous for it’s hateful and pornographic content. How do you pick less hateful stuff from the hateful. I wondered if hate and sexual content are even considered criteria there. On checking their wiki I found the answer to “how to get banned in 4chan” {https://encyclopediadramatica.rs/4chan_bans=>quite an interesting read}. This makes one thing clear, criteria to moderate content is not universal. It depends a lot on the values and culture of the online community.

Having an automated approach to detect users will definitely lessen the burden from the shoulders of human moderators. But I wonder about the false positive cases. How will it affect the community if a series of harmless posts gets some users banned? Also, some users might redeem themselves later. In Fig 6 c) and corresponding explanation in “Characterizing users who were not banned”, we find that even though some FBUs were improving they still get banned. Punishing someone for improving will make sure that person will never improve in life. And the community might loose faith in the moderators. Considering these factors, is it wise to ban a user after observing initial few posts? Even exposing such users to moderators will make the later biased against the former. How long one should wait to form a judgment?

Overall I think it was a good paper, thorough in every aspect: from data collection, annotation, analysis to specifying the future work. I’ll end by mentioning a special app “ReThink[1]” that I saw in an episode of Sharktank (a show where millionaires invest their money in unique ideas). This app detects when a user writes an offensive message and gives him a chance to reconsider sending that message by showing an alert. Aimed for adolescents, the app’s page mentions that 93% of people do change their mind when alerted. Use of such apps by young people might make them responsible adults and might help in reducing the anti social behavior that we see online.

[1] http://www.rethinkwords.com/whatisrethink

Read More