Garrett, R. K. (2009). Echo chambers online?: Politically motivated selective exposure among Internet news users. Journal of Computer-Mediated Communication, 14(2), 265-285.

Summary:

The Internet offers users abundant access to information from around the world, which expose people to more diverse viewpoints, including both similar perspective and attitude-challenging information. With so much information, voice comes up that whether the Internet leads people to adopt increasingly insular political news exposure practices, i.e., “echo chambers” and “filter bubbles”, which refer to cases that individuals tend to be exposed only to information from like-minded individuals or materials similar to previous reading due to the influence of algorithms the news sites employed. This paper tests the viewpoint by conducting a web-administered behavior-tracking study. They find that the effect of opinion-challenging information is small and only marginally significant, which means worry that the Internet will lead to an increasingly fragmented society appears to have been overstated.

Reflection:

The paper shows readers how to conduct an exhausted and comprehensive experimental design when studying a specific problem, from selecting the research subjects to designing the experimental setting/workflow, e.g., using a purpose-built software administrates the study. Appropriate groups should be chosen to study according to the problem itself — two partisan online news services, left AlterNet and right WorldNet Daily, are able to bring immediate benefits to the study with the inherent engagement in selective exposure. Another thing I am concerned about is that there are not enough samples participating in the experiment, i.e., a total of 727 subjects (358 Alternet readers and 369 WorldNetDaily readers respectively), which possibly means the generalization of our observation result is not universal enough. Although the author mentions in the limitation that “when viewing these results is that the sample is not representative of the U.S. population at large”, I think it is very difficult to represent groups with hundreds of subjects, not to mention specific group like U.S. population at large. Why do not we find some volunteers on university campuses or research institutes, e.g., students and faculties, which is easier to get more volunteers?

Bakshy, E., Messing, S., & Adamic, L. A. (2015). Exposure to ideologically diverse news and opinion on Facebook. Science, 348(6239), 1130-1132.

Summary:

Based on the same problem background as the first paper, e.g., “echo chambers” and “filter bubbles”, the paper attempts to explore how this modern online Internet environment will influence people’s exposure to diverse content. Past empirical attempts turn out to be limited because of difficulties in measurement and show mixed results. To overcome previous limitations, they use a large, comprehensive Facebook dataset to measure the extent of ideological homophily and content heterogeneity in friend networks. They found that individual choice plays a stronger role than algorithmic ranking does in content exposure.

Reflection:

One heuristic point to take away is to classify stories as either “hard” (such as national news, politics, or world affairs) or “soft” content (such as sports, entertainment, or travel) by training a support vector machine, which reduces the data size while keeping the most efficient part of the dataset since “hard” content tends to be more controversial than soft content. However, some of the “soft” content is also practicable and appropriate for the task, i.e., hot and controversial topics in sports and entertainment fields. Thus the author may consider augmenting the size of the dataset to find out the result.

Same as the vague distinction between exposure and consumption mentioned in the limitation analysis, I also think how to clarify the line among conservative, neutral, liberal? Can we possibly give a uniform and appropriate criteria for the classification of those three categories?

Read More

Both the papers ask questions along similar lines, however, the question raised by the Kelly is more subtle and in a sense a follow up to the one posed by Bakshy et al.

1. Garrett, R. Kelly. “Echo chambers online?: Politically motivated selective exposure among Internet news users.” Journal of Computer-Mediated Communication2 (2009): 265-285.
2. Bakshy, Eytan, Solomon Messing, and Lada A. Adamic. “Exposure to ideologically diverse news and opinion on Facebook.” Science6239 (2015): 1130-1132.

The first paper asks quite an interesting question that deals with the consequences of technology  i.e. “Will selective new exposure or the ability to limit one’s exposure to articles or sources of a certain kind lead to a dystopian future where democracy will collapse?” The answer  to which was, unsurprisingly, no.

The results of the paper suggest that prospective opinion-reinforcing information greatly influences article selection and that immediately raises a follow up question, “how does fake news framed as opinion-reinforcing content come into play in shaping perception of truth among people?”. If people are more likely to consume articles that support their ideological point of view, then such articles can serve as good tools for derailing the truth. Since there is potential of the truth being manipulated, “does this lead people to very extreme, one-sided and tunneled vision i.e. can people be indirectly be programmed to adopt a certain point of view?if so, how devastating can the effects be?”

The second question has more to do with the design of the study, “does voluntarity (if that’s a word) exaggerate behaviors?” i.e. does passive vs active instrumentation of people lead to different results? will a person’s choice of articles be different if they knew that they were being monitored?

The third question has to do with the choice of the population. People chosen from well known left or right leaning sights are generally quite enthusiastic and/or as compared to the general public. Hence the results that were reported here might be an artifact of the choice of the population. So i guess the question i’m asking is “whether people who read news from partisan sources are democrats for the sake of being democrats or republicans for the sake of being republicans?”. Hence the generalization of the results to the general public is uncertain.

The second paper asks a different question of whether people get isolated from a opinion-challenging information even if they choose to. Given, the amount of data available to the authors i’m surprised the paper is as fleshed out as it could be, but the paper does a good job of quantitatively proving that the fear is unfounded. An interesting thing that i noted was that the neutral group tended to have a smaller proportion of friends who were neutral as compared to liberal of conservative friend, the difference is almost ~20%.

So is it that neutral people have these proportions because they are neutral in leaning, or is that them being neutral is an affect of having equal proportions of conservative or liberal friends?

Another aspect that i think could’ve been easily studied in the paper but was overlooked, is that given hard news that was opinion challenging, how likely are people to share it? This statistic would give deeper insight into how open are each of the three groups to change and/or how likely they are to peruse the truth of the matter and whether they value truth over ego.

Perhaps, this study could also look into different but specific topics e.g. climate change, and look at similar statistics to determine if  “micro echo chambers” (topic based echo chambers) are formed. This sounds certainly seems like an interesting direction to me!

Read More

1. Garrett, R. Kelly. “Echo chambers online?: Politically motivated selective exposure among Internet news users.” Journal of Computer-Mediated Communication2 (2009): 265-285.
2. Bakshy, Eytan, Solomon Messing, and Lada A. Adamic. “Exposure to ideologically diverse news and opinion on Facebook.” Science6239 (2015): 1130-1132.

Both papers talk about how the exposure to news and the civic information is increasing through online social networks and personalization. The emphasis is more on how this is leading to an era of “echo chambers” where people read news or information which favors their ideology and opinions.

Garrett et al. demonstrated that opinion-reinforcing information promotes news story exposure while opinion-challenging information makes exposure only marginally less likely. They conducted a controlled study where the participants were presented with news content and a questionnaire. However, I am not convinced by the fact that the participants were presented with the kind of news or information they already have strong opinions about. This could have led to a possible bias in the conclusion drawn from the study. Although the paper has presented some interesting findings of opinion-reinforcing and opinion-challenging content and how readers perceive information when presented with such content, I was unable to correlate the claims and findings specified by the authors. Also, the study revolves around three issues – gay marriage, social security reform, and civil liberties- which were current topics in 2004. Does this mean that the results presented won’t generalize to other topics? Of all the papers we have read so far, generalizing the results across genres and other geographic location looks like a major roadblock.

Bakshy et al., have used deidentified data to examine how 10.1 million Facebook users interact with socially shared news. Their focus is on identifying how heterogeneous friends could potentially expose individuals to cross-cutting content. Apart from “echo chambers” the authors also talk about “filter bubbles” in which the content is selected by algorithms according to viewer’s previous behaviors. I like the quantitative analysis presented by the authors to compare and quantify the extent to which individuals encounter comparatively more or less diverse content while interacting via Facebook’s algorithmically ranked Newsfeed. Apart from this, in my opinion “how likely is it that an individual will share a cross-cutting post with his friends” should also be considered and “what if an individual doesn’t click on the link containing a cross-cutting post?”

In the end, it makes me wonder how the results will be if authors of both papers would have conducted the study on individuals from outside of the US.

Read More

Summary

The central theme in both these papers talks about echo chambers on the Internet, and specifically, social media websites, such as Facebook. The fundamental premise that the authors build on is that – likes attract, opposites repel. Garrett mentions selective pressure as the main driver behind people’s news choices on the Internet. Algorithms in social media websites and recommender systems suggest similar content to users that drives them towards an extreme, i.e. opinions are reinforced and differing opinions are not read or recommended.

However, the papers differ in their methods. Garrett et al. conduct a small and specific user behavior-tracking study (N = 727), with users recruited from readers of two news websites – AlterNet (left-leaning) and WorldNetDaily (right-leaning). Since readers already visit these websites, ground truths about their political affiliation are assumed. Once these users sign up for the study,  Bakshy et al. perform similar analysis on a significantly larger scale of 10.1 million active users of Facebook who self-report their political affiliations. Their evaluation involves sophisticated data collection involving ~3.8 billion potential exposures, 903 million exposures and 59 million clicks. The paper by Bakshy et. al is very dense in content and I refered to [1, 2] for more explanation.

Both papers conclude by confirming our suspicions about content preference amongst users – they spend more time reading opinion-reinforcing articles over opinion-challenging opinions.

Reflections

Kelly Garrett’s paper, though published in 2009, uses data collected in 2005. This was a time before global social media websites like Facebook and Twitter were prevalent. At the time, the author chose the best means to generate ground truth by looking at left-leaning and right-leaning websites. However, this mechanism of classification feels naïve. It is possible that certain users merely stumbled upon the website or participated in the survey for the reward. Equally importantly, the sample is not truly reflective of the American population, as a vast majority may look for news from a more unbiased source.

One of the undercurrents of the “Echo chambers online?” paper is the effect of Internet in making these biases profound. However, the study does not speak or attempt to measure users’ preferences before the advent of Internet. Would the same citizenry buy newspapers that were partisan or is this behavior reflective only of news on the Internet?

Bakshy et al.’s paper is considerably more recent (2015). While it is evaluating many of the same questions as Garrett’s paper, the methodology and mechanism are fundamentally different, as is the time period. Therefore, comparing the two papers feels a little unfair. Facebook and Twitter are social platforms, and in that sense, very different from news websites. These are platforms where you do not fully choose the content you want to see. The content served to you is an amalgamation of that shared by your friends, and a ranking algorithm. However, a distinction must be made between a website like Facebook and one like Twitter. The authors themselves highlight an important point:

Facebook ties primarily reflect many different offline social contexts: school, family, social activities, and work, which have been found to be fertile ground for fostering cross-cutting social ties.

Therefore, it is substantially more possible to interact with an opinion-challenging article. However, interaction is sometimes poorly defined because there is no real way of knowing if a user merely looked at the article’s summary without clicking on it. Hence, tracking exposure can be tricky and an avenue for further research.

Questions

1. Almost 10 years later, especially after the wave of nationalism across the world, is there more polarization of opinion on the Internet?
2. Is polarization an Internet phenomenon or are we measuring it just because most content is now served digitally? Was this true back in 2005?
3. Can and should recommendation algorithms have individual settings to allow users to modify their feed and allow more diverse content?

References

[1] https://solomonmessing.wordpress.com/2015/05/24/exposure-to-ideologically-diverse-news-and-opinion-future-research/

[2] https://dataverse.harvard.edu/dataset.xhtml?persistentId=doi:10.7910/DVN/LDJ7MS

Read More

Both of these papers examine the trend towards “selective exposure” and away from diverse ideological exposure when viewing news electronically.  At a high level, both papers are touching on the same big idea – that users seem to be creating insular “echo chambers” of polarized news sources based on their ideals, ignoring the viewpoints of the opposing ideology either by their own conscious choice or algorithmically by their past behavior.  The Garrett paper looks at general web browsing for news sources and focuses on the area of opinion reinforcement.  The study details a web-administrated behavioral study in which participants were shown a list of articles (and their summaries) and were given the choice of which ones they wanted to view.  The study findings supported the author’s hypotheses, including that users prefer to view news articles that are more opinion-reinforcing and that users will spend more time viewing those opinion-reinforcing articles.  The Bakshy et al. study was Facebook-centered, examining how users interact with shared news articles on that platform.  Among their findings were that ideologically cross-cutting depended on both the spectrum of friend ideologies and how often those friends shared, but that there was some evidence of ideological isolation in both liberal and conservative groups.

Both of these studies had notable limitations that were discussed by the authors, but I felt that each was addressed insufficiently.  The Garrett study made use of both a liberal and a conservative online news outlet to obtain participants, which obviously will not ensure that the sample is representative of the population.  Garrett justifies this by supposing that if selective reinforcement is common in these groups, then it is likely the same among mainstream news readers; however, (1) no attempt is made to justify that statement (the brief mention in the Limitations section even contradicts this assertion), and (2) my intuition is that the opposite is true: that if selective reinforcement is common among centrists, then it almost certainly will be true at the ideological extremes.  In my opinion, the results from this study do not generalize, and this is a killer limitation of the paper.

Bakshy’s study has a similar limitation that the authors point out: that they are limited to recording engagement based on clicks to interact with articles.  As a result, individuals might spend some time reading the displayed summaries of some articles but never click to open the source, and such interactions are not logged.  To use the authors’ phrasing, “our distinction between exposure and consumption is imperfect.”  This surprised me – there was no way to record the amount of time that a summary was displayed in the browser, to measure the amount of time a viewer may have thought about that summary and decided whether or not to engage?  I know in my experience, my newsfeed is so full and my time is so limited that I purposefully limit the number of articles that I open, though I often pause to read summaries in making that decision.  I do occasionally read the summaries of ideologically-opposing articles, but I rarely if ever engage by clicking to read the full article.  Tracking exposures based on all forms of interaction would be an interesting follow-up study.

Despite the limitations, I thought that both studies were well-performed and well-reported with the data that the authors had gathered.  Garrett’s hypotheses were clearly stated, and the results were presented clearly to back up those hypotheses.  I wish the Bakshy paper had been longer so that more of their results could be presented and discussed, especially with such a large set of users and exposures under study.

Read More

1. Garrett, R. Kelly. “Echo chambers online?: Politically motivated selective exposure among Internet news users.” Journal of Computer-Mediated Communication2 (2009): 265-285.
2. Bakshy, Eytan, Solomon Messing, and Lada A. Adamic. “Exposure to ideologically diverse news and opinion on Facebook.” Science6239 (2015): 1130-1132.

Both the papers discuss about online “echo chambers” or communities/groups/forums on the Internet, that are devoid of differing viewpoints, i.e. places where individuals are exposed only to information from like-minded people. The second paper also talks about “filter bubbles” or the behavior of content-delivery services/algorithms to only deliver or recommend content to users based on their viewing history. Both of these issues are important as they can give rise to fragmented, opinionated and polarized citizenry. While Garrett’s paper mostly focused on the analysis of behavior-tracking data collected from the readers of 2 partisan online news websites, Bakshy et. al. analyzed de-identified, social news sharing data of 10.1 million Facebook users in the U.S.

The results presented in Garrett’s paper suggest that individuals are more likely to read news stories containing “high” opinion-reinforcing information as compared to “high” opinion-challenging information. Additionally, people generally tend to spend more time reading news stories containing “high” opinion-challenging information as compared to those containing “high” opinion- reinforcing information. While reading the paper I felt that it would be interesting to study, how reading opinion-reinforcing news affects the reader’s opinion/attitude versus reading news that conflicts with the reader’s attitude. While both the studies focused on political news which in my opinion can have a wide range of debatable topics, I feel it would be interesting to redo the study on groups/communities whose basis are fanatical, unscientific beliefs, like: anti-vaccination, religious extremism and flat Earth to name a few. We can also think of repeating this study in other geographies (instead of just the U.S.), and also compare the medium of news delivering. For example, people maybe are more likely to read a news story with opinion-challenging information if its presented to them in a physical newspaper vs online news website? This points to a deeper question of, is the Internet making us more opinionated, insular, trapped in our idiosyncratic beliefs and ideologies?

If I have understood it correctly, the participant’s in Garrett’s study complete a post-reading assessment after reading every news story. Given that the participant’s only have 15 minutes to read the stories, it is unclear if the time spent finishing the post-assessment questionnaire was included in these 15 minutes. If the post-assessment was indeed included in the 15 minute reading window, I feel it might bias the post assessment or the choice of the second news story selected. Moreover, it would have been useful to have some statistic about the length of news stories, say the mean and standard deviation of the word-counts. Other than this, I feel it would have been useful to know more about the distribution of age and income in the two subject populations (the author reports the average age and some information about the income). It may also be interesting to analyze the role played by gender, age and income on political opinion as a whole. Overall, I feel the paper presented a very interesting qualitative study for it’s time, a time when users had a lot more control over what they read.

The Science article by Bakshy et. al. presents the quantitative analysis really well and does a good job, explaining the process of media exposure in friendship networks on Facebook. An interesting research question can be to study, how likely are people to share a news story that conflicts with their affiliations/ideology/opinions as compared to one that aligns with their opinions. Another thought/concern is whether the presented results would hold across geographies.

Read More

Zhe Zhao, Paul Resnick, and Qiaozhu Mei. 2015. Enquiring Minds: Early Detection of Rumors in Social Media from Enquiry Posts. In Proceedings of the 24th International Conference on World Wide Web (WWW ’15). International World Wide Web Conferences Steering Committee, Republic and Canton of Geneva, Switzerland, 1395-1405. DOI: https://doi.org/10.1145/2736277.2741637

Summary

The authors aim to identify trending topics in social media, even topics that are not pre-defined. They use Twitter as their source of data. Specifically, they analyze 10,417 tweets related to five rumors. They present a technique to identify trending rumors. Moreover, they define them into topics that include disputed factual claims. Subsequently, they deem the identification of trending rumors as soon as possible important. While it is not easy to identify the factual claims on individual posts, the authors redefine the problem in order to deal with this. Therefore, they find cluster of posts whose topic is a disputed factual claim. Furthermore, when there is a rumor there are usually posts that raise questions by using some signature text phrases. The authors search for these keywords in order to identify them such as: “Is this true?”. As the authors find, many rumor diffusion processes have some posts that have such enquiry phrases quite early in the diffusion. Subsequently, the authors develop a rumor detection method that looks for the enquire phrases. It follows five steps: identification of signal tweets and signal clusters, detect statements, capture non-signal tweets, and rank candidate rumor clusters. Therefore, the method clusters similar posts together and finally collects the related posts that do not contain the enquire phrases. Next, they rank the clusters of posts by their likelihood of really containing a disputed factual claim. The detectors find that the method has a very good performance. About a third of the top 50 clusters were judged to be rumors, a high enough precision

Reflections

The broad success of online social media has created fertile soil for the emergence and fast spread of rumors.  A notable example is that one week after the Boston bombing, the official Twitter account of the Associated Press (AP) was hacked. The hacked account sent out a tweet about two explosions in the White House and the President being injured. Subsequently, the authors have an ambitious goal. They propose that instead of relying solely on human observers to identify trending rumors, it would be helpful to have an automated tool to identify potential rumors. I find the idea of identify rumors in real time, instead of retrieving all the tweets related to them, very novel and intelligent. To their credit, the authors acknowledge that identifying the truth value of an arbitrary statement is very difficult, probably as difficult as any natural language processing problems. They stress that their goal does not make any attempt to assess whether rumors are true or not, or classify or rank them based on the probability that they are true. They rank the clusters based on the probability that they contain a disputed claim, not that they contain a false claim.

I am particularly concerned regarding the adverse effect of automated rumor detection. In particular, its use in either damage control or disinformation campaigns. The authors write: “People who are exposed to a rumor, before deciding whether to believe it or not, will take a step of information enquiry to seek more information or to express skepticism without asserting specifically that it is false”. However, this statement is not self-evident. For instance, what if the flagging mechanism of a rumor, “disputed claim”, does not work for all cases? Government official statements would probably not be flagged as “rumors”. A classic example is the existence, or lack thereof, of WMD’s in Iraq. Most of the media corroborated with the government’s (dis)information. To put things into more technical terms, what if the twitter posts do not have any of the enquiry phrases (i.e. “Is this true?”)? The clusters would then not detect them as “signal tweets”. In that case, the automated algorithm would never find a “rumor” to begin with. The algorithm would do what it was programmed to do, but it would have failed to detect rumors.

Perhaps the greatest controversy is surrounded by how “rumor” is defined. According to the authors, “A rumor is a controversial and fact-checkable statement”. By “Fact-checkable”: In principle, the statement has a truth value that could be determined right now by an observer who had access to all relevant evidence. By “Controversial (or Disputed)”: At some point in the life cycle of the statement, some people express skepticism. I think the “controversial” part might be the weakest part of the definition. Would the statement “earth is round” be controversial because at “some point in the life cycle of the statement, some people express skepticism”? The authors try to recognize such tweets into a category they label as “signal tweets”.

Regardless, I particularly liked the rigorous definitions provided in the “Computational Problem” section that leaves no room for misinterpretation. There is room for research in the automated rumor detection area. Especially if it could broaden the “definition” of rumor and somehow embed it in the detection method.

Questions

1. What if the human annotators are biased in manually labeling rumors?
2. What is the logic regarding the length of the time interval? Is it ad hoc? How sensitive are the results to the choice of time interval?
3. Why was Jaccard similarity coefficient set to a 0.6 threshold? Is this the standard in this type of research?

Mitra, Tanushree, Graham P. Wright, and Eric Gilbert. “A parsimonious language model of social media credibility across disparate events.” Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing. ACM, 2017.

Summary

The main goal of this article is to examine whether the language captured in unfolding Twitter events provide information about the event’s credibility. The data is a corpus of public Twitter messages with 66M messages corresponding to 1,377 real-world events over a span of three months, October 2014 to February 2015. The athors identify 15 theoretically grounded linguistic dimensions and present a parsimonious model that maps language cues to perceived levels of credibility. The results demonstrate that certain linguistic categories and their associated phrases are strong predictors surrounding disparate social media events. The language used by millions of people on Twitter has considerable information about an event’s credibility.

Reflections

With the ever increasing doubt on the credibility of information found on social media, it is important for both citizens and platforms to identify such non-credible information. My intuition before even completing the paper was that the type language used in Twitter posts could be used an indicator to capture the credibility of an event. Furthermore, even though not all non-credible events can be captured just by language, we could still be able to capture a subset. Interestingly enough, the authors indeed verify this hypothesis. This is important in the sense that we can capture non-credible posts with a parsimonious model through a first “screening model”. Then, after discarding these posts we could proceed to more complex models to add additional “filters” that detect non-credible posts. One of the dangers that I find is to make sure not to eliminate credible posts, a false positive error, with “positive” being non-credible error. The second important contribution is that instead of retrospectively identifying whether the information is credible or not, they use CREDBANK in order to overcome dependent variable bias. The choice of Pca Likert scale renders the results interpretable. In order to make sure that the results make sense, they compare this index with hierarchical agglomerative clustering. After comparing the two methods, they find high agreement between our Pca based and HAC-based clustering approaches.

Questions

As the authors discuss, there is no broad consensus of the meaning of “credibility”. In this case credibility is the accuracy of the information.  In turn the accuracy of information is examined by instructed raters. The authors use an objective definition of credibility that is dependent on the instructed raters. Are there other ways to assess “credibility” based on “information quality”? Would that yield different results?

Garrett, R. Kelly, and Brian E. Weeks. “The promise and peril of real-time corrections to political misperceptions.”

Summary

This paper presents an experiment comparing the effects of real-time corrections to corrections that are presented after a short distractor task. Closer inspection reveals that this is only true among individuals predisposed to reject the false claim. The authors find that individuals whose attitudes are supported by the inaccurate information distrust the source more when corrections are presented in real time, yielding beliefs comparable to those never exposed to a correction.

Reflections

I find it interesting Providing factual information is a necessary, but not sufficient, condition for facilitating learning, especially around contentious issues and disputed facts. Furthermore, the authors claim that individual are affected by a variety of biases and that can lead them to reject carefully documented evidence, and correcting misinformation at its source can actually augment the effects of these biases. In Behavioral Economic there is a term that describes this biases. It is called “Bounded Rationality”. Furthermore, economic models used to assume that humans make rational choices. This “rationality” was formalized mathematically and then Economists create optimization problems that takes into account human behavior. However, new Economic models take into account the concept of bounded rationality into their Economic models through various ways. Perhaps it could be useful for the authors to draw some information from this literature.

Question?

1. Would embedding the concept of “Bounded Rationality” provide a theoretical framework for a possible extension of this study?

Read More

[1]. Garrett, R. Kelly, and Brian E. Weeks. “The promise and peril of real-time corrections to political misperceptions.”

[2]. Mitra, Tanushree, Graham P. Wright, and Eric Gilbert. “A parsimonious language model of social media credibility across disparate events.”

These papers are very relevant in this digital age where everyone has a voice and as a result there is plethora of misinformation around the web. In [1], the authors compare the effects of real-time corrections to corrections that are presented after a distraction. To study the implications of correcting the incorrect information, they conducted a between-participants experiment on electronic health records (EHRs) to examine how effective is real-time corrections to corrections that are presented later. Their experiment consisted of demographically diverse sample of 574 participants. In [2], Mitra et. al. present a study to assess the credibility of social media events. In this paper, they present a model that captures language used in Twitter messages of 1,377 real-world events (66M messages) using CREDBANK corpus. The CREDBANK corpus used Mechanical Turks to obtain credibility annotations, after that the authors trained penalized logistic regression using 15 linguistic and other control features present to predict the credibility level (Low, Medium or High) of the event streams.

Garrett et. al. claim that real-time correction even though is more effective than delayed correction but it can have implications especially with people who are predisposed to a certain ideology. First of all, the sample that they had was US-based, which makes me question that will these results hold in other societies? Is sample diverse enough to generalize it? Can we even generalize it for US only? The sample has 86% white people whereas, US has over 14% non-resident immigrants only.

The experiment also does not explain what factors contribute towards people sticking to their preconceived notions? Is it education or age? Are educated people more open to corrections? Are older people less likely to change their opinions?

Also, one experiment on EHRs is inconclusive. Can one topic generalize the results? Can we repeat these experiments with more controversial topics using Mechanical Turks?  

Finally, throughout the paper I felt that delayed correction was not thoroughly discussed. I felt that paper focused so much on psychological aspects of preconceived notions that they neglected (or forgot) to discuss delayed correction. How much delay is suitable? How and when should delayed correction be shown? What if reader closes the article right after reading it? These are the few key questions that should have been answered regarding delayed corrections.

In second paper, Mitra et. al. presented a study to assess the credibility of social media events. They use penalized logistic regression, which in my opinion was a correct choice because linguistic features would add multi co-linearity and penalizing features seems to be the correct approach. But since they use CREDBANK corpus, which used Mechanical Turks, it raises the same questions we discuss in every lecture that did Turkers thoroughly went through every tweet? Can we neglect Turkers bias? Secondly, can we generalize that P_ca based credibility classification technique will always better than data-driven classification approaches?

The creation of features though raises few questions. The authors make a lot of assumption in linguistic features for example, they hypothesize that coherent narrative can be associated with higher level of credibility which even though does make sense but can we hypothesize something and not prove it later? Which makes me questions on feature space that were they right features? Finally, can we extend this study to other social media? Will a corpus generated through twitter events work for other social medias?

Read More

Reflection 4:

1. Garrett, R. Kelly, and Brian E. Weeks. “The promise and peril of real-time corrections to political misperceptions.” Proceedings of the 2013 conference on Computer supported cooperative work. ACM, 2013.
2. Mitra, Tanushree, Graham P. Wright, and Eric Gilbert. “A parsimonious language model of social media credibility across disparate events.” Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing. ACM, 2017.

Summary:

Both papers talk about social media Credibility of the content posted on social media websites like Twitter. Mitra et. al., have presented a parsimonious model that maps language cues to the perceived level of credibility and their results show that certain linguistic categories and their associated phrases are strong predictors surrounding disparate social media events. Their dataset contains 1,377 real-world events. Whereas, Garrett et.al., have presented a study that focuses on comparing the effects of real-time corrections to corrections that are presented after a short distractor task.

Reflection:

Both the papers present interesting findings of information credibility across world wide web can be interpreted.

In the first paper Garrett et. al., have shown how political facts and information can be misstated. According to them, real-time corrections are better than making corrections after a delay. I feel that this is true to a certain level since a user hardly revisits an already read post. If the corrections are made real-time it helps to understand that a mistake has been corrected and hence credible information has now been posted. However, I feel that the experiment about what users perceive – 1. When provided with an inaccurate statement and no correction, 2. When provided a correction after a delay and 3. When provided with messages in which disputed information is highlighted and accompanied by correction. – can be biased based on user’s interest in the content.

An interesting part of this paper was the listing of various tools (Truthy, Videolyzer, etc.,) which can be used to either identify and highlight inaccurate phases.

The second paper Mitra et. al., have tried to map language cues with perceived levels of credibility. They have targeted a problem which is now quite prevalent. Since world wide web is open to everyone, people have the freedom to post any content without caring about the credibility of the information being posted. For example, there are times when I have come across same information (with exact same words) being posted by multiple users. This makes me wonder about the authenticity of the content and raises a doubt about the content credibility. I really liked the approach adopted by the authors to identify expressions which leads to the low or high credibility of the content. However, the authors have focussed on the perceived credibility in this paper. Can “perceived” credibility be considered same as the “actual” credibility of the information? How can the bias be eliminated, if there is any? I feel these are more psychology and theory-based questions and extremely difficult to quantify.

In conclusion, I found both papers very intriguing. I felt that these papers present a perfect amalgamation of human psychology and problems at hand and how they can be addressed using statistical models.

Read More