1. Bond, Robert M., et al. “A 61-million-person experiment in social influence and political mobilization.” Nature 489.7415 (2012): 295.
2. Kramer, Adam DI, Jamie E. Guillory, and Jeffrey T. Hancock. “Experimental evidence of massive-scale emotional contagion through social networks.” Proceedings of the National Academy of Sciences (2014): 201320040.

 

Summary:

The two papers talk about how people are influenced and how this influence spreads through the online social networks that people build. The experiments were focused on Facebook and the network of friends that people build. The first paper tracks the influence of a banner that promoted voting in the election, with and without showing who else stated that they voted. The second paper tracks how the user’s emotions are influenced based on the sentiments represented in the news feed.

 

Reflection:

The whole contagion effect on Facebook influencing people to vote parallels real life unconscious social pressures. For example, I noticed the social pressure affecting me when I felt compelled to buy clothes emblazoned with the VT logo because that is what I see everyone wearing around me. The compulsion to be conforming is extremely strong in humans. So it is not surprising that one is more likely to vote when they are explicitly made aware that people around them, people they know, have voted.

The two experiments make me wonder how much further people can be manipulated in their thoughts and actions, via social media. If you can also manipulate buying habits, advertisers now have a fantastic new weapon. If you can manipulate human thought and initiative, rogue governments now have a fantastic new weapon. This isn’t something that may happen in the far off future either. Oh no! This is something that is happening now. The most extreme example is the massacre and desolation of the Rohingya people in Myanmar. The hatred for these people is being perpetuated through social media, with support from the government (though they would deny such thing). You can spark good effects like getting people to vote through contagion, but remember that this also exists.

We cannot escape talking about the ethical issues surrounding the emotional contagion experiment, especially considering the outrage that it sparked when people were made aware that it happened. I wondered why the emotional contagion paper struck me as something I’d heard about before. Then I realized that I’d read about it in the news and the huge controversy that it caused. One has to ask, is it really informed consent when people are conditioned over the years to simply check the ‘I agree’ box on the terms of service without reading bothering to read it (because who wants to read several pages of impenetrable legal language)? I have to question the researchers’ reasoning that people consented to the experiment. It is a well known fact that in 99.9% of the cases, people don’t read the terms of service. Even if the researchers were unaware of this fact, the problem remains that informed consent did not exist (because few users read the terms of service), so this experiment could be classified as unethical and wrong. The experiment’s results rested on the users not being aware that the experiment is taking place. But it does seem like the user’s choice was taken away from them, which is most certainly not a good standard to set, especially by someone as big and influential as Facebook. Then again, Facebook isn’t exactly a paragon of ethics so I don’t think they care either way.

 

Read More

• Danescu-Niculescu-Mizil, C., Sudhof, M., Jurafsky, D., Leskovec, J., & Potts, C. (2013). A computational approach to politeness with application to social factors.

 

In this research a politeness classifier is constructed that performs with a good accuracy (near-human performance). Authors then use this classifier to explore the relation between politeness and social factors in three different categories namely: “relation to social outcome” (they show that later successful candidates are more polite than failed and non-admins), “politeness and power” (they depicts that requests posted by the question-asker are more polite than the other requests) and “prediction-based interactions” (some results like female Wikipedians are more polite).

Although building a politeness classifier seems very simple and it is similar to other text classifications (such as spam-ham classification), as it is mentioned in this paper, it can be useful for exploring politeness levels across several factors which can be an interesting research in social networks. Some applications of this classifier such as analyzing the relation between politeness and social status are mentioned in this paper, however, I am interested in using this classifier to predict the social status.

As in subsection 5.1 it is concluded, polite Wikipedia editors are more likely to reach high status through elections. I think some other features such as the number of edits that a candidate has done so far or even the candidate’s date of registration in the platform can also play significant roles in success of a candidate. But how is the significance level of the politeness factor on the success of the later successful candidates in the election? I am also curious to know if we can model Wikipedia admin elections (or other elections like this) with several features including politeness levels of the candidates. Can we use the classifier that introduced in this research to make a politeness score for each candidate and then employ it as a feature for prediction of Wikipedia administrator elections?

Also, in this research Amazon Mechanical Turk is used to label a portion of the data which come from the Wikipedia community of editors and the Stack Exchange question-answer community. They only employ annotators who are the U.S. residents (and also some other filtering). However, is this a good approach for labeling the request data? What about other countries’ residents? There are differences in politeness judgments between the two groups of users from two different cultures even with a same languages[1]. How can we use this labeling which is only based on the U.S residents judgment, in Wikipedia admin elections where many editors/users are active and using English language in there but are from other countries with different perceptions of politeness? This also makes me curious about another question. This kind of research are studying the politeness of users but what if we study the perceptions of politeness in social media?

I remember when I came to the U.S. had some difficulties in sending Email to professors or even other students. Although before coming to the U.S I had sent/received many Emails to/from professors, I realized that the politeness in this country is completely different only after I arrived here. In my country, we call each other by last name + Mrs/Ms/Miss/Mr in Emails. Using a lot of gratitude is also very common in our Emails. This way of writing Email sometimes even looks weird in the U.S., but for the people in my country the type of writing that is prevalent here seems impolite or even rude. In total, it seems a behavior study about the perceptions of politeness in social media can be useful. The results of this kind of research can prevent some misunderstandings and their consequences (such as some conflicts happen every day in social networks platforms).

 

[1] Yu-Cheng Lee, ‘Cultural Expectations and Perceptions of Politeness: The “Rude Chinese”?, DOI:10.5539/ass.v7n10p11

Read More

• Danescu-Niculescu-Mizil, C., Sudhof, M., Jurafsky, D., Leskovec, J., & Potts, C. (2013). A computational approach to politeness with application to social factors.

Human speech is a interesting in a sense that, not only is it used to convey information from one person to another but it contains other information that helps us understand dynamics between people and so forth. Such encodings are implicitly imbibed through social behaviors that one is surrounded by.  [1]

In this paper, the author understands one of these encodings, politiness through computational models. In order to build said computational model the author first makes a dataset from two sources wikipedia (feature development) and stackoverflow (feature transfer). The annotations are done by Amozon Mechanical Turkers. A classifier is then built which uses domain independent lexical and syntactic features extracted from the above above dataset to achieve near human levels of accuracy on classifying a text as polite or impolite. This model is then used to identify/reaffirm the following:

• Politeness decreases as one moves the social power dynamic food chain
• Politeness variation by gender and community

One of the interesting points that I thought one could work on after reading the paper, is to see how a language could affect the measure of politeness? One could see how different cultures which therein affecting the language could have an impact. This could depend upon the structure of the society and so forth. [2] [3]

One could also see how if the model described above used in domains where the vocabulary is different let’s say a website where the internet slang is rampant like 4chan would fail to work.

The above two instances can be looked as how politeness understood from one domain could not work in another where the underlying factors that determine politiness is different.

Given the dataset and its content, I belive what the author has built a computational model for is likability rather than it’s superset, politeness which is more encomposing than the former.

The author has used SVM to learn from features extracted separately from sociolinguistic theories of politeness. One could see how a newer deep learning based method could be used to extract these features to jointly learn the phenomenon of politeness [4].

 

[1] https://hbr.org/1995/09/the-power-of-talk-who-gets-heard-and-why

[2] Haugh, Michael. “Revisiting the conceptualisation of politeness in English and Japanese.” Multilingua 23.1/2 (2004): 85-110.

[3] Nelson, Emiko Tajikara. “The expression of politeness in Japan: intercultural implications for Americans.” (1987).

[4] Aubakirova, Malika, and Mohit Bansal. “Interpreting neural networks to improve politeness comprehension.” arXiv preprint arXiv:1610.02683 (2016).

Read More

Cristian Danescu-Niculescu-Mizil, Moritz Sudhof, Dan Jurafsky,
Jure Leskovec, and Christopher Potts, “A computational approach to politeness with application to social factors”

Summary:

The authors in this papers introduced a computational framework for detecting and describing politeness. The built a politeness corpus from annotating request (which embed politeness) from two platforms (Wikipedia and Stack exchange). Then they trained a couple of classifiers based on that corpus (The Wikipedia requests) and tested them using the Stack Exchange annotated request. The classifiers achieved near-human accuracy. Finally, they classify the rest of the requests using those models and discuss the findings from the classified data and the social theory between social outcomes, power and politeness.

Reflection:

• I appreciate all the statistical analysis and validation the authors did in this work. I think this paper gives a lot of statistical guidance to those who need to perform similar linguistic analysis of similar problems
• I wish the real world follow this theories were being polite is appreciated and awarded
• I also suggest if the authors would study the correlation in Stack Exchange between the politeness of the questions being asked and the number of answers/responds they receive. I believe we would find a strong positive correlation between those two
• Although the authors did hard work gathering and annotating and analyzing this data/requests but I think there is a big shortcoming in their work which is the number of annotated request. The number of annotated requests is about 11,000 request out of the whole gathered 409,000 requests which is about only 2.7%. They used 2.7% of the data to train and test their models then used those models to classify the rest (about 97.3%) which has been used in their analysis. Despite that they mentioned in the Human Validation paragraph that they turned to human annotation to validate their methodology but they did not mention how many requests has been validated. I am skeptical about the amount of annotated data and I think they should increase the annotated requests set into a reasonable percentage and then redo their analysis
• I do not find table 8 useful in this paper as I can not find any association between the programming languages and the politeness. I wish the authors gave more explanation about that table
• I admire how the authors employed the politeness theory in order to explain the findings of their analysis. I believe readings and courses in Sociology and Psychology are crucial for the Social Computing course otherwise it would be just data analytics without any social insights

Read More

Paper-

A computational approach to politeness with application to social factors – Danescu-Niculescu-Mizil et. al.

Summary-

The authors created an annotated politeness corpus of requests on Wikipedia and Stack Exchange using Amazon Mechanical Turk, which they used to build a classifier which uses domain independent lexical and syntactic features to achieve near human levels of accuracy on classifying a text as polite or impolite. They confirmed some theoretical ideas about politeness, and found that users who were more polite were win adminship elections on these websites, i.e. an editor/admin rank, and that their politeness levels actually decreased after reaching these levels, and the opposite happened for users who also ran for adminship elections and lost. They also analyzed links between status, gender, interaction type, community type, etc and politeness levels.

Reflection-

The authors designed an experiment with both theoretical and practical importance. The fact that the SVM trained on features derived from socio-linguistic/psychological theories of politeness was better than the bag-of-words model, gives more credibility to these theories. Moreover, the model which they trained also has practical importance since it has applications in auto-moderation of online forums, conversational systems, translation systems, etc.

This paper was an interesting read because it taught me a lot about experimental design for computational social science.What struck me about the approach was how cautious and thorough it was. I learned of a number of interesting statistical corrections which the authors applied to get more accurate results, and to ensure the annotation was being done properly, which may be applicable in my project or in future research. For example, they applied z-score normalization to each Turkers score to correct for personal biases in rating a request’s politeness. They also compared pairwise score correlations for Turkers who reviewed the same request, to ensure that the labelling is not happening randomly. Moreover, the authors even trained a “control” SVM based on bag-of-words unigrams to show that the results were due to the power of the features, and not simply the power of the classifier used. These were all great examples of “science done right”, to help budding researchers like myself understand how to obtain and interpret results.

The paper found that question askers were more significantly polite than answerers. Interestingly, high power users were less polite than low power users even when they themselves were asking questions. However, this difference was not extremely pronounced, but it hints at interesting power dynamics/psychological phenomena in the behavior of high power users, which might be worth computationally exploring.

The authors trained their SVM model on Wikipedia, and used it to classify politeness of requests on Stack Exchange. However, this is not a strong enough indicator of the generalizability of this model, since both Wikipedia and Stack Exchange are broadly similar types of sites, (i.e. both deal with acquisition of information). To further prove the robustness of the model, it should be applied to more diverse and unrelated online communities, like 4chan, gaming forums, reddit, etc. 

The model could also be applied to short exchanges beyond just requests, like comment threads, to further see how generalizable the results are.

The authors also mention that the annotated corpus was only made with requests which had two sentences, with the second sentence being the actual request. The use of this corpus is limited, because of its specificity. A more general politeness corpus could be created taking into consideration exchanges of various lengths.

The model used could also be increased in power, i.e. using a deep neural network instead of an SVM for classification, and results could be observed using the new model.

 

Read More

Paper 1: Danescu-Niculescu-Mizil, Cristian, et al. “A computational approach to politeness with application to social factors.” arXiv preprint arXiv:1306.6078 (2013).

This papers analyzes Wikipedia and StackExchange requests data for politeness markers based on politeness theory and builds a predictive model to judge politeness index of a conversation. There are several strong points about this paper such as data filtering, balanced analysis, and feature justification. I specifically liked the part where strong and weak politeness annotations based on percentile agreements.  I also like the application of politeness to power and status.

Even though the paper claims to be domain independent, what is considered to be polite can change across communities. Very obvious and extreme examples of this can be 4chan and Wikipedia. StackExchange and Wikipedia are both information-driven communities in which conversations are situated on a much different level than other social networks such are Twitter and Facebook. Further, politeness can also change based on the context of the social groups it is analyzed in. For example, what is considered as polite in a group of strangers might differ from politeness norms in a group of friends or colleagues.

Few questions come up after reading the annotation procedure of the paper.

1. Is it realistic to get such perfect 50-50 distribution of politeness scores in both datasets? How both datasets have almost an equal number of polite and impolite requests?  (It might be a coincidence, just looks a bit weird!)
2. The median and variance can depend on how many sample points are chosen. It’s not clear how many randomly scored annotations were used for comparison in Wilcoxon signed rank test.
3. Further, the median of both randomized scores is the same. (Again, might be the coincidence but looks weird! )

Further, the author’s findings that failed candidates get more polite after elections are inconsistent with several works analyzing the effect of the community feedback on user behavior. Considering failure in the election as a negative feedback, users should be expected to get more hostile/deviant. This raises a question: Does the correlation of politeness with power change over incentive driven and punishment driven communities?

Using conversation analysis principles in politeness measure: Even though the turker study was conducted with the context of request and answer pair, the linguistic measures were performed only on the request texts. Politeness is conversation driven. It can change over a series of message exchanges based on how the conversation flows. Principles of conversation analysis can be applied here to get a deeper understanding of politeness in conversation rather than just a part of a text. Specifically, talk-in-turns, attempts to repair problems and action formation aspects of conversation can be important in identifying politeness.

Lastly, the work done in this paper can be used to construct the model for assessing the general concept of “status” on social media. This work proves that politeness and status are correlated. Other aspects of status such as respect, authority, credibility, decisiveness can be assimilated to form a social status model. Such a model, as described in earlier class sessions can nudge people to be more source aware, civil, polite and respectful based on the status index in that community.

 

Read More

PAPER:

Danescu-Niculescu-Mizil, C., Sudhof, M., Jurafsky, D., Leskovec, J., & Potts, C. (2013). A computational approach to politeness with application to social factors.

 

SUMMARY:

In this paper, the authors put forth a computational framework for identification of linguistic cues of politeness on conversational platforms such as Wikipedia and Stack Exchange. They build a classifier that could annotate new request texts with close to human level accuracy. They put forth their findings on the relationship between politeness and (perceived) social status.

 

REFLECTION:

• Firstly, as the authors mention, the amount of politeness currency spent or ‘invested’ decreases the higher is a user’s perceived status. Now the question that can be asked is, why not do something, to ensure the same level of politeness exhibited by a user even after they rise in power and status.
  • These online conversational systems do have markers and provisions for reputations and power and social status, but they fail to implement the same to emulate these concepts as in the real world.
  • The stakes and motivation for a user to invest in politeness must be in proportion to his/her rank/power/reputation.
  • There could be an alternate thought here, that this should not make it permissible for newbies to be less polite. And the point about raising the stakes does not insinuate that, this is because, as in the real world, an asker/imposer requesting for certain knowledge or services will still (have to) be polite considering the social norms and their need of the said information or services.
• Very often, it also happens that people with high power or knowledge or statuses, are rude to new and naive users using sarcasm thus avoiding having to utter rude and/or ‘bad’ things. Thus, if sarcasm detection can be coupled with politeness classifiers, it would be a very robust and deployable system.

This paper by Joshi et al [1] gives a very innovative and interesting way for sarcasm detection, which provides a novel approach that employs sequence labelling to perform sarcasm detection in dialogue.

• Another potential approach that comes to mind for further improving and/or applying the identification of linguistic cues to identify politeness is in the domain speech recognition (via audio/video)
  • One can make use of the intonations, the pauses, volumes and tones in speech combined with the content transcription of what is being said to train and better understand politeness in speech.
  • Now, if the previously addressed problem, of sarcasm in conversational text, is addressed and handled, then one can definitely detect not only politeness but also sarcasm in speech.
• Lastly, Wikipedia and StackExchange are more or less similar platforms, but such an analysis can and should be carried out on platforms like Reddit and 4chan where each subreddit and thread has its different norms and tolerances for rudeness and politeness. Thus more insights and levels of tolerances can be understood with such an analysis on different platforms.
  • Furthermore, it might be wise to perform an ethnographic analysis on the results of rudeness classifier to asses the norms and rudeness followed by different demographics since it is not the same for the entire population/userbase

[1] : Joshi, Aditya, Vaibhav Tripathi, Pushpak Bhattacharyya and Mark James Carman. “Harnessing Sequence Labeling for Sarcasm Detection in Dialogue from TV Series ‘Friends’.” CoNLL (2016).

Read More

[On a tangent to the paper] In the recently concluded Annual HFES Meeting, Matthew Gombolay commented on “context”, that we human’s cannot explain the context behind the actions we take in a uniform manner. Politeness is one such contextual problems according to me. Why do we thank the bus driver who opens the door for us when we get off? Is it not his or her job to ferry us? Do we do this because of an inherent politeness we have as humans? Or do we do this because we saw other people do this and wanted to be a part of the group? A simple thing as thanking another individual can be contextualized in many ways. I feel that this very problem that we have not reached a consensus on contextualizing our actions is why we have not been able to teach a machine agent “context”.

Back to the topic of politeness that the paper deals with. I would like to reflect on the content in two halves, initially talking about the notion of politeness and referring to the paper where necessary and then moving to critique the procedure adopted in the research study.

I would point to two central questions that I wish to address.

What is politeness?

Is it as Brown and Levinson describe the emotional investment that we make to save our ‘face’? Is it an acknowledgment of power disparity with the person we converse with? Is it a complex mix of many things? I feel for us to converse about politeness we need to agree on the notion first. For the sake of this article, I will give the definition to which I will try and stick to.

I will define politeness as the behavioral characteristics an individual portrays, often by linguistic choice, that is civil relative to the people observing this individual.

By this definition, I give a relative measure for the word because it changes with the observers. A group might think to abstain from using cuss words as polite while another might consider the use of words like “Please”, “Thank you”, etc. in one’s conversations as a measure of politeness.

Do humans assume or adopt politeness to reach a higher power level?

From my understanding of the implications of the paper, humans tend to be polite as a means to an end. They are polite essentially to cause a disparity. This raises my question of “Are we subconsciously aware that we will be in a higher power state that the opposite party when we are polite to them?”.

My answer to the above question is that humans are polite to cause a disparity in power, but I am open to having my opinion changed.

Changing directions to critique the paper. I have the following critique:

• The paper deals with requests only. Is this enough to comment on “politeness” as a whole? Is it already sampling in a biased manner? I feel that the work did not defend its choice of requests in a sufficient manner. At least for me. I would have liked to have a mention of why they think requests generalize to other contexts more concretely.
• The test for inter-annotator agreement is interesting. The pairwise correlation test is definitely corroboratory to some of the claims that the authors have made. I failed to understand one aspect of this. When the classifier was compared to human performance by collecting more data. Was the inter-annotator agreement used as a measure of human-performance? If so is it not wrong? Humans are inherently different. A machine is always conformant to a given behavior. Is it not trivial to say based on disagreement that machine performs close to or better than humans?
• I am curious about the linguistic background questionnaire that the authors used and would definitely try to learn more about the same.
• The binary perception section mentioned the ends of the spectrum having more hits than the middle region. This reminded me of signal strength in signal detection theory. This is a Human Information Processing take where they comment that humans detect strong or weak signals very easily. But when it comes to signals that are hard to tell apart from each other, humans are bad at judging whether there is a signal or there isn’t. Only by designing the signal to have redundant dimensions can designers ensure that the right judgment of signal or no signal is made.
• I question the choice of not using any of the second domain for training. Would it not have made the more model-agnostic by using some of the data from the second domain as well?
• The paper talks about analyzing the requests made in these domains. I did not see a mention of the analysis of the responses that these requests got. I feel that an analysis of whether these requests were fulfilled would give valuable insight from the two parties involved rather than the retrospective nature of having a mechanical turker annotate it. An analysis of this sort would present a good baseline for comparison.

REFERENCES:

C. Danescu-niculescu-mizil, M. Sudhof, D. Jurafsky, J. Leskovec, and C. Potts, “A computational approach to politeness with application to social factors,” Proc. 51st Annu. Meet. Assoc. Comput. Linguist., pp. 250–259, 2013.

Penelope Brown and Stephen C. Levinson. 1978. Universals in language use: Politeness phenomena. In Esther N. Goody, editor, Questions and Politeness: Strategies in Social Interaction, pages 56–311, Cambridge. Cambridge University Press.

Read More

In social communication, there are multiple values that people tend to respect in order to gain different types of benefits. Being polite is one of the most important among these values. In modern online communities, politeness plays a great role for the community to ensure healthy interactions and for individuals to maximize their benefits, either being a request for help, conveying an opinion to an audience, or any other types of online social interactions.

In their work “A computational approach to politeness with application to social factors”, Danescu-Niculescu-Mizil et al. presented a valuable approach to computationally analyzing politeness in online communities based on linguistic traits. Their work consisted of labeling a large dataset of requests on Wikipedia and StackOverflow using human annotators, extracting linguistic features, and building a machine learning model that automatically classifies requests as polite or not polite with a close-to-human classification performance. They then use their model to analyze the correlation between politeness and social factors such as power and gender.

My reflection about their work consists of the following points:

1. it is nontrivial to define a norm for politeness. One way of learning this norm is to use human annotators as Danescu-Niculescu-Mizil et al. did. It could be interesting to conduct a similar annotation for the same dataset using human annotators from different cultures ( e.g. different geographic locations ) to understand how the norm for politeness may differ. It could also be interesting to study people’s perception of politeness across different domains. For example, the norm of politeness may differ if the comments are from a political news website versus technical discussions in computer programming.
2. The model evaluation shows a noticeable difference between the in-domain vs. cross-domain settings, as well as another noticeable difference between the cross-domain performance of the model trained with Wikipedia and the that trained with StackExchange. A simple reasoning could be that there are community specific vocabularies that make a model trained on data from one community not to generalize very well on other communities. From this point, we may conclude that the vocabulary used in comments on StackExchange is more generic than that used in the requests to edit on Wikipedia, which gives an advantage to the cross-domain model trained with StackExchange. I believe it is highly important to categorize the communities and to analyze the community-specific linguistic traits in order to make an informed decision when training a cross-domain model.
3. Such a study could be used to help to moderate social platforms that are keen to maintain a certain level of “politeness” in their interactions. It could help moderators automatically detect impolite comments, as well as individuals to tell them how likely are their comments to be perceived as polite or not before sharing them.
4. Given the negative correlation between social power and politeness as inferred by the study, could it be useful to rethink the design of online social systems to encourage maintaining politeness in individuals with higher social power?
5. Although the study incurs some limitations such as the performance of cross-domain models, it represents a robust and coherent analysis that could serve as a guideline for many similar data-driven studies.

To conclude, there are multiple benefits in studying traits of politeness and automatically predict it in online social platforms. This study inspires me to start from where they stopped and work on enhancing their models and applying them to multiple useful domains.

Read More