Paper-
A computational approach to politeness with application to social factors – Danescu-Niculescu-Mizil et. al.
Summary-
The authors created an annotated politeness corpus of requests on Wikipedia and Stack Exchange using Amazon Mechanical Turk, which they used to build a classifier which uses domain independent lexical and syntactic features to achieve near human levels of accuracy on classifying a text as polite or impolite. They confirmed some theoretical ideas about politeness, and found that users who were more polite were win adminship elections on these websites, i.e. an editor/admin rank, and that their politeness levels actually decreased after reaching these levels, and the opposite happened for users who also ran for adminship elections and lost. They also analyzed links between status, gender, interaction type, community type, etc and politeness levels.
Reflection-
The authors designed an experiment with both theoretical and practical importance. The fact that the SVM trained on features derived from socio-linguistic/psychological theories of politeness was better than the bag-of-words model, gives more credibility to these theories. Moreover, the model which they trained also has practical importance since it has applications in auto-moderation of online forums, conversational systems, translation systems, etc.
This paper was an interesting read because it taught me a lot about experimental design for computational social science.What struck me about the approach was how cautious and thorough it was. I learned of a number of interesting statistical corrections which the authors applied to get more accurate results, and to ensure the annotation was being done properly, which may be applicable in my project or in future research. For example, they applied z-score normalization to each Turkers score to correct for personal biases in rating a request’s politeness. They also compared pairwise score correlations for Turkers who reviewed the same request, to ensure that the labelling is not happening randomly. Moreover, the authors even trained a “control” SVM based on bag-of-words unigrams to show that the results were due to the power of the features, and not simply the power of the classifier used. These were all great examples of “science done right”, to help budding researchers like myself understand how to obtain and interpret results.
The paper found that question askers were more significantly polite than answerers. Interestingly, high power users were less polite than low power users even when they themselves were asking questions. However, this difference was not extremely pronounced, but it hints at interesting power dynamics/psychological phenomena in the behavior of high power users, which might be worth computationally exploring.
The authors trained their SVM model on Wikipedia, and used it to classify politeness of requests on Stack Exchange. However, this is not a strong enough indicator of the generalizability of this model, since both Wikipedia and Stack Exchange are broadly similar types of sites, (i.e. both deal with acquisition of information). To further prove the robustness of the model, it should be applied to more diverse and unrelated online communities, like 4chan, gaming forums, reddit, etc.
The model could also be applied to short exchanges beyond just requests, like comment threads, to further see how generalizable the results are.
The authors also mention that the annotated corpus was only made with requests which had two sentences, with the second sentence being the actual request. The use of this corpus is limited, because of its specificity. A more general politeness corpus could be created taking into consideration exchanges of various lengths.
The model used could also be increased in power, i.e. using a deep neural network instead of an SVM for classification, and results could be observed using the new model.
