In very much the same vein as The Language that Gets People to Give: Phrases that Predict Success on Kickstarter by Mitra et al., the authors in this paper look for linguistic cues that foretell betrayal in relationships. Their research focuses on the online game Diplomacy that is set in the pre-World War 1 era. An important aspect of this paper is understanding the game and its intricacies. Each player chooses a country, forms alliances with other players, and tries to win the game by capturing different territories in Europe. Central to the game are these alliances and betrayals, and the conversations that happen when a player becomes disloyal to a friend.
The paper uses draws on prior research work in extracting politeness, sentiment, and linguistic cues for several of its features, and it was instructive to see the uses of some of these social computing tools in their research.
The authors find that there are subtle signs that predict betrayal, namely:
- An imbalance of positive sentiment before the betrayal, where the betrayer uses more positive sentiment;
- Less argumentation and discourse from the betrayer;
- Less planning markers in the betrayer’s language;
- More polite behavior from the betrayer; and
- An imbalance in the number of messages exchanged
Intuitively, I can relate to observations #2, #3, and #5. However, positive sentiment and polite behavior would perhaps not indicate betrayal in an offline context. I do wish that these results were explained better and more examples given to indicate why they made sense.
I also felt that the machine learning model to predict betrayal could have been described better. I could not immediately understand their feature extraction mechanism — were linguistic cues used as binary features or count features? Assuming it wasn’t a thin-slicing study and they used count features, did they normalize the counts over the number of times two players spoke? Additionally, they compared the performance of their model against the players (who were never able to predict a betrayal, i.e. their accuracy was 0%). While 0% -> 57% seems like a big jump, the machine learning model could have predicted at random and still obtained a 50% accuracy rate. This begs the question of how accurate the model really is and what features it found important.
Papers in computational social science often need to define (otherwise abstract) social constructs precisely, and quantitatively. Niculae et al. attempt to define friendships, alliances, and betrayals in this paper. While I like and agree with their definitions with respect to the game, it is important to recognize that these definitions are not necessarily generalizable. The paper studies a small subset of relationships online. I would be interested in seeing how this could be replicated for more offline contexts.
