Lilly C. Irani and M. Six Silberman. 2013. Turkopticon: interrupting worker invisibility in amazon mechanical turk. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ’13). ACM, New York, NY, USA, 611-620. DOI= 10.1145/2470654.2470742http://dl.acm.org/citation.cfm?id=2470742
Discussion leader: Mauricio De La Barra
Summary
This paper provides an analysis of Amazon Mechanical Turk, a human computation system, as a site of technically mediated worker-employer relationships. The authors argue that human computation currently relies on worker invisibility, which in turn leads HCI researchers to pay less attention to crowdwork’s ethics and values. In order to bring to light the relations between requesters and Turkers, they conducted a case study in which they asked 67 Turkers questions about what they would desire as a “Workers’ Bill of Rights.” The points of agreement in the survey were the basis of Turkopticon’s design, which is an activist system that allows workers to do two main things: publicize and evaluate their relations with employers, and engage one another in mutual aid. Turkopticon was developed as a browser extension for Chrome and Firefox. When browsing in Mechanical Turk for HITs, this extension display a CSS button next to the requester’s name. On mouse-over, the workers can see the ratings of the requester according to four qualities: Communicativity, Generosity, Fairness, and Promptness, and also a link to a website to see all the written reviews for that requester. Workers can also leave their own reviews for that requester. This paper continues by explaining that the design of Mechanical Turk favors requesters over workers, which causes the unjust treatment of workers. Turkopticon attempts to provide more fairness in the relationship between requesters and Turkers by holding requesters accountable and enabling help among workers. This piece of software has become an essential tool for many Turkers, as it has been installed over 7,000 times and the Turkopticon website receives 100,000 page views a month. The authors conclude the paper by highlighting the lessons they have learned from intervening in large-scale socio-technical systems, such as Mechanical Turk.
Reflection
As someone who doesn’t use Mechanical Turk often, I find this paper to be a great overview of some of the ethical issues of the use of this platform. While some of them are somewhat obvious (such as whether or not to have a minimum wage for HITs), I didn’t really internalize these issues while using the system. One of the most interesting things I found about the paper is how it highlights that when designing Mechanical Turk, Amazon has prioritized the needs of employers over workers. As an example, by hiding workers behind APIs, Mechanical Turk makes employers see themselves as builders, rather than as employers unconcerned with working conditions of Turkers. Also, because Mechanical Turk’s participation agreement gives requesters intellectual property rights over submission regardless of rejection, workers have no legal recourse against employers who reject work and then use it. One would think that a system such as Mechanical Turk, which is based on relations between requesters and workers, would be designed so that both parties have their concerns addressed. But as Amazon’s goal (like any other company) is to make money, it treats workers interchangeably and since there are so many workers, Mechanical Turk can sustain the loss of workers who don’t abide by the terms of agreements: since Amazon collects money for task volume, they have little reason to prioritize worker needs. I feel that systems such as Turkopticon are a step in the right direction in order to make the workers’ relationships with employers visible to other workers in Mechanical Turk, but I feel that change needs to happen at the infrastructure level – Amazon should also consider the ethical issues that arise through the use of Mechanical Turk.
Questions
• Do you think that if the Turkopticon extension get widely adopted among workers in Mechanical Turk, that Mechanical Turk requesters would move on to a different human computation/crowdsourcing platform? Why?
• Do you agree or disagree with the criteria used to review requesters (do they represent an accurate framing of the interaction with the requester)? Or what other criterion could have been used instead?
• Turkopticon’s developers hope that Amazon would change its system design to include worker safeguards in Mechanical Turk. This has not happened yet. If Amazon becomes aware of Turkopticon, and how useful it is for workers, do you think that it might consider changing its design? In what ways?
• What are some policies that Mechanical Turk should adopt in order to show that it doesn’t just care about the needs of requesters, but also about the needs of workers (e.g., requiring requesters to justify rejections, having a minimum wage for HITs, etc.)? Or do you think that the system is fine as it is currently?
