The authors found that most of current research on crowd work focused on AMT, and there is little exploration of other platforms. To enrich the diversity of crowd work researches and accelerate progress of development of crowd work, the authors contrast the key problems which they have found in AMT versus features of other platforms. With the comparison, the authors found different crowd platforms provide solution to different problems in AMT. However, all the platforms cannot provide sufficient support to well-designed worker analytics. Additionally, they made a conclusion that there is still a number of limitations in AMT and research on crowd work will benefit from diversifying. With the contributions of the authors, future research on alternative platforms can benefit from not considering AMT as the baseline.
Reflection:
AMT is the first platform for crowd work and many people are benefit by using this platform. However, if all companies and workers only use AMT, the lack of diversity will result in a low development progress of crowd work platform.
From AMT’s point of view, if there are no other crowd work platforms, it is hard for AMT to effectively find solutions to its limitations and problems. In this paper, for example, lack of automated tool is one of AMT’s limitations. However, the platform WorkFusion allows usage of machine automated workers, which is an improvement to AMT. If there is only one platform, it is hard for the platform to find its limitation by itself. But with the competitor platforms, to provide better user experience and defeat other adversary companies, they have to do a lot of research to keep their platforms up to date. As a result, the research of crowd work will be pushed to develop rapidly.
For other platforms, they should not just copy the pattern of AMT. Though AMT is the most popular crowd work platform, it still has some limitation. Other platforms can copy the advantages of AMT. However, they should avoid AMT’s limitations and find solutions to these drawbacks. A good baseline can help the initiation of other platforms. But if other platforms are just totally follow the baseline, all the platforms will suffer from same drawbacks and no one can propose any solution to them. To avoid this kind of situation, the companies should develop own platforms, avoid the drawbacks and learn advantages from other platforms.
The researchers should not just focus on AMT. Of course, the researches on AMT will receive more attention as most of the users use AMT. However, it is harmful to the development of crowd work platform. Even some platforms developed solutions for some problems, if the researchers just ignore these platforms, these solutions will not be able to spread out and the researchers and other companies will spend unnecessary effort to do research on similar solutions.
Question:
What is the main reason for most of the companies select AMT as their crowd work service provider?
What is the most significant advantage for each of the platforms? Is there any chance to develop a platform which has all other platforms’ advantages?
Why most of the researchers focused on AMT only? Is it possible to advice more researchers to do cross-platform analysis?
I believe that AMT has gained its popularity due to ease of access (little to no limitations/restrictions). Having little to no restrictions on both the requester and the worker ensured that there are always crowd workers available for tasks and that tasks are posted instantly. This made AMT popular for taskers, workers and more attractive for researchers. Cross-platform analysis is possible for sure, yet it would be skewed given the popularity of AMT compared to other platforms.
It is interesting to see how the reflection pointed out that without any competition, it is difficult to evaluate and discover limitations. I completely agree with this and it’s true that competition helps to better a system and constantly innovate or improve.