{"id":270,"date":"2020-01-30T13:40:18","date_gmt":"2020-01-30T13:40:18","guid":{"rendered":"http:\/\/wordpress.cs.vt.edu\/cs6724s20\/?p=270"},"modified":"2020-01-30T13:40:18","modified_gmt":"2020-01-30T13:40:18","slug":"01-29-20-runge-yan-beyond-mechanical-turk","status":"publish","type":"post","link":"https:\/\/wordpress.cs.vt.edu\/cs6724s20\/2020\/01\/30\/01-29-20-runge-yan-beyond-mechanical-turk\/","title":{"rendered":"01\/29\/20 \u2013 Runge Yan \u2013 Beyond Mechanical Turk"},"content":{"rendered":"\n<p class=\"wp-block-paragraph\"><strong>Analysis on Amazon Mechanic Turk and other paid crowdsourcing platform<\/strong><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Ever since the rise of AMT, research and applications have been focusing on this prominent crowdsourcing platform. With the development of similar platforms, some concerns on the use AMT have been solved in various ways. This paper reviews AMT limitations and compares the solution among 7 other popular platforms: ClickWorker, CloudFactory, CrowdComputing Systems, CrowdFlower, CrowdSource, LeadGenius and oDesk.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">AMT Limitations are presented in four categories: They are\nshort in quality control, management tools, support for fraud prevention and\nautomated tools. These limitations are further mapped to assessment criteria to\nfocus on detailed solution from all the platforms.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">These criteria include the identity and skill of contributors,\nextra workload management, complex task support and quality control by requesters,\nand generalized qualification, task collaboration, task recommendation by\nplatforms. By comparing with AMT, future focus research is addressed. Also, the\nmethod used in this paper can be improved in a way we set an alternative\nplatform as a baseline.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>I tried to work on a platform\u2026<\/strong><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">I\u2019ve been thinking since I tried to work on Amazon Mechanic Turk and ClowdFlower (I believe they changed the name to Figure Eight). How does a requestor post a task? The expected format of the input and output from the platform may not match the interface provided by the requestor. I can see most requestors have to transform\/code for themselves, but the platforms also start to help here. <\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Both platforms require identification through credit card\nand AMT requires SSN. I\u2019m able to use Figure Eight now but AMT refused my\nsignup. I have a relatively new SSN and credit record, which is probably the\nreason I\u2019m refused. Although CrowdFlower comes into people\u2019s sight and was\nmentioned more than before, the difference in scale and functionality can be\neasily spotted in their website layout and structure. <\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Figure Eight provides a basic task for me to start \u2013 A people\u2019s name, current company and position name are given, my job is to go to his LinkedIn profile and make sure of two things: Is he\/she working in the same company, and if so, did he\/she change to another position? How many positions of this people are active in? &nbsp;<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">This should be a simple task, even for a people who\u2019s not familiar with LinkedIn. The reward is relatively low, though. For 10 correct answers I got 1 cent (I think I\u2019m still in an evaluation period). More pay is on the road if I work in a recommended manner, i.e. to try out several simple tasks in different categories, and then put my hands on more complex task, and so on.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Still, I found myself quitting after I made 10 cents. I\u2019m\nnot sure if it\u2019s because I was too casual in sample quiz that I got 8 correct\nout of 10 and that they decided to give me a longer trail. Compared to several fun-inspired\ntasks I tried, experience on Figure Eight is not so welcome as I see it.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Back to the analysis. There\u2019s an example that represent many dilemmas in this tripartite workspace: AMT doesn\u2019t care about workers\u2019 profile while oDesk offers public worker profiles showing their identities, skills, rating, etc. It\u2019s hard to maintain a platform that workers can switch between these two options when identification is preferred for some tasks and not for other tasks. And this may refrain requestors from posting their needs.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Questions<\/strong><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Do platforms cooperate? How to combine existing good\nsolution to improve or come up with a better platform?<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">How does AMT dominate in crowdsourcing for so long and no other platform catches up with it? What are the most significant improvements on AMT in the recent years?<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Analysis on Amazon Mechanic Turk and other paid crowdsourcing platform Ever since the rise of AMT, research and applications have been focusing on this prominent crowdsourcing platform. With the development of similar platforms, some concerns on the use AMT have been solved in various ways. This paper reviews AMT limitations and compares the solution among [&hellip;]<\/p>\n","protected":false},"author":296,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[43,28],"class_list":["post-270","post","type-post","status-publish","format-standard","hentry","category-uncategorized","tag-beyondmturk","tag-class2"],"jetpack_featured_media_url":"","_links":{"self":[{"href":"https:\/\/wordpress.cs.vt.edu\/cs6724s20\/wp-json\/wp\/v2\/posts\/270","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/wordpress.cs.vt.edu\/cs6724s20\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/wordpress.cs.vt.edu\/cs6724s20\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/wordpress.cs.vt.edu\/cs6724s20\/wp-json\/wp\/v2\/users\/296"}],"replies":[{"embeddable":true,"href":"https:\/\/wordpress.cs.vt.edu\/cs6724s20\/wp-json\/wp\/v2\/comments?post=270"}],"version-history":[{"count":2,"href":"https:\/\/wordpress.cs.vt.edu\/cs6724s20\/wp-json\/wp\/v2\/posts\/270\/revisions"}],"predecessor-version":[{"id":280,"href":"https:\/\/wordpress.cs.vt.edu\/cs6724s20\/wp-json\/wp\/v2\/posts\/270\/revisions\/280"}],"wp:attachment":[{"href":"https:\/\/wordpress.cs.vt.edu\/cs6724s20\/wp-json\/wp\/v2\/media?parent=270"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/wordpress.cs.vt.edu\/cs6724s20\/wp-json\/wp\/v2\/categories?post=270"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/wordpress.cs.vt.edu\/cs6724s20\/wp-json\/wp\/v2\/tags?post=270"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}