Summary
The paper discusses crowd work and was presented at CSCW (Computer-supported cooperative work) in 2013. It proposes a framework that takes ideas from organizational behavior and distributed computing along with workers’ feedback. The authors of the paper consider the crowd sourcing platform to be a distributed system’s platform where each worker is considered to be analogous to a node in distributed system. This would help in partitioning tasks like parallel computing does. The ways shared resources can be managed, and allocation is also discussed well in this paper. The paper provides deep analysis on the kind of work crowd workers end up doing, the positives and the negatives of such work.
The paper outlines and identifies 12 research areas that form their model. This takes into account broadly, the future of crowd work processes, crowd computation and the crowd workers. Each of the broad topics addressed various subtopics from quality control to collaboration between workers. The paper also talks about how to create leaders in such systems, the importance of better communication and that learning, and assessment should be an integral part of such systems.
Reflection
It was an interesting read on the future of the crowd work. The approach to define the system as a distributed system was fascinating and a novel way to look at the problem. Workers do have a capability to act as “parallel processors” which make the system more efficient and would enable to do intensive tasks (like application development) effectively. Implementing theories from organizational behavior is interesting that it allows the system to better manage and allocate resources. The authors address various subtopics that talk about various issues in depth. It was a very informative read on where they incorporated background work on each of the research areas. I will be discussing some of the topics or problems that stood out to me.
Firstly, they spoke about processes. Assignment of work, management turns out to be a challenging task. In my opinion, a universal structure or hierarchy is not the way to go. In certain kinds of work or tasks it is needed to have a structure where hierarchy would prove to be useful. Work like software development, would benefit from a structure where the code is reviewed, and the quality is assessed by a separate person. Such work also needs a synchronous as people might have tasks dependent on each other.
Secondly, the paper discussed the future of crowd-computation. This included the discussion of AIs and how they can be used in the future to guide crowd working. AI in recent years have proved to be an important tool. Automatic text summarization can be used to help create “Gold standards”. Similarly, other NLP techniques could very well be used to extract information, annotate, summarize and provide other automatic services that can be used to integrate with the current human framework. This would create a human-in the loop system.
Lastly, the future of crowd workers is also an important topic to ponder. Crowd workers are often not compensated well. Similarly, requesters are often delivered sub-par work. The paper did mention that background verification is not always done properly for such “on-demand worker” as it is done for full-time employees from transcripts to interviews. This is a challenge. However, on-demand workers can be validated like Coursera does to validate students. They can be asked to upload documents for tasks that require specialization. This is in itself a task that can be carried out by contractors who verify documentation or create a turk job for the same.
Overall, this was an interesting read and research should be conducted in each of the areas to see how the system and work improves. It has the potential to create more jobs in the future with recruiters being able to hire people instantaneously.
Questions
- The authors only considered AMT and ODesk to define the framework. Would other platforms (like Amara, LeadGenuis) have greater/lesser issue which differ from the current needs?
- They mentioned about “oDesk Worker Diary” which takes snapshots of workers’ computer screen. How is the privacy and security addressed?
- Can’t credentials be verified digitally for specialized tasks?
“They mentioned about “oDesk Worker Diary” which takes snapshots of workers’ computer screen. How is the privacy and security addressed?”
While the idea behind this strategy is to ensure high-quality work, I completely agree with your concern regarding privacy and security. I feel that examining the way the workers do work periodically rather than waiting for the end result may have its own benefits. Misconceptions about the tasks can be caught at an earlier stage. Having said that, I feel that taking snapshots of the workers’ computer screens may not necessarily be a good solution. Features of version control systems like Git can be leveraged where periodic snapshots of the work are committed and pushed by the workers themselves. This would enable the requesters to monitor the progress and intervene when necessary. However, the adoption of this approach is subject to the nature of the tasks involved.