Summary of the Reading
This paper is a survey of the research in the field of human computation. The paper aims to classify human computation systems so that the similarities between different projects and the holes in current research can be seen more clearly. The paper also explores related fields like crowdsourcing.
The paper starts by defining human computation as “a paradigm for utilizing human processing power to solve problems that computers cannot yet solve.” The paper then goes on to discuss the differences between human computation and crowdsourcing, social computing, data mining, and collective intelligence.
The paper then goes on to classify human computation systems based on 6 dimensions: motivation, quality control, aggregation, human skill, process orders, and task-request cardinality. Each of these dimensions has several discrete options. Putting all of these dimensions together allows for the classification of any arbitrary human computation system. The paper also provides examples of systems that have various values on each dimension.
Reflections and Connections
I think this paper provides a much needed tool for further human computation and crowdsourcing research. The first step to understanding something is being able to classify that thing. This tool will allow current and future researchers to classify human computation systems so that they can apply existing research to other, similar systems and also allow them to see where existing research falls short and where they need to focus future research.
This research also provides an interesting perspective on current human computation systems. It is interesting to see how current human computation systems compare to each other and what each system does differently and what they have in common.
I also like the malleability of the classification system. They say in the future work section that this system is very easy to add to. Future researchers who continue the work on this project could easily add values to each of the dimensions to better classify the human computation systems. They could also add values to the dimensions if new human computation systems are invented and need to be classified using this system. There are a lot of good opportunities for growth from this project.
One thing that I thought this paper was missing is a direct comparison of different human computation systems on more than one dimension. The paper uses human computation systems as examples in the various values for each of the dimensions of the classification system, but it doesn’t put these dimensions together and compare the human computation systems on more than one dimension. I think this would have added a lot to the paper, but it would also make for a great piece of future work for this project. This idea is actually very similar to the other paper from this week’s bunch, titled “Beyond Mechanical Turk: An Analysis of Paid Crowd Work Platforms” and I think it would be helpful and build on both of these papers.
Questions
- Do you think the classification system presented in this paper has enough dimensions? Does it have too many?
- What is one application you see for this classification system?
- Do you think this classification system will help crowdsourcing platforms deal with some of their issues?
I agree with the point that categorization is an important step in understanding a growing research field. Based on a useful categorization, future researchers can benefit a lot from this, like acknowledging the related work. Research questions can be solved in its specific areas derived from other similar studies. In addition, I think the number of dimensions is too many. Because I found lots of applications have been categorized into multiple dimensions. I am not sure this is good for the taxonomy of applications or not. This result can cause confusion.