Summary:
“Human Computation: A Survey and Taxonomy of a Growing Field” by Quinn and Bederson classifies human computation systems different dimensions. They also point out the subtle differences between human computation, crowd sourcing, social computing, data mining and collective intelligence. Traditionally, human computation is defined as “a paradigm for utilizing human processing power to solve problems that computers cannot yet solve.” Although, there is some overlap between human computation and crowd-sourcing, the major idea behind crowds-sourcing is that it works by employing members of the public in place of traditional human workers. Social computing, on the other hand, differs from human computation in the manner such that it studies natural human behavior mediated by technology. The third term, data mining, focuses on using technology to analyse the data generated by humans. All these terms partly fall in the category of collective intelligence. Well-developed human computation systems are also examples of collective intelligence. For example, to create a gold standard dataset for machine translation, several humans have to provide translations for the same system. This is an example of the collective intelligence of the crowd as well as human expertise to solve computationally challenging tasks. The authors present a classification system based on six different dimensions: (i)Motivation, (ii)Quality Control, (iii)Aggregation, (iv)Human Skill, (v)Process Order and, (vi)Task Request Cardinality. The paper further talks about the various sub categories in these dimensions and how these sub categories influence the output. This work presents a useful framework that can be help researchers categorize their work into one of these categories.
Reflections:
This was an interesting read as it presented an overview of the field of human computation. The paper does a commendable job of highlighting the subtle differences in the different sub-fields of HCI. The authors classify human computation into one of the six dimensions depending on the sample values. However, I think there might be some overlap between these sample values and sub categories. For example, the amount of ‘pay’ a person receives definitely determines the motivation for the task but it may also determine the quality of work. If the worker feels that he is adequately compensated, that might prove an incentive to produce quality work. Similarly, ‘redundancy’ and ‘multi-level review’ are part of the ‘quality check’ dimension but they can also fall in the ‘task request cardinality’ dimension as multiple users are required to perform similar tasks. Another point to be considered here is that although, the authors differentiate between crowd-sourcing and human computation, several parallels can be drawn between them using the dimensions presented. For example, it will be interesting to observe whether the features employed by different crowd platforms can be categorized into the dimensions highlighted in this paper and whether they have the same sub class or do they vary depending on the kind of task at hand.
Questions:
1. Can this work extend to related fields like social computing and crowd-sourcing?
2. How can we categorize ethics and labor standards based on these dimensions?
3. Is it possible to add new dimensions to human computation?
Categorizing ethics and labor standards is an interesting idea. For example, some work, such as moderating explicit content, might be considered unethical if people do it for low pay. On the other hand, it may be perfectly ethical to allow a person to do it for *no* pay, i.e. voluntarily. Similarly, this would affect labor laws.
It would be interesting to see how the authors of this paper feel about these two new dimensions, in light of the scandals that have come out regarding cases of mental health trauma and low wages at tech companies.
Hi, I agree with your observations with the sub-classes of the six dimensions. Since the sub-classes are correlated with each other, subtle overlaps are nearly impossible to be avoided in the taxonomy, especially in the qualitative research approach. About the potential new dimension to human computation, I notice that the paper did no discuss much about the work complexity. As crowd work tends to become more complex, I think to what level of work complexity can be handled could be an extension to the classification dimensions. Sub-classes can include 1) naive, 2) advanced, 3) across domains, and 4) distributed computing. I’ll explain these in details. The majority of computing tasks are easy and require short amount of time to finish, which can be defined as naive. Some other computing work needs some reasoning or logic behind them and more difficult to evaluate, which can be categorized as advanced. Probably not current, but in the near future, certain tasks need the interdisciplinary collaboration between experts from different domains. That would require a novel mode to support this new style of computing. Last but not the least, to satisfy the real-time computing demand, distributed computing may be introduced to the framework.