Summary
In this paper, the authors aim to classify the Human Computation systems and compare/contrast the term with other terms like crowdsourcing, social computing, and data mining. The paper starts by presenting some definitions of the human computation where all refer to it as a utilizing the human power to solve problems that can’t be solved by the computers yet.
The paper present different computational systems that share some properties with the human computation and yet they are different. The authors highlight some computational systems like social computing, crowdsourcing, and data mining and show the similarities and distinctions with the human computation systems. All systems grouped together under the collective intelligence where humans’ intel solve a big problem.
The paper presents a classification system for human computation systems that is based on six dimensions. The dimensions include motivation, quality control, aggregation, human skills, process order, and task-request cardinality.
The authors presented different ways to that can motivate people to participate in the systems like pay, altruism, enjoyment and others. And presents the pros and cons for each approach.
The authors also presented different approach to improve the quality of the systems like output or input agreements, expert review, and multilevel reviews and ground truth seeding. All these approaches try to get better quality and ways to measure the performance of the system.
There are different aggregation approaches that collect the results of the tasks completed and to formulate the solution of the global problem.
Other dimensions like human skills, process order, and task-request cardinality discuss that skills required, the way order is processed and the pipeline the request can go through.
Reflection
I found one interesting definition of human computation interesting. It defines it as “systems of computers and large numbers of humans that work together in order to solve problems that can’t be solved by either computers or humans”. It is try that if humans can solve problems then there will be no need to use computers and also if systems can solve the problems and automate the solution then there will be no need for humans so both need to work together to solve bigger problems.
I also found the comparison between different systems including the human computation interesting. I personally was thinking that some systems like crowdsourcing is a human computation system, but it appears it is not.
I agree with the dimensions that define or classify human computation systems as they are accurate measures that help researchers to build new system and to evaluate it.
To connect to other ideas, I found the work is like dynamic programming where we have to solve small problems to eventually solve the global problem. Small tasks are distributed to workers to solve a small problem and then aggregation methods will take these solutions to solve the global problem.
I also found the ground truth seeding quality control approach in similar to the training and testing data in any machine learning algorithm.
Questions
- What other dimensions can we define to classify a human computation system?
- There are different approaches that can measure the quality of a human computation systems. Which one is the best?
- Can we combine to motivation methods together to get better results? Like combining both pay and the enjoyment to solve a global problem?
