Summary:
The paper briefly speaks of the history of human computation. The first dissertation (2005), workshop (2009), and the different backgrounds of scholars in human computation. The authors agree with Von Ahn’s definition of the human computation as: “… a paradigm for utilizing human processing power to solve problems that computers cannot yet solve.” and mention multiple definitions from other papers and scholars. They believe that two conditions need to be satisfied to constitute human computation:
- The problems fit the general paradigm of computation, and so, might someday be solvable by computers.
- The human participation is directed by a computational system or process.
They present a classification for human computation systems made of 6 main factors divided into two groups:
- Motivation, human skill, aggregation.
- Quality control, process, task-request cardinality.
The authors also explain how to find new research problems based on the proposed classification system:
- Combining different dimensions to discover new applications.
- Creating new values for a given dimension.
Reflection:
The interesting issue I found the authors discussing was that they believe that the Wikipedia model does not belong to human computation. Because current Wikipedia articles are created through a dynamic social process of discussion about the facts and presentation of each topic among a network of authors and editors. I never thought of Wikipedia as human computation although there are tasks in there that I believe could be classified as such. Especially when looking at non-English articles. As we all know, the NLP field has created great solutions for the English language, yet some languages, even widely spoken ones, are playing catch up. So, this brings me to disagree with the authors’ opinion about Wikipedia. I agree that some parts of Wikipedia are related to social computing like allowing collaborative writing, but they also have human computation aspects like Arabic articles linked data identification (for the info box). Even though using NLP techniques might work for English articles on Wikipedia, Arabic is still behind when it comes to such task and the machine is unable to complete it correctly.
On another note, I like the way the authors broke up their classification and explained each section. It clarified their point of view and they provided an example for each part. I think that the distinctions were addressed in detail and they left enough room to consider the classification of future work. I believe that this was the reason that other scientists have adapted the classification. Seeing that the paper was cited more than 900 times, it makes me believe that there’s some agreement in the field.
Discussion:
- Give examples of human computation tasks.
- Do you agree/disagree with the author’s opinion about Wikipedia’s articles being excluded from the human computation classification?
- How is human computation different from crowdsourcing, social computing, data mining, and collective intelligence?
- Can you think of a new human computation system that the authors didn’t discuss? Classify it according to the dimensions mentioned in the paper.
- Do you agree with the authors’ classification system? Why/Why not?
- What is something new that you learned from this paper?
You raise a good point that NLP techniques might work for English but there is still a long way to go for other languages. In that case would Wikipedia come under human computation even if the primary goal is to create a resource, not necessarily for computationally intensive tasks?