Summary
In the paper, Quinn and Bederson reflect on the current state of human computation research and define a framework for current and future research in the field. They make sure to impart to the reader that human computation is not crowdsourcing, nor collective intelligence – rather, it is a space where human effort is used where computer may be able to solve the problems in the future. They then define several dimensions on how to classify a human computation study; these are motivation (which can include pay or altruism among others), quality control (or how the study ensures reliable results), how the study aggregates the data, what human skill is used (visual perception etc.), process order (how the tasks are deployed) and task-request cardinality (how many tasks are deployed for how many requests). Using these dimension definitions, the authors define new research areas for growth, through pointing out uncombined dimensions or by creating new dimensions to explore.
Personal Reflection
I read this paper after reading the human computation/human computer collaboration affordances survey, and it was interesting to compare and contrast the two papers for how they approached very similar problems in different ways. This paper did a good job in defining dimensions rather than research areas. It was much easier to understand how one can change the dimensions of research as a sort of toggle on how to tackle the issues they purport to solve.
Also, the beginning seemed to be on a tangent about what human computation is really defined as, but I thought this section helped considerably narrow the scope of what they wanted to define. I had thought of human computation and crowdsourcing as synonyms, so getting them separated early on was a good way of setting the scene for the rest of the paper.
Also, this paper opened my eyes to see how wide the dimensions could be. For example, while I had known of a few methods for quality control, I hadn’t realized there were so many different options.
Lastly, I am very happy they addressed the social issues (in my opinion) plaguing this field of research in the conclusion. Treating these workers as faceless mercenaries is dehumanizing at best. I wish there was a little more interaction between the two parties than there is currently, but it is being at least thought of in these survey studies.
Questions
- What dimension do you think has the most promising potential for new growth, and why?
- Do you think you can start a new research project by just choosing a set of 6 choices (1 for each dimension) and then design a project?
- If a project has the same collection of dimensions as another proven study, is there merit in researching it? Or should it just work?
- Can you think of any study that might fit under two different discrete values of the same dimension? I.E., is there a many (studies) to one dimensional value relationship, or is it many to many?
