Lixiu Yu, Aniket Kittur, and Robert E. Kraut. 2014. Distributed analogical idea generation: inventing with crowds. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ’14). ACM, New York, NY, USA, 1245-1254.
Discussion Leader: Nai-Ching Wang
Summary
This paper introduces a 4-step process, distributed analogical idea generation (identify examples, generate schemas, identify new domains and generate new ideas), to increase the possibility of production of creativity by introducing analogical transfer. There are two issues of current ways of producing new and good ideas which use quantity to exchange quality. The first issue is that rewards are usually only given to the best ideas ignoring contribution made by other participants. The other issue is that the exchange of quantity to quality is usually not stable and inefficient because we do not know how many is good enough. This paper uses three experiments to test the effectiveness of the proposed process. The result of the first experiment shows the quality (composed of practicality, usefulness, novelty) of creativity generation is better with expert-produced schemas. The result of second one shows the number of similar examples increases the quality of induced schemas from the crowd while contrasting examples are not as useful. The result of the third one shows different qualities of schemas produced in Exp. 2 affect the last step, idea generation. The three experiments confirm that the proposed process leads to better ideas than example-based methods.
Reflections
This paper starts to address the “winner takes all” issue we have been discussing in class, especially for the design/creativity domain. It seems that we now have a better way to evaluate/appreciate each person’s contribution and decrease unnecessary/inefficient effort. In general, I like the design of the three experiments, each of which deals with a specific aspect of the overall study. In experiment 3, it is shown that good schemas will help produce better ideas. It will be interesting to see how good the experimenter-generated schemas are, especially when we can compare the quality in terms of scores to the results of experiment 2. Unfortunately, this information is not available in the paper. The distributed process presented in the paper is very impressive because it decomposes a larger process into several smaller components that can be operated separately. It would be interesting if there is a comparison of idea quality between traditional way and the way used in the paper. It would also be interesting to see the quality between assembly line and artisan processes because the latter might provide learning opportunities and thus provide higher quality results although the process is not as flexible/distributed as assembly line.
Questions
- What are the benefits/shortcomings for the raters to discuss or be trained for judging?
- Do you think design guidelines/heuristics are similar to schemas mentioned in the paper? How similar/different?
- In experiment 3, what reasons do you think there is an example associated with either a good or bad schema? Why not just use good or bad schemas?
- This paper mostly focuses on average quality. For creativity tasks, do you think that is a reasonable measure?
