SUMMARY
In this paper, the authors attempt to facilitate the process of finding analogies with a view to boost creative innovations by exploring the value that can be added by incorporating weak structural representations. They leverage the vast body of online information available (for the purpose of this study, product descriptions from Quirky.com). They generate microtasks for crowdworkers to perform that were designed to label the ‘purpose’ and ‘mechanism’ parts of a product description. The authors use GloVe word vectors to represent their purpose and mechanism words and use a BiRNN to learn the purpose and mechanism. In order to collect analogies, they use AMT crowd workers to find analogies for 8000 product descriptions. In the evaluation stage, the authors attempt to weigh the usefulness of their algorithm by having participants redesign a product. 38 AMT workers were recruited for the same and the task was to design a cell phone charger case. 5 graduate students were recruited to evaluate the ideas generated by the workers. Based on a predefined criterion of ‘good’ ideas, 208 were produced out of 749 total ideas (with 2 judges rating it as good) and 154 were produced out of 749 total ideas (with 3 judges rating it as good). In both cases, the analogy approach proposed by the authors out-performed the TF-IDF baseline model and random model.
REFLECTION
I found the motivation of this study to be very good – especially based on ‘bacteria-slot machine’ analogy example highlighted in the introduction of the paper. I agree that given the vast amount of data available, having such a system that would accelerate the process of finding analogies could very well aid in quicker innovation and discovery.
I like that the authors chose to present their approach by using product descriptions. I also like the use of ‘purpose’ and ‘mechanism’ annotations and feel that given the more general domain of this study, the quality of annotations by the crowdworkers would be better than in the case of the paper on ‘SOLVENT: A Mixed Initiative System for Finding Analogies between Research Papers’.
I also liked that the authors presented the results given by using the TF-IDF baseline as it indicated the findings that would have been generated by near-domain results. I felt that it was good that the authors added a criterion to judge the feasibility of an idea, one that could be implemented using existing technologies.
Additionally, while I found that the study and methods proposed by this paper were good, I did not like the organization of the paper.
QUESTIONS
- How would you rate the design of the interface used to collect ‘purpose’ and ‘mechanism’ annotations? What changes might you propose to make this better?
- The authors do not mention the details or experiences of the AMT workers. How much would workers prior experience influence their ability to find ideas using this approach? Can this approach aid more experienced people working with product innovations?
- The authors of SOLVENT leverage the mixed-initiative system proposed by this paper to find analogies between research papers. Which are the domains where this approach would fail to a great extent (even if modifications were to be made)?
Hi,
I also liked the existence of a comparable baseline for the model.
I wonder what you didn’t like about the organization of the paper.
Regarding your first question, the authors collect purpose and mechanism by highlighting
appropriate parts of the text. I think this is reasonable and intuitive.
The only thing I thought was hard to understand was that the purpose and mechanism highlighting were jumbled together, so I would probably have some kind of switch to show only either purpose or mechanism one at a time.
I think to collect purpose and mechanism, one might not need to just pick snippets but rather can write it out themselves based on the research paper.