Authors: Firas Khatib, Seth Cooper, Micheal D. Tyke, Kefan Xu, Ilya Makedon, Zoran Popvic, David Baker
Discussion Leader: Divit Singh
Crowdsourcing Example: http://weathersignal.com
Summary
Foldit is an online puzzle video game. It presents a platform on which multiple players can collaborate and compete on various tasks such as protein folding. It utilizes citizen science: leveraging natural human abilities for scientific purposes. Foldit provides the players with a palette of interactive tools and manipulations to aid them in structuring the protein presented to them. In addition, Foldit also provides players with the ability to create their own “recipes” for augmenting proteins. These recipes are a set of instructions and game play macros that enable the players using the recipe to automatically manipulate the proteins presented to them. User-friendly algorithms from the Rosetta structure prediction methodology were presented as well to aid players in interacting with structures. From observing how players utilized these algorithms, it became apparent that the players used these algorithms to aughment rather than to substitute for human strategizing. There was no one algorithm that was employed. At different stages of interaction, players would use multiple recipes to build their structures which in turn, lead to more recipes being created.
During the time of the study, researchers created the “Fast Relax” algorithm which achieved better results in less time. However, an algorithm was also developed by the Foldit players during this time: “Blue Fuse”. These algorithms were very similar to each other and developed completely independently. On testing these algorithms in side by side, it was revealed that Blue Fuse is more effective than Fast Relax (in Foldit) on time scales best compatible with game play. The discovery of this algorithm was created solely by Foldit players.
Reflection
This paper is about a popular crowdsourcing framework used in the bioinformatics field. It presents a unique way to utilize the brainpower of the general masses to create efficient new and efficient algorithms by introducing a gaming aspect to protein folding. I really liked how they allowed the players to build their algorithsm/simulations by employing the concept of “recipes”. I believe that this was a crucial feature that allowed other players to build off someone else’s work rather than starting from scratch and coming up with either their own small contribution or replicating someone else’s work. They present a clear UI with a helpful suite of tools to help in manipulating the structure as well. In addition, I found that there were videos on YouTube as well as abundant information on their website to really emphasize the purpose of this software.
Figures 3 and 4 really emphasized the power of citizen science as it shows the social evolution of Foldit recipes. New recipes are essentially built on top of each other in hopes to gain marginal efficiency with each iteration. Instead of using machine learning in an attempt to approximate these recipes and simulations, real humans creations were used to develop algorithms. The fact that these recipes resembled that of an algorithm produced by researchers specifically focused on producing an efficient algorithm shows the power of human computation. As it stands, machine learning can only take us so far, especially in visual tasks such as these.
Questions
1. What are your opinions on gamifying problem solving/reasoning tasks such as this to attract a crowd? Do you think it takes away from the task at hand by attracting a crowd that may be too young/old for its purpose? If so, how would you leverage gamification/any other task to try to attract the specified target audience?
2. Assuming there was no “energy” function in which to rate recipes for. Based on visual aesthetics, how would you create a metric to measure how “clean” or “well-produced” a certain recipe is?
3. Would you rather have recipes be built on top of each other, or have individuals try to create their own from scratch? If you want them to be built on top of each other, does it not “tunnel-vision” subsequent creators?
