{"id":1097,"date":"2020-04-17T17:15:10","date_gmt":"2020-04-17T21:15:10","guid":{"rendered":"http:\/\/wordpress.cs.vt.edu\/cs6724s20\/?p=1097"},"modified":"2020-04-17T17:15:11","modified_gmt":"2020-04-17T21:15:11","slug":"04-22-20-myles-frantz-the-knowledge-accelerator-big-picture-thinking-in-small-pieces","status":"publish","type":"post","link":"https:\/\/wordpress.cs.vt.edu\/cs6724s20\/2020\/04\/17\/04-22-20-myles-frantz-the-knowledge-accelerator-big-picture-thinking-in-small-pieces\/","title":{"rendered":"04\/22\/20 \u2013 Myles Frantz \u2013 The Knowledge Accelerator: Big Picture Thinking in Small Pieces"},"content":{"rendered":"\n

Summary<\/h2>\n\n\n\n

Maintaining a public and open source website can be difficult since the website is supported by individuals that are not paid. This team investigated using a crowd sourcing platform to not only support the platform but create articles. These articles (or tasks) were broken down into micro tasks there were manageable and scalable by the crowd sourced workers. These tasks were integrated throughout other HITs and were given extra contributions in order to relieve any extra reluctance on editing other crowd workers work.\u00a0<\/p>\n\n\n\n

Reflection<\/h2>\n\n\n\n

I appreciate the competitive nature of comparing both the supervised learning (SL) and a reinforcement learning (RL) in the same type of game scenario of helping the human succeed by aiding the as best as it can. However as one of their contributions, I have issue with the relative comparison between the SL and RL bots. Within their contributions, they explicitly say they find \u201cno significant difference<\/em> in performance\u201d between the different models. While they continue to describe the two methods performing approximately equally, their self-reported data describes a better model in most measurements. Within Table 1 (the comparison of humans working with each model), SL is reported as having a better (yet small) increase and decrease in Mean Rank and Mean Reciprocal Rank respectively (lower and then higher is better respectively). Within Table 2 (the comparison of the multitude of teams), there was only one scenario where the RL Model performed better than the SL Model. Lastly even in the participants self-reported perceptions, the SL Model only decreased performance in 1 of 6 different categories. Though it may be a small decrease in performance, they\u2019re diction downplays part of the argument their making. Though I admit the SL model having a better Mean Rank by 0.3 (from Table 1 MR difference or Table 2 Human row) doesn\u2019t appear to be a big difference, I believe part of their contribution statement \u201cThis suggests that while self-talk and RL are interesting directions to pursue for building better visual conversational agents\u2026\u201d is not an accurate description since by their own data it\u2019s empirically disproven.\u00a0<\/p>\n\n\n\n

Questions<\/h2>\n\n\n\n