{"id":1157,"date":"2020-04-21T13:42:59","date_gmt":"2020-04-21T17:42:59","guid":{"rendered":"http:\/\/wordpress.cs.vt.edu\/cs6724s20\/?p=1157"},"modified":"2020-04-21T13:43:00","modified_gmt":"2020-04-21T17:43:00","slug":"4-22-20-lee-lisle-solvent-a-mixed-initiative-system-for-finding-analogies-between-research-papers","status":"publish","type":"post","link":"https:\/\/wordpress.cs.vt.edu\/cs6724s20\/2020\/04\/21\/4-22-20-lee-lisle-solvent-a-mixed-initiative-system-for-finding-analogies-between-research-papers\/","title":{"rendered":"4\/22\/20 \u2013 Lee Lisle \u2013 SOLVENT: A Mixed Initiative System for Finding Analogies between Research Papers"},"content":{"rendered":"\n

Summary<\/strong><\/p>\n\n\n\n

Chan et al.\u2019s paper discusses a way to find similarities in\nresearch papers through the use of mixed initiative analysis. They use a\ncombination of humans to identify sections of abstracts and machine learning\nalgorithms to identify key words in those sections in order to distill the\nresearch down into a base analogy. They then compare across abstracts to find\npapers with the same or similar characteristics. This enables researchers to\nfind similar research as well as potentially apply new methods to different\nproblems. They evaluated these techniques through three studies. The first\nstudy used grad students reading and annotating abstracts from their own domain\nas a \u201cbest-case\u201d scenario. Their tool worked very well with the annotated data\nas compared to using all words. The second study looked at helping find\nanalogies to fix similar problems, using out-of-domain experts to annotate\nabstracts. Their tool found more possible new directions than the all words\nbaseline tool. Lastly, the third study sought to scale up using crowdsourcing.\nWhile the annotations were of a lesser quality with mTurkers, they still\noutperformed the all-words baseline.<\/p>\n\n\n\n

Personal Reflection<\/strong><\/p>\n\n\n\n

               I liked\nthis tool quite a bit, as it seems a good way to \u201cunstuck\u201d oneself in the\nresearch black hole and find new ways of solving problems. I also enjoyed that\nthe annotations didn\u2019t necessarily require domain-specific or even\nresearcher-specific knowledge even with the various jargon that is used.\nFurthermore, though it confused me initially, I liked how they used their own\nabstract as an extra figure of sorts \u2013 using their own approach to annotating\ntheir abstract was a good idea. It cleverly showed and explained how their\napproach works quickly without reading the entire paper.<\/p>\n\n\n\n

               I did\nfind a few things confusing about their paper, however. They state that the\nGloVe model doesn\u2019t work very well in one section, but then use it in another.\nWhy go back to using it if it had already disappointed the researchers in one\nphase? Another complication I noticed was that they didn\u2019t define the dataset\nin the third study. Where did the papers come from? I can glean from reading it\nthat it was from one of the prior two studies, but I think its relevant to ask\nif it was the domain-specific or the domain-agnostic datasets (or both).<\/p>\n\n\n\n

               I was\ncurious about total deployment time for this kind of thing. Did they get all of\nthe papers analyzed by the crowd in 10 minutes? 60 minutes? A day? With how\nparallel the task can be performed, I can imagine it could be very quick to get\nthe analysis performed. While this task doesn\u2019t need to be quickly performed,\nit could be an excellent bonus of the approach.<\/p>\n\n\n\n

Questions<\/strong><\/p>\n\n\n\n

  1. This tool seems extremely useful. When would you\nuse it? What would you hope to find using this tool?<\/li>
  2.  Is the\nannotation of 10,000 research papers worth $4000? Why or why not?<\/li>
  3. Based on their future work, what do you think is\nthe best direction to go with this approach? Considering the cost of the\ncrowdworkers, would you pay for a tool like this, and how much would be\nreasonable?<\/li><\/ol>\n","protected":false},"excerpt":{"rendered":"

    Summary Chan et al.\u2019s paper discusses a way to find similarities in research papers through the use of mixed initiative analysis. They use a combination of humans to identify sections of abstracts and machine learning algorithms to identify key words in those sections in order to distill the research down into a base analogy. They […]<\/p>\n","protected":false},"author":105,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[106,109],"class_list":["post-1157","post","type-post","status-publish","format-standard","hentry","category-uncategorized","tag-class14","tag-solvent"],"jetpack_featured_media_url":"","_links":{"self":[{"href":"https:\/\/wordpress.cs.vt.edu\/cs6724s20\/wp-json\/wp\/v2\/posts\/1157","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/wordpress.cs.vt.edu\/cs6724s20\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/wordpress.cs.vt.edu\/cs6724s20\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/wordpress.cs.vt.edu\/cs6724s20\/wp-json\/wp\/v2\/users\/105"}],"replies":[{"embeddable":true,"href":"https:\/\/wordpress.cs.vt.edu\/cs6724s20\/wp-json\/wp\/v2\/comments?post=1157"}],"version-history":[{"count":1,"href":"https:\/\/wordpress.cs.vt.edu\/cs6724s20\/wp-json\/wp\/v2\/posts\/1157\/revisions"}],"predecessor-version":[{"id":1158,"href":"https:\/\/wordpress.cs.vt.edu\/cs6724s20\/wp-json\/wp\/v2\/posts\/1157\/revisions\/1158"}],"wp:attachment":[{"href":"https:\/\/wordpress.cs.vt.edu\/cs6724s20\/wp-json\/wp\/v2\/media?parent=1157"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/wordpress.cs.vt.edu\/cs6724s20\/wp-json\/wp\/v2\/categories?post=1157"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/wordpress.cs.vt.edu\/cs6724s20\/wp-json\/wp\/v2\/tags?post=1157"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}