{"id":869,"date":"2020-03-24T19:13:09","date_gmt":"2020-03-24T19:13:09","guid":{"rendered":"http:\/\/wordpress.cs.vt.edu\/cs6724s20\/?p=869"},"modified":"2020-03-24T23:37:34","modified_gmt":"2020-03-24T23:37:34","slug":"03-25-20-lee-lisle-evorus-a-crowd-powered-conversational-assistant-built-to-automate-itself-over-time","status":"publish","type":"post","link":"https:\/\/wordpress.cs.vt.edu\/cs6724s20\/2020\/03\/24\/03-25-20-lee-lisle-evorus-a-crowd-powered-conversational-assistant-built-to-automate-itself-over-time\/","title":{"rendered":"03\/25\/20 \u2013 Lee Lisle \u2013 Evorus: A Crowd-powered Conversational Assistant Built to Automate Itself Over Time"},"content":{"rendered":"\n<p class=\"wp-block-paragraph\"><strong>Summary<\/strong><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; Often, humans\nhave to be trained on how to talk to AIs such that the AI understands the human\nand what it is supposed to do. However, this certainly puts the onus on the\nhuman to adjust rather than having the AI adjust to the human. One way of\nsolving that issue was to create crowdsourced responses so that the human and\nAI could understand each other with what was essentially a middle-man approach.\nHuang et al.\u2019s work was on creating a hybrid crowd and AI powered conversational\nassistant that aims to create an AI that can be fast enough for a human to\ninteract with naturally but also creates higher-quality natural responses that\nthe crowd could create. It accomplishes this through reusing responses that were\npreviously generated by the crowd over time as high quality responses are identified.\nThey also deployed Evorus over a period of 5 months with 281 conversations that\ngradually moved from crowd responses to chatbot responses.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Personal Reflection<\/strong><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">I liked that the\nauthors took an odd stand in this line of research early on in the paper \u2013 that,\nwhile it has been suggested that AI will eventually take over for the crowd implementations\nof a lot of systems, this hasn\u2019t happened despite a lot of research. This stand\nhighlights that the crowd has been performing tasks that it was supposed to\nstop at some point for a long time.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Also, I found that I\ncould possibly adapt what I\u2019m working on with automatic speech recognition\n(ASR) to improve itself with a similar approach. If I took, for example,\nseveral outputs from different ASR algorithms along with crowd responses and\nhad a ranked voting for which was the best transcription, perhaps it could\neventually wean itself off the crowd as well.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">It was also\ninteresting that they took a reddit or other social website approach with the\nupvote\/downvote system of determination. This approach seems to have long legs\nin fielding appropriate responses via the crowd.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The last observation\nI would like to make is that they had an interesting and diverse set of bots,\nthough I question the usefulness of some of them. I don\u2019t really understand how\nfiller bot can be useful except in situations where the AI doesn\u2019t really\nunderstand what is happening, for example. I had also though the interview bot\nwould be low performing as the types of interviews it describes that it pulled\nits training data from would be particular to certain types of people. <\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Questions<\/strong><\/p>\n\n\n\n<ol class=\"wp-block-list\"><li>Considering they said that the authors felt that\nEvorus wasn\u2019t a complete system but a stepping point to a complete system, what\ndo you think they could do to improve it? I.E., what more can be done?<\/li><li>What other domains within human-AI collaboration\ncould use this approach of the crowd being a scaffold that the AI develops upon\nuntil the scaffold is no longer needed? Is the lack of these deployments\nevidence that the developers don\u2019t want to leave this crutch or is it due to\nthe crowd still being needed?<\/li><li>Does the weighting behind the upvotes and\ndownvotes make sense? Should the votes have equal (yet opposite) weighting or\nshould they be manipulated as the authors did? Why or why not?<\/li><li>Should the workers be incentivized for upvotes\nor downvotes? What does this do the middle-of-the-road responses that could be\nright or wrong?<\/li><\/ol>\n","protected":false},"excerpt":{"rendered":"<p>Summary &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; Often, humans have to be trained on how to talk to AIs such that the AI understands the human and what it is supposed to do. However, this certainly puts the onus on the human to adjust rather than having the AI adjust to the human. One way of solving that issue was [&hellip;]<\/p>\n","protected":false},"author":105,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[87,84],"class_list":["post-869","post","type-post","status-publish","format-standard","hentry","category-uncategorized","tag-class10","tag-evorus"],"jetpack_featured_media_url":"","_links":{"self":[{"href":"https:\/\/wordpress.cs.vt.edu\/cs6724s20\/wp-json\/wp\/v2\/posts\/869","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/wordpress.cs.vt.edu\/cs6724s20\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/wordpress.cs.vt.edu\/cs6724s20\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/wordpress.cs.vt.edu\/cs6724s20\/wp-json\/wp\/v2\/users\/105"}],"replies":[{"embeddable":true,"href":"https:\/\/wordpress.cs.vt.edu\/cs6724s20\/wp-json\/wp\/v2\/comments?post=869"}],"version-history":[{"count":1,"href":"https:\/\/wordpress.cs.vt.edu\/cs6724s20\/wp-json\/wp\/v2\/posts\/869\/revisions"}],"predecessor-version":[{"id":870,"href":"https:\/\/wordpress.cs.vt.edu\/cs6724s20\/wp-json\/wp\/v2\/posts\/869\/revisions\/870"}],"wp:attachment":[{"href":"https:\/\/wordpress.cs.vt.edu\/cs6724s20\/wp-json\/wp\/v2\/media?parent=869"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/wordpress.cs.vt.edu\/cs6724s20\/wp-json\/wp\/v2\/categories?post=869"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/wordpress.cs.vt.edu\/cs6724s20\/wp-json\/wp\/v2\/tags?post=869"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}