{"id":549,"date":"2020-02-22T23:11:19","date_gmt":"2020-02-22T23:11:19","guid":{"rendered":"http:\/\/wordpress.cs.vt.edu\/cs6724s20\/?p=549"},"modified":"2020-02-25T21:23:45","modified_gmt":"2020-02-25T21:23:45","slug":"02-26-20-myles-frantz-will-you-accept-an-imperfect-ai-exploring-designs-for-adjusting-end-user-expectations-of-ai-systems","status":"publish","type":"post","link":"https:\/\/wordpress.cs.vt.edu\/cs6724s20\/2020\/02\/22\/02-26-20-myles-frantz-will-you-accept-an-imperfect-ai-exploring-designs-for-adjusting-end-user-expectations-of-ai-systems\/","title":{"rendered":"02\/26\/20 \u2013 Myles Frantz \u2013 Will You Accept an Imperfect AI? Exploring Designs for Adjusting End-user Expectations of AI Systems"},"content":{"rendered":"\n<h2 class=\"wp-block-heading\">Summation<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Though Machine Learning techniques have advanced greatly within the past few years, human perception on the technology may severely limit the adoption and usage of the technology. To further study how to better encourage and describe the process of using this technology, this team has created a Scheduling Assistant to better monitor and elicit how users would tune and react to from different expectations. Tuning the expectations of the AI (Scheduling Assistant) via a slider (from \u201cFewer detections\u201d on the left to \u201cMore detections\u201d on the right) directly altering the false positive and false negative settings in the AI. This direct feedback loop gave users (mechanical turk workers) more confidence and a better understanding of how the AI works. Though given the various variety of users, having an AI focused on High Precision was not the \u00a0best general setting for the scheduling assistant.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Response<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">I like this kind of raw collaboration between the user and\nthe Machine Learning system. This tailors the algorithm explicitly to the tendencies\nand mannerisms of each user, allowing easier customization and thus a higher\nlikelihood of usage. This is supported due to the team\u2019s Research hypothesis: \u201cH1.1\nAn AI system focused on High Precision \u2026 will result in higher perceptions of\naccuracy \u2026\u201d. In this example each user (mechanical turk worker) was only using\nthe subset of Enron emails to confirm or deny the meeting suggestions. Speculating\nfurther, this type of system being used in an expansive system across many\nusers, being able to tune the AI would greatly encourage use.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">I also strongly agree with the slider bar for ease of use\ntuning by the individual. In this format the user does not neat to have great technological\nskill to be able to use it, and it is relatively fast to use. Having it within\nthe same system easily reachable also ensures a better connection between the\nuser and the AI.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Questions<\/h2>\n\n\n\n<ul class=\"wp-block-list\"><li>I would like to see a greater (and or a beta) study\ndone with a giant email provider. Each email provider likely has their own\nhomegrown Machine Learning model, however providing the capabilities to further\ntune their own AI for their preferences and tendencies would be a great\nimprovement. The only issue would be with the scalability and providing enough\nservices to make this work for all the users.<\/li><li>In tuning the ease of access and usability, I\nwould like to see a study done comparing the different types of interaction\ntools (sliders, buttons, likert scale settings, etc.\u2026). There likely is a study\ndone about the effectiveness of each type of interaction tool upon a system,\nhowever in the context of AI settings it is imperative to have the best tool.\nThis would hopefully be an adopted standard that would be an easy to use tool\naccessible by everyone.<\/li><li>Following along with the first question, I would\nlike to see this kind of study provided to an internal mailing system,\npotentially at an academic level. Though this was studied with 150 mechanical\nturk workers and 400 internally provided workers, this was based on a\nsub-sample on the Enron email dataset. Providing this as a live-beta test in a\nwidely and actively used email system with live emails would be a true test that\nI would like to see.<\/li><\/ul>\n","protected":false},"excerpt":{"rendered":"<p>Summation Though Machine Learning techniques have advanced greatly within the past few years, human perception on the technology may severely limit the adoption and usage of the technology. To further study how to better encourage and describe the process of using this technology, this team has created a Scheduling Assistant to better monitor and elicit [&hellip;]<\/p>\n","protected":false},"author":289,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[71,68],"class_list":["post-549","post","type-post","status-publish","format-standard","hentry","category-uncategorized","tag-class6","tag-imperfectai"],"jetpack_featured_media_url":"","_links":{"self":[{"href":"https:\/\/wordpress.cs.vt.edu\/cs6724s20\/wp-json\/wp\/v2\/posts\/549","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/wordpress.cs.vt.edu\/cs6724s20\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/wordpress.cs.vt.edu\/cs6724s20\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/wordpress.cs.vt.edu\/cs6724s20\/wp-json\/wp\/v2\/users\/289"}],"replies":[{"embeddable":true,"href":"https:\/\/wordpress.cs.vt.edu\/cs6724s20\/wp-json\/wp\/v2\/comments?post=549"}],"version-history":[{"count":1,"href":"https:\/\/wordpress.cs.vt.edu\/cs6724s20\/wp-json\/wp\/v2\/posts\/549\/revisions"}],"predecessor-version":[{"id":551,"href":"https:\/\/wordpress.cs.vt.edu\/cs6724s20\/wp-json\/wp\/v2\/posts\/549\/revisions\/551"}],"wp:attachment":[{"href":"https:\/\/wordpress.cs.vt.edu\/cs6724s20\/wp-json\/wp\/v2\/media?parent=549"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/wordpress.cs.vt.edu\/cs6724s20\/wp-json\/wp\/v2\/categories?post=549"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/wordpress.cs.vt.edu\/cs6724s20\/wp-json\/wp\/v2\/tags?post=549"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}