{"id":810,"date":"2020-03-24T04:48:32","date_gmt":"2020-03-24T04:48:32","guid":{"rendered":"http:\/\/wordpress.cs.vt.edu\/cs6724s20\/?p=810"},"modified":"2020-03-24T23:32:49","modified_gmt":"2020-03-24T23:32:49","slug":"03-25-2020-mohannad-al-ameedi-like-having-a-really-bad-pa-the-gulf-between-user-expectation-and-experience-of-conversational-agents","status":"publish","type":"post","link":"https:\/\/wordpress.cs.vt.edu\/cs6724s20\/2020\/03\/24\/03-25-2020-mohannad-al-ameedi-like-having-a-really-bad-pa-the-gulf-between-user-expectation-and-experience-of-conversational-agents\/","title":{"rendered":"03\/25\/2020 – Mohannad Al Ameedi – \u201cLike Having a Really bad PA\u201d: The Gulf between User Expectation and Experience of Conversational Agents"},"content":{"rendered":"\n

Summary<\/strong><\/h3>\n\n\n\n

In this paper, the authors, try to understand the user experience of conversational agents by examining the factors that motivate users to work with these agents, and also try to propose design considerations that overcome the current limitation and improve human interaction. During their study, they found that there is a huge gap between user expectations and conversational agents\u2019 operations. <\/p>\n\n\n\n

They also found\nthat there are limited studies about how agents are used on a daily bases and\nmost of these studies were not about user experiences and more focus on technical\narchitecture, language learning, and other areas. <\/p>\n\n\n\n

The authors conducted interviews with 14 individuals who use\nconversational agents regularly, and their ages varies from 25 to 60 years. Some\nof these individuals have in depth technical knowledge and the others are\nregular users of technologies.<\/p>\n\n\n\n

They found that the key motivation of using the conversational\nagents was time saving where users ask the CA to execute simple tasks that\nnormally require multiple steps like checking the weather, setting reminders, setting alarms, getting\ndirections. They also found that the users started the engagement through\nplayful interaction like asking the CA to tell them a joke or playing a music. Only\nfew users, who have technical knowledge, reported using these systems on basic work-related\ntasks. <\/p>\n\n\n\n

The user\u2019s interactions were mainly on non-critical tasks and have reported that the agents were not that successful when they are asked to execute complex tasks. The studies shows that users don\u2019t trust conventional agents when it comes to executing critical tasks like sending emails or making a phone calls and they need a visual confirmation to complete these kind tasks. They also mentioned that these systems don\u2019t accept feedback and there are no transparencies of how things are working internally. <\/p>\n\n\n\n

The authors suggest\nconsidering ways reveal system intelligence, reconsidering the interactional\npromise made by humorous engagement, considering how best to indicate\ncapability though interaction, and rethinking system feedback and design goals\nin light of the dominant use case, as areas for future investigation and\ndevelopment. <\/p>\n\n\n\n

Reflection<\/strong><\/h3>\n\n\n\n

I found the results reported\nby the study to be very interesting. Most users learned to use these CA systems\nas they go by trying different words and keywords unit it worked out, and the conversational\nagents failed to have a natural interaction with humans. <\/p>\n\n\n\n

I also thought that companies like Google, Amazon, Microsoft,\nand Facebook have developed conversational systems that can perform much better\nthan answering simple questions and struggling with complex questions, but it appears\nthat is not the case. These companies have developed very sophisticated AI systems\nand services and it seems to me that there are some limitation like\ncomputational power or latency considerations are preventing these systems from\nperforming well. <\/p>\n\n\n\n

I agree with the authors that providing feedback can improve\nhuman interaction with CA systems and communicating the capability can lower\nthe expectation which leads to reducing the gap between the expectation and the\noperation. <\/p>\n\n\n\n

Questions<\/strong><\/h3>\n\n\n\n
  • The authors mentions that\nmost users felt unsure as to whether their conversational agents had a capacity\nto learn, can we use reinforcement learning to help the CA to adapt and learn\nwhile engaging with users in a single session? <\/li>
  • The authors mentioned\nthat CA systems are generally good with simple tasks, but not with complex\ntasks and they are struggling with understanding human requests. Do you think\nthat there are technical limitation or other factors preventing the system from\nperforming well with humans? what are these factors? <\/li>
  • The authors mentioned\nthat most instances, the operation of the CA systems failed to bridge the gap\nbetween user expectation and system operation. If that the case for\nconversational agents, do you think that we are far away from deploying autonomous\ncars, which are far more complicated than CAs, in real time setting since it\nhas direct interaction with environments? <\/li><\/ul>\n","protected":false},"excerpt":{"rendered":"

    Summary In this paper, the authors, try to understand the user experience of conversational agents by examining the factors that motivate users to work with these agents, and also try to propose design considerations that overcome the current limitation and improve human interaction. During their study, they found that there is a huge gap between […]<\/p>\n","protected":false},"author":294,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[88,87],"class_list":["post-810","post","type-post","status-publish","format-standard","hentry","category-uncategorized","tag-badpa","tag-class10"],"jetpack_featured_media_url":"","_links":{"self":[{"href":"https:\/\/wordpress.cs.vt.edu\/cs6724s20\/wp-json\/wp\/v2\/posts\/810","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/wordpress.cs.vt.edu\/cs6724s20\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/wordpress.cs.vt.edu\/cs6724s20\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/wordpress.cs.vt.edu\/cs6724s20\/wp-json\/wp\/v2\/users\/294"}],"replies":[{"embeddable":true,"href":"https:\/\/wordpress.cs.vt.edu\/cs6724s20\/wp-json\/wp\/v2\/comments?post=810"}],"version-history":[{"count":1,"href":"https:\/\/wordpress.cs.vt.edu\/cs6724s20\/wp-json\/wp\/v2\/posts\/810\/revisions"}],"predecessor-version":[{"id":815,"href":"https:\/\/wordpress.cs.vt.edu\/cs6724s20\/wp-json\/wp\/v2\/posts\/810\/revisions\/815"}],"wp:attachment":[{"href":"https:\/\/wordpress.cs.vt.edu\/cs6724s20\/wp-json\/wp\/v2\/media?parent=810"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/wordpress.cs.vt.edu\/cs6724s20\/wp-json\/wp\/v2\/categories?post=810"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/wordpress.cs.vt.edu\/cs6724s20\/wp-json\/wp\/v2\/tags?post=810"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}