Word count: 575
Summary of the Reading
This paper focuses on cataloguing issues that users have with conversational agents and how user expectations of what conversational agents can do differ dramatically from what these systems can actually do. This is where the main issue arises: the difference in perceived and actual usefulness. Conversational agents are the virtual assistants that most of us have on our smartphones. They can do many things, but they will often have trouble with more complicated tasks and they may not be extremely accurate. Many participants in the study said that they would not use their conversation agents to do complicated tasks that required precision, like writing long emails. Other uses assumed that the conversation agents could do something, like book movie tickets, but the system could not accomplish the task for the first few times it was tried. This made the user less likely to try to use those features in the future. This paper lists more of these types of issues and tries to present some solutions to them.
Reflections and Connections
I think that this paper highlights a big problem with conversation agents. It can sometimes be very hard to know what conversational agents can and cannot do. Oftentimes, there is no explicit manual that lists all of the kinds of questions that you can ask or that tells you the limits of what the agent can do. This is unfortunate because being upfront with users is the best way to set expectations to a reasonable level. Conversational agents should do a better job of working expectations into their setups or their instructions.
Companies should also do a better job of telling consumers what these agents can and cannot do. Companies like Apple and Google, some of the companies highlighted in the paper, often build their agents up to be capable of anything. Apple tries to sell you on Siri by promising that it can basically do anything. Apple encourages you to use Siri for as many tasks as you can and advertaties this. But, oftentimes, Siri can’t do everything they imply it can. Or, if Siri can do it, she does it poorly. This compounds the problem even more because it sets user expectations extremely high. Then, users will try to actually use the agents, find out that they can’t do as many things as was advertised, and give up on the system altogether. Companies could do a lot to help solve this problem by just being honest with consumers and saying that there are certain things their agents can do and certain things their agents cannot do.
This is a real problem for people who use these kinds of technologies. When people do not know what kinds of questions the agents can actually answer, they may be more scared to ask any questions, severely limiting the usefulness of the agent. It would vastly improve the user experience if we could solve this issue and make people have more accurate expectations for what conversational agents can do.
Questions
- How can companies better set expectations for their conversational agents?
- Does anyone else have a role to play in educating people on the capabilities of conversation agents besides companies?
- Do we, as computer scientists, have a role to play in educating people about the capabilities of conversational agents?
Hello! I had similar reflections. I also think the big problem with conversational agent is users always don’t know the capability of conversational agent. So,I think for a company, when they set the expectation for a n agent, or propagate the function of agent, they should not exaggerate, do not let people feel upset when they using agent. I think what the company should do now is to cultivate users and improve user satisfaction. When more people use it, there will be more feedback and data that can be collected, at the same time, it can have a good reputation.