Summary
In this paper, the author proposed and evaluated 18 guidelines for Human-AI Interaction. These guidelines were summarized and distilled through four main stages. The author explained these four phases and present partial results by listing several representative examples. First, the author made exhaustive research on AI design and guidelines from different companies, industries, public articles, and papers. Then, they conducted a modified heuristic evaluation to these guidelines and reflect the results. In the third phase, the author conducted a user study with 49 HCI practitioners to evaluate these guidelines from two main aspects: 1) The broad applicability of guidelines. 2) The semantic intelligibility of guidelines. Finally, the author evaluated and revised the guidelines with experts who have work experience in UX/HCI and familiar with discount usability methods such as heuristic evaluation(from paper). These guidelines are analyzed, adjusted and summarized after each stage based on the results of that stage. In the paper, the author even presents the results of each stage through tables and figures. Finally, the author discussed the scope of these guidelines, as well as issues that he found during the evaluation phases.
Reflection:
The main content of this article is an evaluation of the author’s summary. The evaluation process is divided into three phases. There are many times when we need to evaluate our won hypotheses or conclusions in daily studying and research. Thus, the evaluation process present by the author in this paper has many valuable points that are worth learning.
In the first phases, the author’s original version of the guideline was collected from various aspects. The collection is very comprehensive. It is not limited to published papers or journals but also focuses on existing products and applications.
In the next three phases, each stage of the assessment is very detailed and comprehensive. For example, when the author wants to evaluate whether these guidelines are applicable to AI-infused products, there are only 13 products were inspected. The number is not large, but the function of these products is very representative.
In addition, the personnel involved in the inspection in each phase are professionals with experience in the HCI area, which also ensures the professionalism of the evaluation.
During the evaluation, the author not only focused on the applicability and accuracy of the guidelines but also emphasized the quality of semantic expression. This has a great positive effect on the use and dissemination of the guidelines.
In the final discussion of the article, the author also pointed out the development of AI-infused products should always consider ethical issues instead of just adhere to the design guidelines. I don’ have much comment on this, just suddenly realized that no matter in what area, no matter design what kind of product, it is always linked to ethical issues and bias issues. This is always the most complicated topic.
Questions:
- This paper gives a very detailed user study process and results. Have you ever conducted a standard HCI user-study? What can you learn from the user-study in this paper?
- The original version of the guidelines proposed in this article is based on the existing paper and product design summary. However, this summary is more about AI-design than HCI design. How do you think about this? Do you think they should collect more information about the HCI design principle? Or you think the information collected by the author is adequate enough.
- Do you think the inspection process should include more ordinary AI product users?