Summary
The paper argues that users have little to do with application development nowadays. They mention that developers apply machine learning techniques to solve problems but limit their interaction with end users to mediation by practitioners. This results in a long process with multiple iterations which limits the users ability to affect the models. They shed light on the importance of studying users in these systems and present case studies as examples of how these systems could result in better user experiences and more effective learning systems. The authors bring to our attention the advantages of studying user interaction with interactive machine learning systems and some flaws that developers must watch out for. They also present case studies of novel interfaces for interactive machine learning, clarify the different ways that could create richer interactions with users, and emphasize the importance of evaluating them with end users. The authors conclude their paper by underlying that any approach should be appropriately evaluated and tested before deployment since permitting user interactions were often beneficial but not always. They believe that by acknowledging the challenges in this approach, they would produce better machine learning systems as well as better end users.
Reflection
This paper focuses on the importance of the end user’s role in interactive machine learning systems. It raises the questions about how can users effectively influence machine learning systems? and how the machine learning system can appropriately influence the users? The paper also shows case studies that explain how people interact with machine learning systems. In those cases some unexpected results were found, like: people violated assumptions of the machine learning algorithm or they weren’t willing to comply with them. Other cases showed that studies can lead to insights about input and output types that interactive machine learning systems should support. The paper discusses case studies about some novel interfaces for interactive machine learning. Whether the novelty comes from new methods from receiving inputs or giving outputs. They mention that the new input techniques can give users more control over the system while output techniques can make the system more transparent or understandable. The paper does mention though that not all novel interfaces were beneficial, and some certain input and output types lead to obstacles for the user which reduces the accuracy of the learner model. The paper raises a good point about how different end users have different needs and expectations of the systems and therefore, rich interaction techniques must be designed accordingly. I agree with the authors that conducting studies of novel interactive machine learning systems is critical. And that those studies could be the basis of guideline development for future interactive learning systems.
Discussion
- How would you apply interactive machine learning in your project?
- Have you encountered such systems in other research papers you have read?
- What are applications that could benefit from utilizing interactive machine learning systems?
- How would you utilize some case studies suggestions from the paper in a machine learning model rather than the user experience?
Hi Lulwah, good point! I would like to mainly address the 3rd question in the discussion. In nowadays machine learning approaches, the need to include human feedback in the training process is not always there. Take supervised learning for example, labels of the datasets are provided. Through training and testing, machine learning models can achieve fairly good performance in tasks such as image classification and demand prediction. However, a large set of well labelled data is not easy to obtain. In many real-world scenarios, the datasets obtains contain a large amount of noisy samples, which substantially hinder the performance of machine learning models. This challenge can be overcome by getting humans involved in the machine learning process. By incorporating human feedback in the training process, we can obtain accurate classification and prediction results. So I think one of the applications to utilize interactive machine learning is to deal with the data quality issue.
Thank you Fanglan,
I agree. That’s a great example of interactive ML and incorporating it in the training process would yield better input to our models.