Paper Authors: R. Jordon Crouser and Remco Chang

Summary

This paper provides an overview summary of some of the popular systems (back in 2012), which were built around human-computer collaboration. Based on this analysis, the authors uncover different key patterns in human and machine affordances, and propose an affordance-based framework that will help researchers think and strategize better about problems that can benefit from collaboration. Such an affordance-based framework, according to the authors, would enable easy comparison between systems via common metrics (discussed in the paper). In the age of intelligent user interfaces, the paper gives researchers a foundational direction or lens to break down problems and map the solution space in a meaningful manner.

Reflection

1. This paper is a great reference resource for setting some foundational questions on human-computer collaboration – How do we tell if a problem would benefit from a collaborative solution? How do we decide which tasks to delegate to which party, and when? How do we compare different systems solving the same problem? At the same time, it also sets some foundational goals and objectives for a system rooted in human-computer collaboration. The paper illustrates all the concepts through different successful examples of systems, making it easy to visualize the bin in which your (anticipated) research would fit. 
2. This paper makes a great motivating argument about developing systems from the problem space, rather than jumping directly to solutions, which may often lead to investment of significant time and energy into developing inefficient collaboration.
3. The paper makes the case for evolving from a prior established framework (i.e. function allocation) for human-machine systems into the proposed affordance-based one. Even though they proposed this framework in 2012, which is also when deep learning techniques started becoming popular, I feel that this framework is dynamic and broad enough to accommodate the ubiquity of current AI and intelligent user interfaces.
4. Following the paper’s direction of updating theories with technology’s evolution, I would argue for a “sequel” paper to discuss AI affordances as an extension to the machine affordances. This would require an in-depth discussion of the capacities and limitations of state-of-the-art AIs designed for different tasks, some of which currently fall under human affordances, such as visual perception (computer vision), creativity (language models), etc. While AIs may be far from being perfect in these tasks, they still provide imperfect avoidances. Inevitably, this also means re-focusing some of the human affordances described in the paper, and may be part of a bigger question i.e. “what is the role of humans in the age of AI?”. This also pushes the boundaries for what can be achieved with such hybrid interaction, e.g. AI’s last-mile problems [1].
5. Currently, many different algorithms interact with human users via intelligent user interfaces (IUIs), and form a big part of decision-making processes. Over the years, researchers from different communities have pointed out how different algorithms can result in different forms of bias [2, 3] and have pushed for more fairness, accountability, transparency and interpretability of these algorithms in an effort to mitigate these biases. The paper, based in 2012, did not account for algorithms within machine affordances, and thus considered bias-free analysis as a machine affordance. 8 years later, to be able to detect biases still remains as somewhat more of a human affordance.

Questions

1. Now, in 2020, how would you expand upon the machine affordances discussed in the paper?
2. Does AI fit under machine affordances, or deserves a separate section – AI affordances? What kind of affordances does AI provide humans, and vice-a-versa? In other words, how do you envision this paper in current times? 
3. For the folks working on AI or ML systems, is it possible for you to present the inaccuracies of the algorithms you are working on in descriptive, qualitative terms? Do you see human cognition, be it through novice or expert workers, as competent enough to fill in the gaps?
4. Does this paper change the way you view your proposed project? If so, how does it change from before? Is it more in terms of how you present your paper?

Read More

In this paper, the author reviews more than 1200 papers to identify how to best utilize human-machine collaboration. Their field of study was visual analytics, but the paper was well-generalized to fit many other research areas. The paper discusses two foundational factors to consider when designing a human-machine collaborative system: Allocation and affordance. In the many papers that the authors reviewed, systematic methods of trying to appropriately allocate work for each human and computer in a collaborative setting was studied. A good rule was introduced by Fitts, but it was found outdated later due to the increasing computational power of machines. The paper decides that inspecting affordance rather than allocation is a better way to utilize human-machine collaborative systems. Affordance can be best understood as what something an agent is good at than others. For example, humans can provide excellent visual processing skills while computers accel at large-data processing. The paper also introduces some case studies where multiple affordances from each party was utilized.

I greatly enjoyed reading about each of the affordances that human and machine can each provide. The list of affordances that the paper provides will serve as a good resource to come back to when trying to design a human-machine collaborative system. One machine affordance that I do not agree with is bias-free analysis. In machine learning scenarios, a learning model is very often easily biased. Both humans and machines can be biased in analyzing something based on previous experience or data. Of course, it is the responsibility of the designer of the system to ensure unbiased models, but as the designer is a human, it is often impossible to avoid bias of some kind. The case study regarding the reCAPTCHA system was an interesting read. I always thought that CAPTCHAs were only used for security purposes, and not machine learning. After learning how it is actually used, I was impressed how efficient and effective the system is at both securing Internet access as well as digitalizing physical books.

The followings are the questions that I came up with while reading the paper:

1. The paper does a great job at summarizing what each a human and a machine is relatively good at. The designer, therefore, simply needs to select appropriate tasks from the system to assign to each human and machine. Is there a good way to identify what affordance the system’s task needs?

2. There’s another thing that humans are really good at compared to a machine: adapting. Machines, upon their initial programming, does not change their response to an event according to time and era while humans very much do. Is there a human-machine collaborative system that would have a task which would require the affordance “adaptation” from a human collaborator?

3. Many human-machine collaborative systems register the tasks that needs to be processed using an automated machine. For example, the reCAPTCHA system (the machine) samples a question and asks the human user to process it. What if it was the other way around where a human register a task and assigns the task to either a machine or a human collaborator? Would there be any benefits to doing that?

Read More