Authors: Nicholas Diakopoulos
Summary
This paper discusses the challenges involved in algorithmic accountability reporting and the reverse engineering approaches used to frame a story. The author interviewed four journalists who have reported on algorithms, and discusses five different case studies to present the methods and challenges involved. Finally, the paper outlines the need for transparency and potential ethical issues.
Reflection
This paper offers great insights into the decision-making process behind the reporting of different algorithms and applications. It is particularly interesting to see the lengths journalists go to figure out the story and the value in reporting. The paper is a great read even for non-technical folks as it introduces the concepts of association, filtering, classification and prioritization with examples that can be understood universally. While discussing the different case studies, the paper manages to paint a picture of the challenges the journalists encountered in a very easy-to-understand manner (e.g. incorrectly determining that Obama’s campaign targeted by age) and therefore, succeeds in showing why reporting on algorithmic accountability is hard!
In most cases, the space for potential input(s) is large enough not to be figured out easily, making the field more challenging. This somehow necessitates using the skills of computational social scientists to conduct additional studies, collect additional data and come up with inferences. The paper makes a great point about reverse engineering offering more insights than directly asking the algorithm developers, as the unintended consequences would never surface without investigating the algorithms in operation. Another case of “we need more longitudinal studies with ecological validity”!
It was very interesting to see the discussion around last-mile interventions at the user interface stages (in case of the autocomplete case). It shows the fact that (some of the) developers are self-aware and therefore, ensure that the user experience is an ethical experience. Even though they may fall short, it’s a good starting point. This also demonstrates why augmenting an existing pipeline (be it data/AI APIs or models) to make it work for the end-user is desirable (something that some of the papers discussed in the class have shown).
The questions around the ethics, as usual, do not have an easy answer — whether the reporting can enable developers to make it difficult to investigate in the future. However, regulations around transparency can go a long way in holding algorithms accountable. The paper does a great job synthesizing the challenges in all the case studies and outlines four high-level points for how algorithms can become transparent.
Questions
- Would you add anything more to the reverse engineering approaches discussed for the different case studies in the paper? Would you have done anything differently?
- If you were to investigate into the power structures of an algorithm, which application/algorithm would you chose? What methods would you follow?
- Any interesting case studies that this paper misses out on?
