Summary
Dodge et al.’s paper “Explaining Models: An Empirical Study of How Explanations Impact Fairness Judgment” presents an empirical study on how people make fairness judgments of machine learning systems and how different styles of explanation impact their judgments. Fairness issues of ML systems attract research interests during recent years. Mitigating the unfairness in ML systems is challenging, which requires the good cooperation of developers, users, and the general public. The researchers state that how explanations are constructed have an impact on users’ confidence in the systems. To further examine the potential impacts on people’s fairness judgments of ML systems, they conduct empirical experiments involving crowdsourcing workers on four types of programmatically generated explanations (influence, demographic-based, sensitivity, and case-based). Their key findings include: 1) some explanations are considered more fair, while others have negative impact on users’ trust of the algorithm in regards of fairness; 2) varied fairness issues (model-wide fairness and case-specific fairness) can be detected more effectively through an examination of different explanation styles; 3) individual differences (prior positions and judgment criteria of algorithmic fairness) lead to how users react to different styles of explanation.
Reflection
This paper shines light on a very important fact that bias in ML systems can be detected and mitigated. There is a growing attention to the fairness issues in AI-powered technologies in the machine learning research community. Since ML algorithms are widely used to speed up the decision making process in a variety of domains, beyond achieving good performance, they are expected to produce neutral results. There is no denying the fact that algorithms rely on data, “garbage in, garbage out.” Hence, it is incumbent to feed the unbiased data to these systems upon developers in the first place. In many real-world cases, race is actually not used as an input, however, it correlates to other factors that make predictions biased. That case is not as easy as the cases presented in the paper to detect but still requires effort to be corrected. A question here would be in order to counteract this implicit bias, should race be considered and used to calibrate the relative importance of other factors?
Besides the bias introduced by data input, there are other factors that need to be taken into consideration to deal with the fairness issues in ML systems. Firstly, machine bias can never be neglected. The term bias in the context of the high-stakes tasks (e.g. future criminal prediction) is very important because a false positive decision could have a destructive impact on a person’s life. This is why when an AI system deals with the human subject (in this case human life), the system must be highly precise and accurate and ideally provide reasonable explanation. Making a person’s life harder to live in a society or impacting badly a person’s life due to a flawed computer model is never acceptable. Secondly, the proprietary model is another concern. One thing should be kept in mind that many high-stacks tasks such as future criminal prediction is a matter of public matter and should be transparent and fair. That does not mean that the ML systems used for those tasks need to be completely public and open. However, I believe there should be a regulatory board of experts who can verify and validate the ML systems. More specifically, the experts can verify and validate the risk factors used in a system so that the factors could be widely accepted. They can also verify and validate the algorithmic techniques used in a system so that the system incorporates less bias.
Discussion
I think the following questions are worthy of further discussion.
- Besides model unfairness and case-specific disparate impact, are there any other fairness issues?
- What are the benefits and drawbacks of global and local explanations in supporting fairness judgment of AI systems?
- Are there any other style or element of explanations that may impact fairness judgement you can think about?
- If an AI system is not any better than untrained users at predicting recidivism in a fair and accurate way, why do we need the system?
I want to make a comment about your first question. I highly agree with your idea that even though the race is not used as input, there are correlated factors that infer to the race. Actually I have read some news recently about one of the algorithms that commonly used nowadays for predicting recidivism called COMPAS. The paper made the conclusion that despite the COMPAS made the prediction with 137 features, the accuracy of the prediction is the same as the prediction from a simple linear predictor with only two features. And there is no more accurate and fair than the prediction made by people with little or no criminal justice expertise. However, there is a lot of data evidence shows that the algorithm is biased and tend to give black people a high score for risk analysis. So I am wondering if the accuracy and fairness of the algorithm have almost no difference with the prediction made by regular people, how can the output still biased? Does the design of the algorithm not to avoid this bias? Or they just reflect the bias from the public.