In this paper, the authors focused on\nfairness of machine learning system. As machine learning has been widely\napplied, it is important to let the models judge fairly, which needs the\nevaluation from developers, users and general public. With this aim, the\nauthors conduct an empirical study with different generated explanations to understand\nthe meaning of them towards people\u2019s fairness judgement of ML systems. With an\nexperiment involving 160 MTurk workers, they found that the fairness judgement\nof models is a complicated problem. They found that certain explanations are\nconsidered inherently less fair and others will enhance people\u2019s trust of the\nalgorithm, different fairness problems may be more effectively exposed through different\nstyles of explanation and there are individual differences because each\nperson\u2019s unique background and judgment criteria. Finally, they suggested that\nthe evaluation should support different needs of fairness judgment and consider\nindividual differences.<\/p>\n\n\n\n
Reflections:<\/p>\n\n\n\n
This paper provides me three main thoughts.\nFirstly, we should pay attention to explain the machine learning models.\nSecondly, we should consider different needs when evaluating the fairness of\nthe models. Finally, when train models or design human-AI interactions, we\nshould consider the users\u2019 individual differences.<\/p>\n\n\n\n
For the first point, the models are well\ntrained and can perform well. However, the public may not trust them as they\nknow little about the algorithms. If we can provide them fair and friendly\nexplanations, the public trust in the output of machine learning systems may be\nincreased. Also, if they are provided explanations, they may propose\nsuggestions related to practical situations, which will improve the accuracy\nand fairness of the systems reversely. Due to this, all the machine learning\nsystem developers should pay more attention to write appropriate explanations.<\/p>\n\n\n\n
Secondly, the explanation and the models\nshould consider different needs. For the experienced data scientists, we could\nleave comprehensive explanations to let them able to dig deeper. For the people\nwho are experiencing machine learning system for the first time, we should\nleave user-friendly and easy to understand explanations. For the systems which\nwill be used by users with different backgrounds, we may need to write\ndifferent versions of explanations, for example, one user-instruction and one\ndeveloper instruction.<\/p>\n\n\n\n
For the third point, which is the most\ncomplicated one, it is hard to implement a system which will satisfy people\nwith different judgments. Actually, I am thinking if it is possible to develop\nsystems with interface for users to input their bias, which may solve this\nproblem a little bit. It is not possible to train a model which will satisfy\neveryone\u2019s preference. As a result, we could only train a model which will make\nmost of the users or majority of the public satisfied or just leave a place to let\nour users to select their preferences.<\/p>\n\n\n\n
Questions:<\/p>\n\n\n\n
What kinds of information should we provide\nto the users in the explanations?<\/p>\n\n\n\n
Apart from the crowd-sourcing workers,\nwhich groups of people should also be involved in the survey?<\/p>\n\n\n\n
What kinds of information you would like to\nhave if you need to judge a system?<\/p>\n","protected":false},"excerpt":{"rendered":"
In this paper, the authors focused on fairness of machine learning system. As machine learning has been widely applied, it is important to let the models judge fairly, which needs the evaluation from developers, users and general public. With this aim, the authors conduct an empirical study with different generated explanations to understand the meaning […]<\/p>\n","protected":false},"author":297,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[71,70],"class_list":["post-589","post","type-post","status-publish","format-standard","hentry","category-uncategorized","tag-class6","tag-explainingmodels"],"jetpack_featured_media_url":"","_links":{"self":[{"href":"https:\/\/wordpress.cs.vt.edu\/cs6724s20\/wp-json\/wp\/v2\/posts\/589","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/wordpress.cs.vt.edu\/cs6724s20\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/wordpress.cs.vt.edu\/cs6724s20\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/wordpress.cs.vt.edu\/cs6724s20\/wp-json\/wp\/v2\/users\/297"}],"replies":[{"embeddable":true,"href":"https:\/\/wordpress.cs.vt.edu\/cs6724s20\/wp-json\/wp\/v2\/comments?post=589"}],"version-history":[{"count":1,"href":"https:\/\/wordpress.cs.vt.edu\/cs6724s20\/wp-json\/wp\/v2\/posts\/589\/revisions"}],"predecessor-version":[{"id":590,"href":"https:\/\/wordpress.cs.vt.edu\/cs6724s20\/wp-json\/wp\/v2\/posts\/589\/revisions\/590"}],"wp:attachment":[{"href":"https:\/\/wordpress.cs.vt.edu\/cs6724s20\/wp-json\/wp\/v2\/media?parent=589"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/wordpress.cs.vt.edu\/cs6724s20\/wp-json\/wp\/v2\/categories?post=589"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/wordpress.cs.vt.edu\/cs6724s20\/wp-json\/wp\/v2\/tags?post=589"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}