ALG Blog 1: Exception to Data Driven Rules
Published on:
Case Study reading:
The Right to Be an Exception to a Data-Driven Rule —
Summary: Decision-subjects are often held responsible for the failures of data-driven rules. The purpose of this case study is to argue that those who are not accurately captured by a data-driven rule have the right to be the exception.
1.) What is a data-driven rule, and what does it mean to be a data-driven exception? Is an exception the same as an error?
‘Data-driven rules” refer to the decision rules that dictate the inputs and outputs of data-driven decision aids. For any given context, there are many possible data-driven rules. Each rule yields a different set of exceptions, that are mostly impossible to know ahead of time. An exception is not an error, simply a gap.
2.) In addition to those listed above, what other factors differentiate data-driven decisions from human ones?
Data-driven decisions are mainly objective, having only the bias programmed into them. Data-driven decisions are generally faster and more efficient than humans. Data-driven rules can adapt and evolve.
3.) Beyond what is discussed above, what are some of the benefits and downsides of individualization?
Individualization moves data-driven rules away from generalization. Reduces statistical discrimination. There are privacy concerns to individualization. Currently, data-driven methods are incapable of individualizing the same ways that humans naturally do. Encourages attention to each decision-subject’s circumstances.
4.) Why is uncertainty so critical to the right to be an exception? When the stakes are high (e.g., in criminal sentencing), is there any evaluation metric (e.g., accuracy) that can justify the use of a data-driven rule without the consideration of uncertainty?
Because, no amount of individualization can remove all the uncertainty in a data-driven rule. This matters when the risk of harm is high. Even the best data-driven models make mistakes. The main barrier is uncertainty. Accuracy can be determined by combining both the epistemic and aleatoric levels of uncertainty. Without uncertainty, it’s hard to make a justified decision. Even with perfect individualization, there’s no way of knowing all unknowable factors if the result of a decision is directly harmful.
New Discussion Question: What are the ethics of replacing human-made decisions with data-driven rules? What are the pros and cons? What ethical issues might arise, and how might you go about answering them?
I find it so strange how some important decisions can ultimately come down to the whims of a model, and since I’m taking an ethics course, I wanted to think about the ethical implications of data-driven rules.
Reflection: The relationship between individualization, discrimination,and harm is very interesting. Before this case study, I never even considered the implications of an exception when data-driven rules are making decisions. It makes me wonder if I myself might be an exception to a few data-driven rules.