Auditing and Validating Fairness and Ethics in Machine Learning Systems

Abstract

Ethics are embedded in the choices we make every day, whether they are in our personal interactions or when designing computational systems. Ethics should be seen as not just a philosophical pursuit or a legal construct of jurisprudence , instead it is a framework for operating and decision-making. In the context of machine learning, ethical concerns emerge in subtle but rife ways: from how data is collected and labeled, to who defines the contextual fairness. After all, the word 'fair' can mean different things; conforming with the established rules or marked by impartiality and honesty. But, what does it mean in the context of computational frameworks? Despite the frequent framing of ethical concerns and fairness as technical problems, issues like fairness are rooted in value judgments made by real humans. Decisions about inclusion, categorization, and weighting reflect the assumptions and priorities of the people behind the algorithms. At scale, these decisions don’t disappear into abstraction. Instead, they shape outcomes, influencing who benefits from, or is harmed by, automated systems. While ethical theory and legal scholarship have long tackled questions of justice and fairness, computer science often treats these topics as secondary to performance metrics or efficiency. These disconnects risk minimizing the real-world stakes of algorithmic decision-making. My research aims to bridge this gap by examining how fairness interventions can be examined from a broader perspective through the lens of algorithmic auditing. I aim to study how fairness interventions efforts align with the stated goals of the audit framework, and whether they meaningfully address the inequities they aim to solve. The goal should not be to reduce fairness to a checkbox, but to understand it in the context of the data and the broader society the system is embedded within.