Explanation Difference: Bridging Procedural and Distributional Fairness

Abstract

Fairness in Machine Learning (Fair ML) is often presented as a trade-off between predictive performance and equality of predicted values. This view of fairness, commonly referred to as distributional fairness, fails to consider how a model arrives at its predictions. This may lead to Fair ML models evaluating protected groups on differing criteria, creating incentive structures that further perpetuate societal biases. Alternatively, procedural fairness attempts to ensure a fair decision-making process, but often does so at the expense of distributional fairness. In this paper, we propose a new procedural fairness measure, Explanation Difference (EDiff), and further illustrate the importance of treating fairness as a multi-objective optimization problem considering distributional and procedural fairness, and predictive performance. We conduct an extensive experimental evaluation showing 1) the shortcomings of solely optimizing for distributional or procedural fairness, and that 2) our multi-objective approach utilizing EDiff can build fair ML models in both distributional and procedural fairness while retaining strong predictive performance.