Fairness Detectability: Do Explanations Recover Fairness-Relevant Causal Effects?

Abstract

Explainability (XAI) methods can play an essential role in fairness-based auditing of an algorithm. In such cases, an incorrect explanation could have detrimental consequences. In this paper, we begin by categorizing three goals of XAI for fairness: enabling correct epistemic footing, algorithm critique, and enabling change through intervention or policy. We take a pragmatic goal-oriented view where we believe XAI should be evaluated in terms of how well they achieve these goals. Framed by the three fairness goals identified, we 1) use causal data simulations with popular causal and non-causal XAI to highlight limitations through our knowledge of the underlying data generating process, 2) formalize these observations as impossibility propositions for non-causal XAI and 3) propose fairness detectability measures to visualize and quantify the performance of a particular XAI method with respect to a baseline causal effect. We find that non-causal XAI methods are less able to address the goals that fairness asks of explainability, with fairness detectability acting as a useful diagnostic tool to enable informed decision-making about XAI. The paper therefore contributes both empirical and methodological findings. Overall, our work contributes to research on pragmatic XAI as well as work aimed at developing the relationship between fairness and explainability.