Do Personalized Evaluation Functions Reflect Human Preferences? A Study of Weighted Proximity in Algorithmic Recourse

Abstract

Algorithmic recourse focuses on providing individuals (end-users) negatively affected by ML-based decisions with recourse (actionable recommendations) that help them achieve a more favorable outcome in the future. Within this field, counterfactual explanations (CFEs) have gained popularity as a means of providing actionable recourse to end-users by identifying the minimal changes in their feature values needed to flip the ML model’s prediction to a more desirable one. Almost all prior work in CFEs rests on the assumption that minimizing (weighted) proximity (i.e., weighted L1 norm between the original input and the CFE) is a good proxy metric for capturing real-world human preferences with regard to the evaluation of prospective CFEs. Unfortunately, no prior work has empirically evaluated the effectiveness of weighted proximity in capturing real-world user preferences for evaluating CFEs. To address this research gap, this paper makes three novel contributions. First, we develop a novel two-phase study design that helps us effectively learn feature-specific weights for each individual end-user (in Phase 1) and evaluate the effectiveness of weighted proximity using those learned user-specific weights (in Phase 2). Second, we conduct the first-ever set of user studies (with 101 participants) that focus on evaluating the efficacy of global and personalized weighted proximity in capturing user preferences. Our experimental results show that global weighted proximity aligns with user preferences in only half the cases (53.21%), and personalized weighted proximity is not consistently better (56.91%). Finally, we conduct a series of sensitivity analyses to confirm that our findings are robust across different experimental settings and modeling choices.