“How Much Do You Want to Play God in Controlling the Dataset?”: Data Scientists’ Perspectives on Modifying Data to Address Fairness Concerns

Abstract

Machine-learning (ML) models are often trained on data that embed discrimination and biases. We wondered under what circumstances data-science practitioners would want to modify a model's underlying training data directly, instead of attempting to fix a model learned from biased data. We also wondered what tools and practices they would need to do so. To answer these questions, we conducted scenario-based interviews with 20 practitioners about using data modification methods (DMMs)—pre-processing methods that directly change a model's underlying training data—to address fairness issues. Participants acknowledged pressing fairness issues in the scenario we presented, but were reluctant to violate what they perceived as data science and ML norms by modifying training data. Participants frequently felt that their role (both in the scenario and in their own work contexts) meant that they had little responsibility or agency over the ethical outcomes of their models. Our findings suggest that norms within data science and ML practice can act as a barrier to fairness processes. This work calls for the fairness community to reevaluate the community norms and values that create these barriers and to reconsider the range of valid fairness interventions.