Disclosure and Evaluation as Fairness Interventions for General-Purpose AI

Abstract

As generative models are increasingly deployed across diverse settings, fairness interventions must move beyond model-level risk to address system and society-level risk. This expansion is challenging because general-purpose AI (GPAI) is used widely and potential use cases are often not known in advance. As such, we ask: what fairness-related requirements are feasible when fairness is inherently contextual but we lack context for GPAI? We specifically consider the obligations of two major groups: system providers and system deployers. While system providers are natural candidates for regulatory attention, the current state of AI understanding offers limited insight into how upstream fairness harms translate into downstream impacts. Since usage contexts are unknown and fairness is not universally defined across them, rather than imposing context-agnostic requirements, we instead argue for transparency from providers such as to whom they are serving their models and a better understanding of how model development decisions influence fairness. On the other hand, system deployers are closer to real-world contexts. Even if they still lack comprehensive knowledge of all use cases, they can leverage their proximity to end users to address fairness harms in different ways. Here, we argue they should responsibly share information about users and personalization and conduct rigorous evaluations across different levels of fairness. Overall, instead of focusing on enforcing outputs from each group (e.g., that a model has selection rates that are at least 80% representation of each other), we propose a process-oriented breakdown of obligations between system providers and deployers centered on systematic data collection. This allows us to be specific and concrete about the processes even while the contexts remain unknown. Ultimately, this approach can sharpen how we distribute fairness responsibilities and inform more fluid, context-sensitive interventions as AI continues to advance.