Regulatory Policies on Ethics Evaluations for Large-Scale AI Systems

Abstract

The potential ethical harms of large ML systems have expanded model evaluation processes in parallel, and a breadth of research proposes new metrics or measurement techniques for model bias and safety, and comprehensive documentation frameworks for practitioners to use. The creation of community standards and governance structures indicates intrinsic motivation towards ethical AI, but to bolster accountability beyond voluntary participation, a patchwork of regulatory proposals relying on audits has emerged, which preside over varying jurisdictions and domains of applicability. Ultimately, changes to regulatory policies involving AI evaluations affects organization decision making, and also impacts deployed models. My research interests and ongoing work explore the dynamics between regulators and practitioners, as well as the competing priorities involved, with the hope of improving our ability to ensure deployed systems are ethical, and facilitating discussion among stakeholders.