Teaching Parrots to See Red: Self-Audits of Generative Language Models Overlook Sociotechnical Harms

Abstract

The release of ChatGPT as a “low-key research preview” and its viral growth spurred a gold rush among tech companies marketing generative AI (GenAI) as a universal tool. In 2023, the U.S. secured voluntary commitments from top AI developers, including OpenAI, Google, Meta, and Anthropic, to conduct self-audits ensuring model safety before release. However, these models exhibit widespread biases, including by race and gender, unjustly discriminating against users. To inspect this contradiction, we review ten corporate self-audits, finding a notable absence of real-world use cases in sectors like education, creative works, and public policy. Instead, audits focus on thwarting adversarial consumers in hypothetical scenarios and rely on GenAI models to approximate human impacts. This approach places consumers at risk by impairing the mitigation of representational, allocational, and quality-of-service harms. We conclude with recommendations to address audit gaps and protect GenAI consumers.