The Need for Human-AI Collaborative Methods for Conducting Audits of Machine Learning Models

Abstract

Conducting application audits of ML models is essential for ensuring their safe and responsible deployment, particularly in high-stakes applications. However, the auditing of ML models deployed in domain-specific applications remains largely a manual process, relying on domain experts to identify model errors. The manual nature of the process limits scalability of audits and hinders the discovery of problematic model behaviors. We posit that a human-AI collaborative paradigm is key to conducting effective application audits. In this abstract, we propose a research agenda to develop Human-AI collaborative methods for conducting application audits of ML models.