The ‘Wild West’ of Medicine: Exploring the Emergence of ‘Grassroots’ AI Governance in Radiology

Abstract

In the past 10 years, a steady increase in clinical AI adoption has been accompanied by concerns regarding potential risks. Thus, there has been a growing body of literature on the regulatory implications of AI devices, and studies exploring clinician attitudes towards AI. However, there has been limited work examining ‘bottom-up’ hospital-level AI governance approaches. To fill this gap, we conducted a qualitative study interviewing 22 healthcare practitioners with AI governance experience within radiology departments and/or professional societies in the US and UK. We aimed to understand the current state of AI adoption and governance, clinician perspectives on responsibility, and the interaction between ‘top-down’ and ‘bottom-up’ governance approaches. Our findings indicate disparities in resources and AI expertise, as well as differences in the scope, composition, remit, and role of AI governance committees across hospitals. Additionally, we uncover emerging challenges in negotiating responsibility norms for AI outcomes and performance monitoring. We also discuss the AI governance roles taken on by some clinicians, often on a voluntary basis, and the challenges they face in navigating siloed, hierarchical organizations. Finally, we analyze participant recommendations, including the development of streamlined guidance on responsible AI adoption, better staff education/training, and centralized approaches to performance monitoring.