Burying the Lead: Adjusting Goals to Manage Functional Limitations of AI Tools in Healthcare

Abstract

Artificial intelligence based tools are being developed for decision support in healthcare, however, they are frequently found to lack the required functionality to achieve the clinical goals for which they were built. This results in wasted time, money and resources for hospitals attempting to implement and operate such tools. To determine how functionality issues can be resolved prior to tool implementation, it is necessary to understand why such tools are being designed and then built. Our research focuses on clinical decision support tools with functionality issues arising from target variable invalidity. In this paper, we analyze published articles which present clinical decision support tool designs related to clinical goals. These tools use machine learning models trained on electronic health record data. We find that design decisions driven by data availability can introduce construct invalidity in clinical decision support tool designs, leading to an inability of the tool to address the clinical goal. We observe that alternative goals to the main clinical goal are used to justify continued development. We show that functional limitations of the tool related to the clinical goal can be obscured by imprecise terminology in the model’s stated functionality. Finally, we highlight the need for reconsidered approaches to dataset creation, defining success criteria, and the reporting and transparency of research outcomes as they relate to clinical goals.