Technologies for Reliable AI Test and Evaluation
Keywords:Artificial Intelligence, Machine Learning, Verification And Validation, Test And Evaluation, Trustworthy AI, Reliability, Interfaces, Protocols, Interoperability, Deep Neural Networks, Trusted AI
AbstractArtificial intelligence (AI) is revolutionizing many industries, while at the same time facing challenges to safe and reliable use such as vulnerability to adversarial attacks and data drift. Although many AI test and evaluation (T&E) tools exist, integrating them is difficult. Under a program funded by the Chief Digital and AI Office (CDAO), we are developing a library to simplify the AI T&E process by providing user- and developer-friendly interfaces for composing T&E workflows. We illustrate the effectiveness of this approach with an example that compares clean and perturbed accuracy of two models on a computer vision dataset.
Assured and Trustworthy Human-centered AI (ATHAI)