Technologies for Reliable AI Test and Evaluation

Authors

  • Lei Hamilton MIT Lincoln Laboratory
  • Garrett Botkin MIT Lincoln Laboratory
  • Olivia Brown MIT Lincoln Laboratory
  • Justin Goodwin MIT Lincoln Laboratory
  • Michael Yee MIT Lincoln Laboratory
  • Vincent Mancuso MIT Lincoln Laboratory
  • Sanjeev Mohindra MIT Lincoln Laboratory

DOI:

https://doi.org/10.1609/aaaiss.v2i1.27679

Keywords:

Artificial Intelligence, Machine Learning, Verification And Validation, Test And Evaluation, Trustworthy AI, Reliability, Interfaces, Protocols, Interoperability, Deep Neural Networks, Trusted AI

Abstract

Artificial intelligence (AI) is revolutionizing many industries, while at the same time facing challenges to safe and reliable use such as vulnerability to adversarial attacks and data drift. Although many AI test and evaluation (T&E) tools exist, integrating them is difficult. Under a program funded by the Chief Digital and AI Office (CDAO), we are developing a library to simplify the AI T&E process by providing user- and developer-friendly interfaces for composing T&E workflows. We illustrate the effectiveness of this approach with an example that compares clean and perturbed accuracy of two models on a computer vision dataset.

Downloads

Published

2024-01-22

Issue

Section

Assured and Trustworthy Human-centered AI (ATHAI)