Technologies for Reliable AI Test and Evaluation

Authors

  • Lei Hamilton MIT Lincoln Laboratory
  • Garrett Botkin MIT Lincoln Laboratory
  • Olivia Brown MIT Lincoln Laboratory
  • Justin Goodwin MIT Lincoln Laboratory
  • Michael Yee MIT Lincoln Laboratory
  • Vincent Mancuso MIT Lincoln Laboratory
  • Sanjeev Mohindra MIT Lincoln Laboratory

DOI:

https://doi.org/10.1609/aaaiss.v2i1.27679

Keywords:

Artificial Intelligence, Machine Learning, Verification And Validation, Test And Evaluation, Trustworthy AI, Reliability, Interfaces, Protocols, Interoperability, Deep Neural Networks, Trusted AI

Abstract

Artificial intelligence (AI) is revolutionizing many industries, while at the same time facing challenges to safe and reliable use such as vulnerability to adversarial attacks and data drift. Although many AI test and evaluation (T&E) tools exist, integrating them is difficult. Under a program funded by the Chief Digital and AI Office (CDAO), we are developing a library to simplify the AI T&E process by providing user- and developer-friendly interfaces for composing T&E workflows. We illustrate the effectiveness of this approach with an example that compares clean and perturbed accuracy of two models on a computer vision dataset.
AAAI Fall Symposium 2023 Proceedings Cover

Downloads

Published

2024-01-22

How to Cite

Hamilton, L., Botkin, G., Brown, O., Goodwin, J., Yee, M., Mancuso, V., & Mohindra, S. (2024). Technologies for Reliable AI Test and Evaluation. Proceedings of the AAAI Symposium Series, 2(1), 233–235. https://doi.org/10.1609/aaaiss.v2i1.27679

Issue

Vol. 2 No. 1: Proceedings of the 2023 AAAI Fall Symposia

Section

Assured and Trustworthy Human-centered AI (ATHAI)