Debugging a Policy: Automatic Action-Policy Testing in AI Planning

Authors

  • Marcel Steinmetz Saarland University
  • Daniel Fišer Saarland University
  • Hasan Ferit Eniser MPI-SWS
  • Patrick Ferber Saarland University University of Basel
  • Timo P. Gros Saarland University
  • Philippe Heim Saarland University
  • Daniel Höller Saarland University
  • Xandra Schuler Saarland University
  • Valentin Wüstholz ConsenSys
  • Maria Christakis MPI-SWS
  • Jörg Hoffmann Saarland University German Research Center for Artificial Intelligence (DFKI)

Keywords:

Action Policies, Testing, Heuristic Functions

Abstract

Testing is a promising way to gain trust in neural action policies π. Previous work on policy testing in sequential decision making targeted environment behavior leading to failure conditions. But if the failure is unavoidable given that behavior, then π is not actually to blame. For a situation to qualify as a "bug" in π, there must be an alternative policy π' that does better. We introduce a generic policy testing framework based on that intuition. This raises the bug confirmation problem, deciding whether or not a state is a bug. We analyze the use of optimistic and pessimistic bounds for the design of test oracles approximating that problem. We contribute an implementation of our framework in classical planning, experimenting with several test oracles and with random-walk methods generating test states biased to poor policy performance and/or state novelty. We evaluate these techniques on policies π learned with ASNets. We find that they are able to effectively identify bugs in these π, and that our random-walk biases improve over uninformed baselines.

Downloads

Published

2022-06-13

How to Cite

Steinmetz, M., Fišer, D., Eniser, H. F., Ferber, P., Gros, T. P., Heim, P., Höller, D., Schuler, X., Wüstholz, V., Christakis, M., & Hoffmann, J. (2022). Debugging a Policy: Automatic Action-Policy Testing in AI Planning. Proceedings of the International Conference on Automated Planning and Scheduling, 32(1), 353-361. Retrieved from https://ojs.aaai.org/index.php/ICAPS/article/view/19820