Learning of Generalizable and Interpretable Knowledge in Grid-Based Reinforcement Learning Environments


  • Manuel Eberhardinger Hochschule der Medien Stuttgart, Germany University of Tübingen, Germany
  • Johannes Maucher Hochschule der Medien Stuttgart, Germany
  • Setareh Maghsudi University of Tübingen, Germany




Program Synthesis, Imitation Learning, Library Learning, Explainable Reinforcement Learning


Understanding the interactions of agents trained with deep reinforcement learning is crucial for deploying agents in games or the real world. In the former, unreasonable actions confuse players. In the latter, that effect is even more significant, as unexpected behavior cause accidents with potentially grave and long-lasting consequences for the involved individuals. In this work, we propose using program synthesis to imitate reinforcement learning policies after seeing a trajectory of the action sequence. Programs have the advantage that they are inherently interpretable and verifiable for correctness. We adapt the state-of-the-art program synthesis system DreamCoder for learning concepts in grid-based environments, specifically, a navigation task and two miniature versions of Atari games, Space Invaders and Asterix. By inspecting the generated libraries, we can make inferences about the concepts the black-box agent has learned and better understand the agent's behavior. We achieve the same by visualizing the agent's decision-making process for the imitated sequences. We evaluate our approach with different types of program synthesizers based on a search-only method, a neural-guided search, and a language model fine-tuned on code.




How to Cite

Eberhardinger, M., Maucher, J., & Maghsudi, S. (2023). Learning of Generalizable and Interpretable Knowledge in Grid-Based Reinforcement Learning Environments. Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment, 19(1), 203-214. https://doi.org/10.1609/aiide.v19i1.27516