Automatically Defining Game Action Spaces for Exploration Using Program Analysis

Abstract

The capability to automatically explore different possible game states and functionality is valuable for the automated test and analysis of computer games. However, automatic exploration requires an exploration agent to be capable of determining and performing the possible actions in game states, for which a model is typically unavailable in games built with traditional game engines. Therefore, existing work on automatic exploration typically either manually defines a game's action space or imprecisely guesses the possible actions. In this paper we propose a program analysis technique compatible with traditional game engines, which automatically analyzes the user input handling logic present in a game to determine a discrete action space corresponding to the possible user inputs, along with the conditions under which the actions are valid, and the relevant user inputs to simulate on the game to perform a chosen action. We implemented a prototype of our approach capable of producing the action spaces of Gym environments for Unity games, then evaluated the exploration performance enabled by our technique for random exploration and exploration via curiosity-driven reinforcement learning agents. Our results show that for most games, our analysis enables exploration performance that matches or exceeds that of manually engineered action spaces, and the analysis is fast enough for real time game play.