Evaluating AI-Based Games through Retellings

Abstract

We propose a new approach to the human-centered evaluation of AI-based games, grounded in the analysis of player retellings of their play experiences. Retellings offer unique insight into dimensions of player experience that can be hard to get at through existing evaluation methods, such as the typical narrative structures that tend to emerge in the player’s mind when they play a particular game; the variety of subjectively experienced narratives that are possible and probable within a particular game; and the ways in which a game supports, or fails to support, the player’s process of narrativization. We used a grounded theory methodology to analyze retellings of play experiences in Civilization VI, Stellaris, and two distinct versions of the research game Prom Week. We also interviewed the creators of several retellings to gain insight into the subjective experience of story construction in collaboration with these games.