Evaluating Reader Comprehension of Plan-Based Stories Containing Failed Actions

Authors

  • Rushit Sanghrajka University of Utah
  • R. Michael Young University of Utah

DOI:

https://doi.org/10.1609/aiide.v18i1.21962

Keywords:

Narrative Planning, Question Answering For Stories, QUEST, Reader Comprehension

Abstract

A growing number of algorithms for story planning include the ability to create stories with failed actions -- in particular failed actions that occur because of the mistaken beliefs of the characters attempting them. To date, most of these systems have been evaluated analytically, primarily by comparing their expressive range to prior story generation systems. Empirical evaluation of these systems has been preliminary. In this paper, we outline a general comprehension-based approach to the evaluation of plan-based story generation. We describe how we specialize it for use evaluating story plans containing failed actions, and we describe the design and results of an experiment using this approach to evaluate plot lines produced by HeadSpace, a system that models the beliefs of characters and uses that model to generate plot lines containing actions that are attempted but that fail.

Downloads

Published

2022-10-11

How to Cite

Sanghrajka, R., & Young, R. M. (2022). Evaluating Reader Comprehension of Plan-Based Stories Containing Failed Actions. Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment, 18(1), 179-188. https://doi.org/10.1609/aiide.v18i1.21962