Evaluating Plan-Property Dependencies: A Web-Based Platform and User Study

Authors

  • Rebecca Eifler Saarland University
  • Martim Brandao King's College London
  • Amanda Coles King's College London
  • Jeremy Frank NASA Ames Research Center
  • Jörg Hoffmann Saarland University German Research Center for Artificial Intelligence (DFKI)

Keywords:

XAIP, Plan-property Dependencies, User Study

Abstract

The trade-offs between different desirable plan properties -- e.g. PDDL temporal plan preferences -- are often difficult to understand. Recent work addresses this by iterative planning with explanations elucidating the dependencies between such plan properties. Users can ask questions of the form ``Why does the plan not satisfy property p?'', which are answered by ``Because then we would have to forego q''. It has been shown that such dependencies can be computed reasonably efficiently. But is this form of explanation actually useful for users? We run a large crowd-worker user study (N=100 in each of 3 domains) evaluating that question. To enable such a study in the first place, we contribute a Web-based platform for iterative planning with explanations, running in standard browsers. Comparing users with vs. without access to the explanations, we find that the explanations enable users to identify better trade-offs between the plan properties, indicating an improved understanding of the planning task.

Downloads

Published

2022-06-13

How to Cite

Eifler, R., Brandao, M., Coles, A., Frank, J., & Hoffmann, J. (2022). Evaluating Plan-Property Dependencies: A Web-Based Platform and User Study. Proceedings of the International Conference on Automated Planning and Scheduling, 32(1), 687-691. Retrieved from https://ojs.aaai.org/index.php/ICAPS/article/view/19858