Markovian Models Anxious to Stay on the Beaten Path. A Psychology-Grounded Approach to Minimising Exposure to Path Uncertainty

Abstract

Mainstream formalizations of uncertainty in Automated Planning (AP) research tend to revolve on rational probability management and reward maximization, often overlooking psychological factors involved in how humans handle uncertainty, where reward maximization is often discounted in favor of simpler plans yielding to predictable trajectories for the sake of reducing exposure to anxiety. This paper pushes the research frontier by introducing and formalizing the concept of Path Anxiety, which quantifies how much various policies may expose the AP to uncertainty in regards to future trajectories, which we call Path anxiety-sensitive Markov Decision Processes (PAS-MDP). Then, we propose practical PAS-MDP algorithms for generating optimized policies that balance utilitarian reward maximization with path uncertainty minimization. The practical relevance of PAS-MDPs is empirically validated by showing that PAS-MDP policies optimized by our algorithms can dramatically reduce path uncertainty with minimal reduction in utility as well as reducing the number of possible paths the system may reach by several orders of magnitude. The psychological accuracy of PAS-MDPs is validated by showing that the optimized policies replicate human-like anxiety-sensitive behaviors identified by psychology research. Altogether, this research highlights a promising approach towards developing human-friendly AP systems that account for human anxiety in their decision-making processes, hence addressing one of the most central factor of mental health and wellbeing.