Human-AI Collaborative Sub-Goal Optimization in Hierarchical Reinforcement Learning
DOI:
https://doi.org/10.1609/aaaiss.v1i1.27481Keywords:
Human-AI Collaboration, Hierarchical Reinforcement Learning, Sub-Goal Optimization, Variational InferenceAbstract
Hierarchical reinforcement learning often involves human expertise in defining multiple sub-goals to decompose complex objectives into relevant sub-tasks. However, manually specifying these sub-goals is labor-intensive, costly, and prone to introducing biases or misleading the agent. To overcome these challenges, we propose a collaborative human-AI algorithm that seamlessly integrates with hierarchical models to automatically update prior knowledge and optimize candidate sub-goals. Our algorithm can be easily incorporated into a wide range of goal-conditioned frameworks. We evaluate our approach in comparison with relevant baselines, we demonstrate the effectiveness of our algorithm in addressing and preventing negative inferences arising from confusing or conflicting sub-goals. Additionally, our algorithm shows robustness across different levels of human knowledge, accelerating convergence towards optimal sub-goal spaces and hierarchical policies.Downloads
Published
2023-10-03
Issue
Section
Building Connections: From Human-Human to Human-AI Collaboration