On Shallow Planning Under Partial Observability - Lefebvre, Randy; Durand, Audrey (Proceedings of the AAAI Conference on Artificial Intelligence, 2025)
Formulating a real-world problem under the ReinforcementLearning framework involves non-trivial design choices, suchas selecting a discount factor for the learning objective (dis-counted cumulative rewards), which articulates the planninghorizon of the agent. This work investigates the impact of thediscount factor on the bias-variance trade-off given structuralparameters of the underlying Markov Decision Process. Ourresults support the idea that a shorter planning horizon mightbe beneficial, especially under partial observability.