Guidelines for Action Space Definition in Reinforcement Learning-Based Traffic Signal Control Systems

Authors

  • Maxime Treca Institut Vedecom
  • Julian Garbiso Institut Vedecom
  • Dominique Barth David Lab UVSQ

Abstract

Previous works in the field of reinforcement learning applied to traffic signal control (RL-TSC) have focused on optimizing state and reward definitions, leaving the impact of the agent's action space definition largely unexplored. In this paper, we compare different types of TSC controllers – phase-based and step-based – in a simulated network featuring different traffic demand patterns in order to provide guidelines for optimally defining RL-TSC actions. Our results show that an agent's performance and convergence speed both increase with its interaction frequency with the environment. However, certain methods with lower observation frequencies – that can be achieved with realistic sensing technologies – have reasonably similar performance compared to higher frequency ones in all scenarios, and even outperform them under specific traffic conditions.

Downloads

Published

2020-06-01

How to Cite

Treca, M., Garbiso, J., & Barth, D. (2020). Guidelines for Action Space Definition in Reinforcement Learning-Based Traffic Signal Control Systems. Proceedings of the International Conference on Automated Planning and Scheduling, 30(1), 585-589. Retrieved from https://ojs.aaai.org/index.php/ICAPS/article/view/6755