Queue-Learning: A Reinforcement Learning Approach for Providing Quality of Service

Authors

  • Majid Raeis University of Toronto, Canada
  • Ali Tizghadam University of Toronto, Canada
  • Alberto Leon-Garcia University of Toronto, Canada

Keywords:

Cloud, Planning/Scheduling and Learning, Internet of Things, Sensor Networks & Smart Cities

Abstract

End-to-end delay is a critical attribute of quality of service (QoS) in application domains such as cloud computing and computer networks. This metric is particularly important in tandem service systems, where the end-to-end service is provided through a chain of services. Service-rate control is a common mechanism for providing QoS guarantees in service systems. In this paper, we introduce a reinforcement learning-based (RL-based) service-rate controller that provides probabilistic upper-bounds on the end-to-end delay of the system, while preventing the overuse of service resources. In order to have a general framework, we use queueing theory to model the service systems. However, we adopt an RL-based approach to avoid the limitations of queueing-theoretic methods. In particular, we use Deep Deterministic Policy Gradient (DDPG) to learn the service rates (action) as a function of the queue lengths (state) in tandem service systems. In contrast to existing RL-based methods that quantify their performance by the achieved overall reward, which could be hard to interpret or even misleading, our proposed controller provides explicit probabilistic guarantees on the end-to-end delay of the system. The evaluations are presented for a tandem queueing system with non-exponential inter-arrival and service times, the results of which validate our controller's capability in meeting QoS constraints.

Downloads

Published

2021-05-18

How to Cite

Raeis, M., Tizghadam, A., & Leon-Garcia, A. (2021). Queue-Learning: A Reinforcement Learning Approach for Providing Quality of Service. Proceedings of the AAAI Conference on Artificial Intelligence, 35(1), 461-468. Retrieved from https://ojs.aaai.org/index.php/AAAI/article/view/16123

Issue

Section

AAAI Technical Track on Application Domains