Online Optimal Control with Affine Constraints

Authors

  • Yingying Li John A. Paulson School of Engineering and Applied Sciences, Harvard University
  • Subhro Das MIT-IBM Watson AI Lab, IBM Research
  • Na Li John A. Paulson School of Engineering and Applied Sciences, Harvard University

Keywords:

Online Learning & Bandits, Reinforcement Learning, Constraint Optimization

Abstract

This paper considers online optimal control with affine constraints on the states and actions under linear dynamics with bounded random disturbances. The system dynamics and constraints are assumed to be known and time invariant but the convex stage cost functions change adversarially. To solve this problem, we propose Online Gradient Descent with Buffer Zones (OGD-BZ). Theoretically, we show that OGD-BZ with proper parameters can guarantee the system to satisfy all the constraints despite any admissible disturbances. Further, we investigate the policy regret of OGD-BZ, which compares OGD-BZ's performance with the performance of the optimal linear policy in hindsight. We show that OGD-BZ can achieve a policy regret upper bound that is square root of the horizon length multiplied by some logarithmic terms of the horizon length under proper algorithm parameters.

Downloads

Published

2021-05-18

How to Cite

Li, Y., Das, S., & Li, N. (2021). Online Optimal Control with Affine Constraints. Proceedings of the AAAI Conference on Artificial Intelligence, 35(10), 8527-8537. Retrieved from https://ojs.aaai.org/index.php/AAAI/article/view/17035

Issue

Section

AAAI Technical Track on Machine Learning III