Adaptive Gradient Methods for Constrained Convex Optimization and Variational Inequalities

Authors

  • Alina Ene Boston University
  • Huy L. Nguyen Northeastern University
  • Adrian Vladu CNRS IRIF

Keywords:

Optimization

Abstract

We provide new adaptive first-order methods for constrained convex optimization. Our main algorithms AdaACSA and AdaAGD+ are accelerated methods, which are universal in the sense that they achieve nearly-optimal convergence rates for both smooth and non-smooth functions, even when they only have access to stochastic gradients. In addition, they do not require any prior knowledge on how the objective function is parametrized, since they automatically adjust their per-coordinate learning rate. These can be seen as truly accelerated Adagrad methods for constrained optimization. We complement them with a simpler algorithm AdaGrad+ which enjoys the same features, and achieves the standard non-accelerated convergence rate. We also present a set of new results involving adaptive methods for unconstrained optimization and variational inequalities arising from monotone operators.

Downloads

Published

2021-05-18

How to Cite

Ene, A., Nguyen, H. L., & Vladu, A. (2021). Adaptive Gradient Methods for Constrained Convex Optimization and Variational Inequalities. Proceedings of the AAAI Conference on Artificial Intelligence, 35(8), 7314-7321. Retrieved from https://ojs.aaai.org/index.php/AAAI/article/view/16898

Issue

Section

AAAI Technical Track on Machine Learning I