Large-Scale Non-convex Stochastic Constrained Distributionally Robust Optimization
DOI:
https://doi.org/10.1609/aaai.v38i8.28662Keywords:
CSO: Constraint Optimization, SO: Non-convex OptimizationAbstract
Distributionally robust optimization (DRO) is a powerful framework for training robust models against data distribution shifts. This paper focuses on constrained DRO, which has an explicit characterization of the robustness level. Existing studies on constrained DRO mostly focus on convex loss function, and exclude the practical and challenging case with non-convex loss function, e.g., neural network. This paper develops a stochastic algorithm and its performance analysis for non-convex constrained DRO. The computational complexity of our stochastic algorithm at each iteration is independent of the overall dataset size, and thus is suitable for large-scale applications. We focus on the general Cressie-Read family divergence defined uncertainty set which includes chi^2-divergences as a special case. We prove that our algorithm finds an epsilon-stationary point with an improved computational complexity than existing methods. Our method also applies to the smoothed conditional value at risk (CVaR) DRO.Downloads
Published
2024-03-24
How to Cite
Zhang, Q., Zhou, Y., Prater-Bennette, A., Shen, L., & Zou, S. (2024). Large-Scale Non-convex Stochastic Constrained Distributionally Robust Optimization. Proceedings of the AAAI Conference on Artificial Intelligence, 38(8), 8217-8225. https://doi.org/10.1609/aaai.v38i8.28662
Issue
Section
AAAI Technical Track on Constraint Satisfaction and Optimization