Computing an Efficient Exploration Basis for Learning with Univariate Polynomial Features

Authors

  • Chaitanya Amballa TCS Research, Hyderabad, India
  • Manu K. Gupta Department of Management Studies, Indian Institute of Technology, Roorkee, India
  • Sanjay P. Bhat TCS Research, Hyderabad, India

DOI:

https://doi.org/10.1609/aaai.v35i8.16821

Keywords:

Online Learning & Bandits

Abstract

Barycentric spanners have been used as an efficient exploration basis in online linear optimization problems in a bandit framework. We characterise the barycentric spanner for decision problems in which the cost (or reward) is a polynomial in a single decision variable. Our characterisation of the barycentric spanner is two-fold: we show that the barycentric spanner under a polynomial cost function is the unique solution to a set of nonlinear algebraic equations, as well as the solution to a convex optimization problem. We provide numerical results to show that our method computes the barycentric spanner for the polynomial case significantly faster than the only other known algorithm for the purpose. As an application, we consider a dynamic pricing problem in which the revenue is an unknown polynomial function of the price. We then empirically show that the use of a barycentric spanner to initialise the prior distribution in a Thompson sampling setting leads to lower cumulative regret as compared to standard initialisations. We also illustrate the importance of barycentric spanners in adversarial settings by showing, both theoretically and empirically, that a barycentric spanner achieves the minimax value in a static adversarial linear regression problem where the learner selects the training points while an adversary selects the testing points and controls the variance of the noise corrupting the training samples

Downloads

Published

2021-05-18

How to Cite

Amballa, C., Gupta, M. K., & Bhat, S. P. (2021). Computing an Efficient Exploration Basis for Learning with Univariate Polynomial Features. Proceedings of the AAAI Conference on Artificial Intelligence, 35(8), 6636-6643. https://doi.org/10.1609/aaai.v35i8.16821

Issue

Section

AAAI Technical Track on Machine Learning I