Efficient Counterfactual Learning from Bandit Feedback

Authors

  • Yusuke Narita Yale University
  • Shota Yasui Cyberagent
  • Kohei Yata Yale University

DOI:

https://doi.org/10.1609/aaai.v33i01.33014634

Abstract

What is the most statistically efficient way to do off-policy optimization with batch data from bandit feedback? For log data generated by contextual bandit algorithms, we consider offline estimators for the expected reward from a counterfactual policy. Our estimators are shown to have lowest variance in a wide class of estimators, achieving variance reduction relative to standard estimators. We then apply our estimators to improve advertisement design by a major advertisement company. Consistent with the theoretical result, our estimators allow us to improve on the existing bandit algorithm with more statistical confidence compared to a state-of-theart benchmark.

Downloads

Published

2019-07-17

How to Cite

Narita, Y., Yasui, S., & Yata, K. (2019). Efficient Counterfactual Learning from Bandit Feedback. Proceedings of the AAAI Conference on Artificial Intelligence, 33(01), 4634-4641. https://doi.org/10.1609/aaai.v33i01.33014634

Issue

Section

AAAI Technical Track: Machine Learning