Successive Halving Top-k Operator

Authors

  • Michał Pietruszka Applica.ai Jagiellonian University, Poland
  • Łukasz Borchmann Applica.ai Poznan University of Technology, Poland
  • Filip Graliński Applica.ai Adam Mickiewicz University, Poland

Keywords:

Top-k, Differentiable Optimization, Approximation Algorithm, Neural Networks, Subset Sampling, Network Architecture

Abstract

We propose a differentiable successive halving method of relaxing the top-k operator, rendering gradient-based optimization possible. The need to perform softmax iteratively on the entire vector of scores is avoided using a tournament-style selection. As a result, a much better approximation of top-k and lower computational cost is achieved compared to the previous approach.

Downloads

Published

2021-05-18

How to Cite

Pietruszka, M., Borchmann, Łukasz, & Graliński, F. (2021). Successive Halving Top-k Operator. Proceedings of the AAAI Conference on Artificial Intelligence, 35(18), 15869-15870. Retrieved from https://ojs.aaai.org/index.php/AAAI/article/view/17931

Issue

Section

AAAI Student Abstract and Poster Program