Successive Halving Top-k Operator
Keywords:Top-k, Differentiable Optimization, Approximation Algorithm, Neural Networks, Subset Sampling, Network Architecture
AbstractWe propose a differentiable successive halving method of relaxing the top-k operator, rendering gradient-based optimization possible. The need to perform softmax iteratively on the entire vector of scores is avoided using a tournament-style selection. As a result, a much better approximation of top-k and lower computational cost is achieved compared to the previous approach.
How to Cite
Pietruszka, M., Borchmann, Łukasz, & Graliński, F. (2021). Successive Halving Top-k Operator. Proceedings of the AAAI Conference on Artificial Intelligence, 35(18), 15869-15870. Retrieved from https://ojs.aaai.org/index.php/AAAI/article/view/17931
AAAI Student Abstract and Poster Program