Successive Halving Top-k Operator
DOI:
https://doi.org/10.1609/aaai.v35i18.17931Keywords:
Top-k, Differentiable Optimization, Approximation Algorithm, Neural Networks, Subset Sampling, Network ArchitectureAbstract
We propose a differentiable successive halving method of relaxing the top-k operator, rendering gradient-based optimization possible. The need to perform softmax iteratively on the entire vector of scores is avoided using a tournament-style selection. As a result, a much better approximation of top-k and lower computational cost is achieved compared to the previous approach.Downloads
Published
2021-05-18
How to Cite
Pietruszka, M., Borchmann, Łukasz, & Graliński, F. (2021). Successive Halving Top-k Operator. Proceedings of the AAAI Conference on Artificial Intelligence, 35(18), 15869-15870. https://doi.org/10.1609/aaai.v35i18.17931
Issue
Section
AAAI Student Abstract and Poster Program