Neural Machine Translation with Gumbel-Greedy Decoding

Authors

  • Jiatao Gu The University of Hong Kong
  • Daniel Jiwoong Im AIFounded Inc.
  • Victor O.K. Li The University of Hong Kong

DOI:

https://doi.org/10.1609/aaai.v32i1.12016

Keywords:

Machine Translation, Gumbel Softmax, Greedy Decoding, Generator Discriminator

Abstract

Previous neural machine translation models used some heuristic search algorithms (e.g., beam search) in order to avoid solving the maximum a posteriori problem over translation sentences at test phase. In this paper, we propose the \textit{Gumbel-Greedy Decoding} which trains a generative network to predict translation under a trained model. We solve such a problem using the Gumbel-Softmax reparameterization, which makes our generative network differentiable and trainable through standard stochastic gradient methods. We empirically demonstrate that our proposed model is effective for generating sequences of discrete words.

Downloads

Published

2018-04-27

How to Cite

Gu, J., Im, D. J., & Li, V. O. (2018). Neural Machine Translation with Gumbel-Greedy Decoding. Proceedings of the AAAI Conference on Artificial Intelligence, 32(1). https://doi.org/10.1609/aaai.v32i1.12016