Search Engine Guided Neural Machine Translation

Authors

  • Jiatao Gu The University of Hong Kong
  • Yong Wang The University of Hong Kong
  • Kyunghyun Cho New York University
  • Victor O.K. Li The University of Hong Kong

DOI:

https://doi.org/10.1609/aaai.v32i1.12013

Keywords:

Machine Translation, Search Engine, Non-Parametric, Translation Memory

Abstract

In this paper, we extend an attention-based neural machine translation (NMT) model by allowing it to access an entire training set of parallel sentence pairs even after training. The proposed approach consists of two stages. In the first stage –retrieval stage–, an off-the-shelf, black-box search engine is used to retrieve a small subset of sentence pairs from a training set given a source sentence. These pairs are further filtered based on a fuzzy matching score based on edit distance. In the second stage–translation stage–, a novel translation model, called search engine guided NMT (SEG-NMT), seamlessly uses both the source sentence and a set of retrieved sentence pairs to perform the translation. Empirical evaluation on three language pairs (En-Fr, En-De, and En-Es) shows that the proposed approach significantly outperforms the baseline approach and the improvement is more significant when more relevant sentence pairs were retrieved.

Downloads

Published

2018-04-27

How to Cite

Gu, J., Wang, Y., Cho, K., & Li, V. O. (2018). Search Engine Guided Neural Machine Translation. Proceedings of the AAAI Conference on Artificial Intelligence, 32(1). https://doi.org/10.1609/aaai.v32i1.12013