Empirical Regularization for Synthetic Sentence Pairs in Unsupervised Neural Machine Translation

Authors

  • Xi Ai College of Computer Science, Chongqing University
  • Bin Fang College of Computer Science, Chongqing University

Keywords:

Machine Translation & Multilinguality

Abstract

UNMT tackles translation on monolingual corpora in two required languages. Since there is no explicitly cross-lingual signal, pre-training and synthetic sentence pairs are significant to the success of UNMT. In this work, we empirically study the core training procedure of UNMT to analyze the synthetic sentence pairs obtained from back-translation. We introduce new losses to UNMT to regularize the synthetic sentence pairs by jointly training the UNMT objective and the regularization objective. Our comprehensive experiments support that our method can generally improve the performance of currently successful models on three similar pairs {French, German, Romanian} <-> English and one dissimilar pair Russian <-> English with acceptably additional cost.

Downloads

Published

2021-05-18

How to Cite

Ai, X., & Fang, B. (2021). Empirical Regularization for Synthetic Sentence Pairs in Unsupervised Neural Machine Translation. Proceedings of the AAAI Conference on Artificial Intelligence, 35(14), 12471-12479. Retrieved from https://ojs.aaai.org/index.php/AAAI/article/view/17479

Issue

Section

AAAI Technical Track on Speech and Natural Language Processing I