RenewNAT: Renewing Potential Translation for Non-autoregressive Transformer

Authors

  • Pei Guo Soochow University
  • Yisheng Xiao Soochow University
  • Juntao Li Soochow University
  • Min Zhang Soochow University

DOI:

https://doi.org/10.1609/aaai.v37i11.26511

Keywords:

SNLP: Machine Translation & Multilinguality, SNLP: Generation

Abstract

Non-autoregressive neural machine translation (NAT) models are proposed to accelerate the inference process while maintaining relatively high performance. However, existing NAT models are difficult to achieve the desired efficiency-quality trade-off. For one thing, fully NAT models with efficient inference perform inferior to their autoregressive counterparts. For another, iterative NAT models can, though, achieve comparable performance while diminishing the advantage of speed. In this paper, we propose RenewNAT, a flexible framework with high efficiency and effectiveness, to incorporate the merits of fully and iterative NAT models. RenewNAT first generates the potential translation results and then renews them in a single pass. It can achieve significant performance improvements at the same expense as traditional NAT models (without introducing additional model parameters and decoding latency). Experimental results on various translation benchmarks (e.g., 4 WMT) show that our framework consistently improves the performance of strong fully NAT methods (e.g., GLAT and DSLP) without additional speed overhead.

Downloads

Published

2023-06-26

How to Cite

Guo, P., Xiao, Y., Li, J., & Zhang, M. (2023). RenewNAT: Renewing Potential Translation for Non-autoregressive Transformer. Proceedings of the AAAI Conference on Artificial Intelligence, 37(11), 12854-12862. https://doi.org/10.1609/aaai.v37i11.26511

Issue

Section

AAAI Technical Track on Speech & Natural Language Processing