Attention-Informed Mixed-Language Training for Zero-Shot Cross-Lingual Task-Oriented Dialogue Systems

Authors

  • Zihan Liu The Hong Kong University of Science and Technology
  • Genta Indra Winata The Hong Kong University of Science and Technology
  • Zhaojiang Lin The Hong Kong University of Science and Technology
  • Peng Xu The Hong Kong University of Science and Technology
  • Pascale Fung The Hong Kong University of Science and Technology

DOI:

https://doi.org/10.1609/aaai.v34i05.6362

Abstract

Recently, data-driven task-oriented dialogue systems have achieved promising performance in English. However, developing dialogue systems that support low-resource languages remains a long-standing challenge due to the absence of high-quality data. In order to circumvent the expensive and time-consuming data collection, we introduce Attention-Informed Mixed-Language Training (MLT), a novel zero-shot adaptation method for cross-lingual task-oriented dialogue systems. It leverages very few task-related parallel word pairs to generate code-switching sentences for learning the inter-lingual semantics across languages. Instead of manually selecting the word pairs, we propose to extract source words based on the scores computed by the attention layer of a trained English task-related model and then generate word pairs using existing bilingual dictionaries. Furthermore, intensive experiments with different cross-lingual embeddings demonstrate the effectiveness of our approach. Finally, with very few word pairs, our model achieves significant zero-shot adaptation performance improvements in both cross-lingual dialogue state tracking and natural language understanding (i.e., intent detection and slot filling) tasks compared to the current state-of-the-art approaches, which utilize a much larger amount of bilingual data.

Downloads

Published

2020-04-03

How to Cite

Liu, Z., Winata, G. I., Lin, Z., Xu, P., & Fung, P. (2020). Attention-Informed Mixed-Language Training for Zero-Shot Cross-Lingual Task-Oriented Dialogue Systems. Proceedings of the AAAI Conference on Artificial Intelligence, 34(05), 8433-8440. https://doi.org/10.1609/aaai.v34i05.6362

Issue

Section

AAAI Technical Track: Natural Language Processing