Meta Multi-Task Learning for Sequence Modeling

Authors

  • Junkun Chen Fudan University
  • Xipeng Qiu Fudan University
  • Pengfei Liu Fudan University
  • Xuanjing Huang Fudan University

Keywords:

multi-task learning, nature language process, deep learning

Abstract

Semantic composition functions have been playing a pivotal role in neural representation learning of text sequences. In spite of their success, most existing models suffer from the underfitting problem: they use the same shared compositional function on all the positions in the sequence, thereby lacking expressive power due to incapacity to capture the richness of compositionality. Besides, the composition functions of different tasks are independent and learned from scratch. In this paper, we propose a new sharing scheme of composition function across multiple tasks. Specifically, we use a shared meta-network to capture the meta-knowledge of semantic composition and generate the parameters of the task-specific semantic composition models. We conduct extensive experiments on two types of tasks, text classification and sequence tagging, which demonstrate the benefits of our approach. Besides, we show that the shared meta-knowledge learned by our proposed model can be regarded as off-the-shelf knowledge and easily transferred to new tasks.

Downloads

Published

2018-04-27

How to Cite

Chen, J., Qiu, X., Liu, P., & Huang, X. (2018). Meta Multi-Task Learning for Sequence Modeling. Proceedings of the AAAI Conference on Artificial Intelligence, 32(1). Retrieved from https://ojs.aaai.org/index.php/AAAI/article/view/12007