A Graph Reasoning Network for Multi-turn Response Selection via Customized Pre-training

Authors

  • Yongkang Liu Northeastern University
  • Shi Feng Northeastern University
  • Daling Wang Northeastern University
  • Kaisong Song Alibaba Group
  • Feiliang Ren Northeastern University
  • Yifei Zhang Northeastern University

Keywords:

Conversational AI/Dialog Systems

Abstract

We investigate response selection for multi-turn conversation in retrieval-based chatbots. Existing studies pay more attention to the matching between utterances and responses by calculating the matching score based on learned features, leading to insufficient model reasoning ability. In this paper, we propose a graph- reasoning network (GRN) to address the problem. GRN first conducts pre-training based on ALBERT using next utterance prediction and utterance order prediction tasks specifically devised for response selection. These two customized pre-training tasks can endow our model with the ability of capturing semantical and chronological dependency between utterances. We then fine-tune the model on an integrated network with sequence reasoning and graph reasoning structures. The sequence reasoning module conducts inference based on the highly summarized context vector of utterance-response pairs from the global perspective. The graph reasoning module conducts the reasoning on the utterance-level graph neural network from the local perspective. Experiments on two conversational reasoning datasets show that our model can dramatically outperform the strong baseline methods and can achieve performance which is close to human-level.

Downloads

Published

2021-05-18

How to Cite

Liu, Y., Feng, S., Wang, D., Song, K., Ren, F., & Zhang, Y. (2021). A Graph Reasoning Network for Multi-turn Response Selection via Customized Pre-training. Proceedings of the AAAI Conference on Artificial Intelligence, 35(15), 13433-13442. Retrieved from https://ojs.aaai.org/index.php/AAAI/article/view/17585

Issue

Section

AAAI Technical Track on Speech and Natural Language Processing II