Infusing Multi-Source Knowledge with Heterogeneous Graph Neural Network for Emotional Conversation Generation

Authors

  • Yunlong Liang Beijing Jiaotong University, Beijing, China
  • Fandong Meng Tencent WeChat AI - Pattern Recognition Center Tencent Inc, Beijing, China
  • Ying Zhang Beijing Jiaotong University, Beijing, China
  • Yufeng Chen Beijing Jiaotong University, Beijing, China
  • Jinan Xu Beijing Jiaotong University, Beijing, China
  • Jie Zhou Tencent WeChat AI - Pattern Recognition Center Tencent Inc, Beijing, China

Keywords:

Conversational AI/Dialog Systems, Generation

Abstract

The success of emotional conversation systems depends on sufficient perception and appropriate expression of emotions. In a real-world conversation, we firstly instinctively perceive emotions from multi-source information, including the emotion flow of dialogue history, facial expressions, and personalities of speakers, and then express suitable emotions according to our personalities, but these multiple types of information are insufficiently exploited in emotional conversation fields. To address this issue, we propose a heterogeneous graph-based model for emotional conversation generation. Specifically, we design a Heterogeneous Graph-Based Encoder to represent the conversation content (i.e., the dialogue history, its emotion flow, facial expressions, and speakers' personalities) with a heterogeneous graph neural network, and then predict suitable emotions for feedback. After that, we employ an Emotion-Personality-Aware Decoder to generate a response not only relevant to the conversation context but also with appropriate emotions, by taking the encoded graph representations, the predicted emotions from the encoder and the personality of the current speaker as inputs. Experimental results show that our model can effectively perceive emotions from multi-source knowledge and generate a satisfactory response, which significantly outperforms previous state-of-the-art models.

Downloads

Published

2021-05-18

How to Cite

Liang, Y., Meng, F., Zhang, Y., Chen, Y., Xu, J., & Zhou, J. (2021). Infusing Multi-Source Knowledge with Heterogeneous Graph Neural Network for Emotional Conversation Generation. Proceedings of the AAAI Conference on Artificial Intelligence, 35(15), 13343-13352. Retrieved from https://ojs.aaai.org/index.php/AAAI/article/view/17575

Issue

Section

AAAI Technical Track on Speech and Natural Language Processing II