Dialogue Generation: From Imitation Learning to Inverse Reinforcement Learning


  • Ziming Li University of Amsterdam
  • Julia Kiseleva University of Amsterdam
  • Maarten de Rijke University of Amsterdam




The performance of adversarial dialogue generation models relies on the quality of the reward signal produced by the discriminator. The reward signal from a poor discriminator can be very sparse and unstable, which may lead the generator to fall into a local optimum or to produce nonsense replies. To alleviate the first problem, we first extend a recently proposed adversarial dialogue generation method to an adversarial imitation learning solution. Then, in the framework of adversarial inverse reinforcement learning, we propose a new reward model for dialogue generation that can provide a more accurate and precise reward signal for generator training. We evaluate the performance of the resulting model with automatic metrics and human evaluations in two annotation settings. Our experimental results demonstrate that our model can generate more high-quality responses and achieve higher overall performance than the state-of-the-art.




How to Cite

Li, Z., Kiseleva, J., & de Rijke, M. (2019). Dialogue Generation: From Imitation Learning to Inverse Reinforcement Learning. Proceedings of the AAAI Conference on Artificial Intelligence, 33(01), 6722-6729. https://doi.org/10.1609/aaai.v33i01.33016722



AAAI Technical Track: Natural Language Processing