MultiTalk: A Highly-Branching Dialog Testbed for Diverse Conversations

Authors

  • Yao Dou University of Washington
  • Maxwell Forbes University of Washington Allen Institute for AI
  • Ari Holtzman University of Washington
  • Yejin Choi University of Washington Allen Institute for AI

DOI:

https://doi.org/10.1609/aaai.v35i14.17510

Keywords:

Conversational AI/Dialog Systems

Abstract

We study conversational dialog in which there are many possible responses to a given history. We present the MultiTalk Dataset, a corpus of over 320,000 sentences of written conversational dialog that balances a high branching factor (10) with several conversation turns (6) through selective branch continuation. We make multiple contributions to study dialog generation in the highly branching setting. In order to evaluate a diverse set of generations, we propose a simple scoring algorithm, based on bipartite graph matching, to optimally incorporate a set of diverse references. We study multiple language generation tasks at different levels of predictive conversation depth, using textual attributes induced automatically from pretrained classifiers. Our culminating task is a challenging theory of mind problem, a controllable generation task which requires reasoning about the expected reaction of the listener.

Downloads

Published

2021-05-18

How to Cite

Dou, Y., Forbes, M., Holtzman, A., & Choi, Y. (2021). MultiTalk: A Highly-Branching Dialog Testbed for Diverse Conversations. Proceedings of the AAAI Conference on Artificial Intelligence, 35(14), 12760-12767. https://doi.org/10.1609/aaai.v35i14.17510

Issue

Section

AAAI Technical Track on Speech and Natural Language Processing I