Integrating Verbal and Nonvebval Input into a Dynamic Response Spoken Dialogue System

Authors

  • Ting-Yao Hu Carnegie Mellon University
  • Chirag Raman Carnegie Mellon University
  • Salvador Medina Maza Carnegie Mellon University
  • Liangke Gui Carnegie Mellon University
  • Tadas Baltrusaitis Carnegie Mellon University
  • Robert Frederking Carnegie Mellon University
  • Louis-Philippe Morency Carnegie Mellon University
  • Alan Black Carnegie Mellon University
  • Maxine Eskenazi Carnegie Mellon University

DOI:

https://doi.org/10.1609/aaai.v31i1.10538

Keywords:

nonverbal language understanding, dynamic natural language generation, email summarization, emotion detection

Abstract

In this work, we present a dynamic response spoken dialogue system (DRSDS). It is capable of understanding the verbal and nonverbal language of users and making instant, situation-aware response. Incorporating with two external systems, MultiSense and email summarization, we built an email reading agent on mobile device to show the functionality of DRSDS.

AAAI-2017 Proceedings Cover

Downloads

Published

2017-02-12

How to Cite

Hu, T.-Y., Raman, C., Maza, S. M., Gui, L., Baltrusaitis, T., Frederking, R., Morency, L.-P., Black, A., & Eskenazi, M. (2017). Integrating Verbal and Nonvebval Input into a Dynamic Response Spoken Dialogue System. Proceedings of the AAAI Conference on Artificial Intelligence, 31(1). https://doi.org/10.1609/aaai.v31i1.10538

Issue

Vol. 31 No. 1 (2017): Thirty-First AAAI Conference on Artificial Intelligence

Section

Demonstrations