Multi-modal User Intent Classification Under the Scenario of Smart Factory (Student Abstract)


  • Yu-Ching Chiu National Central University, Taiwan
  • Bo-Hao Chang National Central University, Taiwan
  • Tzu-Yu Chen National Central University, Taiwan
  • Cheng-Fu Yang National Taiwan University, Taiwan
  • Nanyi Bi National Taiwan University, Taiwan
  • Richard Tzong-Han Tsai National Central University, Taiwan
  • Hung-yi Lee National Taiwan University, Taiwan
  • Jane Yung-jen Hsu National Taiwan University, Taiwan



Dialog System, Multi-modal, User Intent Classification, Human-agent Interaction


Question-answering systems are becoming increasingly popular in Natural Language Processing, especially when applied in smart factory settings. A common practice in designing those systems is through intent classification. However, in a multiple-stage task commonly seen in those settings, relying solely on intent classification may lead to erroneous answers, as questions rising from different work stages may share the same intent but have different contexts and therefore require different answers. To address this problem, we designed an interactive dialogue system that utilizes contextual information to assist intent classification in a multiple-stage task. Specifically, our system incorporates user’s utterances with real-time video feed to better situate users’ questions and analyze their intent.




How to Cite

Chiu, Y.-C., Chang, B.-H., Chen, T.-Y., Yang, C.-F., Bi, N., Tsai, R. T.-H., Lee, H.- yi, & Hsu, J. Y.- jen. (2021). Multi-modal User Intent Classification Under the Scenario of Smart Factory (Student Abstract). Proceedings of the AAAI Conference on Artificial Intelligence, 35(18), 15771-15772.



AAAI Student Abstract and Poster Program