Robust-MSA: Understanding the Impact of Modality Noise on Multimodal Sentiment Analysis

Authors

  • Huisheng Mao State Key Laboratory of Intelligent Technology and Systems, Department of Computer Science and Technology, Tsinghua University, Beijing 100084, China Beijing National Research Center for Information Science and Technology(BNRist), Beijing 100084, China
  • Baozheng Zhang State Key Laboratory of Intelligent Technology and Systems, Department of Computer Science and Technology, Tsinghua University, Beijing 100084, China School of Information Science and Engineering, Hebei University of Science and Technology, Shijiazhuang 050018, China
  • Hua Xu State Key Laboratory of Intelligent Technology and Systems, Department of Computer Science and Technology, Tsinghua University, Beijing 100084, China Beijing National Research Center for Information Science and Technology(BNRist), Beijing 100084, China
  • Ziqi Yuan State Key Laboratory of Intelligent Technology and Systems, Department of Computer Science and Technology, Tsinghua University, Beijing 100084, China Beijing National Research Center for Information Science and Technology(BNRist), Beijing 100084, China
  • Yihe Liu State Key Laboratory of Intelligent Technology and Systems, Department of Computer Science and Technology, Tsinghua University, Beijing 100084, China School of Information Science and Engineering, Hebei University of Science and Technology, Shijiazhuang 050018, China

DOI:

https://doi.org/10.1609/aaai.v37i13.27078

Keywords:

Multimodal Sentiment Analysis, Robustness Against Noise, Multimodal Applications

Abstract

Improving model robustness against potential modality noise, as an essential step for adapting multimodal models to real-world applications, has received increasing attention among researchers. For Multimodal Sentiment Analysis (MSA), there is also a debate on whether multimodal models are more effective against noisy features than unimodal ones. Stressing on intuitive illustration and in-depth analysis of these concerns, we present Robust-MSA, an interactive platform that visualizes the impact of modality noise as well as simple defence methods to help researchers know better about how their models perform with imperfect real-world data.
AAAI-23 Proceedings Cover

Downloads

Published

2024-07-15

How to Cite

Mao, H., Zhang, B., Xu, H., Yuan, Z., & Liu, Y. (2024). Robust-MSA: Understanding the Impact of Modality Noise on Multimodal Sentiment Analysis. Proceedings of the AAAI Conference on Artificial Intelligence, 37(13), 16458-16460. https://doi.org/10.1609/aaai.v37i13.27078

Issue

Vol. 37 No. 13: AAAI-23 Special Programs, IAAI-23, EAAI-23, Student Papers and Demonstrations

Section

Demonstrations