SVP-T: A Shape-Level Variable-Position Transformer for Multivariate Time Series Classification


  • Rundong Zuo Hong Kong Baptist University
  • Guozhong Li Hong Kong Baptist University
  • Byron Choi Hong Kong Baptist University
  • Sourav S Bhowmick Nanyang Technological University
  • Daphne Ngar-yin Mah Hong Kong Baptist University
  • Grace L.H. Wong Department of Medicine and Therapeutics, The Chinese University of Hong Kong



ML: Time-Series/Data Streams, DMKM: Mining of Spatial, Temporal or Spatio-Temporal Data


Multivariate time series classification (MTSC), one of the most fundamental time series applications, has not only gained substantial research attentions but has also emerged in many real-life applications. Recently, using transformers to solve MTSC has been reported. However, current transformer-based methods take data points of individual timestamps as inputs (timestamp-level), which only capture the temporal dependencies, not the dependencies among variables. In this paper, we propose a novel method, called SVP-T. Specifically, we first propose to take time series subsequences, which can be from different variables and positions (time interval), as the inputs (shape-level). The temporal and variable dependencies are both handled by capturing the long- and short-term dependencies among shapes. Second, we propose a variable-position encoding layer (VP-layer) to utilize both the variable and position information of each shape. Third, we introduce a novel VP-based (Variable-Position) self-attention mechanism to allow the enhancing the attention weights of overlapping shapes. We evaluate our method on all UEA MTS datasets. SVP-T achieves the best accuracy rank when compared with several competitive state-of-the-art methods. Furthermore, we demonstrate the effectiveness of the VP-layer and the VP-based self-attention mechanism. Finally, we present one case study to interpret the result of SVP-T.




How to Cite

Zuo, R., Li, G., Choi, B., S Bhowmick, S., Mah, D. N.- yin, & Wong, G. L. (2023). SVP-T: A Shape-Level Variable-Position Transformer for Multivariate Time Series Classification. Proceedings of the AAAI Conference on Artificial Intelligence, 37(9), 11497-11505.



AAAI Technical Track on Machine Learning IV