Two-Stream Convolution Augmented Transformer for Human Activity Recognition

Authors

  • Bing Li University of New South Wales, Australia
  • Wei Cui Institute for Infocomm Research, Agency for Science, Technology and Research (ASTAR), Singapore
  • Wei Wang Dongguan University of Technology, China University of New South Wales, Australia
  • Le Zhang Institute for Infocomm Research, Agency for Science, Technology and Research (ASTAR), Singapore
  • Zhenghua Chen Institute for Infocomm Research, Agency for Science, Technology and Research (ASTAR), Singapore
  • Min Wu Institute for Infocomm Research, Agency for Science, Technology and Research (ASTAR), Singapore

Keywords:

Internet of Things, Sensor Networks & Smart Cities

Abstract

Recognition of human activities is an important task due to its far-reaching applications such as healthcare system, context-aware applications, and security monitoring. Recently, WiFi based human activity recognition (HAR) is becoming ubiquitous due to its non-invasiveness. Existing WiFi-based HAR methods regard WiFi signals as a temporal sequence of channel state information (CSI), and employ deep sequential models (e.g., RNN, LSTM) to automatically capture channel-over-time features. Although being remarkably effective, they suffer from two major drawbacks. Firstly, the granularity of a single temporal point is blindly elementary for representing meaningful CSI patterns. Secondly, the time-over-channel features are also important, and could be a natural data augmentation. To address the drawbacks, we propose a novel Two-stream Convolution Augmented Human Activity Transformer (THAT) model. Our model proposes to utilize a two-stream structure to capture both time-over-channel and channel-over-time features, and use the multi-scale convolution augmented transformer to capture range-based patterns. Extensive experiments on four real experiment datasets demonstrate that our model outperforms state-of-the-art models in terms of both effectiveness and efficiency.

Downloads

Published

2021-05-18

How to Cite

Li, B., Cui, W., Wang, W., Zhang, L., Chen, Z., & Wu, M. (2021). Two-Stream Convolution Augmented Transformer for Human Activity Recognition. Proceedings of the AAAI Conference on Artificial Intelligence, 35(1), 286-293. Retrieved from https://ojs.aaai.org/index.php/AAAI/article/view/16103

Issue

Section

AAAI Technical Track on Application Domains