Topology-Aware Convolutional Neural Network for Efficient Skeleton-Based Action Recognition

Authors

  • Kailin Xu Zhejiang University
  • Fanfan Ye Hikvision Research Institute
  • Qiaoyong Zhong Hikvision Research Institute
  • Di Xie Hikvision Research Institute

DOI:

https://doi.org/10.1609/aaai.v36i3.20191

Keywords:

Computer Vision (CV), Intelligent Robotics (ROB)

Abstract

In the context of skeleton-based action recognition, graph convolutional networks (GCNs) have been rapidly developed, whereas convolutional neural networks (CNNs) have received less attention. One reason is that CNNs are considered poor in modeling the irregular skeleton topology. To alleviate this limitation, we propose a pure CNN architecture named Topology-aware CNN (Ta-CNN) in this paper. In particular, we develop a novel cross-channel feature augmentation module, which is a combo of map-attend-group-map operations. By applying the module to the coordinate level and the joint level subsequently, the topology feature is effectively enhanced. Notably, we theoretically prove that graph convolution is a special case of normal convolution when the joint dimension is treated as channels. This confirms that the topology modeling power of GCNs can also be implemented by using a CNN. Moreover, we creatively design a SkeletonMix strategy which mixes two persons in a unique manner and further boosts the performance. Extensive experiments are conducted on four widely used datasets, i.e. N-UCLA, SBU, NTU RGB+D and NTU RGB+D 120 to verify the effectiveness of Ta-CNN. We surpass existing CNN-based methods significantly. Compared with leading GCN-based methods, we achieve comparable performance with much less complexity in terms of the required GFLOPs and parameters.

Downloads

Published

2022-06-28

How to Cite

Xu, K., Ye, F., Zhong, Q., & Xie, D. (2022). Topology-Aware Convolutional Neural Network for Efficient Skeleton-Based Action Recognition. Proceedings of the AAAI Conference on Artificial Intelligence, 36(3), 2866-2874. https://doi.org/10.1609/aaai.v36i3.20191

Issue

Section

AAAI Technical Track on Computer Vision III