P^3-Net: Part Mobility Parsing from Point Cloud Sequences via Learning Explicit Point Correspondence

Authors

  • Yahao Shi State Key Laboratory of Virtual Reality Technology and Systems, Beihang University, Beijing, China
  • Xinyu Cao State Key Laboratory of Virtual Reality Technology and Systems, Beihang University, Beijing, China
  • Feixiang Lu Robotics and Autonomous Driving Laboratory, Baidu Research, Beijing, China
  • Bin Zhou State Key Laboratory of Virtual Reality Technology and Systems, Beihang University, Beijing, China Department of Mathematics and Theories, Peng Cheng Laboratory, Shenzhen, China

DOI:

https://doi.org/10.1609/aaai.v36i2.20122

Keywords:

Computer Vision (CV)

Abstract

Understanding an articulated 3D object with its movable parts is an essential skill for an intelligent agent. This paper presents a novel approach to parse 3D part mobility from point cloud sequences. The key innovation is learning explicit point correspondence from a raw unordered point cloud sequence. We propose a novel deep network called P^3-Net to parallelize trajectory feature extraction and point correspondence establishment, performing joint optimization between them. Specifically, we design a Match-LSTM module to reaggregate point features among different frames by a point correspondence matrix, a.k.a. the matching matrix. To obtain this matrix, an attention module is proposed to calculate the point correspondence. Moreover, we implement a Gumbel-Sinkhorn module to reduce the many-to-one relationship for better point correspondence. We conduct comprehensive evaluations on public benchmarks, including the motion dataset and the PartNet dataset. Results demonstrate that our approach outperforms SOTA methods on various 3D parsing tasks of part mobility, including motion flow prediction, motion part segmentation, and motion attribute (i.e. axis & range) estimation. Moreover, we integrate our approach into a robot perception module to validate its robustness.

Downloads

Published

2022-06-28

How to Cite

Shi, Y., Cao, X., Lu, F., & Zhou, B. (2022). P^3-Net: Part Mobility Parsing from Point Cloud Sequences via Learning Explicit Point Correspondence. Proceedings of the AAAI Conference on Artificial Intelligence, 36(2), 2244-2252. https://doi.org/10.1609/aaai.v36i2.20122

Issue

Section

AAAI Technical Track on Computer Vision II