Self-supervised Multi-view Stereo via Effective Co-Segmentation and Data-Augmentation

Authors

  • Hongbin Xu South China University of Technology
  • Zhipeng Zhou Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences
  • Yu Qiao Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences
  • Wenxiong Kang South China University of Technology
  • Qiuxia Wu South China University of Technology, China

Keywords:

3D Computer Vision

Abstract

Recent studies have witnessed that self-supervised methods based on view synthesis obtain clear progress on multi-view stereo (MVS). However, existing methods rely on the assumption that the corresponding points among different views share the same color, which may not always be true in practice. This may lead to unreliable self-supervised signal and harm the final reconstruction performance. To address the issue, we propose a framework integrated with more reliable supervision guided by semantic co-segmentation and data-augmentation. Specially, we excavate mutual semantic from multi-view images to guide the semantic consistency. And we devise effective data-augmentation mechanism which ensures the transformation robustness by treating the prediction of regular samples as pseudo ground truth to regularize the prediction of augmented samples. Experimental results on DTU dataset show that our proposed methods achieve the state-of-the-art performance among unsupervised methods, and even compete on par with supervised methods. Furthermore, extensive experiments on Tanks&Temples dataset demonstrate the effective generalization ability of the proposed method.

Downloads

Published

2021-05-18

How to Cite

Xu, H., Zhou, Z., Qiao, Y., Kang, W., & Wu, Q. (2021). Self-supervised Multi-view Stereo via Effective Co-Segmentation and Data-Augmentation. Proceedings of the AAAI Conference on Artificial Intelligence, 35(4), 3030-3038. Retrieved from https://ojs.aaai.org/index.php/AAAI/article/view/16411

Issue

Section

AAAI Technical Track on Computer Vision III