Recurrent Structure Attention Guidance for Depth Super-resolution

Authors

  • Jiayi Yuan Nanjing University of Science and Technology
  • Haobo Jiang Nanjing University of Science and Technology
  • Xiang Li Nanjing University of Science and Technology
  • Jianjun Qian Nanjing University of Science and Technology
  • Jun Li Nanjing University of Science and Technology
  • Jian Yang Nanjing University of Science and Technology

DOI:

https://doi.org/10.1609/aaai.v37i3.25440

Keywords:

CV: 3D Computer Vision, CV: Multi-modal Vision

Abstract

Image guidance is an effective strategy for depth super-resolution. Generally, most existing methods employ hand-crafted operators to decompose the high-frequency (HF) and low-frequency (LF) ingredients from low-resolution depth maps and guide the HF ingredients by directly concatenating them with image features. However, the hand-designed operators usually cause inferior HF maps (e.g., distorted or structurally missing) due to the diverse appearance of complex depth maps. Moreover, the direct concatenation often results in weak guidance because not all image features have a positive effect on the HF maps. In this paper, we develop a recurrent structure attention guided (RSAG) framework, consisting of two important parts. First, we introduce a deep contrastive network with multi-scale filters for adaptive frequency-domain separation, which adopts contrastive networks from large filters to small ones to calculate the pixel contrasts for adaptive high-quality HF predictions. Second, instead of the coarse concatenation guidance, we propose a recurrent structure attention block, which iteratively utilizes the latest depth estimation and the image features to jointly select clear patterns and boundaries, aiming at providing refined guidance for accurate depth recovery. In addition, we fuse the features of HF maps to enhance the edge structures in the decomposed LF maps. Extensive experiments show that our approach obtains superior performance compared with state-of-the-art depth super-resolution methods. Our code is available at: https://github.com/Yuanjiayii/DSR-RSAG.

Downloads

Published

2023-06-26

How to Cite

Yuan, J., Jiang, H., Li, X., Qian, J., Li, J., & Yang, J. (2023). Recurrent Structure Attention Guidance for Depth Super-resolution. Proceedings of the AAAI Conference on Artificial Intelligence, 37(3), 3331-3339. https://doi.org/10.1609/aaai.v37i3.25440

Issue

Section

AAAI Technical Track on Computer Vision III