Boosting Point Clouds Rendering via Radiance Mapping

Authors

  • Xiaoyang Huang Shanghai Jiao Tong University
  • Yi Zhang Shanghai Jiao Tong University
  • Bingbing Ni Shanghai Jiao Tong University
  • Teng Li Anhui University
  • Kai Chen Shanghai AI Laboratory
  • Wenjun Zhang Shanghai Jiao Tong University

DOI:

https://doi.org/10.1609/aaai.v37i1.25175

Keywords:

CV: Scene Analysis & Understanding, CV: 3D Computer Vision

Abstract

Recent years we have witnessed rapid development in NeRF-based image rendering due to its high quality. However, point clouds rendering is somehow less explored. Compared to NeRF-based rendering which suffers from dense spatial sampling, point clouds rendering is naturally less computation intensive, which enables its deployment in mobile computing device. In this work, we focus on boosting the image quality of point clouds rendering with a compact model design. We first analyze the adaption of the volume rendering formulation on point clouds. Based on the analysis, we simplify the NeRF representation to a spatial mapping function which only requires single evaluation per pixel. Further, motivated by ray marching, we rectify the the noisy raw point clouds to the estimated intersection between rays and surfaces as queried coordinates, which could avoid spatial frequency collapse and neighbor point disturbance. Composed of rasterization, spatial mapping and the refinement stages, our method achieves the state-of-the-art performance on point clouds rendering, outperforming prior works by notable margins, with a smaller model size. We obtain a PSNR of 31.74 on NeRF-Synthetic, 25.88 on ScanNet and 30.81 on DTU. Code and data are publicly available in https://github.com/seanywang0408/RadianceMapping.

Downloads

Published

2023-06-26

How to Cite

Huang, X., Zhang, Y., Ni, B., Li, T., Chen, K., & Zhang, W. (2023). Boosting Point Clouds Rendering via Radiance Mapping. Proceedings of the AAAI Conference on Artificial Intelligence, 37(1), 953-961. https://doi.org/10.1609/aaai.v37i1.25175

Issue

Section

AAAI Technical Track on Computer Vision I