REFINE: Prediction Fusion Network for Panoptic Segmentation

Authors

  • Jiawei Ren SenseTime Research
  • Cunjun Yu SenseTime Research
  • Zhongang Cai SenseTime Research
  • Mingyuan Zhang SenseTime Research
  • Chongsong Chen SenseTime Research Nanyang Technological University
  • Haiyu Zhao SenseTime Research
  • Shuai Yi SenseTime Research
  • Hongsheng Li Multimedia Laboratory, The Chinese University of Hong Kong

Keywords:

Segmentation

Abstract

Panoptic segmentation aims at generating pixel-wise class and instance predictions for each pixel in the input image, which is a challenging task and far more complicated than naively fusing the semantic and instance segmentation results. Prediction fusion is therefore important to achieve accurate panoptic segmentation. In this paper, we present REFINE, pREdiction FusIon NEtwork for panoptic segmentation, to achieve high-quality panoptic segmentation by improving cross-task prediction fusion, and within-task prediction fusion. Our single-model ResNeXt-101 with DCN achieves PQ=51.5 on the COCO dataset, surpassing state-of-the-art performance by a convincing margin and is comparable with ensembled models. Our smaller model with a ResNet-50 backbone achieves PQ=44.9, which is comparable with state-of-the-art methods with larger backbones.

Downloads

Published

2021-05-18

How to Cite

Ren, J., Yu, C., Cai, Z., Zhang, M., Chen, C., Zhao, H., Yi, S., & Li, H. (2021). REFINE: Prediction Fusion Network for Panoptic Segmentation. Proceedings of the AAAI Conference on Artificial Intelligence, 35(3), 2477-2485. Retrieved from https://ojs.aaai.org/index.php/AAAI/article/view/16349

Issue

Section

AAAI Technical Track on Computer Vision II