REFINE: Prediction Fusion Network for Panoptic Segmentation


  • Jiawei Ren SenseTime Research
  • Cunjun Yu SenseTime Research
  • Zhongang Cai SenseTime Research
  • Mingyuan Zhang SenseTime Research
  • Chongsong Chen SenseTime Research Nanyang Technological University
  • Haiyu Zhao SenseTime Research
  • Shuai Yi SenseTime Research
  • Hongsheng Li Multimedia Laboratory, The Chinese University of Hong Kong





Panoptic segmentation aims at generating pixel-wise class and instance predictions for each pixel in the input image, which is a challenging task and far more complicated than naively fusing the semantic and instance segmentation results. Prediction fusion is therefore important to achieve accurate panoptic segmentation. In this paper, we present REFINE, pREdiction FusIon NEtwork for panoptic segmentation, to achieve high-quality panoptic segmentation by improving cross-task prediction fusion, and within-task prediction fusion. Our single-model ResNeXt-101 with DCN achieves PQ=51.5 on the COCO dataset, surpassing state-of-the-art performance by a convincing margin and is comparable with ensembled models. Our smaller model with a ResNet-50 backbone achieves PQ=44.9, which is comparable with state-of-the-art methods with larger backbones.




How to Cite

Ren, J., Yu, C., Cai, Z., Zhang, M., Chen, C., Zhao, H., Yi, S., & Li, H. (2021). REFINE: Prediction Fusion Network for Panoptic Segmentation. Proceedings of the AAAI Conference on Artificial Intelligence, 35(3), 2477-2485.



AAAI Technical Track on Computer Vision II