AutoRemover: Automatic Object Removal for Autonomous Driving Videos


  • Rong Zhang Zhejiang University
  • Wei Li Baidu Research, Baidu Inc.
  • Peng Wang Baidu Research, Baidu Inc.
  • Chenye Guan Baidu Research, Baidu Inc.
  • Jin Fang Baidu Research, Baidu Inc.
  • Yuhang Song University of Southern California
  • Jinhui Yu Zhejiang University
  • Baoquan Chen Peking University
  • Weiwei Xu Zhejiang University
  • Ruigang Yang Baidu Research, Baidu Inc.



Motivated by the need for photo-realistic simulation in autonomous driving, in this paper we present a video inpainting algorithm AutoRemover, designed specifically for generating street-view videos without any moving objects. In our setup we have two challenges: the first is the shadow, shadows are usually unlabeled but tightly coupled with the moving objects. The second is the large ego-motion in the videos. To deal with shadows, we build up an autonomous driving shadow dataset and design a deep neural network to detect shadows automatically. To deal with large ego-motion, we take advantage of the multi-source data, in particular the 3D data, in autonomous driving. More specifically, the geometric relationship between frames is incorporated into an inpainting deep neural network to produce high-quality structurally consistent video output. Experiments show that our method outperforms other state-of-the-art (SOTA) object removal algorithms, reducing the RMSE by over 19%.




How to Cite

Zhang, R., Li, W., Wang, P., Guan, C., Fang, J., Song, Y., Yu, J., Chen, B., Xu, W., & Yang, R. (2020). AutoRemover: Automatic Object Removal for Autonomous Driving Videos. Proceedings of the AAAI Conference on Artificial Intelligence, 34(07), 12853-12861.



AAAI Technical Track: Vision