Deep Parametric 3D Filters for Joint Video Denoising and Illumination Enhancement in Video Super Resolution

Authors

  • Xiaogang Xu The Chinese University of Hong Kong SmartMore
  • Ruixing Wang SmartMore
  • Chi-Wing Fu The Chinese University of Hong Kong
  • Jiaya Jia The Chinese University of Hong Kong SmartMore

DOI:

https://doi.org/10.1609/aaai.v37i3.25409

Keywords:

CV: Computational Photography, Image & Video Synthesis, CV: Low Level & Physics-Based Vision

Abstract

Despite the quality improvement brought by the recent methods, video super-resolution (SR) is still very challenging, especially for videos that are low-light and noisy. The current best solution is to subsequently employ best models of video SR, denoising, and illumination enhancement, but doing so often lowers the image quality, due to the inconsistency between the models. This paper presents a new parametric representation called the Deep Parametric 3D Filters (DP3DF), which incorporates local spatiotemporal information to enable simultaneous denoising, illumination enhancement, and SR efficiently in a single encoder-and-decoder network. Also, a dynamic residual frame is jointly learned with the DP3DF via a shared backbone to further boost the SR quality. We performed extensive experiments, including a large-scale user study, to show our method's effectiveness. Our method consistently surpasses the best state-of-the-art methods on all the challenging real datasets with top PSNR and user ratings, yet having a very fast run time. The code is available at https://github.com/xiaogang00/DP3DF.

Downloads

Published

2023-06-26

How to Cite

Xu, X., Wang, R., Fu, C.-W., & Jia, J. (2023). Deep Parametric 3D Filters for Joint Video Denoising and Illumination Enhancement in Video Super Resolution. Proceedings of the AAAI Conference on Artificial Intelligence, 37(3), 3054-3062. https://doi.org/10.1609/aaai.v37i3.25409

Issue

Section

AAAI Technical Track on Computer Vision III