Deep Recurrent Neural Network with Multi-Scale Bi-directional Propagation for Video Deblurring

Authors

  • Chao Zhu Xi'an Jiaotong University
  • Hang Dong ByteDance
  • Jinshan Pan Nanjing University of Science and Technology
  • Boyang Liang Xi’an Jiaotong University
  • Yuhao Huang Xi'an Jiaotong University
  • Lean Fu ByteDance
  • Fei Wang Xi'an Jiaotong University

DOI:

https://doi.org/10.1609/aaai.v36i3.20272

Keywords:

Computer Vision (CV)

Abstract

The success of the state-of-the-art video deblurring methods stems mainly from implicit or explicit estimation of alignment among the adjacent frames for latent video restoration. However, due to the influence of the blur effect, estimating the alignment information from the blurry adjacent frames is not a trivial task. Inaccurate estimations will interfere the following frame restoration. Instead of estimating alignment information, we propose a simple and effective deep Recurrent Neural Network with Multi-scale Bi-directional Propagation (RNN-MBP) to effectively propagate and gather the information from unaligned neighboring frames for better video deblurring. Specifically, we build a Multi-scale Bi-directional Propagation (MBP) module with two U-Net RNN cells which can directly exploit the inter-frame information from unaligned neighboring hidden states by integrating them in different scales. Moreover, to better evaluate the proposed algorithm and existing state-of-the-art methods on real-world blurry scenes, we also create a Real-World Blurry Video Dataset (RBVD) by a well-designed Digital Video Acquisition System (DVAS) and use it as the training and evaluation dataset. Extensive experimental results demonstrate that the proposed RBVD dataset effectively improve the performance of existing algorithms on real-world blurry videos, and the proposed algorithm performs favorably against the state-of-the-art methods on three typical benchmarks. The code is available at https://github.com/XJTU-CVLAB-LOWLEVEL/RNN-MBP.

Downloads

Published

2022-06-28

How to Cite

Zhu, C., Dong, H., Pan, J., Liang, B., Huang, Y., Fu, L., & Wang, F. (2022). Deep Recurrent Neural Network with Multi-Scale Bi-directional Propagation for Video Deblurring. Proceedings of the AAAI Conference on Artificial Intelligence, 36(3), 3598-3607. https://doi.org/10.1609/aaai.v36i3.20272

Issue

Section

AAAI Technical Track on Computer Vision III