Large Motion Video Super-Resolution with Dual Subnet and Multi-Stage Communicated Upsampling

Authors

  • Hongying Liu Key Laboratory of Intelligent Perception and Image Understanding of Ministry of Education, School of Artificial Intelligence, Xidian University
  • Peng Zhao Key Laboratory of Intelligent Perception and Image Understanding of Ministry of Education, School of Artificial Intelligence, Xidian University
  • Zhubo Ruan Key Laboratory of Intelligent Perception and Image Understanding of Ministry of Education, School of Artificial Intelligence, Xidian University
  • Fanhua Shang Key Laboratory of Intelligent Perception and Image Understanding of Ministry of Education, School of Artificial Intelligence, Xidian University Peng Cheng Lab, Shenzhen
  • Yuanyuan Liu Key Laboratory of Intelligent Perception and Image Understanding of Ministry of Education, School of Artificial Intelligence, Xidian University

DOI:

https://doi.org/10.1609/aaai.v35i3.16310

Keywords:

Computational Photography, Image & Video Synthesis

Abstract

Video super-resolution (VSR) aims at restoring a video in low-resolution (LR) and improving it to higher-resolution (HR). Due to the characteristics of video tasks, it is very important that motion information among frames should be well concerned, summarized and utilized for guidance in a VSR algorithm. Especially, when a video contains large motion, conventional methods easily bring incoherent results or artifacts. In this paper, we propose a novel deep neural network with Dual Subnet and Multi-stage Communicated Upsampling (DSMC) for super-resolution of videos with large motion. We design a new module named U-shaped residual dense network with 3D convolution (U3D-RDN) for fine implicit motion estimation and motion compensation (MEMC) as well as coarse spatial feature extraction. And we present a new Multi-Stage Communicated Upsampling (MSCU) module to make full use of the intermediate results of upsampling for guiding the VSR. Moreover, a novel dual subnet is devised to aid the training of our DSMC, whose dual loss helps to reduce the solution space as well as enhance the generalization ability. Our experimental results confirm that our method achieves superior performance on videos with large motion compared to state-of-the-art methods.

Downloads

Published

2021-05-18

How to Cite

Liu, H., Zhao, P., Ruan, Z., Shang, F., & Liu, Y. (2021). Large Motion Video Super-Resolution with Dual Subnet and Multi-Stage Communicated Upsampling. Proceedings of the AAAI Conference on Artificial Intelligence, 35(3), 2127-2135. https://doi.org/10.1609/aaai.v35i3.16310

Issue

Section

AAAI Technical Track on Computer Vision II