From Coarse to Fine: Hierarchical Pixel Integration for Lightweight Image Super-resolution
Keywords:CV: Low Level & Physics-Based Vision, CV: Computational Photography, Image & Video Synthesis
AbstractImage super-resolution (SR) serves as a fundamental tool for the processing and transmission of multimedia data. Recently, Transformer-based models have achieved competitive performances in image SR. They divide images into fixed-size patches and apply self-attention on these patches to model long-range dependencies among pixels. However, this architecture design is originated for high-level vision tasks, which lacks design guideline from SR knowledge. In this paper, we aim to design a new attention block whose insights are from the interpretation of Local Attribution Map (LAM) for SR networks. Specifically, LAM presents a hierarchical importance map where the most important pixels are located in a fine area of a patch and some less important pixels are spread in a coarse area of the whole image. To access pixels in the coarse area, instead of using a very large patch size, we propose a lightweight Global Pixel Access (GPA) module that applies cross-attention with the most similar patch in an image. In the fine area, we use an Intra-Patch Self-Attention (IPSA) module to model long-range pixel dependencies in a local patch, and then a spatial convolution is applied to process the finest details. In addition, a Cascaded Patch Division (CPD) strategy is proposed to enhance perceptual quality of recovered images. Extensive experiments suggest that our method outperforms state-of-the-art lightweight SR methods by a large margin. Code is available at https://github.com/passerer/HPINet.
How to Cite
Liu, J., Chen, C., Tang, J., & Wu, G. (2023). From Coarse to Fine: Hierarchical Pixel Integration for Lightweight Image Super-resolution. Proceedings of the AAAI Conference on Artificial Intelligence, 37(2), 1666-1674. https://doi.org/10.1609/aaai.v37i2.25254
AAAI Technical Track on Computer Vision II