Densely Connected Parameter-Efficient Tuning for Referring Image Segmentation - Huang, Jiaqi; Xu, Zunnan; Liu, Ting; Liu, Yong; Han, Haonan; Yuan, Kehong; Li, Xiu (Proceedings of the AAAI Conference on Artificial Intelligence, 2025)
In the domain of computer vision, Parameter-Efficient Tuning (PET) is increasingly replacing the traditional paradigm of pre-training followed by full fine-tuning. PET is particularly favored for its effectiveness in large foundation models, as it streamlines transfer learning costs and optimizes hardware utilization.However, the current PET methods are mainly designed for single-modal optimization.While some pioneering studies have undertaken preliminary explorations, they still remain at the level of aligned encoders (e.g., CLIP) and lack exploration of misaligned encoders.These methods show sub-optimal performance with misaligned encoders, as they fail to effectively align the multimodal features during fine-tuning.In this paper, we introduce DETRIS, a parameter-efficient tuning framework designed to enhance low-rank visual feature propagation by establishing dense interconnections between each layer and all preceding layers, which enables effective cross-modal feature interaction and adaptation to misaligned encoders. We also suggest using text adapters to improve textual features. Our simple yet efficient approach greatly surpasses state-of-the-art methods with 0.9% to 1.8% backbone parameter updates, evaluated on challenging benchmarks.