TY - JOUR AU - Weng, Shuchen AU - Wu, Hao AU - Chang, Zheng AU - Tang, Jiajun AU - Li, Si AU - Shi, Boxin PY - 2022/06/28 Y2 - 2024/03/28 TI - L-CoDe:Language-Based Colorization Using Color-Object Decoupled Conditions JF - Proceedings of the AAAI Conference on Artificial Intelligence JA - AAAI VL - 36 IS - 3 SE - AAAI Technical Track on Computer Vision III DO - 10.1609/aaai.v36i3.20170 UR - https://ojs.aaai.org/index.php/AAAI/article/view/20170 SP - 2677-2684 AB - Colorizing a grayscale image is inherently an ill-posed problem with multi-modal uncertainty. Language-based colorization offers a natural way of interaction to reduce such uncertainty via a user-provided caption. However, the color-object coupling and mismatch issues make the mapping from word to color difficult. In this paper, we propose L-CoDe, a Language-based Colorization network using color-object Decoupled conditions. A predictor for object-color corresponding matrix (OCCM) and a novel attention transfer module (ATM) are introduced to solve the color-object coupling problem. To deal with color-object mismatch that results in incorrect color-object correspondence, we adopt a soft-gated injection module (SIM). We further present a new dataset containing annotated color-object pairs to provide supervisory signals for resolving the coupling problem. Experimental results show that our approach outperforms state-of-the-art methods conditioned on captions. ER -