Knowledge-Enhanced Scene Graph Generation with Multimodal Relation Alignment (Student Abstract)
Keywords:Scene Graph Generation, Multimodal Relation Alignment, Knowledge Enhancement
AbstractExisting scene graph generation methods suffer the limitations when the image lacks of sufficient visual contexts. To address this limitation, we propose a knowledge-enhanced scene graph generation model with multimodal relation alignment, which supplements the missing visual contexts by well-aligned textual knowledge. First, we represent the textual information into contextualized knowledge which is guided by the visual objects to enhance the contexts. Furthermore, we align the multimodal relation triplets by co-attention module for better semantics fusion. The experimental results show the effectiveness of our method.
How to Cite
Fu, Z., Feng, J., Zheng, C., & Cai, Y. (2022). Knowledge-Enhanced Scene Graph Generation with Multimodal Relation Alignment (Student Abstract). Proceedings of the AAAI Conference on Artificial Intelligence, 36(11), 12947-12948. https://doi.org/10.1609/aaai.v36i11.21610
AAAI Student Abstract and Poster Program