MyGram: Modality-aware Graph Transformer with Global Distribution for Multi-modal Entity Alignment

Abstract

Multi-modal entity alignment aims to identify equivalent entities between two multi-modal Knowledge graphs by integrating multi-modal data, such as images and text, to enrich the semantic representations of entities. However, existing methods may overlook the structural contextual information within each modality, making them vulnerable to interference from shallow features. To address these challenges, we propose MyGram, a \textbf{m}odalit\textbf{y}-aware \textbf{gra}ph transformer with global distribution for \textbf{m}ulti-modal entity alignment. Specifically, we develop a modality diffusion learning module to capture deep structural contextual information within modalities and enable fine-grained multi-modal fusion. In addition, we introduce a Gram Loss that acts as a regularization constraint by minimizing the volume of a 4-dimensional parallelotope formed by multi-modal features, thereby achieving global distribution consistency across modalities. We conduct experiments on five public datasets. Results show that MyGram outperforms baseline models, achieving a maximum improvement of 4.8\% in Hits@1 on FBDB15K, 9.9\% on FBYG15K, and 4.3\% on DBP15K.