FedMat: A Federated Multimodal Materiality Framework for Trustworthy Financial Document Analysis

Abstract

Financial document analysis has become increasingly essential because it supports business decision-making and investment decisions. However, financial documents have unique characteristics, where the style, tone, and financial indicators can substantially influence the results of the analysis. At the same time, a major challenge in financial document analysis is data security. To address this issue, we develop a new Federated Multimodal Materiality framework, named FedMat, which incorporates not only textual data but also materiality-related signals, including percentage change, monetary amount, qualitative category, and contextual and linguistic features. In addition, FedMat operates in a federated learning environment. Based on our experiments, combining text with materiality scores consistently outperforms text-only approaches for financial text classification, particularly for nonlinear models. In general, FedMat achieves the strongest performance, with AUCs of 90.2%-91.3%, compared to 83.6%-90.3% for text-only methods. The greatest gain is observed for XGBoost, improving by 7.7%. In federated learning settings, FedMat maintains strong utility under mild adversarial conditions, while simple robust aggregation provides an effective and practical defense that improves resilience against stronger poisoning attacks.