Toward Resilient Medical Multimodal AI: A Framework for Missing Modality Recovery Under Clinical Constraints

Abstract

Real-world medical AI systems rarely operate under ideal conditions: multimodal clinical data are frequently incomplete due to resource constraints, equipment limitations, and workflow disruptions—challenges amplified in under-resourced hospitals and community clinics where infrastructure gaps make missing modalities the norm. Existing multimodal learning methods typically assume data completeness or treat missing modalities as a narrow technical problem, limiting their reliability in practice. We posit that missing-modality learning should be reframed as a core challenge of AI resilience in medicine, where systems must maintain reliable performance under real-world constraints while transparently communicating uncertainty. To this end, we propose a two-tier framework aligned with clinical workflows. Tier 1 enables efficient, uncertainty-aware inference under missing modalities using prompt-enhanced modality encoding and learnable pseudo-embeddings. Cases with elevated uncertainty are routed to Tier 2, which performs high-fidelity generative recovery using conditional diffusion models for precision-critical decisions. Component-level validation on a multi-center colorectal cancer cohort (1,679 patients, four centers) demonstrates that each design element contributes measurably to downstream prognostic performance: learnable pseudo-embeddings improve the concordance index (C-index) by 10.1 points over zero-filling in feature space, while missing-aware prompts and modality-aware prompts contribute 4.4 and 3.7 points, respectively. Full evaluation of uncertainty-guided routing and generative recovery across brain tumor, cardiac, and CT–MRI benchmarks is ongoing.