MetaRLEC: Meta-Reinforcement Learning for Discovery of Brain Effective Connectivity
DOI:
https://doi.org/10.1609/aaai.v38i9.28892Keywords:
HAI: Brain-Sensing and Analysis, DMKM: Applications, ML: Applications, ML: Causal Learning, ML: Reinforcement LearningAbstract
In recent years, the discovery of brain effective connectivity (EC) networks through computational analysis of functional magnetic resonance imaging (fMRI) data has gained prominence in neuroscience and neuroimaging. However, owing to the influence of diverse factors during data collection and processing, fMRI data typically exhibits high noise and limited sample characteristics, consequently leading to suboptimal performance of current methods. In this paper, we propose a novel brain effective connectivity discovery method based on meta-reinforcement learning, called MetaRLEC. The method mainly consists of three modules: actor, critic, and meta-critic. MetaRLEC first employs an encoder-decoder framework: the encoder utilizing a Transformer, converts noisy fMRI data into a state embedding; the decoder employing bidirectional LSTM, discovers brain region dependencies from the state and generates actions (EC networks). Then a critic network evaluates these actions, incentivizing the actor to learn higher-reward actions amidst the high-noise setting. Finally, a meta-critic framework facilitates online learning of historical state-action pairs, integrating an action-value neural network and supplementary training losses to enhance the model's adaptability to small-sample fMRI data. We conduct comprehensive experiments on both simulated and real-world data to demonstrate the efficacy of our proposed method.Downloads
Published
2024-03-24
How to Cite
Zhang, Z., Ji, J., & Liu, J. (2024). MetaRLEC: Meta-Reinforcement Learning for Discovery of Brain Effective Connectivity. Proceedings of the AAAI Conference on Artificial Intelligence, 38(9), 10261-10269. https://doi.org/10.1609/aaai.v38i9.28892
Issue
Section
AAAI Technical Track on Humans and AI