Learning the Dynamics of Visual Relational Reasoning via Reinforced Path Routing


  • Chenchen Jing Beijing Institute of Technology
  • Yunde Jia Beijing Institute of Technology
  • Yuwei Wu Beijing Institute of Technology
  • Chuanhao Li Beijing Institute of Technology
  • Qi Wu University of Adelaide




Computer Vision (CV)


Reasoning is a dynamic process. In cognitive theories, the dynamics of reasoning refers to reasoning states over time after successive state transitions. Modeling the cognitive dynamics is of utmost importance to simulate human reasoning capability. In this paper, we propose to learn the reasoning dynamics of visual relational reasoning by casting it as a path routing task. We present a reinforced path routing method that represents an input image via a structured visual graph and introduces a reinforcement learning based model to explore paths (sequences of nodes) over the graph based on an input sentence to infer reasoning results. By exploring such paths, the proposed method represents reasoning states clearly and characterizes state transitions explicitly to fully model the reasoning dynamics for accurate and transparent visual relational reasoning. Extensive experiments on referring expression comprehension and visual question answering demonstrate the effectiveness of our method.




How to Cite

Jing, C., Jia, Y., Wu, Y., Li, C., & Wu, Q. (2022). Learning the Dynamics of Visual Relational Reasoning via Reinforced Path Routing. Proceedings of the AAAI Conference on Artificial Intelligence, 36(1), 1122-1130. https://doi.org/10.1609/aaai.v36i1.19997



AAAI Technical Track on Computer Vision I