Perception Coordination Network: A Framework for Online Multi-Modal Concept Acquisition and Binding

Authors

  • You-Lu Xing Anhui University
  • Fu-Rao Shen Nanjing  University
  • Jin-Xi Zhao Nanjing University
  • Jing-Xin Pan Nanjing University
  • Ah-Hwee Tan Nanyang Technological University

DOI:

https://doi.org/10.1609/aaai.v32i1.11602

Keywords:

multi-modal learning, concept acquisition and binding, online incremental learning

Abstract

A biologically plausible neural network model named Perception Coordination Network (PCN) is proposed for online multi-modal concept acquisition and binding. It is a hierarchical structure inspired by the structure of the brain, and functionally divided into the primary sensory area (PSA), the primary sensory association area (SAA), and the higher order association area (HAA). The PSA processes many elementary features, e.g., colors, shapes, syllables, and basic flavors, etc. The SAA combines these elementary features to represent the unimodal concept of an object, e.g., the image, name and taste of an apple, etc. The HAA connects several primary sensory association areas like a function of synaesthesia, which means associating the image, name and taste of an object. PCN is able to continuously acquire and bind multi-modal concepts in an online way. Experimental results suggest that PCN can handle the multi-modal concept acquisition and binding problem effectively.

Downloads

Published

2018-04-29

How to Cite

Xing, Y.-L., Shen, F.-R., Zhao, J.-X., Pan, J.-X., & Tan, A.-H. (2018). Perception Coordination Network: A Framework for Online Multi-Modal Concept Acquisition and Binding. Proceedings of the AAAI Conference on Artificial Intelligence, 32(1). https://doi.org/10.1609/aaai.v32i1.11602