A Scalable Reasoning and Learning Approach for Neural-Symbolic Stream Fusion


  • Danh Le-Phuoc Technical University Berlin
  • Thomas Eiter TU Wien
  • Anh Le-Tuan Technical University Berlin




Neuro-Symbolic AI (NSAI), Common-Sense Reasoning, Time-Series/Data Streams, Internet of Things, Sensor Networks & Smart Cities


Driven by deep neural networks (DNN), the recent development of computer vision makes vision sensors such as stereo cameras and Lidars ubiquitous in autonomous cars, robotics and traffic monitoring. However, a traditional DNN-based data fusion pipeline like object tracking has to hard-wire an engineered set of DNN models to a fixed processing logic, which makes it difficult to infuse new models to that pipeline. To overcome this, we propose a novel neural-symbolic stream reasoning approach realised by semantic stream reasoning programs which specify DNN-based data fusion pipelines via logic rules with learnable probabilistic degrees as weights. The reasoning task over this program is governed by a novel incremental reasoning algorithm, which lends itself also as a core building block for a scalable and parallel algorithm to learn the weights for such program. Extensive experiments with our first prototype on multi-object tracking benchmarks for autonomous driving and traffic monitoring show that our flexible approach can considerably improve both accuracy and processing throughput compared to the DNN-based counterparts.




How to Cite

Le-Phuoc, D., Eiter, T., & Le-Tuan, A. (2021). A Scalable Reasoning and Learning Approach for Neural-Symbolic Stream Fusion. Proceedings of the AAAI Conference on Artificial Intelligence, 35(6), 4996-5005. https://doi.org/10.1609/aaai.v35i6.16633



AAAI Technical Track Focus Area on Neuro-Symbolic AI