Neurosymbolic Visual Reasoning for Ambiguity Resolution

Abstract

While modern computer vision systems have notched tremendous successes there are still obstacles to deploying such systems in real-world scenarios. Two salient obstacles are the lack of flexibility that comes with systems trained on fixed categories of data and a lack of stability. Indeed, it is not uncommon when using a pre-trained classifier on a video stream to find it will correctly identify an object as a tree (for example) in one frame only to fail to identify the same object as a tree in the next. In this paper we propose a system that can ameliorate both of these issues by introducing reason into the process of object detection. In particular, we will introduce a hybrid computer-vision / logical reasoning system that observes the world, reasons about what it sees, and can change its judgement as a result of that reasoning. We test our system by addressing the challenging problem of distinguishing between real and artificial objects, showing improved performance over our base computer vision model in both cases.