Commonsense visual sensemaking for autonomous driving-On generalised neurosymbolic online abduction integrating vision and semantics

We demonstrate the need and potential of systematically integrated vision and semantics solutions for visual sensemaking in the backdrop of autonomous driving. A general neurosymbolic method for online visual sensemaking using answer set programming (ASP) is systematically formalised and fully implemented. The method integrates state of the art in visual computing, and is developed as a modular framework that is generally usable within hybrid architectures for realtime perception and control. We evaluate and demonstrate with community established benchmarks KITTIMOD, MOT2017, and MOT-2020. As use-case, we focus on the significance of human-centred visual sensemaking -e.g., involving semantic representation and explainability, questionanswering, commonsense interpolation- in safety-critical autonomous driving situations. The developed neurosymbolic framework is domain-independent, with the case of autonomous driving designed to serve as an exemplar for online visual sensemaking in diverse cognitive interaction settings in the backdrop of select human-centred AI technology design considerations. (c) 2021 The Author(s). Published by Elsevier B.V. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/).

Automated summary

The study demonstrates the necessity and potential of systematically integrating vision and semantics solutions for visual sensemaking in autonomous driving scenarios, presenting a neurosymbolic method formalized with answer set programming. Evaluation and demonstrations are conducted using established benchmarks, with a focus on human-centered visual sensemaking in critical autonomous driving situations. The developed framework is domain-independent and serves as an exemplar for online visual sensemaking in diverse cognitive interaction settings.