Human-centered deep compositional model for handling occlusions

Despite their powerful discriminative abilities, Convolutional Neural Networks (CNNs) lack the properties of generative models. This leads to a decreased performance in environments where objects are poorly visible. Solving such a problem by adding more training samples can quickly lead to a combinatorial ex-plosion, therefore the underlying architecture has to be changed instead. This work proposes a Human -Centered Deep Compositional model (HCDC) that combines low-level visual discrimination of a CNN and the high-level reasoning of a Hierarchical Compositional model (HCM). Defined as a transparent model, it can be optimized to real-world environments by adding compactly encoded domain knowledge from hu-man studies and physical laws. The new FridgeNetv2 dataset and a mixture of publicly available datasets are used as a benchmark. The experimental results show the proposed model is explainable, has higher discriminative and generative power, and better handles the occlusion than the current state-of-the-art Mask-RCNN in instance segmentation tasks. (c) 2023 The Author(s). Published by Elsevier Ltd. This is an open access article under the CC BY-NC-ND license ( http://creativecommons.org/licenses/by-nc-nd/4.0/ )

Automated summary

This study proposes a Human-Centered Deep Compositional model (HCDC) that combines the visual discrimination of a CNN and the reasoning of a Hierarchical Compositional model (HCM). The experimental results show that the HCDC model outperforms the current state-of-the-art Mask-RCNN in instance segmentation tasks, with higher discriminative and generative power, as well as better occlusion handling.