Real-Time Semantic Segmentation via a Densely Aggregated Bilateral Network

With the growing demands of applications on online devices, the speed-accuracy trade-off is critical in the seman-tic segmentation system. Recently, the bilateral segmentation network has shown promising capacity to achieve the balance between favorable accuracy and fast speed, and has become the mainstream backbone in real-time semantic segmentation. Segmentation of target objects relies on high-level semantics, whereasit requires detailed low-level features to model specific local patterns for accurate location. However, the lightweight backboneof bilateral architecture limits the extraction of semantic contextand spatial details. And the late fusion of the bilateral streamsincurs the insufficient aggregation of semantic context and spatial details. In this article, we propose a densely aggregated bilateral network (DAB-Net) for real-time semantic segmentation. In the context path, a patch wise context enhancement (PCE) moduleis proposed to efficiently capture the local semantic contextual information from spatialwise and channelwise, respectively.Meanwhile, a context-guided spatial path (CGSP) is designedto exploit more spatial information by encoding finer details from the raw image and the transition from the context path.Finally, with multiple interactions between bilateral branches,the intertwined outputs from bilateral streams are combined ina unified decoder for a final interaction to further enhance thefeature representation, which generates the final segmentation prediction. Experimental results on three public benchmarks demonstrate that our proposed method achieves higher accuracy with a limited decay in speed, which performs favorably againststate-of-the-art real-time approaches and runs at 31.1 frames/s(FPS) on the high resolution of 2048x1024. The source code is released at https://github.com/isyangshu/DABNet.

Automated summary

In this article, a densely aggregated bilateral network (DAB-Net) is proposed for real-time semantic segmentation. The network captures local semantic contextual information through a patch wise context enhancement (PCE) module in the context path, and exploits more spatial information through a context-guided spatial path (CGSP). Experimental results demonstrate that the proposed method achieves high accuracy with limited decay in speed.