Journal
IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS
Volume 23, Issue 10, Pages 17224-17240Publisher
IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
DOI: 10.1109/TITS.2022.3150350
Keywords
Semantics; Real-time systems; Image segmentation; Lattices; Decoding; Task analysis; Feature extraction; Deep learning; real-time semantic segmentation; lightweight convolutional neural networks; multi-branch aggregation
Categories
Funding
- National Natural Science Foundation of China [62071404, U21A20514, 61872307, 62172372]
- Open Research Projects of Zhejiang Laboratory [2021KG0AB02]
- Youth Innovation Foundation of Xiamen City [3502Z20206046]
- Zhejiang Provincial Natural Science Foundation [LZ21F030001]
- China Postdoctoral Science Foundation [2021M692957]
Ask authors/readers for more resources
This paper proposes a novel Deep Multi-branch Aggregation Network (DMA-Net) for real-time semantic segmentation in street scenes. The DMA-Net achieves a good tradeoff between segmentation quality and speed by effectively aggregating different levels of feature maps and capturing multi-scale information. Experimental results on Cityscapes and CamVid datasets demonstrate the superior performance of DMA-Net.
Real-time semantic segmentation, which aims to achieve high segmentation accuracy at real-time inference speed, has received substantial attention over the past few years. However, many state-of-the-art real-time semantic segmentation methods tend to sacrifice some spatial details or contextual information for fast inference, thus leading to degradation in segmentation quality. In this paper, we propose a novel Deep Multi-branch Aggregation Network (called DMA-Net) based on the encoder-decoder structure to perform real-time semantic segmentation in street scenes. Specifically, we first adopt ResNet-18 as the encoder to efficiently generate various levels of feature maps from different stages of convolutions. Then, we develop a Multi-branch Aggregation Network (MAN) as the decoder to effectively aggregate different levels of feature maps and capture the multi-scale information. In MAN, a lattice enhanced residual block is designed to enhance feature representations of the network by taking advantage of the lattice structure. Meanwhile, a feature transformation block is introduced to explicitly transform the feature map from the neighboring branch before feature aggregation. Moreover, a global context block is used to exploit the global contextual information. These key components are tightly combined and jointly optimized in a unified network. Extensive experimental results on the challenging Cityscapes and CamVid datasets demonstrate that our proposed DMA-Net respectively obtains 77.0% and 73.6% mean Intersection over Union (mIoU) at the inference speed of 46.7 FPS and 119.8 FPS by only using a single NVIDIA GTX 1080Ti GPU. This shows that DMA-Net provides a good tradeoff between segmentation quality and speed for semantic segmentation in street scenes.
Authors
I am an author on this paper
Click your name to claim this paper and add it to your profile.
Reviews
Recommended
No Data Available