HANA: Hierarchical Attention Network Assembling for Semantic Segmentation

Semantic segmentation is a crucial issue in the field of computer vision, and it aims to assign each pixel in an image to a semantic object category. Modern cognitive research has presented that the biological system contains hidden features and explicit features, although they both contain useful information, the hidden features need further processing to make them explicit or clear. Inspired by this theory, a semantic segmentation framework named hierarchical attention network assembling is proposed. Multiple auxilary information of different levels corresponding to the two kinds of features of the cognitive system are exploited. Then we further process the hidden information to make them explicit for the semantic segmentation. While in the traditional methods, limited assistance of the auxiliary tasks with only hidden information is provided. In this study, the attention mechanism is utilized and two auxiliary tasks are introduced as attention modules to give explicit guidance to the semantic segmentation task. Two hierarchical sub-networks-an object-level bounding box attention network and an edge-level boundary attention network together serve as explicit auxiliary tasks, of which the first network driven by the object detection aims to aggrandize the consistency constraint of pixels belonging to the same object, and the second one driven by the boundary detection aims to improve the segmentation accuracy within the boundary regions. With the proposed method, the performance achieves 78.3% mean IOU on PASCAL VOC 2012. The explicit guidance of the two auxiliary tasks can well assist the semantic segmentation task.

Automated summary

This study proposed a semantic segmentation framework named hierarchical attention network assembling, which utilizes auxiliary information of different levels corresponding to the hidden and explicit features in the cognitive system, and further processes hidden information to assist the semantic segmentation task.