期刊
2019 IEEE/CVF INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV 2019)
卷 -, 期 -, 页码 9166-9175出版社
IEEE
DOI: 10.1109/ICCV.2019.00926
关键词
-
资金
- National Basic Research Program of China (973 Program) [2015CB352502]
- National Natural Science Foundation (NSF) of China [61625301, 61731018]
- Qualcomm
- National Natural Science Foundation of China [U1613209, 61673030]
- Shenzhen Key Laboratory for Intelligent Multimedia and Virtual Reality [ZDSYS201703031405467]
- Microsoft Research Asia
Self-attention mechanism has been widely used for various tasks. It is designed to compute the representation of each position by a weighted sum of the features at all positions. Thus, it can capture long-range relations for computer vision tasks. However, it is computationally consuming. Since the attention maps are computed w.r.t all other positions. In this paper, we formulate the attention mechanism into an expectation-maximization manner and iteratively estimate a much more compact set of bases upon which the attention maps are computed. By a weighted summation upon these bases, the resulting representation is low-rank and deprecates noisy information from the input. The proposed Expectation-Maximization Attention (EMA) module is robust to the variance of input and is also friendly in memory and computation. Moreover, we set up the bases maintenance and normalization methods to stabilize its training procedure. We conduct extensive experiments on popular semantic segmentation benchmarks including PASCAL VOC, PAS-CAL Context and COCO Stuff, on which we set new records(1).
作者
我是这篇论文的作者
点击您的名字以认领此论文并将其添加到您的个人资料中。
推荐
暂无数据