3.8 Proceedings Paper

MixFormer: End-to-End Tracking with Iterative Mixed Attention

Publisher

IEEE COMPUTER SOC
DOI: 10.1109/CVPR52688.2022.01324

Keywords

-

Funding

  1. National Natural Science Foundation of China [62076119, 61921006]
  2. Program for Innovative Talents and Entrepreneur in Jiangsu Province
  3. Collaborative Innovation Center of Novel Software Technology and Industrialization

Ask authors/readers for more resources

This paper presents a compact tracking framework called MixFormer, which utilizes mixed attention modules for simultaneous feature extraction and target information integration. The framework achieves state-of-the-art performance on five tracking benchmarks, demonstrating the effectiveness of the proposed approach.
Tracking often uses a multi-stage pipeline of feature extraction, target information integration, and bounding box estimation. To simplify this pipeline and unify the process of feature extraction and target information integration, we present a compact tracking framework, termed as MixFormer, built upon transformers. Our core design is to utilize the flexibility of attention operations, and propose a Mixed Attention Module (MAM) for simultaneous feature extraction and target information integration. This synchronous modeling scheme allows to extract target-specific discriminative features and perform extensive communication between target and search area. Based on MAM, we build our MixFormer tracking framework simply by stacking multiple MAMs with progressive patch embedding and placing a localization head on top. In addition, to handle multiple target templates during online tracking, we devise an asymmetric attention scheme in MAM to reduce computational cost, and propose an effective score prediction module to select high-quality templates. Our MixFormer sets a new state-of-the-art performance on five tracking benchmarks, including LaSOT, TrackingNet, VOT2020, GOT-10k, and UAV123. In particular, our MixFormer-L achieves NP score of 79.9% on LaSOT, 88.9% on TrackingNet and EAO of 0.555 on VOT2020. We also perform in-depth ablation studies to demonstrate the effectiveness of simultaneous feature extraction and information integration. Code and trained models are publicly available at https://github.com/MCG-NJU/MixFormer.

Authors

I am an author on this paper
Click your name to claim this paper and add it to your profile.

Reviews

Primary Rating

3.8
Not enough ratings

Secondary Ratings

Novelty
-
Significance
-
Scientific rigor
-
Rate this paper

Recommended

No Data Available
No Data Available