ReMOT: A model-agnostic refinement for multiple object tracking

Although refinement is commonly used in visual tasks to improve pre-obtained results, it has not been studied for Multiple Object Tracking (MOT) tasks. This could be attributed to two reasons: i) it has not been explored what kinds of errors should - and could - be reduced in MOT refinement; ii) the refinement target, namely, the tracklets, are intertwined and interactive in a 3D spatio-temporal space, and therefore changing one tracklet may affect the others. To tackle these issues, i) we define two types of errors in imperfect tracklets, as Mix-up Error and Cut-off Error, to clarify the refinement goal; ii) we propose a Refining MOT Framework (ReMOT), which first splits imperfect tracklets and then merges the split tracklets with appearance features improved by self-supervised learning. Experiments demonstrate that ReMOT can make significant improvements to state-of-the-art MOT results as powerful post-processing. As a new application, we demonstrate that ReMOT has the potential of being used to assist semi-automatic MOT data annotation and partially release humans from the tedious work. (C) 2020 Elsevier B.V. All rights reserved.

Automated summary

The refinement method was studied for Multiple Object Tracking (MOT) tasks, defining Mix-up Error and Cut-off Error in imperfect tracklets, proposing the ReMOT framework to improve appearance features by splitting and merging tracklets, significantly improving MOT results, and assisting semi-automatic MOT data annotation.