CrowdMLP: Weakly-supervised crowd counting via multi-granularity MLP

Currently, state-of-the-art crowd counting algorithms rely excessively on location-level annotations, which are burdensome to acquire. When only weak supervisory signals at the count level are available, it is arduous and error-prone to regress total counts due to the lack of explicit spatial constraints. To address this issue, we propose a novel and efficient counter, CrowdMLP, which explores the modelling of global dependencies of embeddings and regresses total counts by designing a multi-granularity MLP regressor. Specifically, a locally-focused pre-trained frontend is used to extract crude feature maps with intrinsic spatial cues, preventing the model from collapsing into trivial outcomes. The crude embeddings, along with the raw crowd scenes, are tokenized at different granularity levels. Next, the multi-granularity MLP mixes tokens at the dimensions of cardinality, channel, and spatial for mining global information. We also propose an effective proxy task called Split-Counting to overcome the limited samples and the lack of spatial hints in a self-supervised manner. Extensive experiments demonstrate that CrowdMLP significantly outperforms existing weakly-supervised counting algorithms and performs better than state-of-the-art location-level supervised approaches.

Automated summary

This paper proposes a novel and efficient counting model, CrowdMLP, which regresses total counts by designing a multi-granularity MLP regressor that models global dependencies of embeddings. The model uses a locally-focused pre-trained frontend to extract crude feature maps with spatial cues and tokenizes the crude embeddings and raw crowd scenes at different granularities. The study also introduces a self-supervised proxy task, Split-Counting, to overcome limited samples and the lack of spatial hints.