Counting challenging crowds robustly using a multi-column multi-task convolutional neural network

Counting challenging crowds from still images has a wide range of applications, such as surveillance event detection, public safety control, traffic monitoring, and urban planning. Early studies on crowd counting focused on extracting hand-crafted features and building effective regression models. However, previous approaches may encounter many challenges, such as partial occlusion, non-uniform density distribution, and variations in scale and perspective. A multi-column multi-task convolutional neural network (MMCNN) is proposed for robust crowd counting, which is achieved through summing up the density map estimated by the proposed network. A novel approach is used to generate the ground truth of density map that focuses on location and detailed information. A multi-column CNN is designed to address drastic scale variation exists in crowds. Per-scale loss is minimized to make the features of different scales highly discriminative. Meanwhile, a multi-task strategy is utilized to simultaneously estimate the density map, crowd density level, and background/foreground mask. Contrastive evaluations in benchmarking datasets are implemented with several state-of-the-art CNN-based crowd counting approaches. Results reveal the accuracy and robustness of our approach in counting challenging crowds. The proposed approach achieves the state-of-the-art performance in terms of mean absolute error and mean squared error. The counting approach can be also extended to other related tasks, such as anomaly detection.