Benchmarking a large-scale FIR dataset for on-road pedestrian detection

Far infrared (FIR) pedestrian detection is an essential module of advanced driver assistance systems (ADAS) at nighttime. Although recent deep learning-based detectors have achieved excellent results on visible images in the daytime, their performance on nighttime FIR images is still unidentified, due to the existing nighttime FIR data set is not sufficient to fully train a deep learning detector. To this end, a nighttime FIR pedestrian dataset with the largest scale at present is introduced in this paper, which is called SCUT (South China University of Technology) dataset. The dataset contains fine-grained annotated videos recorded from variable road scenes. In addition, a detailed statistical analysis of the dataset is provided and four representative pedestrian detection methods are evaluated. Benefit from the volume and diversity of training data, the experiment results show that convolutional neural networks (CNN) based detectors obtained good performance on FIR image. To further explore the performance of pedestrian detection on FIR images, four important modifications on Faster R-CNN were studied and a strong baseline for SCUT dataset was proposed, which achieves the best detection result by reducing log-average miss rate from 36.78% to 17.73%. The dataset will be public online (SCUT dataset will be published on: https://github.com/SCUT-CV/SCUT_FIR_Pedestrian_Dataset).