Pedestrian detection based on faster R-CNN in nighttime by fusing deep convolutional features of successive images

Existing studies using visible-light cameras have mainly focused on methods of pedestrian detection during daytime. However, these studies found it difficult to detect pedestrians during nighttime with low external light. The NIR illuminator has limitations in terms of illumination angle and distance, and the illuminator's power needs to be adjusted depending on whether an object is near or distant. Although, thermal cameras were used for nighttime pedestrian detection, thermal cameras are currently expensive and thus difficult to install in many places. To solve these problems, attempts have been made to use visible-light cameras for nighttime pedestrian detection. However, most of these attempts considered an indoor environment where the distance to the object was short. This study proposes a method of pedestrian detection at nighttime using a visible-light camera and faster region-based convolutional neural network (R-CNN). In addition, as pedestrians cannot be reliably detected from a single nighttime image, we combined deep convolutional features in successive frames. Using Korea advanced institute of science and technology (KAIST) open database, we conducted experiments and observed that the proposed method performed better than the baseline methods at all times (day and night). In addition, through the experiments with national ICT Australia Ltd. (NICTA) open database, we confirm that the proposed method is effective for pedestrian detection at all times. Finally, we present theoretical grounds for the proposed fusion. (C) 2018 Elsevier Ltd. All rights reserved.