Temporal Shift and Spatial Attention-Based Two-Stream Network for Traffic Risk Assessment

On-board vision based traffic risk assessment is a challenging task for intelligent driving systems, which has some special challenges including spatial-temporal feature extraction, different judgements of risks, real-time requirement, lacking data, etc. To overcome these difficulties, we propose a novel Temporal Shift and Spatial Attention based Two-stream Network (TSSAT-Net) for on-board vision based traffic risk assessment. Firstly, we build new judgement measures that integrate actual driving experience and scenario complexity, and release an on-board vision based traffic risk assessment dataset. Secondly, a novel weighted Temporal Shift Module (weighted-TSM) based two-stream network is proposed; unlike previous methods that rely on complex and time-consuming 3D CNN or LSTM calculation, the proposed two-stream network can effectively extract spatial-temporal features using a weighted temporal shift mechanism with just 2D CNN calculation requirement. Thirdly, a spatial and channel attention mechanism is proposed to make the TSSAT-Net more focus on the features closely related to traffic risks, avoiding redundant information in complex traffic scenarios. Experiments based on the released comprehensive dataset show that our method achieves the state-of-the-art classification accuracy in real-time.

Automated summary

This study introduces a novel Temporal Shift and Spatial Attention based Two-stream Network (TSSAT-Net) for on-board vision based traffic risk assessment, incorporating new judgement measures integrating actual driving experience and scenario complexity. A weighted Temporal Shift Module is proposed to extract spatial-temporal features effectively, while a spatial and channel attention mechanism is used to focus on features closely related to traffic risks and avoid redundant information.