Self-attention eidetic 3D-LSTM: Video prediction models for traffic flow forecasting

Video prediction is extremely challenging in a traffic flow forecasting problem due to dynamic spatiotem-poral dependence. Eidetic 3D convolutional long short-term memory (E3D-LSTM) network, a state-of-the-art video prediction model, proposes a gate-controlled self-attention module called recall gate in the LSTM mechanism to make the present memory state interact with its historical records for long-term relations. Instead of using the gate-controlled self-attention mechanism, we introduce a query -key-value self-attention mechanism into E3D-LSTM for long-term relations from the perspectives of algo-rithm (internal) and network architecture (external). As for the algorithm (internal) perspective, we replace the original recall gate inside the E3D-LSTM cell with a query-key-value self-attention (SA) mod-ule. While for the network architecture (external) perspective, we propose an independent residual query-key-value self-attention (RSA) block outside E3D-LSTM networks in conjunction with the original recall gate. In a traffic flow forecasting problem, we find that both the models from the internal perspec-tive and the external one, named SAE3D-LSTM and RSA-E3D-LSTM, outperform E3D-LSTM and seven other baseline models on three traffic datasets. This validates the effectiveness of the query-key-value self-attention mechanism for long-term relations. Furthermore, we find that SAE3D-LSTM performs bet-ter than RSA-E3D-LSTM. This indicates that the query-key-value self-attention mechanism alone can cap-ture long-term relations, dispensing with the gate-controlled self-attention mechanism.(c) 2022 Elsevier B.V. All rights reserved.

Automated summary

This study proposes a video prediction model using a query-key-value self-attention mechanism in traffic flow forecasting. The model optimizes the long-term relationship by introducing self-attention mechanism from both algorithm and network architecture perspectives.