Dual Disentangled Attention for Multi-Information Utilization in Sequential Recommendation

Sequential recommendation systems predict the next item that a user will interact with according to the sequential patterns inferred from historical interacted items. Benefiting from the self-attention mechanism, BERT has achieved success in sequential recommendation. Nevertheless, it is still challenging to leverage multiple types of information (e.g., item IDs, attributes, timestamps) effectively under the BERT framework. Existing methods fuse different types of information as the input of BERT by directly adding or concatenating their embedding vectors. This way of fusion ignores the differences of different types of information in sequential pattern inference, which can cause the mutual interference problem and hinder the effective utilization of multi-information. In this work, we propose a Dual Disentangled Attention (DDA) based BERT model, called DDA-BERT, for better leveraging multi-information. Our model performs disentangled information modeling from two aspects: 1) different views to describe items using content information (e.g., using IDs or using attributes) are modeled in different attention heads, respectively; 2) different sequential information (time intervals, relative position distances) modeling is disentangled from content information modeling while calculating the attention scores. As a result, DDA-BERT can effectively prevent the mutual interference problem to improve the prediction accuracy. Extensive experiments on three benchmark datasets show that DDA-BERT consistently outperforms the state-of-the-art baselines by up to 30%.

Automated summary

This paper proposes a Dual Disentangled Attention (DDA) based BERT model, called DDA-BERT, to better leverage multi-information in sequential recommendation systems. Extensive experiments on three benchmark datasets demonstrate that DDA-BERT consistently outperforms the state-of-the-art baselines by up to 30%.