Improving abstractive summarization based on dynamic residual network with reinforce dependency

The Seq2Seq abstract summarization model based on long short-term memory (LSTM) is very effective for short text summarization. However, LSTM is limited by long-term dependencies, which can potentially result in salient information loss when long text is processed by the Seq2Seq model based on LSTM. To overcome the long-term dependence limitation, an encoder-decoder model based on the dynamic residual network is proposed in this work. The model can dynamically select an optimal state from the state history to establish a connection with the current state to improve the LSTM long sequence dependencies according to the current decoding environment. Because the dynamic residual connections will result in long-term connection-dependent words, a new method based on reinforcement learning is proposed to simulate the dependence between words, which is then implemented into the training process of the model. This model is verified using the CNN/Daily Mail and New York Times datasets, and the experimental results show that the proposed model achieves significant improvements in capturing longterm dependencies compared with the traditional LSTM-based Seq2Seq abstractive summarization model.& nbsp; (c) 2021 Elsevier B.V. All rights reserved.

Automated summary

A new encoder-decoder model based on dynamic residual network is proposed in this work to improve LSTM's long sequence dependencies by dynamically selecting an optimal state and simulating word dependence using reinforcement learning. Experimental results demonstrate significant improvements in capturing long-term dependencies compared to traditional LSTM-based Seq2Seq abstractive summarization model.