Technological forecasting based on estimation of word embedding matrix using LSTM networks

There are a vast number of quantitative and qualitative technological forecasting methods. In the last decade, advanced quantitative technological forecasting methods based on the various applications of data science ap-proaches have been proposed. Text mining is one of the key approaches used to examine large datasets consisting of scientific publications and patent documents with the aim of offering foresight for a selected area. However, the existing related studies either perform a qualitative approach by analysing the recent data to identify the emerging topics or use extrapolation techniques to predict the future values of some statistical terms or the future frequency of some important keywords. In this study, different from such related studies, we propose a deep learning-based framework to predict future co-similarity matrix representing the possible new and disappearing interactions between the words in the future. For this purpose, word vectors are generated using a word embedding technique and the temporal changes of the associations between the words are modelled using Long Short-Term Memory networks for the future estimation of the word embedding matrix. The text mining area is chosen as a case study. The clusters of the terms extracted from the predicted word embedding matrices were analysed and potentially emerging areas were identified for different prediction horizon lengths. The accuracy of the proposed model was analysed based on a set of evaluation metrics that measure the amount of overlapping between the actual and predicted word maps. The quantitative analysis showed that the proposed system can successfully identify the emerging and disappearing areas and can be used as a decision-making tool for the future projection of other areas.

Automated summary

This study proposes a deep learning-based framework to predict the possible new and disappearing interactions between words in the future. Word vectors are generated using a word embedding technique, and the temporal changes of word associations are modeled using Long Short-Term Memory networks. The accuracy of the proposed model is analyzed based on evaluation metrics, indicating its ability to identify emerging and disappearing areas and serve as a decision-making tool for future projections in other fields.