Understanding the meanings of citations using sentiment, role, and citation function classifications

Traditional citation analyses use quantitative methods only, even though there is meaning in the sentences containing citations within the text. This article analyzes three citation meanings: sentiment, role, and function. We compare citation meanings patterns between fields of science and propose an appropriate deep learning model to classify the three meanings automatically at once. The data comes from Indonesian journal articles covering five different areas of science: food, energy, health, computer, and social science. The sentences in the article text were classified manually and used as training data for an automatic classification model. Several classic models were compared with the proposed multi-output convolutional neural network model. The manual classification revealed similar patterns in citation meaning across the science fields: (1) not many authors exhibit polarity when citing, (2) citations are still rarely used, and (3) citations are used mostly for introductions and establishing relations instead of for comparisons with and utilizing previous research. The proposed model's automatic classification metric achieved a macro F1 score of 0.80 for citation sentiment, 0.84 for citation role, and 0.88 for citation function. The model can classify minority classes well concerning the unbalanced dataset. A machine model that can classify several citation meanings automatically is essential for analyzing big data of journal citations.

Automated summary

Traditional citation analyses have limitations in using quantitative methods only. This article proposes a deep learning model to classify citation meanings automatically, including sentiment, role, and function. The proposed model is compared with classic models and shows good performance in classifying citation meanings. The study also reveals similar patterns of citation meaning across different fields of science. The automatic classification metric achieves high scores, especially for unbalanced datasets. The ability to classify citation meanings automatically is important for analyzing big data of journal citations.