BERT for Arabic Topic Modeling: An Experimental Study on BERTopic Technique

Topic modeling is an unsupervised machine learning technique for finding abstract topics in a large collection of documents. It helps in organizing, understanding and summarizing large collections of textual information and discovering the latent topics that vary among documents in a given corpus. Latent Dirichlet allocation (LDA) and Non-Negative Matrix Factorization (NMF) are two of the most popular topic modeling techniques. LDA uses a probabilistic approach whereas NMF uses matrix factorization approach, however, new techniques that are based on BERT for topic modeling do exist. In this paper, we aim to experiment with BERTopic using different Pre-Trained Arabic Language Models as embeddings, and compare its results against LDA and NMF techniques. We used Normalized Pointwise Mutual Information (NPMI) measure to evaluate the results of topic modeling techniques. The overall results generated by BERTopic showed better results compared to NMF and LDA. (C) 2021 The Authors. Published by Elsevier B.V.

Automated summary

Topic modeling is an unsupervised machine learning technique that helps find abstract topics in a large collection of documents. Popular techniques like LDA and NMF are used for this purpose, with emerging approaches using BERT-based methods gaining traction. Overall, the results from BERTopic outperformed NMF and LDA in this study.