Development of Sindhi text corpus

Sindhi language is a rich language with plenty of literary and general texts. There are number of books, newspapers, magazines and internet material available to develop Sindhi text corpus but yet proper and useful text corpus could not be developed and presented online for research, language features analysis, linguistics analysis and information retrieval systems. The lack of resources for research on computational linguistics and NLP applications for Sindhi language are challenging tasks at this stage. However, we have developed Sindhi text corpora in order to provide text resources to computational linguists, Natural Languages process (NLP) experts and researchers. Online books, newspapers, magazines, blogs and social websites are utilized to build Sindhi text corpus. Sindhi sentiment based text corpus is developed and analyzed with Document Term Matrix and TF-IDF models using 2-gram technique of n-gram model. The corpus may be useful for research on language variation analysis, sentiment analysis, aspect based sentiment analysis, semantic analysis, machine translation, information retrieval, Word2Vec, topic modeling and cluster analysis. (c) 2019 The Authors. Production and hosting by Elsevier B.V. on behalf of King Saud University. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/).

Automated summary

Developing and analyzing Sindhi text corpora is challenging due to the lack of resources, but text resources have been developed for computational linguists and researchers.