Towards Malay named entity recognition: an open-source dataset and a multi-task framework

Named entity recognition (NER) is a key component of many natural language processing (NLP) applications. The majority of advanced research, however, has not been widely applied to low-resource languages represented by Malay due to the data-hungry problem. In this paper, we present a system for building a Malay NER dataset (MS-NER) of 20,146 sentences through labelled datasets of homologous languages and iterative optimisation. Additionally, we propose a Multi-Task framework, namely MTBR, to integrate boundary information more effectively for NER. Specifically, boundary detection is treated as an auxiliary task and an enhanced Bidirectional Revision module with a gated ignoring mechanism is proposed to undertake conditional label transfer. This can reduce error propagation by the auxiliary task. We conduct extensive experiments on Malay, Indonesian, and English. Experimental results show that MTBR could achieve competitive performance and tends to outperform multiple baselines. The constructed dataset and model would be made available to the public as a new, reliable benchmark for Malay NER.

Automated summary

Named entity recognition (NER) plays a crucial role in various natural language processing (NLP) applications. However, applying advanced NER research to low-resource languages like Malay has been challenging due to the lack of sufficient data. This paper presents a system for building a Malay NER dataset (MS-NER) with 20,146 sentences through labeled datasets in related languages and iterative optimization. Additionally, a Multi-Task framework (MTBR) is proposed to effectively integrate boundary information for improved NER performance.