ClinicalRadioBERT: Knowledge-Infused Few Shot Learning for Clinical Notes Named Entity Recognition

Transformer based language models such as BERT have been widely applied to many domains through model pretraining and fine tuning. However, in low-resource scenarios such as clinical cases, customizing a BERT-based language model is still a challenging task. In this paper, we focus on the radiotherapy domain and train a ClinicalRadioBERT model for analyzing clinical notes through a two-step procedure. First, we fine tune a BioBERT model by exploiting full texts of radiotherapy literature and name this model as RadioBERT. Second, we propose a knowledge-infused few-shot learning (KI-FSL) approach that leverages domain knowledge and trains the ClinicalRadioBERT model for understanding radiotherapy clinical notes. We evaluate ClinicalRadioBERT on a newly collected clinical notes dataset and demonstrate its superiority over baselines on few-shot named entity recognition. We will apply the ClinicalRadioBERT to link BERT and medical imaging for radiotherapy.

Automated summary

This paper presents ClinicalRadioBERT, a BERT-based model for analyzing clinical notes. By pretraining and fine-tuning on radiotherapy literature, as well as incorporating knowledge-infused few-shot learning, the model demonstrates superior performance in few-shot named entity recognition.