Contextualized medication event extraction with levitated markers

Automatic extraction of patient medication histories from free-text clinical notes can increase the amount of relevant information to clinicians for developing treatment plans. In addition to detecting medication events, clinical text mining systems must also be able to predict event context, such as negation, uncertainty, and time of occurrence, in order to construct accurate patient timelines. Towards this goal, we introduce Levitated Context Markers (LCMs), a novel transformer-based model for contextualized event extraction. LCMs are an adaptation of levitated markers -originally developed for relation extraction- that allow pretrained transformer models to utilize global input representations while also focusing on event-related subspans using a sparse attention mechanism. In addition to outperforming a strong baseline model on the Contextualized Medication Event Dataset, we show that LCMs' sparse attention can provide interpretable predictions by detecting relevant context cues in an unsupervised manner.

Automated summary

Automatic extraction of patient medication histories from clinical notes can enhance clinicians' access to relevant information. To accurately construct patient timelines, clinical text mining systems need to predict event context, including negation, uncertainty, and time of occurrence. In this study, we present Levitated Context Markers (LCMs), a transformer-based model that enables global representation utilization and event-focused attention mechanism for contextualized event extraction. LCMs outperform a strong baseline model on the Contextualized Medication Event Dataset and demonstrate interpretable predictions by detecting relevant context cues in an unsupervised manner.