Lexical knowledge enhanced text matching via distilled word sense disambiguation

This study proposes a method to improve text matching through integration of lexical knowledge from external resources to model the senses of potentially ambiguous words. Specifically, a sense-aware mechanism is designed wherein a word sense disambiguation (WSD) model is introduced into text matching and both tasks (WSD and matching) are simultaneously optimized via multi-task learning. The proposed WSD is a lightweight model that distills a pre-trained BERT-based model by leveraging the lexical knowledge obtained from WordNet. The sense information obtained from the WSD is integrated into matching explicitly and adaptively through the fusion of the learned sense representations with the word context representations generated from the baseline matching model. The effectiveness of the proposed approach is verified through extensive experiments with three distinct matching-based tasks: natural language inference, paraphrase identification, and answer selection. The results obtained for the respective datasets indicate that the proposed sense knowledge-enhanced matching mechanism outperforms several BERT-based baselines and other recent matching approaches.(c) 2023 Elsevier B.V. All rights reserved.

Automated summary

This study proposes a method that integrates external lexical knowledge to improve text matching by modeling the senses of potentially ambiguous words. A lightweight word sense disambiguation (WSD) model based on BERT and WordNet is designed and integrated into a matching mechanism. Experimental results on three matching-based tasks show that the sense knowledge-enhanced matching mechanism outperforms BERT-based baselines and other recent approaches.