Journal
INFORMATION PROCESSING & MANAGEMENT
Volume 59, Issue 2, Pages -Publisher
ELSEVIER SCI LTD
DOI: 10.1016/j.ipm.2021.102788
Keywords
Passage ranking; Attention mechanism; Information retrieval; Question answering; Pre-trained model
Funding
- National Natural Science Foundation of China [62002373, 61690203]
Ask authors/readers for more resources
Passage ranking is crucial in information retrieval and question answering, and pre-trained language models like BERT have been shown to improve performance. However, these models can be easily fooled by overlapping but irrelevant passages. To address this issue, a self-matching attention-pooling mechanism (SMAP) and a hybrid passage ranking architecture called BERT-SMAP have been proposed to better identify distracting passages.
Passage ranking has attracted considerable attention due to its importance in information retrieval (IR) and question answering (QA). Prior works have shown that pre-trained language models (e.g. BERT) can improve ranking performance. However, these simple BERT-based methods tend to focus on passage terms that exactly match the question, which makes them easily fooled by the overlapping but irrelevant (distracting) passages. To solve this problem, we propose a self-matching attention-pooling mechanism (SMAP) to highlight the Essential Terms in the question-passage pairs. Further, we propose a hybrid passage ranking architecture, called BERT-SMAP, which combines SMAP with BERT to more effectively identify distracting passages and downplay their influence. BERT-SMAP uses the representations obtained through SMAP to enhance BERT's classification mechanism as an interaction-focused neural ranker, and as the inputs of a matching function. Experimental results on three evaluation datasets show that our model outperforms the previous best BERTbase-based approaches, and is comparable to the state-of-the-art method that utilizes a much stronger pre-trained language model.
Authors
I am an author on this paper
Click your name to claim this paper and add it to your profile.
Reviews
Recommended
No Data Available