Integrating N-Gram Features into Pre-trained Model: A Novel Ensemble Model for Multi-target Stance Detection

Multi-target stance detection in tweets aims to detect the stance of given texts towards a specific target entity. Most existing models on stance detection consider word embedding as input, however, recent developments pointed out that it would be beneficial to incorporate feature-based information appropriately. Motivated by the strong performance of the pre-trained models in many Natural Language Processing field, and n-gram features that have been proved to be effective in prior competition, we present a novel combination module to obtain both advantages. This paper has proposed a pre-trained model integrated with n-gram features module (PMINFM) to better utilize multi-scale feature representation information and semantic features. Then connect it to a Bidirectional Long Short-Term Memory networks with target-specific attention mechanism. The experimental results show that our proposed model outperforms other baseline models in the SemEval-2016 stance detection dataset and achieves state-of-the-art performance.

Automated summary

This paper introduces a novel module that combines pre-trained models with n-gram features to leverage multi-scale feature representation and semantic features. Experimental results demonstrate that this model outperforms other baseline models and achieves state-of-the-art performance in the stance detection dataset.