Pretrained Embeddings for Stance Detection with Hierarchical Capsule Network on Social Media

Stance detection on social media aims to identify the stance of social media users toward a topic or claim, which can provide powerful information for various downstream tasks. Many existing stance detection approaches neglect to model the deep semantic representation information in tweets and do not explore aggregating the hierarchical features among words, thus degrading performance. To address these issues, this article proposes a novel deep learning approach Pretrained Embeddings for Stance Detection with Hierarchical Capsule Network (PE-HCN) without complicated preprocessing. Specifically, PE-HCN first adopts a pretrained language model and then uses a related textual entailment task for fine-tuning to obtain the deep textual representations of tweets. The PE-HCN approach extends the dynamic routing scheme to cope with these deep textual representations by utilizing primary capsules for routing the information among words in each tweet and applying secondary capsules to transmit the aggregated features to each category capsule accordingly. Moreover, to improve the confidences of the category capsules, we design an adaptive feedback mechanism to dynamically strengthen the routing signals. Through experiments on three benchmark datascts, compared with the state-of-the-art baselines, the extensive results exhibit that PE-HCN achieves competitive improvements of up to 6.32%, 2.09%, and 1.8%, respectively.