Journal
INFORMATION PROCESSING & MANAGEMENT
Volume 59, Issue 5, Pages -Publisher
ELSEVIER SCI LTD
DOI: 10.1016/j.ipm.2022.103038
Keywords
Fine-grained opinion mining; Aspect-Based Sentiment Analysis; Multimodal Sentiment Analysis
Funding
- Natural Science Foundation of China [62006117]
- Jiangsu Province [BK20200463]
Ask authors/readers for more resources
In this paper, we propose a multi-task learning framework called CMMT for End-to-End Multimodal Aspect-Based Sentiment Analysis. Experimental results demonstrate that CMMT consistently outperforms the state-of-the-art approach JML and achieves superior performance in aspect extraction and sentiment classification compared to other systems.
As an emerging task in opinion mining, End-to-End Multimodal Aspect-Based Sentiment Analysis (MABSA) aims to extract all the aspect-sentiment pairs mentioned in a pair of sentence and image. Most existing methods of MABSA do not explicitly incorporate aspect and sentiment information in their textual and visual representations and fail to consider the different contributions of visual representations to each word or aspect in the text. To tackle these limitations, we propose a multi-task learning framework named Cross-Modal Multitask Transformer (CMMT), which incorporates two auxiliary tasks to learn the aspect/sentiment-aware intra-modal representations and introduces a Text-Guided Cross-Modal Interaction Module to dynamically control the contributions of the visual information to the representation of each word in the inter-modal interaction. Experimental results demonstrate that CMMT consistently outperforms the state-of-the-art approach JML by 3.1, 3.3, and 4.1 absolute percentage points on three Twitter datasets for the End-to-End MABSA task, respectively. Moreover, further analysis shows that CMMT is superior to comparison systems in both aspect extraction (AE) and sentiment classification (SC), which would move the development of multimodal AE and SC algorithms forward with improved performance.
Authors
I am an author on this paper
Click your name to claim this paper and add it to your profile.
Reviews
Recommended
No Data Available