Journal
PROCEEDINGS OF THE 60TH ANNUAL MEETING OF THE ASSOCIATION FOR COMPUTATIONAL LINGUISTICS (ACL 2022), VOL 1: (LONG PAPERS)
Volume -, Issue -, Pages 1767-1777Publisher
ASSOC COMPUTATIONAL LINGUISTICS-ACL
Keywords
-
Categories
Funding
- National Natural Science Foundation of China [61876053, 62006062, 62176076, 62006060]
- UK Engineering and Physical Sciences Research Council [EP/V048597/1, EP/T017112/1]
- Natural Science Foundation of Guangdong Province of China [2019A1515011705]
- Shenzhen Foundational Research Funding [JCYJ20200109113441941, JCYJ20210324115614039]
- Shenzhen Science and Technology Innovation Program [KQTD20190929172835662]
- Turing AI Fellowship - UK Research and Innovation (UKRI) [EP/V020579/1]
- Joint Lab of Lab of HITSZ
Ask authors/readers for more resources
In this paper, the authors investigate multimodal sarcasm detection from a novel perspective by constructing a cross-modal graph to explicitly capture the ironic relations between textual and visual modalities. They propose a cross-modal graph convolutional network which achieves state-of-the-art performance in multimodal sarcasm detection.
With the increasing popularity of posting multimodal messages online, many recent studies have been carried out utilizing both textual and visual information for multi-modal sarcasm detection. In this paper, we investigate multimodal sarcasm detection from a novel perspective by constructing a cross-modal graph for each instance to explicitly draw the ironic relations between textual and visual modalities. Specifically, we first detect the objects paired with descriptions of the image modality, enabling the learning of important visual information. Then, the descriptions of the objects are served as a bridge to determine the importance of the association between the objects of image modality and the contextual words of text modality, so as to build a cross-modal graph for each multi-modal instance. Furthermore, we devise a cross-modal graph convolutional network to make sense of the incongruity relations between modalities for multi-modal sarcasm detection. Extensive experimental results and in-depth analysis show that our model achieves state-of-the-art performance in multi-modal sarcasm detection(1).
Authors
I am an author on this paper
Click your name to claim this paper and add it to your profile.
Reviews
Recommended
No Data Available