期刊
IEEE TRANSACTIONS ON IMAGE PROCESSING
卷 27, 期 11, 页码 5600-5611出版社
IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
DOI: 10.1109/TIP.2018.2855422
关键词
Video captioning; adversarial training; LSTM
资金
- National Natural Science Foundation of China [61572108, 61632007]
- 111 Project [B17008]
In this paper, we propose a novel approach to video captioning based on adversarial learning and long shortterm memory (LSTM). With this solution concept, we aim at compensating for the deficiencies of LSTM-based video captioning methods that generally show potential to effectively handle temporal nature of video data when generating captions but also typically suffer from exponential error accumulation. Specifically, we adopt a standard generative adversarial network (GAN) architecture, characterized by an interplay of two competing processes: a generator that generates textual sentences given the visual content of a video and a discriminator that controls the accuracy of the generated sentences. The discriminator acts as an adversary toward the generator, and with its controlling mechanism, it helps the generator to become more accurate. For the generator module, we take an existing video captioning concept using LSTM network. For the discriminator, we propose a novel realization specifically tuned for the video captioning problem and taking both the sentences and video features as input. This leads to our proposed LSTM-GAN system architecture, for which we show experimentally to significantly outperform the existing methods on standard public datasets.
作者
我是这篇论文的作者
点击您的名字以认领此论文并将其添加到您的个人资料中。
推荐
暂无数据