期刊
ENGINEERING APPLICATIONS OF ARTIFICIAL INTELLIGENCE
卷 114, 期 -, 页码 -出版社
PERGAMON-ELSEVIER SCIENCE LTD
DOI: 10.1016/j.engappai.2022.105055
关键词
Deep learning; Activity detection; Graph convolution network
类别
资金
- Chinese National Natural Science Foun-dation [62076033, U1931202]
The study introduces a Multi-actor Activity Detection Framework (MADF) for modeling the interactive relationship among multiple actors in extended videos. By utilizing detection, classification, and post-processing stages, the system achieves accurate detection and classification of multi-actor activities.
We present a Multi-actor Activity Detection Framework (MADF) to model the interactive relationship among multiple actors for activity detection in extended videos. MADF can detect 3 groups of multi-actor activities with different kinds of actors, which involves three stages: detection, classification and post-processing. In the detection stage, both interaction proposals and actor proposals are generated in each video clip, in order to eliminate irrelevant background in the scene. In the classification stage, 3 different classification networks are proposed to classify the 3 groups of activities. And further, for person-object interaction, an attention mechanism is adopted to help the person-object classification network to pay more attention to the small-scale objects; for person-person interaction, a suppression module is used to improve the accuracy of the person-person activity detection; for person-vehicle interaction, a spatial-temporal graph convolution network (GCN) module is embedded to model the fine-grained relationship between the person and vehicle in the person-vehicle classification network, with a proposed Mutually Exclusive Category Loss (MECLoss) helping this network distinguish mutually exclusive activities. At last, we use the off-the-shelf post-processing methods to re-score the proposals for more stable results. The proposed system achieves a great progress on our baseline and achieves the state-of-the-art results in TRECVID 2021 ActEV challenge.
作者
我是这篇论文的作者
点击您的名字以认领此论文并将其添加到您的个人资料中。
推荐
暂无数据