Journal
ENGINEERING APPLICATIONS OF ARTIFICIAL INTELLIGENCE
Volume 114, Issue -, Pages -Publisher
PERGAMON-ELSEVIER SCIENCE LTD
DOI: 10.1016/j.engappai.2022.105055
Keywords
Deep learning; Activity detection; Graph convolution network
Categories
Funding
- Chinese National Natural Science Foun-dation [62076033, U1931202]
Ask authors/readers for more resources
The study introduces a Multi-actor Activity Detection Framework (MADF) for modeling the interactive relationship among multiple actors in extended videos. By utilizing detection, classification, and post-processing stages, the system achieves accurate detection and classification of multi-actor activities.
We present a Multi-actor Activity Detection Framework (MADF) to model the interactive relationship among multiple actors for activity detection in extended videos. MADF can detect 3 groups of multi-actor activities with different kinds of actors, which involves three stages: detection, classification and post-processing. In the detection stage, both interaction proposals and actor proposals are generated in each video clip, in order to eliminate irrelevant background in the scene. In the classification stage, 3 different classification networks are proposed to classify the 3 groups of activities. And further, for person-object interaction, an attention mechanism is adopted to help the person-object classification network to pay more attention to the small-scale objects; for person-person interaction, a suppression module is used to improve the accuracy of the person-person activity detection; for person-vehicle interaction, a spatial-temporal graph convolution network (GCN) module is embedded to model the fine-grained relationship between the person and vehicle in the person-vehicle classification network, with a proposed Mutually Exclusive Category Loss (MECLoss) helping this network distinguish mutually exclusive activities. At last, we use the off-the-shelf post-processing methods to re-score the proposals for more stable results. The proposed system achieves a great progress on our baseline and achieves the state-of-the-art results in TRECVID 2021 ActEV challenge.
Authors
I am an author on this paper
Click your name to claim this paper and add it to your profile.
Reviews
Recommended
No Data Available