期刊
IEEE SIGNAL PROCESSING LETTERS
卷 29, 期 -, 页码 2178-2182出版社
IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
DOI: 10.1109/LSP.2022.3216500
关键词
Feature extraction; Anomaly detection; Benchmark testing; Training; Task analysis; Visualization; Surveillance; Video anomaly detection; multimodal information; supervised attention; weakly supervised learning
资金
- Shanghai Key Research Laboratory of NSAI
This paper proposes a novel fusion method and a Multimodal Supervise-Attention enhanced Fusion (MSAF) framework under weak supervision to address the lack of exploration of multimodal data and the ignore of implicit alignment of multimodal features in video anomaly detection. Extensive experiments on four challenging datasets demonstrate the effectiveness of the framework.
The complementarity of multimodal signal is essential for video anomaly detection. However, existing methods either lack exploration to multimodal data or ignore the implicit alignment of multimodal features. In our work, we address this problem using a novel fusion method and propose a Multimodal Supervise-Attention enhanced Fusion (MSAF) framework under weak supervision. Our framework can be divided into two parts: 1) the multimodal labels refinement part refines video-level ground truth into pseudo clip-level labels for subsequent training, 2) the multimodal supervise-attention fusion network enhances features via implicitly aligning different information, then fusing them effectively to predict anomaly scores with the help of refined labels. We validate our framework on four challenging datasets: ShanghaiTech, UCF-Crime, LAD, and XD-Violence. Extensive experiments on the benchmarks demonstrate the effectiveness of our framework, which achieves comparable results on several benchmarks and outperforms current state-of-the-art methods on the XD-Violence audiovisual multimodal dataset.
作者
我是这篇论文的作者
点击您的名字以认领此论文并将其添加到您的个人资料中。
推荐
暂无数据