Journal
2021 IEEE/CVF INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV 2021)
Volume -, Issue -, Pages 10201-10210Publisher
IEEE
DOI: 10.1109/ICCV48922.2021.01006
Keywords
-
Funding
- National Key R&D Program of China [2018AAA0102600, 2018YEF0200200]
- National Natural Science Foundation of China [61876215, 12002254]
- Beijing Academy of Artificial Intelligence (BAAI)
- open project of Zhejiang Laboratory
- Institute for Guo Qiang of Tsinghua University
- Peng Cheng Laboratory
Ask authors/readers for more resources
In this research, a temporal-wise attention SNN model is proposed to enhance the accuracy of processing event streams. By extending the attention concept to temporal-wise input, the model has shown significant improvements in accuracy across three different classification tasks. The results demonstrate the potential of the temporal-wise attention SNN model for effective event stream processing.
How to effectively and efficiently deal with spatio-temporal event streams, where the events are generally sparse and non-uniform and have the mu s temporal resolution, is of great value and has various real-life applications. Spiking neural network (SNN), as one of the brain-inspired event-triggered computing models, has the potential to extract effective spatio-temporal features from the event streams. However, when aggregating individual events into frames with a new higher temporal resolution, existing SNN models do not attach importance to that the serial frames have different signal-to-noise ratios since event streams are sparse and non-uniform. This situation interferes with the performance of existing SNNs. In this work, we propose a temporal-wise attention SNN (TA-SNN) model to learn frame-based representation for processing event streams. Concretely, we extend the attention concept to temporal-wise input to judge the significance of frames for the final decision at the training stage, and discard the irrelevant frames at the inference stage. We demonstrate that TA-SNN models improve the accuracy of event streams classification tasks. We also study the impact of multiple-scale temporal resolutions for frame-based representation. Our approach is tested on three different classification tasks: gesture recognition, image classification, and spoken digit recognition. We report the state-of-the-art results on these tasks, and get the essential improvement of accuracy (almost 19%) for gesture recognition with only 60 ms.
Authors
I am an author on this paper
Click your name to claim this paper and add it to your profile.
Reviews
Recommended
No Data Available