☆ 4.8 Article

Attention Spiking Neural Networks

IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE (2023)

Related references

Note: Only part of the references are listed.

Article Computer Science, Artificial Intelligence

A Comprehensive and Modularized Statistical Framework for Gradient Norm Equality in Deep Neural Networks

Zhaodong Chen et al.

Summary: This paper proposes a novel metric called Block Dynamical Isometry to measure the change of gradient norm in neural networks and presents a statistical framework based on free probability. Several existing methods are improved based on the analysis, and a new normalization technique is proposed. The conclusions and methods are validated by extensive experiments on multiple models.

IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE (2022)

Add to Collection

Article Computer Science, Artificial Intelligence

Event-Based Vision: A Survey

Guillermo Gallego et al.

Summary: This paper provides a comprehensive overview of event-based vision, focusing on the applications and algorithms developed for event cameras. Event cameras differ from traditional cameras in their asynchronous measurement of per-pixel brightness changes, offering high temporal resolution, very high dynamic range, low power consumption, and high pixel bandwidth. The paper discusses techniques for processing events, including learning-based methods and specialized processors. It also highlights the remaining challenges and opportunities in the field of bio-inspired perception and interaction for machines.

IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE (2022)

Add to Collection

Review Computer Science, Software Engineering

Attention mechanisms in computer vision: A survey

Meng-Hao Guo et al.

Summary: Attention mechanisms, inspired by the human visual system, have been successfully applied in various computer vision tasks. This survey provides a comprehensive review of different types of attention mechanisms and suggests future research directions.

COMPUTATIONAL VISUAL MEDIA (2022)

Add to Collection

Article Computer Science, Artificial Intelligence

Progressive Tandem Learning for Pattern Recognition With Deep Spiking Neural Networks

Jibin Wu et al.

Summary: This paper proposes a novel framework for rapid and efficient pattern recognition in spiking neural networks (SNNs), which involves an ANN-to-SNN conversion and layer-wise learning approach called progressive tandem learning. By studying the equivalence between ANNs and SNNs, a spike count-based approximation method is introduced to convert ANN neurons into SNN neurons. Additionally, a layer-wise learning method with adaptive training scheduler is used to fine-tune the network weights and compensate for approximation errors. The framework also allows hardware constraints to be progressively imposed during training and demonstrates remarkable performance on various tasks.

IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE (2022)

Add to Collection

Article Computer Science, Artificial Intelligence

Rethinking Pretraining as a Bridge From ANNs to SNNs

Yihan Lin et al.

Summary: This paper proposes a new training paradigm for SNNs by combining concepts from different training methods, pretraining techniques, and BP-based deep SNN training mechanisms. The proposed paradigm achieves efficient training and obtains state-of-the-art results in experiments.

IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS (2022)

Add to Collection

Article Multidisciplinary Sciences

Visual explanations from spiking neural networks using inter-spike intervals

Youngeun Kim et al.

Summary: Spiking Neural Networks (SNNs) offer an energy-efficient alternative to conventional deep learning by emulating biological features in the brain. The visual explanation tool SAM highlights neurons with short inter-spike interval activity to provide a transparent understanding for end-users on how SNNs make predictions and decisions without the use of gradients and ground truth. This approach opens up a new research area of 'explainable neuromorphic computing' that aims to establish trust in predictions from SNNs.

SCIENTIFIC REPORTS (2021)

Add to Collection

Article Computer Science, Artificial Intelligence

High Speed and High Dynamic Range Video with an Event Camera

Henri Rebecq et al.

Summary: Event cameras are sensors that report brightness changes in the form of asynchronous events, offering advantages such as high temporal resolution, dynamic range, and lack of motion blur. This study introduces a data-driven approach to reconstruct intensity images from event streams, showcasing significant improvements in terms of image quality and real-time performance compared to existing methods.

IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE (2021)

Add to Collection

Proceedings Paper Computer Science, Artificial Intelligence

Incorporating Learnable Membrane Time Constant to Enhance Learning of Spiking Neural Networks

Wei Fang et al.

Summary: This paper introduces a novel training algorithm for spiking neural networks that can learn both synaptic weights and membrane time constants, leading to improved learning efficiency and robustness. The study shows that adaptable membrane time constants can reduce network sensitivity to initial values, accelerate learning, and reevaluates pooling methods in SNNs.

2021 IEEE/CVF INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV 2021) (2021)

Add to Collection

Proceedings Paper Computer Science, Artificial Intelligence

Temporal-wise Attention Spiking Neural Networks for Event Streams Classification

Man Yao et al.

Summary: In this research, a temporal-wise attention SNN model is proposed to enhance the accuracy of processing event streams. By extending the attention concept to temporal-wise input, the model has shown significant improvements in accuracy across three different classification tasks. The results demonstrate the potential of the temporal-wise attention SNN model for effective event stream processing.

2021 IEEE/CVF INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV 2021) (2021)

Add to Collection

Article Computer Science, Artificial Intelligence

Accurate and efficient time-domain classification with adaptive spiking recurrent neural networks

Bojian Yin et al.

Summary: Spiking neural networks show promise as biologically plausible and efficient models for neural computation, achieving state-of-the-art performance in challenging time domain benchmarks. The use of sparse and binary communication between neurons enables powerful and energy-efficient networks.

NATURE MACHINE INTELLIGENCE (2021)

Add to Collection

Proceedings Paper Computer Science, Artificial Intelligence

Spike-Thrift: Towards Energy-Efficient Deep Spiking Neural Networks by Limiting Spiking Activity via Attention-Guided Compression

Souvik Kundu et al.

Summary: This paper introduces a novel two-step SNN compression technique that reduces spiking activity while maintaining accuracy, achieving significant compression ratios and energy-efficiency improvements in experiments.

2021 IEEE WINTER CONFERENCE ON APPLICATIONS OF COMPUTER VISION WACV 2021 (2021)

Add to Collection

Article Computer Science, Artificial Intelligence

Optimized spiking neurons can classify images with high accuracy through temporal coding with two spikes

Christoph Stoeckl et al.

Summary: Optimizing spiking neuron models for information transmission enhances the efficiency and accuracy of deep learning applications through reducing the number of spikes emitted per neuron. This new method improves latency and throughput of resulting spiking networks, offering a low-energy solution for edge and mobile devices in image classification tasks.

NATURE MACHINE INTELLIGENCE (2021)

Add to Collection

Proceedings Paper Computer Science, Artificial Intelligence

ViViT: A Video Vision Transformer

Anurag Arnab et al.

Summary: The study introduces Transformer-based video classification models and achieves state-of-the-art results on multiple benchmarks. Researchers demonstrate how to train the model on smaller datasets and improve performance by leveraging pretrained image models.

2021 IEEE/CVF INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV 2021) (2021)

Add to Collection

Article Computer Science, Artificial Intelligence

A Simple and Light-Weight Attention Module for Convolutional Neural Networks

Jongchan Park et al.

INTERNATIONAL JOURNAL OF COMPUTER VISION (2020)

Add to Collection

Article Computer Science, Artificial Intelligence

Rethinking the performance comparison between SNNS and ANNS

Lei Deng et al.

NEURAL NETWORKS (2020)

Add to Collection

Article Neurosciences

Going Deeper in Spiking Neural Networks: VGG and Residual Architectures

Abhronil Sengupta et al.

FRONTIERS IN NEUROSCIENCE (2019)

Add to Collection

Article Multidisciplinary Sciences

Towards artificial general intelligence with hybrid Tianjic chip architecture

Jing Pei et al.

NATURE (2019)

Add to Collection

Article Engineering, Electrical & Electronic

Surrogate Gradient Learning in Spiking Neural Networks: Bringing the Power of Gradient-based optimization to spiking neural networks

Emre O. Neftci et al.

IEEE SIGNAL PROCESSING MAGAZINE (2019)

Add to Collection

Article Multidisciplinary Sciences

Towards spike-based machine intelligence with neuromorphic computing

Kaushik Roy et al.

NATURE (2019)

Add to Collection

Proceedings Paper Computer Science, Artificial Intelligence

EV-Gait: Event-based Robust Gait Recognition using Dynamic Vision Sensors

Yanxiang Wang et al.

2019 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR 2019) (2019)

Add to Collection

Article Computer Science, Hardware & Architecture

Loihi: A Neuromorphic Manycore Processor with On-Chip Learning

Mike Davies et al.

IEEE MICRO (2018)

Add to Collection

Article Neurosciences

Spatio-Temporal Backpropagation for Training High-Performance Spiking Neural Networks

Yujie Wu et al.

FRONTIERS IN NEUROSCIENCE (2018)

Add to Collection

Article Multidisciplinary Sciences

Dermatologist-level classification of skin cancer with deep neural networks

Andre Esteva et al.

NATURE (2017)

Add to Collection

Proceedings Paper Computer Science, Artificial Intelligence

A Low Power, Fully Event-Based Gesture Recognition System

Arnon Amir et al.

30TH IEEE CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR 2017) (2017)

Add to Collection

Article Multidisciplinary Sciences

Mastering the game of Go with deep neural networks and tree search

David Silver et al.

NATURE (2016)

Add to Collection

Article Neurosciences

Neuronal Mechanisms of Visual Attention

John H. R. Maunsell

ANNUAL REVIEW OF VISION SCIENCE, VOL 1 (2015)

Add to Collection

Article Multidisciplinary Sciences

A million spiking-neuron integrated circuit with a scalable communication network and interface

Paul A. Merolla et al.

SCIENCE (2014)

Add to Collection

Article Multidisciplinary Sciences

Attention enhances synaptic efficacy and the signal-to-noise ratio in neural circuits

Farran Briggs et al.

NATURE (2013)

Add to Collection

Article Computer Science, Artificial Intelligence

Selective Attention and Consciousness: Investigating Their Relation Through Computational Modelling

Kleanthis C. Neokleous et al.

COGNITIVE COMPUTATION (2011)

Add to Collection

Review Multidisciplinary Sciences

Communication in neuronal networks

SB Laughlin et al.

SCIENCE (2003)

Add to Collection

© Peeref 2019-2024. All rights reserved.