4.8 Article

Brain-inspired multimodal hybrid neural network for robot place recognition

Related references

Note: Only part of the references are listed.
Article Neurosciences

Contextual and pure time coding for self and other in the hippocampus

David B. Omer et al.

Summary: This study identified two distinct populations of time cells in the bat hippocampal area CA1. One population encoded different temporal sequences based on the bat's location, thus encoding both spatial context and time. The other population encoded similar preferred times across different spatial contexts, purely encoding elapsed time. Additionally, time cells were found to encode temporal sequences aligned to another bat's landing in a social imitation task. These diverse time codes may support interval timing perception, episodic memory, and temporal coordination between self and others.

NATURE NEUROSCIENCE (2023)

Article Automation & Control Systems

Brain-inspired Intelligent Robotics: Theoretical Analysis and Systematic Application

Hong Qiao et al.

Summary: Traditional joint-link robots are widely used in production lines due to their high precision for single tasks. However, the growing demand for comprehensive robotic performance in manufacturing and service industries has led to the exploration of bio-inspired robotics. This paper presents a brain-inspired intelligent robotic system that incorporates visual cognition, decision-making, motion control, and musculoskeletal structures, and reviews cutting-edge research in these areas.

MACHINE INTELLIGENCE RESEARCH (2023)

Article Computer Science, Artificial Intelligence

Event-Based Vision: A Survey

Guillermo Gallego et al.

Summary: This paper provides a comprehensive overview of event-based vision, focusing on the applications and algorithms developed for event cameras. Event cameras differ from traditional cameras in their asynchronous measurement of per-pixel brightness changes, offering high temporal resolution, very high dynamic range, low power consumption, and high pixel bandwidth. The paper discusses techniques for processing events, including learning-based methods and specialized processors. It also highlights the remaining challenges and opportunities in the field of bio-inspired perception and interaction for machines.

IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE (2022)

Article Multidisciplinary Sciences

Evolution of inner ear neuroanatomy of bats and implications for echolocation

R. Benjamin Sulser et al.

Summary: Phylogenomics of bats suggests that their echolocation either evolved separately in the bat suborders Yinpterochiroptera and Yangochiroptera, or had a single origin in bat ancestors and was later lost in some yinpterochiropterans. The observation of highly derived structures of the spiral ganglion in yangochiropteran bats suggests a neuroanatomical evolutionary driver for their diverse echolocating strategies. These features are highly variable across major clades and may be associated with the explosive diversification of yangochiropterans.

NATURE (2022)

Article Multidisciplinary Sciences

Spatial maps in piriform cortex during olfactory navigation

Cindy Poo et al.

Summary: The study demonstrates that posterior piriform cortex neurons can carry a spatial representation of the environment, showing features of a learned cognitive map that is stable across behavioral contexts and independent of olfactory drive or reward availability. The accuracy of spatial information carried by individual piriform neurons is predicted by their functional coupling strength to the hippocampal theta rhythm, and ensembles of piriform neurons concurrently represent odor identity and spatial locations of animals.

NATURE (2022)

Article Multidisciplinary Sciences

Embodied neuromorphic intelligence

Chiara Bartolozzi et al.

Summary: This Perspective discusses the potential, challenges and future direction of research aimed at demonstrating embodied intelligent robotics via neuromorphic technology. Neuromorphic engineering studies neural computational principles to develop compact and low-power processing systems. Endowing robots with neuromorphic technologies represents a promising approach for creating robots that can seamlessly integrate in society.

NATURE COMMUNICATIONS (2022)

Article Robotics

EchoVPR: Echo State Networks for Visual Place Recognition

Anil Ozdemir et al.

Summary: Recognizing previously visited locations is an important but unsolved task in autonomous navigation. Recent research has shown that Echo State Networks (ESNs) can effectively solve this task and improve performance and generalization abilities.

IEEE ROBOTICS AND AUTOMATION LETTERS (2022)

Article Robotics

Spiking Neural Networks for Visual Place Recognition Via Weighted Neuronal Assignments

Somayeh Hussaini et al.

Summary: SNNs offer energy efficiency, low latency, and have potential in template matching and image recognition. This research introduces a high-performance SNN for VPR task, utilizing a novel assignment scheme to implement ambiguity-informed salience.

IEEE ROBOTICS AND AUTOMATION LETTERS (2022)

Article Robotics

Neuromorphic computing chip with spatiotemporal elasticity for multi-intelligent-tasking robots

Songchen Ma et al.

Summary: Recent advances in artificial intelligence have improved the capabilities of mobile robots in complex and dynamic scenarios. To enable computationally intensive algorithms to be executed locally in multi-task robots with low latency and high efficiency, innovative computing hardware is necessary. Researchers have developed TianjicX, a neuromorphic computing hardware that can support concurrent execution of different neural network models with various coordination manners for robotics. Using a high-level model called Rivulet, they have bridged the gap between robotic-level requirements and hardware implementations, allowing for independent and interactive execution. With the TianjicX chip, a multi-intelligent-tasking mobile robot called Tianjicat was built, demonstrating substantial reductions in latency and power compared to NVIDIA Jetson TX2.

SCIENCE ROBOTICS (2022)

Article Robotics

Neuromorphic computing hardware and neural architectures for robotics

Yulia Sandamirskaya et al.

Summary: This viewpoint provides an overview of recent insights from neuroscience that could enhance signal processing in artificial neural networks on chip, unlocking innovative applications in robotics and autonomous intelligent systems.

SCIENCE ROBOTICS (2022)

Article Robotics

How Many Events Do You Need? Event-Based Visual Place Recognition Using Sparse But Varying Pixels

Tobias Fischer et al.

Summary: Event cameras are of interest due to their characteristics such as high dynamic range, low latency, minimal motion blur, and energy efficiency. This letter explores the distinctiveness of event streams from a small subset of pixels for visual place recognition, demonstrating effectiveness in matching query observations to reference places.

IEEE ROBOTICS AND AUTOMATION LETTERS (2022)

Article Automation & Control Systems

Towards a New Paradigm for Brain-inspired Computer Vision

Xiao-Long Zou et al.

Summary: Brain-inspired computer vision aims to develop advanced image processing techniques by learning from biological systems. However, the current paradigm in this field has not fully captured the fundamental nature of biological vision. A new paradigm that emphasizes the spatio-temporal nature of visual signals and brain-inspired models for processing such data is emerging.

MACHINE INTELLIGENCE RESEARCH (2022)

Article Engineering, Electrical & Electronic

Event-VPR: End-to-End Weakly Supervised Deep Network Architecture for Visual Place Recognition Using Event-Based Vision Sensor

Delei Kong et al.

Summary: In this study, an end-to-end VPR network using event cameras is proposed, which achieves good recognition performance in challenging driving scenes. By characterizing and aggregating event streams, the algorithm demonstrates stronger advantages in handling challenging environments.

IEEE TRANSACTIONS ON INSTRUMENTATION AND MEASUREMENT (2022)

Article Neurosciences

End-to-End Implementation of Various Hybrid Neural Networks on a Cross-Paradigm Neuromorphic Chip

Guanrui Wang et al.

Summary: The integration of computer-science oriented artificial neural networks (ANNs) and neuroscience oriented spiking neural networks (SNNs) has emerged as a highly promising direction in achieving further breakthroughs in artificial intelligence. The proposed end-to-end mapping framework for implementing hybrid neural networks on many-core neuromorphic architectures shows low execution latency and power consumption with nearly no accuracy degradation. This approach provides a new method for developing hybrid neural network models for brain-inspired computing chips.

FRONTIERS IN NEUROSCIENCE (2021)

Article Computer Science, Artificial Intelligence

VPR-Bench: An Open-Source Visual Place Recognition Evaluation Framework with Quantifiable Viewpoint and Appearance Change

Mubariz Zaffar et al.

Summary: VPR is the process of recognizing previously visited places using visual information, crucial for autonomous navigation systems, but the field lacks standardization in performance evaluation, leading to fragmentation.

INTERNATIONAL JOURNAL OF COMPUTER VISION (2021)

Article Robotics

Probabilistic Appearance-Invariant Topometric Localization With New Place Awareness

Ming Xu et al.

Summary: Probabilistic state-estimation approaches provide a principled foundation for localization systems, and the new probabilistic topometric localization system effectively addresses the shortcomings of existing systems, achieving superior performance in the presence of appearance change and route deviations.

IEEE ROBOTICS AND AUTOMATION LETTERS (2021)

Article Engineering, Electrical & Electronic

Advancing Neuromorphic Computing With Loihi: A Survey of Results and Outlook

Mike Davies et al.

Summary: Neuromorphic computing aims to develop chips inspired by biological neural circuits to process new knowledge, adapt, behave, and learn in real time at low power levels, with recent advancements showing promising results with Intel's Loihi processor. Compelling neuromorphic networks using spike-based hardware demonstrate significantly lower latency and energy consumption compared to state-of-the-art conventional approaches, solving diverse problems representative of brain-like computation.

PROCEEDINGS OF THE IEEE (2021)

Article Biochemistry & Molecular Biology

The spatiotemporal organization of experience dictates hippocampal involvement in primary visual cortical plasticity

Peter S. B. Finnie et al.

Summary: The hippocampus plays a crucial role in specific experience-dependent plasticity processes in the primary visual cortex of mice, but low-level recognition memory can be formed independently of the hippocampus. The hippocampus is crucial for encoding temporally structured experiences.

CURRENT BIOLOGY (2021)

Article Multidisciplinary Sciences

Locally ordered representation of 3D space in the entorhinal cortex

Gily Ginosar et al.

Summary: Researchers discovered various types of spatial neurons in the brains of freely flying bats, including 3D border cells, 3D head-direction cells, and neurons with multiple 3D firing fields. The 3D grid cells exhibited only local order, unlike the global lattice arrangement seen in 2D grid cells, providing a locally ordered metric for space.

NATURE (2021)

Article Multidisciplinary Sciences

Nonlocal spatiotemporal representation in the hippocampus of freely flying bats

Nicholas M. Dotson et al.

Summary: The study finds that neural activity in the hippocampus of bats primarily encodes spatial information beyond the present moment, extending up to meters away from the bat's current position. This spatiotemporal representation extends both backward and forward in time, with emphasis on future locations, and is observed during both random exploration and goal-directed navigation.

SCIENCE (2021)

Proceedings Paper Automation & Control Systems

Event-driven Vision and Control for UAVs on a Neuromorphic Chip

Antonio Vitale et al.

Summary: Event-based vision sensors provide a significant speed and power advantage in high-speed UAV control, enabling faster vision-driven control. Implementing an event-based perception algorithm as a spiking neuronal network on a chip can lead to more efficient control and lower latency.

2021 IEEE INTERNATIONAL CONFERENCE ON ROBOTICS AND AUTOMATION (ICRA 2021) (2021)

Proceedings Paper Automation & Control Systems

EventVLAD: Visual Place Recognition with Reconstructed Edges from Event Cameras

Alex Junho Lee et al.

Summary: Event cameras can capture high dynamic range images with low latency, but integration of multiple events is necessary for spatial information; to achieve robust visual place recognition, reconstructed event edges denoised are suggested for use.

2021 IEEE/RSJ INTERNATIONAL CONFERENCE ON INTELLIGENT ROBOTS AND SYSTEMS (IROS) (2021)

Article Engineering, Electrical & Electronic

Multi-Modal Visual Place Recognition in Dynamics-Invariant Perception Space

Lin Wu et al.

Summary: This research successfully improves place recognition in dynamic environments by utilizing multi-modal fusion, deep learning architecture, and spatial-pyramid-matching model.

IEEE SIGNAL PROCESSING LETTERS (2021)

Review Neurosciences

Navigating Through Time: A Spatial Navigation Perspective on How the Brain May Encode Time

John B. Issa et al.

ANNUAL REVIEW OF NEUROSCIENCE, VOL 43 (2020)

Article Robotics

Dynamic obstacle avoidance for quadrotors with event cameras

Davide Falanga et al.

SCIENCE ROBOTICS (2020)

Article Multidisciplinary Sciences

Humans with stereo olfaction

Yuli Wu et al.

PROCEEDINGS OF THE NATIONAL ACADEMY OF SCIENCES OF THE UNITED STATES OF AMERICA (2020)

Article Engineering, Electrical & Electronic

Tianjic: A Unified and Scalable Chip Bridging Spike-Based and Continuous Neural Computation

Lei Deng et al.

IEEE JOURNAL OF SOLID-STATE CIRCUITS (2020)

Article Robotics

Event-Based Visual Place Recognition With Ensembles of Temporal Windows

Tobias Fischer et al.

IEEE ROBOTICS AND AUTOMATION LETTERS (2020)

Proceedings Paper Automation & Control Systems

Reinforcement co-Learning of Deep and Spiking Neural Networks for Energy-Efficient Mapless Navigation with Neuromorphic Hardware

Guangzhi Tang et al.

2020 IEEE/RSJ INTERNATIONAL CONFERENCE ON INTELLIGENT ROBOTS AND SYSTEMS (IROS) (2020)

Article Robotics

A Hybrid Compact Neural Architecture for Visual Place Recognition

Marvin Chancan et al.

IEEE ROBOTICS AND AUTOMATION LETTERS (2020)

Article Computer Science, Artificial Intelligence

Rapid online learning and robust recall in a neuromorphic olfactory circuit

Nabil Imam et al.

NATURE MACHINE INTELLIGENCE (2020)

Article Multidisciplinary Sciences

Towards artificial general intelligence with hybrid Tianjic chip architecture

Jing Pei et al.

NATURE (2019)

Article Multidisciplinary Sciences

Integrating vision and echolocation for navigation and perception in bats

S. Danilovich et al.

SCIENCE ADVANCES (2019)

Article Computer Science, Cybernetics

NeuroSLAM: a brain-inspired SLAM system for 3D environments

Fangwen Yu et al.

BIOLOGICAL CYBERNETICS (2019)

Article Computer Science, Cybernetics

Leveraging variable sensor spatial acuity with a homogeneous, multi-scale place recognition framework

Adam Jacobson et al.

BIOLOGICAL CYBERNETICS (2018)

Article Multidisciplinary Sciences

Generalized leaky integrate-and-fire models classify multiple neuron types

Corinne Teeter et al.

NATURE COMMUNICATIONS (2018)

Article Neurosciences

Spatio-Temporal Backpropagation for Training High-Performance Spiking Neural Networks

Yujie Wu et al.

FRONTIERS IN NEUROSCIENCE (2018)

Article Multidisciplinary Sciences

Integrating time from experience in the lateral entorhinal cortex

Albert Tsao et al.

NATURE (2018)

Article Robotics

Visual Place Recognition: A Survey

Stephanie Lowry et al.

IEEE TRANSACTIONS ON ROBOTICS (2016)

Article Mathematical & Computational Biology

Unsupervised learning of digit recognition using spike-timing-dependent plasticity

Peter U. Diehl et al.

FRONTIERS IN COMPUTATIONAL NEUROSCIENCE (2015)

Article Multidisciplinary Sciences

Three-dimensional head-direction coding in the bat brain

Arseny Finkelstein et al.

NATURE (2015)

Article Multidisciplinary Sciences

Representation of Three-Dimensional Space in the Hippocampus of Flying Bats

Michael M. Yartsev et al.

SCIENCE (2013)

Article Multidisciplinary Sciences

A map of visual space in the primate entorhinal cortex

Nathaniel J. Killian et al.

NATURE (2012)

Review Neurosciences

How Does the Brain Solve Visual Object Recognition?

James J. DiCarlo et al.

NEURON (2012)

Article Robotics

Persistent Navigation and Mapping using a Biologically Inspired SLAM System

Michael Milford et al.

INTERNATIONAL JOURNAL OF ROBOTICS RESEARCH (2010)

Review Neurosciences

The sensory and motor roles of auditory hair cells

R Fettiplace et al.

NATURE REVIEWS NEUROSCIENCE (2006)

Article Computer Science, Artificial Intelligence

Spatial view cells in the hippocampus, and their idlothetic update based on place and head direction

ET Rolls et al.

NEURAL NETWORKS (2005)

Article Multidisciplinary Sciences

Microstructure of a spatial map in the entorhinal cortex

T Hafting et al.

NATURE (2005)