4.6 Article

End-to-End RGB-D SLAM With Multi-MLPs Dense Neural Implicit Representations

Related references

Note: Only part of the references are listed.
Proceedings Paper Computer Science, Artificial Intelligence

ESLAM: Efficient Dense SLAM System Based on Hybrid Representation of Signed Distance Fields

Mohammad Mahdi Johari et al.

Summary: We present ESLAM, an efficient implicit neural representation method for Simultaneous Localization and Mapping (SLAM). Our method incorporates Neural Radiance Fields (NeRF) into a SLAM system, resulting in an efficient and accurate dense visual SLAM method. Extensive experiments demonstrate that ESLAM improves the accuracy of 3D reconstruction and camera localization by more than 50% compared to state-of-the-art methods, while running up to 10 times faster and not requiring pre-training.

2023 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR) (2023)

Proceedings Paper Computer Science, Artificial Intelligence

NeRF-RPN: A general framework for object detection in NeRFs

Benran Hu et al.

Summary: This paper introduces NeRF-RPN, the first significant object detection framework that operates directly on NeRF. NeRF-RPN aims to detect bounding boxes of all objects in a scene by using a novel voxel representation and regressing the 3D bounding boxes without rendering NeRF. It is a general framework that can be applied to detect objects without class labels, and can be trained in an end-to-end manner with various backbone architectures, RPN head designs, and loss functions to estimate high quality 3D bounding boxes. The authors also provide a new benchmark dataset for object detection in NeRF.

2023 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR) (2023)

Proceedings Paper Computer Science, Artificial Intelligence

Co-SLAM: Joint Coordinate and Sparse Parametric Encodings for Neural Real-Time SLAM

Hengyi Wang et al.

Summary: Co-SLAM is a neural RGB-D SLAM system that performs real-time camera tracking and high-fidelity surface reconstruction. It utilizes a hybrid representation with a multi-resolution hash-grid and one-blob encoding to achieve fast convergence, surface coherence, and hole filling.

2023 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR) (2023)

Editorial Material Computer Science, Hardware & Architecture

Neural Radiance Fields Explode on the Scene

Frank Dellaert

COMMUNICATIONS OF THE ACM (2022)

Article Robotics

SO-SLAM: Semantic Object SLAM With Scale Proportional and Symmetrical Texture Constraints

Ziwei Liao et al.

Summary: This letter introduces the concept of object SLAM and proposes a novel monocular Semantic Object SLAM (SO-SLAM) system that addresses the challenges in mapping accuracy and robustness. The algorithm achieves significant improvement in mapping effects according to the experiments.

IEEE ROBOTICS AND AUTOMATION LETTERS (2022)

Article Computer Science, Software Engineering

Instant Neural Graphics Primitives with a Multiresolution Hash Encoding

Thomas Mueller et al.

Summary: The research introduces a versatile new input encoding that allows for a reduction in the cost of training and evaluation by using a multiresolution hash table in neural networks.

ACM TRANSACTIONS ON GRAPHICS (2022)

Proceedings Paper Computer Science, Artificial Intelligence

NeRFReN: Neural Radiance Fields with Reflections

Yuan-Chen Guo et al.

Summary: Neural Radiance Fields (NeRF) is a coordinate-based neural scene representation method that achieves unprecedented view synthesis quality. However, NeRF's view dependency is limited to simple reflections and cannot handle complex reflections such as those from glass and mirrors. To address this issue, we propose NeRFReN, which models scenes with reflections by splitting them into transmitted and reflected components and using separate neural radiance fields for each component. We exploit geometric priors and carefully-designed training strategies to achieve reasonable decomposition results.

2022 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR 2022) (2022)

Proceedings Paper Computer Science, Artificial Intelligence

NeRF in the Dark: High Dynamic Range View Synthesis from Noisy Raw Images

Ben Mildenhall et al.

Summary: Neural Radiance Fields (NeRF) is a technique that synthesizes high-quality novel views by training on linear raw images, allowing for HDR view synthesis and robust handling of noise.

2022 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR 2022) (2022)

Proceedings Paper Computer Science, Artificial Intelligence

HumanNeRF: Free-viewpoint Rendering of Moving People from Monocular Video

Chung-Yi Weng et al.

Summary: The method introduces a free-viewpoint rendering technique called HumanNeRF, which allows rendering the subject from arbitrary camera viewpoints based on a given monocular video. It synthesizes photorealistic details of the body, including fine details such as cloth folds and facial appearance, even from camera angles that may not exist in the input video. The method demonstrates significant performance improvements compared to previous work and provides compelling examples of free-viewpoint renderings from monocular videos of moving humans.

2022 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR 2022) (2022)

Proceedings Paper Computer Science, Artificial Intelligence

Mega-NeRF: Scalable Construction of Large-Scale NeRFs for Virtual Fly-Throughs

Haithem Turki et al.

Summary: This research utilizes neural radiance fields to create interactive 3D environments at a large scale, addressing challenges in modeling, training, and rendering speed. By analyzing visibility statistics and introducing a sparse network structure, the study improves training speed and quality, and introduces a novel method leveraging temporal coherence for faster rendering.

2022 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR) (2022)

Proceedings Paper Computer Science, Artificial Intelligence

Point-NeRF: Point-based Neural Radiance Fields

Qiangeng Xu et al.

Summary: Point-NeRF combines the advantages of high-quality view synthesis and fast scene reconstruction by using neural 3D point clouds and features for radiance field modeling, optimizing rendering efficiency and training time, and handling errors and outliers from other 3D reconstruction methods.

2022 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR 2022) (2022)

Proceedings Paper Computer Science, Artificial Intelligence

NICE-SLAM: Neural Implicit Scalable Encoding for SLAM

Zihan Zhu et al.

Summary: This paper presents NICE-SLAM, a dense SLAM system that incorporates multi-level local information through a hierarchical scene representation. It achieves detailed reconstruction on large indoor scenes and demonstrates competitive results in both mapping and tracking quality.

2022 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR) (2022)

Proceedings Paper Computer Science, Artificial Intelligence

BNV-Fusion: Dense 3D Reconstruction using Bi-level Neural Volume Fusion

Kejie Li et al.

Summary: This article presents a neural network-based method for dense 3D reconstruction, which introduces a bi-level fusion strategy to balance efficiency and reconstruction quality, and achieves significant improvements on multiple datasets.

2022 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR 2022) (2022)

Proceedings Paper Computer Science, Artificial Intelligence

NAN: Noise-Aware NeRFs for Burst-Denoising

Naama Pearl et al.

Summary: Burst denoising is crucial in computational photography for mobile phones and small cameras. We propose a method called NAN1 that utilizes inter-view and spatial information in Neural Radiance Fields (NeRFs) to effectively handle noise, particularly in the presence of large motion and occlusions under high levels of noise.

2022 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR) (2022)

Proceedings Paper Computer Science, Artificial Intelligence

LOLNeRF: Learn from One Look

Daniel Rebain et al.

Summary: This paper presents a method for learning a generative 3D model based on neural radiance fields, which can be trained solely from single view data and reconstructs the shape and appearance of objects for rendering from different views. Experimental results show state-of-the-art performance in novel view synthesis and high-quality results in monocular depth prediction.

2022 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR 2022) (2022)

Article Robotics

DynaSLAM II: Tightly-Coupled Multi-Object Tracking and SLAM

Berta Bescos et al.

Summary: The paper introduces DynaSLAM II, a visual SLAM system for stereo and RGB-D camera configurations with tight integration of multi-object tracking ability, utilizing instance semantic segmentation and ORB features to track dynamic objects. The system not only provides rich clues for scene understanding but also benefits camera tracking.

IEEE ROBOTICS AND AUTOMATION LETTERS (2021)

Proceedings Paper Computer Science, Artificial Intelligence

iMAP: Implicit Mapping and Positioning in Real-Time

Edgar Sucar et al.

Summary: For the first time, a multilayer perceptron is used as the sole scene representation in a real-time SLAM system for a handheld RGB-D camera. The network is trained in live operation to build a dense, scene-specific implicit 3D model of occupancy and color, allowing for immediate tracking. The iMAP algorithm achieves real-time SLAM through continual training with dynamic information-guided pixel sampling for speed and efficient geometry representation.

2021 IEEE/CVF INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV 2021) (2021)

Proceedings Paper Computer Science, Artificial Intelligence

Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fields

Jonathan T. Barron et al.

Summary: Mip-NeRF is an extension of NeRF that reduces aliasing artifacts, improves fine detail representation, and is more efficient in speed and size compared to NeRF.

2021 IEEE/CVF INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV 2021) (2021)

Article Robotics

DeepFactors: Real-Time Probabilistic Dense Monocular SLAM

Jan Czarnowski et al.

IEEE ROBOTICS AND AUTOMATION LETTERS (2020)

Article Robotics

CubeSLAM: Monocular 3-D Object SLAM

Shichao Yang et al.

IEEE TRANSACTIONS ON ROBOTICS (2019)

Article Robotics

ORB-SLAM2: An Open-Source SLAM System for Monocular, Stereo, and RGB-D Cameras

Raul Mur-Artal et al.

IEEE TRANSACTIONS ON ROBOTICS (2017)

Article Automation & Control Systems

S-PTAM: Stereo Parallel Tracking and Mapping

Taihu Pire et al.

ROBOTICS AND AUTONOMOUS SYSTEMS (2017)

Proceedings Paper Computer Science, Artificial Intelligence

ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes

Angela Dai et al.

30TH IEEE CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR 2017) (2017)

Article Robotics

ORB-SLAM: A Versatile and Accurate Monocular SLAM System

Raul Mur-Artal et al.

IEEE TRANSACTIONS ON ROBOTICS (2015)

Article Biochemical Research Methods

TANDEM: matching proteins with tandem mass spectra

R Craig et al.

BIOINFORMATICS (2004)