How to Train Your HERON

In this letter we apply Deep Reinforcement Learning (Deep RL) and Domain Randomization to solve a navigation task in a natural environment relying solely on a 2D laser scanner. We train a model-based RL agent in simulation to follow lake and river shores and apply it on a real Unmanned Surface Vehicle in a zero-shot setup. We demonstrate that even though the agent has not been trained in the real world, it can fulfill its task successfully and adapt to changes in the robot's environment and dynamics. Finally, we show that the RL agent is more robust, faster, and more accurate than a state-aware Model-Predictive-Controller. Code, simulation environments, pre-trained models, and datasets are available at https://github.com/AntoineRichard/Heron-RL-ICRA.git.

Automated summary

This letter demonstrates using Deep RL and Domain Randomization to successfully solve a navigation task in a natural environment, showing the model's ability to adapt and perform well without real world training. Additionally, the RL agent is proven to be more robust, faster, and more accurate compared to other methods.