Anypath Routing Protocol Design via Q-Learning for Underwater Sensor Networks

As a promising technology in the Internet of Underwater Things, underwater sensor networks (UWSNs) have drawn a widespread attention from both academia and industry. However, designing a routing protocol for UWSNs is a great challenge due to high energy consumption and large latency in the underwater environment. This article proposes a Q-learning-based localization-free anypath routing (QLFR) protocol to prolong the lifetime as well as reduce the end-toend delay for UWSNs. Aiming at optimal routing policies, the Q-value is calculated by jointly considering the residual energy and depth information of sensor nodes throughout the routing process. More specifically, we define two reward functions (i.e., depth-related and energy-related rewards) for Q-learning with the objective of reducing latency and extending network lifetime. In addition, a new holding time mechanism for packet forwarding is designed according to the priority of forwarding candidate nodes. Furthermore, mathematical analyses are presented to analyze the performance and computational complexity of the proposed routing protocol. Extensive simulation results demonstrate the superiority performance of the proposed routing protocol in terms of the end-to-end delay and the network lifetime.

Automated summary

This article proposes a Q-learning-based localization-free anypath routing (QLFR) protocol for UWSNs, aiming to prolong the network lifetime and reduce end-to-end delay. The protocol calculates Q-values to determine optimal routing policies by considering residual energy and depth information of sensor nodes. By defining reward functions and designing a holding time mechanism based on priority, the protocol shows superior performance in terms of end-to-end delay and network lifetime through mathematical analyses and simulation results.