Relay Selection in Underwater Acoustic Cooperative Networks: A Contextual Bandit Approach

Cooperative relay transmission is an attractive architecture for underwater acoustic networks. However, designing relay selection policies in the harsh underwater environment is difficult. In this letter, we model relay selection as a contextual bandit problem-an important extension of multi-armed bandit. Through this way, we can achieve relay selection based on a bit of contextual communication environment information about relay nodes instead of instantaneous or statistical channel state information. Our proposed relay selection technique enables highly stable performance of the cooperative system in a complex and changeable underwater environment, and the process of relay selection can be simplified and easily facilitate efficient cooperative transmission. Simulation results illustrate the effectiveness and the robustness of this relay selection technique.