Data driven hybrid edge computing-based hierarchical task guidance for efficient maritime escorting with multiple unmanned surface vehicles

The advancement of hardware and software technology makes multiple cooperative unmanned surface vehicles (USVs) utilized in maritime escorting with low cost and high efficiency. USVs can work as edge computing devices to locally and cooperatively perform heavy computational tasks without dependence of remote cloud servers. As such, we organize a team of USVs to escort a high value ship (e.g., mother ship) in a complex maritime environment with hostile intruder ships, where the significant challenge is to learn cooperation of USVs and assign each USV tasks to achieve optimal performance. To this end, in this paper, a hierarchical scheme is proposed for the USV team to guard a valuable ship, which belongs to problems of sparse rewards and long-time horizons in multi-agent reinforcement learning. The core idea utilized in the proposed scheme is centralized training with decentralized execution, where USVs learn policies to guard a high-value ship with extra shared environmental data from other USVs through communication. Specifically, the ships are divided into two groups, i.e., high-level ship and low-level USVs. The high-level ship optimizes decision-level policy to predict intercept points, while each low-level USV utilizes multi-agent reinforcement learning to learn task-level policy to intercept intruders. The hierarchical task guidance is exploited in maritime escorting, whereby high-level ship's decision-level policy guides the training and execution of task-level policies of USVs. Simulation results and experiment analysis show that the proposed hierarchical scheme can efficiently execute the escort mission.