A Distributed Multi-Agent RL-Based Autonomous Spectrum Allocation Scheme in D2D Enabled Multi-Tier HetNets

Multi-tier heterogeneous networks (HetNets) and device-to-device (D2D) communication are vastly considered in 5G networks. The interference mitigation and resource allocation in the D2D enabled multi-tier HetNets is a cumbersome and challenging task that cannot be solved by the conventional centralized resource allocation techniques proposed in the literature. In this paper, we propose a distributed multi-agent learning-based spectrum allocation scheme in which D2D users learn the wireless environment and select spectrum resources autonomously to maximize their throughput and spectral efficiency (SE) while causing minimum interference to the cellular users. We have employed the distributed learning in a stochastic geometry-based realistic multi-tier heterogeneous network to validate the performance of our scheme. The proposed scheme enables the D2D users to achieve higher throughput and SE, higher signal-to-interference-plus-noise ratio and low outage ratio for cellular users, and better computational time efficiency and performs well in the dense multi-tier HetNets without affecting network coverage compared with the distance based resource criterion and joint-resource allocation and link adaptation schemes.