Linear-shrinkage-based DOA estimation for coherently distributed sources considering mutual coupling in massive MIMO systems

In this paper, an approach of robust direction-of-arrival (DOA) estimation for coherently distributed (CD) sources considering the influence of array mutual coupling is proposed in a large-scale/massive multiple-input-multiple-output (MIMO) system. By exploiting the special structure of coupling matrix, the proposed approach first eliminates its influence through the inherent mechanism. Considering the realistic scenario that the number of available samples is of the same order of magnitude than the number of sensors in massive MIMO systems, the approach then improves the estimation of the sample covariance matrix (SCM) via the linear shrinkage operation, which is known as an efficient strategy to enhance the estimation of SCM in large-scale uniform linear arrays. Finally, according to the linear shrinkage based minimum description length (LS-MDL) criterion and the improved SCM, DOAs are obtained in closed-form expressions. Numerical results are included for illustrating the effectiveness of the proposed approach.