Steepest descent algorithms for optimization under unitary matrix constraint

In many engineering applications we deal with constrained optimization problems with respect to complex-valued matrices. This paper proposes a Riemannian geometry approach for optimization of a real-valued cost function J of complex-valued matrix argument W, under the constraint that W is an n x n unitary matrix. We derive steepest descent (SD) algorithms on the Lie group of unitary matrices U(n). The proposed algorithms move towards the optimum along the geodesics, but other alternatives are also considered. We also address the computational complexity and the numerical stability issues considering both the geodesic and the nongeodesic SD algorithms. Armijo step size [1] adaptation rule is used similarly to [2], but with reduced complexity. The theoretical results are validated by computer simulations. The proposed algorithms are applied to blind source separation in MIMO systems by using the joint diagonalization approach [3]. We show that the proposed algorithms outperform other widely used algorithms.