Mixture-of-Gaussian clustering-based decision technique for a coherent optical communication system

A decision technique using mixture-of-Gaussian (MoG) clustering algorithms is proposed in the context of a coherent optical communication system. For an 80-G b/s single-carrier polarization-division multiplexed 16 quadrature amplitude modulation (QAM) transmission system at 975 km, an improvement in Q factors up to 0.41 dB is observed for the entire range of the considered optical signal-to-noise SNR. Compared with the traditional minimum Euclidean distance-based decision, the MoG clustering-based decision achieves a transmission distance increase of175 km at a Q factor of 9.96 dB. Experiment results show that the proposed decision technique is insensitive to the system's nonlinear impairments and can effectively improve nonlinear tolerance of the system. We also propose a majorization method for decision-directed least mean square (LMS) using the MoG clustering-based decision algorithm, called MoG-LMS. The performance improvement of the MoG-LMS algorithm is verified by experiments. (C) 2019 Optical Society of America