End-to-End Optimization of Coherent Optical Communications Over the Split-Step Fourier Method Guided by the Nonlinear Fourier Transform Theory

Optimizing modulation and detection strategies for a given channel is critical to maximizing the throughput of a communication system. Such an optimization can be easily carried out analytically for channels that admit closed-form analytical models. However, this task becomes extremely challenging for nonlinear dispersive channels such as the optical fiber. End-to-end optimization through autoencoders (AEs) can be applied to define symbol-to-waveform (modulation) and waveform-to-symbol (detection) mappings, but so far it has been mainly shown for systems relying on approximate channel models. Here, for the first time, we propose an AE scheme applied to the full optical channel described by the nonlinear Schrodinger equation (NLSE). Transmitter and receiver are jointly optimized through the split-step Fourier method (SSFM) which accurately models an optical fiber. In this first numerical analysis, the detection is performed by a neural network (NN), whereas the symbol-to-waveform mapping is aided by the nonlinear Fourier transform (NFT) theory in order to simplify and guide the optimization on the modulation side. This proof-of-concept AE scheme is thus benchmarked against a manually-optimized NFT-based system and a three-fold increase in achievable distance (from 2000 to 6640 km) is demonstrated.

Automated summary

Optimizing modulation and detection strategies is crucial for maximizing communication system throughput, which can be easily carried out analytically for linear channels but becomes challenging for nonlinear channels such as optical fibers. A novel autoencoder scheme applied to the full optical channel described by the nonlinear Schrodinger equation is proposed in this study, demonstrating a significant improvement in system performance.