Reconfigurable Activation Functions in Integrated Optical Neural Networks

The implementation of nonlinear activation functions is one of the key challenges that optical neural networks face. To the date, different approaches have been proposed, including switching to digital implementations, electro-optical or all optical. In this article, we compare the response of different electro-optic architectures where part of the input optical signal is converted into the electrical domain and used to self-phase modulate the intensity of the remaining optical signal. These architectures are made up of Mach Zehnder Interferometers (MZI) and microring resonators (MRR). We have compared the corresponding transfer functions with commonly used activation functions in state-of-the-art machine learning models and carried out an in-depth analysis of the capabilities of those architectures to generate the proposed activation functions. We demonstrate that a ring assisted MZI and a two-ring assisted MZI present the highest expressivity among the proposed structures. To the best of our knowledge, this is the first time that a quantified analysis of the capabilities of optical devices to mimic state-of-the-art activation functions is presented. The obtained activation functions are benchmarked on two machine learning examples: classification task using the Iris dataset, and image recognition using the MNIST dataset. We use complex-valued feed-forward neural networks and get test accuracies of 97% and 95% respectively.

Automated summary

This article discusses the implementation of nonlinear activation functions in optical neural networks and compares the response of different electro-optic architectures. The study demonstrates that ring assisted MZI and two-ring assisted MZI have the highest expressivity among the proposed structures. The article also presents a quantitative analysis of the capabilities of optical devices to mimic state-of-the-art activation functions, and benchmarks the obtained activation functions on two machine learning examples.