Benchmarking Neural Capacity Estimation: Viability and Reliability

Recently, several methods have been proposed for estimating the mutual information from sample data using deep neural networks. This approach is referred to as neural mutual information estimation (NMIE). NMIEs differ from other approaches in the literature as they are data-driven estimators. As such, they have the potential to perform well on a large class of capacity problems. To test the performance across various NMIEs, it is desirable to establish a benchmark encompassing the different challenges of capacity estimation. This is the objective of this paper. We consider three scenarios for benchmarking: (i) the classic AWGN channel, (ii) channels continuous inputs - the optical intensity and peak-power constrained AWGN channel (iii) channels with a discrete output - i.e., the Poisson channel. We also consider the extension to the multi-terminal case with (iv) the AWGN and optical MAC models. We argue that benchmarking a certain NMIE across these four scenarios provides a substantive test of performance. We study the performance of mutual information neural estimator (MINE), smoothed mutual information lower-bound estimator (SMILE), and directed information neural estimator (DINE) and provide insights into the performance of other methods as well. To summarize our benchmarking results, MINE provides the most reliable performance.

Automated summary

Recently, deep neural networks have been used to estimate the mutual information from sample data, a method known as Neural Mutual Information Estimation (NMIE). NMIEs are different from other approaches as they are data-driven estimators, thus they have the potential to perform well on a wide range of capacity problems. This paper aims to establish a benchmark for testing the performance of various NMIEs across different challenges of capacity estimation. The benchmark includes scenarios such as the classic AWGN channel, channels with continuous inputs and discrete outputs, and the extension to multi-terminal cases. The results show that the Mutual Information Neural Estimator (MINE) provides the most reliable performance.