Journal
ENTROPY
Volume 22, Issue 2, Pages -Publisher
MDPI
DOI: 10.3390/e22020151
Keywords
information bottleneck; rate distortion theory; logarithmic loss; representation learning
Categories
Funding
- European Union's Horizon 2020 Research And Innovation Programme [694630]
- WIN consortium
Ask authors/readers for more resources
This tutorial paper focuses on the variants of the bottleneck problem taking an information theoretic perspective and discusses practical methods to solve it, as well as its connection to coding and learning aspects. The intimate connections of this setting to remote source-coding under logarithmic loss distortion measure, information combining, common reconstruction, the Wyner-Ahlswede-Korner problem, the efficiency of investment information, as well as, generalization, variational inference, representation learning, autoencoders, and others are highlighted. We discuss its extension to the distributed information bottleneck problem with emphasis on the Gaussian model and highlight the basic connections to the uplink Cloud Radio Access Networks (CRAN) with oblivious processing. For this model, the optimal trade-offs between relevance (i.e., information) and complexity (i.e., rates) in the discrete and vector Gaussian frameworks is determined. In the concluding outlook, some interesting problems are mentioned such as the characterization of the optimal inputs (features) distributions under power limitations maximizing the relevance for the Gaussian information bottleneck, under complexity constraints.
Authors
I am an author on this paper
Click your name to claim this paper and add it to your profile.
Reviews
Recommended
No Data Available