Restricted Boltzmann machine: Recent advances and mean-field theory*

This review deals with restricted Boltzmann machine (RBM) under the light of statistical physics. The RBM is a classical family of machine learning (ML) models which played a central role in the development of deep learning. Viewing it as a spin glass model and exhibiting various links with other models of statistical physics, we gather recent results dealing with mean-field theory in this context. First the functioning of the RBM can be analyzed via the phase diagrams obtained for various statistical ensembles of RBM, leading in particular to identify a compositional phase where a small number of features or modes are combined to form complex patterns. Then we discuss recent works either able to devise mean-field based learning algorithms; either able to reproduce generic aspects of the learning process from some ensemble dynamics equations or/and from linear stability arguments.

Automated summary

This review focuses on the application of restricted Boltzmann machines in statistical physics, discussing mean-field theory and phase diagram analysis of RBMs in machine learning. Recent works on mean-field based learning algorithms and reproducing aspects of the learning process from ensemble dynamics equations or linear stability arguments are also discussed.