A Gaussian RBM with binary auxiliary units

Restricted Boltzmann Machines (RBM) have been widely applied in image processing. For RBM-based models on image recognition and image generation tasks, extracting expressive real-valued features and alleviating the overfitting problem are extremely important. In this paper, we propose a Gaussian Restricted Boltzmann Machine with binary Auxiliary units (GARBM), which designs binary auxiliary units in its visible layer and constructs parameterized real-valued features in its hidden layer. Specifically, based on the designed energy function in GARBM, activated auxiliary units are directly used to control probabilities of visible units and hidden units to extract real-valued features. Moreover, auxiliary units and their resulting feature selection mechanism not only alleviate the gradient-variance problem, but also provide certain randomness to other units to alleviate overfitting without introducing more hyperparameters. To build more effective deep models, we propose GARBM-based deep neural networks, and the effectiveness of proposed neural networks is verified in experiments.

Automated summary

The paper proposes a Gaussian Restricted Boltzmann Machine with binary Auxiliary units (GARBM) for image processing, which can extract real-valued features and alleviate the overfitting problem. By designing binary auxiliary units in the visible layer and constructing parameterized real-valued features in the hidden layer, the proposed GARBM-based deep neural networks achieve effective image recognition and generation tasks.