A semi-supervised auto-encoder using label and sparse regularizations for classification

The semi-supervised auto-encoder (SSAE) is a promising deep-learning method that integrates the advantages of unsupervised and supervised learning processes. The former learning process is designed to extract the underlying concepts of data as intrinsic information and enhance its generalization ability to express data. Furthermore, the supervised process tends to describe the rules of categorization with labels that further improve categorization accuracy. In this paper, we propose a novel semi-supervised learning method, namely, label and sparse regularization AE (LSRAE), by integrating label and sparse constraints to update the structure of the AE. The sparse regularization activates a minority of important neurons, while most of the other neurons are inhibited. Such a method ensures that LSRAE can yield a more local and informative structure of the data. Moreover, by implementing the label constraint, the supervised learning process can extract the features regulated by category rules and enhance the performance of the classifier in depth. To extensively test the performances of LSRAE, we perform our experiments on the benchmark datasets USPS, ISOLET and MNIST. The experimental results demonstrate the superiority of LSRAE in comparison with state-of-the-art feature extraction methods including AE, LSAE, SAE, ELM, DBN, and adaptive DBN. (C) 2019 Elsevier B.V. All rights reserved.