4.7 Article

Extracting and inserting knowledge into stacked denoising auto-encoders

Journal

NEURAL NETWORKS
Volume 137, Issue -, Pages 31-42

Publisher

PERGAMON-ELSEVIER SCIENCE LTD
DOI: 10.1016/j.neunet.2021.01.010

Keywords

Deep learning; Feature learning; Stacked denoised auto-encoders; Knowledge discovery; Knowledge insertion

Funding

  1. National Natural Science Foundation of China [71777173]
  2. Fund Projects in Common Technology Field of Equipment Pre Research [61400020119]
  3. Fundamental Research Funds for the Central Universities

Ask authors/readers for more resources

This paper introduces a new deep neural network model KBSDAE, which enhances the understanding of representations learned by the deep network and improves the learning performance of stacked denoising auto-encoder. It achieves this by inserting knowledge into the deep network structure, offering a novel method for knowledge insertion and showing better feature learning performance compared to typical DNNs.
Deep neural networks (DNNs) with a complex structure and multiple nonlinear processing units have achieved great successes for feature learning in image and visualization analysis. Due to interpretability of the black box problem in DNNs, however, there are still many obstacles to applications of DNNs in various real-world cases. This paper proposes a new DNN model, knowledge-based deep stacked denoising auto-encoders (KBSDAE), which inserts the knowledge (i.e., confidence and classification rules) into the deep network structure. This model not only can offer a good understanding of the representations learned by the deep network but also can produce an improvement in the learning performance of stacked denoising auto-encoder (SDAE). The knowledge discovery algorithm is proposed to extract confidence rules to interpret the layerwise network (i.e., denoising auto-encoder (DAE)). The symbolic language is developed to describe the deep network and shows that it is suitable for the representation of quantitative reasoning in a deep network. The confidence rule insertion to the deep network is able to produce an improvement in feature learning of DAEs. The classification rules extracted from the data offer a novel method for knowledge insertion to the classification layer of SDAE. The testing results of KBSDAE on various benchmark data indicate that the proposed method not only effectively extracts knowledge from the deep network, but also shows better feature learning performance than that of those typical DNNs (e.g., SDAE). (C) 2021 Elsevier Ltd. All rights reserved.

Authors

I am an author on this paper
Click your name to claim this paper and add it to your profile.

Reviews

Primary Rating

4.7
Not enough ratings

Secondary Ratings

Novelty
-
Significance
-
Scientific rigor
-
Rate this paper

Recommended

No Data Available
No Data Available