Journal
ENTROPY
Volume 20, Issue 8, Pages -Publisher
MDPI
DOI: 10.3390/e20080583
Keywords
born machine; tensor network; mutual information
Categories
Funding
- National R&D Program of China [2017YFA0302901]
- National Natural Science Foundation of China [11190024, 11474331, 11774398]
- Ministry of Science and Technology of China [2016YFA0300603]
Ask authors/readers for more resources
We compare and contrast the statistical physics and quantum physics inspired approaches for unsupervised generative modeling of classical data. The two approaches represent probabilities of observed data using energy-based models and quantum states, respectively. Classical and quantum information patterns of the target datasets therefore provide principled guidelines for structural design and learning in these two approaches. Taking the Restricted Boltzmann Machines (RBM) as an example, we analyze the information theoretical bounds of the two approaches. We also estimate the classical mutual information of the standard MNIST datasets and the quantum Renyi entropy of corresponding Matrix Product States (MPS) representations. Both information measures are much smaller compared to their theoretical upper bound and exhibit similar patterns, which imply a common inductive bias of low information complexity. By comparing the performance of RBM with various architectures on the standard MNIST datasets, we found that the RBM with local sparse connection exhibit high learning efficiency, which supports the application of tensor network states in machine learning problems.
Authors
I am an author on this paper
Click your name to claim this paper and add it to your profile.
Reviews
Recommended
No Data Available