Dynamical Mechanism of Sampling-Based Probabilistic Inference Under Probabilistic Population Codes

Animals make efficient probabilistic inferences based on uncertain and noisy information from the outside environment. It is known that probabilistic population codes, which have been proposed as a neural basis for encoding probability distributions, allow general neural networks (NNs) to perform near-optimal point estimation. However, the mechanism of sampling-based probabilistic inference has not been clarified. In this study, we trained two types of artificial NNs, feedforward NN (FFNN) and recurrent NN (RNN), to perform sampling-based probabilistic inference. Then we analyzed and compared their mechanisms of sampling. We found that sampling in RNN was performed by a mechanism that efficiently uses the properties of dynamical systems, unlike FFNN. In addition, we found that sampling in RNNs acted as an inductive bias, enabling a more accurate estimation than in maximum a posteriori estimation. These results provide important arguments for discussing the relationship between dynamical systems and information processing in NNs.

Automated summary

This study trained two types of artificial neural networks, feedforward neural network (FFNN) and recurrent neural network (RNN), to perform sampling-based probabilistic inference. It found that the sampling mechanism in RNN efficiently utilizes the properties of dynamical systems, unlike FFNN. Additionally, the study found that sampling in RNNs provides an inductive bias and enables more accurate estimation than maximum a posteriori estimation.