A convolutional neural network provides a generalizable model of natural sound coding by neural populations in auditory cortex

Author summarySounds in the natural world are composed of complex, dynamic spectro-temporal features. The brain's auditory system is able to identify and extract meaningful patterns from acoustic inputs, including in the presence of noise and other competing sounds. A better understanding of auditory neural computation may inform algorithms for speech processing and auditory prosthetics. Despite their importance, current computational models have limited success explaining neural sound coding, particularly in auditory cortex. This study used convolutional neural networks (CNNs) to model the functional relationship between a large set of natural sounds and the activity of neurons in auditory cortex. The CNNs substantially outperformed several previously proposed models of auditory coding. Moreover, they were able to generalize. After training on data from one set of neurons, they could be trained easily on a new set of neurons. This finding suggests that the CNN models characterize a general space of spectro-temporal patterns encoded by the population of neurons in auditory cortex. A better understanding of this brain-derived space may be useful for improved signal processing algorithms. Convolutional neural networks (CNNs) can provide powerful and flexible models of neural sensory processing. However, the utility of CNNs in studying the auditory system has been limited by their requirement for large datasets and the complex response properties of single auditory neurons. To address these limitations, we developed a population encoding model: a CNN that simultaneously predicts activity of several hundred neurons recorded during presentation of a large set of natural sounds. This approach defines a shared spectro-temporal space and pools statistical power across neurons. Population models of varying architecture performed consistently and substantially better than traditional linear-nonlinear models on data from primary and non-primary auditory cortex. Moreover, population models were highly generalizable. The output layer of a model pre-trained on one population of neurons could be fit to data from novel single units, achieving performance equivalent to that of neurons in the original fit data. This ability to generalize suggests that population encoding models capture a complete representational space across neurons in an auditory cortical field.

Automated summary

This study used convolutional neural networks (CNNs) to model the functional relationship between natural sounds and the activity of neurons in auditory cortex. CNNs outperformed previous models of auditory coding and could be easily trained on new sets of neurons. This finding suggests that CNN models capture the complete representational space across neurons in the auditory cortex, which is useful for improving signal processing algorithms.