Integration of deep learning and soft robotics for a biomimetic approach to nonlinear sensing

Bats with sophisticated biosonar systems move their ears at a high speed to help localize sound sources. Yin and Muller present a system inspired by this strategy, which can localize sounds with high accuracy and with a single detector, using a flexible silicone model of a bat's ear and a deep convolutional neural network to process the complex Doppler signatures. Traditional approaches to sensing have often been aimed at simple sensor characteristics to make interpretation of the sensor outputs easier, but this has also limited the quality of the encoded sensory information. Integrating a complex sensor with deep learning could hence be a strategy for removing current limitations on the information that sensory inputs can carry. Here, we demonstrate this concept with a soft-robotic sensor that mimics fast non-rigid deformation of the ears in certain bat species. We show that a deep convolutional neural network can use the nonlinear Doppler shift signatures generated by these motions to estimate the direction of a sound source with an estimation error of similar to 0.5 degrees. Previously, determining the direction of a sound source based on pressure receivers required either multiple frequencies or multiple receivers. Our current results demonstrate a third approach that makes do with only a single frequency and a single receiver.

Automated summary

Researchers have developed a system inspired by bats' biosonar systems, using a deep convolutional neural network to accurately localize sound sources. Their innovation allows sensors to better process complex sound features, enabling precise sound localization with a single detector.