A hierarchical 3D-motion learning framework for animal spontaneous behavior mapping

Animal behavior usually has a hierarchical structure and dynamics. Therefore, to understand how the neural system coordinates with behaviors, neuroscientists need a quantitative description of the hierarchical dynamics of different behaviors. However, the recent end-to-end machine-learning-based methods for behavior analysis mostly focus on recognizing behavioral identities on a static timescale or based on limited observations. These approaches usually lose rich dynamic information on cross-scale behaviors. Here, inspired by the natural structure of animal behaviors, we address this challenge by proposing a parallel and multi-layered framework to learn the hierarchical dynamics and generate an objective metric to map the behavior into the feature space. In addition, we characterize the animal 3D kinematics with our low-cost and efficient multi-view 3D animal motion-capture system. Finally, we demonstrate that this framework can monitor spontaneous behavior and automatically identify the behavioral phenotypes of the transgenic animal disease model. The extensive experiment results suggest that our framework has a wide range of applications, including animal disease model phenotyping and the relationships modeling between the neural circuits and behavior. Animal behavior usually has a hierarchical structure and dynamics. Here, the authors propose a parallel and multi-layered framework to learn the hierarchical dynamics and generate an objective metric to map the behaviour into the feature space.

Automated summary

The study introduces a parallel and multi-layered framework to learn the hierarchical dynamics of animal behavior and map behavior into the feature space objectively. Experimental results suggest that this framework has wide applications, including modeling animal disease model phenotypes and relationships between neural circuits and behavior.