Machine learning-based segmentation of the rodent hippocampal CA2 area from Nissl-stained sections

The hippocampus is a center of learning, memory, and spatial navigation. This region is divided into the CA1, CA2, and CA3 areas, which are anatomically different from each other. Among these divisions, the CA2 area is unique in terms of functional relevance to sociality. The CA2 area is often manually detected based on the size, shape, and density of neurons in the hippocampal pyramidal cell layer, but this manual segmentation relying on cytoarchitecture is impractical to apply to a large number of samples and dependent on experimenters' proficiency. Moreover, the CA2 area has been defined based on expression pattern of molecular marker proteins, but it generally takes days to complete immunostaining for such proteins. Thus, we asked whether the CA2 area can be systematically segmented based on cytoarchitecture alone. Since the expression pattern of regulator of G-protein signaling 14 (RGS14) signifies the CA2 area, we visualized the CA2 area in the mouse hippocampus by RGS14-immunostaining and Nissl-counterstaining and manually delineated the CA2 area. We then established CAseg, a machine learning-based automated algorithm to segment the CA2 area with the F1-score of approximately 0.8 solely from Nissl-counterstained images that visualized cytoarchitecture. CAseg was extended to the segmentation of the prairie vole CA2 area, which raises the possibility that the use of this algorithm can be expanded to other species. Thus, CAseg will be beneficial for investigating unique properties of the hippocampal CA2 area.

Automated summary

The hippocampus is a key area for learning, memory, and spatial navigation. The CA2 region is particularly important for social behavior. Manual segmentation methods are impractical and dependent on experimenters' proficiency, so we developed CAseg, an automated algorithm based on machine learning, to accurately segment the CA2 area from microscopic images. This algorithm will be valuable for investigating the unique properties of the hippocampal CA2 area.