Comparison of accuracy between FSL's FIRST and Freesurfer for caudate nucleus and putamen segmentation

Although several methods have been developed to automatically delineate subcortical gray matter structures from MR images, the accuracy of these algorithms has not been comprehensively examined. Most of earlier studies focused primarily on the hippocampus. Here, we assessed the accuracy of two widely used non-commercial programs (FSL-FIRST and Freesurfer) for segmenting the caudate and putamen. T1-weighted 1 mm(3) isotropic resolution MR images were acquired for thirty healthy subjects (15 females). Caudate nucleus and putamen were segmented manually by two independent observers and automatically by FIRST and Freesurfer (v4.5 and v5.3). Utilizing manual labels as reference standard the following measures were studied: Dice coefficient (D), percentage volume difference (PVD), absolute volume difference as well as intraclass correlation coefficient (ICC) for consistency and absolute agreement. For putamen segmentation, FIRST achieved higher D, lower PVD and higher ICC for absolute agreement with manual tracing than either version of Freesurfer. Freesurfer overestimated the putamen, while FIRST was not statistically different from manual tracing. The ICC for consistency with manual tracing was similar between the two methods. For caudate segmentation, FIRST and Freesurfer performed more similarly. In conclusion, Freesurfer and FIRST are not equivalent when comparing to manual tracing. FIRST was superior for putaminal segmentation.