Reliability and comparability of human brain structural covariance networks

Structural covariance analysis is a widely used structural MRI analysis method which characterises the co-relations of morphology between brain regions over a group of subjects. To our knowledge, little has been investigated in terms of the comparability of results between different data sets of healthy human subjects, as well as the reliability of results over the same subjects in different rescan sessions, image resolutions, or FreeSurfer versions. In terms of comparability, our results show substantial differences in the structural covariance matrix between data sets of age- and sex-matched healthy human adults. These differences persist after univariate site correction, they are exacerbated by low sample sizes, and they are most pronounced when using average cortical thickness as a morphological measure. Down-stream graph theoretic analyses further show statistically significant differences. In terms of reliability, substantial differences were also found when comparing repeated scan sessions of the same subjects, image resolutions, and even FreeSurfer versions of the same image. We could further estimate the relative measurement error and showed that it is largest when using cortical thickness as a morphological measure. Using simulated data, we argue that cortical thickness is least reliable because of larger relative measurement errors. Practically, we make the following recommendations (1) combining subjects across sites into one group should be avoided, particularly if sites differ in image resolutions, subject demographics, or preprocessing steps; (2) surface area and volume should be preferred as morphological measures over cortical thickness; (3) a large number of subjects (n >> 30 for the Desikan-Killiany parcellation) should be used to estimate structural covariance; (4) measurement error should be assessed where repeated measurements are available; (5) if combining sites is critical, univariate (per ROI) site-correction is insufficient, but error covariance (between ROIs) should be explicitly measured and modelled.