Reproducible brain-wide association studies require thousands of individuals

Magnetic resonance imaging (MRI) hastransformed our understanding ofthe human brain through well-replicated mapping of abilitiesto specific structures (for example, lesion studies) and functions(1-3) (for example, task functional MRI (fMRI)). Mental health research and care have yet to realize similar advances from MRI. A primary challenge has been replicating associations between inter-individual differences in brain structure or function and complex cognitive or mental health phenotypes (brain-wide association studies (BWAS)). Such BWAS have typically relied on sample sizes appropriate for classical brain mapping(4) (the median neuroimaging study sample size is about 25), but potentially too small for capturing reproducible brainbehavioural phenotype associations(5,6). Here we used three ofthe largest neuroimaging datasets currently available-with a total sample size of around 50,000 individuals-to quantify BWAS effect sizes and reproducibility as a function of sample size. BWAS associations were smallerthan previously thought, resulting in statistically underpowered studies, inflated effect sizes and replication failures at typical sample sizes. As sample sizes grew into the thousands, replication rates began to improve and effect size inflation decreased. More robust BWAS effects were detected for functional MRI (versus structural), cognitive tests (versus mental health questionnaires) and multivariate methods (versus univariate). Smaller than expected brain-phenotype associations and variability across population subsamples can explain widespread BWAS replication failures. In contrast to non-BWAS approaches with larger effects (for example, lesions, interventions and within-person), BWAS reproducibility requires samples with thousands of individuals.

Automated summary

Magnetic resonance imaging (MRI) has greatly advanced our understanding of the human brain, but similar progress has not been made in mental health research. By analyzing three large neuroimaging datasets, researchers found that brain-wide association studies (BWAS) had smaller effect sizes than previously thought, leading to underpowered studies, inflated effect sizes, and replication failures. As sample sizes increased, replication rates improved and effect size inflation decreased. Functional MRI, cognitive tests, and multivariate methods showed more robust effects in BWAS. Smaller than expected brain-phenotype associations and population subsample variability explain the widespread replication failures in BWAS. Reproducibility in BWAS requires samples with thousands of individuals.