Comparing Network Structures on Three Aspects: A Permutation Test

Translational Abstract The network approach, in which psychological constructs are modeled in terms of interactions between their constituent factors, have rapidly gained popularity in psychology. Applications of such network approaches to various psychological constructs have recently moved from a descriptive stance, in which the goal is to estimate the network structure, to a more comparative stance, in which the goal is to compare network structures across groups. However, the statistical tools to do so are lacking. In this article, we present the network comparison test (NCT). NCT is a statistical test that compares two network structures on three types of characteristics. Performance of NCT is evaluated by means of a simulation study. Simulated data shows that NCT performs well in various circumstances for all three tests: when the groups are simulated to be similar, the error rate (i.e., NCT indicating that they are different, while the simulated networks are similar) is adequately low, and when the groups are simulated to be different, the ability to detect a difference is sufficiently high when the difference between simulated networks and the sample size are substantial. We illustrate NCT by comparing depression symptom networks of males and females. Possible extensions of NCT are discussed. Network approaches to psychometric constructs, in which constructs are modeled in terms of interactions between their constituent factors, have rapidly gained popularity in psychology. Applications of such network approaches to various psychological constructs have recently moved from a descriptive stance, in which the goal is to estimate the network structure that pertains to a construct, to a more comparative stance, in which the goal is to compare network structures across populations. However, the statistical tools to do so are lacking. In this article, we present the network comparison test (NCT), which uses resampling-based permutation testing to compare network structures from two independent, cross-sectional data sets on invariance of (a) network structure, (b) edge (connection) strength, and (c) global strength. Performance of NCT is evaluated in simulations that show NCT to perform well in various circumstances for all three tests: The Type I error rate is close to the nominal significance level, and power proves sufficiently high if sample size and difference between networks are substantial. We illustrate NCT by comparing depression symptom networks of males and females. Possible extensions of NCT are discussed.

Automated summary

The network approach, which models psychological constructs in terms of the interactions between their constituent factors, has gained popularity in psychology. Recently, the focus has shifted from describing network structures to comparing them across different groups. However, there is a lack of statistical tools for this purpose. In this article, the authors present the network comparison test (NCT), a statistical test that compares network structures on three different characteristics. The performance of NCT is evaluated through simulations, showing that it performs well in various circumstances. The authors illustrate the application of NCT by comparing depression symptom networks of males and females. Possible extensions of NCT are also discussed.