Misbegotten Methodologies and Forgotten Lessons From Tom Swift's Electric Factor Analysis Machine: A Demonstration With Competing Structural Models of Psychopathology

Translational Abstract Confirmatory factor analysis (CFA) is the most popular method in model comparison studies involving bifactor structures of psychological constructs. However, notable pitfalls are associated with myopic approaches to building and testing bifactor CFA models, which can distort results, limit generalizability, and impede theory development. The overarching aims of this study are to provide applied researchers with a conceptually clear roadmap for evaluating latent factor structures, present a comprehensive accounting of theoretical and methodological challenges that arise in both confirmatory and exploratory factor analytic scenarios, and highlight the benefits of their combined use. These general issues are illustrated in an applied example featuring a frequently used dataset in the field of quantitative psychopathology. Several statistical and conceptual similarities emerged across bifactor CFA and EFA models. Results are situated within the broader literature on structural models of psychopathology, and implications for p-factor theories are discussed. The historical narrative that is woven into this study traces recurring themes of the century-old general factor debate to caution readers against the development of theories on the basis of a single factor analytic solution. Confirmatory factor analysis (CFA) and its bifactor models are popular in empirical investigations of the factor structure of psychological constructs. CFA offers straightforward hypothesis testing but has notable pitfalls, such as the imposition of strict assumptions (i.e., simple structure) that obscure unmodeled complexity. Due to the limitations of bifactor CFAs, they have yielded anomalous results across samples and studies that suggest model misspecification (e.g., evaporating specific factors and unexpected loadings). We propose the use of exploratory factor analysis (EFA) to evaluate the structural validity of CFA solutions-either before or after the estimation of more restrictive CFA models-to (a) identify model misspecifications that may drive anomalous estimates and (b) confirm CFA models by examining whether hypothesized structures emerge with limited researcher input. We evaluated the degree to which predominant factor structures were invariant across contexts along the exploratory-confirmatory continuum and demonstrate how poor methodological choices can distort results and impede theory development. All CFA models fit well, but there were numerous differences in replicability and substantive interpretability. Several similarities emerged between bifactor CFA and EFA models, including evidence of overextraction, the collapse of specific factors onto the general factor, and subsequent shifts in how the general factor was defined. We situate these methodological shortcomings within the broader literature on structural models of psychopathology, articulate implications for theories (such as the p-factor) that are borne out of factor analysis, outline several remedies for problems encountered when performing exploratory bifactor analysis, and propose alternative specifications for confirmatory bifactor models.

Automated summary

This study aims to provide applied researchers with a clear roadmap for evaluating latent factor structures and discusses the challenges in both confirmatory and exploratory factor analytic scenarios. The similarities between bifactor CFA and EFA models are illustrated through an applied example and the implications for psychopathology structural models are discussed.