The generalizability of machine learning models of personality across two text domains

Machine learning of high-dimensional models have received attention for their ability to predict psychological variables, such as personality. However, it has been less examined to what degree such models are capable of generalizing across domains. Across two text domains (Reddit message and personal essays), compared to lowdimensional- and theoretical models, atheoretical high-dimensional models provided superior predictive accuracy within but poor/non-significant predictive accuracy across domains. Thus, complex models depended more on the specifics of the trained domain. Further, when examining predictors of models, few survived across domains. We argue that theory remains important when conducting prediction-focused studies and that research on both high- and low-dimensional models benefit from establishing conditions under which they generalize.

Automated summary

Machine learning of high-dimensional models can predict psychological variables accurately within specific domains, but their ability to generalize across domains is limited. Complex models are highly influenced by the specifics of the trained domain, and only a few predictors can survive in different domains.