Optimizing the Length, Width, and Balance of a Personality Scale: How Do Internal Characteristics Affect External Validity?

How well can scores on a personality scale predict criteria such as behaviors and life outcomes? This question concerns external validity, which is a core aspect of personality assessment. The present research was conducted to examine how external validity is influenced by a trait scale's internal characteristics, such as its length (number of items), width (breadth of content), and balance (between positively and negatively keyed items). Participants completed the Big Five Inventory-2 (BFI-2), and were also assessed on a set of self-reported and peer-reported validity criteria. We used the BFI-2 item pool to construct multiple versions, or iterations, of each Big Five trait scale that varied in terms of length, width, and balance. We then identified systematic effects of these internal scale characteristics on external validity associations. Regarding length, we find that longer trait scales tend to have greater validity, with a scale length sweet spot of approximately 6 to 9 items. Regarding width, we find that broad trait scales tend to have slightly stronger, and much more consistent, associations with external validity criteria than do narrow scales; broad scales thus represent relatively safe bets for personality assessment, whereas narrow scales carry greater risks but offer potentially greater rewards. Regarding balance, we find that associations between imbalanced trait and criterion scales can be substantially inflated or suppressed by acquiescent responding; trait scales that include an equal number of positively and negatively keyed items can minimize such acquiescence bias. We conclude by translating these findings into practical advice regarding psychological assessment.