Effective Dimensionality: A Tutorial

The topic of this tutorial is the effective dimensionality (ED) of a dataset, that is, the equivalent number of orthogonal dimensions that would produce the same overall pattern of covariation. The ED quantifies the total dimensionality of a set of variables, with no assumptions about their underlying structure. The ED of a dataset has important implications for the curse of dimensionality; it can be used to inform decisions about data analysis and answer meaningful empirical questions. The tutorial offers an accessible introduction to ED, distinguishes it from the related but distinct concept of intrinsic dimensionality, critically reviews various ED estimators, and gives indications for practical use with examples from personality research. An R function is provided to implement the techniques described in the tutorial.

Automated summary

This tutorial discusses the concept, importance, and evaluation of the effective dimensionality (ED) of a dataset. It highlights the importance of ED in guiding decisions for data analysis and answering empirical questions.