A new statistical approach for assessing similarity of species composition with incidence and abundance data

The classic Jaccard and Sorensen indices of compositional similarity (and other indices that depend upon the same variables) are notoriously sensitive to sample size, especially for assemblages with numerous rare species. Further, because these indices are based solely on presence-absence data, accurate estimators for them are unattainable. We provide a probabilistic derivation for the classic, incidence-based forms of these indices and extend this approach to formulate new Jaccard-type or Sorensen-type indices based on species abundance data. We then propose estimators for these indices that include the effect of unseen shared species, based on either (replicated) incidence- or abundance-based sample data. In sampling simulations, these new estimators prove to be considerably less biased than classic indices when a substantial proportion of species are missing from samples. Based on species-rich empirical datasets, we show how incorporating the effect of unseen shared species not only increases accuracy but also can change the interpretation of results.