Properties of bias and variance of two multiallelic estimators of FST

This study compares theoretical and simulated properties of two estimators of fixation indices F-ST for multiallelic data from a single locus. The estimators are due to Weir and Cockerham (<(theta)over cap>(WC)) and to Robertson and Hill (<(theta)over cap>(RH)), respectively. Both estimators are linear combinations of biallelic estimators which differ in the way frequent and rare alleles are weighted. Consequently, their sampling properties differ as far as bias and variance are concerned. In the infinite island model at migration-drift equilibrium, in the case of one multiallelic locus, we analytically approximate the bias of the two estimators and show that <(theta)over cap>(WC) is nearly unbiased, whereas <(theta)over cap>(RH) is negatively biased. Hence, we propose a correction of bias of the latter. Moreover, we reexamine the properties of variance of the initial estimators: due to their construction, their variances are minimal, each over different parameter ranges, <(theta)over cap>(RH) performing better for low differentiation and <(theta)over cap>(WC) for high differentiation. These theoretical properties are confirmed by simulations, which show that our correction of bias of <(theta)over cap>(RH) performs well and does not affect its property of minimal variance for low differentiation. Hence, we advocate the use of <(theta)over cap>(WC) for high values of differentiation, and the use of <(theta)over cap>(RH), with our correction of bias, for low or moderate differentiation. (C) 2000 Academic Press.