On stochastic versions of the EM algorithm

A previously proposed stochastic modification of the EM algorithm is discussed, in which an intractable E-step is replaced by a single simulation of the complete data, followed by averaging of the resulting Markov chain iterative sequence. A connection is drawn between this approach and a modified Eh I algorithm in which the E- and M-steps are carried out in reverse order. Since this modified EM algorithm is equivalent to solving a biased estimating equation in finite samples, a simple modification of the stochastic EM algorithm is suggested. The modified stochastic algorithm is applicable when the E-step of an EM algorithm is intractable, and it is related to a deterministic algorithm that solves an unbiased estimating equation. In small-sample simulation studies of standard censoring and mixture problems, the modified stochastic algorithm outperforms the usual stochastic EM algorithm and the maximum likelihood estimator. In large samples all approaches perform similarly.