Two decades of statistical language modeling: Where do we go from here?

Statistical language models estimate the distribution of various natural language phenomena;for the purpose of speech recognition and other language technologies. Since the first significant model was proposed in 1980, many attempts have been made to improve the state-of-the-art. We review them here, point to a few promising directions, and argue for a Bayesian approach to integration of linguistic theories with data.