The integration of quantitative genetics, paleontology, and neontology reveals genetic underpinnings of primate dental evolution

Developmental genetics research on mice provides a relatively sound understanding of the genes necessary and sufficient to make mammalian teeth. However, mouse dentitions are highly derived compared with human dentitions, complicating the application of these insights to human biology. We used quantitative genetic analyses of data from living nonhuman primates and extensive osteological and paleontological collections to refine our assessment of dental phenotypes so that they better represent how the underlying genetic mechanisms actually influence anatomical variation. We identify ratios that better characterize the output of two dental genetic patterning mechanisms for primate dentitions. These two newly defined phenotypes are heritable with no measurable pleiotropic effects. When we consider how these two phenotypes vary across neontological and paleontological datasets, we find that the major Middle Miocene taxonomic shift in primate diversity is characterized by a shift in these two genetic outputs. Our results build on the mouse model by combining quantitative genetics and paleontology, and thereby elucidate how genetic mechanisms likely underlie major events in primate evolution.